[crawler series] 2. Open the App reverse "Pandora's box"
1, Foreword

• In recent years, most products either have limited Web-side functions or have no Web-side directly. It is becoming more and more difficult to grasp data directly from the Web-side
• Grasp the basic operation of HTTP requests through the proxy software (whistle | fiddler|Charles), and learn the relevant tutorials by yourself.
• After Android 7, the system directly does not trust the cert certificate installed by the user under non Root conditions, which directly makes it more troublesome for App HTTP packet capturing; iOS packet capture is much simpler. After the trust certificate, the green light is all the way. You can even spend 168 to buy a local packet capture tool on the iOS side
• The opposite is that sometimes you "work hard" to get HTTP packets and take a look. The small "sign" field lies in the request body, and each request will change. The door of "data acquisition" opens a crack and closes again, which is really uncomfortable

So
Is there any way?
What's next?

PS: all operations in this article are based on Android App, and iOS is not included in this tutorial (neither Chen Qie)

2, Prior knowledge

2.0 core ideas

Pasting block s outside Docs is not supported
2.1 tips for persuasion

• Can understand java code and know JAVA_HOME ,ANDROID_HOME, which can independently configure Android Studio and Maven warehouse
• Understand the command line operation, understand the basic operation of Git, and get code from Github (don't TM Download zip)
• I have a lot of Android development knowledge, at least know the adb operation and what the apk file is, and can run gralwe

PS: if not, I suggest you wash and go to bed early.
PPS: or ask me for one-on-one guidance.

2.2 preparation of working environment

• JDK8 local environment (don't ask why JDK8 is capricious)
  ➜ ~ java -version
  openjdk version "1.8.0_292"
  OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_292-b10)
  OpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.292-b10, mixed mode)
• skylot/jadx (Dex to Java decompiler) git clone + jdax tool configured - > user decompiles Apk
• install Android Studio download + install. If the network is good, it's best to directly fix Android NDK 21.0
• I almost forgot to say that there should be an Android phone and a data cable

3, Open up
3.0 it's a nice day today
PS: let me see, whose App is more interesting
PPS: not for anyone, just "learn" excellent code

3.1 signature cracking of Douban App (Java Native)

• After looking through the previous projects, it seems that Douban App is a local signature, so it's it
• Learning purpose: Douban group API cracking
  Get a curl first and see the http interface request
  curl --location --request GET 'https://frodo.douban.com/api/...' \
  --header 'Authorization;' \
  --header 'User-Agent: api-client/1 com.douban.frodo/7.1.0(205) Android/29 product/perseus vendor/Xiaomi model/Mi MIX 3 rom/miui6 network/wifi udid/a0f9cde79ec841a748625f766273e8f4333ed9c1 platform/mobile nd/1' \
  --header 'Cookie: bid=EGo8Z7aSUAI'

You can see that the request returns normally, and you can also see the data.

However, if we switch the data of other groups, we will soon see the prompt of "invalid_request_996" and signature error.

Careful friends, I have probably seen the field "_sig", which is obviously the abbreviation of "sign".
All right, get to work.

Douban App - Download

Download apk

wget https://img2.doubanio.com/dae...

Decompile apk with jadx and output to the double SRC folder

jadx -d douban-src com.douban.frodo_douban_7.18.1_231.apk

Open the doublan SRC project using Android studio

studio douban-src

Decompile result

Android project code

The simplest way is to search the field "_sig" directly“

You can probably find this Code:

@android.webkit.JavascriptInterface
public java.lang.String decoratorUrl(java.lang.String str) {

try {
    if (!com.douban.frodo.baseproject.rexxar.RexxarConstant.a(android.net.Uri.parse(str).getHost())) {
        return str;
    }
    android.net.Uri.Builder appendQueryParameter = android.net.Uri.parse(str).buildUpon().appendQueryParameter("udid", 
    com.douban.frodo.baseproject.util.FrodoUtils.a()).appendQueryParameter("rom", 
    com.douban.frodo.baseproject.util.Utils.i()).appendQueryParameter("apikey", 
    com.douban.frodo.baseproject.util.FrodoUtils.c()).appendQueryParameter(com.umeng.commonsdk.proguard.d.ao, "rexxar_new");
    if (com.douban.frodo.utils.AppContext.b() != null) {
        appendQueryParameter.appendQueryParameter("channel", com.douban.frodo.utils.AppContext.b().market);
    }
    android.util.Pair<java.lang.String, java.lang.String> a 
           = com.douban.frodo.network.ApiSignatureHelper.a(str, "GET", null);
    if (a != null) {
        appendQueryParameter.appendQueryParameter("_sig", (java.lang.String) a.first);
        appendQueryParameter.appendQueryParameter("_ts", (java.lang.String) a.second);
    }
    str = appendQueryParameter.build().toString();
    return str;
} catch (java.lang.Exception e) {
    e.printStackTrace();
}

}

The obvious signature is implemented in the class "com.doublan.frodo.network.apisignaturehelper",
The a method is used to sign the str variable, and the_ sig and_ ts, and then throw it to QueryParameter.

package com.douban.frodo.network;

public class ApiSignatureHelper {

static android.util.Pair<java.lang.String, java.lang.String> a(okhttp3.Request request) {
    if (request == null) {
        return null;
    }
    java.lang.String header = request.header(com.douban.push.internal.api.Request.HEADER_AUTHORIZATION);
    if (!android.text.TextUtils.isEmpty(header)) {
        header = header.substring(7);
    }
    return a(request.url().toString(), request.method(), header);
}

public static android.util.Pair<java.lang.String, java.lang.String> a(
        java.lang.String str, java.lang.String str2, java.lang.String str3) {
    if (android.text.TextUtils.isEmpty(str)) {
        return null;
    }
    java.lang.String str4 = com.douban.frodo.network.FrodoApi.a().e.b;
    if (android.text.TextUtils.isEmpty(str4)) {
        return null;
    }
    java.lang.StringBuilder sb = new java.lang.StringBuilder();
    sb.append(str2);
    java.lang.String encodedPath = okhttp3.HttpUrl.parse(str).encodedPath();
    if (encodedPath == null) {
        return null;
    }
    java.lang.String decode = android.net.Uri.decode(encodedPath);
    if (decode == null) {
        return null;
    }
    if (decode.endsWith("/")) {
        decode = decode.substring(0, decode.length() - 1);
    }
    sb.append(jodd.util.StringPool.AMPERSAND);
    sb.append(android.net.Uri.encode(decode));
    if (!android.text.TextUtils.isEmpty(str3)) {
        sb.append(jodd.util.StringPool.AMPERSAND);
        sb.append(str3);
    }
    long currentTimeMillis = java.lang.System.currentTimeMillis() / 1000;
    sb.append(jodd.util.StringPool.AMPERSAND);
    sb.append(currentTimeMillis);
    return new android.util.Pair<>(com.douban.frodo.utils.crypto.HMACHash1.a(
            str4, sb.toString()), java.lang.String.valueOf(currentTimeMillis));
}

}

You should be happy to see this code. Lines 17 - 45 are processing the incoming parameters,
It mainly uses Android net. After URI extracts the parameters, it is assembled into a string divided by "&", which can be processed by itself in a familiar language.
Finally, the signature calls "com. Doublan. Frodo. Utils. Crypto. Hmachash1. A", which needs to be followed.
However, the name "HMAC-SHA1" has given a lot of information.

Follow the code to enter com douban. frodo. utils. crypto. Hmachash1 class, so you see the following code:
package com.douban.frodo.utils.crypto;

public class HMACHash1 {

public static final java.lang.String a(java.lang.String str, 
java.lang.String str2) {
    try {
        javax.crypto.spec.SecretKeySpec secretKeySpec = new javax.crypto.spec.SecretKeySpec(
                str.getBytes(), com.douban.live.internal.LiveHelper.HMAC_SHA1);
        javax.crypto.Mac instance = javax.crypto.Mac.getInstance(
                com.douban.live.internal.LiveHelper.HMAC_SHA1);
        instance.init(secretKeySpec);
        return android.util.Base64.encodeToString(
                instance.doFinal(str2.getBytes()), 2);
    } catch (java.lang.Exception e) {
        e.printStackTrace();
        return null;
    }
}

}

A HMAC implemented in pure Java_ SHA1 encryption algorithm.
str is the SecretKey (a variable of ZenoConfig), and str2 is the incoming signature data.

Now there's only one last question. How do I get the secret key?
Returning to the above code, you can see:
java.lang.String str4 = com.douban.frodo.network.FrodoApi.a().e.b;
//str4 is the key passed to HMACHash1. e here is ZenoConfig

Here you can see zenoconfig B is the str3 passed in, that is, the secret key we are looking for
Keep turning and you'll see

java.lang.String d2 = com.douban.frodo.baseproject.util.FrodoUtils.d();

builder.c = d2;

com.douban.zeno.ZenoConfig zenoConfig = new com.douban.zeno.ZenoConfig(

    builder.a, builder.b, builder.c, builder.d, builder.e, 
    builder.f, builder.g, builder.h, builder.i, builder.j);

package com.douban.frodo.baseproject.util;

public class FrodoUtils {

private static java.lang.String a;
private static java.lang.String b;
private static java.lang.String c;
private static java.lang.String d;
private static java.lang.String e;

public static java.lang.String e() {
    return "frodo://app/oauth/callback/";
}

public static java.lang.String a() {
    if (android.text.TextUtils.isEmpty(a)) {
        a = com.douban.amonsul.MobileStat.g((android.content.Context) com.douban.frodo.utils.AppContext.a());
    }
    return a;
}

public static void a(java.lang.String str) {
    e = str;
}

public static java.lang.String b() {
    return e;
}

public static java.lang.String c() {
    return b;
}

public static java.lang.String d() {
    return c;
}

@android.annotation.SuppressLint({"PackageManagerGetSignatures"})
public static void a(boolean z) {
    if (android.text.TextUtils.isEmpty(b)) {
        b = "74CwfJd4+7LYgFhXi1cx0IQC35UQqYVFycCE+EVyw1E=";
    }
    if (android.text.TextUtils.isEmpty(c)) {
        c = "bHUvfbiVZUmm2sQRKwiAcw==";
    }
    if (z) {
        try {
            java.lang.String encodeToString = android.util.Base64.encodeToString(
            com.douban.frodo.utils.AppContext.a().getPackageManager().getPackageInfo(
            com.douban.frodo.utils.AppContext.a().getPackageName(),
             64).signatures[0].toByteArray(), 0);
            b = com.douban.frodo.utils.crypto.AES.a(b, encodeToString);
            c = com.douban.frodo.utils.crypto.AES.a(c, encodeToString);
        } catch (android.content.pm.PackageManager.NameNotFoundException e2) {
            e2.printStackTrace();
        }
    }
}

public static java.lang.String f() {
    if (android.text.TextUtils.isEmpty(d)) {
        return "d40568d833";
    }
    return d;
}

}

So we know that the so-called "secret key" is actually C = "bhuvfbivzumm2sqrkwiacw = ="; After AES encryption, encodeToString is com douban. frodo. utils. AppContext. a(). Getpackagename() package name information. If you have developed android App, you will probably know that this code is used to obtain the signature of the current application, which is an anti tamper security mechanism of Android.
Although we can't get com directly douban. frodo. utils. AppContext. a(). Getpackagename(), however, other applications can also obtain the signature information of the installed application. They only need to pass in the package name of the corresponding app as a parameter.
therefore...
Application application=(Application)getApplicationContext();
PackageInfo packageInfo=application.getPackageManager().getPackageInfo("com.douban.frodo",PackageManager.GET_SIGNATURES);
String sign=Base64.encodeToString(packageInfo.signatures[0].toByteArray(),0);

Finally, we run the above signature code to get the secret key = "bf7dddc7c9cfe6f7"

Very good, very awesome, obviously can be off duty.
code?
I...
Also
Lazy
I have to
Write

Friends in need can go to the Douban app signature algorithm analysis and decryption for themselves.

3.2 ratel core Android reverse analysis tool suite

• GitHub - virjarratel / ratel core: the core code of Flathead
• Introduction · Ratel document
  ratel is an Android reverse analysis tool suite, which provides a series of progressive app reverse analysis tools.
  At the same time, pingtouge is also a sandbox environment for app secondary development. It supports hook and redefine app functions in a root free environment.
  For most apps, Flathead has opened Pandora's box. Please don't illegally use Flathead outside the authorization (only recommended for personal customized use, app attack and defense security research, etc.). Some consequences caused by illegal use of ratel outside the official authorization of ratel shall be borne by the user
  Pingtouge is an app reverse analysis ecology, and the development progress lasted 3 years. At present, we are considering launching a commercial version of open source.

At the same time, as a complete closed-loop tool chain, Flathead has many related functions.

• Basic hook arbitrary app function, root free ability
• Split and multi opening capability (at present, 100 devices of a mobile phone have been verified in production)
• Fingerprint countermeasure capability of equipment: it is found that it can deal with some large factories
• Group control capability: built in supplappium module, and its open source solution: https://bbs.pediy.com/thread-...
• Timed task management is mostly to support group control without computer (offline group control without USB)
• Hot release: the plug-in module takes effect on all devices in the cluster through the back-end hot release. And supports rollback
• RDP: at present, it is the only functional module on the market that can also realize smali repackaging for wechat and other large app s
• Shelling: the built-in instruction dump level shelling machine can avoid root shelling
• Compatibility and adaptation: in more than 2000 devices, 500 sample apps from the application market have been tested. Overwrite Android 5 0-Android10. 0 (Android 11 is already under internal test)
• No root IDA debugging, built-in JustTrustMe
• Built in socketmonitor (appeared 3 years earlier than R0Cpature of shredded meat, and even supported thread jump tracking in the early stage to solve asynchronous problems)
• SplitApk: Google App Store Android installation package distribution format
• Multiple repackaging schemes are supported: appendDex, rebuildDex, zelda and shell
• Ecology: wechat robot, simulated positioning, multi account resource backup and restore

One sentence summary: in the case of non ROOT, the existing App functions can be changed arbitrarily through the plug-in mechanism.

So let's try the water.

3.2. 1. Prepare ratel core compilation environment

• JDK8
• Android Studio + NDK21.0 (NDK revision history | Android NDK | Android Developers)
• adb
  export ANDROID_NDK_HOME=/Users/liguobao/Library/Android/sdk/ndk/21.0.6113669
  export ANDROID_SDK_ROOT="/Users/liguobao/Library/Android/sdk"
  export PATH="${PATH}:${ANDROID_SDK_ROOT}/tools:${ANDROID_SDK_ROOT}/platform-tools"

Refer to the above to configure the local ANDROID_NDK_HOME + ANDROID_SDK_ROOT
PS: if you are not familiar with this thing, turn to the tutorial.

download https://github.com/virjarRate... Source code
$ git clone https://github.com/virjarRate...
$ cd ratel-core/

Compile code + check environment configuration

$ ./script/create-dist.sh

All the above operations are OK. After normal compilation,

You can use it/ script/ratel.sh to repackage the App.

➜ script git:(master) cd dist
➜ dist git:(master) ./ratel.sh ~/Downloads/leyoujia.apk

Normally, a new apk file will be generated. The last step is to install the apk on the mobile phone.

After installation, open the App on the mobile terminal. If the App does not crash, it means success.
PS: in case of crash, go to GitHub to mention Issues.

3.2. 2. Prepare ratel module project

• The repackaged Apk is the App that opens the "back door"
• The plug-in gives us the ability to customize App + manipulate App
• https://github.com/virjarRate...
  At the same time, you can "rate manager" to the mobile terminal to manage and view App infection information through this App management plug-in.
• Ratel Manager can also be compiled in the source code or directly download the old version.

"Happy home" is the App that has just been repackaged.
$ git clone https://github.com/virjarRate...
$ cd ratel-module-template/
$ ./template.sh ~/Downloads/leyoujia.apk

Then open the entire ratel module template project in Android studio,
Wait a minute, Index code and restore related package files.

Try compiling and installing "crack happy home" to the mobile terminal.

Under normal circumstances, the plug-in App will start directly,
In addition, you can see the new plug-in App in the Ratel Manager module

• Switch the rate status and pull down the page several times to trigger the refresh mechanism

The default plug-in project already has an interesting feature - "insert hover button"
//Add floating window
private static void addFloatingButtonForActivity(final RC_LoadPackage.LoadPackageParam lpparam) {

RposedHelpers.findAndHookMethod(Activity.class, "onCreate", Bundle.class, new RC_MethodHook() {
    @Override
    protected void afterHookedMethod(final MethodHookParam param) throws Throwable {
        new Handler(Looper.getMainLooper())
                .postDelayed(new Runnable() {
                    @Override
                    public void run() {
                        createAndAttachFloatingButtonOnActivity((Activity) param.thisObject);
                    }
                }, 1000);
    }

    private void createAndAttachFloatingButtonOnActivity(Activity activity) {
        Context context = RatelToolKit.ratelResourceInterface.createContext(lpparam.modulePath, HookEntry.class.getClassLoader(), RatelToolKit.sContext);

        FrameLayout frameLayout = (FrameLayout) activity.getWindow().getDecorView();
        LayoutInflater.from(context).cloneInContext(context)
                .inflate(R.layout.float_button, frameLayout);

    }
});

}

So far, the ratel module plug-in project has been running normally,
Demo code, we have already run.

3.2. Do something interesting?
A. Directly trust all local users cert certificates
// package ratel.com.jjs.android.butler;
// public class HookEntry implements IRposedHookLoadPackage {}
//Add a new line of code in the handleLoadPackage function, and then recompile and install,
//Re open the App after operating the refresh plug-in on the mobile phone
JustTrustMe.trustAllCertificate();

So there is no hiding place for HTTPS requests.

curl --location --request POST 'https://steward.leyoujia.com/...' \
--header 'host: steward.leyoujia.com' \
--header 'clientid: e14becf6-2e13-43ae-b453-bd9cd35354a4' \
--header 'd: 0' \
--header 'latitude;' \
--header 'channel: online_32' \
--header 'imsi: 460110136201976' \
--header 'uuid: e14becf6-2e13-43ae-b453-bd9cd35354a4' \
--header 'ssid: 00000000378d5761ffffffffa488b92e' \
--header 'version: 8.1.9' \
--header 'mac: 64:BC:0C:44:64:01' \
--header 'network: WIFI' \
--header 'cit: 001729' \
--header 'sid: 38e5d3e9be36f0508dff7415ffffc076' \
--header 'phonemodel: Maru on the Nexus 5X' \
--header 'phoneos: android' \
--header 'carries: 0' \
--header 'imei: 35362607298355' \
--header 'aid: APP001' \
--header 'clientsign: 39ea04d3954db18df716e21978f009df' \
--header 'androidid: aabfdc5bb198e3b7' \
--header 'oaid: 00000000378d5761ffffffffa488b92e' \
--header 'longitude;' \
--header 'timestamp: 1639312110572' \
--header 'content-type: application/x-www-form-urlencoded' \
--header 'user-agent: okhttp/3.9.1' \
--header 'Connection: close' \
--data-urlencode 'cityCode=001729'

Take a closer look, the clientsign signature stands out.
Good guy, it's time to start doing things again
Return to the operation of 3.1, generate a "leyoujia SRC" and start
B. Reopen the leyoujia SRC code
Search "clientSign" and you will probably see the code

java.lang.String encode32 = com.jjshome.common.utils.MD5.encode32(
com.jjshome.common.utils.MD5.encode32(sb.toString()));

package com.jjshome.common.utils;

public class MD5 {

/* JADX WARNING: type inference failed for: r2v2, types: [int] */
/* JADX WARNING: type inference failed for: r2v5 */
/* JADX WARNING: Multi-variable type inference failed */
public static java.lang.String encode32(java.lang.String str) {
    java.lang.StringBuffer stringBuffer = new java.lang.StringBuffer("");
    try {
        java.security.MessageDigest instance = java.security.MessageDigest.getInstance("MD5");
        instance.update(str.getBytes());
        byte[] digest = instance.digest();
        for (int i = 0; i < digest.length; i++) {
            byte b = digest[i];
            if (b < 0) {
                b += 256;
            }
            if (b < 16) {
                stringBuffer.append("0");
            }
            stringBuffer.append(java.lang.Integer.toHexString(b));
        }
    } catch (java.lang.Exception e) {
        e.printStackTrace();
    }
    return stringBuffer.toString();
}

}

Maybe the story has become something. I see how the sb string came.
Omit here and toss on your own.
PS:

• In fact, it took an afternoon to do this thing. It's a little painful how to splice the parameters.
• Read the code and look at the logic. You can always fix it. Come on, friends!

N1. Summary

• Just study. Don't have dangerous ideas.
• Attack and defense are always relative and can never be done once and for all.
• Every day from the "basket bridge" further.
  N2. Summary
• Learned the use of jadx reverse tools and reviewed the basics of Java
• I learned the Ratel Flathead tool and the basic knowledge of Android

Reference reading:

• Analysis and decryption of Douban app signature algorithm - Tianci network
• How does Android call so files
• What is the so file of Linux? Analysis of Linux dynamic link library
• Android so file loading mechanism - please call me da Su - blog Park
• Use of ratel

To be continued

• Rate + sekiro high-level usage of "roll dead" App reverse peers.
  Coming soon...