What is RPC
RPC, English RangPaCong, Chinese let the reptile, designed to open the way for the reptile, kill everything every second, and let the reptile unobstructed!
Just kidding, in fact, RPC is a Remote Procedure Call, which is a technical idea rather than a specification or protocol. In fact, the birth of RPC is inseparable from the development of distribution. RPC mainly solves two problems:
- It solves the problem of calling each other between services in distributed system;
- RPC makes the remote call as convenient as the local call, and makes the caller unaware of the logic of the remote call.
The existence of RPC makes it easier to build a distributed system. Compared with HTTP protocol, RPC adopts binary bytecode transmission, so it is also more efficient and secure. A typical RPC usage scenario includes service discovery, load, fault tolerance, network transmission, serialization and other components. The complete RPC architecture is shown in the following figure:
JSRPC
RPC technology is very complex. For those of us engaged in crawler and reverse, we don't need to fully understand it. We just need to know how to apply this technology in reverse.
In reverse, RPC simply means that the local and browser are regarded as server and client. They communicate with each other through WebSocket protocol, expose the encryption function in the browser, and directly call the corresponding encryption function in the browser locally, so as to obtain the encryption result. There is no need to care about the specific execution logic of the function, and the deduction code is omitted Operations such as environment repair can save a lot of reverse debugging time. We take the login of a group of web pages as an example to demonstrate the specific use of RPC in reverse. (assuming you have a certain reverse foundation and understand the WebSocket protocol, pure Xiaobai can look at brother K's previous articles first)
- Home page (base64): aHR0cHM6Ly9wYXNzcG9ydC5tZWl0dWFuLmNvbS9hY2NvdW50L3VuaXRpdmVsb2dpbg==
- Parameter: h5Fingerprint
First grab the package. The login interface has a super long parameter h5Fingerprint, as shown in the following figure:
Search directly to find the encryption function:
Where utility Geth5fingerprint() passed in parameter window location. After the origin + URL is formatted, the parameters are as follows:
url = "https://passport. Desensitization treatment com/account/unitivelogin"
params = {
"risk_partner": "0",
"risk_platform": "1",
"risk_app": "-1",
"uuid": "96309b5f00ba4143b920.1644805104.1.0.0",
"token_id": "DNCmLoBpSbBD6leXFdqIxA",
"service": "www",
"continue": "https://www. desensitization treatment com/account/settoken?continue=https%3A%2F%2Fwww. Desensitization treatment com%2F"
}uuid and token_id can be found directly, which is not the focus of this research. I won't elaborate here. Next, we use RPC technology to directly call the utility in the browser Geth5fingerprint() method, first write the server code locally, so that it can always input the string to be encrypted, and receive and print the encrypted string:
# ==================================
# --*-- coding: utf-8 --*--
# @Time : 2022-02-14
# @Author: WeChat official account: K brother crawler
# @FileName: ws_server.py
# @Software: PyCharm
# ==================================
import sys
import asyncio
import websockets
async def receive_massage(websocket):
while True:
send_text = input("Please enter a string to encrypt: ")
if send_text == "exit":
print("Exit, goodbye!")
await websocket.send(send_text)
await websocket.close()
sys.exit()
else:
await websocket.send(send_text)
response_text = await websocket.recv()
print("\n Encryption result:", response_text)
start_server = websockets.serve(receive_massage, '127.0.0.1', 5678) # Custom port
asyncio.get_event_loop().run_until_complete(start_server)
asyncio.get_event_loop().run_forever()Write the JS code of the browser client and directly use it when you receive the message Geth5fingerprint() gets the encryption parameters and sends them to the server:
/* ==================================
# @Time : 2022-02-14
# @Author : WeChat official account: K brother crawler
# @FileName: ws_client.js
# @Software: PyCharm
# ================================== */
var ws = new WebSocket("ws://127.0.0.1:5678 "); / / custom port
ws.onmessage = function (evt) {
console.log("Received Message: " + evt.data);
if (evt.data == "exit") {
ws.close();
} else {
ws.send(utility.getH5fingerprint(evt.data))
}
};Then we need to inject the client code into the web page. There are many methods here, such as replacing the response with Fiddler, replacing JS with rres, rewriting the browser developer tool Overrides, etc. we can also insert it by injecting Hook into plug-ins and oil monkeys. Anyway, there are many methods, Friends who don't know much about these methods can go to see brother K's previous articles, which are introduced.
Here, we use the override rewriting function of the browser developer tool to add the WebSocket client code to the encrypted JS file and save it with Ctrl+S. here, it is written as IIFE self-execution mode. The reason for this is to prevent pollution of global variables. Of course, it is OK not to use self-execution mode.
Then run the local server code first, log in on the web page first, log in on the web page first, log in on the web page first, and say the important steps three times! Then you can pass in the string to be encrypted locally to obtain the utility The encrypted result of geth5fingerprint():
Sekiro
Through the previous example, you can find that writing the server is too cumbersome and difficult to expand. Is there a ready-made wheel in this regard? The answer is yes. Here are two items:
- JsRPC-hliang: https://github.com/jxhczhl/JsRpc
- Sekiro: https://github.com/virjar/sekiro
JSRPC hliang is written in go language and is a project specially designed for JS reverse. Sekiro is more powerful. Sekiro is an Android Private API exposure framework based on long link and code injection written by Deng Weijia, commonly known as president slag. It can be used in APP reverse, APP data capture, Android group control and other scenarios, At the same time, sekiro is also the only stable JSRPC framework for the current open scheme. In fact, the use methods of both in JS reverse are similar. This paper mainly introduces the application of sekiro in Web JS reverse.
Referring to the Sekiro documentation, first compile the project locally:
- Linux & Mac: execute script build_demo_server.sh, and then get the output release compressed package: sekiro service demo / target / sekiro release demo zip
- Windows: can be downloaded directly: https://oss.virjar.com/sekiro...
Then run it locally (Java environment is required, self configuring):
- Linux & Mac: bin/sekiro.sh
- Windows: bin/sekiro.bat
Take Windows as an example. After startup, it is as follows:
Next, you need to inject code into the browser. You need to use the sekiro provided by the author_ web_ client. JS (download address: https://sekiro.virjar.com/sek... )Inject it into the browser environment, and then communicate with SekiroClient and Sekiro server to directly call the browser's internal methods through RPC. The official SekiroClient code example is as follows:
function guid() {
function S4() {
return (((1+Math.random())*0x10000)|0).toString(16).substring(1);
}
return (S4()+S4()+"-"+S4()+"-"+S4()+"-"+S4()+"-"+S4()+S4()+S4());
}
var client = new SekiroClient("wss://sekiro.virjar.com/business/register?group=ws-group&clientId="+guid());
client.registerAction("clientTime",function(request, resolve, reject){
resolve(""+new Date());
})In the wss link, if it is a free version, change business to business demo, and explain the terms involved:
- Group: business type (Interface Group). Each business has a group. Under the group, multiple terminals (SekiroClient) can be registered, and the group can mount multiple actions;
- clientId: refers to a device. Multiple devices use multiple machines to provide API services and provide group control capability and load balancing capability;
- SekiroClient: service provider client. The main scenarios are mobile phones / browsers, etc. The final Sekiro call is forwarded to SekiroClient. Each client needs to have a unique clientId;
- registerAction: interface. There can be multiple interfaces under the same group to perform different functions;
- resolve: the method to send the content back to the client;
- Request: if there are multiple parameters in the request sent by the client, the parameters can be extracted from it in the form of key value pairs and then processed.
It may not be easy to understand after saying so much. Direct actual combat, or take the web login of a regiment as an example. We will sekiro_web_client.js and SekiroClient communication code together, and then rewrite the communication code according to the requirements:
- The ws link is changed to: ws: / / 127.0.0.1:5620 / business demo / register? Group = RPC test & ClientID =, user-defined group is RPC test;
- Register an event registerAction as getH5fingerprint;
- The result returned by resolve is utility Geth5fingerprint (request ["url"]), that is, encrypt and return the url parameter passed from the client.
Pay attention to the end of the communication code of SekiroClient:
/* ==================================
# @Time : 2022-02-14
# @Author : WeChat official account: K brother crawler
# @FileName: sekiro.js
# @Software: PyCharm
# ================================== */
(function () {
'use strict';
function SekiroClient(wsURL) {
this.wsURL = wsURL;
this.handlers = {};
this.socket = {};
// check
if (!wsURL) {
throw new Error('wsURL can not be empty!!')
}
this.webSocketFactory = this.resolveWebSocketFactory();
this.connect()
}
SekiroClient.prototype.resolveWebSocketFactory = function () {
if (typeof window === 'object') {
var theWebSocket = window.WebSocket ? window.WebSocket : window.MozWebSocket;
return function (wsURL) {
function WindowWebSocketWrapper(wsURL) {
this.mSocket = new theWebSocket(wsURL);
}
WindowWebSocketWrapper.prototype.close = function () {
this.mSocket.close();
};
WindowWebSocketWrapper.prototype.onmessage = function (onMessageFunction) {
this.mSocket.onmessage = onMessageFunction;
};
WindowWebSocketWrapper.prototype.onopen = function (onOpenFunction) {
this.mSocket.onopen = onOpenFunction;
};
WindowWebSocketWrapper.prototype.onclose = function (onCloseFunction) {
this.mSocket.onclose = onCloseFunction;
};
WindowWebSocketWrapper.prototype.send = function (message) {
this.mSocket.send(message);
};
return new WindowWebSocketWrapper(wsURL);
}
}
if (typeof weex === 'object') {
// this is weex env : https://weex.apache.org/zh/docs/modules/websockets.html
try {
console.log("test webSocket for weex");
var ws = weex.requireModule('webSocket');
console.log("find webSocket for weex:" + ws);
return function (wsURL) {
try {
ws.close();
} catch (e) {
}
ws.WebSocket(wsURL, '');
return ws;
}
} catch (e) {
console.log(e);
//ignore
}
}
//TODO support ReactNative
if (typeof WebSocket === 'object') {
return function (wsURL) {
return new theWebSocket(wsURL);
}
}
// weex is not completely consistent with the websocket API of PC environment, so it is abstractly compatible
throw new Error("the js environment do not support websocket");
};
SekiroClient.prototype.connect = function () {
console.log('sekiro: begin of connect to wsURL: ' + this.wsURL);
var _this = this;
// No check close, let
// if (this.socket && this.socket.readyState === 1) {
// this.socket.close();
// }
try {
this.socket = this.webSocketFactory(this.wsURL);
} catch (e) {
console.log("sekiro: create connection failed,reconnect after 2s");
setTimeout(function () {
_this.connect()
}, 2000)
}
this.socket.onmessage(function (event) {
_this.handleSekiroRequest(event.data)
});
this.socket.onopen(function (event) {
console.log('sekiro: open a sekiro client connection')
});
this.socket.onclose(function (event) {
console.log('sekiro: disconnected ,reconnection after 2s');
setTimeout(function () {
_this.connect()
}, 2000)
});
};
SekiroClient.prototype.handleSekiroRequest = function (requestJson) {
console.log("receive sekiro request: " + requestJson);
var request = JSON.parse(requestJson);
var seq = request['__sekiro_seq__'];
if (!request['action']) {
this.sendFailed(seq, 'need request param {action}');
return
}
var action = request['action'];
if (!this.handlers[action]) {
this.sendFailed(seq, 'no action handler: ' + action + ' defined');
return
}
var theHandler = this.handlers[action];
var _this = this;
try {
theHandler(request, function (response) {
try {
_this.sendSuccess(seq, response)
} catch (e) {
_this.sendFailed(seq, "e:" + e);
}
}, function (errorMessage) {
_this.sendFailed(seq, errorMessage)
})
} catch (e) {
console.log("error: " + e);
_this.sendFailed(seq, ":" + e);
}
};
SekiroClient.prototype.sendSuccess = function (seq, response) {
var responseJson;
if (typeof response == 'string') {
try {
responseJson = JSON.parse(response);
} catch (e) {
responseJson = {};
responseJson['data'] = response;
}
} else if (typeof response == 'object') {
responseJson = response;
} else {
responseJson = {};
responseJson['data'] = response;
}
if (Array.isArray(responseJson)) {
responseJson = {
data: responseJson,
code: 0
}
}
if (responseJson['code']) {
responseJson['code'] = 0;
} else if (responseJson['status']) {
responseJson['status'] = 0;
} else {
responseJson['status'] = 0;
}
responseJson['__sekiro_seq__'] = seq;
var responseText = JSON.stringify(responseJson);
console.log("response :" + responseText);
this.socket.send(responseText);
};
SekiroClient.prototype.sendFailed = function (seq, errorMessage) {
if (typeof errorMessage != 'string') {
errorMessage = JSON.stringify(errorMessage);
}
var responseJson = {};
responseJson['message'] = errorMessage;
responseJson['status'] = -1;
responseJson['__sekiro_seq__'] = seq;
var responseText = JSON.stringify(responseJson);
console.log("sekiro: response :" + responseText);
this.socket.send(responseText)
};
SekiroClient.prototype.registerAction = function (action, handler) {
if (typeof action !== 'string') {
throw new Error("an action must be string");
}
if (typeof handler !== 'function') {
throw new Error("a handler must be function");
}
console.log("sekiro: register action: " + action);
this.handlers[action] = handler;
return this;
};
function guid() {
function S4() {
return (((1 + Math.random()) * 0x10000) | 0).toString(16).substring(1);
}
return (S4() + S4() + "-" + S4() + "-" + S4() + "-" + S4() + "-" + S4() + S4() + S4());
}
var client = new SekiroClient("ws://127.0.0.1:5620/business-demo/register?group=rpc-test&clientId=" + guid());
client.registerAction("getH5fingerprint", function (request, resolve, reject) {
resolve(utility.getH5fingerprint(request["url"]));
})
})();As in the previous method, use the override rewriting function of the browser developer tool to inject the above code into the web page JS:
Then Sekiro provided us with some API s:
- To view a list of groups: http://127.0.0.1:5620/busines...
- To view queue status: http://127.0.0.1:5620/busines...
- Call forwarding: http://127.0.0.1:5620/busines...
For example, we are now going to call utility What should the geth5fingerprint () encryption method do? Very simple. After the code is injected into the browser, you should first log in manually, log in manually, log in manually, and say important things three times! Then refer to the call forwarding API above for rewriting:
- Our customized group is RPC test;
- The event action is getH5fingerprint;
- The name of the parameter to be encrypted is url, and its value is, for example: https://www.baidu.com/
Then our call link should be: http://127.0.0.1:5620/business -demo/invoke? group=rpc-test&action=getH5fingerprint&url= https://www.baidu.com/ , open the browser directly, and the returned dictionary is the encryption result in data:
Similarly, if you use Python locally, you can directly complete the requests:
In front of us is sekiro_web_client.js is copied and injected into the browser together with the communication code. Here we can also have a more elegant method to directly create a new script for the document and insert sekiro in the form of link_ web_ client. JS, here are some problems to pay attention to:
- The first is the question of timing. It is necessary to wait for the completion of these elements in document to complete the SekiroClient communication. Otherwise, calling SekiroClient will be wrong. Here we can use the setTimeout method, which is used to call functions or calculate expressions after the specified milliseconds, and separately encapsulate the SekiroClient communication code into a function. For example, function startSekiro(), and then wait 1-2 seconds before executing sekiroclient communication code;
- Since SekiroClient communication code is encapsulated into a function, utility. Is called directly at this time Geth5fingerprint will prompt undefined, so we need to import it as a global variable, such as window getH5fingerprint = utility.getH5fingerprint, and then directly call window Geth5fingerprint is enough.
The complete code is as follows:
/* ==================================
# @Time : 2022-02-14
# @Author : WeChat official account: K brother crawler
# @FileName: sekiro.js
# @Software: PyCharm
# ================================== */
(function () {
var newElement = document.createElement("script");
newElement.setAttribute("type", "text/javascript");
newElement.setAttribute("src", "https://sekiro.virjar.com/sekiro-doc/assets/sekiro_web_client.js");
document.body.appendChild(newElement);
window.getH5fingerprint = utility.getH5fingerprint
function guid() {
function S4() {
return (((1 + Math.random()) * 0x10000) | 0).toString(16).substring(1);
}
return (S4() + S4() + "-" + S4() + "-" + S4() + "-" + S4() + "-" + S4() + S4() + S4());
}
function startSekiro() {
var client = new SekiroClient("ws://127.0.0.1:5620/business-demo/register?group=rpc-test&clientId=" + guid());
client.registerAction("getH5fingerprint", function (request, resolve, reject) {
resolve(window.getH5fingerprint(request["url"]));
})
}
setTimeout(startSekiro, 2000)
})();
Advantages and disadvantages
At present, if we don't reverse JS to implement encryption parameters, automation tools are most used, such as Selenium and puppeter. Obviously, these automation tools have cumbersome configuration and low operation efficiency. RPC technology doesn't need to load redundant resources, and its stability and efficiency are obviously higher. RPC doesn't need to consider browser fingerprint and various environments, If the risk control is not strict, high concurrency can be easily realized. On the contrary, because RPC is always mounted on the same browser, for sites with strict risk control, such as detecting the binding of UA, IP and encryption parameters, it is not feasible to call PRC too frequently. Of course, we can also study browser group control technology, Manipulating multiple different browsers can alleviate this problem to some extent. In short, RPC technology is still very good. In addition to JS reverse, it can be said to be a more versatile and efficient method at present. To a certain extent, it has achieved a shuttle of encryption parameters!