WeChat has already started mars, but there are few related articles on the market, even if there are many such as using xlog, so this time I hope to be able to analyze from stn, which is directly used for the bottom communication of im.
To be more comprehensive, let's start with samples.
First of all, WeChat explicitly replaced JSON and xml with google's open source protocol protobuf library.The reason for using this is also in terms of efficiency and throughput, which is nearly 10 times more efficient than json, and based on binary rather than text, the size of the transmission is more advantageous, no more specific, and you can check it yourself if you are interested.
Let's start with samples and see how we get listing data from http. Look directly at / mars-master/samples/android/marsSampleChat/app/src/main/java/com/tencent/mars/sample/ConversationActivity.java, which is the initial listing interface. Just look at this:
/** * pull conversation list from server */ private void updateConversationTopics() { if (taskGetConvList != null) { MarsServiceProxy.cancel(taskGetConvList); } mTextView.setVisibility(View.INVISIBLE); progressBar.setVisibility(View.VISIBLE); swipeRefreshLayout.setRefreshing(true); taskGetConvList = new NanoMarsTaskWrapper<Main.ConversationListRequest, Main.ConversationListResponse>( new Main.ConversationListRequest(), new Main.ConversationListResponse() ) { private List<Conversation> dataList = new LinkedList<>(); @Override public void onPreEncode(Main.ConversationListRequest req) { req.type = conversationFilterType; req.accessToken = ""; // TODO: Log.d("xxx", "onPreEncode: " + req.toString()); } @Override public void onPostDecode(Main.ConversationListResponse response) { Log.d("xxx", "onPostDecode: " + response.toString()); } @Override public void onTaskEnd(int errType, int errCode) { Log.d("xxx", "onTaskEnd: " + errType + " " + errCode); runOnUiThread(new Runnable() { @Override public void run() { if (response != null) { for (Main.Conversation conv : response.list) { dataList.add(new Conversation(conv.name, conv.topic, conv.notice)); Log.d("xxx", conv.toString()); } } if (!dataList.isEmpty()) { progressBar.setVisibility(View.INVISIBLE); conversationListAdapter.list.clear(); conversationListAdapter.list.addAll(dataList); conversationListAdapter.notifyDataSetChanged(); swipeRefreshLayout.setRefreshing(false); } else { Log.i(TAG, "getconvlist: empty response list"); progressBar.setVisibility(View.INVISIBLE); mTextView.setVisibility(View.VISIBLE); } } }); } }; MarsServiceProxy.send(taskGetConvList.setHttpRequest(CONVERSATION_HOST, "/mars/getconvlist")); }
Newa NanoMarsTaskWrapper object and Override several methods: onPreEncode, onPostDecode, onTaskEnd.They are callback before encoding and transmission, callback after receiving the result decoding and callback after the task is finished.
Set the http url address of the NanoMarsTaskWrapper object;
Send is executed through the send method of MarsServiceProxy;
Through these, we can generally understand that, through a built-in task system, the distribution and invocation of transmission; through services to drive the entire system to operate, and ensure independence;
As you can see in the directory, samples are divided into two parts: app, wrapper and jar.
Okay, let's start with wrapper and look at the basic structure.
First, manifest:
<manifest xmlns:android="http://schemas.android.com/apk/res/android" package="com.tencent.mars.sample.wrapper"> <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" /> <uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" /> <uses-permission android:name="android.permission.INTERNET" /> <uses-permission android:name="android.permission.WAKE_LOCK" /> <application> <service android:name=".service.MarsServiceNative" android:process=":marsservice" /> <receiver android:name="com.tencent.mars.BaseEvent$ConnectionReceiver" android:process=":marsservice"/> </application> </manifest>
As you can see, services for independent processes are agreed upon here.Broadcast recipients have agreed here that they are in the same process as the service.
What is the MarsServiceProxy used in the app above?
public class MarsServiceProxy implements ServiceConnection { ...... private MarsServiceProxy() { worker = new Worker(); worker.start(); } public static void init(Context context, Looper looper, String packageName) { if (inst != null) { // TODO: Already initialized return; } gContext = context.getApplicationContext(); gPackageName = (packageName == null ? context.getPackageName() : packageName); gClassName = SERVICE_DEFUALT_CLASSNAME; inst = new MarsServiceProxy(); } ...... }
It's actually a service connection object inherited from ServiceConnection, but it's not just a connection object.We see that he is a singleton that was initialized in the app's SamplleApplicaton onCreate:
// NOTE: MarsServiceProxy is for client/caller // Initialize MarsServiceProxy for local client, can be moved to other place MarsServiceProxy.init(this, getMainLooper(), null);
send is the static method called in app:
public static void send(MarsTaskWrapper marsTaskWrapper) { inst.queue.offer(marsTaskWrapper); }
This method actually operates on the queue LinkedBlockingQueue <MarsTaskWrapper>.See, this MarsServiceProxy is actually an api proxy with a cached task queue inside. send actually adds a task MarsTaskWrapper to the thread-safe queue.
For a moment, we'll look at his service capabilities.At the time of construction, new a Worker and start.This worker is a thread:
private static class Worker extends Thread { @Override public void run() { while (true) { inst.continueProcessTaskWrappers(); try { Thread.sleep(50); } catch (InterruptedException e) { // } } } }
That is, when this class was created, a worker thread was created, calling continueProcessTaskWrappers in a loop at an interval of 50 ms.Look again at continueProcessTaskWrappers:
private void continueProcessTaskWrappers() { try { if (service == null) { Log.d(TAG, "try to bind remote mars service, packageName: %s, className: %s", gPackageName, gClassName); Intent i = new Intent().setClassName(gPackageName, gClassName); gContext.startService(i); if (!gContext.bindService(i, inst, Service.BIND_AUTO_CREATE)) { Log.e(TAG, "remote mars service bind failed"); } // Waiting for service connected return; } MarsTaskWrapper taskWrapper = queue.take(); if (taskWrapper == null) { // Stop, no more task return; } try { Log.d(TAG, "sending task = %s", taskWrapper); final String cgiPath = taskWrapper.getProperties().getString(MarsTaskProperty.OPTIONS_CGI_PATH); final Integer globalCmdID = GLOBAL_CMD_ID_MAP.get(cgiPath); if (globalCmdID != null) { taskWrapper.getProperties().putInt(MarsTaskProperty.OPTIONS_CMD_ID, globalCmdID); Log.i(TAG, "overwrite cmdID with global cmdID Map: %s -> %d", cgiPath, globalCmdID); } service.send(taskWrapper, taskWrapper.getProperties()); } catch (Exception e) { // RemoteExceptionHandler e.printStackTrace(); } } catch (Exception e) { } }
1. Check whether the service is started or not, then start and return to wait for the next 50ms to continue;
2. Get a task from the queue, assign it a cmdID, and then call the send method of MarsService to execute the real send event.
In fact, from the above point of view, this service agent is doing these things, and the deeper things are actually handed over to the specific service process to do.This is a proxy api.
Okay, let's look down at the specific service.
First MarsService is a definition of aidl, but as we can see from this thread loop above, services are started according to Intent i = new Intent().setClassName(gPackageName, gClassName); this gClassName = SERVICE_DEFUALT_CLASSNAME; that is, public final String SERVICE_DEFUALT_CLASSNAME = "com.tencent.mars.sample.wrapper.service.MarsServiceNative"; see, MarsServiceNative.
Get into the service now.
public class MarsServiceNative extends Service implements MarsService { private static final String TAG = "Mars.Sample.MarsServiceNative"; private MarsServiceStub stub; ...... }
A MarsServiceStub is saved here. The send that follows calls it to implement it. Now put down send for the moment and look at onCreate:
@Override public void onCreate() { super.onCreate(); final MarsServiceProfile profile = gFactory.createMarsServiceProfile(); stub = new MarsServiceStub(this, profile); // set callback AppLogic.setCallBack(stub); StnLogic.setCallBack(stub); SdtLogic.setCallBack(stub); // Initialize the Mars PlatformComm Mars.init(getApplicationContext(), new Handler(Looper.getMainLooper())); // Initialize the Mars StnLogic.setLonglinkSvrAddr(profile.longLinkHost(), profile.longLinkPorts()); StnLogic.setShortlinkSvrAddr(profile.shortLinkPort()); StnLogic.setClientVersion(profile.productID()); Mars.onCreate(true); StnLogic.makesureLongLinkConnected(); // Log.d(TAG, "mars service native created"); }
1. Create the configuration information class MarsServiceProfile;
2.new comes out of MarsServiceStub;
3. Set various callbacks;
4. Initialize Mars;
5.Mars.onCreate(true);
6.StnLogic.makesureLongLinkConnected(); confirm long connection.
Mars is used here, this is the core, and it's not in this project.Let's lay down the core part and go into the next one.
Back to MarsServiceStub, see his send method:
@Override public void send(final MarsTaskWrapper taskWrapper, Bundle taskProperties) throws RemoteException { final StnLogic.Task _task = new StnLogic.Task(StnLogic.Task.EShort, 0, "", null); // Set host & cgi path final String host = taskProperties.getString(MarsTaskProperty.OPTIONS_HOST); final String cgiPath = taskProperties.getString(MarsTaskProperty.OPTIONS_CGI_PATH); _task.shortLinkHostList = new ArrayList<>(); _task.shortLinkHostList.add(host); _task.cgi = cgiPath; final boolean shortSupport = taskProperties.getBoolean(MarsTaskProperty.OPTIONS_CHANNEL_SHORT_SUPPORT, true); final boolean longSupport = taskProperties.getBoolean(MarsTaskProperty.OPTIONS_CHANNEL_LONG_SUPPORT, false); if (shortSupport && longSupport) { _task.channelSelect = StnLogic.Task.EBoth; } else if (shortSupport) { _task.channelSelect = StnLogic.Task.EShort; } else if (longSupport) { _task.channelSelect = StnLogic.Task.ELong; } else { Log.e(TAG, "invalid channel strategy"); throw new RemoteException("Invalid Channel Strategy"); } // Set cmdID if necessary int cmdID = taskProperties.getInt(MarsTaskProperty.OPTIONS_CMD_ID, -1); if (cmdID != -1) { _task.cmdID = cmdID; } TASK_ID_TO_WRAPPER.put(_task.taskID, taskWrapper); WRAPPER_TO_TASK_ID.put(taskWrapper, _task.taskID); // Send Log.i(TAG, "now start task with id %d", _task.taskID); StnLogic.startTask(_task); if (StnLogic.hasTask(_task.taskID)) { Log.i(TAG, "stn task started with id %d", _task.taskID); } else { Log.e(TAG, "stn task start failed with id %d", _task.taskID); } }
1.new a StnLogic.Task;
2. Set the parameters of task according to the Bundle of the entry;
3.2 map s preserve the relationship between taskID and task.
4.StnLogic.startTask(_task); Start task execution;
The content here goes deep into the Mars core, and you can see that the key processing is done in the core part of Mars, regardless of whether the service or anything is doing the work of parameter transfer and relationship maintenance.
Okay, let's bring it back to MarsServiceStub, which implements the interface StnLogic.ICallBack.Defined in mars:
public interface ICallBack { /** * SDK Require upper level to authenticate (a new AUTH CGI may be launched) * @return */ boolean makesureAuthed(); /** * SDK Require domain name resolution from the upper level. The upper level can implement traditional DNS resolution or its own domain name/IP mapping * @param host * @return */ String[] onNewDns(final String host); /** * Received message from SVR PUSH * @param cmdid * @param data */ void onPush(final int cmdid, final byte[] data); /** * SDK Require upper level to TASK group packages * @param taskID Task Identification * @param userContext * @param reqBuffer BUFFER for Group Packages * @param errCode Error codes for packages * @return */ boolean req2Buf(final int taskID, Object userContext, ByteArrayOutputStream reqBuffer, int[] errCode, int channelSelect); /** * SDK Require upper level to unpack TASK * @param taskID Task Identification * @param userContext * @param respBuffer BUFFER to unpack * @param errCode Error code for unpacking * @return int */ int buf2Resp(final int taskID, Object userContext, final byte[] respBuffer, int[] errCode, int channelSelect); /** * Task End Callback * @param taskID Task Identification * @param userContext * @param errType Error Type * @param errCode Error Code * @return */ int onTaskEnd(final int taskID, Object userContext, final int errType, final int errCode); /** * Traffic Statistics * @param send * @param recv */ void trafficData(final int send, final int recv); /** * Connection Status Notification * @param status The state of synthesis, that is, the state of long company + short company * @param longlinkstatus Long-running status only */ void reportConnectInfo(int status, int longlinkstatus); /** * SDK Requires upper-level generation of long-Link data checkpackages to be used after long-Link connections to authenticate SVR * @param identifyReqBuf Check package data content * @param hashCodeBuffer HASH of Check Pack * @param reqRespCmdID CMD ID for data validation * @return ECHECK_NOW(Need checking, ECHECK_NEVER (no checking), ECHECK_NEXT (ask again next time) */ int getLongLinkIdentifyCheckBuffer(ByteArrayOutputStream identifyReqBuf, ByteArrayOutputStream hashCodeBuffer, int[] reqRespCmdID); /** * SDK Require upper level disconnection check backpack. * @param buffer SVR Connection Check Pack Replied * @param hashCodeBuffer CLIENT HASH value of the requested connection checkpack * @return */ boolean onLongLinkIdentifyResp(final byte[] buffer, final byte[] hashCodeBuffer); /** * Request to sync */ void requestDoSync(); String[] requestNetCheckShortLinkHosts(); /** * Whether to sign in * @return true Login false not logged in */ boolean isLogoned(); void reportTaskProfile(String taskString); }
You can see that they are callbacks. Through mars callbacks, MarsServiceStub receives taskend and executes:
@Override public int onTaskEnd(int taskID, Object userContext, int errType, int errCode) { final MarsTaskWrapper wrapper = TASK_ID_TO_WRAPPER.remove(taskID); if (wrapper == null) { Log.w(TAG, "stn task onTaskEnd callback may fail, null wrapper, taskID=%d", taskID); return 0; // TODO: ??? } try { wrapper.onTaskEnd(errType, errCode); } catch (RemoteException e) { e.printStackTrace(); } finally { WRAPPER_TO_TASK_ID.remove(wrapper); // onTaskEnd will be called only once for each task } return 0; }
Remove the task from the map and execute task's own onTaskEnd.This allows us to see the code for subsequent updates to the ui in our original updateConversationTopics.
Next, we're going back to the updateConversationTopics to see NanoMars TaskWrapper:
public abstract class NanoMarsTaskWrapper<T extends MessageNano, R extends MessageNano> extends AbstractTaskWrapper { private static final String TAG = "Mars.Sample.NanoMarsTaskWrapper"; protected T request; protected R response; public NanoMarsTaskWrapper(T req, R resp) { super(); this.request = req; this.response = resp; } @Override public byte[] req2buf() { try { onPreEncode(request); final byte[] flatArray = new byte[request.getSerializedSize()]; final CodedOutputByteBufferNano output = CodedOutputByteBufferNano.newInstance(flatArray); request.writeTo(output); Log.d(TAG, "encoded request to buffer, [%s]", MemoryDump.dumpHex(flatArray)); return flatArray; } catch (Exception e) { e.printStackTrace(); } return new byte[0]; } @Override public int buf2resp(byte[] buf) { try { Log.d(TAG, "decode response buffer, [%s]", MemoryDump.dumpHex(buf)); response = MessageNano.mergeFrom(response, buf); onPostDecode(response); return StnLogic.RESP_FAIL_HANDLE_NORMAL; } catch (Exception e) { Log.e(TAG, "%s", e); } return StnLogic.RESP_FAIL_HANDLE_TASK_END; } public abstract void onPreEncode(T request); public abstract void onPostDecode(R response);
}
1. inherited from AbstractTaskWrapper;
2. Save both request and response, which are of type MessageNano (message data class in google's protobuf);
3. Two interfaces are implemented to convert request s to bufs and bufs to response s.In fact, the object is converted to byte[], and byte to object;
3. During req2buf conversion, the writeTo method of request was called;
4. In buf2resp, the call to MessageNano.mergeFrom actually ends up calling the mergeFrom of the response, see below:
/** * Parse {@code data} as a message of this type and merge it with the * message being built. */ public static final <T extends MessageNano> T mergeFrom(T msg, final byte[] data) throws InvalidProtocolBufferNanoException { return mergeFrom(msg, data, 0, data.length); }
You can see from the four points above that this is a serialization and deserialization process.google's open source protobuf is of no concern to us, but what we need to know is that it uses a configuration file with the proto suffix name to generate code for the class at compile time.
So what is the role of this base class of AbstractTaskWrapper?
public abstract class AbstractTaskWrapper extends MarsTaskWrapper.Stub { private Bundle properties = new Bundle(); public AbstractTaskWrapper() { // Reflects task properties final TaskProperty taskProperty = this.getClass().getAnnotation(TaskProperty.class); if (taskProperty != null) { setHttpRequest(taskProperty.host(), taskProperty.path()); setShortChannelSupport(taskProperty.shortChannelSupport()); setLongChannelSupport(taskProperty.longChannelSupport()); setCmdID(taskProperty.cmdID()); } } @Override public Bundle getProperties() { return properties; } @Override public abstract void onTaskEnd(int errType, int errCode); public AbstractTaskWrapper setHttpRequest(String host, String path) { properties.putString(MarsTaskProperty.OPTIONS_HOST, ("".equals(host) ? null : host)); properties.putString(MarsTaskProperty.OPTIONS_CGI_PATH, path); return this; } public AbstractTaskWrapper setShortChannelSupport(boolean support) { properties.putBoolean(MarsTaskProperty.OPTIONS_CHANNEL_SHORT_SUPPORT, support); return this; } public AbstractTaskWrapper setLongChannelSupport(boolean support) { properties.putBoolean(MarsTaskProperty.OPTIONS_CHANNEL_LONG_SUPPORT, support); return this; } public AbstractTaskWrapper setCmdID(int cmdID) { properties.putInt(MarsTaskProperty.OPTIONS_CMD_ID, cmdID); return this; } @Override public String toString() { return "AbsMarsTask: " + BundleFormat.toString(properties); }
}
Simply, it provides some interfaces to set the transport protocol types, such as long or short connections, http, and so on.
In summary, this demo uses a separate service framework to guarantee transmission; uses a task system to host each transmission and response; a large number of callbacks to monitor key points in the operation process; encapsulates a separate jar wrapper for easy change and use at the upper level; and introduces a separate configuration class to support http and tThe use of CP long and short connections; the introduction of protobuf greatly improves the efficiency of serialization and deserialization, and reduces the size of data transferred;
This is for the time being, and we'll take a closer look at the core of mars later.