Principle of Flutter and practice of meituan

Alibaba P7 mobile Internet architect advanced video (in daily update) for free, please click: https://space.bilibili.com/474380680

Take out full category page practice

After investigating the features and implementation principles of Flutter, the take out plan is to launch the full category page of Flutter in grayscale. For the integration mode of using the fluent page as a part of the App, it is not officially provided Perfect support , so we first need to understand how Flutter is compiled, packaged, and running.

Flutter App build process

The simplest shuttle project consists of at least two files:

When running the Flutter program, you need the hosting project of the corresponding platform. On Android, Flutter automatically creates a Gradle project to generate the host, and executes the flitter create. In the project directory, Flutter will create two directories, ios and Android, respectively, to build the hosting project of the corresponding platform. The contents of Android directory are as follows:

There is only one app module in this Gradle project, and the building product is the host APK. When Flutter is running locally, Debug mode is used by default. After executing flitter run in the project directory, it can be installed in the device and run automatically. In Debug mode, Flutter uses JIT mode to execute Dart code. All Dart codes will be packaged in the APK file under the assets directory, read and executed by DartVM provided in libflitter.so:

kernel_blob.bin is the underlying interface of the Flutter engine and the basic function code of Dart language

third_party/dart/runtime/bin/*.dart
third_party/dart/runtime/lib/*.dart
third_party/dart/sdk/lib/_http/*.dart
third_party/dart/sdk/lib/async/*.dart
third_party/dart/sdk/lib/collection/*.dart
third_party/dart/sdk/lib/convert/*.dart
third_party/dart/sdk/lib/core/*.dart
third_party/dart/sdk/lib/developer/*.dart
third_party/dart/sdk/lib/html/*.dart
third_party/dart/sdk/lib/internal/*.dart
third_party/dart/sdk/lib/io/*.dart
third_party/dart/sdk/lib/isolate/*.dart
third_party/dart/sdk/lib/math/*.dart
third_party/dart/sdk/lib/mirrors/*.dart
third_party/dart/sdk/lib/profiler/*.dart
third_party/dart/sdk/lib/typed_data/*.dart
third_party/dart/sdk/lib/vmservice/*.dart
flutter/lib/ui/*.dart

platform.dill is the code that implements the page logic, including the Flutter Framework and other library codes dependent on pub:

flutter_tutorial_2/lib/main.dart
flutter/packages/flutter/lib/src/widgets/*.dart
flutter/packages/flutter/lib/src/services/*.dart
flutter/packages/flutter/lib/src/semantics/*.dart
flutter/packages/flutter/lib/src/scheduler/*.dart
flutter/packages/flutter/lib/src/rendering/*.dart
flutter/packages/flutter/lib/src/physics/*.dart
flutter/packages/flutter/lib/src/painting/*.dart
flutter/packages/flutter/lib/src/gestures/*.dart
flutter/packages/flutter/lib/src/foundation/*.dart
flutter/packages/flutter/lib/src/animation/*.dart
.pub-cache/hosted/pub.flutter-io.cn/collection-1.14.6/lib/*.dart
.pub-cache/hosted/pub.flutter-io.cn/meta-1.1.5/lib/*.dart
.pub-cache/hosted/pub.flutter-io.cn/shared_preferences-0.4.2/*.dart

Both kernel_blob.bin and platform.dill are created by the bundle.dart Call in KernelCompiler Generation.

In Release mode (flitter run -- Release), flitter will use Dart's AOT running mode, and convert Dart code into ARM instruction during compilation:

Kernel ﹣ blob.bin and platform.dill are not in the packaged APK. Instead, they are (isolate / VM) ﹣ snapshot ﹣ four files (data / instr). The snapshot file is generated by the flutter / bin / cache / artifacts / engine / Android arm release / darwin-x64 / gen_snapshot command in the Flutter SDK. vm_snapshot ﹣ is the data and code instructions required for the operation of the Dart virtual machine, while isolate﹣ snapshot ﹣ is the data and code instructions required for each isolate operation.

Running mechanism of Flutter App

The APK built by flutter will extract all the resource files under the assets directory to the flitter directory in the App private file directory at runtime, mainly including the icudtl.dat processing character encoding, as well as the four snapshot files under the Debug mode of kernel_blob.bin, platform.dill and Release mode. By default, Flutter calls FlutterMain#startInitialization to start the decompression task at Application#onCreate, and then calls FlutterMain#ensureInitializationComplete in FlutterActivityDelegate#onCreate to wait for the end of the decompression task.

The JIT execution mode of Flutter in Debug mode is mainly to support the popular hot refresh function:

When the touch heat refresh occurs, the Flutter will detect the changed Dart file and synchronize it to the App private cache directory. The DartVM loads and modifies the corresponding class or method. After the control tree is rebuilt, the effect can be seen on the device immediately.

In Release mode, the shuttle will directly map the snapshot file to the memory to execute its instructions:

In the Release mode, calling the FlutterMain#ensureInitializationComplete method in FlutterActivityDelegate#onCreate will set the snapshot parameters set up in AndroidManifest (the default settings mentioned above) and the file name and other running parameters to the corresponding C++ class name object. When nativeAttach instance is constructed, nativeAttach is used to initialize DartVM. Compiled Dart code.

Package Android Library

After understanding the construction and operation mechanism of the Flutter project, we can package it into AAR according to its requirements and then integrate it into the existing native App. First, modify it in andorid/app/build.gradle:

After simple modification, we can use Android Studio or Gradle command-line tools to package the Flutter code into aar. All the resources needed by the Flutter runtime will be included in aar. After it is published to maven server or local maven warehouse, it can be referenced in the native App project.

But this is only the first step of integration. In order to make the fluent page seamlessly connect to the take out App, we need to do a lot more.

Picture resource reuse

By default, Flutter packs all image resource files into the assets directory, but we do not use Flutter to develop new pages. Image resources were originally placed in each drawable directory according to Android specifications. Even new pages will have many scenarios of image resource reuse, so it is not appropriate to add new image resources in the assets directory.

Flutter doesn't provide a way to directly call the image resources under the drawable directory. After all, the processing of drawable files will involve a lot of Android platform related logic (screen density, system version, language, etc.), and the reading operation of assets directory files is also implemented in the engine using C + +, so it is difficult to read the drawable files at the Dart level. Flutter also supports adding when processing files in the assets directory Multiple rate picture resources , and can be used when Automatic selection , but Flutter requires that each picture must be provided with a 1x picture before it can recognize the pictures under the corresponding other magnification Directory:

flutter:
  assets:
    - images/cat.png
    - images/2x/cat.png
    - images/3.5x/cat.png

new Image.asset('images/cat.png');

After this configuration, the corresponding density pictures can be correctly used on devices with different resolutions. However, in order to reduce the APK package volume, our bitmap resources generally only provide the commonly used 2x resolution, and other resolution devices will automatically scale to the corresponding size at runtime. For this special case, we also provide the same capabilities as the native App without increasing the package volume:

  1. Before calling the shutter page, scale the specified image resources according to the screen density of the device, and store them in the App private directory.
  2. The user-defined WMImage control is used to load in Flutter. In fact, it is loaded by converting to FileImage and automatically setting scale to devicePixelRatio.

In this way, we can solve the problem of both the APK package size and the lack of image resources.

Communication between Flutter and native code

We only use Flutter to implement a page, and a large number of existing logics are implemented in Java. When running, there will be many scenarios that must use the logic and functions in the native application, such as network requests. Our unified network library will add many common parameters to each network request, and it will also be responsible for the monitoring of success rate and other indicators, as well as exception reporting. We need to capture Report its stack and environment information to the server when critical exceptions arrive. It is unlikely that these functions can be implemented in one set by Dart immediately, so we need to use the Platform Channel function provided by Dart to realize the mutual call between Dart and Java.

Taking the network request as an example, we define a MethodChannel object in Dart:

import 'dart:async';
import 'package:flutter/services.dart';
const MethodChannel _channel = const MethodChannel('com.sankuai.waimai/network');
Future<Map<String, dynamic>> post(String path, [Map<String, dynamic> form]) async {
  return _channel.invokeMethod("post", {'path': path, 'body': form}).then((result) {
    return new Map<String, dynamic>.from(result);
  }).catchError((_) => null);
}

Then implement the MethodChannel with the same name on the Java side:

public class FlutterNetworkPlugin implements MethodChannel.MethodCallHandler {

    private static final String CHANNEL_NAME = "com.sankuai.waimai/network";

    @Override
    public void onMethodCall(MethodCall methodCall, final MethodChannel.Result result) {
        switch (methodCall.method) {
            case "post":
                RetrofitManager.performRequest(post((String) methodCall.argument("path"), (Map) methodCall.argument("body")),
                        new DefaultSubscriber<Map>() {
                            @Override
                            public void onError(Throwable e) {
                                result.error(e.getClass().getCanonicalName(), e.getMessage(), null);
                            }

                            @Override
                            public void onNext(Map stringBaseResponse) {
                                result.success(stringBaseResponse);
                            }
                        }, tag);
                break;

            default:
                result.notImplemented();
                break;
        }
    }
}

After registering in the Flutter page, you can call the corresponding Java implementation by calling the post method:

loadData: (callback) async {
    Map<String, dynamic> data = await post("home/groups");
    if (data == null) {
      callback(false);
      return;
    }
    _data = AllCategoryResponse.fromJson(data);
    if (_data == null || _data.code != 0) {
      callback(false);
      return;
    }
    callback(true);
  }),

SO library compatibility

Flutter officially provides only four CPU architecture SO libraries: armeabi-v7a, arm64-v8a, x86 and x86-64. Among them, x86 series only supports Debug mode, but a large number of SDK s used for takeout only provide armeabi architecture libraries.

Although we can compile the Fletter engine of armeabi version by modifying the configuration files of build/config/arm.gni and BUILD.gn under the src root directory and the third party / dart directory of the engine, in fact, most of the devices on the market already support armeabi-v7a, and the hardware acceleration floating-point operation instructions provided by it can greatly improve the running speed of the Fletter In the grayscale stage, we can actively shield the devices that do not support armeabi-v7a, and directly use the engine of armeabi-v7a version.

To do this, we first need to modify the engine provided by Flutter. Under bin/cache/artifacts/engine in the installation directory of Flutter, there are engines of all platforms downloaded by Flutter:

We just need to modify flitter.jar under Android arm, Android arm profile and Android arm release, and move lib / armeabi-v7a / libflitter.so to Lib / armeabi / libflitter.so:

cd $FLUTTER_ROOT/bin/cache/artifacts/engine
for arch in android-arm android-arm-profile android-arm-release; do
  pushd $arch
  cp flutter.jar flutter-armeabi-v7a.jar # backups
  unzip flutter.jar lib/armeabi-v7a/libflutter.so
  mv lib/armeabi-v7a lib/armeabi
  zip -d flutter.jar lib/armeabi-v7a/libflutter.so
  zip flutter.jar lib/armeabi/libflutter.so
  popd
done

In this way, after packing, the SO Library of Flutter will be printed to the lib/armeabi directory of APK. If the device does not support armeabi-v7a at runtime, it may crash, SO we need to actively identify and block such devices. It is also very simple to judge whether the device supports armeabi-v7a on Android:

public static boolean isARMv7Compatible() {
    try {
        if (SDK_INT >= LOLLIPOP) {
            for (String abi : Build.SUPPORTED_32_BIT_ABIS) {
                if (abi.equals("armeabi-v7a")) {
                    return true;
                }
            }
        } else {
            if (CPU_ABI.equals("armeabi-v7a") || CPU_ABI.equals("arm64-v8a")) {
                return true;
            }
        }
    } catch (Throwable e) {
        L.wtf(e);
    }
    return false;
}

Grayscale and auto degradation strategy

Horn is a cross platform SDK distributed in meituan. You can easily specify the gray level switch by using horn:

On the condition configuration page, define a series of conditions, and then add a new field fluent on the parameter configuration page

The ABI rules defined here are not enabled because the ABI policy is implemented on the client.

At present, Flutter is still in the Beta stage, and it is inevitable to Crash during the gray-scale process. It is observed that after the Crash, we can degrade the model or device ID as much as possible, but we can do it more quickly. The take out Crash collection SDK also supports JNI Crash collection. We specially registered a Crash listener for the Flutter. Once JNI Crash related to the Flutter is collected, the Flutter function of the device will be stopped immediately. Before starting the Flutter, the existence of the flitter_native_Crash_flag file will be determined first. If it exists, it means that the device has experienced a Crash related to the Flutter, which is likely to be Problems caused by incompatibility. In the current version cycle, the Flutter function will no longer be used on this device.

In addition to crashing, dart code in the Flutter page may also have exceptions, such as parsing failure caused by data format error issued by the server. Dart also provides global exception capture function:

import 'package:wm_app/plugins/wm_metrics.dart';

void main() {
  runZoned(() => runApp(WaimaiApp()), onError: (Object obj, StackTrace stack) {
    uploadException("$obj\n$stack");
  });
}

In this way, we can achieve all-round abnormal monitoring and perfect degradation strategy, and minimize the possible impact on the user when the gray level.

Analyze crash stack and exception data

The engine part of Flutter is implemented in C/C + +. In order to reduce the package size, all SO libraries will remove the symbol table information when they are published. Like other JNI crash stacks, only memory address offset and other information can be seen in the reported stack information:

*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
Build fingerprint: 'Rock/odin/odin:7.1.1/NMF26F/1527007828:user/dev-keys'
Revision: '0'
Author: collect by 'libunwind'
ABI: 'arm64-v8a'
pid: 28937, tid: 29314, name: 1.ui  >>> com.sankuai.meituan.takeoutnew <<<
signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x0

backtrace:
    r0 00000000  r1 ffffffff  r2 c0e7cb2c  r3 c15affcc
    r4 c15aff88  r5 c0e7cb2c  r6 c15aff90  r7 bf567800
    r8 c0e7cc58  r9 00000000  sl c15aff0c  fp 00000001
    ip 80000000  sp c0e7cb28  lr c11a03f9  pc c1254088  cpsr 200c0030
    #00 pc 002d7088  /data/app/com.sankuai.meituan.takeoutnew-1/lib/arm/libflutter.so
    #01 pc 002d5a23  /data/app/com.sankuai.meituan.takeoutnew-1/lib/arm/libflutter.so
    #02 pc 002d95b5  /data/app/com.sankuai.meituan.takeoutnew-1/lib/arm/libflutter.so
    #03 pc 002d9f33  /data/app/com.sankuai.meituan.takeoutnew-1/lib/arm/libflutter.so
    #04 pc 00068e6d  /data/app/com.sankuai.meituan.takeoutnew-1/lib/arm/libflutter.so
    #05 pc 00067da5  /data/app/com.sankuai.meituan.takeoutnew-1/lib/arm/libflutter.so
    #06 pc 00067d5f  /data/app/com.sankuai.meituan.takeoutnew-1/lib/arm/libflutter.so
    #07 pc 003b1877  /data/app/com.sankuai.meituan.takeoutnew-1/lib/arm/libflutter.so
    #08 pc 003b1db5  /data/app/com.sankuai.meituan.takeoutnew-1/lib/arm/libflutter.so
    #09 pc 0000241c  /data/data/com.sankuai.meituan.takeoutnew/app_flutter/vm_snapshot_instr

It's difficult to locate the problem simply with these information, so we need to use NDK stack provided by NDK to analyze the specific code location:

ndk-stack -sym PATH [-dump PATH]
Symbolizes the stack trace from an Android native crash.
  -sym PATH   sets the root directory for symbols
  -dump PATH  sets the file containing the crash dump (default stdin)

If a customized engine is used, the libflitter.so file compiled under engine / SRC / out / Android release must be used. In general, we use the official version of the engine, which can be flutter_infra The SO file with symbol table can be downloaded directly from the page, and the corresponding file can be downloaded according to the version of Flutter tool used in packaging. For example, the 0.4.4 beta version:

$ flutter --version # The version command can see the version 06afdfe54e corresponding to the Engine
Flutter 0.4.4 • channel beta • https://github.com/flutter/flutter.git
Framework • revision f9bb4289e9 (5 weeks ago) • 2018-05-11 21:44:54 -0700
Engine • revision 06afdfe54e
Tools • Dart 2.0.0-dev.54.0.flutter-46ab040e58
$ cat flutter/bin/internal/engine.version # You can also see the complete version information in the engine.version file in the flitter installation directory 06afdfe54ebef9168a90ca00a6721c2d36e6aafa
06afdfe54ebef9168a90ca00a6721c2d36e6aafa

After getting the engine version number https://console.cloud.google.com/storage/browser/flutter_infra/flutter/06afdfe54ebef9168a90ca00a6721c2d36e6aafa/ See all the construction products corresponding to this version, download the symbols.zip under the android-arm-release, android-arm64-release and android-x86 directories, and store it in the corresponding directory:

Execute NDK stack to see the actual crash code and specific line number information:

ndk-stack -sym flutter-production-syms/06afdfe54ebef9168a90ca00a6721c2d36e6aafa/armeabi-v7a -dump flutter_jni_crash.txt 
********** Crash dump: **********
Build fingerprint: 'Rock/odin/odin:7.1.1/NMF26F/1527007828:user/dev-keys'
pid: 28937, tid: 29314, name: 1.ui  >>> com.sankuai.meituan.takeoutnew <<<
signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x0
Stack frame #00 pc 002d7088  /data/app/com.sankuai.meituan.takeoutnew-1/lib/arm/libflutter.so: Routine minikin::WordBreaker::setText(unsigned short const*, unsigned int) at /b/build/slave/Linux_Engine/build/src/out/android_release/../../flutter/third_party/txt/src/minikin/WordBreaker.cpp:55
Stack frame #01 pc 002d5a23  /data/app/com.sankuai.meituan.takeoutnew-1/lib/arm/libflutter.so: Routine minikin::LineBreaker::setText() at /b/build/slave/Linux_Engine/build/src/out/android_release/../../flutter/third_party/txt/src/minikin/LineBreaker.cpp:74
Stack frame #02 pc 002d95b5  /data/app/com.sankuai.meituan.takeoutnew-1/lib/arm/libflutter.so: Routine txt::Paragraph::ComputeLineBreaks() at /b/build/slave/Linux_Engine/build/src/out/android_release/../../flutter/third_party/txt/src/txt/paragraph.cc:273
Stack frame #03 pc 002d9f33  /data/app/com.sankuai.meituan.takeoutnew-1/lib/arm/libflutter.so: Routine txt::Paragraph::Layout(double, bool) at /b/build/slave/Linux_Engine/build/src/out/android_release/../../flutter/third_party/txt/src/txt/paragraph.cc:428
Stack frame #04 pc 00068e6d  /data/app/com.sankuai.meituan.takeoutnew-1/lib/arm/libflutter.so: Routine blink::ParagraphImplTxt::layout(double) at /b/build/slave/Linux_Engine/build/src/out/android_release/../../flutter/lib/ui/text/paragraph_impl_txt.cc:54
Stack frame #05 pc 00067da5  /data/app/com.sankuai.meituan.takeoutnew-1/lib/arm/libflutter.so: Routine tonic::DartDispatcher<tonic::IndicesHolder<0u>, void (blink::Paragraph::*)(double)>::Dispatch(void (blink::Paragraph::*)(double)) at /b/build/slave/Linux_Engine/build/src/out/android_release/../../topaz/lib/tonic/dart_args.h:150
Stack frame #06 pc 00067d5f  /data/app/com.sankuai.meituan.takeoutnew-1/lib/arm/libflutter.so: Routine void tonic::DartCall<void (blink::Paragraph::*)(double)>(void (blink::Paragraph::*)(double), _Dart_NativeArguments*) at /b/build/slave/Linux_Engine/build/src/out/android_release/../../topaz/lib/tonic/dart_args.h:198
Stack frame #07 pc 003b1877  /data/app/com.sankuai.meituan.takeoutnew-1/lib/arm/libflutter.so: Routine dart::NativeEntry::AutoScopeNativeCallWrapperNoStackCheck(_Dart_NativeArguments*, void (*)(_Dart_NativeArguments*)) at /b/build/slave/Linux_Engine/build/src/out/android_release/../../third_party/dart/runtime/vm/native_entry.cc:198
Stack frame #08 pc 003b1db5  /data/app/com.sankuai.meituan.takeoutnew-1/lib/arm/libflutter.so: Routine dart::NativeEntry::LinkNativeCall(_Dart_NativeArguments*) at /b/build/slave/Linux_Engine/build/src/out/android_release/../../third_party/dart/runtime/vm/native_entry.cc:348
Stack frame #09 pc 0000241c  /data/data/com.sankuai.meituan.takeoutnew/app_flutter/vm_snapshot_instr

Dart exceptions are relatively simple. By default, the symbol table information is not removed when the dart code is compiled into machine code. Therefore, the exception stack of dart itself can identify the code file and line number information of the real exception

FlutterException: type '_InternalLinkedHashMap<dynamic, dynamic>' is not a subtype of type 'num' in type cast
#0      _$CategoryGroupFromJson (package:wm_app/lib/all_category/model/category_model.g.dart:29)
#1      new CategoryGroup.fromJson (package:wm_app/all_category/model/category_model.dart:51)
#2      _$CategoryListDataFromJson.<anonymous closure> (package:wm_app/lib/all_category/model/category_model.g.dart:5)
#3      MappedListIterable.elementAt (dart:_internal/iterable.dart:414)
#4      ListIterable.toList (dart:_internal/iterable.dart:219)
#5      _$CategoryListDataFromJson (package:wm_app/lib/all_category/model/category_model.g.dart:6)
#6      new CategoryListData.fromJson (package:wm_app/all_category/model/category_model.dart:19)
#7      _$AllCategoryResponseFromJson (package:wm_app/lib/all_category/model/category_model.g.dart:19)
#8      new AllCategoryResponse.fromJson (package:wm_app/all_category/model/category_model.dart:29)
#9      AllCategoryPage.build.<anonymous closure> (package:wm_app/all_category/category_page.dart:46)
<asynchronous suspension>
#10     _WaimaiLoadingState.build (package:wm_app/all_category/widgets/progressive_loading_page.dart:51)
#11     StatefulElement.build (package:flutter/src/widgets/framework.dart:3730)
#12     ComponentElement.performRebuild (package:flutter/src/widgets/framework.dart:3642)
#13     Element.rebuild (package:flutter/src/widgets/framework.dart:3495)
#14     BuildOwner.buildScope (package:flutter/src/widgets/framework.dart:2242)
#15     _WidgetsFlutterBinding&BindingBase&GestureBinding&ServicesBinding&SchedulerBinding&PaintingBinding&RendererBinding&WidgetsBinding.drawFrame (package:flutter/src/widgets/binding.dart:626)
#16     _WidgetsFlutterBinding&BindingBase&GestureBinding&ServicesBinding&SchedulerBinding&PaintingBinding&RendererBinding._handlePersistentFrameCallback (package:flutter/src/rendering/binding.dart:208)
#17     _WidgetsFlutterBinding&BindingBase&GestureBinding&ServicesBinding&SchedulerBinding._invokeFrameCallback (package:flutter/src/scheduler/binding.dart:990)
#18     _WidgetsFlutterBinding&BindingBase&GestureBinding&ServicesBinding&SchedulerBinding.handleDrawFrame (package:flutter/src/scheduler/binding.dart:930)
#19     _WidgetsFlutterBinding&BindingBase&GestureBinding&ServicesBinding&SchedulerBinding._handleDrawFrame (package:flutter/src/scheduler/binding.dart:842)
#20     _rootRun (dart:async/zone.dart:1126)
#21     _CustomZone.run (dart:async/zone.dart:1023)
#22     _CustomZone.runGuarded (dart:async/zone.dart:925)
#23     _invoke (dart:ui/hooks.dart:122)
#24     _drawFrame (dart:ui/hooks.dart:109)

Original author: meituan technical team
Original link: https://zhuanlan.zhihu.com/p/41731950
Alibaba P7 mobile Internet architect advanced video (in daily update) for free, please click: https://space.bilibili.com/474380680

Keywords: Operation & Maintenance Android SDK snapshot network

Added by Fakcon on Thu, 12 Dec 2019 08:30:19 +0200