background

Last November, our team announced its dissolution, but due to the problem of privacy supervision, we have to maintain the last version to meet the requirements of privacy supervision.
I am mainly responsible for the privacy issues of our team. Looking back on this year, I was really tossed about by various privacy issues,
Fortunately, the last version has finally been launched. Now we summarize various ways to deal with privacy issues.

Privacy detection process

Process 1.0

At the beginning of being asked for privacy detection, our process is as follows:

Submit the privacy test before ordering the package, and then make corresponding changes according to the test results. However, during the actual operation, the following problems are found:

1. After submitting the privacy detection, a new code is submitted, which may cause new privacy problems;
2. If the problem is detected and then changed, the test time is short and the risk is high, which may cause the delay of package ordering;
3. Modify before ordering the package, and there is not enough manpower to support the test;

Therefore, after discussion with PM, we made some changes to the process.

Process 2.0

We submit privacy detection during the system test after the package is ordered for each version, and then convert the detected problems into the requirements of the next version.

However, after a period of implementation, it is found that there are still problems.
Our team is divided into two teams: Beijing and Shenzhen. Generally, we issue a version every two weeks. Shenzhen and Beijing release it alternately. For example, Shenzhen releases 4.7.0, and then Beijing releases 4.7.5.
Usually, after we wait for the privacy detection results, the requirements for the next version are already arranged. As shown in the figure below, the privacy test takes about three days. When we submit the privacy test for version 4.7.0 and get the test results, the requirements for version 4.7.5 have been scheduled for a long time. At this time, it is difficult to follow version 4.7.5, so we can only schedule the requirements into version 4.8.0.

Therefore, the second process still has the following problems:

1. The distribution of versions is frequent. The problems detected in 470 cannot be solved in 475 and need to be delayed to 480;
2. 470 detected problems, 475 detected the same problems, wasting detection resources;
3. During the system test, it is necessary to manually submit privacy detection to increase the test workload;

After discussing with our privacy detection colleagues, we made some changes to the process.

Process 3.0

Since privacy detection cannot be avoided, it is better to embrace change actively than passively accepting detection;

It includes the following items:

1. We still use process 2.0 to solve the privacy problem of stock;
2. In view of the new requirements, after discussion with PM, we require to increase privacy self-examination in the stage of product internal audit and requirements review to expose privacy issues in advance as much as possible, including the following:

	Inspection items
1	Whether the requirement needs to call permission, whether it has permission or has not been marked before. At the same time, it needs to state the process after the user refuses authorization in the requirement document
2	Whether the requirement will access a new sdk needs to be specified
3	Whether there is any other data sharing / sharing of this demand, which needs to be supplemented by words

1. Pipeline introduces privacy detection
   We added the step of submitting privacy detection in the release pipeline,
   This step will be executed after the first automation test is passed. It will first judge whether the version has been submitted for testing. If it has been submitted, it will not be submitted. If not, the corresponding apk will be uploaded to the testing platform.
   

In addition to process problems, there are also various technical problems.

Various postures to deal with privacy detection

1. Obtain data and frequency in advance

In view of the situation of obtaining data in advance, I created a class PrivacyInit,
When the user is not authorized, first put the code to be initialized into the privacy callback, and then initialize after the user is authorized;
If the user has been authorized, it can be executed directly.

//Call after user authorization
public static void privacyAllInit() {
    for (PrivacyCallBack callBack : callBackList) {
        try {
            callBack.call();
        } catch (Exception e) {
            e.printStackTrace();
            ABCLog.e(TAG, e.getMessage());
        }
    }
}

//Judge whether to authorize or not. If not, do not initialize first
public static void privacyInit(PrivacyCallBack callBack) {
    if (isAuthorized()) {
        try {
            if (callBack != null) {
                callBack.call();
            }
        } catch (Exception e) {
            e.printStackTrace();
            ABCLog.e(TAG, e.getMessage());
        }
    } else {
        callBackList.add(callBack);
    }
}

//Privacy callback interface
public interface PrivacyCallBack {
    void call();
}

The problem of acquisition frequency is relatively simple. You can directly use a global static variable or save the relevant private data in a single instance to avoid multiple acquisition.

2. Authority issues

The new privacy specification requires us to give a pop-up prompt every time we use a permission, and then we can really apply for permission.
If you overwrite and modify each scenario, it will obviously cost a lot, but fortunately, you have done a collapse operation for permission applications before.
Android native application permission is as follows:
Usually, we apply for permission somewhere, and then get the result of permission application in the corresponding onRequestPermissionsResult

public class xxxActivity extends AppCompatActivity {
    
    protected void onCreate(@Nullable Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        //Apply for permission
        this.requestPermissions(permissions, 100);
    }

    @Override
    public void onRequestPermissionsResult(int requestCode, @NonNull String[] permissions,
            @NonNull int[] grantResults) {
            //Obtain the result of permission application     
    }
}

The disadvantage of this is that the permission application and acquisition results are scattered in different places. In addition, different activities apply for permission, which is not convenient for unified management.
Therefore, we have created a transparent Activity to apply for permission, and then the applied results are returned through callback.

<activity
    android:name="***.***.***.PermissionActivity"
    android:theme="@style/Transparent" >
</activity>

The final results are as follows:

//Apply for permission
PermissionHelper.with(context).permission(Manifest.permission.CAMERA)
        .callBack((int requestCode, String[] permissions, int[] grantResults) -> {
            //Get the result of permission application
        }).request();

At the same time, because all permission applications are gathered in one place, it is convenient to deal with all kinds of fancy needs of permission supervision.

3. sdk problem

If we detect a privacy problem with the sdk, we will directly find the corresponding team and ask them to help solve it and update the version of the sdk.
However, there are some special cases. For example, the sdk is no longer maintained.

Decompile

In this case, my heart breaks down. If I remove the sdk, it may cause exceptions to the existing functions. If I introduce a new sdk, it will cost a lot to change the scene of the last version,
How to solve privacy problems at the least cost?
I thought of decompilation.
You can decompile the jar package into smali, modify smali, and then compile it into a new jar package, replacing the original jar package.
However, this is not a conventional means. Use it with caution!

Ultimate trick

Decompilation is not a conventional method after all, and smali code needs to be modified, and the corresponding syntax is not familiar, but it doesn't matter. We can also use Transfrom to directly modify the code in the sdk.
Generally, the privacy problems we encounter with sdk can be divided into two categories: accessing privacy related variables or accessing privacy related methods.

override fun onCreate(savedInstanceState: Bundle?) {
    super.onCreate(savedInstanceState)
    setContentView(R.layout.activity_main)
    var btn = findViewById<Button>(R.id.btn)
    btn.setOnClickListener {
        println("getdevice:"+Build.DEVICE)//Access to privacy related variables
        println("getImei:"+getImei())//Access to privacy related methods
    }
}

1. Access privacy related variables
In this case, we can copy the visitFieldInsn method in the AdviceAdapter. When accessing the corresponding field, we can directly return the variables set by ourselves

public void visitFieldInsn(int opcode, String owner, String name, String descriptor) {
    if ("android.os.Build".equals(owner) || "DEVICE".equals(name) ) {
        //Replace with our own variables
        super.visitFieldInsn(opcode, "***/***/***/Privacy", "device", "Ljava/lang/String;");
    } else {
        super.visitFieldInsn(opcode, owner, name, descriptor);
    }
}

After replacement, we can find that visit Android os. Build. The device field has been replaced with privacy Device

2. Access privacy related methods
Assume mainactivity Getimei () method is a privacy related method. We want to replace it with privacy Newgetimei() method,
We can duplicate the visitMethodInsn method in the AdviceAdapter, as shown below, which can be replaced by the method we want.

@Override
public void visitMethodInsn(int opcode, String owner, String methodName, String descriptor, boolean isInterface) {
    if ("getImei".equals(methodName) && "()Ljava/lang/String;".equals(descriptor)&&""***/"***/"***/MainActivity".equals(owner)) {
        mv.visitMethodInsn(Opcodes.INVOKESTATIC, "***/"***/"***/Privacy", "newGetImei", "()Ljava/lang/String;", false);
    } else {
        super.visitMethodInsn(opcode, owner, methodName, descriptor, isInterface);
    }
}

After replacement, we found that the access method has become privacy Newgetimei()

summary

Privacy compliance has come to an end. What makes me collapse is that the privacy policy has been changing. For example, Android ID was not included in the privacy field at the beginning, followed by, and then the access frequency was added to the detection.
In addition, the problems detected by privacy detection are different every time. For example, the ABC problem is detected for the first time. After we change it, we detect the DE problem, which makes it difficult to converge.
However, the darkest moment has passed.
In addition, I would like to thank the product students who have been fighting side by side. Although the running water has been sent to other departments, we have persisted to the end. I feel that we have made a revolutionary friendship 😄 .