theme: smartblue
Opening
Let's introduce it first Xu Gongda If there is a need for advance, it is recommended to take a look at this series.
You can have a good look at this series of startup optimization. Thank you, Mr. Xu.
I will introduce some more interesting points about DAG and how to adjust it in detail.
At present, the warehouse is still in an iterative state, which is not a particularly stable state, so the article is more to open up some small ideas for you.
Students with ideas can leave messages. I personally feel that an iterative library can evolve continuously.
demo address AndroidStartup
There are many code references in the demo android-startup , thank you, u1s1 this guy is very strong.
Task granularity
This is actually quite important. I believe that many people, after accessing the startup framework, do more to wrap up the previous code with several tasks, which is equivalent to completing the simple startup framework access.
In fact, this basically violates the original intention of the startup framework design. Let me put forward a point first. The startup framework will not really help you speed up the startup speed. The scenario he solves is to make the initialization of your sdk more orderly, so that you can add some new SDKs more safely in the long-term iterative process.
For example, when your buried point framework relies on the network library, the abtest configuration center also relies on the network library, and then the network library depends on dns, etc., and then all businesses rely on the buried point configuration center, picture library, etc. after the initialization of sdk is completed.
Of course, there will be dependency ring problems in extreme cases. At this time, it may be necessary for development students to solve this dependency problem manually. For example, in special cases, the network library needs a unique id, the reporting library depends on the network library, and the reporting library depends on the unique id, and the only id needs to be reported
Therefore, in my personal opinion, the granularity of the startup framework should be refined to the initialization of each sdk. If the granularity can be more detailed, of course, the better. In fact, the general startup framework will count the time consumption of each task, so that it will be easier for us to follow up the corresponding problems later, such as checking whether the time consumption of some tasks has increased.
At present, when designing, we may split the initialization of an sdk into three parts to solve the problem of dependency ring.
Wait between child threads
Previously, it was found that the startup framework in the project only ensures that the sequence of putting threads is executed according to dag. This is ok if only the main thread and the pool size is 1 thread pool. However, if multiple threads are concurrent, this becomes a dangerous operation.
Therefore, we need to add a wait in the concurrent scenario. We must wait until the dependent tasks are completed before continuing to execute the initialization code downward.
The CountDownLatch mechanism is still used. When the dependent tasks are completed, the await will be released and continue to be executed downward. In terms of design, I still adopt the decorator, which can continue to be used without the user changing the original logic.
The code is as follows, which mainly refers to the distribution of a task. If it is found that the current dependency has the task, then latch-1 When the latch reaches 0, the current thread will be released.
class StartupAwaitTask(val task: StartupTask) : StartupTask {
private var dependencies = task.dependencies()
private lateinit var countDownLatch: CountDownLatch
private lateinit var rightDependencies: List<String>
var awaitDuration: Long = 0
override fun run(context: Context) {
val timeUsage = SystemClock.elapsedRealtime()
countDownLatch.await()
awaitDuration = (SystemClock.elapsedRealtime() - timeUsage) / 1000
KLogger.i(
TAG, "taskName:${task.tag()} await costa:${awaitDuration} "
)
task.run(context)
}
override fun dependencies(): MutableList<String> {
return dependencies
}
fun allTaskTag(tags: HashSet<String>) {
rightDependencies = dependencies.filter { tags.contains(it) }
countDownLatch = CountDownLatch(rightDependencies.size)
}
fun dispatcher(taskName: String) {
if (rightDependencies.contains(taskName)) {
countDownLatch.countDown()
}
}
override fun mainThread(): Boolean {
return task.mainThread()
}
override fun await(): Boolean {
return task.await()
}
override fun tag(): String {
return task.tag()
}
override fun onTaskStart() {
task.onTaskStart()
}
override fun onTaskCompleted() {
task.onTaskCompleted()
}
override fun toString(): String {
return task.toString()
}
companion object {
const val TAG = "StartupAwaitTask"
}
}This is not only a supplement to the capability, but also a part of multithreading dependency.
At the same time, the dependency mode is changed from class to tag, but the final design of this place has not been completed. At present, it is still a little unclear. It mainly solves the problem of componentization, which can be more arbitrary.
Thread pool shutdown
Here is my personal consideration. When the whole startup process is over, should I consider turning off the thread pool by default. I found that many of them did not write these, which will cause the leakage of thread usage.
fun dispatcherEnd() {
if (executor != mExecutor) {
KLogger.i(TAG, "auto shutdown default executor")
mExecutor.shutdown()
}
}The code is as above. If the current thread pool is not the incoming thread pool, consider closing the thread pool after execution.
dsl + anchor
Because I am both a developer and a user of the framework. Therefore, in the process of using, I found that there were still many problems in the original design. It was very inconvenient for me to insert a task after all SDKs were completed.
Then I consider this part to write dynamic add task through dsl. kotlin is really fragrant. If there is no sugar in the follow-up development, I guess it will be a loser.
I just jumped from here. The sugar is so delicious.
fun Application.createStartup(): Startup.Builder = run {
startUp(this) {
addTask {
simpleTask("taskA") {
info("taskA")
}
}
addTask {
simpleTask("taskB") {
info("taskB")
}
}
addTask {
simpleTask("taskC") {
info("taskC")
}
}
addTask {
simpleTaskBuilder("taskD") {
info("taskD")
}.apply {
dependOn("taskC")
}.build()
}
addTask("taskC") {
info("taskC")
}
setAnchorTask {
MyAnchorTask()
}
addTask {
asyncTask("asyncTaskA", {
info("asyncTaskA")
}, {
dependOn("asyncTaskD")
})
}
addAnchorTask {
asyncTask("asyncTaskB", {
info("asyncTaskB")
}, {
dependOn("asyncTaskA")
await = true
})
}
addAnchorTask {
asyncTaskBuilder("asyncTaskC") {
info("asyncTaskC")
sleep(1000)
}.apply {
await = true
dependOn("asyncTaskE")
}.build()
}
addTaskGroup { taskGroup() }
addTaskGroup { StartupTaskGroupApplicationKspMain() }
addMainProcTaskGroup { StartupTaskGroupApplicationKspAll() }
addProcTaskGroup { StartupProcTaskGroupApplicationKsp() }
}
}This DSL writing method is suitable for inserting some simple tasks, which can be tasks without dependencies, or you just want to write like this. The advantage is that you can avoid writing too much redundant code in the form of inheritance, and then you can see what you have done in this startup process.
Generally, several anchor points will be set up after the project is stable. Their role is to perform the follow-up tasks as long as they are attached to the anchor point, and set some standards so that the follow-up students can access them more quickly.
We will set some task groups as benchmarks, such as network library, image library, embedded point framework, abtest, etc. after these tasks are completed, other business codes can be initialized here. In this way, we don't need everyone to write some basic dependencies, and we can also make the development students a little more comfortable.
Why is it a ring again
In the previous sorting stage, there was a very ghost problem. If the task you depend on does not exist in the current graph, you will report the dependency ring problem, but you do not know why it is ring.
This is very inconvenient for the development students to debug the problem, so I added the validity judgment of the front task. If it does not exist, the Log log will be printed directly, and the debugmode will also be added. If the test case is tested, the crash of the non-existent task can be ended directly.
ksp
I want to be lazy, so I generated some code with ksp. At the same time, I hope my startup framework can also be applied to the component and plug-in of the project, which is awesome anyway.
Start task grouping
One of the functions currently completed is to generate a start task group through annotation + ksp. This time, we use the version of ksp of 1.5.30, and there are some changes in the api.
The problem of dead cycle of process was mentioned in ksp's article before. Recently, I found that the system provides a finish method when communicating with mihuyou ujige (bragging). As long as there is class generation during process, the process method will be restarted, resulting in stackoverflow. Therefore, subsequent code generation can be considered to migrate to the new method.
class StartupProcessor(
val codeGenerator: CodeGenerator,
private val logger: KSPLogger,
val moduleName: String
) : SymbolProcessor {
private lateinit var startupType: KSType
private var isload = false
private val taskGroupMap = hashMapOf<String, MutableList<ClassName>>()
private val procTaskGroupMap =
hashMapOf<String, MutableList<Pair<ClassName, ArrayList<String>>>>()
override fun process(resolver: Resolver): List<KSAnnotated> {
logger.info("StartupProcessor start")
val symbols = resolver.getSymbolsWithAnnotation(StartupGroup::class.java.name)
startupType = resolver.getClassDeclarationByName(
resolver.getKSNameFromString(StartupGroup::class.java.name)
)?.asType() ?: kotlin.run {
logger.error("JsonClass type not found on the classpath.")
return emptyList()
}
symbols.asSequence().forEach {
add(it)
}
return emptyList()
}
private fun add(type: KSAnnotated) {
logger.check(type is KSClassDeclaration && type.origin == Origin.KOTLIN, type) {
"@JsonClass can't be applied to $type: must be a Kotlin class"
}
if (type !is KSClassDeclaration) return
//class type
val routerAnnotation = type.findAnnotationWithType(startupType) ?: return
val groupName = routerAnnotation.getMember<String>("group")
val strategy = routerAnnotation.arguments.firstOrNull {
it.name?.asString() == "strategy"
}?.value.toString().toValue() ?: return
if (strategy.equals("other", true)) {
val key = groupName
if (procTaskGroupMap[key] == null) {
procTaskGroupMap[key] = mutableListOf()
}
val list = procTaskGroupMap[key] ?: return
list.add(type.toClassName() to (routerAnnotation.getMember("processName")))
} else {
val key = "${groupName}${strategy}"
if (taskGroupMap[key] == null) {
taskGroupMap[key] = mutableListOf()
}
val list = taskGroupMap[key] ?: return
list.add(type.toClassName())
}
}
private fun String.toValue(): String {
var lastIndex = lastIndexOf(".") + 1
if (lastIndex <= 0) {
lastIndex = 0
}
return subSequence(lastIndex, length).toString().lowercase().upCaseKeyFirstChar()
}
// Start code generation logic
override fun finish() {
super.finish()
// logger.error("className:${moduleName}")
try {
taskGroupMap.forEach { it ->
val generateKt = GenerateGroupKt(
"${moduleName.upCaseKeyFirstChar()}${it.key.upCaseKeyFirstChar()}",
codeGenerator
)
it.value.forEach { className ->
generateKt.addStatement(className)
}
generateKt.generateKt()
}
procTaskGroupMap.forEach {
val generateKt = GenerateProcGroupKt(
"${moduleName.upCaseKeyFirstChar()}${it.key.upCaseKeyFirstChar()}",
codeGenerator
)
it.value.forEach { pair ->
generateKt.addStatement(pair.first, pair.second)
}
generateKt.generateKt()
}
} catch (e: Exception) {
logger.error(
"Error preparing :" + " ${e.stackTrace.joinToString("\n")}"
)
}
}
}
class StartupProcessorProvider : SymbolProcessorProvider {
override fun create(
environment: SymbolProcessorEnvironment
): SymbolProcessor {
return StartupProcessor(
environment.codeGenerator,
environment.logger,
environment.options[KEY_MODULE_NAME] ?: "application"
)
}
}
fun String.upCaseKeyFirstChar(): String {
return if (Character.isUpperCase(this[0])) {
this
} else {
StringBuilder().append(Character.toUpperCase(this[0])).append(this.substring(1)).toString()
}
}
const val KEY_MODULE_NAME = "MODULE_NAME"The processor is divided into two parts. The SymbolProcessorProvider is responsible for constructing and the SymbolProcessor is responsible for processing ast logic. The previous initapi has been moved to the SymbolProcessorProvider.
The logic is also relatively simple. Collect annotations, and then generate a taskGroup logic based on the input parameters of annotations. This group will be manually added to the startup process by me.
hang in the air
Another thing I want to do is to generate a task through annotations, and then combine a new task through the arrangement and combination of different annotations.
This part of the function is still under design. After the follow-up is completed, we will give you a piece of water.
Debug components
This part is the top priority of my recent design. After taking over the task of starting the framework, you need to trace the problem of slow startup more often. We call this situation deterioration. How to quickly locate the degradation problem is also the concern of the startup framework.
At first, we intend to report through the log, and then deduce the online task time-consuming after the release of the version. However, because the calculated value is the average value, and our automation test students will run the automation case before the release of each version to observe the start-up time. If the average time becomes longer, they will notify us, At this time, it is difficult to find the problem by looking at the buried point data.
The core reason is that I want to be lazy, because the troubleshooting must be based on the comparison between the previous version and the current version, and compare the time-consuming situation between various tasks. At present, we should have 30 + startup tasks. Isn't NIMA killing me.
Therefore, I communicated with my boss and set up a project for this part. I intend to toss a debugging tool to record the time-consuming of startup tasks and the list of startup tasks. Through local comparison, we can quickly deduce the problem tasks, which is convenient for us to quickly locate the problem.
Tips: it's better not to rely too much on the development of debugging tools, and then add them through the buildtype of debug, so use the content provider to initialize
Start timeline
My nickname ui Dashi has been circulating all the time in the Jianghu. I'm not in vain. The figure drawn by ui Dashi is as beautiful as a picture.
The principle of this part is relatively simple. We collect the data of the current startup task, then distribute it according to the thread name, record the start and end nodes of the task, and then display it graphically.
If you don't understand it for the first time, you can refer to the stock selection list. Each column represents the timeline of a thread.
Whether the startup sequence is changed
We will record the current startup sequence in the database at each startup, and then find out the tasks different from the current hashcode through the database, and then compare them and display them in the form of textview, so as to test the students' feedback.
The principle of this place is that I splice the whole startup task through strings, and then generate a string, and then use the hashcode of the string as the unique identifier. The hashcode generated by different strings is also different.
Here's a stupid thing. At first, I compared the hashcode of stringbuilder, and then found that the value of the same task changed. I'm really stupid.
Don't ask me if it's wet or not?
Average task time
The database design of this place made me think for a while. Then I took the day as the dimension, recorded the time and times, and then took the average value when rendering.
Then take out the previous historical data, summarize and count, and then regenerate the list. A current task is followed by a historical task. Then perform awesome ui rendering.
You have to spray at this time. Why do you call yourself ui wet when you are all textview.
Have you ever heard of shrimp bullshit? Yes, that's it.
summary
Roll, the sky is not born, I catch shrimp households, and the rolling road is as long as night.
Share with you.
Really sum up
In terms of UI, I will still iterate in the future. After all, the first version is ugly, mainly for the mobile phone that wants to complete the data, and the development doesn't seem particularly conspicuous. The difference may be output directly later.
Make it bigger and stronger and make a wave of big news.