preface
This case is a real case I encountered. The process of finding the cause once made me collapse. I believe many people have encountered the same problem, so I want to record it and help you. This case uses RxHttp 2.6.4 + OkHttp 4.9.1 + Android Studio 4.2.2. Of course, if you use other frameworks based on OkHttp encapsulation such as Retrofit, If you use the function of monitoring the upload progress, you will also encounter this problem. Please read it patiently. If you want to see the results directly, just draw to the end of the article.
https://github.com/liujingxing/rxhttp
1. Problem description
The thing is, there is a file upload code, as follows:
fun uploadFiles(fileList: List<File>) {
RxHttp.postForm("/server/...")
.add("key", "value")
.addFiles("files", fileList)
.upload {
//Upload progress callback
}
.asString()
.subscribe({
//Successful callback
}, {
//Failed callback
})
}
This code was ok for a long time after it was written. Suddenly one day, an error was reported when executing this code. The log is as follows:
This exception is 100% familiar. The specific reason is that the data flow is closed, but the data is still written into it. Let's take a look at the last exception thrown, as follows:
You can see that the first line of code in the method determines whether the data flow has been closed. If so, an exception is thrown.
Note: if you are an RxHttp user and are trying this code, don't be surprised to find that there is no problem, because it needs to be executed in a specific scenario of Android Studio, and it is a relatively high-frequency scenario. Please wait for me to reveal the answer step by step.
2. Find out
Based on the principle of locating your own code when a problem occurs, open the ProgressRequestBody class 76 line to see, as follows:
fun uploadFiles(fileList: List<File>) {
RxHttp.postForm("/server/...")
.add("key", "value")
.addFiles("files", fileList)
.upload {
//Upload progress callback
}
.asString()
.subscribe({
//Successful callback
}, {
//Failed callback
})
}
ProgressRequestBody inherits okhttp3 The requestbody class is used to monitor the upload progress; Obviously, the data flow has been closed when the last execution is here. From the log, you can see that the last call to the ProgressRequestBody#writeTo(BufferedSink) method is in line 59 of the CallServerInterceptor interceptor. Open it and have a look.
class CallServerInterceptor(private val forWebSocket: Boolean) : Interceptor {
//Omit relevant codes
@Throws(IOException::class)
override fun intercept(chain: Interceptor.Chain): Response {
//Omit relevant codes
if (responseBuilder == null) {
if (requestBody.isDuplex()) {
exchange.flushRequest()
val bufferedRequestBody = exchange.createRequestBody(request, true).buffer()
requestBody.writeTo(bufferedRequestBody)
} else {
val bufferedRequestBody = exchange.createRequestBody(request, false).buffer()
requestBody.writeTo(bufferedRequestBody) //This is line 59
bufferedRequestBody.close() //When the data is written, close the data flow
}
}
}
}
Students familiar with the okhttp principle should know that the CallServerInterceptor interceptor is the last interceptor in the okhttp interceptor chain. It writes the client data to the server, which is implemented here, that is, 59 lines. The problem is, how can the data flow be closed before the data is written out? This makes me puzzled and confused.
So I did a lot of useless work. For example, I rechecked the code to see if there was a place to manually close the data flow. Obviously, I didn't find it; Then, there is no way to roll back the code and roll back to the version of the code originally written. I am full of expectation and think that there should be no problem now. After trying, I still report Java Lang. IllegalStateException: closed, the collapse of adults. At this moment, I was in a desperate situation. It has taken me five hours on this issue. At this time, it is 23:30 p.m. and it seems to be another sleepless night.
Habit tells me that if I haven't found out a problem for a long time, I can give up first. Well, pull out my cell phone, turn off the computer, take a bath and sleep.
Half an hour later, I was lying in bed. I felt very uncomfortable, so I took out my mobile phone, opened the app, tried the upload function again, and was surprised to find that it was OK. The upload was successful, which.... With a confused face, I asked someone to reason. Although there was no problem, the problem was not found. As a junior programmer, I couldn't accept it.
The power of spirit helped me out of bed, turned on the computer again and connected my mobile phone. This time, I really got a new harvest and refreshed my world outlook; When I open the app again and try to upload files, the same error appears in front of me. What??? It was fine just now. You can't connect to the computer?
ok, I'm completely out of temper. Unplug my mobile phone, restart the app, try again, no problem, connect to the computer again, try again, the problem comes out again..
At this time, my state of mind has improved a little. After all, I have a new investigation direction. I checked the error log again and found a very strange place, as follows:
com. android. tools. profiler. agent. okhttp. Where did okhttp3interceptor come from? In my understanding, okhttp3 does not have this interceptor. To verify my understanding, check the okhttp3 source code again, as follows:
It is confirmed that the interceptor was not added. A closer look at the log shows that OkHttp3Interceptor executes between CallServerInterceptor and ConnectInterceptor. There is only one explanation. OkHttp3Interceptor is added through the addNetworkInterceptor method. It's easy to do now. It's a pity to know who added it and where it was added by searching addNetworkInterceptor globally, The source code for calling this method is not found, and it seems to be in a desperate situation again.
Then you can only start debugging to see if OkHttp3Interceptor is in the networkInterceptors network interceptor list of OkHttpClient object. After debugging, it is found as follows:
Debugging click next, and something magical happens, as follows:
How does that explain? networkInterceptors.size is always 0, interceptors How does size change from 1 to 5? Let's see what 1 is added, as follows:
Very familiar with the OkHttp3Interceptor we mentioned earlier. How does this work? There is only one explanation. The OkHttpClient#networkInterceptors() method is inserted with new code by bytecode piling technology. In order to verify my idea, I did the following experiments:
You can see that I directly new an OkHttpClient object without configuring anything. By calling the networkInterceptors() method, I get the OkHttp3Interceptor interceptor, but there is no such interceptor in the networkInterceptors list in the OkHttpClient object, which confirms my idea.
The question now is, who injected OkHttp3Interceptor? Is it directly related to the failure of file upload?
Who injected OkHttp3Interceptor?
First, let's explore the first problem through the package name of OkHttp3Interceptor class # com android. tools. profiler. agent. Okhttp, I have the following three guesses:
1. The package name is com android. Tools, it should have something to do with the Android official.
2. The package name includes agent and interceptor, which should be related to network agent, that is, network monitoring.
3. Last but not least, the package name has profiler, which reminds me of the profiler network analyzer in Android studio (hereinafter referred to as as AS).
Sure enough, the OkHttp3Interceptor class was found in Google's source code. Look at the relevant code:
https://android.googlesource.com/platform/tools/base/+/studio-master-dev/profiler/app/perfa-okhttp/src/main/java/com/android/tools/profiler/agent/okhttp/OkHttp3Interceptor.java
public final class OkHttp3Interceptor implements Interceptor {
//Omit relevant codes
@Override
public Response intercept(Interceptor.Chain chain) throws IOException {
Request request = chain.request();
HttpConnectionTracker tracker = null;
try {
tracker = trackRequest(request); //1. Tracking request body
} catch (Exception ex) {
StudioLog.e("Could not track an OkHttp3 request", ex);
}
Response response;
try {
response = chain.proceed(request);
} catch (IOException ex) {
}
try {
if (tracker != null) {
response = trackResponse(tracker, response); //2. Tracking responder
}
} catch (Exception ex) {
StudioLog.e("Could not track an OkHttp3 response", ex);
}
return response;
}
I'm sure it's a network monitor, but I'm still skeptical about whether it's an AS network monitor, because I haven't started the Profiler analyzer in this project, but I've recently been developing room database related functions and started the data analyzer database inspector. Is this related? I tried to shut down the database inspector, restart the app and try to upload the file again. It was really successful. Can you believe it? AS like AS two peas, I started to open the Database Inspector again, and tried again to upload the file. Then, I closed the database inspector, opened the Profiler analyzer, and tried to upload the file again, but it also failed.
When I thought of this, I can basically conclude that OkHttp3Interceptor is the network monitor in Profiler, but there seems to be a lack of direct evidence. Therefore, I tried to change the ProgressRequestBody class as follows:
public class ProgressRequestBody extends RequestBody {
//Omit relevant codes
private BufferedSink bufferedSink;
@Override
public void writeTo(BufferedSink sink) throws IOException {
//If the caller is OkHttp3Interceptor, it does not write the request body and returns directly
if (sink.toString().contains(
"com.android.tools.profiler.support.network.HttpTracker$OutputStreamTracker"))
return;
if (bufferedSink == null) {
bufferedSink = Okio.buffer(sink(sink));
}
requestBody.writeTo(bufferedSink);
bufferedSink.flush();
}
}
The above code only adds a if statement, which can determine whether the current caller is OkHttp3Interceptor or not. If okhttp3interceptor is the network monitor in Profiler, the request body, that is, the request parameters, should not be visible in Profiler, as follows:
It can be seen that the network monitor in Profiler does not monitor the request parameters.
This confirms that OkHttp3Interceptor is indeed a network monitor in Profiler, that is, AS dynamic injection.
Is OkHttp3Interceptor directly related to file upload?
Through the above case analysis, it is obvious that it is directly related. When you do not open Database Inspector and Profiler, the file upload is normal.
How does OkHttp3Interceptor affect file upload?
Back to the point, how does OkHttp3Interceptor affect file upload? For this, we need to continue to analyze the source code of OkHttp3Interceptor to see the code that tracks the request body:
public final class OkHttp3Interceptor implements Interceptor {
private HttpConnectionTracker trackRequest(Request request) throws IOException {
StackTraceElement[] callstack =
OkHttpUtils.getCallstack(request.getClass().getPackage().getName());
HttpConnectionTracker tracker =
HttpTracker.trackConnection(request.url().toString(), callstack);
tracker.trackRequest(request.method(), toMultimap(request.headers()));
if (request.body() != null) {
OutputStream outputStream =
tracker.trackRequestBody(OkHttpUtils.createNullOutputStream());
BufferedSink bufferedSink = Okio.buffer(Okio.sink(outputStream));
request.body().writeTo(bufferedSink); //1. Write the request body to BufferedSink
bufferedSink.close(); //2. Close BufferedSink
}
return tracker;
}
}
The problem is clear when you think of it. The first code in the above remarks is request Body (), gets the ProgressRequestBody object, then calls its writeTo(BufferedSink) method and passes it to the BufferedSink object. After executing the method, it closes the BufferedSink object. However, ProgressRequestBody declares BufferedSink as a member variable and assigns it only when it is empty. This will cause the CallServerInterceptor to call its writeTo(BufferedSink) method using the last closed bufferedsink object, and then write data to it, which is Java Lang.illegalstateexception: closed exception.
3. How to solve it
If you know the specific reason, you can solve it. Change the BufferedSink object in the ProgressRequestBody to a local variable, as follows:
public class ProgressRequestBody extends RequestBody {
//Omit relevant codes
@Override
public void writeTo(BufferedSink sink) throws IOException {
BufferedSink bufferedSink = Okio.buffer(sink(sink));
requestBody.writeTo(bufferedSink);
bufferedSink.colse();
}
}
After changing it, open the network monitor in Profiler, try the file upload again, ok is successful, but there is a new problem. ProgressRequestBody is used to monitor the upload progress. OkHttp3Interceptor and CallServerInterceptor have called the writeTo(BufferedSink) method successively, which will cause the request body to write two times, that is, the progress monitor will receive two times. What we really need is the call to the callserverinterceptor. How? Easy to handle. We have judged whether the caller is okhttp3interceptor.
Therefore, the following changes are made:
You think it's over? I believe many people will use COM squareup. Okhttp3: logging interceptor log interceptor. When you add the log interceptor and upload the file again, you will find that the progress callback has been executed twice. Why? Because the log interceptor will also call the ProgressRequestBody#writeTo(BufferedSink) method, look at the code:
//Omit some codes
class HttpLoggingInterceptor @JvmOverloads constructor(
private val logger: Logger = Logger.DEFAULT
) : Interceptor {
@Throws(IOException::class)
override fun intercept(chain: Interceptor.Chain): Response {
val request = chain.request()
val requestBody = request.body
if (logHeaders) {
if (!logBody || requestBody == null) {
logger.log("--> END ${request.method}")
} else if (bodyHasUnknownEncoding(request.headers)) {
logger.log("--> END ${request.method} (encoded body omitted)")
} else if (requestBody.isDuplex()) {
logger.log("--> END ${request.method} (duplex request body omitted)")
} else if (requestBody.isOneShot()) {
logger.log("--> END ${request.method} (one-shot body omitted)")
} else {
val buffer = Buffer()
//1. Here, the writeTo method of RequestBody is called and the Buffer object is passed in
requestBody.writeTo(buffer)
}
}
val response: Response
try {
response = chain.proceed(request)
} catch (e: Exception) {
throw e
}
return response
}
}
It can be seen that the HttpLoggingInterceptor will also call the RequestBody#writeTo method and pass in the Buffer object. It's easy to do. Add a Buffer judgment logic in the ProgressRequestBody class, as follows:
public class ProgressRequestBody extends RequestBody {
//Omit relevant codes
@Override
public void writeTo(BufferedSink sink) throws IOException {
//If the caller is OkHttp3Interceptor or the Buffer object is passed in, the request body is written directly, and the request progress is no longer processed through the wrapper class
if (sink instanceof Buffer
|| sink.toString().contains(
"com.android.tools.profiler.support.network.HttpTracker$OutputStreamTracker")) {
requestBody.writeTo(bufferedSink);
} else {
BufferedSink bufferedSink = Okio.buffer(sink(sink));
requestBody.writeTo(bufferedSink);
bufferedSink.colse();
}
}
}
That's it? It's not certain that if any interceptor calls its writeTo method later, the progress callback will be executed twice. You can only add the corresponding judgment logic in this case.
At this point, some people may ask why they don't directly judge whether the caller is a CallServerInterceptor. If so, listen for the progress callback. Otherwise, write it directly to the request body. The idea is good and feasible, as follows:
public class ProgressRequestBody extends RequestBody {
//Omit relevant codes
@Override
public void writeTo(BufferedSink sink) throws IOException {
//If the caller is CallServerInterceptor, monitor the upload progress
if (sink.toString().contains("RequestBodySink(okhttp3.internal")) {
BufferedSink bufferedSink = Okio.buffer(sink(sink));
requestBody.writeTo(bufferedSink);
bufferedSink.colse();
} else {
requestBody.writeTo(bufferedSink);
}
}
}
However, this scheme has a fatal flaw. If the directory structure is changed in a future version of okhttp, the ProgressRequestBody class will be completely invalid.
The two schemes are up to you to choose. Here is the complete source code of ProgressRequestBody, which needs to be taken by yourself.
https://github.com/liujingxing/rxhttp/blob/master/rxhttp/src/main/java/rxhttp/wrapper/progress/ProgressRequestBody.java
Summary
The direct reason for the failure of uploading this case is that when the AS starts the Database Inspector database analyzer or Profiler network monitor, the AS will inject new bytecode into the OkHttpClient#networkInterceptors() method through bytecode stake insertion technology to make it return one more com android. tools. Profiler. agent. okhttp. Okhttp3interceptor interceptor (used to listen to the network). The interceptor will call the ProgressRequestBody#writeTo(BufferedSink) method and pass in the BufferedSink object. After the writeTo method is executed, close the BufferedSink object immediately, In the subsequent CallServerInterceptor interception, the ProgressRequestBody#writeTo(BufferedSink) method is called to write data to the closed BufferedSink object, resulting in Java Lang.illegalstateexception: closed exception.
But there is a doubt, but I haven't found the answer, that is, why opening the Database Inspector will lead the AS to monitor the network? If you know, you can leave a message in the comment area.
Reprinted from: https://mp.weixin.qq.com/s/lwn5RIK_G_R8UV-S_tQcBw