GC thinking caused by a thread pool bug!

Author: empty https://segmentfault.com/a/1190000021109130

Problem description

A few days ago, I was helping my colleagues to troubleshoot the occasional thread pool errors on the production line. The logic was very simple. The thread pool performed an asynchronous task with results.

However, there have been occasional errors recently:

java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@a5acd19 rejected from java.util.concurrent.ThreadPoolExecutor@30890a38\[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0\]

The simulation code in this article has been in the hotspot java8 (1.8.0) version of the simulation & appeared

The following is the simulation code. Create a single thread pool through executors.newsingthreadexector, and then get the result of Future at the caller

public class ThreadPoolTest {    public static void main(String\[\] args) {        final ThreadPoolTest threadPoolTest = new ThreadPoolTest();        for (int i = 0; i < 8; i++) {            new Thread(new Runnable() {                @Override
                public void run() {                    while (true) {

                        Future<String> future = threadPoolTest.submit();                        try {
                            String s = future.get();
                        } catch (InterruptedException e) {
                            e.printStackTrace();
                        } catch (ExecutionException e) {
                            e.printStackTrace();
                        } catch (Error e) {
                            e.printStackTrace();
                        }
                    }
                }
            }).start();
        }        
        //Sub thread keeps gc, simulating sporadic gc
        new Thread(new Runnable() {            @Override
            public void run() {                while (true) {
                    System.gc();
                }
            }
        }).start();
    }    /**
     * Asynchronous task execution
     * @return
     */
    public Future<String> submit() {        //Key point, create a single thread pool through executors.newsingthreadexector
        ExecutorService executorService = Executors.newSingleThreadExecutor();
        FutureTask<String> futureTask = new FutureTask(new Callable() {            @Override
            public Object call() throws Exception {
                Thread.sleep(50);                return System.currentTimeMillis() + "";
            }
        });
        executorService.execute(futureTask);        return futureTask;
    }

}

Analysis & questions

The first question is: why the thread pool is closed? There is no place in the code to close it manually. Take a look at the source implementation of executors.newsinglethreadexecutor:

public static ExecutorService newSingleThreadExecutor() {    return new FinalizableDelegatedExecutorService
            (new ThreadPoolExecutor(1, 1,                    0L, TimeUnit.MILLISECONDS,                    new LinkedBlockingQueue<Runnable>()));
}

What is created here is actually a FinalizableDelegatedExecutorService. This wrapper class overrides the finalize function, which means that the class will execute the shutdown method of thread pool before it is recycled by GC.

Here comes the problem. GC will only recycle unreachable objects. executorService should be reachable before the stack frame of submit function is finished.

For this problem, first throw a conclusion:

finalize may also be performed when the object still exists in the stack frame

There is an introduction to finalize in the oracle jdk document:

A reachable object is any object that can be accessed in any potential continuing computation from any live thread.

Optimizing transformations of a program can be designed that reduce the number of objects that are reachable to be less than those which would naively be considered reachable. For example, a Java compiler or code generator may choose to set a variable or parameter that will no longer be used to null to cause the storage for such an object to be potentially reclaimable sooner.

The reachable object is any object that can be accessed continuously from any active thread. The java compiler or code generator may set the object that is no longer accessed to null in advance, so that the object can be recycled in advance

That is to say, under the optimization of jvm, there may be an empty and recycling situation after the object is not reachable. Pay attention to WeChat official account: Java technology stack, reply in the background: Java, get the latest Java tutorial I have finished N, all dry cargo.

Take an example to verify (excerpt from https://stackoverflow.com/questions/24376768/can-java-finalize-an-object-when-it-is-still-in-scope):

class A {
    @Override protected void finalize() {
        System.out.println(this + " was finalized!");
    }    public static void main(String\[\] args) throws InterruptedException {
        A a = new A();
        System.out.println("Created " + a);        for (int i = 0; i < 1\_000\_000\_000; i++) {            if (i % 1\_000_00 == 0)
                System.gc();
        }
        System.out.println("done.");
    }
}//Print result Created A@1be6f5c3
A@1be6f5c3 was finalized!//finalize method output done

As can be seen from the example, if a is no longer used after the completion of the loop, it will execute finalize first. Although the method is not finished and the stack frame is not out of the stack from the object scope, it will be executed in advance.

Now add a line of code to print object a on the last line, and let the compiler / code generator think that there is a reference to object a after it

...System.out.println(a);//Print result Created A@1be6f5c3
done.
A@1be6f5c3

As a result, none of the finalize methods are executed (because the process ends directly after the main method is executed), let alone the problem of finalize in advance

Based on the above test results, another test is to set the object a to null before the loop, and print the reference to keep the object a at last

A a = new A();System.out.println("Created " + a);
a = null;//Manually set nullfor (int i = 0; I < 1 \ \000 \ \\\\\\; I + +) {if (I% 1 \ \\\;
}System.out.println("done.");System.out.println(a);//Print result Created A@1be6f5c3
A@1be6f5c3 was finalized!
done.null

As a result, if you manually set null, the object will be recycled in advance. Although there is a reference at the end, it is also null

Now go back to the above thread pool problem. According to the mechanism described above, after analyzing no reference, the object will be finalize d in advance

In the above code, it is clear that there is a reference executorService.execute(futureTask) before return. Why does it also finalize in advance?

It is speculated that in the execute method, the threadPoolExecutor will be called, and a new thread will be created and started. At this time, an active thread switch will occur, resulting in the unreachable objects in the active thread

Combined with the description in the Oracle Jdk document above that "reachable object is any object that can be accessed from any potential continuous access of any active thread", it can be considered that the object is considered unreachable due to a displayed thread switch, which causes the thread pool to be finalize d in advance

Let's test our conjecture:

//The entry function public class finalizedtest {public static void main(String \ [\] args) {final finalizedtest finalizedtest = new FinalizedTest(); for (int i = 0; I < 8; I + +) {new Thread(new Runnable() {@ Override
                public void run() {                    while (true) {
                        TFutureTask future = finalizedTest.submit();
                    }
                }
            }).start();
        }        new Thread(new Runnable() {            @Override
            public void run() {                while (true) {
                    System.gc();
                }
            }
        }).start();
    }    public TFutureTask submit(){
        TExecutorService TExecutorService = Executors.create();
        TExecutorService.execute();        return null;
    }
}//Executors.java, simulating Executorspublic class executors {of juc/**
     * simulation Executors.createSingleExecutor
     * @return
     */
    public static TExecutorService create(){        return new FinalizableDelegatedTExecutorService(new TThreadPoolExecutor());
    }    static class FinalizableDelegatedTExecutorService extends DelegatedTExecutorService {

        FinalizableDelegatedTExecutorService(TExecutorService executor) {            super(executor);
        }        
        /**
         * Execute shutdown in destructor to modify thread pool status
         * @throws Throwable
         */
        @Override
        protected void finalize() throws Throwable {            super.shutdown();
        }
    }    static class DelegatedTExecutorService extends TExecutorService {        protected TExecutorService e;        public DelegatedTExecutorService(TExecutorService executor) {            this.e = executor;
        }        @Override
        public void execute() {
            e.execute();
        }        @Override
        public void shutdown() {
            e.shutdown();
        }
    }
}//TThreadPoolExecutor.java, to simulate the ThreadPoolExecutor public class of juc, TThreadPoolExecutor extensions TExecutorService {/**
     * Thread pool status, false: Not closed. true Closed
     */
    private AtomicBoolean ctl = new AtomicBoolean();    @Override
    public void execute() {        //Start a new thread and simulate ThreadPoolExecutor.execute
        new Thread(new Runnable() {            @Override
            public void run() {

            }
        }).start();        //Simulate the ThreadPoolExecutor, start the new thread, cycle to check the thread pool status, and verify whether it will shut down in finalize
        //If the thread pool is shut down in advance, an exception is thrown
        for (int i = 0; i < 1\_000\_000; i++) {            if(ctl.get()){                throw new RuntimeException("reject!!!\["+ctl.get()+"\]");
            }
        }
    }    @Override
    public void shutdown() {
        ctl.compareAndSet(false,true);
    }
}

Error is reported after several times of execution:

Exception in thread "Thread-1" java.lang.RuntimeException: reject!!!\[true\]

From the error point of view, "thread pool" is also shut down in advance, so it must be caused by new threads?

Next, change the new thread to Thread.sleep to test:

//TThreadPoolExecutor.java, the modified execute method public void execute() {try {/ / explicit sleep 1 ns, to actively switch threads
        TimeUnit.NANOSECONDS.sleep(1);
    } catch (InterruptedException e) {
        e.printStackTrace();
    }    //Simulate the ThreadPoolExecutor, start the new thread, cycle to check the thread pool status, and verify whether it will shut down in finalize
    //If the thread pool is shut down in advance, an exception is thrown
    for (int i = 0; i < 1\_000\_000; i++) {        if(ctl.get()){            throw new RuntimeException("reject!!!\["+ctl.get()+"\]");
        }
    }
}

The execution result is the same as error reporting

Exception in thread "Thread-3" java.lang.RuntimeException: reject!!!\[true\]

Thus, if an explicit thread switch occurs during execution, the compiler / code generator will think that the outer wrapper object is not reachable

summary

Although GC can only recycle the objects that can't reach GC ROOT, under the optimization of compiler (not explicitly pointed out, it may also be JIT) / code generator, there may be the situation that the object is set to null in advance, or the "advanced object can't reach" caused by thread switching.

So if you want to do something in the finalize method, you must refer to the object in the last display (toString/hashcode is OK), and keep the reachable of the object

As for the above, the object caused by thread switching is not reachable, and there is no support from official literature. It is just a test result of an individual. If there is any problem, please point out

To sum up, this recycling mechanism is not a bug of JDK, but an optimization strategy, just recycling in advance. However, there is a bug in the implementation of executors.newsingthreadexecutor that automatically shuts down the thread pool by finalizing. After optimization, it may cause the thread pool to shut down in advance, leading to exceptions.

This problem of thread pool is also an open but unresolved problem in the JDK forum https://bugs.openjdk.java.net/browse/JDK-8145304.

However, under JDK11, the problem has been fixed:

JUC  Executors.FinalizableDelegatedExecutorServicepublic void execute(Runnable command) {    try {
        e.execute(command);
    } finally { reachabilityFence(this); }
}

WeChat official account: Java technology stack, back in the background: Java, can get the latest Java tutorial I have compiled N, all dry cargo.

Recommend to my blog to read more:

1.Java JVM, collection, multithreading, new features series

2.Spring MVC, Spring Boot, Spring Cloud series tutorials

3.Maven, Git, Eclipse, Intellij IDEA series tools tutorial

4.Latest interview questions of Java, backend, architecture, Alibaba and other large factories

Life is beautiful. See you tomorrow

Keywords: Programming Java JDK Spring Oracle

Added by poscribes on Tue, 17 Mar 2020 11:18:15 +0200