preface
During the project two days ago, I wanted to improve the performance optimization of inserting tables, because there are two tables. Insert the old table first, and then insert the new table. More than 10000 data are a little slow
The thread pool ThreadPoolExecutor comes to mind later, and the Spring Boot project is used. The thread pool ThreadPoolTaskExecutor encapsulated by ThreadPoolExecutor provided by Spring can be directly enabled with annotations
Use steps
First create a thread pool Configuration and let Spring Boot load it to define how to create a ThreadPoolTaskExecutor. Use @ Configuration and @ EnableAsync annotations to indicate that this is a Configuration class and a thread pool Configuration class
@Configuration
@EnableAsync
public class ExecutorConfig {
private static final Logger logger = LoggerFactory.getLogger(ExecutorConfig.class);
@Value("${async.executor.thread.core_pool_size}")
private int corePoolSize;
@Value("${async.executor.thread.max_pool_size}")
private int maxPoolSize;
@Value("${async.executor.thread.queue_capacity}")
private int queueCapacity;
@Value("${async.executor.thread.name.prefix}")
private String namePrefix;
@Bean(name = "asyncServiceExecutor")
public Executor asyncServiceExecutor() {
logger.info("start asyncServiceExecutor");
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
//Configure the number of core threads
executor.setCorePoolSize(corePoolSize);
//Configure maximum threads
executor.setMaxPoolSize(maxPoolSize);
//Configure queue size
executor.setQueueCapacity(queueCapacity);
//Configure the name prefix of threads in the thread pool
executor.setThreadNamePrefix(namePrefix);
// Rejection policy: how to handle new tasks when the pool has reached max size
// CALLER_RUNS: the task is not executed in the new thread, but in the thread of the caller
executor.setRejectedExecutionHandler(new ThreadPoolExecutor.CallerRunsPolicy());
//Perform initialization
executor.initialize();
return executor;
}
}
@Value is configured in application.yml. You can refer to the configuration and define it freely
# Asynchronous thread configuration async: executor: thread: # Configure the number of core threads core_pool_size: 5 # Configure maximum threads max_pool_size: 5 # Configure queue size queue_capacity: 99999 # Configure the name prefix of threads in the thread pool name: prefix: async-service-
Create a Service interface, which is the interface of asynchronous threads
public interface AsyncService {
/**
* Perform asynchronous tasks
* You can add parameters according to your needs. I'll make a test demonstration here
*/
void executeAsync();
}
Implementation class
@Service
public class AsyncServiceImpl implements AsyncService {
private static final Logger logger = LoggerFactory.getLogger(AsyncServiceImpl.class);
@Override
@Async("asyncServiceExecutor")
public void executeAsync() {
logger.info("start executeAsync");
System.out.println("What asynchronous threads do");
System.out.println("You can perform time-consuming things such as batch insertion here");
logger.info("end executeAsync");
}
}
Asynchronize the services of the Service layer, and add the annotation @ Async("asyncServiceExecutor") on the executeAsync() method. The asyncServiceExecutor method is the method name in the previous ExecutorConfig.java, indicating that the thread pool entered by the executeAsync method is created by the asyncServiceExecutor method
The next step is to inject the Service in the Controller or where through the annotation @ Autowired
@Autowired
private AsyncService asyncService;
@GetMapping("/async")
public void async(){
asyncService.executeAsync();
}
Use postmain or other tools to test the request multiple times
2018-07-16 22:15:47.655 INFO 10516 --- [async-service-5] c.u.d.e.executor.impl.AsyncServiceImpl : start executeAsync What asynchronous threads do You can perform time-consuming things such as batch insertion here 2018-07-16 22:15:47.655 INFO 10516 --- [async-service-5] c.u.d.e.executor.impl.AsyncServiceImpl : end executeAsync 2018-07-16 22:15:47.770 INFO 10516 --- [async-service-1] c.u.d.e.executor.impl.AsyncServiceImpl : start executeAsync What asynchronous threads do You can perform time-consuming things such as batch insertion here 2018-07-16 22:15:47.770 INFO 10516 --- [async-service-1] c.u.d.e.executor.impl.AsyncServiceImpl : end executeAsync 2018-07-16 22:15:47.816 INFO 10516 --- [async-service-2] c.u.d.e.executor.impl.AsyncServiceImpl : start executeAsync What asynchronous threads do You can perform time-consuming things such as batch insertion here 2018-07-16 22:15:47.816 INFO 10516 --- [async-service-2] c.u.d.e.executor.impl.AsyncServiceImpl : end executeAsync 2018-07-16 22:15:48.833 INFO 10516 --- [async-service-3] c.u.d.e.executor.impl.AsyncServiceImpl : start executeAsync What asynchronous threads do You can perform time-consuming things such as batch insertion here 2018-07-16 22:15:48.834 INFO 10516 --- [async-service-3] c.u.d.e.executor.impl.AsyncServiceImpl : end executeAsync 2018-07-16 22:15:48.986 INFO 10516 --- [async-service-4] c.u.d.e.executor.impl.AsyncServiceImpl : start executeAsync What asynchronous threads do You can perform time-consuming things such as batch insertion here 2018-07-16 22:15:48.987 INFO 10516 --- [async-service-4] c.u.d.e.executor.impl.AsyncServiceImpl : end executeAsync
It can be found from the above logs that [async Service -] has multiple threads, which have obviously been executed in the thread pool configured by us, and the start and end logs of the controller are printed continuously in each request, indicating that each request is responded quickly, and the time-consuming operations are left to the threads in the thread pool to execute asynchronously;
Although we have used the thread pool, it is not clear how many threads are executing and how many are waiting in the queue? A subclass of ThreadPoolTaskExecutor is created here. The running status of the current thread pool will be printed every time the thread is submitted
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;
import org.springframework.util.concurrent.ListenableFuture;
import java.util.concurrent.Callable;
import java.util.concurrent.Future;
import java.util.concurrent.ThreadPoolExecutor;
/**
* @Author: ChenBin
* @Date: 2018/7/16/0016 22:19
*/
public class VisiableThreadPoolTaskExecutor extends ThreadPoolTaskExecutor {
private static final Logger logger = LoggerFactory.getLogger(VisiableThreadPoolTaskExecutor.class);
private void showThreadPoolInfo(String prefix) {
ThreadPoolExecutor threadPoolExecutor = getThreadPoolExecutor();
if (null == threadPoolExecutor) {
return;
}
logger.info("{}, {},taskCount [{}], completedTaskCount [{}], activeCount [{}], queueSize [{}]",
this.getThreadNamePrefix(),
prefix,
threadPoolExecutor.getTaskCount(),
threadPoolExecutor.getCompletedTaskCount(),
threadPoolExecutor.getActiveCount(),
threadPoolExecutor.getQueue().size());
}
@Override
public void execute(Runnable task) {
showThreadPoolInfo("1. do execute");
super.execute(task);
}
@Override
public void execute(Runnable task, long startTimeout) {
showThreadPoolInfo("2. do execute");
super.execute(task, startTimeout);
}
@Override
public Future<?> submit(Runnable task) {
showThreadPoolInfo("1. do submit");
return super.submit(task);
}
@Override
public <T> Future<T> submit(Callable<T> task) {
showThreadPoolInfo("2. do submit");
return super.submit(task);
}
@Override
public ListenableFuture<?> submitListenable(Runnable task) {
showThreadPoolInfo("1. do submitListenable");
return super.submitListenable(task);
}
@Override
public <T> ListenableFuture<T> submitListenable(Callable<T> task) {
showThreadPoolInfo("2. do submitListenable");
return super.submitListenable(task);
}
}
As shown above, the showThreadPoolInfo method prints out the total number of tasks, the number of completed threads, the number of active threads, and the queue size, and then overrides the execute, submit, and other methods of the parent class, where the showThreadPoolInfo method is called, so that each time a task is submitted to the thread pool, the basic information of the current thread pool will be printed to the log;
Modify the asyncServiceExecutor method of ExecutorConfig.java and change threadpooltaskexecutor = new threadpooltaskexecutor() to threadpooltaskexecutor = new visiblethreadpooltaskexecutor()
@Bean(name = "asyncServiceExecutor")
public Executor asyncServiceExecutor() {
logger.info("start asyncServiceExecutor");
//Modify here
ThreadPoolTaskExecutor executor = new VisiableThreadPoolTaskExecutor();
//Configure the number of core threads
executor.setCorePoolSize(corePoolSize);
//Configure maximum threads
executor.setMaxPoolSize(maxPoolSize);
//Configure queue size
executor.setQueueCapacity(queueCapacity);
//Configure the name prefix of threads in the thread pool
executor.setThreadNamePrefix(namePrefix);
// Rejection policy: how to handle new tasks when the pool has reached max size
// CALLER_RUNS: the task is not executed in the new thread, but in the thread of the caller
executor.setRejectedExecutionHandler(new ThreadPoolExecutor.CallerRunsPolicy());
//Perform initialization
executor.initialize();
return executor;
}
Start the project test again
2018-07-16 22:23:30.951 INFO 14088 --- [nio-8087-exec-2] u.d.e.e.i.VisiableThreadPoolTaskExecutor : async-service-, 2. do submit,taskCount [0], completedTaskCount [0], activeCount [0], queueSize [0] 2018-07-16 22:23:30.952 INFO 14088 --- [async-service-1] c.u.d.e.executor.impl.AsyncServiceImpl : start executeAsync What asynchronous threads do You can perform time-consuming things such as batch insertion here 2018-07-16 22:23:30.953 INFO 14088 --- [async-service-1] c.u.d.e.executor.impl.AsyncServiceImpl : end executeAsync 2018-07-16 22:23:31.351 INFO 14088 --- [nio-8087-exec-3] u.d.e.e.i.VisiableThreadPoolTaskExecutor : async-service-, 2. do submit,taskCount [1], completedTaskCount [1], activeCount [0], queueSize [0] 2018-07-16 22:23:31.353 INFO 14088 --- [async-service-2] c.u.d.e.executor.impl.AsyncServiceImpl : start executeAsync What asynchronous threads do You can perform time-consuming things such as batch insertion here 2018-07-16 22:23:31.353 INFO 14088 --- [async-service-2] c.u.d.e.executor.impl.AsyncServiceImpl : end executeAsync 2018-07-16 22:23:31.927 INFO 14088 --- [nio-8087-exec-5] u.d.e.e.i.VisiableThreadPoolTaskExecutor : async-service-, 2. do submit,taskCount [2], completedTaskCount [2], activeCount [0], queueSize [0] 2018-07-16 22:23:31.929 INFO 14088 --- [async-service-3] c.u.d.e.executor.impl.AsyncServiceImpl : start executeAsync What asynchronous threads do You can perform time-consuming things such as batch insertion here 2018-07-16 22:23:31.930 INFO 14088 --- [async-service-3] c.u.d.e.executor.impl.AsyncServiceImpl : end executeAsync 2018-07-16 22:23:32.496 INFO 14088 --- [nio-8087-exec-7] u.d.e.e.i.VisiableThreadPoolTaskExecutor : async-service-, 2. do submit,taskCount [3], completedTaskCount [3], activeCount [0], queueSize [0] 2018-07-16 22:23:32.498 INFO 14088 --- [async-service-4] c.u.d.e.executor.impl.AsyncServiceImpl : start executeAsync What asynchronous threads do You can perform time-consuming things such as batch insertion here 2018-07-16 22:23:32.499 INFO 14088 --- [async-service-4] c.u.d.e.executor.impl.AsyncServiceImpl : end executeAsync
Note this line of log:
2018-07-16 22:23:32.496 INFO 14088 --- [nio-8087-exec-7] u.d.e.e.i.VisiableThreadPoolTaskExecutor : async-service-, 2. do submit,taskCount [3], completedTaskCount [3], activeCount [0], queueSize [0]
This indicates that when submitting a task to the thread pool, the method submit(Callable task) is called. Currently, three tasks have been submitted and completed. At present, there are 0 threads processing tasks, and there are 0 tasks waiting in the queue. The basic situation of the thread pool has been all the way;