diyTomcat series 3 introduces thread pool and handles long TCP connections
Several problems to be solved in this section:
Frequent creation and destruction of threads, every time HTTP Every request generates a new Socket Connection. When the number of requests is too large, the overhead is large
Solution:
Introduction of thread pool and utilization Connection=keep-alive,Connection timeout control TCP Long connection to handle http request
1. Introduce thread pool
In normal network requests, a large number of http requests need to be processed at the same time. If each http request is handed over to a separate thread for processing, the thread will be destroyed immediately after processing, which will consume performance and cause unnecessary overhead. Therefore, we introduce thread pool to process requests
//Let's first look at the parameters to be passed in by the thread pool construction method ThreadPoolExecutor(corePoolSize,maxPoolSize,keepAliveTime,unit,workQueue,threadFactory,handler); //The parameters are: /The size of the core thread pool to obtain the number of cores of the server processor int corePoolSize = Runtime.getRuntime().availableProcessors(); //Maximum number of threads in the core thread pool int maxPoolSize=corePoolSize * 2; //Maximum idle time of thread long keepAliveTime=10; //The time unit is set to seconds TimeUnit unit=TimeUnit.SECONDS; //The capacity of the blocking queue is maxPoolSize * 4, and a maximum of maxPoolSize * 4 idle tasks are allowed BlockingQueue<Runnable> workQueue=new ArrayBlockingQueue<>(maxPoolSize * 4); //Thread creation factory ThreadFactory threadFactory = new NameThreadFactory(); //Thread pool reject policy RejectedExecutionHandler handler = new MyIgnorePolicy();
If the number of connections is small, using thread pool is actually a waste of resources. By default, all Tomcat does not use thread pool. Wait until a certain number of requests to use thread pool. Of course, based on the design idea of high cohesion and low coupling, we can also use thread pool on server Set the default startup thread pool manually in the configuration file in line 74 of XML
2. Handle long TCP connections
First of all, we should know that HTTP protocol is an application layer protocol. The application layer protocol is to define message dialogue, ensure that the message being sent gets the expected response, and call the correct service when transmitting data. The HTTP protocol is based on the request / response pattern. The long connection we usually discuss is a TCP long connection. The application layer does not pay attention to whether it is a long connection or a short connection. As long as the server gives a response, the HTTP connection ends, or more accurately, the HTTP request ends, and the next time is a new request and a new response. Therefore, there is no long connection at all. Then naturally, there is no short connection.
According to the code, the TCP long connection, that is, the socket has not been closed and is always connected.
How to set up a long connection?
We only need to set connection: keep alive in the request header, so that when our server receives the request, it will not take the initiative to close the socket connection. When a web page is opened, the TCP connection between the client and the server for transmitting HTTP data will not be closed. If the client accesses the web page on the server again, Will continue to use this established connection. Of course, the server set to keep alive will not permanently maintain the connection. It has a retention time, which can be set in different server software (such as Tomcat). If the client does not make an HTTP request within the specified time, it is good to disconnect and implement it in the code, that is, close the socket. To realize long connection, both client and server should support long connection. The long connection and short connection of HTTP protocol are essentially the long connection and short connection of TCP protocol.
The connection timeout can be set in Tomcat server Configure the Connector in line 69 of the XML. You can see that the default time of Tomcat long connection is 20s
In the author's second article, long connections are not handled. You can see that the connection is closed after each http request, and then a new connection will be made
Next, let's optimize it (in fact, it's very simple, just don't turn off the socket connection and set the timeout):
The main code is modified in TaskService in the second article. The modified code and code are pasted here first, and the complete code is pasted at the end of the article. The author also limits the number of connections here to prevent exceptions caused by a thread processing too many requests. With the line pool, the number of threads in active state can be increased to process requests.
Neither HttpServletRequest nor HttpServletResponse can close the flow connection, because if the flow obtained through the socket is closed, the socket will also be closed, which will cause an exception.
public void run() {
long connectionTime = 0L;//Record connection time
while (flag){
try {
connectionCount++;
logger.info(Thread.currentThread().getName()+",This long connection has been processed"+connectionCount+"Secondary request");
if(connectionCount > MAX_REQUEST_COUNT){
connection = "close";
}
//http request encapsulation. If there is no request, it will be blocked here
HttpServletRequest req = new HttpServletRequest(socket);
//Respond to requests
HttpServletResponse resp = new HttpServletResponse(req, socket);
} catch (Exception e) {
logger.error("Thread running error",e);
connection = "close";
}finally {
//Timeout close connection
if(connectionTime == 0){
connectionTime = System.currentTimeMillis();
}else {
connectionTime = System.currentTimeMillis() - connectionTime;
if(connectionTime > 20000){//The timeout can be read from the xml configuration file
flag = false;
connection = "close";//Close connection
}
}
//The number of connections exceeded and the connection was closed
if("close".equals(connection)){
flag = false;
//Close the resource if it is not a long link
try {
if(socket != null){
socket.close();
}
} catch (IOException ioException) {
logger.error(ioException.getMessage());
}
try {
if(in != null){
in.close();
}
} catch (IOException ioException) {
logger.error(ioException.getMessage());
}
try {
if(out != null){
out.close();
}
} catch (IOException ioException) {
logger.error(ioException.getMessage());
}
}
}
}
}
3. Complete project code
Referring to the code of phase II, only a few modifications have been made here.
Remember that neither HttpServletRequest nor HttpServletResponse can close the flow connection, because if the flow obtained through the socket is closed, the socket will also be closed, which will cause exceptions. Close the socket connection and process it in TaskService
3.1 MyCatServer
package com.fx.tomcat;
import org.apache.log4j.Logger;
import org.dom4j.Document;
import org.dom4j.DocumentException;
import org.dom4j.Element;
import org.dom4j.io.SAXReader;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.parsers.ParserConfigurationException;
import java.io.*;
import java.net.ServerSocket;
import java.net.Socket;
import java.util.HashMap;
import java.util.Map;
import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicInteger;
/**
* @author:fengxian
* @date: 2022/1/17 17:02
* Description:
*/
public class MyCatServer {
private final static Logger logger = Logger.getLogger(MyCatServer.class);
private static Map<String, Map<String,String>> map;//Used to wrap configuration files
private static boolean flag=true;
public static void main(String[] args) {
MyCatServer myCatServer = new MyCatServer();
//Get the port of the service from the configuration file
int tomcatServicePort = Integer.parseInt(map.get("Connector").get("port"));
//Use different multithreading policies according to the configuration file
if("true".equalsIgnoreCase(map.get("Connector").get("executor"))){
myCatServer.startServerByThreadPool(tomcatServicePort);
}else {
myCatServer.startServerByThread(tomcatServicePort);
}
}
/**
* Not through thread pool
*/
private void startServerByThread(int port){
try (ServerSocket ss = new ServerSocket(port)) {
while (flag){
Socket socket = ss.accept();
logger.debug("There are user requests,He is"+socket.getLocalSocketAddress());
new Thread(new TaskService(socket)).start();
}
} catch (Exception e) {
logger.error("Server startup exception"+e);
}
}
/**
* Thread pool open thread
*/
private void startServerByThreadPool(int port) {
//The size of the core thread pool to obtain the number of cores of the server processor
int corePoolSize = Runtime.getRuntime().availableProcessors();
//Maximum number of threads in the core thread pool
int maxPoolSize=corePoolSize * 2;
//Maximum idle time of thread
long keepAliveTime=10;
//Time unit
TimeUnit unit=TimeUnit.SECONDS;
//The capacity of the blocking queue is 2. At most two idle tasks can be put in
BlockingQueue<Runnable> workQueue=new ArrayBlockingQueue<>(maxPoolSize * 4);
//Thread creation factory
ThreadFactory threadFactory = new NameThreadFactory();
//Thread pool reject policy
RejectedExecutionHandler handler = new MyIgnorePolicy();
//Thread pool performer
ThreadPoolExecutor executor;
//Uncontrolled thread creation is not allowed. Thread pool must be used
executor = new ThreadPoolExecutor(corePoolSize,maxPoolSize,keepAliveTime,unit,workQueue,threadFactory,handler);
//Pre start all core threads to improve efficiency
executor.prestartAllCoreThreads();
try (ServerSocket ss = new ServerSocket(port)) {
while(flag){
Socket socket = ss.accept();
logger.debug("There are user requests,He is"+socket.getLocalSocketAddress());
//Perform tasks
executor.submit(new TaskService(socket));
}
} catch (Exception e) {
logger.error("Server startup exception:{}",e);
} finally {
//Thread pool shutdown
executor.shutdown();
}
}
/**
* Thread factory
*/
static class NameThreadFactory implements ThreadFactory {
Logger logger = Logger.getLogger(NameThreadFactory.class);
//Thread id, AtomicInteger, atomic class
private final AtomicInteger threadId=new AtomicInteger(1);
@Override
public Thread newThread(Runnable r) {
Thread t = new Thread(r, "thread " + threadId.getAndIncrement());//Equivalent to i++
logger.info(t.getName()+"Has been created");
return t;
}
}
/**
* Reject policy after thread pool BlockingQueue is full
*/
public static class MyIgnorePolicy implements RejectedExecutionHandler {
@Override
public void rejectedExecution(Runnable r, ThreadPoolExecutor e) {
logger.error("Thread pool+"+ e.toString()+r.toString()+"Rejected");
}
}
//dom4j parsing xml
static {
map = new ConcurrentHashMap<>();
Map<String,String> sonMap = new HashMap<>();
SAXReader saxReader = new SAXReader();
Document document = null;
try {
document = saxReader.read("conf/server.xml");
} catch (DocumentException e) {
logger.error("Configuration file parsing failed{}",e);
}
assert document != null;
Element root = document.getRootElement();
Element connector = root.element("Service").element("Connector");
sonMap.put("port",connector.attributeValue("port"));
sonMap.put("protocol",connector.attributeValue("protocol"));
sonMap.put("connectionTimeout",connector.attributeValue("connectionTimeout"));
sonMap.put("redirectPort",connector.attributeValue("redirectPort"));
sonMap.put("executor",connector.attributeValue("executor"));
map.put("Connector",sonMap);
}
}
3.2 TaskService
package com.fx.tomcat;
import org.apache.log4j.Logger;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.Socket;
import java.time.Duration;
import java.time.Instant;
/**
* @author: fengxian
* @date: 2022/1/18 9:09
* Description:
*/
public class TaskService implements Runnable{
private final Logger logger = Logger.getLogger(TaskService.class);
private Socket socket;
private InputStream in = null;
private OutputStream out = null;
private String connection="keep-alive";
private boolean flag;
private final int MAX_REQUEST_COUNT = 30;
private int connectionCount;//Number of connections
public TaskService(Socket socket){
this.socket=socket;
flag = true;
}
@Override
public void run() {
long connectionTime = 0L;
while (flag){
try {
connectionCount++;
logger.info(Thread.currentThread().getName()+",This long connection has been processed"+connectionCount+"Secondary request");
if(connectionCount > MAX_REQUEST_COUNT){
connection = "close";
}
//http request encapsulation. If there is no request, it will be blocked here
HttpServletRequest req = new HttpServletRequest(socket);
//Respond to requests
HttpServletResponse resp = new HttpServletResponse(req, socket);
} catch (Exception e) {
logger.error("Thread running error",e);
connection = "close";
}finally {
//Timeout close connection
if(connectionTime == 0){
connectionTime = System.currentTimeMillis();
}else {
connectionTime = System.currentTimeMillis() - connectionTime;
if(connectionTime > 20000){//The timeout can be read from the xml configuration file
flag = false;
connection = "close";//Close connection
}
}
//The number of connections exceeded and the connection was closed
if("close".equals(connection)){
flag = false;
//Close the resource if it is not a long link
try {
if(socket != null){
socket.close();
}
} catch (IOException ioException) {
logger.error(ioException.getMessage());
}
try {
if(in != null){
in.close();
}
} catch (IOException ioException) {
logger.error(ioException.getMessage());
}
try {
if(out != null){
out.close();
}
} catch (IOException ioException) {
logger.error(ioException.getMessage());
}
}
}
}
}
}
HttpServletResponse and HttpServletRequest only eliminate all the codes that close the flow, and others are not modified
4. Project demonstration effect and new problem thinking
Here's another interesting point
**Why do you set up long connections or generate so many new socket connections** Did you time out?
Let's set the timeout as long as the sky and long as the earth, and test it again
Test results:
We will find that six socket connections will still be generated, but we will find that six socket connections will be generated each time after we test several times
Why?
That's because the browser can't request resources only by one thread, which will cause slow speed. Therefore, the browser will set the default maximum number of HTTP concurrent connections for the same server. The default number of concurrent connections for mainstream browsers is as follows:
| Browser | HTTP/1.1 | HTTP/1.0 |
|---|---|---|
| IE 8,9 | 6 | 6 |
| IE 6,7 | 2 | 4 |
| Firefox 17 | 6 | 6 |
| Firefox 3 | 6 | 6 |
| Firefox 2 | 2 | 8 |
| Safari 3,4 | 4 | 4 |
| Chrome 1,2 | 6 | ? |
| Chrome 3 | 4 | 4 |
| Chrome 4+ | 6 | ? |
We all know that @ WebServlet can be used instead of xml file for address mapping in Tomcat. In the next article, the author will customize this annotation and realize the access of Servlet dynamic resources. The process of customizing annotations can be seen in the writer's article, Annotation learning 1. Java built-in annotation and annotation writing and Annotation learning II. Using annotations to imitate junit test framework First, try to write a few comments.