Spring cloud gateway is a gateway product that replaces zuul. It is based on Spring 5, Spring boot 2.0 and above, and Reactor, and provides arbitrary routing matching, assertion, and filtering functions.
The author's previous system also used Spring cloud gateway as the gateway for background application access. The version information adopted is:
| assembly | edition | other |
|---|---|---|
| spring boot | 2.1.7.RELEASE | |
| spring cloud | Greenwich.SR2 | |
| spring cloud gateway | 2.1.2.RELEASE |
The code of one route is as follows:
@Bean
public RouteLocator customRouteLocator(RouteLocatorBuilder builder) {
RouteLocatorBuilder.Builder routes = builder.routes();
RouteLocatorBuilder.Builder serviceProvider = routes
.route("accept",
r -> r
.header(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_UTF8_VALUE)
.and()
.method(HttpMethod.POST)
.and()
.readBody(String.class, readBody -> true)
.and()
.path("/gateway-accept/**")
.filters(f -> {
f.rewritePath("/gateway-accept/(?<path>.*)", "/${path}");
return f;
})
.uri("lb://ACCEPT-MICROSERVICE"));
return serviceProvider.build();
}
The background version is spring boot 2.1.2 Release, a service with built-in Tomcat.
At the beginning, the system was running well. Recently, the business has been busy. The front-end call said that it occasionally returned such problems:
{"timestamp":"2021-01-06T01:50:00.468+0000","path":"/gateway-accept/policy","status":500,"error":"Internal Server Error","message":"Connection prematurely closed BEFORE response"}
It literally means "close the connection prematurely before responding". Check the log of the background service. There is no call information at all. There is no problem calling again. There is no problem with the service and gateway. What's the matter?
reason
Because this is a problem with the Spring cloud gateway, someone must have met it. First, go to the issues on the github of the gateway to take a chance.
Sure enough, find "Connection prematurely closed BEFORE response" in issues, and list more than a dozen, including seven or eight related ones. Read them one by one, and finally an issue mentioned the same problem:
https://github.com/spring-cloud/spring-cloud-gateway/issues/1148
The summary is as follows:
When the gateway calls the background service, it will use the connections in the httpclient connection pool
The connection of the httpclient connection pool used by the gateway has a parameter: Max idle time, which roughly refers to how long the connection will be closed without use. If set to null, the connection will not close.
The background service also has a corresponding connection, corresponding to the connection of the connection pool. The parameter keepAliveTimeout means that the connection of the background service will be closed automatically when it is idle. The default value is the value of the connection timeout parameter. If it is - 1, there will be no time limit. The default value is 60s, but the general server XML is set to 20s
Important: if the connection Max idle time of the connection pool of the gateway is not set (null), the connection timeout of the background service is 20s
-
Suppose the network connection pool has a connection (gateway-connect-1) corresponding to the connection of the background service (server-connect-1)
-
When the front end requests, the connection assigned to the request by the gateway is exactly (gateway-connect-1), and the back end initiates a request call
-
At the same time, the server-connect-1 has been idle for 20 seconds and is automatically closed;
-
It is conceivable that the server does not have a connection corresponding to (gateway-connect-1), so an exception occurs.
Spring.com needs to be set in the gateway layer cloud. gateway. httpclient. pool. max-idle-time
The server needs to be set on the server The connection timeout value should be appropriately greater than the max idle time of the gateway layer, which means that the idle time of the gateway layer for the back-end connection should be less than that of the back-end service, so that invalid gateway layer connections will not be obtained.
solve
According to the above description, I add in yml:
spring: cloud: gateway: httpclient: pool: max-idle-time: 5000 max-connections: 30
The yellow base price of Max idle time is found in the idea. This configuration cannot be found. Click the OK Max connections to locate it
picture
Originally, the connection pool of the gateway version 2.1.2 I used did not provide the max idle time parameter at all. Which version can provide the max idle time parameter?
I have created a new gateway service with the following version:
| assembly | edition | other |
|---|---|---|
| spring boot | 2.3.4.RELEASE | |
| spring cloud | Hoxton.SR1 | |
| spring cloud gateway | 2.2.1.RELEASE |
Set in gateway service layer:
spring: cloud: gateway: httpclient: pool: max-idle-time: 10000
i click Max idle time to see that this parameter has been provided:
picture
Backend service settings (embedded Tomcat for backend):
server: tomcat: connection-timeout: 20000
Service call interface:
@GetMapping(value = "/test")
public String get() throws InterruptedException {
Thread.sleep(10);
return "laza";
}
First setting
The gateway does not set Max idle time
Backend service setting connection time: 100
Use jmeter to test, and the configuration is as follows:
picture
Click start, and an error appears in the background:
picture
Second setting
Gateway setting Max idle time: 10000
Backend service setting connection time: 20000
jmeter settings are the same as above, and everything is normal.
It also has something to do with the version. The version I use in production does not support the setting of the max idle time parameter, so I need to upgrade the version used by the gateway.
follow-up
At the end of issues, I found this:
picture
After clicking in, I found that it was Lovnx, a classmate of ant Jinfu, who explained this problem in detail. If you are interested, you can go and have a look. In his article, he mentioned:
reactor.netty.pool.leasingStrategy=lifo the default FIFO is changed to LIFO, because LIFO can ensure that the maximum probability of obtained connections is recently used, that is, hot connections are always hot connections, and connections that are never used can be recycled. The idea of LRU (in Mandarin)
I checked the reactor netty version. Spring cloud gateway 2.2.1 The reactor netty version provided by release is 0.9.2 RELEASE
<dependency> <groupId>io.projectreactor.netty</groupId> <artifactId>reactor-netty</artifactId> <version>0.9.2.RELEASE</version> <scope>compile</scope> </dependency>
- END -