Hystrix enables service isolation
- Here is how to prevent avalanche effect in RPC service call
- In microservices, we use Hystrix to realize protection services and improve fault tolerance
- Hystrix is A service protection framework that can realize fault tolerance of services in distributed (fault tolerance refers to the processing scheme after an error occurs when the service is unavailable, which is the preparation scheme), It can reduce the dependency relationship with the service (this is not A business dependency. It means that A calls B, B calls C, and C are unavailable. It should lead to the unavailability of A and B, but the problem of unavailability of C service leading to unavailability of A and B service can be avoided by using the hystrix framework)
- Avalanche effect of service: when a service has high concurrent requests, the server can't stand so many concurrent requests, resulting in service accumulation. A calls B, B calls C, and C service is unavailable, which will directly lead to the unavailability of B service and a service.
- During the experiment, we can access an interface under the condition of high concurrency. A large number of high concurrency accesses of this interface cannot affect the access of other interfaces of the service.
Introduction to Hystrix
- What is the accumulation of services?
It is the blocking of service requests. A large number of requests are waiting and cannot be accessed. Service a will be blocked when the number of requests that can be called by service B is greater than that of service C. that is, service a will be blocked when the number of requests that can be called by service B is not allowed.
For example, in the case of high concurrency, if the maximum number of threads allowed by tomcat is 50, the 51st request will be blocked when we send it, because tomcat only allows 50 threads. If the number of threads exceeds, it will wait.
- Under what circumstances did Hystrix come into being?
This framework is generated in the high concurrency scenario of microservices.
- Suppose there are two services: order service and member service. The client invokes the order service, invokes the member service through the order service, the member service returns data to the order service, and the order service returns data to the client. Here, member services are producers and order services are consumers.
If the member service generates a wait, the order service will also generate a wait. If the client has many requests to access the order service, the order service will also have a large number of requests to access the member service. At this time, it is assumed that the member service needs to wait 1.5s for each response. At this time, the order service and the client will have an avalanche effect, and the order service and the client will wait. - If the avalanche effect is not properly handled in microservices, it may lead to a series of dependent services waiting
- The role of Hystrix?
(1) Service protection (not security protection, but user experience)
- When services accumulate
(2) Isolation of services - Different service interfaces do not affect each other. Generally, semaphores (not commonly used) and thread pools are used to isolate services.
(3) Service degradation - With the fuse of service, there must be service degradation. The so-called degradation means that when a service is blown out and the server no longer calls it, the client can prepare a local fallback function callback to return a default value. Although the service level drops, it can be used at any rate, which is better than hanging up directly. Can improve the user experience
- Objective: to improve the user experience and prevent the avalanche effect of the server
(4) Current limit of service
(5) Fusing of services - Similar to the fuse in the real world, when an abnormal condition is triggered, the whole service will be blown directly instead of waiting for the service to timeout. The triggering conditions of fusing can vary according to different scenarios.
- Set a limit, for example: up to 100 requests can be accessed at the same time, and the excess requests will be placed in the cache queue. If the cache queue is full, access will be directly denied and access will not be possible (only when the cache queue is full will it be broken)
- The service fuse protects the service. It is usually used in the case of high concurrency.
Note: in general, service fusing and service degradation are used together. When the request exceeds the maximum limit, a reminder will be given. Simply put, if the service is blown out, the service will be degraded
- Why is there a service avalanche?
- The bottom layer of tomcat uses the technology of thread pool. By default, the bottom layer of tomcat will create a thread pool to help us process requests. One thread pool manages all requests and one thread is used to manage one request. Assuming that the thread pool can create 50 threads at most, that is, there can be 50 requests at most. If there are more than 50, it will wait; If there is an order micro service with two interfaces, / orderIndex and / findOrderIndex, then 50 threads call the / orderIndex interface at the same time. When another thread (that is, another request) accesses / findOrderIndex, because tomcat has only one thread pool, and the maximum number of threads is assumed to be 50, Because all threads handle the / orderIndex interface, no thread can serve the interface accessing / findOrderIndex, so we will have service accumulation (i.e. service blocking) at this time.
- The avalanche effect of services causes services to accumulate in the same thread pool, because in the same thread pool, all requests are accessed in one service. At this time, other services will not have threads to accept the requested access, so the avalanche effect of services will occur
- Service isolation:
- Each service interface does not affect each other.
- There are two ways to implement service isolation:
(1) Thread pool mode: each service interface has its own independent thread pool to manage and run its own interface. - That is, each interface has its own independent thread pool, and each interface does not affect each other.
- Using thread pool isolation can completely isolate third-party applications, and request threads can be quickly put back (a request is processed by one thread, now a request is processed by the thread pool of the request, and the thread that originally processed the request can continue to accept other requests)
- The requesting thread can continue to accept new requests. If there is a problem, the thread pool isolation is independent and will not affect other applications.
- Disadvantages: the use of thread pool will cause great CPU overhead,
(2) Counter mode (also called semaphore) - Use atomic counters (or semaphores) to record how many threads are currently running.
- Not much. The bottom layer uses atomic counters to set its own threshold for each service (that is, the maximum number of requests that the current service can receive). When the number of accesses to our interface is greater than the threshold, we can implement our own rejection policy and degrade the service. If it does not exceed the threshold, it will pass. At this time, the counter requests to increase by 1, and the counter decreases by 1 after returning data
Project integration Hystrix
- The startup should be placed in the run method of the class that inherits Hystrix.
- When the execute() method is called, the run method will be called automatically.
- The purpose is that each interface has its own independent thread pool. This method is used to realize the isolation of services. When the cache queue is full, first meet the conditions of service fusing, and then downgrade the service.
- The code is as follows:
(1) The code structure is as follows
(1) Import pom file
- High pogfm file
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.xiyou</groupId>
<artifactId>highbingfa</artifactId>
<version>1.0-SNAPSHOT</version>
<modules>
<module>membeer</module>
<module>order</module>
</modules>
<packaging>pom</packaging>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.0.0.RELEASE</version>
</parent>
</project>
- pom file of order
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<artifactId>highbingfa</artifactId>
<groupId>com.xiyou</groupId>
<version>1.0-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>order</artifactId>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.httpcomponents/httpclient -->
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
</dependency>
<!-- https://mvnrepository.com/artifact/com.alibaba/fastjson -->
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>fastjson</artifactId>
<version>1.2.47</version>
</dependency>
<dependency>
<groupId>com.netflix.hystrix</groupId>
<artifactId>hystrix-metrics-event-stream</artifactId>
<version>1.5.12</version>
</dependency>
<dependency>
<groupId>com.netflix.hystrix</groupId>
<artifactId>hystrix-javanica</artifactId>
<version>1.5.12</version>
</dependency>
</dependencies>
</project>
- pom file of member
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<artifactId>highbingfa</artifactId>
<groupId>com.xiyou</groupId>
<version>1.0-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>membeer</artifactId>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.httpcomponents/httpclient -->
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
</dependency>
<!-- https://mvnrepository.com/artifact/com.alibaba/fastjson -->
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>fastjson</artifactId>
<version>1.2.47</version>
</dependency>
</dependencies>
</project>
(2) Java file
- Member
(1)MemberController
package com.xiyou.controller;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import java.util.HashMap;
import java.util.Map;
@RestController
@RequestMapping("/member")
public class MemberController {
/**
* Call the member service and do a simple test
* @return
* @throws InterruptedException
*/
@GetMapping("/memberIndex")
public Object memberIndex() throws InterruptedException {
Map<String, Object> hashMap = new HashMap<>();
hashMap.put("code", "200");
hashMap.put("msg", "call member Return success");
Thread.sleep(1500);
int i = 1/0;
return hashMap;
}
}
(2)MemberApplication
package com.xiyou;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
@SpringBootApplication
public class MemberApplication {
public static void main(String[] args) {
SpringApplication.run(MemberApplication.class, args);
}
}
(3) yml file
server: port: 8888
- Order
(1)OrderApplication
package com.xiyou;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
@SpringBootApplication
public class OrderApplication {
public static void main(String[] args) {
SpringApplication.run(OrderApplication.class, args);
}
}
(2) yml file
server:
port: 8889
# Configure the maximum number of tomcat threads (maximum 20)
tomcat:
max-threads: 20
(4) Controller file
OrderController
package com.xiyou.controller;
import com.xiyou.hystrix.OrderHystrixCommand;
import com.xiyou.service.OrderService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
/**
* Order service
*/
@RestController
@RequestMapping("/order")
public class OrderController {
@Autowired
private OrderService orderService;
@GetMapping("/orderIndexHystrix")
public Object orderIndexHystrix() {
return new OrderHystrixCommand(orderService).execute();
}
}
(4)OrderHystrixCommand
package com.xiyou.hystrix;
import com.alibaba.fastjson.JSONObject;
import com.netflix.hystrix.*;
import com.xiyou.service.OrderService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
/**
* Isolation of services with Hystrix
* You need to inherit the Hystrix class. Its generic type is the return result of the function that returns the result. The return result of the run method is because it calls the execute call (the run method is executed by default)
*/
@Service
public class OrderHystrixCommand extends HystrixCommand<JSONObject> {
@Autowired
private OrderService orderService;
/**
* Parametric structure
* We also need to customize some parameters here, because Hystrix does not provide parameterless construction
* @param orderService
*/
public OrderHystrixCommand(OrderService orderService){
super(setter());
this.orderService = orderService;
}
/**
* When the main method calls its execute method, the run method will be executed by default
* @return
* @throws Exception
*/
@Override
protected JSONObject run() throws Exception {
JSONObject member = orderService.getMember();
System.out.println("The name of the current thread is: " + Thread.currentThread().getName() + " ,Order service calls member service: member: " + member);
return member;
}
/**
* Assign the relevant value of Hystrix
* @return
*/
public static Setter setter() {
// Service grouping
HystrixCommandGroupKey groupKey = HystrixCommandGroupKey.Factory.asKey("members");
// Service identification
HystrixCommandKey commandKey = HystrixCommandKey.Factory.asKey("member");
// The name of the thread pool
// The name of each thread pool should not be the same, because an interface should provide a thread pool
HystrixThreadPoolKey threadPoolKey = HystrixThreadPoolKey.Factory.asKey("member-pool");
// Configure the thread pool. The size of each thread pool is 10, the thread survival time is 15s, and the queue waiting threshold is 100. If it exceeds 100 times, execute will be rejected. In fact, we can receive 110 threads per interface at most (thread pool plus cache queue)
HystrixThreadPoolProperties.Setter threadPoolProperties = HystrixThreadPoolProperties.Setter().withCoreSize(10)
.withKeepAliveTimeMinutes(15).withQueueSizeRejectionThreshold(100);
// Set related properties of Hystrix
HystrixCommandProperties.Setter commandProperties = HystrixCommandProperties.Setter()
// Use thread pool to realize service isolation
.withExecutionIsolationStrategy(HystrixCommandProperties.ExecutionIsolationStrategy.THREAD)
// Whether the timeout setting is enabled for execution, which means that the current thread will not timeout no matter how long the program is executed
.withExecutionTimeoutEnabled(false);
return HystrixCommand.Setter.withGroupKey(groupKey).andCommandKey(commandKey).andThreadPoolKey(threadPoolKey)
.andThreadPoolPropertiesDefaults(threadPoolProperties)
.andCommandPropertiesDefaults(commandProperties);
}
@Override
protected JSONObject getFallback() {
// If the fuse breaks in the Hystrix, when the service is unavailable, directly execute the FallBack method
System.out.println("System error");
JSONObject jsonObject = new JSONObject();
jsonObject.put("code", "500");
jsonObject.put("msg", "System error!");
return jsonObject;
}
}
(5)orderService
package com.xiyou.service;
import com.alibaba.fastjson.JSONObject;
import com.xiyou.utils.HttpClientUtils;
import org.springframework.stereotype.Service;
@Service
public class OrderService {
/**
* Remote call to get the data of member end
* @return
*/
public JSONObject getMember() {
JSONObject jsonObject = HttpClientUtils.httpGet("http://localhost:8888/member/memberIndex");
return jsonObject;
}
}
(6)HttpClientUtils
package com.xiyou.utils;
import com.alibaba.fastjson.JSONObject;
import org.apache.http.HttpEntity;
import org.apache.http.HttpStatus;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.util.EntityUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.IOException;
/**
* HttpClient4.3 Tool class
*
* @author hang.luo
*/
public class HttpClientUtils {
private static Logger logger = LoggerFactory.getLogger(HttpClientUtils.class); // Logging
private static RequestConfig requestConfig = null;
static {
// Set request and transmission timeout
requestConfig = RequestConfig.custom().setSocketTimeout(2000).setConnectTimeout(2000).build();
}
/**
* post Request transfer json parameters
*
* @param url
* url address
* @param json
* parameter
* @return
*/
public static JSONObject httpPost(String url, JSONObject jsonParam) {
// Return result of post request
CloseableHttpClient httpClient = HttpClients.createDefault();
JSONObject jsonResult = null;
HttpPost httpPost = new HttpPost(url);
// Set request and transmission timeout
httpPost.setConfig(requestConfig);
try {
if (null != jsonParam) {
// Solve the problem of Chinese garbled code
StringEntity entity = new StringEntity(jsonParam.toString(), "utf-8");
entity.setContentEncoding("UTF-8");
entity.setContentType("application/json");
httpPost.setEntity(entity);
}
CloseableHttpResponse result = httpClient.execute(httpPost);
// The request was sent successfully and received a response
if (result.getStatusLine().getStatusCode() == HttpStatus.SC_OK) {
String str = "";
try {
// Read the json string data returned by the server
str = EntityUtils.toString(result.getEntity(), "utf-8");
// Convert json string to json object
jsonResult = JSONObject.parseObject(str);
} catch (Exception e) {
logger.error("post Request Submission failed:" + url, e);
}
}
} catch (IOException e) {
logger.error("post Request Submission failed:" + url, e);
} finally {
httpPost.releaseConnection();
}
return jsonResult;
}
/**
* post Request to transfer String parameters, for example: name = Jack & sex = 1 & type = 2
* Content-type:application/x-www-form-urlencoded
*
* @param url
* url address
* @param strParam
* parameter
* @return
*/
public static JSONObject httpPost(String url, String strParam) {
// Return result of post request
CloseableHttpClient httpClient = HttpClients.createDefault();
JSONObject jsonResult = null;
HttpPost httpPost = new HttpPost(url);
httpPost.setConfig(requestConfig);
try {
if (null != strParam) {
// Solve the problem of Chinese garbled code
StringEntity entity = new StringEntity(strParam, "utf-8");
entity.setContentEncoding("UTF-8");
entity.setContentType("application/x-www-form-urlencoded");
httpPost.setEntity(entity);
}
CloseableHttpResponse result = httpClient.execute(httpPost);
// The request was sent successfully and received a response
if (result.getStatusLine().getStatusCode() == HttpStatus.SC_OK) {
String str = "";
try {
// Read the json string data returned by the server
str = EntityUtils.toString(result.getEntity(), "utf-8");
// Convert json string to json object
jsonResult = JSONObject.parseObject(str);
} catch (Exception e) {
logger.error("post Request Submission failed:" + url, e);
}
}
} catch (IOException e) {
logger.error("post Request Submission failed:" + url, e);
} finally {
httpPost.releaseConnection();
}
return jsonResult;
}
/**
* Send get request
*
* @param url
* route
* @return
*/
public static JSONObject httpGet(String url) {
// get request return result
JSONObject jsonResult = null;
CloseableHttpClient client = HttpClients.createDefault();
// Send get request
HttpGet request = new HttpGet(url);
request.setConfig(requestConfig);
try {
CloseableHttpResponse response = client.execute(request);
// The request was sent successfully and received a response
if (response.getStatusLine().getStatusCode() == HttpStatus.SC_OK) {
// Read the json string data returned by the server
HttpEntity entity = response.getEntity();
String strResult = EntityUtils.toString(entity, "utf-8");
// Convert json string to json object
jsonResult = JSONObject.parseObject(strResult);
} else {
logger.error("get Request Submission failed:" + url);
}
} catch (IOException e) {
logger.error("get Request Submission failed:" + url, e);
} finally {
request.releaseConnection();
}
return jsonResult;
}
}