Circuit breaker function of Spring Cloud Gateway

Welcome to my GitHub

Content: classification and summary of all original articles and supporting source code, involving Java, Docker, Kubernetes, DevOPS, etc;

Overview of this article

Learn more about the circuit breaker function of Spring Cloud Gateway:

Let's talk about theory first
Then determine the technology stack by combining the information of the official and the great God
Then start to develop, first implement and then verify
Strike while the iron is hot and see its source code
Finally, review the shortcomings (which will be addressed in the next article)

About circuit breaker

The following figure is from the official document of resilience4j, which introduces what is a circuit breaker:

In CLOSED status, normal release is requested
When the request failure rate reaches the set threshold, it changes to OPEN status. At this time, all requests are not released
After the duration of the OPEN state is set, enter the half OPEN state (HALE_OPEN) and let go of some requests
In the half open state, if the failure rate is lower than the set threshold, it will enter the CLOSE state, that is, all release
In the half OPEN state, if the failure rate is higher than the set threshold, it will enter the OPEN state, that is, all will not be released

Confirm concept

There is a concept to be confirmed first, that is, spring cloud circuit breaker and Spring Cloud Gateway circuit breaker function are not the same concept. The Spring Cloud Gateway circuit breaker function also involves a filter, that is, the circuit breaker is used under the rules of the filter:

This article focuses on how the Spring Cloud Gateway configures and uses the circuit breaker. Therefore, the details of Resilience4J will not be discussed. If you want to know more about Resilience4J, the recommended information is Spring Cloud Circuit Breaker

About Spring Cloud circuit breaker

First look at the Spring Cloud circuit breaker, as shown in the following figure. Hystrix and Sentinel are familiar concepts:

About the circuit breaker function of Spring Cloud Gateway

Take a look at the official documents of Spring Cloud Gateway, as shown in the following figure. Several key points will be introduced later:

The figure above reveals several key information:

Spring Cloud Gateway has built-in circuit breaker filter,
The specific method is to use the API of Spring Cloud circuit breaker to encapsulate the routing logic of gateway into the circuit breaker
Libraries with multiple circuit breakers can be used in Spring Cloud Gateway (unfortunately, they are not listed)
Resilience4J is out of the box for Spring Cloud

In short, the circuit breaker function of Spring Cloud Gateway is realized through the built-in filter, which uses the Spring Cloud circuit breaker;
Officials say that multiple circuit breaker libraries can be used in Spring Cloud Gateway, but they don't say what they are, which is depressing. At this time, let's learn about a cow's point of view: Piotr Mi ń kowski is the author of the following book:

Piotr Mi ń kowski's Blog The circuit breaker function of Spring Cloud Gateway is introduced in detail, as shown in the figure below. Several important information will be mentioned later:

In the figure above, you can get three key information:

Since version 2.2.1, Spring Cloud Gateway has integrated the circuit breaker implementation of Resilience4J
Netflix's Hystrix has entered the maintenance phase (can it be understood as retiring?)
Netflix's Hystrix is still available but deprecated, and future versions of Spring Cloud may not support it

When it comes to official documents, take resilience4 as an example (as shown in the figure below). Timid I seem to have no other choice, just Resilience4J:

That's all for the theoretical analysis. Next, start the actual combat. The specific steps are as follows:

Preparations: the service provider adds a new web interface / Account / {ID} , which can return immediately or delay 500 milliseconds according to different input parameters
Add a sub project named circuitbreaker gateway , which is a Spring Cloud Gateway application with circuit breaker function
Write unit test code in circuit breaker gateway to verify whether the circuit breaker is normal
Run the unit test code and observe whether the circuit breaker is effective
Add fallback to the circuit breaker and verify that it is effective
Do a simple source code analysis, one for students who want to deeply understand the circuit breaker to clear the source code path, and the other to test their previous knowledge of springboot. Is it helpful to read the source code

Source download

The complete source code in this actual combat can be downloaded from GitHub. The address and link information are shown in the table below( https://github.com/zq2599/blog_demos):

name	link	remarks
Project Home	https://github.com/zq2599/blog_demos	The project is on the GitHub home page
git warehouse address (https)	https://github.com/zq2599/blog_demos.git	The warehouse address of the source code of the project, https protocol
git warehouse address (ssh)	git@github.com:zq2599/blog_demos.git	The project source code warehouse address, ssh protocol

There are multiple folders in this git project. The source code of this article is in the spring cloud tutorials folder, as shown in the red box below:

There are several sub projects under the spring cloud tutorials folder. The code of this chapter is circuit breaker gateway , as shown in the red box below:

preparation

We should prepare a controllable web interface and control its success or failure through parameters, so as to trigger the circuit breaker
In the actual combat of this article, the service provider is still provider Hello . In order to meet the needs of this actual combat, we add a web interface in Hello.java file, and the corresponding source code is as follows:

    @RequestMapping(value = "/account/{id}", method = RequestMethod.GET)
    public String account(@PathVariable("id") int id) throws InterruptedException {
        if(1==id) {
            Thread.sleep(500);
        }

        return Constants.ACCOUNT_PREFIX + dateStr();
    }

The above code is very simple: it is to receive the id parameter. If it is equal to 1, it will delay for 500 milliseconds, and if it is not equal to 1, it will return immediately
If the circuit breaker is set to more than 200ms, it will fail. By controlling the value of the id parameter, we can simulate whether the request succeeds or fails, this is the key to verify the function of the circuit breaker
Ready to finish, start writing code

actual combat

Add a sub project circuit breaker gateway under the parent project spring cloud tutorials
Add the following dependencies

<dependency>
	<groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-circuitbreaker-reactor-resilience4j</artifactId>
</dependency>

The configuration file application.yml is as follows:

server:
  #Service port
  port: 8081
spring:
  application:
    name: circuitbreaker-gateway
  cloud:
    gateway:
      routes:
        - id: path_route
          uri: http://127.0.0.1:8082
          predicates:
            - Path=/hello/**
          filters:
            - name: CircuitBreaker
              args:
                name: myCircuitBreaker

Startup class:

package com.bolingcavalry.circuitbreakergateway;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class CircuitbreakerApplication {
    public static void main(String[] args) {
        SpringApplication.run(CircuitbreakerApplication.class,args);
    }
}

The configuration class is as follows, which is the parameter configuration related to the circuit breaker:

package com.bolingcavalry.circuitbreakergateway.config;

import io.github.resilience4j.circuitbreaker.CircuitBreakerConfig;
import io.github.resilience4j.timelimiter.TimeLimiterConfig;
import org.springframework.cloud.circuitbreaker.resilience4j.ReactiveResilience4JCircuitBreakerFactory;
import org.springframework.cloud.circuitbreaker.resilience4j.Resilience4JConfigBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import java.time.Duration;

@Configuration
public class CustomizeCircuitBreakerConfig {

    @Bean
    public ReactiveResilience4JCircuitBreakerFactory defaultCustomizer() {

        CircuitBreakerConfig circuitBreakerConfig = CircuitBreakerConfig.custom() //
                .slidingWindowType(CircuitBreakerConfig.SlidingWindowType.TIME_BASED) // The type of sliding window is time window
                .slidingWindowSize(10) // The size of the time window is 60 seconds
                .minimumNumberOfCalls(5) // At least 5 calls are required in the unit time window to start statistical calculation
                .failureRateThreshold(50) // When the call failure rate reaches 50% within the unit time window, the circuit breaker will be started
                .enableAutomaticTransitionFromOpenToHalfOpen() // The circuit breaker is allowed to automatically change from open state to half open state
                .permittedNumberOfCallsInHalfOpenState(5) // The number of normal calls allowed in the half open state
                .waitDurationInOpenState(Duration.ofSeconds(5)) // It takes 60 seconds for the circuit breaker to change from open state to half open state
                .recordExceptions(Throwable.class) // All exceptions are treated as failures
                .build();

        ReactiveResilience4JCircuitBreakerFactory factory = new ReactiveResilience4JCircuitBreakerFactory();
        factory.configureDefault(id -> new Resilience4JConfigBuilder(id)
                .timeLimiterConfig(TimeLimiterConfig.custom().timeoutDuration(Duration.ofMillis(200)).build())
                .circuitBreakerConfig(circuitBreakerConfig).build());

        return factory;
    }
}

The above code needs to be noted once: the timelimiterconfig method sets the timeout. If the service provider does not respond for more than 200 milliseconds, the Spring Cloud Gateway will return a failure to the caller
After the development is completed, the next thing to consider is how to verify

Unit test class

In order to verify the circuit breaker function of Spring Cloud Gateway, we can use Junit unit test to accurately control the request parameters and request times. The test class is as follows. It can be seen that the test class will send 100 requests continuously. In the first 50 times, the request parameters are always switched between 0 and 1. When the parameter is equal to 1, the interface will have a delay of 500 milliseconds, If the 200 ms timeout limit of Spring Cloud Gateway is exceeded, a failure will be returned. If there are many failures, the circuit breaker will be disconnected:

package com.bolingcavalry.circuitbreakergateway;

import io.github.resilience4j.circuitbreaker.CircuitBreaker;
import org.junit.jupiter.api.RepeatedTest;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.autoconfigure.web.reactive.AutoConfigureWebTestClient;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.http.MediaType;
import org.springframework.test.context.junit.jupiter.SpringExtension;
import org.springframework.test.web.reactive.server.WebTestClient;

@SpringBootTest
@ExtendWith(SpringExtension.class)
@AutoConfigureWebTestClient
public class CircuitbreakerTest {

    // Total number of tests
    private static int i=0;

    @Autowired
    private WebTestClient webClient;

    @Test
    @RepeatedTest(100)
    void testHelloPredicates() throws InterruptedException {
        // When it is less than 50 times, gen switches between 0 and 1, that is, one normal time and one timeout,
        // When more than 50 times, gen is fixed to 0, and each request will not time out
        int gen = (i<50) ? (i % 2) : 0;

        // Times plus one
        i++;

        final String tag = "[" + i + "]";

        // Initiate web request
        webClient.get()
                .uri("/hello/account/" + gen)
                .accept(MediaType.APPLICATION_JSON)
                .exchange()
                .expectBody(String.class).consumeWith(result  -> System.out.println(tag + result.getRawStatusCode() + " - " + result.getResponseBody()));

        Thread.sleep(1000);
    }
}

verification

Start nacos (service provider dependent)
Promoter project provider Hello
Run the unit test class we just developed. The content intercepted on the console is as follows, which will be analyzed later:

[2]504 - {"timestamp":"2021-08-28T02:55:42.920+00:00","path":"/hello/account/1","status":504,"error":"Gateway Timeout","message":"","requestId":"594efed1"}
[3]200 - Account2021-08-28 10:55:43
[4]504 - {"timestamp":"2021-08-28T02:55:45.177+00:00","path":"/hello/account/1","status":504,"error":"Gateway Timeout","message":"","requestId":"427720b"}
[5]200 - Account2021-08-28 10:55:46
[6]503 - {"timestamp":"2021-08-28T02:55:47.227+00:00","path":"/hello/account/1","status":503,"error":"Service Unavailable","message":"","requestId":"6595d7f4"}
[7]503 - {"timestamp":"2021-08-28T02:55:48.250+00:00","path":"/hello/account/0","status":503,"error":"Service Unavailable","message":"","requestId":"169ae1c"}
[8]503 - {"timestamp":"2021-08-28T02:55:49.259+00:00","path":"/hello/account/1","status":503,"error":"Service Unavailable","message":"","requestId":"53b695a1"}
[9]503 - {"timestamp":"2021-08-28T02:55:50.269+00:00","path":"/hello/account/0","status":503,"error":"Service Unavailable","message":"","requestId":"4a072f52"}
[10]504 - {"timestamp":"2021-08-28T02:55:51.499+00:00","path":"/hello/account/1","status":504,"error":"Gateway Timeout","message":"","requestId":"4bdd96c4"}
[11]200 - Account2021-08-28 10:55:52
[12]504 - {"timestamp":"2021-08-28T02:55:53.745+00:00","path":"/hello/account/1","status":504,"error":"Gateway Timeout","message":"","requestId":"4e0e7eab"}
[13]200 - Account2021-08-28 10:55:54
[14]504 - {"timestamp":"2021-08-28T02:55:56.013+00:00","path":"/hello/account/1","status":504,"error":"Gateway Timeout","message":"","requestId":"27685405"}
[15]503 - {"timestamp":"2021-08-28T02:55:57.035+00:00","path":"/hello/account/0","status":503,"error":"Service Unavailable","message":"","requestId":"3e40c5db"}
[16]503 - {"timestamp":"2021-08-28T02:55:58.053+00:00","path":"/hello/account/1","status":503,"error":"Service Unavailable","message":"","requestId":"2bf2698b"}
[17]503 - {"timestamp":"2021-08-28T02:55:59.075+00:00","path":"/hello/account/0","status":503,"error":"Service Unavailable","message":"","requestId":"38cb1840"}
[18]503 - {"timestamp":"2021-08-28T02:56:00.091+00:00","path":"/hello/account/1","status":503,"error":"Service Unavailable","message":"","requestId":"21586fa"}
[19]200 - Account2021-08-28 10:56:01
[20]504 - {"timestamp":"2021-08-28T02:56:02.325+00:00","path":"/hello/account/1","status":504,"error":"Gateway Timeout","message":"","requestId":"4014d6d4"}
[21]200 - Account2021-08-28 10:56:03
[22]504 - {"timestamp":"2021-08-28T02:56:04.557+00:00","path":"/hello/account/1","status":504,"error":"Gateway Timeout","message":"","requestId":"173a3b9d"}
[23]200 - Account2021-08-28 10:56:05
[24]504 - {"timestamp":"2021-08-28T02:56:06.811+00:00","path":"/hello/account/1","status":504,"error":"Gateway Timeout","message":"","requestId":"aa8761f"}
[25]200 - Account2021-08-28 10:56:07
[26]504 - {"timestamp":"2021-08-28T02:56:09.057+00:00","path":"/hello/account/1","status":504,"error":"Gateway Timeout","message":"","requestId":"769bfefc"}
[27]200 - Account2021-08-28 10:56:10
[28]504 - {"timestamp":"2021-08-28T02:56:11.314+00:00","path":"/hello/account/1","status":504,"error":"Gateway Timeout","message":"","requestId":"2fbcb6c0"}
[29]503 - {"timestamp":"2021-08-28T02:56:12.332+00:00","path":"/hello/account/0","status":503,"error":"Service Unavailable","message":"","requestId":"58e4e70f"}
[30]503 - {"timestamp":"2021-08-28T02:56:13.342+00:00","path":"/hello/account/1","status":503,"error":"Service Unavailable","message":"","requestId":"367651c5"}

Analyze the return code of the above output:

504 is the error returned after timeout, and 200 is the normal return of the service provider
Both 504 and 200 return codes indicate that the request has reached the service provider, so the circuit breaker is closed at this time
After repeated 504 errors, the configured threshold is reached and the circuit breaker is triggered to open
503 that appears continuously is the return code after the circuit breaker is turned on. At this time, the request cannot reach the service provider
After consecutive 503, 504 and 200 appear alternately again to prove that they enter the half open state at this time, and then 504 reaches the threshold again to trigger the circuit breaker to turn from half open to open. After 50 times, the circuit breaker enters the closed state because it is not sending the timeout request

fallback

It can be seen from the above tests that the Spring Cloud Gateway uses the return code to inform the caller of the error information. This method is not friendly enough. We can customize the fallback to build the return information when an error is returned
To develop another web interface, yes, add a web interface to the circuit breaker gateway Project:

package com.bolingcavalry.circuitbreakergateway.controller;

import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;
import java.text.SimpleDateFormat;
import java.util.Date;

@RestController
public class Fallback {

    private String dateStr(){
        return new SimpleDateFormat("yyyy-MM-dd hh:mm:ss").format(new Date());
    }

    /**
     * Return string type
     * @return
     */
    @GetMapping("/myfallback")
    public String helloStr() {
        return "myfallback, " + dateStr();
    }
}

The configuration of application.yml is as follows. It can be seen that the fallbackuri attribute is added to the filter:

server:
  #Service port
  port: 8081
spring:
  application:
    name: circuitbreaker-gateway
  cloud:
    gateway:
      routes:
        - id: path_route
          uri: http://127.0.0.1:8082
          predicates:
            - Path=/hello/**
          filters:
            - name: CircuitBreaker
              args:
                name: myCircuitBreaker
                fallbackUri: forward:/myfallback

Run the unit test again. It can be seen that the return codes are all 200. The original errors have now become the return contents of the newly added interface:

[2]200 - myfallback, 2021-08-28 11:15:02
[3]200 - Account2021-08-28 11:15:03
[4]200 - myfallback, 2021-08-28 11:15:04
[5]200 - Account2021-08-28 11:15:05
[6]200 - myfallback, 2021-08-28 11:15:06
[7]200 - myfallback, 2021-08-28 11:15:08
[8]200 - myfallback, 2021-08-28 11:15:09
[9]200 - myfallback, 2021-08-28 11:15:10
[10]200 - myfallback, 2021-08-28 11:15:11
[11]200 - Account2021-08-28 11:15:12
[12]200 - myfallback, 2021-08-28 11:15:13
[13]200 - Account2021-08-28 11:15:14
[14]200 - myfallback, 2021-08-28 11:15:15

So far, we have completed the development and testing of the circuit breaker function of Spring Cloud Gateway. If you are smart and eager to learn, you are not satisfied with these few lines of configuration and code. If you want to have an in-depth understanding of the interior of the circuit breaker, please go on and let's talk about its source code;

Source code analysis

The construction method (bean injection) of RouteDefinitionRouteLocator has the following code to bind name and instance:

gatewayFilterFactories.forEach(factory -> this.gatewayFilterFactories.put(factory.name(), factory));

Then use this map in the loadGatewayFilters method to find the put bean above;
Final effect: if the name is equal to CircuitBreaker specified in the routing configuration, it can correspond to a bean of SpringCloudCircuitBreakerFilterFactory type, because its name method returns "CircuitBreaker", as shown in the following figure:

Question now: what is a bean of type SpringCloudCircuitBreakerFilterFactory? As shown in the red box below, SpringCloudCircuitBreakerFilterFactory is the only subclass of SpringCloudCircuitBreakerFilterFactory:

From the above figure, the filter of CircuitBreaker type should be springcloudcircuitbreakerrelience4jfilterfactory, but it is only inferred from the inheritance relationship. There is still a key evidence: does spring have a bean of springcloudcircuitbreakerrelience4jfilterfactory type?
Finally, the configuration in gatewayresilience 4jcircuitbreakerautoconfiguration is found, which proves that springcloudcircuitbreakerrelience4jfilterfactory will be instantiated and registered with spring:

@Bean
	@ConditionalOnBean(ReactiveResilience4JCircuitBreakerFactory.class)
	@ConditionalOnEnabledFilter
	public SpringCloudCircuitBreakerResilience4JFilterFactory springCloudCircuitBreakerResilience4JFilterFactory(
			ReactiveResilience4JCircuitBreakerFactory reactiveCircuitBreakerFactory,
			ObjectProvider<DispatcherHandler> dispatcherHandler) {
		return new SpringCloudCircuitBreakerResilience4JFilterFactory(reactiveCircuitBreakerFactory, dispatcherHandler);
	}

To sum up, when you configure the CircuitBreaker filter, the springcloudcircuitbreakerrelience4jfilterfactory class actually serves you, and the key code is concentrated in its parent class SpringCloudCircuitBreakerFilterFactory;
Therefore, to learn more about the breaker function of Spring Cloud Gateway, please read the SpringCloudCircuitBreakerFilterFactory.apply method

A little regret

Do you remember the content of analyzing the console output just now? It is the section in the red box in the figure below. At that time, we used the return code to infer the state of the circuit breaker:

I believe you still have doubts about Xinchen's analysis when reading this pure text. Have you determined the state of the circuit breaker according to the return code? For example, is 504 closed or half open? It's possible, so this speculation can only prove that the circuit breaker is working, but it can't determine the specific state at a certain time
Therefore, we need a more accurate way to know the state of the circuit breaker at each time, so as to have a deep understanding of the circuit breaker
In the following articles, we will go further on today's achievements and print out the status of the circuit breaker in the request. Then... Please look forward to it. Xinchen was original and never let you down;

You're not alone. Xinchen's original accompanies you all the way

Welcome to the official account: programmer Xin Chen

Wechat search "programmer Xinchen", I'm Xinchen, looking forward to traveling with you in the Java World https://github.com/zq2599/blog_demos

Keywords: Java git github Junit Spring

Added by oneofayykind on Fri, 19 Nov 2021 06:47:56 +0200

Programming VIP