load balancing
The English name of Load Balance is Load Balance, which means that the load (work task) is balanced and distributed to multiple operating units for operation.
In the distributed architecture, there must be multiple copies of a service background. At this time, the load balancing component is required to allocate the traffic to these copies.
Load balancing can be realized by hardware or software, such as F5, Nginx, API gateway, etc.
Server load balancing
Server side load balancing is to process requests on the server side. For example, when using Nginx, requests arrive at Nginx, which performs load balancing and forwards requests to specific background services.
The client does not know the specific algorithm of load balancing, nor does it know the specific background service address.
Client load balancing
Client load balancing is that the load balancer pulls the service information of the server from the registry or configuration at the client, maintains a list of available services locally, and directly selects a service address locally when sending a request. The client can control the load balancing algorithm by itself.
When making RPC calls between services, we need a client equalizer, so that when multiple providers are implemented, consumers can choose which provider service.
Load balancing algorithm
1. Polling method
The requests are distributed to the back-end servers in turn. It treats each server in the back-end evenly, regardless of the actual number of connections and the current system load.
2. Random method
Through the random algorithm of the system, one of the servers is randomly selected for access according to the list size value of the back-end server. According to the theory of probability and statistics, as the number of calls from the client to the server increases,
The actual effect is closer and closer to the average allocation of calls to each server at the back end, that is, the result of polling.
3. Source address hashifa
The idea of source address hash is to obtain a value calculated by the hash function according to the IP address of the client, use this value to modulo the size of the server list, and the result is the serial number of the customer service side to access the server. The source address hash method is used for load balancing. When the back-end server list remains unchanged, the client with the same IP address will be mapped to the same back-end server for access every time.
4. Weighted polling method
Different back-end servers may have different machine configurations and the load of the current system, so their stress resistance is also different. Configure higher weights for machines with high configuration and low load to handle more requests; The machines with low configuration and high load are assigned a lower weight to reduce their system load. Weighted polling can deal with this problem well and distribute the requests to the back end in order and according to the weight.
5. Weighted random method
Like the weighted polling method, the weighted random method also distributes different weights to the system load according to the configuration of the back-end machine. The difference is that it randomly requests the back-end servers according to the weight, not in order.
6. Minimum connection number method
The minimum number of connections algorithm is flexible and intelligent. Due to the different configurations of the back-end servers, the processing of requests is fast and slow. It dynamically selects the current connection according to the current connection of the back-end servers
The server with the least number of backlog connections will process the current requests, improve the utilization efficiency of back-end services as much as possible, and reasonably divert the responsibility to each server
Ribbion
brief introduction
Ribbon is a client load balancer that provides the following functions
- load balancing
- fault-tolerant
- Multi protocol (HTTP, TCP, UDP) support in asynchronous and reactive models
- Caching and batch processing
Stop maintenance statement
At the end of 2018, Netflix announced that Hystrix entered maintenance mode. Ribbon has been in a similar state since 2016.
In spring cloud 2020 In version x, these modules have been completely removed, and all ribbons have become the past tense, but they have been used on a large scale. It is still possible to study...
Use case
1. Modified version
Modify feign demo POM file to reduce the relevant version.
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.pearl</groupId>
<artifactId>feign-demo</artifactId>
<packaging>pom</packaging>
<version>1.0-SNAPSHOT</version>
<modules>
<module>account-service</module>
<module>order-service</module>
</modules>
<properties>
<java.version>1.8</java.version>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<!--Spring-->
<!-- <spring.boot.version>2.5.2</spring.boot.version>
<spring.cloud.version>2020.0.3</spring.cloud.version>-->
<spring.cloud.alibaba.version>2.2.6.RELEASE</spring.cloud.alibaba.version>
<spring.cloud.version>Hoxton.SR9</spring.cloud.version>
<spring.boot.version>2.2.13.RELEASE</spring.boot.version>
</properties>
<!--Spring edition-->
<dependencyManagement>
<dependencies>
<dependency>
<groupId>com.alibaba.cloud</groupId>
<artifactId>spring-cloud-alibaba-dependencies</artifactId>
<version>${spring.cloud.alibaba.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-dependencies</artifactId>
<version>${spring.boot.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>${spring.cloud.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>com.alibaba.cloud</groupId>
<artifactId>spring-cloud-starter-alibaba-nacos-discovery</artifactId>
<!--Not used Ribbon Client load balancing-->
<!-- <exclusions>
<exclusion>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-ribbon</artifactId>
</exclusion>
</exclusions>-->
</dependency>
<!--SpringCloud Feign stay Hoxton.M2 RELEASED No longer used after version Ribbon But use spring-cloud-loadbalancer-->
<!-- <dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-loadbalancer</artifactId>
</dependency>-->
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-openfeign</artifactId>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<optional>true</optional>
</dependency>
</dependencies>
</project>
2. Test
After adding the above dependencies, because Feign (lower version) integrates the Ribbon by default, there is no need to add the Ribbon.
First, modify the order service to make its interface return to the port of the current service, and then start the order service with two different ports.
@Value("${server.port}")
String port;
@GetMapping("insert")
public Order insertOrder(Long accountId, String commodityCode, Long count, Long money) {
// Simulate order placement and return
Order order = new Order();
order.setAccountId(accountId);
order.setCommodityCode(commodityCode);
order.setCount(count);
order.setMoney(1L);
order.setPort(port);
return order;
}
Access the account service and find that the port is constantly switching, indicating that the Ribbon has taken effect and the polling algorithm is used.
3. Ribbon execution process analysis
After the Feign interface generates the code object, the method is executed. The MethodHandler (method processor) calls the Client to execute. You can see that the Client in the synchronous method executor is LoadBalancerFeignClient (load balancing Client), which acts as the agent for the actual Http Client.
The execute method of LoadBalancerFeignClient,
// Create request URI = > http://order-service/order/insert?accountId=111&commodityCode=iphone11&count=1&money=100
URI asUri = URI.create(request.url());
// Host name, here is the service registration, so it is the service name = "order service"
String clientName = asUri.getHost();
// Remove the URI of the host name
URI uriWithoutHost = cleanUrl(request.url(), clientName);
// Create Ribbon request object
RibbonRequest ribbonRequest = new RibbonRequest(this.delegate, request, uriWithoutHost);
// Get Client configuration information, such as timeout
IClientConfig requestConfig = this.getClientConfig(options, clientName);
// Get the specific load balancer and execute the load balancing request.
return ((RibbonResponse)this.lbClient(clientName).executeWithLoadBalancer(ribbonRequest, requestConfig)).toResponse();
In the executeWithLoadBalancer method, a LoadBalancerCommand object (load balancing execution command class) is created, and then its submit is called for the request to submit, and the response is obtained. After further processing, the Feign decoding is finally returned to the client.
public T executeWithLoadBalancer(final S request, final IClientConfig requestConfig) throws ClientException {
// LoadBalancerCommand = "load balancing command line", which encapsulates the request, list of available servers and other information.
LoadBalancerCommand<T> command = buildLoadBalancerCommand(request, requestConfig);
// Submit request command
try {
return command.submit(
new ServerOperation<T>() {
@Override
public Observable<T> call(Server server) {
// URI address of the final call: http://192.168.0.100:9112/order/insert?accountId=111&commodityCode=iphone11&count=1&money=100
URI finalUri = reconstructURIWithServer(server, request.getUri());
S requestForServer = (S) request.replaceUri(finalUri);
try {
// Call OK HTTP to execute the remote request
return
Observable.just(AbstractLoadBalancerAwareClient.this.execute(requestForServer, requestConfig));
}
catch (Exception e) {
return Observable.error(e);
}
}
})
.toBlocking()
.single();
} catch (Exception e) {
// Omit
}
}
4. Ribbon load balancing algorithm flow
In the selectServer select server method in step 3, an available service will be queried in the load balancer and returned.
private Observable<Server> selectServer() {
return Observable.create(new OnSubscribe<Server>() {
@Override
public void call(Subscriber<? super Server> next) {
try {
// FeignLoadBalancer.getServerFromLoadBalancer
Server server = loadBalancerContext.getServerFromLoadBalancer(loadBalancerURI, loadBalancerKey);
next.onNext(server);
next.onCompleted();
} catch (Exception e) {
next.onError(e);
}
}
});
}
getServerFromLoadBalancer method will eventually call baseloadbalancer Chooseserver () method, in which the load balancing algorithm rule class will be called to calculate which service needs to be called in the end. You can see that the default rule class is ZoneAvoidanceRule, which maintains a combined assertion and a polling algorithm rule.
The choose method of ZoneAvoidanceRule calls the choose method of the parent class PredicateBasedRule. This method will obtain the ILoadBalancer object and maintain the service name, available services, all service lists corresponding to the service name, ribbon client configuration and other related information in this object.
@Override
public Server choose(Object key) {
// 1. Get ILoadBalancer
ILoadBalancer lb = getLoadBalancer();
// 2. Access to services
Optional<Server> server = getPredicate().chooseRoundRobinAfterFiltering(lb.getAllServers(), key);
if (server.isPresent()) {
return server.get();
} else {
return null;
}
}
In the chooseround robinafterfiltering method, the incrementAndGetModulo method will be called. This method is the implementation of the polling algorithm to calculate the number of coordinates that should be used by the current service.
private int incrementAndGetModulo(int modulo) {
// modulo is the number of background copies
for (;;) {
// Get the current coordinate number, which is AtomicInteger, so it is thread safe.
int current = nextIndex.get();
// Calculate next coordinate
int next = (current + 1) % modulo;
// After the operation is completed, assign next to nextServerCyclicCounter with CAS operation
if (nextIndex.compareAndSet(current, next) && current < modulo)
return current;
}
}
Finally, the polling algorithm returns the real IP and port information of the service we currently need to call, and then forwards the request to the HTTP Client.
