SpringBoot distributed, Dubbo, zookeeper

In the book "principles and models of distributed systems", there is the following definition: "a distributed system is a collection of several independent computers, which are like a single related system to users";

Distributed system is a system composed of a group of computer nodes that communicate through the network and coordinate to complete common tasks. The emergence of distributed system is to use cheap and ordinary machines to complete the calculation and storage tasks that can not be completed by a single computer. Its purpose is to use more machines to process more data.

distributed system is a software system based on network.

First of all, it should be clear that we need to consider distributed systems only when the processing capacity of a single node can not meet the increasing computing and storage tasks, and the improvement of hardware (adding memory, adding disk and using better CPU) is too high to pay off, and the application can not be further optimized. Because the problems to be solved by the distributed system itself are the same as those of the single machine system. Due to the multi node and network communication topology of the distributed system, many problems that the single machine system does not have will be introduced. In order to solve these problems, more mechanisms and agreements will be introduced to bring more problems...

Dubbo document

With the development of the Internet and the continuous expansion of the scale of website applications, the conventional vertical application architecture can not cope with it. The distributed service architecture and mobile computing architecture are imperative, and a governance system is urgently needed to ensure the orderly evolution of the architecture.

There is such a picture on Dubbo's official website

Single application architecture

When the website traffic is very small, only one application is needed to deploy all functions together to reduce deployment nodes and costs. At this time, the data access framework (ORM) used to simplify the workload of addition, deletion, modification and query is the key.

It is applicable to small websites and small management systems. All functions are deployed into one function, which is simple and easy to use.

Disadvantages:

1. Performance expansion is difficult

2. Collaborative development problem

3. Not conducive to upgrade and maintenance

Vertical application architecture

When the number of visits increases gradually, the acceleration brought by a single application increases and the machine decreases. Split the application into several unrelated applications to improve efficiency. At this time, the Web framework (MVC) for accelerating front-end page development is the key.

The independent deployment of each module is realized through business segmentation, which reduces the difficulty of maintenance and deployment. The team performs its own duties, which is easier to manage, and the performance expansion is more convenient and targeted.

Disadvantages: common modules cannot be reused, which is a waste of development

Distributed service architecture

When there are more and more vertical applications, the interaction between applications is inevitable. Extract the core business as an independent service, and gradually form a stable service center, so that the front-end application can respond to the changing market demand more quickly. At this time, the distributed service framework (RPC) used to improve business reuse and integration is the key.

Flow computing architecture

When there are more and more services, problems such as capacity evaluation and waste of small service resources gradually appear. At this time, it is necessary to add a dispatching center to manage the cluster capacity in real time based on the access pressure and improve the cluster utilization. At this time, the resource scheduling and Governance Center (SOA) [Service Oriented Architecture] used to improve machine utilization is the key.

RPC

What is PRC

RPC [Remote Procedure Call] refers to Remote Procedure Call. It is a way of inter process communication. It is a technical idea, not a specification. It allows a program to call a procedure or function in another address space (usually on another machine sharing a network) without the programmer explicitly encoding the details of the remote call. That is, whether programmers call local or remote functions, the calling code written by programmers is basically the same.

In other words, two servers a and B, one application is deployed on server A. if you want to call the functions / methods provided by the application on server B, they cannot be called directly because they are not in the same memory space. You need to express the semantics of the call and convey the data of the call through the network. Why use RPC? It is a requirement that cannot be completed in a process or even in a computer through local call, such as communication between different systems, or even communication between different organizations. Due to the horizontal expansion of computing power, applications need to be deployed on a cluster composed of multiple machines. RPC is to call remote functions like calling local functions;

RPC Fundamentals

Step analysis:

RPC has two core modules: communication and serialization.
Serialization: data transfer requires conversion

Test environment construction

Dubbo

Apache Dubbo | ˈ d ʌ b əʊ| It is a high-performance and lightweight open source Java RPC framework. It provides three core capabilities: interface oriented remote method invocation, intelligent fault tolerance and load balancing, and automatic service registration and discovery.

dubbo official website Apache Dubbo

1. Understand the characteristics of Dubbo

2. View official documents

Basic concepts of dubbo

Service Provider: the service Provider that exposes the service. When the service Provider starts, it registers the service it provides with the registry.

Service Consumer: the service Consumer who calls the remote service. When starting, the service Consumer subscribes to the service required by the registry. The service Consumer selects one provider from the list of provider addresses based on the soft load balancing algorithm to call. If the call fails, select another one to call.

Registry: the registry returns the address list of service providers to consumers. If there is any change, the registry will push the change data to consumers based on the long connection

Monitor ing center: service consumers and providers accumulate call times and call times in memory, and regularly send statistical data to the monitoring center every minute

Call relation description

l the service container is responsible for starting, loading and running the service provider.

l when the service provider starts, it registers its services with the registry.

l service consumers subscribe to the services they need from the registry when they start.

l the registration center returns the service provider address list to the consumer. If there is any change, the registration center will push the change data to the consumer based on the long connection.

l the service consumer selects one provider from the provider address list based on the soft load balancing algorithm to call. If the call fails, select another one to call.

l service consumers and providers accumulate call times and call times in memory, and regularly send statistical data to the monitoring center every minute.

Dubbo environment construction

Click the official dubbo document and recommend that we use Zookeeper registration center Zookeeper registry | Apache Dubbo

What is zookeeper? You can view official documents Apache ZooKeeper

Installing zookeeper under Window

1. Download zookeeper: Apache ZooKeeper the latest version! Unzip zookeeper

2. Run / bin / zkserver CMD, an error will be reported during the initial operation, and there is no zoo CFG configuration file;

Possible problems: flash back!

Solution: edit zkserver Add pause at the end of CMD file. In this way, if there is an error in operation, it will not exit, and an error message will be prompted to find the reason.

3. Modify CFG configuration file

Put the zoo under the conf folder_ sample. CFG copy and rename it zoo CFG is enough.

Pay attention to several important points:

dataDir=./ Directory of temporary data store (writable relative path)

clientPort=2181; the port number of zookeeper

After the modification is completed, start zookeeper again

4. Use zkcli CMD test

ls /: lists all nodes saved under the root of zookeeper

Create – e / gh 123: create a gh node with a value of 123

get /gh: get the value of the / gh node

Install Dubbo admin under Windows

dubbo itself is not a service software. It is actually a jar package, which can help your java program connect to zookeeper and use zookeeper to consume and provide services.

However, in order to allow users to better manage and monitor many dubbo services, the official provides a visual monitoring program dubbo admin, but this monitoring will not affect the use even if it is not installed.

Let's install it here:

1. Download Dubbo admin

Address: GitHub - apache/dubbo-admin at master-0.2.0

2. Unzip into directory

Modify Dubbo admin \ SRC \ main \ resources \ application Properties specifies the zookeeper address

server.port=7001
spring.velocity.cache=false
spring.velocity.charset=UTF-8
spring.velocity.layout-url=/templates/default.vm
spring.messages.fallback-to-system-locale=false
spring.messages.basename=i18n/message
spring.root.password=root
spring.guest.password=guest

dubbo.registry.address=zookeeper://127.0.0.1:2181

3. Package Dubbo admin in the project directory

mvn clean package -Dmaven.test.skip=true

The first packaging process is a little slow and requires patience! Until success!

4. Execute dubbo-admin-0.0.1-snapshot. Under Dubbo admin \ target jar

java -jar dubbo-admin-0.0.1-SNAPSHOT.jar

[Note: zookeeper's service must be turned on!]

After execution, let's visit http://localhost:7001/ At this time, we need to enter the login account and password. We are the default root root;

After successful login, view the interface

Installation complete!

zookeeper: Registry

Dubbo admin: it is a monitoring and management background ~ check which services we have registered and which services are consumed ~ [optional]

Dubbo:jar package~

SpringBoot + Dubbo + zookeeper

Steps:

Premise: zookeeper service has been started!

1. Provision of services by providers

1. Import dependency

2. Configure the address of the registry, the service discovery name, and the package to be scanned~

3. Add a comment on the Service you want to register under @ Service dubbo

2. How consumers consume

1. Import dependency

2. Configure the address of the registry and configure your own service name~

3. Inject Service ~ @ Reference from remote

Frame construction

1. Start zookeeper!

2. IDEA creates an empty project;

3. Create a module to realize the service provider: provider server, and select web dependency

4. After the project is created, we write a service, such as ticket selling service;

Write interface

 //Ticket Service
public interface TicketService {
    String getTicket();
}

Writing implementation classes

 //zookeeper service registration and discovery

@Service    //The service annotation under the dubbo package can be scanned and automatically registered with the registration center when the project moves together
@Component  //After using Dubbo, try not to use Service annotation, for fear of pouring the wrong package
public class TicketServiceImpl implements TicketService {
    @Override
    public String getTicket() {
        return "Hum~Sample ,Do you want to buy a ticket  No way";
    }
}

5. Create a module to realize the service consumer: consumer server, and select web dependency

6. After the project is created, we write a service, such as the user's service;

Write service

package com.gh.service;

public class UserService {
   //We need to get the service from the registration center
}

Demand: now our users want to use the ticket buying service. How can we do this?

Service provider

1. To register the service provider in the registry, we need to integrate Dubbo and zookeeper, so we need to import the package

We enter github from dubbo's official website, look at the help document below, find dubbo springboot, and find the dependent package

         <!-- Introduce dependency Dubbo + zookeeper -->
        <!-- https://mvnrepository.com/artifact/org.apache.dubbo/dubbo-spring-boot-starter -->
        <dependency>
            <groupId>org.apache.dubbo</groupId>
            <artifactId>dubbo-spring-boot-starter</artifactId>
            <version>2.7.3</version>
        </dependency>

We'll go to maven warehouse to download the package of zookeeper, zkclient;

         <!-- introduce zookeeper  zkclient -->
        <!-- https://mvnrepository.com/artifact/com.github.sgroschupf/zkclient -->
        <dependency>
            <groupId>com.github.sgroschupf</groupId>
            <artifactId>zkclient</artifactId>
            <version>0.1</version>
        </dependency>

[new version of pit] zookeeper and its dependency package. To solve log conflicts, you also need to eliminate log dependencies;

        <!-- introduce zookeeper -->
        <!-- https://mvnrepository.com/artifact/org.apache.curator/curator-framework -->
        <dependency>
            <groupId>org.apache.curator</groupId>
            <artifactId>curator-framework</artifactId>
            <version>4.2.0</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/org.apache.curator/curator-recipes -->
        <dependency>
            <groupId>org.apache.curator</groupId>
            <artifactId>curator-recipes</artifactId>
            <version>4.2.0</version>
        </dependency>
        <dependency>
            <groupId>org.apache.zookeeper</groupId>
            <artifactId>zookeeper</artifactId>
            <version>3.6.2</version>
            <!--Exclude this slf4j-log4j12-->
            <exclusions>
                <exclusion>
                    <groupId>org.slf4j</groupId>
                    <artifactId>slf4j-log4j12</artifactId>
                </exclusion>
            </exclusions>
        </dependency>

2. Configure dubbo related properties in the springboot configuration file!

 #Current app name
dubbo.application.name=provider-server
#Address of Registration Center
dubbo.registry.address=zookeeper://127.0.0.1:2181
#Scan the services under the specified package
dubbo.scan.base-packages=com.gh.service

3. Configure the service annotation in the service implementation class and publish the service! Pay attention to the problem of guide package

 //zookeeper service registration and discovery

@Service    //The service annotation under the dubbo package can be scanned and automatically registered with the registration center when the project moves together
@Component  //After using Dubbo, try not to use Service annotation, for fear of pouring the wrong package
public class TicketServiceImpl implements TicketService {
    @Override
    public String getTicket() {
        return "Hum~Sample ,Do you want to buy a ticket  No way";
    }
}

Logical understanding: when the application starts, dubbo will scan the service with @ component annotation under the specified package and publish it in the specified registry!

Serving consumers

1. Import dependency, the same as the previous dependency;

         <!-- Introduce dependency Dubbo + zookeeper -->
        <!-- https://mvnrepository.com/artifact/org.apache.dubbo/dubbo-spring-boot-starter -->
        <dependency>
            <groupId>org.apache.dubbo</groupId>
            <artifactId>dubbo-spring-boot-starter</artifactId>
            <version>2.7.3</version>
        </dependency>

        <!-- introduce zookeeper  zkclient -->
        <!-- https://mvnrepository.com/artifact/com.github.sgroschupf/zkclient -->
        <dependency>
            <groupId>com.github.sgroschupf</groupId>
            <artifactId>zkclient</artifactId>
            <version>0.1</version>
        </dependency>


        <!-- introduce zookeeper -->
        <!-- https://mvnrepository.com/artifact/org.apache.curator/curator-framework -->
        <dependency>
            <groupId>org.apache.curator</groupId>
            <artifactId>curator-framework</artifactId>
            <version>4.2.0</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/org.apache.curator/curator-recipes -->
        <dependency>
            <groupId>org.apache.curator</groupId>
            <artifactId>curator-recipes</artifactId>
            <version>4.2.0</version>
        </dependency>
        <dependency>
            <groupId>org.apache.zookeeper</groupId>
            <artifactId>zookeeper</artifactId>
            <version>3.6.2</version>
            <!--Exclude this slf4j-log4j12-->
            <exclusions>
                <exclusion>
                    <groupId>org.slf4j</groupId>
                    <artifactId>slf4j-log4j12</artifactId>
                </exclusion>
            </exclusions>
        </dependency>

2. Configuration parameters

 #Current app name
dubbo.application.name=consumer-server
#Address of Registration Center
dubbo.registry.address=zookeeper://127.0.0.1:2181

3. The original normal step is to package the service provider's interface and then import it with pom file. Here, we use a simple way to directly take the service interface. The path must be correct, that is, the same as that of the service provider;

4. Improve consumer services

 @Service   //Put it in a container
public class UserService {

    //To get the classes provided by the provider server, go to the registration center to get the services
    @Reference   //Method 1 of referencing pom coordinates: you can locate the interface name with the same path
    private TicketService ticketService;

    public void getTicket(){
        String ticket = ticketService.getTicket();
        System.out.println("Get it at the registry=>" + ticket);
    }


}

5. Test preparation

 @SpringBootTest(classes = Application.class)
class ApplicationTests {

    @Autowired
    UserService userService;

    @Test
    void contextLoads() {

        userService.getTicket();
    }

}

Start test

1. Start zookeeper

2. Open Dubbo admin to realize monitoring [no need to do]

3. Open the service provider

4. Consumer consumption test, results:

Monitoring center:

ok, this is the application of SpingBoot + dubbo + zookeeper to realize distributed development. In fact, it is an idea of service splitting;

Microservice Architecture -- > New Architecture

Modularization, functionalization!

User, payment, check-in, entertainment;

More and more people: one server can't solve it; Adding servers! transverse

Suppose that server A occupies 98% of resources and server B only occupies 10%-- Load balancing;

Divide the original overall project into modules. The user is a separate project, and the check-in is also a separate project. Before the project and the project need to communicate, how to communicate?

There are many users and very few sign in! Give users more servers and fewer servers for check-in!

Microservice architecture problem?

Four core problems encountered by distributed architecture?

1. How can clients access so many services?

2. With so many services, how can services communicate with each other?

3. How to manage so many services?

4. What if the service hangs up?

Solution: spring cloud is a set of ecology, which is to solve the above four problems of distributed architecture

If you want to use spring cloud, you must master spring boot, because spring cloud is based on spring boot;

1.Spring Cloud NetF1ix has come up with a set of solutions! One stop solution. We can all get it right here?

Api gateway, zuu1 component

Feign -- > httpc1ient -- > HTTP communication mode, synchronization and blocking

Service registration and discovery, Eureka

Fuse mechanism, Hystrix

At the end of 2018. NetF1ix announced to stop maintenance indefinitely. I f the ecology is no longer maintained, it will be disconnected

2.Apache Dubbo zookeeper, the second set of solution system

API: no! Either find a third-party component or implement it yourself

Dubbo is a high-performance RPC communication framework based on Java implementation! three

Service registration and discovery, zookeeper: Hadoop, hive, elephant, bee

No circuit breaker mechanism: with the help of Hystrix

3.SpringCloud Alibaba one-stop solution!

At present, another scheme is proposed:

Service Grid: next generation microservice standard, Server Mesh

Representative solution: istio (you may need to master it in the future!)

All changes are inseparable from their ancestors. They are all accessible!

1.API gateway, service routing

2.HTTP, RPC framework, asynchronous call

3. Service registration and discovery, high availability

4. Circuit breaker mechanism, service degradation

If you develop a solution based on these four problems, it is also called spring cloud!

Why solve this problem? Essence: the network is unreliable!

Keywords: Java Spring Boot Distribution

Added by nielskg on Tue, 15 Feb 2022 04:31:06 +0200

Programming VIP