Container service update and discovery of docker consumer

What is service registration and discovery

Service registration and discovery is an indispensable component in microservice architecture.
At first, services are single node, which does not guarantee high availability, and does not consider the pressure bearing of services. Calls between services are simply accessed through interfaces. Until the distributed architecture of multiple nodes appeared later, the initial solution was load balancing at the service front end. In this way, the front end must know the network location of all back-end services and configure them in the configuration file. Here are a few questions:

  • If you need to call back-end services A-N, you need to configure the network location of N services, which is very troublesome
  • When the network location of back-end services changes, the configuration of each caller needs to be changed

Since there are these problems, service registration and discovery solve these problems. The back-end service A-M can register its current network location with the service discovery module, and the service discovery is recorded in the form of K-v. K is generally the service name, and V is IP:PORT. The service discovery module conducts regular health checks and polls to see if these back-end services can be accessed. When the front end calls the back-end services A-M, it runs to the service discovery module to ask for their network location, and then calls their services. In this way, the above problems can be solved. The front end does not need to record the network location of these back-end services, and the front end and back-end are completely decoupled!

What is consumer

Consumer is an open source service management software developed by google using go language. Support multi data center, distributed high availability, service discovery and configuration sharing. Raft algorithm is adopted to ensure high availability of services. Built in service registration and discovery framework, distributed consistency protocol implementation, health check, Key/Value storage, multi data center scheme, and no need to rely on other tools (such as ZooKeeper). Service deployment is simple, with only one runnable binary package. Each node needs to run agent, which has two running modes: server and client. The official recommendation of each data center requires 3 or 5 server nodes to ensure data security and ensure that the server leader election can be carried out correctly.

In the client mode, all services registered with the current node will be forwarded to the server node, and the information itself is not persistent.
In the server mode, the function is similar to the client mode. The only difference is that it will persist all information locally, so that in case of failure, the information can be retained.
Server leader is the boss of all server nodes. Unlike other server nodes, it needs to be responsible for synchronizing the registered information to other server nodes, as well as monitoring the health of each node.

Some key features provided by consumer

Service registration and discovery: consumer makes service registration and discovery easy through DNS or HTTP interfaces. Some external services, such as those provided by saas, can also be registered.
Health check: health check enables consumer to quickly alarm the operation in the cluster. Integration with service discovery can prevent services from being forwarded to failed services.
Key/Value storage: a system used to store dynamic configuration. Provides a simple HTTP interface that can be operated anywhere.
Multiple data centers: any number of areas can be supported without complex configuration.

Install consumer is used for service registration, that is, some information of the container itself is registered in consumer. Other programs can obtain the registered service information through consumer, which is service registration and discovery.

Imagine an environment: there are two containers, docker1/docker2, which have nginx services installed, They are connected to the third server (installed with nginx and consumer) through the Registrar (proxy), and send their IP and port registration information to the consumer for registration. At this time, the template in the consumer (profile template) the IP and port discovered by the service can be generated in the template format and in the directory of the nginx configuration file. In this case, nginx can also identify the IP and port of the two docker containers. At this time, the external client accesses the nginx server and the nginx server forwards it, so as to achieve the effect of reverse proxy load balancing.

Consumer deployment

consul The server       one hundred and ninety-two.168.121.17        function Docker,consul Services nginx Services consul-template Daemon

registrator The server    one hundred and ninety-two.168.121.19 function registrator Container, operation nginx container

Turn off firewall

systemctl stop firewalld
systemctl disable firewalld
setenforce 0

Consumer server (

1. Build Consul service

mkdir /opt/consul
cd /opt/consul
mv consul /usr/local/bin/

//Set the agent and start the consumer server in the background
consul agent \
-server \
-bootstrap \
-ui \
-data-dir=/var/lib/consul-data \
-bind= \
-client= \
-node=consul-server01 &> /var/log/consul.log &

-server:with server Start as. The default is client. 
-bootstrap :Used to control a server Is it bootstrap There can only be one mode in a data center server be in bootstrap Mode, when a server be in bootstrap In mode, you can select as server-leader. 
-bootstrap-expect=2 :Minimum cluster requirements server Quantity. When it is lower than this quantity, the cluster will fail.
-ui :Specify on UI Interface, so that you can http://Use an address like localhost:8500/ui to access the web UI provided by consumer.
-data-dir :Specify the data store directory.
-bind :Specify the communication address within the cluster. All nodes in the cluster must be reachable to this address. The default is 
-client : appoint consul Where is the binding client address.On, this address provides HTTP,DNS,RPC And other services. The default is 
-node :The name of a node in a cluster must be unique in a cluster. The default is the host name of the node.
-datacenter :Specify the data center name. The default is dc1. 

netstat -natp | grep consul

start-up consul By default, it will listen to 5 ports:
8300: replication, lcader farwarding Port
8301: lan cossip Port
3302: wan gossip Port
8500:web ui Interface port
8600:use dns Port for viewing node information by protocol

2. View cluster information

#View members information
consul members

#View cluster status
consul operator raft list-peers
consul info | grep leader

3. Obtain cluster information through http api

curl      //View cluster server members
curl     //Cluster Raf leader
curl  //All registered services
curl     //View nginx service information
curl     //Cluster node details

Registrar server (

The container service automatically joins the Nginx cluster

1. Install gliderlabs / Registrar

Gliderlabs / Registrar can check the running status of the container, register automatically, and log off the service of the docker container to the service configuration center.
Consul, Etcd and SkyDNS2 are currently supported

docker run -d \
--name=registrator \
--net=host \
-v /var/run/docker.sock:/tmp/docker.sock \
--restart=always \
gliderlabs/registrator:latest \
-ip= \

2. Test whether the service discovery function is normal

docker run -itd -p:83:80 --name test-01 -h test01 nginx
docker run -itd -p:84:80 --name test-02 -h test02 nginx
docker run -itd -p:99:80 --name test-03 -h test03 httpd
docker run -itd -p:88:80 --name test-04 -h test04 httpd

--net=host :Put the running docker Container set to host Network mode.
-v/var/run/docker.sock:/tmp/docker.sock:Put the host Docker Daemon(Docker daemon)Default listening Onix Mount the domain socket into the container.
--restart=always :Set to always restart the container when it exits.
--ip : Just put network Specified host Mode, so we specify ip For the host ip. 
consul :appoint consul Server IP And ports.

3. Test whether the service discovery function is normal (consumer server)

reach consul On the server


Consul template is an application based on consul to automatically replace configuration files.
consul template is a daemon used to query the consumer cluster information in real time, update any number of specified templates on the file system, and generate configuration files. After the update is completed, you can choose to run the shell command to perform the update operation and reload Nginx.

The consult template can query the service directory, Key, Key values, etc. in the consumer. This powerful abstraction function and query language template can make consul template particularly suitable for dynamic configuration file creation.
For example, create Apache/Nginx Proxy Balancers, haproxy bags, etc.

1. Prepare template nginx template file

//Operate on the consumer server
vim /opt/consul/nginx.ctmpl

upstream http_backend {
  {{range service "nginx"}}
   server {{.Address}}:{{.Port}};

server {
  listen 8000;
  server_name localhost;
  access_log /var/log/nginx/;
  index index.html index.php;
  location / {
    proxy_set_header HOST $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header Client-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_pass http://http_backend;

2. Compile and install nginx

//Operate on the consumer server
yum install gcc pcre-devel zlib-devel -y
useradd -M -s /sbin/nologin nginx
tar zxvf nginx-1.12.0.tar.gz -C /opt/
cd //opt/nginx-1.12.0/
./configure --prefix=/usr/local/nginx --user=nginx --group=nginx && make && make install

ln -s /usr/local/nginx/sbin/nginx /usr/local/sbin/

3. Configure nginx

vim /usr/local/nginx/conf/nginx.conf

http {
   include  mime.types;
   include  vhost/*.conf;   #Add virtual host directory
   default_type application/octet-stream;

//Create virtual host directory
mkdir /usr/local/nginx/conf/vhost
//Create log file directory
mkdir /var/log/nginx

//start nginx 

4. Configure and start the template

cd /root

#Upload consumer template_ 0.19. 3_ linux_ amd64. Zip the package to the / root directory

mv consul-template /usr/bin/

//Start the template service in the foreground. After starting, do not press ctrl+C to stop the consumption template process
consul-template --consul-addr --template "/opt/consul/nginx.ctmpl:/usr/local/nginx/conf/vhost/song.conf:/usr/local/nginx/sbin/nginx -s reload" --log-level=info

Add an nginx container node

#Add an nginx container node to test the service discovery and configuration update functions
//Register at the server of the Registrar (
docker run -itd -p:85:80 --name test-05 -h test05 nginx

//When the consumer server monitors the loading, there will be a prompt for automatic update
2021/03/26 05:51:59.624677 [INFO] (child) spawning: /usr/local/nginx/sbin/nginx -s reload
2021/03/26 05:53:09.576538 [INFO] (runner) initiating run
2021/03/26 05:53:09.577487 [INFO] (runner) rendered "/root/consul/nginx.ctmpl" => "/usr/local/nginx/conf/vhost/test.conf"
2021/03/26 05:53:09.577505 [INFO] (runner) executing command "/usr/local/nginx/sbin/nginx -s reload" from "/root/consul/nginx.ctmpl" => "/usr/local/nginx/conf/vhost/test.conf"
2021/03/26 05:53:09.577534 [INFO] (child) spawning: /usr/local/nginx/sbin/nginx -s reload

//Check the logs of three nginx containers and request normal polling to each container node
docker logs -f test-01
docker logs -f test-02
docker logs -f test-05

Keywords: Linux Docker

Added by robert_gsfame on Mon, 27 Dec 2021 17:49:59 +0200