Docker compose installation You can refer to this
Brief introduction to zookeeper
ZooKeeper is a high-performance coordination service for distributed applications, which is used to maintain configuration information, name, provide distributed synchronization and provide group services. As we all know, coordination services are difficult to do well. They are particularly prone to errors such as race conditions and deadlocks. The motivation behind ZooKeeper is to reduce the responsibility of distributed applications to implement coordination services from scratch.
zookeeper is a typical distributed data consistency solution. Based on it, distributed applications can realize functions such as data publish / subscribe, load balancing, naming service, distributed coordination / notification, cluster management, Master election, distributed lock and distributed queue.
ZooKeeper is commonly used as a service registry. The service provider registers its own information with ZooKeeper. When calling a service, the service consumer first looks up the service in ZooKeeper, obtains the information of the service provider, and then calls the interface of the service provider.
Search zookeeper for available images
docker search zookeeper
Install the latest version of the official image (you can install it here without manually pulling it first. Later, docker compose will not pull it from the remote warehouse, otherwise it will pull the image from the remote warehouse first)
docker pull zookeeper:latest
Basic documents
Deploy three zookeeper nodes on the server
The server creates the following folders to facilitate the attachment of the specified folders (log files, configuration files, etc.) of the following container zookeeper
mkdir -p /home/xt/zookeeper/zookeeper-1/data mkdir -p /home/xt/zookeeper/zookeeper-1/datalog mkdir -p /home/xt/zookeeper/zookeeper-1/log mkdir -p /home/xt/zookeeper/zookeeper-1/conf mkdir -p /home/xt/zookeeper/zookeeper-2/data mkdir -p /home/xt/zookeeper/zookeeper-2/datalog mkdir -p /home/xt/zookeeper/zookeeper-2/log mkdir -p /home/xt/zookeeper/zookeeper-2/conf mkdir -p /home/xt/zookeeper/zookeeper-3/data mkdir -p /home/xt/zookeeper/zookeeper-3/datalog mkdir -p /home/xt/zookeeper/zookeeper-3/log mkdir -p /home/xt/zookeeper/zookeeper-3/conf
Give all users read-write executable permissions to all files under the / home/xt/zookeeper file
-R make the same permission change for all files and subdirectories in the current directory (i.e. change one by one in a recursive manner)
During real development, it is not recommended to give all users the highest permission (divide user groups and give different permissions to different user groups to avoid other users accidentally modifying or deleting files)
chmod -R 777 /home/xt/zookeeper
Create the configuration file zoo. Exe in the conf folder under each zookeeper folder cfg
clientPort=2181 dataDir=/data dataLogDir=/datalog tickTime=2000 initLimit=5 syncLimit=2 autopurge.snapRetainCount=3 autopurge.purgeInterval=0 maxClientCnxns=60 standaloneEnabled=true admin.enableServer=true #It corresponds to the docker container instance created by docker compose below server.1=zookeeper-1:2888:3888 server.2=zookeeper-2:2888:3888 server.3=zookeeper-3:2888:3888
Create docker compose yml
Create docker-compose.exe in the zookeeper folder The YML file is as follows
version: '3.9'
# Configuring zk clusters
# Each sub configuration under container services corresponds to the docker container of a zk node
# Configure zk cluster with a network named zookeeper net
networks:
zookeeper-net:
name: zookeeper-net
driver: bridge
services:
zookeeper-1:
image: zookeeper
container_name: zookeeper-1
restart: always
# Configure the port mapping of docker container and host
ports:
- 2181:2181
- 8081:8080
# Mount the path on the docker container to the host to share data between the host and the docker container
volumes:
- "/home/xt/zookeeper/zookeeper-1/data:/data"
- "/home/xt/zookeeper/zookeeper-1/datalog:/datalog"
- "/home/xt/zookeeper/zookeeper-1/log:/log"
- "/home/xt/zookeeper/zookeeper-1/conf:/conf"
# Configure environment variables for docker container
environment:
# id of the current zk instance
ZOO_MY_ID: 1
# List of machines and ports of the whole zk cluster
ZOO_SERVERS: server.1=zookeeper-1:2888:3888 server.2=zookeeper-2:2888:3888 server.3=zookeeper-3:2888:3888
networks:
- zookeeper-net
zookeeper-2:
image: zookeeper
# Configure the port mapping of docker container and host
container_name: zookeeper-2
restart: always
ports:
- 2182:2181
- 8082:8080
# Mount the path on the docker container to the host to share data between the host and the docker container
volumes:
- "/home/xt/zookeeper/zookeeper-2/data:/data"
- "/home/xt/zookeeper/zookeeper-2/datalog:/datalog"
- "/home/xt/zookeeper/zookeeper-2/log:/log"
- "/home/xt/zookeeper/zookeeper-2/conf:/conf"
# Configure environment variables for docker container
environment:
# id of the current zk instance
ZOO_MY_ID: 2
# List of machines and ports of the whole zk cluster
ZOO_SERVERS: server.1=zookeeper-1:2888:3888 server.2=zookeeper-2:2888:3888 server.3=zookeeper-3:2888:3888
networks:
- zookeeper-net
zookeeper-3:
image: zookeeper
container_name: zookeeper-3
restart: always
# Configure the port mapping of docker container and host
ports:
- 2183:2181
- 8083:8080
# Mount the path on the docker container to the host to share data between the host and the docker container
volumes:
- "/home/xt/zookeeper/zookeeper-3/data:/data"
- "/home/xt/zookeeper/zookeeper-3/datalog:/datalog"
- "/home/xt/zookeeper/zookeeper-3/log:/log"
- "/home/xt/zookeeper/zookeeper-3/conf:/conf"
# Configure environment variables for docker container
environment:
# id of the current zk instance
ZOO_MY_ID: 3
# List of machines and ports of the whole zk cluster
ZOO_SERVERS: server.1=zookeeper-1:2888:3888 server.2=zookeeper-2:2888:3888 server.3=zookeeper-3:2888:3888
networks:
- zookeeper-net
Zookeeper requires a total of three ports:
1. 2181: the port that provides services to the client
2. 3888: election leader usage
3. 2888: communication use of machines in the cluster (Leader listens to this port)
Start container
In the zookeeper folder, that is, docker - compose Where the YML file is located
-d background operation
docker-compose up -d
After the container is started successfully, you can view the container status through the following command
docker ps -a
The container turns to UP and starts successfully
The firewall opens ports to facilitate external access to the server. If the server is deployed with a security group, The security group should also open these ports (if it is useless to run the following commands, it is recommended to manually add ports on the server console. I don't quite understand. It seems that the effects of adding ports in the two ways are different. The ports manually added on the server console can't be viewed with the firewall CMD command. If anyone knows, please comment in the comment area)
firewall-cmd --permanent --add-port=2181/tcp firewall-cmd --permanent --add-port=2182/tcp firewall-cmd --permanent --add-port=2183/tcp firewall-cmd --permanent --add-port=8081/tcp firewall-cmd --permanent --add-port=8082/tcp firewall-cmd --permanent --add-port=8083/tcp
Restart the firewall (restart the firewall after modifying the configuration)
firewall-cmd --reload
See which ports are open
firewall-cmd --list-ports
After the ports are opened, the 8080 ports inside the containers of zookeeper-1, zookeeper-2 and zookeeper-3 are mapped to 8081, 8082 and 8083 ports on the host computer respectively in the configuration file. Therefore, we can access the embedded web consoles of the three zk examples through the browser on our own computer:
http://Server ip:8081/commands
In the configuration directory, we can see the basic configuration information of zk instance, such as server_id, port providing client service, data directory, log directory, etc
In the stats directory, you can see the running status information of zk instance, such as node identity, data size, log size, etc There are three nodes (one leader and two follower s)
docker-compose. Explanation of YML document
After starting the container, we can view some information
1. Network configuration
The top-level networks tab specifies that an isolated network identified as zookeeper net is created and named zookeeper net. Although the identification and naming are set to zookeeper net here, they are actually different configurations:
The network flag is used to configure the flag that the docker container belongs to a network, and the naming is used to enable developers to view and distinguish all network environments on the host, so as to understand the system operation on the host
View all networks on the host
docker network ls
Each sub tag under the services tag represents a docker container instance. Take zookeeper-1 as an example. The networks sub tag under the zookeeper-1 tag indicates that the docker container corresponding to zookeeper-1 is added to the subnet identified as zookeeper net.
So here are three nodes in a network called zookeeper net.
View the container metadata with the following command
docker inspect zookeeper-1
docker inspect zookeeper-2
docker inspect zookeeper-3
Zookeeper-1, zookeeper-2 and zookeeper-3 belong to the network called zookeeper net, and the gateway of the network is 192.168 32.1. The ip addresses of zookeeper-1, zookeeper-2 and zookeeper-3 are respectively
192.168.32.4,192.168.32.3,192.168.32.2
2. Environmental variables
There is an environment tag under the zookeeper-1, zookeeper-2 and zookeeper-3 tags, which configures the environment variables of the container. The environment variables configured under the environment tab will eventually be added to the environment variables of docker container. You can also use the
docker inspect zookeeper-1
In addition to the environment variables explicitly configured in the yml file, there are also some environment variables that come with the container under the Env tag
ZOO_MY_ID indicates the number of the current zookeeper instance in the zookeeper cluster, ranging from 1 to 255, so a zk cluster can have up to 255 nodes
ZOO_SERVERS indicates the number, port, host name (or IP address) configuration of all nodes in the zookeeper cluster where the current zookeeper instance is located
3. Mount files
View the mounting of the container folder, and the other two zookeeper s are the same. (persist logs, configuration files, etc. to the host)
docker inspect zookeeper-1
4. Port mapping
Check the container port mapping information, and the other two zookeeper s are the same
Accessing the 2181 port of the host is equivalent to accessing the 2181 port of the container
docker inspect zookeeper-1
References:
- https://blog.csdn.net/qq_37960603/article/details/121321242
- https://blog.csdn.net/ypp91zr/article/details/89423878
- https://blog.csdn.net/weixin_41622183/article/details/90714190
- https://blog.csdn.net/qq_39340792/article/details/109139406?spm=1001.2101.3001.6661.1&utm_medium=distribute.pc_relevant_t0.none-task-blog-2%7Edefault%7ECTRLIST%7Edefault-1.pc_relevant_paycolumn_v2&depth_1-utm_source=distribute.pc_relevant_t0.none-task-blog-2%7Edefault%7ECTRLIST%7Edefault-1.pc_relevant_paycolumn_v2&utm_relevant_index=1
- https://www.cnblogs.com/kingkoo/p/8732448.html
- https://blog.csdn.net/baidu_38432732/article/details/116763281
- https://blog.csdn.net/killer_999/article/details/93762479
- https://blog.csdn.net/milhua/article/details/78931672
- https://zhuanlan.zhihu.com/p/121728783
(blogging is mainly to summarize and sort out their own learning. Most of the materials come from books, online materials and their own practice. It is not easy to sort out, but it is inevitable to have shortcomings. If there are mistakes, please comment and correct them in the comment area. At the same time, thank the bloggers and authors for their hard sorting out resources.)