Redis Cluster is a Redis Cluster scheme launched by redis version 3.0. It distributes data in different service areas to reduce the system's dependence on a single master node and greatly improve the read-write performance of redis services.
Redis divides all data into 16384 slots, and each node is responsible for a part of the slots. When a redis client connects to the cluster, it will get a copy of the slot configuration information of the cluster, so that it can directly send the request command to the corresponding node for processing.
Redis Cluster is an agent-free and decentralized operation mode. The vast majority of commands sent by the client will be directly handed over to relevant nodes for execution. In most cases, the request command does not need to be forwarded, or the request and response can be completed only once. Therefore, the performance of a single node in the cluster is very close to that of a single redis server, Therefore, in theory, when the master node is expanded twice horizontally, the performance of request processing is doubled, so the performance of Redis Cluster is very high.
Redis Cluster architecture is as follows:
Set up Redis Cluster
There are two ways to build Redis Cluster: one is to use the create cluster tool provided in Redis source code to quickly build Redis Cluster environment, and the other is to manually create Redis Cluster environment by configuration file.
Quickly build Redis Cluster
The create cluster tool is in the utils / create cluster directory, as shown in the following figure:
Use the command/ Create cluster start to quickly create a Redis cluster, as follows:
$ ./create-cluster start # Create cluster Starting 30001 Starting 30002 Starting 30003 Starting 30004 Starting 30005 Starting 30006
Next, we need to form a cluster of the six nodes created above through the create command, as follows:
[@iZ2ze0nc5n41zomzyqtksmZ:create-cluster]$ ./create-cluster create # Cluster formation >>> Performing hash slots allocation on 6 nodes... Master[0] -> Slots 0 - 5460 Master[1] -> Slots 5461 - 10922 Master[2] -> Slots 10923 - 16383 Adding replica 127.0.0.1:30005 to 127.0.0.1:30001 Adding replica 127.0.0.1:30006 to 127.0.0.1:30002 Adding replica 127.0.0.1:30004 to 127.0.0.1:30003 >>> Trying to optimize slaves allocation for anti-affinity [WARNING] Some slaves are in the same host as their master M: 445f2a86fe36d397613839d8cc1ae6702c976593 127.0.0.1:30001 slots:[0-5460] (5461 slots) master M: 63bb14023c0bf58926738cbf857ea304bff8eb50 127.0.0.1:30002 slots:[5461-10922] (5462 slots) master M: 864d4dfe32e3e0b81a64cec8b393bbd26a65cbcc 127.0.0.1:30003 slots:[10923-16383] (5461 slots) master S: 64828ab44566fc5ad656e831fd33de87be1387a0 127.0.0.1:30004 replicates 445f2a86fe36d397613839d8cc1ae6702c976593 S: 0b17b00542706343583aa73149ec5ff63419f140 127.0.0.1:30005 replicates 63bb14023c0bf58926738cbf857ea304bff8eb50 S: e35f06ca9b700073472d72001a39ea4dfcb541cd 127.0.0.1:30006 replicates 864d4dfe32e3e0b81a64cec8b393bbd26a65cbcc Can I set the above configuration? (type 'yes' to accept): yes >>> Nodes configuration updated >>> Assign a different config epoch to each node >>> Sending CLUSTER MEET messages to join the cluster Waiting for the cluster to join . >>> Performing Cluster Check (using node 127.0.0.1:30001) M: 445f2a86fe36d397613839d8cc1ae6702c976593 127.0.0.1:30001 slots:[0-5460] (5461 slots) master 1 additional replica(s) M: 864d4dfe32e3e0b81a64cec8b393bbd26a65cbcc 127.0.0.1:30003 slots:[10923-16383] (5461 slots) master 1 additional replica(s) S: e35f06ca9b700073472d72001a39ea4dfcb541cd 127.0.0.1:30006 slots: (0 slots) slave replicates 864d4dfe32e3e0b81a64cec8b393bbd26a65cbcc S: 0b17b00542706343583aa73149ec5ff63419f140 127.0.0.1:30005 slots: (0 slots) slave replicates 63bb14023c0bf58926738cbf857ea304bff8eb50 M: 63bb14023c0bf58926738cbf857ea304bff8eb50 127.0.0.1:30002 slots:[5461-10922] (5462 slots) master 1 additional replica(s) S: 64828ab44566fc5ad656e831fd33de87be1387a0 127.0.0.1:30004 slots: (0 slots) slave replicates 445f2a86fe36d397613839d8cc1ae6702c976593 [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered.
In the process of execution, you will be asked whether you can complete the execution by taking 30001, 30002 and 30003 as the master nodes and 30004, 30005 and 30006 as their slave nodes.
We can use redis cli to connect to the cluster. The command is as follows:
$ redis-cli -c -p 30001
Use the nodes command to view the node information of the cluster. The command is as follows:
127.0.0.1:30001> cluster nodes 864d4dfe32e3e0b81a64cec8b393bbd26a65cbcc 127.0.0.1:30003@40003 master - 0 1585125835078 3 connected 10923-16383 e35f06ca9b700073472d72001a39ea4dfcb541cd 127.0.0.1:30006@40006 slave 864d4dfe32e3e0b81a64cec8b393bbd26a65cbcc 0 1585125835078 6 connected 0b17b00542706343583aa73149ec5ff63419f140 127.0.0.1:30005@40005 slave 63bb14023c0bf58926738cbf857ea304bff8eb50 0 1585125835078 5 connected 63bb14023c0bf58926738cbf857ea304bff8eb50 127.0.0.1:30002@40002 master - 0 1585125834175 2 connected 5461-10922 445f2a86fe36d397613839d8cc1ae6702c976593 127.0.0.1:30001@40001 myself,master - 0 1585125835000 1 connected 0-5460 64828ab44566fc5ad656e831fd33de87be1387a0 127.0.0.1:30004@40004 slave 445f2a86fe36d397613839d8cc1ae6702c976593 0 1585125835000 4 connected
It can be seen that 30001, 30002 and 30003 are the main nodes. The corresponding slot positions of 30001 are 0 ~ 546030002, 5461 ~ 1092230003 and 10923 ~ 16383, with a total of 16384 slots (0 ~ 16383).
Although the method of create cluster is fast, the number of primary and secondary nodes and slot allocation mode of the cluster built in this method are fixed, and they are installed on the same server, so they can only be used in the test environment.
After our test, we can use the following command to shut down and clean up the cluster:
$ ./create-cluster stop # Shutdown cluster Stopping 30001 Stopping 30002 Stopping 30003 Stopping 30004 Stopping 30005 Stopping 30006 $ ./create-cluster clean # Clean up cluster
Manually set up Redis Cluster
Due to the limitations of create cluster itself, we need to manually add configuration to build Redis cluster in the actual production environment. Therefore, we first need to copy the Redis installation package to node1 to node6 files, because we need to install 6 nodes, 3 master and 3 slave, as shown in the following figure:
Next, we configure and start the Redis cluster.
1. Set profile
We need to modify the redis In the conf file, setting "cluster enabled yes" means that the cluster mode is enabled and the respective ports are modified. We continue to use 30001 to 30006 and set it through "port 3000X".
2. Start each node
redis. After conf is configured, we can start all nodes. The commands are as follows:
cd /usr/local/soft/mycluster/node1 ./src/redis-server redis.conf
3. Create a cluster and allocate slots
We have started 6 nodes before, but these nodes are not interconnected within their respective clusters. Therefore, we need to connect these nodes in series into a cluster and specify the corresponding slot for them. Execute the following command:
redis-cli --cluster create 127.0.0.1:30001 127.0.0.1:30002 127.0.0.1:30003 127.0.0.1:30004 127.0.0.1:30005 127.0.0.1:30006 --cluster-replicas 1
create followed by multiple nodes indicates that these nodes are used as nodes of the whole cluster, while cluster replicas indicates that the number of slave nodes is specified for the master node in the cluster, and 1 indicates that one slave node is set for each master node.
After executing the create command, the system will specify the role of the node and the slot allocation plan for us, as shown below:
>>> Performing hash slots allocation on 6 nodes... Master[0] -> Slots 0 - 5460 Master[1] -> Slots 5461 - 10922 Master[2] -> Slots 10923 - 16383 Adding replica 127.0.0.1:30005 to 127.0.0.1:30001 Adding replica 127.0.0.1:30006 to 127.0.0.1:30002 Adding replica 127.0.0.1:30004 to 127.0.0.1:30003 >>> Trying to optimize slaves allocation for anti-affinity [WARNING] Some slaves are in the same host as their master M: bdd1c913f87eacbdfeabc71befd0d06c913c891c 127.0.0.1:30001 slots:[0-5460] (5461 slots) master M: bdd1c913f87eacbdfeabc71befd0d06c913c891c 127.0.0.1:30002 slots:[5461-10922] (5462 slots) master M: bdd1c913f87eacbdfeabc71befd0d06c913c891c 127.0.0.1:30003 slots:[10923-16383] (5461 slots) master S: bdd1c913f87eacbdfeabc71befd0d06c913c891c 127.0.0.1:30004 replicates bdd1c913f87eacbdfeabc71befd0d06c913c891c S: bdd1c913f87eacbdfeabc71befd0d06c913c891c 127.0.0.1:30005 replicates bdd1c913f87eacbdfeabc71befd0d06c913c891c S: bdd1c913f87eacbdfeabc71befd0d06c913c891c 127.0.0.1:30006 replicates bdd1c913f87eacbdfeabc71befd0d06c913c891c Can I set the above configuration? (type 'yes' to accept):
It can be seen from the above information that Redis intends to set 30001, 30002 and 30003 as master nodes and allocate slots to them. The corresponding slots of 30001 are 0 ~ 546030002, 5461 ~ 1092230003, 10923 ~ 16383, 30005 as slave nodes of 30001, 30006 as slave nodes of 30002 and 30004 as slave nodes of 30003, We only need to enter "yes" to confirm and execute the allocation, as shown below:
Can I set the above configuration? (type 'yes' to accept): yes >>> Nodes configuration updated >>> Assign a different config epoch to each node >>> Sending CLUSTER MEET messages to join the cluster Waiting for the cluster to join .... >>> Performing Cluster Check (using node 127.0.0.1:30001) M: 887397e6fefe8ad19ea7569e99f5eb8a803e3785 127.0.0.1:30001 slots:[0-5460] (5461 slots) master 1 additional replica(s) S: abec9f98f9c01208ba77346959bc35e8e274b6a3 127.0.0.1:30005 slots: (0 slots) slave replicates 887397e6fefe8ad19ea7569e99f5eb8a803e3785 S: 1a324d828430f61be6eaca7eb2a90728dd5049de 127.0.0.1:30004 slots: (0 slots) slave replicates f5958382af41d4e1f5b0217c1413fe19f390b55f S: dc0702625743c48c75ea935c87813c4060547cef 127.0.0.1:30006 slots: (0 slots) slave replicates 3da35c40c43b457a113b539259f17e7ed616d13d M: 3da35c40c43b457a113b539259f17e7ed616d13d 127.0.0.1:30002 slots:[5461-10922] (5462 slots) master 1 additional replica(s) M: f5958382af41d4e1f5b0217c1413fe19f390b55f 127.0.0.1:30003 slots:[10923-16383] (5461 slots) master 1 additional replica(s) [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered.
The display of OK indicates that the whole cluster has been started successfully.
Next, we use redis cli to connect and test the running status of the cluster. The code is as follows:
$ redis-cli -c -p 30001 # Connect to cluster 127.0.0.1:30001> cluster info # View cluster information cluster_state:ok # Status normal cluster_slots_assigned:16384 # Slot number cluster_slots_ok:16384 # Normal slot number cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 # Number of nodes in the cluster cluster_size:3 # Number of cluster main nodes cluster_current_epoch:6 cluster_my_epoch:1 cluster_stats_messages_ping_sent:130 cluster_stats_messages_pong_sent:127 cluster_stats_messages_sent:257 cluster_stats_messages_ping_received:122 cluster_stats_messages_pong_received:130 cluster_stats_messages_meet_received:5 cluster_stats_messages_received:257
The description of relevant fields has been identified in the above code, so it will not be repeated here.
Dynamic adding and deleting nodes
In some cases, we need to dynamically add or delete nodes to the running cluster according to the actual business situation, so we need to do the following operations.
Add master node
Add method 1: cluster meet
You can add a node to the cluster by using the command cluster meet ip:port. Execute the command as follows:
127.0.0.1:30001> cluster meet 127.0.0.1 30007 OK 127.0.0.1:30001> cluster nodes dc0702625743c48c75ea935c87813c4060547cef 127.0.0.1:30006@40006 slave 3da35c40c43b457a113b539259f17e7ed616d13d 0 1585142916000 6 connected df0190853a53d8e078205d0e2fa56046f20362a7 127.0.0.1:30007@40007 master - 0 1585142917740 0 connected f5958382af41d4e1f5b0217c1413fe19f390b55f 127.0.0.1:30003@40003 master - 0 1585142916738 3 connected 10923-16383 3da35c40c43b457a113b539259f17e7ed616d13d 127.0.0.1:30002@40002 master - 0 1585142913000 2 connected 5461-10922 abec9f98f9c01208ba77346959bc35e8e274b6a3 127.0.0.1:30005@40005 slave 887397e6fefe8ad19ea7569e99f5eb8a803e3785 0 1585142917000 5 connected 887397e6fefe8ad19ea7569e99f5eb8a803e3785 127.0.0.1:30001@40001 myself,master - 0 1585142915000 1 connected 0-5460 1a324d828430f61be6eaca7eb2a90728dd5049de 127.0.0.1:30004@40004 slave f5958382af41d4e1f5b0217c1413fe19f390b55f 0 1585142916000 4 connected
It can be seen that the node with port 30007 is added to the cluster and set as the master node.
Add method 2: add node
Use redis cli -- cluster add node to add a node ip:port. A node ip:port in the cluster can also add a node to the cluster. Execute the following commands:
$ redis-cli --cluster add-node 127.0.0.1:30008 127.0.0.1:30001 >>> Adding node 127.0.0.1:30008 to cluster 127.0.0.1:30001 >>> Performing Cluster Check (using node 127.0.0.1:30001) M: 887397e6fefe8ad19ea7569e99f5eb8a803e3785 127.0.0.1:30001 slots:[0-5460] (5461 slots) master 1 additional replica(s) S: dc0702625743c48c75ea935c87813c4060547cef 127.0.0.1:30006 slots: (0 slots) slave replicates 3da35c40c43b457a113b539259f17e7ed616d13d M: df0190853a53d8e078205d0e2fa56046f20362a7 127.0.0.1:30007 slots: (0 slots) master M: f5958382af41d4e1f5b0217c1413fe19f390b55f 127.0.0.1:30003 slots:[10923-16383] (5461 slots) master 1 additional replica(s) M: 1d09d26fd755298709efe60278457eaa09cefc26 127.0.0.1:30008 slots: (0 slots) master M: 3da35c40c43b457a113b539259f17e7ed616d13d 127.0.0.1:30002 slots:[5461-10922] (5462 slots) master 1 additional replica(s) S: abec9f98f9c01208ba77346959bc35e8e274b6a3 127.0.0.1:30005 slots: (0 slots) slave replicates 887397e6fefe8ad19ea7569e99f5eb8a803e3785 S: 1a324d828430f61be6eaca7eb2a90728dd5049de 127.0.0.1:30004 slots: (0 slots) slave replicates f5958382af41d4e1f5b0217c1413fe19f390b55f [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered. [ERR] Node 127.0.0.1:30008 is not empty. Either the node already knows other nodes (check with CLUSTER NODES) or contains some key in database 0.
From the above results, it can be seen that 30008 node is also set as the master node.
Add slave node
Use the command cluster replicate nodeId to set the current node as the slave node of the target node. Execute the command as follows:
127.0.0.1:30008> cluster replicate df0190853a53d8e078205d0e2fa56046f20362a7 OK 127.0.0.1:30008> cluster nodes df0190853a53d8e078205d0e2fa56046f20362a7 127.0.0.1:30007@40007 master - 0 1585147827000 0 connected abec9f98f9c01208ba77346959bc35e8e274b6a3 127.0.0.1:30005@40005 slave 887397e6fefe8ad19ea7569e99f5eb8a803e3785 0 1585147827000 1 connected 1a324d828430f61be6eaca7eb2a90728dd5049de 127.0.0.1:30004@40004 slave f5958382af41d4e1f5b0217c1413fe19f390b55f 0 1585147823000 3 connected 887397e6fefe8ad19ea7569e99f5eb8a803e3785 127.0.0.1:30001@40001 master - 0 1585147826000 1 connected 0-5460 dc0702625743c48c75ea935c87813c4060547cef 127.0.0.1:30006@40006 slave 3da35c40c43b457a113b539259f17e7ed616d13d 0 1585147826930 2 connected f5958382af41d4e1f5b0217c1413fe19f390b55f 127.0.0.1:30003@40003 master - 0 1585147826000 3 connected 10923-16383 1d09d26fd755298709efe60278457eaa09cefc26 127.0.0.1:30008@40008 myself,slave df0190853a53d8e078205d0e2fa56046f20362a7 0 1585147823000 7 connected 3da35c40c43b457a113b539259f17e7ed616d13d 127.0.0.1:30002@40002 master - 0 1585147827933 2 connected 5461-10922
It can be seen that 30008 has become a slave node of 30007.
Delete node
A node can be removed from the cluster by using the ， cluster forget nodeId ， command.
When this command is different from the meet command, you need to delete the Id of the used node to delete the node. You can view the Id information of all nodes through the "cluster nodes" command. The combination of the first 40 letters of each line and the array is the Id of the node, as shown in the following figure:
Execute the following command:
127.0.0.1:30001> cluster forget df0190853a53d8e078205d0e2fa56046f20362a7 OK
At this point, we use the command "cluster nodes" to view the information of all nodes in the cluster:
127.0.0.1:30001> cluster nodes dc0702625743c48c75ea935c87813c4060547cef 127.0.0.1:30006@40006 slave 3da35c40c43b457a113b539259f17e7ed616d13d 0 1585143789940 6 connected f5958382af41d4e1f5b0217c1413fe19f390b55f 127.0.0.1:30003@40003 master - 0 1585143791000 3 connected 10923-16383 3da35c40c43b457a113b539259f17e7ed616d13d 127.0.0.1:30002@40002 master - 0 1585143789000 2 connected 5461-10922 abec9f98f9c01208ba77346959bc35e8e274b6a3 127.0.0.1:30005@40005 slave 887397e6fefe8ad19ea7569e99f5eb8a803e3785 0 1585143789000 5 connected 887397e6fefe8ad19ea7569e99f5eb8a803e3785 127.0.0.1:30001@40001 myself,master - 0 1585143786000 1 connected 0-5460 1a324d828430f61be6eaca7eb2a90728dd5049de 127.0.0.1:30004@40004 slave f5958382af41d4e1f5b0217c1413fe19f390b55f 0 1585143791945 4 connected
It can be seen that the previous node with port 30007 has been successfully removed.
Summary
This paper describes two ways to build Redis clusters: create cluster start and cluster create. Although the former method is fast, it can only create a fixed number of master-slave nodes, and all nodes are on the same server, so it can only be used in the test environment. We also talked about the Redis cluster's functions of dynamically adding master and slave nodes and deleting any node.
I hope this article is helpful to you. Let's practice it~