redis master-slave replication, sentinel mode, cluster mode

Although redis can realize the data persistence of a single machine, neither RDB nor AOF can solve the problem of single point of downtime, that is, once a single redis server has system failure, hardware failure and other problems, it will directly cause data loss. In addition, the performance of a single machine is limited, Therefore, it is necessary to use other technologies to solve the problems of single point of failure and performance expansion.

1. Master slave replication:Master slave replication is highly available Redis Sentinels and clusters are highly available based on master-slave replication. Master-slave replication mainly realizes multi machine backup of data, load balancing for read operation and simple fault recovery. defect:Failure recovery cannot be automated;Write operations cannot be load balanced;The storage capacity is limited by a single machine.
2. sentry:Based on master-slave replication, sentinel realizes automatic fault recovery. defect:Write operations cannot be load balanced:Storage capacity is limited by a single machine;The sentinel cannot automatically fail over the slave node. In the read-write separation scenario, the failure of the slave node will lead to the unavailability of the read service, and additional monitoring and switching operations need to be performed on the slave node.
3. colony:Through the cluster, Redis It solves the problem that the write operation cannot be load balanced and the storage capacity is limited by a single machine, and realizes a relatively perfect high availability scheme.

redis master-slave replication

Master slave replication refers to copying data from one redis server to other redis servers. The former is called the master node and the latter is called the slave node; data replication is one-way and can only be from the master node to the slave node.

By default, each redis server is the primary node; A master node can have multiple slave nodes (or no slave nodes), but each slave node can only have one master node.

Role of master-slave replication:

Data redundancy: master-slave replication realizes the hot backup of data, which is a data redundancy method other than persistence.
Fault recovery: when the master node has problems, the slave node can provide services to achieve rapid fault recovery; It is actually a kind of service redundancy.
Load balancing: Based on Master - slave replication In, with the separation of read and write, the master node can provide write services, and the slave node can provide read services (that is, the application connects to the master node when writing Redis data, and the application connects to the slave node when reading Redis data), sharing the server load; Especially in the scenario of less writing and more reading, the concurrency of Redis server can be greatly improved by sharing the read load among multiple slave nodes.
High availability cornerstone: in addition to the above functions, master-slave replication is also the basis for sentinels and clusters to implement. Therefore, master-slave replication is the basis for Redis high availability.

Master slave replication process

If a Slave machine is started, it will send a "sync command" command to the Master machine to request synchronous connection. From to Master
Whether it is the first connection or reconnection, the Master machine will start a background process to save the data snapshot to the data file (perform rdb operation). At the same time, the Master will record all commands to modify the data and cache them in the data file.
After the background process completes the cache operation, the Master machine will send the data file storage to the slave machine, and then the Master machine will send all operations of modifying the data to the slave machine. If the slave goes down due to failure, it will automatically reconnect after it returns to normal.
After receiving the connection from slave machines, the Master machine sends its complete data files to slave machines. If the Master receives synchronization requests from multiple slave machines at the same time, the Master will start a process in the background to save the data files, and then send them to all slave machines to ensure that all slave machines are normal.

Set up master-slave replication:

Environment configuration:

host	operating system	IP address	Software / installation package / tools
Master	CentOS7	192.168.159.104	redis-5.0.7.tar.gz
Slave1	CentOS7	192.168.159.105	redis-5.0.7.tar.gz
Slave2	CentOS7	192.168.159.106	redis-5.0.7.tar.gz

Install Redis

systemctl stop firewalld
setenforce 0

yum install -y gcc gcc-c++ make

tar zxvf redis-5.0.7.tar.gz -C /opt/

cd /opt/redis-5.0.7/
make
make PREFIX=/usr/local/redis install

cd /opt/redis-5.0.7/utils
./install_server.sh

Press enter four times. The next step requires manual input

Please select the redis executable path [] /usr/local/redis/bin/redis-server  	

ln -s /usr/local/redis/bin/* /usr/local/bin/

Master node Redis profile

192.168.159.104

vim /etc/redis/6379.conf
bind 0.0.0.0						#Line 70, modify the bind item, 0.0 0.0 monitor all network segments
daemonize yes						#Line 137, start the daemon
logfile /var/log/redis_6379.log		#Line 172, specify the log file directory
dir /var/lib/redis/6379				#Line 264, specify the working directory
appendonly yes						#Line 700, enable AOF persistence function

/etc/init.d/redis_6379 restart

Slave node Redis profile

(192.168.159.105),(192.168.159.106)

vim /etc/redis/6379.conf
bind 0.0.0.0						#Line 70, modify the bind item, 0.0 0.0 monitor all network cards
daemonize yes						#Line 137, start the daemon
logfile /var/log/redis_6379.log		#Line 172, specify the log file directory
dir /var/lib/redis/6379				#Line 264, specify the working directory
replicaof 192.168.184.10 6379		#Line 288, specifying the Master node IP and port to synchronize
appendonly yes						#Line 700, enable AOF persistence function

/etc/init.d/redis_6379 restart

Verify master-slave effect

View logs on the Master node

tail -f /var/log/redis_6379.log

Verify the slave node on the Master node

redis-cli info replication

Redis sentinel mode

Core function of sentinel: Based on master-slave replication, sentinel introduces automatic failover of master node

1. Principle of sentinel mode

Sentinel: it is a distributed system used to monitor each server in the Master-Slave structure. In case of failure, it selects a new Master through the voting mechanism and connects all Slave servers to the new Master. Therefore, the number of clusters running sentinels shall not be less than 3 nodes.

2. Role of sentinel mode

● monitoring: the Sentry will constantly check whether the master node and slave node are operating normally.

● automatic failover: when the master node cannot work normally, the Sentry will start the automatic failover operation, which will upgrade one of the slave nodes of the failed master node to a new master node, and change other slave nodes to copy the new master node.

● notification (reminder): the sentinel can send the result of failover to the client.

3. Structure of sentinel mode

The sentinel structure consists of two parts, sentinel node and data node:
● sentinel node: the sentinel system is composed of one or more sentinel nodes. The sentinel node is a special redis node and does not store data.
● data node: both master node and slave node are data nodes.

The start of sentry depends on the Master-slave mode. Therefore, the sentry mode must be implemented after the Master-slave mode is installed. The sentry mode needs to be deployed on all nodes. The sentry mode will monitor whether all Redis working nodes are normal. When the Master has a problem, other nodes will vote because they lose contact with the Master node, After half of the vote, it is considered that there is a problem with this Master, and then it will notify the sentinel room, and then select one from the slave as a new Master.

It should be noted that the concept of objective offline is unique to the master node; If the slave node and sentinel node fail, there will be no subsequent objective offline and failover operations after the sentinel subjectively goes offline.

1. Environment configuration

Master slave replication has been set up

host	operating system	IP address	Software / installation package / tools
Master	CentOS7	192.168.159.104	redis-5.0.7.tar.gz
Slave1	CentOS7	192.168.159.105	redis-5.0.7.tar.gz
Slave2	CentOS7	192.168.159.106	redis-5.0.7.tar.gz

2. Modify Redis configuration file (all node operations)

systemctl stop firewalld
setenforce 0

vim /opt/redis-5.0.7/sentinel.conf
protected-mode no								#Line 17, turn off protection mode
port 26379										#Line 21, the default listening port of Redis sentry
daemonize yes									#Line 26, specify sentinel as the background startup
logfile "/var/log/sentinel.log"					#Line 36, specify the log storage path
dir "/var/lib/redis/6379"						#Line 65, specify the database storage path
sentinel monitor mymaster 192.168.184.10 6379 2	#Line 84, modify and specify the sentinel node to monitor 192.168 184.10:6379 the name of the master node is mymaster. The meaning of the last 2 is related to the fault determination of the master node: at least two sentinel nodes need to agree to determine the fault of the master node and failover
sentinel down-after-milliseconds mymaster 30000	#In line 113, determine the time period when the server goes down. The default is 30000 milliseconds (30 seconds)
sentinel failover-timeout mymaster 180000		#146 lines, the maximum timeout time of the failed node is 180000 (180 seconds)

Start sentinel mode

Master: 192.168.159.104
Slave1: 192.168.159.105
Slave2: 192.168.159.106
Note: start the master first and then the slave

cd /opt/redis-5.0.7/
redis-sentinel sentinel.conf &

View sentry mode information

Master: 192.168.159.104

redis-cli -p 26379 info Sentinel

Fault simulation

Master: 192.168.159.104

#View redis server process number
ps aux | grep redis
root      46451  0.1  0.4 156404  7776 ?        Ssl  09:17   0:04 /usr/local/redis/bin/redis-server 0.0.0.0:6379
root      46884  0.2  0.4 153844  7744 ?        Ssl  09:54   0:01 redis-sentinel *:26379 [sentinel]
root      46999  0.0  0.0 112676   984 pts/2    R+   10:04   0:00 grep --color=auto redis


#Kill the process number of redis server on the Master node and simulate the fault
kill -9 46451			#The process number of the redis server on the Master node

Verification results

Master: 192.168.159.104

tail -f /var/log/sentinel.log

Redis cluster mode

Cluster, namely Redis Cluster, is a distributed storage scheme introduced by Redis 3.0.

The cluster consists of multiple nodes in which Redis data is distributed. Nodes in the cluster are divided into master nodes and slave nodes: only the master node is responsible for reading and writing requests and maintaining cluster information; The slave node only copies the master node data and status information.

1. Role of clusters

(1) Data partition: data partition (or data partition) is the core function of the cluster.
The cluster distributes data to multiple nodes. On the one hand, it breaks through the limitation of Redis single machine memory size and greatly increases the storage capacity; On the other hand, each master node can provide external read and write services, which greatly improves the response ability of the cluster.
The problem of limited memory size of Redis stand-alone is mentioned when introducing persistence and master-slave replication; For example, if the stand-alone memory is too large, the fork operations of bgsave and bgrewriteaof may cause the master process to block. In the master-slave environment, the slave node may be unable to provide services for a long time during host switching, and the replication buffer of the master node may overflow in the full replication phase.

(2) High availability: the cluster supports master-slave replication and automatic failover of the master node (similar to sentinel); when any node fails, the cluster can still provide external services.

2. Data fragmentation of Redis cluster

Redis cluster introduces the concept of hash slot
Redis cluster has 16384 hash slots (No. 0-16383)
Each node of the cluster is responsible for a portion of the hash slot
After each Key is verified by CRC16, it takes the remainder of 16384 to determine which hash slot to place. Through this value, it finds the node corresponding to the corresponding slot, and then directly and automatically jumps to the corresponding node for access operation

#Take a cluster composed of three nodes as an example:
Node A contains hash slots 0 to 5460
Node B contains hash slots 5461 to 10922
Node C contains hash slots 10923 to 16383

#Master slave replication model of Redis cluster
There are three nodes A, B and C in the cluster. If node B fails, the whole cluster will be unavailable due to the lack of slots in the range of 5461-10922.
Add a slave node A1, B1 and C1 to each node. The whole cluster consists of three Master nodes and three slave nodes. After node B fails, the cluster elects the Master node with B1 to continue serving. When both B and B1 fail, the cluster will not be available

Build Redis cluster mode

A redis cluster generally requires 6 nodes, 3 master nodes and 3 slave nodes.
First install the redis database on six servers

host	operating system	IP: Port	Installation package
Master1	CentOS7	192.168.159.101:6371	redis-5.0.7.tar.gz
Master2	CentOS7	192.168.159.102:6372	redis-5.0.7.tar.gz
Master3	CentOS7	192.168.159.103:6373	redis-5.0.7.tar.gz
Slave1	CentOS7	192.168.159.104:6374	redis-5.0.7.tar.gz
Slave2	CentOS7	192.168.159.105:6375	redis-5.0.7.tar.gz
Slave3	CentOS7	192.168.159.106:6376	redis-5.0.7.tar.gz

Create and copy related files

All nodes

#Create a file. The file name should be created according to the port to facilitate distinction
cd /etc/redis/
mkdir -p redis-cluster/redis6371 
cp /opt/redis-5.0.7/redis.conf /etc/redis/redis-cluster/redis6371/
cp /opt/redis-5.0.7/src/redis-cli /opt/redis-5.0.7/src/redis-server /etc/redis/redis-cluster/redis6371/

Modify the configuration file to enable the cluster function

All nodes

cd /etc/redis/redis-cluster/redis6371
vim redis.conf
#Line 69, modify the bind item and listen to your own IP address
bind 192.168.163.11
#Line 88, modify, turn off protection mode
protected-mode no
#Line 92, modify, redis listening port
port 6371
#136 line to start as a separate process
daemonize yes
#Line 699, modify and enable AOF persistence
appendonly yes
#Line 832, uncomment and turn on the cluster function
cluster-enabled yes
#Line 840, uncomment, modify, cluster name, file settings
cluster-config-file nodes-6371.conf
#Line 846, uncomment, cluster timeout setting
cluster-node-timeout 15000

Copy files are configured on the node

scp /etc/redis/redis-cluster/redis6371/redis.conf root@192.168.159.102:/etc/redis/redis-cluster/redis6372/redis.conf

On other servers

cd /etc/redis/redis-cluster/redis6372
vim redis.conf
#Line 69, modify the bind item and listen to your own IP address
bind 192.168.159.102
#Line 92, modify, redis listening port
port 6372
#Line 840, uncomment, modify, cluster name, file settings
cluster-config-file nodes-6372.conf

Start redis node

All nodes

#Each server enters the corresponding file and executes the command
cd /etc/redis/redis-cluster/redis6371/
redis-server redis.conf

ps -ef |  grep redis

Start cluster

Master1 192.168.159.101:6371

redis-cli --cluster create 192.168.159.101:6371 192.168.159.102:6372 192.168.159.103:6373 192.168.159.104:6374 192.168.159.105:6375 192.168.159.106:6376 --cluster-replicas 1

Test cluster

edis-cli -h 192.168.159.101 -p 6371 -c   #With the - c parameter, nodes can jump to each other	
cluster slots			#View the hash slot number range of the node
set test lisi
cluster keyslot test	#View the slot number of the name key

Keywords: Database Redis cloud computing

Added by dml on Tue, 14 Dec 2021 09:29:53 +0200

Programming VIP

redis master-slave replication, sentinel mode, cluster mode

redis cluster