redis master-slave replication

concept

Master slave replication refers to copying data from one Redis server to other Redis servers. The former is called Master/Leader and the latter is called Slave/Follower. Data replication is one-way! It can only be copied from the master node to the slave node (the master node is dominated by writing and the slave node is dominated by reading).

By default, each Redis server is a master node. A master node can have 0 or more slave nodes, but each slave node can only have one master node.

effect

1. Data redundancy: master-slave replication realizes the hot backup of data, which is a way of data redundancy other than persistence.
2. Fault recovery: when the master node fails, the slave node can temporarily replace the master node to provide services, which is a way of service redundancy
3. Load balancing: on the basis of master-slave replication, with read-write separation, the master node performs write operations and the slave node performs read operations to share the load of the server; Especially in the scenario of more reads and less writes, multiple slave nodes share the load and improve the concurrency.
4. High availability cornerstone: master-slave replication is also the basis for sentinel and cluster implementation.

Why cluster

1. It is difficult for a single server to load a large number of requests
2. The failure rate of a single server is high and the system collapse probability is high
3. The memory capacity of a single server is limited.

Environment configuration

When explaining the configuration file, we noticed a replication module (see article 8 in Redis.conf)

View the information of the current library: info replication

127.0.0.1:6379> info replication
# Replication
role:master # role
connected_slaves:0 # Number of slaves
master_replid:3b54deef5b7b7b7f7dd8acefa23be48879b4fcff
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:0
second_repl_offset:-1
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0

Since you need to start multiple services, you need multiple configuration files. Modify the following information for each configuration file:

• Port number
• pid file name
• Log file name
• rdb file name

Start single machine multi service cluster:

One master and two slave configuration

==By default, each Redis server is the master node== In general, we only need to configure the slave!

Recognize the boss! One master (79) and two slaves (80, 81)

Use SLAVEOF host port to configure the host for the slave.

Then the status of the slave can also be seen on the host:

Here we use commands to build, which is temporary, = = in real development, it should be configured in the configuration file of the slave, = = in this case, it is permanent.

Usage rules

1. The slave can only read and cannot write. The host can read and write, but it is mostly used for writing.

    127.0.0.1:6381> set name sakura # Slave 6381 write failed
   (error) READONLY You can't write against a read only replica.
   
   127.0.0.1:6380> set name sakura # Slave 6380 write failed
   (error) READONLY You can't write against a read only replica.
   
   127.0.0.1:6379> set name sakura
   OK
   127.0.0.1:6379> get name
   "sakura"
   

2. When the host is powered off and down, the role of the slave will not change by default. The cluster only loses the write operation. After the host is restored, the slave will be connected and restored to its original state.

3. When the slave is powered off and down, if the slave is not configured with the configuration file, the data of the previous host cannot be obtained as the host after restart. If it is reconfigured at this time, it is called the slave, and all the data of the host can be obtained. Here we will mention a synchronization principle.

4. As mentioned in the second article, by default, new hosts will not appear after host failure. There are two ways to generate new hosts:

   • The slave machine manually executes the command slave of no one. After execution, the slave machine becomes a master machine independently
   • Sentinel mode (automatic)

If there is no boss, can you choose a boss at this time? Manual!

If the host is disconnected, we can use SLAVEOF no one to turn ourselves into a host! Other nodes can be manually connected to the latest master node (manual)! If the boss fixes it at this time, reconnect it for so long!

Sentinel mode

The method of master-slave switching technology is: when the master server is down, a slave server needs to be manually switched to the master server, which requires manual intervention, which is laborious and laborious, and the service will not be available for a period of time. This is not a recommended way. More often, we give priority to sentinel mode.

The role of sentinels:

• Send commands to let Redis server return to monitor its running status, including master server and slave server.
• When the sentinel detects that the master is down, it will automatically switch the slave to the master, and then notify other slave servers through publish subscribe mode to modify the configuration file and let them switch hosts.

The core configuration of sentry

sentinel monitor mymaster 127.0.0.1 6379 1

• The number 1 indicates that when a sentinel subjectively thinks that the host is disconnected, he can objectively think that the host is faulty, and then start electing a new host.

test

redis-sentinel xxx/sentinel.conf

Sentry mode started successfully

At this time, the sentry is monitoring our host 6379. When we disconnect the host:

Advantages and disadvantages of sentinel mode

advantage:

1. Sentinel cluster is based on master-slave replication mode. It has all the advantages of master-slave replication
2. The master and slave can be switched, the fault can be transferred, and the availability of the system is better
3. Sentinel mode is the upgrade of master-slave mode. It is more robust from manual to automatic

Disadvantages:

1. Redis is not easy to expand online. Once the cluster capacity reaches the upper limit, online expansion will be very troublesome
2. The configuration of sentinel mode is actually very troublesome. There are many configuration items

Full configuration of sentinel mode

Complete sentinel mode configuration file sentinel conf

# Example sentinel.conf
 
# The port on which the sentinel sentinel instance runs is 26379 by default
port 26379
 
# sentinel's working directory
dir /tmp
 
# The ip port of the redis master node monitored by sentry sentinel 
# The master name can be named by itself. The name of the master node can only be composed of letters A-z, numbers 0-9 and the three characters ". -" form.
# Quorumwhen the sentinel of these quorum s thinks that the master master node is lost, it objectively thinks that the master node is lost
# sentinel monitor <master-name> <ip> <redis-port> <quorum>
sentinel monitor mymaster 127.0.0.1 6379 1
 
# When the requirepass foobared authorization password is enabled in the Redis instance, all clients connecting to the Redis instance must provide the password
# Set the password of sentinel sentinel connecting master and slave. Note that the same authentication password must be set for master and slave
# sentinel auth-pass <master-name> <password>
sentinel auth-pass mymaster MySUPER--secret-0123passw0rd
 
 
# After the specified number of milliseconds, the master node does not respond to the sentinel sentinel. At this time, the sentinel subjectively thinks that the master node goes offline for 30 seconds by default
# sentinel down-after-milliseconds <master-name> <milliseconds>
sentinel down-after-milliseconds mymaster 30000
 
# This configuration item specifies the maximum number of slave s that can synchronize the new master at the same time when a failover active / standby switch occurs,
The smaller the number, the better failover The longer it takes,
But if this number is bigger, it means more slave because replication Not available.
You can ensure that there is only one at a time by setting this value to 1 slave Is in a state where the command request cannot be processed.
# sentinel parallel-syncs <master-name> <numslaves>
sentinel parallel-syncs mymaster 1
 
 
 
# Failover timeout can be used in the following aspects: 
#1. The interval between two failover s of the same sentinel to the same master.
#2. When a slave synchronizes data from a wrong master, the time is calculated. Until the slave is corrected to synchronize data to the correct master.
#3. The time required to cancel an ongoing failover.  
#4. During failover, configure the maximum time required for all slaves to point to the new master. However, even after this timeout, the slave will still be correctly configured to point to the master, but it will not follow the rules configured by parallel syncs
# The default is three minutes
# sentinel failover-timeout <master-name> <milliseconds>
sentinel failover-timeout mymaster 180000
 
# SCRIPTS EXECUTION
 
#Configure the script to be executed when an event occurs. You can notify the administrator through the script. For example, send an email to notify relevant personnel when the system is not running normally.
#There are the following rules for the running results of scripts:
#If the script returns 1 after execution, the script will be executed again later. The number of repetitions is currently 10 by default
#If the script returns 2 after execution, or a return value higher than 2, the script will not be executed repeatedly.
#If the script is terminated due to receiving the system interrupt signal during execution, the behavior is the same as when the return value is 1.
#The maximum execution time of a script is 60s. If it exceeds this time, the script will be terminated by a SIGKILL signal and then re executed.
 
#Notification script: this script will be called when any warning level event occurs in sentinel (such as subjective failure and objective failure of redis instance),
#At this time, the script should notify the system administrator about the abnormal operation of the system through e-mail, SMS, etc. When calling the script, two parameters will be passed to the script,
#One is the type of event,
#One is the description of the event.
#If sentinel If the script path is configured in the conf configuration file, you must ensure that the script exists in this path and is executable, otherwise sentinel cannot be started successfully.
#Notification script
# sentinel notification-script <master-name> <script-path>
  sentinel notification-script mymaster /var/redis/notify.sh
 
# Client reconfiguration master node parameter script
# When a master changes due to failover, this script will be called to notify the relevant clients of the change of the master address.
# The following parameters will be passed to the script when calling the script:
# <master-name> <role> <state> <from-ip> <from-port> <to-ip> <to-port>
# At present, < state > is always "failover",
# < role > is one of "leader" or "observer". 
# The parameters from IP, from port, to IP and to port are used to communicate with the old master and the new master (i.e. the old slave)
# This script should be generic and can be called multiple times, not targeted.
# sentinel client-reconfig-script <master-name> <script-path>
sentinel client-reconfig-script mymaster /var/redis/reconfig.sh