Redis master-slave replication

brief introduction

The read-write capability supported by a single redis is still limited. At this time, we can use multiple redis to improve the concurrent processing capability of redis. How these redis cooperate requires a certain architecture design. Here, we first analyze and implement the master / slave architecture

Basic architecture

redis master-slave architecture is shown in the figure below:

The master is responsible for reading and writing, synchronizing the data to the slave, and the slave node is responsible for reading

Quick start practice

Based on Redis, a Master-Slave architecture is designed, including one Master and two slaves. The Master is responsible for reading and writing Redis and synchronizing data to the Slave. The Slave is only responsible for reading. The steps are as follows:

First, duplicate redis01, for example:

cp -r redis01/ redis02

cp -r redis01/ redis03

Step 2: if there are existing redis services, first stop all the original redis services (docker rm -f redis container name) and start a new redis container, for example:

docker run -p 6379:6379 --name redis6379 \
-v /usr/local/docker/redis01/data:/data \
-v /usr/local/docker/redis01/conf/redis.conf:/etc/redis/redis.conf \
-d redis redis-server /etc/redis/redis.conf \
--appendonly yes

docker run -p 6380:6379 --name redis6380 \
-v /usr/local/docker/redis02/data:/data \
-v /usr/local/docker/redis02/conf/redis.conf:/etc/redis/redis.conf \
-d redis redis-server /etc/redis/redis.conf \
--appendonly yes

docker run -p 6381:6379 --name redis6381 \
-v /usr/local/docker/redis03/data:/data \
-v /usr/local/docker/redis03/conf/redis.conf:/etc/redis/redis.conf \
-d redis redis-server /etc/redis/redis.conf \
--appendonly yes

Step 3: check the redis service role

Start the three clients, log in to the three redis container services respectively, and view the roles through the info command. By default, the three newly started redis service roles are master

127.0.0.1:6379> info replication

\# Replication
role:master
connected_slaves:0
master_repl_offset:3860
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:2
repl_backlog_histlen:3859

Step 4: check the ip settings of redis6379

docker inspect redis6379

......
"Networks": {
    "bridge": {
        "IPAMConfig": null,
        "Links": null,
        "Aliases": null,
        "NetworkID": "c33071765cb48acb1efed6611615c767b04b98e6e298caa0dc845420e6112b73",
        "EndpointID": "4c77e3f458ea64b7fc45062c5b2b3481fa32005153b7afc211117d0f7603e154",
        "Gateway": "172.17.0.1",
        "IPAddress": "172.17.0.2",
        "IPPrefixLen": 16,
        "IPv6Gateway": "",
        "GlobalIPv6Address": "",
        "GlobalIPv6PrefixLen": 0,
        "MacAddress": "02:42:ac:11:00:02",
        "DriverOpts": null
    }
}

Step 5: set Master/Slave architecture

Log in to redis6380/redis6381 respectively, and then execute the following statement

slaveof ip port # IP: the IP address of the master node, port: the port number of the master node

slaveof 172.17.0.2 6379 

Note: if the master has a password, you need to add the statement "masterauth your password" in the redis.conf configuration file of the slave, restart redis, and then execute the slaveof command

Step 6: log in to redis6379 again and check info

[root@centos7964 ~]# docker exec -it redis6379 redis-cli
127.0.0.1:6379> info replication

\# Replication
role:master
connected_slaves:2
slave0:ip=172.17.0.3,port=6379,state=online,offset=2004,lag=1
slave1:ip=172.17.0.4,port=6379,state=online,offset=2004,lag=1
master_failover_state:no-failover
master_replid:5baf174fd40e97663998abf5d8e89a51f7458488
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:2004
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:2004

Step 7: log in to redis6379 to test. The master can read and write

[root@centos7964 ~]# docker exec -it redis6379 redis-cli
127.0.0.1:6379> set role master6379
OK
127.0.0.1:6379> get role
"master6379"
127.0.0.1:6379>

Step 8: log in to redis6380/6381 to test. slave can only read but not write.

[root@centos7964 ~]# docker exec -it redis6380 redis-cli
127.0.0.1:6379> get role
"master6379"
127.0.0.1:6379> set role slave6380
(error) READONLY You can't write against a read only replica.
127.0.0.1:6379>

Read / write test analysis in Java. The code is as follows:

package com.jt;

import org.junit.Test;
import redis.clients.jedis.Jedis;

/**
 * Analysis of master-slave architecture
 */
public class MasterSlaveTests {

    @Test //Master node (supports read and write operations)
    public void testWriteRead(){
        Jedis jedis = new Jedis("192.168.126.129",6379);
        jedis.set("a1", "100");
        String a1 = jedis.get("a1");
        System.out.println(a1);
        jedis.close();
    }

    @Test //Slave node (only supports read operations)
    public void testRead(){
        Jedis jedis = new Jedis("192.168.126.129",6380);
        //jedis.set("a1", "200");// Write operation is not allowed here
        String a1 = jedis.get("a1");
        System.out.println(a1);
        jedis.close();
    }
}
//reliability

Principle analysis of master-slave synchronization

Redis master-slave structure can adopt one master-slave structure. Redis master-slave replication can be divided into full synchronization and incremental synchronization according to whether it is full or not.

• Redis full synchronization

Redis full replication usually occurs in the Slave initialization stage. At this time, the Slave needs to copy all the data on the Master. The specific steps are as follows:
1) Connect the master server from the server and send the sync command;
2) After receiving the sync naming, the master server starts executing the bgsave command to generate an rdb file and uses the buffer to record all write commands executed thereafter;
3) After the master server bgsave executes, it sends snapshot files to all slave servers, and continues to record the executed write commands during sending;
4) After receiving the snapshot file from the server, discard all old data and load the received snapshot;
5) After sending the snapshot from the master server, start sending the write command in the buffer to the slave server;
6) Loading the snapshot from the server, receiving the command request, and executing the write command from the main server buffer;

• Redis incremental synchronization

Redis incremental replication refers to the process of synchronizing the write operations of the master server to the Slave server when the Slave starts working normally after the Slave is initialized. The main process of incremental replication is that the master server sends the same write command to the Slave server every time it executes a write command, and the Slave server receives and executes the received write command.

Section interview analysis

• What would you do if redis wanted to support 100000 + concurrency?

It's almost impossible for a single redis to say that the QPS exceeds 100000 +, unless there are some special circumstances, such as your machine's performance is particularly good, the configuration is particularly high, the physical machine and maintenance are particularly good, and your overall operation is not too complex. The average single machine is tens of thousands. To truly realize the high concurrency of redis, read-write separation is required. For cache, it is generally used to support high read concurrency. There are relatively few write requests, and write requests may be thousands of times a second. There will be relatively more requests to read, for example, 200000 times a second. Therefore, the high concurrency of redis can be realized based on the master-slave architecture and the read-write separation mechanism.

• What is the replication mechanism of Redis?

(1) redis copies data to the slave node asynchronously.
(2) A master node can be configured with multiple slave node s.
(3) Slave nodes can also be connected to other slave node s.
(4) When a slave node is replicated, the block master node will not work normally.
(5) When copying, slave node will not block its own query operations. It will use the old data set to provide services; However, when the copy is completed, the old dataset needs to be deleted and the new dataset needs to be loaded. At this time, the external service will be suspended.
(6) The slave node is mainly used for horizontal expansion and read-write separation. The expanded slave node can improve the read throughput.

Redis sentinel mode

brief introduction

Sentinel is a mechanism to achieve high availability under the master-slave architecture mode of Redis.
The Sentinel system composed of one or more Sentinel instance s can monitor any number of master servers and all slave servers under these master servers, and automatically upgrade a slave server under the offline master server to a new master server when the monitored master server enters the offline state, Then, the new master server continues to process the command request instead of the offline master server.

Basic architecture