Redis cluster
https://www.cnblogs.com/ruiyeclub/p/12355073.html
1, Redis related
1. Introduction to Redis
Redis (Remote Dictionary Server), namely remote dictionary service, is an ey value storage system that supports master-slave synchronization. It is an open source log and key value database written in ANSI C language, supporting network, memory based and persistent, and provides API s in multiple languages.
Redis is a key value storage system. It supports stored value types: string, list, set, Zset, and hash. These data types (data types) support push/pop, add/remove, intersection, union, difference and richer operations, and these operations are atomic. On this basis, redis supports various sorting methods. Like memcached, data is cached in memory to ensure efficiency. The difference is that redis will periodically write the updated data to the disk or write the modification operation to the additional record file, and on this basis, it realizes master-slave synchronization.
2. Redis features
Redis and other key - value caching products have the following three characteristics:
- Redis supports data persistence. It can save the data in memory on disk and can be loaded again for use when restarting.
- Redis not only supports simple key value data, but also provides storage of list, set, zset, hash and other data structures.
- Redis supports data backup, that is, data backup in master slave mode.
3. Redis advantages
- Extremely high performance – Redis can read 110000 times / s and write 81000 times / s.
- Rich data types – Redis supports string, lists, hashes, sets and Ordered Sets data type operations of binary cases.
- Atomic – all Redis operations are atomic, which means that they are either executed successfully or not executed at all. A single operation is atomic. Multiple operations also support transactions, that is, atomicity, which are packaged through MULTI and EXEC instructions.
- Rich features – Redis also supports publish/subscribe, notification, key expiration and other features.
2, Cluster construction
1. Docker compose file writing
version: '2'
services:
master:
image: redis
container_name: redis-master
command: redis-server --requirepass 123456
ports:
- "6379:6379"
networks:
sentinel-master:
ipv4_address: 192.169.0.2
slave1:
image: redis
container_name: redis-slave-1
ports:
- "6380:6379"
command: redis-server --slaveof redis-master 6379 --requirepass 123456 --masterauth 123456
depends_on:
- master
networks:
sentinel-master:
ipv4_address: 192.169.0.3
slave2:
image: redis
container_name: redis-slave-2
ports:
- "6381:6379"
command: redis-server --slaveof redis-master 6379 --requirepass 123456 --masterauth 123456
depends_on:
- master
networks:
sentinel-master:
ipv4_address: 192.169.0.4
networks:
sentinel-master:
driver: bridge
ipam:
config:
- subnet: 192.169.0.0/16
2. Run cluster
[root@iZp06cqz6zbn9jZ redis_clusters]# docker-compose up -d
Creating network "redis_clusters_sentinel-master" with driver "bridge"
Creating redis-master ... done
Creating redis-slave-1 ... done
Creating redis-slave-2 ... done
[root@iZp06cqz6zbn9jZ redis_clusters]# docker-compose ps
Name Command State Ports
-------------------------------------------------------------------------------
redis-master docker-entrypoint.sh redis ... Up 0.0.0.0:6379->6379/tcp
redis-slave-1 docker-entrypoint.sh redis ... Up 0.0.0.0:6380->6379/tcp
redis-slave-2 docker-entrypoint.sh redis ... Up 0.0.0.0:6381->6379/tcp
[root@iZp06cqz6zbn9jZ redis_clusters]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
acdfccf7f0b4 redis "docker-entrypoint.s..." 16 seconds ago Up 14 seconds 0.0.0.0:6381->6379/tcp redis-slave-2
76b74b0a640f redis "docker-entrypoint.s..." 16 seconds ago Up 14 seconds 0.0.0.0:6380->6379/tcp redis-slave-1
f0ec343bc3b4 redis "docker-entrypoint.s..." 16 seconds ago Up 15 seconds 0.0.0.0:6379->6379/tcp redis-master
3. Enter the cluster
# Enter the master node [root@iZp06cqz6zbn9jZ redis_clusters]# docker exec -it f0ec343bc3b4 bash root@f0ec343bc3b4:/data# redis-cli 127.0.0.1:6379> keys * (error) NOAUTH Authentication required. 127.0.0.1:6379> ahth 123456 (error) ERR unknown command `ahth`, with args beginning with: `123456`, 127.0.0.1:6379> auth 123456 OK 127.0.0.1:6379> keys * (empty array) # Enter the master node > set key/value 127.0.0.1:6379> set name dyl OK 127.0.0.1:6379> keys * 1) "name" 127.0.0.1:6379> get name "dyl" 127.0.0.1:6379> exit root@f0ec343bc3b4:/data# exit # Enter slave1 node [root@iZp06cqz6zbn9jZ redis_clusters]# docker exec -it 76b74b0a640f bash root@76b74b0a640f:/data# redis-cli 127.0.0.1:6379> auth 123456 OK 127.0.0.1:6379> keys * 1) "name" # Go to slave2 node > View key/value [root@iZp06cqz6zbn9jZ redis_clusters]# docker exec -it acdfccf7f0b4 bash root@acdfccf7f0b4:/data# redis-cli 127.0.0.1:6379> auth 123456 OK 127.0.0.1:6379> keys * 1) "name" 127.0.0.1:6379> get name "dyl" 127.0.0.1:6379>
3, Six data structures in Redis
https://www.cnblogs.com/xrq730/p/8944539.html
https://www.cnblogs.com/ysocean/p/9080940.html
1. string data type
As like as two peas, string is the most basic data type of Redis, which can be understood as a type exactly like Memcached and a key corresponding to a value. The string type is binary safe, which means that the string of redis can contain any data, such as pictures or serialized objects. The string value in a redis can be 512M at most.
| command | describe | usage |
|---|---|---|
| SET | (1) Associate string Value to Key (2) If the Key is associated, it will be overwritten, regardless of the type (3) If the original Key has a lifetime TTL, the TTL is cleared | SET key value [EX seconds] [PX milliseconds] [NX|XX] |
| GET | (1) Return the string value associated with the key (2) if the key does not exist, return nil (3) Key does not store a string and returns an error, because GET is only used to process strings | GET key |
| MSET | (1) Set one or more key value pairs at the same time (2) If a given Key already exists, the new value of MSET will overwrite the old value (3) If the above overwrite is not what you want, use the MSETNX command to overwrite only when all keys do not exist (4) MSET is an atomic operation. All keys will be set at the same time. There will be no update or no update | MSET key value [key value ...] |
| MGET | (1) Returns the Value corresponding to one or more given keys. (2) if a Key does not exist, the Key returns nil | MGET key [key ...] |
| SETEX | (1) Associate the Value with the Key (2) and set the Key lifetime to seconds, in seconds (3) If the Value corresponding to the Key already exists, the old Value will be overwritten (4) SET can also SET the expiration time, but the difference is that SETNX is an atomic operation, that is, the associated value is completed at the same time as the SET survival time | SETEX key seconds value |
| SETNX | (1) Set the Value of the Key to Value, if and only if the Key does not exist (2) if the given Key already exists, SEXNX will not take any action | SETNX key value |
| APPEND | String append | APPEND key value |
| DEL | Delete key | DEL key [key ...] |
| INCR | Increment the numeric value stored in the key by one | INCR key |
| DECR | Subtract the numeric value stored in the key by one | DECR key |
[root@iZp06cqz6zbn9jZ redis_clusters]# docker exec -it 5a1cc01437ff bash root@5a1cc01437ff:/data# redis-cli 127.0.0.1:6379> auth 123456 OK 127.0.0.1:6379> kyes * (error) ERR unknown command `kyes`, with args beginning with: `*`, 127.0.0.1:6379> keys * (empty array) 127.0.0.1:6379> set string1 dyl1 OK 127.0.0.1:6379> 127.0.0.1:6379> keys * 1) "string1" 127.0.0.1:6379> get string1 "dyl1" 127.0.0.1:6379> mset string2 dyl2 string3 dyl3 OK 127.0.0.1:6379> keys * 1) "string2" 2) "string1" 3) "string3" 127.0.0.1:6379> mget string1 string2 1) "dyl1" 2) "dyl2" 127.0.0.1:6379> setex strtime 10 time1 OK 127.0.0.1:6379> keys * 1) "string2" 2) "string1" 3) "strtime" 4) "string3" Enter again after ten seconds 127.0.0.1:6379> keys * 1) "string2" 2) "string1" 3) "string3" 127.0.0.1:6379> append string1 123 (integer) 7 127.0.0.1:6379> get string1 "dyl1123" 127.0.0.1:6379> del string3 (integer) 1 127.0.0.1:6379> keys * 1) "string2" 2) "string1" # It can be used as a counter for the number of times the page has been accessed 127.0.0.1:6379> set one 1 OK 127.0.0.1:6379> keys * 1) "one" 127.0.0.1:6379> incr one (integer) 2 127.0.0.1:6379> get one "2" 127.0.0.1:6379> decr one (integer) 1 127.0.0.1:6379> get one "1"
2. hash data type
3. list data type
ist list, which is a simple string list, sorted according to the insertion order. You can add an element to the head (left) or tail (right) of the list. Its bottom layer is actually a linked list.
| command | describe | usage |
|---|---|---|
| LPUSH | (1) Insert one or more values value into the header of the list key (2) If there are multiple value values, each value value is inserted into the header from left to right (3) If the key does not exist, an empty list will be created and LPUSH operation will be executed (4) The key exists but is not a list type. An error is returned | LPUSH key value [value ...] |
| LPUSHX | (1) Insert the value value into the header of the list key. If the upgrade key exists and is a list (2) When the key does not exist, the LPUSHX command does nothing | LPUSHX key value |
| LPOP | (1) Removes and returns the header element of the list key | LPOP key |
| LRANGE | (1) Returns the elements within the specified interval in the list key. The interval is specified by offset start and stop (2) Both start and stop are based on 0 bits (3) Negative subscripts can be used, - 1 for the last element of the list, - 2 for the penultimate element of the list, and so on (4) If start is greater than the maximum subscript of the list, an empty list will be returned (5) Stop is greater than the maximum subscript of the list, stop = the maximum subscript of the list | LRANGE key start stop |
| LREM | (1) Remove the elements in the list equal to value according to the value of count (2) Count > 0 means to search from beginning to end and remove elements equal to value. The number is count (3) Count < 0 means to search from end to end and remove elements equal to value. The number is count (4) count=0 means to remove all elements equal to value in the table | LREM key count value |
| LSET | (1) Set the value of the element whose index is the index of the list key to value (2) The index parameter is out of range, or an error is returned when LSET is performed on an empty list | LSET key index value |
| LINDEX | (1) Returns the elements in the key list with index as the subscript | LINDEX key index |
| LINSERT | (1) Insert the value value into the list key, before or after pivot (2) If pivot does not exist in the list key, no operation will be performed. (3) if key does not exist, no operation will be performed | LINSERT key BEFORE|AFTER pivot value |
| LLEN | (1) The length of the returned list key (2) if the key does not exist, 0 is returned | LLEN key |
| LTRIM | (1) Prune a list so that the list only returns elements within the specified interval, and those that do not exist within the specified interval will be removed | LTRIM key start stop |
| RPOP | (1) Remove and return the tail element of the list key | RPOP key |
| RPOPLPUSH | In an atomic time, perform two actions: (1) pop up the last element in the list source and return it to the client (2) Insert the pop-up element of source into the destination list as the header element of the destination list | RPOPLPUSH source destination |
| RPUSH | (1) Insert one or more values value at the end of the list key | RPUSH key value [value ...] |
| RPUSHX | (1) Insert value at the end of the list key, if and only if the key exists and is a list (2) key does not exist. RPUSHX does nothing | RPUSHX key value |
- Stack command lpush+lpop
- Queue command lpush+rpop
- Finite set command lpush+ltrim
- Message queuing command lpush+rpop
root@5a1cc01437ff:/data# redis-cli 127.0.0.1:6379> auth 123456 OK 127.0.0.1:6379> keys * 1) "int1" 2) "string2" 3) "string1" 127.0.0.1:6379> del int1 string1 string2 (integer) 3 127.0.0.1:6379> keys * (empty array) 127.0.0.1:6379> lpush list1 1 (integer) 1 127.0.0.1:6379> lpush list1 2 (integer) 2 127.0.0.1:6379> keys * 1) "list1" 127.0.0.1:6379> llen list1 (integer) 2 127.0.0.1:6379> lindex list1 0 "2" 127.0.0.1:6379> lindex list1 1 "1" 127.0.0.1:6379> lpop list1 "2" 127.0.0.1:6379> lindex list1 0 "1" 127.0.0.1:6379> llen list1 (integer) 1 127.0.0.1:6379> lpush list1 2 (integer) 2 127.0.0.1:6379> rpop list1 "1"
4. set data type
Redis's set is an unordered and non repeatable set of string type.
Usage scenario: using the intersection and union feature of the set, for example, in the social field, we can easily find the common friends and fields of common interest of multiple users.
| command | describe | usage |
|---|---|---|
| SADD | Add: sadd set1 1 2 3 | SADD key number [member ...] |
| SCARD | Number of collection elements | SCARD key |
| SDIFF | Difference set: sdiff set1 set2 sets the difference set between set1 and set2 | SDIFF key [key ...] |
| SDIFFSTORE | (1) Similar to SDIFF, but the result is saved to the destination set instead of simply returning the result set (2) destination if it already exists, overwrite it | SDIFFSTORE destionation key [key ...] |
| SINTER | Intersection: sinter set1 set2 | SINTER key [key ...] |
| SINTERSTORE | (1) Similar to SINTER, but the result is saved in the early destination set instead of simply returning the result set (2) Overwrite if destination already exists (3) destination can be the key itself | SINTERSTORE destination key [key ...] |
| SISMEMBER | (1) Judge whether the member element is a member of the key. 0 means no and 1 means yes | SISMEMBER key member |
| SMEMBERS | (1) Returns all members in the set key (2) Nonexistent key s are treated as empty sets | SMEMBERS key |
| SMOVE | (1) Atomically move the member element from the source set to the destination set (2) The source collection does not contain the member element. The SMOVE command does not perform any operation and only returns 0 (3) The destination already contains the member element. The SMOVE command simply removes the member element of the source set | SMOVE source desination member |
| SPOP | (1) Remove and return a random element in the collection. If count is not specified, a random element will be returned randomly (2) If count is positive and less than the number of collection elements, an array of count elements is returned, and the elements in the array are different (3) If count is a positive number and is greater than or equal to the number of set elements, the entire set is returned (4) If count is negative, the command returns an array. The elements in the array may be repeated multiple times, and the number is the absolute value of count | SPOP key [count] |
| SRANDMEMBER | (1) If count is not specified, a random element (2) in the set will be returned. The count is the same as above | SRANDMEMBER key [count] |
| SREM | Delete: removes one or more member elements from the collection. Nonexistent members will be ignored | SREM key member [member ...] |
| SUNION | Union: sunion set1 set2 | SUNION key [key ...] |
| SUNIONSTORE | (1) Similar to SUNION, but the results are saved to the destination set instead of simply returning the result set (2) destination already exists, overwrite old value (3) destination can be the key itself | SUNION destination key [key ...] |
127.0.0.1:6379> sadd set1 1 2 3 127.0.0.1:6379> sadd set 2 3 4 5 127.0.0.1:6379> sdiff set1 set 1) "1" 127.0.0.1:6379> sunion set1 set 1) "1" 2) "2" 3) "3" 4) "4" 5) "5" 127.0.0.1:6379> sinter set set1 1) "2" 2) "3" 127.0.0.1:6379> srem set 4 (integer) 1 127.0.0.1:6379> smembers set 1) "2" 2) "3" 3) "5"
5. zset data type
| command | describe | usage |
|---|---|---|
| ZADD | (1) Add one or more member elements and their score values to the ordered set key (2) If the member is already a member of the ordered set, update the score corresponding to the member and re insert the member to ensure that the member is in the correct position (3) score can be an integer value or a double precision floating-point number | ZADD key score member [[score member] [score member] ...] |
| ZCARD | (1) Returns the number of elements of the ordered set key | ZCARD key |
| ZCOUNT | (1) Returns the number of members in the ordered set key with score value > = min and < = max | ZCOUNT key min max |
| ZRANGE | (1) Returns the members within the specified interval in the ordered set key. The member positions are sorted from small to large by score (2) Members with the same score value are arranged in dictionary order (3) The members are required to be arranged from large to small according to the score. Use the ZREVRANGE command( (4) The subscript parameters start and stop are based on 0, or negative numbers can be used, - 1 represents the last member and - 2 represents the penultimate member (5) The with scores option allows the member to be returned with its score value | ZRANGE key start stop [WITHSCORES] |
| ZRANK | (1) Returns the ranking of members in the ordered set key. The members of the ordered set are arranged from small to large according to the score value (2) The bottom of the ranking is 0, that is, the member with the smallest score is ranked 0 (3) The ZREVRANK command ranks members by score value from large to small | ZRANK key number |
| ZREM | (1) Remove one or more members from the ordered set key. Nonexistent members will be ignored (2) When the key exists but is not an ordered set, an error is returned | ZREM key member [member ...] |
| ZREMRANGEBYRANK | (1) Remove all members within the specified ranking range in the ordered set key | ZREMRANGEBYRANK key start stop |
| ZREMRANGEBYSCORE | (1) Remove all members in the ordered set key whose score value is > = min and < = max | ZREMRANGEBYSCORE key min max |
127.0.0.1:6379> zadd sort1 1 a 3 b 2 c 4 d (integer) 4 127.0.0.1:6379> zrange sort1 0 -1 1) "a" 2) "c" 3) "b" 4) "d" 127.0.0.1:6379> zcard sort1 (integer) 4 127.0.0.1:6379> zrem sort1 d (integer) 1 127.0.0.1:6379> zrange sort1 0 -1 1) "a" 2) "c" 3) "b" 127.0.0.1:6379> zcount sort1 2 3 (integer) 2 127.0.0.1:6379>
6. stream data type
Redis based Stream type perfect message queue solution https://zhuanlan.zhihu.com/p/60501638
How to choose Redis and professional message queues? https://zhuanlan.zhihu.com/p/367452637
When Redis is used as a queue, it always faces two problems:
-
Redis itself may lose data
-
In the face of message backlog, Redis is short of memory resources. Message queues such as Kafka and RabbitMQ are different. Their data will be stored on disk
If your business scenario is simple enough, insensitive to data loss, and the message backlog probability is small, it is entirely possible to treat redis as a queue. Redis is lighter in deployment and operation and maintenance than Kafka and RabbitMQ.
Redis Stream is a newly added data structure in Redis 5.0.
Redis Stream is mainly used for Message Queue (MQ). Redis itself has a redis publish / subscribe (pub/sub) to realize the function of Message Queue, but its disadvantage is that messages cannot be persistent. If the network is disconnected or redis is down, messages will be discarded.
- In short, publish / subscribe (pub/sub) can distribute messages, but cannot record historical messages.
- Redis Stream provides message persistence and active / standby replication functions, allowing any client to access data at any time, remembering the access location of each client, and ensuring that messages are not lost.
It's very good about redis: https://www.jianshu.com/p/efae507f03cf
127.0.0.1:6379> xadd codehole * name laoqian age 30
"1624075334164-0"
127.0.0.1:6379> xadd codehole * name xiaoyu age 29
"1624075342341-0"
127.0.0.1:6379> xadd codehole * name xiaoqian age 1
"1624075348213-0"
127.0.0.1:6379> xlen codehole
(integer) 3
127.0.0.1:6379> xrange codehole - +
1) 1) "1624075334164-0"
2) 1) "name"
2) "laoqian"
3) "age"
4) "30"
2) 1) "1624075342341-0"
2) 1) "name"
2) "xiaoyu"
3) "age"
4) "29"
3) 1) "1624075348213-0"
2) 1) "name"
2) "xiaoqian"
3) "age"
4) "1"
127.0.0.1:6379> xrange codehole 1624075342341-0 +
1) 1) "1624075342341-0"
2) 1) "name"
2) "xiaoyu"
3) "age"
4) "29"
2) 1) "1624075348213-0"
2) 1) "name"
2) "xiaoqian"
3) "age"
4) "1"
127.0.0.1:6379> xrange codehole 1624075348213-0 +
1) 1) "1624075348213-0"
2) 1) "name"
2) "xiaoqian"
3) "age"
4) "1"
127.0.0.1:6379> xdel codehole 1624075348213-0
(integer) 1
127.0.0.1:6379> xrange codehole - +
1) 1) "1624075334164-0"
2) 1) "name"
2) "laoqian"
3) "age"
4) "30"
2) 1) "1624075342341-0"
2) 1) "name"
2) "xiaoyu"
3) "age"
4) "29"
127.0.0.1:6379> del codehole
(integer) 1