If you already know the basic commands, you can This link View specific usage
Redis
Redis is the abbreviation of remote dictionary service, that is, remote dictionary service;
Redis is an in memory database (data is stored in memory, and the main data in mysql is stored in disk), KV database (key value), and data structure database (value provides rich data structure);
Redis is widely used, such as Twitter, Blizzard Entertainment, Github, Stack Overflow, Tencent, Alibaba, JD, etc., and many small and medium-sized companies are also using it;
Redis has 16 databases (dictionaries) and is single threaded, so only one database is used. A key corresponds to only one value;
Steps of using Redis
1. connect, the client connects to Redis;
2. auth, input login information, user name and password; If the password is not set, you do not need to enter the password;
3. Select: select the database to use (Redis has 16 databases). If it is not selected, the 0th database will be used by default
value
value code in Redis
string
Character array. The string is a dynamic string raw. When the length of the string is less than 1M, the capacity will be doubled; Over 1M, only 1M more each time; The maximum length of the string is 512M;
Note: redis string is a binary security string; Can store pictures, binary protocols and other binary data;
Basic command
# Set the value of key SET key val # Get the value of key GET key # Perform an atomic plus one operation INCR key # Performs the operation of adding an integer to an atom INCRBY key increment # Perform atomic minus one operation DECR key # Performs the atomic subtraction of an integer DECRBY key decrement # If the key does not exist, it is equivalent to the SET command. When the key exists, do nothing SETNX key value # Delete key val key value pair DEL key # Set or clear the bit value of the value (string) of the key at offset. SETBIT key offset value # Returns the bit value of the string corresponding to the key at offset GETBIT key offset # Count the number of bit s whose string is set to 1 BITCOUNT key
storage structure
If the string length is less than or equal to 20 and can be converted into an integer, use int storage;
If the string length is less than or equal to 44, use embstr to store;
If the string length is greater than 44, raw storage is used;
application
# Object storage
SET role:10001 '{["name"]:"mark",["sex"]:"male",["age"]:30}'
GET role:10001
# Generally, Redis clients can recognize ':', role:10001 refers to 10001 rows of data
# This value is in json format. If it needs to be modified frequently, it's better to use hash to store it
# accumulator
# Add 1 to the total number of reading statistics
incr reads
# Add 100 in total
incrby reads 100
# Distributed locks are briefly shown in this example and will be explained in detail in the subsequent articles
# Lock
setnx lock 1
# Release lock
del lock
# 1. Exclusive function 2 Definition of locking behavior 3 Release behavior definition
# Bit operation
# Monthly check-in function 10001 user ID 202106 check-in in June 2021 the first day of June
setbit sign:10001:202106 1 1
# Calculate the attendance in June 2021
bitcount sign:10001:202106
# Get the check-in status on the second day of June 2021. 1 has checked in and 0 has not checked in
getbit sign:10001:202106 2
list
Two way linked list implementation, list head and tail operation (delete and increase) time complexity O(1); The time complexity of finding intermediate elements is O(n);
Whether the data in the list is compressed is based on:
- If the element length is less than 48, it will not be compressed;
- The length difference of elements before and after compression shall not exceed 8, and they shall not be compressed;
Basic command
# Queue one or more elements from the left side of the queue LPUSH key value [value ...] # Pop up an element from the left side of the queue LPOP key # Queue one or more elements from the right side of the queue RPUSH key value [value ...] # Pop up an element from the right side of the queue RPOP key # Returns the elements 0, 1, 2 from the queue between start and end LRANGE key start end # Remove the element with the value of the previous count from the list stored in the key LREM key count value # It is a blocking version of RPOP, because this command blocks the connection when a given list cannot pop up any elements BRPOP key timeout # Timeout + delay queue
storage structure
/* Minimum ziplist size in bytes for attempting compression. */
#define MIN_COMPRESS_BYTES 48
/* quicklistNode is a 32 byte struct describing a ziplist for a quicklist.
* We use bit fields keep the quicklistNode at 32 bytes.
* count: 16 bits,
* max 65536 (max zl bytes is 65k, so max count actually < 32k).
* encoding: 2 bits, RAW=1, LZF=2.
* container: 2 bits, NONE=1, ZIPLIST=2.
* recompress: 1 bit, bool, true if node is temporary decompressed for usage.
* attempted_compress: 1 bit, boolean, used for verifying during testing.
* extra: 10 bits, free for future use; pads out the remainder of 32 bits */
typedef struct quicklistNode {
struct quicklistNode *prev;
struct quicklistNode *next;
unsigned char *zl;
unsigned int sz; /* ziplist size in bytes */
unsigned int count : 16; /* count of items in ziplist */
unsigned int encoding : 2; /* RAW==1 or LZF==2 */
unsigned int container : 2; /* NONE==1 or ZIPLIST==2 */
unsigned int recompress : 1; /* was this node previous compressed? */
unsigned int attempted_compress : 1; /* node can't compress; too small */
unsigned int extra : 10; /* more bits to steal for future usage */
} quicklistNode;
typedef struct quicklist {
quicklistNode *head; quicklistNode *tail;
unsigned long count; /* total count of all entries in all ziplists */
unsigned long len; /* number of quicklistNodes */
int fill : QL_FILL_BITS; /* fill factor for individual nodes */
unsigned int compress : QL_COMP_BITS; /* depth of end nodes not to compress;0=off */
unsigned int bookmark_count: QL_BM_BITS; quicklistBookmark bookmarks[];
} quicklist;
application
# Stack (first in and last out FILO) LPUSH + LPOP # perhaps RPUSH + RPOP # Queue (FIFO) LPUSH + RPOP # perhaps RPUSH + LPOP # blocking queue LPUSH + BRPOP # perhaps RPUSH + BLPOP
Asynchronous message queuing:
The operation is the same as the queue, but between different systems;
hash
Hash table, which contains this data structure in many high-level languages; c++ unordered_map quickly indexes value through key;
Basic command
# Get the value corresponding to the field in the hash corresponding to the key HGET key field # Set the value corresponding to the field in the hash corresponding to the key HSET key field value # Set multiple hash key value pairs HMSET key field1 value1 field2 value2 ... fieldn valuen # Get the value of multiple field s HMGET key field1 field2 ... fieldn # Add an integer value to the value corresponding to the field in the hash corresponding to the key HINCRBY key field increment # Get the number of key value pairs in the hash corresponding to the key HLEN key # Delete the key value pair of the hash corresponding to the key. The key is field HDEL key field
storage structure
if the number of nodes is greater than 512 (hash Max ziplist entries) or the length of all strings is greater than 64 (hash Max ziplistvalue), use dict;
if the number of nodes is less than or equal to 512 and the length of a string is less than 64, use ziplost to implement it;
application
# Storage object
{
hmset hash:10001 name mark age 18 sex male
# Compare with string
set hash:10001 '{["name"]:"mark",["sex"]:"male",["age"]:18}'
# Suppose the age of mark is 19
# hash:
hset hash:10001 age 19
# string:
get role:10001
# Call json to decrypt the obtained string, take out the field and modify the age value
# Then call json encryption
set role:10001 '{["name"]:"mark",["sex"]:"male",["age"]:19}'
}
# Shopping Cart
{
# Use user id as key
# Commodity id as field
# Quantity of goods as value
# Note: these items are displayed in the order we add them;
# Add item:
hset MyCart:10001 40001 1
lpush MyItem:10001 40001
# Increase quantity:
hincrby MyCart:10001 40001 1
hincrby MyCart:10001 40001 -1 # Reduce quantity 1
# Show all item quantities:
hlen MyCart:10001
# Delete item:
hdel MyCart:10001 40001
lrem MyItem:10001 1 40001
# Get all items:
lrange MyItem:10001
# 40001 40002 40003
hget MyCart:10001 40001
hget MyCart:10001 40002
hget MyCart:10001 40003
}
set
Assembly; It is used to store unique fields and does not require order;
Basic command
# Add one or more specified member elements to the key of the collection SADD key member [member ...] # Calculate the number of collection elements SCARD key # Return all value s in key SMEMBERS key # Returns whether the member member is a member of the stored set key SISMEMBER key member # Randomly return one or more elements in the key set without deleting them SRANDMEMBER key [count] # Removes and returns one or more random elements from the collection stored in the key SPOP key [count] # Returns the element of the difference between a set and a given set SDIFF key [key ...] # Returns the intersection of members of all specified sets SINTER key [key ...] # Returns all members of a given set of multiple sets SUNION key [key ...]
storage structure
if all elements are integers and the number of nodes is less than or equal to 512 (set Max intset entries), the integer array is used for storage;
If one of the elements is not an integer or the number of nodes is greater than 512, the dictionary is used for storage;
application
# luck draw
{
# Add lottery user
sadd Award:1 10001 10002 10003 10004 10005 10006
sadd Award:1 10009
# View all lottery users
smembers Award:1
# Extract multiple lucky users
srandmember Award:1 10
}
# Common concern
{
sadd follow:A mark king darren mole vico
sadd follow:C mark king darren
sinter follow:A follow:C
}
# Recommend friends
{
sadd follow:A mark king darren mole vico
sadd follow:C mark king darren
# C possible acquaintances:
sdiff follow:A follow:C
}
zset
Ordered set; Used to realize ranking; It is an orderly and unique;
Basic command
# Added to the sorted set with key ZADD key [NX|XX] [CH] [INCR] score member [score member ...] # Delete the key value pair of member from the ordered set with key as key ZREM key member [member ...] # Returns the score value of the member in the ordered set key ZSCORE key member # Add increment to the score value of the member of the ordered set key ZINCRBY key increment member # Returns the number of ordered set elements of the key ZCARD key # Returns the ranking of member s in the ordered set key ZRANK key member # Returns the elements of the specified range stored in the ordered set key order by id limit 1,100 ZRANGE key start stop [WITHSCORES] # Returns the members in the specified interval in the ordered set key (in reverse order) ZREVRANGE key start stop [WITHSCORES]
storage structure
If the number of nodes is greater than 128 or the length of a string is greater than 64, a skip list is used;
If the number of nodes is less than or equal to 128 (Zset Max ziplost entries) and the length of all strings is less than or equal to 64 (Zset maxziplost value), ziplost storage is used;
Save space when there is little data; O(n)
When the number is large, access performance; O(1) o(logn)
application
# Hot list
{
# Click news:
zincrby hot:20210601 1 10001
zincrby hot:20210601 1 10002
zincrby hot:20210601 1 10003
zincrby hot:20210601 1 10004
zincrby hot:20210601 1 10005
zincrby hot:20210601 1 10006
zincrby hot:20210601 1 10007
zincrby hot:20210601 1 10008
zincrby hot:20210601 1 10009
zincrby hot:20210601 1 10010
# Get leaderboard:
zrevrange hot:20210601 0 9 withscores
}
# Delay queue
# Sequence the message into a string as the member of zset; The expiration processing time of this message is taken as score,
# Then use multiple threads to poll zset to get the expired tasks for processing.
{
def delay(msg):
msg.id = str(uuid.uuid4()) #Ensure member uniqueness
value = json.dumps(msg)
retry_ts = time.time() + 5 # Try again in 5s
redis.zadd("delay-queue", retry_ts, value)
# Use connection pool
def loop():
while True:
values = redis.zrangebyscore("delay-queue", 0, time.time(), start=0, num=1)
if not values:
time.sleep(1)
continue
value = values[0]
success = redis.zrem("delay-queue", value)
if success:
msg = json.loads(value)
handle_msg(msg)
# Disadvantages: loop is a multi-threaded competition. Both threads get data from zrangebyscore, but zrem succeeds and fails
# Optimization: to avoid redundant operations, you can use lua script atom to execute these two commands
# Solution: funnel current limiting
}
# Time window current limiting
# The system limits that a user's behavior can only occur N times within the specified time range (dynamic);
{
# Specify user_ A behavior action of ID can only occur for the number of times Max during the period of a specific time_ count
local function is_action_allowed(red, userid, action, period, max_count)
local key = tab_concat({"hist", userid, action}, ":")
local now = zv.time()
red:init_pipeline()
# Record behavior
red:zadd(key, now, now)
# Remove the behavior records before the time window, and the rest are records in the time window
red:zremrangebyscore(key, 0, now - period *100)
# Gets the number of behaviors in the time window
red:zcard(key)
# Set the expiration time to prevent cold users from continuously occupying memory. The length of the time window is + 1 second
red:expire(key, period + 1)
local res = red:commit_pipeline()
return res[3] <= max_count
end
# Maintain a time window, clear all records outside the window, and only keep records in the window;
# Disadvantages: the data in all time windows are recorded. If this amount is large, it is not suitable for such current limiting; Funnel current limiting
# Note: if you use key + expire, it can also be realized, but the realization is fusing, and the maintenance time window is the function of current limiting;
}
Distributed timer:
the producer hash es the scheduled tasks to different redis entities, and assigns a dispatcher process to each redis entity to periodically obtain the timeout events in redis and publish them to different consumers;