Redis 06 redis Zset structure and Application

1 zset

zset (ordered set) is the most frequently asked data structure in Redis. This ordered set is similar to the set container of C + +, but the underlying structure is different. The underlying structure of C + + is implemented using RB tree (red black tree). Unlike zset, which is implemented using a hop table.

On the one hand, zset ensures the uniqueness of the internal value value through set, and on the other hand, it sorts through the score (weight) of value. This sorting function is realized through Skip List.

Using zset's effect of de duplication and order can be used in many usage scenarios, which are usually used to realize ranking. Two examples:

Store the list of fans. value is the ID of fans and score is the time stamp of attention. In this way, the attention of fans can be sorted.
Store the student's score, value makes the student's ID and score is the student's score, so that the student's score can be ranked.

For skip table, please refer to my article: Redis -- 01 -- Introduction to redis (redis installation and download, analysis of underlying storage structure principle).

2 basic command

2.1 ZADD,ZREM,ZSCORE

# Add to the sorted set with key.
# 1) If the specified added member is already a member in the ordered set, the score of the member will be updated and updated to the correct sorting position.
# 2) If the key does not exist, a new key will be created and the score/member pair will be added to the ordered set,
# 	Just like there was an empty ordered set. If the key exists but the type is not an ordered set, an error response will be returned.
#	The fractional value is a double precision floating-point number string+ Inf and - inf are valid values.
# 3) The ZADD command supports some parameters (> = redis 3.0.2) in front of the score/member pair after the key. They are:
# 	3.1) XX: only update existing members without adding new members.
# 	3.2) NX: do not update existing members. Add only new members.
# 	3.3) ch: modify the return value to the total number of changed members. The original value is to return the total number of newly added members (CH means changed).
#			The changed element is a newly added member or an existing member update score. So the members specified in the command have the same score
#			Will not be counted. Note: under normal circumstances, the return value of ZADD only calculates the number of newly added members.
# 	CH is actually not difficult and not used much, but it will be introduced in detail below.
# 	3.4) INCR: when ZADD specifies this option, the member's operation is the same as the ZINCRBY command, which increments the member's score.
# 4) Members in an ordered set cannot be repeated and are unique. However, different members may have the same score.
#	When multiple members have the same score, they will be ordered lexically (the score is still used as the first sorting condition, and then the members with the same score are sorted relatively according to the dictionary rules).
# 5) Return value Integer reply, including:
# 5.1) the number of members added to the ordered set, excluding members that already have update scores (i.e. when the CH parameter is not used).
# 5.2) if the INCR parameter is specified, the return will become bulk string reply. The new fraction (double precision floating-point number) string of the member.
# 6) Redis > = 2.4: accept multiple members. Before Redis 2.4, the command can only add or update one member.
ZADD key [NX|XX] [CH] [INCR] score member [score member ...]

# Delete the key value pair of member from the ordered set with key as key.
# 1) When the key exists, but it is not an ordered collection type, an error is returned.
# 2) The return value is integer reply, which is the following integer:
# 	Returns the number of members deleted from the ordered collection, excluding nonexistent members.
# 3) = 2.4: accept multiple elements. In versions prior to 2.4, only one member could be deleted at a time.
ZREM key member [member ...]

# Returns the score value of the member in the ordered set key.
# 1) If the member element is not a member of the ordered set key, or the key does not exist, nil is returned.
# 2) Return value bulk string reply: returns the score value (double type floating-point number) of the member, expressed in string form.
ZSCORE key member

1) Demonstrate zadd (mainly according to the above text). The following commands are executed from top to bottom, but I have made comments.

# 1) If the specified added member is already a member in the ordered set, the score of the member will be updated and updated to the correct sorting position.
192.168.1.9:6379> ZADD z1 1 one
(integer) 1
192.168.1.9:6379> ZADD z1 2 one
(integer) 0
192.168.1.9:6379> ZSCORE z1 one
"2"
192.168.1.9:6379>

# 2) If the key does not exist, a new key will be created and the score/member pair will be added to the ordered set,
# 	Just like there was an empty ordered set. If the key exists but the type is not an ordered set, an error response will be returned.
#	The fractional value is a double precision floating-point number string+ Inf and - inf are valid values.
192.168.1.9:6379> EXISTS z2
(integer) 0
192.168.1.9:6379> ZADD z2 1 one 2 two
(integer) 2
192.168.1.9:6379> ZADD z2 hello three
(error) ERR value is not a valid float
192.168.1.9:6379>

# 3) The ZADD command supports some parameters (> = redis 3.0.2) in front of the score/member pair after the key. They are:
# 	3.1) XX: only update existing members without adding new members.
# As you can see below, z2 originally had one and two members, and then the following command wants to add three members, but because there is XX option, it will not be added.
192.168.1.9:6379> ZADD z2 XX 11 one 22 two 3 three
(integer) 0
192.168.1.9:6379> ZRANGE z2 0 -1
1) "one"
2) "two"
192.168.1.9:6379> 

# 	3.2) NX: do not update existing members. Add only new members.
# As you can see below, because of the NX option, the values of one and two have not changed, and only three members will be added.
# Because the above command sets the scores of one to 11 and the scores of two to 22 If you want to see more details, you can actually add the ZRANGE z2 0 -1 when measuring XX above to the with scores option to see the score.
192.168.1.9:6379> ZADD z2 NX 111 one 222 two 3 three
(integer) 1
192.168.1.9:6379> ZRANGE z2 0 -1 WITHSCORES
1) "three"
2) "3"
3) "one"
4) "11"
5) "two"
6) "22"
192.168.1.9:6379> 

# 	3.3) ch: modify the return value to the total number of changed members. The original value is to return the total number of newly added members (CH means changed).
#			The changed element is a newly added member or an existing member update score. So the members specified in the command have the same score
#			Will not be counted. Note: under normal circumstances, the return value of ZADD only calculates the number of newly added members.
# 	The idea of testing CH options is simple:
#	1. I just want to test the option without CH for comparison. Then, because there are four possibilities between scores and member. With member as the independent variable and scores as the dependent variable, there are:
# 	1) Both member and scores are the same. 2) Members are the same, but scores are different. 3) Members are different and scores are the same. 4) Member and scores are different.
#	2. Then add the CH option and test again in the way without CH option above.
# First, add two elements to the z3 set to indicate that there are already.
192.168.1.9:6379> ZADD z3  1 one 2 two
(integer) 2
# Take two members as an example, and z3 collect the existing elements for testing.
# 1) Both member and scores are the same. Because no new member is added, 0.0 is returned
192.168.1.9:6379> ZADD z3 2 two
(integer) 0
# 2) Members are the same, but scores are different. Because no new member is added, 0.0 is returned
192.168.1.9:6379> ZADD z3 22 two
(integer) 0
# 3) Members are different and scores are the same. Because a new member has been added, 1. 0 is returned
192.168.1.9:6379> ZADD z3 22 three
(integer) 1
# 4) Member and scores are different. Because a new member has been added, 1. 0 is returned
192.168.1.9:6379> ZADD z3 44 four
(integer) 1

# 2. Added CH option.
# First, add two elements to the z4 set to indicate that there are already.
192.168.1.9:6379> ZADD z4 CH 1 one 2 two
(integer) 2
# 1) Both member and scores are the same. Because no new members have been added or scores have changed, 0. 0 is returned
192.168.1.9:6379> ZADD z4 CH 2 two
(integer) 0
# 2) Members are the same, but scores are different. No new member s are added, but the scores are changed, so 1. 0 is returned
192.168.1.9:6379> ZADD z4 CH 22 two
(integer) 1
# 3) Members are different and scores are the same. Because a new member has been added, 1. 0 is returned
192.168.1.9:6379> ZADD z4 CH 22 three
(integer) 1
# 4) Member and scores are different. Because a new member has been added, 1. 0 is returned
192.168.1.9:6379> ZADD z4 CH 44 four
(integer) 1
192.168.1.9:6379> 
# Summary of CH options: through the above test of CH, in fact, we only need to consider the situation when the member s are the same in ch, which is different from the return value without ch option.
#	Because new members will be added when the members are different, it is the same as without CH option.


# 	3.4) INCR: when ZADD specifies this option, the member's operation is the same as the ZINCRBY command, which increments the member's score.
# It can be seen that after the INCR option is added, the operation of members is the same as the ZINCRBY command, and only the scores of members will be incremented. If the member does not exist, the initial value is 0
192.168.1.9:6379> ZADD z5 INCR 5 five
"5"
192.168.1.9:6379> ZADD z5 INCR 10 five
"15"
192.168.1.9:6379> 

# 4) Members in an ordered set cannot be repeated and are unique. However, different members may have the same score.
#	When multiple members have the same score, they will be ordered lexically (the score is still used as the first sorting condition, and then the members with the same score are sorted relatively according to the dictionary rules).
# As shown below, when the scores of multiple members are the same, they will be sorted according to the first capital letter of the members. If the first letter is the same, the next letter will be compared, and so on.
192.168.1.9:6379> ZADD z6 1 one 1 two 1 three
(integer) 3
192.168.1.9:6379> ZRANGE z6 0 -1 WITHSCORES
1) "one"
2) "1"
3) "three"
4) "1"
5) "two"
6) "1"

# 5) Return value Integer reply, including:
# 5.1) the number of members added to the ordered set, excluding members that already have update scores (i.e. when the CH parameter is not used).
Look at the top 3.3)No CH The return value of the option or in the command shown above, use ZADD The return value of the command.

# 5.2) if the INCR parameter is specified, the return will become bulk string reply. The new fraction (double precision floating-point number) string of the member.
Look at the top 3.4)INCR The test is enough.

# 6) Redis > = 2.4: accept multiple members. Before Redis 2.4, the command can only add or update one member.
Just remember, there is no need to test.

2) Demonstrate ZREM:

# 1) When the key exists, but it is not an ordered collection type, an error is returned.
# The string structure is treated as zset to use ZREM incorrectly.
192.168.1.9:6379> set tyy hello
OK
192.168.1.9:6379> type tyy
string
192.168.1.9:6379> ZREM tyy hello
(error) WRONGTYPE Operation against a key holding the wrong kind of value
192.168.1.9:6379> 
# Or the hash structure mistakenly uses ZREM as zset. Report the same error.
192.168.1.9:6379> HSET role:10001 name lqq
(integer) 1
192.168.1.9:6379> ZREM role:10001 name
(error) WRONGTYPE Operation against a key holding the wrong kind of value
192.168.1.9:6379> 

# 2) The return value is integer reply, which is the following integer:
# 	Returns the number of members deleted from the ordered collection, excluding nonexistent members.
192.168.1.9:6379> EXISTS z1
(integer) 0
192.168.1.9:6379> ZADD z1 1 one 2 two 3 three 4 four
(integer) 4
# As you can see below, six and ten do not exist, so they will be ignored. Finally delete the existing member: one two three.
192.168.1.9:6379> ZREM z1 one two three six ten
(integer) 3
192.168.1.9:6379> ZRANGE z1 0 -1 WITHSCORES
1) "four"
2) "4"

3) Demonstrate ZSCORE:

# 1) If the member element is not a member of the ordered set key, or the key does not exist, nil is returned.
192.168.1.9:6379> EXISTS z1
(integer) 0
192.168.1.9:6379> ZADD z1 1 one
(integer) 1
192.168.1.9:6379> ZSCORE z1 two
(nil)
192.168.1.9:6379> EXISTS z2
(integer) 0
192.168.1.9:6379> ZSCORE z2 one
(nil)
192.168.1.9:6379> 

# 2) Return value bulk string reply: returns the score value (double type floating-point number) of the member, expressed in string form.
192.168.1.9:6379> ZADD z3 1 one 2 two 3 three
(integer) 3
192.168.1.9:6379> ZSCORE z3 one
"1"
192.168.1.9:6379> ZSCORE z3 two
"2"
192.168.1.9:6379> ZSCORE z3 three
"3"

2.2 ZINCRBY,ZCARD

# Add increment to the score value of the member of the ordered set key.
# 1) If the key exists but there is no member, add a member in the key, and the score is increment (as if its previous score was 0.0).
# 2) If the key does not exist, an ordered collection containing only the specified member members is created. When the key is not an ordered set type, an error is returned.
# 3) The score value must be an integer value or double precision floating-point number represented by a string, and can accept floating-point numbers with double precision. You can also give a negative number to reduce the value of score. 
#	In fact, the first half of the sentence does not need to be demonstrated, because the score of the members of the ordered set must be of integer or double type, otherwise an error will be reported in ZADD. ZINCRBY a nonexistent member,
#	The default score of this member is 0.0, so the first half sentence can be ignored. Therefore, you only need to show a negative number to reduce the value of score. 
# 4) Return value bulk string reply: the new score value of member, expressed in string form.
ZINCRBY key increment member 

# Returns the number of ordered set elements of the key.
# Return value integer reply: returns the number of elements of the ordered set when the key exists; Otherwise, it returns 0.
ZCARD key

1) Demonstrate ZINCRBY:

# 1) If the key exists but there is no member, add a member in the key, and the score is increment (as if its previous score was 0.0).
192.168.1.9:6379> EXISTS  z1
(integer) 0
192.168.1.9:6379> ZADD z1 1 one
(integer) 1
192.168.1.9:6379> ZRANGE z1 0 -1 withscores
1) "one"
2) "1"
192.168.1.9:6379> ZINCRBY z1 5 two
"5"
192.168.1.9:6379> ZRANGE z1 0 -1 withscores
1) "one"
2) "1"
3) "two"
4) "5"

# 2) If the key does not exist, an ordered collection containing only the specified member members is created. When the key is not an ordered set type, an error is returned.
192.168.1.9:6379> EXISTS z2
(integer) 0
192.168.1.9:6379> ZINCRBY z2 1 one
"1"
192.168.1.9:6379> ZRANGE z2 0 -1 withscores
1) "one"
2) "1"
192.168.1.9:6379> set hello world
OK
192.168.1.9:6379> type hello
string
192.168.1.9:6379> ZINCRBY hello 1 one
(error) WRONGTYPE Operation against a key holding the wrong kind of value
192.168.1.9:6379> 

# 3) The score value must be an integer value or double precision floating-point number represented by a string, and can accept floating-point numbers with double precision. You can also give a negative number to reduce the value of score. 
#	In fact, the first half of the sentence does not need to be demonstrated, because the score of the members of the ordered set must be of integer or double type, otherwise an error will be reported in ZADD. ZINCRBY a nonexistent member,
#	The default score of this member is 0.0, so the first half sentence can be ignored. Therefore, you only need to show a negative number to reduce the value of score. 
192.168.1.9:6379> ZRANGE z2 0 -1 withscores
1) "one"
2) "1"
192.168.1.9:6379> ZINCRBY z2 -11 one
"-10"
192.168.1.9:6379> ZRANGE z2 0 -1 withscores
1) "one"
2) "-10"
192.168.1.9:6379> 

# 4) Return value bulk string reply: the new score value of member, expressed in string form.
Just look at the return value above.

2) Demo ZCARD:

# Return value integer reply: returns the number of elements of the ordered set when the key exists; Otherwise, it returns 0.
192.168.1.9:6379> EXISTS z3
(integer) 0
192.168.1.9:6379> ZADD z3 1 one 2 two
(integer) 2
192.168.1.9:6379> ZRANGE z3 0 -1 withscores
1) "one"
2) "1"
3) "two"
4) "2"
192.168.1.9:6379> ZCARD z3
(integer) 2
192.168.1.9:6379> ZADD z3 3 three
(integer) 1
192.168.1.9:6379> ZRANGE z3 0 -1 withscores
1) "one"
2) "1"
3) "two"
4) "2"
5) "three"
6) "3"
192.168.1.9:6379> ZCARD z3
(integer) 3
192.168.1.9:6379> 

# When key doesn't exist.
192.168.1.9:6379> EXISTS z4
(integer) 0
192.168.1.9:6379> ZCARD z4
(integer) 0

2.3 ZRANK,ZRANGE,ZRANGEBYSCORE

# Returns the ranking of member s in the ordered set key.
# 1) Returns the ranking of members in the ordered set key. The members of the ordered set are arranged in the order of increasing score value (from small to large). The ranking is based on 0, that is, the member with the lowest score is ranked as 0.
#	Using the ZREVRANK command, you can get the ranking of members in descending order (from large to small) of the score value.
# 2) Return value:
# 	2.1) if member is a member of the ordered set key, return integer reply: the ranking of member.
# 	2.2) if the member is not a member of the ordered set key, return bulk string reply: nil.
ZRANK key member 

# Returns the element order by ID limit 1100 of the specified range stored in the ordered set key.
# 1) Returns the specified range of elements stored in the ordered set key. The returned elements can be considered to be ranked from the lowest to the highest score. If the scores are the same, they will be sorted by dictionary.
# 	When you need to rank elements from the highest score to the lowest score, please refer to zrevrank (the same scores will be sorted in reverse order of the dictionary).
# 2) The parameters start and stop are zero based indexes, that is, 0 is the first element, 1 is the second element, and so on. 
#	They can also be negative numbers, indicating the offset from the end of the ordered set, where - 1 is the last element of the ordered set, - 2 is the penultimate element, and so on.
# 3) start and stop are all inclusive intervals, so for example, ZRANGE myzset 0 1 will return the first and second elements of the ordered set.
# 4) Out of range indexes do not produce errors. If the value of the start parameter is greater than the maximum index in the ordered set, or start > stop, an empty list will be returned. 
# 5) If the value of stop is greater than the end of the ordered set, Redis will treat it as the last element of the ordered set.
# 6) You can pass the with scores option to return the element's score with the element. In this way, the returned list will contain value1, Score1, Valuen, scoren, not value1, valueN.  
#	Client class libraries are free to return more appropriate data types (recommended: arrays or records with values and scores).
# 7) Return value array reply: returns the list of elements within the given range (if the with scores option is specified, their scores will be returned at the same time).
ZRANGE key start stop [WITHSCORES]

# 1) Returns all elements in the ordered set key with scores between min and max (including min and max); Elements are considered to be sorted from low to high.
#	limit specifies how many elements to return from the first few.
# 2) The optional LIMIT parameter specifies the number and interval of the returned results (similar to SELECT LIMIT offset, count in SQL).
#	Note that if the offset is too large (because it is too large, you need for to traverse to this position), locating the offset may traverse the entire ordered set, which will increase the complexity of O(N).
# 3) The optional parameter WITHSCORES returns the element and its score, not just the element. This option is available in redis2 Versions after 0 are available.
# 4) min and max can be - inf and + inf, so you can use commands like ZRANGEBYSCORE without knowing the lowest and highest score values of the ordered set.
# 5) By default, the value of the interval uses the closed interval (less than or equal to or greater than or equal to). You can also use the optional open interval (less than or greater than) by adding (symbol) before the parameter.
# 	For example: "zrangebyscore Zset (1,5)". 	 It represents the return of all members that meet the criteria of 1 < score < = 5.
#	"ZRANGEBYSCORE zset (5 (10".  		 It represents the return of all members that meet the criteria of 5 < score < 10.
# 6) Return value array reply: a list of elements with a specified score range (their scores can also be returned).
ZRANGEBYSCORE key min max [WITHSCORES] [LIMIT offset count]

1) Demonstrate ZRANK:

# 1) Returns the ranking of members in the ordered set key. The members of the ordered set are arranged in the order of increasing score value (from small to large). The ranking is based on 0, that is, the member with the lowest score is ranked as 0.
192.168.1.9:6379> ZADD z1 1 one 2 two 3 three
(integer) 3
192.168.1.9:6379> ZRANGE z1 0 -1 withscores
1) "one"
2) "1"
3) "two"
4) "2"
5) "three"
6) "3"
192.168.1.9:6379> ZRANK z1 one
(integer) 0
192.168.1.9:6379> ZRANK z1 two
(integer) 1
192.168.1.9:6379> ZRANK z1 three
(integer) 2

# 2) Return value:
# 	2.1) if member is a member of the ordered set key, return integer reply: the ranking of member.
Just look at the return value above.

# 	2.2) if the member is not a member of the ordered set key, return bulk string reply: nil.
192.168.1.9:6379> ZRANK z1 five
(nil)
192.168.1.9:6379>

2) Demonstrate ZRANGE:

# 1) Returns the specified range of elements stored in the ordered set key. The returned elements can be considered to be ranked from the lowest to the highest score. If the scores are the same, they will be sorted by dictionary.
# 	When you need to rank elements from the highest score to the lowest score, please refer to zrevrank (the same scores will be sorted in reverse order of the dictionary).
192.168.1.9:6379> ZRANGE z1 0 -1 withscores
1) "one"
2) "1"
3) "two"
4) "2"
5) "three"
6) "3"
192.168.1.9:6379> ZADD z1 2 b
(integer) 1
# As you can see below, the score s are all 2. Because the initial letter b is ahead of t, member b is ranked first.
192.168.1.9:6379> ZRANGE z1 0 -1 withscores
1) "one"
2) "1"
3) "b"
4) "2"
5) "two"
6) "2"
7) "three"
8) "3"
192.168.1.9:6379> 

# 2) The parameters start and stop are zero based indexes, that is, 0 is the first element, 1 is the second element, and so on. 
#	They can also be negative numbers, indicating the offset from the end of the ordered set, where - 1 is the last element of the ordered set, - 2 is the penultimate element, and so on.
For example: ZRANGE z1 0 -1 withscores

# 3) start and stop are all inclusive intervals, so for example, ZRANGE myzset 0 1 will return the first and second elements of the ordered set.
192.168.1.9:6379> ZRANGE z1 0 1 withscores
1) "one"
2) "1"
3) "b"
4) "2"
192.168.1.9:6379> 

# 4) Out of range indexes do not produce errors. If the value of the start parameter is greater than the maximum index in the ordered set, or start > stop, an empty list will be returned. 
192.168.1.9:6379> ZRANGE z1 2 1 withscores
(empty array)
192.168.1.9:6379>

# 5) If the value of stop is greater than the end of the ordered set, Redis will treat it as the last element of the ordered set.
192.168.1.9:6379> ZRANGE z1 1 10 withscores
1) "b"
2) "2"
3) "two"
4) "2"
5) "three"
6) "3"

# 6) You can pass the with scores option to return the element's score with the element. In this way, the returned list will contain value1, Score1, Valuen, scoren, not value1, valueN.  
#	Client class libraries are free to return more appropriate data types (recommended: arrays or records with values and scores).
Look at the tape above withscores The result of the option is OK.

# 7) Return value array reply: returns the list of elements within the given range (if the with scores option is specified, their scores will be returned at the same time).
Look up ZRANGE The result of command execution.

3) Demonstrate ZRANGEBYSCORE:

# 1) Returns all elements in the ordered set key with scores between min and max (including min and max); Elements are considered to be sorted from low to high.
#	limit specifies how many elements to return from the first few.
Just look at the execution results below.

# 2) The optional LIMIT parameter specifies the number and interval of the returned results (similar to SELECT LIMIT offset, count in SQL).
#	Note that if the offset is too large (because it is too large, you need for to traverse to this position), locating the offset may traverse the entire ordered set, which will increase the complexity of O(N).
# Note that offset starts with the 0 subscript. If you add the LIMIT option to further filter according to the returned results, there are four possibilities:
# Take offset as the independent variable and count as the dependent variable:
	# 1) In the returned result set, neither offset nor count exceeds the number of elements in the result set. Return normally according to the LIMIT.
	# 2) In the returned result set, offset does not exceed the number of elements in the result set, but count exceeds. Returns all the remaining elements of the result set starting from offset.
	# 3) In the returned result set, offset exceeds the number of elements in the result set, but count does not exceed. Returns an empty array.
	# 4) In the returned result set, offset exceeds the number of elements in the result set and count exceeds. Returns an empty array.
# Add elements to the ordered set first.
192.168.1.9:6379> ZADD z1 1 one 2 two 10 ten 60 sixty 100 hundred
(integer) 5
# The first possibility of test 2). When LIMIT is not added, because there is one two ten in socers in the range of 1-10, but LIMIT 0 2 is added at this time, the two elements of one two will be returned.
192.168.1.9:6379> ZRANGEBYSCORE z1 1 10 LIMIT 0 2 
1) "one"
2) "two"
192.168.1.9:6379>

# The second possibility of test 2). Because socers have one two ten in the range of 1-10, but LIMIT 1 3 is added at this time, that is, three elements are returned from two, but because it is not enough, all the remaining elements from offset=1 are returned.
192.168.1.9:6379> ZRANGEBYSCORE z1 1 10 LIMIT 1 3
1) "two"
2) "ten"
192.168.1.9:6379>

# The third possibility of test 2). Because socers has one two ten in the range of 1-10, but limit 32 is added at this time, and the offset is directly greater than the subscript 2 of the maximum number of elements in the result set, so count has no reference value at this time, so an empty array is returned.
192.168.1.9:6379> ZRANGEBYSCORE z1 1 10 LIMIT 3 2
(empty array)
192.168.1.9:6379>

# The fourth possibility of test 2). Because socers have one two ten in the range of 1-10, but limit 3.5 is added at this time, and the offset is directly greater than the subscript 2 of the maximum number of elements in the result set, so count has no reference value at this time, so it also returns an empty array.
192.168.1.9:6379> ZRANGEBYSCORE z1 1 10 LIMIT 3 5
(empty array)
192.168.1.9:6379>

# 3) The optional parameter WITHSCORES returns the element and its score, not just the element. This option is available in redis2 Versions after 0 are available.
192.168.1.9:6379> ZRANGEBYSCORE z1 1 10 WITHSCORES
1) "one"
2) "1"
3) "two"
4) "2"
5) "ten"
6) "10"
192.168.1.9:6379> ZRANGEBYSCORE z1 1 100 WITHSCORES
 1) "one"
 2) "1"
 3) "two"
 4) "2"
 5) "ten"
 6) "10"
 7) "sixty"
 8) "60"
 9) "hundred"
10) "100"
192.168.1.9:6379> 

# 4) min and max can be - inf and + inf, so you can use commands like ZRANGEBYSCORE without knowing the lowest and highest score values of the ordered set.
192.168.1.9:6379> ZRANGEBYSCORE z1 -inf +inf
1) "one"
2) "two"
3) "ten"
4) "sixty"
5) "hundred"
192.168.1.9:6379> 

# 5) By default, the value of the interval uses the closed interval (less than or equal to or greater than or equal to). You can also use the optional open interval (less than or greater than) by adding (symbol) before the parameter.
# 	For example: "zrangebyscore Zset (1,5)". 	 It represents the return of all members that meet the criteria of 1 < score < = 5.
#	"ZRANGEBYSCORE zset (5 (10".  		 It represents the return of all members that meet the criteria of 5 < score < 10.
192.168.1.9:6379> ZRANGEBYSCORE z1 1 10
1) "one"
2) "two"
3) "ten"
192.168.1.9:6379> ZRANGEBYSCORE z1 (1 10
1) "two"
2) "ten"
192.168.1.9:6379> ZRANGEBYSCORE z1 (1 10)
(error) ERR min or max is not a float
192.168.1.9:6379> ZRANGEBYSCORE z1 (1 (10
1) "two"
192.168.1.9:6379> 

# 6) Return value array reply: a list of elements with a specified score range (their scores can also be returned).
Just look at the results of the above implementation.

Now let's summarize some key subscripts in the above basic command design. In fact, only ZRANK, ZRANGE and ZRANGEBYSCORE are involved.

1) It should be noted that the return ranking of ZRANK starts from 0. For example, look at the first example of ZRANK shown above.
2) ZRANGE should note that the subscript of start stop starts from 0.
3) ZRANGEBYSCORE should note that min and max are specified as intervals, not subscripts. For example, 0 and 100 represent members with scores within [0, 100]. 1, 100 represents the member whose score range is within [1, 100], which is only related to the score of the member of the ordered set and has nothing to do with the subscript of the ordered set. Note that the value of offset starts from the subscript 0, and the value of this offset represents the offset position of the result set obtained by the operation without LIMIT.

2.4 ZREMRANGEBYSCORE

# Remove all members in the ordered set key whose score value is between min and max (including those equal to min or max). 
#	Since version 2.1.6, members whose score value is equal to min or max can also be excluded. For the syntax, see the ZRANGEBYSCORE command.
# 1) Return value integer reply: the number of elements deleted.
ZREMRANGEBYSCORE key min max

1) Demonstrate ZREMRANGEBYSCORE:

192.168.1.9:6379> ZADD z1 1 one 2 two 3 three 4 four 5 five
(integer) 5
192.168.1.9:6379> ZREMRANGEBYSCORE z1 1 3
(integer) 3
192.168.1.9:6379> ZRANGE z1 0 -1
1) "four"
2) "five"
192.168.1.9:6379> ZREMRANGEBYSCORE z1 4 (5
(integer) 1
192.168.1.9:6379> ZRANGE z1 0 -1
1) "five"
192.168.1.9:6379>

2.5 sorting rules of ordered sets

Note that ordered sets are compared regularly. First, the sorting is determined by comparing scores. If the scores are the same, membe is compared.
The member comparison rule is to compare in alphabetical order. The above command introduction has also been mentioned many times.

3 storage structure

zset is implemented using a jump table, and the bottom layer of the jump table is actually implemented using a linked list (internal order). Therefore, when the number of nodes is small (128) and the string length is small (64), it will use a more efficient linked list, that is, compressed list storage, to provide efficiency.
But in the final analysis, they all use linked lists for storage.
This is similar to the list in the first section of redis.

However, during the interview, the interviewer asks you what structure is used to realize the bottom of zset. You should say jump list instead of linked list. Only by asking you what data structure is used to realize jump list, you won't have too much problem with the linked list structure at this time.

4 Application

4.1 Baidu hot list

I believe you will have read the hot list information of various applications, such as jitter, micro-blog, Baidu tiktok. So how did they achieve it?

It's not hard. For example, in the baidu hot list above, it is obvious that the number of tens of thousands on the right is a basis for sorting, so it represents the score option of zset. And the left side is the member member of zset. We will first use id to represent this member, for example, 10001 represents "No. 1 central document speed reading", 10002 represents "China supports Russia and Ukraine to solve the problem through negotiations". In this way, you can use zset to rank the hot search list.
Although the above member is id, how to save the theme text and how to display the content after clicking on it is obvious that an additional hash structure is needed to store these contents. The key of hash is id. For example, hash:id, which can be hot: 20226:10001.
Then the hash structure of the hot search of this id can store its own id, text topic, click to display the url of the hot search and other hash fields.

Through the above analysis, a hot search list can be realized.

# Hot: 20226: hot stands for hot search. 20226 represents the hot search on February 26, 22.

# 1. Suppose there are 10 hot search lists, and the first reading is 1. Users change the score by clicking on the news times, so as to change the order of hot search lists.
192.168.1.9:6379> zincrby hot:20220226 1 10001
"1"
192.168.1.9:6379> zincrby hot:20220226 1 10002
"1"
192.168.1.9:6379> zincrby hot:20220226 1 10003
"1"
192.168.1.9:6379> zincrby hot:20220226 1 10004
"1"
192.168.1.9:6379> zincrby hot:20220226 1 10005
"1"
192.168.1.9:6379> zincrby hot:20220226 1 10006
"1"
192.168.1.9:6379> zincrby hot:20220226 1 10007
"1"
192.168.1.9:6379> zincrby hot:20220226 1 10008
"1"
192.168.1.9:6379> zincrby hot:20220226 1 10009
"1"
192.168.1.9:6379> zincrby hot:20220226 1 10010
"1"
192.168.1.9:6379> 
# 2. Then change the score through the user's click news behavior, so as to change the hot search list.
# Suppose we want to increase the number of clicks by 30 for 10001, 20 for 10005 and 10 for 10010. Others remain unchanged.
192.168.1.9:6379> zincrby hot:20220226 10 10010
"11"
192.168.1.9:6379> zincrby hot:20220226 20 10005
"21"
192.168.1.9:6379> zincrby hot:20220226 30 10001
"30"
192.168.1.9:6379>

# 3. Get the top 10 rankings.
# Because zrang returns from small to large by default, that is, the number of hits from small to large, our hot search list needs to return the number of hits from large to small, which is the hot search list.
# As we can see below, 10001 has the most hits, so it ranks first, followed by 10005 and 10010. The rest can be ignored because I give the default values. They will sort according to the letter of member when the score is the same, and then reverse it according to zrevrange, and finally display the following results.
192.168.1.9:6379> zrevrange hot:20220226 0 9 withscores
 1) "10001"
 2) "31"
 3) "10005"
 4) "21"
 5) "10010"
 6) "11"
 7) "10009"
 8) "1"
 9) "10008"
10) "1"
11) "10007"
12) "1"
13) "10006"
14) "1"
15) "10004"
16) "1"
17) "10003"
18) "1"
19) "10002"
20) "1"
192.168.1.9:6379>
# You can actually use the ZRANG command, but you need to add the REV option. The result is the same as zrevrange.
192.168.1.9:6379> zrange hot:20220226 0 9 withscores rev
 1) "10001"
 2) "31"
 3) "10005"
 4) "21"
 5) "10010"
 6) "11"
 7) "10009"
 8) "1"
 9) "10008"
10) "1"
11) "10007"
12) "1"
13) "10006"
14) "1"
15) "10004"
16) "1"
17) "10003"
18) "1"
19) "10002"
20) "1"
192.168.1.9:6379>

4.2 delay queue

1) Delay queue: suppose there is a delay task to be processed in a back-end process, which needs to notify other distributed nodes to complete a certain work, then the back-end process needs to submit delay tasks to redis, and these delay tasks are uniformly handed over to a check service to be distributed to other distributed nodes for processing.
2) Processing of general zset structure: sequence messages into a string as the member of zset. The expiration processing time of this message is regarded as a score, and then multiple threads poll zset to obtain the expired tasks for processing.

For example, the following example. Explain the following figure:

S1, S2, S3 and S4 are our four back-end services. They are all connected to the same redis service. Then there is a check service that also connects with the redis service.
Suppose S1 has a delayed task submitted to redis (for this task, we can't simply add a timer to S1 for processing, and there will be delay tasks in S2, S3 and S4. If we simply implement them with a timer, it certainly won't work, because we require that this task needs a specific distributed feature, which is not available if we use a timer. Therefore, we need to submit the task to redis and a Che CK service, which schedules the delayed task at the same time, needs to notify S2, S3 and S4 to process the task, so an additional check service area needs to be transferred to these delayed tasks to realize unified processing.

So how to implement delay queue? See the pseudo code explanation below.

# 1. delay thread. The thread that submits the deferred task. It may be S1, S2, S3, S4.
# delay:1 represents the function and 1 represents the unique id, so delay:1 can be considered as a delay queue.
# socre is represented by timestamp, and now+5 means that the task is delayed for 5 seconds.
# task1: it may be a json string, which may record in detail how to operate the task. For example, this task needs to be broadcast to S2, S3 and S4. Or it can only be forwarded to a certain node, such as S2 for processing.
zadd delay:1 now+5 task1
zadd delay:1 now+10 task2

# 2. check thread. Threads that uniformly handle delayed tasks.
# Note: the interval [0,now] below can be used to execute the delayed task at that time.
# For example:
# 1) Add a task above. now=0s, and the detection interval of the check service below is [0,0], so this task will not be returned.
# 2) After 5s, now=5s, and the detection interval of the following check service is [0,5], so it just returns this delayed task, and then it is deleted from the delayed queue and executed. Task 2 of now+10 is the same.
# 3) It is emphasized here that even if there is a delay queue with the same timestamp, that is, the score is the same, it can be executed, just in the order of execution.
# For example, there are two delayed tasks t1, t2 and score that are executed after now+5, that is, 5s. Suppose t1 executes first and the execution time takes 1s, then
# The next time you traverse zrangebyscore, the interval is [0,6] seconds. t2 can also be returned, because t1 is deleted from the delay queue before execution. Therefore, t2 can also realize delayed tasks,
# It's only delayed for one second, because it was originally executed after 5s. Because the time-consuming of t1 becomes executed after 6s, the time-consuming duration of this task can be limited according to the specific scenario.
for {
# vals contains at most one member, because the limit is 0 1. The returned result set does not contain score.
vals := zrangebyscore delay:1 0 now limit 0 1
# Get the task at that time.
val := vals[0]
# Delete the task.
zrem delay:1 val
# Perform delayed tasks.
handle(val)
# }

The above is the pseudo code explanation of delay queue (or delay task).

4.3 distributed timer

The above distributed delay task queue is only for relatively simple development scenarios. When some development scenarios are complex and extensive, we need to introduce our distributed timer.
Let's briefly introduce the design principle of distributed timer.

1) The producer hash es the scheduled tasks to different redis entities and assigns a dispatcher to each redis entity
Process, which is used to regularly obtain timeout events in redis and publish them to different consumers.
2) It is assumed that after the producer submits the task, there is only one redis service and check service according to the processing of the delay queue. If both of them are down at this time, it obviously does not meet the reliability in the distributed system, so it needs a distributed timer to process it. That is, after the producer generates a task, it is submitted to the redis cluster according to the appropriate situation, and then the corresponding dispatcher takes the task from its corresponding redis service and assigns it to consumers for consumption.
Distributed reliability should refer to P in the principle of distributed CAP, that is, partition fault tolerance (because I don't know much about distributed). The principle of distributed CAP can be referred to Distributed CAP theorem, why can't it meet three characteristics at the same time?.

4.4 time window current limiting

Before talking about time window current limiting, let's first understand the concepts of current limiting and fusing.

1) Current limiting: the window is moving. For example, if a behavior is limited to 10 times in 5 seconds, it is calculated as follows: suppose that the beginning is 1-6 seconds, then after 1 second, it is 2-7 seconds, and after 1 second, it is 3-8 seconds. That is, the behavior of current limiting statistics here may be repeated, and the window is moving.
For example, in 1-6 seconds, it occurs once in the first second, three times in the second second to sixth seconds, and a total of 1 + 3 = 4 times in 1-6 seconds.
Then, in the statistics of 2-7 seconds, since the second second to sixth seconds of 1-6 seconds represent the first six seconds of 2-7 seconds, the number of occurrences in the first six seconds of 2-7 seconds is 3. Assuming that it occurs twice in the seventh second, there will be a total of 3 + 2 = 5 occurrences in 2-7 seconds.
Similarly, the number of occurrences in the 3rd-7th seconds of 3-8 seconds is the number of occurrences in 1-6 (i.e. 3-6 seconds) and 2-7 (i.e. 3-7 seconds) (because the specific number of seconds is not assumed here, it is impossible to calculate the specific number of occurrences), and then add the number of occurrences in the 8th second, which is the total number of occurrences in 3-8 seconds.
2) Fuse: the window is fixed. Similarly, for example, if a certain behavior occurs 10 times in 5 seconds, which is easier to understand than the above, then it is calculated as follows: assuming that the beginning is 1-6 seconds, if a certain behavior does not occur 10 times in 5 seconds, it will be cleared; Then, count whether a certain behavior has occurred 10 times within 5 seconds in 7-12 seconds, and do not continue to clear. and so on. That is, the number of previous behaviors will not affect the next statistics, and the window is fixed.
3) Concept of time window current limiting: the system limits that a user's behavior can only occur N times within the specified time range (dynamic).

# The current limiting methods include:
# Method 1: use zset.
# 1. Analysis. The above concept has four key points: specify user_ A behavior action of ID can only occur for the number of times Max during the period of a specific time_ count. user_id,action,period,max_count.
# Key limit: 10001: the meaning of action1: limit represents the function of current limiting. 10001 represents the unique id of the user. Action1 represents a specific behavior of the user.

# 2. Use zset to record the number of actions of a user. Both score and member use now (the purpose of this will be explained below).
zadd limit:10001:action1 now now

# 3. Delete the number of behaviors that are not within the period from the zset set, so that the number of elements remaining in the zset set represents the number of behaviors that occur within the period.
# Because the time unit of now is milliseconds and the incoming period is seconds, the period multiplied by 1000 is converted into milliseconds.
zremrangebyscore limit:10001:action1 0 now - period*1000

# 4. Get the number of times of a user's behavior in the period time.
count = zcard limit:10001:action1
# expire limit:10001:action1 60+1

# 5. After obtaining the number of times, you can compare whether the current limit is exceeded and finally judge whether the user is allowed to do so.
# Compare count and max_count . If count > max_count indicates the number of times exceeded; Otherwise, the limited number of times is not exceeded.

For example, explain the above example.
1)Suppose the system is set period=2s，max_count=5 Times. Re hypothesis zadd When, now=5s，After 1 s After, zremrangebyscore Start execution, then now=5+1=6s，
therefore zremrangebyscore The truncated interval is:[0, 4]，0 was eventually deleted-4s The number of behaviors, so that the remaining elements in the ordered set are the nearest period The number of times within, i.e. the 5th and 6th seconds. This also explains why zadd When, score and member All use now.
Similarly, go through 1 s，Interval is[0,5]，The remaining are the 6th and 7th seconds. Pass like this zcard After, you can get period Number of behaviors in the. Then with max_count Compare.
2)And see, use zset In this way, its window is moving.

# Use method 2: expire + string.
val = incr limit:10001:action1
val = incr limit:10001:action1
expire limit:10001:action1 60+1

For example, explain the above example.
1)Specifically https://blog.csdn.net/liyunlong41/article/details/89854808.
About this article incr and expire It is not an atomic operation method. It is recommended to use it lua Script, he said ttl After all, the method is not 100% safe.
2)this string + expire The method is fusing(Window does not move)Processing or current limiting(window moving)At present, I am not very clear.
Look at this later when you have time string + expire Implementation of. After all, only understanding the principle is not enough to be used flexibly in practical development.
Namely string+expire For example, I have two more questions:
1)string+expire The specific execution order of the command call.
2)this string + expire The method is fusing(Window does not move)Processing or current limiting(window moving)Treatment of.

Keywords: Database Redis

Added by svenski on Sun, 27 Feb 2022 07:16:40 +0200

Programming VIP