Collection object
set types are also used to hold multiple string elements
Unlike list types, duplicate elements are not allowed in a collection, and elements in a collection are out of order and cannot be retrieved through an index subscript
A collection can store up to 232-1 elements
Redis not only supports addition, deletion, and censoring within a set, but also supports intersection, Union and difference of multiple sets, and makes good use of set types.
The encoding of a collection object can be intset or hashtable
Reading Reference: Redis Data Structure (5) - Integer Set
The intset-encoded collection object uses an integer collection as the underlying implementation, and all elements contained in the collection object are stored in the integer collection
For example, the following code creates an intset-encoded collection object
redis> SADD numbers 1 3 5 (integer) 3
On the other hand, hashtable-encoded collection objects use a dictionary as the underlying implementation, where each key is a string object, each string object contains a collection element, and the values of the dictionary are all set to NULL
For example, the following code creates a hashtable-encoded collection object
Encoding Conversion of Collection Objects
When a collection object can satisfy both of the following conditions, it uses intset encoding
- All elements saved by a collection object are integer values
- Collection objects hold no more than 512 elements
Collection objects that do not meet these two conditions need hashtable encoding
Be careful:
The upper limit of the second condition can be modified, see the description of the set-max-intset-entries option in the configuration file
For collection objects that use intset encoding, when either of the two conditions required for intset encoding cannot be met, the object's encoding conversion operation is performed, all elements that were originally stored in the integer collection are transferred and saved in the dictionary, and the object's encoding changes from intset to hashtable
For example, the following code creates a collection object that contains only integer elements and is coded as intset
redis> SADD numbers 1 3 5 (integer) 3 redis> OBJECT ENCODING numbers "intset"
However, as long as a string element is added to this collection object that contains only integer elements, the encoding transfer of the collection object will be performed
redis> SADD numbers "seven" integer) 1 redis> OBJECT ENCODING numbers "hashtable"
In addition, if you create a collection object with 512 integer elements, the object's encoding should be intset
redis> EVAL "for i=1, 512 do redis.call('SADD', KEYS[1], i) end" 1 integers
(nil)
redis> SCARD integers
(integer) 512
redis> OBJECT ENCODING integers
"intset"
However, as long as a new integer element is added to the set, making the number of elements in the set 513, the object's encoding conversion operation will be performed
redis> SADD integers 10086 (integer) 1 redis> SCARD integers (integer) 513 redis> OBJECT ENCODING integers "hashtable"
Commands for Collection Objects
Operation within collection
1. Add Element-sadd
sadd key element [element ...]
Returns the number of elements successfully added, such as
127.0.0.1:6379> exists myset (integer) 0 127.0.0.1:6379> sadd myset a b c (integer) 3 127.0.0.1:6379> sadd myset a b (integer) 0
2. Delete element-srem
srem key element [element ...]
Returns the number of elements successfully deleted, for example
127.0.0.1:6379> srem myset a b (integer) 2 127.0.0.1:6379> srem myset hello (integer) 0
3. Calculate Number of Elements - scard
scard key
The time complexity of scard is O(1), which does not traverse all elements of the collection, but uses variables directly inside Redis
127.0.0.1:6379> scard myset (integer) 1
4. Determine if an element is in the set-sismember
sismember key element
If a given element element element element returns 1 within a collection and vice versa returns 0, for example
127.0.0.1:6379> sismember myset c (integer) 1
5. Randomly returns a specified number of elements from a set - srandmember
srandmember key [count]
[count] is an optional parameter if it is not written to default to 1, for example
127.0.0.1:6379> srandmember myset 2 1) "a" 2) "c" 127.0.0.1:6379> srandmember myset "d"
6. Randomly eject element-spop from set
spop key
A spop operation can randomly pop up an element from a collection, for example, the following code is a spop and the collection element becomes "d b a"
127.0.0.1:6379> spop myset "c" 127.0.0.1:6379> smembers myset 1) "d" 2) "b" 3) "a"
It is important to note that Redis started at version 3.2 and spop also supports the [count] parameter
Both srandmember and spop randomly select elements from a set, unlike srandmember, which deletes elements from the set after the spop command executes
7. Get all elements - smembers
smembers key
The following code takes all elements of the collection myset and returns the result out of order
127.0.0.1:6379> smembers myset 1) "d" 2) "b" 3) "a"
smembers and lrange, hgetall are heavier commands, which can be accomplished using sscan if there is a possibility of blocking Redis with too many elements
Inter-Collection Operations
There are now two collections, user:1:follow and user:2:follow.
127.0.0.1:6379> sadd user:1:follow it music his sports (integer) 4 127.0.0.1:6379> sadd user:2:follow it news ent sports (integer) 4
1. Find the intersection of multiple sets-sinter
sinter key [key ...]
For example, the following code finds the intersection of two sets of user:1:follow and user:2:follow and returns the result sports, it
127.0.0.1:6379> sinter user:1:follow user:2:follow 1) "sports" 2) "it"
2. Seek the union of multiple sets-suinon
suinon key [key ...]
For example, the following code finds the union of two sets of user:1:follow and user:2:follow and returns the result sports, it, his, news, music, ent
127.0.0.1:6379> sunion user:1:follow user:2:follow 1) "sports" 2) "it" 3) "his" 4) "news" 5) "music" 6) "ent"
3. Find the difference of multiple sets-sdiff
sdiff key [key ...]
For example, the following code finds the difference between two sets of user:1:follow and user:2:follow and returns music and his
127.0.0.1:6379> sdiff user:1:follow user:2:follow 1) "music" 2) "his"
4. Save the results of intersection, Union and difference sets
sinterstore destination key [key ...] suionstore destination key [key ...] sdiffstore destination key [key ...]
Operations between sets can be time consuming when there are many elements, so Redis provides the three commands above (the original command + store) to save the results of intersections, unions, and differences between sets in destination key
For example, the following saves the result of the intersection of two sets of user:1:follow and user:2:follow in user:1_2:inter, and user:1_2:inter itself is a collection type
127.0.0.1:6379> sinterstore user:1_2:inter user:1:follow user:2:follow (integer) 2 127.0.0.1:6379> type user:1_2:inter set 127.0.0.1:6379> smembers user:1_2:inter 1) "it" 2) "sports"
Scenarios for collecting objects (Important!!!)
Typical usage scenarios for collection types are tag s
For example, one user may be more interested in entertainment, sports, and another in history, news. These are the tags.
With this data, you can get tags that people who like the same tag and users'common preferences. This data is important for the user experience and for enhancing user stickiness.
For example, an e-commerce website will make different types of recommendations for users with different labels, such as people who are interested in digital products, recommending the latest digital products to them on various pages or by email, which usually brings more benefits to the website.
Here are some of the tag functions implemented using collection types
(1) Label users
sadd user:1:tags tag1 tag2 tag5 sadd user:2:tags tag2 tag3 tag5 ... sadd user:k:tags tag1 tag2 tag4 ...
(2) Add users to labels
sadd tag1:users user:1 user:3 sadd tag2:users user:1 user:2 user:3 ... sadd tagk:users user:1 user:2 ...
(3) Delete labels under users
srem user:1:tags tag1 tag5 ...
(4) Delete users under Tags
srem tag1:users user:1 srem tag5:users user:1 ...
(3) Calculate tags of common interest to users
You can use the sinter command to calculate tags of common interest to users, as shown in the code below
sinter user:1:tags user:2:tags
summary
Collection types are typically used in the following scenarios
- sadd=Tagging (label)
- spop/srandmember=Random item
- sadd+sinter=Social Graph
Ordered Collection Object
Ordered collections are a little unfamiliar to hashes, lists, and collections, but since they are called ordered collections, they must be related to collections. They retain the feature that collections cannot have duplicate members, but the elements in ordered collections can be ordered.
But unlike lists that use index subscripts as their sorting criteria, it sets a score for each element to be sorted by
Elements in an ordered set cannot be duplicated, but score can
Similarities and differences among list, set, ordered set
The encoding of an ordered set can be ziplist or skiplist
Reading Reference: Redis Data Structure (6) - Compressed List ziplist
ziplist-encoded compressed list objects use a compressed list as the underlying implementation, where each collection element is saved with two compact list nodes next to each other, the first node holds the member of the element, and the second element holds the score of the element.
Collection elements in a compressed list are sorted from smallest to largest, with smaller elements placed near the header and larger elements placed near the tail of the table
For example, if the following ZADD command is executed, the server will create an ordered collection object as the value of the price key
redis> ZADD price 8.5 apple 5.0 banana 6.0 cherry (integer) 3
Skplist-encoded ordered collection objects use the zset structure as the underlying implementation, a zset structure containing a dictionary and a jump table
typedef struct zset {
zskiplist *zsl;
dict *dict;
} zset;
The zsl jump table in the zset structure saves all set elements by score from smallest to largest, and each jump table node saves one set element
The object attribute of a skip table node holds the members of an element
The score attribute of a skip table node holds the element's score
With this jump table, programs can perform range operations on ordered collections, such as ZRANK, ZRANGE and other commands, which are based on the jump table API.
In addition, dict dictionaries in the zset structure create a mapping from members to scores for ordered collections
Each key-value pair in a dictionary holds a collection element, the key of the dictionary holds the members of the element, and the value of the dictionary holds the score of the element.
Using this dictionary, programs can find the score of a given member with O(1) complexity, and the ZSCORE command is based on this feature, which is used internally by many other ordered set commands.
Ordered set Each element's member is a string object, and each element's score is a floating point number of type double
Although the zset structure uses both a skip table and a dictionary to hold ordered collection elements, both data structures share members and values of the same element through pointers, so using both a skip table and a dictionary to hold collection elements does not result in any duplicate members or scores, nor does it waste additional memory
For example, if the preceding price key creates an ordered set object that is not ziplist-coded but skiplist-coded, then the ordered set object and object use the zset structure shown below.
Why do ordered collections need to be implemented using both skip tables and dictionaries?
In theory, ordered collections can be implemented using either a dictionary or one of the data structures of a jump table alone, but using either a dictionary or a jump table alone can reduce performance compared to using both a dictionary and a jump table
For example, if only dictionaries are used to implement ordered collections, then O(1) is used instead.Complexity Find Members'Scoring is preserved, but because dictionaries store collection elements in an out-of-order manner, the program needs to sort all the elements saved by the dictionary each time it performs a range operation, such as ZRANK, ZRANGE, and so on. This sort requires at least O(NlogN) time complexity and additional O(N) memory space.(because you want to create an array to hold the sorted elements)
On the other hand, if only a skip table is used to implement an ordered set, all the advantages of a skip table performing a range operation are preserved, but because there is no dictionary, the complexity of finding scores based on members increases from O(1) to O(logN)
For these reasons, Redis chose to use both dictionary and skip table data structures to implement ordered collections in order to make both the lookup of ordered collections and the scoped operation as fast as possible.
Coding Conversion of Ordered Collection Objects
An ordered collection object uses ziplist encoding when it satisfies both of the following conditions
- Ordered collections hold fewer than 128 elements
- All element members stored in an ordered collection are less than 64 bytes long
Ordered collection objects that do not meet these two conditions will use skiplist encoding
The upper limit of the above two conditions can be modified. See the description of the zset-max-ziplist-entries option and the zset-max-ziplist-value option in the configuration file
For ordered collection objects using ziplist encoding, when either of the two conditions required for ziplist encoding cannot be met, the object's encoding conversion operation is performed. All collection elements originally stored in the compressed list are transferred and saved in the zset structure, and the object's encoding changes from ziplist to skiplist.
The following code shows how an ordered collection object can cause encoding conversions because it contains too many elements
#
Object contains 128
Elements
redis> EVAL "for i=1, 128 do redis.call('ZADD', KEYS[1], i, i) end" 1 numbers
(nil)
redis> ZCARD numbers
(integer) 128
redis> OBJECT ENCODING numbers
"ziplist"
#
Add a new element
redis> ZADD numbers 3.14 pi
(integer) 1
#
Object contains 129 elements
individual
redis> ZCARD numbers
(integer) 129
#
Encoding changed
redis> OBJECT ENCODING numbers
"skiplist"
The following code shows an ordered collection object that causes encoding conversions because its members are too long
# Add an element to an ordered collection that has only three bytes of members redis> ZADD blah 1.0 www (integer) 1 redis> OBJECT ENCODING blah "ziplist" # Add a member to an ordered collection of 66 Elements of byte length redis> ZADD blah 2.0 ooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo (integer) 1 # Encoding changed redis> OBJECT ENCODING blah "skiplist"
Commands for Ordered Collection Objects (to be perfected)
Within a collection
1. Add Member-zadd
zadd key score member [score member ...]
Add user tom and his score 251 to the ordered collection user:ranking
127.0.0.1:6379> zadd user:ranking 251 tom (integer) 1
Return results represent the number of successfully added members
127.0.0.1:6379> zadd user:ranking 1 kris 91 mike 200 frank 220 tim 250 martin (integer) 5
There are two things to note about the zadd command:
- Redis3.2 adds four options for the zadd command: nx, xx, ch, incr:
- nx:member must not exist before it can be set successfully for addition
- xx:member must exist before it can be set up successfully for updating
- ch: The number of ordered collection elements and scores that have changed since the operation was returned
- incr: add score, equivalent to zincrby described later.
- An ordered set provides a sort field over a set, but at a cost, zadd has O(log(n)) time complexity and sadd has O(1)
2. Calculate the number of members - zcard
zcard key
For example, the following operation returns an ordered set user:ranking with a membership of 5, and as with scard commands of the set type, zcard has an O(1) time complexity
127.0.0.1:6379> zcard user:ranking (integer) 5
3. Calculate a member's score-zscore
zscore key member
tom has a score of 251 and returns nil if the member does not exist
127.0.0.1:6379> zscore user:ranking tom "251" 127.0.0.1:6379> zscore user:ranking test (nil)
4. Calculate the rank of members - zrank, zrevrank
zrank key member zrevrank key member
zrank returns rank from low to high, whereas zrevrank returns
For example, in the following operations, tom ranks 5th and 0th in zrank and zrevrank, respectively (ranking starts at 0).
127.0.0.1:6379> zrank user:ranking tom (integer) 5 127.0.0.1:6379> zrevrank user:ranking tom (integer) 0
5. Delete member-zrem
zrem key member [member ...]
The following action removes the member mike from the ordered collection user:ranking
127.0.0.1:6379> zrem user:ranking mike (integer) 1
Number of successful deletions returned
6. Increase Membership Score-zincrby
zincrby key increment member
The following adds 9 points to tom and changes the score to 260 points
127.0.0.1:6379> zincrby user:ranking 9 tom "260"
7. Return members of the specified ranking range - zrange, zrevrange
zrange key start end [withscores] zrevrange key start end [withscores]
Ordered sets are ranked by score, zrange returns from low to high, and zrevrange returns
The following code returns the lowest ranked three members, with the withscores option added, returning the member's score
127.0.0.1:6379> zrange user:ranking 0 2 withscores 1) "kris" 2) "1" 3) "frank" 4) "200" 5) "tim" 6) "220" 127.0.0.1:6379> zrevrange user:ranking 0 2 withscores 1) "tom" 2) "260" 3) "martin" 4) "250" 5) "tim" 6) "220"
8. Returns members of the specified score range - zrangebyscore, zrevrangebyscore
zrangebyscore key min max [withscores] [limit offset count] zrevrangebyscore key max min [withscores] [limit offset count]
Where zrangebyscore returns from low to high, whereas zrevrangebyscore returns
For example, if the following operation returns members with a score of 200 to 221 from low to high, the withscores option returns each member's score at the same time
The [limit offset count] option limits the starting location and number of outputs
127.0.0.1:6379> zrangebyscore user:ranking 200 tinf withscores 1) "frank" 2) "200" 3) "tim" 4) "220" 127.0.0.1:6379> zrevrangebyscore user:ranking 221 200 withscores 1) "tim" 2) "220" 3) "frank" 4) "200"
min and max also support open (parentheses) and closed (middle parentheses), where -inf and + inf represent infinite small and infinite large, respectively.
127.0.0.1:6379> zrangebyscore user:ranking (200 +inf withscores 1) "tim" 2) "220" 3) "martin" 4) "250" 5) "tom" 6) "260"
9. Returns the number of members in the specified score range - zcount
zcount key min max
The following returns the number of members with a score of 200 to 221
127.0.0.1:6379> zcount user:ranking 200 221 (integer) 2
10. Delete the ascending element in the specified rank - zremrangebyrank
zremrangebyrank key start end
The following action deletes the members from start to end
127.0.0.1:6379> zremrangebyrank user:ranking 0 2 (integer) 3
11. Delete members of the specified score range - zremrangebyscore
zremrangebyscore key min max
The following action deletes all members with more than 250 points and returns the number of successful deletions
127.0.0.1:6379> zremrangebyscore user:ranking (250 +inf (integer) 2
Operations between collections
Suppose the following two ordered sets are imported into Redis
user:ranking:1 and user:ranking:2
127.0.0.1:6379> zadd user:ranking:1 1 kris 91 mike 200 frank 220 tim 250 martin
251 tom
(integer) 6
127.0.0.1:6379> zadd user:ranking:2 8 james 77 mike 625 martin 888 tom
(integer) 4
1. Intersection-zinterstore
zinterstore destination numkeys key [key ...] [weights weight [weight ...]] [aggregate sum|min|max]
This command has more parameters
- destination, the intersection result is saved to this key
- numkeys, the number of computed keys that need to be intersected
- key[key...], the key that needs to be intersected
- weights weight[weight...], the weight of each key, each member in each key multiplies its score by this weight when doing intersection calculations, and the weight of each key defaults to 1
- aggregate sum|min|max, after calculating member intersection, the score can be summarized by sum (sum), min (minimum), Max (maximum), default value is sum
The following intersects user:ranking:1 and user:ranking:2, weights and aggregate s use the default configuration, and you can see that the target key user:anking:1_inter_2 sum s the value
127.0.0.1:6379> zinterstore user:ranking:1_inter_2 2 user:ranking:1
user:ranking:2
(integer) 3
127.0.0.1:6379> zrange user:ranking:1_inter_2 0 -1 withscores
1) "mike"
2) "168"
3) "martin"
4) "875"
5) "tom"
6) "1139"
If you want the weight of user:ranking:2 to be 0.5 and the aggregation effect to use max, you can do the following
127.0.0.1:6379> zinterstore user:ranking:1_inter_2 2 user:ranking:1 user:ranking:2 weights 1 0.5 aggregate max integer) 3 127.0.0.1:6379> zrange user:ranking:1_inter_2 0 -1 withscores 1) "mike" 2) "91" 3) "martin" 4) "312.5" 5) "tom" 6) "444"
2. Union-zunionstore
zunionstore destination numkeys key [key ...] [weights weight [weight ...]] [aggregate sum|min|max]
All the parameters of the command are identical to the zinterstore, but only a union calculation is done, such as calculating the union of user:ranking:1 and user:ranking:2, weights and aggregate using the default configuration, and you can see that the target key user:ranking:1_union_2 sum s the value
127.0.0.1:6379> zunionstore user:ranking:1_union_2 2 user:ranking:1
user:ranking:2
(integer) 7
127.0.0.1:6379> zrange user:ranking:1_union_2 0 -1 withscores
1) "kris"
2) "1"
3) "james"
4) "8"
5) "mike"
6) "168"
7) "frank"
8) "200"
9) "tim"
10) "220"
11) "martin"
12) "875"
13) "tom"
14) "1139"
Scenarios for ordered collection of objects (Important!!!)
Ordered collections are typically used in a leaderboard system
Video websites, for example, need to rank videos uploaded by users. The dimensions of the rankings can be in many ways: by time, by number of playback, by number of favorites received.
The following uses the favorite dimension to record the rankings of daily video uploads by users. There are four main functions that need to be implemented
(1) Add user favorites
For example, user mike uploaded a video and got three reviews for using zadd and zincrby functions in an ordered collection
zadd user:ranking:2016_03_15 mike 3
If you get another favor later, you can use zincrby
zincrby user:ranking:2016_03_15 mike 1
(2) Cancel user approval
For various reasons (such as user logoff, user cheating) you need to delete the user, at which point you need to delete the user from the list, you can use zrem. For example, delete member tom
zrem user:ranking:2016_03_15 mike
(3) Display the ten most popular users
This function is implemented using the zrevrange command
zrevrangebyrank user:ranking:2016_03_15 0 9
(4) Display user information and user scores
This function uses the user name as a key suffix and saves the user information in a hash type. You can use zscore and zrank for score and rank
hgetall user:info:tom zscore user:ranking:2016_03_15 mike zrank user:ranking:2016_03_15 mike