Redis learning object

1. Introduction

We can see that the main data structures used in redis include: simple string (SDS), linked list, dictionary, compressed list, integer set, etc. Redis objects are implemented by these data structures. Moreover, in order to select more appropriate data structures for different objects in different use scenarios, redis has created an object system based on these data structures, which includes five types of objects: String object, list object, hash object, collection object and ordered collection object.
The benefits of using object systems are:

  1. Before executing a command, judge whether an object can execute a given command according to the type of object;
  2. Set different data structures for objects in different use scenarios, so as to optimize the use efficiency of objects in different scenarios;
  3. The memory recycling mechanism of reference counting technology is implemented to release the memory of objects that are no longer used;
  4. The object shared memory mechanism is realized;
  5. The idling duration of the object is recorded.

Before the specific introduction, these functions sound a little confused. With in-depth study, you will gradually realize the power of the object system.

2. Object type and code

Redis uses objects to represent the keys and values in the database. Every time we create a new key value pair in redis's database, we will create at least two objects, one as the key of the key value pair and the other as the value of the key value pair.
For example, as shown in the figure:

  • Red font: the key of key value pair;
  • Green font: the value of key value pair.

    From this, we can know that the key of the key value pair always exists in the form of string, so we focus on the object type corresponding to the value of the key value pair.
    Before explaining the object type, you need to know what its data structure looks like (some fields are temporarily hidden and only key fields are listed):
typedef struct redisObject {
    // type
    unsigned type:4;
    
    // code
    unsigned encoding:4;
    
    // Pointer to the underlying implementation data structure
    void *ptr;
}

Each object in Redis is represented by the redisObject structure. The three attributes related to saving data in this structure are type, encoding and ptr.

2.1 object type - type

The type attribute of the object records the type of the object. The values of this attribute are as follows:

typed constant The name of the object
REDIS_STRINGString object
REDIS_LISTList object
REDIS_HASHhash objects
REDIS_SETCollection object
REDIS_ZSETOrdered collection object

As mentioned above, the object TYPE we mentioned actually refers to the object TYPE of the value of a key value pair. You can query the object TYPE of the value of a key value pair by using the TYPE command:

127.0.0.1:6379> TYPE msg
string

127.0.0.1:6379> TYPE list
list

127.0.0.1:6379> TYPE hashmap
hash

127.0.0.1:6379> TYPE set
set

127.0.0.1:6379> TYPE price
zset

2.2 object code - encoding

REDIS_ Object types such as string are composed of different codes, which use different codes according to different stored contents, and the encoded information is stored in encoding. ptr pointer is used to point to the underlying data structure of the object, which is determined by encoding. The cross reference table of detailed object code and object type is as follows:

typed constant Encoding constantSpecific object
REDIS_STRINGREDIS_ENCODING_INTString object implemented using integer values
REDIS_STRINGREDIS_ENCODING_EMBSTRString object implemented using simple dynamic string encoded by embstr
REDIS_STRINGREDIS_ENCODING_RAWString object implemented using simple dynamic string
REDIS_LISTREDIS_ENCODING_LINKEDLISTList object implemented using double ended linked list
REDIS_HASHREDIS_ENCODING_ZIPLISTHash object implemented using compressed list
REDIS_SETREDIS_ENCODING_INTSETA collection object implemented using an integer collection
REDIS_SETREDIS_ENCODING_HTCollection objects implemented using dictionaries
REDIS_ZSETREDIS_ENCODING_ZIPLISTAn ordered collection object implemented using a compressed list
REDIS_ZSETREDIS_ENCODING_SKIPLISTOrdered collection objects implemented using jump tables and dictionaries

We can use the OBJECT ENCODING key command to view the code corresponding to the value of the key value pair, as follows:

127.0.0.1:6379> SET msg 1
OK
127.0.0.1:6379> OBJECT ENCODING msg
"int"

127.0.0.1:6379> SET student "wifi"
OK
127.0.0.1:6379> OBJECT ENCODING student
"embstr"

127.0.0.1:6379> SET numbers "1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20"
OK
127.0.0.1:6379> OBJECT ENCODING numbers
"raw"

127.0.0.1:6379> HSET hash a 1 b 2 c 3
(integer) 3
127.0.0.1:6379> OBJECT ENCODING hash
"ziplist"

The encoding attribute is used to determine the encoding used by the object, rather than associating a fixed encoding for a specific type of object, which greatly improves the flexibility and efficiency of Redis, because Reis can set different encoding for an object according to different usage scenarios, so as to optimize the efficiency of the object in a certain scenario. Later, we will talk about when the coding of different data structures will change.

3. String object

The encoding of string object can be int, raw or embstr.
If a string object stores an integer value and the integer value can be represented by long, the string object will save the integer value in the ptr attribute of the string object structure (convert void * to long) and set the encoding of the string object to int.

For example:

redis> SET number 10086
OK
    
redis> OBJECT ENCODING number
"int"

The structure is as follows:

If the string object saves a string with a string length less than or equal to 44 bytes, the string object will use embstr encoding to save the string value.

127.0.0.1:6379> SET str "hello world! hello world! hello world! hello"
OK
127.0.0.1:6379> STRLEN str
(integer) 44
127.0.0.1:6379> OBJECT ENCODING str
"embstr"

If the string object saves a string and the length of the string value is greater than 44 bytes, the string object will use a simple dynamic string (SDS) to save the string value and set the encoding of the object to raw.

For example:

127.0.0.1:6379> SET msg "hello world! hello world! hello world! hello "
OK
127.0.0.1:6379> STRLEN msg
(integer) 45
127.0.0.1:6379> OBJECT ENCODING msg
"raw"

The structure is as follows:

3.1 what is the difference between raw and embstr?

  • difference:
    embstr encoding is an optimized encoding method specially used to save short strings. Like raw encoding, this encoding uses readObject structure and sdshdr structure to represent string objects, but raw encoding will call the memory allocation function twice to create readObject and sdshdr structures respectively, embstr code allocates a continuous space by calling the memory allocation function once. The space contains redisObject and sdshdr structures in turn, as shown in the following figure:

  • Similarities:
    When an embstr encoded string object executes a command, the effect is the same as that of a raw encoded string object.

Advantages of embstr:

  1. embstr encoding reduces the number of memory allocations required to create string objects from two to one;
  2. To release the string object encoded by embstr, you only need to call the memory release function once;
  3. Because all the data of embstr encoded string objects are stored in a continuous memory, this encoded string object can make better use of the advantages of caching than raw encoded string objects.

For more detailed information, please refer to: Redis development and operation and maintenance: SDS

3.2 conversion of string encoding

Specific convertible forms include:

  • int -> int
  • int -> raw
  • embstr -> raw
  • raw -> raw

Int - > int example:

redis> SET number 10086
OK

redis> OBJECT ENCODING number
"int"

127.0.0.1:6379> incr number
(integer) 10087

127.0.0.1:6379> OBJECT ENCODING number
"int"

Int - > raw example:

127.0.0.1:6379> SET number 10086
OK

127.0.0.1:6379> OBJECT ENCODING number
"int"

127.0.0.1:6379> APPEND number " is a good number!"
(integer) 23

127.0.0.1:6379> GET number
"10086 is a good number!"

127.0.0.1:6379> OBJECT ENCODING number
"raw"

Embstr - > raw example:

127.0.0.1:6379> SET msg "hello world"
OK

127.0.0.1:6379> OBJECT ENCODING msg
"embstr"

127.0.0.1:6379> APPEND msg " again!"
(integer) 18

127.0.0.1:6379> OBJECT ENCODING msg
"raw"

Doesn't it mean that the encoding of strings within 44 bytes is embstr? How come it's only 18 bytes, and it's raw code?
This is because Redis does not write any corresponding modification program for the embstr encoded string object (only int and raw encoded strings exist), so the embstr encoded string object is actually read-only. When we execute any modification command on the string object encoded by embstr, the program will first convert the encoding of the object from embstr to raw, and then execute the command. For this reason, the embstr encoded string object will always become a raw encoded string object when executing the modify command.

Raw - > raw example:

127.0.0.1:6379> SET msg "hello world! hello world! hello world! hello "
OK

127.0.0.1:6379> OBJECT ENCODING msg
"raw"

127.0.0.1:6379> APPEND msg "TEST!!!!!!!"
(integer) 56

127.0.0.1:6379> OBJECT ENCODING msg
"raw"

3.3 command implementation corresponding to different string encoding

commandImplementation of int codingImplementation of embstr codingImplementation of raw coding
SETSave values using int encodingSave values using embstr encodingSave values using raw encoding
GETCopy the integer value saved by the object, convert the copy into a string value, and then return the string to the clientReturns a string directly to the clientReturns a string directly to the client
APPENDConvert the object to raw encoding, and then do so in raw encodingConvert the object to raw encoding, and then do so in raw encodingCall the sdscatlen function to append the given string to the end of the existing string
INCRBYFLOATTake out the integer value and convert it into a floating-point number of long double type, add the floating-point number, and then save the result of the floating-point numberTake out the string and try to convert it into a floating-point number of type long double, add the floating-point number, and then save the result of the floating-point number.
If the string cannot be converted to a floating-point number, an error is returned to the client		Take out the string and try to convert it into a floating-point number of type long double, add the floating-point number, and then save the result of the floating-point number.
If the string cannot be converted to a floating-point number, an error is returned to the client
INCRBYAdd the integer value and the result will be saved as an integerembstr encoding cannot execute this command and returns an error to the clientraw encoding cannot execute this command and returns an error to the client
DECRBYSubtract the integer value, and the result will be saved as an integerembstr encoding cannot execute this command and returns an error to the clientraw encoding cannot execute this command and returns an error to the client
STRLENCopy the integer value saved by the object, convert the copy into a string value, and calculate and return the length of the string valueCall the strlen function to return the length of the stringCall the strlen function to return the length of the string
SETRANGEConvert the object to raw encoding, and then do so in raw encodingConvert the object to raw encoding, and then do so in raw encodingSets the value on the string index to the given character
GETRANGECopy the integer value saved by the object, convert the copy into a string value, and then take out and return the characters on the specified index of the stringDirectly fetch and return the characters on the specified index of the stringDirectly fetch and return the characters on the specified index of the string

4. Hash object

The encoding of the hash object is:

  • ziplist
  • hashtable

4.1 hash object encoded by ziplost

The zip list encoded hash object uses the compressed list as the underlying implementation. Whenever a new key value pair is added to the hash object, the program will first push the compressed list node with the saved key to the end of the compressed list, and then push the compressed list node with the saved value to the end of the compressed list. Therefore:

  • The two nodes that save the same key value pair are always close together. The node that saves the key is first and the node that saves the value is last;
  • The key value pairs first added to the hash object will be placed in the header direction of the compressed list, while the key value pairs later added to the hash object will be placed in the footer direction of the compressed list.

For example:

127.0.0.1:6379> HMSET profile name "Tom" age 25 career "programmer"
OK
127.0.0.1:6379> OBJECT ENCODING profile
"ziplist"

The structure is as follows:

4.2 hash object encoded with hashtable

The hashtable encoded hash object uses a dictionary as the underlying implementation, and each key value pair in the hash object uses a dictionary key value pair to save:

  • Each key of the dictionary is a string object, which holds the key of the key value pair;
  • Each value of the dictionary is a string object, which stores the value of the key value pair;

The structure is as follows:

be careful:

  1. The hashtable in the figure above is only a simplified version, and the real hashtable structure will be a little more complex;
  2. The above example does not allow Redis to use hashtable encoding as the encoding of hash objects, but is used to display the structure of hashtable encoded hash objects;
  3. When to use zip list coding and hashtable coding will be mentioned in the next section.

4.3 code conversion

When the hash object can meet the following two conditions at the same time, the hash object is encoded with ziplist:

  • The string length of keys and values of all key value pairs saved by the hash object is less than 64 bytes;
  • The number of key value pairs saved by the hash object is less than 512;

Hash objects that cannot meet these two conditions will be encoded with hashtable.

4.3.1 code conversion caused by too long key

Example:

127.0.0.1:6379> HSET book name "Mastering C++ in 21 days"
(integer) 1
127.0.0.1:6379> OBJECT ENCODING book
"ziplist"

127.0.0.1:6379> HSET book long_long_long_long_long_long_long_long_long_long_long_description "content"
(integer) 1
127.0.0.1:6379> OBJECT ENCODING book
"hashtable"

4.3.2 code conversion caused by too long value

Example:

127.0.0.1:6379> HSET hs greeting "hello world"
(integer) 1
127.0.0.1:6379> OBJECT ENCODING hs
"ziplist"

# 65 bytes
127.0.0.1:6379> HSET hs story "11111111111111111111111111111111111111111111111111111111111111111"
(integer) 1
127.0.0.1:6379> OBJECT ENCODING hs
"hashtable"

Only quicklist can be converted into hashtable, but hashtable cannot be converted into quicklist.

5. Collection object

The encoding of the collection object is

  • intset
  • hashtable

5.1 set object encoded by intset

The set object encoded by intset uses the integer set as the underlying implementation, and all elements contained in the set object are saved in the integer set.

For example, a collection object encoded with intset:

127.0.0.1:6379> SADD numbers 1 3 5
(integer) 3
127.0.0.1:6379> OBJECT ENCODING numbers
"intset"

The structure is as follows:

5.2 set object encoded with hashtable

The hashtable encoded collection object uses the dictionary as the underlying implementation. Each key of the dictionary is a string object, each string object contains a collection element, and all the values of the dictionary are set to NULL.

For example, a collection object encoded with hashtable:

127.0.0.1:6379> SADD fruits "apple" "banana" "cherry"
(integer) 3
127.0.0.1:6379> OBJECT ENCODING fruits
"hashtable"

The structure is as follows:

The hashtable in the figure above is only a simplified version, and the real hashtable structure will be a little more complex;

5.3 code conversion

When the set object can meet the following two conditions at the same time, the object uses intset encoding; otherwise, it uses hashtable encoding:

  • All elements saved by the collection object are integer values;
  • The number of elements saved by the collection object does not exceed 512.

5.3.1 set objects contain non integer elements

For set objects encoded with intset, when either of the two conditions required for intset encoding cannot be met, the encoding conversion operation of the object will be performed. All elements originally saved in the integer set will be transferred and saved into the dictionary, and the encoding of the object will also be changed from intset to hashtable.

Example:

127.0.0.1:6379> SADD numbers 1 3 5
(integer) 3
127.0.0.1:6379> OBJECT ENCODING numbers
"intset"

127.0.0.1:6379> SADD numbers "test"
(integer) 1
127.0.0.1:6379> OBJECT ENCODING numbers
"hashtable"

5.3.2 the set object stores more than 512 elements

127.0.0.1:6379> EVAL "for i = 1, 512 do redis.call('SADD', KEYS[1], i) end" 1 integers
(nil)
127.0.0.1:6379> SCARD integers
(integer) 512
127.0.0.1:6379> OBJECT ENCODING integers
"intset"

127.0.0.1:6379> SADD integers 513
(integer) 1
127.0.0.1:6379> SCARD integers
(integer) 513
127.0.0.1:6379> OBJECT ENCODING integers
"hashtable"

6. Ordered collection objects

The codes of ordered sets are

  • ziplist
  • skiplist

6.1 ordered collection object encoded by ziplist

The ordered collection object encoded by ziplost uses the compressed list as the underlying implementation. Each collection element is saved with the compressed list node next to each other. The first node saves the member of the element and the second element saves the score of the element.
The set elements in the compressed list are sorted from small to large. The elements with smaller scores are placed near the header, while the elements with larger scores are placed near the footer.

For example, an ordered collection encoded in ziplost:

127.0.0.1:6379> ZADD price 8.5 apple 5.0 banana 6.0 cherry
(integer) 3
127.0.0.1:6379> OBJECT ENCODING price
"ziplist"

The structure is as follows:

6.2 ordered collection object encoded by skiplist

The ordered set object encoded by skiplist uses zset structure as the underlying implementation. A zset structure contains a dictionary and a jump table at the same time. The structure is as follows:

typedef struct zset {
    zskiplist *zsl;
    dict *dict;
} zset;

The zsl jump table in the zset structure saves all set elements from small to large according to the score, and each jump table node saves a set element:

  • The object attribute of the jump table node saves the members of the element;
  • The score attribute of the jump table node saves the score of the element.

Through this jump table, the program can perform range operations on ordered sets, such as ZRANK, ZRANGE and other commands.

In addition, the dict dictionary in the zset structure creates a mapping from members to scores for ordered sets. Each key value pair in the dictionary stores a set element:

  • The key of the dictionary holds the members of the element;
  • The value of the dictionary holds the score of the element.

Through this dictionary, the program can find the score of a given member with O(1) complexity. ZSCORE command is implemented according to this feature.

Although zset structure uses jump table and dictionary to save ordered set elements at the same time, these two data structures will share the members and scores of the same element through pointers. Therefore, using jump table and dictionary to save set elements at the same time will not produce any duplicate members or scores, and will not waste memory.

The structure is as follows:

6.2.1 why should an ordered set be implemented using both jump tables and dictionaries?

Theoretically, the ordered set can be realized by using one of the data structures of dictionary or jump table alone, but no matter using dictionary or jump table alone, the performance will be reduced compared with using dictionary and jump table at the same time.

  • Use dictionary only
    • Although the score of members can be found by O(1) complexity, because the dictionary saves elements in an unordered way, the program needs to sort all the elements saved by the dictionary every time when executing range operations (such as ZRANGE, ZRANK and other commands). Completing this sorting requires at least O(NlogN) time complexity, And additional O(N) memory space for storing the sorted elements;
  • Use only jump tables
    • All the advantages of performing range operations on jump tables will be preserved, but because there is no dictionary, the complexity of finding scores according to members will increase from O(1) to O(logN).

Therefore, in order to make the search and range operations of the ordered set as fast as possible, Redis chose to use both dictionary and jump table data structures to realize the ordered set.

6.3 code conversion

When the ordered collection object can meet the following two conditions at the same time, the object is encoded with ziplist

  • The number of elements saved in the ordered set is less than 128;
  • The length of all element members saved in an ordered collection is less than 64 bytes.

Ordered collection objects that cannot meet the above two conditions will be encoded with skiplist

6.3.1 the number of ordered collections exceeds 128

Example:

127.0.0.1:6379> EVAL "for i = 1, 128 do redis.call('ZADD', KEYS[1], i, i) end" 1 numbers
(nil)
127.0.0.1:6379> ZCARD numbers
(integer) 128
127.0.0.1:6379> OBJECT ENCODING numbers
"ziplist"

127.0.0.1:6379> ZADD numbers 129 129
(integer) 1
127.0.0.1:6379> ZCARD numbers
(integer) 129
127.0.0.1:6379> OBJECT ENCODING numbers
"skiplist"

6.3.2 the length of ordered set element members is greater than 64 bytes

Example:

127.0.0.1:6379> ZADD numbers 1.0 a
(integer) 1
127.0.0.1:6379> OBJECT ENCODING numbers
"ziplist"

127.0.0.1:6379> ZADD numbers 2.0 aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
(integer) 1
127.0.0.1:6379> OBJECT ENCODING numbers
"skiplist"

7. Object type check and command polymorphism

There are basically two types of commands used to operate keys in Redis.

  1. RedisObject oriented commands, such as DEL, experiment, RENAME, TYPE, OBJECT, etc;
# We create five types of data
127.0.0.1:6379> SET string "value"
OK
127.0.0.1:6379> RPUSH list 1 3 5
(integer) 3
127.0.0.1:6379> HSET hash a 1
(integer) 1
127.0.0.1:6379> SADD set 1 3 5
(integer) 3
127.0.0.1:6379> ZADD zset 1 a 2 b
(integer) 2

# Using the TYPE command
127.0.0.1:6379> TYPE string
string
127.0.0.1:6379> TYPE list
list
127.0.0.1:6379> TYPE set
set
127.0.0.1:6379> TYPE zset
zset
127.0.0.1:6379> TYPE hash
hash

RedisObject oriented encoding commands, such as

  1. SET, GET, APPEND, STRLEN and other commands can only be executed on string keys;
  2. HDEL, HSET, HGET, HLEN and other commands can only be executed on hash keys;
  3. RPUSH, LPOP, LINSERT, LLEN and other commands can only be executed for list creation;
  4. SADD, SPOP, sintert, scar and other commands can only be executed on set keys;
  5. ZADD, ZCARD, ZRANK, ZSCORE and other commands can only be executed on ordered set keys.
# Manipulate specific objects with specific commands
127.0.0.1:6379> SET string "value"
OK
127.0.0.1:6379> RPUSH list 1 3 5
(integer) 3
127.0.0.1:6379> HSET hash a 1
(integer) 1
127.0.0.1:6379> SADD set 1 3 5
(integer) 3
127.0.0.1:6379> ZADD zset 1 a 2 b
(integer) 2

7.1 implementation of object type check

In order to ensure that only the specified type of key can execute some specific commands, Redis will check whether the type of input key is correct before executing a specific type of command, and then decide whether to execute the given command.

The type checking of type specific commands is realized through the type attribute of redisObject structure:

  1. Before executing a type specific command, the server will first check whether the value object of the input data key is the type required to execute the command. If so, the server will execute the specified command on the key;
  2. Otherwise, the server will refuse to execute the command and return a type error to the client.

The flow chart is as follows:

Example:

127.0.0.1:6379> HSET profile name "wifi"
(integer) 1
127.0.0.1:6379> OBJECT ENCODING profile
"ziplist"
127.0.0.1:6379> GET profile
(error) WRONGTYPE Operation against a key holding the wrong kind of value

7.2 implementation of polymorphic commands

Redis will not only judge whether the key can execute the specified command according to the type of value object, but also select the correct command implementation code according to the coding method of value object to execute the command.

In fact, we can also call DEL, export, TYPE and other commands polymorphic commands, because these commands can be executed correctly regardless of the TYPE of key entered.

The difference between DEL, express, TYPE and SET, GET, APPEND, STRLEN and other commands is that the former is based on TYPE polymorphism and the latter is based on coding polymorphism.

The execution flow of polymorphic commands is shown in the figure:

8. Memory recycling

Because C language does not have the function of automatic memory recovery, Redis has built a memory recovery mechanism based on reference counting in its own object system. Through this mechanism, the program can automatically release the object and recover the memory at an appropriate time by tracking the reference counting information of the object.

The reference count information of each object is recorded by the refcount attribute of redisObject structure:

typedef struct redisObject {
    // Reference count
    int refcount;
} robj;

The reference count information of the object changes with the use status of the object:

  • When creating a new object, the value of reference count will be initialized to: 1;
  • When an object is used by a new program, its reference count value will be + 1;
  • When an object is no longer used by a program, its reference count value will be: - 1;
  • When the reference count value of the object becomes 0, the memory occupied by the object will be released.

The whole life cycle of objects can be divided into three stages: creating objects, operating objects and releasing objects.
As an example, the following code shows the whole process from creation to release of a string object:

// Create a string object s with a reference count of 1
robj *s = createStringObject(...)
    
// Object s performs various operations

// The reference count of object s is - 1, making the reference count of object 0
// Cause object s to be released
decrRefCount(s)

9. Object sharing

In addition to the reference count memory recycling mechanism, the reference count attribute of an object also plays the role of object sharing.

For example, suppose key A creates A string object containing an integer value of 100 as A value object, as shown in the figure:

If key B also needs to create a string object with an integer value of 100 as the value object, the server has the following two methods:

  1. Create a new string object containing an integer value of 100 for key B;
  2. Let key A and key B share the same string object.

The above two methods are obviously the second, which is more memory saving. In Redis, there are two steps to make multiple keys share the same value object:

  1. Point the value pointer of the database key to an existing value object;
  2. Increment the reference count of the shared value object by one.

As shown in the following figure, key A and key B share the string object of integer value 100 at the same time. The only change is that refcount changes from 1 to 2.

The shared object mechanism is very helpful to save memory. The object sharing mechanism can save more memory if the objects with the same value are stored in the database.

At present, Redis will create 10000 string objects when initializing the server. These objects contain all integer values from 0 to 9999. When the server needs to use integer value string objects in this range, the server will use these shared objects instead of newly created objects.

However, during actual operation, it will be found that the value reference count is 2147483647 instead of a simple 1 or 2.

127.0.0.1:6379> SET A 100
OK
127.0.0.1:6379> OBJECT REFCOUNT A
(integer) 2147483647

My Redis version is: 5.0.8

By checking the source code, it is found that in object In the robj *makeObjectShared(robj *o) method in the C file:

robj *makeObjectShared(robj *o) {
    serverAssert(o->refcount == 1);
    # When sharing objects, refcount points to OBJ_SHARED_REFCOUNT
    o->refcount = OBJ_SHARED_REFCOUNT;
    return o;
}

Then check the server Obj in H file_ SHARED_ The refcount variable definition shows:

# OBJ_SHARED_REFCOUNT initialization value: INT_MAX
# define OBJ_SHARED_REFCOUNT INT_MAX     /* Global object never destroyed. */

Therefore, in the newer version, the refcount value corresponding to the integer value string object from 0 to 9999 is INT_MAX.

9.1 why doesn't Redis share objects containing strings?

Through simple experiments, Redis creates new string objects every time instead of sharing string objects:

127.0.0.1:6379> SET B "HELLO"
OK
127.0.0.1:6379> OBJECT REFCOUNT B
(integer) 1
127.0.0.1:6379> SET C "HELLO"
OK
127.0.0.1:6379> OBJECT REFCOUNT C
(integer) 1

This is because:

When the server considers setting a shared object as the value object of the key, the program needs to check whether the given shared object and the target object to be created by the key are exactly the same. Only when the shared object and the target object are exactly the same, the program will use the shared object as the value object of the key. The more complex the value saved by a shared object, The higher the complexity required to verify whether the shared object and the target object are the same, the more CPU time will be consumed:

  • If the shared object is a string object that holds integer values, the complexity of the verification operation is: O(1);
  • If the shared object is a string object that holds string values, the complexity of the verification operation is: O(N);
  • If the shared object is an object that contains multiple values (or objects), such as a list object or a hash object, the complexity of the verification operation is: O(N2).

Therefore, although sharing more complex objects can save more memory, due to the limitation of CPU time, Redis only shares string objects containing integer values.

10. Idling duration of the object

In addition to the four attributes of type, encoding, ptr and refcount described above, the last attribute contained in the redisObject structure is lru attribute, which records the last time the object was accessed by the command program:

typedef struct redisObject {
    unsigned lru:24; /* LRU time (relative to global lru_clock) or
                            * LFU data (least significant 8 bits frequency
                            * and most significant 16 bits access time). */
} robj;

The idle time of a given key can be printed by using the OBJECT IDLETIME command. This idle time is calculated by subtracting the lru time of the value object of the key from the current time:

127.0.0.1:6379> SET msg "test"
OK
127.0.0.1:6379> OBJECT IDLETIME msg
(integer) 10
127.0.0.1:6379> OBJECT IDLETIME msg
(integer) 11
127.0.0.1:6379> OBJECT IDLETIME msg
(integer) 12
127.0.0.1:6379> GET msg
"test"
127.0.0.1:6379> OBJECT IDLETIME msg
(integer) 2

In addition to being printed by the OBJECT IDLETIME command, the idle time of the key has another function:

If the maxmemory option is enabled on the server, and the algorithm used by the server to reclaim memory is volatile LRU or allkeys LRU, when the amount of memory occupied by the server exceeds the upper limit set by the maxmemory option, the key with higher idle time will be released by the server first, so as to reclaim memory.

Question: will the integer value string from 0 to 9999 be recycled?

10.1 "pit" of object idletime command

From the previous knowledge, we know that in Redis, 0 ~ 9999 integer value strings are shared, so when using the OBJECT IDELTIME command, the lru value of the object whose key value is integer value string is also shared, as shown in the following example:

127.0.0.1:6379> set a 1
OK
127.0.0.1:6379> set b 1
OK
# When 1 has not been called for a long time, current_ The value of time - LRU is very large
127.0.0.1:6379> OBJECT IDLETIME a
(integer) 365481
127.0.0.1:6379> OBJECT IDLETIME b
(integer) 365487
        
# Call key a
127.0.0.1:6379> get a
"1"
# At this time, the idle time value of key a and key b is affected at the same time
127.0.0.1:6379> OBJECT IDLETIME b
(integer) 2
127.0.0.1:6379> OBJECT IDLETIME a
(integer) 5

# Non integer numeric strings are not affected
127.0.0.1:6379> set msg "hello"
OK
127.0.0.1:6379> set string "hello"
OK
127.0.0.1:6379> OBJECT IDLETIME msg
(integer) 18
127.0.0.1:6379> OBJECT IDLETIME string
(integer) 17
127.0.0.1:6379> get msg
"hello"
127.0.0.1:6379> OBJECT IDLETIME string
(integer) 24
127.0.0.1:6379> OBJECT IDLETIME msg
(integer) 9

Therefore, when using the OBJECT IDLETIME command, you need to pay attention to sharing integer value characters.

11. Complete redisObject object structure

Finally, show the complete data structure of redisObject object:

typedef struct redisObject {
    unsigned type:4;
    unsigned encoding:4;
    unsigned lru:LRU_BITS; /* LRU time (relative to global lru_clock) or
                            * LFU data (least significant 8 bits frequency
                            * and most significant 16 bits access time). */
    int refcount;
    void *ptr;
} robj;

Source location: server H medium

reference resources

Keywords: Database Redis data structure

Added by brokeDUstudent on Mon, 31 Jan 2022 00:53:48 +0200

Popular Keywords

©2024 Programming VIP