Distributed cache - detailed explanation of Redis basic functions
1. Preface
After several years of development, the concept of cache should be familiar. Common cache components include redis, mongodb and memcached. This article briefly introduces the common usage scenarios of redis and the use of basic APIs.
redis is a database based on key value pairs. Value can be String, hash, list, set, zset and other data structures. It can meet many application scenarios. It also provides additional functions such as key expiration, publish and subscribe, transaction, pipeline and so on.
characteristic:
1) the speed is fast, the data is stored in the memory, and the official reading and writing performance is 100000 / S, which is also related to the machine performance
a. Putting data in memory is the main reason for fast speed
b. C language implementation, close to the operating system
c. Single thread architecture is used to prevent possible competition problems caused by multithreading
2) data structure server of key value pair
3) rich functions: see the above functions
4) simple and stable: single thread
5) persistence: in case of power failure or machine failure, data may be lost and persisted to the hard disk
6) master slave replication: realize multiple redis copies of the same data
8) high availability and distributed: sentinel mechanism realizes high availability to ensure redis node fault discovery and automatic transfer
9) many client languages: Java PHP Python C + + nodejs
2. Installation and use of api
2.1 installation
1.Redisdownload , I use redis-6.2 5 for this version, linux is recommended. After all, most of the time, our servers also use linux.
2. Unzip. Mine is to put it in the opt directory. According to your preferences, unzip the command
tar -zvxf redis-6.2.5.tar.gz
3. Decompress and compile into the redis directory
make PREFIX=/opt/redis-6.2.5 install if an error is reported, execute the following steps
make MALLOC=libc
Description of redis memory allocator selection: when building redis, select a non default memory allocator by setting the "MALLOC" environment variable. Redis compiles and links to libc malloc by default, but the default memory allocator on Linux system is jemalloc. Linux systems choose jemalloc as the default memory allocator because jemalloc has proved to have fewer fragmentation problems than libc malloc.
To forcibly compile redis using libc malloc, use the: make MALLOC=libc instruction
Then execute make prefix = / opt / redis-6.2 5 install
4. At this time, you can see an additional bin directory in the redis directory
5. Start command
./redis-server /opt/redis-6.2.5/redis.conf
Specify / opt / redis-6.2 5/redis. Conf this configuration file
Here, the installation of redis is basically completed.
2.1 use
Use/ redis-server /opt/redis-6.2. 5/redis. When conf starts, you will find that redis stops as soon as you exit. So we have to add the & symbol to the command/ redis-server& /opt/redis-6.2. 5/redis. Conf & symbol with table background start
2.1 configuration file
After compilation, an additional redis.exe will appear in the redis directory The conf file is the configuration file. Here are some general configuration items
| Configuration item | value | explain |
|---|---|---|
| daemonize | yes/no | yes indicates that the daemon is enabled. The default is no, that is, it does not run as a daemon. The daemon mode is not supported under Windows system |
| port | 6379 | Program running port |
| protected-mode | yes/no | Protection mode. This mode controls whether the external network can connect to the redis service. The default is yes, so our external network cannot be accessed by default. If the external network needs to connect to the rendis service, this attribute needs to be changed to no |
| timeout | 1000 | When the client is idle for how long, close the connection. If 0 is specified, it means that the function is closed |
| databases | 16 | Set the number of databases. The default database is 16, arranged by subscript 0-15, and there is only one 0 in the cluster environment |
| dir | /opt/redis-data | Local database storage directory |
| bind | 127.0.0.1 | Bound host address |
| requirepass | 123456 | Set the Redis connection password. If the connection password is configured, the client needs to pass password verification when connecting to Redis. It is closed by default |
There are other configuration items, which will not be listed here. For detailed configuration, please refer to the official documents.
2.2 modify configuration:
Comment out bind Configuration of protected-mode Change to no daemonize Change to yes Set password requirepass 12345678
2.3 connect to redis using the client
1.stay bin Enter command./redis-cli -p 6379 I use port 6379 here,
//Verify password [root@localhost bin]# ./redis-cli -p 6379 127.0.0.1:6379> auth 12345678
3. Data types and api usage supported by redis
Redis supports String, hash, list, set, zset, etc
3.1 use of string type
String: a string can actually be a string (including XML JSON),
There are also digital (integer floating-point number), binary (picture, audio and video), and the maximum can not exceed 512MB;
//Insert a key value into set. This command will overwrite the existing key value pairs. Using this command will overwrite the original values.
//For string storage, set a key as name and a value of 12345 to expire after 10 seconds. ex is in seconds and ps is in milliseconds 127.0.0.1:6379> set name 12345 ex 10 127.0.0.1:6379> set name 12345 px 10000
//get gets the value of a key
127.0.0.1:6379> get name "12345"
//Exists checks whether a key exists. If it exists, it returns 1 and if it does not exist, it returns 0; It is often used as a cache. Judge whether the cache does not exist and then check it in the database
127.0.0.1:6379> exists name (integer) 1
//del delete a key
//Delete a specified key and return the deleted record line 127.0.0.1:6379> del name (integer) 1
//setnx when the key exists, insertion is not allowed; For some business data, use this command to prevent data from being overwritten; This function is similar to the temporary node of zookeeper and can be used to implement distributed locks. (Redisson is a complete distributed lock implementation scheme officially recommended by redis. Interested friends can take a look and study it by themselves.)
//When the name key exists, the insertion fails and returns 0. If it does not exist, the insertion succeeds and returns 1 127.0.0.1:6379> setnx name 12345 (integer) 1
//mset inserts multiple strings at one time; When multiple inserts are involved, try to use this method to reduce the network overhead between the client and the server.
// The one-time insertion key is two key value pairs with country value of china and city value of chongqing. 127.0.0.1:6379> mset country china city chongqing OK
//incr self increment and decr self decrement can add and subtract integer type data
127.0.0.1:6379> set age 18 OK 127.0.0.1:6379> incr age (integer) 19 127.0.0.1:6379> decr age (integer) 18
//incrby increases the specified quantity decrby decreases the specified quantity as above, adding and subtracting values of numeric type
127.0.0.1:6379> set age 18 OK 127.0.0.1:6379> incrby age 2 (integer) 20 127.0.0.1:6379> decrby age 2 (integer) 18
//incrbyfloat floating point addition
127.0.0.1:6379> set score 18.2 OK 127.0.0.1:6379> incrbyfloat score 2.3 "20.5"
//Append string append, similar to the append() operation of StringBuffer
127.0.0.1:6379> set name zhangsan OK 127.0.0.1:6379> append name " is a teacher" (integer) 21 127.0.0.1:6379> get name "zhangsan is a teacher"
3.2 use of hash type
hash is actually a mapping table of string keys and values To put it bluntly, one key corresponds to multiple key values, which is particularly suitable for storing objects. Of course, we can also format objects into json strings and store strings directly.
hset stores a hash type of data
//Set a hash type data whose key is user: 1, name attribute is Zhangsan, and age attribute is 18 127.0.0.1:6379> hset user:1 name zhangsan age 18 (integer) 2
hget gets a property corresponding to a key value
//Gets the name attribute with the key user:1 127.0.0.1:6379> hget user:1 name "zhangsan"
hmget gets multiple properties of a key value in batch
//Get the name attribute and age attribute with the key user:1 127.0.0.1:6379> hmget user:1 name age 1) "zhangsan" 2) "18"
You can see that I specifically used user: 1 as the key and set the name and age attributes. If we use mysql, it is similar to select name,age from user where id = 1. User represents the user table, and 1 represents the primary key. We use this mapping. Store MySQL data perfectly in redis. This storage method is simple and intuitive. Rational use can reduce memory consumption, The disadvantage is the need to control the encoding conversion (redis will convert all stored data into redisObejct objects, and there are 10 corresponding encoding methods according to the data type and size. The actual purpose is to improve memory utilization; anyway, the key value and value should not be too long. If they are too long, it is recommended to split them). If it is not controlled, hashtable will consume more memory; in addition to hash, there is another way to store directly Serialization is used to store objects. This method has high memory utilization. The disadvantage is that I only change a certain value of the object, and I also need to deserialize the whole object, and then serialize it to redis after the operation. Serialization and deserialization have some overhead. How to choose depends on the actual situation. If your resources need to be modified frequently, hash is recommended.
hdel Deletes one or more fields corresponding to a key value
127.0.0.1:6379> hdel user:1 name (integer) 1
There is a big difference between hash operation and string operation, but the attribute must be specified after the specified key, and the command must be preceded by an h. I won't introduce them one by one here.
3.3 use of list data type
It is used to store multiple ordered strings. A list can store up to the 32nd power of 2 - 1 elements. Because it is ordered, you can obtain elements or a list of elements in a range through the index subscript
rpush and lrange
// Insert Id1 Id2 ID3 strings into the order list 127.0.0.1:6379> rpush order id1 id2 id3 (integer) 3 //Remove all elements in the order list. 0 - 1 means to take out all elements 127.0.0.1:6379> lrange order 0 -1 1) "id1" 2) "id2" 3) "id3"
lpush and lrange
127.0.0.1:6379> lpush order1 id1 id2 id3 (integer) 3 127.0.0.1:6379> lrange order1 0 -1 1) "id3" 2) "id2" 3) "id1"
From the above, we can see that the insertion order is Id1, Id2 and ID3, but the order of lrange is different. This is because this is actually a two-way linked list, one is inserted from left to right (lpush) and the other is inserted from right to left (rpush). Therefore, the order of results from lrange is reversed, because we read (lgange) from left to right
If you don't understand, let's look at the following example. Just now we rpush and lpush an order list element, both of which are Id1, Id2 and ID3. Now I insert an element into it
linsert inserts an element in the order list and an id4 element before the id2 element;
And insert an element id4 in the order1 list, also before id2.
127.0.0.1:6379> linsert order before id2 id4 (integer) 4 127.0.0.1:6379> lrange order 0 -1 1) "id1" 2) "id4" 3) "id2" 4) "id3" 127.0.0.1:6379> linsert order1 before id2 id4 (integer) 4 127.0.0.1:6379> lrange order1 0 -1 1) "id3" 2) "id4" 3) "id2" 4) "id1"
Because the order of lpush and rpush is reverse, we add a new element before the same element, and the order is still reverse;
lpop pops an element from the left
127.0.0.1:6379> lpop order1 "id3"
rpop pops an element from the right
127.0.0.1:6379> lpop order "id1"
A message queue can be implemented using rpush and lpop. (however, redis is not recommended as a message queue. After all, we only use it as a cache, and the message should be handed over to mq). There are other commands for list type data operations, such as lidnex to view the elements of the specified subscript, len to view the length of the list or the length of an element in the list, lset to modify the elements in the list, and lrem to delete the specified number of elements of the specified content in the list And so on.
//Delete the two elements with id4 in the order set and return the number of affected elements 127.0.0.1:6379> lrem order1 2 id4 (integer) 2
3.4 use of set
Cannot duplicate the set set in java. redis is the same, and the set is unordered. In addition to supporting addition, deletion, modification and query, it also supports intersection, union and difference sets of sets. In social networking, recommendation and common hobbies, you can use the set set.
sadd new element
//Add four elements to the user's set collection and return the number of affected elements 127.0.0.1:6379> sadd user zhangsan lisi wangwu zhaoliu (integer) 4
smember queries the elements in the set set. It can also be seen from the following query results that the set set is unordered
127.0.0.1:6379> smembers user 1) "wangwu" 2) "zhangsan" 3) "zhaoliu" 4) "lisi" //Insert zhangsan and lisi elements into the user set again. It is found that the return is 0, indicating that the set set is not allowed to be repeated. 127.0.0.1:6379> sadd user zhangsan lisi (integer) 0
srem delete element
//Delete the wangwu element in the user collection 127.0.0.1:6379> srem user wangwu (integer) 1
sismember determines whether an element exists in the collection
//Query whether the element zhangsan exists in the user collection, and return 1 yes 0 No 127.0.0.1:6379> sismember user zhangsan (integer) 1
spop randomly pops up a specified number of elements. This thing can be used as a lottery function
//Pop up an element at random, 127.0.0.1:6379> spop user 1 1) "lisi"
sinter finds the intersection of two sets. Below, we can see that the intersection of user set and user1 is the element zhangsan.
127.0.0.1:6379> smembers user 1) "zhangsan" 2) "zhaoliu" 127.0.0.1:6379> sadd user1 zhangsan wangwu (integer) 2 127.0.0.1:6379> smembers user1 1) "wangwu" 2) "zhangsan" 127.0.0.1:6379> sinter user user1 1) "zhangsan"
sunion finds the union of two sets, or the user1 and user elements of the above two sets remain unchanged. There are only two elements in both sets. zhangsan has both sets. The union set is three elements.
127.0.0.1:6379> sunion user user1 1) "wangwu" 2) "zhangsan" 3) "zhaoliu"
sdiff finds the difference set of two sets; Or the above two sets, user and user1. Their difference sets are zhaoliu and wangwu respectively
127.0.0.1:6379> sdiff user user1 1) "zhaoliu" 127.0.0.1:6379> sdiff user1 user 1) "wangwu"
Store get set operation, whether it is to get intersection, union or difference set. The store operation is to regenerate the extracted collection into another collection. The corresponding commands are sinterstore, sunionstore and sdiffstore
//Take out the difference set between user and user1 and put it into the usersdiff set 127.0.0.1:6379> sdiffstore usersdiff user user1 (integer) 1 127.0.0.1:6379> smembers usersdiff 1) "zhaoliu"
3.5 use of Zset set
zset is an ordered set, which is actually sorted by a score. Using zset, you can realize a function of like ranking. The following simulates the implementation of a like counting operation.
zadd adds a zset set. The following operations can be understood as recording the likes of 20180808 users.
//Create a collection with the key user:1:20180808 and add an element with zhangsan score of 0 and Lisi score of 0 127.0.0.1:6379> zadd user:1:20180808 0 zhangsan (integer) 1 127.0.0.1:6379> zadd user:1:20180808 0 lisi (integer) 1
Zincrby adds a specified score to an element; To put it bluntly, every praise is zincrby
127.0.0.1:6379> zincrby user:1:20180808 1 zhangsan "1"
zscore obtains the score of an element in the collection, that is, the number of likes of a user.
127.0.0.1:6379> zscore user:1:20180808 zhangsan "1"
zrange view ranking
127.0.0.1:6379> zrange user:1:20180808 0 -1 withscores 1) "lisi" 2) "0" 3) "zhangsan" 4) "1"
The above describes the different data types and related api operations of redis. The following describes our common redis functions.
4.Redis key management
Rename rename key
//Change the key user:1:20180808 to user:20180808 127.0.0.1:6379> rename user:1:20180808 user:20180808 OK
expire sets the expiration time of the key value. Some time limited functions can realize the validation code aging, session expiration time, etc.
127.0.0.1:6379> expire user 30 (integer) 1
Expierat sets the fixed expiration time of a key. The parameter is time stamp. Through this, the function of collecting vouchers within a limited time can be realized
127.0.0.1:6379> expireat user:1 1630481350920 (integer) 1
persist cancels the timing expiration of a key, that is, cancels the timing expiration of the key for which we have set the expiration time, and sets it as permanently valid.
127.0.0.1:6379> persist age (integer) 1
migrate key migration. This function enables data migration between multiple redis. However, it should be noted that the network between redis must be connected, that is, the bind configured by redis and the ip configured must be connected, otherwise it won't work.
//Put 192.168 0.6 copy the name key in this machine to 192.168 0.4 the timeout of the 0th Library in this machine is 1000 192.168.0.6:6379> migrate 192.168.0.4 6379 name 0 1000 copy OK
3. Summary
This article mainly introduces the installation and startup of redis, some configuration items and the use of main APIs. There are many functions of redis, such as persistence, master-slave replication, sentinel mechanism to achieve high availability, and ensure the failure and automatic transfer of redis nodes. The Java api of redis is not introduced here. In fact, it is no different from using redis cli. To put it bluntly, it just encapsulates the above commands into methods. That's all for the introduction of this article. Later, we will introduce some other redis functions and specific application scenarios in detail.