Database: redis

1, Introduction to Redis

   NoSQL (Not Only SQL), which means Not Only SQL, but also non relational database.

  with the development of internet web2 With the rise of web2.0 website, the traditional relational database is dealing with web2.0 0 websites, especially the super large-scale and highly concurrent SNS type web2 0 pure dynamic website has been unable to meet its needs, exposing many insurmountable problems, while non relational database has developed very rapidly due to its own characteristics. The generation of NoSQL database is to solve the challenges brought by large-scale data collection and multiple data types, especially the problems of big data application, including the storage of large-scale data. (for example, Google or Facebook collects trillions of bits of data for their users every day). These types of data storage do not need a fixed mode and can be expanded horizontally without redundant operations.

  NoSQL representatives: MongDB, Redis, Memcache

   Redis is an open source log and key value database written in ANSIC language, supporting network, memory based and persistent, and provides API s in multiple languages. Since March 15, 2010, the development of Redis has been hosted by VMware.

Redis is a key value storage system. Five data types are supported: string, hash, list, set and zset. These data types support push/pop, add/remove, intersection, union, difference and richer operations, and these operations are atomic. On this basis, redis supports various sorting methods. Like memcached, data is cached in memory to ensure efficiency. The difference is that redis will periodically write the updated data to the disk or write the modification operation to the additional record file, and on this basis, it realizes master-slave synchronization.

  redis is a high-performance key value database. The emergence of redis largely compensates for the shortage of keyvalue storage such as memcached, and can play a good supplementary role to relational databases on some occasions. It provides Java, Python, Ruby, Erlang and PHP clients, which is very convenient to use.

   all data of redis is saved in memory and then saved to disk asynchronously from time to time (this is called "semi persistent mode"); You can also write every data change to an append only file (AOF) (this is called "full persistence mode"). Since the data of redis is stored in memory, if persistence is not configured, all the data will be lost after redis is restarted. Therefore, it is necessary to turn on the persistence function of redis to save the data to the disk. After redis is restarted, the data can be recovered from the disk. Redis provides two methods for persistence. One is RDB persistence (the principle is to periodically dump the database records of Reids in memory to the RDB persistence on disk), and the other is AOF (append only file) persistence (the principle is to write the operation logs of Reids to the file in the form of append).

2, Install redis service

1. Use the Xiaopi system to install and start the redis service.

You can use RedisClient or redis cli Exe to view the data in the database.

Common Redis commands:

ping  Is the test database connected
select 6   Select the 6th database
set  a 15  with a Save a value of 15 for the key
get a   obtain a Value of
keys * View all keys in the library
flushdb Empty the current database
flushall Empty all databases

Create a new Java project based on maven. And import the redis connection dependency.


Create a Java class and add the main method test

public static void main(String[] args) {
    Jedis jedis = new Jedis("", 6379);

When the return value is PONG, the database connection is successful. For faster performance, connections are generally obtained through the connection pool.

3, Use connection pool

The following connection pool uses the hungry man mode in the singleton mode. The relevant configuration is limited to jedis-2.1.0, and the configuration of other versions is different.

import redis.clients.jedis.Jedis;
import redis.clients.jedis.JedisPool;
import redis.clients.jedis.JedisPoolConfig;

public class JedisPoolUtil {
    private static volatile JedisPool jedisPool = null;

    private JedisPoolUtil() {

    public static Jedis getJedisInstance() {
        if (null == jedisPool) {
            synchronized (JedisPoolUtil.class) {
                JedisPoolConfig poolConfig = new JedisPoolConfig();
                //The maximum number of idle connections is 8 by default
                //Gets the maximum number of milliseconds to wait for a connection. If it times out, throw an exception. Less than zero: the blocking time is uncertain. The default is - 1
                poolConfig.setMaxWaitMillis(100 * 1000);
                //Check the validity when obtaining the connection. The default is false
                jedisPool = new JedisPool(poolConfig, "", 6379);
        return jedisPool.getResource();

    public static void release(Jedis jedis) {
        if (null != jedis) {


Get redis connection from connection pool when using

Jedis jedis = JedisPoolUtil.getJedisInstance();

4, String type operation

redis is a key value storage system. Five data types are supported: string, hash, list, set and zset.
The string in Redis is a sequence of bytes. Strings in Redis are binary safe, which means that their length is not determined by any special termination character. Therefore, anything up to 512 megabytes can be stored in a string.

public static void main(String[] args) {
    Jedis jedis = new Jedis( "", 6379 ); 0 );//Select the database for the current operation
    System.out.println( jedis.keys( "*" ) );//View all keys in the current database
    jedis.set( "hello", "helloword" );//Store data
    jedis.append( "hello", " haiwen" );//Add
    System.out.println( jedis.get( "hello" ) );//Fetch data
    jedis.del( "user" );//Delete key
    jedis.mset( "bb","bb1","cc","cc1");
    //Store multiple key value pairs at one time (it must be an even number. If it is an odd number, it will throw ERR wrong number of arguments for MSET)
    //jedis.incr( "hello" );
    //jedis.incrBy( "hello",50 );
    //Add 1 to the value of aa (the modified object must be a number. If it is a character, throw ERR value is not an integer or out of range)
    //jedis.decr( "aa" );// Subtract the value of AA by 1
    jedis.setex( "ff",12,"From the president of the Republic" );//Set the survival time of the key value pair to 12 seconds
    jedis.flushDB();//Delete all data in the current database
    jedis.flushAll();//Delete all data in the database

5, List type operation

Redis list is just a string list, sorted by insertion order. You can add elements to the head or tail of redis list.

public static void main(String[] args) {

    Jedis jedis = new Jedis("", 6379);
    //jedis.lpush( "list", "Java" );
    //jedis.lpush( "list", "UI" );
    //jedis. Lpush ("list", "graphic design")// Press an object into the top of the list
    //String list = jedis.lpop( "list" );// Take an object from the top, and there will be one less thing in it after taking it
    //The first is the key, the second is the start position, and the third is the end position, jedis Len get length - 1 means get all
    List<String> list = jedis.lrange("list", 0, -1);//Check the values in the list
    for (String s : list) {
    System.out.println(jedis.llen("list"));//Get length
    // jedis. Rpush ("list", "last")// Insert data at the bottom of the stack
    //jedis.lset( "list" ,2,"aaa" );// Set the value with index 2 under the list key (overwrite if it exists)
    jedis.lrem("list", 2, "aaa");
    //Delete key value pairs with list key and aaa value. The number of deleted pairs is 2. If the number of deleted pairs is 0, all pairs will be deleted. After deletion, the indexes inside are sorted automatically


6, Hash type operation

Redis Hashes are a collection of key value pairs. Redis hash / hash is the mapping between string fields and string values. Therefore, you can think of Hashes as objects.

public static void main(String[] args) {
    Jedis jedis = new Jedis("", 6379);

    Map<String, String> map = new HashMap<>();
    map.put("name", "xinxin");
    map.put("age", "22");
    map.put("qq", "123456");
    jedis.hmset("user", map);

    //The delete key is the subkey in user, and there can be multiple subkeys
    //jedis.hdel( "user","qq","..." );

    //The first parameter is the key of the whole map, and the latter parameter is the key in the map. You can write whatever value you need,
    // If the corresponding attribute is not found, the position is empty
    List<String> hmget = jedis.hmget("user", "qq", "age", "name", "aa");

    System.out.println(jedis.hlen("user")); // Returns the number of values stored in the key whose key is user. 2
    System.out.println(jedis.exists("user"));// Whether there is a record whose key is user returns true
    System.out.println(jedis.hkeys("user"));// Returns all key s in the map object
    System.out.println(jedis.hvals("user"));// Returns all value s in the map object

7, Collection type operation

Redis collection is an unordered collection of strings. In redis, you can add, delete and test the time O(1) complexity of members.

public static void main(String[] args) {

    Jedis jedis = new Jedis( "", 6379 );
    jedis.sadd( "user", "liuling" );
    jedis.sadd( "user", "xinxin" );
    jedis.sadd( "user", "ling" );
    jedis.sadd( "user", "zhangxinxin" );
    jedis.sadd( "user", "who" );
    jedis.sadd( "user", "who" );//Duplicate is not allowed, and sorting
    Set<String> user = jedis.smembers( "user" );
    for (String s : user) {
        System.out.println( s );
    jedis.srem( "user", "who" );// remove
    System.out.println( jedis.sismember( "user", "who" ) );// Determine whether who is an element of the user collection
    System.out.println( jedis.scard( "user" ) );// Returns the number of elements in the collection

8, Sortable set operation

Redis sortable sets are similar to redis sets and are non duplicate character sets. The difference is that each member of the sort set is associated with a score, which is used to sort the sort set from the minimum score to the maximum score. Although the member is unique, the score value can be repeated.

public static void main(String[] args) {
    Jedis jedis = new Jedis( "", 6379 );
    jedis.zadd( "hackers", 1940, "Alan Kay" );
    jedis.zadd( "hackers", 1953, "Richard Stallman" );
    jedis.zadd( "hackers", 1965, "Yukihiro Matsumoto" );
    jedis.zadd( "hackers", 1916, "Claude Shannon" );
    jedis.zadd( "hackers", 1969, "Linus Torvalds" );
    jedis.zadd( "hackers", 1912, "Alan Turing" );
    Set<String> setValues = jedis.zrange( "hackers", 0, -1 );
   System.out.println( setValues );
    Set<String> setValues2 = jedis.zrevrange( "hackers", 0, -1 );
   System.out.println( setValues2 );

    // wipe data 
    System.out.println( jedis.flushDB() );
    // Add data
    jedis.zadd( "zset", 10.1, "hello" );
    jedis.zadd( "zset", 10.0, ":" );
    jedis.zadd( "zset", 9.0, "zset*" );
    jedis.zadd( "zset", 11.0, "zset!" );
    // Number of elements
    System.out.println( jedis.zcard( "zset" ) );
    // Element subscript
    System.out.println( jedis.zscore( "zset", "zset*" ) );
    // Set subset
    System.out.println( jedis.zrange( "zset", 0, -1 ) );
    // Delete element
    System.out.println( jedis.zrem( "zset", "zset!" ) );
    System.out.println( jedis.zcount( "zset", 9.5, 10.5 ) );
    // Entire set value
    System.out.println( jedis.zrange( "zset", 0, -1 ) );

9, List sorting

In development, we often use Jedis to sort the list.

public static void main(String[] args) {

    Jedis jedis = new Jedis( "", 6379 );
    jedis.rpush("a", "1");
    System.out.println(jedis.lrange("a",0,-1));// [9, 3, 6, 1]
    System.out.println(jedis.sort("a")); //[1, 3, 6, 9] / / enter the sorted result
    System.out.println(jedis.lrange("a",0,-1));//[9, 3, 6, 1]


10, Transactions and locks

Multiple redis operations can also support transactions.

public static void main(String[] args) {

    Jedis jedis = new Jedis( "", 6379 );
    Transaction transaction = jedis.multi();
    try {
        transaction.set("aa", "aa12");
        int i= 1/0;
        transaction.set("bb", "bb12");
    } catch (Exception e) {

redis supports optimistic locking mechanism by default

public boolean transMethod() throws InterruptedException {
    Jedis jedis = new Jedis("", 6379);
    int balance;// Available balance
    int debt;// bill
    int amtToSubtract = 10;// Actual amount"balance");
    // jedis.set("balance","5");// This sentence should not appear. It is convenient to give a lecture. Simulate that other programs have modified this entry
    balance = Integer.parseInt(jedis.get("balance"));		
    if (balance < amtToSubtract) {
        System.out.println("Sorry, your credit is running low");
        return false;
    } else {
        Transaction transaction = jedis.multi();
        transaction.decrBy("balance", amtToSubtract);
        transaction.incrBy("debt", amtToSubtract);
        if (transaction.exec() == null) {
            System.out.println("Modified midway");
        } else {
            System.out.println("Successful execution");

        balance = Integer.parseInt(jedis.get("balance"));
        debt = Integer.parseInt(jedis.get("debt"));
        System.out.println("*******" + balance);
        System.out.println("*******" + debt);
        return true;

 * Generally speaking, the watch command is to mark a key. If a key is marked, if the key is modified by others before committing the transaction, the transaction will fail. In this case, you can usually try again in the program.
 * First, mark the key balance, and then check whether the balance is sufficient. If it is insufficient, cancel the mark and make no deduction; If enough, start the transaction for update operation,
 * If the key balance is modified by others during this period, it will not be modified when exec is executed. Usually, such errors can be caught in the program and executed again until it succeeds.
public static void main(String[] args) throws InterruptedException {
    TestWatch test = new TestWatch();
    boolean retValue = test.transMethod();
    System.out.println("main retValue-------: " + retValue);

11, Integration with spring

Add the host and port that Redis depends on and configures jedis

<bean id="jedis" class="redis.clients.jedis.Jedis">
    <constructor-arg name="host" value=""></constructor-arg>
    <constructor-arg name="port" value="6379"></constructor-arg>

Get Jedis from the spring container or inject it directly

public static void main(String[] args) {
    ApplicationContext applicationContext = new ClassPathXmlApplicationContext( "spring-context.xml" );
    Jedis bean = applicationContext.getBean(Jedis.class);

12, Cache avalanche

For system A, assuming 5000 requests per second in the peak period every day, the cache could have carried 4000 requests per second in the peak period, but the cache machine unexpectedly went down completely. The cache hangs. At this time, all 5000 requests in one second fall into the database. The database will not be able to carry them. It will call the police and then hang up. At this time, if no special scheme is adopted to deal with the fault, the DBA is very anxious and restarts the database, but the database is immediately killed by new traffic.

This is cache avalanche.

About three years ago, a well-known Internet company in China had an avalanche due to a cache accident, and all the background systems collapsed. The accident lasted from that afternoon to 3 ~ 4 a.m. in the evening, and the company lost tens of millions. The solutions of cache avalanche are as follows.

In advance: redis is highly available, master-slave + sentinel, and redis cluster to avoid total collapse.
In fact: local ehcache cache + hystrix current limiting & degradation to avoid MySQL being killed.
Afterwards: redis is persistent. Once restarted, it will automatically load data from the disk and quickly recover cached data.

The user sends A request. After receiving the request, system A first checks the local ehcache cache, and then checks redis if it is not found. If neither ehcache nor redis exists, check the database again and write the results in the database into ehcache and redis.

The current limiting component can set the requests per second, how many can pass the component, and what about the remaining failed requests? Let's go! You can return some default values, or friendly tips, or blank values.

The database will never die. The current limiting component ensures that only how many requests can pass per second.
As long as the database does not die, that is, for users, 2 / 5 of the requests can be processed.
As long as 2 / 5 of the requests can be processed, it means that your system is not dead. For users, it may be that they can't swipe the page after clicking several times, but they can swipe it once more.

Cache penetration

For system A, assume 5000 requests per second, and 4000 of them are malicious attacks by hackers.
The 4000 attacks sent by hackers can't be found in the cache. Every time you go to the database, you can't find them.
Take a chestnut. The database id starts from 1. As a result, all the request IDS sent by hackers are negative numbers. In this way, there will be no requests in the cache. Each time, the request will "treat the cache as nothing" and query the database directly. The cache penetration of this malicious attack scenario will directly kill the database.

The solution is very simple. Every time system A does not find it in the database, it writes A null value to the cache, such as set -999 UNKNOWN. Then set an expiration time, so that the next time the same key is accessed, you can directly get data from the cache before the cache expires.

Buffer breakdown

Cache breakdown means that a key is very hot and frequently accessed. It is in a centralized and highly concurrent access situation. When the key fails, a large number of requests break through the cache and directly request the database, which is like cutting a hole in a barrier.

The solution is also very simple. You can set the hotspot data to never expire; Or implement the mutex lock based on redis or zookeeper, wait for the first request to build the cache, and then release the lock, so that other requests can access data through the key.

13, Install redis cluster (from)

Redis cluster management tool redis trib RB relies on the ruby environment

First, you need to install the ruby environment: install ruby

yum install ruby 
yum install rubygems

Install ruby and redis interface programs

Copy redis-3.0.0 Gem to / usr/local:

gem install /usr/local/redis-3.0.0.gem

Create the redis cluster directory under / usr/local, and create 7001 and 7002 under it.. 7006 catalog

6 copies of the installed redis

Copy the files in the redis installation directory bin to each 700X directory, and redis trib. In the redis source code directory src Copy RB to the redis cluster directory.

Modify each 700 X Directory redis.conf Profile:
port 700X
cluster-enabled yes

Start each redis

Enter 7001, 7002,... 7006 directories respectively and execute:

./redis-server ./redis.conf

After all nodes are started, check the startup of all nodes

ps aux|grep redis
root       1281  0.0  0.9 140888  9544 ?        Ssl  20:26   0:00 ./redis-server *:7001 [cluster]
root       1288  0.0  0.9 140888  9228 ?        Ssl  20:26   0:00 ./redis-server *:7002 [cluster]
root       1295  0.0  0.9 140888  9548 ?        Ssl  20:26   0:00 ./redis-server *:7003 [cluster]
root       1302  0.0  0.9 140888  9536 ?        Ssl  20:26   0:00 ./redis-server *:7004 [cluster]
root       1309  0.0  0.9 140888  9528 ?        Ssl  20:26   0:00 ./redis-server *:7005 [cluster]
root       1316  0.0  0.9 140888  9380 ?        Ssl  20:26   0:00 ./redis-server *:7006 [cluster]

Execute cluster command

./redis-trib.rb create --replicas 1

If you execute the cluster command for the second time, you need to delete the previously generated data

cd 7001/bin/
rm -rf *.aof
rm -rf *.rdb
rm -rf nodes.conf
./redis-server ./redis.conf
cd ..
cd ..
cd 7002/bin/
rm -rf *.aof
rm -rf *.rdb
rm -rf nodes.conf
./redis-server ./redis.conf
cd ..
cd ..
cd 7003/bin/
rm -rf *.aof
rm -rf *.rdb
rm -rf nodes.conf
./redis-server ./redis.conf
cd ..
cd ..
cd 7004/bin/
rm -rf *.aof
rm -rf *.rdb
rm -rf nodes.conf
./redis-server ./redis.conf
cd ..
cd ..
cd 7005/bin/
rm -rf *.aof
rm -rf *.rdb
rm -rf nodes.conf
./redis-server ./redis.conf
cd ..
cd ..
cd 7006/bin/
rm -rf *.aof
rm -rf *.rdb
rm -rf nodes.conf
./redis-server ./redis.conf
cd ..
cd ..
./redis-trib.rb create --replicas 1

When you see

M: 9ead02ced1e47fc92b2d43f39eadaaeebe110664
   slots:0-5460 (5461 slots) master
M: c4590e1a1428a363545a29b9910b880791630416
   slots:5461-10922 (5462 slots) master
M: fd4abd8efab2864cd1e0066843e10f96161dc289
   slots:10923-16383 (5461 slots) master
S: b3ec510d2859203fb2ceabba35fc3d38207847a6
   replicates 9ead02ced1e47fc92b2d43f39eadaaeebe110664
S: a195de8609128e4bcd0d2c5bc236cf79935748f3
   replicates c4590e1a1428a363545a29b9910b880791630416
S: fb8c46bca7e4d9b13a75dfa81f1448953087b6f0
   replicates fd4abd8efab2864cd1e0066843e10f96161dc289
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join..
>>> Performing Cluster Check (using node
M: 9ead02ced1e47fc92b2d43f39eadaaeebe110664
   slots:0-5460 (5461 slots) master
M: c4590e1a1428a363545a29b9910b880791630416
   slots:5461-10922 (5462 slots) master
M: fd4abd8efab2864cd1e0066843e10f96161dc289
   slots:10923-16383 (5461 slots) master
M: b3ec510d2859203fb2ceabba35fc3d38207847a6
   slots: (0 slots) master
   replicates 9ead02ced1e47fc92b2d43f39eadaaeebe110664
M: a195de8609128e4bcd0d2c5bc236cf79935748f3
   slots: (0 slots) master
   replicates c4590e1a1428a363545a29b9910b880791630416
M: fb8c46bca7e4d9b13a75dfa81f1448953087b6f0
   slots: (0 slots) master
   replicates fd4abd8efab2864cd1e0066843e10f96161dc289
[OK] All nodes agree about slots configuration.

Indicates that the cluster is started successfully

redis cluster clear cache

redis-cli -c -h -p 3433 -a redis_2018

redis_2018 database password (all passwords on the cluster need to be cleared)

14, redis master-slave replication

Due to the short reading interval, unreadability may occur for the first time

public static void main(String[] args) {
	Jedis jedis_M = new Jedis("",6379);
	Jedis jedis_S = new Jedis("",6380);	
	String result = jedis_S.get("class");

Keywords: Database Redis nosql memcached

Added by witty on Fri, 25 Feb 2022 04:21:39 +0200