Basics: introduction to distributed locks

catalogue

Fundamentals: distributed locks

Why do I need distributed locks?

Characteristics of distributed locks

Common scenarios

Redis implements distributed locks

Redisson distributed lock entry practice

reference

Fundamentals: distributed locks

The synchronized keyword and ReentrantLock reentrant lock in Java can control concurrent access to resources in a multi-threaded environment, but they are JVM level local locks that will fail in distributed scenarios. Distributed locking can avoid different nodes from repeating the same work, so as to avoid multiple nodes operating on data at the same time and affecting the correctness.

1. If a scheduled task needs to be executed under cluster deployment and there is no distributed scheduling center such as xxljob, an exception will occur if the task is executed on multiple servers at the same time.

2. If the user places an order on the e-commerce platform and clicks twice continuously due to network problems, how should the back end respond? Both the front and back ends need to be processed, and the back end needs to consider the problem of processing duplicate orders to ensure idempotency.

The distributed lock solves the problem of resource competition when multiple machines are deployed and the same request is accessed concurrently.

Why do I need distributed locks?

If we need a function to count the number of logins, what should we do?

A global counter can be designed. Each time a user logs in to onlineCount + +, but this operation is not atomic. It will be compiled into onlineCount = onlineCount +1. In fact, the following three steps are done:

Read memory to register: read the value of onlineCount from memory to the CPU register;
Auto increment in register: perform the operation of onlineCount + 1 in the register of CPU;

Write data back to memory: write the calculated value in the register back to memory.

If two clients execute this function at the same time, client 1 and client 2 first read the value of onlineCount from memory to the CPU register, assuming that they are both 0, and then both clients operate + 1 in the register to become 1. Finally, client 1 writes the self incremented data 1 back to memory, and client 2 also writes the self incremented data 1 back to memory. Is it normal? Not normal. The correct result should be 2, but the result is 1.

Lock can ensure the data security of competing for the same resource in multi-threaded environment. Distributed lock is a means to ensure the security of multi-threaded concurrent threads in the whole cluster. In the distributed cluster environment, it is necessary to use distributed locks to ensure the thread safety of the whole cluster;

Characteristics of distributed locks

Distributed locks must have four characteristics: high performance, reentrant, anti deadlock and mutual exclusion;

High performance: the performance of distributed locks must be high. If the time-consuming process of unlocking and unlocking will affect the performance of the system in the scenario of high concurrency seckill;
Reentrant: if the thread holding the lock is the current thread, it should execute logic normally instead of blocking and waiting;

Deadlock prevention: client 1 is locked and has not been released in time. The service is down. How can I release this lock? If it is not released, other clients will never execute this method. When the machine is restored, client 1 cannot execute it, which is also a manifestation of deadlock;
Mutex: multiple clients are not allowed to execute a piece of code at the same time;

Common scenarios

Prevent cache breakdown: add a distributed lock when checking the database, and then cache the data to Redis, so as to ensure that when large traffic requests come in, even if Redis fails, only one request will hit the database at the same time;
Ensure the idempotency of the interface. Take the simplest example: the form is submitted repeatedly. After clicking submit, it has not been processed. Click the submit button again. At this time, it is equivalent to submitting two copies. Distributed locks can also solve this scenario. A distributed lock is added to the submission interface. It will be locked during the first submission, and then the business logic will not be executed if a lock is found during the second submission;

Task scheduling: use distributed locks to ensure that only one machine is working when the cluster is deployed, and other machines will not execute when they find locks;
Second kill inventory reduction and other similar businesses to prevent oversold

Redis implements distributed locks

Redis, as a distributed lock, has four key points: atomicity, expiration time, lock renewal and correct release of locks;

1. Atomicity - both success and failure

The core of implementing distributed locks is to find a visible place in the cluster to store the locks. At the same time, it is also necessary to consider the problem of deadlock and set the expiration time for the locks. Then, only two commands are required in redis:

redis.set(key,value);
redis.expire(key,time,unit);

redis supports lua scripts. You can use lua scripts for two commands to ensure atomicity;

2. Expiration time

The expiration time is lofty to avoid deadlock and service downtime, which can not be released all the time;

3. Lock renewal

Suppose the lock expiration time is set to 3s, but the business code has not been executed for 4s, then the lock is automatically released after expiration. When other threads find that there is no lock at present when requesting the interface, they add the lock. At this time, won't the two clients execute concurrently? Equivalent to or thread unsafe. Open up another thread for lock renewal. When locking, start a thread for endless cycle renewal. The core process is to judge that the lock time is over one third and renew it as the lock time. For example, if the set lock is 3s and the check has not been executed for more than 1s, renew the lock for 3s to prevent the lock from expiring before the method is executed.

4. Release the lock correctly

The lock renewal mechanism is used to prevent the lock from expiring and being released in advance before the business is completed. When releasing the lock, it is also necessary to judge whether the lock is added by yourself, so as to avoid releasing the lock of others by mistake.

Redisson distributed lock entry practice

Reisson official address

As we all know, ReentrantLock has good Lock performance and implementation. It has good performance and implementation in mutually exclusive, reentrant, Lock timeout, blocking support and fair Lock support, but it is not suitable for distributed scenarios. Reisson is a distributed Lock to make up for this deficiency (there are many distributed locks, and others are not discussed here). RLock interface inherits the Lock interface and naturally and elegantly implements the above Lock requirements.

1. Introduce related dependencies into the project

        <dependency>
            <groupId>org.apache.commons</groupId>
            <artifactId>commons-pool2</artifactId>
            <version>2.9.0</version>
        </dependency>
        <dependency>
            <groupId>org.redisson</groupId>
            <artifactId>redisson</artifactId>
            <version>3.13.1</version>
        </dependency>

2. Add the redis configuration in the configuration file

spring:
  redis:
    database: 0
    host: Your hostname
    password: Your password
    port: 6379
    lettuce:
      pool:
        max-active: 100
        max-wait: -1
        max-idle: 8
        min-idle: 0

3. RedissonConfig configuration

import lombok.extern.slf4j.Slf4j;
import org.redisson.Redisson;
import org.redisson.api.RedissonClient;
import org.redisson.config.Config;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;


@Slf4j
@Configuration
public class RedissonConfig {


    @Value("${spring.redis.password}")
    private String password;

    @Value("${spring.redis.host}")
    private String host;

    @Value("${spring.redis.port}")
    private String port;

    @Value("${spring.redis.database}")
    private int database;

    @Bean
    public RedissonClient redissonClient() {
        Config config = new Config();
        String REDISSON_PREFIX = "redis://";
        String url = REDISSON_PREFIX + host + ":" + port;
        // Single Redis
        config.useSingleServer()
                .setAddress(url)
                .setPassword(password)
                .setDatabase(database);
        try {
            return Redisson.create(config);
        } catch (Exception e) {
            log.error("RedissonClient init redis url:[{}], Exception:", url, e);
            return null;
        }
    }
}

4,DistributedRedisLock

import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;
import org.redisson.api.RLock;
import org.redisson.api.RedissonClient;

import java.util.concurrent.TimeUnit;


@Slf4j
@Component
public class DistributedRedisLock {


    @Autowired
    RedissonClient redissonClient;

    public Boolean lock(String lockName){
        if(null == redissonClient){
            log.info("DistributedRedisLock redissonClient is null");
            return false;
        }
        try{
            RLock lock = redissonClient.getLock(lockName);
            lock.lock(10,TimeUnit.SECONDS);
            log.info("Thread [{}] DistributedRedisLock lock [{}] success Lock succeeded", Thread.currentThread().getName(), lockName);
            return true;
        } catch (Exception e){
            log.error("DistributedRedisLock lock [{}] Exception:", lockName, e);
            return false;
        }
    }

    public Boolean unlock(String lockName){
        if (redissonClient == null) {
            log.info("DistributedRedisLock redissonClient is null");
            return false;
        }
        try{
            RLock lock = redissonClient.getLock(lockName);
            lock.unlock();
            log.info("Thread [{}] DistributedRedisLock unlock [{}] success Unlock", Thread.currentThread().getName(), lockName);
            return true;
        } catch (Exception e){
            log.error("DistributedRedisLock unlock [{}] Exception:", lockName, e);
            return false;
        }
    }
}

5,LockController

import com.example.lock.util.DistributedRedisLock;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

import java.util.concurrent.BrokenBarrierException;
import java.util.concurrent.CyclicBarrier;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicInteger;


@Slf4j
@RestController
@RequestMapping("/lock")
public class LockController {

    @Autowired
    DistributedRedisLock distributedRedisLock;

    AtomicInteger ID = new AtomicInteger(0);
    AtomicInteger ID1 = new AtomicInteger(0);

    // Test Do Not Release Lock
    @GetMapping("/testLock")
    public void testLock() {
        CyclicBarrier cyclicBarrier = new CyclicBarrier(5);
        for (int i = 0; i < 5; i++) {
            new Thread(() -> {
//                distributedRedisLock.lock(LOCK);
                try {
                    System.out.println(ID.addAndGet(1)+"Enter Wait");
                    cyclicBarrier.await();
                    System.out.println("Start execution");
                    post();
                } catch (InterruptedException e) {
                    e.printStackTrace();
                } catch (BrokenBarrierException e) {
                    e.printStackTrace();
                }
            }).start();
        }
    }

    // Test Do Not Release Lock
    @GetMapping("/testLock1")
    public void testLock1() {
        CyclicBarrier cyclicBarrier = new CyclicBarrier(5);
        for (int i = 0; i < 5; i++) {
            new Thread(() -> {
//                distributedRedisLock.lock(LOCK);
                try {
                    System.out.println(ID1.addAndGet(1)+"Enter Wait");
                    cyclicBarrier.await();
                    System.out.println("Start execution");
                    post1();
                } catch (InterruptedException e) {
                    e.printStackTrace();
                } catch (BrokenBarrierException e) {
                    e.printStackTrace();
                }
            }).start();
        }
    }

    // How distributed locks are used in real-world business development
    public void post() throws InterruptedException {
        final String LOCK = "LOCK2LOCK";
        try {
            if (distributedRedisLock.lock(LOCK)) {
                log.info("No. e Two ready to start business logic");
                TimeUnit.SECONDS.sleep(1);
                // Business logic
                log.info("No. e Two Start Business Logics");
                TimeUnit.SECONDS.sleep(1);
            } else {
                // Handling logic that failed to acquire locks
                log.info("Failed to acquire lock");
            }
        } catch (Exception e) {
            log.error("Handle exceptions:", e);
        } finally {
            distributedRedisLock.unlock(LOCK);
            TimeUnit.SECONDS.sleep(1);
        }
    }


    // How distributed locks are used in real-world business development
    public void post1() throws InterruptedException {
        final String LOCK = "LOCK1LOCK";
        try {
            if (distributedRedisLock.lock(LOCK)) {
                // Business logic
                log.info("First Start Business Logic");
                TimeUnit.SECONDS.sleep(1);
            } else {
                // Handling logic that failed to acquire locks
                log.info("Failed to acquire lock");
            }
        } catch (Exception e) {
            log.error("Handle exceptions:", e);
        } finally {
            distributedRedisLock.unlock(LOCK);
            TimeUnit.SECONDS.sleep(1);
        }
    }

}