There are three ways to implement the distributed lock: 1. Optimistic database lock; 2. Redis based distributed lock; 3. ZooKeeper based distributed lock. This blog will introduce the second way to implement distributed locks based on redis. Although there are various blogs about the implementation of redis distributed lock on the Internet, their implementation has various problems. In order to avoid misusing children, this blog will detail how to correctly implement redis distributed lock.
reliability
First of all, in order to ensure the availability of distributed locks, we need to ensure that the implementation of locks meets the following four conditions at the same time:
- Mutual exclusion. At any time, only one client can hold the lock.
- No deadlock will occur. Even if one client crashes during the lock holding period without active unlocking, it can ensure that other clients can lock later.
- Fault tolerance. As long as most Redis nodes operate normally, the client can be locked and unlocked.
- You have to tie the bell. Locking and unlocking must be the same client. The client cannot unlock the lock added by others.
code implementation
Component dependency
First, we need to introduce the Jedis open source component through Maven, and add the following code in pom.xml file:
<dependency>
<groupId>redis.clients</groupId>
<artifactId>jedis</artifactId>
<version>2.9.0</version>
</dependency>
Lock code
Correct posture
Talk is cheap, show me the code. First, show the code, and then slowly explain why it is implemented like this:
public class RedisTool {
private static final String LOCK_SUCCESS = "OK";
private static final String SET_IF_NOT_EXIST = "NX";
private static final String SET_WITH_EXPIRE_TIME = "PX";
/**
*Try to get distributed lock
*@ param jedis Redis client
*@ param lockKey lock
*@ param requestId request ID
*@ param expireTime
*@ return success
*/
public static boolean tryGetDistributedLock(Jedis jedis, String lockKey, String requestId, int expireTime) {
String result = jedis.set(lockKey, requestId, SET_IF_NOT_EXIST, SET_WITH_EXPIRE_TIME, expireTime);
if (LOCK_SUCCESS.equals(result)) {
return true;
}
return false;
}
}
As you can see, we lock with one line of code: jedis.set(String key, String value, String nxxx, String expx, int time). This set() method has five parameters in total:
-
The first one is the key. We use the key as the lock, because the key is unique.
-
The second one is value. We pass the requestId. Many children's shoes may not understand that having a key as a lock is enough. Why use value? The reason is that when we talk about reliability, the fourth condition that a distributed lock needs to be unlocked must be solved by the ringer. By assigning value to requestId, we can know which request this lock is added to. When unlocking, we can have a basis. The requestId can be generated using the UUID.randomUUID().toString() method.
-
The third parameter is nxxx, which means SET IF NOT EXIST, that is, when the key does not exist, we perform set operation; if the key already exists, we do not perform any operation;
-
The fourth parameter is expx, which is PX, which means that we need to add an expired setting to the key. The specific time is determined by the fifth parameter.
-
The fifth is time, which corresponds to the fourth parameter and represents the expiration time of the key.
In general, executing the set() method above will only lead to two results: 1. There is no lock at present (the key does not exist), then perform the lock operation, and set a validity period for the lock. At the same time, value represents the locked client. 2. The existing lock exists and no operation is performed.
The careful children's shoes will find that our lock code meets the three conditions described in our reliability. First of all, set() adds NX parameter to ensure that if the existing key exists, the function will not be called successfully, that is, only one client can hold the lock to meet the mutual exclusion. Secondly, because we set the expiration time for the lock, even if the lock holder subsequently crashes and does not unlock, the lock will automatically unlock (that is, the key is deleted) because it reaches the expiration time, and no deadlock will occur. Finally, because we assign value to requestId, which represents the request ID of the locked client, when the client is unlocked, we can verify whether it is the same client. Because we only consider the scenario of Redis single machine deployment, we do not consider the fault tolerance temporarily.
Error example 1
A common error example is to use the combination of jedis.setnx() and jedis.expire() to implement locking. The code is as follows:
public static void wrongGetLock1(Jedis jedis, String lockKey, String requestId, int expireTime) {
Long result = jedis.setnx(lockKey, requestId);
if (result == 1) {
//If the program crashes suddenly here, the expiration time cannot be set, and a deadlock will occur
jedis.expire(lockKey, expireTime);
}
}
The setnx() method is used to SET IF NOT EXIST, and the expire() method is used to add an expiration time to the lock. At first glance, it seems to be the same as the result of the previous set() method. However, since these are two Redis commands, they are not atomic. If the program suddenly crashes after the execution of setnx(), the lock does not have an expiration time set. Then a deadlock will occur. The reason why it is implemented on the Internet is that the lower version of jedis does not support the set () method with multiple parameters.
Error example 2
public static boolean wrongGetLock2(Jedis jedis, String lockKey, int expireTime) {
long expires = System.currentTimeMillis() + expireTime;
String expiresStr = String.valueOf(expires);
//If the current lock does not exist, the lock is successfully added
if (jedis.setnx(lockKey, expiresStr) == 1) {
return true;
}
//If the lock exists, get the expiration time of the lock
String currentValueStr = jedis.get(lockKey);
if (currentValueStr != null && Long.parseLong(currentValueStr) < System.currentTimeMillis()) {
//The lock has expired. Obtain the expiration time of the previous lock and set the expiration time of the current lock
String oldValueStr = jedis.getSet(lockKey, expiresStr);
if (oldValueStr != null && oldValueStr.equals(currentValueStr)) {
//Considering the situation of multithreading concurrency, only when the setting value of a thread is the same as the current value, can it have the right to lock
return true;
}
}
//In other cases, all return to lock failure
return false;
}
This kind of error example is more difficult to find problems, and the implementation is also more complex. Implementation idea: use the jedis.setnx() command to implement locking, where key is the lock and value is the expiration time of the lock. Execution process: 1. Try to lock through the setnx() method. If the current lock does not exist, the lock is returned successfully. 2. If the lock already exists, obtain the expiration time of the lock, and compare it with the current time. If the lock has expired, set a new expiration time, and return to lock successfully. The code is as follows:
So what's the problem with this code? 1. Because the client generates the expiration time by itself, the time of each client in the distributed system must be synchronized. 2. When the lock expires, if multiple clients execute the jedis.getSet() method at the same time, only one client can lock at last, but the expiration time of this client's lock may be overwritten by other clients. 3. The lock does not have the owner identity, that is, any client can be unlocked.
unlock code
Correct posture
Let's show the code first, and then explain why it is implemented in this way
public class RedisTool {
private static final Long RELEASE_SUCCESS = 1L;
/**
*Release distributed lock
*@ param jedis Redis client
*@ param lockKey lock
*@ param requestId request ID
*@ return whether to release successfully
*/
public static boolean releaseDistributedLock(Jedis jedis, String lockKey, String requestId) {
String script = "if redis.call('get', KEYS[1]) == ARGV[1] then return redis.call('del', KEYS[1]) else return 0 end";
Object result = jedis.eval(script, Collections.singletonList(lockKey), Collections.singletonList(requestId));
if (RELEASE_SUCCESS.equals(result)) {
return true;
}
return false;
}
}
As you can see, we only need two lines of code to unlock! The first line of code, we wrote a simple Lua script code. The last time I saw this programming language was in hacker and painter, I didn't expect to use it this time. In the second line, we pass the Lua code to the jedis.eval() method, and assign the key [1] parameter to lockKey, and the ARGV[1] parameter to requestId. The eval() method is to give the Lua code to the Redis server for execution.
So what is the function of this Lua code? In fact, it is very simple. First, obtain the value corresponding to the lock, check whether it is equal to the requestId. If it is equal, delete the lock (unlock). So why use Lua? Because make sure the above operations are atomic. You can read about the problems caused by nonatomism [unlock code - error example 2] . Then why the execution of eval() method can ensure atomicity, which is derived from the feature of Redis. Here is a partial explanation of Eval command on the official website:
In short, when the eval command executes Lua code, the Lua code will be executed as a command, and Redis will not execute other commands until the eval command is executed.
Error example 1
The most common unlocking code is to directly use the jedis.del() method to delete the lock. This way of directly unlocking without first judging the owner of the lock will lead to any client can unlock at any time, even if the lock is not its own.
public static void wrongReleaseLock1(Jedis jedis, String lockKey) {
jedis.del(lockKey);
}
Error example 2
At first glance, this unlocking code is no problem. Even before, I almost realized it like this. It is similar to the correct posture. The only difference is that it is executed by two commands. The code is as follows:
public static void wrongReleaseLock2(Jedis jedis, String lockKey, String requestId) {
//Judge whether lock and unlock are the same client
if (requestId.equals(jedis.get(lockKey))) {
//If at this time, the lock is suddenly not owned by the client, it will be unlocked by mistake
jedis.del(lockKey);
}
}
For example, code comments, the problem is that if you call the jedis.del() method, the lock will be released when it is not part of the current client. So is this really the case? The answer is yes. For example, client A is locked. After A period of time, client A unlocks. Before executing jedis.del(), the lock suddenly expires. At this time, client B attempts to lock successfully, and then client A executes del() method again, which unlocks client B.
summary
This paper mainly introduces how to use Java code to implement Redis distributed lock correctly, and gives two classic error examples for locking and unlocking. In fact, it's not difficult to realize distributed lock through Redis, as long as four conditions in reliability can be met. Although the Internet has brought us convenience, we can google as long as we have questions, but the answer on the Internet must be right? In fact, it is not, so we should always maintain the spirit of questioning, think more and verify more.
If Redis is deployed on multiple computers in your project, you can try to use Redisson to implement distributed locks. This is the official Java component of Redis, which is linked to Reference reading Section has been given.
Example: only one distributed scheduled task can succeed
Full code:
package com.boonya.webservice.util; import java.util.Collections; import redis.clients.jedis.Jedis; /** * * @function Function: REDIS distributed lock * @author PJL * @package com.boonya.webservice.util * @filename RedisDistributedLock.java * @time 2019 11:40:10 a.m., December 20, 2010 */ public class RedisDistributedLock { private static final String LOCK_SUCCESS = "OK"; private static final String SET_IF_NOT_EXIST = "NX"; private static final String SET_WITH_EXPIRE_TIME = "PX"; private static final Long RELEASE_SUCCESS = 1L; /** * Try to get distributed lock * * @param lockKey * lock * @param requestId * Request ID * @param expireTime * Overdue time * @return Success or not */ public static boolean tryGetDistributedLock(String lockKey, String requestId, int expireTime) { Jedis jedis = RedisPool.getJedis(); String result = jedis.set(lockKey, requestId, SET_IF_NOT_EXIST, SET_WITH_EXPIRE_TIME, expireTime); if (LOCK_SUCCESS.equals(result)) { return true; } return false; } /** * Release distributed lock * * @param lockKey * lock * @param requestId * Request ID * @return Whether to release successfully */ public static boolean releaseDistributedLock(String lockKey,String requestId) { Jedis jedis = RedisPool.getJedis(); String script = "if redis.call('get', KEYS[1]) == ARGV[1] then return redis.call('del', KEYS[1]) else return 0 end"; Object result = jedis.eval(script, Collections.singletonList(lockKey), Collections.singletonList(requestId)); if (RELEASE_SUCCESS.equals(result)) { return true; } return false; } }
quartz job configuration:
<?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:aop="http://www.springframework.org/schema/aop" xmlns:tx="http://www.springframework.org/schema/tx" xmlns:context="http://www.springframework.org/schema/context" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.0.xsd http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-2.5.xsd http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-2.5.xsd"> <!-- Thread executor configuration:Consider multitasking--> <bean id="executor" class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor"> <property name="corePoolSize" value="10" /> <property name="maxPoolSize" value="100" /> <property name="queueCapacity" value="500" /> </bean> <!-- Task object --> <bean id="ssgjTask" class="com.boonya.cache.XHTCacheQuartzClearTask" /> <bean id="tokenTask" class="com.boonya.cache.XHTMobileTokenManagerTask" /> <!-- Configure tasks --> <bean id="ssgjClearJob" class="org.springframework.scheduling.quartz.MethodInvokingJobDetailFactoryBean"> <property name="targetObject" ref="ssgjTask" /> <property name="targetMethod" value="ssgjClear" /> </bean> <bean id="tokenClearJob" class="org.springframework.scheduling.quartz.MethodInvokingJobDetailFactoryBean"> <property name="targetObject" ref="tokenTask" /> <property name="targetMethod" value="tokenClear" /> </bean> <!-- Configure triggers --> <bean id="ssgjCornTrigger" class="org.springframework.scheduling.quartz.CronTriggerBean"> <property name="jobDetail" ref="ssgjClearJob" /> <!-- 00 per day:00:00 implement --> <property name="cronExpression" value="0 0/3 * * * ?" /> </bean> <bean id="tokenCornTrigger" class="org.springframework.scheduling.quartz.CronTriggerBean"> <property name="jobDetail" ref="tokenClearJob" /> <!-- Every 1 hour --> <property name="cronExpression" value="0 0 0/1 * * ?" /> </bean> <!-- Register trigger --> <bean class="org.springframework.scheduling.quartz.SchedulerFactoryBean" > <property name="taskExecutor" ref="executor"/> <property name="triggers"> <!-- Register multiple Trigger --> <list> <!-- <ref bean="saveRoomList" /> --> <ref bean="ssgjCornTrigger" /> <ref bean="tokenCornTrigger" /> </list> </property> </bean> </beans>
Execution task Code:
package com.boonya.cache; import java.util.UUID; import org.apache.log4j.Logger; import org.springframework.stereotype.Component; import com.boonya.webservice.manager.MobileTokenManager; import com.boonya.webservice.util.RedisDistributedLock; import com.boonya.xht.util.Constants; /** * Clear MobileToken every hour * @author Administrator * */ /////////////////Modifying class names and method calls/////////////////// /** * * @author PJL * * @note Function Description: TODO token cleaning task * @package com.boonya.webservice.manager * @filename MobileTokenManagerTask.java * @date 2019 3:34:21 PM, April 16, 2016 */ @Component public class XHTMobileTokenManagerTask{ private static Logger logger = Logger.getLogger(XHTMobileTokenManagerTask.class); /** * Execute token cleanup -- pay attention to concurrency */ public void tokenClear(){ if(XHTSystemConfig.clusterModeForTomcat){ String requestId=UUID.randomUUID().toString(); // Get distributed lock boolean success=RedisDistributedLock.tryGetDistributedLock( Constants.CLUSTER_APPLICATION_JOB_LOCK, requestId, Constants.CLUSTER_APPLICATION_JOB_LOCK_TIME); logger.error("tokenClear Get timed task distributed lock:"+success); if(success){ MobileTokenManager.getInstance().removeInvalidToken(); // Release distributed lock RedisDistributedLock.releaseDistributedLock( Constants.CLUSTER_APPLICATION_JOB_LOCK, requestId); logger.error("tokenClear Timed task successfully released the distributed lock!"); } }else{ MobileTokenManager.getInstance().removeInvalidToken(); } } }