In one work, there was a small accident. The result of a redis write and read method was always different from what was expected. After careful consideration, it was found that because all nodes in the cluster use shared caches and queues, resource competition may occur among nodes in some scenarios, There may be a "thread insecurity problem" between various nodes. In a single machine, locks can be used to solve it. In a distributed environment, distributed locks will be used. At that time, I checked on the Internet and found many ideas and codes given by the great God. I tried to realize it myself and shared my ideas with you here.
Redis can do distributed lock conditions
Because redis is single threaded, and the instructions accessed in parallel will be executed in series, redis can be used as a technology to realize distributed locking.
Commands used
In the implementation of distributed locks, redis's string type data structure and the following operations are mainly used:
set key value [ex seconds] [px milliseconds] [nx/xx]: save key value
ex and px refer to the validity period of the key. nx: create a key and save value when the key does not exist XX: save value when the key exists
localhost:0>set hello world OK localhost:0>set hello world ex 100 OK localhost:0>set hello world px 1000 OK Here is the main look nx And xx Effect of localhost:0>set hello worldnx nx NULL localhost:0>set hello worldxx xx OK because hello this key It already exists, so bring the parameter nx After that, I returned NULL，No operation succeeded. close xx After this parameter, now hello this key of value Has been replaced by worldxx
The nx parameter of the set instruction is used in the implementation to save a key value. If the key does not exist, the key is created and the value is saved. If the key exists, NULL is returned without operation. In the java code, this command is encapsulated as a method of Jedis, and I have made another encapsulation in the code:
/**
* If the key does not exist, the establishment returns 1
* If the key is saved, no operation will be performed and 0 will be returned
* @param key
* @param value
* @return
*/
public Long setNx(final String key,final String value){
return this.execute(new Function<Long, Jedis>() {
@Override
public Long callback(Jedis jedis) {
//The set command with nx parameters is encapsulated into the setnx method in Jedis
return jedis.setnx(key, value);
}
});
}
It is mainly used to obtain the lock. setnx returns 1 to prove that the lock has been obtained and 0 to prove that the lock already exists.
expire key integer value: sets the life cycle of the key in seconds
If you feel that your learning efficiency is low and you lack correct guidance, you can join the technology circle with rich resources and strong learning atmosphere to learn and communicate together!
[Java architecture group]
There are many technological giants from the front line in the group, as well as code farmers struggling in small factories or outsourcing companies. We are committed to building an equal and high-quality JAVA Communication circle, which may not make everyone's technology advance by leaps and bounds in the short term, but in the long run, vision, pattern and long-term development direction are the most important.
It is mainly used to set the expiration time of the lock and post the method expire encapsulated in java:
/**
* Set the expiration time of the key
* @param key
* @param exp
* @return
*/
public Long expire(final String key,final int exp){
return this.execute(new Function<Long, Jedis>() {
@Override
public Long callback(Jedis jedis) {
return jedis.expire(key, exp);
}
});
}
get key: get the value corresponding to the key
Take out the stored lock value and paste the method get encapsulated in java:
/**
* Gets a value of type String
*
* @param key
* @return
*/
public String get(final String key) {
return this.execute(new Function<String, Jedis>() {
@Override
public String callback(Jedis jedis) {
return jedis.get(key);
}
});
}
getset key newvalue: returns the old value of the key and saves the new value newvalue
localhost:0>getset hello wolrd wogldxx localhost:0>get hello wolrd
It is used to store the new lock value after obtaining the lock and paste the method getset encapsulated in java:
/**
* Get and return the old value, and set the new value
* @param key
* @param value
* @return
*/
public String getSet(final String key, final String value){
return this.execute(new Function<String, Jedis>() {
@Override
public String callback(Jedis jedis) {
return jedis.getSet(key, value);
}
});
}
Implementation ideas
1. Add the current timestamp and the expiration time of the lock to get the time when the lock will expire. Take this time as value and the lock as key to call setnx method. If 1 is returned, it proves that the original key, that is, the lock does not exist and the lock is obtained successfully, then call expire method to set the timeout time of the lock and return the success of obtaining the lock.
2. If setnx returns 0, it proves that the original lock exists and the lock is not obtained, and then blocks the acquisition of other locks. (I originally thought that step 2 was over. After looking at the realization idea of great God, I found that my idea was inappropriate. Let's talk about step 3 of great God.).
3. if setnx returns 0, it proves that the original lock does not get the lock, then calls the get method to get the value stored in the first step, that is, the lock will expire. Compared with the current timestamp, if the current timestamp is greater than the time that the lock will expire, it means that the lock has expired, calling the getset method to save the new time, and returning the lock to be successful. This is mainly to deal with the failure of expire execution or the failure to release the lock when the server is restarted.
The code implemented according to the idea of great God is posted below to realize the lock of blocking mechanism and an exclusive lock:
import com.eduapi.common.component.RedisComponent;
import com.eduapi.common.util.BeanUtils;
import org.apache.commons.lang3.StringUtils;
/**
* @Description: Using redis to realize distributed lock
* @Author: ZhaoWeiNan .
* @CreatedTime: 2017/3/20 .
* @Version: 1.0 .
*/
public class RedisLock {
private RedisComponent redisComponent;
private static final int DEFAULT_ACQUIRY_RESOLUTION_MILLIS = 100;
private String lockKey;
/**
* The lock timeout prevents the thread from waiting for unlimited execution after entering the lock
*/
private int expireMillisCond = 60 * 1000;
/**
* Lock wait time to prevent thread starvation
*/
private int timeoutMillisCond = 10 * 1000;
private volatile boolean isLocked = false;
public RedisLock(RedisComponent redisComponent, String lockKey) {
this.redisComponent = redisComponent;
this.lockKey = lockKey;
}
public RedisLock(RedisComponent redisComponent, String lockKey, int timeoutMillisCond) {
this(redisComponent, lockKey);
this.timeoutMillisCond = timeoutMillisCond;
}
public RedisLock(RedisComponent redisComponent, String lockKey, int timeoutMillisCond, int expireMillisCond) {
this(redisComponent, lockKey, timeoutMillisCond);
this.expireMillisCond = expireMillisCond;
}
public RedisLock(RedisComponent redisComponent, int expireMillisCond, String lockKey) {
this(redisComponent, lockKey);
this.expireMillisCond = expireMillisCond;
}
public String getLockKey() {
return lockKey;
}
/**
* Get lock (glued the idea of the great God)
* Implementation idea: it mainly uses the setnx command of redis to cache locks
* reids The cached key is the key of the lock, all shares, and value is the expiration time of the lock (Note: the expiration time is set at value here, and there is no time to set its timeout)
* Execution process:
* 1.Try to set the value of a key through setnx. If it succeeds (there is no lock at present), it will return and the lock will be obtained successfully
* 2.If the lock already exists, get the expiration time of the lock. Compare it with the current time. If it times out, set a new value
*
* @return true if lock is acquired, false acquire timeouted
* @throws InterruptedException in case of thread interruption
*/
public synchronized boolean lock() throws InterruptedException {
int timeout = timeoutMillisCond;
boolean flag = false;
while (timeout > 0){
//Set the expiration time of the income
Long expires = System.currentTimeMillis() + expireMillisCond;
String expiresStr = BeanUtils.convertObject2String(expires);
//It turns out that there is no lock in redis. The lock is obtained successfully
if (this.redisComponent.setNx(lockKey,expiresStr) > 0){
//Set the expiration time of the lock
this.redisComponent.expire(lockKey,expireMillisCond);
isLocked = true;
return true;
}
flag = compareLock(expiresStr);
if (flag){
return flag;
}
timeout -= DEFAULT_ACQUIRY_RESOLUTION_MILLIS;
/*
The delay is 100 milliseconds. It may be better to use random time here to prevent hungry processes, that is, when multiple processes arrive at the same time,
Only one process will get the lock, and others will try with the same frequency. Later, some processes will apply for the lock with the same frequency, which may lead to the dissatisfaction of the previous lock
Using random waiting time can ensure fairness to a certain extent
*/
Thread.sleep(DEFAULT_ACQUIRY_RESOLUTION_MILLIS);
}
return false;
}
/**
* Exclusive lock. The function is equivalent to synchronized fast synchronization
* @return
* @throws InterruptedException
*/
public synchronized boolean excludeLock() {
//Set the expiration time of the income
long expires = System.currentTimeMillis() + expireMillisCond;
String expiresStr = BeanUtils.convertObject2String(expires);
//It turns out that there is no lock in redis. The lock is obtained successfully
if (this.redisComponent.setNx(lockKey,expiresStr) > 0){
//Set the expiration time of the lock
this.redisComponent.expire(lockKey,expireMillisCond);
isLocked = true;
return true;
}
return compareLock(expiresStr);
}
/**
* Compare whether locks can be acquired
* Lock timeout acquisition
* @param expiresStr
* @return
*/
private boolean compareLock(String expiresStr){
//If two threads come here
//Because redis is obtained by a single thread
// A thread gets currentValueStr = 1. B thread gets currentValueStr = 1
String currentValueStr = this.redisComponent.get(lockKey);
//lock
if (StringUtils.isNotEmpty(currentValueStr) && Long.parseLong(currentValueStr) < System.currentTimeMillis()){
//Obtain the last lock expiration time and set the current lock expiration time,
//Only one thread can get the setting time of the previous thread, because jedis GetSet is synchronized
//Only thread A saved 2. Take out 1 and get the lock
//B thread bar 2 is saved. Get 2. Compared with not getting the lock,
String oldValue = this.redisComponent.getSet(lockKey,expiresStr);
if (StringUtils.isNotEmpty(oldValue) && StringUtils.equals(oldValue,currentValueStr)){
//Prevent the lock of others from being deleted (overwritten, because the key is the same) - the effect cannot be achieved here, and the value here will be overwritten, but it is acceptable because there is little time difference
//[distributed situation]: if multiple threads happen to be here after this time, but only one thread has the same setting value as the current value, can it have the right to obtain the lock
isLocked = true;
return true;
}
}
return false;
}
/**
* Release lock
*/
public synchronized void unlock(){
if (isLocked){
this.redisComponent.delete(lockKey);
isLocked = false;
}
}
}
In the code, the idea of writing by the great God is put into it. The great God still understands what he said. Look at the calling code:
//Create a lock object. redisComponent is the key of the object expiration time lock of redis component
RedisLock redisLock = new RedisLock(redisComponent,1000 * 60,RedisCacheKey.REDIS_LOCK_KEY + now_mm);
//Acquire lock
if (redisLock.excludeLock()){
try {
//Get the lock, read the timed SMS and collect them in order
set = this.redisComponent.zRangeByScore(RedisCacheKey.MSG_TIME_LIST,0,end);
if (set != null && set.size() > 0){
flag = true;
}
} catch (Exception e) {
LOGGER.error("Exception in getting the ordered collection of timed short messages. The exception is{}",e.toString());
}
...
Here is an idea of using redis to realize distributed lock. Welcome to communicate and point out some mistakes in the article, so that I can deepen my understanding.
last
Bald brother will share with you an immortal document of Java high concurrency core programming compiled by front-line development Daniel, which mainly contains knowledge points: multithreading, thread pool, built-in lock, JMM, CAS, JUC, high concurrency design mode, Java asynchronous callback, completable future class, etc.
Document address: A divine article explains java multithreading, lock, JMM, JUC and high concurrency design pattern clearly
Code words are not easy. If you think this article is useful to you, please give me one button three times! Pay attention to the author, there will be more dry goods to share in the future, please continue to pay attention!