Single point timed task

JDK native

Since jdk1 After 5, ScheduledExecutorService is provided to replace TimerTask to execute scheduled tasks, which provides good reliability.

public class SomeScheduledExecutorService {
    public static void main(String[] args) {
        //Create a task queue with a total of {10 threads
        ScheduledExecutorService scheduledExecutorService =
                Executors.newScheduledThreadPool(10);
        //Task execution: start after 1 second, and execute every 30 seconds
        scheduledExecutorService.scheduleAtFixedRate(() -> {
            System.out.println("Perform tasks:" + new Date());
        }, 10, 30, TimeUnit.SECONDS);
    }
}

Spring Task

The Spring Framework comes with timed tasks and provides cron expressions to realize rich timed task configuration. Recommended for beginners https://cron.qqe2.com/ This site to match your cron expression.

@Configuration
@EnableScheduling
public class SomeJob {
    private static final Logger LOGGER = LoggerFactory.getLogger(SomeJob.class);

    /**
     * Every minute (example: 18:01:00, 18:02:00)
     * Second, minute, hour, day, month, week, year
     */
    @Scheduled(cron = "0 0/1 * * * ? *")
    public void someTask() {
       //...
    }
}

Single point timing service in the current microservice environment, the application scenarios are becoming more and more limited, so try distributed timing tasks.

Redis based implementation

Compared with the previous two methods, this Redis based implementation can increase scheduled tasks and consumption through multiple points. However, we should be prepared to prevent repeated consumption.

By means of ZSet

Store the scheduled task in the ZSet set, and store the expiration time in the Score field of ZSet, and then judge whether there is a scheduled task to be executed within the current time through a cycle. If so, execute it.

The specific implementation code is as follows:

/**
 * Description: ZSet timing task based on Redis< br>
 *
 * @author mxy
 */
@Configuration
@EnableScheduling
public class RedisJob {
    public static final String JOB_KEY = "redis.job.task";
    private static final Logger LOGGER = LoggerFactory.getLogger(RedisJob.class);
    @Autowired private StringRedisTemplate stringRedisTemplate;

    /**
     * Add task
     *
     * @param task
     */
    public void addTask(String task, Instant instant) {
        stringRedisTemplate.opsForZSet().add(JOB_KEY, task, instant.getEpochSecond());
    }

    /**
     * Scheduled task queue consumption
     * Consume once per minute (the interval can be shortened to 1s)
     */
    @Scheduled(cron = "0 0/1 * * * ? *")
    public void doDelayQueue() {
        long nowSecond = Instant.now().getEpochSecond();
        //Query all tasks at the current time
        Set<String> strings = stringRedisTemplate.opsForZSet().range(JOB_KEY, 0, nowSecond);
        for (String task : strings) {
            //Start consuming {task
            LOGGER.info("Perform tasks:{}", task);
        }
        //Delete tasks already executed
        stringRedisTemplate.opsForZSet().remove(JOB_KEY, 0, nowSecond);
    }
}

The applicable scenarios are as follows:

• 15 minutes after the order is placed, if the user does not pay, the system needs to automatically cancel the order.
• The red envelope has not been checked for 24 hours, and the return business needs to be delayed;
• An activity is specified to be effective & invalid within a certain time;

The advantages are:

• The query operation of MySQL is omitted, and Redis with higher performance is used instead;
• The tasks to be performed will not be missed due to shutdown and other reasons;

Key space notification mode

We can implement scheduled tasks through Redis's key space notification. Its implementation idea is to set an expiration time for all scheduled tasks. After expiration, we can perceive that the scheduled tasks need to be executed by subscribing to the expiration message. At this time, we can execute the scheduled tasks.

By default, Redis does not enable keyspace notifications. We need to manually enable them through the command of config set notify keyspace events ex. The code of the scheduled task after opening is as follows:

Custom listener

 /**
  * Custom listener
  */
public class KeyExpiredListener extends KeyExpirationEventMessageListener {
    public KeyExpiredListener(RedisMessageListenerContainer listenerContainer) {
        super(listenerContainer);
    }

    @Override
    public void onMessage(Message message, byte[] pattern) {
        // channel
        String channel = new String(message.getChannel(), StandardCharsets.UTF_8);
        //Expired key
        String key = new String(message.getBody(), StandardCharsets.UTF_8);
        //todo your handling
    }
}

Set this listener

/**
 * Description: The scheduled task is realized by subscribing to the expiration notice of Redis< br>
 *
 * @author mxy
 */
@Configuration
public class RedisExJob {
    @Autowired private RedisConnectionFactory redisConnectionFactory;
    @Bean
    public RedisMessageListenerContainer redisMessageListenerContainer() {
        RedisMessageListenerContainer redisMessageListenerContainer = new RedisMessageListenerContainer();
        redisMessageListenerContainer.setConnectionFactory(redisConnectionFactory);
        return redisMessageListenerContainer;
    }

    @Bean
    public KeyExpiredListener keyExpiredListener() {
        return new KeyExpiredListener(this.redisMessageListenerContainer());
    }
}

Spring will listen for Redis messages in the following format

private static final Topic TOPIC_ALL_KEYEVENTS = new PatternTopic("__keyevent@*");

Redis based timing tasks can be applied to limited scenarios, but the implementation is relatively simple, but there are great requirements for functional idempotence. In terms of usage scenario, it should be called delayed task.

Scenario example:

• 15 minutes after the order is placed, if the user does not pay, the system needs to automatically cancel the order.
• The red envelope has not been checked for 24 hours, and the return business needs to be delayed;

The advantages and disadvantages are:

• Passive triggering, less resource consumption for services;
• Redis's Pub/Sub is unreliable and there is no ACK mechanism, but it can be tolerated in general;
• The keyspace notification function consumes some CPU

Distributed timed task

Introducing distributed timed task component or middleware

It is beneficial to the independent consumption and maintenance as well as the independent service.

quartz

Relying on MySQL, it is relatively simple to use and can be deployed on multiple nodes. Only one node can perform tasks by competing for database locks. There is no graphical management page, which is relatively troublesome.

elastic-job-lite

Depending on zookeeper, servers can be added dynamically through the registration and discovery of zookeeper.

• Multiple operation modes
• Failure transfer
• Running status collection
• Multithreading data processing
• Idempotency
• fault tolerant
• Support spring namespace
• There is a graphical management page

LTS

Depending on Zookeeper and cluster deployment, servers can be added dynamically. You can manually add scheduled tasks, start and pause tasks.

• Business log recorder
• SPI extended support
• Failover
• Node monitoring
• Diversified task execution result support
• FailStore fault tolerance
• Dynamic capacity expansion
• Relatively friendly to spring
• Graphical interface for monitoring and management

xxl-job

Domestic, relying on MySQL and based on competitive database lock, it ensures that only one node performs tasks and supports horizontal capacity expansion. You can manually add scheduled tasks, start and pause tasks.

• Elastic expansion
• Split broadcast
• Failover
• Rolling real time log
• GLUE (support online code editing, no release)
• Task progress monitoring
• Task dependency
• data encryption
• Mail alarm
• Run Report
• Graceful shutdown
• Internationalization (Chinese friendly)

summary

Under microservices, it is recommended to use component services such as XXL job to manage scheduled tasks reasonably and effectively. The single point timing task has its limitations, which is suitable for services with small scale and low requirements for future expansion.

Relatively speaking, the scheduled tasks based on spring task are the simplest and fastest, while the difficulty of XXL job is mainly reflected in integration and debugging. No matter what kind of scheduled tasks, you need to ensure that:

• Tasks will not be executed multiple times due to cluster deployment.
• Exceptions in the task are handled effectively
• The slow processing of tasks leads to a large backlog
• The task should be performed at the expected point in time

Middleware can decouple services, but it increases the complexity