Why?

In the actual development, we often encounter the situation that we need to wait for background events for a long time, such as the common code scanning login function, the event that the QR code interface needs to wait for the successful code scanning login in the background, and the task that takes a long time to complete, such as import and export. At this time, we need to put the task in the background for asynchronous task processing, After completion, push the completion event to the foreground interface. For the above requirements, we need a long connection to complete the push. However, the long connection push status management is complex, and an independent system needs to be deployed. The system process is complex and horizontal expansion is difficult. At this time, it is easier to choose long polling waiting. http requests directly wait for return. Obviously, the logic is simpler, Availability and maintainability will also be higher.

Openresty is a high-performance system built on nginx. Generally, we also need to deploy nginx in front of our own services as a gateway, so it is obviously a good choice to choose openresty to build a high-performance long polling service. Slock is a high-performance state and atomic operation database, and redis is a high-performance memory cache database. Using the following nginx configuration file, you can quickly build a high-performance and available long polling service based on slock and redis. The long polling service built at the same time is a general service, which can be used to push the completion status of requirements such as code scanning login, and also to push messages such as message system and private message system.

slock project address: https://github.com/snower/slock

The slock profile can be seen as follows: https://segmentfault.com/a/11...

Quick configuration build

First, install the lua client package of slock in the installed openresty service.

Project address: https://github.com/snower/slo...

The installation method is slock in slock Lua nginx Copy Lua to lualib / in the openresty directory, then add the following nginx configuration file and modify the relevant parameters.

init_worker_by_lua_block {
    local slock = require "slock"
    slock:connect("server1", "127.0.0.1", 5658)
}

server {
    listen 8081;
    default_type application/json;

    location /poll/event {
        content_by_lua_block {
            local cjson = require "cjson"
            local slock = require "slock"
            local slock_client = slock:get("server1")
            local default_type = ngx.var.arg_default_type or "clear"
            local wait_type = ngx.var.arg_wait_type or ""
            local event_key = ngx.var.arg_event or ""
            local wait_timeout = tonumber(ngx.var.arg_timeout) or 60

            local sendResult = function(err_code, err_message)
                ngx.say(cjson.encode({
                    err_code = err_code,
                    err_message = err_message,
                }))
            end

            if event_key == "" then
                return sendResult(400, "event key is empty")
            end

            local event = nil
            if default_type == "set" then
                event = slock_client:newDefaultSetEvent(event_key, 5, wait_timeout * 2)
            else
                event = slock_client:newDefaultClearEvent(event_key, 5, wait_timeout * 2)
            end

            if wait_type == "reset" then
                local ok, err = event:waitAndTimeoutRetryClear(wait_timeout)
                if not ok then
                    return sendResult(504, "wait event timeout")
                end
                return sendResult(0, "succed")
            end

            local ok, err = event:wait(wait_timeout)
            if not ok then
                return sendResult(504, "wait event timeout")
            end
            return sendResult(0, "succed")
        }
    }

    location /poll/message {
        content_by_lua_block {
            local cjson = require "cjson"
            local redis = require "resty.redis"
            local slock = require "slock"
            local redis_client = redis:new()
            local slock_client = slock:get("server1")

            local default_type = ngx.var.arg_default_type or "clear"
            local wait_type = ngx.var.arg_wait_type or ""
            local event_key = ngx.var.arg_event or ""
            local wait_timeout = tonumber(ngx.var.arg_timeout) or 60

            local sendResult = function(err_code, err_message, data)
                ngx.say(cjson.encode({
                    err_code = err_code,
                    err_message = err_message,
                    data = data,
                }))
            end

            if event_key == "" then
                return sendResult(400, "event key is empty")
            end

            redis_client:set_timeouts(5000, wait_timeout * 500, wait_timeout * 500)
            local ok, err = redis_client:connect("10.10.10.251", 6379)
            if not ok then
                return sendResult(502, "redis connect fail")
            end
            local message, err = redis_client:lpop(event_key)
            if err ~= nil then
                return sendResult(500, "redis lpop fail")
            end
            if message ~= ngx.null then
                redis_client:set_keepalive(7200000, 16)
                return sendResult(0, "", message)
            end

            local event = nil
            if default_type == "set" then
                event = slock_client:newDefaultSetEvent(event_key, 5, wait_timeout * 2)
            else
                event = slock_client:newDefaultClearEvent(event_key, 5, wait_timeout * 2)
            end

            if wait_type == "reset" then
                local ok, err = event:waitAndTimeoutRetryClear(wait_timeout)
                if not ok then
                    return sendResult(504, "wait timeout")
                end

                local message, err = redis_client:lpop(event_key)
                if err ~= nil then
                    return sendResult(500, "redis lpop fail")
                end
                redis_client:set_keepalive(7200000, 16)
                return sendResult(0, "succed", message)
            end

            local ok, err = event:wait(wait_timeout)
            if not ok then
                return sendResult(504, "wait timeout")
            end

            local message, err = redis_client:lpop(event_key)
            if err ~= nil then
                return sendResult(500, "redis lpop fail")
            end
            redis_client:set_keepalive(7200000, 16)
            return sendResult(0, "succed", message)
        }
    }
}

/The poll/event interface only waits for the event to trigger and does not return data.

/poll/message obtains data from redis first, and returns if it succeeds. Otherwise, wait for the event to trigger, and then obtain data from redis and return.

Interface Query String parameter:

• default_type creates whether the initial state of the Event is set or cleared. wait for the set state to trigger. That is, if the initial state is set, other systems need to perform the clear operation first. The optional values are set and clear, and the default value is clear
• wait_ Whether to reset the Event status after the type Event is triggered. If it is not reset, it can be re entered within the expiration time. If it is reset, it can be used for the circular acquisition of messages or events in the private messaging system. Set reset to reset. The default value is that the null character is not reset
• The event key that event waits for cannot be empty. redis also uses this key to save messages in the List data structure
• Timeout number, the waiting timeout, in seconds. The default is 60 seconds

Special attention:

• openresty uses a single tcp connection to slock to handle all requests, and nginx only has init_ worker_ by_ lua_ The socket created by the block can only exist in the whole worker life cycle, so the slock connect needs to be in init_ worker_ by_ lua_ After block is completed, the first parameter is the connection name, which can be used to obtain the connection for use later.
• When slock is configured in replset mode, it can be connected in replset mode. For example, slock: connectreplset ("Server1", {"127.0.0.1", 5658}, {"127.0.0.1", 5659}). When connecting in this mode, nginx will automatically track available nodes to maintain high availability.

After the configuration is completed, execute the following shell to wait for the return:

curl "http://localhost:8081/poll/message?event=test&default_type=clear"

How do other systems push events?

Push events using slock java client

java client project address: https://github.com/snower/jas...

package main;

import io.github.snower.jaslock.Client;
import io.github.snower.jaslock.Event;
import io.github.snower.jaslock.exceptions.SlockException;
import redis.clients.jedis.Jedis;

import java.io.IOException;
import java.nio.charset.StandardCharsets;

public class App {
    public static void main(String[] args) {
        Client slock = new Client("localhost", 5658);
        Jedis jedis = new Jedis("10.10.10.251", 6379);
        try {
            byte[] eventKey = "test".getBytes(StandardCharsets.UTF_8);
            slock.open();
            jedis.rpush(eventKey, "hello".getBytes(StandardCharsets.UTF_8));
            jedis.expire(eventKey, 120);
            Event event = slock.newEvent(eventKey, 5, 120, false);
            event.set();
        } catch (IOException | SlockException e) {
            e.printStackTrace();
        } finally {
            slock.close();
            jedis.close();
        }
    }
}

newEvent parameter:

• eventKey event name, consistent with the front-end request
• timeout set operation timeout event
• Expanded if the initial state is cleared, it indicates the state holding time after set. If the initial state is set, it indicates the state holding time after clear. If the initial state is set, it will be automatically recovered after the change time
• Defaultset ture indicates that the initial state is set and false is cleared, which shall be consistent with the parameters passed by the front end

Note: only the redis operation can be removed when pushing events.

php, python and golang operate similarly.

php client project address: https://github.com/snower/pys...

python client project address: https://github.com/snower/phs...

Push events using redis custom command

redis user-defined commands can also be used to execute the set and clear operations of slock Event to complete event triggering.

When the initial is set:

#clear operation
lock ${EVENT_KEY} lock_id ${EVENT_KEY} flag 2 timeout ${TIMEOUT} expried ${EXPRIED}
#Such as lock test lock_id test flag 2 timeout 5 expried 120


#set operation
unlock ${EVENT_KEY} lock_id ${EVENT_KEY}
#Such as unlock test lock_id test

When the initial is cleared:

#clear operation
unlock ${EVENT_KEY} lock_id ${EVENT_KEY}
#Such as unlock test lock_id test


#set operation
lock ${EVENT_KEY} lock_id ${EVENT_KEY} flag 2 timeout ${TIMEOUT} expried ${EXPRIED} count 2
#Such as lock test lock_id test flag 2 timeout 5 expried 120 count 2
#After connecting to slock with redis cli - P 5658 and executing the sample command, you can see that the curl command is returned successfully.

About high availability and expansion

For high availability, slock supports the configuration of cluster mode. When the master node is abnormal during the operation of centralized mode, it can automatically select a new master node. At this time, if openresty uses the resplset mode to connect, it can automatically use the new available node to ensure high availability.

As for horizontal expansion, slock's good multi-core support and million level qps ensure that there is no need to consider the problem of horizontal expansion, while openresty still maintains the normal stateless characteristics of Web services. It can continuously improve the bearing performance of the system by expanding the capacity according to the normal horizontal expansion mode of the web.