github

https://github.com/idealvin/co

Reference documents

• chinese: github gitee
• English: github gitee

What is CO

CO is an elegant and efficient C + + basic library, which supports Linux, Windows, Mac and other platforms. It implements a series of high-quality basic components such as golang like coprocess, coprocess based network programming framework, command line parameter and configuration file parsing library, high-performance log library, unit test framework, JSON library and so on.

CO in github Above MIT The license is open source. It uses some third-party code and may have different licenses. For details, see LICENSE File. For the convenience of domestic users, gitee The code on github will also be synchronized on a regular basis.

Development of CO

Alvin(idealvin) CO has been developed since 2013. The initial purpose is to reduce the tripartite dependence in C + + projects and improve the development efficiency of C + +. Since 2015, Alvin has introduced CO into the actual project for use by himself and his colleagues, greatly reducing the development cycle of the project, and CO has been tested by industrial projects.

After years of accumulation and precipitation, by 2019, Alvin realized the co process mechanism in golang in C + +, and provided a set of network programming framework based on CO process. At the beginning of the birth of CO process, it has been used in embedded network program development and achieved immediate results.

By 2021, CO collaboration has made great progress. At present, hook has been supported on Linux/Windows/Mac platforms, and collaboration lock, collaboration synchronization event, collaboration pool, channel and waitgroup in golang have been realized. Users can write golang experience with Co.

Get started quickly

compile

Recommended installation xmake , execute the following command in the CO root directory to build all subprojects:

xmake -a

If you need to use http::Client, SSL or HTTPS features, you can use the following command to build:

xmake f --with_libcurl=true --with_openssl=true
xmake -a

xmake will automatically install libcurl and openssl from the network. Depending on the network, this process may be slow. xmake -a will build libco, gen, co/unitest as well as co/test All test codes below. The user can execute the following command to run the test program in CO:

xmake r unitest -a
xmake r flag
xmake r log -cout
xmake r co

Developing C + + projects using CO

The simplest can directly include co/all.h , use all characteristics in CO. If you are worried about affecting the compilation speed, you can also include only the header files you need, such as co/co.h , you can use co/flag, co/log, and all the features related to coprocessing.

#include "co/all.h"

DEF_string(s, "nice", "");

int main(int argc, char** argv) {
    flag::init(argc, argv);
    log::init();

    LOG << FLG_s;
    return 0;
}

The above is a simple example. The first two lines of the main function are used to initialize the flag and log libraries respectively. Some components in CO will define configuration items with flag and print logs with log. Therefore, it is generally necessary to call flag::init() and log::init() at the beginning of main function for initialization.

You can also use the macro def_ Maindefine the main function:

#include "co/all.h"

DEF_string(s, "nice", "");

DEF_main(argc, argv) {
    LOG << FLG_s;
    return 0;
}

DEF_main has internally called flag::init() and log::init(), and the user does not need to call it again. In addition, DEF_main will run the code in the main function in the coroutine, which is consistent with golang, and the main function in golang is also in the coroutine. Some CO process related components must be used in the CO process. When developing CO process based applications, DEF is generally recommended_ Maindefines the main function.

Core components

co/flag

co/flag It is an easy-to-use command-line parameter and configuration file parsing library. Some components in CO will use it to define configuration items.

co/flag provides a default value for each configuration item. Without configuration parameters, the program can run according to the default configuration. The user can also pass in configuration parameters from the command line or configuration file, which can be executed when a configuration file is required/ exe -mkconf automatically generates configuration files.

// xx.cc
#include "co/flag.h"
#include "co/log.h"

DEF_bool(x, false, "bool x");
DEF_bool(y, false, "bool y");
DEF_uint32(u32, 0, "...");
DEF_string(s, "hello world", "string");

int main(int argc, char** argv) {
    flag::init(argc, argv);

    COUT << "x: " << FLG_x;
    COUT << "y: " << FLG_y;
    COUT << "u32: " << FLG_u32;
    COUT << FLG_s << "|" << FLG_s.size();

    return 0;
}

The above is an example of using co/flag. DEF is in the code_ The macro at the beginning defines four configuration items. Each configuration item is equivalent to a global variable, and the variable name is FLG_ Add configuration name. After the above code is compiled, it can be run as follows:

./xx                  # Run as default
./xx -xy -s good      # Single letter named bool flag can be set to true together
./xx -s "I'm ok"      # String with spaces
./xx -u32 8k          # Integers can have units: k,m,g,t,p, case insensitive

./xx -mkconf          # Auto generate configuration file XX conf
./xx xx.conf          # Pass in parameters from configuration file
./xx -config xx.conf  # Same as above

co/log

co/log It is a high-performance local log system, which is used by some components in CO to print logs.

co/log divides the log into five levels: debug, info, warning, error and fatal. Printing the log of fatal level will terminate the operation of the program. Users can print different levels of logs as follows:

DLOG << "hello " << 23;  // debug
LOG << "hello " << 23;   // info
WLOG << "hello " << 23;  // warning
ELOG << "hello " << 23;  // error
FLOG << "hello " << 23;  // fatal

co/log also provides a series of CHECK macros, which can be regarded as enhanced assert, and they will not be cleared in debug mode.

void* p = malloc(32);
CHECK(p != NULL) << "malloc failed..";
CHECK_NE(p, NULL) << "malloc failed..";

When the CHECK assertion fails, co/log will print the function call stack information, and then terminate the program.

co/log is very fast. After the program runs stably, there is almost no need for memory allocation. The following are some test results for reference only:

• co/log vs glog (single thread)

  platform	google glog	co/log
  win2012 HHD	1.6MB/s	180MB/s
  win10 SSD	3.7MB/s	560MB/s
  mac SSD	17MB/s	450MB/s
  linux SSD	54MB/s	1023MB/s

• co/log vs spdlog (Windows)

  threads	total logs	co/log time(seconds)	spdlog time(seconds)
  1	1000000	0.103619	0.482525
  2	1000000	0.202246	0.565262
  4	1000000	0.330694	0.722709
  8	1000000	0.386760	1.322471

• co/log vs spdlog (Linux)

  threads	total logs	co/log time(seconds)	spdlog time(seconds)
  1	1000000	0.096445	2.006087
  2	1000000	0.142160	3.276006
  4	1000000	0.181407	4.339714
  8	1000000	0.303968	4.700860

co/unitest

co/unitest It is a simple and easy-to-use unit test framework. Many components in CO will use it to write unit test code, which provides a guarantee for the stability of CO.

#include "co/unitest.h"
#include "co/os.h"

namespace test {
    
DEF_test(os) {
    DEF_case(homedir) {
        EXPECT_NE(os::homedir(), "");
    }

    DEF_case(cpunum) {
        EXPECT_GT(os::cpunum(), 0);
    }
}
    
} // namespace test

The above is a simple example, def_ The test macro defines a test unit, which is actually a function (a method in a class). DEF_ The case macro defines test cases, and each test case is actually a code block. Multiple test units can be put into the same C + + project. Generally, the main function only needs the following lines:

#include "co/unitest.h"

int main(int argc, char** argv) {
    flag::init(argc, argv);
    unitest::run_all_tests();
    return 0;
}

co/unitest The following is the unit test code in CO. after compilation, the following commands can be executed:

xmake r unitest -a   # Run all unit test cases
xmake r unitest -os  # Run only the test cases in the os unit

Synergetic process

CO achieves similar results golang It has the following characteristics:

• For multithreading scheduling, the default number of threads is the number of system CPU cores.
• Shared stack: the coroutines in the same thread share several stacks (the default size is 1MB), and the memory occupation is low. The test on Linux shows that 10 million coroutines only use 2.8G memory (for reference only).
• Each collaboration process has a horizontal relationship, and a new collaboration process can be created anywhere (including in the collaboration process).
• Support system API hook (Windows/Linux/Mac), and you can directly use the third-party network library in the collaboration process.
• Synergetic socket API.
• Co process synchronization event co::Event.
• Co process lock co::Mutex.
• Synergetic pool co::Pool.
• channel co::Chan.
• waitgroup co::WaitGroup.

Create collaboration

go(ku);            // void ku();
go(f, 7);          // void f(int);
go(&T::f, &o);     // void T::f(); T o;
go(&T::f, &o, 7);  // void T::f(int); T o;
go([](){
    LOG << "hello go";
});

The above is an example of creating a coroutine with go(). go() is a function that accepts 1 to 3 parameters. The first parameter f is any callable object. These parameters can be called as long as f (), (* f) (), f (P), (* f) (P), (o - > * f) () or (o - > * f) (P) are met.

The coroutines created by go() will be evenly distributed to different scheduling threads. If you want some of the collaborations to run in the same thread, you can create them in the following way:

auto s = co::next_scheduler();
s->go(f1);
s->go(f2);

If you want to create a collaboration in all scheduling threads, you can use the following method:

auto& s = co::all_schedulers();
for (size_t i = 0; i < s.size(); ++i) {
    s[i]->go(f);
}

channel

co::Chan , similar to the channel in golang, can be used to transfer data between processes.

#include "co/co.h"

DEF_main(argc, argv) {
    co::Chan<int> ch;
    go([ch]() {
        ch << 7;
    });

    int v = 0;
    ch >> v;
    LOG << "v: " << v;

    return 0;
}

The read-write operation of channel must be carried out in the collaboration, so DEF is used in the above code_ Main defines the main function so that the code in the main function also runs in the coroutine.

The channel object in the code is on the stack, while CO adopts the shared stack implementation method. The data on one process stack may be covered by other processes. Generally, the processes cannot communicate directly through the data on the stack. Therefore, the lambda in the code adopts the method of capturing by value to copy a copy of the channel and transfer it to the new process. The copy operation of channel only increases the internal reference count by 1, which has little impact on performance.

When creating a channel, you can add a timeout as follows:

co::Chan<int> ch(8, 1000);

After the channel read / write operation is completed, you can call co::timeout() to determine whether it times out. This method is simpler than the select based implementation in golang.

The channel in CO is implemented based on memory copy. The data type passed can be built-in type, pointer type, or structure type with simple memory copy semantics. Like the container type in std::string or STL, the copy operation is not a simple memory copy. Generally, it cannot be passed directly in the channel. See for details co::Chan reference document.

waitgroup

co::WaitGroup , similar to sync. In golang Waitgroup, which can be used to wait for the exit of a process or thread.

#include "co/co.h"

DEF_main(argc, argv) {
    FLG_cout = true;

    co::WaitGroup wg;
    wg.add(8);

    for (int i = 0; i < 8; ++i) {
        go([wg]() {
            LOG << "co: " << co::coroutine_id();
            wg.done();
        });
    }

    wg.wait();
    return 0;
}

Network programming

CO provides a set of collaborative socket API , most of them are basically consistent with the native socket API in form. Users familiar with socket programming can easily write high-performance network programs in a synchronous way. In addition, CO also implements higher-level network programming components, including TCP,HTTP And based on JSON of RPC Framework, which is compatible with IPv6 and supports SSL. It is more convenient to use than socket API. Here is a brief demonstration of the usage of HTTP, and the rest can be viewed in the reference documents.

Static web server

#include "co/flag.h"
#include "co/log.h"
#include "co/so.h"

DEF_string(d, ".", "root dir"); // Specify the root directory of the web server

int main(int argc, char** argv) {
    flag::init(argc, argv);
    log::init();

    so::easy(FLG_d.c_str()); // mum never have to worry again

    return 0;
}

HTTP server

http::Server serv;

serv.on_req(
    [](const http::Req& req, http::Res& res) {
        if (req.is_method_get()) {
            if (req.url() == "/hello") {
                res.set_status(200);
                res.set_body("hello world");
            } else {
                res.set_status(404);
            }
        } else {
            res.set_status(405); // method not allowed
        }
    }
);

serv.start("0.0.0.0", 80);                                    // http
serv.start("0.0.0.0", 443, "privkey.pem", "certificate.pem"); // https

HTTP client

void f() {
    http::Client c("https://github.com");

    c.get("/");
    LOG << "response code: "<< c.response_code();
    LOG << "body size: "<< c.body_size();
    LOG << "Content-Length: "<< c.header("Content-Length");
    LOG << c.header();

    c.post("/hello", "data xxx");
    LOG << "response code: "<< c.response_code();
}

go(f);