muduo log library is an asynchronous high-performance log library. Its performance cost is about 1.0us~1.6us for each log message written by the front end.
Log base model
double buffering interaction technology is adopted. The basic idea is to prepare two parts of buffer: A and B. the front end thread fills buffer A with data (log messages), and the back end thread is responsible for writing buffer B to the log file. When a is full, swap a and B. So back and forth.
During implementation, a full buffer queue (Buffer1~n, 2 < = n < = 16) is set at the back end to cache the log messages to be written temporarily in a cycle.
The benefits of this are:
1) Thread safety; 2) Non blocking.
In this way, when two buffer s write logs at the front end, they do not have to wait for disk file operation, and avoid triggering the back-end thread every time they write a log message.
Exception handling:
When too many buffers are generated into the queue in a cycle, when the upper limit number of queue elements exceeds 25, the excess part is directly discarded and recorded.
component
The muduo log library consists of a front-end and a back-end.
front end
The front end mainly includes Logger, LogStream, FixedBuffer and SourceFile.
The class diagram relationship is as follows:
Logger class
The Logger is located in logging h/Logging. CC, which mainly provides an interface for users (front-end threads) to use the log library, is an implementation of pointer to impl (i.e. GoF bridging mode), which is implemented in detail by the internal class Impl.
The Logger can be based on the information provided by the user__ FILE__，__ LINE__ And other macro construction objects to record the information of the log code itself (the number of files and lines); It also provides the ability to construct different levels of log message objects. Each Logger object represents a log message.
The Logger defines the log level (enum LogLevel) internally and provides the interface for obtaining and setting the global log level (g_logLevel); Provides an interface to access the internal LogStream object.
Log level type LogLevel
definition:
enum LogLevel
{
TRACE = 0,
DEBUG,
INFO,
WARN,
ERROR,
FATAL,
NUM_LOG_LEVELS
};
User interface
Logging.h, a series of logs are also defined_ The macro at the beginning is convenient for users to record logs in C + + style:
#define LOG_TRACE if (muduo::Logger::logLevel() <= muduo::Logger::TRACE) \ muduo::Logger(__FILE__, __LINE__, muduo::Logger::TRACE, __func__).stream() #define LOG_DEBUG if (muduo::Logger::logLevel() <= muduo::Logger::DEBUG) \ muduo::Logger(__FILE__, __LINE__, muduo::Logger::DEBUG, __func__).stream() #define LOG_INFO if (muduo::Logger::logLevel() <= muduo::Logger::INFO) \ muduo::Logger(__FILE__, __LINE__).stream() #define LOG_WARN muduo::Logger(__FILE__, __LINE__, muduo::Logger::WARN).stream() #define LOG_ERROR muduo::Logger(__FILE__, __LINE__, muduo::Logger::ERROR).stream() #define LOG_FATAL muduo::Logger(__FILE__, __LINE__, muduo::Logger::FATAL).stream() #define LOG_SYSERR muduo::Logger(__FILE__, __LINE__, false).stream() #define LOG_SYSFATAL muduo::Logger(__FILE__, __LINE__, true).stream()
For example, users can use logs in this way:
LOG_TRACE << "trace" << 1;
Constructor
It is not difficult to find that each macro definition constructs a temporary Logger object, and then achieves the function of writing logs through stream().
Select the Logger constructor with the most complete parameters to construct a temporary Logger object:
muduo::Logger(__FILE__, __LINE__, muduo::Logger::TRACE, __func__) __FILE__ Is a macro, Indicates the file name (including path) of the current code __LINE__ Is a macro, Indicates the number of lines in the file where the current code is located muduo::Logger::TRACE Log level TRACE __func__ Is a macro, Indicates the name of the function where the current code is located
Corresponding prototype:
Logger(SourceFile file, int line, LogLevel level, const char* func);
Here, SourceFile is also an internal class, which is used to wrap the file name of the code that constructs the Logger object, and only record the basic file name (excluding the path) to save the length of log messages.
Output location, scour log
An application usually has only one global Logger. The Logger class defines two function pointers, which are used to set the output position of the log (g_output) and flush the log (g_flush).
Type:
typedef void (*OutputFunc)(const char* msg, int len); typedef void (*FlushFunc)();
The Logger outputs and flushes to stdout by default:
void defaultOutput(const char* msg, int len)
{
size_t n = fwrite(msg, 1, static_cast<size_t>(len), stdout);
//FIXME check n
(void)n;
}
void defaultFlush()
{
fflush(stdout);
}
Logger::OutputFunc g_output = defaultOutput;
Logger::FlushFunc g_flush = defaultFlush;
Logger also provides two static functions to set g_output and g_flush.
static void setOutput(OutputFunc); static void setFlush(FlushFunc);
The user code can use these two functions to modify the output position of the Logger (which needs to be modified synchronously). A typical application is g_output is relocated to the back-end AsyncLogging::append(), so that the back-end thread can fetch data from the buffer and write it to the log file (asynchronously with the front-end thread) when the buffer is full or timed.
muduo::AsyncLogging* g_asyncLog = NULL;
void asyncOutput(const char* msg, int len)
{
g_asyncLog->append(msg, len);
}
void func()
{
muduo::Logger::setOutput(asyncOutput);
LOG_INFO << "123456";
}
int main()
{
char name[256] = { '\0' };
strncpy(name, argv[0], sizeof name - 1);
muduo::AsyncLogging log(::basename(name), kRollSize);
log.start();
g_asyncLog = &log;
func();
}
Log level, time zone
Two global variables are also defined to store the log level (g_logLevel) and time zone (g_logTimeZone). Current log message level, if lower than g_ Loglevel, there will be no operation and almost no overhead; Only not less than g_ Only log messages of loglevel level can be recorded. This is through log_ The if statement defined by XXX macro is implemented.
#define LOG_TRACE if (muduo::Logger::logLevel() <= muduo::Logger::TRACE) \ ...
g_logTimeZone will affect the time zone used for logging. The default time zone is UTC time (GMT time zone). For example:
20220306 07:37:08.031441Z 3779 WARN Hello - Logging_test.cpp:75
The "20220306 07:37:08.031441Z" will be affected by the log time zone.
According to g_logTimeZone is the time when the log record is generated, which is located in formatTime(). Since GMT is not the default time zone for subsequent research, it is not the default time zone.
Destructor
As mentioned earlier, Logger is a bridge mode, and the specific implementation is entrusted to Impl. The Logger destructor code is as follows:
Logger::~Logger()
{
impl_.finish(); // Add suffix file name to Small Buffer: number of lines
const LogStream::Buffer& buf(stream().buffer());
g_output(buf.data(), buf.length()); // Callback saved g_output: output the Small Buffer to the specified file stream
if (impl_.level_ == FATAL) // In case of fatal error, output log and terminate the program
{
g_flush(); // Callback scour
abort();
}
}
In the destructor, the Logger mainly completes the work: creating a LogStream object stream_ Add the suffix (file name: line number, LF refers to line break '\ n') to the log message in the stream_ The cached log message passes through G_ The output callback writes to the specified file stream. In addition, if there is a FATAL error (FATAL level log), terminate the program.
The size of the Small Buffer buffer is 4KB by default. Each log message is actually saved. See the description of LogStream for details.
Impl class
Logger::Impl is the internal class of logger. It is responsible for the main implementation of logger and provides the function of assembling a complete log message.
Here are three complete log s:
20220306 09:15:44.681220Z 4013 WARN Hello - Logging_test.cpp:75 20220306 09:15:44.681289Z 4013 ERROR Error - Logging_test.cpp:76 20220306 09:15:44.681296Z 4013 INFO 4056 - Logging_test.cpp:77 Format: Date + time + Microseconds + thread id + level + text + original file name + Line number date time Microseconds Thread level body Source file name: Line number 20220306 09:15:44.681220Z 4013 WARN Hello - Logging_test.cpp:75 ...
Data structure of Impl
class Impl
{
public:
typedef Logger::LogLevel LogLevel;
Impl(LogLevel level, int old_errno, const SourceFile& file, int line);
void formatTime(); // Format the current time string according to the time zone, which is also the beginning of a log message
void finish(); // Add the suffix of a log message
Timestamp time_; // Used to get the current time
LogStream stream_; // It is used to format user log data, provide operator < < interface and save log messages
LogLevel level_; // Log level
int line_; // Line of source code
SourceFile basename_; // Source code file name (excluding path) information
};
It contains all components that need to be assembled into a complete log information. Of course, the body part is directly passed to the stream by the user thread through logstream:: operator < <_ of
Impl constructor
In addition to the initial construction of each member, it also generates thread tid, formats time string, etc., and uses stream_ Join the Samall Buffer.
Logger::Impl::Impl(LogLevel level, int savedErrno, const SourceFile &file, int line)
: time_(Timestamp::now()),
stream_(),
level_(level),
line_(line),
basename_(file)
{
formatTime();
CurrentThread::tid();
stream_ << T(CurrentThread::tidString(), static_cast<unsigned int>(CurrentThread::tidStringLength()));
stream_ << T(LogLevelName[level], kLogLevelNameLength); // 6
if (savedErrno != 0) // A system call error occurred
{
stream_ << strerror_tl(savedErrno) << " (errno=" << savedErrno << ") "; // Custom function strerror_tl converts the error number into a string, which is equivalent to strerror_r(3)
}
}
LogStream class implementation
LogStream mainly provides the operator < < operation, which formats the integer number, floating point number, character, string, character array, binary memory and another Small Buffer provided by the user into a string, and adds the Small Buffer of the current class.
Small Buffer stores log messages
Small Buffer is an embodiment of the template class fixedbuffer < >. i.e.FixedBuffer, with a default size of 4KB, is used to store a log message. Held for the front-end class LogStream.
In contrast, there is also a Large Buffer, which is also an embodiment of FixedBuffer. FixedBuffer, with a default size of 4MB, is used to store multiple log messages. Held for the backend class AsyncLogging.
const int kSmallBuffer = 4000;
const int kLargeBuffer = 4000 * 1000;
class LogStream : noncopyable
{
...
typedef detail::FixedBuffer<detail::kSmallBuffer> Buffer; // Small Buffer Type
...
Buffer buffer_; // Small Buffer for storing log messages
}
class AsyncLogging: noncopyable
{
...
typedef muduo::detail::FixedBuffer<muduo::detail::kLargeBuffer> Buffer; // Large Buffer Type
...
}
The template class FixedBuffer uses the array char data internally_ [size] storage, with pointer char* cur_ Indicates the location of the current data to be written. Various operations on FixedBuffer < > are actually on data_ Array and cur_ Pointer operation.
template<int SIZE>
class FixedBuffer : noncopyable
{
public:
...
private:
...
char data_[SIZE];
char* cur_;
}
For example, add data to FixedBuffer. FixedBuffer < >:: append():
void append(const char* buf, size_t len)
{
// FIXME: append partially
if (implicit_cast<size_t>(avail()) > len) // implicit_cast implicitly converts int avail() to size_t
{
memcpy(cur_, buf, len);
cur_ += len;
}
}
int avail() const { return static_cast<int>(end() - cur_); } // Return the remaining free space size of Buffer
const char* end() const { return data_ + sizeof(data_); } // Returns the pointer to the end of the Buffer internal array
Operator < < format data
For different types of data, LogStream overloads a series of operator < < operators, which are used to format the data into a string and store it in LogStream::buffer_.
{
typedef LogStream self;
public:
...
self& operator<<(bool v)
self& operator<<(short);
self& operator<<(unsigned short);
self& operator<<(int);
self& operator<<(unsigned int);
self& operator<<(long);
self& operator<<(unsigned long);
self& operator<<(long long);
self& operator<<(unsigned long long);
self& operator<<(const void*);
self& operator<<(float v);
self& operator<<(double);
self& operator<<(char v);
self& operator<<(const char* str);
self& operator<<(const unsigned char* str);
self& operator<<(const string& v);
self& operator<<(const StringPiece& v);
self& operator<<(const Buffer& v);
...
}
1) For string type parameters, operator < < essentially calls buffer_ The corresponding fixedbuffer < >:: append(), and store it in the Small Buffer.
self& operator<<(const char* str)
{
if (str)
{
buffer_.append(str, strlen(str));
}
else
{
buffer_.append("(null)", 6);
}
return *this;
}
2) For the character type, the difference between the parameter and the string type is that the length is only 1, and there is no need to judge whether the pointer is empty.
self& operator<<(char v)
{
buffer_.append(&v, 1);
return *this;
}
3) For decimal integers, such as int/long, the template function formatInteger() is used to convert them into strings and fill them directly into the tail of Small Buffer.
formatInteger() does not use snprintf to convert the format of integer data, but uses the efficient conversion method convert() proposed by Matthew Wilson. The basic idea is: from the end, the integer number to be converted is converted from decimal bit by bit to char type, and then filled into the cache until the remaining value to be converted is 0.
template<typename T>
void LogStream::formatInteger(T v)
{
if (buffer_.avail() >= kMaxNumericSize) // Small Buffer has enough space left
{
size_t len = convert(buffer_.current(), v);
buffer_.add(len);
}
}
const char digits[] = "9876543210123456789";
const char* zero = digits + 9; // zero pointer to '0'
static_assert(sizeof(digits) == 20, "wrong number of digits");
/* Efficient Integer to String Conversions, by Matthew Wilson. */
template<typename T>
size_t convert(char buf[], T value)
{
T i = value;
char* p = buf;
do {
int lsd = static_cast<int>(i % 10);
i /= 10;
*p++ = zero[lsd];
} while (i != 0);
if (value < 0)
{
*p++ = '-';
}
*p = '\0';
std::reverse(buf, p);
return static_cast<size_t>(p - buf);
}
4) For double type, use the library function snprintf to convert to const char *, and fill in the tail of Small Buffer directly.
LogStream::self &LogStream::operator<<(double v)
{
if (buffer_.avail() >= kMaxNumericSize)
{
int len = snprintf(buffer_.current(), kMaxNumericSize, "%.12g", v ); // Convert v to a string and fill in buffer_ Current tail.% G automatically selects% F,% e format and does not output meaningless 0.% 12g keep up to 12 decimal places
buffer_.add(static_cast<size_t>(len));
}
return *this;
}
5) For binary numbers, the principle is the same as that of integer numbers. However, they are not stored in Small Buffer in hexadecimal format, but in the form of hexadecimal string (not NUL ending). Each number will be prefixed with "0x".
The core function convertHex, which converts binary memory into hexadecimal number, uses an efficient conversion algorithm similar to convert.
LogStream::self &LogStream::operator<<(const void* p)
{
uintptr_t v = reinterpret_cast<uintptr_t>(p); // uintptr_ The number of T digits is the same as the number of address digits, which is convenient for cross platform use
if (buffer_.avail() >= kMaxNumericSize) // Small Buffer has enough space left
{
char* buf = buffer_.current();
buf[0] = '0';
buf[1] = 'x';
size_t len = convertHex(&buf[2], v);
buffer_.add(len + 2);
}
return *this;
}
const char digitsHex[] = "0123465789ABCDEF";
static_assert(sizeof(digitsHex) == 17, "wrong number of digitsHex");
size_t convertHex(char buf[], uintptr_t value)
{
uintptr_t i = value;
char* p = buf;
do
{
int lsd = static_cast<int>(i % 16); // last digit for hex number
i /= 16;
*p++ = digitsHex[lsd];
} while (i != 0);
*p = '\0';
std::reverse(buf, p);
return static_cast<size_t>(p - buf);
}
Note: uintptr_ The number of T bits is the same as that of the platform address, accounting for 64 bits in the 64 bit system; In 32-bit system, it occupies 32 bits. Using uintptr_t is to improve portability.
6) For other types, they are converted to the above basic types, then converted to strings and added to the end of Small Buffer.
staticCheck() static check
When formatting binary memory data and integer numbers in operator < < (char void *) and formatInteger(T v), there is a judgment: whether the remaining space of Small Buffer is enough. There is a static constant kMaxNumericSize (default 48). So, how to get the value of kMaxNumericSize? 48 is it reasonable and how to verify it?
This can be verified using staticCheck(). The purpose is to ensure that the value of kMaxNumericSize can meet the requirement that the remaining space of Small Buffer can store the data to be formatted. Take the double, long double, long, long long with long data bits and perform static_assert assertion.
void LogStream::staticCheck()
{
static_assert(kMaxNumericSize - 10 > std::numeric_limits<double>::digits10,
"kMaxNumericSize is large enough");
static_assert(kMaxNumericSize - 10 > std::numeric_limits<long double>::digits10,
"kMaxNumericSize is large enough");
static_assert(kMaxNumericSize - 10 > std::numeric_limits<long>::digits10,
"kMaxNumericSize is large enough");
static_assert(kMaxNumericSize - 10 > std::numeric_limits<long long>::digits10,
"kMaxNumericSize is large enough");
}
std::numeric_limits::digits10 returns the number of significant digits of a decimal number of type T. for example, float has 6 significant digits, double has 15 significant digits, and int has 9 significant digits. The purpose of kMaxNumericSize-10 is to ensure that the kMaxNumericSize is large enough to make the available space of Small Buffer 10 bytes more than the type with the longest number of bits to be converted (1 byte can accommodate one bit).
Summary
So far, the core part of the front end of muduo log library has been finished. However, by default, data can only be output to stdout in a non thread safe manner, and asynchronous log messages cannot be recorded.
Key points:
1) The Logger provides a user interface to hide the implementation details in Impl. The Logger defines a set of macros to define LOG_XXX is convenient for users to use the log library at the front end;
2) Impl realizes the assembly of a complete log message except the body part;
3) LogStream provides operator < < format user body content, convert it into a string, and add it to the end of Small Buffer (4KB);
reference resources
https://blog.csdn.net/luotuo44/article/details/19252535
https://docs.microsoft.com/en-us/previous-versions/af6x78h6(v=vs.140)?redirectedfrom=MSDN