[ten thousand words long text] let you know the past and present life of the log

Logs are like vehicle insurance. No one is willing to pay for insurance, but once something goes wrong, everyone wants to have insurance available

Function and purpose of log

log file

Log file is a collection of files used to record system operation events, which can be divided into event log and message log. It plays an important role in processing historical data, diagnosing problem tracking and understanding system activities.

In a computer, a log file is a file that records events that occur in a running operating system or other software, or messages sent between users of network chat software. Logging refers to the behavior of saving logs. The simplest way is to write the log to a single log file.

Why print logs

Why or when to print the log depends on the purpose of printing. Different printing purposes determine the format, location and frequency of log output

Debugging and development: the purpose is to use it when developing debugging programs. It should only appear in the development cycle, not on-line system output
User behavior log: records user operation behavior, which is mostly used for big data analysis, such as monitoring, risk control, recommendation, etc
Program running log: records the running conditions of the program, especially the unexpected behavior and abnormal conditions, which are mainly used for development, maintenance and use
Machine log: it mainly records the network request, system CPU, memory, IO usage, etc. for operation and maintenance or monitoring

What should be included in the log

Analysis with 4W1H

When: print the time stamp of the log. The time at this time should be the time when the events recorded in the log occur. The specific time can help us analyze the time point at which the time occurs
Where: where is the log recorded, which module, which file, which function and which line of code
What: what is the main body of the log? Briefly describe the events recorded in the log
Who: the unique identifier of the event producer. Taking the order as an example, it is the order id. of course, it can also be the declaration of an action
How: the importance of logs is classified by error > warning > info > debug > trace

Past and present life of Java log

Why use log framework

The development of software system has been very complex, especially in server-side software, which involves too many knowledge and content problems. Using other people's mature framework in some aspects is equivalent to letting others help you complete some basic work. You only need to concentrate on completing the business logic design of the system. Moreover, the framework is generally mature and robust. It can help you deal with many details, such as asynchronous processing of logs, dynamic control and so on. In addition, the framework is generally used by many people, so its structure and scalability are very good.

Existing logging framework

According to the log facade and log implementation, there are several existing Java log frameworks

Log facade: JCL, Slf4j
Log implementation: JUL, Logback, Log4j, Log4j2

picture

Why log facade

When our system becomes more complex, our logs are prone to confusion. With the development of the system, different log frameworks may be updated, resulting in different log dependencies in the current system, which makes it difficult for us to manage and control uniformly. Even if we force the projects developed in our company to use the same logging framework, other third-party frameworks such as Spring or Mybatis will be referenced in the system. They depend on us to specify different logging frameworks. Moreover, their own logging systems are inconsistent, and there will still be confusion in the logging system.

Therefore, we can learn from the idea of JDBC and provide a set of facade for the logging system, so we can develop for these interface specifications and avoid directly relying on the specific logging framework. In this way, our system has a log facade and log implementation in the log.

Relationship between log facade and log implementation

picture

Log4j

Apache Log4j is a Java based logging tool, which is a project of the Apache Software Foundation. In jdk1 Before 3, there was no ready-made logging framework, and Java engineers could only use the original system out. println(), System. err. Println() or e.printStackTrace(). By writing the debug log to the StdOut stream and the error log to the ErrOut stream, the running state of the application is recorded. This original logging method has obvious defects. It can not be customized, and the output granularity of the log is not fine enough. In view of this, in 1999, Daniel Ceki G ü lc ü created the Log4j project and almost became the actual standard of the Java logging framework.

JUL

Log4j as a member of Apache foundation, Apache hopes to introduce log4j into JDK, but Sun company refused. Subsequently, sun imitated log4j and created a new version in jdk1 JUL (java.util.logging) is introduced in 4.

JCL

In order to decouple the log interface and implementation, Apache launched JCL (Jakarta common logging) in 2002, that is, common logging. Common logging defines a set of logging interfaces, and the specific implementation is completed by log4j or JUL. The logging interface defined by the underlying jloader or the commlogs library can be used to realize the function of dynamic logging. Therefore, it can only use the logging interface defined by the underlying jloader or the commlogs library when running.

SlF4j and Logback

Daniel Ceki G ü lc ü disagreed with the Apache foundation on the standards for common logging. Later, Ceki G ü lc ü left Apache and created slf4j and logback projects successively. Slf4j is a log facade that only provides interfaces. It can support log implementations such as logback, JUL and log4j. Logback provides specific implementations. Compared with log4j, it has faster execution speed and more perfect functions.

Log4j2

In order to maintain its position in the Java log arena and prevent JCL and log4j from being replaced by the combination of Slf4j and Logback, Apache launched log4j 2 in 2014. Log4j 2 is incompatible with log4j. After a lot of in-depth optimization, its performance is significantly improved.

Introduction to the principles of each log framework

Log4j

Log4j is an open source logging framework under Apache. By using log4j in the project, we can control the output of log information to the console, files and even databases. We can control the output format of each log. By defining the output level of the log, we can control the output process of the log more flexibly and conveniently.

Log4j's official website: http://logging.apache.org/log4j/1.2/

If you want to use Log4j in the project, you need to introduce the corresponding Jar package

1<dependency>
2    <groupId>log4j</groupId>
3    <artifactId>log4j</artifactId>
4    <version>1.2.17</version>
5</dependency>

Log4j is mainly composed of Loggers, Appenders, and Layout

Loggers

Loggers are mainly responsible for processing log records. The naming of loggers has an inheritance mechanism, such as com test. The logger of log will inherit the name com The logger of test.

There is a special logger called "root" in Log4j. It is the root of all loggers, which means that all other loggers will inherit from root directly or indirectly.

Appenders

Appender is used to specify where logs are output. You can specify the output destination of multiple logs at the same time. The output destinations of Log4j include the following sets.

Output type	effect
ConsoleAppender	Output log to console
FileAppender	Output log to file
DailyRollingFileAppender	Output the log to a log file, and output to a new file every day
RollingFileAppender	Output the log information to the log file, and according to the specified file size, when the file size reaches the specified size, the file will be renamed automatically and a new file will be generated at the same time
JDBCAppender	Save the log information to the database

Layouts

Layouts is used to control the format of log output content, so that we can output logs in various required formats. Log4j common layouts:

Formatter type	effect
HTMLLayout	Format log output in HTML tabular form
SimpleLayout	Simple log output formatting
PatternLayout	You can output logs in a custom format

 1* log4j Adopt similar C Linguistic printf The print format of the function formats the log information. The specific placeholders and their meanings are as follows:
 2        %m Output the log information specified in the code
 3        %p Output priority, and DEBUG,INFO etc.
 4        %n Line feed( Windows The newline character of the platform is "\n"，Unix Platform is "\n")
 5        %r Output from application startup to output log The number of milliseconds the message took
 6        %c Output the full name of the class to which the print statement belongs
 7        %t Output the full name of the thread that generated the log
 8        %d Output the current time of the server. The default is ISO8601，You can also specify a format, such as:%d{yyyy year MM month dd day
 9        HH:mm:ss}
10        %l The location where the output log time occurs, including class name, thread, and the number of lines in the code. For example:
11        Test.main(Test.java:10)
12        %F The name of the file where the output log message is generated
13        %L Line number in output code
14        %% Output one "%" character
15* Can be in % And characters to control the minimum width, maximum width, and how the text is aligned. For example:
16        %5c output category Name, minimum width is 5, category<5，Right justified by default
17        %-5c output category Name, minimum width is 5, category<5，"-"Number specifies the left alignment,There will be spaces
18        %.5c output category Name, maximum width is 5, category>5，Will cut off the extra characters on the left,<5 There will be no spaces
19        %20.30c category name<20 Fill in the blanks and align right,>30 Characters are cut off from the characters handed in for export on the left

JUL

The full name of JUL is Java util Logging. It is a native logging framework of Java. It does not need to refer to a third-party class library. Compared with other logging frameworks, JUL is easy to use, easy to learn, and can be used flexibly in small applications.

Architecture of JUL

picture

Logger: it is called a logger. The application obtains the logger object and calls its API to publish log information. Logger is usually the entry program for applications to access the logging system
Handler (similar to the Appenders of Log4j): each Logger will be associated with a group of Handlers, and the Logger will hand over the log to the associated Handlers for processing. This handler is an abstraction. Its specific implementation determines the location of logging, which can be console, file, database, etc
Layouts: also known as Formatters, it is responsible for formatting the log. Layouts determine the final form of data in a log record
Filters: filters, which can be customized according to needs, and which information will be recorded

To sum up, the user uses the Logger to record the log. The Logger holds several handlers. The log output operation is completed by the Handler. Before the Handler outputs, the information that does not need to be output will be filtered out through the custom Filter filter rules. Finally, the Handler decides what Layout to use to format the log and decide where to output it.

Next, let's write a simple introductory case to see how JUL handles logs

JUL log processing does not need to refer to any log framework. It is a function of Java

 1// 1. Get logger object
 2Logger logger = Logger.getLogger("com.macaque.JulLogTest");
 3//Turn off system default configuration
 4logger.setUseParentHandlers(false);
 5//Custom configuration log level
 6//Create console output handler
 7ConsoleHandler consoleHandler = new ConsoleHandler();
 8//Create a simple format conversion object
 9SimpleFormatter simpleFormatter = new SimpleFormatter();
10//Associate
11consoleHandler.setFormatter(simpleFormatter);
12logger.addHandler(consoleHandler);
13//Configure log specific level
14logger.setLevel(Level.ALL);
15consoleHandler.setLevel(Level.ALL);
16logger.severe("severe");
17logger.warning("waring");
18logger.info("info");
19logger.config("config");
20logger.fine("fine");
21logger.finer("finer");
22logger.finest("finest");

JCL

The full name is Jakarta common logging, which is a general logging API provided by Apache.

Its goal is to "provide a unified interface for all Java log implementations". It also provides a log implementation itself, but the function is very weak (SimpleLog). Therefore, JCL is generally not used alone. It allows developers to use different log implementation tools: Log4j and JDK's own log (JUL)

JCL has two basic abstract classes: Log and LogFactory

picture

How to use

If you want to use JCL in your project, you need to introduce the corresponding jar package

1<dependency>
2    <groupId>commons-logging</groupId>
3    <artifactId>commons-logging</artifactId>
4    <version>1.2</version>
5</dependency>

This only introduces the corresponding log facade, and the specific log implementation needs to be introduced by itself.

Principle introduction

When using JCL to print logs, the implementation class of Log is dynamically loaded by calling its LogFactory

1Log log = LogFactory.getLog(xxxx.class);

picture

Then, during initialization, you can traverse the array to find whether there is a matching implementation class. The initialization of the traversed array is

 1/**
 2 * The names of classes that will be tried (in order) as logging
 3 * adapters. Each class is expected to implement the Log interface,
 4 * and to throw NoClassDefFound or ExceptionInInitializerError when
 5 * loaded if the underlying logging library is not available. Any
 6 * other error indicates that the underlying logging library is available
 7 * but broken/unusable for some reason.
 8 */
 9private static final String[] classesToDiscover = {
10        "org.apache.commons.logging.impl.Log4JLogger",
11        "org.apache.commons.logging.impl.Jdk14Logger",
12        "org.apache.commons.logging.impl.Jdk13LumberjackLogger",
13        "org.apache.commons.logging.impl.SimpleLog"
14};

The logic that traverses this array

1for(int i=0; i<classesToDiscover.length && result == null; ++i) {
2    result = createLogFromClass(classesToDiscover[i], logCategory, true);
3}

SlF4j

Simple Logging Facade For Java SlF4j is mainly used to provide a set of standard and standardized API framework for Java log access. Its main significance is to provide interfaces, and the specific implementation can be handed over to other logging frameworks. For general Java projects, the log framework will choose SlF4j API as the facade, coupled with a specific implementation framework, and use a bridge in the middle.

Official website: http://www.slf4j.org/

Slf4j is the most popular log facade in the market, which mainly provides two functions:

Binding of log framework
Bridging of log frame

Binding of logs

Slf4j supports various log frameworks, but slf4j only exists as a log facade and defines a log printing specification. There will be two cases. The categories of packages introduced for these two cases are slightly different.

Comply with the specification defined by Slf4j: if you comply with the log specification defined by Slf4j, you only need to introduce two packages, one is the dependency of Slf4j and the implementation of the log jar package that complies with its specification
Failure to comply with the specification defined by Slf4j: if the log specification defined by Slf4j is not complied with, three packages need to be introduced, one is the Slf4j dependency, one is the adapter package, and the other is the package that does not comply with the log specification defined by Slf4j

This is a picture given on the official website, which describes the binding process.

picture

Introduction to the underlying principle of log binding

Based on the underlying binding principle of JCL introduced above, we know that JCL is implemented through the log of detecting binding at startup through the polling mechanism, but it is different in Slf4j. We can use LoggerFactory Start with the getlogger method, and finally locate the findPossibleStaticLoggerBinderPathSet method of LoggerFactory, as shown below.

 1private static String STATIC_LOGGER_BINDER_PATH = "org/slf4j/impl/StaticLoggerBinder.class";
 2static Set<URL> findPossibleStaticLoggerBinderPathSet() {
 3    Set<URL> staticLoggerBinderPathSet = new LinkedHashSet<URL>();
 4    try {
 5        ClassLoader loggerFactoryClassLoader = LoggerFactory.class.getClassLoader();
 6        Enumeration<URL> paths;
 7        if (loggerFactoryClassLoader == null) {
 8            paths = ClassLoader.getSystemResources(STATIC_LOGGER_BINDER_PATH);
 9        } else {
10        //This is the key point. Find all org / slf4j / impl / staticloggerbinder. Org through the class loader Class
11            paths = loggerFactoryClassLoader.getResources(STATIC_LOGGER_BINDER_PATH);
12        }
13        while (paths.hasMoreElements()) {
14            URL path = paths.nextElement();
15            staticLoggerBinderPathSet.add(path);
16        }
17    } catch (IOException ioe) {
18        Util.report("Error getting resources from path", ioe);
19    }
20    return staticLoggerBinderPathSet;
21}

Therefore, the loading process is as follows

Slf4j specific implementation of loading logs through LoggerFactory
During the initialization of LoggerFactory, the specific log implementation will be bound through the performeinitialization () method
When binding the specific implementation, load org / slf4j / impl / staticloggerbinder. Com through the class loader Class class
So, as long as it's a log implementation framework, it's on org Slf4j. The impl package provides its own StaticLoggerBinder class, in which the LoggerFactory that provides the specific log implementation can be loaded and managed by Slf4j

Bridging of log frame

In some old projects, the Slf4j framework may not be used at the beginning. If you want to upgrade the log at this time, Slf4j also provides such a function and provides a corresponding bridge to replace the original log framework. The following figure shows how to bridge the log on the official website. In fact, it is simply to redirect the original log to Slf4j and then hand it over to Slf4j for management.

picture

It may be difficult to understand the meaning of bridging. Let's directly use an example to demonstrate how Slf4j replaces the original logging framework.

First, we set up a project. First, we use Log4j to print logs and introduce the jar package of Log4j

1<dependency>
2    <groupId>log4j</groupId>
3    <artifactId>log4j</artifactId>
4    <version>1.2.17</version>
5</dependency>

Then simply add the configuration of Log4j to print the log

1@Test
2public void testLog4jToSlf4j(){
3    Logger logger = Logger.getLogger(TestSlf4jBridge.class);
4    logger.info("testLog4jToSlf4j");
5}

The output of the console is as follows. Because the log format is not processed, it simply outputs a string.

picture

Next, we need to upgrade the log framework by adding and removing some dependent packages without changing the code. Let's assume that we want to upgrade to Logback, and follow the following steps.

Remove the original log framework (here is the log framework of Log4j)
After removing the original log framework, the code must report an error, so add the log bridge of Log4j
Add slf4j API dependency
Then add the log implementation dependency of Logback

After completing these four steps, the log framework is upgraded. Next, let's take a look at the effect. Here, format processing is added to the log output of Logback. You can see that the log has been printed by Logback.

picture

Logback

Logback is another open source log component designed by the founder of Log4j. Its performance is better than that of Log4j. The official website: https://logback.qos.ch/index.html

Logback is mainly divided into three modules

Logback core: the basic module of the other two modules
Logback classic: it is an improved version of Log4j, and it fully implements the API of Slf4j
Logback access: the access module is integrated with the Servlet container to access logs through Http

The subsequent logs build the log system through the Slf4j log facade, so there is no difference in the code, mainly by changing the configuration file and pom dependency.

pom dependency

 1<dependency>
 2    <groupId>org.slf4j</groupId>
 3    <artifactId>slf4j-api</artifactId>
 4    <version>1.7.26</version>
 5</dependency>
 6<dependency>
 7    <groupId>ch.qos.logback</groupId>
 8    <artifactId>logback-classic</artifactId>
 9    <version>1.2.3</version>
10</dependency>

Basic configuration

logback will read the following types of configuration files in turn:

logback.groovy
logabck-test.xml
logback.xml

 1<?xml version="1.0" encoding="UTF-8"?>
 2<configuration>
 3<!--
 4Log output format:
 5%-5level
 6%d{yyyy-MM-dd HH:mm:ss.SSS}date
 7%c The full name of the class
 8%M by method
 9%L Line number
10%thread Thread name
11%m perhaps%msg For information
12%n Line feed
13-->
14<!--Format output:%d Indicates the date,%thread Represents the thread name,%-5level: The level is displayed 5 characters wide from the left
15%msg: Log messages,%n Is a newline character-->
16<property name="pattern" value="%d{yyyy-MM-dd HH:mm:ss.SSS} %c [%thread]
17%-5level %msg%n"/>
18<!--
19Appender: Set the destination of log information,The following are commonly used
20ch.qos.logback.core.ConsoleAppender (Console)
213. FileAppender to configure
22ch.qos.logback.core.rolling.RollingFileAppender (The file size reaches the specified size
23When a new file is generated)
24ch.qos.logback.core.FileAppender (file)
25-->
26<appender name="console" class="ch.qos.logback.core.ConsoleAppender">
27<!--Output stream object default System.out Change to System.err-->
28<target>System.err</target>
29<!--Log format configuration-->
30<encoder
31class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
32<pattern>${pattern}</pattern>
33</encoder>
34</appender>
35<!--
36Used to set the log printing level of a package or a specific class, and specify<appender>. 
37<loger>Only one name Property, an optional level And an optional addtivity attribute
38name:
39Used to specify that this logger A package or a specific class of constraints.
40level:
41Used to set the printing level, regardless of case: TRACE, DEBUG, INFO, WARN, ERROR, ALL and
42OFF，
43If this property is not set, the current logger Will inherit the level of the superior.
44additivity:
45Report to superior loger Transfer printing information. Default is true. 
46<logger>Can contain zero or more<appender-ref>Element that identifies this appender Will be added to this
47logger
48-->
49<!--
50Also<logger>Element, but it is the root logger. default debug
51level:Used to set the printing level, regardless of case: TRACE, DEBUG, INFO, WARN, ERROR, ALL
52and OFF，
53<root>Can contain zero or more<appender-ref>Element that identifies this appender Will be added to this
54logger. 
55-->
56<root level="ALL">
57<appender-ref ref="console"/>
58</root>
59</configuration>

FileAppender configuration

 1<?xml version="1.0" encoding="UTF-8"?>
 2<configuration>
 3<!-- Custom properties Can pass ${name}Reference-->
 4<property name="pattern" value="[%-5level] %d{yyyy-MM-dd HH:mm:ss} %c %M
 5%L [%thread] %m %n"/>
 6      <!--
 7        Log output format:
 8        %d{pattern}date
 9        %m perhaps%msg For information
10        %M by method
11        %L Line number
12        %c The full name of the class
13        %thread Thread name
14        %n Line feed
15        %-5level
16      -->
17<!-- Log file storage directory -->
18<property name="log_dir" value="d:/logs"></property>
19<!--console output  appender object-->
20<appender name="console" class="ch.qos.logback.core.ConsoleAppender">
21<!--Output stream object default System.out Change to System.err-->
22<target>System.err</target>
23<!--Log format configuration-->
24<encoder
25class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
26<pattern>${pattern}</pattern>
27</encoder>
28</appender>
29<!--Log file output appender object-->
30<appender name="file" class="ch.qos.logback.core.FileAppender">
31<!--Log format configuration-->
32<encoder
33class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
34<pattern>${pattern}</pattern>
35</encoder>
36<!--Log output path-->
37<file>${log_dir}/logback.log</file>
38</appender>
39<!-- generate html format appender object -->
40<appender name="htmlFile" class="ch.qos.logback.core.FileAppender">
41<!--Log format configuration-->
42<encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder">
43<layout class="ch.qos.logback.classic.html.HTMLLayout">
44<pattern>%level%d{yyyy-MM-dd
45HH:mm:ss}%c%M%L%thread%m</pattern>
46</layout>
47</encoder>
48<!--Log output path-->
49<file>${log_dir}/logback.html</file>
50</appender>
51<!--RootLogger object-->
52<root level="all">
53<appender-ref ref="console"/>
54<appender-ref ref="file"/>
55<appender-ref ref="htmlFile"/>
56</root>
57</configuration>

RollingFileAppender configuration

 1<?xml version="1.0" encoding="UTF-8"?>
 2<configuration>
 3<!-- Custom properties Can pass ${name}Reference-->
 4<property name="pattern" value="[%-5level] %d{yyyy-MM-dd HH:mm:ss} %c %M
 5%L [%thread] %m %n"/>
 6      <!--
 7        Log output format:
 8        %d{pattern}date
 9        %m perhaps%msg For information
10        %M by method
11        %L Line number
12        %c The full name of the class
13        %thread Thread name
14        %n Line feed
15        %-5level
16      -->
17<!-- Log file storage directory -->
18<property name="log_dir" value="d:/logs"></property>
19<!--console output  appender object-->
20<appender name="console" class="ch.qos.logback.core.ConsoleAppender">
21  <!--Output stream object default System.out Change to System.err-->
22  <target>System.err</target>
23  <!--Log format configuration-->
24  <encoder
25  class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
26  <pattern>${pattern}</pattern>
27  </encoder>
28</appender>
29<!-- Log file splitting and archiving appender object-->
30<appender name="rollFile"
31class="ch.qos.logback.core.rolling.RollingFileAppender">
32  <!--Log format configuration-->
33  <encoder
34  class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
35  <pattern>${pattern}</pattern>
36  </encoder>
37  <!--Log output path-->
38  <file>${log_dir}/roll_logback.log</file>
39  <!--Specify log file splitting and compression rules-->
40  <rollingPolicy
41    class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
42    <!--Determine how to split the file by specifying the name of the compressed file-->
43    <fileNamePattern>${log_dir}/rolling.%d{yyyy-MMdd}.
44    log%i.gz</fileNamePattern>
45    <!--File split size-->
46    <maxFileSize>1MB</maxFileSize>
47  </rollingPolicy>
48</appender>
49<!--RootLogger object-->
50<root level="all">
51  <appender-ref ref="console"/>
52  <appender-ref ref="rollFile"/>
53</root>
54</configuration>

Filter and asynchronous log configuration

 1<?xml version="1.0" encoding="UTF-8"?>
 2<configuration>
 3<!-- Custom properties Can pass ${name}Reference-->
 4<property name="pattern" value="[%-5level] %d{yyyy-MM-dd HH:mm:ss} %c %M
 5%L [%thread] %m %n"/>
 6<!--
 7Log output format:
 8%d{pattern}date
 9%m perhaps%msg For information
10%M by method
11%L Line number
12%c The full name of the class
13%thread Thread name
14%n Line feed
15%-5level
16-->
17<!-- Log file storage directory -->
18<property name="log_dir" value="d:/logs/"></property>
19<!--console output  appender object-->
20<appender name="console" class="ch.qos.logback.core.ConsoleAppender">
21<!--Output stream object default System.out Change to System.err-->
22<target>System.err</target>
23<!--Log format configuration-->
24<encoder
25class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
26<pattern>${pattern}</pattern>
27</encoder>
28</appender>
29<!-- Log file splitting and archiving appender object-->
30<appender name="rollFile"
31class="ch.qos.logback.core.rolling.RollingFileAppender">
32<!--Log format configuration-->
33<encoder
34class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
35<pattern>${pattern}</pattern>
36</encoder>
37<!--Log output path-->
38<file>${log_dir}roll_logback.log</file>
39<!--Specify log file splitting and compression rules-->
40<rollingPolicy
41class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
42<!--Determine how to split the file by specifying the name of the compressed file-->
43<fileNamePattern>${log_dir}rolling.%d{yyyy-MMdd}.
44log%i.gz</fileNamePattern>
45<!--File split size-->
46<maxFileSize>1MB</maxFileSize>
47</rollingPolicy>
48<!--filter to configure-->
49<filter class="ch.qos.logback.classic.filter.LevelFilter">
50<!--Set interception log level-->
51<level>error</level>
52<onMatch>ACCEPT</onMatch>
53<onMismatch>DENY</onMismatch>
54</filter>
55</appender>
56<!--Asynchronous log-->
57<appender name="async" class="ch.qos.logback.classic.AsyncAppender">
58<appender-ref ref="rollFile"/>
59</appender>
60<!--RootLogger object-->
61<root level="all">
62<appender-ref ref="console"/>
63<appender-ref ref="async"/>
64</root>
65<!--custom logger additivity Indicates whether to start from rootLogger Inheritance configuration-->
66<logger name="com.macaque" level="debug" additivity="false">
67<appender-ref ref="console"/>
68</logger>
69</configuration>

Log4j steering Logback

Official log4j Convert properties to logback Tools for XML file configuration: http://logback.qos.ch/translator/

Log4j2

Apache Log4j2 is an upgraded version of Log4j. It refers to some excellent designs of logback, fixes some problems and brings some significant improvements, mainly including:

Exception handling: in logback, exceptions in the Appender will not be perceived by the application, but some exception handling mechanisms are provided in log4j 2
Performance improvement: compared with log4j and logback, log4j2 has a significant performance improvement. There will be official test data later
Automatic reload configuration refers to the configuration of logback. Of course, automatic refresh parameter configuration will be provided. The most practical is to dynamically modify the log level in our production environment without restarting the application
No garbage mechanism. In most cases, log4j can use its designed garbage free mechanism to avoid jvm gc caused by frequent log collection

Official website: https://logging.apache.org/log4j/2.x/

How to use Log4j2

At present, the most mainstream log facade in the market is slf4j. Although Log4j2 itself is also a log facade, its log implementation function is very powerful and its performance is superior. Therefore, Log4j2 is generally regarded as the realization of log amount. Slf4j+Log4j2 should be the general trend in the future.

Add dependency (used with Slf4j)

 1<!--use slf4j As the facade of the log,use log4j2 To log -->
 2<dependency>
 3    <groupId>org.slf4j</groupId>
 4    <artifactId>slf4j-api</artifactId>
 5    <version>1.7.25</version>
 6</dependency>
 7<!--by slf4j Binding log implementation log4j2 Adapter for -->
 8<dependency>
 9    <groupId>org.apache.logging.log4j</groupId>
10    <artifactId>log4j-slf4j-impl</artifactId>
11    <version>2.10.0</version>
12</dependency>
13<!-- Log4j2 Facade API-->
14<dependency>
15    <groupId>org.apache.logging.log4j</groupId>
16    <artifactId>log4j-api</artifactId>
17    <version>2.11.1</version>
18</dependency>
19<!-- Log4j2 Log implementation -->
20<dependency>
21    <groupId>org.apache.logging.log4j</groupId>
22    <artifactId>log4j-core</artifactId>
23    <version>2.11.1</version>
24</dependency>

Configuration of Log4j2

The configuration of Log4j2 is the same as that of Logback

 1<?xml version="1.0" encoding="UTF-8"?>
 2<!--
 3    status="warn" The output log level of the log framework itself
 4    monitorInterval="5" The interval between automatic loading of configuration files shall not be less than 5 second
 5-->
 6<Configuration status="debug" monitorInterval="5">
 7    <!--
 8        Centralized configuration attribute management
 9        Use through:${name}
10    -->
11    <properties>
12        <property name="LOG_HOME">/logs</property>
13    </properties>
14    <!--Log processing-->
15    <Appenders>
16        <!--console output  appender-->
17        <Console name="Console" target="SYSTEM_ERR">
18            <PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] [%-5level] %c{36}:%L --- %m%n" />
19        </Console>
20        <!--Log file output appender-->
21        <File name="file" fileName="${LOG_HOME}/myfile.log">
22            <PatternLayout pattern="[%d{yyyy-MM-dd HH:mm:ss.SSS}] [%-5level] %l %c{36} - %m%n" />
23        </File>
24        <!--<Async name="Async">-->
25            <!--<AppenderRef ref="file"/>-->
26        <!--</Async>-->
27        <!--Log file output using random read and write appender，Performance improvement-->
28        <RandomAccessFile name="accessFile" fileName="${LOG_HOME}/myAcclog.log">
29            <PatternLayout pattern="[%d{yyyy-MM-dd HH:mm:ss.SSS}] [%-5level] %l %c{36} - %m%n" />
30        </RandomAccessFile>
31        <!--Of log files split according to certain rules appender-->
32        <RollingFile name="rollingFile" fileName="${LOG_HOME}/myrollog.log"
33                     filePattern="/logs/$${date:yyyy-MM-dd}/myrollog-%d{yyyy-MM-dd-HH-mm}-%i.log">
34            <!--Log level filter-->
35            <ThresholdFilter level="debug" onMatch="ACCEPT" onMismatch="DENY" />
36            <!--Log message format-->
37            <PatternLayout pattern="[%d{yyyy-MM-dd HH:mm:ss.SSS}] [%-5level] %l %c{36} - %msg%n" />
38            <Policies>
39                <!--When the system starts, start the split rule and produce a new log file-->
40                <OnStartupTriggeringPolicy />
41                <!--Split by file size, 10 MB -->
42                <SizeBasedTriggeringPolicy size="10 MB" />
43                <!--According to the node split rule, according to filePattern Defined-->
44                <TimeBasedTriggeringPolicy />
45            </Policies>
46            <!--In the same directory, the number of files is limited to 30 More than, overwrite-->
47            <DefaultRolloverStrategy max="30" />
48        </RollingFile>
49    </Appenders>
50    <!--logger definition-->
51    <Loggers>
52        <!--Custom asynchronous logger object
53            includeLocation="false" Turn off the line number information of logging
54            additivity="false" Not in inheritance rootlogger object
55        -->
56        <AsyncLogger name="com.macaque" level="trace" includeLocation="false" additivity="false">
57            <AppenderRef ref="Console"/>
58        </AsyncLogger>
59        <!--use rootLogger to configure log level level="trace"-->
60        <Root level="trace">
61            <!--Specifies the processor used by the log-->
62            <AppenderRef ref="Console" />
63            <!--Use asynchronous appender-->
64            <AppenderRef ref="Async" />
65        </Root>
66    </Loggers>
67</Configuration>

Asynchronous log

The biggest feature of log4j2 is asynchronous logging, and its performance improvement also benefits from asynchronous logging. Log4j2 provides two implementations of asynchronous logging, one is AsyncAppender, and the other is asyncloger, which corresponds to the Apperder component and Logger component mentioned above.

If you want to use asynchronous logging, you need to introduce an additional Jar package

1<dependency>
2    <groupId>com.lmax</groupId>
3    <artifactId>disruptor</artifactId>
4    <version>3.4.2</version>
5</dependency>

The official website does not recommend using the AsyncAppender mode at present, so I won't introduce it here. I'll focus on the logs of asyncloger. There are two options for asyncloger: Global asynchrony and hybrid asynchrony.

Global asynchrony means that all logs are asynchronous records. There is no need to change the configuration file. Just add a global system configuration: - dlog4j2 contextSelector=org. apache. logging. log4j. core. async. AsyncLoggerContextSelector
Hybrid asynchrony means that you can use both synchronous and asynchronous logs in your application, which makes the log configuration more flexible

 1<?xml version="1.0" encoding="UTF-8"?>
 2<Configuration status="WARN">
 3    <properties>
 4        <property name="LOG_HOME">D:/logs</property>
 5    </properties>
 6    <Appenders>
 7        <File name="file" fileName="${LOG_HOME}/myfile.log">
 8            <PatternLayout>
 9                <Pattern>%d %p %c{1.} [%t] %m%n</Pattern>
10            </PatternLayout>
11        </File>
12        <Async name="Async">
13            <AppenderRef ref="file"/>
14        </Async>
15    </Appenders>
16    <Loggers>
17        <AsyncLogger name="com.macaque" level="trace"
18                     includeLocation="false" additivity="false">
19            <AppenderRef ref="file"/>
20        </AsyncLogger>
21        <Root level="info" includeLocation="true">
22            <AppenderRef ref="file"/>
23        </Root>
24    </Loggers>
25</Configuration>

The above configuration: com The macaque log is asynchronous and the root log is synchronous

There are two issues to note when using asynchronous logging

If asynchronous logging is used, AsyncApperder, AsyncLogger and global logging should not appear at the same time. OK, it will be consistent with AsyncApperder and minimized.
If includeLocation=false is set, printing location information will dramatically reduce the performance of asynchronous logs, which is slower than synchronous logs

Performance of Log4j2

The most powerful thing about Log4j2 is its performance when outputting logs asynchronously. The throughput of Log4j2 in a multi-threaded environment is compared with that of Log4j and Logback. The figure provided on the official website. You can see that the performance of using global asynchronous mode is the best, followed by using hybrid asynchronous mode.

picture

Best practices for printing logs

Insisting on doing simple things well is not simple, and insisting on doing ordinary things well is not ordinary. The so-called success is to make extraordinary persistence in the ordinary!

Good logging can provide us with enough basis for locating problems. Everyone will think that logging is very simple, but how to efficiently locate the problem through logging is not a simple thing.

How to keep a log is more convenient for us to check problems

Encapsulation of external calls

The dependence on external systems and modules in the program is recorded before and after the call, which is convenient for interface debugging. When something goes wrong, you can quickly sort out the problem.

 1boolean debugEnabled = logger.isDebugEnabled();
 2if (debugEnabled){
 3    logger.debug("Calling external system : {}",requestParam);
 4}
 5try{
 6    result = callRemoteSystem(requestParam);
 7    if (debugEnabled){
 8        logger.debug("Called successfully result is :{}",result);
 9    }
10}catch (BusinessException e){
11    logger.warn("Failed at calling xxx system request:{}",requestParam,e);
12}catch (Exception e){
13    logger.error("Failed at calling xxx system Exception request:{}",requestParam,e);
14}

State change

Important status information changes in the program should be recorded to facilitate the restoration of the scene when checking problems and inferring the running process of the program.

System inlet and outlet

This granularity can be important at the method or module level, recording its input and output for easy positioning.

Business exception

Any business exception should be recorded and the exception stack should be output.

Rare else situation

Rare else situations may swallow your request or give an incomprehensible end result

What logging methods should be avoided

Log of confusing information

The log should be clear and accurate. For example, when you see the following log, do you know the problem caused by the connection pool unable to get the connection?

1  Connection connection = ConnectionFactory.getConnection();  
2  if (connection == null) {  
3      LOG.warn("System initialized unsuccessfully");  
4  }

Logging at all levels

Whether it is an exception or an input parameter request, the level of using the print log is info level, and there is no distinction between levels. There are two bad things about this.

Print logs cannot be physically distinguished into different files
A large number of invalid logs are output, which is not conducive to the improvement of system performance and the rapid positioning of error points

Missing key information

There may be two situations

Under normal circumstances, key information, such as the order ID of the order placing process, is not printed
The exception stack is not printed under abnormal conditions

Dynamic splicing string

Splicing using String strings will use the append() method of StringBuilder, which has a certain performance loss. Using placeholders is just a replacement action, which can effectively improve performance.

Duplicate print log

Avoid printing logs repeatedly and wasting disk space. Be sure to set additivity = false in the log configuration file

Unswitched log output

1logger.debug("Called successfully result is :{}", JSONObject.toJSONString(result));

In this case, the debug method will be called and the log output will be changed to JSON. In other words, the debug parameter will not be printed. It wastes the performance of method calls.

Input all logs into one file

Different levels of log information should be output to different log files. Distinguishing the information can not only effectively locate the problem, but also retain the site for a longer time.

source code

All the source codes involved in the log are listed in: https://github.com/modouxiansheng/Macaque/tree/master/macaque-log In, you can download and modify the configuration file to understand for yourself.

Reference articles

Added by jaytux on Wed, 09 Mar 2022 15:39:40 +0200

Programming VIP

[ten thousand words long text] let you know the past and present life of the log

Function and purpose of log

log file

Why print logs

What should be included in the log

Past and present life of Java log

Why use log framework

Existing logging framework

Why log facade

Relationship between log facade and log implementation

Log4j

JUL

JCL

SlF4j and Logback

Log4j2

Introduction to the principles of each log framework

Log4j

Loggers

Appenders

Layouts

JUL

JCL

How to use

Principle introduction

SlF4j

Binding of logs

Introduction to the underlying principle of log binding

Bridging of log frame

Logback

pom dependency

Basic configuration

FileAppender configuration

RollingFileAppender configuration

Filter and asynchronous log configuration

Log4j steering Logback

Log4j2

How to use Log4j2

Configuration of Log4j2

Asynchronous log

Performance of Log4j2

Best practices for printing logs

How to keep a log is more convenient for us to check problems

What logging methods should be avoided

source code

Reference articles

Popular Keywords