1, Log introduction
01 why do I need logs?
The code needs to go through different stages such as development, debugging, review, testing or online. The type of information you want to print during development may be completely different from the type of information you want to see after online. In other words, you may only want to see warning and error messages during "testing", but you may also want to see debugging related information during "debugging". If you want to print out the modules used and the running time of the code, the code can easily become chaotic. These problems can be easily solved by using the logging module.
02 what is a log?
Log is used to record information during system operation. It is also called log for recording an event.
03 what is the purpose of the log?
The basic purpose of the log is as follows:
- Record errors during program operation, facilitate tracking and positioning problems, and reduce debugging and maintenance costs;
- The log can restore the execution process of the whole program and understand the overall state of the program;
- Analyze user behavior and data statistics, and know which module the information comes from;
- When designing a test framework, you can also record the whole test process of the framework through the design log;
04 what are the levels of logs?
Common log levels are as follows:
Debug: debug level (Value=10), which prints very detailed log information. It is usually only used during debug, such as the intermediate state of each cycle in the algorithm;
INFO: information level (Value=20), printing general log information, highlighting the running process of the program, which is mainly used to deal with daily affairs such as requests or status changes;
ERROR: ERROR level (Value=40), printing ERROR and exception information. This level of ERROR may cause some functions of the system to fail to work normally, such as IO operation failure or connection problem;
CRITICAL: serious error (Value=50). A serious error may make the system unable to continue to run. For example, memory is exhausted and disk space is empty, which is rarely used;
05 implementation of log function
Almost all development languages have built-in log related functions, or there are excellent third-party libraries to provide log operation functions, such as log4j, log4php, etc. They are powerful and easy to use.
Python itself also provides a standard library module for logging - logging.
2, Logging module
01 introduction to logging module
The logging module is a built-in standard module in Python. It is mainly used to output operation logs. You can set the level of output logs, log saving path, log file rollback, etc.
02 advantages of logging module
Compared with print, it has the following advantages:
- You can set the log level in the logging module. You can set different output levels on different relase versions (such as development environment and production environment) to record the corresponding logs. Only important information is output without displaying a large amount of debugging information;
- The output information of print will be output to the standard output stream, which seriously affects developers to view other data from the standard output. The logging module is more flexible and can be set to output to any location, such as writing files, writing to remote servers, etc;
- The logging module has flexible configuration and formatting functions, such as configuring and outputting the current module information and running time. The developer decides where to output the information and how to output it. Compared with the string formatting of print, it is more convenient and easy to use.
03 composition of logging framework
Logger: the log exposes the function to the application, and determines which logs are valid based on the logger and filter level.
LogRecord: log recorder, which transfers the log to the corresponding processor for processing.
Handler: the processor that sends the log records (generated by the logger) to the appropriate destination.
Filter: filter, which provides better granularity control. It can determine which log records to output.
Formatter: formatter that indicates the layout of the log records in the final output.
04 specific parameters in logging function
filename: create a FiledHandler with the specified file name, so that the log will be stored in the specified file;
filemode: file opening method. This parameter is used when filename is specified. The default value is "w" and can also be specified as "a";
Format: Specifies the log display format used by the handler;
datefmt: Specifies the date time format
Level: sets the log level of the rootlogger;
Stream: create a StreamHandler with the specified stream;
05 simple log example
We try to output a log record of different log levels:
import logging
logging.debug("This is a debug log.")
logging.info("This is a info log.")
logging.warning("This is a warning log.")
logging.error("This is a error log.")
logging.critical("This is a critical log.")The output result is:
WARNING:root:This is a warning log.
ERROR:root:This is a error log.
CRITICAL:root:This is a critical log.
Why are debug and info level logs not output? The default log level provided by the logging module is WARNING, so only WARNING and above log levels are output.
Output content format description: log level: logger Name: log content. If the logger name is not customized, the default is root.
The following is the source code:
def getLogger(name=None):
"""
Return a logger with the specified name, creating it if necessary.
If no name is specified, return the root logger.
"""
if name:
return Logger.manager.getLogger(name)
else:
return root06 custom logger log
Set log collector and level:
# Define a log collector
logger = logging.getLogger('ITester')
# Set the collector level. If it is not set, it will collect logs of warning level and above by default
logger.setLevel('DEBUG')Format log:
fmt =logging.Formatter('%(filename)s-%(lineno)d-%(asctime)s-%(levelname)s-%(message)s')
Common log output formats:
Set log processor - output to file:
# output to a file
file_handler = logging.FileHandler('../log/mylog.txt')
# Set log processor level
file_handler.setLevel("DEBUG")
# The processor outputs the log in the specified format
file_handler.setFormatter(fmt)Create a new folder log under the project, and a new file mylog.txt under the folder to store logs.
Set log processor - output to console:
# Output to console
ch = logging.StreamHandler()
# Set log processor level
ch.setLevel("DEBUG")
# The processor outputs the log in the specified format
ch.setFormatter(fmt)Connect the collector to the processor and specify the output channel:
# Log output to file logger.addHandler(file_handler) # Log output to console logger.addHandler(ch)
The general flow chart of logger log is as follows:
We will combine the above independent introduction to test whether the function is normal. In the common directory, create a new file logger_handler.py.
import logging
# Define a log collector
logger = logging.getLogger('ITester')
# Set the collector level. If it is not set, it will collect logs of warning level and above by default
logger.setLevel('DEBUG')
# Format log
fmt =logging.Formatter('%(filename)s-%(lineno)d-%(asctime)s-%(levelname)s-%(message)s')
# Set log processor - output to file
file_handler = logging.FileHandler('../log/mylog.txt')
# Set log processor level
file_handler.setLevel("DEBUG")
# The processor outputs the log in the specified format
file_handler.setFormatter(fmt)
# Output to console
ch = logging.StreamHandler()
# Set log processor level
ch.setLevel("DEBUG")
# The processor outputs the log in the specified format
ch.setFormatter(fmt)
# The collector interfaces with the processor to specify the output channel
# Log output to file
logger.addHandler(file_handler)
# Log output to console
logger.addHandler(ch)
if __name__ == '__main__':
logger.debug('Custom debug journal')
logger.info('Custom info journal')
logger.warning('Custom warning journal')
logger.error('Custom error journal')
logger.critical('Custom critical journal')Console output effect:
File output effect:
