Turn:
Detailed explanation of python log logging module
Article catalogue
- 1. Log related concepts
-
- 1.1 function of log
- 1.2 log level
- 1.3 there are two ways to use the logging module
- 2 use the module level functions provided by logging
-
- 2.1 the logging module defines common functions
- 2.2 usage 1: simple configuration
- 2.3 usage 2: using logging Basicconfig() function
- 3. Use the four components of the Logging system
-
- 3.1 Logger class
- 3.2 Handler class
- 3.3 Formater class
- 3.4 Filter class (just understand)
- 3.5 brief process of log stream processing
1. Log related concepts
1.1 function of log
- Program debugging
- Know whether the program runs normally
- Fault analysis and problem location
- User behavior analysis
1.2 log level
Grade | meaning |
---|---|
DEBUG | The most detailed log information. The typical application scenario is problem diagnosis |
INFO | The level of detail of information is second only to DEBUG, which usually only records the information of key nodes to confirm that everything is working as expected |
WARNING | Information recorded when something unexpected happens (for example, low disk free space), but the application is still running normally |
ERROR | A message recorded when some functions cannot function properly due to a more serious problem |
CRITICAL | Information recorded when a serious error occurs that prevents the application from continuing to run |
By default, the logging module prints the log information with the level of WARNING and above to the console
1.3 there are two ways to use the logging module
The logging module can be used in two ways
- The first way is to use the module level functions provided by logging
- The second way is to use the four components of the Logging system
2 use the module level functions provided by logging
2.1 the logging module defines common functions
function | explain |
---|---|
logging.debug(msg,*args,**kwargs) | Create a log record with a severity of DEBUG |
logging.info(msg,*args,*kwargs) | Create a log record with severity INFO |
logging.warning(msg,*args,*kwargs) | Create a log record with severity WARNING |
logging.error(msg,*args,*kwargs) | Create a log record with severity ERROR |
logging.critical(msg,*args,**kwargs) | Create a log record with CRITICAL severity |
logging.log(level,*args,*kwargs) | Create a log record with severity level |
logging.basicConfig(**kwargs) | One time configuration of root logger |
Here is a demonstration:
2.2 usage 1: simple configuration
import logging logging.debug("debug message") logging.info("info message") logging.warning("warning message") logging.error("error message") logging.critical("critical message") logging.log(level=logging.ERROR, msg = "error in logging.log function")
Output result:
WARNING:root:warning message ERROR:root:error message CRITICAL:root:critical message ERROR:root:error in logging.log function
By default, the logging module prints logs to the standard output and only displays logs with WARNING level greater than or equal to, which indicates that the default log level is set to WARNING (log level critical > error > WARNING > info > debug)
2.3 usage 2: using logging Basicconfig() function
Use logging Basicconfig() function can adjust log level, output format, etc
logging.basicConfig() function description
Parameter name | describe |
---|---|
filename | Specify the file name of the log output target file. After specifying this setting, the log information will not be output to the console |
format | Specify the log format string, that is, specify the field information contained in the log output and their order. The format fields defined by the logging module are listed below. |
datefmt | Specify the date / time format. It should be noted that this option is only valid when the format contains the time field% (actime) s |
level | Specifies the log level of the logger |
stream | Specify the log output target stream, such as sys stdout,sys.stderr and network stream. It should be noted that stream and filename cannot be provided at the same time, otherwise ValueError exception will be thrown |
style | Python3.2 newly added configuration item. Specify the style of format string. The values can be '%', '{' and '$', and the default is'% ' |
handlers | New configuration item added in Python 3.3. If this option is specified, it should be an iterative object that creates multiple handlers, which will be added to the rootlogger. It should be noted that only one of the three configuration items filename, stream and handlers can exist, and two or three cannot occur at the same time. Otherwise, ValueError exception will be thrown. |
Format string of logging module
Field / attribute name | Use format | describe |
---|---|---|
asctime | %(asctime)s | Time of log event – human readable time, such as: 2003-07-08 16:49:45896 |
created | %(created)f | The time when the log event occurs – timestamp, which is the time when it is called The value returned by the time() function |
relativeCreated | %(relativeCreated)d | The number of milliseconds relative to the loading time of the logging module when the log event occurs (I don't know why to use it at present) |
msecs | %(msecs)d | The millisecond portion of the log event occurrence event |
levelname | %(levelname)s | The log level in text form of the log record ('DEBUG ',' INFO ',' WARNING ',' ERROR ',' CRITICAL ') |
levelno | %(levelno)s | The log level (10, 20, 30, 40, 50) of the digital form of the log record |
name | %(name)s | The name of the logger used is "root" by default, because rootLogger is used by default |
message | %(message)s | The text content of the log record is calculated by MSG% args |
pathname | %(pathname)s | The full path of the source file that calls the logging function |
filename | %(filename)s | The file name part of pathname, including the file suffix |
module | %(module)s | The name part of filename, excluding the suffix |
lineno | %(lineno)d | The line number of the source code that calls the logging function |
funcName | %(funcName)s | The name of the function that called the logging function |
process | %(process)d | Process ID |
processName | %(processName)s | Process name, Python 3.1 NEW |
thread | %(thread)d | Thread ID |
threadName | %(thread)s | Thread name |
# coding=utf-8 import logging MY_FORMAT = "%(asctime)s %(name)s %(levelname)s %(pathname)s %(lineno)d %(message)s" # Configure output log format DATE_FORMAT = '%Y-%m-%d %H:%M:%S %a ' #Configure the format of the output time logging.basicConfig( filename="my.log", # Specifies that the log is written to a file level=logging.INFO, datefmt=DATE_FORMAT, format=MY_FORMAT, ) logging.debug("debug") logging.info("info") logging.warning("warning") logging.error("error") logging.critical("critical")
Open the file my Log, as follows:
2020-11-22 19:18:58 Sun root INFO E:/prapy/python_project/testcase/test1.py 15 info 2020-11-22 19:18:58 Sun root WARNING E:/prapy/python_project/testcase/test1.py 16 warning 2020-11-22 19:18:58 Sun root ERROR E:/prapy/python_project/testcase/test1.py 17 error 2020-11-22 19:18:58 Sun root CRITICAL E:/prapy/python_project/testcase/test1.py 18 critical
explain:
- logging.basicConfig() function is a one-time simple configuration tool, which means that it will work only when the function is called for the first time, and no operation will be generated when the function is called again later. The setting of multiple calls is not an accumulation operation.
- The Logger has a hierarchical relationship. The Logger used by the logging module level function called above is an instance of RootLogger class, whose name is' root '. It is the Logger at the top level of the Logger hierarchical relationship, and the instance exists in singleton mode.
- If the log to be recorded contains variable data, you can use a format string as the description message of the event (the first parameter of logging.debug, logging.info and other functions), and then pass the variable data as the value of the second parameter * args,
>>> import logging >>> logging.warning('%s is %d years old.', 'Tom', 10) WARNING:root:Tom is 10 years old.
3. Use the four components of the Logging system
Above, we learned about logging debug(),logging.info(),logging.warning(),logging.error(),logging.critical() (used to record different levels of log information respectively), logging Basicconfig() (use the default log format to establish a default stream handler for the log system, set the basic configuration (such as log level) and add it to the root logger) functions at the logging module level.
The following describes the second method of printing logs, log stream processing, using the function logging Getlogger ([name]) (returns a logger object. If no name is specified, it will return root logger).
Before introducing the log stream processing flow of the logging module, let's introduce the four components of the logging module:
Component name | Corresponding class name | Function description |
---|---|---|
Logger | Logger | Provides an interface that applications can always use |
processor | Handler | Send the log record created by logger to the appropriate destination for output |
filter | Filter | It provides finer grained control tools to determine which log records are output and which log records are discarded |
Formatter | Formatter | Determines the final output format for logging |
Description of the relationship between these components:
- The logger needs to output the log information to the target location through the handler, such as file and sys Stdout, network, etc;
- Different handler s can output logs to different locations;
- The logger can set multiple handler s to output the same log record to different locations;
- Each handler can set its own filter to filter logs, so that only interested logs can be retained;
- Each handler can set its own formatter to output the same log to different places in different formats.
To put it simply: the logger is the entry, and the real work is the handler. The handler can also filter and format the log content to be output through the filter and formatter.
Introduction to related classes and common methods of logging module
Classes related to the four components of logging: logger, handler, filter and formatter.
3.1 Logger class
The Logger object has three tasks to do:
- Expose several methods to the application code so that the application can record log messages at run time;
- Decide which logs to perform subsequent processing based on the log severity level (default filtering facility) or filter object;
- Send log messages to all interested log handlers.
The most commonly used methods for Logger objects are divided into two categories: configuration methods and message sending methods
Logger class related methods
method | describe |
---|---|
Logger.setLevel() | Sets the minimum severity level of log messages that the logger will process |
Logger.addHandler() and logger removeHandler() | Add and remove a handler object for the logger object |
Logger.addFilter() and logger removeFilter() | Add and remove a filter object for the logger object |
After the logger object is configured, you can use the following method to create log records:
method | describe |
---|---|
Logger.debug(), Logger.info(), Logger.warning(), Logger.error(), Logger.critical() |
Create a log record corresponding to the level of their method name |
Logger.exception() | Similar to creating a logger Log message for error() |
Logger.log() | You need to get an explicit log level parameter to create a log record |
What about a Logger object? One way is to create an instance of the Logger class through the instantiation method of the Logger class, but we usually use the second way - logging Getlogger () method.
logging. The getLogger() method has an optional parameter name, which indicates the name and identification of the logger to be returned. If this parameter is not provided, its value is' root '. If the getLogger() method is called multiple times with the same name parameter value, a reference to the same logger object will be returned.
Note that multiple logger s cannot be created after multiple use, otherwise the log will be output repeatedly.
Description of the hierarchy and effective level of the logger:
- The name of the logger is one with '.' Split hierarchy, each ' The following loggers are '. " The children of the previous logger, for example, have a logger named foo, and the other names are foo bar, foo. bar. Baz and foo BAM is the offspring of foo.
- Logger has a concept of "effective level". If a level is not explicitly set on a logger, the logger uses the level of its parent; If the level of its parent is not explicitly set, continue to look up the valid level of its parent, and so on until an ancestor with level explicitly set is found. It should be noted that the root logger always has a clear level setting (WARNING by default). When deciding whether to handle an event that has occurred, the validity level of the logger will be used to decide whether to pass the event to the handlers of the logger for processing.
- After processing the log messages, child loggers will pass the log messages to the handlers related to their ancestor loggers by default. Therefore, we do not need to define and configure handlers for all loggers used in an application. We only need to configure handlers for a top-level logger and then create child loggers as needed. We can also turn off this delivery mechanism by setting the propagate property of a logger to False.
3.2 Handler class
The handler object is used to distribute messages (based on the level of log messages) to the location specified by the handler (file, network, mail, etc.). The Logger object can add 0 or more handler objects for itself through the addHandler() method. For example, an application may want to implement the following logging requirements:
method | describe |
---|---|
Handler.setLevel(lel) | Specify the information level to be processed. Information below lel level will be ignored |
Handler.setFormatter() | Choose a format for this handler |
Handler.addFilter(filt),Handler.removeFilter(filt) | Add or delete a filter object |
It should be noted that the application code should not directly instantiate and use the Handler instance. Because Handler is a base class, it only defines the interfaces that all handlers should have, and provides some default behaviors that subclasses can directly use or override. Here are some commonly used handlers:
Handler | describe |
---|---|
logging.StreamHandler | Send log messages to the output Stream, such as std.out, std.err or any file like object. |
logging.FileHandler | Send log messages to disk files. By default, the file size will grow indefinitely |
logging.handlers.RotatingFileHandler | Send log messages to disk files and support cutting log files by size |
logging.hanlders.TimedRotatingFileHandler | Send log messages to disk files and support log file cutting by time |
logging.handlers.HTTPHandler | Send log messages to an HTTP server in the form of GET or POST |
logging.handlers.SMTPHandler | Send the log message to a specified email address |
logging.NullHandler | This Handler instance ignores error messages, which is usually used by library developers who want to use logging to avoid the message "No handlers could be found for logger XXX". |
3.3 Formater class
The Formater object is used to configure the final order, structure and content of log information. And logging Unlike the handler base class, the application code can directly instantiate the Formatter class. In addition, if your application needs some special processing behavior, you can also implement a subclass of Formatter.
The construction method of Formatter class is defined as follows:
logging.Formatter.__init__(fmt=None, datefmt=None, style='%')
It can be seen that the construction method receives three optional parameters:
- fmt: Specifies the message format string. If this parameter is not specified, the original value of message will be used by default
- datefmt: Specifies the date format string. If this parameter is not specified, "% Y -% m -% d% H:% m:% s" will be used by default
- style: a new parameter in Python 3.2, which can be taken as'% ',' {'and' $'. If this parameter is not specified,'% 'will be used by default
Generally, you can directly use logging Formatter(fmt, datefmt)
3.4 Filter class (just understand)
Filter can be used by Handler and logger to perform finer grained and more complex filtering functions than level. Filter is a filter base class, which only allows log events under a logger level to be filtered. This category is defined as follows:
class logging.Filter(name='') filter(record)
For example, if the value of the name parameter passed during the instantiation of a filter is' A.B ', the filter instance will only allow the log records generated by loggers with names similar to the following rules to be filtered:' A.B ',' A.B,C ',' A.B.C.D ',' A.B.D ', while the logs generated by loggers with names of' A.BB 'and' B.A.B 'will be filtered out. If the value of name is an empty string, all log events are allowed to pass the filter.
The filter method is used to specifically control whether the transmitted record can pass the filtering. If the return value of this method is 0, it means it cannot pass the filtering, and if the return value is non-0, it means it can pass the filtering.
3.5 brief process of log stream processing
1. Create a logger
2. Set the log level of the logger
3. Create an appropriate Handler(FileHandler should have a path)
4. Set the log level of each Handler
5. Create the format of the next log
6. Add the format created above to the Handler
7. Add the Handler created above to the logger
8. Printout logger debuglogger. infologger. warninglogger. errorlogger. critical
# coding=utf-8 import logging # Create a logger. If the parameter is empty, the root logger is returned logger = logging.getLogger("mylogger") logger.setLevel(logging.DEBUG) # Set logger log level # Create handler fh = logging.FileHandler("test.log", encoding="utf-8") ch = logging.StreamHandler() # Set the output log format and pay attention to logging Case of formatter formatter = logging.Formatter( fmt="%(asctime)s %(name)s %(filename)s %(message)s", datefmt="%Y/%m/%d %X" ) # Specify the output format for the handler, paying attention to case fh.setFormatter(formatter) ch.setFormatter(formatter) # Log processor added for logger logger.addHandler(fh) logger.addHandler(ch) # Output log s of different levels logger.warning("warning message") logger.info("info message") logger.error("error message")
Operation results
2020/11/22 21:00:24 mylogger test3.py warning message 2020/11/22 21:00:24 mylogger test3.py info message 2020/11/22 21:00:24 mylogger test3.py error message
python logging repeated log writing
When logging with Python's logging module, you may encounter the problem of repeatedly logging. The first record is written once, the second record is written twice, and the third record is written three times
Cause: the handler was not removed. Solution: removeHandler was removed after logging
# coding=utf-8 import logging def log(msg): #Create a logger. If the parameter is empty, the root logger is returned logger = logging.getLogger("mylogger") logger.setLevel(logging.DEBUG) #Set logger log level #Create handler fh = logging.FileHandler("test.log",encoding="utf-8") ch = logging.StreamHandler() #Set output log format formatter = logging.Formatter( fmt="%(asctime)s %(name)s %(filename)s %(message)s", datefmt="%Y/%m/%d %X" ) #Specify the output format for the handler fh.setFormatter(formatter) ch.setFormatter(formatter) #Log processor added for logger logger.addHandler(fh) logger.addHandler(ch) # Output log s of different levels logger.info(msg) # Output log s of different levels log("message1") log("message2") log("message3")
Operation results
2020/11/22 21:08:04 mylogger test3.py message1 2020/11/22 21:08:04 mylogger test3.py message2 2020/11/22 21:08:04 mylogger test3.py message2 2020/11/22 21:08:04 mylogger test3.py message3 2020/11/22 21:08:04 mylogger test3.py message3 2020/11/22 21:08:04 mylogger test3.py message3
Analysis: you can see that the output results are printed repeatedly
Reason: when calling log for the second time, get the same logger according to the name in getLogger(name), and there is already a handler you added for the first time in this logger, and a handler is added for the second call. Therefore, there are two identical handlers in this logger, and so on. There will be several handlers after calling several times.
Solution 1: add removeHandler statement
# coding=utf-8 import logging def log(msg): # Create a logger. If the parameter is empty, the root logger is returned logger = logging.getLogger("mylogger") logger.setLevel(logging.DEBUG) # Set logger log level # Create handler fh = logging.FileHandler("test.log", encoding="utf-8") ch = logging.StreamHandler() # Set output log format formatter = logging.Formatter( fmt="%(asctime)s %(name)s %(filename)s %(message)s", datefmt="%Y/%m/%d %X" ) # Specify the output format for the handler fh.setFormatter(formatter) ch.setFormatter(formatter) # Log processor added for logger logger.addHandler(fh) logger.addHandler(ch) # Output log s of different levels logger.info(msg) # Solution 1: add a removeHandler statement and remove the Handler after each use logger.removeHandler(fh) logger.removeHandler(ch) # Output log s of different levels log("message1") log("message2") log("message3")
Solution 2: make a judgment in the log method. If the logger already has a handler, no handler will be added.
# coding=utf-8 import logging def log(msg): # Create a logger. If the parameter is empty, the root logger is returned logger = logging.getLogger("mylogger") logger.setLevel(logging.DEBUG) # Set logger log level if not logger.handlers: # Create handler fh = logging.FileHandler("test.log", encoding="utf-8") ch = logging.StreamHandler() # Set output log format formatter = logging.Formatter( fmt="%(asctime)s %(name)s %(filename)s %(message)s", datefmt="%Y/%m/%d %X" ) # Specify the output format for the handler fh.setFormatter(formatter) ch.setFormatter(formatter) # Log processor added for logger logger.addHandler(fh) logger.addHandler(ch) # Output log s of different levels logger.info(msg) # Output log s of different levels log("message1") log("message2") log("message3")
Example of logger calling method
# coding=utf-8 import logging.handlers import datetime def get_logger(): logger = logging.getLogger('mylogger') # mylogger is the name and ID of the logger. If this parameter is not provided, it defaults to 'root' logger.setLevel(logging.DEBUG) # Set logger processing level # Judge here if logger If the handlers list is empty, add it. Otherwise, write the log directly if not logger.handlers: # rf_handler writes all log information to all Log # when: string, which defines the interval time unit of log segmentation # Interval: the number of interval time units. It refers to the number of when time to wait. The Logger will automatically rebuild the news and continue logging # backupCount is the number of log files to keep. The maximum number of log files is backupCount. Redundant files are deleted. The default is 0, which means they will not be deleted automatically rf_handler = logging.handlers.TimedRotatingFileHandler('all.log', when='midnight', interval=1, backupCount=7, atTime=datetime.time(0, 0, 0, 0)) # Set output log format rf_formatter = logging.Formatter("%(asctime)s - %(levelname)s - %(message)s") # Specify the output format for the handler rf_handler.setFormatter(rf_formatter) # f_ The handler writes information with a level greater than or equal to error to error Log file f_handler = logging.FileHandler('error.log') f_handler.setLevel(logging.ERROR) # Set output log format f_formatter = logging.Formatter("%(asctime)s - %(levelname)s - %(filename)s[:%(lineno)d] - %(message)s") # Specify the output format for the handler f_handler.setFormatter(f_formatter) # Log processor added for logger logger.addHandler(rf_handler) logger.addHandler(f_handler) return logger logger = get_logger() logger.debug('debug message') logger.info('info message') logger.warning('warning message') logger.error('error message') logger.critical('critical message') logger.log(level=logging.ERROR, msg="logger.log message")
reference resources: https://www.cnblogs.com/Nicholas0707/p/9021672.html#_label1_1
Turn:
Detailed explanation of python log logging module
--Posted from Rpc