WeChat official account is searching for the program yuan Xiaozhuang. It pays attention to how to develop a family breadwinner by python.
preface
When developing with Python, we often use different modules to help us complete the implementation of some functions. Therefore, mastering the common methods of some common modules can help us speed up program development.
time module
In Python, there are usually several ways to represent time:
1. Timestamp refers to the offset calculated in seconds from 00:00:00 on January 1, 1970. It is a floating-point type and can be used to calculate the time interval.
>>> import time >>> time.time() 1622362862.679771
2. The formatted time string is mainly used to display the time.
>>> import time
>>> time.strftime('%Y-%m-%d %H:%M:%S %p') # Y-year m-month d-Day H-Hour m-minute S-second p-AM/PM
'2021-05-30 16:22:44 PM'
# The above code can be abbreviated, using X to represent hours, minutes and seconds
>>> time.strftime('%Y-%m-%d %X')
'2021-05-30 16:23:34'
3. Structured time
>>> print(time.localtime()) #Struct of local time zone_ time >>> print(time.gmtime()) #Struct of UTC time zone_ time
Among the above three time formats, the time format that can be recognized by computer is only timestamp, while the time that human can understand is formatted time string or structured time, so there is a conversion relationship between the three.
1. Timestamp - > structured time - > formatted time
# Timestamp -- structured time
import time
time1 = time.time() # time stamp
res = time.localtime(time1) # Convert timestamp to structured time
print(res)
# Structured time -- format time
res = time.localtime() # # Get structured time
time1 = time.strftime('%Y-%m-%d %X',res) # Convert structured time to formatted time
print(time1)
2. Format time - > structured time - > timestamp
# Format time --- structured time
res = time.strftime('%Y-%m-%d %X',res) # Get format time
print(res)
time1 = time.strptime(res,'%Y-%m-%d %X') # Convert formatted time to structured time
print(time1)
# Conversion of structured time to timestamp
res = time.localtime() # Get structured time
time1 = time.mktime(res)
print(time1)
datetime module
The datetime module can be used to add or subtract time
import datetime print(datetime.datetime.now()) # Format output time print(datetime.datetime.now() + datetime.timedelta(3)) #Current time + 3 days print(datetime.datetime.now() + datetime.timedelta(-3)) #Current time - 3 days print(datetime.datetime.now() + datetime.timedelta(hours=3)) #Current time + 3 hours print(datetime.datetime.now() + datetime.timedelta(minutes=30)) #Current time + 30 minutes
random module
Random module can be used to generate random numbers, random verification codes and other functions.
import random
# Get floating point numbers between 0 and 1 at random
print(random.random())
# Randomly obtain integer numbers greater than or equal to 1 and less than or equal to 6
print(random.randint(1,6))
# Randomly obtain integer numbers greater than or equal to 1 and less than 6
print(random.randrange(1,6))
# Randomly obtain one of the parameters
print(random.choice((111,'aaa',456,[1234,'qq'])))
# The number of specified elements can be combined freely and output randomly. When the number of specified elements is 1, it has the same effect as choice
print(random.sample([111, 'aaa', 'ccc','ddd'],1))
print(random.sample([1,3,5,7,89,],3))
# shuffle the original list
list1 = [1,2,3,5]
random.shuffle(list1)
print(list1)
# Generate verification code using random module
def func(n=6):
# Obtain 26 random uppercase English letters according to the ASC code table
al_A = chr(random.randint(65,90))
# Obtain 26 random lowercase English letters according to the ASC code table
al_a = chr(random.randint(97,122))
# Get random number
num = random.randint(0,9)
# Generate random verification code
res = ''
for i in range(n):
res += random.choice([al_a,al_A,str(num)])
return res
os module
os module provides very rich methods to deal with files and directories. It is a very large module to interact with the operating system. Here are some common methods.
import os
os.listdir(path) # Get the names of all subfolders and file names under a folder
os.remove(path) # Delete files in the specified path
os.rename("oldname","newname") # Rename file / directory
os.path.dirname(__file__) # Gets the folder where the current file is located
os.path.basename(__file__) # Gets the file name of the current file
os.path.split(path) # The path is divided into directory and file name binary and returned
os.path.exists(path) # If path exists, return True; If path does not exist, False is returned
os.listdir('dirname') # List all files and subdirectories in the specified directory, including hidden files, and print them in list mode
os.mkdir('dirname') # Generate a single level directory under the current file path
os.getcwd() # Get the current working directory, that is, the path of the working directory of the current python script
os.path.abspath(path) # Returns the absolute path normalized by path
os.path.join('a','b') # Splice file paths
sys module
sys.version # Get the version information of Python interpreter sys.path # Returns the search path of the module. The value of PYTHONPATH environment variable is used during initialization sys.platform # Returns the operating system platform name
Serialization and deserialization module
json module and pickle module are modules used to serialize and deserialize data.
Serialization refers to converting the data type of memory into the content of a specific format, which can be used for storage or transmission to other platforms (other programming languages). Deserialization is to convert the data on other platforms into a data format recognized by python.
Serialization: Data type in memory---->serialize---->Specific format( json/pickle (format) Deserialization: Data type in memory<----Deserialization<----Specific format( json/pickle (format)
Serialization of data serves two purposes:
First, it can persistently save the data generated during the operation of the program and write the data to disk.
Second, after serialization, the serialized content can not only be written to disk, but also be transmitted to other machines through the network. If the sending and receiving parties agree to use a serialization format, the restrictions brought by platform / language differentiation will be broken and cross platform data interaction will be realized.
Serialization and deserialization can be done with json and pickle modules.
json module
Data in json format is a common data type of all programming languages. If we want to transfer objects between different programming languages, we must serialize the objects into standard format, such as XML, but a better way is to serialize them into json format, because json format represents a string that can be read by all languages, It can also be conveniently stored on disk or transmitted through the network. However, json cannot be used for serialization or deserialization of data types unique to a language, such as the collection type of python.
You should also pay attention to the difference between json format string and Python syntax. Don't confuse it. For example, json format string does not support single quotation marks, but the dictionary supports single quotation marks~
import json
# Serialization operation
res = json.dumps([1,2,3,4,True,False])
print('Serialization result',res) # Serialization result [1, 2, 3, 4, true, false], 'true'/false --- string type
# Serialization results can exist in files: complex methods
with open('json.txt', 'w', encoding='utf-8') as f:
f.write(res)
# An easy way to write the result of serialization to a file:
with open('json.txt', 'w', encoding='utf-8') as f:
json.dump([1,2,3,4,True,False],f)
# Deserialization
res1 = json.loads(res)
print('Deserialization',res1,type(res1))
# Deserializing data in a file: a complex method
with open('json.txt', 'r', encoding='utf -8') as f:
json_res = f.read()
res = json.loads(json_res)
print(res)
# Simple method of deserialization
with open('json.txt', 'r', encoding='utf -8') as f:
res = json.load(f)
print(res,type(res))
pickle module
The usage of pickle module is the same as that of json module, except that it can only be used in Python. You can serialize data types unique to python.
import pickle
# serialize
res = pickle.dumps({1,2,3})
# Collections are unique to python and can be serialized using pickle
print(res)
# Deserialization
res1 = pickle.loads(res)
print(res)
hashlib module
Hash is a kind of algorithm, which receives the incoming content and obtains a string of hash values after operation. Hash values have the following characteristics:
I as long as the incoming content is the same, the obtained hash value must be the same;
II. The hash value cannot be returned to the content;
III no matter how big the content is, as long as the hash algorithm used remains unchanged, the length of the hash value obtained is certain;
Based on the above characteristics, hash can be used for password transmission and verification. The commonly used encryption algorithm is md5.
import hashlib
m = hashlib.md5() # It is equivalent to building a factory to receive data that needs to be encrypted
m.update('hello'.encode('utf-8')) # Receive encrypted data
m.update('world'.encode('utf-8')) # Encrypted data can be received multiple times
res = m.hexdigest() # Process the data 'helloworld' to be encrypted and get the encrypted result
print(res) # fc5e038d38a57032085441e7fe7010b0
m1 = hashlib.md5('he'.encode('utf-8'))
m1.update('llo'.encode('utf-8'))
m1.update('w'.encode('utf-8'))
m1.update('orld'.encode('utf-8'))
res = m1.hexdigest() # # Process the data 'helloworld' to be encrypted and get the encrypted result
print(res) # fc5e038d38a57032085441e7fe7010b0
# Note: updating a long piece of data multiple times will get the same result as updating this long piece of data once.
Although the above encryption algorithm can not be solved inversely, it can be solved inversely by hitting the library. It can be solved inversely by encrypting the guess password and comparing it with the known md5 string, as shown in the following code:
cryptograph='aee949757a2e698417463d47acac93df' # Known md5 encrypted string
import hashlib
# Make password field
passwds=[
'asdfgh',
'asdfh66',
'asdfh6678'
]
dic={}
for p in passwds:
res=hashlib.md5(p.encode('utf-8'))
dic[p]=res.hexdigest()
# Simulate hitting the library to get the password
for k,v in dic.items():
if v == cryptograph:
print('Hit the library successfully. The plaintext password is:%s' %k)
break
Therefore, a user-defined key can be added to the encryption algorithm and then encrypted, commonly known as adding salt. Adding salt can increase the cost of hitting the library.
import hashlib
m = hashlib.md5()
m.update('King'.encode('utf-8'))
m.update('hello'.encode('utf-8'))
m.update('Gedi tiger'.encode('utf-8'))
m.update('world'.encode('utf-8'))
print(m.hexdigest()
logging module
The logging module can be used to record the log of program operation. The log is a very important part in program development. You can use the log to check whether the program is running normally in the online production environment or in the test environment. If the program is abnormal, you can quickly locate the abnormal code location through the log. Therefore, the log recording needs to be as detailed as possible, In python development, logging module can be used to record logs.
import logging
# The number size represents different levels
CRITICAL = 50
ERROR = 40
WARNING = 30
INFO = 20
DEBUG = 10
NOTSET = 0
logging.debug('debugging debug')
logging.info('news info')
logging.warning('warning warn')
logging.error('error error')
logging.critical('serious critical')
'''
WARNING:root:warning warn
ERROR:root:error error
CRITICAL:root:serious critical
'''
The log level above warning.warning can be output to the log in the terminal by default Basicconfig() specifies the global configuration and can save the log to a file.
import logging
logging.basicConfig(filename='test.log',
format='%(asctime)s - %(name)s - %(levelname)s -%(module)s: %(message)s',
datefmt='%Y-%m-%d %H:%M:%S %p',
level=10)
The above parameters and other unused parameters are described as follows:
filename: Creates a file with the specified file name FiledHandler，Used to print to a file
datefmt: Specifies the date time format.
level: Log level set by number
format Possible formatting strings in parameters:
%(name)s Logger Name of
%(levelno)s Log level in digital form
%(levelname)s Log level in text form
%(pathname)s The full pathname of the module calling the log output function, which may not be available
%(filename)s The file name of the module that called the log output function
%(module)s The name of the module that called the log output function
%(funcName)s The name of the function that calls the log output function
%(lineno)d The code line of the statement that calls the log output function
%(asctime)s The current time in string form. The default format is 2003-07-08 16:49:45,896". The comma is followed by milliseconds
%(thread)d thread ID. Probably not
%(threadName)s Thread name. Probably not
%(process)d process ID. Probably not
%(message)s User output message
In the process of using the logging module, Formatter, Handler, Logger and Filter objects need to be used:
logger: the object that generates the log;
Filter: the object used to filter logs, which is not commonly used;
Handler: receive the log and control the printing to different places. FileHandler is used to print to the file and StreamHandler is used to print to the terminal;
Formatter object: you can customize different log format objects and bind them to different Handler objects to control the log format of different handlers.
After all, how to use the logging module to record logs in python projects? In project development, just remember the following usage.
First, configure the log format in the project configuration file (settings.py):
# settings.py
# log file
LOG_PATH = os.path.join(BASE_DIR,'log')
LOG_UER_PATH = os.path.join(LOG_PATH,'access.log')
# Define three log output formats
standard_format = '%(asctime)s - %(threadName)s:%(thread)d - Log name:%(name)s - %(filename)s:%(lineno)d -' \
'%(levelname)s - %(message)s'
simple_format = '[%(levelname)s][%(asctime)s][%(filename)s:%(lineno)d]%(message)s'
test_format = '%(asctime)s] %(message)s'
# Log configuration dictionary
LOGGING_DIC = {
'version': 1,
'disable_existing_loggers': False,
'formatters': {
'standard': {
'format': standard_format
},
'simple': {
'format': simple_format
},
'test': {
'format': test_format
},
},
'filters': {},
# handlers are the recipients of logs. Different handler s will output logs to different locations
'handlers': {
#Log printed to terminal
'console': {
'level': 'DEBUG',
'class': 'logging.StreamHandler', # Print to screen
'formatter': 'simple'
},
'default': {
'level': 'DEBUG',
'class': 'logging.handlers.RotatingFileHandler', # Save to file
# 'maxBytes': 1024*1024*5, # Log size 5M
# 'maxBytes': 1000,
# 'backupCount': 5,
'filename': LOG_UER_PATH, # os.path.join(os.path.dirname(os.path.dirname(__file__)),'log','a2.log')
'encoding': 'utf-8',
'formatter': 'standard',
},
#Print the log to the file and collect the log of info and above
'other': {
'level': 'DEBUG',
'class': 'logging.FileHandler', # Save to file
'filename': LOG_UER_PATH, # os.path.join(os.path.dirname(os.path.dirname(__file__)),'log','a2.log')
'encoding': 'utf-8',
'formatter': 'test',
},
},
# loggers are the producers of logs. The generated logs will be passed to the handler and then control the output
'loggers': {
# logging.getLogger(__name__) If we want the logger objects with different logger names to share the same configuration, we can't define n keys in the loggers sub dictionary. The solution is to define an empty key, so when we get the logger object, logging getLogger(__name__)， Different files__ name__ Different. This ensures that the identification information is different when printing logs. However, when you find the key name in loggers with this name, you find that it cannot be found. Therefore, the configuration of key = '' is used by default
'': {
'handlers': ['default', ], # Here we add the two handler s defined above, that is, the log data is written to the file and printed to the screen
'level': 'DEBUG', # Loggers (Level 1 log level limit) - > handlers (Level 2 log level limit)
'propagate': False, # The default value is True. It is passed upward (the logger at a higher level). It is usually set to False. Otherwise, a log will be passed upward layer by layer
},
},
}
Then, use the logging module to log in other files:
# test.py
# logging is a package. You need to use config and getLogger under it
from logging import config
from logging import getLogger
# Load log dictionary configuration
logging.config.dictConfig(settings.LOGGING_DIC)
# Generate log object
logger = logging.getLogger('test')
# Log
logger.info('Record a log')
The logging module looks very complex, but it can be used in the above way. During project development, you only need to copy it for use.