preface

Starting from this chapter, we enter the second stage of python learning, mainly explaining the functions and modules in python language. This chapter mainly explains the content of file operation in python language.

1, Functions and modules

1.1 function

Function is the content that must be mastered in every programming language. A function often realizes a function, and the function is often used many times. The significance of function is to reduce the amount of code and improve the efficiency of programmer writing code.
There are two types of functions in python:

• Built in function
  There are many built-in functions in python, such as len(), bin(), oct(), hex(), etc. when writing programs, they can be called directly without declaration or definition or introducing special modules.
• Custom function
  Custom functions are often code blocks written by programmers in order to realize some functions.

def send_email():
    # Wrote 10 lines of code to send mail.
    pass

send_email()

# Defines a function, function code block
def send_email():
    # Wrote 10 lines of code to send mail.
    pass

goods = [
    {"name": "computer", "price": 1999},
    {"name": "mouse", "price": 10},
    {"name": "yacht", "price": 20},
    {"name": "beauty", "price": 998}
]
for index in range(len(goods)):
    item = goods[index]
    print(index + 1, item['name'], item['price'])

# Call and execute functions
send_email()

while True:
    num = input("Please enter the serial number of the item you want to select(Q/q): ")  # "1"
    if num.upper() == "Q":
        break
    if not num.isdecimal():
        print("The input format is incorrect")
        break
    num = int(num)
    send_email()
    if num > 4 or num < 0:
        print("Range selection error")
        break
    target_index = num - 1
    choice_item = goods[target_index]
    print(choice_item["name"], choice_item['price'])
    send_email()

1.2 module

The module is a collection of functions of various parts of the airport.

• Built in modules, Python helps us provide good modules internally.

import random

num = random.randint(0,19)

import decimal

v1 = decimal.Decimal("0.1")
v2 = decimal.Decimal("0.2")
v3 = v1 + v2
print(v3) # 0.3

• Third party modules, such as tensorflow, numpy, and so on.
• Custom module.

3, File operation

Before learning about file operations, let's review the knowledge of coding and related data types.

• String type (str), used to represent text information in programs, is essentially binary in unicode coding.

  name = "Wu Peiqi"
  

• Byte type (bytes)

  • It can represent text information, which is essentially binary encoded by utf-8/gbk (compress unicode to facilitate file storage and network transmission.)

    name = "Wu Peiqi"
    data = name.encode('utf-8')
    print(data) # b'\xe6\xad\xa6\xe6\xb2\x9b\xe9\xbd\x90'
    
    result = data.decode('utf-8')
    print(result) # "Wu Peiqi"
    

  • Can represent original binary (picture, file and other information)

3.1 file opening and closing

For opening and closing files, you need to use python's built-in file opening function () and closing function (), which are often used together. The open() function returns a file object.
Python provides the necessary functions and methods to perform basic file operations by default. You can use the file object to do most file operations.

open function
You must use Python's built-in open() function to open a file and create a file object before the relevant methods can call it for reading and writing.

Syntax:

file object = open(file_name [, access_mode][, buffering])

The details of each parameter are as follows:

• file_name: file_ The name variable is a string value that contains the name of the file you want to access.
• access_mode: access_ Mode determines the mode of opening files: read-only, write, append, etc. All desirable values are shown in the complete list below. This parameter is optional. The default file access mode is read-only ®.
• Buffering: if the value of buffering is set to 0, there will be no deposit. If the value of buffering is 1, the line will be registered when accessing the file. If the value of buffering is set to an integer greater than 1, it indicates that this is the buffer size of the register. If a negative value is taken, the buffer size of the deposit area is the system default.

Full list of files opened in different modes:

pattern	describe
t	Text mode
x	In write mode, create a new file. If the file already exists, an error will be reported.
+	Open a file for update (readable and writable).
r	Open the file as read-only. The pointer to the file will be placed at the beginning of the file. This is the default mode.
rb	Open a file in binary format for read-only. The file pointer will be placed at the beginning of the file. This is the default mode. It is generally used for non text files, such as pictures.
r+	Open a file for reading and writing. The file pointer will be placed at the beginning of the file.
rb+	Open a file in binary format for reading and writing. The file pointer will be placed at the beginning of the file. It is generally used for non text files, such as pictures.
w	Open a file for writing only. If the file already exists, open the file and edit it from the beginning, that is, the original content will be deleted. If the file does not exist, create a new file.
wb	Open a file in binary format for writing only. If the file already exists, open the file and edit it from the beginning, that is, the original content will be deleted. If the file does not exist, create a new file. It is generally used for non text files, such as pictures.
w+	Open a file for reading and writing. If the file already exists, open the file and edit it from the beginning, that is, the original content will be deleted. If the file does not exist, create a new file.
wb+	Open a file in binary format for reading and writing. If the file already exists, open the file and edit it from the beginning, that is, the original content will be deleted. If the file does not exist, create a new file. It is generally used for non text files, such as pictures.
a	Open a file for append. If the file already exists, the file pointer will be placed at the end of the file. That is, the new content will be written after the existing content. If the file does not exist, create a new file for writing.
ab	Open a file in binary format for appending. If the file already exists, the file pointer will be placed at the end of the file. That is, the new content will be written after the existing content. If the file does not exist, create a new file for writing.
a+	Open a file for reading and writing. If the file already exists, the file pointer will be placed at the end of the file. The file is opened in append mode. If the file does not exist, create a new file for reading and writing.
ab+	Open a file in binary format for appending. If the file already exists, the file pointer will be placed at the end of the file. If the file does not exist, create a new file for reading and writing.

The following figure is a good summary of these modes:

pattern	r	r+	w	w+	a	a+
read	+	+		+		+
write		+	+	+	+	+
establish			+	+	+	+
cover			+	+
Pointer at start	+	+	+	+
Pointer at end					+	+

Properties of the File object
After a file is opened, you have a file object, and you can get all kinds of information about the file.

The following is a list of all attributes related to the file object (note not methods):

attribute	describe
file.closed	Returns true if the file has been closed, otherwise returns false.
file.mode	Returns the access mode of the open file.
file.name	Returns the name of the file.
file.softspace	If the output with print must be followed by a space character, false is returned. Otherwise, return true.

Code example

# -*- coding: UTF-8 -*-
 
# Open a file
fo = open("foo.txt", "w")
print "file name: ", fo.name
print "Is it closed : ", fo.closed
print "Access mode : ", fo.mode
print "Whether to force spaces at the end : ", fo.softspace

#output
 file name:  foo.txt
 Is it closed :  False
 Access mode :  w
 Whether to force spaces at the end :  0

close() method
The close () method of the File object flushes any information that has not been written in the buffer and closes the File. After that, it can no longer be written.

When a reference to a file object is reassigned to another file, Python closes the previous file. It is a good habit to close files with the close () method.

Syntax:

fileObject.close()

# Open a file
fo = open("foo.txt", "w")
print "file name: ", fo.name
 
# Close open files
fo.close()

#Output:
file name:  foo.txt

3.2 file content reading

read() method
The read () method reads a string from an open file. It is important to note that Python strings can be binary data, not just text. That is, if it is opened in text mode, the minimum unit of reading is characters. If the opening mode is binary mode, the minimum unit of reading is bytes.

Syntax:

fileObject.read([count])

Here, the parameter passed is the character / byte count to be read from the open file. This method reads in from the beginning of the file. If no count is passed in, it will try to read as much content as possible, probably up to the end of the file.

Code example

# -*- coding: UTF-8 -*-
 
# Open a file
fo = open("foo.txt", "r+")
str = fo.read(10)
print "The read string is : ", str
# Close open files
fo.close()

# 1. Open the file
#	-Path:
#		Relative path: 'file / info txt'
#		Absolute path: 'F:\python Luffy learning \ python project - module 2 \ file \ info txt'
# 	-Pattern
# 		rb indicates the original binary of the read file (r, read; b, binary;)
# 1. Open the file
file_object = open('file/info.txt', mode='rb')
# 2. Read the contents of the file and assign it to data
data = file_object.read()
# 3. Close the file
file_object.close()

print(data) # b'alex-123\n\xe6\xad\xa6\xe6\xb2\x9b\xe9\xbd\x90-123'
text = data.decode("utf-8")
print(text)

Create a py file in the current directory for file reading. The files to be read are placed in info. In the file folder Txt file. The first parameter in the open() function is the text path. Here, relative path or absolute path can be used. When pycharm opens the project, the current directory has been entered into the project directory by default, so there will be no error when using the relative path. However, if the script is run directly without entering the project directory on the terminal, the text file will not be found, which should be noted.

file_object = open('a1.png', mode='rb')
data = file_object.read()
file_object.close()

print(data) # \x91\xf6\xf2\x83\x8aQFfv\x8b7\xcc\xed\xc3}\x7fT\x9d{.3.\xf1{\xe8\...

supplement
In liunx or mac systems, the separator of the file path is /, which will not be escaped in the unloading string. However, for windows systems, the separator of the file path is ` ` `, which will be escaped by the string by default. Therefore, when writing the file path under windows system, the following two methods can be used to solve this problem:

1. String plus R, e.g. file_object = open(r'C:\new\info.txt', mode='rt', encoding='utf-8')
2. Use \ \ instead of \, such as file_object = open('C:\new\info.txt', mode='rt', encoding='utf-8')

3.2 judge whether the document exists

Call the exists function in the os module to determine whether the file path exists:

# Determine whether the path exists?
import os

file_path = "/Users/wupeiqi/PycharmProjects/luffyCourse/day09/info.txt"
exists = os.path.exists(file_path)
if exists:
    # 1. Open the file
    file_object = open('infower.txt', mode='rt', encoding='utf-8')
    # 2. Read the contents of the file and assign it to data
    data = file_object.read()
    # 3. Close the file
    file_object.close()
    print(data)
else:
    print("file does not exist")

3.3 writing documents

write() method
The write() method writes any string to an open file. It is important to note that Python strings can be binary data, not just text.

The write() method does not add a newline character ('\ n') to the end of the string.
Syntax:

fileObject.write(string)

Here, the parameter passed is the content to be written to the open file.

Code example

# -*- coding: UTF-8 -*-
 
# Open a file
fo = open("foo.txt", "w")
fo.write( "www.runoob.com!\nVery good site!\n")
 
# Close open files
fo.close()

The above method creates foo Txt file, and write the received content to the file, and finally close the file. If you open this file, you will see the following:

$ cat foo.txt 
www.runoob.com!
Very good site!

# 1. Open the file
#	-Path:
#		Relative path: 'file / info txt'
#		Absolute path: 'F:\python Luffy learning \ python project - module 2 \ file \ info txt'
# 	-Pattern
# 		w. Indicates write file
# 1. Open the file
file_object = open('file/info.txt', mode='wt',encoding="utf-8")# Note that the encoding parameter should be written clearly

# 2. Write the contents of the document
file_object.write("Little pig loves learning!!!!!!!!")
# 3. Close the file
file_object.close()

# 1. Open the file
# Path: T1 txt
# Mode: wb (the content to be written needs to be byte type)
file_object = open("t1.txt", mode='wb')

# 2. Write content
file_object.write(    "Wu Peiqi".encode("utf-8")    )

# 3. File closing
file_object.close()

file_object = open("t1.txt", mode='wt', encoding='utf-8')

file_object.write("Wu Peiqi")

file_object.close()

You can also write picture files. Note that for non text files, it is best to use byte type to open the file:

f1 = open('a1.png',mode='rb')
content = f1.read()
f1.close()

f2 = open('a2.png',mode='wb')
f2.write(content)
f2.close()

Basic case

# Case 1: user registration

user = input("Please enter user name:")
pwd = input("Please input a password:")
data = "{}-{}".format(user, pwd)
file_object = open("files/info.txt", mode='wt', encoding='utf-8')
file_object.write(data)
file_object.close()


# Case 2: multi user registration

# w to write a file, empty the file first; Then write the content in the file.
file_object = open("files/info.txt", mode='wt', encoding='utf-8')
while True:
    user = input("Please enter user name:")
    if user.upper() == "Q":
        break
    pwd = input("Please input a password:")
    data = "{}-{}\n".format(user, pwd)

    file_object.write(data)
file_object.close()

Use crawler to get data and write to file:

# Case 1: go online to download some text and write the text information into the file.
import requests

res = requests.get(
    url="https://movie.douban.com/j/search_subjects?type=movie&tag=%E7%83%AD%E9%97%A8&sort=recommend&page_limit=20&page_start=20",
    headers={
        "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36"
    }
)

# Original binary information transmitted by network (bytes)
# res.content

file_object = open('files/log1.txt', mode='wb')
file_object.write(res.content)
file_object.close()



# Case 2: download a picture from the Internet and write the picture into the local file.
import requests

res = requests.get(
    url="https://hbimg.huabanimg.com/c7e1461e4b15735fbe625c4dc85bd19904d96daf6de9fb-tosv1r_fw1200",
    headers={
        "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36"
    }
)

# Original binary information transmitted by network (bytes)
# res.content

file_object = open('files/picture.png', mode='wb')
file_object.write(res.content)
file_object.close()

3.4 document positioning

The tell() method tells you the current location in the file. In other words, the next read and write will occur after so many bytes at the beginning of the file.

The seek (offset [,from]) method changes the location of the current file.

• The Offset variable indicates the number of bytes to move.
• The From variable specifies the reference position at which the bytes are to be moved.

If from is set to 0, this means that the beginning of the file is used as the reference position for the moving bytes. If set to 1, the current position is used as the reference position. If it is set to 2, the end of the file will be used as the reference location.

It must be noted that Chinese characters generally occupy three bytes and English characters generally occupy one byte, so reading a Chinese character cursor will move three characters!

file_object = open('file/info.txt', mode='r+',encoding="utf-8")
# 2. Write the contents of the document
data = file_object.read(4)
print(data)
print(file_object.tell())
# 3. Close the file
file_object.close()

#output
 Xiaoxiang a pig
10 #Three Chinese characters plus one English character, a total of 3 * 3 + 1 = 10 bytes

3.5 renaming or deleting files

Python's os module provides methods to help you perform file processing operations, such as renaming and deleting files. To use this module, you must import it first, and then you can call various related functions.

rename() method
The rename() method requires two parameters, the current file name and the new file name.

Syntax:

os.rename(current_file_name, new_file_name)

example:

The following example will rename an existing file test1 txt.

# -*- coding: UTF-8 -*-

import os
 
# Rename the file test1 Txt to test2 txt. 
os.rename( "test1.txt", "test2.txt" )

remove() method
You can use the remove() method to delete a file. You need to provide the name of the file to be deleted as a parameter.

Syntax:

os.remove(file_name)

example:

The following example will delete an existing file test2 txt.

# -*- coding: UTF-8 -*-


import os
 
# Delete an existing file test2 txt
os.remove("test2.txt")

3.6 creating, deleting or changing directories

All files are contained in different directories, but Python can handle them easily. os module has many methods to help you create, delete and change directories.

mkdir() method
You can use the mkdir() method of the os module to create a new directory in the current directory. You need to provide a parameter that contains the name of the directory to be created.

Syntax:

os.mkdir("newdir")

example:

The following example will create a new directory test under the current directory.

# -*- coding: UTF-8 -*-

import os
 
# Create directory test
os.mkdir("test")

chdir() method
You can use the chdir() method to change the current directory. One of the parameters required by the chdir() method is the directory name you want to set to the current directory.

Syntax:

os.chdir("newdir")

example:

The following example will enter the "/ home/newdir" directory.

# -*- coding: UTF-8 -*-

import os
 
# Change the current directory to "/ home/newdir"
os.chdir("/home/newdir")

getcwd() method:

The getcwd() method displays the current working directory.

Syntax:

os.getcwd()

example:

The following example shows the current directory:

# -*- coding: UTF-8 -*-

import os
 
# Give the current directory
print os.getcwd()

rmdir() method
The rmdir() method deletes the directory, and the directory name is passed as a parameter.

All contents of this directory should be cleared before deleting it.

Syntax:

os.rmdir('dirname')

example:

The following is an example of deleting the "/ tmp/test" directory. The fully compliant name of the directory must be given, otherwise the directory will be searched under the current directory.

# -*- coding: UTF-8 -*-

import os
 
# Delete the "/ tmp/test" directory
os.rmdir( "/tmp/test"  )

4, Supplement to common methods of file object

4.1 Python 3 file flush() method

summary
The flush() method is used to flush the buffer, that is, the data in the buffer is written to the file immediately, and the buffer is emptied at the same time. There is no need to wait passively for the output buffer to be written.

Normally, the buffer will be refreshed automatically after the file is closed, but sometimes you need to refresh it before closing, so you can use the flush() method.

grammar
The syntax of the flush() method is as follows:

fileObject.flush();

parameter
nothing

Return value
The method has no return value.

example
The following example demonstrates the use of the flush() method:

# Open file
fo = open("runoob.txt", "wb")
print ("The file name is: ", fo.name)

# refresh buffer 
fo.flush()

# Close file
fo.close()

# The output result of the above example is:

The file name is:  runoob.txt

4.2 Python 3 file readline() method

summary
The readline() method is used to read the entire line from the file, including the "\ n" character. If a non negative parameter is specified, the number of bytes of the specified size is returned, including the "\ n" character.

grammar
The syntax of the readline() method is as follows:

fileObject.readline(); 

parameter

• size – the number of bytes read from the file.

Return value
Returns the bytes read from the string.

example
The following example demonstrates the use of the readline() method:

file runoob.txt The contents are as follows:

1:www.runoob.com
2:www.runoob.com
3:www.runoob.com
4:www.runoob.com
5:www.runoob.com

Read the contents of the file:

example

# Open file
fo = open("runoob.txt", "r+")
print ("The file name is: ", fo.name)

line = fo.readline()
print ("Read the first line %s" % (line))

line = fo.readline(5)
print ("The read string is: %s" % (line))

# Close file
fo.close()

#The output result of the above example is:

The file name is:  runoob.txt
 Read first line 1:www.runoob.com

The read string is: 2:www

4.3 Python 3 file readlines() method

summary
The readlines() method is used to read all lines (up to the terminator EOF) and return a list that can be processed by Python's for... in... Structure. If the terminator EOF is encountered, an empty string is returned.

If the terminator EOF is encountered, an empty string is returned.

grammar
The syntax of the readlines() method is as follows:

fileObject.readlines()

parameter
None.

Return value
Returns a list containing all rows.

example
The following example demonstrates the use of the readline() method:

file runoob.txt The contents are as follows:

1:www.runoob.com
2:www.runoob.com
3:www.runoob.com
4:www.runoob.com
5:www.runoob.com

Cycle through the contents of the file:

# Open file
fo = open("runoob.txt", "r")
print ("The file name is: ", fo.name)
 
for line in fo.readlines():                          #Read each row in turn  
    line = line.strip()                             #Remove the blanks at the beginning and end of each line  
    print ("The data read is: %s" % (line))
 
# Close file
fo.close()

The output result of the above example is:

The file name is:  runoob.txt
 The data read is: 1:www.runoob.com
 The data read is: 2:www.runoob.com
 The data read is: 3:www.runoob.com
 The data read is: 4:www.runoob.com
 The data read is: 5:www.runoob.com

Supplement: cycle, read large files (readline enhanced version) [common]

f = open('info.txt', mode='r', encoding='utf-8')
for line in f:
    print(line.strip())
f.close()

4.4 Python 3 file seek() method

summary
The seek() method is used to move the file read pointer to the specified location.

grammar
The syntax of seek() method is as follows:

fileObject.seek(offset[, whence])

parameter

• Offset – the starting offset, that is, the number of bytes that need to be shifted. If it is a negative number, it means starting from the penultimate bit.
• Where: optional. The default value is 0. Define a parameter for offset, indicating where to start offset; 0 means 1 from the beginning of the file
  Represents from the current position, and 2 represents from the end of the file.

Return value
If the operation is successful, the new file location is returned. If the operation fails, the function returns - 1.

example
The following example demonstrates the use of the seek() method:

>>> f = open('workfile', 'rb+')
>>> f.write(b'0123456789abcdef')
16
>>> f.seek(5)      # Move to the sixth byte of the file
5
>>> f.read(1)
b'5'
>>> f.seek(-3, 2)  # Move to the penultimate byte of the file
13
>>> f.read(1)
b'd'

File runoob Txt is as follows:

1:www.runoob.com
2:www.runoob.com
3:www.runoob.com
4:www.runoob.com
5:www.runoob.com

Cycle through the contents of the file:

# Open file
fo = open("runoob.txt", "r+")
print ("The file name is: ", fo.name)
 
line = fo.readline()
print ("The data read is: %s" % (line))
 
# Reset the file read pointer to the beginning
fo.seek(0, 0)
line = fo.readline()
print ("The data read is: %s" % (line))
 
 
# Close file
fo.close()


#The output result of the above example is:

The file name is:  runoob.txt
 The data read is: 1:www.runoob.com

The data read is: 1:www.runoob.com

4.5 Python 3 file writelines() method

summary
The writelines() method is used to write a sequence of strings to a file.

This sequence string can be generated by iterative objects, such as a string list.

Newline requires a newline \ n.

grammar
The syntax of writelines() method is as follows:

fileObject.writelines( [ str ])

parameter

• str – the sequence of strings to write to the file.

Return value
The method has no return value.

example
The following example demonstrates the use of the writelines() method:

# Open file
fo = open("test.txt", "w")
print ("The file name is: ", fo.name)
seq = ["Little pig loves learning\n", "Little pig has money"]
fo.writelines( seq )

# Close file
fo.close()
The output result of the above example is:

The file name is:  test.txt
 View file contents:

$ cat test.txt 
Little pig loves learning
 Little pig has money

5, The with keyword is used to open a file

In order to prevent the program from crashing due to an error in the middle of the file opening, and the file is not closed eventually, resulting in the loss or damage of file data, we choose to hand over the appropriate time of file closing to the python interpreter to decide that it is safer.

with open("xxxx.txt", mode='rb') as file_object:
    data = file_object.read()
    print(data)

After Python 2.7, with supports the context management of multiple files at the same time, that is:

with open("xxxx.txt", mode='rb') as f1, open("xxxx.txt", mode='rb') as f2:
    pass