File operation
File is a kind of data source (where data is saved), such as word document, txt document, excel file, which are often used. The main function of file is to save data. It can save a picture, video and sound
Input and output streams
Files are operated as streams in a program
Flow: the path the data goes through between the data source (file) and the program (memory)
Input stream: path of data from data source (file) to program (memory)
Output stream: path of data from program (memory) to data source (file)
os.File encapsulates all file related operations. File is a structure
The os.File structure is often used to manipulate files later
Open and close files
Functions and methods used
Case demonstration
import ( "fmt" "os" ) func main() { //Open file //Concept description: the name of file //1. file is called file object //2. file is called file pointer //3. file is called file handle file, err := os.Open("e:/test.txt") if err != nil { fmt.Println("Open file err = ", err) } //Output the file, see what the file is, and see that the file is a pointer * Filr fmt.Printf("file = %v", file) //file = &{0xc000070780} //Close err = file.Close() if err != nil { fmt.Println("Close file err = ", err) } }
Read file operation application case
- Read the contents of the file and display it on the terminal (with buffer). Use os.Open, file.Close, bufio.NewReader(), reader.ReadString function and method
import ( "bufio" "fmt" "io" "os" ) func main() { //Open file //Concept description: the name of file //1. file is called file object //2. file is called file pointer //3. file is called file handle file, err := os.Open("E:/gostudent/src/2020-04-02/utils/utils.go") if err != nil { fmt.Println("Open file err = ", err) } //When the function exits, close the file in time defer file.Close() //Close the file handle in time, or there will be a memory leak //Create a * Reader with buffer /* const ( defaultBufSize = 4096 //The default buffer is 4096 ) */ reader := bufio.NewReader(file) //Loop through the contents of a file for { str, err := reader.ReadString('\n') //Read a new line and it's over if err == io.EOF { //io.EOF indicates the end of the file break } //Output content fmt.Print(str) } fmt.Println("End of file read...") }
- Read the contents of the file and display it on the terminal (use ioutil to read the whole file into memory at one time), which is suitable for the case of small file. Related methods and functions ioutil.ReadFile
import ( "fmt" "io/ioutil" ) func main() { //Use ioutil.ReadFile to read the file in place at one time file := "E:/gostudent/src/2020-04-02/utils/utils.go" content, err := ioutil.ReadFile(file) if err != nil { fmt.Printf("read file err = %v", err) } //Display the read content to the terminal //fmt.Printf("%v", content) //[]byte fmt.Printf("%v", string(content)) // []byte //There is no Open file shown here, so there is no need to show the Close file //Because the Open and Close of the file are encapsulated inside the ReadFile function }
Write file operation application case
os.OpenFile function
- Create a new file, write the content: 5 sentences "Hello,zisefeizhu"
import ( "bufio" "fmt" "os" ) func main() { filePath := "E:/gostudent/src/2020-04-05/abc.txt" file, err := os.OpenFile(filePath, os.O_CREATE | os.O_WRONLY, 0666) if err != nil { fmt.Printf("open file err = %v \n", err) return } //Close file handle in time defer file.Close() //Prepare to write 5 sentences: "hello,zisefeizhu" str := "hello,zisefeizhu\n" // \n for newline //When writing, use * Writer with cache writer := bufio.NewWriter(file) for i := 0; i< 5; i++ { writer.WriteString(str) } //Because write is cached, when the WriterString method is called //In fact, the content is written to the cache first, so you need to call the Flush method to buffer the data //Really write to the file, otherwise there will be no data in the file!!! writer.Flush() }
- Open an existing file and overwrite the original content with 10 new sentences "Hello, purple flying pig"
import ( "bufio" "fmt" "os" ) func main() { //2) Open an existing file and overwrite the original content with 10 new sentences "Hello, purple flying pig" //1. Open the existing file E:/gostudent/src/2020-04-05/abc.txt filePath := "E:/gostudent/src/2020-04-05/abc.txt" file, err := os.OpenFile(filePath, os.O_WRONLY | os.O_TRUNC, 0666) if err != nil { fmt.Printf("open file err = %v \n", err) return } //Close file handle in time defer file.Close() //Prepare to write 10 sentences: "hello,zisefeizhu" str := "Hello, purple flying pig!\n" // \n for newline //When writing, use * Writer with cache writer := bufio.NewWriter(file) for i := 0; i< 10; i++ { writer.WriteString(str) } //Because write is cached, when the WriterString method is called //In fact, the content is written to the cache first, so you need to call the Flush method to buffer the data //Really write to the file, otherwise there will be no data in the file!!! writer.Flush() }
- Open an existing file and append "Hello, jingxing" to the original content
import ( "bufio" "fmt" "os" ) func main() { //1. Open the existing file E:/gostudent/src/2020-04-05/abc.txt filePath := "E:/gostudent/src/2020-04-05/abc.txt" file, err := os.OpenFile(filePath, os.O_WRONLY | os.O_APPEND, 0666) if err != nil { fmt.Printf("open file err = %v \n", err) return } //Close file handle in time defer file.Close() //Additional content str := "Hello, jingxing\n" // \n for newline //When writing, use * Writer with cache writer := bufio.NewWriter(file) for i := 0; i< 10; i++ { writer.WriteString(str) } //Because write is cached, when the WriterString method is called //In fact, the content is written to the cache first, so you need to call the Flush method to buffer the data //Really write to the file, otherwise there will be no data in the file!!! writer.Flush() }
- Open an existing file, read out and display the original content on the terminal, and add 5 sentences "Hello, Shenzhen"
import ( "bufio" "fmt" "io" "os" ) func main() { //1. Open the existing file E:/gostudent/src/2020-04-05/abc.txt filePath := "E:/gostudent/src/2020-04-05/abc.txt" file, err := os.OpenFile(filePath, os.O_RDWR | os.O_APPEND, 0666) if err != nil { fmt.Printf("open file err = %v \n", err) return } //Close file handle in time defer file.Close() //First read the contents of the original file and display it on the terminal reader := bufio.NewReader(file) for { str, err := reader.ReadString('\n') if err == io.EOF { //If read to the end of the file break } //Display to terminal fmt.Print(str) } //Additional content str := "Hello, Shenzhen\n" // \n for newline //When writing, use * Writer with cache writer := bufio.NewWriter(file) for i := 0; i< 5; i++ { writer.WriteString(str) } //Because write is cached, when the WriterString method is called //In fact, the content is written to the cache first, so you need to call the Flush method to buffer the data //Really write to the file, otherwise there will be no data in the file!!! writer.Flush() }
5) Write a program to write the contents of one file to another. Note: these two files already exist
Note: use ioutil.readfile/ioutil.writefile to complete the task of writing files
import ( "fmt" "io/ioutil" ) func main() { //Import the contents of the e:/abc.txt file to e:/abc.txt //1. First read the contents of e:/abc.txt into memory //2. Write the read content to d:/abc.txt file1Path := "E:/gostudent/src/2020-04-05/abc.txt" file2Path := "D:/abc.txt" data, err := ioutil.ReadFile(file1Path) if err != nil { //Error reading file fmt.Printf("read file err = %v\n", err) return } err = ioutil.WriteFile(file2Path, data, 0666) if err != nil { fmt.Printf("write file error = %v \n", err) } }
Judge whether the file exists
The method for Go to determine whether a file or folder exists is to use the os.Stat() function to determine the returned error value:
-
If the error returned is nil, the file or folder exists
-
If the returned error type is judged to be true by using os.IsNotExist(), the file or folder does not exist
-
If the error returned is of another type, it is not sure if it exists
Application example of file programming
Copy file
Note: copy a picture / movie / mp3 to another file e:/abc.jpg
func Copy(dst Writer,src Reader)(written int64, err error)
Note: the Copy function is provided by the io package
import ( "bufio" "fmt" "io" "os" ) //Write a function to receive two file paths srcFileName dstFileName func CopyFile(srcFileName string, dstFileName string) (written int64, err error) { srcFile, err := os.Open(srcFileName) if err != nil { fmt.Printf("open file err = %v\n", err) } defer srcFile.Close() //Get Reader through srcfile reader := bufio.NewReader(srcFile) //Open dstFileName dstFile,err := os.OpenFile(dstFileName, os.O_WRONLY | os.O_CREATE, 0666) if err != nil { fmt.Printf("open file err = %v\n", err) return } //Get Writer through dstFile writer := bufio.NewWriter(dstFile) defer dstFile.Close() return io.Copy(writer, reader) } func main() { //Copy the d:/abc.jpg file to e:/abc.jpg //Call CopyFile to complete file copy srcFile := "d:/abc.jpeg" dstFile := "e:/abc.jpg" _, err := CopyFile(srcFile, dstFile) if err == nil { fmt.Printf("copy complete\n") } else { fmt.Printf("Copy error err = %v\n", err) } }
Count the number of English, numbers, spaces and other characters
import ( "bufio" "fmt" "io" "os" ) //Define a structure for saving statistics results type CharCount struct { ChCount int //Record the number of English NumCount int //Number of recorded numbers SpaceCount int //Record the number of spaces OtherCount int //Record the number of other characters } func main() { //Idea: open a file and create a Reader //For each line read, count the number of English, numbers, spaces and other characters in the line //Then save the results to a structure fileName := "E:/gostudent/src/2020-04-05/abc.txt" file,err := os.Open(fileName) if err != nil { fmt.Printf("open file err = %v \n", err) return } defer file.Close() //Define CharCount instances var count CharCount //Create a Reader reader := bufio.NewReader(file) //Start looping through the contents of filename for { str, err := reader.ReadString('\n') if err == io.EOF { //Exit at the end of the file break } //To be compatible with Chinese characters, you can convert str to [] run strChange := []rune(str) //Traverse str for statistics for _,v := range strChange { switch { case v >= 'a' && v <= 'z' : fallthrough //Pierce through case v >= 'A' && v <= 'Z' : count.ChCount++ case v == ' ' || v == '\t' : count.SpaceCount++ case v >= '0' && v <= '9' : count.NumCount++ default: count.OtherCount++ } } } //Output statistical results to see if they are correct fmt.Printf("The number of characters is = %v The number of numbers is = %v The number of spaces is = %v Number of other characters = %v", count.ChCount, count.NumCount, count.SpaceCount, count.OtherCount) }
Command line arguments
What should I do if I want to be able to get various parameters of command line input?
os.Args is a slice of string that stores all command line parameters
Illustrate with examples
import ( "fmt" "os" ) func main() { fmt.Println("The parameters on the command line are:", len(os.Args)) //By traversing the os.Args slice, you can get all the command-line input parameter values for i, v := range os.Args { fmt.Printf("args[%v] = %v \n", i ,v) } } //E:\gostudent\src\2020-04-05>go run main.go 999 //The parameters on the command line are: 2 //args[0] = C:\Users\lxxxxn\AppData\Local\Temp\go-build133979866\b001\exe\main. //exe //args[1] = 999
The flag package is used to parse command line parameters
Note: the previous method is more native, which is not particularly convenient for parsing parameters, especially the command line with the specified parameter form
For example: CMD > main. Exe - F C: / AAA. Txtx - P 200 - U root. The go designer provides a flag package, which can easily parse command line parameters, and the order of parameters can be arbitrary
import ( "flag" "fmt" ) func main() { //Define several variables to receive parameter values from the command line var user string var pwd string var host string var port int //&User is the parameter value after - u entered in the receiving user command line //"u" is - u specifies the parameter //"" default //"User name, empty by default" description flag.StringVar(&user, "u","","User name, empty by default") flag.StringVar(&pwd,"pwd","","Password, blank by default") flag.StringVar(&host,"h","localhost","Hostname, default is localhost") flag.IntVar(&port,"port",3306,"Port number, 3306 by default") //There is a very important operation transformation that must be called flag.Parse() //Output result fmt.Printf("user = %v pwd = %v host = %v port = %v", user, pwd, host, port) } //E:\gostudent\src\2020-04-05>go run main.go -u root -pwd zisefeizhu -h 20.0.0.201 -port 3306 //user = root pwd = zisefeizhu host = 20.0.0.201 port = 3306
Json
Basic introduction to Jason
Json(JavaScript Object Notation) is a lightweight data exchange format. Easy to read and write. It is also easy for machine analysis and generation. key - val
Json is a data format that was popularized in 2001 and has become the mainstream data format
JSON is easy to be parsed and generated by machine, and it can effectively improve the efficiency of network transmission. Generally, when the program is transmitting on the network, it will first sequence the data (structure, map, etc.) into JSON string, and when the receiver gets the JSON string, it will be deserialized back to the original data type (structure, map, etc.). This has become the standard of every language
Application scenario
Json data format description
In JS, everything is an object. Therefore, any data type can be represented by JSON, such as string, number, object, array, map, structure, etc
JSON key value pair is a way to save data
The key names in the key / value pair combination are written before and wrapped in double quotation marks "", separated by colons: and then followed by the value:
[{"key1":val1,"key2":val2,"key3":val3,"key4":[val4,val5]}, {"key1":val1,"key2":val2,"key3":val3,"key4":[val4,val5]}] such as {"firstName":"Json"} {"name":"tom","age":18,"address": ["Beijing", "Shanghai"]} [{"name":"zisefeizhu","age":18,"address": ["Beijing", "Shanghai"]}, {"name":"jingxing","age":18,"address": ["Beijing", "Shanghai"]}]
Online analysis of Jsnon data
https://www.json.cn/ The website can verify whether the data in json format is correct. Especially useful when writing more complex json format data
Serialization of Json
json serialization refers to the operation of serializing data types with key value structure (such as structure, map, slice) into json strings
Application case
Demonstrate the serialization of structs, map s, and slices. The serialization of other data types is similar
import ( "encoding/json" "fmt" ) //Define a structure type Monster struct { Name string Age int Bithday string Sal float64 Skill string } func testStruct() { //Demonstration monster := Monster{ Name : "King of cattle", Age : 500, Bithday : "2001-11-11", Sal : 8000.0, Skill : "Ngau Tau boxing", } //Serialize monster data, err := json.Marshal(&monster) if err != nil { fmt.Printf("Serial number error err = %v \n", err) } //Output serialized results fmt.Printf("monster After serialization = %v \n", string(data)) } //Serialize map func testMap() { //Define a map var a map[string]interface{} //To use map, you need to make first a = make(map[string]interface{}) a["name"] = "Red child" a["age"] = 30 a["address"] = "Hongya cave" //Serialize the map a data, err := json.Marshal(a) if err != nil { fmt.Printf("Serial number error err = %v \n", err) } //Output serialized results fmt.Printf("monster After serialization = %v \n", string(data)) } //Demonstrate serialization of slices, which are [] map[string]interface {} func testSlice() { var slice []map[string]interface{} var m1 map[string]interface{} //Before using map, you need to make m1 = make(map[string]interface{}) m1["name"] = "jack" m1["age"] = "7" m1["address"] = "Beijing" slice = append(slice, m1) var m2 map[string]interface{} //Before using map, you need to make m2 = make(map[string]interface{}) m2["name"] = "tom" m2["age"] = "20" m2["address"] = [2]string{"Mexico","Hawaii"} slice = append(slice, m2) //Serialize slice data, err := json.Marshal(slice) if err != nil { fmt.Printf("Serial number error err = %v \n", err) } //Output serialized results fmt.Printf("monster After serialization = %v \n", string(data)) } //It is not significant for basic data type serialization func testFloat64() { var num1 float64 = 2345.67 //Serializing num1 data, err := json.Marshal(num1) if err != nil { fmt.Printf("Serial number error err = %v \n", err) } //Output serialized results fmt.Printf("monster After serialization = %v \n", string(data)) } func main() { //Demonstrate serialization of structure, map and slice testStruct() testMap() testSlice() testFloat64() } //output // After monster serialization = {"Name": "the demon", "Age":500,"Bithday":"2001-11-11","Sal":8000,"Skill": "niutouquan"} //After monster serialization = {"address": "Hongya cave", "age":30,"name": "red boy"} //After monster serialization = [{"address": "Beijing", "age":"7","name":"jack"},{"address": ["Mexico", "Hawaii"], "age":"20","name":"tom"}] //After monster serialization = 2345.67
Matters needing attention
For the serialization of structs, if you want the name of the serialized key to be reformatted, you can make a tag for struct
import ( "encoding/json" "fmt" ) //Define a structure type Monster struct { Name string `json:"monster_name"` //Reflection mechanism / /: do not separate the two sides Age int `json:"monster_age"` Bithday string Sal float64 Skill string } func testStruct() { //Demonstration monster := Monster{ Name : "King of cattle", Age : 500, Bithday : "2001-11-11", Sal : 8000.0, Skill : "Ngau Tau boxing", } //Serialize monster data, err := json.Marshal(&monster) if err != nil { fmt.Printf("Serial number error err = %v \n", err) } //Output serialized results fmt.Printf("monster After serialization = %v \n", string(data)) } func main() { //Demonstrate serialization of structs, map s, slices testStruct() } //output // After monster serialization = {"monster_name": "the devil of the ox", "monster_age":500,"Bithday":"2001-11-11","Sal":8000,"Skill": "niutouquan"}
Deserialization of Json
json deserialization refers to the operation of deserializing json strings into corresponding data types (such as structure, map, slice)
Application case
Show us how to de sequence json strings into structs, map s, and slices
import ( "encoding/json" "fmt" ) //Define a structure type Monster struct { Name string Age int Birthday string Sal float64 Skill string } //Demonstrate deserializing json string into struct func unmarshalStruct() { //In project development, str is obtained by network transmission or by reading files str := "{\"Name\":\"King of cattle\",\"Age\":500,\"Birthday\":\"2001-11-11\",\"Sal\":8000,\"Skill\":\"Ngau Tau boxing\"}" //Define a Monster instance var monster Monster err := json.Unmarshal([]byte(str), &monster) if err != nil { fmt.Printf("unmarshal err = %v\n", err) } fmt.Printf("After deserialization monster = %v monster.Name = %v \n", monster, monster.Name) } //Demonstrate deserializing json strings into map s func unmarshalMap() { str := "{\"address\":\"Hongya cave\",\"age\":30,\"name\":\"Red child\"}" //Define a map var a map[string]interface{} //De serialization //Note: to deserialize a map, make is not required because the make operation is encapsulated in the Unmarshal function err := json.Unmarshal([]byte(str), &a) if err != nil { fmt.Printf("unmarshal err = %v\n", err) } fmt.Printf("After deserialization a = %v\n",a) } //Demonstrate deserializing json strings into slice 1 func unmarshalSlice() { str := "[{\"address\":\"Beijing\",\"age\":\"7\",\"name\":\"jack\"},"+ "{\"address\":[\"Mexico\",\"Hawaii\"],\"age\":\"20\",\"name\":\"tom\"}]" //Define a slice var slice []map[string]interface{} //Deserialization, no make is required, because the make operation is encapsulated in the Unmarshal function err := json.Unmarshal([]byte(str), &slice) if err != nil { fmt.Printf("unmarshal err = %v\n", err) } fmt.Printf("After deserialization slice = %v\n", slice) } func main() { unmarshalStruct() unmarshalMap() unmarshalSlice() } //output //After deserialization, monster = {Bull Demon 500 2001-11-11 8000 bull fist} monster.Name = Bull Demon //After deserialization, a = map[address: Hongyadong age:30 name: honger] //Slice after deserialization = [map [address: Beijing age:7 name:jack] map[address: [Mexico Hawaii] age:20 name:tom]]
Matters needing attention
1) when deserializing a json string, make sure that the data type after deserialization is the same as that before the original serialization
2) if the json string is obtained through the program, you do not need to escape the
unit testing
First look at a need
In our work, we will encounter a situation where we need to confirm whether the result of a function or a module is correct
Such as:
func addUpper(n int) int { res := 0 for i := 1; i <= n; i++ { res += i } return res }
Traditional approach
In the main function, call the addUpper function to see whether the actual output result is consistent with the expected result. If it is consistent, it means that the function is correct. Otherwise, the function has an error, and then modify the error
//A tested function func addUpper(n int) int { res := 0 for i := 1; i <= n - 1; i++ { res += i } return res } func main() { //The traditional test method is to use the main function to see if the result is correct res := addUpper(10) if res != 55 { fmt.Printf("addUpper Error return value = %v expected value = %v\n ", res, 55) } else { fmt.Printf("addUpper Correct return value = %v expected value = %v\n", res, 55) } } //addUpper error return value = 45 expected value = 55
Analysis of the shortcomings of traditional methods
-
It's not convenient. You need to call it in the main function, so you need to modify the main function. If the project is running now, you may stop the project
-
It's not conducive to management, because when we test multiple functions or modules, we need to write them in the main function, which is not conducive to our management and clear thinking
-
Lead out unit test. ->Testing framework can solve problems
unit testing
Basic introduction
There is a lightweight test framework testing and its own go test command in Go language to implement unit test and performance test. The testing framework is similar to the test framework in other languages. You can write test cases for corresponding functions based on this framework, or write corresponding stress tests based on this framework. Through unit test, the following problems can be solved:
-
Make sure that each function is runnable and that the results are correct
-
Make sure that the written code performs well
-
Unit test can discover the logic errors of program design or implementation in time, make the problems exposed as early as possible, and facilitate the problem location and solution. The focus of performance test is to discover some problems in program design, so that the program can remain stable under the condition of high concurrency
quick get start
Use the unit test of Go to test the addUpper and sub functions
Special note: when testing, you may need to exit 360 temporarily (because 360 may think the generated test case program is a Trojan horse)
Show me how to unit test
Schematic diagram of unit test operation
Unit test quick start summary
-
The test case file name must end with 'ou test.go'. For example, cal_test.go, CAL is not fixed
-
The Test case function must start with Test. Generally speaking, it is the function name of Test + to be tested, such as TestAddUpper
-
Parameter type of TestAddUpper(t *testing.T) must be * testing.T
-
In a test case file, there can be multiple test cases Korean, such as TestAddUpper, TestSub
-
Run test case instructions
(1) CMD > go test
(2) CMD > go test - V
-
When an error occurs, you can use t.Fatalf to format the output error message and exit the program
-
t. The logf method can output corresponding logs
-
The test case function, which is not in the main function, can also be executed, which is the convenience of the test case
-
PASS indicates that the test case runs successfully, and fail indicates that the test case fails
-
Test a single file, be sure to bring the source file to be tested
go test -v cal_test.go cal.go
- Test a single method
go test -v -test.run TestAddUpper
Comprehensive case
Requirements for integrated case of unit test:
-
Write a Monster structure, field Name, Age, Skill
-
Bind the method Store to the Monster. You can serialize a Monster variable (object) and save it to a file
-
To bind the Monster method ReStore, you can read a serialized Monster from a file and deserialize it to a Monster object. Check that the deserialization is correct
-
Program the test case file store_test.go, write the test case functions TestStore and TestRestore to test
Code area
monster/monster.go
package monster import ( "encoding/json" "io/ioutil" "fmt" ) type Monster struct { Name string Age int Skill string } //Bind the method Store to the Monster. You can serialize a Monster variable (object) and save it to a file func (this *Monster) Store() bool { //Serialization first data, err := json.Marshal(this) if err != nil { fmt.Println("marshal err =", err) return false } //Save to file filePath := "e:/monster.ser" err = ioutil.WriteFile(filePath, data, 0666) if err != nil { fmt.Println("write file err =",err) return false } return true } //To bind the Monster method ReStore, you can read a serialized Monster from a file, //And deserialize to the Monster object. Check that the name is correct func (this *Monster) ReStore() bool { //1. First, read the serialized string from the file filePath := "e:/monster.ser" data, err := ioutil.ReadFile(filePath) if err != nil { fmt.Println("ReadFile err =", err) return false } //2. Use read data []byte to deserialize err = json.Unmarshal(data, this) if err != nil { fmt.Println("UnMarshal err = ", err) return false } return true }
monster/monster_test.go
package monster import "testing" //Test case, test Store method func TestStore(t *testing.T) { //Create a Monster instance first monster := &Monster{ Name: "Red child", Age: 10, Skill: "Spitting fire", } res := monster.Store() if !res { t.Fatalf("monster.Store() Error, want to be = %v Example is = %v",true,res) } t.Logf("monster.Store() Test successful!") } func TestReStore(t *testing.T) { //There are a lot of test data, and only after many tests can functions and modules be determined //Create a Monster instance first, without setting the value of the field var monster = &Monster{} res := monster.ReStore() if !res { t.Fatalf("monster.ReStore() Error, want to be = %v Example is = %v",true,res) } //Further judgment if monster.Name != "Red child" { t.Fatalf("monster.ReStore() Error, want to be = %v Example is = %v","Red child",monster.Name) } t.Logf("monster.ReStore() Test successful!") }