Background and overview

1. The author is a new recruit in the school. The company uses Golang. He has Web development experience in Java and PHP and is familiar with the front-end Vue technology stack. He has not written Golang project before. He just read Golang's syntax according to the rookie tutorial a few months ago to understand its naturally high concurrency characteristics and its two important characteristics: collaboration and pipeline.

2. On the day of entry, you are required to do a task of Golang's novice village by yourself and a distributed login and registration that can achieve 1000QPS under 2000 concurrent access. The deadline is one week. It is inconvenient to disclose the details, which will not be carried out in detail here. The dry goods shared in this article mainly come from the accumulation of tasks in this novice village.

3. This article mainly shares the pit I stepped in the process of converting Java to Golang. The main differences from the existing Golang tutorials are as follows:

   • Do not pursue systematic and comprehensive (greedy for more than chew)
   • It belongs to the introductory tutorial
   • Be friendly to small partners with Java experience and rapid development needs

4. Through this article, you will learn:

   • Golang's common syntax and project organization
   • Use of common components of Go language: rpc, gin, redis, mysql
   • Basic use of Go language pressure measuring tool wrk
   • Unit testing under Go technology stack

5. Although it is a quick tutorial, it also takes several hours to read this article slowly. There is no more nonsense. Let's start now~

Common syntax and code organization of Golang

Grammar

1. If you have systematically read Golang's grammar, here I recommend it to you y-minute learning x language series tutorials , and its Domestic version of handling , you can quickly review the grammar through this (there is no need to memorize the grammar, but you can remember it when you use it more)
2. If you haven't read Golang syntax systematically, it doesn't matter. It's enough to read the following code carefully

// Single-Line Comments 
/* Multiline
    notes */
// I recommend vscade for go's IDE, and Golan is also good. The former is for my own use, and the latter is used by my colleagues
// When developing with go, you will often see document annotations, such as @ Version 1.0.0

// The clause of the import package is at the beginning of each source file.
// Main is special. It is used to declare executable files instead of a library.
package main

// The Import statement declares the package referenced by the current file.
import (
    "fmt"       // Packages in Go language standard library
    "net/http"  // A web server package
    "strconv"   // String conversion
    "strings"   // String standard library
)
// When you are used to writing go without points, writing Java will be very twisted

// Function declaration: Main is the entry of program execution.
// Whether you like it or not, Go uses curly braces to wrap the function body. The curly braces are combined with strict indentation, and the code is very easy to read.
func main() {
    // Print a line to standard output.
    // Restrict the print function with the package name fmt.
    fmt.Println("Hello world!")
	// Use log when writing items Println("Hello world!")  Will be more
	
    // Call another function of the current package.
    beyondHello()
}

// Functions can have arguments in parentheses.
// If there are no parameters, an empty bracket is also required.
func beyondHello() {
    var x int   // Variable declaration, variables must be declared before use.
    x = 3       // Variable assignment.
    // You can use: = to be lazy. It automatically completes the variable type, declaration and assignment, which is most commonly used in projects.
    y := 4
    sum, prod := learnMultiple(x, y)        // Function that returns multiple variables
    fmt.Println("sum:", sum, "prod:", prod) // Simple output
    learnTypes()                            // Less than y minutes, learn more!
}

// Functions with multiple variables and multiple return values
func learnMultiple(x, y int) (sum, prod int) {
    return x + y, x * y // Returns two values
    /*
	// Because the return value here has not only the type, but also the name, so the function body can also be written in this way
	sum = x + y // Note that this is not:= 
	prod = x * y
	return
	*/
}

// Built in variable types and keywords
func learnTypes() {
    // Short statements give you what you want.
    s := "Learn Go!" // String type

    s2 := `A "raw" string literal
can include line breaks.` // It is also a String type

    // Non ascii characters. Go uses UTF-8 encoding.
    g := 'Σ' // rune type, alias of int32, encoded in UTF-8

    f := 3.14195 // float64 type, IEEE-754 64 bit floating point number
    c := 3 + 4i  // complex128 type, internally represented by two float64

    // Var variables can be initialized directly.
    var u uint = 7  // Unsigned unsigned variable, but the implementation depends on the length of int variable
    var pi float32 = 22. / 7

    // Character conversion
    n := byte('\n') // byte is an alias for uint8

    // The size of the array type is fixed when compiling.
    var a4 [4] int              // An array with 4 int variables, initially 0
    a3 := [...]int{3, 1, 5}     // An array with three int variables is initialized at the same time

    // Slice can be dynamically added or deleted. Array and slice have their own advantages, but slice is used in more places.
    s3 := []int{4, 5, 9}        // Compared with a3, there is no ellipsis here
    s4 := make([]int, 4)        // Allocate a slice with four int variables, all of which are initialized to 0

	// Common operation
	str := "With Jiahao and even dilute slag"
	bs := []byte(str) // Convert string to byte slice
	str2 := string(bs) // byte slice to string
	println(str, len(str), bs, len(bs), str2, strings.Count(str, "")-1) // The println function is more like Java's sysout
	fmt.Println(str, len(str), bs, len(bs), str2, strings.Count(str2, "residue")) // This effect is better
	
    var d2 [][]float64          // It's just a statement. There's nothing assigned
    bs := []byte("a slice")     // Syntax of type conversion

    p, q := learnMemory()       // Declare P and Q as pointers to int variables
    fmt.Println(*p, *q)         // *Value

    // Map is a dynamic and increasable associative array, which is similar to hash or dictionary in other languages.
    m := map[string]int{"three": 3, "four": 4}
    m["one"] = 1

    // Variables that are not used in Go language will report errors during compilation, not warning.
    // Underline_ You can "use" a variable, but discard its value.
    _,_,_,_,_,_,_,_,_ = s2, g, f, u, pi, n, a3, s4, bs
    // Output variables. It can be seen from the results that the above operation will not clear the variables
    fmt.Println(s, c, a4, s3, d2, m)

    learnFlowControl() // Return to process control 
}

// Go fully supports garbage collection. Go has pointers, but does not support pointer operations.
// You will make mistakes because of null pointers, but you will not make mistakes because of adding pointers.
func learnMemory() (p, q *int) {
    // Returns pointer p and q of int type variable
    p = new(int)    // The built-in function new allocates memory
    // Automatically assign the assigned int to 0, and p is no longer empty.
    s := make([]int, 20)    // Allocate a block of memory for 20 int variables
    s[3] = 7                // assignment
    r := -2                 // Declare another local variable
    return &s[3], &r        // &Take address
}

func expensiveComputation() int {
    return 1e6 // 1 * 10^6
}

func learnFlowControl() {
    // If you need curly braces, you don't need them
    if true {
        fmt.Println("told ya")
    }
    // Using go fmt command can help you format code, so you don't have to make complaints about the style of code.
    // You don't have to tolerate the style of code.
    if false {
        // pout
    } else {
        // gloat
    }
    // If there are too many nested if statements, switch is recommended
    x := 1
    switch x {
    case 0:
    case 1:
        // Implicitly call the break statement and stop when matching the previous one
    case 2:
        // Will not run
    case 3:
    	// If you have to match the previous one, don't stop, it's not impossible
    	fallthrough // However, it is not recommended
    default:
    	// No matching execution
    }
    
    // Like if, for doesn't use parentheses
    for x := 0; x < 3; x++ { // ++Self increasing
        fmt.Println("iteration", x)
    }
    // x is still 1 here. Why?

    // for is the only circular keyword in go, but it has many variants
    for { // Dead cycle
        break    // I lied to you 
        continue // It won't run
    }
    // Like for, in if: = first assign a value to y, and then compare it with x.
    if y := expensiveComputation(); y > x {
        x = y
    }
    
    // Closure function
    xBig := func() bool {
        return x > 100 // x is the variable reference declared above
    }
    fmt.Println("xBig:", xBig()) // true (it assigns y to x) 
    x /= 1e5                     // x becomes 10
    fmt.Println("xBig:", xBig()) // Now it's false

    // When you need goto, you will love it!
    goto love
love:

    learnInterfaces() // Here comes the good thing!
}

// Define Stringer as an interface type with a method String
type Stringer interface {
    String() string // There can be multiple method declarations
}

// pair is defined as a structure with two int variables x and y.
// This pair is equivalent to a class in Java
type pair struct {
    x, y int
}

// Define the member method of pair type and implement the Stringer interface. There is no need to display the specified Stringer here
func (p pair) String() string { // p is called "receiver"
    // Sprintf is another public function in the fmt package.
    // Use Call the element in p.
    return fmt.Sprintf("(%d, %d)", p.x, p.y)
}

func learnInterfaces() {
    // Curly braces are used to define structural variables,: = assign a structural variable to p here.
    p := pair{3, 4}
    fmt.Println(p.String()) // Call the String method of pair type p 
    var i Stringer          // Declare i as Stringer interface type 
    i = p                   // Effective! Because p implements the Stringer interface (similar to the shaping in java) 
    // Call the String method of i, and the output is the same as above
    fmt.Println(i.String())

    // The Println function in the fmt package asks for their string output from objects. It can be used by implementing the string method.
    // (similar to serialization in java)
    fmt.Println(p) // The output is the same as above, and the String function is called automatically.
    fmt.Println(i) // The output is the same as above.

    learnErrorHandling()
}

// You can define other structures as receivers to implement the String() string method and realize polymorphism,
// When instantiating, take Stringer as its type (var obj Stringer) and assign different structures
// For detailed reference https://www.jianshu.com/p/1227c5145cd8
// I think Golang can barely be object-oriented, but there's no need to worry too much about Java
func learnErrorHandling() {
	// At first, you don't adapt to try catch. If you get used to it, you will fall in love with her
    // ", ok" is used to judge whether it is working normally 
    m := map[int]string{3: "three", 4: "four"}
    if x, ok := m[1]; !ok { // ok is false because there is no 1 in m
        fmt.Println("no one there")
    } else {
        fmt.Print(x) // If x is in the map, X is the value.
    }
    // The error is not just ok, it can also give more details about the problem.
    if _, err := strconv.Atoi("non-int"); err != nil { // _ discards value
        // Output "strconv.parseint: parsing" non int ": invalid syntax"
        fmt.Println(err)
    }
    // I'll talk about the interface later. At the same time,
    learnConcurrency()
}

// c is a channel type, a concurrent and secure communication object.
func inc(i int, c chan int) {
    c <- i + 1 // < - send the right to the left channel.
}

// We will use the inc function to add some numbers concurrently.
func learnConcurrency() {
    // Use make to declare a slice. Make will allocate and initialize slice, map and channel.
    c := make(chan int)
    // Start three concurrent goroutine s with the go keyword. If the machine supports it, it may also be executed in parallel.
    // All three are sent to the same channel.
    go inc(0, c) // go is a statement that starts a new goroutine.
    go inc(10, c)
    go inc(-805, c)
    // Leave the results alone from the channel and print them.
    // Printing out something is unpredictable.
    fmt.Println(<-c, <-c, <-c) // When the channel is on the right, < - is a read operation.

    cs := make(chan string)       // channel of operation string
    cc := make(chan chan string)  // Operating channel
    go func() { c <- 84 }()       // Start a goroutine to send a new number asynchronously 
    go func() { cs <- "wordy" }() // Send to cs
    
    // Select is similar to switch, but each case includes a channel operation.
    // It randomly selects a case ready for communication.
    select {
    case i := <-c: // The value received from channel can be assigned to other variables
        fmt.Println("it's a", i)
    case <-cs: // Or discard it directly
        fmt.Println("it's a string")
    case <-cc: // Empty, not ready for communication 
        fmt.Println("didn't happen.")
    }
    // The value of c or cs above is taken. One goroutine ends and the other is blocked all the time.

    learnWebProgramming() // Go is very suitable for web programming. I know you also want to learn!
}
// The above functions involve multithreading and multithreaded communication, but they are simple and clear

// A simple function in the http package can start the web server.
func learnWebProgramming() {
    // The first parameter of ListenAndServe specifies the listening port, and the second parameter is an interface, specifically http Handler. 
    err := http.ListenAndServe(":8080", pair{})

    fmt.Println(err) // Don't ignore mistakes.
    // How to gracefully close http server: https://www.cnblogs.com/oxspirt/p/7058812.html
}

// Make pair realize http The serverhttp method of the handler interface.
func (p pair) ServeHTTP(w http.ResponseWriter, r *http.Request) {
    // Use HTTP Responsewriter returns data
    w.Write([]byte("You learned Go in Y minutes!"))
    defer fmt.Println("Last execution")
	// The defer statement is often used to close io. It is executed successively after the code in the function is executed
	// The execution sequence follows the stack pressing and stack out sequence
	defer func(){
		fmt.Println("Penultimate execution")
	}()
}

So far, you have mastered the key contents of most Go language tutorials, but the following Golang code organization is more important, which is the author's valuable experience in taming Go language in the early stage.

Code organization

1. For Java or any other programming language, there are scope limited keywords such as global, var, public and private, but this is not the case for Golang. The methods starting with uppercase letters in Go language are public and those starting with lowercase letters are private. That's why FMT The reason why the method name of println() starts with an uppercase letter

Understand this, as a Javaer, do you think Golang has a beautiful face?

1. This is very important. The author used JSON in his early days When Marshal serializes a struct, the serialized string is an empty object {} without member variables. The problem was not solved until I changed the name of member variables to start with uppercase letters!
2. Classes and methods in Java will be decorated with the static keyword. In Golang, package variables and package functions are static, that is, in the same package import ed from multiple places, the directly defined variables are global.
3. Package and import are quite different from those in Java. Package in Java is a complete path string, which is consistent with the directory structure. The path in Go is the path, and the package name is the package name. When importing, import does not import the package name, but the package path. The package name is used in the code importing the package, such as FMT Println().
4. This is also very important. In the early stage, the author tried to import a file under the package of Golang like importing a Java class. However, he could not succeed. He tried and tried again and again until he found that the scope of several files under a package of Golang was interconnected, that is, several files under a directory (excluding subdirectories) were the same package, Just import the directory path where the package is located. Functions and variable scopes starting with lowercase letters in different files under the same package are interconnected, just like in the same file.
5. About GOPATH and go mod, GOPATH is an environment variable used in go language. It uses absolute path to provide the working directory of the project. When our third-party package is not the official library, it can be used only under GOPATH/src. This scheme causes the third-party libraries and their own projects to be in the GOPATH/src directory, which is mixed. If different packages under GOPATH/src depend on different versions of the same third-party library, there will be version conflicts. In short, it is unfriendly to multi project support. So in go1 After version 11, go mod appeared, which is similar to Maven in Java. go.mod is in the root directory of the project The mod dependency configuration file is similar to pom.com in the root directory of Maven project xml.

You have solved the problem of code organization, and you have mastered the basic skills of living with Golang

Use of common components of Go language

As we all know, the early implementation of go language was implemented in C and assembly. In version 1.5, it implemented bootstrapping (using go to implement go compiler), which means that it is not difficult to understand the Golang syntax and then understand the Golang source code. Looking at the source code, there will always be some unexpected receipts.

rpc

• from Golang You can see two rpc packages in the source code of. One is net/rpc and the other is net/rpc/jsonrpc
  
• The corresponding package has test code, such as server The test code of go is server_test.go can be regarded as a very high-quality official Demo

ginweb

• recommend Official Chinese Course , which provides Many example s , following handling
• demo: run one interface

package main

import "github.com/gin-gonic/gin"

func main() {
	r := gin.Default()
	r.GET("/ping", func(c *gin.Context) {
		c.JSON(200, gin.H{
			"message": "pong",
		})
	})
	r.Run() // Listen and start the service on 0.0.0.0:8080
}

• In recent years, the development of front-end and back-end separation and front-end pure static file service are popular:

func main() {
	router := gin.Default()
	router.Static("/assets", "./assets")
	router.StaticFS("/more_static", http.Dir("my_file_system"))
	router.StaticFile("/favicon.ico", "./resources/favicon.ico")

	// Listen and start the service on 0.0.0.0:8080
	router.Run(":8080")
}

redis

• go-redis just so so
• Redis single point failure does occur in my development experience, so it is necessary to have high availability of redis, which is often asked in the interview. About the redis cluster scheme, It is recommended to read this article Finally, the article concludes that the official redis cluster is a more reliable choice, but I think Codis is more friendly to developers, solves some key pain points and is more recommended.

mysql

• gorm document

Basic use of Go language pressure measuring tool wrk

• Golang has high performance. Using Jmeter at a single point may not reach the upper performance limit, so I use it wrk
• The execution process and life cycle of the lua script of wrk are rather complicated. Here is a summary

    1. Life cycle:
    
    First, it is generally divided into three stages: setup, ruuning, done
    Each stage can then be subdivided:
    setup(thread) Once per thread
    runing:
        init(args) Once per thread
            delay() Once per request
            request() Once per request
            response(status, headers, body) Once per request
    done(summary, latency, requests)

    It is worth noting here that statements outside the periodic function will be executed thread + 1 Therefore, if you want to initialize something once, you should use the counter to judge whether it is executed for the first time

    2. Variable scope
        a. lua In script, External variable of life cycle function
        b. setup Within stage function
        c. runing Within the function of the stage
        d. done Within the function of the stage
        e. In other custom functions
    In here 
        - abd Scope interworking
        - c The scopes of all the functions of are interconnected, but each thread holds them separately
        - d Can pass thread Get local variables for each thread
        
    issue: How in each thread Get global variables from,Such as test data set?
    - Method 1: in the life cycle function setup Given in thread Incoming data, disadvantage, not suitable for a large amount of data
    - Method 2: Citation transmission (its feasibility needs to be investigated) wrk Memory call model and lua How to cross process reference passing) from the error reporting situation, guess init Heap memory is used in the stage, which can load more data

• example:

samples = {} -- lua Function transfer in table Pass by reference
local counter = 1 -- Global variables
local threads = {}


function setup(thread)
    -- implement t second
    if counter == 1 then 
        loadSamples()
    end
    thread:set("id", counter)
    math.randomseed(os.time())
    -- The use case setting is completed when the thread is initialized, so it is a fixed user
    thread:set("sample", samples[math.random(0, #samples)]) 
    counter = counter + 1
end

function init(args)
    faild = 0
    responses = 0
end

-- Load data
function loadSamples()
    local j = 0
    for line in io.lines("../pressure/loginbody4lua.txt")
    do
        samples[j] = line
    end
    -- print("Total loaded"..j.."Use cases")
end

-- Unit test to verify the correctness. If there is no comment during the formal pressure test, the script performance will not be improved
-- function response(status, headers, body)
--     -- It belongs to thread, and each request is executed once, Statistics failed
--     if status == 200 then
--         if(not string.find(body,'"code":0')) then
--             faild = faild + 1
--         end
--         wrk.thread:stop()
--     else 
--         faild = faild + 1
--     end
--     responses = responses + 1
-- end

wrk.method = "POST"
wrk.host = "127.0.0.1"
wrk.port = 8000
wrk.path = "/api/v1/login"
wrk.headers["Content-Type"] = "application/json"
wrk.headers["Connection"] = "keep-alive"

function request(args)
    wrk.body = sample
    return wrk.format()
end


-- 200 Concurrent request 
-- redis-cli flushall
-- redis-cli flushall & wrk -t200 -c200 -d60s -s login_fixuser.lua http://127.0.0.1:8000   --latency
-- wrk -t200 -c200 -d120s -s login_fixuser.lua http://127.0.0.1:8000   --latency

-- 2000 Concurrent request 
-- redis-cli flushall
-- wrk -t2000 -c2000 -d30s -s login_fixuser.lua http://127.0.0.1:8000  --latency
-- wrk -t2000 -c2000 -d60s -s login_fixuser.lua http://127.0.0.1:8000  --latency
-- wrk -t2000 -c2000 -d120s -s login_fixuser.lua http://127.0.0.1:8000  --latency

Unit testing under Go technology stack

• During unit testing, the core work is to mock off external dependencies. The best examples of go technology stack are: go-monkey