socket programming and tcp

Since tcp/ip protocols are encapsulated on the bottom layer of the operating system, in order to avoid the application layer directly calling the methods of the operating system, the operating system provides a socket interface, which is called by the application layer. The socket interface is divided into tcpSocket and udpSocket. It can be understood that socket is actually a facade of the application layer and the transport layer, Calling the socket method is just to call the transmission of our tcp/udp protocol

tcp features:

1. Reliable transmission (checksum, timeout retry mechanism, flow control, congestion control)
2. Byte oriented stream
3. Three handshakes and four waves

socket programming of go

server side:

func main()  {

	//Listening port
	listen,err:=net.Listen("tcp","localhost:20000")
	if(err!=nil){
		fmt.Println("err:",err)
		return
	}

	
	for {
	//Request received
		coon, err := listen.Accept()
		if (err != nil) {
			fmt.Println("err:", err)
			return
		}
	//A request to start a Ctrip
		go slove(coon)
	}

}

func slove(conn net.Conn) {

	defer conn.Close()
	for {
		reader := bufio.NewReader(conn)
		var buf [128]byte
		n, err := reader.Read(buf[:])
		if (err != nil) {
			fmt.Println("err:", err)
			return
		}
		s := string(buf[:n])
		fmt.Println(s)
		conn.Write([]byte(s))
	}
}

client:

func main()  {
	//Connect server
	conn, err := net.Dial("tcp", "localhost:20000")
	if(err!=nil){
		fmt.Println(err)
		return
	}
	for{
	//Read and write data
		var s string
		fmt.Scanln(&s)
		//It is only sent to the buffer and will not be sent to the client immediately
		_, err := conn.Write([]byte(s))
		if(err!=nil){
			return
		}
		reader := bufio.NewReader(conn)
		var buf [128]byte

		n, err := reader.Read(buf[:])
		if(err!=nil){
			fmt.Printf(err.Error())
			return

		}
		fmt.Println(string(buf[:n]))
	}
}

The above socket programming can realize concurrent requests. What happens if we directly use socket programming to transmit our application layer messages.
1. Packet gluing: if the client sends a message with very small data, and the client sends several messages, these messages will not be sent to the server immediately, but will be accumulated in the client's cache. Similarly, the server will not be sent to the application layer immediately after receiving the message ack confirmation, but will be accumulated in our buffer. In the above case, the server will read all the messages from the client.
2. Packet breakage: the data sent by the client is very large. If it exceeds the buffer, it will be written several times, while the server will read it several times. The server thinks that the customer service has sent several messages.

The above reason is that the server does not know what the boundary of the client message is. A simple solution is that the client adds a header in front of the message to tell how long the message is, and the server reads the message in this way. This is an agreement between the two sides. Before sending the message, the client serializes the message into an identifiable binary stream, The server de serializes into messages. The message transmission of this kind of serialization and de serialization is a process that rpc Protocol or application layer protocol must achieve

go's http package

http applications are very concise

func main()  {
//Route registration
	http.HandleFunc("/", func(re http.ResponseWriter, rs *http.Request) {
		re.Write([]byte("Hello World"))
	})
//Bind the port, receive the requested connection, and then process the requested connection
	http.ListenAndServe(":8080",nil)
}

Here, the path is registered on the http default route

func HandleFunc(pattern string, handler func(ResponseWriter, *Request)) {
	DefaultServeMux.HandleFunc(pattern, handler)
}

Listening and receiving http requests

func ListenAndServe(addr string, handler Handler) error {
//Encapsulate a server class
	server := &Server{Addr: addr, Handler: handler}
	return server.ListenAndServe()
}

func (srv *Server) ListenAndServe() error {
	if srv.shuttingDown() {
		return ErrServerClosed
	}
	addr := srv.Addr
	if addr == "" {
		addr = ":http"
	}
	//Listen to the request port in tcp mode
	ln, err := net.Listen("tcp", addr)
	if err != nil {
		return err
	}
	//Processing request connections
	return srv.Serve(ln)
}

func (srv *Server) Serve(l net.Listener) error {
    // ... omit code
    for {
      rw, e := l.Accept() //Look here accept
      ctx := context.WithValue(baseCtx, ServerContextKey, srv)
      //Omit a lot
     c := srv.newConn(rw)
	 c.setState(c.rwc, StateNew, runHooks) // before Serve can return
	 //Start a process to process the request
	 go c.serve(connCtx) 
    }
}

func (c *conn) serve(ctx context.Context) {
    // ... omit the code. Here, we mainly process the request information and convert it into an httpRequest and httpResponse object
    serverHandler{c.server}.ServeHTTP(w, w.req)
    w.cancelCtx()
    if c.hijacked() {
      return
    }
    w.finishRequest()
    // ... omit code
}

func (sh serverHandler) ServeHTTP(rw ResponseWriter, req *Request) {
	handler := sh.srv.Handler
	if handler == nil {
	//If there is no other route, use http to customize the route
		handler = DefaultServeMux
	}
	if req.RequestURI == "*" && req.Method == "OPTIONS" {
		handler = globalOptionsHandler{}
	}
	//Processing http information with routing
	handler.ServeHTTP(rw, req)
}

Review the calling process of http
http.ListenAndServe(":8080",nil)

1. Establish a tcpSocket connection and listen to port 8080
2. After accepting the request, start a process to process the request
3. The request information is processed and encapsulated into httpReqeust and httpResponse objects, which are processed by serverhandler {c.server}. Serverhttp (W, w.req)
4. Then find the real http route (if not, use the default), and find the corresponding handler according to the request path
5. Response request

http summary

1. The http package has helped us to encapsulate the socket. The request message is encapsulated into an httpRequest object and given to the router to process the request. After the route is processed, it will help us respond to the request
2. However, the HTTP default route matching is very simple, which is equivalent to matching with a map. It does not support restful style, nor does it have the concept of middleware and routing group. The clever design is that HTTP will not give priority to the default route, but will use our custom route. It is also very simple to implement the custom route. You only need to implement the Handler interface of the HTTP package. For example, the Engine object like gin implements the Handler interface to realize the custom route of http

type Handler interface {
	ServeHTTP(ResponseWriter, *Request)
}

Support restful style gin

Basic use of gin

type Login struct {
	// binding:"required" modified field. If the received value is null, an error will be reported. It is a required field
	User     *string `form:"username" json:"user" `
	Password *string `form:"password" json:"password"`
}

func main() {
	// 1. Create a route
	// Two middleware loggers () and recovery () are used by default
	r := gin.Default()
	r.POST("/loginForm", func(c *gin.Context) {
		// Declare received variables
		var form Login
		// Bind() parses and binds the form format by default
		if err := c.ShouldBind(&form); err != nil {
			fmt.Println(form)
			c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
			return
		}
		fmt.Println(*form.Password)
		fmt.Println(*form.User)
		c.JSON(http.StatusOK, gin.H{})

	})
	r.Run(":8000")
}

Start service

func (engine *Engine) Run(addr ...string) (err error) {
	defer func() { debugPrintError(err) }()

	address := resolveAddress(addr)
	debugPrint("Listening and serving HTTP on %s\n", address)
	err = http.ListenAndServe(address, engine)
	return
}

Get request connection, http request will request here

// ServeHTTP conforms to the http.Handler interface.
func (engine *Engine) ServeHTTP(w http.ResponseWriter, req *http.Request) {
    //Get the context of the object pool
	c := engine.pool.Get().(*Context)
	//Reset context object
	c.writermem.reset(w)
	c.Request = req
	c.reset()
	//Find the corresponding handler call chain through the routing tree and execute the handler call chain
	engine.handleHTTPRequest(c)
	//Put context into object pool
	engine.pool.Put(c)
}

Take a quick look at the handleHTTPRequest

func (engine *Engine) handleHTTPRequest(c *Context) {
	httpMethod := c.Request.Method
	rPath := c.Request.URL.Path
	unescape := false
	if engine.UseRawPath && len(c.Request.URL.RawPath) > 0 {
		rPath = c.Request.URL.RawPath
		unescape = engine.UnescapePathValues
	}

	if engine.RemoveExtraSlash {
		rPath = cleanPath(rPath)
	}

	// Find root of the tree for the given HTTP method
	t := engine.trees
	for i, tl := 0, len(t); i < tl; i++ {
		if t[i].method != httpMethod {
			continue
		}
		root := t[i].root
		// Find route in tree
		value := root.getValue(rPath, c.Params, unescape)
		if value.handlers != nil {
			c.handlers = value.handlers
			c.Params = value.params
			c.fullPath = value.fullPath
			//Execute handler
			c.Next()
			c.writermem.WriteHeaderNow()
			return
		}
		if httpMethod != "CONNECT" && rPath != "/" {
			if value.tsr && engine.RedirectTrailingSlash {
				redirectTrailingSlash(c)
				return
			}
			if engine.RedirectFixedPath && redirectFixedPath(c, root, engine.RedirectFixedPath) {
				return
			}
		}
		break
	}
	//Omit a lot
}

Advantages of gin

1. gin is just a custom router of http. It inherits the high-performance characteristics of http and naturally supports high concurrency
2. gin has the concept of routeGroup, such as (user). All route registrations under this routing group will inherit all middleware of the current routing group.
3. The concept of middleware is introduced to facilitate the secondary expansion of gin. For example, our project uses middleware for authority authentication and login authentication
4. For friendly parameter binding, you only need to bind the parameters to the structure through the tag form and json.
5. It supports restful style, uses multiple cardinal trees for route matching, and compresses the memory to the extreme. The advantages of using cardinal trees can also deal with the general path matching characters "*" and ":", such as (user/:id),(user/*id)
6. Validator is integrated for parameter verification

The above only introduces http streaming to gin, but it doesn't explain how gin registers the routing tree and finds the handler through the routing tree. You can simply see what it looks like first

reference

Official documents
How does gin flow to http
gin cardinal tree data structure