Implementation principle of defer in GO

Let's review our last sharing and share some knowledge about channels

• What is the channel in GO
• Detailed analysis of the underlying data structure of the channel
• How is the channel implemented in the GO source code
• Basic principles of Chan reading and writing
• What exceptions will occur when the channel is closed, panic
• Simple application of select

If you are still interested in chan channel, please check the article Chan realizes principle sharing in GO

What is defer?

Let's see what defer is

Is a keyword in GO

This keyword is usually used to release resources. It will be called before return

If there are multiple defers in the program, the call order of defers is in a stack like manner, last in first out LIFO, which is written here by the way

• Stack

Follow the principle of last in first out

Last in, first out

First in the stack, then out of the stack

• queue

Following the principle of first in first out, we can imagine a one-way pipe, which goes in from the left and out from the right

First in, first out

Come in later, go out later. Don't jump in the queue

Implementation principle of defer

Let's draw a conclusion first and have a little bottom in mind:

• The position of the defer is declared in the code. When compiling, a function called deferproc will be inserted. A return function will be inserted in front of the function where the defer is located, not return, but deferreturn

The specific implementation principle of defer is the same. Let's see what the underlying data structure of defer is,

In Src / Runtime / runtime2 Type of go_ Defer struct {structure

// A _defer holds an entry on the list of deferred calls.
// If you add a field here, add code to clear it in freedefer and deferProcStack
// This struct must match the code in cmd/compile/internal/gc/reflect.go:deferstruct
// and cmd/compile/internal/gc/ssa.go:(*state).call.
// Some defers will be allocated on the stack and some on the heap.
// All defers are logically part of the stack, so write barriers to
// initialize them are not required. All defers must be manually scanned,
// and for heap defers, marked.
type _defer struct {
   siz     int32 // includes both arguments and results
   started bool
   heap    bool
   // openDefer indicates that this _defer is for a frame with open-coded
   // defers. We have only one defer record for the entire frame (which may
   // currently have 0, 1, or more defers active).
   openDefer bool
   sp        uintptr  // sp at time of defer
   pc        uintptr  // pc at time of defer
   fn        *funcval // can be nil for open-coded defers
   _panic    *_panic  // panic that is running defer
   link      *_defer

   // If openDefer is true, the fields below record values about the stack
   // frame and associated function that has the open-coded defer(s). sp
   // above will be the sp for the frame, and pc will be address of the
   // deferreturn call in the function.
   fd   unsafe.Pointer // funcdata for the function associated with the frame
   varp uintptr        // value of varp for the stack frame
   // framepc is the current pc associated with the stack frame. Together,
   // with sp above (which is the sp associated with the stack frame),
   // framepc/sp can be used as pc/sp pair to continue a stack trace via
   // gentraceback().
   framepc uintptr
}

_ defer holds an entry in the deferred call list. Let's see what the parameters of the above data structure mean

The defer keyword must be followed by a function, which we should remember

Through the description of the above parameters, we can know that the data structure of defer is similar to that of the function. It also has the following three parameters:

• Stack pointer SP
• Program counter PC
• Address of the function

But have we also found that there is a link in the member, and the defer delay function in the same coroutine will be connected together through this pointer

This link pointer refers to the header of a defer single linked list. Every time we declare a defer, we will insert the data of the defer into the header of the single linked list,

So, when we execute defer, can we guess how the defer was achieved?

As mentioned earlier, defer is a last in first out. Of course, this principle is followed here. When taking defer for execution, it is taken from the head of the single linked list.

Let's draw a picture

Declare 2 defers in the process A, and declare defer test1() first

Redeclare defer test2()

It can be seen that the later declared defer will be inserted into the header of the single linked list, and the first declared defer will be listed later

When we take it, we always take the head down and execute it until the single linked list is empty.

Let's take a look at the concrete implementation of defer

The source file is in Src / Runtime / panic Go to view the function deferproc

// Create a new deferred function fn with siz bytes of arguments.
// The compiler turns a defer statement into a call to this.
//go:nosplit
func deferproc(siz int32, fn *funcval) { // arguments of fn follow fn
   gp := getg()
   if gp.m.curg != gp {
      // go code on the system stack can't defer
      throw("defer on system stack")
   }

   // the arguments of fn are in a perilous state. The stack map
   // for deferproc does not describe them. So we can't let garbage
   // collection or stack copying trigger until we've copied them out
   // to somewhere safe. The memmove below does that.
   // Until the copy completes, we can only call nosplit routines.
   sp := getcallersp()
   argp := uintptr(unsafe.Pointer(&fn)) + unsafe.Sizeof(fn)
   callerpc := getcallerpc()

   d := newdefer(siz)
   if d._panic != nil {
      throw("deferproc: d.panic != nil after newdefer")
   }
   d.link = gp._defer
   gp._defer = d
   d.fn = fn
   d.pc = callerpc
   d.sp = sp
   switch siz {
   case 0:
      // Do nothing.
   case sys.PtrSize:
      *(*uintptr)(deferArgs(d)) = *(*uintptr)(unsafe.Pointer(argp))
   default:
      memmove(deferArgs(d), unsafe.Pointer(argp), uintptr(siz))
   }

   // deferproc returns 0 normally.
   // a deferred func that stops a panic
   // makes the deferproc return 1.
   // the code the compiler generates always
   // checks the return value and jumps to the
   // end of the function if deferproc returns != 0.
   return0()
   // No code can go here - the C return register has
   // been set and must not be clobbered.
}

The function of deferproc is:

Create a new deferred function fn with a parameter of siz bytes. The compiler converts a deferred statement into a call to this

getcallersp():

Get the value of rsp register before deferproc, and the implementation method is the same for all platforms

//go:noescape
func getcallersp() uintptr // implemented as an intrinsic on all platforms

callerpc := getcallerpc():

The rsp obtained here is stored in callerpc to call the next instruction of deferproc

d := newdefer(siz):

D: = newdefer (siz) create a new defer structure. The subsequent code is assigning values to the members of the defer structure

Let's take a look at the general process of deferproc:

• Gets the value of the rsp register before deferproc
• Use newdefer to assign a_ Defer structure object and put it into the current_ Header of defer linked list
• Initialize_ Related member parameters of defer
• return0

Let's take a look at the source code of newdefer

The source file is in Src / Runtime / panic Go to view the function newdefer

// Allocate a Defer, usually using per-P pool.
// Each defer must be released with freedefer.  The defer is not
// added to any defer chain yet.
//
// This must not grow the stack because there may be a frame without
// stack map information when this is called.
//
//go:nosplit
func newdefer(siz int32) *_defer {
    var d *_defer
    sc := deferclass(uintptr(siz))
    gp := getg()
    if sc < uintptr(len(p{}.deferpool)) {
        pp := gp.m.p.ptr()
        if len(pp.deferpool[sc]) == 0 && sched.deferpool[sc] != nil {
            // Take the slow path on the system stack so
            // we don't grow newdefer's stack.
            systemstack(func() {
                lock(&sched.deferlock)
                for len(pp.deferpool[sc]) < cap(pp.deferpool[sc])/2 && sched.deferpool[sc] != nil {
                    d := sched.deferpool[sc]
                    sched.deferpool[sc] = d.link
                    d.link = nil
                    pp.deferpool[sc] = append(pp.deferpool[sc], d)
                }
                unlock(&sched.deferlock)
            })
        }
        if n := len(pp.deferpool[sc]); n > 0 {
            d = pp.deferpool[sc][n-1]
            pp.deferpool[sc][n-1] = nil
            pp.deferpool[sc] = pp.deferpool[sc][:n-1]
        }
    }
    if d == nil {
        // Allocate new defer+args.
        systemstack(func() {
            total := roundupsize(totaldefersize(uintptr(siz)))
            d = (*_defer)(mallocgc(total, deferType, true))
        })
    }
    d.siz = siz
    d.heap = true
    return d
}

Function of newderfer:

Usually, a per-P pool is used to allocate a Defer

Each defer can be freely released. Currently, defers will not be added to any defer chain

getg():

Gets the structure pointer of the current collaboration

// getg returns the pointer to the current g.
// The compiler rewrites calls to this function into instructions
// that fetch the g directly (from TLS or from the dedicated register).
func getg() *g

pp := gp.m.p.ptr():

Get the P in the current working thread

Then take some objects from the global object pool to P's pool

for len(pp.deferpool[sc]) < cap(pp.deferpool[sc])/2 && sched.deferpool[sc] != nil {
                    d := sched.deferpool[sc]
                    sched.deferpool[sc] = d.link
                    d.link = nil
                    pp.deferpool[sc] = append(pp.deferpool[sc], d)
                }

Click to see the data structure of the pool. In fact, the members in the pool are what we mentioned earlier_ defer pointer

Where sched Deferpool [SC] is a global pool, and pp.deferpool[sc] is a local pool

mallocgc allocate space

In the above operation, if d does not get the value, it will directly use mallocgc to reallocate and set the corresponding members siz and heap

if d == nil {
        // Allocate new defer+args.
        systemstack(func() {
            total := roundupsize(totaldefersize(uintptr(siz)))
            d = (*_defer)(mallocgc(total, deferType, true))
        })
    }
d.siz = siz
d.heap = true

mallocgc is implemented in Src / Runtime / malloc Go, if you are interested, you can take a deep look at this one. Today we won't focus on this function

// Allocate an object of size bytes.
// Small objects are allocated from the per-P cache's free lists.
// Large objects (> 32 kB) are allocated straight from the heap.
func mallocgc(size uintptr, typ *_type, needzero bool) unsafe.Pointer {}

Finally, let's look at return0

Finally, let's take a look at the result in the deferproc function, which returns return0()

// return0 is a stub used to return 0 from deferproc.
// It is called at the very end of deferproc to signal
// the calling Go function that it should not jump
// to deferreturn.
// in asm_*.s
func return0()

return0 is the stub used to return 0 from deferproc

It is called at the end of the deferproc function to inform the function calling Go that it should not jump to deferreturn.

Under normal conditions, return0 returns 0

However, under abnormal circumstances, the return0 function will return 1, and GO will jump to execute deferreturn

Briefly, deferreturn

The function of deferreturn is to return the corresponding buffer to the linked list in defer, or let the GC reclaim and adjust the corresponding space

Rules of defer in GO

After analyzing the implementation principle of def in GO, let's now understand that the application of def in GO needs to abide by three rules. Let's list:

• The function followed by defer is called delay function. The parameters in the function have been determined when the defer statement is declared
• The delay function is executed according to the last in, first out method, which has been mentioned many times in the article. This impression should be very deep. The first defer is executed after the later defer is executed first
• Delaying a function may affect the return value of the entire function

Let's explain. The second point above should be easy to understand. The above figure also shows the execution sequence

First, let's write a little DEMO

The parameters in the delay function are determined when the defer statement is declared

func main() {
   num := 1
   defer fmt.Println(num)

   num++

   return
}

Don't guess. The running result is 1. The partners can copy the code and run it by themselves

Third, a DEMO

Delaying a function may affect the return value of the entire function

func test3() (res int) {
   defer func() {
      res++
   }()

   return 1
}
func main() {

   fmt.Println(test3())

   return
}

In the above code, we named the return value in test3 function in advance. It should have returned 1

But here in return, the execution order is like this

res = 1

res++

Therefore, the result is 2

summary

• Shared what a defer is
• Simple illustration of stack and queue
• defer data structure and implementation principle, specific source code display
• Three rules of defer in GO

Welcome to like, follow and collect

My friends, your support and encouragement are the driving force for me to insist on sharing and improve quality

OK, that's all for this time. Next time, let's play the verification code with GO

Technology is open, and our mentality should be open. Embrace change, grow into the sun and strive to move forward.

I'm Nezha, the Little Devil boy. Welcome to praise and pay attention to the collection. See you next time~