本从以 go-1.16 版本源码为根底,介绍了 defer 关键字的应用规定、实现原理和优化路线,最初介绍了几种将近的应用场景。试图对 go defer 关键字利用到实现原理有一个全面的理解。
defer 概述
Go 提供关键字 defer
解决提早调用问题。在语法上,defer
与一般的函数调用没有什么区别。正如官网文档形容的那样:
A “defer” statement invokes a function whose execution is deferred to the moment the surrounding function returns, either because the surrounding function executed a return statement, reached the end of its function body, or because the corresponding goroutine is panicking.
DeferStmt = "defer" Expression .
The expression must be a function or method call; it cannot be parenthesized. Calls of built-in functions are restricted as for expression statements.
简略了解一下:
defer
提早了函数执行(留神,不是主调函数,而是提早函数)-
被提早的函数被调用的机会:
- 函数
return
- 函数体开端
- 产生
panic
- 函数
-
语法规定:
- 表达式必须是函数或者办法调用
- 不能被括号括起来
- 内置函数的调用受表达式语句的限度
另外,在《effective go》中也有相干形容:
Go’s
defer
statement schedules a function call (the deferred function) to be run immediately before the function executing thedefer
returns. It’s an unusual but effective way to deal with situations such as resources that must be released regardless of which path a function takes to return. The canonical examples are unlocking a mutex or closing a file.
翻译过去大略是:Go 的 defer
语句会在执行 defer
的函数返回之前安顿一个函数调用(提早函数)立刻运行。这是一种不寻常但无效的办法来解决诸如必须开释资源的状况,而不论函数采纳哪条门路返回。典型示例是解锁互斥锁或敞开文件。
这里用很简略的话形容了 defer
的威力和应用场景:高效的开释资源,如锁开释、文件敞开等。
defer
机制到是有点相似 C++
等语言的析构函数。当函数退出或者对象销毁时做一些开头工作。当然 defer
更为灵便。
defer 应用规定
go
官网文档用一段简略的话,清晰明了的介绍了 defer
的特点:
Each time a “defer” statement executes, the function value and parameters to the call are evaluated as usual and saved anew but the actual function is not invoked. Instead, deferred functions are invoked immediately before the surrounding function returns, in the reverse order they were deferred. That is, if the surrounding function returns through an explicit return statement, deferred functions are executed after any result parameters are set by that return statement but before the function returns to its caller. If a deferred function value evaluates to
nil
, execution panics when the function is invoked, not when the “defer” statement is executed.
然而了解起来,可能并没有那么容易,演绎其余次要有如下特点:
- 参数预计算:每次
defer
语句执行时,会先计算出函数值和入参并放弃起来;即,在执行defer
语句时,提早函数
的入参曾经确定,并保留了正本。 -
提早调用机会:
defer
语句没有真正的被调用提早函数
,提早函数
真正被调用是在主调函数 返回前。- 如果主调函数有明确的
return
语句,则提早函数
将在所有返回值被设置(即return
语句被执行)之后,主调函数返回之前被执行; - 如果
提早函数
是nil
,在提早函数
被调用时,而非执行defer
语句时触发panic
。
- 如果主调函数有明确的
- 执行程序:依照
defer
语句的逆序执行。
本大节,将通过具体的实例代码展现上述特点,下一大节,将通过源码剖析 defer
机制这些特点的背地原理与实现细节。
执行程序
defer
语句的执行程序是先进后出LIFO
。
上面的代码展现了 defer
的执行程序。main
函数中顺次通过 defer
调用了 deferA
、deferB
、deferC
三个函数,执行后果的确顺次执行了 deferC
、deferB
、deferA
。在执行程序上,和C++
析构函数极为相似。
示例代码:
// 演示 defer 执行程序
package main
import "fmt"
func deferA() {fmt.Println("deferA")
}
func deferB() {fmt.Println("deferB")
}
func deferC() {fmt.Println("deferC")
}
func main() {defer deferA()
defer deferB()
defer deferC()
fmt.Println("main")
}
上述示例代码执行后果:
$ go run defer_1.go
main
deferC
deferB
deferA
defer 与 return 程序
示例代码:
// 验证 defer 与 return 执行程序
package main
import "fmt"
func deferFunc() int {fmt.Println("defer func is called")
return 0
}
func returnFunc() int {fmt.Println("return func is called")
return 0
}
func returnAndDefer() int {defer deferFunc()
return returnFunc()}
func main() {returnAndDefer()
}
上述示例代码执行后果:
$ go run defer_4.go
return func is called
defer func is called
下面的示例代码证实了 defer
机制真正执行 提早函数
(示例中为deferFunc()
函数),是在 return
语句执行 returnFunc()
之后执行的。大抵流程程序如 图 1
图 1 defer 与 return 执行程序
预计算
在上面这个例子中,变量 a
在 defer
被调用的时候就曾经确定了,而非在 defer
执行时。下述代码所以上面代码输入的是 1。
示例代码:
// 演示 defer 预计算
package main
import "fmt"
func deferD(d int) {fmt.Println(d)
}
func main() {
a := 1
defer deferD(a)
a = a + 1
}
上述代示例码执行后果:
$ go run defer_2.go
1
这里还有一个典型的示例,在 提早函数
的实参中调用函数。
// 验证 defer 预计算示例 2
package main
import "fmt"
func f(index int, value int) int {fmt.Printf("index=%d,value=%d\n", index, value)
return index
}
func main() {defer f(1, f(3, 1))
defer f(2, f(4, 2))
}
上述示例代码执行后果:
$ go run defer_2_2.go
index=3,value=1
index=4,value=2
index=2,value=4
index=1,value=3
这个示例从另一个角度阐明了 defer
机制的预计算个性。
defer
压栈f(1,f(3,1))
,压栈函数地址、形参 1、形参 2(调用f(3,1)
) –> 打印index=3,value=1
defer
压栈f(2,f(4,2))
,压栈函数地址、形参 1、形参 2(调用f(4,2)
) –> 打印index=4,value=2
defer
出栈f(2,f(4,2))
,–> 打印index=2,value=4
defer
压栈f(1,f(3,1))
,–> 打印index=1,value=3
批改命名返回值
示例代码:
// 演示 defer 批改有命名返回值函数的返回值
package main
import "fmt"
func func1(a int) (e int) {defer func() {e = a + 1}()
return a
}
func main() {e := func1(1)
fmt.Println(e)
}
上述示例代码执行后果:
$ go run defer_3.go
2
通过上述代码,能够察看到,func1
的返回值 e
并不是 a
的值。依据前文介绍 defer
特点的 2.1
能够晓得 defer
理论调用 提早函数
是在计算 a
之后返回给主调函数 main
函数之前执行的。因而,命名返回值 e
是在 return a
执行之后,真正返回给主调函数 main
之前批改为 a+1
的。
此处,须要延长一个知识点:go
语言中函数返回值的初始化话机会。这里须要从两个点思考:
- 命名返回值,如果指定了一个返回值的名字,则会在函数起始处被初始化为对应类型的零值并且作用域为整个函数。能够视为在该函数的第一行中定义了该名字的变量。
- 匿名返回值,如果没有指定返回值的名字,则是在返回时创立一个长期变量来接管返回值。
示例代码:
package main
import "fmt"
func funcE() (t int) {fmt.Printf("t = %v\n", t)
return 2
}
func main() {funcE()
}
上述示例代码执行后果:
$ go run defer_5.go
t = 0
defer 与 panic
panic
机制也会导致函数提前结束执行,将后续流程交给 defer
语句(如果有的话)。这里 defer
机制可能 recover
这个 panic
也可能不做解决。还有一种状况就是 提早函数
也可能panic
。接下来将通过示例代码介绍一下这三种状况。
defer 函数不捕捉异样
示例代码:
// 演示 defer 不捕捉异样
package main
import ("fmt")
func deferCall() {defer func() {fmt.Println("defer 1: before panic print") }()
defer func() { fmt.Println("defer 2: before panic print") }()
panic("panic error") // trigger defer out stack
defer func() { fmt.Println("defer 3: after panic, never exec") }()}
func main() {deferCall()
fmt.Println("main exec ok")
}
上述示例代码执行后果:
$ go run defer_panic_1.go
defer 2: before panic print
defer 1: before panic print
panic: panic error
goroutine 1 [running]:
main.deferCall()
/home/work/workspace/defer/defer_panic_1.go:11 +0x68
main.main()
/home/work/workspace/defer/defer_panic_1.go:16 +0x25
exit status 2
上述代码的执行后果证实,panic
语句之前的 defer
语句会依照 先进后出 的程序顺次执行;而 panic
语句后的不会只执行。
defer 函数捕捉异样
示例代码:
// 演示 defer 捕捉异样
package main
import ("fmt")
func deferCall() {defer func() {fmt.Println("defer 1: before panic print") }()
defer func() {fmt.Println("defer 2: before panic print,recover")
if err := recover(); err != nil {fmt.Println(err)
}
}()
defer func() { fmt.Println("defer 3: before panic print") }()
panic("panic error") // trigger defer out stack
defer func() { fmt.Println("defer 4: after panic, never exec") }()}
func main() {deferCall()
fmt.Println("main exec ok")
}
上述示例代码执行后果:
$ go run defer_panic_2.go
defer 3: before panic print
defer 2: before panic print,recover
panic error
defer 1: before panic print
main exec ok
上述示例代码表明:和不捕捉异样的状况一样,panic
语句之前的 defer
都会依照先进后出顺次执行;不同的是在第二个 defer
语句中 提早函数
捕捉了异样,并得当解决,因而 deferCall
函数中 panic
语句前的 defer
依旧会执行,并且 deferCall
函数平安退出,main
函数也失常执行。
在 提早函数
中通过 recover()
捕捉 panic()
抛出的异样,是比拟常见的异样解决形式。
defer 函数中蕴含 panic
示例代码:
// 演示 defer 抛出异样
package main
import ("fmt")
func main() {defer func() {if err := recover(); err != nil {fmt.Println("defer1 recover:", err)
} else {fmt.Println("fatal")
}
}()
defer func() {if err := recover(); err != nil {fmt.Println(err)
}
panic("defer panic")
}()
panic("main panic")
}
上述示例代码执行后果:
$ go run defer_panic_3.go
main panic
defer1 recover: defer panic
上述示例中,main
函数通过 panic
抛出了异样 main panic
该异样被第二个 defer
语句捕捉并解决,同时第二个 defer
本人也通过 panic
抛出了异样,该异样被第一个 defer
语句捕捉并处。
图 2 panic 能够沿着 defer 执行门路上抛或者被 recover
defer 实现
本节将通过源码来深刻理解 defer
的设计与实现,从原理和实现层面探讨 defer
机制。本节波及 go
源代码均是 go 1.6
版本。
defer 执行机制
数据结构
defer
数据结构 runtime._defer
type _defer struct {
siz int32 // includes both arguments and results
started bool
heap bool
// openDefer indicates that this _defer is for a frame with open-coded
// defers. We have only one defer record for the entire frame (which may
// currently have 0, 1, or more defers active).
openDefer bool
sp uintptr // sp at time of defer
pc uintptr // pc at time of defer
fn *funcval // can be nil for open-coded defers
_panic *_panic // panic that is running defer
link *_defer
// If openDefer is true, the fields below record values about the stack
// frame and associated function that has the open-coded defer(s). sp
// above will be the sp for the frame, and pc will be address of the
// deferreturn call in the function.
fd unsafe.Pointer // funcdata for the function associated with the frame
varp uintptr // value of varp for the stack frame
// framepc is the current pc associated with the stack frame. Together,
// with sp above (which is the sp associated with the stack frame),
// framepc/sp can be used as pc/sp pair to continue a stack trace via
// gentraceback().
framepc uintptr
}
_defer
构造中字段含意:
siz
参数和返回值共占多少字节,会间接调配在_defer 前面,在注册时保留参数,在执行实现时拷贝到调用者参数和返回值空间started
标记是否曾经执行heap
go1.13 优化,标识是否为堆调配openDefer
示意以后defer
是否通过凋谢编码的优化sp
记录调用者栈指针,能够通过它判断本人注册的 defer 是否曾经执行完了pc
deferproc 的返回地址fn
是defer
关键字中传入的函数,即提早函数,如果开启了凋谢编码优化,可能为空;_panic
是触发提早调用的构造体,可能为空;_panic
指向以后的panic
,示意这个 defer 是由这个 panic 触发的link
链表串联字段, 链到前一个注册的 defer 构造体
其余字段为 open-coded
配套字段,通过这些信息能够找到未注册到链表的 defer 函数
图 3 defer 构造
执行机制
两头代码生成阶段的 gc.state.stmt
会负责处理程序中的 defer
,该函数会依据条件的不同,应用三种不同的机制解决该关键字:
// stmt converts the statement n to SSA and adds it to s.
func (s *state) stmt(n *Node) {
// ...
case ODEFER:
// ...
if s.hasOpenDefers {s.openDeferRecord(n.Left) // 凋谢编码
} else {
d := callDefer // 默认是堆上实现
if n.Esc == EscNever {d = callDeferStack // 栈上实现}
s.callResult(n.Left, d)
}
// ...
}
留神:这里是 go 1.16 版本,在 go 1.17 之后,这里重构了,然而逻辑根本保持一致,能够参考 ssagen.state.stmt
堆调配、栈调配和凋谢编码是解决 defer
关键字的三种办法,晚期的 Go 语言会在堆上调配 runtime._defer
构造体,不过该实现的性能较差,Go 语言在 1.13 中引入栈上调配的构造体,缩小了 30% 的额定开销,并在 1.14 中引入了基于凋谢编码的 defer
,使得该关键字的额定开销能够忽略不计
依据 gc.state.stmt
能够看出:
- 如果开启凋谢编码(且符合条件
s.hasOpenDefers==true
)则调用openDeferRecord
依照凋谢编码实现形式解决; - 如果满足
n.Esc == EscNever
则将callKind
设置为callDeferStack
而后调用callResult
依照栈上实现来解决; - 否则走默认逻辑,则将
callKind
设置为callDefer
,而后调用callResult
依照堆上实现来解决。
接下来会别离介绍三种不同类型 defer
的设计与实现原理。
堆上实现
堆上实现是 golang 最早的 defer
实现形式。go1.12 引入。
当该计划被启用时,编译器会调用 gc.state.callResult
,该函数会调用gc.state.call
,因而 defer
在编译器看来也是函数调用。
gc.state.call
会负责为所有函数和办法调用生成中间代码,它的工作包含以下内容:
- 获取须要执行的函数名、闭包指针、代码指针和函数调用的接管方;
- 获取栈地址并将函数或者办法的参数写入栈中;
- 调用用
gc.state.newValue1A
以及相干函数生成函数调用的中间代码; - 如果以后调用的函数是
defer
,那么会独自生成相干的完结代码块; - 获取函数的返回值地址并完结以后调用;
// Calls the function n using the specified call type.
// Returns the address of the return value (or nil if none).
func (s *state) call(n *Node, k callKind, returnResultAddr bool) *ssa.Value {
// ...
if k == callDeferStack { // 栈上实现逻辑分支,前面会介绍
// ....
} else {
// ...
// call target
switch {
case k == callDefer:
aux := ssa.StaticAuxCall(deferproc, ACArgs, ACResults) // deferproc defer 创立函数
if testLateExpansion {call = s.newValue0A(ssa.OpStaticLECall, aux.LateExpansionResultType(), aux)
call.AddArgs(callArgs...)
} else {call = s.newValue1A(ssa.OpStaticCall, types.TypeMem, aux, s.mem())
}
// ....
call.AuxInt = stksize // Call operations carry the argsize of the callee along with them
}
// ...
}
从上述代码中咱们能看到实现与 go 其余关键字的实现相似,调用的是gc.state.call
。
核心思想:
- 在 defer 呈现的中央插入了指令 CALL
runtime.deferproc
; - 在函数返回的中央插入了 CALL
runtime.deferreturn
; - goroutine 的控制结构中,有一张表记录 defer,调用
runtime.deferproc
时会将须要 defer 的表达式记录在表中,而在调用runtime.deferreturn
的时候,则会顺次从 defer 表中“出栈”并执行; - 如果有多个 defer,调用程序相似栈,越前面的 defer 表达式越先被调用。
编译器通过以下三个步骤为所有调用 defer
的函数开端插入 runtime.deferreturn
的函数调用:
gc.walkstmt
在遇到ODEFER
节点时会执行Curfn.Func.SetHasDefer(true)
设置以后函数的hasdefer
属性;gc.buildssa
会执行s.hasdefer = fn.Func.HasDefer()
更新state
的hasdefer
;cgc.state.exit
会依据state
的hasdefer
在函数返回之前插入runtime.deferreturn
的函数调用;
// exit processes any code that needs to be generated just before returning.
// It returns a BlockRet block that ends the control flow. Its control value
// will be set to the final memory state.
func (s *state) exit() *ssa.Block {
if s.hasdefer {if s.hasOpenDefers { // 凋谢编码实现解决逻辑,后续会介绍} else {s.rtcall(Deferreturn, true, nil)
}
}
}
下面介绍了在编译阶段 defer
的相干逻辑——如果进行代码革新。那么在运行时又是怎么运行的呢?
能够留神到 runtime.deferproc
、runtime.deferreturn
是运行时包的函数,这两个运行时函数是 defer
关键字运行时机制的入口:
runtime.deferproc
负责创立新的提早调用;runtime.deferreturn
负责在函数调用完结时执行所有的提早调用;
创立提早调用
runtime.deferproc
次要工作:
- 调用
runtime.newdefer
创立一个新的runtime._defer
对象; - 将新创建的
runtime._defer
对象插入到runtime.g
对象的_defer
链表上; - 设置它的函数指针
fn
、程序计数器pc
和栈指针sp
并将相干的参数拷贝到相邻的内存空间中。 - 最初调用的
runtime.return0
返回,runtime.return0
是惟一一个不会触发提早调用的函数,它能够防止递归runtime.deferreturn
的递归调用。
// Create a new deferred function fn with siz bytes of arguments.
// The compiler turns a defer statement into a call to this.
//go:nosplit
func deferproc(siz int32, fn *funcval) { // arguments of fn follow fn
// 获取以后 goroutine
gp := getg()
if gp.m.curg != gp {
// go code on the system stack can't defer
throw("defer on system stack")
}
// the arguments of fn are in a perilous state. The stack map
// for deferproc does not describe them. So we can't let garbage
// collection or stack copying trigger until we've copied them out
// to somewhere safe. The memmove below does that.
// Until the copy completes, we can only call nosplit routines.
// 获取调用者指针
sp := getcallersp()
// 通过偏移取得参数
argp := uintptr(unsafe.Pointer(&fn)) + unsafe.Sizeof(fn)
callerpc := getcallerpc()
d := newdefer(siz) // 创立了一个新的_defer 对象
if d._panic != nil {throw("deferproc: d.panic != nil after newdefer")
}
// 留神这里,能够看出,_defer 链表是头插的,这是为什么 defer 是逆序执行的起因
d.link = gp._defer
gp._defer = d
d.fn = fn
d.pc = callerpc
d.sp = sp
switch siz {
case 0:
// Do nothing.
case sys.PtrSize:
*(*uintptr)(deferArgs(d)) = *(*uintptr)(unsafe.Pointer(argp))
default:
memmove(deferArgs(d), unsafe.Pointer(argp), uintptr(siz))
}
// deferproc returns 0 normally.
// a deferred func that stops a panic
// makes the deferproc return 1.
// the code the compiler generates always
// checks the return value and jumps to the
// end of the function if deferproc returns != 0.
return0()
// No code can go here - the C return register has
// been set and must not be clobbered.
}
runtime.newdefer
顺从从三个中央构建 runtime._defer
:
- 尝试从调度器的提早调用缓存池
sched.deferpool
中回收一批_defer
对象并将该对象追加到以后goroutine
的缓存池中; - 而后在从以后
goroutine
的提早调用缓存池pp.deferpool
中取出一个闲暇的_defer
对象; - 如果从
pp.deferpool
没有取到可用的_defer
对象,则通过runtime.mallocgc
在堆上创立一个新的_defer
对象。
// Allocate a Defer, usually using per-P pool.
// Each defer must be released with freedefer. The defer is not
// added to any defer chain yet.
//
// This must not grow the stack because there may be a frame without
// stack map information when this is called.
//
//go:nosplit
func newdefer(siz int32) *_defer {
var d *_defer
sc := deferclass(uintptr(siz))
gp := getg()
if sc < uintptr(len(p{}.deferpool)) {// 从 deferpool 中
pp := gp.m.p.ptr()
if len(pp.deferpool[sc]) == 0 && sched.deferpool[sc] != nil {
// Take the slow path on the system stack so
// we don't grow newdefer's stack.
systemstack(func() {lock(&sched.deferlock)
// 先去 sched.deferpool 回收一批_defer 对象,转移到 pp.deferpool 中
for len(pp.deferpool[sc]) < cap(pp.deferpool[sc])/2 && sched.deferpool[sc] != nil {d := sched.deferpool[sc]
sched.deferpool[sc] = d.link
d.link = nil
pp.deferpool[sc] = append(pp.deferpool[sc], d)
}
unlock(&sched.deferlock)
})
}
// 尝试从 pp.deferpool 中取个闲暇的_defer 对象
if n := len(pp.deferpool[sc]); n > 0 {d = pp.deferpool[sc][n-1]
pp.deferpool[sc][n-1] = nil
pp.deferpool[sc] = pp.deferpool[sc][:n-1]
}
}
// 切实取不到,则生成一个
if d == nil {
// Allocate new defer+args.
systemstack(func() {total := roundupsize(totaldefersize(uintptr(siz)))
d = (*_defer)(mallocgc(total, deferType, true))
})
}
d.siz = siz
d.heap = true
return d
}
留神:将新创建的 runtime._defer
对象插入到 runtime.g
对象的 _defer
链表上,应用的是头插法,因而,defer
的执行程序是逆序的。
执行提早调用
runtime.deferreturn
会从以后 Goroutine 的 _defer
链表中取出最后面的 runtime._defer
并调用 runtime.jmpdefer
传入须要执行的函数和参数:
// Run a deferred function if there is one.
// The compiler inserts a call to this at the end of any
// function which calls defer.
// If there is a deferred function, this will call runtime·jmpdefer,
// which will jump to the deferred function such that it appears
// to have been called by the caller of deferreturn at the point
// just before deferreturn was called. The effect is that deferreturn
// is called again and again until there are no more deferred functions.
//
// Declared as nosplit, because the function should not be preempted once we start
// modifying the caller's frame in order to reuse the frame to call the deferred
// function.
//
// The single argument isn't actually used - it just has its address
// taken so it can be matched against pending defers.
//go:nosplit
func deferreturn(arg0 uintptr) {gp := getg()
d := gp._defer // 取出第一个_defer 对象
if d == nil {return}
// ...
// 凋谢编码实现解决逻辑
if d.openDefer { }
fn := d.fn
d.fn = nil
gp._defer = d.link // 从 g._defer 链表中删除以后_defer 对象
freedefer(d) // 开释_defer 对象
// If the defer function pointer is nil, force the seg fault to happen
// here rather than in jmpdefer. gentraceback() throws an error if it is
// called with a callback on an LR architecture and jmpdefer is on the
// stack, because the stack trace can be incorrect in that case - see
// issue #8153).
_ = fn.fn
jmpdefer(fn, uintptr(unsafe.Pointer(&arg0))) // 调用 jmpdefer
}
runtime.jmpdefer
是一个用汇编语言实现的运行时函数。它的次要工作是:
- 跳转到
defer
所在的代码段 - 并在执行完结之后跳转回
runtime.deferreturn
。
runtime.deferreturn
会屡次判断以后 goroutine
的 _defer
链表中是否有未执行的 _defer
对象,该函数只有在所有提早函数都执行后才会返回。
栈上实现
堆上实现的 defer
存在如下问题:
defer
信息次要存储在堆上,要在堆和栈上来回拷贝返回值和参数很慢;defer
构造体通过链表链起来,而链表的操作也很慢。
在 go1.13 中对 defer
的实现进行了优化:
-
缩小了
defer
信息的堆调配。再通过runtime.deferprocStack
将整个defer
注册到defer
链表中:- 将个别状况的
defer
信息存储在函数栈帧的局部变量区域; - 显示循环或者是隐式循环的
defer
还是须要用到 go1.12 中defer
信息的堆调配。
- 将个别状况的
func (s *state) call(n *Node, k callKind, returnResultAddr bool) *ssa.Value {
if k == callDeferStack {testLateExpansion = ssa.LateCallExpansionEnabledWithin(s.f)
// Make a defer struct d on the stack. 在栈上创立_defer 构造
t := deferstruct(stksize)
d := tempAt(n.Pos, s.curfn, t)
s.vars[&memVar] = s.newValue1A(ssa.OpVarDef, types.TypeMem, d, s.mem())
addr := s.addr(d)
// Must match reflect.go:deferstruct and src/runtime/runtime2.go:_defer.
// 0: siz
s.store(types.Types[TUINT32],
s.newValue1I(ssa.OpOffPtr, types.Types[TUINT32].PtrTo(), t.FieldOff(0), addr),
s.constInt32(types.Types[TUINT32], int32(stksize)))
// 1: started, set in deferprocStack
// 2: heap, set in deferprocStack
// 3: openDefer
// 4: sp, set in deferprocStack
// 5: pc, set in deferprocStack
// 6: fn
s.store(closure.Type,
s.newValue1I(ssa.OpOffPtr, closure.Type.PtrTo(), t.FieldOff(6), addr),
closure)
// 7: panic, set in deferprocStack
// 8: link, set in deferprocStack
// 9: framepc
// 10: varp
// 11: fd
// Then, store all the arguments of the defer call.
ft := fn.Type
off := t.FieldOff(12)
args := n.Rlist.Slice()
// Set receiver (for interface calls). Always a pointer.
if rcvr != nil {p := s.newValue1I(ssa.OpOffPtr, ft.Recv().Type.PtrTo(), off, addr)
s.store(types.Types[TUINTPTR], p, rcvr)
}
// Set receiver (for method calls).
if n.Op == OCALLMETH {f := ft.Recv()
s.storeArgWithBase(args[0], f.Type, addr, off+f.Offset)
args = args[1:]
}
// Set other args.
for _, f := range ft.Params().Fields().Slice() {s.storeArgWithBase(args[0], f.Type, addr, off+f.Offset)
args = args[1:]
}
// Call runtime.deferprocStack with pointer to _defer record.
ACArgs = append(ACArgs, ssa.Param{Type: types.Types[TUINTPTR], Offset: int32(Ctxt.FixedFrameSize())})
aux := ssa.StaticAuxCall(deferprocStack, ACArgs, ACResults) // 调用 deferprocStack
// ...
} else {// 堆上实现}
// 栈上和堆上实现的独特逻辑
}
因为在编译期间咱们曾经创立了 runtime._defer
对象,所以在运行期间 runtime.deferprocStack
只须要设置一些未在编译期间初始化的字段,就能够将栈上的 runtime._defer
追加到函数的链表上。
// deferprocStack queues a new deferred function with a defer record on the stack.
// The defer record must have its siz and fn fields initialized.
// All other fields can contain junk.
// The defer record must be immediately followed in memory by
// the arguments of the defer.
// Nosplit because the arguments on the stack won't be scanned
// until the defer record is spliced into the gp._defer list.
//go:nosplit
func deferprocStack(d *_defer) { // 留神这里入参曾经是_defer 了,因而 deferprocStack 只是做一些简略的初始化,而后将初始化好的_defer 对象插入以后 goroutine 的_defer 链表中
gp := getg()
if gp.m.curg != gp {
// go code on the system stack can't defer
throw("defer on system stack")
}
// siz and fn are already set.
// The other fields are junk on entry to deferprocStack and
// are initialized here.
d.started = false
d.heap = false
d.openDefer = false
d.sp = getcallersp()
d.pc = getcallerpc()
d.framepc = 0
d.varp = 0
*(*uintptr)(unsafe.Pointer(&d._panic)) = 0
*(*uintptr)(unsafe.Pointer(&d.fd)) = 0
// 将初始化好的_defer 对象插入以后 goroutine 的_defer 链表中
*(*uintptr)(unsafe.Pointer(&d.link)) = uintptr(unsafe.Pointer(gp._defer))
*(*uintptr)(unsafe.Pointer(&gp._defer)) = uintptr(unsafe.Pointer(d))
return0()}
栈上调配和堆上调配的 runtime._defer
并没有实质的不同,只是调配地位的不同,余下逻辑共用。因而该办法能够实用于绝大多数的场景。
凋谢编码实现
go1.14 中进一步对 defer
的进行了优化:
- 在编译阶段插入代码,把
defer
函数的执行逻辑开展在所属函数内,防止创立_defer
对象,而且不须要注册到_defer
链表。称为open coded defer
。 -
与 1.13 一样不适用于循环中的
defer
- 性能简直晋升了一个数量级。
open coded defer
中产生panic
或 调用runtime.Goexit()
,前面未注册到的defer
函数无奈执行到,须要栈扫描。defer
构造体中就多增加了一些字段,借助这些字段能够找到未注册到链表中的defer
函数。- 后果就是 defer 变快了,然而 panic 变慢了。
开启凋谢编码
Go1.14 对 defer
的优化,其实就是内联。因而,它有很多内联函数相似的限度条件:
- 函数的
defer
数量少于或者等于 8 个; - 函数的
defer
关键字不能在循环中执行; - 函数的
return
语句与defer
语句的乘积小于或者等于 15 个。
如 gc.walkstmt 函数所示,defer
关键字的数量多于 8 个或者 defer
关键字处于 for
循环中,那么咱们在这里都会禁用凋谢编码优化。
// The max number of defers in a function using open-coded defers. We enforce this
// limit because the deferBits bitmask is currently a single byte (to minimize code size)
const maxOpenDefers = 8
// The result of walkstmt MUST be assigned back to n, e.g.
// n.Left = walkstmt(n.Left)
func walkstmt(n *Node) *Node {
// ...
switch n.Op {
// ...
case ODEFER:
Curfn.Func.SetHasDefer(true)
Curfn.Func.numDefers++
if Curfn.Func.numDefers > maxOpenDefers { // maxOpenDefers == 8 defer 个数大于 8 个,不行
// Don't allow open-coded defers if there are more than
// 8 defers in the function, since we use a single
// byte to record active defers.
Curfn.Func.SetOpenCodedDeferDisallowed(true)
}
if n.Esc != EscNever { // defer 在循环中也不行
// If n.Esc is not EscNever, then this defer occurs in a loop,
// so open-coded defers cannot be used in this function.
Curfn.Func.SetOpenCodedDeferDisallowed(true)
}
fallthrough
// ...
return n
}
在 SSA 两头代码生成阶段,如 gc.buildssa
函数所示,启用凋谢编码优化的其余条件,也就是返回语句的数量与 defer
数量的乘积须要小于 15。
/ buildssa builds an SSA function for fn.
// worker indicates which of the backend workers is doing the processing.
func buildssa(fn *Node, worker int) *ssa.Func {
// ...
s.hasOpenDefers = Debug.N == 0 && s.hasdefer && !s.curfn.Func.OpenCodedDeferDisallowed()
switch {case s.hasOpenDefers && (Ctxt.Flag_shared || Ctxt.Flag_dynlink) && thearch.LinkArch.Name == "386":
// Don't support open-coded defers for 386 ONLY when using shared
// libraries, because there is extra code (added by rewriteToUseGot())
// preceding the deferreturn/ret code that is generated by gencallret()
// that we don't track correctly.
s.hasOpenDefers = false
}
if s.hasOpenDefers && s.curfn.Func.Exit.Len() > 0 {
// Skip doing open defers if there is any extra exit code (likely
// copying heap-allocated return values or race detection), since
// we will not generate that code in the case of the extra
// deferreturn/ret segment.
s.hasOpenDefers = false
}
if s.hasOpenDefers &&
s.curfn.Func.numReturns*s.curfn.Func.numDefers > 15 { // 返回语句的数量与 defer 数量的乘积须要小于 15
// Since we are generating defer calls at every exit for
// open-coded defers, skip doing open-coded defers if there are
// too many returns (especially if there are multiple defers).
// Open-coded defers are most important for improving performance
// for smaller functions (which don't have many returns).
s.hasOpenDefers = false
}
// ...
}
设置
通过上述一些列的条件判断,如果最终s.hasOpenDefers == true
即开启凋谢编码实现。接下来将会做如下工作:
设置 deferBits
,deferBits
是一个 bitmask
激励了哪个 defer
须要被执行,相似在堆和栈上实现的 _defer
链表
func buildssa(fn *Node, worker int) *ssa.Func {
// ...
if s.hasOpenDefers {
// Create the deferBits variable and stack slot. deferBits is a
// bitmask showing which of the open-coded defers in this function
// have been activated.
deferBitsTemp := tempAt(src.NoXPos, s.curfn, types.Types[TUINT8])
s.deferBitsTemp = deferBitsTemp
// For this value, AuxInt is initialized to zero by default
startDeferBits := s.entryNewValue0(ssa.OpConst8, types.Types[TUINT8])
s.vars[&deferBitsVar] = startDeferBits
s.deferBitsAddr = s.addr(deferBitsTemp)
s.store(types.Types[TUINT8], s.deferBitsAddr, startDeferBits)
// Make sure that the deferBits stack slot is kept alive (for use
// by panics) and stores to deferBits are not eliminated, even if
// all checking code on deferBits in the function exit can be
// eliminated, because the defer statements were all
// unconditional.
s.vars[&memVar] = s.newValue1Apos(ssa.OpVarLive, types.TypeMem, deferBitsTemp, s.mem(), false)
}
// ...
}
deferBits
中的每一个比特位都示意该位对应的 defer
关键字是否须要被执行,如下图所示,其中 8 个比特的第二、第三个比特在函数返回前被设置成了 1,那么该比特位对应的函数会在函数返回前执行。
图 4 Golang deferBits 示意
两头代码生成阶段的 gc.state.stmt
函数调用 gc.state.openDeferRecord
结构 gc.openDeferInfo
对象,该构造体的 closure
中存储着调用的函数,rcvr
中存储着办法的接收者,而最初的 argVals
中存储了函数的参数。
// Information about each open-coded defer.
type openDeferInfo struct {
// The ODEFER node representing the function call of the defer
n *Node
// If defer call is closure call, the address of the argtmp where the
// closure is stored.
closure *ssa.Value
// The node representing the argtmp where the closure is stored - used for
// function, method, or interface call, to store a closure that panic
// processing can use for this defer.
closureNode *Node
// If defer call is interface call, the address of the argtmp where the
// receiver is stored
rcvr *ssa.Value
// The node representing the argtmp where the receiver is stored
rcvrNode *Node
// The addresses of the argtmps where the evaluated arguments of the defer
// function call are stored.
argVals []*ssa.Value
// The nodes representing the argtmps where the args of the defer are stored
argNodes []*Node}
结构 gc.openDeferInfo
对象。
// openDeferRecord adds code to evaluate and store the args for an open-code defer
// call, and records info about the defer, so we can generate proper code on the
// exit paths. n is the sub-node of the defer node that is the actual function
// call. We will also record funcdata information on where the args are stored
// (as well as the deferBits variable), and this will enable us to run the proper
// defer calls during panics.
func (s *state) openDeferRecord(n *Node) {
// ...
opendefer := &openDeferInfo{n: n,}
fn := n.Left
if n.Op == OCALLFUNC {
// We must always store the function value in a stack slot for the
// runtime panic code to use. But in the defer exit code, we will
// call the function directly if it is a static function.
closureVal := s.expr(fn)
closure := s.openDeferSave(nil, fn.Type, closureVal)
opendefer.closureNode = closure.Aux.(*Node)
if !(fn.Op == ONAME && fn.Class() == PFUNC) {opendefer.closure = closure}
} else if n.Op == OCALLMETH {
if fn.Op != ODOTMETH {Fatalf("OCALLMETH: n.Left not an ODOTMETH: %v", fn)
}
closureVal := s.getMethodClosure(fn)
// We must always store the function value in a stack slot for the
// runtime panic code to use. But in the defer exit code, we will
// call the method directly.
closure := s.openDeferSave(nil, fn.Type, closureVal)
opendefer.closureNode = closure.Aux.(*Node)
} else {
if fn.Op != ODOTINTER {Fatalf("OCALLINTER: n.Left not an ODOTINTER: %v", fn.Op)
}
closure, rcvr := s.getClosureAndRcvr(fn)
opendefer.closure = s.openDeferSave(nil, closure.Type, closure)
// Important to get the receiver type correct, so it is recognized
// as a pointer for GC purposes.
opendefer.rcvr = s.openDeferSave(nil, fn.Type.Recv().Type, rcvr)
opendefer.closureNode = opendefer.closure.Aux.(*Node)
opendefer.rcvrNode = opendefer.rcvr.Aux.(*Node)
}
for _, argn := range n.Rlist.Slice() {
var v *ssa.Value
if canSSAType(argn.Type) {v = s.openDeferSave(nil, argn.Type, s.expr(argn))
} else {v = s.openDeferSave(argn, argn.Type, nil)
}
args = append(args, v)
argNodes = append(argNodes, v.Aux.(*Node))
}
opendefer.argVals = args
opendefer.argNodes = argNodes
index := len(s.openDefers)
s.openDefers = append(s.openDefers, opendefer)
// Update deferBits only after evaluation and storage to stack of
// args/receiver/interface is successful.
bitvalue := s.constInt8(types.Types[TUINT8], 1<<uint(index))
newDeferBits := s.newValue2(ssa.OpOr8, types.Types[TUINT8], s.variable(&deferBitsVar, types.Types[TUINT8]), bitvalue)
s.vars[&deferBitsVar] = newDeferBits
s.store(types.Types[TUINT8], s.deferBitsAddr, newDeferBits)
}
很多 defer
语句能够在编译期间判断是否被执行,如果函数中的 defer
语句能够在编译期间确定,两头代码生成阶段就会间接通过 gc.state.exit
调用 gc.state.openDeferExit
在函数返回前生成判断 deferBits
的代码。
// exit processes any code that needs to be generated just before returning.
// It returns a BlockRet block that ends the control flow. Its control value
// will be set to the final memory state.
func (s *state) exit() *ssa.Block {
if s.hasdefer {
if s.hasOpenDefers {if shareDeferExits && s.lastDeferExit != nil && len(s.openDefers) == s.lastDeferCount {
if s.curBlock.Kind != ssa.BlockPlain {panic("Block for an exit should be BlockPlain")
}
s.curBlock.AddEdgeTo(s.lastDeferExit)
s.endBlock()
return s.lastDeferFinalBlock
}
s.openDeferExit()} else {s.rtcall(Deferreturn, true, nil)
}
}
// ...
}
执行
当程序遇到运行时能力判断的条件语句时,咱们依然须要由运行时的 runtime.deferreturn
决定是否执行 defer
关键字:
func deferreturn(arg0 uintptr) {gp := getg()
d := gp._defer
if d.openDefer {done := runOpenDeferFrame(gp, d)
if !done {throw("unfinished open-coded defers in deferreturn")
}
gp._defer = d.link
freedefer(d)
return
}
}
该函数为凋谢编码做了非凡的优化,运行时会调用 runtime.runOpenDeferFrame
执行沉闷的凋谢编码提早函数,该函数会执行以下的工作:
- 从
runtime._defer
构造体中读取deferBits
、函数defer
数量等信息; - 在循环中顺次读取函数的地址和参数信息并通过
deferBits
判断该函数是否须要被执行; - 调用
runtime.reflectcallSave
调用须要执行的defer
函数。
// runOpenDeferFrame runs the active open-coded defers in the frame specified by
// d. It normally processes all active defers in the frame, but stops immediately
// if a defer does a successful recover. It returns true if there are no
// remaining defers to run in the frame.
func runOpenDeferFrame(gp *g, d *_defer) bool {
done := true
fd := d.fd
// Skip the maxargsize
_, fd = readvarintUnsafe(fd)
deferBitsOffset, fd := readvarintUnsafe(fd)
nDefers, fd := readvarintUnsafe(fd)
deferBits := *(*uint8)(unsafe.Pointer(d.varp - uintptr(deferBitsOffset))) // 拿到 deferBits
for i := int(nDefers) - 1; i >= 0; i-- { // 遍历 deferBits
// read the funcdata info for this defer
if deferBits&(1<<i) == 0 {
// 遍历,跳过不须要执行的 defer
continue
}
closure := *(**funcval)(unsafe.Pointer(d.varp - uintptr(closureOffset)))
d.fn = closure
// ...
deferBits = deferBits &^ (1 << i)
*(*uint8)(unsafe.Pointer(d.varp - uintptr(deferBitsOffset))) = deferBits
p := d._panic
reflectcallSave(p, unsafe.Pointer(closure), deferArgs, argWidth) // 解决须要被执行提早函数
if p != nil && p.aborted {break}
d.fn = nil
// These args are just a copy, so can be cleared immediately
memclrNoHeapPointers(deferArgs, uintptr(argWidth))
if d._panic != nil && d._panic.recovered {
done = deferBits == 0
break
}
}
return done
}
runtime.reflectcallSave
最终通过调用 runtime.reflectcall
来执行提早函数。
// reflectcallSave calls reflectcall after saving the caller's pc and sp in the
// panic record. This allows the runtime to return to the Goexit defer processing
// loop, in the unusual case where the Goexit may be bypassed by a successful
// recover.
func reflectcallSave(p *_panic, fn, arg unsafe.Pointer, argsize uint32) {
if p != nil { // 解决 panic
p.argp = unsafe.Pointer(getargp(0))
p.pc = getcallerpc()
p.sp = unsafe.Pointer(getcallersp())
}
reflectcall(nil, fn, arg, argsize, argsize)
if p != nil {
p.pc = 0
p.sp = unsafe.Pointer(nil)
}
}
open coded defer
中产生 panic
或调用runtime.Goexit
,前面未注册到的defer
函数无奈执行到,须要栈扫描。_defer
构造体中就多增加了一些字段,借助这些字段能够找到未注册到链表中的 defer
函数,后果就是 defer
变快了,然而 panic
变慢了。
至此,defer
实现原理根本梳理结束,上面介绍一下一些应用场景。
应用场景案例
因为 defer
相似 C ++ 中的析构函数的作用,因而能够用来做些开头的工作。
资源开释
C++
中,利用 RAIIRAII(Resource Acquisition Is Initialization)
资源获取即初始化机制来确保资源分配后能够被回收。这种机制关键点就是利用了对象来到生命周期时,会主动调用析构函数,通过在析构函数中实现资源回收操作即可。golang
中没有这种机制,然而能够利用 defer
来实现,确保对象在来到生命周期时被销毁。
// defer 敞开文件
package main
import (
"fmt"
"os"
)
func main() {fileHandler, err := os.Open("./test.txt")
if nil != err {panic(err)
}
// 查看完, 发现没有谬误,就能够敞开应用 defer 来敞开
defer func() {err := fileHandler.Close()
if nil != err {fmt.Println("defer 敞开文件失败:", err)
} else {fmt.Println("defer 敞开文件胜利")
}
}()}
defer 敞开文件胜利
上报
上报 / 日志解决,应用 defer
可能节俭大量的代码工作量,尤其是对于失败和胜利都须要上报 / 日志的场景。
// defer 日志解决
package main
import (
"fmt"
"math/rand"
"time"
)
func testDefer() {
a, b := 0, 1
defer func(a, b *int) {fmt.Printf("a=%d,b=%d\n", *a, *b)
}(&a, &b)
rand.Seed(time.Now().Unix())
if rand.Int()%2 == 0 {a, b = 1, 2} else {return}
}
func main() {testDefer()
}
a=1,b=2
a=0,b=1
将上报 / 日志解决操作放在 defer
中可能确保即便在函数提前返回的状况下也能够失常执行,不至于脱漏。
函数执行工夫
有这样的一个场景,咱们须要获取一个函数的耗时,在 go 语言中咱们会怎么做呢?
在其余语言中,可能是这样操作:
- 在函数块的第一行获取并记录函数开始执行的工夫
start
- 在函数完结时,再次获取以后工夫
end
,end-start
就是函数执行的大略总工夫
在 go 语言中,咱们能够借助 defer
机制来优雅的实现。
func slowOperation() {defer trace("slowOperation")() // 留神函数调用,不能漏掉最初的圆括号
time.Sleep(10 * time.Second)
}
func trace(msg string) func() {start := time.Now()
return func() { log.Printf("exit %s time_cost=%s", msg, time.Since(start)) }
}
2021/12/04 23:06:42 exit slowOperation time_cost=10.000143172s
小结
通过下面的梳理,咱们晓得 defer
关键字的实现次要依附 编译器 和运行时 的合作来实现。defer
的实现并不是一步到位,间接就是当初的样子,而是通过了数年,几个版本的迭代才出现当初的风貌的:
-
go 1.12 堆上调配
- 编译期将
defer
关键字转换成runtime.deferproc
并在调用defer
关键字的函数返回之前插入runtime.deferreturn
; - 运行时调用
runtime.deferproc
会将一个新的runtime._defer
对象插入到以后g._defer
的链表头; - 运行时调用
runtime.deferreturn
会从g._defer
的链表中取出runtime._defer
构造并顺次执行;
- 编译期将
-
go 1.13 栈上调配
- 当
defer
关键字在函数体中最多执行一次时,编译期间的gc.state.call
会将构造体调配到栈上并调用runtime.deferprocStack
;
- 当
-
go 1.14 凋谢编码
- 编译期间判断
defer
关键字、return
语句的个数确定是否开启凋谢编码优化; - 通过
deferBits
和gc.openDeferInfo
存储defer
关键字的相干信息; - 如果
defer
关键字的执行能够在编译期间确定,会在函数返回前直接插入相应的代码,否则会由运行时的runtime.deferreturn
解决。
- 编译期间判断
图 5 Golang defer 优化路线
三种实现机制,并不是代替关系,而是,特殊化解决的关系,条件越来越刻薄。尽管性能一直晋升,但机制适用范围越来越窄。
参考文献
- [1] https://go.dev/ref/spec#Defer…
- [2] https://go.dev/doc/effective_…
- [3] Defer, Panic, and Recover
- [4] defer
- [3] https://segmentfault.com/a/11…
- [5] https://zhuanlan.zhihu.com/p/…
- [6]https://juejin.cn/post/710188…
- [7]引入栈 https://go-review.googlesourc…
- [8]引入凋谢编码 https://go-review.googlesourc…
- [9]优化成果 https://github.com/golang/pro…
- [10] https://www.topgoer.com/%E5%8…
- [11] https://www.luozhiyun.com/arc…