原文链接:Go语言如何在测试发现goroutine透露

前言

哈喽,大家好,我是asong

家喻户晓,gorourtine的设计是Go语言并发实现的外围组成部分,易上手,然而也会遭逢各种疑难杂症,其中goroutine透露就是重症之一,其呈现往往须要排查很久,有人说能够应用pprof来排查,尽管其能够达到目标,然而这些性能剖析工具往往是在呈现问题后借助其辅助排查应用的,有没有一款能够防患于未然的工具吗?当然有,goleak他来了,其由 Uber 团队开源,能够用来检测goroutine透露,并且能够联合单元测试,能够达到防备于未然的目标,本文咱们就一起来看一看goleak

goroutine透露

不晓得你们在日常开发中是否有遇到过goroutine透露,goroutine透露其实就是goroutine阻塞,这些阻塞的goroutine会始终存活直到过程终结,他们占用的栈内存始终无奈开释,从而导致系统的可用内存会越来越少,直至解体!简略总结了几种常见的透露起因:

  • Goroutine内的逻辑进入死循坏,始终占用资源
  • Goroutine配合channel/mutex应用时,因为使用不当导致始终被阻塞
  • Goroutine内的逻辑长时间期待,导致Goroutine数量暴增

接下来咱们应用Goroutine+channel的经典组合来展现goroutine透露;

func GetData() {    var ch chan struct{}    go func() {        <- ch    }()}func main()  {    defer func() {        fmt.Println("goroutines: ", runtime.NumGoroutine())    }()    GetData()    time.Sleep(2 * time.Second)}

这个例子是channel遗记初始化,无论是读写操作都会造成阻塞,这个办法如果是写单测也是查看不进去问题的:

func TestGetData(t *testing.T) {    GetData()}

运行后果:

=== RUN   TestGetData--- PASS: TestGetData (0.00s)PASS

内置测试无奈满足,接下来咱们引入goleak来测试一下。

goleak

github地址:https://github.com/uber-go/go...

应用goleak次要关注两个办法即可:VerifyNoneVerifyTestMainVerifyNone用于繁多测试用例中测试,VerifyTestMain能够在TestMain中增加,能够缩小对测试代码的入侵,举例如下:

应用VerifyNone:

func TestGetDataWithGoleak(t *testing.T) {    defer goleak.VerifyNone(t)    GetData()}

运行后果:

=== RUN   TestGetDataWithGoleak    leaks.go:78: found unexpected goroutines:        [Goroutine 35 in state chan receive (nil chan), with asong.cloud/Golang_Dream/code_demo/goroutine_oos_detector.GetData.func1 on top of the stack:        goroutine 35 [chan receive (nil chan)]:        asong.cloud/Golang_Dream/code_demo/goroutine_oos_detector.GetData.func1()            /Users/go/src/asong.cloud/Golang_Dream/code_demo/goroutine_oos_detector/main.go:12 +0x1f        created by asong.cloud/Golang_Dream/code_demo/goroutine_oos_detector.GetData            /Users/go/src/asong.cloud/Golang_Dream/code_demo/goroutine_oos_detector/main.go:11 +0x3c        ]--- FAIL: TestGetDataWithGoleak (0.45s)FAILProcess finished with the exit code 1

通过运行后果看到具体产生goroutine透露的具体代码段;应用VerifyNone会对咱们的测试代码有入侵,能够采纳VerifyTestMain办法能够更快的集成到测试中:

func TestMain(m *testing.M) {    goleak.VerifyTestMain(m)}

运行后果:

=== RUN   TestGetData--- PASS: TestGetData (0.00s)PASSgoleak: Errors on successful test run: found unexpected goroutines:[Goroutine 5 in state chan receive (nil chan), with asong.cloud/Golang_Dream/code_demo/goroutine_oos_detector.GetData.func1 on top of the stack:goroutine 5 [chan receive (nil chan)]:asong.cloud/Golang_Dream/code_demo/goroutine_oos_detector.GetData.func1()    /Users/go/src/asong.cloud/Golang_Dream/code_demo/goroutine_oos_detector/main.go:12 +0x1fcreated by asong.cloud/Golang_Dream/code_demo/goroutine_oos_detector.GetData    /Users/go/src/asong.cloud/Golang_Dream/code_demo/goroutine_oos_detector/main.go:11 +0x3c]Process finished with the exit code 1

VerifyTestMain的运行后果与VerifyNone有一点不同,VerifyTestMain会先报告测试用例执行后果,而后报告透露剖析,如果测试的用例中有多个goroutine透露,无奈精确定位到产生透露的具体test,须要应用如下脚本进一步剖析:

# Create a test binary which will be used to run each test individually$ go test -c -o tests# Run each test individually, printing "." for successful tests, or the test name# for failing tests.$ for test in $(go test -list . | grep -E "^(Test|Example)"); do ./tests -test.run "^$test\$" &>/dev/null && echo -n "." || echo -e "\n$test failed"; done

这样会打印出具体哪个测试用例失败。

goleak实现原理

VerifyNone入口,咱们查看源代码,其调用了Find办法:

// Find looks for extra goroutines, and returns a descriptive error if// any are found.func Find(options ...Option) error {  // 获取以后goroutine的ID    cur := stack.Current().ID()    opts := buildOpts(options...)    var stacks []stack.Stack    retry := true    for i := 0; retry; i++ {    // 过滤无用的goroutine        stacks = filterStacks(stack.All(), cur, opts)        if len(stacks) == 0 {            return nil        }        retry = opts.retry(i)    }    return fmt.Errorf("found unexpected goroutines:\n%s", stacks)}

咱们在看一下filterStacks办法:

// filterStacks will filter any stacks excluded by the given opts.// filterStacks modifies the passed in stacks slice.func filterStacks(stacks []stack.Stack, skipID int, opts *opts) []stack.Stack {    filtered := stacks[:0]    for _, stack := range stacks {        // Always skip the running goroutine.        if stack.ID() == skipID {            continue        }        // Run any default or user-specified filters.        if opts.filter(stack) {            continue        }        filtered = append(filtered, stack)    }    return filtered}

这里次要是过滤掉一些不参加检测的goroutine stack,如果没有自定义filters,则应用默认的filters

func buildOpts(options ...Option) *opts {    opts := &opts{        maxRetries: _defaultRetries,        maxSleep:   100 * time.Millisecond,    }    opts.filters = append(opts.filters,        isTestStack,        isSyscallStack,        isStdLibStack,        isTraceStack,    )    for _, option := range options {        option.apply(opts)    }    return opts}

从这里能够看出,默认检测20次,每次默认距离100ms;增加默认filters;

总结一下goleak的实现原理:

应用runtime.Stack()办法获取以后运行的所有goroutine的栈信息,默认定义不须要检测的过滤项,默认定义检测次数+检测距离,一直周期进行检测,最终在屡次查看后仍没有找到剩下的goroutine则判断没有产生goroutine透露。

总结

本文咱们分享了一个能够在测试中发现goroutine透露的工具,然而其还是须要齐备的测试用例反对,这就暴露出测试用例的重要性,敌人们好的工具能够助咱们更快的发现问题,然而代码品质还是把握在咱们本人的手中,加油吧,少年们~。

好啦,本文到这里就完结了,我是asong,咱们下期见。

欢送关注公众号:Golang梦工厂

参考资料

  • https://github.com/uber-go/go...
  • https://segmentfault.com/a/11...
  • https://blog.schwarzeni.com/2...
  • https://zhuanlan.zhihu.com/p/...