前往小程序,Get更优阅读体验!
立即前往
首页
学习
活动
专区
工具
TVP
发布
社区首页 >专栏 >聊聊storagetapper的pool

聊聊storagetapper的pool

原创
作者头像
code4it
修改2021-03-04 09:47:46
4440
修改2021-03-04 09:47:46
举报
文章被收录于专栏:码匠的流水账

本文主要研究一下storagetapper的pool

Thread

storagetapper/pool/pool.go

代码语言:javascript
复制
type Thread interface {
    Start(m uint, f func())
    Adjust(m uint)
    Terminate() bool
    NumProcs() uint
}

Thread接口定义了Start、Adjust、Terminate、NumProcs方法

pool

storagetapper/pool/pool.go

代码语言:javascript
复制
/*Create helps to hide poolImpl in the package, but not really required */
func Create() Thread {
    return &poolImpl{}
}

type poolImpl struct {
    mutex       sync.Mutex
    numProcs    uint
    maxNumProcs uint
    fn          func()
}

/*Start instantiates a pool of size of 'm' of 'f' goroutines */
/*Start and Create separation allows to pass pool instance to 'f' goroutine */
func (p *poolImpl) Start(m uint, f func()) {
    p.fn = f
    p.Adjust(m)
}

/*Adjust resizes the pool. It creates new threads if requested size is bigger
* then current size, while it assumes threads cooperation when requested size is
* smaller then current size. Threads should periodically call Terminate function
* and obey the result. */
func (p *poolImpl) Adjust(m uint) {
    p.mutex.Lock()
    defer p.mutex.Unlock()
    log.Debugf("Current size=%v, current maximum size=%v, requested size=%v", p.numProcs, p.maxNumProcs, m)
    p.maxNumProcs = m
    if p.numProcs < p.maxNumProcs {
        adj := p.maxNumProcs - p.numProcs
        shutdown.Register(int32(adj))
        for i := uint(0); i < adj; i++ {
            go func() { defer shutdown.Done(); p.fn() }()
        }
        p.numProcs = m
    }
}

/*Terminate return true if the caller thread need to terminate */
func (p *poolImpl) Terminate() bool {
    //Uncomment if Terminate is called frequently
    //Introduces a race when thread can miss Pool resize event, that's ok, so as
    //some other threads may see the event, or we will see it on the next
    //iteration
    //  if p.numProcs <= p.maxNumProcs {
    //      return false
    //  }

    p.mutex.Lock()
    defer p.mutex.Unlock()

    if p.numProcs > p.maxNumProcs {
        p.numProcs--
        log.Debugf("Terminating. Current size=%v, current maximum size=%v", p.numProcs, p.maxNumProcs)
        return true
    }

    return false
}

/*NumProcs return current size of the pool */
func (p *poolImpl) NumProcs() uint {
    p.mutex.Lock()
    defer p.mutex.Unlock()
    return p.numProcs
}

poolImpl定义了mutex、numProcs、maxNumProcs、fn属性;它实现了Thread接口,其Start方法设置了fn,同时执行Adjust方法;Adjust方法在numProcs小于maxNumProcs时会执行shutdown.Register,然后挨个执行shutdown.Done();Terminate方法对于numProcs大于maxNumProcs的情况递减numProcs

实例

storagetapper/pool/pool_test.go

代码语言:javascript
复制
func TestBasic(t *testing.T) {
    var m sync.Mutex
    var nProcs int32

    sig := make(chan bool)

    p := Create()

    if p.NumProcs() != 0 {
        t.Fatalf("Initially not zero")
    }

    p.Start(2, func() {
        m.Lock()
        atomic.AddInt32(&nProcs, 1)
        log.Debugf("Starting new proc, nProcs=%v", nProcs)
        m.Unlock()
        for !p.Terminate() {
            <-sig
            log.Debugf("Woken up")
        }
        m.Lock()
        atomic.AddInt32(&nProcs, -1)
        log.Debugf("Terminating proc, nProcs=%v", nProcs)
        m.Unlock()
    })

    /* Check that both real number and reported by thread pool equal to expected
    * value */
    waitFor(&nProcs, 2, 5, t)
    if p.NumProcs() != 2 {
        t.Fatalf("numProcs != 2")
    }

    p.Adjust(8)

    waitFor(&nProcs, 8, 5, t)
    if p.NumProcs() != 8 {
        t.Fatalf("numProcs != 8")
    }

    p.Adjust(3)

    for i := 0; i < 5; i++ {
        sig <- true
    }

    waitFor(&nProcs, 3, 5, t)
    if p.NumProcs() != 3 {
        t.Fatalf("numProcs != 3")
    }

    p.Adjust(0)
    for i := 0; i < 3; i++ {
        sig <- true
    }

    waitFor(&nProcs, 0, 5, t)
    if p.NumProcs() != 0 {
        t.Fatalf("numProcs != 0")
    }
}

小结

storagetapper的Thread接口定义了Start、Adjust、Terminate、NumProcs方法;poolImpl实现了Thread接口;其Adjust可以在numProcs小于maxNumProcs的时候进行扩容;Terminate会在numProcs大于maxNumProcs的时候递减numProcs。

doc

原创声明:本文系作者授权腾讯云开发者社区发表,未经许可,不得转载。

如有侵权,请联系 cloudcommunity@tencent.com 删除。

原创声明:本文系作者授权腾讯云开发者社区发表,未经许可,不得转载。

如有侵权,请联系 cloudcommunity@tencent.com 删除。

评论
登录后参与评论
0 条评论
热度
最新
推荐阅读
目录
  • Thread
  • pool
  • 实例
  • 小结
  • doc
领券
问题归档专栏文章快讯文章归档关键词归档开发者手册归档开发者手册 Section 归档