←back to thread

Go Optimization Guide

(goperf.dev)
480 points jedeusus | 1 comments | 31 Mar 25 20:29 UTC | HN request time: 0s | source
Show context
jensneuse ◴[31 Mar 25 23:06 UTC] No.43540964[source]▶
You can often fool yourself by using sync.Pool. pprof looks great because no allocs in benchmarks but memory usage goes through the roof. It's important to measure real world benefits, if any, and not just synthetic benchmarks.
replies(2): >>43541261 #>>43543697 #
makeworld ◴[31 Mar 25 23:43 UTC] No.43541261[source]▶
Why would Pool increase memory usage?
replies(2): >>43541282 #>>43543225 #
xyproto ◴[31 Mar 25 23:46 UTC] No.43541282[source]▶
I guess if you allocate more than you need upfront that it could increase memory usage.
replies(1): >>43542705 #
throwaway127482 ◴[01 Apr 25 03:59 UTC] No.43542705[source]▶
I don't get it. The pool uses weak pointers under the hood right? If you allocate too much up front, the stuff you don't need will get garbage collected. It's no worse than doing the same without a pool, right?
replies(1): >>43543649 #
1. cplli ◴[01 Apr 25 06:56 UTC] No.43543649{3}[source]▶
What the top commenter probably failed to mention, and jensneuse tried to explain is that sync.Pool makes an assumption that the size cost of pooled items are similar. If you are pooling buffers (eg: []byte) or any other type with backing memory which during use can/will grow beyond their initial capacity, can lead to a scenario where backing arrays which have grown to MB capacities are returned by the pool to be used for a few KB, and the KB buffers are returned to high memory jobs which in turn grow the backing arrays to MB and return to the pool.

If that's the case, it's usually better to have non-global pools, pool ranges, drop things after a certain capacity, etc.:

https://github.com/golang/go/issues/23199 https://github.com/golang/go/blob/7e394a2/src/net/http/h2_bu...