Noticed the object pooling doc, had me wondering: are there any plans to make packages like `sync` generic?
replies(1):
- https://go101.org/optimizations/101.html
- https://github.com/uber-go/guide
I wish this content existed as a model context protocol (MCP) tool to connect to my IDE along w/ local LLM.
After 6 months or switching between different language projects, it's challenging to remember all the important things.
Allocations of any kind have an effect on triggering GC earlier, but in real apps it is almost hopeless to avoid GC, except for very carefully written programs with no dependenciesm, and if GC happens, then reducing GC mark times gives bigger bang for the buck.
This has saved Uber a lot of money on compute (I'm one of the devs). If your compute fleet is large and has memory to spare (stateless), performing dynamic GOGC tuning to tradeoff higher memory utilization for fewer GC events will save quite a lot of compute.
https://github.com/mark3labs/mcp-go/blob/main/examples/every...
Taken to the extreme, Go is still nice even with constraints. For example, tinygo is pretty nice for microcontroller projects. You can say upfront that you don't want GC, and just allocate everything at the start of the program (kind of like how DJB writes C programs) and writing the rest of the program is still a pleasant experience.
Anytime you use an interface it forces a heap allocation, even if the object is only used read only and within the same scope. That includes calls to things like fmt.Printf() so doing a for loop that prints the value of i forces the integer backing i to be heap allocated, along with every other value that you printed. So if you helpfully make every api in your library use an interface you are forcing the callers to use heap allocations for every single operation.
Go's network interfaces and slices makes this kind of thing particularly simple - I had to do the same thing in Java and it was a lot more awkward.
Doing huge allocations which go to LOH is quite punishing, but even substantial inter-generational traffic won't kill it.
So, if you have predictable object sizes, the pool will stay flat. If the workloads are random, you have a new problem because, like in this scenario, your pool grows 5x more.
You can solve this problem. E.g. you can only give back items into the pool that are small enough. Alternatively, you could have a small pool and a big pool, but now you're playing cat and mouse.
In such a scenario, it could also work to simply allocate and use GC to clean up. Then you don't have to worry about memory and the lifetime of objects, which makes your code much simpler to read and reason about.
pool := sync.Pool{
New: func() any { return 42 }
}
a := pool.Get()
pool.Put("hello")
pool.Put(struct{}{})
b := pool.Get()
c := pool.Get()
d := pool.Get()
fmt.Println(a, b, c, d)
Is a type system all that helpful if you have to keep turning it off any time you want to do something even slightly interesting?
Also I can't help but notice that there's no API to reset values to some initialized default. Shouldn't there be some sort of (perhaps optional) `Clear` callback that resets values back to a sane default, rather than forcing every caller to remember to do so themselves?
If that's the case, it's usually better to have non-global pools, pool ranges, drop things after a certain capacity, etc.:
https://github.com/golang/go/issues/23199 https://github.com/golang/go/blob/7e394a2/src/net/http/h2_bu...
My comment is more about "reduce allocations to reduce GC pressure" advice seen everywhere. It doesn't tell the whole story. Short lived allocation doesn't introduce any GC pressure: you'll be hard pressed to see GC sweep phase on pprof without zooming. People take this advice, spend time and energy hunting down allocations, just to see that total GC time remained the same after all that effort, because they were focusing on wrong type of allocations.
> Also I can't help but notice that there's no API to reset values to some initialized default.
That's what the New function does, isn't it?
BTW, the code you posted isn't syntactically correct. It needs a comma on the second line.
But that's only run when the pool needs to allocate more space. What GP seems to expect is that sync.Pool() would always return a zeroed structure, just as Golang allocation does.
I think Golang's implementation does make sense, as sync.Pool() is clearly an optimization you use when performance is an issue; and in that case you almost certainly want to only initialize parts of the struct that are needed. But I can see why it would be surprising.
> [any] is a type
It's typed the way Python is typed, not the way Rust or C are typed; so loses the "if it compiles there's a good chance it's correct" property that people want from statically typed languages.
I don't use sync.Pool, but it does seem like now that we have generics, having a typed pool would be better.
Go is pretty strict about breaking changes, so they probably won't change the current implementations; maybe we'll see a v2 version, or maybe not. The more code you have, the more code you have to maintain, and given Go's backward-compatibility promises, that's a lot of work.
If that's what people actually wanted, Coq and friends would be household names, not the obscure oddities that they are. All the languages that people actually use on any kind of regular basis require you to write tests in order to gain that sense of correctness, which also ends up validating type-correctness as a natural byproduct.
"A machine can help me refactor my code" is the property that most attracts people to the statically typed languages that are normally used. With "I can write posts about it on the internet" being the secondary property of interest.
Might be, but that's a design decision that has nothing to do with type or generics, isn't it? You seem to refer to a function to drain the pool, which is not needed, and frankly, rather unusual.
> It's typed the way Python is typed
Not in the slightest.
> "if it compiles there's a good chance it's correct"
If you want to compare it to something, it's more like Rust's unwrap(), which will panic if you apply it to the wrong result.
You know, it's this kind of comment on USENET forums which prompted the creation of StackOverflow. It's not curious and adds nothing to the discussion.
I like Go and use it extensively; and I like having the option to fall back to the `any` type. But it's simply a fact that using the `any` type means that certain properties of the program can't be checked at compile time, in the same way that Python isn't able to check certain properties of the program at compile time.
> If you want to compare it to something, it's more like Rust's unwrap(), which will panic if you apply it to the wrong result.
Rust's unwrap() is used when a type can have one of exactly two underlying types (which is why no type is specified). In this case, getting back an `any` type means the underlying type could literally be anything -- as demonstrated by the example, where they put an integer, a string, and an empty struct into the pool. That's almost certainly not what you wanted, but the compiler won't prevent you from doing it.
One could define a new "Pool[T]" type (extending sync.Pool) to get these guarantees:
type Pool[T any] sync.Pool // typed def
func (p *Pool[T]) Get() T { // typed Get
pp := (*sync.Pool)(p)
return pp.Get().(T)
}
func (p *Pool[T]) Put(v T) { // typed Put
pp := (*sync.Pool)(p)
pp.Put(v)
}
intpool := Pool[int]{ // alias New
New: func() any { var zz int; return zz },
}
boolpool := Pool[bool]{ // alias New
New: func() any { var zz bool; return zz },
}
Map/List<T> etc are erased to basically an array of Objects (or a more specific supertype) at compile-time, but you can still use the non-generic version (with a warning) if you want and put any object into a map/list, and get it out as any other type, you having to cast it as the correct type.
Also, write-only barriers don't have that big of an overhead.
It's static vs. dynamic and strong vs. weak.
If you can avoid allocs in a hot loop, it definitely pays to do so. If you can't for some reason, and can use sync.Pool there, measure it.
Cutting allocs in half may not matter much, but if you can cut them by 99% because you were allocating in every iteration of a 1 million loop, and now aren't, it will make a difference, even if all those allocs die instantly.
I've gotten better than two fold performance increases on real code with both techniques.
That’s not a strong argument. You can easily (but sometimes tediously) wrap any API with one that (further) restricts what types you can use with it. Generics make it possible to avoid doing that work, and code you don’t write won’t have errors.
There have recently been some optimizations to `runtime.mallocgc`, which may have decrease that 3-4x estimate a bit.
You might want to step off that extremely high horse for a second, buddy. It's extremely reasonable to expect a type-safe pool that only holds a single type, since that's the most common use case.
> certain properties of the program can't be checked at compile time
Neither can you check if a number is positive or negative, or if a string is empty or not at compile time, but that doesn't make Go similar to COBOL or Forth. `var v any` declares v to be of the type any, not of any arbitrary type, which is what Python does. Writing `v + 1` gives a compiler error, unlike Python, which may or may not turn it into a runtime error. It is rather different, and especially so when you look at interfacing. Even though you may declare a variable to be an integer in Python, there is no guarantee that it actually is, whereas in Go that is the case, which has significant implications for how you handle e.g. json.
> the compiler won't prevent you from doing it.
It will prevent you from using e.g. an array of strings as if it were an array ints. Python does not. They are quite different.
> You know, it's this kind of comment on USENET forums which prompted the creation of StackOverflow. It's not curious and adds nothing to the discussion.
Ffs.
GC frequency is directly driven by allocation rate (in terms of bytes) and live heap size. Some examples:
- If you halve the allocation rate, you halve the GC frequency.
- If you double the live heap size, you halve the GC frequency (barring changes away from the default `GOGC=100`).
It is true that sweeping is a lot cheaper than marking, which makes your next statement:
> Short lived allocations, those which GC mark phase will never reach, has almost neglible effect on GC times.
...technically correct. Usually, this is the best kind of correct, but it omits two important considerations:
- If you generate a ton of short-lived allocations instead of keeping them around, the GC will trigger more frequently.
- If you reduce the live heap size (by not keeping anything around), the GC will trigger more frequently.
It is not a priori clear to me this is a win. In my experience, it isn't.
From the Godoc:
> The Map type is optimized for two common use cases: (1) when the entry for a given key is only ever written once but read many times, as in caches that only grow, or (2) when multiple goroutines read, write, and overwrite entries for disjoint sets of keys. In these two cases, use of a Map may significantly reduce lock contention compared to a Go map paired with a separate Mutex or RWMutex.
Source: https://pkg.go.dev/sync#Map
And regarding slow writes, those were recently improved in Go 1.24:
> The implementation of sync.Map has been changed, improving performance, particularly for map modifications. For instance, modifications of disjoint sets of keys are much less likely to contend on larger maps, and there is no longer any ramp-up time required to achieve low-contention loads from the map.
Source: https://go.dev/doc/go1.24#minor_library_changes ("sync" section)
I assume the thinking was that this is pretty easy to optimise if you care, and if it's on by default there'd then have to be some opt-out which there isn't a good mechanism for.
No, you can't decide on heap vs stack. Go's compiler decides that. You can get feedback about the decision if you pass the right debug flags, and then based on that you may be able to tickle the optimizer into changing its mind based on code changes you make, but it'll always be an optimization decision subject to change without notice in any future versions of Go, just like any other language where you program to the optimizer.
If you need that level of control, Go is generally not the right language. However, I would encourage developers to be sure they need that level of control before taking it, and that's not special pleading for Go but special pleading for the entire class of "languages that are pretty fast but don't offer quite that level of control". There's still a lot of programmers running around with very 200x ideas of performance, even programmers who weren't programmers at the time, who must have picked it up by osmosis.
(My favorite example to show 200x perf ideas is paginated APIs where the "pages" are generally chosen from the set {25, 50, 100} for "performance reasons". In 2025, those are terribly, terribly small numbers. Presenting that many results to humans makes sense, but my default size for paginating API calls nowadays is closer to 1000, and that's the bottom end, for relatively expensive things. If I have no reason to think it's expensive, tack another order of magnitude on to my minimum.)
[1]: https://github.com/teh-cmc/go-internals/blob/master/chapter2...
[2]: https://cs.opensource.google/go/go/+/refs/tags/go1.24.1:src/...
So long as that one is not you? You completely forgot to address the expectation:
type Foo struct{ V int }
pool := Pool[*Foo]{ // Your Pool type.
New: func() any { return new(Foo) },
}
a := pool.Get()
a.V = 10
pool.Put(a)
b := pool.Get()
fmt.Println(b.V) // Prints: 10; 0 was expected.You make it look like it's a good thing to be able to express it.
There's no way in Java, Rust or C++ to express this, praised be the language designers.
As for expressing a pool value that may be multiple things without a horrible any type and an horrible cast, you could make an union type in Rust, or an interface in Java implemented by multiple concrete objects. Both ways would force the consumer to explicitly check the value without requiring unchecked duck typing.
It serves as a way around a limitation in the type system which you don't want to deal with.
You can still have the rest of the code base be safe, as long as you create a wrapper which is.
The same can be said about having imperative implementations with functional interfaces wrapping said implementation. From the outside, you have a view of a system which is functionally sound. Internally, it might break the rules and use imperative code (usually for the case of efficiency).
This array is also used for single-byte strings (which previously had its own array): https://go-review.googlesource.com/c/go/+/221979/3/src/runti...
What people do is what this article suggested, pool.Get/pool.Put, which makes it only grow in size even if load profile changes. App literally accumulated now unwanted garbage in pool and no app I have seen made and attempt to GC it.
> fmt.Println(b.V) // Prints: 10; 0 was expected.
Sorry, I don't get what else one expects when pooling pointers to a type? In fact, pooling *[]uint8 or *[]byte is common place; Pool.Put() or Pool.Get() then must zero its contents.
I am against blanket statements "reduce allocations to reduce GC pressure", which lead people wrong way: they compare libraries based on "allocs/op" from go bench, they trust rediculous (who allocates 8KB per iteration in tight loop??) microbenchmarks of sync.Pool like in the article above, hoping to resolve their GC problem. Spend considerabe amount of effort just to find that they barely moved a needle on GC times.
If we generalize then my "avoid long-lived allocations" or yours "reduce allocation rate in terms of bytes" are much more useful in practice, than what this and many other articles preach.
I haven't personally found it to be problematic; just keep it private, give it a default new func, and be cautious about only putting things in it that you got out.
As seen in your previous comment, the expectation is that the zero value will always be returned: "What GP seems to expect is that sync.Pool() would always return a zeroed structure, just as Golang allocation does." To which you offered a guarantee.
> Pool.Put() or Pool.Get() then must zero its contents.
Right. That is the solution (as was also echoed in the top comment in this thread) if one needs that expectation to hold. But you completely forgot to do it, which questions what your code was for? It behaves exactly the same as sync.Pool itself... And, unfortunately, doesn't even get the generic constraints right, as demonstrated with the int and bool examples.
Go’s type system has your back when you’re writing easy stuff.
But it throws up its hands and leaves you to fend for yourself when you need to do nearly anything interesting or complex, which is precisely when I want the type system to have my back.
I should not have to worry (or worse, not worry and be caught off guard) that my pool of database connections suddenly starts handing back strings.
If those constraints don't hold (like you say) it should manifest as runtime panic, no?
> What GP seems to expect is that sync.Pool() would always return a zeroed structure
Ah, well. You gots to be careful when Pooling addresses.
> But you completely forgot to do it, which questions what your code was for?
OK. If anyone expects zero values for pointers, then the New func should return nil (but this is almost always useless), or if one expects values to be zeroed-out, then Pool.Get/Put must zero it out. Thanks for the code review.
Not quite that. Imagine I have a pool of buffers with a length and capacity, say when writing code to handle receiving data from the network.
When I put one of those buffers back, I would like the next user of that buffer to get it back emptied. The capacity should stay the same, but the length should be zero.
I think it’s reasonable to have a callback to do this. One, it doesn’t force every consumer of the pool to have to remember themselves; it’s now a guarantee of the system itself. Two, it’s not much work but it does prevent me from re-emptying freshly-allocated items (in this case reinitialzing is fast, but in some cases it may not be).
This also should be an optional callback since there are many cases where you don’t want any form of object reset.
No. Your int and bool pools run just fine – I can't imagine you would have posted the code if it panicked – but are not correct.
> I did not forget?
Then your guarantee is bunk: "One could define a new "Pool[T]" type (extending sync.Pool) to get these guarantees:" Why are you making claims you won't stand behind?
> but are not correct.
I don't follow what you're saying. You asserted, "And, unfortunately, doesn't even get the generic constraints right, as demonstrated with the int and bool examples." What does it even mean? I guess, this bikeshed has been so thoroughly built that the discussion points aren't even getting through.
Values are copied in Go. Your code will function, but it won't work.
You've left it up to the user of the pool to not screw things up. Which is okay to some degree, but sync.Pool already does that alone, so what is your code for?
Gotcha. Thanks for clearing it up.
> so what is your code for?
If that's not rhetorical, then the code was to demonstrate that sync.Pool could be "extended" with typedefs/embeds + custom logic. Whether it got pooling itself right was not the intended focus (as shown by the fact that it created int & bool pools).
Wherein lies the aforementioned guarantee? The code guarantees neither the ask (zero values) nor even proper usage if you somehow didn't read what you quoted and thought that some kind of type safety was the guarantee being offered.
Furthermore, who, exactly, do you think would be familiar enough with Go to get all the other things right that you left out but be unaware of that standard, widely used feature?
I think you should re-read what I wrote. You seem to be upset that I did not solve everyone's problem with sync.Pool with my 10 liner (when I claimed no such thing).
One could define a new "Pool[T]" type (extending sync.Pool) to get these guarantees
Yes, structural typing removes the ability to check certain properties at compile-time. That doesn't make it typed like Python, though.
> Converting a small integer value into an interface value no longer causes allocation
I forgot that it can also be used for single byte strings, That's not an optimization I ever encountered being useful, but it's there!
You "forgot" to copy the colon from the original statement. A curious exclusion given the semantic meaning it carries. Were you hoping I didn't read your original comment and wouldn't notice?
> You seem to be upset
How could one ever possibly become upset on an internet forum? Even if for some bizarre and unlikely reason you were on the path to becoming upset, you'd turn off the computer long before ever becoming upset. There is absolutely no reason to use this tool if it isn't providing joy.
> One could define / extend sync.Pool to get those guarantees [for their custom types] ...
What audience would be interested in this? Is there anyone who understands all the intricacies of sync.Pool but doesn't know how to define types or how to write functions?
Good is subjective, but the mechanism is something already implemented: https://pkg.go.dev/structs#HostLayout
The spec defines alignment for numeric types, but that's about it. There is nothing in the spec about struct layout. That is implementation dependent. If you are relying on a particular implementation, you are decidedly in unsafe territory.
> so any change to the default would break a lot of code.
The compiler can disable optimization on cgo calls automatically and most other places where it matters are via the standard library, so it might not be as much as you think. And if you still have a case where it matters, that is what this is for: https://pkg.go.dev/structs#HostLayout
You got me!
> If the Pool holds the only reference when this happens, the item might be deallocated.
Conceptually, the pool is holding a weak pointer to the items inside it. The GC is free to clean them up if it wants to, when it gets triggered.
But the instrument itself is really sharp and is indeed kind of last resort
A truly typeless language (or maybe more accurately single type for everything language) is ASM, particularly for older CPU designs. You get a word - the bitfield in a register, and can do any operation on it. Is that 64 bits loaded from an array of characters and the programmer intended it to be a string? Cool you can bitwise and it with some other register. Was it a u64, a pointer, a pair of u32s? Same thing - the semantics don't change.
binary.Read/Write already uses reflect as far as I can tell, so it wouldn't matter in that case. There is an optimization for numeric values in there, which may be what you are thinking of? But they are specified in the spec and that's not what we're talking about anyway (and if an optimization really had to go, for whatever reason, it wouldn't be the end of the world).
Did you mean something else?
Basically, I'm trying to come up with an real world example of a style change (like create wrappers for every error, or use naked integers instead of time.Time) to estimate its impact. And my feeling is that any such example would affect one of your points way more than the other, so we can still argument that e.g. "creating short-lived iterators is totally fine".
That's not even the case. In Java, you'd just use Object, which is for all practical purposes equivalent to `interface{}` aka `any` in Go. And then you downcast. Indeed, code exactly like this was necessary in Java to work with collections before generics were added to the language.
In C++, there's no common supertype, but there std::any, which can contain a value of any type and be downcast if you know what the actual type is.
As such, short-lived objects have little impact in Java (thank god for that!). They will have second order effects in Go.
The GC calls out to sync.Pool's cleanup.
By default the elastic-search results were paginated and defaulted to some small number in the order of 25..100. I increased this steadily upwards beyond 100,000 to the point where every request always returned the entire result in the first page. And it _transformed_ the performance of the service. From one that was unbearably slow for human users to one that _felt_ instantaneous. I had real perf numbers at the time, but now all I have are the impressions.
But the lesson on the impact of the overhead of those paginated calls was important. Obviously everything is specific and YMMV, but this something worth having in the back of your mind.
This is the same n00b trap that derailed the llama.cpp project last year because people don’t understand how memory maps and paging works, and the tradeoffs.
Also, it's no accident that every high-performance GC runtime went the moving, generational way.
Generations are used almost exclusively in moving GCs — precisely to reduce the negative performance impact of data relocation. Non-moving GCs are less invasive, which is why they don’t need generations and can be fully concurrent.
- For C++ programs, the allocator (allocating+freeing) consumes roughly 5% of cycles.
- For Go programs, the allocator (runtime.mallocgc) used to consume ~20% of cycles (this is the data I referenced). I checked and recently it's become closer to 15%, thanks to optimizations.
And which GC is fully concurrent? I don't think that's possible (though I will preface that I am no expert, only read into the topic on a hobby level) - I believe the most concurrent GC out there is ZGC, which does read barriers and some pointer tricks to make the stop-the-world time independent of the heap size.
Okay... but the reply was to the parent who questioned if the typing is static. It is by way of structural typing. The compiler enforces that the ducks, so to speak, are compatible. But as an empty interface has no constraints, all types are compatible. Whatever some other comment was talking about is irrelevant.
> but about sync.Pool type erasing
The type isn't erased...?
p := sync.Pool{New: func() interface{} { return 1 }}
fmt.Printf("%T", p.Get()) // Prints: int
And you think that makes it dynamically typed? It does not.
Non-moving GCs, however, can be fully concurrent — as demonstrated by the SGCL project for C++.
In my opinion, the GC for Go is the most likely to become fully concurrent in the future.
In that case, are you doing atomic writes for managed pointers/the read flag on them? I have read a few of your comments on reddit and your flags seem to be per memory page? Still, the synchronization on them may or may not have a more serious performance impact than alternative methods and without a good way to compare it to something like Java which is the state of the art in GC research we can't really comment much on whether it's a net benefit.
Also, have you perhaps tried modeling your design in something like TLA+?
You can't write concurrent code without atomic operations — you need them to ensure memory consistency, and concurrent GCs for Java also rely on them. However, atomic loads and stores are cheap, especially on x86. What’s expensive are atomic counters and CAS operations — and SGCL uses those only occasionally.
Java’s GCs do use state-of-the-art technology, but it's technology specifically optimized for moving collectors. SGCL is optimized for non-moving GC, and some operations can be implemented in ways that are simply not applicable to Java’s approach.
I’ve never tried modeling SGCL's algorithms in TLA+.
What I was hoping not to find (and fortunately didn't!) was one of Go's magical syntactic comments.