> Zero dependencies
This is something seldom attempted, but I congratulate you. Go is one of a few languages that really is batteries-included. Just about anything you could need is included in the stdlib.
replies(4):
> ... a command-line utility designed to simplify the process of generating boilerplate code for your project based on the Ergo Framework
Why is there any boiler plate code at all? Why isn't hello world just a five line programme that spawns and sends hello world somewhere 5 times?
https://fasterthanli.me/articles/i-want-off-mr-golangs-wild-...
Not to mention nothing prevents anyone from using or writing their own library only for the parts that need specialization. You're free to do that and many have.
And standard libraries can be versioned and deprecated too.
I've been thinking for years that if a project existed like this for Python it would take over the world. Golang is close, I guess.
The time API, on the other hand, is that bad, and worse. Besides the bizarre monotonic time implementation, there's also the bloated size of the time.Time struct, and the lack of actual "dates" (YYYY-MM-DD) and "times" (HH:MM:SS) that you can do arithmetic on. You can't really roll your own robustly either because time.Time isn't a great foundation to build on and //go:linkname is getting locked down.
Thankfully, build constraints are much better now, since they just use regular boolean operators, e.g.:
//go:build windows && (i386 || amd64)
Also, one of my personal gripes is that the standard library exposes no guaranteed way to access the system CSPRNG. Any code can replace (crypto/rand).Reader with another implementation and compromise cryptographic operations. It's a trivial supply-chain attack vector, and even in non-malicious scenarios, it can be done mistakenly and break things all over the place in a subtle but dangerous way. The language developers have so far refused to fix it too.
Then there's log/slog with no TRACE or FATAL levels built-in; yeah, you can roll your own levels, but why should you have to?
It looks like a close copy of Erlang APIs, albeit with the usual golang language limitations and corresponding boilerplate and some additional stuff.
Most interesting to me is it has integration with actual Erlang processes. That could fill a nice gap as Erlang lacks in some areas like media processing - so you could use this to handle those kind of CPU bound / native tasks.
func (a *actorA) HandleMessage(from gen.PID, message any) error {
switch message.(type) {
case doCallLocal:
local := gen.Atom("b")
a.Log().Info("making request to local process %s", local)
if result, err := a.Call(local, MyRequest{MyString: "abc"}); err == nil {
a.Log().Info("received result from local process %s: %#v", local, result)
} else {
a.Log().Error("call local process failed: %s", err)
}
a.SendAfter(a.PID(), doCallRemote{}, time.Second)
return nilThat golang is a mess, and demonstrates just what a huge conceptual gap there really is between the two. Erlang relies on many tricks to end up greater than the sum of its parts, like how receiving messages is actually pattern matching over a mailbox, and using a tail recursive pattern to return the next handling state. You could conceivably do that in golang syntax but it will be horrible and absolutely not play nicely with the golang runtime.
How equivalent are BEAM processes and goroutines?
Some of the above will never be in Go due to how the community and language designers are philosophically against them.
I've written before about how I think the more FBP-style concurrency of Go and the message passing one in Erlang complement each other as much as streaming DNA processing inside the cell and fire and forget cell to cell signaling between cells do in biology:
https://livesys.se/posts/flowbased-vs-erlang-message-passing...
The FBP/CSP-style in Go being more suited for high performance streaming operations inside apps and compute nodes (or cells), while the message passing mechanism shines over more unreliable channels such as over the network (or between cells).
Ergo seems like it might allow both of these mechanisms to be implemented in the same language, which should be a big deal.
Writing safe NIFs has a certain intrinsic amount of complication. Farming off some intensive work to what is actually a Go node (or any other kind of node, this isn't specific to Go) is somewhat safer, and while there is the caveat of getting the data into your non-BEAM process up front, once the data is there you're quite free.
Then again, I think the better answer is just to make some sort of normal server call rather than trying to wrap the service code into the BEAM cluster. There's not actually a lot of compelling reasons to be "in" the cluster like that. If anything it's the wrong direction, you want to reduce your dependency on the BEAM cluster as your message bus.
(Non-BEAM nodes have no need to copy using tail recursion to process the next state. That's a detail of BEAM, not an absolute requirement. Pattern matching out of the mailbox is a requirement... a degenerate network service that is pure request/response might be able to coincidentally ignore it but it would be necessary in general.)
[1]: you usually use channels instead
[2]: you usually use sync.WaitGroup (or similar, e.g. errgroup.Group) instead
var sent bool
select {
case channel<-message:
sent = true
default:
sent = false
}A while back, I tried to solve no.2 on your list by having a dynamically expanding and shrinking go-routines. POC here: https://github.com/didip/laborunion. It is meant to be used in-conjunction with config library & application object that's update-able over the wire. This idea is good enough to hot-reload a small IoT/metrics agent, but I never got around to truly solve the problem completely.
It would have been interesting if the author had a suggestion on what the Golang team should've / could've done instead, beyond correctly communicating the nature of the breaking change in the Go 1.9 release notes.
It's been a while since I played with the furry thing but is that even possible in Golang?
Eg I don’t find the stdlib logging library particularly great; not bad, but not impressive. Ditto for the stdlib errors package before they added error wrapping
Also an overlooked part here is that the global Erlang GC is easier to parallellize and/or keep incremental since it won't have object cycles sans PID's (that probably have special handling anyhow).
TlDr; GC's become way harder as soon as you have cyclic objects, Erlang avoids it and thus parts of it being good is more about Erlang being "simple".
Go is also not supposed to be used in a complicated way. Its use cases must remain simple.
Why a function that takes an Actor instead of each Actor being a type that implements a receive function? There’s so much Feature Envy (Refactoring, Fowler) in this example. There is no world where having one function handle three kinds of actors makes any sense. This is designed for footguns.
I also doubt very much that the Log() call is helping anything. Surely lathe API is thin enough to inline a that child.
In Go you can naturally do it, by using the manual constructor approach, however there is no magic auto wiring like you can do with attributes and compiler plugins, plus standard libraries infrastructure for locating services, from those three ecosystems above.
But that's separate from per process GC. Per process GC is possible because processes don't share memory[1], so each process can compact its own memory without coordination with other processes. GC becomes stop the process, not stop the world, and it's effectively preemptable, so one process doing a lot of GC will not block other processes from getting cpu time.
Also, per process GC enables a pattern where a well tuned short lived process is spawned to do some work, then die, and all its garbage can be thrown away without a complex collection. With shared GC, it can be harder to avoid the impact of short lived tasks on the overall system.
[1] yes yes, shared refcounted binaries, which are allocated separately from process memory.
This may be true only for some implementations. Good GC implementations operate on the concept of object graph roots. Whether the graph has cyclic references or not is irrelevant as the GC scans the relevant memory linearly. As long as the graph is unrooted, such GC implementations are able to still easily collect it (or, to be more precise, ignore it - the generational moving GCs the cost is the live objects that need to be relocated to an older/tenured generation).
BEAM has 4 ways to closely integrate with native code: NIFs, linked in ports, OS process ports (fork/ecommunicate over a pipe), and foreign nodes (C Nodes). You can also integrate through normal networking or pipes too. Everything has plusses and minusses.
BeamVM languages complete side step this problem all together.
If your Go version doesn't know of such a build tag as "bar", then foo_bar.go is unconditionally compiled. If Go in a later version adds "bar" as a known build tag, then foo_bar.go becomes conditionally compiled. Better hope you know this is how things work (reality: lots of Go devs don't).
Build tag lines don't have this problem. They always specify compilation conditions, even if the build tag is not already known to the compiler. They also apply to the whole file, the same as _tag in the name; there's no preprocessor spaghetti possible.
> Why a function that takes an Actor instead of each Actor being a type that implements a receive function?
That function is a method with receiver type `Actor` - IE `Actor` implements this HandleMessage function.
Granted it is exactly equivalent to ``` func HandleMessage(a *Actor, from gen.PID, message any) error { ... } ```
But I'm happy sticking with composition over inheritance
It’s a deliberate decision but very easy with the BEAM.
On Kubernetes, running with regular pods it was not desirable. Maybe if we were deploying with StatefulStates.
For more context, most of my work is around real-time streaming / monitoring of trades and trading metrics.
Go's GC already has a lot of work done to minimize "stop the world" time down to very small values.
defmodule KV.Supervisor do
use Supervisor
def start_link(opts) do
Supervisor.start_link(__MODULE__, :ok, opts)
end
@impl true
def init(:ok) do
children = [
KV.Registry
]
Supervisor.init(children, strategy: :one_for_one)
end
end
(I can also tell you from personal experience Mnesia isn't a "durable storage" solution. Rather the opposite, honestly.)
However Go doesn't like magic, thus that isn't something that will ever happen on the standard library, like on Python, Java, .NET.
For example you could parallelize a for loop by creating a goroutine for each item in a list. Or separate actors for logging, session tracking, connection pools, etc.
Doing that with processes and Kafka would be massive overhead.
So the BEAM VM definitely handles things a bit better but Golang can get quite close.
The Java gc's are doing some crazy stuff, my point however was that the acyclic nature of the Erlang object graph enables them to do fairly "simple" optimizations to the GC that in practice should remove most need for pauses without hardware or otherwise expensive read barriers.
It doesn't have to do a lot of things to be good, once you have cycles you need a lot more machinery to be able to do the same things.
Build constraints with "//go:build" already exist, I'm not making them up or proposing something new: https://pkg.go.dev/go/build#hdr-Build_Constraints
This has nothing to do with preprocessor spaghetti, which is impossible in Go. Either a file is included or it is excluded. Neither with file naming nor "//go:build" can you cause only portions of a file to be compiled.
Really, the _tag.go mechanism is just a very simple kind of build constraint, equivalent to "//go:build tag". The problem is that it relies on a magic list of known build tags, and the contents of that list change over time. Apart from _test.go, which is a little too convenient to give up in my opinion, the rest of the tags could be pushed into build constraints, or at the very least, the list of known build tags could be frozen in its current state.
And if one goes back 20 years, constructor based injection was the norm, the magic only came later thanks to Aspect Oriented Programming, yet another tool that Go will never adopt.
Erlang/Elixir has plenty of performance for plenty of problems, but Go is definitely generally a cut above. And there's definitely another cut above Go in performance, it's not the fastest, and there's a cut below Erlang/Elixir as well because they're generally faster than the dynamic scripting languages. And even the dynamic scripting languages are often fast enough for plenty of loads themselves.
But yes, you do have to determine state and recovery patterns. Depends entirely on the situation but things like making sure your data will survive a process crash is straightforward.
When it comes to Java - it has multiple GC implementations with different tradeoffs and degree of sophistication. I'm not very well versed in their details besides the fact that pretty much all of them are quite liberal with the use of host memory. So the way I approach it is by assuming that at least some of them resemble the GC implementation in .NET, given extensive evidence that under allocation-heavy scenarios they have similar (throughput) performance characteristics.
As for .NET itself, in server scenarios, it uses SRV GC which has per-core heaps (the count is sizing is now dynamically scalable per workload profile, leading to much smaller RAM footprint) and multi-threaded collection, which lends itself to very high throughput and linear scaling with cores even on very large hosts thanks to minimal contention (think 128C 1TiB RAM, stometimes you need to massage it with flags for this, but it's nowhere near the amount of ceremony required by Java).
Both SRV and WKS GC implementations use background collection for Gen2, large and pinned object heaps. Collection of Gen0 and Gen1 is pausing by design as it lends itself for much better throughput and pause times are short enough anyway.
They are short enough that modern .NET versions end up having better p99 latency than Go on multi-core throughput saturated nodes. Given decent enough codebase, you only ever need to worry about GC pause impact once you go into the territory of systems with hard realtime requirements. One of the better practical examples of this that exists in open source is Osu! which must run its game loop 1000hz - only 1ms of budget! This does pose challenges and requires much more hands-on interaction with GC like dynamically switching GC behaviopr depending on scenario: https://github.com/dotnet/runtime/issues/96213#issuecomment-... This, however, would be true with any language with automatic memory management, if it's possible to implement such a system in it in the first place.
Goroutines communicate through channels, all you need is a queue (eg buffered chan).
> you cannot terminate a goroutine from another goroutine
Termination is propagated via context cancellation. go-A cancels ctx, go-B waits with `select`, reads from `<-ctx.Done()` and does a `return`, or checks it after each blocking call.
> there are no "goroutine-local variables"
Not sure if I got this one, but every var in a function's scope, which has been `go`-routined, would qualify.
I'm currently working on a lib/framework somehow related to Ergo, but taking a more "generic" approach of a state machine[0]. It may solve some of the mentioned issues with Go, like addressing and queues for communication.
You seem to be very attached to an idea of using the same goroutine for a long time, whereas it's usually more dynamic and only schedulers are long lived `go`-s.
I appreciate the mention of context cancelation, that is a good example of how to terminate a goroutine, provided that a) the goroutine has a context to cancel and b) the goroutine does not spend too much time in operations that can't be canceled (e.g., file system calls, time.Sleep, blocking select without case <-ctx.Done(), etc.). This is still cooperative multitasking though, while BEAM processes are fully preemptive.
A "goroutine-local variable" would be a variable accessible by other goroutines which is still scoped to a particular goroutine. Ordinary local variables are only accessible by the goroutine in whose stack they belong. Something like the process dictionary can be constructed in Go using a synchronized map with atomic values, but it certainly isn't a built-in part of the runtime like it is in BEAM.
I generally don't regard goroutines with much attachment. There are, however, some situations in which goroutines must be long-lived: the main goroutine, the "scheduler" goroutines you mentioned, and in general any goroutine performing a supervisory role over other goroutines. Monitoring these important goroutines is more complicated in Go than BEAM languages, but certainly isn't impossible.
The more problematic aspect of goroutines vs. BEAM processes is that, due to the lack of application-accessible preemption, when a critical-path goroutine does get stuck, the only real solution most of the time is to kill the entire program. This is rarer than it was in Go's early days, at least.
Small things like tuples (combining multiple values in outputs) and out parameters relaxes the burden on the runtime since programmers don't need to create objects just to send several things back out from a function.
But the real kicker probably comes since the lowlevel components, be it the Http server with ValueTasks and the C# dictionary type getting memory savings just by having struct types and proper generics. I remember reading some article from years ago about re-writing the C# dictionary class that they reduced memory allocations by something like 90%.
On allocation traffic, I doubt average code in C# allocates less than Go - the latter puts quite a lot of emphasis on plain structs, and because Go has very poor GC throughput, the only way to explain tolerable performance in the common case is that Go still allocates less. Of course this will change now that more teams adopt Go and start classic interface spam and write abstractions that box structs into interfaces to cope with inexpressive and repetition-heavy nature of Go.
Otherwise, both .NET and Java GC implementations are throughput-focused, even the ones that target few-core smaller applications, while Go GC focuses on low to moderate allocation traffic on smaller hosts with consistent performance, and regresses severely when its capacity to reclaim memory in time is exceeded. You can expect from ~4 up to ~16-32x and more (SRV GC scales linearly with cores) difference in maximum allocation throughput between Go and .NET: https://gist.github.com/neon-sunset/c6c35230e75c89a8f6592cac...