Most active commenters
  • aatd86(9)
  • rowanseymour(7)
  • kbolino(6)

←back to thread

Go subtleties

(harrisoncramer.me)
235 points darccio | 47 comments | | HN request time: 0.421s | source | bottom
1. rowanseymour ◴[] No.45666258[source]
Ah the old nil values boxed into non-nil interfaces. Even after 8 years writing go code almost every day this still bites me occasionally. I've never seen code that actually uses this. I understand why it is the way it is but I hate it.
replies(4): >>45666614 #>>45666792 #>>45666859 #>>45669163 #
2. amelius ◴[] No.45666614[source]
I ditched Go after an evaluation years ago. I can remember it was an issue with nil pointers being non-intuitive that turned me off. And exception handling. A pity because the runtime and ecosystem/community seemed pretty good.
replies(2): >>45666666 #>>45667034 #
3. rowanseymour ◴[] No.45666666[source]
It's fantastic concise language and standard library steered by people who are determined to keep it simple and intuitive... which IMO makes it all the more odd that it has this obvious foot gun trap where `!= nil` doesn't always mean what you might think.
replies(1): >>45667163 #
4. aatd86 ◴[] No.45666792[source]
Yes, that'a bit too late after ten+ years perhaps but I wished we had a nil type and checking whether the interface is empty was a type assertion. In all other cases, like any(2) == 2, we compare the values.

Then again that would mean that the nil identifier would be coerced into a typed nil and we would check for the nilness of what is inside an interface in any(somepointer) == nil.

wrt the current behavior, it also makes sense to have a nil value that remains untyped. But in many other cases we do have that automatic inference/coercion, for instance when we set a pointer to nil.(p = nil)

That's quite subtle and that ship has sailed though.

replies(2): >>45666878 #>>45668625 #
5. kitd ◴[] No.45666859[source]
The advice I've read (and follow) is always to return values, not interfaces, from functions and test for nil against them. That IME tends to nip the majority of nil interface problems in the bud.
replies(4): >>45667017 #>>45667105 #>>45667244 #>>45667694 #
6. rowanseymour ◴[] No.45666878[source]
Agree the ship has likely sailed, but if it could be addressed wouldn't it be nice to remove nil value interfaces altogether? Maybe start by letting new interface types declare/annotate that they don't box nil values? Then one day that becomes the default. Oh well.
replies(2): >>45667583 #>>45670974 #
7. YesThatTom2 ◴[] No.45667017[source]
Exactly.

I find that people try to use interfaces like they’re using an OO language. Go is not OO.

8. ignoramous ◴[] No.45667034[source]
> And exception handling

If you read & write Go regularly, the rather verbose error handling simply fades into the background.

That said, errors in Go don't really translate to Exceptions as generally thought of; panic, however; may be does.

Making changes to error handling wasn't for the lack of trying, though: https://news.ycombinator.com/item?id=44171677

> issue with nil pointers

This is why most APIs strive for a non-nil zero value, where possible, as methods (on structs) can still dictate if it will act on a pointer. Though, I get what you're saying with Go missing Optional / Maybe / ? operator, as the only other way to warn about nil types is through documentation; ex: https://github.com/tailscale/tailscale/blob/afaa23c3b4/syncs... (a recent example I stumbled upon).

Static code analysers like nilaway (https://news.ycombinator.com/item?id=38300425) help, but these aren't without false positives (annoying) & false negatives (fatal).

replies(1): >>45678512 #
9. leetrout ◴[] No.45667105[source]
Return concrete types, accept interfaces. Returning interfaces hides behavior and hampers evolution; accept interfaces so callers can swap implementations. For testing, mock the dependency, not the return value.

Loudest arguments against returning concrete types were on the terraform core team and the excuse was it makes testing easier. I disagree.

replies(1): >>45667214 #
10. amw-zero ◴[] No.45667163{3}[source]
The “simplicity” of Go is just virtue signaling. It has gotchas like that all over the language, because it’s not actually simple.
replies(3): >>45667506 #>>45669557 #>>45673702 #
11. lenkite ◴[] No.45667214{3}[source]
This advice of "returning concrete types" is in most cases a horrible anti-pattern that prevents evolution due to lack of information hiding. It has been also deliberately broken in the standard library in several places. This "advice" cannot be generically applied. Places where it is has been deliberately broken:

   net.Dial (Conn, error) 
   image.Decode(r io.Reader) (Image, string, error)
   sha256.NewXXX() hash.Hash
   flate.NewReader(r io.Reader) io.ReadCloser
   http.NewFileTransport(fs FileSystem) RoundTripper
Regarding `os.File`, the Go team even said: “If we were starting from scratch, we might do it differently.”

That’s why Go added abstractions later like fs.FS and fs.File.

   embed/fs.Open again deliberately breaks this.
Whereas consider its counterpart net.Conn. net.Conn is one of the most successful interfaces in the Go standard library. It’s the foundation of the net, net/http, tls, and net/rpc packages, and has been stable since Go 1.0. It didn't need a replacement fs.Fs.

If you will always only ever have one implementation in absolute permanence and no mocking/fake/alternative implementation is ever required in eternity, return a concrete type. Otherwise, consider whether returning an interface makes more sense.

replies(2): >>45667420 #>>45667824 #
12. rowanseymour ◴[] No.45667244[source]
That works until 1) you don't want to export the value types 2) the return values aren't simple structs but slices or maps because []x is not a []X even if x implements X.
replies(2): >>45667431 #>>45667624 #
13. arccy ◴[] No.45667420{4}[source]
On the contrary, in recent proposal reviews, returning interfaces has been discouraged unless you're trying to make a generic interface like fs.FS, or dispatch functions like net.Dial / image.Decoder.

The advice of returning concrete types is paired with defining interfaces when you need them on the consumer side.

It's returning interfaces that prevents good evolution, since the standard library will not add methods to interfaces, it can only document things like: all current standard library implementations additionally satisfy XXX interfaces.

replies(1): >>45667807 #
14. formerly_proven ◴[] No.45667431{3}[source]
> the return values aren't simple structs but slices or maps because []x is not a []X even if x implements X.

I assume this is because on is an array of struct pointers and the other is an array of fat pointers, since Go has reified interfaces (unlike higher-level languages).

15. LandR ◴[] No.45667506{4}[source]
Yep.

The lack of features means all the complexity is offloaded to the programmer. Where other languages can take some of the complexity burden off the programmer.

Go isn't simple, it's basic.

replies(1): >>45667557 #
16. amelius ◴[] No.45667557{5}[source]
Perhaps Go is a nice target language for a transpiler, so you could still benefit from the runtime and ecosystem while fixing the bugs in the language itself. Anyone working on this?
replies(1): >>45669862 #
17. aatd86 ◴[] No.45667583{3}[source]
Oh that's probably doable. Introducing something like this is a bit orthogonal to the point above, but yes.

It's not straightforward but probably something that will be considered at some point I reckon when thinking about making union interfaces first class. That will require to track a not nil typestate/predicate in the backend, something like that I guess.

replies(1): >>45667667 #
18. bpicolo ◴[] No.45667624{3}[source]
For 1/, you can return a struct value type without exporting it. If it satisfies the receiving interface they won’t have a problem.

That’s exactly the pattern I use for most Go development

replies(1): >>45672442 #
19. rowanseymour ◴[] No.45667667{4}[source]
Having pondered on a bit more.. I think it's the struct that would declare that it's not usable as nil, and that in turn would tell the runtime not to box it if it's nil. That would also help the compiler (and copilot etc) spot calls on nil pointers which will panic.
replies(1): >>45668969 #
20. Magnolia9438 ◴[] No.45667694[source]
On the happy path downstream, yes, and it does work really well. But the error flow back upstream flips that, as errors are returned as, often nested, interfaces.

This is fine for a lot of general purpose code that exits when running into problems. But when errors are an expected part of a long lived process, like an API, it’s painful to build logic around and conditionally handle them.

The ergonomics of errors.Is and As are pretty bad and there doesn’t seem to be a clear indication as when to expect a sentinel, concrete, or pointer to a concrete error.

All that to say, I think Go’s errors really illustrate the benefit of “return values, not interfaces”. Though for errors specifically, I’m not sure you could improve them without some pretty bad tradeoffs around flexibility.

21. lenkite ◴[] No.45667807{5}[source]
It seems there is a dichotomy in the real implemented world and your hypothetical advice world.

Due to lack of native support of defaults for optional methods , many interfaces in Go are using hacks for optional methods added by evolution.

The Value interface has a `IsBoolFlag()` optional method not part of the interface signature

The other way for evolution is just add sub-interfaces. Like `io.WriterTo` and `io.ReaderFrom` which are effectively just extensions of `io.Writer` and `io.Reader` with `WriteTo` and `ReadFrom` methods - which are checked for in consumers like `io.Copy`.

Anyways, my point was specifically about generic interfaces and alternative implementations, so it appears you agree.

22. leetrout ◴[] No.45667824{4}[source]
Like anything else there are exceptions to the rule. Pointing to the standard library is a weak position because it is consistently inconsistent.

Go's standard library interfaces (like net.Conn) earned their place.

Premature interfaces calcify mistakes and that's what the guideline pushes back on.

23. kbolino ◴[] No.45668625[source]
> In all other cases, like any(2) == 2, we compare the values.

But any(nil) == nil returns true like you'd expect.

The reason that any((*int)(nil)) == nil is false is the same reason that any(uint(2)) == 2 is false: interfaces compare values and types.

replies(1): >>45668792 #
24. aatd86 ◴[] No.45668792{3}[source]
that's another thing that makes it difficult to fix. Same thing here. 2 is an untyped constant so it should have returned true. (even if int is the default picked on short assignment)

any(uint(2)) == int(2) should return false indeed however.

replies(1): >>45669545 #
25. aatd86 ◴[] No.45668969{5}[source]
But that information disappears when you assign to an interface variable/container that is nillable. It requires an assertion to recover the info about the value inside the interface being not nil.

basically `if v.(nil){...}

creates two branches. In one we know v is not nil (outside the if block) and it can therefore be assigned to non nillable variables so to speak...

26. thomashabets2 ◴[] No.45669163[source]
Be prepared to be called a newbie for criticising typed nils.

My post https://news.ycombinator.com/item?id=44982491 got a lot of hate from people who defend Go by saying "so just don't do that!", and people trying to explain my own blog post to me.

27. kbolino ◴[] No.45669545{4}[source]
Untyped constants deserve an entry of their own in a list of the language's subtleties, that's for sure.

Importantly, untyped constants don't exist at runtime, and non-primitive types like interfaces aren't constants, so any(uint(2)) == 2 can't behave the way you want without some pretty significant changes to the language's semantics. Either untyped constants would have to get a runtime representation--and equality comparisons would have to introduce some heavyweight reflection--or else interfaces would have to be hoisted into the constant part of the language--which is quite tricky to get right--and then you just end up in a situation where any(uint(2)) == 2 works but x == 2 doesn't when x turns out to be any(uint(2)) at runtime.

replies(1): >>45670011 #
28. laumars ◴[] No.45669557{4}[source]
As someone who's written commercial software in well over a dozen different languages for nearly 40 years, I completely disagree.

Go has its warts for sure. But saying the simplicity of Go is "just virtue signaling" is so far beyond ignorant that I can only conclude this opinion of yours is nothing more than the typical pseudo-religious biases that lesser experienced developers smugly cling to.

Go has one of the easiest tool chains to get started. There's no esconfig, virtualenv and other bullshit to deal with. You don't need a dozen `use` headers just to define the runtime version nor trust your luck with a thousand dependencies that are impossible to realistically audit because nobody bothered to bundle a useful standard library with it. You don't have multi-page indecipherable template errors, 50 different ways to accomplish the same simple problem nor arguments about what subset of the language is allowed to be used when reviewing pull requests. There isn't undefined behaviour nor subtle incompatibilities between different runtime implementations causing fragmentation of the language.

The problem with Go is that it is boring and that's boring for developers. But it's also the reason why it is simple.

So it's not virtue signaling at all. It's not flawless and it's definitely boring. But that doesn't mean it isn't also simple.

Edit: In case anyone accuses me of being a fanboy, I'm not. I much preferred the ALGOL lineage of languages to the B lineage. I definitely don't like a lot of the recent additions to Go, particularly around range iteration. But that's my personal preference.

replies(1): >>45669909 #
29. bobbylarrybobby ◴[] No.45669862{6}[source]
https://github.com/borgo-lang/borgo
30. bobbylarrybobby ◴[] No.45669909{5}[source]
You are comparing Go to Python, JS, and C++, arguably the three most complex languages to build. (JS isn't actually hard, but there are a lot of seemingly arbitrary decisions that have to be made before you can begin.) There are languages out there that are easy to build, have a reasonable std lib, and don't offload the complexity of the world onto the programmer.
replies(1): >>45670373 #
31. aatd86 ◴[] No.45670011{5}[source]
Not sure that reflection would be needed. They are exclusively on the RHS. But you're right. They would have a sort of type of their own instead of basically being int under the hood. type conversions do not require reflection. Or maybe you are thinking about something I have overlooked? In any case, not very likely a change anyway.
replies(1): >>45670884 #
32. laumars ◴[] No.45670373{6}[source]
> You are comparing Go to Python, JS, and C++, arguably the three most complex languages to build.

No, I'm comparing to more than a dozen different languages that I've used commercially. And there were direct references there to Perl, Java, Pascal, procedural SQL, and many, many others too.

> There are languages out there that are easy to build, have a reasonable std lib

Sure. And the existence of them doesn't mean Go isn't also simple.

> and don't offload the complexity of the world onto the programmer.

I disagree. Every language makes tradeoffs, and those tradeoffs always end up being complexities that the programmer has to negotiate. This is something I've seen, without exception, in my 40 years of language agnosticism and part-time language designer.

33. kbolino ◴[] No.45670884{6}[source]
Let's assume the runtime representation case, as it's the most flexible. You'd need to do an assignability check to compare it to a typed number. Keep LHS as the interface, and RHS as the untyped constant.

That means following the type pointer of LHS, switching on its underlying type (with 15 valid possibilities [1]) or similar, and then casting either RHS to LHS's type, or LHS to the untyped representation, and finally doing the equality check. Something like this (modulo choice of representation and possible optimizations):

  import ("math/big"; "reflect")
  type untypedInt struct { i *big.Int }
  func (x untypedInt) equals(y any) bool {
    val := reflect.ValueOf(y)
    if val.Type() == reflect.TypeOf(x) {
      return x.i.Cmp(val.Interface().(untypedInt).i) == 0
    } else if val.CanInt() {
      if !x.i.IsInt64() { return false }
      return x.i.Int64() == val.Int()
    } else if val.CanUint() {
      if !x.i.IsUint64() { return false }
      return x.i.Uint64() == val.Uint()
    } else {
      var yf float64
      if val.CanFloat() {
        yf = val.Float()
      } else if val.CanComplex() {
        yc := val.Complex()
        if imag(yc) != 0 { return false }
        yf = real(yc)
      } else { return false }
      xf, acc := x.i.Float64()
      if acc != big.Exact { return false }
      return xf == yf
    }
  }
[1]: Untyped integer constants can be compared with any of uint8..uint64, int8..int64, int, uint, uintptr, float32, float64, complex64, or complex128
replies(1): >>45671471 #
34. jerf ◴[] No.45670974{3}[source]
It's not that the ship has sailed, it is that if you sit down and sketch out what people think they want it is logically incoherent. What Go does is the logically-coherent result of the way interfaces work and the fact that "nil" values are not invalid. It is perfectly legal for a "nil" pointer to validly implement an interface. For instance, see https://go.dev/play/p/JBsa8XXxeJP , where a nil pointer of "*Repeater" is a completely valid implementation of the io.Reader interface; it represents the "don't repeat anything at all" value.

In light of that fact, it would cause the interface rules to grow a unique wart that doesn't accomplish anything if interfaces tried to ban putting "nil" pointers into them. The correct answer is to not to create invalid values in the first place [1] and basically "don't do that", but that's not a "don't do that because it ought to do what you think and it just doesn't for some reason", it's a "don't do that because what you think should happen is in fact wrong and you need to learn to think the right thing".

Interfaces can not decide to not box nil values, because interfaces are not supposed to "know" what is and is not a legal value that implements them. It is the responsibility of the code that puts a value into the interface to ensure that the value correctly implements the interface. Note how you could not have io.Reader label itself as "not containing a nil" in my example above, because io.Reader has no way to "know" what my Repeater is. The job of an io.Reader value is to Read([]byte) (int error), and if it can't do that, it is not io.Reader's "fault". It is the fault of the code that made a promise that some value fits into the io.Reader interface when it doesn't.

In Go, nil is not the same thing as invalid [2] and until you stop forcing that idea into the language from other previous languages you've used you're going to not just have a bad time here, but elsewhere as well, e.g., in the behavior of the various nil values for slice and map and such.

One can more justifiably make the complaint that there is often no easy way to make a clearly-invalid value in Go the way a sum type can clearly declare an "Invalid/None/Empty/NULL", or even declare multiple such values in a single type if the semantics call for it, but that's a separate issue and doesn't make "nil" be the invalid value in current Go. Go does not have a dedicated "invalid" value, nor does it have a value of a given type that methods can not be called on.

(You can also ask for Go to have more features that make it harder to stick invalid values into an interface, but if you try to follow that to the point where it is literally impossible, you end up in dependently-typed languages, which currently have no practical implementations. Nothing can prevent you, in any current popular language, from labelling a bit of code as implementing an interface/trait/set of methods and simply being wrong about that fact. So it's all a question of where the tradeoffs are in the end, since "totally accurately correct interfaces" are not currently known to even be possible.)

[1]: https://jerf.org/iri/post/2957/

[2]: https://jerf.org/iri/post/2023/value_validity/

replies(2): >>45671338 #>>45673235 #
35. rowanseymour ◴[] No.45671338{4}[source]
I don't know why every time people complain about this there is an assumption that we just don't understand why it is the way it is. I get that x can implement X and x can have methods that work with nil. I sometimes write methods that work with nils. It's a neat feature.

What's frustrating is that 99.99% of written go code doesn't work this way and so people _do_ shoot themselves in the foot all the time, and so at some point you have to concede that what we have might be logical but it isn't intuitive. And that kinda sucks for a language that prides itself on simplicity.

I also get that there's no easy way to address this. The best I can imagine is a way to declare that a method Y on type x can't take nil so (*x)(nil) shouldn't be considered as satisfying that method on an interface.. and thus not boxed automatically into that interface type. But yeah I get that's gonna get messy. If I could think of a good solution I'd make a proposal.

replies(1): >>45671687 #
36. aatd86 ◴[] No.45671471{7}[source]
If it is because of overflow, the idea was that there could be size classes at compile time. A bit like sub/supertyping but for numeric types. A simple type pointer check would be sufficient.
replies(1): >>45671725 #
37. jerf ◴[] No.45671687{5}[source]
Because in the last dozen times I've handled this question the root cause is lack of understanding of why. Inductively it is logical to conclude that's the reason next time. It is probably also the case the bulk of readers of this conversation are still in the camp that don't understand the problem correctly.

If you understand that there isn't really a fix and just wish there was one anyhow, while I still disagree in some details it's in the range I wouldn't fuss about. I understand that sort of wishing perfectly; don't think there's ever been a language I've used for a long time that I've had similar sorts of "I just wish it could work this way even though I understand why it can't." Maybe someday we'll be "blessed" with some sort of LLM-based language that can do things like that... for better or for worse.

replies(1): >>45671946 #
38. kbolino ◴[] No.45671725{8}[source]
Size classes would save some space and speed up like-to-like comparisons, but wouldn't really do much for unlike comparisons (especially vs. float or complex). Looking only at type pointers fails to account for custom types (e.g., type Foo int); remember that an untyped integer constant can be compared with these. If you want the same semantics at runtime as you get at compile time, I don't see how you can get much simpler than what I wrote, in terms of the high-level logic. Though there are undoubtedly ways to optimize it, both because Go's compiler favors speed of compilation over efficiency of generated code, and because if this were the real code, it could poke at internals while my (probably working) example has to rely on the public reflect package, which is more abstract.
replies(1): >>45673035 #
39. rowanseymour ◴[] No.45671946{6}[source]
I can't think of good way to give programmers control over boxing without adding a bunch of complexity that nobody wants.. but it doesn't seem out of the realm of possibility that the linter could detect issues like this. It should be able to spot methods that aren't nil-safe and spot nil values of those types ending up in interfaces with those methods. Then you'd have less explaining to do!
replies(1): >>45677465 #
40. kbolino ◴[] No.45672442{4}[source]
This affects discoverability, though. Your unexported type won't have public documentation. So you end up having to publish an interface anyway (even if you don't return it) or document in words what the method set looks like.
41. aatd86 ◴[] No.45673035{9}[source]
But the LHS can determine how it compares to the RHS when the RHS is determined to be an untyped constant? Or instead of saying RHS (my mistake), let's say the typed side since comparisons are symmetric. A bit like having a special method attached to the type strictly for comparisons? That would be much less expensive than such a type switch if I am not mistaken. Would handle custom types as well. If promotable from the underlying type, that would not even bloat the executable. Unless I'm confused...
replies(1): >>45673527 #
42. ngrilly ◴[] No.45673235{4}[source]
Your blog posts (that I read a few weeks ago) and your comment here are the best explanations I've ever read on this topic. You're not just looking at the surface of the problem, but diving in the why it is like that, semantically. I really like that you mentioned dependent typing in your conclusion.
43. kbolino ◴[] No.45673527{10}[source]
Ok, I think I follow. Instead of putting the comparison logic on the untyped side, you'd put it on the typed side. In code, reusing imports and untypedInt declaration, but replacing the method from before, you'd have:

  type intType interface { ~int | ~int8 | ~int16 | ~int32 | ~int64 }
  func equals[I intType](x I, y any) bool {
    switch val := y.(type) {
    case I: return x == val
    case untypedInt: return val.i.IsInt64() && val.i.Int64() == int64(x)
    default: return false
    }
  }
And this would need a separate specialization for unsigned integers, floats, and complex numbers. This approach saves us from having to introspect the underlying type at runtime, but the example is incomplete. We also have float and complex untyped constants, so now each concrete type has to switch on all of the untyped constant forms it compares with. Still, it might be faster, though I'm not sure how much it reduces code bloat in practice (it's nice to not need the reflect package though).

[edit: side note, I was trying to actually write out all the code that would be needed, and I discovered that you can't call real or imag on generic types: https://github.com/golang/go/issues/50937]

44. ignoramous ◴[] No.45673702{4}[source]
> because it’s not actually simple

Cue Rich Hickey's Simple made Easy: https://www.youtube-nocookie.com/embed/SxdOUGdseq4 / https://ghostarchive.org/varchive/SxdOUGdseq4

45. aatd86 ◴[] No.45677465{7}[source]
It's a bit difficult statically because it is a flow sensitive analysis.

You are not wrong that it is a sharp edge. Completely removing nils from interfaces is not possible because: 1. not backward compatible

However I would nuance a little. Having an empty interface ie. a untyped nil is useful. Having typed nils in interfaces is arguable. Because every value type that has methods can make pointer. That means potential deref if any such pointer is passed to an interface variable instead of the value itself.

Being able to keep nil from some interfaces would be useful.

You're not wrong. In general there is not much value in having working methods on a typed nil pointer.

If we think in terms of bottom wrt type theory, yes it is supposed to implement every type. But that would be closer to untyped nil and that's not how go's type system works either. It is close though. We just don't have a language concept for nillable int because variables are auto initialized to 0. And because it would be difficult to encode such an information purely virtually. But that could be possible in theory, without mechanical sympathy. I digress. The takeaway is that I don't think a linter can do the trick easily but there has been good attempts. And it is worth pondering, you're right.

replies(1): >>45679621 #
46. peterashford ◴[] No.45678512{3}[source]
I read & write Go regularly, and I hate its error handling intensely
47. aatd86 ◴[] No.45679621{8}[source]
"empty interface" meaning an interface value that is empty i.e. nil interface... ofc