Most active commenters
  • aatd86(9)
  • rowanseymour(5)
  • kbolino(5)

←back to thread

Go subtleties

(harrisoncramer.me)
234 points darccio | 21 comments | | HN request time: 0.001s | source | bottom
Show context
rowanseymour ◴[] No.45666258[source]
Ah the old nil values boxed into non-nil interfaces. Even after 8 years writing go code almost every day this still bites me occasionally. I've never seen code that actually uses this. I understand why it is the way it is but I hate it.
replies(4): >>45666614 #>>45666792 #>>45666859 #>>45669163 #
1. aatd86 ◴[] No.45666792[source]
Yes, that'a bit too late after ten+ years perhaps but I wished we had a nil type and checking whether the interface is empty was a type assertion. In all other cases, like any(2) == 2, we compare the values.

Then again that would mean that the nil identifier would be coerced into a typed nil and we would check for the nilness of what is inside an interface in any(somepointer) == nil.

wrt the current behavior, it also makes sense to have a nil value that remains untyped. But in many other cases we do have that automatic inference/coercion, for instance when we set a pointer to nil.(p = nil)

That's quite subtle and that ship has sailed though.

replies(2): >>45666878 #>>45668625 #
2. rowanseymour ◴[] No.45666878[source]
Agree the ship has likely sailed, but if it could be addressed wouldn't it be nice to remove nil value interfaces altogether? Maybe start by letting new interface types declare/annotate that they don't box nil values? Then one day that becomes the default. Oh well.
replies(2): >>45667583 #>>45670974 #
3. aatd86 ◴[] No.45667583[source]
Oh that's probably doable. Introducing something like this is a bit orthogonal to the point above, but yes.

It's not straightforward but probably something that will be considered at some point I reckon when thinking about making union interfaces first class. That will require to track a not nil typestate/predicate in the backend, something like that I guess.

replies(1): >>45667667 #
4. rowanseymour ◴[] No.45667667{3}[source]
Having pondered on a bit more.. I think it's the struct that would declare that it's not usable as nil, and that in turn would tell the runtime not to box it if it's nil. That would also help the compiler (and copilot etc) spot calls on nil pointers which will panic.
replies(1): >>45668969 #
5. kbolino ◴[] No.45668625[source]
> In all other cases, like any(2) == 2, we compare the values.

But any(nil) == nil returns true like you'd expect.

The reason that any((*int)(nil)) == nil is false is the same reason that any(uint(2)) == 2 is false: interfaces compare values and types.

replies(1): >>45668792 #
6. aatd86 ◴[] No.45668792[source]
that's another thing that makes it difficult to fix. Same thing here. 2 is an untyped constant so it should have returned true. (even if int is the default picked on short assignment)

any(uint(2)) == int(2) should return false indeed however.

replies(1): >>45669545 #
7. aatd86 ◴[] No.45668969{4}[source]
But that information disappears when you assign to an interface variable/container that is nillable. It requires an assertion to recover the info about the value inside the interface being not nil.

basically `if v.(nil){...}

creates two branches. In one we know v is not nil (outside the if block) and it can therefore be assigned to non nillable variables so to speak...

8. kbolino ◴[] No.45669545{3}[source]
Untyped constants deserve an entry of their own in a list of the language's subtleties, that's for sure.

Importantly, untyped constants don't exist at runtime, and non-primitive types like interfaces aren't constants, so any(uint(2)) == 2 can't behave the way you want without some pretty significant changes to the language's semantics. Either untyped constants would have to get a runtime representation--and equality comparisons would have to introduce some heavyweight reflection--or else interfaces would have to be hoisted into the constant part of the language--which is quite tricky to get right--and then you just end up in a situation where any(uint(2)) == 2 works but x == 2 doesn't when x turns out to be any(uint(2)) at runtime.

replies(1): >>45670011 #
9. aatd86 ◴[] No.45670011{4}[source]
Not sure that reflection would be needed. They are exclusively on the RHS. But you're right. They would have a sort of type of their own instead of basically being int under the hood. type conversions do not require reflection. Or maybe you are thinking about something I have overlooked? In any case, not very likely a change anyway.
replies(1): >>45670884 #
10. kbolino ◴[] No.45670884{5}[source]
Let's assume the runtime representation case, as it's the most flexible. You'd need to do an assignability check to compare it to a typed number. Keep LHS as the interface, and RHS as the untyped constant.

That means following the type pointer of LHS, switching on its underlying type (with 15 valid possibilities [1]) or similar, and then casting either RHS to LHS's type, or LHS to the untyped representation, and finally doing the equality check. Something like this (modulo choice of representation and possible optimizations):

  import ("math/big"; "reflect")
  type untypedInt struct { i *big.Int }
  func (x untypedInt) equals(y any) bool {
    val := reflect.ValueOf(y)
    if val.Type() == reflect.TypeOf(x) {
      return x.i.Cmp(val.Interface().(untypedInt).i) == 0
    } else if val.CanInt() {
      if !x.i.IsInt64() { return false }
      return x.i.Int64() == val.Int()
    } else if val.CanUint() {
      if !x.i.IsUint64() { return false }
      return x.i.Uint64() == val.Uint()
    } else {
      var yf float64
      if val.CanFloat() {
        yf = val.Float()
      } else if val.CanComplex() {
        yc := val.Complex()
        if imag(yc) != 0 { return false }
        yf = real(yc)
      } else { return false }
      xf, acc := x.i.Float64()
      if acc != big.Exact { return false }
      return xf == yf
    }
  }
[1]: Untyped integer constants can be compared with any of uint8..uint64, int8..int64, int, uint, uintptr, float32, float64, complex64, or complex128
replies(1): >>45671471 #
11. jerf ◴[] No.45670974[source]
It's not that the ship has sailed, it is that if you sit down and sketch out what people think they want it is logically incoherent. What Go does is the logically-coherent result of the way interfaces work and the fact that "nil" values are not invalid. It is perfectly legal for a "nil" pointer to validly implement an interface. For instance, see https://go.dev/play/p/JBsa8XXxeJP , where a nil pointer of "*Repeater" is a completely valid implementation of the io.Reader interface; it represents the "don't repeat anything at all" value.

In light of that fact, it would cause the interface rules to grow a unique wart that doesn't accomplish anything if interfaces tried to ban putting "nil" pointers into them. The correct answer is to not to create invalid values in the first place [1] and basically "don't do that", but that's not a "don't do that because it ought to do what you think and it just doesn't for some reason", it's a "don't do that because what you think should happen is in fact wrong and you need to learn to think the right thing".

Interfaces can not decide to not box nil values, because interfaces are not supposed to "know" what is and is not a legal value that implements them. It is the responsibility of the code that puts a value into the interface to ensure that the value correctly implements the interface. Note how you could not have io.Reader label itself as "not containing a nil" in my example above, because io.Reader has no way to "know" what my Repeater is. The job of an io.Reader value is to Read([]byte) (int error), and if it can't do that, it is not io.Reader's "fault". It is the fault of the code that made a promise that some value fits into the io.Reader interface when it doesn't.

In Go, nil is not the same thing as invalid [2] and until you stop forcing that idea into the language from other previous languages you've used you're going to not just have a bad time here, but elsewhere as well, e.g., in the behavior of the various nil values for slice and map and such.

One can more justifiably make the complaint that there is often no easy way to make a clearly-invalid value in Go the way a sum type can clearly declare an "Invalid/None/Empty/NULL", or even declare multiple such values in a single type if the semantics call for it, but that's a separate issue and doesn't make "nil" be the invalid value in current Go. Go does not have a dedicated "invalid" value, nor does it have a value of a given type that methods can not be called on.

(You can also ask for Go to have more features that make it harder to stick invalid values into an interface, but if you try to follow that to the point where it is literally impossible, you end up in dependently-typed languages, which currently have no practical implementations. Nothing can prevent you, in any current popular language, from labelling a bit of code as implementing an interface/trait/set of methods and simply being wrong about that fact. So it's all a question of where the tradeoffs are in the end, since "totally accurately correct interfaces" are not currently known to even be possible.)

[1]: https://jerf.org/iri/post/2957/

[2]: https://jerf.org/iri/post/2023/value_validity/

replies(2): >>45671338 #>>45673235 #
12. rowanseymour ◴[] No.45671338{3}[source]
I don't know why every time people complain about this there is an assumption that we just don't understand why it is the way it is. I get that x can implement X and x can have methods that work with nil. I sometimes write methods that work with nils. It's a neat feature.

What's frustrating is that 99.99% of written go code doesn't work this way and so people _do_ shoot themselves in the foot all the time, and so at some point you have to concede that what we have might be logical but it isn't intuitive. And that kinda sucks for a language that prides itself on simplicity.

I also get that there's no easy way to address this. The best I can imagine is a way to declare that a method Y on type x can't take nil so (*x)(nil) shouldn't be considered as satisfying that method on an interface.. and thus not boxed automatically into that interface type. But yeah I get that's gonna get messy. If I could think of a good solution I'd make a proposal.

replies(1): >>45671687 #
13. aatd86 ◴[] No.45671471{6}[source]
If it is because of overflow, the idea was that there could be size classes at compile time. A bit like sub/supertyping but for numeric types. A simple type pointer check would be sufficient.
replies(1): >>45671725 #
14. jerf ◴[] No.45671687{4}[source]
Because in the last dozen times I've handled this question the root cause is lack of understanding of why. Inductively it is logical to conclude that's the reason next time. It is probably also the case the bulk of readers of this conversation are still in the camp that don't understand the problem correctly.

If you understand that there isn't really a fix and just wish there was one anyhow, while I still disagree in some details it's in the range I wouldn't fuss about. I understand that sort of wishing perfectly; don't think there's ever been a language I've used for a long time that I've had similar sorts of "I just wish it could work this way even though I understand why it can't." Maybe someday we'll be "blessed" with some sort of LLM-based language that can do things like that... for better or for worse.

replies(1): >>45671946 #
15. kbolino ◴[] No.45671725{7}[source]
Size classes would save some space and speed up like-to-like comparisons, but wouldn't really do much for unlike comparisons (especially vs. float or complex). Looking only at type pointers fails to account for custom types (e.g., type Foo int); remember that an untyped integer constant can be compared with these. If you want the same semantics at runtime as you get at compile time, I don't see how you can get much simpler than what I wrote, in terms of the high-level logic. Though there are undoubtedly ways to optimize it, both because Go's compiler favors speed of compilation over efficiency of generated code, and because if this were the real code, it could poke at internals while my (probably working) example has to rely on the public reflect package, which is more abstract.
replies(1): >>45673035 #
16. rowanseymour ◴[] No.45671946{5}[source]
I can't think of good way to give programmers control over boxing without adding a bunch of complexity that nobody wants.. but it doesn't seem out of the realm of possibility that the linter could detect issues like this. It should be able to spot methods that aren't nil-safe and spot nil values of those types ending up in interfaces with those methods. Then you'd have less explaining to do!
replies(1): >>45677465 #
17. aatd86 ◴[] No.45673035{8}[source]
But the LHS can determine how it compares to the RHS when the RHS is determined to be an untyped constant? Or instead of saying RHS (my mistake), let's say the typed side since comparisons are symmetric. A bit like having a special method attached to the type strictly for comparisons? That would be much less expensive than such a type switch if I am not mistaken. Would handle custom types as well. If promotable from the underlying type, that would not even bloat the executable. Unless I'm confused...
replies(1): >>45673527 #
18. ngrilly ◴[] No.45673235{3}[source]
Your blog posts (that I read a few weeks ago) and your comment here are the best explanations I've ever read on this topic. You're not just looking at the surface of the problem, but diving in the why it is like that, semantically. I really like that you mentioned dependent typing in your conclusion.
19. kbolino ◴[] No.45673527{9}[source]
Ok, I think I follow. Instead of putting the comparison logic on the untyped side, you'd put it on the typed side. In code, reusing imports and untypedInt declaration, but replacing the method from before, you'd have:

  type intType interface { ~int | ~int8 | ~int16 | ~int32 | ~int64 }
  func equals[I intType](x I, y any) bool {
    switch val := y.(type) {
    case I: return x == val
    case untypedInt: return val.i.IsInt64() && val.i.Int64() == int64(x)
    default: return false
    }
  }
And this would need a separate specialization for unsigned integers, floats, and complex numbers. This approach saves us from having to introspect the underlying type at runtime, but the example is incomplete. We also have float and complex untyped constants, so now each concrete type has to switch on all of the untyped constant forms it compares with. Still, it might be faster, though I'm not sure how much it reduces code bloat in practice (it's nice to not need the reflect package though).

[edit: side note, I was trying to actually write out all the code that would be needed, and I discovered that you can't call real or imag on generic types: https://github.com/golang/go/issues/50937]

20. aatd86 ◴[] No.45677465{6}[source]
It's a bit difficult statically because it is a flow sensitive analysis.

You are not wrong that it is a sharp edge. Completely removing nils from interfaces is not possible because: 1. not backward compatible

However I would nuance a little. Having an empty interface ie. a untyped nil is useful. Having typed nils in interfaces is arguable. Because every value type that has methods can make pointer. That means potential deref if any such pointer is passed to an interface variable instead of the value itself.

Being able to keep nil from some interfaces would be useful.

You're not wrong. In general there is not much value in having working methods on a typed nil pointer.

If we think in terms of bottom wrt type theory, yes it is supposed to implement every type. But that would be closer to untyped nil and that's not how go's type system works either. It is close though. We just don't have a language concept for nillable int because variables are auto initialized to 0. And because it would be difficult to encode such an information purely virtually. But that could be possible in theory, without mechanical sympathy. I digress. The takeaway is that I don't think a linter can do the trick easily but there has been good attempts. And it is worth pondering, you're right.

replies(1): >>45679621 #
21. aatd86 ◴[] No.45679621{7}[source]
"empty interface" meaning an interface value that is empty i.e. nil interface... ofc