Where the type system isn't that expressive, you still have to fall back to testing to fill in the places where the type system isn't sufficient, and your tests are also going to 'deal with "what if this input is null?" everywhere' cases and whatnot by virtue of execution constraints anyway.
I'm not sure I agree with your prior that "your tests are also going to 'deal with "what if this input is null?" everywhere' cases and whatnot." Invalid input is at the very edge of the program where it goes through a parser. If I parse a string value into a type with no room for null, that eliminates a whole class of errors throughout the program. I do need to test that my parser does what I assume it will, sure. But once I have a type that cannot represent illegal data, I can focus my testing away from playing defense on every function.
You're asking about a circumstance that's just very very different from the one I'm in.
You seem to misunderstand. The question was centred around the fact that unexpected null cases end up being tested by virtue of you covering normal test cases due to the constraints on execution. Explicitly testing for null is something else. Something else I suggest unnecessary — at least where null is not actually part of the contract, which for the purposes of our discussion is the case. Again, we're specifically talking about the testing that is necessary to the extent of covering what is missing in the type system when you don't have a formal proof language in hand.
> You're asking about a circumstance that's just very very different from the one I'm in.
But one you've clearly had trouble with in the past given your claims that you weren't able to deal with it. Otherwise, how would you know?
Perhaps I can phrase my request in another way: If null isn't expected in your codebase, and your testing over where the type system is lacking has covered your bases to know that the behaviour is as documented, where are the errant nulls going to magically appear from?
I can confirm that at least 30% of the prod alerts I've seen come from NullReferenceExceptions. I insist on writing new C# code with null-checking enabled which mostly solves the problem, but there's still plenty of code in the wild crashing on null bugs all the time.
Of those who are concerned about type theory? 99%. With a delusional 1% thinking that a gimped type system (read: insufficient for formal proofs) is some kind of magic that negates the need to write tests, somehow not noticing that many of the lessons on how to write good tests come from those language ecosystems (e.g. Haskell).
> I can confirm that at least 30% of the prod alerts I've seen come from NullReferenceExceptions.
I don't think I've ever seen a null exception (or closest language analog) occur in production, and I spent a lot of years involved in projects that used dynamically typed languages even. I'd still love for someone to show actual code and associated tests to see how they ended up in that situation. The other commenter, despite being adamant, has become avoidant when it comes down to it.
The "how" is almost always lack of discipline (or as I sometimes couch it, "imagination") but usually shit like https://github.com/microsoft/SynapseML/issues/405#:~:text=cl...
I've had to learn in my career not to open other people's projects in a real IDE because of all the screaming it does about "value can be null"
Same thing on the front end in JS: change the shape of some record that has more places using it than I could remember. Better tests would have caught these. A compiler would be even better.
FWIW I’ve written a lot of tests of code written in all of the languages I like. You absolutely need tests, you just don’t need them to be as paranoid when writing them.
Right, but the condition here is that the languages that are expressive enough to negate the need for testing are, shall we say, unusable. In the real world people are going to be using, at best, languages with gimped type systems that still require testing to fill in the gaps.
Given that, we're trying to understand your rejection of the premise that the tests you will write to fill in those gaps will also catch things like null exceptions in the due course of execution. It remains unclear where you think these errant nulls are magically coming from.
I'm not convinced "Didn’t help that we were testing by hand.", "Better tests would have caught these." is that rejection. Those assertions, while no doubt applicable to your circumstances, is kind of like saying that static type systems don't help either because you can write something like this:
data MaybeString = Null | Str String
len :: MaybeString -> Int
len (Str s) = length s
main :: IO ()
main = do
print (len Null) -- exception thrown
There is admittedly a lot in the update and I may have simply missed it, but I don't see any modifications, even additions, to tests to denote recognition of the problem in the earlier version. Which, while not knowing much about the project, makes me think that there really isn't any meaningful testing going on. That maybe be interesting for what it is, I suppose, but not really in the vein of the discussion here about where one is using type systems and testing to overcome their personal limitations.
More careful use of the type system might have caught the null pointer case specifically, but the compiler still wouldn't have caught that they were doing the wrong thing beyond the null pointer error. In other words, the failure would have still occurred due to the next problem down the line from a problem that is impossible to catch with a partial type system.
While a developer full of hubris who thinks they can do no wrong may let that slide, our discussion is specifically about a developer who fully understands his personal limitations. He recognizes that he needs the machine to hold his hand. While that includes leveraging types, he understands that the type system in any language he will use in the real world isn't expressive enough to cover all of the conditions necessary. Thus he will also write tests to fill in the gaps.
Now, the premise proposed was that once you write the tests to cover that behaviour in order to fill in the gaps where the type system isn't expressive enough, you naturally also ensure that you don't end up with things like null pointer cases by way of the constraints of test execution. The parent rejected that notion, but it remains unclear why, and we don't yet have a concrete example showing how errant nulls "magically" appear under those conditions.
"I don't need testing" isn't the same thing, and has nothing to do with the discussion taking place here.
To head off (hah) discussion of taking the head of [], we used a prelude where head returned a Maybe. As far as I know, there were no incomplete functions in the prelude. https://hackage.haskell.org/package/nri-prelude
Yeah, when you have extra tools like that it can certainly help. The thing is that you can ignore any warning! I like it to be a compiler error because then there's no way to put that on the tech-debt credit card and still pass CI. If you are able to put those warnings into your CI so a PR with them cannot merge, then that's like 99% of what I like in my code: make the computer think of the low-hanging-fruit things that can go wrong.
With all of that said, solving for null does not get you everything a tagged union type does, but that's a different story.
"Better flag choices would have caught it" is a poor take given what came before. Of course that's true, but the same as your "better tests would have caught it". However, that really has nothing to do with our discussion.
Again, the premise here is that you are writing tests to close the gaps where the type system is insufficient to describe the full program. Of course you are as the astute observation of "My hair is too shiny and my muscles are too large to have to deal with [error prone things] everywhere. Can't do it." is true and valid. Null checks alone, or even all type checks alone, are not sufficient to satisfy all cases of [error prone things]. That is at least outside of formal proof languages, but we already established that we aren't talking about those.
So... Given the tests you are writing to fill in those gaps (again, not tests specifically looking for null pointer cases, but the normal tests you are writing), how would null pointer cases slip through, even if the compiler didn't notice? What is the concrete example that demonstrates how that is possible?
Because frankly I have no idea how it could be possible and I am starting to think your rejection was just made up on the spot and thrown out there without you giving any thought, or perhaps reiterating some made up nonsense you read elsewhere without considering if it were valid? It is seemingly telling that every attempt to dismiss that idea has devolved into bizarre statements along the lines of "well, if you don't write tests then null pointer exceptions might make it into production" even though it is clear that's not what we are talking about.