←back to thread

Falsify: Hypothesis-Inspired Shrinking for Haskell (2023)

(www.well-typed.com)
90 points birdculture | 1 comments | 20 Apr 25 19:41 UTC | HN request time: 0.293s | source
Show context
mjw1007 ◴[20 Apr 25 22:51 UTC] No.43747119[source]▶
I've found in practice that shrinking to get the "smallest amount of detail" is often unhelpful.

Suppose I have a function which takes four string parameters, and I have a bug which means it crashes if the third is empty.

I'd rather see this in the failure report:

("ldiuhuh!skdfh", "nd#lkgjdflkgdfg", "", "dc9ofugdl ifugidlugfoidufog")

than this:

("", "", "", "")

replies(4): >>43747380 #>>43748889 #>>43749509 #>>43749569 #
gwern ◴[20 Apr 25 23:43 UTC] No.43747380[source]▶
Really? Your examples seem the opposite. I am left immediately thinking, "hm, is it failing on a '!', some sort of shell issue? Or is it truncating the string on '#', maybe? Or wait, there's a space in the third one, that looks pretty dangerous, as well as noticeably longer so there could be a length issue..." As opposed to the shrunk version where I immediately think, "uh oh: one of them is not handling an empty input correctly." Also, way easier to read, copy-paste, and type.
replies(3): >>43747816 #>>43749072 #>>43749633 #
dullcrisp ◴[21 Apr 25 01:27 UTC] No.43747816[source]▶
Their point is that in the unshrunk example the “special” value stands out.

I guess if we were even more clever we could get to something more like (…, …, "", …).

replies(2): >>43747837 #>>43748112 #
1. gwern ◴[21 Apr 25 01:33 UTC] No.43747837[source]▶
The special value doesn't stand out, though. All three examples I gave were what I thought skimming his comment before my brain caught up to his caveat about an empty third argument. The empty string looked like it was by far the most harmless part... Whereas if they are all empty strings, then by definition the empty string stands out as the most suspicious possible part.