←back to thread

LLMs understand nullability

(dmodel.ai)
169 points mattmarcus | 1 comments | 07 Apr 25 14:52 UTC | HN request time: 0s | source
Show context
tanvach ◴[07 Apr 25 17:34 UTC] No.43613913[source]▶
Dear future authors: please run multiple iterations and report the probability.

From: ‘Keep training it, though, and eventually it will learn to insert the None test’

To: ‘Keep training it, though, and eventually the probability of inserting the None test goes up to xx%’

The former is just horse poop, we all know LLMs generate big variance in output.

replies(1): >>43614733 #
1. aSanchezStern ◴[07 Apr 25 19:02 UTC] No.43614733[source]▶
If you're interested in a more scientific treatment of the topic, the post links to a technical report which reports the numbers in detail. This post is instead an attempt to explain the topics to a more general audience, so digging into the weeds isn't very useful.