←back to thread

Can LLMs accurately recall the Bible?

(benkaiser.dev)
223 points benkaiser | 1 comments | 29 Dec 24 03:44 UTC | HN request time: 0s | source
Show context
Animats ◴[29 Dec 24 20:52 UTC] No.42542976[source]▶
It's discouraging that an LLM can accurately recall a book. That is, in a sense, overfitting. The LLM is supposed to be much smaller than the training set, having in some sense abstracted the training inputs.

Did they try this on obscure bible excerpts, or just ones likely to be well known and quoted elsewhere? Well known quotes would be reinforced by all the copies.

replies(4): >>42543124 #>>42543534 #>>42544514 #>>42545640 #
1. bluGill ◴[30 Dec 24 01:55 UTC] No.42545640[source]▶
The bible is probably in enough different training sets (not just in whole, various papers making some religious argument that quote a few verses to make their point) that the model should have most of the bible.