←back to thread

223 points benkaiser | 1 comments | | HN request time: 0.225s | source
Show context
Animats ◴[] No.42542976[source]
It's discouraging that an LLM can accurately recall a book. That is, in a sense, overfitting. The LLM is supposed to be much smaller than the training set, having in some sense abstracted the training inputs.

Did they try this on obscure bible excerpts, or just ones likely to be well known and quoted elsewhere? Well known quotes would be reinforced by all the copies.

replies(4): >>42543124 #>>42543534 #>>42544514 #>>42545640 #
1. amelius ◴[] No.42544514[source]
This is actually a good point.

Are reciting and not-overfitting at odds?