←back to thread

223 points benkaiser | 2 comments | | HN request time: 0.413s | source
Show context
ChuckMcM ◴[] No.42544992[source]
Interesting that it takes an LLM with 405 BILLION parameters to accurately recall text from a document with slightly less than 728 THOUSAND words. (not quite three decimal orders of magnitude smaller but still).
replies(3): >>42545195 #>>42545228 #>>42545577 #
1. Sabinus ◴[] No.42545195[source]
I don't think it's necessarily about the parameter count, but the amount of training material about the Bible relative to the rest of the training material, with higher parameter models able to retain more Bible information with a higher proportion of training on other topics.
replies(1): >>42545225 #
2. ChuckMcM ◴[] No.42545225[source]
I would be interested to hear your thoughts on what a parameter in an LLM model represents.