←back to thread

Can LLMs accurately recall the Bible?

(benkaiser.dev)
223 points benkaiser | 3 comments | 29 Dec 24 03:44 UTC | HN request time: 0.881s | source
Show context
danpalmer ◴[30 Dec 24 03:18 UTC] No.42546115[source]▶
LLMs are bad databases, so for something like a bible which is so easily and precisely referenced, why not just... look it up?

This is playing against their strengths. By all means ask them for a summary, or some analysis, or textual comparison, but please, please stop treating LLMs as databases.

replies(2): >>42546183 #>>42546545 #
1. mcswell ◴[30 Dec 24 04:42 UTC] No.42546545[source]▶
A year or so ago, there was a complaint from the NY Times (IIRC) that by asking about some event, they were able to get back one of their articles almost verbatim--and alleging that this was a copyright violation. This appears to be a similar outcome, where you do get back the verbatim text. That to me is a good reason to do tests like this, although feel free to do it with the WaPo or some other news outlet instead.
replies(1): >>42547257 #
2. mmooss ◴[30 Dec 24 07:18 UTC] No.42547257[source]▶
That is an essential point: Can you ask the application to return other copyright content, such as from books, transcripts, etc.?

If so, what happens to their IP risks?

replies(1): >>42547756 #
3. fph ◴[30 Dec 24 09:10 UTC] No.42547756[source]▶
There is a well-known case from 2021 where Copilot returned Quake's famous `invsqrt` code (GPL licensed) from a minimal prompt: https://news.ycombinator.com/item?id=27710287 .