Can LLMs accurately recall the Bible?

(benkaiser.dev)

223 points benkaiser | 2 comments | 29 Dec 24 03:44 UTC | HN request time: 0.564s | source

Show context

asim ◴[29 Dec 24 06:22 UTC] No.42538054[source]▶

I had similar thoughts about using it for the Quran. I think this highlights you have to be very specific in your use cases especially when expecting an exact response on static text that shouldn't change. This is why I'm trying something a bit different. I've generated embeddings for the Quran and use chromem-go for this. So I'll ask the index the question first based on a similarity search and then feed the results in as context to an LLM. But in the response I'll still sight the references so I can see what they were. It's not perfect but a first step towards something. I think they call this RAG.

What I'm working on https://reminder.dev

replies(6): >>42538137 #>>42538188 #>>42538217 #>>42543891 #>>42545257 #>>42548243 #

1. aorth ◴[29 Dec 24 06:56 UTC] No.42538188[source]▶

>>42538054 #

Cool project! I asked some questions and got good answers. I was surprised to get results from the Hadith though, as I thought it would be restricted to the Quran and they are (as you know) not the same thing.

replies(1): >>42538372 #

2. asim ◴[29 Dec 24 07:41 UTC] No.42538372[source]▶

>>42538188 (TP) #

Thanks, yea I thought about separating the search but felt like when you're looking for that piece of knowledge it can extend to the hadith and it was important to include it. So much of the Quran is explained in the context of the hadith and prophet's life.

↑