←back to thread

Production RAG: what I learned from processing 5M+ documents

(blog.abdellatif.io)
548 points tifa2up | 1 comments | 20 Oct 25 15:55 UTC | HN request time: 0.207s | source
Show context
captainregex ◴[21 Oct 25 03:31 UTC] No.45652171[source]▶
How much of a hit would you take on quality if you moved the processing local? have you experimented with it? don’t think llamaindex has local sadly
replies(1): >>45653478 #
1. tifa2up ◴[21 Oct 25 07:51 UTC] No.45653478[source]▶
Quite a decent hit. Local models don't perform very well in long contexts. We're planning to support a local-only offline set-up for people to host w/o additional dependencies