←back to thread

Production RAG: what I learned from processing 5M+ documents

(blog.abdellatif.io)
548 points tifa2up | 2 comments | 20 Oct 25 15:55 UTC | HN request time: 0.482s | source
Show context
pietz ◴[20 Oct 25 18:16 UTC] No.45647213[source]▶
I find it interesting that so many services and tools were investigated except for embedding models. I would have thought that's one of the biggest levers.
replies(2): >>45647238 #>>45656990 #
1. Trias11 ◴[20 Oct 25 18:18 UTC] No.45647238[source]▶
they just grabbed the better one (3-large) right off the bat. 6x cost to 3-small, but it's still tiny.
replies(1): >>45649644 #
2. pietz ◴[20 Oct 25 21:31 UTC] No.45649644[source]▶
But the model is like 18 months old. and recently we've seen big leaps on MTEB. Not sure how well those translate to reality, but I'm a little surpised this wasn't worth looking into.