←back to thread

Production RAG: what I learned from processing 5M+ documents

(blog.abdellatif.io)
548 points tifa2up | 1 comments | 20 Oct 25 15:55 UTC | HN request time: 0.211s | source
Show context
leetharris ◴[20 Oct 25 17:05 UTC] No.45646303[source]▶
Embedding based RAG will always just be OK at best. It is useful for little parts of a chain or tech demos, but in real life use it will always falter.
replies(6): >>45646470 #>>45646482 #>>45646495 #>>45646758 #>>45646892 #>>45656450 #
1. phillipcarter ◴[20 Oct 25 17:50 UTC] No.45646892[source]▶
Not necessarily? It's been the basis of one of the major ways people would query their data since 2023 on a product I worked on: https://www.honeycomb.io/blog/introducing-query-assistant

The difference is this feature explicitly isn't designed to do a whole lot, which is still the best way to build most LLM-based products and sandwich it between non-LLM stuff.