Production RAG: what I learned from processing 5M+ documents

My biggest RAG learning is to use agentic RAG. (Sorry for buzzword dropping)

- Classic RAG: `User -> Search -> LLM -> User`

- Agentic RAG: `User <-> LLM <-> Search`

Essentially instead of having a fixed loop, you provide the search as a tool to the LLM, which does three things:

- The LLM can search multiple times

- The LLM can adjust the search query

- The LLM can use multiple tools

The combination of these three things has solved a majority of classic RAG problems. It improves user queries, it can map abbreviations, it can correct bad results on its own, you can also let it list directories and load files directly.