What do other HNers make out of this? Would you use this? Responsible for a legaltech startup here.
replies(2):
If you want a transformational shift in terms of accuracy and reasoning, the answer is different. Many a times RAG accuracy suffers because the text is out of distribution, and ICL does not work well. You get away with it if all your data is in public domain in some form (ergo, llm was trained on it), else you keep seeing the gaps with no way to bridge them. I published a paper around it and how to effciently solve it, if interested. Here is a simplified blog post on the same: https://medium.com/@ankit_94177/expanding-knowledge-in-large...
Edit: Please reach out here or on email if you would like further details. I might have skipped too many things in the above comment.