So they trained LLM's on a bunch of junk and then notice that it got worse? I don't understand how that's a surprising, or even interesting result?
replies(3):
just use a different model?
dont train it with bad data and just start a new session if your RAG muffins went off the rails?
what am I missing here
I guess I don't actually have an issue with this research paper existing, but I do have an issue with its clickbait-y title that gets it a bunch of attention, even though the actual research is really not that interesting.