←back to thread

Will We Run Out of Data? Limits of LLM Scaling Based on Human-Generated Data

(epochai.org)
56 points trott | 1 comments | 18 Jun 24 02:04 UTC | HN request time: 0.215s | source
Show context
makapuf ◴[18 Jun 24 06:52 UTC] No.40714795[source]▶
Funny that it does not need that much data to train your average 20th century human genius. I'd say that if we are dreaming of the future of ai, learning and reasoning seems the greatest issue, not data. That said, the article title is about LLMs, so that's what will need changing I guess.
replies(3): >>40715430 #>>40715643 #>>40716666 #
1. verdverm ◴[18 Jun 24 09:25 UTC] No.40715643[source]▶
This has become my thinking as well. There is plenty of data for humans, so this is really a hardware / algos problem, not a data one. Using lots of text and energy is just a step on the path. We are already moving on to text+image and not long until video and later robots (with sensors) too