←back to thread

The wall confronting large language models

(arxiv.org)
170 points PaulHoule | 5 comments | 03 Sep 25 11:40 UTC | HN request time: 0.3s | source
Show context
dcre ◴[03 Sep 25 20:10 UTC] No.45119904[source]▶
Always fun to see a theoretical argument that something clearly already happening is impossible.
replies(2): >>45120040 #>>45120369 #
1. ahartmetz ◴[03 Sep 25 20:28 UTC] No.45120040[source]▶
So where are the recent improvements in LLMs proportional to the billions invested?
replies(1): >>45120367 #
2. dcre ◴[03 Sep 25 21:00 UTC] No.45120367[source]▶
Value for the money is not at issue in the paper!
replies(2): >>45120399 #>>45120659 #
3. ahartmetz ◴[03 Sep 25 21:03 UTC] No.45120399[source]▶
I believe it is. They are saying that LLMs don't improve all that much from giving them more resources - and computing power (and input corpus size) is pretty proportional to money.
4. 42lux ◴[03 Sep 25 21:35 UTC] No.45120659[source]▶
It's not about value it's about the stagnation while throwing compute at the problem.
replies(1): >>45120725 #
5. dcre ◴[03 Sep 25 21:45 UTC] No.45120725{3}[source]▶
Exactly.