The wall confronting large language models

Always fun to see a theoretical argument that something clearly already happening is impossible.

So where are the recent improvements in LLMs proportional to the billions invested?

Value for the money is not at issue in the paper!

Really? It sure seems like we're at the top of the S curve with LLMs. Wiring them up to talk the themselves as reasoning isn't scaling the core models, which have only made incremental gains for all the billions invested.

There's plenty more room to grow with agents and tooling, but the core models are only slightly bumping YoY rather than the rocketship changes of 2022/23.

I believe it is. They are saying that LLMs don't improve all that much from giving them more resources - and computing power (and input corpus size) is pretty proportional to money.

It's not about value it's about the stagnation while throwing compute at the problem.

Exactly.

> the core models are only slightly bumping YoY rather than the rocketship changes of 2022/23

From Anthropic's press release yesterday after raising another $13 billion:

"Anthropic has seen rapid growth since the launch of Claude in March 2023. At the beginning of 2025, less than two years after launch, Anthropic’s run-rate revenue had grown to approximately $1 billion. By August 2025, just eight months later, our run-rate revenue reached over $5 billion—making Anthropic one of the fastest-growing technology companies in history."

$4 billion increase in 8 months. $1 billion every two months.

They’re talking about model quality. I still think they’re wrong, but the revenue is only indirectly relevant.

And relevant to the summary of this paper, LLM incremental improvement doesn't really seem to include the described wall.

If work produced by LLMs forever has to be checked for accuracy, the applicability will be limited.

This is perhaps analogous to all the "self-driving cars" that still have to be monitored by humans, and in that case the self-driving system might as well not exist at all.

> There's plenty more room to grow with agents and tooling, but the core models are only slightly bumping YoY rather than the rocketship changes of 2022/23.

understandable. the real innovation was the process/technique underlying LLMs. the rest is just programmers automating it. similar happened with blockchain, everything after was just tinkering the initial idea