Grok 3: Another win for the bitter lesson

This is a weird takeaway from the recent changes. Right now companies can scale because there's stupid amount of stupid money flowing into the AI craze, but that's going to end. Companies are already discovering the issues with monetising those systems. Sure, they can "let go" and burn the available cash, but the investors will come knocking finally. Since everyone figures out similar tech anyway, it's the people with most tech improvement experience that will be in the best position long term, while openai will be stuck trying to squeeze adverts and monitoring into their chat for cash flow.

Until we see progress slowing down, I don’t see venture capital disappearing in the race to ASI and beyond.

Adjustment: not progress, the hype.

People belive that LLM progress will become foundation of the future economy expansion the same way as microelectronics did. But for now there are few signs of that economic benefit from AI/LLM stuff. If one do math of what productivity increase tech should give in order to have positive ROI, one would be surprised how reality is far from feasibility of investments https://www.bruegel.org/working-paper/tension-between-explod.... Yes, anecdotally people tell stories how they can code twice/trice/ten times faster, or how they atumated their whole workflow or replaced support with LLM. But that's far not enough for AI investment feasibility in existing businesses (AI startups will flurish for a while on venture money). Also anecdotally there are many failed attempts to replace people with LLMs (like mcdonalds ordering which resulted in crazy orders).

So what we have is a hype on top of beliefs in progress as continious phenomena. But progress itself has slowed greately. Where are all breakthroughs which change our way of living? Pocket computers and consumer electronics (which is not a discovery rather an optimisation) and internet (also more about scaling than inventing) were the last. 3d printing, cancer treatment, robotics thought to be the new factors. Till AI/LLM. Now AIs/LLMs are the last resourt for believers in progress and technooptimists like Musk.

> So what we have is a hype on top of beliefs in progress as continious phenomena. But progress itself has slowed greately.

I think we can split the two.

a) I don't think there's anyone seriously doubting there's a lot of hype going around. But that's to be expected. Trust not the trust me bros, and so on. I think we agree here.

b) Progress hasn't slowed, and at the same time progress (in research) can stop tomorrow and we'd still have work to do for 20years, simply using what's on the table today. LLMs have solved "intent comprehension". RL seems to have solved "verifiable problems". The two can and will be used together to tackle "open ended questions". And together we'll see advancements in every aspect of our lives, on every repetitive mundane and boring task we have ever done.

> on every repetitive mundane and boring task we have ever done.

A lot of them could be solved with pre-ai things. Many were self-inflicted by people badly designing and approving existing processes. I really don't see how AI is going to get us out of this one. I've been paid money to automate a few existing self-inflicted repetitive, mundane and boring tasks and the companies that created them are not even interested in talking about solving that. Some work the opposite way - in the US the value keeps being extracted from the tax filing system for example, even though we know how to remove the issue itself.

It's weird to discuss how the AI will automate everything when we're actively fighting simplification right now.

>>Pocket computers and consumer electronics (which is not a discovery rather an optimisation)

Can you really describe a process like EUV lithography an optimisation? I mean it requires matter to be controlled and manipulated in a way that would have been regarded as pure science fiction 20 years ago. Also the material science that provided Gallium Nitride electronics in our communcation system is rather amazing. There are other things as well - I have an electric car with a battery that lets me travel for hundreds of kilometers, if I trusted it, it could take me there without me operating it (much). I know where I am in London and where I am going because of satellites in geosynchronous orbits and calculations that use relativity. Last year I got an mRNA vaccine, that's pretty new... and pretty pervasive. I've seen rockets that fly up into space and then turn around and come back down to land, and I spend my days talking face to face with people on the other side of the world. I've never met half of them, but they greet me as an old friend.

How is it that you can't see any of these wonders that have sprung into being in our life times?