←back to thread

LLM Inevitabilism

(tomrenner.com)
1611 points SwoopsFromAbove | 4 comments | | HN request time: 0s | source
Show context
lsy ◴[] No.44568114[source]
I think two things can be true simultaneously:

1. LLMs are a new technology and it's hard to put the genie back in the bottle with that. It's difficult to imagine a future where they don't continue to exist in some form, with all the timesaving benefits and social issues that come with them.

2. Almost three years in, companies investing in LLMs have not yet discovered a business model that justifies the massive expenditure of training and hosting them, the majority of consumer usage is at the free tier, the industry is seeing the first signs of pulling back investments, and model capabilities are plateauing at a level where most people agree that the output is trite and unpleasant to consume.

There are many technologies that have seemed inevitable and seen retreats under the lack of commensurate business return (the supersonic jetliner), and several that seemed poised to displace both old tech and labor but have settled into specific use cases (the microwave oven). Given the lack of a sufficiently profitable business model, it feels as likely as not that LLMs settle somewhere a little less remarkable, and hopefully less annoying, than today's almost universally disliked attempts to cram it everywhere.

replies(26): >>44568145 #>>44568416 #>>44568799 #>>44569151 #>>44569734 #>>44570520 #>>44570663 #>>44570711 #>>44570870 #>>44571050 #>>44571189 #>>44571513 #>>44571570 #>>44572142 #>>44572326 #>>44572360 #>>44572627 #>>44572898 #>>44573137 #>>44573370 #>>44573406 #>>44574774 #>>44575820 #>>44577486 #>>44577751 #>>44577911 #
moffkalast ◴[] No.44569151[source]
ML models have the good property of only requiring investment once and can then be used till the end of history or until something better replaces them.

Granted the initial investment is immense, and the results are not guaranteed which makes it risky, but it's like building a dam or a bridge. Being in the age where bridge technology evolves massively on a weekly basis is a recipe for being wasteful if you keep starting a new megaproject every other month though. The R&D phase for just about anything always results in a lot of waste. The Apollo programme wasn't profitable either, but without it we wouldn't have the knowledge for modern launch vehicles to be either. Or to even exist.

I'm pretty sure one day we'll have an LLM/LMM/VLA/etc. that's so good that pretraining a new one will seem pointless, and that'll finally be the time we get to (as a society) reap the benefits of our collective investment in the tech. The profitability of a single technology demonstrator model (which is what all current models are) is immaterial from that standpoint.

replies(1): >>44569360 #
1. wincy ◴[] No.44569360[source]
Nah, if TSMC got exploded and there was a world war, in 20 years all the LLMs would bit rot.
replies(1): >>44569507 #
2. moffkalast ◴[] No.44569507[source]
Eh, I doubt it, tech only got massively better in each world war so far, through unlimited reckless strategic spending. We'd probably get a TSMC-like fab on every continent by the end of it. Maybe even optical computers. Quadrotor UAV are the future of warfare after all, and they require lots of compute.

Adjusted for inflation it took over 120 billion to build the fleet of liberty ships during WW2, that's like at least 10 TSMC fabs.

replies(1): >>44572240 #
3. aydyn ◴[] No.44572240[source]
Technology is an exponential process, and the thing about exponentials is that they are chaotic. You cant use inductive reasoning vis a vis war and technology. The next big one could truly reset us to zero or worse.
replies(1): >>44572604 #
4. moffkalast ◴[] No.44572604{3}[source]
Sure you can't plan for black swan events, so the only choice you have is to plan for their absence. If we all nuke ourselves tomorrow well at least we don't have to worry about anything anymore. But in case we don't, those plans will be useful.