←back to thread

LLM Inevitabilism

(tomrenner.com)
1616 points SwoopsFromAbove | 9 comments | 15 Jul 25 04:35 UTC | HN request time: 0.002s | source | bottom
Show context
lsy ◴[15 Jul 25 05:33 UTC] No.44568114[source]▶
I think two things can be true simultaneously:

1. LLMs are a new technology and it's hard to put the genie back in the bottle with that. It's difficult to imagine a future where they don't continue to exist in some form, with all the timesaving benefits and social issues that come with them.

2. Almost three years in, companies investing in LLMs have not yet discovered a business model that justifies the massive expenditure of training and hosting them, the majority of consumer usage is at the free tier, the industry is seeing the first signs of pulling back investments, and model capabilities are plateauing at a level where most people agree that the output is trite and unpleasant to consume.

There are many technologies that have seemed inevitable and seen retreats under the lack of commensurate business return (the supersonic jetliner), and several that seemed poised to displace both old tech and labor but have settled into specific use cases (the microwave oven). Given the lack of a sufficiently profitable business model, it feels as likely as not that LLMs settle somewhere a little less remarkable, and hopefully less annoying, than today's almost universally disliked attempts to cram it everywhere.

replies(26): >>44568145 #>>44568416 #>>44568799 #>>44569151 #>>44569734 #>>44570520 #>>44570663 #>>44570711 #>>44570870 #>>44571050 #>>44571189 #>>44571513 #>>44571570 #>>44572142 #>>44572326 #>>44572360 #>>44572627 #>>44572898 #>>44573137 #>>44573370 #>>44573406 #>>44574774 #>>44575820 #>>44577486 #>>44577751 #>>44577911 #
alonsonic ◴[15 Jul 25 13:05 UTC] No.44570711[source]▶
I'm confused with your second point. LLM companies are not making any money from current models? Openai generates 10b USD ARR and has 100M MAUs. Yes they are running at a loss right now but that's because they are racing to improve models. If they stopped today to focus on optimization of their current models to minimize operating cost and monetizing their massive user base you think they don't have a successful business model? People use this tools daily, this is inevitable.
replies(11): >>44570725 #>>44570756 #>>44570760 #>>44570772 #>>44570780 #>>44570853 #>>44570896 #>>44570964 #>>44571007 #>>44571541 #>>44571655 #
dvfjsdhgfv ◴[15 Jul 25 13:24 UTC] No.44570896{3}[source]▶
> If they stopped today to focus on optimization of their current models to minimize operating cost and monetizing their user base you think they don't have a successful business model?

Actually, I'd be very curious to know this. Because we already have a few relatively capable models that I can run on my MBP with 128 GB of RAM (and a few less capable models I can run much faster on my 5090).

In order to break even they would have to minimize the operating costs (by throttling, maiming models etc.) and/or increase prices. This would be the reality check.

But the cynic in me feels they prefer to avoid this reality check and use the tried and tested Uber model of permanent money influx with the "profitability is just around the corner" justification but at an even bigger scale.

replies(1): >>44570940 #
ghc ◴[15 Jul 25 13:30 UTC] No.44570940{4}[source]▶
> In order to break even they would have to minimize the operating costs (by throttling, maiming models etc.) and/or increase prices. This would be the reality check.

Is that true? Are they operating inference at a loss or are they incurring losses entirely on R&D? I guess we'll probably never know, but I wouldn't take as a given that inference is operating at a loss.

I found this: https://semianalysis.com/2023/02/09/the-inference-cost-of-se...

which estimates that it costs $250M/year to operate ChatGPT. If even remotely true $10B in revenue on $250M of COGS would be a great business.

replies(1): >>44571028 #
1. dvfjsdhgfv ◴[15 Jul 25 13:36 UTC] No.44571028{5}[source]▶
As you say, we will never know, but this article[0] claims:

> The cost of the compute to train models alone ($3 billion) obliterates the entirety of its subscription revenue, and the compute from running models ($2 billion) takes the rest, and then some. It doesn’t just cost more to run OpenAI than it makes — it costs the company a billion dollars more than the entirety of its revenue to run the software it sells before any other costs.

[0] https://www.lesswrong.com/posts/CCQsQnCMWhJcCFY9x/openai-los...

replies(2): >>44571100 #>>44571236 #
2. ghc ◴[15 Jul 25 13:43 UTC] No.44571100[source]▶
Obviously you don't need to train new models to operate existing ones.

I think I trust the semianalysis estimate ($250M) more than this estimate ($2B), but who knows? I do see my revenue estimate was for this year, though. However, $4B revenue on $250M COGS...is still staggeringly good. No wonder amazon, google, and Microsoft are tripping over themselves to offer these models for a fee.

replies(3): >>44571326 #>>44572365 #>>44575101 #
3. matwood ◴[15 Jul 25 13:58 UTC] No.44571236[source]▶
CapEx vs. OpEx.

If they stop training today what happens? Does training always have to be at these same levels or will it level off? Is training fixed? IE, you can add 10x the subs and training costs stay static.

IMO, there is a great business in there, but the market will likely shrink to ~2 players. ChatGPT has a huge lead and is already Kleenex/Google of the LLMs. I think the battle is really for second place and that is likely dictated by who runs out of runway first. I would say that Google has the inside track, but they are so bad at product they may fumble. Makes me wonder sometimes how Google ever became a product and verb.

replies(1): >>44572608 #
4. hamburga ◴[15 Jul 25 14:07 UTC] No.44571326[source]▶
But assuming no new models are trained, this competitive effect drives down the profit margin on the current SOTA models to zero.
replies(1): >>44571604 #
5. ghc ◴[15 Jul 25 14:37 UTC] No.44571604{3}[source]▶
Even if the profit margin is driven to zero, that does not mean competitors will cease to offer the models. It just means the models will be bundled with other services. Case in point: Subversion & Git drove VCS margin to zero (remember BitKeeper?), but Bitbucket and Github wound up becoming good businesses. I think Claude Code might be the start of how companies evolve here.
6. singron ◴[15 Jul 25 15:45 UTC] No.44572365[source]▶
You need to train new models to advance the knowledge cutoff. You don't necessarily need to R&D new architectures, and maybe you can infuse a model with new knowledge without completely training from scratch, but if you do nothing the model will become obsolete.

Also the semianalysis estimate is from Feb 2023, which is before the release of gpt4, and it assumes 13 million DAU. ChatGPT has 800 million WAU, so that's somewhere between 115 million and 800 million DAU. E.g. if we prorate the cogs estimate for 200 DAU, then that's 15x higher or $3.75B.

replies(1): >>44573670 #
7. marcosdumay ◴[15 Jul 25 16:06 UTC] No.44572608[source]▶
That paragraph is quite clear.

OpEx is larger than revenue. CapEx is also larger than the total revenue on the lifetime of a model.

8. ghc ◴[15 Jul 25 17:28 UTC] No.44573670{3}[source]▶
> You need to train new models to advance the knowledge cutoff

That's a great point, but I think it's less important now with MCP and RAG. If VC money dried up and the bubble burst, we'd still have broadly useful models that wouldn't be obsolete for years. Releasing a new model every year might be a lot cheaper if a company converts GPU opex to capex and accepts a long training time.

> Also the semianalysis estimate is from Feb 2023,

Oh! I missed the date. You're right, that's a lot more expensive. On the other hand, inference has likely gotten a lot cheaper (in terms of GPU TOPS) too. Still, I think there's a profitable business model there if VC funding dries up and most of the model companies collapse.

9. dvfjsdhgfv ◴[15 Jul 25 19:47 UTC] No.44575101[source]▶
> Obviously you don't need to train new models to operate existing ones.

For a few months, maybe. Then they become obsolete and, in some cases like coding, useless.