←back to thread

507 points martinald | 2 comments | | HN request time: 0.022s | source
Show context
JCM9 ◴[] No.45051717[source]
These articles (of which there are many) all make the same basic accounting mistakes. You have to include all the costs associated with the model, not just inference compute.

This article is like saying an apartment complex isn’t “losing money” because the monthly rents cover operating costs but ignoring the cost of the building. Most real estate developments go bust because the developers can’t pay the mortgage payment, not because they’re negative on operating costs.

If the cash flow was truly healthy these companies wouldn’t need to raise money. If you have healthy positive cash flow you have much better mechanisms available to fund capital investment other than selling shares at increasingly inflated valuations. Eg issue a bond against that healthy cash flow.

Fact remains when all costs are considered these companies are losing money and so long as the lifespan of a model is limited it’s going to stay ugly. Using that apartment building analogy it’s like having to knock down and rebuild the building every 6 months to stay relevant, but saying all is well because the rents cover the cost of garbage collection and the water bill. That’s simply not a viable business model.

Update Edit: A lot of commentary below re the R&D and training costs and if it’s fair to exclude that on inference costs or “unit economics.” I’d simply say inference is just selling compute and that should be high margin, which the article concludes it is. The issue behind the growing concerns about a giant AI bubble is if that margin is sufficient to cover the costs of everything else. I’d also say that excluding the cost of the model from “unit economics” calculations doesn’t make business/math/economics since it’s literally the thing being sold. It’s not some bit of fungible equipment or long term capital expense when they become obsolete after a few months. Take away the model and you’re just selling compute so it’s really not a great metric to use to say these companies are OK.

replies(17): >>45051757 #>>45051787 #>>45051841 #>>45051851 #>>45051914 #>>45052000 #>>45052124 #>>45052133 #>>45052139 #>>45052319 #>>45052370 #>>45052582 #>>45052624 #>>45052648 #>>45052702 #>>45053815 #>>45054029 #
martinald ◴[] No.45051841[source]
(Author here). Yes I am aware of that and did mention it. However - what I wanted to push back in this article was that claude code was completely unsustainable and therefore a flash in the pan and devs aren't at risk (I know you are not saying this).

The models as is are still hugely useful, even if no further training was done.

replies(2): >>45052012 #>>45052637 #
Aurornis ◴[] No.45052012[source]
> The models as is are still hugely useful, even if no further training was done.

Exactly. The parent comment has an incorrect understanding of what unit economics means.

The cost of training is not a factor in the marginal cost of each inference or each new customer.

It’s unfortunate this comment thread is the highest upvoted right now when it’s based on a basic misunderstanding of unit economics.

replies(2): >>45052084 #>>45052661 #
esafak ◴[] No.45052084[source]
The marginal cost is not the salient factor when the model has to be frequently retrained at great cost. Even if the marginal cost was driven to zero, would they profit?
replies(2): >>45052157 #>>45052608 #
wongarsu ◴[] No.45052608[source]
But they don't have to be retained frequently at great cost. Right now they are retrained frequently because everyone is frequently coming out with new models and nobody wants to fall behind. But if investment for AI were to dry up everyone would stop throwing so much money at R&D, and if everyone else isn't investing in new models you don't have to either. The models are powerful as they are, most of the knowledge in them isn't going to rapidly obsolete, and where that is a concern you can paper over it with RAG or MCP servers. If everyone runs out of money for R&D at the same time we could easily cut back to a situation where we get an updated version of the same model every 3 years instead of a bigger/better model twice a year.

And whether companies can survive in that scenario depends almost entirely on their unit economics of inference, ignoring current R&D costs

replies(2): >>45052928 #>>45052970 #
1. re-thc ◴[] No.45052970[source]
> But if investment for AI were to dry up everyone would stop throwing so much money at R&D, and if everyone else isn't investing in new models you don't have to either

IF.

If you do stagnate for years someone will eventually decide to invest and beat you. Intel has proven so.

replies(1): >>45053755 #
2. simianwords ◴[] No.45053755[source]
Yeah so? How does that change anything?