←back to thread

507 points martinald | 10 comments | | HN request time: 0.29s | source | bottom
Show context
JCM9 ◴[] No.45051717[source]
These articles (of which there are many) all make the same basic accounting mistakes. You have to include all the costs associated with the model, not just inference compute.

This article is like saying an apartment complex isn’t “losing money” because the monthly rents cover operating costs but ignoring the cost of the building. Most real estate developments go bust because the developers can’t pay the mortgage payment, not because they’re negative on operating costs.

If the cash flow was truly healthy these companies wouldn’t need to raise money. If you have healthy positive cash flow you have much better mechanisms available to fund capital investment other than selling shares at increasingly inflated valuations. Eg issue a bond against that healthy cash flow.

Fact remains when all costs are considered these companies are losing money and so long as the lifespan of a model is limited it’s going to stay ugly. Using that apartment building analogy it’s like having to knock down and rebuild the building every 6 months to stay relevant, but saying all is well because the rents cover the cost of garbage collection and the water bill. That’s simply not a viable business model.

Update Edit: A lot of commentary below re the R&D and training costs and if it’s fair to exclude that on inference costs or “unit economics.” I’d simply say inference is just selling compute and that should be high margin, which the article concludes it is. The issue behind the growing concerns about a giant AI bubble is if that margin is sufficient to cover the costs of everything else. I’d also say that excluding the cost of the model from “unit economics” calculations doesn’t make business/math/economics since it’s literally the thing being sold. It’s not some bit of fungible equipment or long term capital expense when they become obsolete after a few months. Take away the model and you’re just selling compute so it’s really not a great metric to use to say these companies are OK.

replies(17): >>45051757 #>>45051787 #>>45051841 #>>45051851 #>>45051914 #>>45052000 #>>45052124 #>>45052133 #>>45052139 #>>45052319 #>>45052370 #>>45052582 #>>45052624 #>>45052648 #>>45052702 #>>45053815 #>>45054029 #
gruez ◴[] No.45051787[source]
I think the point isn't to argue AI companies are money printers or even that they're fairly valued, it's that at least the unit economics work out. Contrast this to something like moviepass, where they were actually losing money on each subscriber. Sure, a company that requires huge capital investments that might never be paid back isn't great either, but at least it's better than moviepass.
replies(1): >>45051831 #
1. JCM9 ◴[] No.45051831[source]
Unit economics needs to include the cost of the thing being sold, not just the direct cost of selling it.

Unit economics is mostly a manufacturing concept and the only reason it looks OK here is because of not really factoring in the cost of building the thing into the cost of the thing.

Someone might say I don’t understand “unit economics” but I’d simply argue applying a unit economics argument saying it’s good without including the cost of model training is abusing the concept of unit economics in a way that’s not realistic from a business/economics sense.

The model is what’s being sold. You can’t just sell “inference” as a thing with no model. Thats just selling compute, which should be high margin. The article is simply affirming that by saying yes when you’re just selling compute in micro-chunks that’s a decent margin business which is a nice analysis but not surprising.

replies(6): >>45051876 #>>45051913 #>>45051937 #>>45051971 #>>45051975 #>>45054691 #
2. martinald ◴[] No.45051876[source]
But what about running Deepseek R1 or (insert other open weights model here)? There is no training cost for that.
replies(1): >>45051926 #
3. ascorbic ◴[] No.45051913[source]
You can amortise the training cost across billions of inference requests though. It's the marginal cost for inference that's most interesting here.
4. JCM9 ◴[] No.45051926[source]
1. Someone is still paying for that cost.

2. “Open source” is great but then it’s just a commodity. It would be very hard to build a sustainable business purely on the back of commoditized models. Adding a feature to an actual product that does something else though? Sure.

replies(1): >>45052547 #
5. Aurornis ◴[] No.45051937[source]
The cost of “manufacturing” an AI response is the inference cost, which this article covers.

> That would be like saying the unit economics of selling software is good because the only cost is some bandwidth and credit card processing fees. You need to include the cost of making the software

Unit economics is about the incremental value and costs of each additional customer.

You do not amortize the cost of software into the unit economics calculations. You only include the incremental costs of additional customers.

> just like you need to include the cost of making the models.

The cost of making the models is important overall, but it’s not included in the unit economics or when calculating the cost of inference.

6. voxic11 ◴[] No.45051971[source]
That isn't what unit economics is. The purpose of unit economics is to answer: "How much money do I make (or lose) if I add one more customer or transaction?". Since adding an additional user/transaction doesn't increase the cost of training the models you would not include the cost of training the models in a unit economics analysis. The entire point of unit economics is that it excludes such "fixed costs".
7. barrkel ◴[] No.45051975[source]
There is no marginal cost for training, just like there's no marginal cost for software. This is why you don't generally use unit economics for analyzing software company breakeven.
replies(1): >>45052508 #
8. cj ◴[] No.45052508[source]
The only reason unit economics aren't generally used for software companies is the profit margin is typically 80%+. The cost of posting a Tweet on Twitter/X is close to $0.

Compare the cost of tweeting to the cost of submitting a question to ChatGPT. The fact that ChatGPT rate limits (and now sells additional credits to keep using it after you hit the limit) indicates there are serious unit economic considerations.

We can't think of OpenAI/Anthropic as software businesses. At least from a financial perspective, it's more similar to a company selling compute (e.g. AWS) than a company selling software (e.g. Twitter/X).

9. scarface_74 ◴[] No.45052547{3}[source]
There is plenty of money to be made from hosting open source software. AWS for instance makes tons of money from Linux, MySQL, Postgres, Redis, hosting AI models like DeepSeek (Bedrock) etc.
10. cwyers ◴[] No.45054691[source]
The thing about large fixed costs is that you can just solve them with growth. If they were losing money on inference alone no amount of growth would help. It's not clear to me there's enough growth that everybody makes it out of this AI boom alive, but at least some companies are going to be able to grow their way to profitability at some point, presumably.