Are OpenAI and Anthropic losing money on inference?

The Amodei quote in my other reply explains why this is wrong. The point is not to compare the training of the current model to inference on the current model. The thing that makes them lose so much money is that they are training the next model while making back their training cost on the current model. So it's not COGS at all.

Well, only if the one training model continued to function as a going business. Their amortization window for the training cost is 2 months or so. They can't just keep that up and collect $.

They have to build the next model, or else people will go to someone else.

So is OpenAI capable of not making a new model at some point? They've been training the next model continuously as long as they've existed AFAIK.

Our software house spends a lot on R&D sure, but we're still incredibly profitable all the same. If OpenAI is in a position where they effectively have to stop iterating the product to be profitable, I wouldn't call that a very good place to be when you're on the verge of having several hundred billion in debt.

Why two months? It was almost a year between Claude 3.5 and 4. (Not sure how much it costs to go from 3.5 to 3.7.)

I think at that point there is strong financial pressure to figure out how to continuously evolve models instead of changing new ones, for example by building models out of smaller modules that can be trained individually and swapped out. Jeff Dean and Noam Shazeer talked about that a bit in their interview with Dwarkesh: https://www.dwarkesh.com/p/jeff-dean-and-noam-shazeer

So,if they stopped training they’d be profitable? Only in some incremental sense, ignoring all sunk costs.

There’s still untapped value in deeper integrations. They might hit a jackpot of exponentially increasing value from network effects caused by tight integration with e.g. disjoint business processes.

We know that businesses with tight network effects can grow to about 2 trillion in valuation.

Even being generous, and saying it's a year, most capital expenditures depreciate over a period of 5-7 years. To state the obvious, training one model a year is not a saving grace

Don't they need to accelerate that, though? Having a 1 year old model isn't really great, it's just tolerable.

I don't understand why the absolute time period matters — all that matters is that you get enough time making money on inference to make up for the cost of training.

I think this is debatable as more models become good enough for more tasks. Maybe a smaller proportion of tasks will require SOTA models. On the other hand, the set of tasks people want to use LLMs for will expand along with the capabilities of SOTA models.

How would that look with at least 3 US companies, probably 2 Chinese ones and at least 1 European company developing state of the art LLMs?

Like a very over-served market, I think. I see perhaps three survivors long term, or at most one gorilla, two chimps, and perhaps a few very small niche-focused monkeys.

Network effects usually destroy or marginalized competition until they themselves start stagnating decaying. Sometimes they produce partially-overlapping duopolies, but maintain their monopoly-like power.

Facebook marginalized linkedin and sent twitter into a niche.

Internet Explorer and Windows destroyed competition, for a long while.

Google Search marginalized everyone for over 20 years.

These are multi-trillion-dollar businesses. If OpenAI creates a network effect of some sort they can join the league.