Are OpenAI and Anthropic losing money on inference?

https://www.axios.com/2025/08/15/sam-altman-gpt5-launch-chat... quotes Sam Altman saying:

> Most of what we're building out at this point is the inference [...] We're profitable on inference. If we didn't pay for training, we'd be a very profitable company.

ICYMI, Amodei said the same in much greater detail:

"If you consider each model to be a company, the model that was trained in 2023 was profitable. You paid $100 million, and then it made $200 million of revenue. There's some cost to inference with the model, but let's just assume, in this cartoonish cartoon example, that even if you add those two up, you're kind of in a good state. So, if every model was a company, the model, in this example, is actually profitable.

What's going on is that at the same time as you're reaping the benefits from one company, you're founding another company that's much more expensive and requires much more upfront R&D investment. And so the way that it's going to shake out is this will keep going up until the numbers go very large and the models can't get larger, and then it'll be a large, very profitable business, or, at some point, the models will stop getting better, right? The march to AGI will be halted for some reason, and then perhaps it'll be some overhang. So, there'll be a one-time, 'Oh man, we spent a lot of money and we didn't get anything for it.' And then the business returns to whatever scale it was at."

https://cheekypint.substack.com/p/a-cheeky-pint-with-anthrop...

That might be the case, but inference times have only gone up since GPT-3 (GPT-5 is regularly 20+ seconds for me).

Which is like saying, “If all we did is charge people money and didn’t have any COGS, we’d be a very profitable company.” That’s a truism of every business and therefore basically meaningless.

This can be technically true without being actually true.

IE OpenAI invests in Cursor/Windsurf/Startups that give away credits to users and make heavy use of inference API. Money flows back to OpenAI then OpenAI sends it back to those companies via credits/investment $.

It's even more circular in this case because nvidia is also funding companies that generate significant inference.

It'll be quite difficult to figure out whether it's actually profitable until the new investment dollars start to dry up.

And by GPT-5 you mean through their API? Directly through Azure OpenAI services? or are you talking about ChatGPT set to using GPT-5.

All of these alternatives means different things when you say it takes +20 seconds for a full response.

The Amodei quote in my other reply explains why this is wrong. The point is not to compare the training of the current model to inference on the current model. The thing that makes them lose so much money is that they are training the next model while making back their training cost on the current model. So it's not COGS at all.

Sure, apologies. I mean ChatGPT UI

Well, only if the one training model continued to function as a going business. Their amortization window for the training cost is 2 months or so. They can't just keep that up and collect $.

They have to build the next model, or else people will go to someone else.

So is OpenAI capable of not making a new model at some point? They've been training the next model continuously as long as they've existed AFAIK.

Our software house spends a lot on R&D sure, but we're still incredibly profitable all the same. If OpenAI is in a position where they effectively have to stop iterating the product to be profitable, I wouldn't call that a very good place to be when you're on the verge of having several hundred billion in debt.

I can't imagine the hoops an accountant would have to go through to argue training cost is COGS. In the most obvious stick-figures-for-beginners interpretation, as in, "If I had to explain how a P&L statement works to an AI engineer", training is R&D cost and inference cost is COGS.

there's not a bright line there, though.

From the latest NYT Hard Fork podcast [1]. The hosts were invited to a dinner hosted by Sam, where Sam said "we're profitable if we remove training from the equation", they report he turned to Lightcap (COO) and asked "right?" and Lightcap gave an "eeekk we're close".

They aren't yet profitable even just on inference, and its possible Sam didn't know that until very recently.

[1] https://www.nytimes.com/2025/08/22/podcasts/is-this-an-ai-bu...

This sounds like fabs.

I don't see why the declining marginal returns can't be continuous.

While this could be true, I don't think OpenAI is investing the $hundreds of millions-to-billions that would be required otherwise make it actually true.

OpenAI's fund is ~$250-300mm Nvidia reportedly invested $1b last year - still way less than Open AI revenue

It's even more circular, because Microsoft and Amazon also fund ChatGPT and Anthropic with Azure and AWS credits.

OpenAI and Anthropic have very different customer bases and usage profiles. I'd estimate a significantly higher percentage of Anthropic's tokens are paid by the customer than OpenAI's. The ChatGPT free tier is magnitudes more popular than Claude's free tier, and Anthropic in all likelihood does a higher percentage of API business versus consumer business than OpenAI does.

In other words, its possible this story is correct and true for Anthropic, but not true for OpenAI.

Copy laundering as a service is only profitable when you discount future settlements:

https://www.reuters.com/legal/government/anthropics-surprise...

The "model as company" metaphor makes no sense. It should actually be models are products, like a shoe. Nike spends money developing a shoe, then building it, then they sell it, and ideally those R&D costs are made up in shoe sales. But you still have to run the whole company outside of that.

Also, in Nike's case, as they grow they get better at making more shoes for cheaper. LLM model providers tell us that every new model (shoe) costs multiples more than the last one to develop. If they make 2x revenue on training, like he's said, to be profitable they have to either double prices or double users every year, or stop making new models.

Also Amodei has an assumption that a 100m model will make 200m of revenue but a 1B model will make 2B of revenue. Does that really hold up? There's no phenomenon that prevents them from only making 200m of revenue off a $1B model.

Except these tech billionaires lie the most of the time. This is still the "grow at any cost" phase, so I don't even genuinely believe he has a confident understanding of how or at what point anything will be profitable. This just strikes me as the best answer he has at the moment.

GPT-4.5 has entered the chat..

Why two months? It was almost a year between Claude 3.5 and 4. (Not sure how much it costs to go from 3.5 to 3.7.)

I think at that point there is strong financial pressure to figure out how to continuously evolve models instead of changing new ones, for example by building models out of smaller modules that can be trained individually and swapped out. Jeff Dean and Noam Shazeer talked about that a bit in their interview with Dwarkesh: https://www.dwarkesh.com/p/jeff-dean-and-noam-shazeer

So,if they stopped training they’d be profitable? Only in some incremental sense, ignoring all sunk costs.

I wasn’t using COGS in a GAAP sense, but rather as a synonym for unspecified “costs.” My bad. I suppose you would classify training as development and ongoing datacenter and GPU costs as actual GAAP COGS. My point was, if all you focus on is revenue and ignore the costs of creating your business and keeping it running, it’s pretty easy for any business to be “profitable.”

But new models to date have cost more than the previous ones to create, often by an order of magnitude, so the shoe metaphor falls apart.

A better metaphor would be oil and gas production, where existing oil and gas fields are either already finished (i.e. model is no longer SOTA -- no longer making a return on investment) or currently producing (SOTA inference -- making a return on investment). The key similarity with AI is new oil and gas fields are increasingly expensive to bring online because they are harder to make economical than the first ones we stumbled across bubbling up in the desert, and that's even with technological innovation. That is to say, the low hanging fruit is long gone.

If you're going to use shoes as the metaphor, a model would be more like a shoe factory. A shoe would be a LLM answer, i.e. inference. In which case it totally makes sense to consider each factory as an autonomous economic unit, like a company.

Good point, very possible that Altman is excluding free tier as a marketing cost even if it loses more than they make on paid customers. On the other hand they may be able to cut free tier costs a lot by having the model router send queries to gpt-5-mini where before they were going to 4o.

“We’re not profitable even if we discount training costs.”

and

“Inference revenue significantly exceeds inference costs.”

are not incompatible statements.

So maybe only the first part of Sam’s comment was correct.

> So, there'll be a one-time, 'Oh man, we spent a lot of money and we didn't get anything for it.'

There a journalist ed zittron

https://www.wheresyoured.at/

That is an openai skeptic. His research if correct says not only is openai unprofitable but it likely never will be. Can't be ,its various finance ratios make early uber, amazon ect look downright fiscally frugal.

He is not a tech person for what that means to you.

Free tier provides a lot of training material. Every time you correct ChatGPT on its mistakes you’re giving them knowledge that’s not in any book or website.

Thats a moat, albeit one that is slow to build.

Okay but noticeably he invents two numbers then pretends that a third number is irrelevant in order to claim that each model (which is not a company) is a profitable company.

You'd think maybe the CEO might be able to give a ball park on the profit made off that 2023 model.

ETA: "You paid $100 million... There's some cost to inference with the model, but let's just assume ... that even if you add those two up, you're kind of in a good state."

You see this right? He literally says that if you assume revenue exceeds costs then it's profitable. He doesn't actually say that it does though.

Fantastic perspective.

Basically each new company puts competitive pressure on the previous company, and together they compress margins.

They are racing themselves to the bottom. I imagine they know this and bet on AGI primacy.

It's model as a company because people are using the VC mentality, and also explaining competition.

Model as a product is the reality, but each model competes with previous models and is only successful if it's both more cost effective, and also more effective in general at its tasks. By the time you get to model Z, you'll never use model A for any task as the model lineage cannibalizes sales of itself.

It’s generally useful to consider unit economy separate from whole company. If your unit economy is negative thing are very bleak. If it’s positive, your chance are going up by a lot - scaling the business amortizes fixed (non-unit) costs, such as admin and R&D, and slightly improves unit margins as well.

However this does not work as well if your fixed (non-unit) cost is growing exponentially. You can’t get out of this unless your user base grows exponentially or the customer value (and price) per user grows exponentially.

I think this is what Altman is saying - this is an unusual situation: unit economy is positive but fixed costs are exploding faster than economy if scale can absorb it.

You can say it’s splitting hair, but insightful perspective often requires teasing things apart.

There’s still untapped value in deeper integrations. They might hit a jackpot of exponentially increasing value from network effects caused by tight integration with e.g. disjoint business processes.

We know that businesses with tight network effects can grow to about 2 trillion in valuation.

>Also, in Nike's case, as they grow they get better at making more shoes for cheaper.

This is clearly the case for models as well. Training and serving inference for GPT4 level models is probably > 100x cheaper than they used to be. Nike has been making Jordan 1's for 40+ years! OpenAI would be incredibly profitable if they could live off the profit from improved inference efficiency on a GPT4 level model!

Analogies don't prove anything, but they're still useful for suggesting possibilities for thinking about a problem.

If you don't like "model as company," how about "model as making a movie?" Any given movie could be profitable or not. It's not necessarily the case that movie budgets always get bigger or that an increased budget is what you need to attract an audience.

> I imagine they know this and bet on AGI primacy.

Just like Uber and Tesla are betting on self driving cars. I think it's been 10 years now ("any minute now").

>>This is clearly the case ... probably

>>OpenAI would be incredibly profitable if they could live off the profit from improved inference efficiency on a GPT4 level model!

If gpt4 was basically free money at this point it's real weird that their first instinct was to cut it off after gpt5

exactly: it’s like making shoes if you’re really bad at making shoes :)

I believe better analogy is CPU development on next process node.

each node is much more expensive to design for, but when you finally have it you basically print money.

and of course you always have to develop next more powerful and power efficient CPU to keep competitive

>> If we didn't pay for training, we'd be a very profitable company.

> ICYMI, Amodei said the same

No. He says that even paying for training a model is profitable. It makes more revenue that it costs - all things considered. A much stronger claim.

Even being generous, and saying it's a year, most capital expenditures depreciate over a period of 5-7 years. To state the obvious, training one model a year is not a saving grace

Don't they need to accelerate that, though? Having a 1 year old model isn't really great, it's just tolerable.

I don't understand why the absolute time period matters — all that matters is that you get enough time making money on inference to make up for the cost of training.

It’s splitting a hair, but a pretty important hair. Does anyone think that models won’t need continuous retraining? Does anyone think models won’t continue to try to scale? Personally, I think we’re reaching diminishing returns with scaling, which is probably good because we’ve basically run out of content to train on, and so perhaps that does stop or at least slow down drastically. But I don’t see a scenario where constant retraining isn’t the norm, even if the rough amount of content we’re using for it grows only slightly.

I think this is debatable as more models become good enough for more tasks. Maybe a smaller proportion of tasks will require SOTA models. On the other hand, the set of tasks people want to use LLMs for will expand along with the capabilities of SOTA models.

I think the idea here is that gpt-5-mini is the cheap gpt-4 quality model they want to serve and make money on.

How would that look with at least 3 US companies, probably 2 Chinese ones and at least 1 European company developing state of the art LLMs?

That's interesting, though you have to imagine the data set is very low quality on average and distilling high quality training pairs out of it is very costly.

Amazon was very frugal. If you look at Amazon losses for the first 10 years, they were all basically under 5% of revenue and many years were break even or slightly net positive.

Uber burnt through a lot of money and even now I'm not sure their lifetime revenue is positive (it's possible that since their foundation they've lost more money than they've made).

I take them to be saying the same thing — the difference is that Altman is referring to the training of the next model happening now, while Amodei is referring to the training months ago of the model you're currently earning money back on through inference.

Exactly Zittrons point.

Like a very over-served market, I think. I see perhaps three survivors long term, or at most one gorilla, two chimps, and perhaps a few very small niche-focused monkeys.

Zitron is not a serious analyst.

https://bsky.app/profile/davidcrespo.bsky.social/post/3lxale...

https://bsky.app/profile/davidcrespo.bsky.social/post/3lo22k...

https://bsky.app/profile/davidcrespo.bsky.social/post/3lwhhz...

https://bsky.app/profile/davidcrespo.bsky.social/post/3lv2dx...

> new models to date have cost more than the previous ones to create

This largely was the case in software in the '80s-'10s (when versions largely disappeared) and still is the case in hardware. iPhone 17 will certainly cost far more to develop than did iPhone 10 or 5. iPhone 5 cost far more than 3G, etc.

Notably, Uber switched horses and now runs Waymos with no human drivers.

I don't think it's the case if you take inflation into account.

You could see here: https://www.reddit.com/r/dataisbeautiful/comments/16dr1kb/oc...

new ones are generally cheaper if adjusted for inflation. This is a sale price, but assuming that margins stay the same it should reflect the manufacturing price. And from what I remember about apple earnings their margins increased over time, so it means the new phones are even cheaper. Which kind of makes sense.

Maybe he means that but the quote says “We're profitable on inference.” - not “We're profitable on inference including training of that model.”

I should have addressed this. This thread is about the capital costs of getting to the first sale, so that's model training for an LLM vs all the R&D in an iPhone.

Recent iPhones use Apple's own custom silicon for a number of components, and are generally vastly more complex. The estimates I have seen for iPhone 1 development range from $150 million to $2.5 billion. Even adjusting for inflation, a current iPhone generation costs more than the older versions.

And it absolutely makes sense for Apple to spend more in total to develop successive generations, because they have less overall product risk and larger scale to recoup.

Ed Zitron: I don’t think OpenAI will become profitable

The link you posted: I think it is very plausible that it will be hard for OpenAI to become profitable

I imagine that one of the largest costs for openai is the wages they pay.

> If gpt4 was basically free money at this point it's real weird that their first instinct was to cut it off after gpt5

People find the UX of choosing a model very confusing, the idea with 5 is that it would route things appropriately and so eliminate this confusion. That was the motivation for removing 4. But people were upset enough that they decided to bring it back for a while, at least.

This is very true. ChatGPT has a very generous free tier. I used to pay for it, but realized I was never really hitting the limits of what is needed to pay for it.

However, at the same time, I was using Claude much less, really preferring the answers from it most of the time, and constantly being hit with limits. So guess what I did. I cancelled my OpenAI subscription and moved to Anthropic. Not only do i get Claude Code, which OpenAI really has no serious competitor for.

I still use both models but never run into problems with OpenAI, so i see no reason to pay for it.

Are you referring to the post where I listed 4 claims and marked one ridiculous, one wrong, one unlikely, and one plausible?

He is not wrong about everything. For example, after Sam Altman said in January that OpenAI would introduce a model picker, Zitron was able to predict in March that OpenAI would introduce a model picker. And he was right about that.

Yes. In this thread about the profitability (or lack thereof) of OpenAI’s business model, I pointed out the part where you appeared to agree with Ed Zitron about the profitability (or lack thereof) of OpenAI’s business model. Like it seems like all of those posts were pretty clearly motivated by wanting to poke holes in Zitron’s criticism, their lack of profitability (which is central to his criticism) is where you declined to push back with any real argument.

Since only the first one responds to any of Zitron's content that I've actually read, I'll respond only to that one:

It's not responsive at all to Zitron's point. Zitron's broader contention is that AI tools are not profitable because the cost of AI use is too high for users to justify spending money on the output, given the quality of output. And furthermore, he argues that this basic fact is being obscured by lots of shell games around numbers to hide the basic cash flow issue. For example, focusing on cost in terms of cost per token rather than cost per task. And finally, there's an implicit assumption that the AI just isn't getting tremendously better, as might be exemplified by... burning twice as money tokens on the task in the hopes the quality goes up.

And in that context, the response is "Aha, he admits that there is a knob to trade off cost and quality! Entire argument debunked!" The existence of a cost-quality tradeoff doesn't speak to whether or not that line will intersect the quality-value tradeoff. I grant that a lot turns on how good you think AI is and/or will shortly be, and Zitron is definitely a pessimist there.

Already in your first point you are mixing up two claims Ed also likes to mix up. The funny thing is these claims are in direct conflict with each other. There is the question of whether people find AI worth paying for given what they get. You seem to think this is in some doubt, meanwhile here are tons of people paying for it, some even begging to be allowed to pay more in order to get more. The labs have revenue growing 20% per month. So I think that version of the point is absurd on its face. (And that's exactly why my thing about the cost-quality tradeoff being real is relevant. At least we agree on the relationship between these points.)

Ed doesn’t really make that argument anymore. The more recent form of the point is: yes, clearly people are willing to pay for it, but only because the providers are burning VC money to sell it below cost. If sold at a profit, customers would no longer find it worth it. But that’s completely different from what you’re saying. And I also think that’s not true, for a few reasons: mostly that selling near cost is the simplest explanation for the similarity of prices between providers. And now recently we have both Altman and Amodei saying their companies are selling inference at a profit.

The drivers are remote, but they are still there, dropping in as needed.

I should have provided a direct quote:

> At first, he answered, no, we would be profitable, if not for training new models. Essentially, if you take away all the stuff, all the money we’re spending on building new models and just look at the cost of serving the existing models, we are profitable on that basis. And then he looked at Brad Lightcap, who is the COO. And he sort of said, right? And Brad kind of squirmed in his seat a little bit and was like, well — He’s like, we’re pretty close.

I don't think you can square that with what he stated in the Axios article:

> "We're profitable on inference. If we didn't pay for training, we'd be a very profitable company."

Except, if the NYT dinner happened after the Axios article interview, which is possible given when each was published, and he was actually literally unaware of the company's financials.

Personally: it feels like it should reflect very poorly on OpenAI that their CEO has been, charitably, entirely unaware how close they are to profitability (and uncharitably, that he actively lies about it). But I'm not sure if the broader news cycle caught it; the only place I've heard this mentioned is literally this NYT Hard Fork podcast which is hosted by the people who were at the dinner.

They picked the worst possible time to make the change if money wasn’t involved (which is why I assumed GPT-5 must be massively cheaper to run). The backlash from being forced to use it cost a fair bit of the model’s reputation.

Yeah it didnt work out for them, for sure.

Well yes. He is a journalist. Not an analyst. I don't think he is a prophet or anything nor is is a tech person so yeah his details on operation are off. The money business side though he seem to be the only person willing to poke holes in the whole AI thing publicly.

I'm not an AI hater. I genuinely hope it take over every single white collar job that exists. I'm not being sarcastic or hyperbolic. Only then will we be able to re-discuss what society is in a more humane way.

Hence exponential increase in model training costs. Also hallucinations in the long tail of knowledge.

Network effects usually destroy or marginalized competition until they themselves start stagnating decaying. Sometimes they produce partially-overlapping duopolies, but maintain their monopoly-like power.

Facebook marginalized linkedin and sent twitter into a niche.

Internet Explorer and Windows destroyed competition, for a long while.

Google Search marginalized everyone for over 20 years.

These are multi-trillion-dollar businesses. If OpenAI creates a network effect of some sort they can join the league.

Well, models are definitely good enough for some things in their current state, without needing to be retrained (computer translation for example was a solved problem with GPT3)

Got it, it's just an awfully specific term to use as a generic replacement for "cost" when the whole concept of COGS is essentially "not any cost, but specifically this kind" :)

That’s true but irrelevant. No AI company is stopping training and further model development. OpenAI didn’t stop with GPT3, and they won’t stop with GPT5. No company, AI company or not, stops innovating in their market segment. You need to keep innovating to stay competitive.