Are OpenAI and Anthropic losing money on inference?

Will these companies ever stop training new models? What does it mean if we get there. Feels like they will have to constantly train and improve the models, not sure what that means either. What ncremental improvements can these models show?

Another question is - will it ever become less costly to train?

Let to see opinions from someone in the know

current way the models works is that they don't have memory, it's included in training (or has to be provided as context).

So to keep up with times the models have to be constantly trained.

One thing though is that right now it's not just incremental training, the whole thing gets updated - multiple parameters and how the model is trained is different.

This might not be the case in the future where the training could become more efficient and switch to incremental updates where you don't have to re-feed all the training data but only the new things.

I am simplifying here for brevity, but I think the gist is still there.