Qwen2.5-VL-32B: Smarter and Lighter

Silly question: how can OpenAI, Claude and all, have a valuation so large considering all the open source models? Not saying they will disappear or be tiny (closed models), but why so so so valuable?

It's user base and brand. Just like with Pepsi and Coca Cola. There's a reason OpenAI ran a Super Bowl ad.

Yeah but cheaper alternatives (and open source and local ones) it would be super easy for most of the customers to migrate to a different provider. I am not saying they don't provide any value, but it's like paid software vs open source alternative. Open source alternative ends up imposing, especially among tech people.

People cannot normally invest in their competitors.

It's not unlikely that chinese products may be banned / tarriff'd

There are non-Chinese open LLMs (Mistral, LLama, etc), so I don't think that explains it.

Most "normies" I know only recognize ChatGPT with AI, so for sure, brand recognition is the only thing that matters.

ChatGPT's o1 is still really good and the free options are not compelling enough to switch if you've been using it for a while. They've positioned themselves to be a good mainstream default.

Because what would seem like a tiny difference in those benchmark graphs is the difference between worth paying for and complete waste of time in practice

The average user won't self-host a model.

...yet

Because they offer extremely powerful models at pretty modest prices.

The hardware for a local model would cost years and years of a $20/mo subscription, would output lower quality work, and would be much slower.

3.7 Thinking is an insane programming model. Maybe it cannot do an SWE's job, but it sure as hell can write functional narrow-scope programs with a GUI.

Valuation can depend on lots of different things, including hype. However, it ultimately comes down to an estimated discounted cash flow from the future, i.e. those who buy their shares (through private equity methods) at the current valuation believe the company will earn such and such money in the future to justify the valuation.

OpenAI is worth >$100B because of the "ChatGPT" name which it turns out, over 400M+ users use it weekly.

That name alone holds the most mindshare in it's product category, and is close to the level of name recognition just like Google.

The competition isn't self-hosting. If you can just pick a capable model from any provider inference just turns into a infrastructure/PaaS game -> The majority of the profits will be captured by the cloud providers.

Their valuation is not marked to market. We know their previous round valuation, but at this point it is speculative until they go through another round that will mark them again.

That being said, they have a user base and integrations. As long as they stay close or a bit ahead of the Chinese models they'll be fine. If the Chinese models significantly jumps ahead of them, well, then they are pretty much dead. Add open source to the mix and they become history.

For coding and other integrations people pay per token on api key, not subscription. Claude code costs few $ per task on your code - it gets expensive quite quickly.

...according to investors. (ps. it's even >$150B)

In reality OpenAI is loosing money per user.

Cost per token is tanking like crazy due to competition.

They guesstimate break even and then profit in couple of years.

Their guesses seem to not account for progress much especially on open weight models.

Frankly I have no idea what they're thinking there – they can barely keep up with investor subsidized, non sustainable model.

I'm not sure how it'll ever make sense unless you need a lot of customizations or care a lot about data leaks.

For small guys and everyone else.. it'll probably be cost neutral to keep paying OpenAi, Google etc directly rather than paying some cloud provider to host an at best on-par model at equivalent prices.

But something comparable to a local hosted model in the 32-70b range costs pennies on the dollar compared to Claude, will be 50x faster than your gpu, and with a much larger context window.

Local hosting on GPU only really makes sense if you're doing many hours of training/inference daily.

> unless you need a lot of customizations or care a lot about data leaks

And both those needs are very normal. "Customization" in this case can just be "specializing the LLM on local material for specialized responses".

I've tried self hosting. It is quite difficult, and either you are limited to low models, either you need a very expensive setup. I couldn't run this model on my gaming computer.

If I try other models, I basically end up with a very bad version of AI. Even if I'm someone who uses Anthropic APIs a lot, it's absolutely not worth it to try and self host it. The APIs are much better and you get much cheaper results.

Self hosting for AI might be useful for 0.001% of people honestly.

...or working for company which forbids sending IP over wire somewhere.

Also "many hours of inference daily" may mean you're doing your usual stuff daily while running some processing in the background that takes hours/days or you've put together some reactive automation that runs often all the time.

ps. local training rarely makes sense.

ps. 2. not sure where you got 50x slower from; 4090 is actually faster than A100 for example and 5090 is ~75% faster than 4090