How OpenAI uses complex and circular deals to fuel its multibillion-dollar rise

Why is ML knowledge "in the engineers" while chip manufacturing apparently sits in the company/hardware/something else than the engineers/humans?

Read up a bit on the effort needed to get a fab going, and the yield rates. While engineers are crucial in the setup, the fab itself is not as 'fungible' as the employees involved.

I can spin up a strong ML team through hiring in probably 6-12 months with the right funding. Building a chip fab and getting it to a sensible yield would take 3-5 years, significantly more funding, strong supply lines, etc.

First-order: because of the capex and lead times. If you grab a bunch of world-class ML folks and put them in a room together, they're going to be able to start producing world-class work together. If you grab a bunch of world-class chip designers in the same scenario but don't have world-class fabs for them to use, they're not going to be able to ship competitive designs.

One person can implement a transformer model from scratch in a weekend. Hardware is not the valuable part of machine learning. Data and how it is used are.

The "magic of AI" doesn't live inside an Nvidia GPU. There are billions of dollars of marketing being deployed to convince you it does. As soon as the market realizes that nvidia != magic AI box, the music should stop pretty quickly.

The start-up costs of creating a new chip manufacture are significantly higher (you can't just SAAS your way into factories) and the chips themselves more subject to IP and patents owned by that company.

That's true, but without the kind of horsepower provided by modern hardware, even though I'm skeptical that it's all needed, especially given DeepSeek's amazing results, AI would be nearly impossible.

There are some important innovations on the algorithm / network structure side, but all these ideas are only able to be tried because the hardware supports it. This stuff has been around for decades.

> If you grab a bunch of world-class chip designers in the same scenario but don't have world-class fabs for them to use, they're not going to be able to ship competitive designs.

But why such an unfair comparison?

Instead of comparing "skilled people with hardware VS skilled people without hardware", why not compare it to "a bunch of world-class ML folks" without any computers to do the work, how could they produce world-class work then?

But the fabs don't belong to NVIDIA, they belong to TSMC. I have no doubt that Taiwan and maybe even the US government would step in to save TSMC if for some reason it got existential problems, but that doesn't provide an argument for saving NVIDIA

Right. I could spin up a strong ML team, an AI startup, build a foundational model, etc give a reasonable amount of seed capital.

Build a chip fab? I’ve got no idea where to start, where to even find people to hire, and i know the equipment we’d need to acquire would be also quite difficult to get at any price.

Chip manufacturing is extremely time consuming, especially when we are talking about masks for lithography.

The rights on masks for chips and their parts (IPs) belong to companies.

And one definitely does not want these masks to be sold during bankruptcy process to (arbitrary) higher bidders.

> I can spin up a strong ML team through hiring in probably 6-12 months with the right funding.

Mark Zuckerberg would like a word with you

Nvidia isn't a fab.

Chip designs have strong IP protections.

AI models do not. Sure you can't just copy the exact floating point values without permission. But with enough capital you can train a model just as good, as the training and inference techniques are well known.

Much easier and cheaper to source computers than a fab.

Umm, part of it does. It necessary but not sufficient, at least to achieve it on the timescales we've seen. Scale is part of the "magic".

Deepseek required existing models that required the horsepower.

Right, but to source a fab you need experience as well, nothing you can just hire a random person to do exactly.

> I can spin up a strong ML team through hiring in probably 6-12 months with the right funding

Not sure what to call this except "HN hubris" or something.

There are hundreds of companies who thought (and still think) the exact same thing, and even after 24 months or more of "the right funding" they still haven't delivered the results.

I think you're misunderstanding how difficult all of this is, if you think it's merely a money problem. Otherwise we'd see SOTA models from new groups every month, which we obviously aren't, we have a few big labs iteratively progressing SOTA, with some upstarts appearing sometimes (DeepSeek, Kimi et al) but it isn't as easy as you're trying to make it out to be.

> But with enough capital you can train a model just as good, as the training and inference techniques are well known

You're not alone in believing just money can train a good model, and I've already answered elsewhere why things aren't so easy as you believe, but besides this, where are y'all getting that from? Is there some popular social media influencer that keeps parroting this or where it comes from? Clearly you're not involved in those processes/workflows yourself, then you wouldn't claim it's just a money problem, so where are you all getting this from?

There’s a lot in LLM training that is pretty commodity at this point. The difficulty is in data - and a large part of why it has gotten more challenging is simply that some of the best sources of data have locked down against scraping post-2022 and it is less permissible to use copyrighted data than the “move fast and break things” pre-2023 era.

As you mentioned, multiple no name chinese companies have done it and published many of their results. There is a commodity recipe for dense transformer training. The difference between Chinese and US is that they have less data restrictions.

I think people overindex on the Meta example. It’s hard to fully understand why Meta/llama have failed as hard as they have - but they are an outlier case. Microsoft AI only just started their efforts in earnest and are already beating Meta shockingly.

Fully agree. I also think we are deep into the diminishing returns territory.

If I have to guess OAI and others pay top dollars for talent that has a higher probability of discovering the next "attention" mechanism and investors are betting this is coming soon (hence the hige capitalizations and willing to loive with 11B losses/quarter). If they lose patience in throwing money at the problem I see only few players remaining in the race because they have other revenue streams

To simplify it down even more:

- For the ML team, you need money. Money to pay them and money to get access to GPUs. You might buy the GPUs and make your own server farm (which also takes time) or you might just burn all that money with AWS and use their GPUs. You can trade off money vs. time.

- For the chip design team, you need money and time. There's no workaround for the time aspect of it. You can't spend more money and get a fab quicker.

> - For the ML team, you need money. Money to pay them and money to get access to GPUs. You might buy the GPUs and make your own server farm (which also takes time) or you might just burn all that money with AWS and use their GPUs. You can trade off money vs. time.

Even if you do those things though, it doesn't guarantee success or you'll be able to train something bigger. For that you need knowledge, hard work and expertise, regardless of how much money you have. It's not a problem you can solve by throwing money at it, although many are trying. You can increase the chances of hopefully discovering something novel that helps you build something SOTA, but as current history tells us, it isn't as easy as "ML Team + Money == SOTA model in a few months".

>Otherwise we'd see SOTA models from new groups every month

We do.

It's just that startups don't go after the frontier models but niche spaces which are under served and can be explored with a few million in hardware.

Just like how open AI made gpt2 before they made gpt3.

> We do.

> It's just that startups don't go after the frontier models but niche spaces

But both of "New SOTA models every month" and "Startups don't go for SOTA" cannot be true at the same time. Either we get new SOTA models from new groups every month (not true today at least) or we don't, maybe because the labs are focusing on non-SOTA instead.

State of the art doesn't mean frontier.

Sure. No guarantees that you could throw money at putting an ML team together and have a new SOTA model in a few months. You might, you might not.

You know what I can guarantee? No matter how much money you throw at it, you will not have a new SOTA fab in a few months.

I've always taken that term literally, basically "top of the top". If you're not getting the best responses from that LLM, then it's not "top of the top" anymore, regardless of size.

Then something could be "SOTA in it's class" I suppose, but personally that's less interesting and also not what the parent commentator claimed, which was basically "anyone with money can get SOTA models up and running".

Edit: Wikipedia seems to agree with me too:

> The state of the art (SOTA or SotA, sometimes cutting edge, leading edge, or bleeding edge) refers to the highest level of general development, as of a device, technique, or scientific field achieved at a particular time

I haven't heard of anyone using SOTA to not mean "at the front of the pack", but maybe people outside of ML use the word differently.

A sota decoder model is a bigger deal than yet another trillion parameter encoder only model trained on benchmarks.

I don't get why you think that the only way that you can beat the big guys is by having more parameters than them.

That was claimed but never proven. I personally don't believe the American companies making this claim. I suspect they made this up to protect their valuations when they were hideously embarassed and lost a trillion dollars in equity.

> I don't get why you think that the only way that you can beat the big guys is by having more parameters than them.

Yeah, and I don't understand why people have to argue against some point others haven't made, kind of makes it less fun to participate in any discussions.

Whatever gets the best responses (no matter parameter size, specific architecture, addition of other things) is what I'd consider SOTA, then I guess you can go by your own definition.