←back to thread

387 points reaperducer | 1 comments | | HN request time: 0.207s | source
Show context
SubiculumCode ◴[] No.45772210[source]
Given that AI is a national security matter now, I'd expect the U.S.A to step in and rescue certain companies in the event of a crash. However, I'd give higher chances to NVIDIA than OpenAI. Weights are easily transferrable and the expertise is in the engineers, but ability to continue making advanced chips is not as easily transferred.
replies(4): >>45772241 #>>45772328 #>>45772343 #>>45772651 #
embedding-shape ◴[] No.45772241[source]
Why is ML knowledge "in the engineers" while chip manufacturing apparently sits in the company/hardware/something else than the engineers/humans?
replies(6): >>45772325 #>>45772346 #>>45772355 #>>45772369 #>>45772507 #>>45772729 #
NBJack ◴[] No.45772325[source]
Read up a bit on the effort needed to get a fab going, and the yield rates. While engineers are crucial in the setup, the fab itself is not as 'fungible' as the employees involved.

I can spin up a strong ML team through hiring in probably 6-12 months with the right funding. Building a chip fab and getting it to a sensible yield would take 3-5 years, significantly more funding, strong supply lines, etc.

replies(5): >>45772443 #>>45772496 #>>45772509 #>>45772514 #>>45773390 #
embedding-shape ◴[] No.45773390[source]
> I can spin up a strong ML team through hiring in probably 6-12 months with the right funding

Not sure what to call this except "HN hubris" or something.

There are hundreds of companies who thought (and still think) the exact same thing, and even after 24 months or more of "the right funding" they still haven't delivered the results.

I think you're misunderstanding how difficult all of this is, if you think it's merely a money problem. Otherwise we'd see SOTA models from new groups every month, which we obviously aren't, we have a few big labs iteratively progressing SOTA, with some upstarts appearing sometimes (DeepSeek, Kimi et al) but it isn't as easy as you're trying to make it out to be.

replies(3): >>45773610 #>>45773835 #>>45774090 #
1. whimsicalism ◴[] No.45773610[source]
There’s a lot in LLM training that is pretty commodity at this point. The difficulty is in data - and a large part of why it has gotten more challenging is simply that some of the best sources of data have locked down against scraping post-2022 and it is less permissible to use copyrighted data than the “move fast and break things” pre-2023 era.

As you mentioned, multiple no name chinese companies have done it and published many of their results. There is a commodity recipe for dense transformer training. The difference between Chinese and US is that they have less data restrictions.

I think people overindex on the Meta example. It’s hard to fully understand why Meta/llama have failed as hard as they have - but they are an outlier case. Microsoft AI only just started their efforts in earnest and are already beating Meta shockingly.