←back to thread

Meta's open AI hardware vision

(engineering.fb.com)
240 points GavCo | 4 comments | 15 Oct 24 18:08 UTC | HN request time: 0.933s | source
Show context
TechDebtDevin ◴[15 Oct 24 19:43 UTC] No.41852366[source]▶
> "This effort pushed our infrastructure to operate across more than 16,000 NVIDIA H100 GPUs, making Llama 3.1 405B the first model in the Llama series to be trained at such a massive scale."

So at 20k a pop (assuming meta has a decent wholesale price from Nividia) they spent $320 MILLION on the 405B model (not including probably 5-10 million in electricity for the training process, water, staff, infra).

Do we think that brings more than 400+ million in value to Meta? I think so. I don't want to do the math, so I'll ask Perplexity to look it up:

> "How much has Meta's valuation increased since they released their first open source model"

Answer (edited):

> Closing price on February 23, 2023: $509.50 > Closing price on October 11, 2024: $573.68 > The increase in stock price is $64.18 per share. > Total increase = Price increase per share × Number of outstanding shares > Total increase = $64.18 × 2,534,000,000 = $162,632,100,000 > Meta's stock valuation has increased by approximately $162.63 billion since the release of their first open source model on February 24, 2023.

They seem to be making the right choices!

replies(11): >>41852405 #>>41852476 #>>41852557 #>>41852616 #>>41852649 #>>41853410 #>>41853466 #>>41853527 #>>41853550 #>>41853606 #>>41857468 #
JumpCrisscross ◴[15 Oct 24 19:48 UTC] No.41852405[source]▶
> Do we think that brings more than 400+ million in value to Meta?

Tough to tell, given nobody is turning a net profit on LLMs yet.

Companies have a tendency to develop neuroses, though, just like people. Apple’s near miss with bankruptcy fuelled cash hoarding. For Facebook, their disastrous IPO and near miss of mobile seems to have made them hyper aware of the Innovator’s Dilemma. $400mm spent on a defensive move is certainly wider than tens of billions on the metaverse.

replies(2): >>41852505 #>>41852901 #
1. sangnoir ◴[15 Oct 24 20:44 UTC] No.41852901[source]▶
> Tough to tell, given nobody is turning a net profit on LLMs yet

Perhaps, but Meta is definitely getting some money back from ad impressions supported by AI-generated content.

replies(1): >>41853022 #
2. JumpCrisscross ◴[15 Oct 24 20:56 UTC] No.41853022[source]▶
Correct. We know these models are producing fucktonnes of revenue. At least some of them can be run at a gross profit, i.e. where marginal power costs and capital costs are less than marginal revenues. (Put another way: if OpenAI were an absolute monopoly and stopped training new models, could it turn a profit?)

What’s unclear is if this is a short-term revenue benefit from people fucking around with the newest, shiniest model, or recurring revenue that is only appearing unprofitable because the frontier is advancing so quickly.

replies(1): >>41853435 #
3. TechDebtDevin ◴[15 Oct 24 21:44 UTC] No.41853435[source]▶
From the little we know about OpenAIs inference infra, I feel like I can confidently say that if training stopped today, and they got cut off Azure subsidies, their $20.00 subscription model would probably not cover the cost of Inference.

I know nothing about the enterprise side of OpenAI but I'm sure they're profitable there. I doubt the subscription cost of a single power user of ChatGPT Plus covers the water they consume as a single user (This is probably an exaggeration, but I think I'm in the ballpark).

replies(1): >>41853482 #
4. JumpCrisscross ◴[15 Oct 24 21:50 UTC] No.41853482{3}[source]▶
It may be that extra-large LLMs don’t make sense for ChatGPT. They’re for enterprise use, like supercomputers. The reason I say “at least some” is I’ve found use in running a local instance of Llama, which seems to imply there is some niche of (legal) activities AI can support sustainably. (Versus, e.g. crypto.)