←back to thread

Meta's open AI hardware vision

(engineering.fb.com)
212 points GavCo | 2 comments | 15 Oct 24 18:08 UTC | HN request time: 0s | source
Show context
TechDebtDevin ◴[15 Oct 24 19:43 UTC] No.41852366[source]▶
> "This effort pushed our infrastructure to operate across more than 16,000 NVIDIA H100 GPUs, making Llama 3.1 405B the first model in the Llama series to be trained at such a massive scale."

So at 20k a pop (assuming meta has a decent wholesale price from Nividia) they spent $320 MILLION on the 405B model (not including probably 5-10 million in electricity for the training process, water, staff, infra).

Do we think that brings more than 400+ million in value to Meta? I think so. I don't want to do the math, so I'll ask Perplexity to look it up:

> "How much has Meta's valuation increased since they released their first open source model"

Answer (edited):

> Closing price on February 23, 2023: $509.50 > Closing price on October 11, 2024: $573.68 > The increase in stock price is $64.18 per share. > Total increase = Price increase per share × Number of outstanding shares > Total increase = $64.18 × 2,534,000,000 = $162,632,100,000 > Meta's stock valuation has increased by approximately $162.63 billion since the release of their first open source model on February 24, 2023.

They seem to be making the right choices!

replies(10): >>41852405 #>>41852476 #>>41852557 #>>41852616 #>>41852649 #>>41853410 #>>41853466 #>>41853527 #>>41853550 #>>41853606 #
1. trsohmers ◴[15 Oct 24 19:56 UTC] No.41852476[source]▶
Do you think that the 16k GPUs get used once and then are thrown away? Llama 405B was trained over 56 days on the 16k GPUs; if I round that up to 60 days and assume the current mainstream hourly rate of $2/H100/hour from the Neoclouds (which are obviously making margin), that comes out to a total cost of ~$47M. Obviously Meta is training a lot of models using their GPU equipment, and would expect it to be in service for at least 3 years, and their cost is obviously less than what the public pricing on clouds is.
replies(1): >>41853367 #
2. lossolo ◴[15 Oct 24 21:35 UTC] No.41853367[source]▶
And Meta is using a lot of GPUs for offline ML and online ML features on Instagram, FB etc. So nothing is "wasted".