←back to thread

Deploying DeepSeek on 96 H100 GPUs

(lmsys.org)
281 points GabrielBianconi | 1 comments | 29 Aug 25 14:07 UTC | HN request time: 0.2s | source
Show context
34679 ◴[29 Aug 25 14:44 UTC] No.45064819[source]▶
"By deploying this implementation locally, it translates to a cost of $0.20/1M output tokens"

Is that just the cost of electricity, or does it include the cost of the GPUs spread out over their predicted lifetime?

replies(3): >>45064954 #>>45066023 #>>45071720 #
dragonslayer56 ◴[29 Aug 25 14:54 UTC] No.45064954[source]▶
” Our implementation, shown in the figure above, runs on 12 nodes in the Atlas Cloud, each equipped with 8 H100 GPUs.”

Maybe the cost of renting?

replies(2): >>45065147 #>>45065503 #
34679 ◴[29 Aug 25 15:09 UTC] No.45065147[source]▶
I'm confused because I wouldn't consider a cloud implementation to be local.
replies(3): >>45065380 #>>45065518 #>>45065549 #
randomjoe2 ◴[29 Aug 25 15:39 UTC] No.45065518[source]▶
Local doesn't refer to "on metal" anymore to many people
replies(3): >>45065653 #>>45065663 #>>45067202 #
1. bee_rider ◴[29 Aug 25 17:45 UTC] No.45067202[source]▶
Local doesn’t need to be “on metal,” but I’m still confused as to what they are saying. Are they running some local cloud system?