←back to thread

Deploying DeepSeek on 96 H100 GPUs

(lmsys.org)
281 points GabrielBianconi | 1 comments | 29 Aug 25 14:07 UTC | HN request time: 0.208s | source
Show context
34679 ◴[29 Aug 25 14:44 UTC] No.45064819[source]▶
"By deploying this implementation locally, it translates to a cost of $0.20/1M output tokens"

Is that just the cost of electricity, or does it include the cost of the GPUs spread out over their predicted lifetime?

replies(3): >>45064954 #>>45066023 #>>45071720 #
dragonslayer56 ◴[29 Aug 25 14:54 UTC] No.45064954[source]▶
” Our implementation, shown in the figure above, runs on 12 nodes in the Atlas Cloud, each equipped with 8 H100 GPUs.”

Maybe the cost of renting?

replies(2): >>45065147 #>>45065503 #
34679 ◴[29 Aug 25 15:09 UTC] No.45065147[source]▶
I'm confused because I wouldn't consider a cloud implementation to be local.
replies(3): >>45065380 #>>45065518 #>>45065549 #
randomjoe2 ◴[29 Aug 25 15:39 UTC] No.45065518[source]▶
Local doesn't refer to "on metal" anymore to many people
replies(3): >>45065653 #>>45065663 #>>45067202 #
mwcz ◴[29 Aug 25 15:51 UTC] No.45065663[source]▶
"On metal" is muddied too. I've heard people refer to web apps running in an OCI container as being "bare metal" deployment, as opposed to AWS or whatever hosting platform.

That's silly, but the idea that "local" is not the opposite of remote is even sillier.

replies(2): >>45065742 #>>45065883 #
1. dtech ◴[29 Aug 25 16:06 UTC] No.45065883[source]▶
If you do bare metal as not being under a VM it fits. OCI on linux is cgroup so that counts as not a VM I'd say. Or at least it's a layer closer to the metal than a typical VM running OCI images.

I a Java app running on Linux bare metal?