←back to thread

I regret building this $3000 Pi AI cluster

(www.jeffgeerling.com)
468 points speckx | 2 comments | 19 Sep 25 14:28 UTC | HN request time: 0s | source
Show context
Aurornis ◴[19 Sep 25 14:52 UTC] No.45302320[source]▶
I thought the conclusion should have been obvious: A cluster of Raspberry Pi units is an expensive nerd indulgence for fun, not an actual pathway to high performance compute. I don’t know if anyone building a Pi cluster actually goes into it thinking it’s going to be a cost effective endeavor, do they? Maybe this is just YouTube-style headline writing spilling over to the blog for the clicks.

If your goal is to play with or learn on a cluster of Linux machines, the cost effective way to do it is to buy a desktop consumer CPU, install a hypervisor, and create a lot of VMs. It’s not as satisfying as plugging cables into different Raspberry Pi units and connecting them all together if that’s your thing, but once you’re in the terminal the desktop CPU, RAM, and flexibility of the system will be appreciated.

replies(11): >>45302356 #>>45302424 #>>45302433 #>>45302531 #>>45302676 #>>45302770 #>>45303057 #>>45303061 #>>45303424 #>>45304502 #>>45304568 #
glitchc ◴[19 Sep 25 15:01 UTC] No.45302424[source]▶
I did some calculations on this. Procuring a Mac Studio with the latest Mx Ultra processor and maxing out the memory seems to be the most cost effective way to break into 100b+ parameter model space.
replies(8): >>45302483 #>>45302490 #>>45302620 #>>45302698 #>>45302777 #>>45302916 #>>45302937 #>>45304489 #
1. the8472 ◴[19 Sep 25 15:20 UTC] No.45302620[source]▶
You could try getting a DGX Thor devkit with 128GB unified memory. Cheaper than the 96GB mac studio and more FLOPs.
replies(1): >>45305690 #
2. glitchc ◴[19 Sep 25 19:46 UTC] No.45305690[source]▶
Yeah but slower memory compared to the M3 Ultra. There's a big difference in memory bandwidth, which seems to be a driving factor for inferencing. Training on the other hand, it's probably a lot faster.