←back to thread

I regret building this $3000 Pi AI cluster

(www.jeffgeerling.com)
468 points speckx | 2 comments | 19 Sep 25 14:28 UTC | HN request time: 0.513s | source
Show context
densh ◴[19 Sep 25 18:08 UTC] No.45304632[source]▶
For anyone interested in playing with distributed systems, I'd really recommend getting a single machine with latest 16-core CPU from AMD and just running 8 virtual machines on it. 8 virtual machines, with 4 hyper threads pinned per machine, and 1/8 of total RAM per machine. Create a network between them virtually within your virtualization software of choice (such as Proxmox).

And suddenly you can start playing with distributed software, even though it's running on a single machine. For resiliency tests you can unplug one machine at a time with a single click. It will annihilate a Pi cluster in Perf/W as well, and you don't have to assemble a complex web of components to make it work. Just a single CPU, motherboard, m.2 SSD, and two sticks of RAM.

Naturally, using a high core count machine without virtualization will get you best overall Perf/W in most benchmarks. What's also important but often not highlighted in benchmarks in Idle W if you'd like to keep your cluster running, and only use it occasionally.

replies(6): >>45305155 #>>45305387 #>>45305468 #>>45305628 #>>45307364 #>>45313651 #
qmr ◴[19 Sep 25 19:42 UTC] No.45305628[source]▶
No need for so much CPU power, any old quad core would work.
replies(2): >>45306905 #>>45309795 #
anaganisk ◴[20 Sep 25 02:46 UTC] No.45309795[source]▶
Aren’t newer CPUs especially AMDs more energy efficient?
replies(2): >>45310940 #>>45314677 #
AnthonyMouse ◴[20 Sep 25 06:22 UTC] No.45310940[source]▶
Newer CPUs have significantly better performance per watt under load, essentially by being a lot faster while using a similar amount of power. Idle CPU power consumption hasn't changed much in 10+ years simply because by that point it was already a single digit number of watts.

The thing that matters more than the CPU for idle power consumption is how efficient the system's power supply is under light loads. The variance between them is large and newer power supplies aren't all inherently better at it.

replies(1): >>45314741 #
1. 0manrho ◴[20 Sep 25 16:26 UTC] No.45314741[source]▶
Also worth noting, as this is a common point for the homelabbers out there, fans in surplus enterprise hardware can actually be a significant source of not just noise, but power usage, even at idle.

I remember back in the R710 days (circa 2008 and Nehalem/Westmere cpu's) that under like 30% cpu load, most of your power draw came from fans that you couldn't spin down below a certain threshold without an firmware/idrac script, as well as what you mentioned about those PSU's being optimized for high sustained loads and thus being inefficient at near idle and low usage.

IIRC System Idle power profile on those was only like 15% CPU (that's combined for both CPUs), with the rest being fans, ram and the various other vendor stuff (iDrac, PERC etc) and low-load PSU inefficiencies.

Newer hardware has gotten better, but servers are still generally engineered for above 50% sustained loads rather than under, and those fans still can easily pull a dozen plus watts even at very low usage each in those servers (of course, depends on exact model), so, point being, splitting hairs over a dozen watts or so between CPU's is a bit silly when your power floor from fans and PSU inefficiencies alone puts you at 80W+ draw anyway, not to mention the other components (NIC, Drives, Storage controller, OoB, RAM etc). Also, this is primarily relevant for surplus servers, but lot of people building systems at home for the usecase relevant to this discussion often turn to or are recommended these servers, so just wanted to add this food for thought.

replies(1): >>45316353 #
2. AnthonyMouse ◴[20 Sep 25 19:10 UTC] No.45316353[source]▶
Yeah, the server vendors give negative fucks about idle power consumption. I have a ~10 year old enterprise desktop quad core with a full-system AC power consumption of 6 watts while powered on and idle. I've seen enterprise servers of a similar vintage -- from the same vendor -- draw 40 watts when they're off.