Unless you can keep your compute at 70% average utilization for 5 years - you will never save money purchasing your hardware compared to renting it.
2) Hardware optimization (the exact GPU you want may not always be available for some providers)
3) Not subject to price changes
4) Not subject to sudden Terms of Use changes
5) Know exactly who is responsible if something isn't working.
6) Sense of pride and accomplishment + Heating in the winter
$3,000 is well under many "oopsie billsies" from cloud providers.
And that's outside of the whole "I own it" side of the conversation, where things like latency, control, flexibility, & privacy are all compelling reasons to be willing to spend slightly more.
I still run quite a number of LLM services locally on hardware I bought mid-covid (right around 3k for a dual RTX3090 + 124gb system ram machine).
It's not that much more than you'd spend if you're building a gaming machine anyways, and the nifty thing about hardware I own is that it usually doesn't stop working at the 5 year mark. I have desktops from pre-2008 still running in my basement. 5 year amortization might have the cloud win, but the cloud stops winning long before most hardware dies. Just be careful about watts.
Personally - I don't think pi clusters really make much sense. I love them individually for certain things, and with a management plane like k8s, they're useful little devices to have around. But I definitely wouldn't plan to get good performance from 10 of them in a box. Much better off spending roughly the same money for a single large machine unless you're intentionally trying to learn.
But also when it comes to Vast/RunPod it can be annoying and genuinely become more expensive if you have to rent 2x the number of hours because you constantly have to upload and download data, checkpoints, continuous storage costs, transfer data to another server because the GPU is no longer available, etc. It's just less of a headache if you have an always available GPU with a hard drive plugged into the machine and that's it
If it's for personal use, do whatever... there's nothing wrong with buying a $60,000 sports car if you get a lot of enjoyment out of driving it. (you could also lease if you want to trade up to the "faster model" next year) For business, renting (and managed hosting) makes more sense.
Plus cloud gaming is always limited in range of games, there are restrictions on how you can use the PC (like no modding and no swapping savegames in or out).
Like, if you buy that card it can still be processing things for you a decade from now.
Or you can get 3 months of rental time.
---
And yes, there is definitely a point where renting makes more sense because the capital outlay becomes prohibitive, and you're not reasonably capable of consuming the full output of the hardware.
But the cloud is a huge cash cow for a reason... You're paying exorbitant prices to rent compared to the cost of ownership.
If I spill something on my own hardware, the max out-of-pocket amount I lose is the amount I spent on that hardware.
If I run up an AWS/GCP/Azure bill accidentally... the max out-of-pocket amount I lose is often literally unbounded. Are there some guardrails you can put around this? Sure. But they're often confusing, misleading, delayed, or riddled with "holes" which they don't catch.
Ex - the literal best AWS offers you is delayed "billing alarms" which need to be manually enabled and configured, and even then don't cover all the services you might incur billing charges for.
It's not that "Oopsies" can't happen locally - it's that even if they do, I have a clear understanding of the potential costs by default, and they're much less intangible than "I left a thing running overnight and I now I owe AWS a new car worth of cash".
The worst case for a misconfigured bit of software locally is that my machine stalls and my services go down (ex - overloaded). The worst case for a misconfigured bit of software in AWS is literal bankruptcy.
Think about that for a minute.