Use One Big Server (2022)

I've been doing hybrid colo+public cloud for over a decade and it's always been the most cost effective route at a certain scale. That specific break even point is lowering over time with the density and cost effectiveness of hardware.

Sure you need net/infra admins but the software and hardware these days are pretty management friendly and you'll find you still need (often more expensive "cloud") admins so you're not offsetting much management cost there.

Colocation is plentiful and providers often aggregate and resell bandwidth from their preferred carriers.

At one point we were up to 8 dell vrtx clusters and a few SANs, with 500+ VMs from huge msSQL servers to kube clusters the public cloud bill would have been well into the 6 figures even with preferred pricing and reserved instances. Our colocation bill was $2400/mo and that was mostly for power. The one thing that always surprised me was how much faster everything was - every time we had to scale-over into the cloud the public cloud node was noticably slower even for identical CPU generations and vcpu.

You need to be very keen about server deals, updates, support contracts and licenses - but it's really manageable and interconnecting with the cloud is trivial at this point - you can get a "cloud connect" fiber drop to your preferred cloud provider and connect your colo infra to your vpc.

Colocation to me means you buy your own hardware and rent only the rack space (and power and connectivity) from the datacenter. Is that really what you're talking about? If so, why do you choose this over renting bare metal servers?

Because it's your hardware in the colo, so if money becomes dire, you can extend the servers lifetime beyond the standard depreciation schedule. Your rented bare metal servers might be slightly cheaper than a respective EC2 instance, but you stop paying that bill, it's gonna go poof, same as the EC2 instance.

Not always - you can lease your servers from the vendor as well, in which case you're renting the rack space, power and cooling from the datacenter and you're renting the servers from the vendor - most of the leases are designed so you can refresh your hardware every 4-5 years and it's usually still cheaper than renting from a dedicated hosting company.

Once you have an established baseline for your server needs - it's almost always more capital friendly to buy the servers and keep them running for the ~5 reliable years you'll get out of them - usually break even here is 2-3 years vs renting from a provider. If you're running your servers until they fail you'll get 7-10 years out of them, provided the power cost is still worth running them (usually that is also around the 8-10 year mark depending on your power cost).

So there are many reasons you'd buy vs rent - including capital deductions and access to cheap interest rates. You can also get some pretty crazy deals (like 33% of new price) by buying 2-3 year old equipment, then continue to run them for another 4-5 years, which is the lowest cost scenario if you don't need bleeding edge.

What about the cost of having people actually go to the datacenter to install hardware, and go again whenever there is a hardware problem, possibly resulting in much longer downtimes than with a rented server?

Especially for the "one (or a few) big server" scenario in the article, that would seem to me a pretty big factor.

At 1 rack scale you're saving ~20-30k/mo in cloud fees - you can hire an excellent sysadmin in the 12-15k/mo range and they can do a lot more than just go to the datacenter as needed.

But we're not comparing colo to cloud fees, we're comparing colo to renting a server.

I went with buying and colocation because I found I sleep better this way than when I used to rent servers in a distant datacenter and have to count on techs I'd never met working on hardware I'd never seen if anything went wrong. In my case, I live near the datacenter, so I can be hands-on fairly quickly if something goes wrong that I can't handle remotely.

And I can do whatever I want with the hardware. When I bought my servers, they came with disk controllers with non-optional RAID, as almost all of them do. I wanted to run RAIDz2 in FreeBSD/ZFS, so I swapped in non-RAID controllers. They were just a few bucks, but having that ability meant I could choose from a wider range of servers.