Fly.io Postgres cluster down for 3 days, no word from them about it

Not a sarcastic or rhetorical question - how come the three big A clouds or even smaller ones (Hetzner,my favorite) are mostly so stable (give or take some outages) and anyone knows their internal engineering, architecture and practices to keep systems that much stable?

IMHO it is their approach. I use Hetzner and OVH (and their other variants for lower budget clients) for our EU clients. They do not use buzz words like "deploy app server", "cloud clusters", "turbo charge this app". They are simply providing VPS and similarly configured droplets. They are also established and don't want to mess around with very modern experimental infrastructures.

Same goes for Digital Ocean. No buzz words. Just hosting with droplets. They simply say "here pick a linux distro, configure whatever and don't ask us much about app support". I use their Linux distros for my own apps and if want anything extra I just install it and suffer my own actions' consequences. Not theirs.

DO, OVH, and Hetzner are more stable because they don't use buzzwords?

There isn't really secret sauce to it in 2023. The techniques, processes, and etc have pretty much been documented over the past 20 years.

But if you are wondering how AWS manages to be so good at it at such scale? Hosting infrastructure is incredibly complicated and AWS employs something like 100k people. Seemingly small AWS services employ more engineers than Fly.io.

That being said my take is that what's happening at Fly.io is a lack of leadership. There are not the right people in the right positions clearly. I've worked infra at companies from 5 people to, well Rackspace, and I'm having a hard time imagining so much time passing with.. Essentially a piece of infra MIA and impacting users.

I guess what OP is getting at is that these providers stick to the battle tested proven bedrock and nothing like "run your app where your users are" which I find interesting because that too can be done with any cloud that has a Datacenter in the region where you happen to have users.

So this "closer to your users" voodoo is a little beyond me.

The 'where each user is' is implicit, the expectation is that you're some kind of global SaaS, and you want low latency whereever your users are.

Sure you can do that with any cloud (or multiple) that has datacenters in a suitable spread of regions, but I suppose the point (or claimed point, selling point, if you like) is that that's more difficult or more expensive to coordinate. Fly says 'give us one container spec and tell us in which regions to run it', not 'we give you machines/VMs in which regions you want, figure it out'. It's an abstraction on top of 'battle tested proven bedrock' providers in a sense, except that I believe they run their own metal (to keep costs down, presumably).

Some workloads are surely latency sensitive but some of those transactional CRUD systems don't need that much closer to the edge is my possibly flawed opinion.

I mean chat or e-commerce yes, the edge and all.

But for a ticketing system, invoicing solution or such, a few hundred millisecons are not that much of a big deal but compliance, regulations matter more.

Scale the technical difficulty and innovation of the product with the size and competency of the team. The market will always say they want more and the job of the company leaders is to know when to say no. AWS did not begin with everything it offers now but rather started with fairly boring things (even for the time) that they expanded over time. This was after a decade of learning how to do this internally so they weren't starting from scratch.

I think the core issue is that they venomously don't want to act like a corporation. Which is great for early marketing and adoption but there's a reason successful B2B corporations act like they do. It's less fun and it's less endearing but it also annoys customers significantly less. I mean, the CEO has "Interim Food Taster" as his title on LinkedIn.