Fly.io Postgres cluster down for 3 days, no word from them about it

I adore DO. They’re seriously underrated. I love how they’ll just give you a server and say here, have at it. No abstractions, no fancy crap, just get out of my way and let me do my thing.

I wish I could say the same. My ISP and DO have absolutely terrible peering, unfortunately a lot of our internal stuff is hosted there. It’s always fun to git push/pull with 40kb/s on a gigabit connection.

Maybe you could VPN to or proxy through a box with good peering to you and DO?

I'm using Digital Ocean App platform, which does pretty much everything for me. It's very simple to use. I can run my app as a single developer without caring about infrastructure for 99% of the time.

Same! I've had my first server there for 10 years now. They added a lot of stuff in the meantime, they have AWS-like things you can do. But in terms of launching a VM that just works, they are a great choice.

When I’ve run into this in the past Cloudflare Warp has been a bit of a saviour. It’s a hassle free way to flick a switch and follow a different path over the network.

Do they offer authentication/authorization?

This is the one thing I need in every app and don't want to do myself.

wow! sub mbps indicates that there is indeed no peering at all (political issues?) but just a transit connection via an overloaded carryall.

collect some evidence, maybe someone wants to do something about it.

I've been using Supabase for authentication/authorization in my recent side project.

The main app is node/express running on Digital Ocean and it connects to directly to the Supabase hosted Postgres for most operations, but then uses the Supabase auth API for auth related stuff.

Saves a lot of time sending password reset emails etc and the entire project costs less than $5/mo in hosting costs.

I love their high value content about dev ops, I have learned most of what I know in this field tinkering with a VPS with their great tutorials on how to set up stuff.

They filled the Slicehost vacuum nicely in this area. That's where I got my start in running my own servers about 15 years ago and the tutorials were the driving factor.

I love DO for projects where I don't need control. For my side project, I eventually migrated to AWS after running into a lot of issues with DO.

Things like they don't give you the postgres root user on their managed postgres. And I ran into issues trying to capture the deployments in code. Their terraform providers are pretty good, but still leave something to be desired. For all its many warts, I'm much happier back on AWS. It did end up more expensive, but it's worth it for the fine grained control in my case.

But I spent the last 5 years as a DevOps/SRE, so... uh... I'm picky.

Yeah I hadn't seen those newer features until recently, the one-click deployments are super cool.

That's interesting, because granular control is why I enjoy DO, although I'm thinking about it from the server perspective. They set up a machine, give me root access, and that's literally it. I set up my own ssh keys, firewalls, and there's no additional abstraction that I have to learn. I might just be reminiscing because right now I'm on a team where we're writing terraform/helm/k8s in GCP and it makes me want to cry myself to sleep each night lol.

Would you consider a project like https://github.com/authcompanion/authcompanion2 for the authentication side? Missing anything?

historically, I've used Vultr, but I don't see anyone talking about it—I'm curious if anyone else has thoughts on them? (I've been happy, but then again my usage has been exceedingly basic)

In addition to Supabase Auth the sibling mentions (which I played with very briefly) I've been using clerk.dev (no affiliation) and it's great. Depending on your definition of doing it yourself it could be just want you want. You have to set some things up, you're not going to get things like row-level permissions you get out of the box w/ Supabase, but if you're looking for a quick implementation where things like password reset etc. are handled for you, it might be a good fit.

I've used Vultr for several years (hobby projects) with no issues. My favorite feature is having a BGP session from my VM, which is unusual among cloud providers. I have an AS and am able to advertise my own IPs from multiple Vultr instances (anycast).

I went to DO's site due to your comment and I don't see anywhere where I can just get a server. Do you mean a VPS/Droplet? (I'm looking under Products and Solutions.)

No I would not.

I don't like self hosting anything that requires its own process. And if I did decide to self host I would choose a more mature project.

This is a very young one man project delegating the heavy lifting to another one man project. And it doesn't appear to support social logins.

Not GP, but yes -- Droplets are DigitalOcean's "servers" (virtual, but nonetheless).

You boot one up in less than 30 seconds, and get ssh access to it almost immediately. It's very BS-free.

How do you get an AS?

The other commenter was correct - I meant a droplet. Should have been more explicit, apologies. But yeah if you're looking to learn how to work with backends, going through a droplet set up is by far the best way to get started IMO.

I like https://github.com/goauthentik It has Helm charts and a Terraform provider.

Have used both DO and Vultr for years. Put simply, DO is better, but Vultr isn’t terrible.

Higher number of outages at Vultr over 5 years, but none longer than a few hours. I can’t remember the last DO outage lasting more than a few minutes.

Experienced a Vultr routing problem that lasted several hours; they communicated about it, but it was still a long time to fix.

DO once did an auto-migration of a server to another cluster with an attendant outage that lasted a few minutes at most. No IP changes, completely transparent.

thanks for the feedback.

Seriously! They have an amazing article I followed one time to set up a k8s cluster to run any container I wanted with full automatic ssl provisioning/management and dns. Make a quick little yml file that includes what subdomain it wants to be and kubectl apply. The cluster was like $100 a month all-in and performed like a beast at huge traffic levels, and all I did was follow a tutorial.

I know that’s probably pretty easy for many, but I was pretty new to k8s and it felt like magic.

I agree. I can either abstract with the app platform or kubernetes, or I can go straight into the box myself and do whatever needs doing. It has been a real pleasure.

I think fly’s tooling feels better than doctl, but the infrastructure is incomparable at the end of the day. doctl has improved over time too, and with added pressure from newcomers I don’t doubt that it’ll continue to improve.

Same, it works really well.

Part of what inspired me to give fly.io a shot was that I didn’t love the monorepo deployment story on the app platform. Fly doesn’t have a solution to that, but I suppose I felt less tied to DO at the time because I wasn’t totally content anyways. I’ve discovered since then that I was actually doing it wrong, so I’m way happier. I’m pretty big on monorepos so their whole system fits my workflow remarkably well now.

I’d like to figure out how to prevent deployments when my code doesn’t change in one app, but does in another. At the moment, pushing anything at all will trigger all apps to rebuild and deploy again. Not a huge deal and several orders of magnitude less painful than not being able to deploy at all, haha.

Those are good things to know. I’ve been wondering about their managed databases recently, so I’ll keep that in mind.

I’m nowhere near as picky as you are, but maybe I’ll need to be at some point. As it is I mostly just build stuff and send it to the internet. If it builds and it does what I expected, I’m pretty happy! I don’t often need anything too special.

I really love DO except for one thing - you can't run your own firewall/router there (like opnSense). Really hard to link systems together.

I find myself going to DO docs on various setup things even when I'm not using said thing on DO (although I'm also a DO customer, and love them for the reasons you've stated).