Most active commenters
  • Aeolun(3)

←back to thread

Use One Big Server (2022)

(specbranch.com)
343 points antov825 | 31 comments | | HN request time: 0.844s | source | bottom
Show context
runako ◴[] No.45085915[source]
One of the more detrimental aspects of the Cloud Tax is that it constrains the types of solutions engineers even consider.

Picking an arbitrary price point of $200/mo, you can get 4(!) vCPUs and 16GB of RAM at AWS. Architectures are different etc., but this is roughly a mid-spec dev laptop of 5 or so years ago.

At Hetzner, you can rent a machine with 48 cores and 128GB of RAM for the same money. It's hard to overstate how far apart these machines are in raw computational capacity.

There are approaches to problems that make sense with 10x the capacity that don't make sense on the much smaller node. Critically, those approaches can sometimes save engineering time that would otherwise go into building a more complex system to manage around artificial constraints.

Yes, there are other factors like durability etc. that need to be designed for. But going the other way, dedicated boxes can deliver more consistent performance without worries of noisy neighbors.

replies(11): >>45086252 #>>45086272 #>>45086760 #>>45087388 #>>45088476 #>>45089414 #>>45091154 #>>45091413 #>>45092146 #>>45092305 #>>45095302 #
shrubble ◴[] No.45086760[source]
It's more than that - it's all the latency that you can remove from the equation with your bare-metal server.

No network latency between nodes, less memory bandwidth latency/contention as there is in VMs, no caching architecture latency needed when you can just tell e.g. Postgres to use gigs of RAM and then let Linux's disk caching take care of the rest (and not need a separate caching architecture).

replies(1): >>45086889 #
matt-p ◴[] No.45086889[source]
The difference between a fairly expensive ($300) RDS instance + EC2 in the same region vs a $90 dedicated server with a NVME drive and postgres in a container is absolutely insane.
replies(2): >>45087248 #>>45088681 #
1. bspammer ◴[] No.45087248[source]
A fair comparison would include the cost of the DBA who will be responsible for backups, updates, monitoring, security and access control. That’s what RDS is actually competing with.
replies(9): >>45087378 #>>45087484 #>>45087756 #>>45088306 #>>45088314 #>>45090125 #>>45090795 #>>45091984 #>>45092441 #
2. matt-p ◴[] No.45087378[source]
Totally. My frustration isn't even price though RDS is literally just dog slow.
3. shrubble ◴[] No.45087484[source]
Paying someone $2000 to set that up once should result in the costs being recovered in what, 18 months?

If you’re running Postgres locally you can turn off the TCP/IP part; nothing more to audit there.

SSH based copying of backups to a remote server is simple.

If not accessible via network, you can stay on whatever version of Postgres you want.

I’ve heard these arguments since AWS launched, and all that time I’ve been running Postgres (since 2004 actually) and have never encountered all these phantom issues that are claimed as being expensive or extremely difficult.

replies(2): >>45089151 #>>45092739 #
4. yjftsjthsd-h ◴[] No.45087756[source]
As long as you also include the Cloud Certified DevOps Engineer™[0] to set up that RDS instance.

[0] A normal sysadmin remains vaguely bemused at their job title and the way it changes every couple years.

replies(1): >>45090765 #
5. vidarh ◴[] No.45088306[source]
I do consulting in this space, and we consistently make more money from people who insist on using cloud services, because their setups tend to need far more work.
replies(2): >>45091746 #>>45092002 #
6. sgarland ◴[] No.45088314[source]
You don’t need a DBA for any of those, you need someone who can read some docs. It’s not witchcraft.
replies(1): >>45091161 #
7. applied_heat ◴[] No.45089151[source]
$2k? That’s a $100k project for a medium size Corp
replies(2): >>45090118 #>>45091859 #
8. sysguest ◴[] No.45090118{3}[source]
hmm where did you get the numbers?

(what's "medium-size corp" and how did you come up with $100k ?)

replies(1): >>45091127 #
9. lelanthran ◴[] No.45090125[source]
The RDS solution doesn't need a technical person to set it up?

It doesn't need someone who knows how to use the labrythine AWS services and console?

replies(1): >>45092992 #
10. mrweasel ◴[] No.45090765[source]
It's also interesting that the cloud engineer can apparently be both a DBA, network-, storage- and backup engineer, but if you move the same services on-prem, you apparently need specialists for each task.

Sometimes even the certified cloud engineers can't tell you why an RDS behaves the way it does, nor can they really fix it. Sometimes you really do need a DBA, but that applies equally to on-prem and cloud.

I'm a sysadmin, but have been labelled and sold as: Consultant (sounds expensive), DevOps engineer, Cloud Engineer, Operations Expert and right now a Site Reliability Engineer.... I'm a systems administrator.

replies(3): >>45091144 #>>45092283 #>>45092497 #
11. Cthulhu_ ◴[] No.45090795[source]
While that's fair, most organizations I've worked at in the past decade have had a dedicated team for managing their cloud setup, which is also responsible for backups, updates, monitoring, security and access control. I don't think they're competing.
12. Aeolun ◴[] No.45091127{4}[source]
I’m assuming he’s talking about the corporate team of DBA’s that will spend weeks discussing the best way to copy a bunch of SQL files to S3
13. Aeolun ◴[] No.45091144{3}[source]
If you’ve started working in the industry more than about 15 years ago all the titles sound quaint.
14. Aeolun ◴[] No.45091161[source]
I’d argue that AWS is witchcraft a lot of the time. They’ll have all these they claim will work for everything, but you’ll always find one of the things you’d expect to be unavailable.
15. kiney ◴[] No.45091746[source]
same for me
16. christophilus ◴[] No.45091859{3}[source]
$200 does seem too low. $100k seems waaay too high. That sounds like an AWS talking point.
17. benterix ◴[] No.45091984[source]
You are aware that RDS needs backups, setting up monitoring properly, defining access, providing secrets management etc., and updates between major versions are not automatic?

RDS has a value. But for many teams the price paid for this value is ridiculously high when compared to other options.

replies(1): >>45094773 #
18. benterix ◴[] No.45092002[source]
Similar here - but in my case the reason is because of vendor lock-in - they spent years getting into AWS and any thought of getting out seems dreadful.
19. data_marsupial ◴[] No.45092283{3}[source]
Need to get Platform Engineer for a full house
20. steveBK123 ◴[] No.45092441[source]
My firm paid DBAs for RDS as well so..
21. icedchai ◴[] No.45092497{3}[source]
I haven't seen a company that hired DBAs in over 15 years. I think the "DevOps" movement sent them packing, along with SysAdmins.
replies(1): >>45097093 #
22. sahilagarwal ◴[] No.45092739[source]
I guess my non-management / non-business side is show here, but how can it be that much?? I still remember I designed a fairly simple cron job that took database backups when I was a junior developer.

It gets even easier now that you have cheap s3 - just upload the dump to s3 every day and set the s3 deletion policy to whatever is feasible for you.

replies(3): >>45093271 #>>45093583 #>>45097767 #
23. whstl ◴[] No.45092992[source]
Agree.

These comments sound super absurd to me, because RDS is difficult as hell to setup, unless you do it very frequently or already have it in IoC format, since one needs setting up a VPC, subnets, security groups, internet gateway, etc.

It's not like creating a DynamoDB, Lambda or S3 where a non-technical person can learn it in a few hours.

Sure, one might find some random Terraform file online to do this or vibe-code some CloudFormation, but that's not really a fair comparison.

24. fragmede ◴[] No.45093271{3}[source]
How much were you paid as a jr developer, and how long did it take you to set up? Then round up to mid-level developer, and add in hardware and software costs.
replies(1): >>45093572 #
25. dijit ◴[] No.45093572{4}[source]
That's a deflection. The question isn't about a developer's salary; it's about the fundamental difference between a one-time investment and a permanent cost.

Either way: 1 day of a mid-level developer in the majority of the world (basically: anywhere except Zurich, NYC or SF) is between €208 and €291. (Yearly salary of €50-€70k)

A junior developer's time for setup and the cost of hardware is practically a one-off expense. It's a few days of work at most.

The alternative you're advocating for (a recurring SaaS fee) is a permanent rent trap. That money is gone forever, with no asset or investment to show for it. Over a few years, you'll have spent tens of thousands of dollars for nothing. The real cost is not what you pay a developer; it's what you lose by never owning your tools.

replies(1): >>45098290 #
26. alemanek ◴[] No.45093583{3}[source]
I am not an expert here but I am currently researching for a planned project.

For backups, including Postgres, I was planning on paying Veeam ~$500 a year for a software license to backup the active node and Postgres database to s3/r2. Standby node would be getting streaming updates via logical replication.

There are free options as well but I didn’t want to cheap out on the backups.

It looks pretty turnkey. I am a software engineer not a sysadmin though. Still just theory as well as I haven’t built it out yet

27. pdhborges ◴[] No.45094773[source]
AWS can make major version upgrades automatically now with less downtime. I think they do the logical replication dance internally.
28. dijit ◴[] No.45097093{4}[source]
Sysadmins never left, they just got rebranded.
replies(1): >>45102739 #
29. nine_k ◴[] No.45097767{3}[source]
Taking database backups is relatively simple. What differentiates a good solution is the ease of restoring from a backup. This includes the certainty that the restored state would be a correct point-in-time state from the past, not an amalgamation of several such states.
30. fragmede ◴[] No.45098290{5}[source]
> The alternative you're advocating for

Not sure where I advocated for that. Could you point it out please?

31. icedchai ◴[] No.45102739{5}[source]
I actually agree with this. I meant you never seen roles with the "system administrator" job title, not that it actually disappeared as a function. DBAs on the other hand, I do think that has mostly been absorbed into other roles.