←back to thread

Distributed systems programming has stalled

(www.shadaj.me)
287 points shadaj | 2 comments | 27 Feb 25 16:12 UTC | HN request time: 0.441s | source
Show context
bsnnkv ◴[27 Feb 25 16:52 UTC] No.43196091[source]▶
Last month I switched from a role working on a distributed system (FAANG) to a role working on embedded software which runs on cards in data center racks.

I was in my last role for a year, and 90%+ of my time was spent investigating things that went "missing" at one of many failure points between one of the many distributed components.

I wrote less than 200 lines of code that year and I experienced the highest level of burnout in my professional career.

The technical aspect that contributed the most to this burnout was both the lack of observability tooling and the lack of organizational desire to invest in it. Whenever I would bring up this gap I would be told that we can't spend time/money and wait for people to create "magic tools".

So far the culture in my new embedded (Rust, fwiw) position is the complete opposite. If you're burnt out working on distributed systems and you care about some of the same things that I do, it's worth giving embedded software dev a shot.

replies(24): >>43196122 #>>43196159 #>>43196163 #>>43196180 #>>43196239 #>>43196674 #>>43196899 #>>43196910 #>>43196931 #>>43197177 #>>43197902 #>>43198895 #>>43199169 #>>43199589 #>>43199688 #>>43199980 #>>43200186 #>>43200596 #>>43200725 #>>43200890 #>>43202090 #>>43202165 #>>43205115 #>>43208643 #
bob1029 ◴[27 Feb 25 18:18 UTC] No.43196910[source]▶
> Whenever I would bring up this gap I would be told that we can't spend time/money and wait for people to create "magic tools".

I've never once been granted explicit permission to try a different path without being burdened by a mountain of constraints that ultimately render the effort pointless.

If you want to try a new thing, just build it. No one is going to encourage you to shoot holes through things that they hang their own egos from.

replies(1): >>43200044 #
1. DrFalkyn ◴[28 Feb 25 00:02 UTC] No.43200044[source]▶
Hope you can justify that during sprint planning / standup
replies(1): >>43200280 #
2. bob1029 ◴[28 Feb 25 00:35 UTC] No.43200280[source]▶
If you are going to just build it in the absence of explicit buy-in, you certainly shouldn't spend time on the standup talking about it. Wait until your idea is completely formed and then drop a 5 minute demo on the team.

It can be challenging to push through to a completed demo without someone cheering you on every morning. I find this to be helpful more than hurtful if we are interested in the greater good. If you want to go against the grain (everyone else on the team), then you need to be really sure before you start wasting everyone else's time. Prove it to yourself first.