←back to thread

Distributed systems programming has stalled

(www.shadaj.me)
287 points shadaj | 2 comments | 27 Feb 25 16:12 UTC | HN request time: 0.485s | source
Show context
margorczynski ◴[27 Feb 25 18:20 UTC] No.43196933[source]▶
Distributed systems are cool but most people don't really get how much complexity it introduces which leads them to fad-driven decisions like using Event Sourcing where there is no fundamental need to use it. I've seen projects getting burned because of the complexity and overhead it introduces where "simpler" approaches worked well and were easy to extend/fix. Hard to find and fix bugs, much slower feature addition and lots of more goodies the blogs with toy examples don't speak about.
replies(3): >>43196997 #>>43197027 #>>43204750 #
1. nine_k ◴[27 Feb 25 18:27 UTC] No.43196997[source]▶
The best recipe I know is to start from a modular monolith [1] and split it when and if you need to scale way past a few dozen nodes.

Event sourcing is a logical structure; you can implement it with SQLite or even flat files, locally, if you your problem domain is served well by it. Adding Kafka as the first step is most likely a costly overkill.

[1]: https://awesome-architecture.com/modular-monolith/

replies(1): >>43197153 #
2. margorczynski ◴[27 Feb 25 18:43 UTC] No.43197153[source]▶
What you're speaking of is a need/usability-based design and extension where you design the solution with certain "safety valves" that let you scale it up when needed.

This is in contrast to the fad-driven design and over-engineering that I'm speaking of (here I simply used ES as an example) that is usually introduced because someone in power saw a blog post or 1h talk and it looked cool. And Kafka will be used because it is the most "scalable" and shiny solution, there is no pros-vs-cons analysis.