←back to thread

39 points ingve | 1 comments | | HN request time: 0s | source
Show context
cultofmetatron ◴[] No.45793910[source]
Id argue that if you are in the position where you legitimately NEED kafka, you hopefully also know what you're doing. You're outside the audience for the "just use postres" crowd. That said, if you're in a startup with a few thousand users, just use postgres is still solid advice.
replies(1): >>45793967 #
threatofrain ◴[] No.45793967[source]
If you need some kind of event streaming system there are other choices which have less dev ops burden, such as just using any particular cloud's proprietary or managed offerings. I've seen two companies on NATS so far, I'm trying it out myself for size as well.

There are plenty of choices between PSQL & Kafka. It's not like you take one step north and you're in the "oh no you better know what you're doing" territory.

replies(1): >>45794284 #
strken ◴[] No.45794284[source]
The problem with taking one step north and leaving the border of Postgres is what you lose, not the direct ops burden.

Postgres land is a comfy place filled with transactions across all your data at once, one backup solution that you (hopefully) have had running for months or years and has been thoroughly tested, and ACID compliance. You have a single host, probably, which means that you are neither Available nor Partion-tolerant, but at least you are Consistent.

The moment you expand beyond a single database host you now have a distributed system, and woe unto you if you don't understand what that means.

replies(3): >>45794354 #>>45794363 #>>45796327 #
threatofrain ◴[] No.45794363[source]
If you wanted such simplicity then nothing is stopping you from running single-node NATS or even just Redis. You always had all the simplicity and consistency you wanted.
replies(1): >>45794498 #
strken ◴[] No.45794498[source]
The problem is that now you use Postgres for 95% of your system, and also Redis or NATS, which means you lose the ability to atomically commit changes to your database and send a message in one transaction.

You can work around this, but to the best of my knowledge you can't have consistency (between your existing Postgres database and your separate queue or event log) and simplicity.

replies(2): >>45796999 #>>45803218 #
1. hbogert ◴[] No.45796999{5}[source]
Indeed, only eventual consistency. The article approaches this subject and mentions the use of the outbox pattern and/or using tools like Debezium.