←back to thread

66 points enether | 3 comments | | HN request time: 0.001s | source

The space is confusing to say the least.

Message queues are usually a core part of any distributed architecture, and the options are endless: Kafka, RabbitMQ, NATS, Redis Streams, SQS, ZeroMQ... and then there's the “just use Postgres” camp for simpler use cases.

I’m trying to make sense of the tradeoffs between:

- async fire-and-forget pub/sub vs. sync RPC-like point to point communication

- simple FIFO vs. priority queues and delay queues

- intelligent brokers (e.g. RabbitMQ, NATS with filters) vs. minimal brokers (e.g. Kafka’s client-driven model)

There's also a fair amount of ideology/emotional attachment - some folks root for underdogs written in their favorite programming language, others reflexively dismiss anything that's not "enterprise-grade". And of course, vendors are always in the mix trying to steer the conversation toward their own solution.

If you’ve built a production system in the last few years:

1. What queue did you choose?

2. What didn't work out?

3. Where did you regret adding complexity?

4. And if you stuck with a DB-based queue — did it scale?

I’d love to hear war stories, regrets, and opinions.

1. tacostakohashi ◴[] No.44018500[source]
UUCP
replies(1): >>44019268 #
2. RedShift1 ◴[] No.44019268[source]
People will call me crazy but why not SMTP for message queueing?
replies(1): >>44022660 #
3. mdaniel ◴[] No.44022660[source]
Because it's only INSERT not SELECT nor DELETE? Maybe you meant IMAP which does have APPEND <https://www.rfc-editor.org/rfc/rfc3501.html#section-6.3.11> to insert a new message into a folder, and a bazillion SELECT options, plus of course DELETE

I still would call that crazy, because of the mental tax of explaining to every new employee "wait, you're using IMAP for what?" but if it works for you, then great