←back to thread

66 points enether | 6 comments | | HN request time: 0.947s | source | bottom

The space is confusing to say the least.

Message queues are usually a core part of any distributed architecture, and the options are endless: Kafka, RabbitMQ, NATS, Redis Streams, SQS, ZeroMQ... and then there's the “just use Postgres” camp for simpler use cases.

I’m trying to make sense of the tradeoffs between:

- async fire-and-forget pub/sub vs. sync RPC-like point to point communication

- simple FIFO vs. priority queues and delay queues

- intelligent brokers (e.g. RabbitMQ, NATS with filters) vs. minimal brokers (e.g. Kafka’s client-driven model)

There's also a fair amount of ideology/emotional attachment - some folks root for underdogs written in their favorite programming language, others reflexively dismiss anything that's not "enterprise-grade". And of course, vendors are always in the mix trying to steer the conversation toward their own solution.

If you’ve built a production system in the last few years:

1. What queue did you choose?

2. What didn't work out?

3. Where did you regret adding complexity?

4. And if you stuck with a DB-based queue — did it scale?

I’d love to hear war stories, regrets, and opinions.

1. speedgoose ◴[] No.44003058[source]
I played with most message queues and I go with RabbitMQ in production.

Mostly because it has been very reliable for years in production at a previous company, and doesn’t require babysitting. Its recent versions also has new features that make it is a descent alternative to Kafka if you don’t need to scale to the moon.

And the logo is a rabbit.

replies(2): >>44019404 #>>44027228 #
2. swyx ◴[] No.44019404[source]
Datadog too. i often wonder how come more companies dont pick cute mascots. gives a logo, makes everyone have warm fuzzies immediately, creates pun opportunities.

inb4 "oh but you wont be taken seriously" well... datadog.

replies(1): >>44022601 #
3. DonsDiscountGas ◴[] No.44022601[source]
Hugging face clearly shares the same philosophy
replies(1): >>44053061 #
4. aitchnyu ◴[] No.44027228[source]
Just used it as a Celery (job queue) backend. How is it a Kafka alternative?
replies(1): >>44027948 #
5. speedgoose ◴[] No.44027948[source]
RabbitMQ streams: https://www.rabbitmq.com/docs/streams
6. taskforcegemini ◴[] No.44053061{3}[source]
but usually I only see the namw "huggingface" written, and I think of headcrabs from half-life instead