←back to thread

Anything can be a message queue if you use it wrongly enough (2023)

(xeiaso.net)
183 points crescit_eundo | 3 comments | 28 Aug 25 15:14 UTC | HN request time: 0s | source
Show context
adamcharnock ◴[28 Aug 25 18:12 UTC] No.45055206[source]▶
I had a developer colleague a while back who was toying with an idea that would require emitting and consuming a _lot_ of messages. I think it was somewhere on the order of 10k-100k/second. He was looking at some pretty expensive solutions IIRC.

I asked if the messages were all under 1.5kb, he said yes. I asked if at-most-one delivery was ok, he said yes. So I proposed he just grab a router and fire messages through it as UDP packets, then use BGP/ECMP to balance the packets between receivers. Add some queues on the router, then just let the receivers pick up the packets as fast as they could. You'd need some kind of feedback to manage back pressure, but ¯\_(ツ)_/¯

A fairly cheap way to achieve 1M+ messages per second.

I never got the chance to flesh-out the idea fully, but the simplicity of it tickled me. Maybe it would have worked, maybe not.

replies(1): >>45055387 #
HeyLaughingBoy ◴[28 Aug 25 18:29 UTC] No.45055387[source]▶
Isn't a fundamental property of a queue that it's FIFO?

UDP message delivery order is not guaranteed. Hell, UDP delivery itself is not guaranteed (although IME, messages don't usually get dropped unless they cross subnets).

replies(3): >>45055437 #>>45055745 #>>45081133 #
bdcravens ◴[28 Aug 25 18:34 UTC] No.45055437[source]▶
> UDP delivery itself is not guaranteed

> I asked if at-most-one delivery was ok, he said yes.

Use case satisfied.

replies(1): >>45056254 #
trelane ◴[28 Aug 25 19:45 UTC] No.45056254[source]▶
>> UDP delivery itself is not guaranteed

>> I asked if at-most-one delivery was ok, he said yes.

> Use case satisfied.

No. https://wiki.wireshark.org/DuplicatePackets

"ConnectionlessProtocols such as UDP won't detect duplicate packets, because there's no information in, for example, the UDP header to identify a packet so that packets can be recognized as duplicates. The data from that packet will be indicated twice (or even more) to the application; it's the responsibility of the application to detect duplicates (perhaps by supplying enough information in its headers to do so) and process them appropriately, if necessary"

replies(1): >>45064699 #
ranger_danger ◴[29 Aug 25 14:36 UTC] No.45064699[source]▶
protection from duplicate packets was never a requirement... you also assume there is no other identifying info inside the packets that might provide an inherent protection anyways
replies(1): >>45066986 #
trelane ◴[29 Aug 25 17:29 UTC] No.45066986[source]▶
> protection from duplicate packets was never a requirement..

Nope. This is what "at-most-one" means. Either zero or one. Not two.

Three is right out.

> you also assume there is no other identifying info inside the packets that might provide an inherent protection anyways

Yes, but then if we're positing a system on top of UDP, then it doesn't matter what the protocol is, TCP or UDP. We can also assume the underlying system we're relying on can also detect missing requests, or even ensure we have exactly one.

replies(1): >>45070660 #
1. ranger_danger ◴[29 Aug 25 23:47 UTC] No.45070660[source]▶
> Nope. This is what "at-most-one" means. Either zero or one. Not two.

But "is at-most-one ok?" doesn't necessarily mean that more is also unacceptable. All the valid and invalid conditions are not completely specified IMO.

replies(1): >>45070716 #
2. dragonwriter ◴[29 Aug 25 23:56 UTC] No.45070716[source]▶
> But "is at-most-one ok?" doesn't necessarily mean that more is also unacceptable.

If you propose delivering an at-most-once solution, the customer agrees that that is an acceptable parameter, and then you deliver a solution that has no at-most-once guarantee, you haven't delivered what you proposed, and you certainly have no reason to believe you've met the customer's needs.

replies(1): >>45070812 #
3. ranger_danger ◴[30 Aug 25 00:14 UTC] No.45070812[source]▶
The way OP framed it IMO sounded like a very informal conversation, not a written contract with formal specifications... I think it's possible the person agreeing at-most-one was "ok" quite possibly didn't care either way, and might even view supporting duplicates as a positive extra feature... but I don't think this is something either of us can speak to authoritatively.

> if we're positing a system on top of UDP, then it doesn't matter what the protocol is

And this is still true, and neither of us know if they would have designed it this way or not, but they could.