Anything can be a message queue if you use it wrongly enough (2023)

I had a developer colleague a while back who was toying with an idea that would require emitting and consuming a _lot_ of messages. I think it was somewhere on the order of 10k-100k/second. He was looking at some pretty expensive solutions IIRC.

I asked if the messages were all under 1.5kb, he said yes. I asked if at-most-one delivery was ok, he said yes. So I proposed he just grab a router and fire messages through it as UDP packets, then use BGP/ECMP to balance the packets between receivers. Add some queues on the router, then just let the receivers pick up the packets as fast as they could. You'd need some kind of feedback to manage back pressure, but ¯\_(ツ)_/¯

A fairly cheap way to achieve 1M+ messages per second.

I never got the chance to flesh-out the idea fully, but the simplicity of it tickled me. Maybe it would have worked, maybe not.

Isn't a fundamental property of a queue that it's FIFO?

UDP message delivery order is not guaranteed. Hell, UDP delivery itself is not guaranteed (although IME, messages don't usually get dropped unless they cross subnets).

> UDP delivery itself is not guaranteed

> I asked if at-most-one delivery was ok, he said yes.

Use case satisfied.

> UDP message delivery order is not guaranteed

My thinking was that ordering would be pretty unaffected when there is only a single hop. But yeah, we would have needed to test that under load.

>> UDP delivery itself is not guaranteed

>> I asked if at-most-one delivery was ok, he said yes.

> Use case satisfied.

No. https://wiki.wireshark.org/DuplicatePackets

"ConnectionlessProtocols such as UDP won't detect duplicate packets, because there's no information in, for example, the UDP header to identify a packet so that packets can be recognized as duplicates. The data from that packet will be indicated twice (or even more) to the application; it's the responsibility of the application to detect duplicates (perhaps by supplying enough information in its headers to do so) and process them appropriately, if necessary"

protection from duplicate packets was never a requirement... you also assume there is no other identifying info inside the packets that might provide an inherent protection anyways

> protection from duplicate packets was never a requirement..

Nope. This is what "at-most-one" means. Either zero or one. Not two.

Three is right out.

> you also assume there is no other identifying info inside the packets that might provide an inherent protection anyways

Yes, but then if we're positing a system on top of UDP, then it doesn't matter what the protocol is, TCP or UDP. We can also assume the underlying system we're relying on can also detect missing requests, or even ensure we have exactly one.

> Nope. This is what "at-most-one" means. Either zero or one. Not two.

But "is at-most-one ok?" doesn't necessarily mean that more is also unacceptable. All the valid and invalid conditions are not completely specified IMO.

> But "is at-most-one ok?" doesn't necessarily mean that more is also unacceptable.

If you propose delivering an at-most-once solution, the customer agrees that that is an acceptable parameter, and then you deliver a solution that has no at-most-once guarantee, you haven't delivered what you proposed, and you certainly have no reason to believe you've met the customer's needs.

The way OP framed it IMO sounded like a very informal conversation, not a written contract with formal specifications... I think it's possible the person agreeing at-most-one was "ok" quite possibly didn't care either way, and might even view supporting duplicates as a positive extra feature... but I don't think this is something either of us can speak to authoritatively.

> if we're positing a system on top of UDP, then it doesn't matter what the protocol is

And this is still true, and neither of us know if they would have designed it this way or not, but they could.

Even Azures native queue systems can guarantee that, because it gets hard to scale. You can only have one processor for any given queue, because otherwise one might take so long to process a message that others have finished processing others in the meantime.

But given OPs need for at-most-once, this doesn't really seem to matter in the first place.