←back to thread

WebSockets cost us $1M on our AWS bill

(www.recall.ai)
261 points tosh | 2 comments | 06 Nov 24 18:50 UTC | HN request time: 0.001s | source
Show context
trollied ◴[06 Nov 24 21:16 UTC] No.42069524[source]▶
>In a typical TCP/IP network connected via ethernet, the standard MTU (Maximum Transmission Unit) is 1500 bytes, resulting in a TCP MSS (Maximum Segment Size) of 1448 bytes. This is much smaller than our 3MB+ raw video frames.

> Even the theoretical maximum size of a TCP/IP packet, 64k, is much smaller than the data we need to send, so there's no way for us to use TCP/IP without suffering from fragmentation.

Just highlights that they do not have enough technical knowledge in house. Should spend the $1m/year saving on hiring some good devs.

replies(5): >>42069956 #>>42070181 #>>42070248 #>>42070804 #>>42070811 #
1. hathawsh ◴[06 Nov 24 22:09 UTC] No.42070248[source]▶
Why do you say that? Their solution of using shared memory (structured as a ring buffer) sounds perfect for their use case. Bonus points for using Rust to do it. How would you do it?

Edit: I guess perhaps you're saying that they don't know all the networking configuration knobs they could exercise, and that's probably true. However, they landed on a more optimal solution that avoided networking altogether, so they no longer had any need to research network configuration. I'd say they made the right choice.

replies(1): >>42070439 #
2. maxmcd ◴[06 Nov 24 22:22 UTC] No.42070439[source]▶
Yes, maybe they're talking about this: https://en.wikipedia.org/wiki/TCP_window_scale_option