←back to thread

WebSockets cost us $1M on our AWS bill

(www.recall.ai)
362 points tosh | 1 comments | 06 Nov 24 18:50 UTC | HN request time: 0.218s | source
Show context
trollied ◴[06 Nov 24 21:16 UTC] No.42069524[source]▶
>In a typical TCP/IP network connected via ethernet, the standard MTU (Maximum Transmission Unit) is 1500 bytes, resulting in a TCP MSS (Maximum Segment Size) of 1448 bytes. This is much smaller than our 3MB+ raw video frames.

> Even the theoretical maximum size of a TCP/IP packet, 64k, is much smaller than the data we need to send, so there's no way for us to use TCP/IP without suffering from fragmentation.

Just highlights that they do not have enough technical knowledge in house. Should spend the $1m/year saving on hiring some good devs.

replies(5): >>42069956 #>>42070181 #>>42070248 #>>42070804 #>>42070811 #
karamanolev ◴[06 Nov 24 21:45 UTC] No.42069956[source]▶
I fail to see how TCP/IP fragmentation really affects this use case. I don't know why it's mentioned and given that there aren't multiple network devices with different MTUs it will cause issues. Am I right? Is that the lack of technical knowledge you're referring to or am I missing something?
replies(1): >>42069979 #
drowsspa ◴[06 Nov 24 21:47 UTC] No.42069979[source]▶
Sounds weird that apparently they expected to send 3 MB in a single TCP packet
replies(2): >>42070420 #>>42074549 #
ahoka ◴[07 Nov 24 07:57 UTC] No.42074549[source]▶
Especially considering there are no packets in TCP.
replies(1): >>42086080 #
1. rstuart4133 ◴[08 Nov 24 11:45 UTC] No.42086080[source]▶
There are no packets on the user's API. But under the hood everything is sent in packets, numbered, ACK'ed and checksumed. The maximum packet size supported by IP is 64KB, as they say. I'm surprised the kernel supports that because I'm not aware of any real device that supports packets that big (Ethernet Jumbo Frames are only 9KB), but I guess it must.