←back to thread

It's time to replace TCP in the datacenter (2023)

(arxiv.org)
188 points ilove_banh_mi | 2 comments | 18 Nov 24 01:42 UTC | HN request time: 0.001s | source
Show context
parasubvert ◴[18 Nov 24 07:18 UTC] No.42170451[source]▶
This has already been done at scale with HTTP/3 (QUIC), it's just not widely distributed beyond the largest sites & most popular web browsers. gRPC for example is still on multiplexed TCP via HTTP/2, which is "good enough" for many.

Though it doesn't really replace TCP, it's just that the predominant requirements have changed (as Ousterhout points out). Bruce Davie has a series of articles on this: https://systemsapproach.substack.com/p/quic-is-not-a-tcp-rep...

Also see Ivan Pepelnjak's commentary (he disagrees with Ousterhout): https://blog.ipspace.net/2023/01/data-center-tcp-replacement...

replies(3): >>42171930 #>>42174573 #>>42177673 #
1. aseipp ◴[18 Nov 24 22:11 UTC] No.42177673[source]▶
QUIC and Homa are not remotely similar and have completely different design constraints. I have no idea why people keep bringing up QUIC in this thread other than "It's another thing that isn't TCP." Yes, many things are not-TCP. The details are what matter.
replies(1): >>42185943 #
2. parasubvert ◴[19 Nov 24 17:25 UTC] No.42185943[source]▶
Not remotely similar? Both are RPC (request/response) optimized and are focused on removing head of line blocking and strict need for FIFO message ordering in favour of multiplexing.

QUIC is more focused on the global web applications, but most datacentres also leverage the web protocols (REST on HTTP 1.1 or HTTP/2, gRPC HTTP/2) for their inter-process communication, just with a a lot more east-west traffic (arguably 10x for every 1x N/S flow). There's also a fair amount app-specific messaging stacks (usually L7 over TCP) like Kafka, NATS or AMQP which have their own L7 facilities for dealing with TCP drawbacks that might benefit from a retrofit like Homa, but it's not clear if it's worth the effort.

They are design approaches for solving similar requirements. Yes, homa deals with other things (makes ECMP load balancing easier) but also has blindspots on datacenter requirements like security: a lot of data centre traffic requires hop by hop TLS for authentication, integrity and privacy, QUIC explicitly focuses on improving latency of TLS handshakes.