←back to thread

It's time to replace TCP in the datacenter (2023)

(arxiv.org)
188 points ilove_banh_mi | 1 comments | 18 Nov 24 01:42 UTC | HN request time: 0s | source
Show context
UltraSane ◴[18 Nov 24 05:43 UTC] No.42170007[source]▶
I wonder why Fibre Channel isn't used as a replacement for TCP in the datacenter. It is a very robust L3 protocol. It was designed to connect block storage devices to servers while making the OS think they are directly connected. OSs do NOT tolerate dropped data when reading and writing to block devices and so Fibre Channel has a extremely robust Token Bucket algorithm. The algo prevents congestion by allowing receivers to control how much data senders can send. I have worked with a lot of VMware clusters that use FC to connect servers to storage arrays and it has ALWAYS worked perfectly.
replies(9): >>42170384 #>>42170465 #>>42170698 #>>42171057 #>>42171576 #>>42171890 #>>42174071 #>>42174140 #>>42175585 #
tonetegeatinst ◴[18 Nov 24 16:44 UTC] No.42174140[source]▶
Fiber is attractive. As someone who wants to upgrade to fiber, the main barrier to entry is the cost of switches and a router.

Granted I'm also trying to find a switch that supports ROCm and rdma. Not easy to find a high bandwidth switch that supports this stuff without breaking the bank.

replies(3): >>42174857 #>>42175957 #>>42176517 #
1. UltraSane ◴[18 Nov 24 20:23 UTC] No.42176517[source]▶
Fibre Channel is a routed protocol invented specifically to connect block storage arrays to remote servers while making the block storage look locally attached to the OS. And it works REALLY REALLY well.