←back to thread

Kvcached: Virtualized, elastic KV cache for LLM serving on shared GPUs

(www.notion.so)
69 points Jrxing | 1 comments | 21 Oct 25 17:29 UTC | HN request time: 0.216s | source
Show context
CharlesW ◴[21 Oct 25 20:06 UTC] No.45661001[source]▶
Actual title: "Solve the GPU Cost Crisis with kvcached: A library to enable virtualized, elastic KV cache for LLM serving on shared GPUs"
replies(1): >>45672497 #
1. dang ◴[22 Oct 25 17:37 UTC] No.45672497[source]▶
Yes, we've put that in the title above (shortened to fit HN's 80 char limit). Submitted title was "Time to build a GPU OS? Here is the first step".