Does this get us closer to fully and performant distributed LLMs?
Instead of spending ludicrous amounts on Nvidia GPUs, can just deploy commodity clusters of AMD EPYC servers with tons of memory, NVMe disks, and 40G or 100G networking which is vastly less expensive.
Goodbye Nvidia moat!