←back to thread

Llama 3.1 405B now runs at 969 tokens/s on Cerebras Inference

(cerebras.ai)
426 points benchmarkist | 1 comments | 19 Nov 24 00:15 UTC | HN request time: 2.136s | source
Show context
brcmthrowaway ◴[19 Nov 24 03:03 UTC] No.42179727[source]▶
So out of all AI chip startups, Cerebras is probably the real deal
replies(2): >>42179835 #>>42179935 #
icelancer ◴[19 Nov 24 03:46 UTC] No.42179935[source]▶
Groq is legitimate. Cerebras so far doesn't scale (wide) nearly as good as Groq. We'll see how it goes.
replies(2): >>42180141 #>>42180942 #
1. hendler ◴[19 Nov 24 04:41 UTC] No.42180141[source]▶
Google TPUs, Amazon, a YC funded ASIC/FPGA company, a Chinese Co. all have custom hardware too that might scale well.