Llama 3.1 405B now runs at 969 tokens/s on Cerebras Inference

I am wondering how much cost is needed for serving at such a latency. Of course for customers, static cost depends on the pricing strategy. But still, the cost really determines how widely this can be adopted. Is it only for those business that really need the latency, or this can be generally deployed.

Maybe it could become standard for everyone to make giant chips and use SRAM?

How many SRAM manufacturers are there? Or does it somehow need to be fully integrated into the chip?

SRAM is usually produced on the same wafer as the rest of the logic. SRAM on an external chip would lose many of the advantages without being significantly cheaper.