←back to thread

AI PCs Aren't Good at AI: The CPU Beats the NPU

(github.com)
486 points dbreunig | 1 comments | 16 Oct 24 19:44 UTC | HN request time: 0.205s | source
1. ein0p ◴[17 Oct 24 03:49 UTC] No.41866247[source]▶
Memory bound workload is memory bound. Doesn’t matter how many TOPS you have if you’re sitting idle waiting on DRAM during generation. You will, however notice a difference in prefill for long prompts.