←back to thread

AI PCs Aren't Good at AI: The CPU Beats the NPU

(github.com)
486 points dbreunig | 1 comments | 16 Oct 24 19:44 UTC | HN request time: 0.621s | source
Show context
freehorse ◴[17 Oct 24 01:48 UTC] No.41865681[source]▶
I always thought that the main point of NPUs is energy efficiency (and being able to run ML models without taking over all computer resources, making it practical to integrate ML applications in the OS itself in ways that it does not disturb the user or the workflow) rather than being exceptionally faster. At least this has been my experience with running stable diffusion on macs. Similar to using other specialised hardware like media encoders; they are not necessarily faster than a CPU if you throw a dozen+ cpu cores on the task, but it will draw a minuscule part of the power.
replies(1): >>41899887 #
1. tokyolights ◴[21 Oct 24 01:20 UTC] No.41899887[source]▶
Not sure why this isn't discussed more here. I think exactly the same, the NPU occupies more silicon area because it has custom circuits specifically to reduce the number of cycles (and thus energy) needed to perform those calculations. Doesn't necessarily mean that a CPU wouldn't be able to bulldoze through it faster (with much more energy).