←back to thread

AI PCs Aren't Good at AI: The CPU Beats the NPU

(github.com)
486 points dbreunig | 4 comments | 16 Oct 24 19:44 UTC | HN request time: 0s | source
Show context
jsheard ◴[16 Oct 24 20:16 UTC] No.41863390[source]▶
These NPUs are tying up a substantial amount of silicon area so it would be a real shame if they end up not being used for much. I can't find a die analysis of the Snapdragon X which isolates the NPU specifically but AMDs equivalent with the same ~50 TOPS performance target can be seen here, and takes up about as much area as three high performance CPU cores:

https://www.techpowerup.com/325035/amd-strix-point-silicon-p...

replies(4): >>41863880 #>>41863905 #>>41864412 #>>41865466 #
ezst ◴[16 Oct 24 21:13 UTC] No.41863905[source]▶
I can't wait for the LLM fad to be over so we get some sanity (and efficiency) back. I personally have no use for this extra hardware ("GenAI" doesn't help me in any way nor supports any work-related tasks). Worse, most people have no use for that (and recent surveys even show predominant hostility towards AI creep). We shouldn't be paying extra for that, it should be opt-in, and then it would become clear (by looking at the sales and how few are willing to pay a premium for "AI") how overblown and unnecessary this is.
replies(6): >>41863966 #>>41864134 #>>41865168 #>>41865589 #>>41865651 #>>41875051 #
1. kalleboo ◴[17 Oct 24 01:30 UTC] No.41865589[source]▶
> most people have no use for that

Apple originally added their NPUs before the current LLM wave to support things like indexing your photo library so that objects and people are searchable. These features are still very popular. I don't think these NPUs are fast enough for GenAI anyway.

replies(2): >>41865887 #>>41865902 #
2. wmf ◴[17 Oct 24 02:31 UTC] No.41865887[source]▶
MS Copilot and "Apple Intelligence" are running a small language model and image generation on the NPU so that should count as "GenAI".
replies(1): >>41866463 #
3. grugagag ◴[17 Oct 24 02:34 UTC] No.41865902[source]▶
I wish I could turn that off on my phone.
4. kalleboo ◴[17 Oct 24 04:41 UTC] No.41866463[source]▶
It's still in beta so we'll see how things go but I saw someone testing what Apple Intelligence ran on-device vs sent off to the "private secure cloud" and even stuff like text summaries were being sent to the cloud.