Most active commenters
  • ezst(3)

←back to thread

AI PCs Aren't Good at AI: The CPU Beats the NPU

(github.com)
486 points dbreunig | 18 comments | 16 Oct 24 19:44 UTC | HN request time: 0s | source | bottom
Show context
jsheard ◴[16 Oct 24 20:16 UTC] No.41863390[source]▶
These NPUs are tying up a substantial amount of silicon area so it would be a real shame if they end up not being used for much. I can't find a die analysis of the Snapdragon X which isolates the NPU specifically but AMDs equivalent with the same ~50 TOPS performance target can be seen here, and takes up about as much area as three high performance CPU cores:

https://www.techpowerup.com/325035/amd-strix-point-silicon-p...

replies(4): >>41863880 #>>41863905 #>>41864412 #>>41865466 #
ezst ◴[16 Oct 24 21:13 UTC] No.41863905[source]▶
I can't wait for the LLM fad to be over so we get some sanity (and efficiency) back. I personally have no use for this extra hardware ("GenAI" doesn't help me in any way nor supports any work-related tasks). Worse, most people have no use for that (and recent surveys even show predominant hostility towards AI creep). We shouldn't be paying extra for that, it should be opt-in, and then it would become clear (by looking at the sales and how few are willing to pay a premium for "AI") how overblown and unnecessary this is.
replies(6): >>41863966 #>>41864134 #>>41865168 #>>41865589 #>>41865651 #>>41875051 #
1. renewiltord ◴[16 Oct 24 21:42 UTC] No.41864134[source]▶
I was telling someone this and they gave me link to a laptop with higher battery life and better performance than my own, but I kept explaining to them that the feature I cared most about was die size. They couldn't understand it so I just had to leave them alone. Non-technical people don't get it. Die size is what I care about. It's a critical feature and so many mainstream companies are missing out on my money because they won't optimize die size. Disgusting.
replies(5): >>41864304 #>>41864691 #>>41864921 #>>41866254 #>>41866907 #
2. nl ◴[16 Oct 24 22:04 UTC] No.41864304[source]▶
Is this a parody?

Why would anyone care about die size? And if you do why not get one of the many low power laptops with Atoms etc that do have small die size?

replies(3): >>41864414 #>>41864462 #>>41864496 #
3. throwaway48476 ◴[16 Oct 24 22:19 UTC] No.41864414[source]▶
Maybe through a game of telephone they confused die size and node size?
4. thfuran ◴[16 Oct 24 22:25 UTC] No.41864462[source]▶
Yes, they're making fun of the comment they replied to.
replies(1): >>41866239 #
5. tedunangst ◴[16 Oct 24 22:29 UTC] No.41864496[source]▶
No, no, no, you just don't get it. The only thing Dell will sell me is a laptop 324mm wide, which is totally appalling, but if they offered me a laptop that's 320mm wide, I'd immediately buy it. In my line of work, which is totally serious business, every millimeter counts.
6. _zoltan_ ◴[16 Oct 24 22:52 UTC] No.41864691[source]▶
News flash: you're in the niche of the niche. People don't care about die size.

I'd be willing to bet that the amount of money they are missing out on is miniscule and is by far offset by people's money who care about other stuff. Like you know, performance and battery life, just to stick to your examples.

replies(1): >>41865605 #
7. waveBidder ◴[16 Oct 24 23:25 UTC] No.41864921[source]▶
your satire is off base enough that people don't understand it's satire.
replies(2): >>41865190 #>>41866598 #
8. heavyset_go ◴[17 Oct 24 00:12 UTC] No.41865190[source]▶
The Poe's Law means it's working.
9. mattnewton ◴[17 Oct 24 01:33 UTC] No.41865605[source]▶
That’s exactly what the poster is arguing- they are being sarcastic.
replies(1): >>41870152 #
10. singlepaynews ◴[17 Oct 24 03:47 UTC] No.41866239{3}[source]▶
Would you do me the favor of explaining the joke? I get the premise—nobody cares about die size, but the comment being mocked seems perfectly innocuous to me? They want a laptop without an NPU b/c according to link we get more out of CPU anyways? What am I missing here?
replies(1): >>41867686 #
11. fijiaarone ◴[17 Oct 24 03:50 UTC] No.41866254[source]▶
Yeah, I know what you mean. I hate lugging around a big CPU core.
12. 0xDEAFBEAD ◴[17 Oct 24 05:10 UTC] No.41866598[source]▶
Says a lot about HN that so many believed he was genuine.
13. ezst ◴[17 Oct 24 06:19 UTC] No.41866907[source]▶
I'm fine with the mockery, I genuinely hadn't realized that "wanting to pay for what one needs" was such a hot and controversial take.
replies(1): >>41867899 #
14. michaelt ◴[17 Oct 24 08:57 UTC] No.41867686{4}[source]▶
It has been the norm for several decades to have hardware features that go unused.

The realities of mass manufacturing and supply chains and whatnot mean it's cheaper to get a laptop with a webcam I don't use, a fingerprint reader I don't use, and an SD card reader I don't use. It's cheaper to get a CPU with integrated graphics I don't use, a trusted execution environment I don't use, remote management features I don't use. It's cheaper to get a discrete GPU with RGB LEDs I don't use, directx support I don't use, four outputs when I only need one. It's cheaper to get a motherboard with integrated wifi than one without.

15. ginko ◴[17 Oct 24 09:36 UTC] No.41867899[source]▶
The extra cost of the area spent on npu cores is pretty hard to quantify. I guess removing it would allow for higher yields and number of chips per wafer but then you’d need to set up tooling for two separate runs (one with npu and one without) Add to that that most of the cost is actually the design of the chip and it’s clear why manufacturers just always add the extra features. Maybe they could sell a chip with the NPU permanently disabled but I guess that wouldn’t be what you want either?

Fwiw there should be no power downside to having an unused unit. It’ll just not be powered.

replies(1): >>41870106 #
16. ezst ◴[17 Oct 24 14:39 UTC] No.41870106{3}[source]▶
The argument boils down to "since it's there, better to keep it because making a version without it would defeat economies of scale and not save much, if at all", and that's a sensible take… under the assumption that there's a general demand for NPUs, which I contest.

In practice, everyone is paying a premium for NPUs that only a minority desires, and only a fraction of that minority essentially does "something" with it.

This thread really helps to show that the use-cases are few, non-essential, and that the general application landscape hasn't adopted NPUs and has very little incentive to do so (because of the alien programming model, because of hardware compat across vendors, because of the ecosystem being a moving target with little stability in sight, and because of the high-effort/low-reward in general).

I do want to be wrong, of course. Tech generally is exciting because it offers new tools to crack old problems, opening new venues and opportunities in the process. Here it looks like we have a solution in search for a problem that was set by marketing departments.

replies(1): >>41872019 #
17. shermantanktop ◴[17 Oct 24 14:44 UTC] No.41870152{3}[source]▶
It whooshed over my head too. That’s the danger of sarcasm…it’s a cooperative form of humor but the other party might not get it.
18. Miraste ◴[17 Oct 24 18:01 UTC] No.41872019{4}[source]▶
Modern SoCs already have all kinds of features with use-cases that are few and non-essential. Granted they don't take as much space as NPUS, but manufacturers are betting that if NPUs are available, software will evolve to use them regularly. If it doesn't, they'll probably go away in a few generations. But at a minimum, Microsoft and Apple seem highly committed to using them.