←back to thread

AI PCs Aren't Good at AI: The CPU Beats the NPU

(github.com)
486 points dbreunig | 7 comments | 16 Oct 24 19:44 UTC | HN request time: 0.906s | source | bottom
Show context
jsheard ◴[16 Oct 24 20:16 UTC] No.41863390[source]▶
These NPUs are tying up a substantial amount of silicon area so it would be a real shame if they end up not being used for much. I can't find a die analysis of the Snapdragon X which isolates the NPU specifically but AMDs equivalent with the same ~50 TOPS performance target can be seen here, and takes up about as much area as three high performance CPU cores:

https://www.techpowerup.com/325035/amd-strix-point-silicon-p...

replies(4): >>41863880 #>>41863905 #>>41864412 #>>41865466 #
Kon-Peki ◴[16 Oct 24 21:10 UTC] No.41863880[source]▶
Modern chips have to dedicate a certain percentage of the die to dark silicon [1] (or else they melt/throttle to uselessness), and these kinds of components count towards that amount. So the point of these components is to be used, but not to be used too much.

Instead of an NPU, they could have used those transistors and die space for any number of things. But they wouldn't have put additional high performance CPU cores there - that would increase the power density too much and cause thermal issues that can only be solved with permanent throttling.

[1] https://en.wikipedia.org/wiki/Dark_silicon

replies(2): >>41864171 #>>41865813 #
1. IshKebab ◴[16 Oct 24 21:47 UTC] No.41864171[source]▶
If they aren't being used it would be better to dedicate the space to more SRAM.
replies(2): >>41864316 #>>41870432 #
2. a2l3aQ ◴[16 Oct 24 22:05 UTC] No.41864316[source]▶
The point is parts of the CPU have to be off or throttled down when other components are under load to maintain TDP, adding cache that would almost certainly be being used defeats the point of that.
replies(2): >>41864415 #>>41866947 #
3. jsheard ◴[16 Oct 24 22:19 UTC] No.41864415[source]▶
Doesn't SRAM have much lower power density than logic with the same area though? Hence why AMD can get away with physically stacking cache on top of more cache in their X3D parts, without the bottom layer melting.
replies(2): >>41864778 #>>41864937 #
4. Kon-Peki ◴[16 Oct 24 23:03 UTC] No.41864778{3}[source]▶
Yes, cache has a much lower power density and could have been a candidate for that space.

But I wasn’t on the design team and have no basis for second-guessing them. I’m just saying that cramming more performance CPU cores onto this die isn’t a realistic option.

5. wtallis ◴[16 Oct 24 23:28 UTC] No.41864937{3}[source]▶
The SRAM that AMD is stacking also has the benefit of being last-level cache, so it doesn't need to run at anywhere near the frequency and voltage that eg. L1 cache operates at.
6. IshKebab ◴[17 Oct 24 06:26 UTC] No.41866947[source]▶
Cache doesn't use nearly as much power as active computation; that was my point.
7. VHRanger ◴[17 Oct 24 15:14 UTC] No.41870432[source]▶
SRAM is extremely hot, it's the very opposite of dark silicon