Most active commenters

    ←back to thread

    AI PCs Aren't Good at AI: The CPU Beats the NPU

    (github.com)
    486 points dbreunig | 13 comments | 16 Oct 24 19:44 UTC | HN request time: 0.842s | source | bottom
    Show context
    jsheard ◴[16 Oct 24 20:16 UTC] No.41863390[source]▶
    These NPUs are tying up a substantial amount of silicon area so it would be a real shame if they end up not being used for much. I can't find a die analysis of the Snapdragon X which isolates the NPU specifically but AMDs equivalent with the same ~50 TOPS performance target can be seen here, and takes up about as much area as three high performance CPU cores:

    https://www.techpowerup.com/325035/amd-strix-point-silicon-p...

    replies(4): >>41863880 #>>41863905 #>>41864412 #>>41865466 #
    1. Kon-Peki ◴[16 Oct 24 21:10 UTC] No.41863880[source]▶
    Modern chips have to dedicate a certain percentage of the die to dark silicon [1] (or else they melt/throttle to uselessness), and these kinds of components count towards that amount. So the point of these components is to be used, but not to be used too much.

    Instead of an NPU, they could have used those transistors and die space for any number of things. But they wouldn't have put additional high performance CPU cores there - that would increase the power density too much and cause thermal issues that can only be solved with permanent throttling.

    [1] https://en.wikipedia.org/wiki/Dark_silicon

    replies(2): >>41864171 #>>41865813 #
    2. IshKebab ◴[16 Oct 24 21:47 UTC] No.41864171[source]▶
    If they aren't being used it would be better to dedicate the space to more SRAM.
    replies(2): >>41864316 #>>41870432 #
    3. a2l3aQ ◴[16 Oct 24 22:05 UTC] No.41864316[source]▶
    The point is parts of the CPU have to be off or throttled down when other components are under load to maintain TDP, adding cache that would almost certainly be being used defeats the point of that.
    replies(2): >>41864415 #>>41866947 #
    4. jsheard ◴[16 Oct 24 22:19 UTC] No.41864415{3}[source]▶
    Doesn't SRAM have much lower power density than logic with the same area though? Hence why AMD can get away with physically stacking cache on top of more cache in their X3D parts, without the bottom layer melting.
    replies(2): >>41864778 #>>41864937 #
    5. Kon-Peki ◴[16 Oct 24 23:03 UTC] No.41864778{4}[source]▶
    Yes, cache has a much lower power density and could have been a candidate for that space.

    But I wasn’t on the design team and have no basis for second-guessing them. I’m just saying that cramming more performance CPU cores onto this die isn’t a realistic option.

    6. wtallis ◴[16 Oct 24 23:28 UTC] No.41864937{4}[source]▶
    The SRAM that AMD is stacking also has the benefit of being last-level cache, so it doesn't need to run at anywhere near the frequency and voltage that eg. L1 cache operates at.
    7. jcgrillo ◴[17 Oct 24 02:16 UTC] No.41865813[source]▶
    Question--what's to be lost by making your features sufficiently not dense to allow them to cool at full tilt?
    replies(2): >>41865917 #>>41866644 #
    8. AlotOfReading ◴[17 Oct 24 02:36 UTC] No.41865917[source]▶
    Messes with timing, among other things. A lot of those structures are relatively fixed blocks that are designed for specific sizes. Signals take more time to propagate longer distances, and longer conductors have worse properties. Dense and hot is faster and more broadly useful.
    replies(1): >>41866001 #
    9. jcgrillo ◴[17 Oct 24 02:52 UTC] No.41866001{3}[source]▶
    Interesting, so does that mean we're basically out of runway without aggressive cooling?
    replies(1): >>41867074 #
    10. positr0n ◴[17 Oct 24 05:19 UTC] No.41866644[source]▶
    Good discussion on how at multi GHz clock speeds, the speed of light is actually limiting on some circuit design choices: https://news.ycombinator.com/item?id=12384596
    11. IshKebab ◴[17 Oct 24 06:26 UTC] No.41866947{3}[source]▶
    Cache doesn't use nearly as much power as active computation; that was my point.
    12. joha4270 ◴[17 Oct 24 06:53 UTC] No.41867074{4}[source]▶
    No.

    Every successive semiconductor node uses less power than the previous per transistor at the same clock speed. Its just that we then immediately use this headroom to pack more transistors closer and run them faster, so every chip keeps running into power limits, even if they continually do more with said power.

    13. VHRanger ◴[17 Oct 24 15:14 UTC] No.41870432[source]▶
    SRAM is extremely hot, it's the very opposite of dark silicon