←back to thread

AI PCs Aren't Good at AI: The CPU Beats the NPU

(github.com)
486 points dbreunig | 2 comments | 16 Oct 24 19:44 UTC | HN request time: 0.603s | source
Show context
eightysixfour ◴[16 Oct 24 20:32 UTC] No.41863546[source]▶
I thought the purpose of these things was not to be fast, but to be able to run small models with very little power usage? I have a newer AMD laptop with an NPU, and my power usage doesn't change using the video effects that supposedly run on it, but goes up when using the nvidia studio effects.

It seems like the NPUs are for very optimized models that do small tasks, like eye contact, background blur, autocorrect models, transcription, and OCR. In particular, on Windows, I assumed they were running the full screen OCR (and maybe embeddings for search) for the rewind feature.

replies(7): >>41863632 #>>41863779 #>>41863821 #>>41863886 #>>41864628 #>>41864828 #>>41869772 #
boomskats ◴[16 Oct 24 20:56 UTC] No.41863779[source]▶
That's especially true because yours is a Xilinx FPGA. The one that they just attached to the latest gen mobile ryzens is 5x more capable too.

AMD are doing some fantastic work at the moment, they just don't seem to be shouting about it. This one is particularly interesting https://lore.kernel.org/lkml/DM6PR12MB3993D5ECA50B27682AEBE1...

edit: not an FPGA. TIL. :'(

replies(5): >>41863852 #>>41863876 #>>41864048 #>>41864435 #>>41865733 #
errantspark ◴[16 Oct 24 21:04 UTC] No.41863852[source]▶
Wait sorry back up a bit here. I can buy a laptop that has a daughter FPGA in it? Does it have GPIO??? Are we seriously building hardware worth buying again in 2024? Do you have a link?
replies(2): >>41863959 #>>41864293 #
dekhn ◴[16 Oct 24 22:02 UTC] No.41864293[source]▶
If you want GPIOs, you don't need (or want) an FPGA.

I don't know the details of your use case, but I work with low level hardware driven by GPIOs and after a bit of investigation, concluded that having direect GPIO access in a modern PC was not necessary or desirable compared to the alternatives.

replies(1): >>41866390 #
1. errantspark ◴[17 Oct 24 04:18 UTC] No.41866390[source]▶
I get a lot of use out of the PRUs on the BeagleboneBlack, I would absolutely get use out of an FPGA in a laptop.
replies(1): >>41866503 #
2. dekhn ◴[17 Oct 24 04:49 UTC] No.41866503[source]▶
It makes more sense to me to just use the BeagleboneBlack in concert with the FPGA. Unless you have highly specific compute or data movement needs that can't be satisfied over a USB serial link. If you have those needs, and you need a laptop, I guess an FPGA makes sense but that's a teeny market.