←back to thread

AI PCs Aren't Good at AI: The CPU Beats the NPU

(github.com)
486 points dbreunig | 2 comments | 16 Oct 24 19:44 UTC | HN request time: 0s | source
Show context
eightysixfour ◴[16 Oct 24 20:32 UTC] No.41863546[source]▶
I thought the purpose of these things was not to be fast, but to be able to run small models with very little power usage? I have a newer AMD laptop with an NPU, and my power usage doesn't change using the video effects that supposedly run on it, but goes up when using the nvidia studio effects.

It seems like the NPUs are for very optimized models that do small tasks, like eye contact, background blur, autocorrect models, transcription, and OCR. In particular, on Windows, I assumed they were running the full screen OCR (and maybe embeddings for search) for the rewind feature.

replies(7): >>41863632 #>>41863779 #>>41863821 #>>41863886 #>>41864628 #>>41864828 #>>41869772 #
boomskats ◴[16 Oct 24 20:56 UTC] No.41863779[source]▶
That's especially true because yours is a Xilinx FPGA. The one that they just attached to the latest gen mobile ryzens is 5x more capable too.

AMD are doing some fantastic work at the moment, they just don't seem to be shouting about it. This one is particularly interesting https://lore.kernel.org/lkml/DM6PR12MB3993D5ECA50B27682AEBE1...

edit: not an FPGA. TIL. :'(

replies(5): >>41863852 #>>41863876 #>>41864048 #>>41864435 #>>41865733 #
1. beeflet ◴[16 Oct 24 21:08 UTC] No.41863876[source]▶
It would be cool if most PCs had a general purpose FPGA that could be repurposed by the operating system. For example you could use it as a security processor like a TPM or as a bootrom, or you could repurpose it for DSP or something.

It just seems like this would be better in terms of firmware/security/bootloading because you would be more able to fix it if an exploit gets discovered, and it would be leaner because different operating systems can implement their own stuff (for example linux might not want pluton in-chip security, windows might not want coreboot or linux-based boot, bare metal applications can have much simpler boot).

replies(1): >>41864617 #
2. walterbell ◴[16 Oct 24 22:44 UTC] No.41864617[source]▶
Xilinx Artix 7-series PicoEVB fits in M.2 wifi slot and has an OSS toolchain, http://www.enjoy-digital.fr/