AMD unveils Ryzen Pro 8000-series processors

Even top of the line DDR5 is around 128GB/s vs a M1 at 400GB/s.

At the end of the day, it still seems like AI in consumer chips is chasing a buzzword, what is the killer feature?

On mobile there are image processing benefits and voice to text, translation... but on desktop those are no where near common use cases.

That says M1 is 68.25 GB/s

Especially since most desktop applications people use are web apps. Of the native apps people use that leverage this sort of stuff, almost all are GPU accelerated already (eg. image and video editing AI tools)

Local context aware search

Offline Speech to text and TTS

Offline generation of clip art or stock images for document editing

Offline LLM that can work with your documents as context and access application and OS APIs

Improved enemy AI in gaming

Webcam effects like background removal or filters.

Audio upscaling and interpolation like for bad video call connections.

Already exists on all three major GPU manufacturers, and it definitely makes sense as a GPU workload.

> Local context aware search

You don't need an AI processor to do this, Windows search used to work better and had even less compute resources to work with.

> Offline Speech to text and TTS

See my point about not a very common use case for desktops & laptops vs cell phones.

> Offline LLM that can work with your documents as context and access application and OS APIs

Maybe for some sort of background task or only using really small models <13B parameters. Anything real time is going to run at 1-2t/s with a large model.

Small models are pretty terrible though, I doubt people want even more incorrect information and hallucinations.

> Improved enemy AI in gaming

See Ageia PhysX

> Webcam effects like background removal or filters.

We already have this without NPUs.

> Audio upscaling and interpolation like for bad video call connections.

I could see this, or noise cancellation.

Maybe your local machine can run, I don't know, a model to make suggestions as you're editing a Google Doc, which frees up the Big Machine in the Sky to do other things.

As this becomes more technically feasible, it reduces the effective cost of inference for a new service provider, since you, the client, are now running their code.

The Jevons paradox might kick in, causing more and more uses of LLMs for use cases that were too expensive before.

Intel's bottom of the barrel N95 & N100 CPUs have Gaussian & Neural accelerators for simple image processing and object detection tasks, plus a voice processor for low power voice based activation and command capture and process.

You can always add more power hungry, general purpose components to add capabilities. Heck, video post processing entered hardware era with ATI Radeon 8500. But doing these things with negligible power costs is the new front.

Apple is not adding coprocessors to their iPhones because it looks nice. All of these coprocessors reduce CPU wake-up cycles tremendously and allows the device to monitor tons of things out of bands with negligible power costs.

> Already exists on all three major GPU manufacturers, and it definitely makes sense as a GPU workload.

"makes sense as a GPU workload" is underselling it a bit. Doing it on the CPU is basically insane. Games typically upscale only the world view (the expensive part to render) while rendering the UI at full res. So to do CPU-side upscaling we're talking about a game rendering a surface on the GPU, sending it to the CPU, upscaling it there, sending it back to the GPU, then compositing with the UI. It's just needlessly complicated.

Their cores/shaders can be programmed to do that.

Also, name a current gen GPU which doesn't have video encoding/decoding capabilities/facilities in silicon, even ones which do not allow shaders to be used in this process for post-processing. It's impossible (to not to have these capabilities) at this point in time.

> Local context aware search > You don't need an AI processor to do this, Windows search used to work better and had even less compute resources to work with. You could still improve it with increased natural language understanding instead of simple keyword. “Give me all documents about dogs” instead of searching for each breed as a keyword.

> Offline Speech to text and TTS > See my point about not a very common use case for desktops & laptops vs cell phones. Maybe not for you but accessibility is a key feature for many users. You think blind users should suffer through bad TTS?

> Offline LLM that can work with your documents as context and access application and OS APIs > Maybe for some sort of background task or only using really small models <13B parameters. Anything real time is going to run at 1-2t/s with a large model. > Small models are pretty terrible though, I doubt people want even more incorrect information and hallucinations. Small model have been improving and better capabilities in consumer chips will allow larger models to run faster.

> Improved enemy AI in gaming > See Ageia PhysX Surely you’re not suggesting that enemy AI is solved problem in gaming?

> Webcam effects like background removal or filters. > We already have this without NPUs. Sure but it could go from obvious and distracting to seamless and convincing.

> Audio upscaling and interpolation like for bad video call connections. I could see this, or noise cancellation.

I said that the core (3D rendering hardware) of a GPU with shaders is the AI block already, and said that other tasks like video encoders have their own blocks, but still pull capabilities from the "core" to improve things.

I like the latter. Why even use a new word if it's just going to be the same as "client"?