←back to thread

Gemma 3 QAT Models: Bringing AI to Consumer GPUs

(developers.googleblog.com)
602 points emrah | 1 comments | 20 Apr 25 12:22 UTC | HN request time: 0.212s | source
Show context
porphyra ◴[20 Apr 25 17:31 UTC] No.43745158[source]▶
It is funny that Microsoft had been peddling "AI PCs" and Apple had been peddling "made for Apple Intelligence" for a while now, when in fact usable models for consumer GPUs are only barely starting to be a thing on extremely high end GPUs like the 3090.
replies(4): >>43745683 #>>43746489 #>>43746510 #>>43746780 #
1. dragonwriter ◴[20 Apr 25 21:54 UTC] No.43746780[source]▶
AI PCs aren't about running the kind of models that take a 3090-class GPU, or even running on GPU at all, but systems where the local end is running something like Phi-3.5-vision-instruct, on system RAM using a CPU with an integrated NPU, which is why the AI PC requirements specify an NPU, a certain amount of processing capacity, and a minimum amount of DDR5/LPDDR5 system RAM.