←back to thread

Gemma 3 QAT Models: Bringing AI to Consumer GPUs

(developers.googleblog.com)
602 points emrah | 2 comments | 20 Apr 25 12:22 UTC | HN request time: 0.405s | source
Show context
porphyra ◴[20 Apr 25 17:31 UTC] No.43745158[source]▶
It is funny that Microsoft had been peddling "AI PCs" and Apple had been peddling "made for Apple Intelligence" for a while now, when in fact usable models for consumer GPUs are only barely starting to be a thing on extremely high end GPUs like the 3090.
replies(4): >>43745683 #>>43746489 #>>43746510 #>>43746780 #
NorwegianDude ◴[20 Apr 25 21:04 UTC] No.43746510[source]▶
A 3090 is not a extremely high end GPU. Is a consumer GPU launched in 2020, and even in price and compute it's around a mid-range consumer GPU these days.

The high end consumer card from Nvidia is the RTX 5090, and the professional version of the card is the RTX PRO 6000.

replies(2): >>43746662 #>>43746696 #
1. dragonwriter ◴[20 Apr 25 21:36 UTC] No.43746696[source]▶
For model usability as a binary yes/no, pretty much the only dimension that matters is VRAM, and at 24GB the 3090 is still high end for a consumer NVidia GPUs, yes, the 5090 (and only the 5090) is above it, at 32GB, but 24GB is way ahead of the mid-range.
replies(1): >>43747062 #
2. NorwegianDude ◴[20 Apr 25 22:41 UTC] No.43747062[source]▶
24 GB of VRAM is a large amount of VRAM on a consumer GPU, that I totally agree with you on. But it's definitely not an extremely high end GPU these days. It is suitable, yes, but not high end. The high end alternative for a consumer GPU would be the RTX 5090, but that is only available for €3000 now, while used 3090s are around €650.