←back to thread

Gemma 3 QAT Models: Bringing AI to Consumer GPUs

(developers.googleblog.com)
602 points emrah | 1 comments | 20 Apr 25 12:22 UTC | HN request time: 0.207s | source
Show context
wtcactus ◴[20 Apr 25 13:33 UTC] No.43743666[source]▶
They keep mentioning the RTX 3090 (with 24 GB VRAM), but the model is only 14.1 GB.

Shouldn’t it fit a 5060 Ti 16GB, for instance?

replies(3): >>43743691 #>>43743768 #>>43747505 #
1. Havoc ◴[21 Apr 25 00:09 UTC] No.43747505[source]▶
Just checked - 19 gigs with 8k context @ q8 kv.Plus another 2.5-ish or so for OS etc.

...so yeah 3090