←back to thread

Gemma 3 QAT Models: Bringing AI to Consumer GPUs

(developers.googleblog.com)
602 points emrah | 6 comments | 20 Apr 25 12:22 UTC | HN request time: 0.001s | source | bottom
1. jarbus ◴[20 Apr 25 13:31 UTC] No.43743656[source]▶
Very excited to see these kinds of techniques, I think getting a 30B level reasoning model usable on consumer hardware is going to be a game changer, especially if it uses less power.
replies(1): >>43743674 #
2. apples_oranges ◴[20 Apr 25 13:34 UTC] No.43743674[source]▶
Deepseek does reasoning on my home Linux pc but not sure how power hungry it is
replies(1): >>43743696 #
3. gcr ◴[20 Apr 25 13:38 UTC] No.43743696[source]▶
what variant? I’d considered DeepSeek far too large for any consumer GPUs
replies(1): >>43743721 #
4. scosman ◴[20 Apr 25 13:43 UTC] No.43743721{3}[source]▶
Some people run Deepseek on CPU. 37B active params - it isn't fast but it's passible.
replies(1): >>43743842 #
5. danielbln ◴[20 Apr 25 14:05 UTC] No.43743842{4}[source]▶
Actual deepseek or some qwen/llama reasoning fine-tune?
replies(1): >>43744550 #
6. scosman ◴[20 Apr 25 15:50 UTC] No.43744550{5}[source]▶
Actual Deepseek. 500gb of memory and a threadripper works. Not a standard PC spec, but a common ish home brew setup for single user Deepseek.