←back to thread

Gemma 3 QAT Models: Bringing AI to Consumer GPUs

(developers.googleblog.com)
602 points emrah | 1 comments | 20 Apr 25 12:22 UTC | HN request time: 0.339s | source
Show context
emrah ◴[20 Apr 25 12:22 UTC] No.43743338[source]▶
Available on ollama: https://ollama.com/library/gemma3
replies(2): >>43743657 #>>43743658 #
Der_Einzige ◴[20 Apr 25 13:32 UTC] No.43743658[source]▶
How many times do I have to say this? Ollama, llamacpp, and many other projects are slower than vLLM/sglang. vLLM is a much superior inference engine and is fully supported by the only LLM frontends that matter (sillytavern).

The community getting obsessed with Ollama has done huge damage to the field, as it's ineffecient compared to vLLM. Many people can get far more tok/s than they think they could if only they knew the right tools.

replies(9): >>43743672 #>>43743695 #>>43743760 #>>43743819 #>>43743824 #>>43743859 #>>43743860 #>>43749101 #>>43753155 #
oezi ◴[20 Apr 25 14:02 UTC] No.43743819[source]▶
Why is sillytavern the only LLM frontend which matters?
replies(2): >>43744110 #>>43744916 #
1. Der_Einzige ◴[20 Apr 25 16:50 UTC] No.43744916[source]▶
It supports more sampler and other settings than anyone else.