←back to thread

Gemma 3 QAT Models: Bringing AI to Consumer GPUs

(developers.googleblog.com)
602 points emrah | 1 comments | 20 Apr 25 12:22 UTC | HN request time: 0.001s | source
Show context
justanotheratom ◴[20 Apr 25 14:23 UTC] No.43743956[source]▶
Anyone packaged one of these in an iPhone App? I am sure it is doable, but I am curious what tokens/sec is possible these days. I would love to ship "private" AI Apps if we can get reasonable tokens/sec.
replies(4): >>43743983 #>>43744244 #>>43744274 #>>43744863 #
Alifatisk ◴[20 Apr 25 14:26 UTC] No.43743983[source]▶
If you ever ship a private AI app, don't forget to implement the export functionality, please!
replies(2): >>43744861 #>>43747697 #
idonotknowwhy ◴[21 Apr 25 00:56 UTC] No.43747697[source]▶
You mean conversations? Just the jsonl of the standard hf dataset format to import into other systems?
replies(1): >>43750298 #
1. Alifatisk ◴[21 Apr 25 10:32 UTC] No.43750298[source]▶
Yeah I mean conversations.