Recent AI model progress feels mostly like bullshit

I'd say most of the recent AI model progress has been on price.

A 4-bit quant of QwQ-32B is surprisingly close to Claude 3.5 in coding performance. But it's small enough to run on a consumer GPU, which means deployment price is now down to $0.10 per hour. (from $12+ for models requiring 8x H100)

Have you compared it with 8-bit QwQ-17B?

In my evals 8 bit quantized smaller Qwen models were better, but again evaluating is hard.

There’s no QwQ 17B that I’m aware of. Do you have a HF link?

You're right, sorry...I just tested Qwen models, not QwQ, I see QwQ only has 32B.