←back to thread

Qwen2.5-VL-32B: Smarter and Lighter

(qwenlm.github.io)
544 points tosh | 1 comments | 24 Mar 25 18:35 UTC | HN request time: 0.2s | source
Show context
jauntywundrkind ◴[24 Mar 25 18:47 UTC] No.43464180[source]▶
Wish I knew better how to estimate what sized video card one needs. HuggingFace link says this is bfloat16, so at least 64GB?

I guess the -7B might run on my 16GB AMD card?

replies(4): >>43464207 #>>43464240 #>>43464303 #>>43464853 #
1. xiphias2 ◴[24 Mar 25 18:53 UTC] No.43464240[source]▶
I wish they would start producing graphs with quantized version performances as well. What matters is RAM/bandwidth vs performance, not number of parameters.