←back to thread

Qwen2.5-VL-32B: Smarter and Lighter

(qwenlm.github.io)
544 points tosh | 1 comments | 24 Mar 25 18:35 UTC | HN request time: 0.212s | source
Show context
jauntywundrkind ◴[24 Mar 25 18:47 UTC] No.43464180[source]▶
Wish I knew better how to estimate what sized video card one needs. HuggingFace link says this is bfloat16, so at least 64GB?

I guess the -7B might run on my 16GB AMD card?

replies(4): >>43464207 #>>43464240 #>>43464303 #>>43464853 #
1. clear_view ◴[24 Mar 25 19:00 UTC] No.43464303[source]▶
deepseek-r1:14b/mistral-small:24b/qwen2.5-coder:14b fit 16GB VRAM with fast generation. 32b versions bleed into RAM and take a serious performance hit but still usable.