←back to thread

Qwen2.5-VL-32B: Smarter and Lighter

(qwenlm.github.io)
544 points tosh | 1 comments | 24 Mar 25 18:35 UTC | HN request time: 0.2s | source
Show context
simonw ◴[24 Mar 25 18:53 UTC] No.43464243[source]▶
32B is one of my favourite model sizes at this point - large enough to be extremely capable (generally equivalent to GPT-4 March 2023 level performance, which is when LLMs first got really useful) but small enough you can run them on a single GPU or a reasonably well specced Mac laptop (32GB or more).
replies(9): >>43464289 #>>43464380 #>>43464443 #>>43464588 #>>43464688 #>>43467991 #>>43468940 #>>43469099 #>>43470619 #
1. Tepix ◴[25 Mar 25 12:55 UTC] No.43470619[source]▶
32B is also great for two 24GB GPUs if you want a nice context size and/or Q8 quantization which is usually very good.