←back to thread

Qwen2.5-VL-32B: Smarter and Lighter

(qwenlm.github.io)
544 points tosh | 2 comments | 24 Mar 25 18:35 UTC | HN request time: 0.42s | source
Show context
nodesocket ◴[24 Mar 25 19:22 UTC] No.43464517[source]▶
What is the recommended model to process a RAG of PDF text documents? I've seen some recommendations for Mistral:7b. Looking to run on a consumer pedestrian home PC (ollama) with a Nvidia 4060ti and Ryzen 5700x.
replies(1): >>43464814 #
unquietwiki ◴[24 Mar 25 19:57 UTC] No.43464814[source]▶
Apparently there are two versions of the 4060Ti, with 8GB and 16GB of VRAM respectively. I've got an 8GB 3060 that runs gemma2:9b nicely, and that will parse PDF files; gemma3:4b also seems to analyze PDFs decently.
replies(1): >>43464986 #
nodesocket ◴[24 Mar 25 20:16 UTC] No.43464986[source]▶
I also have the 8GB 4060ti variant. Want to upgrade to a 4070 super, but prices on them are still ridiculous. Could be had for $599 a handful of months ago, now on ebay going for $750 plus.

Thanks for the recommendations. I'll give gemma3:12b a try and if needed go down to gemma:4b.

replies(1): >>43466291 #
1. manmal ◴[24 Mar 25 22:58 UTC] No.43466291[source]▶
May I ask why you don’t get a used 3090 with 24GB VRAM?
replies(1): >>43468631 #
2. nodesocket ◴[25 Mar 25 06:50 UTC] No.43468631[source]▶
The rig also is my gaming PC, so want the latest generation chips.