Most active commenters
  • simonw(3)

←back to thread

Qwen2.5-VL-32B: Smarter and Lighter

(qwenlm.github.io)
544 points tosh | 35 comments | 24 Mar 25 18:35 UTC | HN request time: 0.383s | source | bottom
Show context
simonw ◴[24 Mar 25 18:52 UTC] No.43464227[source]▶
Big day for open source Chinese model releases - DeepSeek-v3-0324 came out today too, an updated version of DeepSeek v3 now under an MIT license (previously it was a custom DeepSeek license). https://simonwillison.net/2025/Mar/24/deepseek/
replies(5): >>43464375 #>>43464498 #>>43464686 #>>43465383 #>>43467111 #
1. chaosprint ◴[24 Mar 25 19:06 UTC] No.43464375[source]▶
it seems that this free version "may use your prompts and completions to train new models"

https://openrouter.ai/deepseek/deepseek-chat-v3-0324:free

do you think this needs attention?

replies(7): >>43464399 #>>43464480 #>>43464512 #>>43464616 #>>43464961 #>>43468548 #>>43470210 #
2. wgd ◴[24 Mar 25 19:08 UTC] No.43464399[source]▶
That's typical of the free options on OpenRouter, if you don't want your inputs used for training you use the paid one: https://openrouter.ai/deepseek/deepseek-chat-v3-0324
replies(1): >>43466292 #
3. ◴[24 Mar 25 19:19 UTC] No.43464480[source]▶
4. huijzer ◴[24 Mar 25 19:21 UTC] No.43464512[source]▶
Since we are on HN here, I can highly recommend open-webui with some OpenAI-compatible provider. I'm running with Deep Infra for more than a year now and am very happy. New models are usually available within one or two days after release. Also have some friends who use the service almost daily.
replies(7): >>43464718 #>>43465081 #>>43466430 #>>43466464 #>>43466949 #>>43469369 #>>43473139 #
5. TechDebtDevin ◴[24 Mar 25 19:33 UTC] No.43464616[source]▶
Thats because its a 3rd party API someone is hosting and trying to arb the infra cost or mine training data, or maybe something even more sinister. I stay away from open router API's that aren't served by reputable well known companies, and even then...
6. unquietwiki ◴[24 Mar 25 19:45 UTC] No.43464718[source]▶
I'm using open-webui at home with a couple of different models. gemma2-9b fits in VRAM on a NV 3060 card + performs nicely.
replies(2): >>43465594 #>>43469350 #
7. behnamoh ◴[24 Mar 25 20:13 UTC] No.43464961[source]▶
good grief! people are okay with it when OpenAI and Google do it, but as soon as open source providers do it, people get defensive about it...
replies(2): >>43465002 #>>43465838 #
8. chaosprint ◴[24 Mar 25 20:18 UTC] No.43465002[source]▶
no. it's nothing to do with deepseek. it's openrouter and providers there
9. l72 ◴[24 Mar 25 20:28 UTC] No.43465081[source]▶
I too run openweb-ui locally and use deepinfra.com as my backend. It has been working very well, and I am quite happy with deepinfra's pricing and privacy policy.

I have set up the same thing at work for my colleagues, and they find it better than openai for their tasks.

replies(1): >>43468529 #
10. zakki ◴[24 Mar 25 21:30 UTC] No.43465594{3}[source]▶
What is the memory of your NV3060? 8GB?
replies(1): >>43466243 #
11. londons_explore ◴[24 Mar 25 21:57 UTC] No.43465838[source]▶
I trust big companies far more with my data than small ones.

Big companies have so much data they won't be having a human look at mine specifically. Some small place probably has the engineer looking at my logs as user #4.

Also, big companies have security teams whose job is securing the data, and it won't be going over some unencrypted link to cloudflare because OP was too lazy to set up Https certs.

replies(1): >>43466206 #
12. henry2023 ◴[24 Mar 25 22:48 UTC] No.43466206{3}[source]▶
Equifax.
replies(1): >>43467325 #
13. ngvjmfgb ◴[24 Mar 25 22:53 UTC] No.43466243{4}[source]▶
12GB (edit: that is what mine is)
14. overfeed ◴[24 Mar 25 22:58 UTC] No.43466292[source]▶
Is OpenRouter planning on distilling models off the prompts and responses from frontier models? That's smart - a little gross - but smart.
replies(1): >>43466375 #
15. numlocked ◴[24 Mar 25 23:09 UTC] No.43466375{3}[source]▶
COO of OpenRouter here. We are simply stating the WE can’t vouch for the behavior of the upstream provider’s retention and training policy. We don’t save your prompt data, regardless of the model you use, unless you explicitly opt-in to logging (in exchange for a 1% inference discount).
replies(2): >>43466481 #>>43466652 #
16. ◴[24 Mar 25 23:17 UTC] No.43466430[source]▶
17. totetsu ◴[24 Mar 25 23:22 UTC] No.43466464[source]▶
And it’s quite easy to set up a Cloudflare tunnel to make your open-webui instance accessible online too just you
replies(1): >>43466484 #
18. simonw ◴[24 Mar 25 23:24 UTC] No.43466481{4}[source]▶
That 1% discount feels a bit cheap to me - if it was a 25% or 50% discount I would be much more likely to sign up for it.
replies(1): >>43466533 #
19. simonw ◴[24 Mar 25 23:25 UTC] No.43466484{3}[source]▶
... or a TailScale network. I've been leaving open-webui running on my laptop on my desk and then going out into the word and accessing it from my phone via TailScale, works great.
replies(2): >>43466956 #>>43467521 #
20. numlocked ◴[24 Mar 25 23:32 UTC] No.43466533{5}[source]▶
We don’t particularly want our customers’ data :)
replies(2): >>43467799 #>>43469383 #
21. overfeed ◴[24 Mar 25 23:51 UTC] No.43466652{4}[source]▶
I'm glad to hear you are not hoovering up this data for your own purposes.
22. wkat4242 ◴[25 Mar 25 00:40 UTC] No.43466949[source]▶
Yeah OpenWebUI is great with local models too. I love it. You can even do a combo, send the same prompt to local and cloud and even various providers and compare the results.
23. wkat4242 ◴[25 Mar 25 00:40 UTC] No.43466956{4}[source]▶
Yeah this sounds like the more secure option, you don't want to be dependent on a single flaw in a web service
24. jimmygrapes ◴[25 Mar 25 01:43 UTC] No.43467325{4}[source]▶
I'm not convinced any humans have worked there for most of my lifetime.
25. totetsu ◴[25 Mar 25 02:15 UTC] No.43467521{4}[source]▶
I would use tail scale. But I specifically want to use open web-ui from a place I can’t install a Tailscale client
replies(1): >>43468733 #
26. oofbaroomf ◴[25 Mar 25 03:14 UTC] No.43467799{6}[source]▶
Yeah, but Openrouter has a 5% surcharge anyway.
replies(1): >>43468205 #
27. YetAnotherNick ◴[25 Mar 25 04:49 UTC] No.43468205{7}[source]▶
Better way to state is 20% of surcharge then :)
28. jychang ◴[25 Mar 25 06:22 UTC] No.43468529{3}[source]▶
Yeah, openweb-ui is the best frontend for API queries. Everything seems to work well.

I've tried LibreChat before, but the app is terrible at generating titles for chats instead of leaving it as "New Chat". Also it lacks a working Code Interpreter.

29. madduci ◴[25 Mar 25 06:29 UTC] No.43468548[source]▶
As always, avoid using sensitive information and you are good to go
30. fragmede ◴[25 Mar 25 07:17 UTC] No.43468733{5}[source]▶
where's that?
31. mdp2021 ◴[25 Mar 25 09:21 UTC] No.43469350{3}[source]▶
> performs nicely

Do you have rough indication of token/s ?

32. eurekin ◴[25 Mar 25 09:24 UTC] No.43469369[source]▶
I've tried using it, but it's browser tab seems to peg one core to 100% after some time. Anyone else experienced it?
33. vintermann ◴[25 Mar 25 09:26 UTC] No.43469383{6}[source]▶
You clearly want it a little if you give a discount for it?
34. indigodaddy ◴[25 Mar 25 16:27 UTC] No.43473139[source]▶
Can open-webui update code on your local computer ala cursor etc?
replies(1): >>43473514 #
35. cess11 ◴[25 Mar 25 17:04 UTC] No.43473514{3}[source]▶
It has a module system so maybe it can but it seems more people are using Aider or Continue for that. There's a bit of stitching things together regardless of whether you show your project to some SaaS or run local models but if you can manage a Linux system it'll be easy.

Personally I heavily dislike the experience though, so I might not be the best one to answer.