330 points threeturn | 2 comments | 31 Oct 25 13:39 UTC | HN request time: 0.567s | source

Dear Hackers, I’m interested in your real-world workflows for using open-source LLMs and open-source coding assistants on your laptop (not just cloud/enterprise SaaS). Specifically:

Which model(s) are you running (e.g., Ollama, LM Studio, or others) and which open-source coding assistant/integration (for example, a VS Code plugin) you’re using?

What laptop hardware do you have (CPU, GPU/NPU, memory, whether discrete GPU or integrated, OS) and how it performs for your workflow?

What kinds of tasks you use it for (code completion, refactoring, debugging, code review) and how reliable it is (what works well / where it falls short).

I'm conducting my own investigation, which I will be happy to share as well when over.

Thanks! Andrea.

1. codingbear ◴[31 Oct 25 19:22 UTC] No.45775708[source]▶

>>45771870 (OP) #

I use local for code completions only. Which means models supporting FIM tokens.

My current setup is the llama-vscode plugin + llama-server running Qwen/Qwen2.5-Coder-7B-Instruct. It leads to very fast completions, and don't have to worry about internet outages which take me out of the zone.

I do wish qwen-3 released a 7B model supporting FIM tokens. 7B seems to be the sweet spot for fast and usable completions

replies(1): >>45777958 #

2. Mostlygeek ◴[31 Oct 25 23:48 UTC] No.45777958[source]▶

>>45775708 (TP) #

qwen3-coder-30B-A3B supports FIM and should be faster than the 7B if you got the vram.

I use bartowkski’s Q8 quant over dual 3090s and it gets up to 100tok/sec. The Q4 quant on a single 3090 is very fast and decently smart.

↑

Ask HN: Who uses open LLMs and coding assistants locally? Share setup and laptop