←back to thread

Ask HN: Who uses open LLMs and coding assistants locally? Share setup and laptop

326 points threeturn | 2 comments | 31 Oct 25 13:39 UTC | HN request time: 0.524s | source

Dear Hackers, I’m interested in your real-world workflows for using open-source LLMs and open-source coding assistants on your laptop (not just cloud/enterprise SaaS). Specifically:

Which model(s) are you running (e.g., Ollama, LM Studio, or others) and which open-source coding assistant/integration (for example, a VS Code plugin) you’re using?

What laptop hardware do you have (CPU, GPU/NPU, memory, whether discrete GPU or integrated, OS) and how it performs for your workflow?

What kinds of tasks you use it for (code completion, refactoring, debugging, code review) and how reliable it is (what works well / where it falls short).

I'm conducting my own investigation, which I will be happy to share as well when over.

Thanks! Andrea.

Show context
scosman ◴[31 Oct 25 17:15 UTC] No.45774360[source]▶
What are folks motivation for using local coding models? Is it privacy and there's no cloud host you trust?

I love local models for some use cases. However for coding there is a big gap between the quality of models you can run at home and those you can't (at least on hardware I can afford) like GLM 4.6, Sonnet 4.5, Codex 5, Qwen Coder 408.

What makes local coding models compelling?

replies(8): >>45774482 #>>45774556 #>>45774772 #>>45775037 #>>45775069 #>>45775189 #>>45775715 #>>45776913 #
voakbasda ◴[31 Oct 25 17:27 UTC] No.45774482[source]▶
Zero trust in remote systems run by others with unknowable or questionable motives.
replies(2): >>45774580 #>>45774611 #
1. scosman ◴[31 Oct 25 17:37 UTC] No.45774580[source]▶
Makes sense that you'd run locally then.

But really no host you trust to not keep data? Big tech with no-log guarantees and contractual liability? Companies with no-log guarantees and clear inference business model to protect like Together/Fireworks? Motives seem aligned.

I'd run locally if I could without compromise. But the gap from GLM 4.5 Air to GLM 4.6 is huge for productivity.

replies(1): >>45776548 #
2. xemdetia ◴[31 Oct 25 20:50 UTC] No.45776548[source]▶
This really isn't an all or nothing sort of situation. Many of the AI players have a proven record of simply not following existing norms. Until there is a consumer oriented player who is not presuming that training on my private data and ideas is permitted it only makes sense to do some stuff things locally. Beyond that many of the companies providing AI have either weird limits or limitations that interrupt me. I just know as an individual or a fledgling company I am simply not big enough to fight some of these players and win, and the compliance around companies running AI transparently is too new for me to rely on so the rules of engagement are all over the place. Also don't forget in a few years when the dust settles that company with that policy you like is highly likely to be consumed by a company who may not share the same ethics but your data is still held by them.

Why take a chance?