←back to thread

Ask HN: Is anyone doing anything cool with tiny language models?

684 points prettyblocks | 1 comments | 21 Jan 25 19:39 UTC | HN request time: 0.263s | source

I mean anything in the 0.5B-3B range that's available on Ollama (for example). Have you built any cool tooling that uses these models as part of your work flow?
Show context
iamnotagenius ◴[21 Jan 25 20:42 UTC] No.42784970[source]▶
No, but I use llama 3.2 1b and qwen2.5 1.5 as bash oneliner generator, always runnimg in console.
replies(2): >>42785424 #>>42786003 #
andai ◴[21 Jan 25 21:30 UTC] No.42785424[source]▶
Could you elaborate?
replies(2): >>42785998 #>>42792097 #
1. XMasterrrr ◴[21 Jan 25 22:39 UTC] No.42785998[source]▶
I think I know what he means. I use AI Chat. I load Qwen2.5-1.5B-Instruct with llama.cpp server, fully offloaded to the CPU, and then I config AI Chat to connect to the llama.cpp endpoint.

Checkout the demo they have below

https://github.com/sigoden/aichat#shell-assistant