←back to thread

Ask HN: Is anyone doing anything cool with tiny language models?

684 points prettyblocks | 1 comments | 21 Jan 25 19:39 UTC | HN request time: 0.314s | source

I mean anything in the 0.5B-3B range that's available on Ollama (for example). Have you built any cool tooling that uses these models as part of your work flow?
Show context
behohippy ◴[21 Jan 25 20:57 UTC] No.42785105[source]▶
I have a mini PC with an n100 CPU connected to a small 7" monitor sitting on my desk, under the regular PC. I have llama 3b (q4) generating endless stories in different genres and styles. It's fun to glance over at it and read whatever it's in the middle of making. I gave llama.cpp one CPU core and it generates slow enough to just read at a normal pace, and the CPU fans don't go nuts. Totally not productive or really useful but I like it.
replies(6): >>42785192 #>>42785253 #>>42785325 #>>42786081 #>>42786114 #>>42787856 #
1. keeganpoppen ◴[21 Jan 25 22:44 UTC] No.42786081[source]▶
oh wow that is actually such a brilliant little use case-- really cuts to the core of the real "magic" of ai: that it can just keep running continuously. it never gets tired, and never gets tired of thinking.