←back to thread

Ask HN: Is anyone doing anything cool with tiny language models?

684 points prettyblocks | 2 comments | 21 Jan 25 19:39 UTC | HN request time: 0.001s | source

I mean anything in the 0.5B-3B range that's available on Ollama (for example). Have you built any cool tooling that uses these models as part of your work flow?
Show context
behohippy ◴[21 Jan 25 20:57 UTC] No.42785105[source]▶
I have a mini PC with an n100 CPU connected to a small 7" monitor sitting on my desk, under the regular PC. I have llama 3b (q4) generating endless stories in different genres and styles. It's fun to glance over at it and read whatever it's in the middle of making. I gave llama.cpp one CPU core and it generates slow enough to just read at a normal pace, and the CPU fans don't go nuts. Totally not productive or really useful but I like it.
replies(6): >>42785192 #>>42785253 #>>42785325 #>>42786081 #>>42786114 #>>42787856 #
1. bithavoc ◴[21 Jan 25 21:11 UTC] No.42785253[source]▶
this is so cool, any chance you post a video?
replies(1): >>42792165 #
2. behohippy ◴[22 Jan 25 12:41 UTC] No.42792165[source]▶
Just this pic: https://imgur.com/ip8GWIh