←back to thread

Ask HN: Is anyone doing anything cool with tiny language models?

684 points prettyblocks | 1 comments | 21 Jan 25 19:39 UTC | HN request time: 0s | source

I mean anything in the 0.5B-3B range that's available on Ollama (for example). Have you built any cool tooling that uses these models as part of your work flow?
Show context
behohippy ◴[21 Jan 25 20:57 UTC] No.42785105[source]▶
I have a mini PC with an n100 CPU connected to a small 7" monitor sitting on my desk, under the regular PC. I have llama 3b (q4) generating endless stories in different genres and styles. It's fun to glance over at it and read whatever it's in the middle of making. I gave llama.cpp one CPU core and it generates slow enough to just read at a normal pace, and the CPU fans don't go nuts. Totally not productive or really useful but I like it.
replies(6): >>42785192 #>>42785253 #>>42785325 #>>42786081 #>>42786114 #>>42787856 #
Dansvidania ◴[21 Jan 25 21:05 UTC] No.42785192[source]▶
this sounds pretty cool, do you have any video/media of it?
replies(1): >>42792159 #
behohippy ◴[22 Jan 25 12:40 UTC] No.42792159[source]▶
I don't have a video but here's a pic of the output: https://imgur.com/ip8GWIh
replies(1): >>42799809 #
1. sky2224 ◴[23 Jan 25 02:00 UTC] No.42799809{3}[source]▶
The next step is to format it so it looks like an endless starwars intro.