←back to thread

Ask HN: Is anyone doing anything cool with tiny language models?

684 points prettyblocks | 1 comments | 21 Jan 25 19:39 UTC | HN request time: 0.204s | source

I mean anything in the 0.5B-3B range that's available on Ollama (for example). Have you built any cool tooling that uses these models as part of your work flow?
Show context
behohippy ◴[21 Jan 25 20:57 UTC] No.42785105[source]▶
I have a mini PC with an n100 CPU connected to a small 7" monitor sitting on my desk, under the regular PC. I have llama 3b (q4) generating endless stories in different genres and styles. It's fun to glance over at it and read whatever it's in the middle of making. I gave llama.cpp one CPU core and it generates slow enough to just read at a normal pace, and the CPU fans don't go nuts. Totally not productive or really useful but I like it.
replies(6): >>42785192 #>>42785253 #>>42785325 #>>42786081 #>>42786114 #>>42787856 #
Uehreka ◴[21 Jan 25 21:20 UTC] No.42785325[source]▶
Do you find that it actually generates varied and diverse stories? Or does it just fall into the same 3 grooves?

Last week I tried to get an LLM (one of the recent Llama models running through Groq, it was 70B I believe) to produce randomly generated prompts in a variety of styles and it kept producing cyberpunk scifi stuff. When I told it to stop doing cyberpunk scifi stuff it went completely to wild west.

replies(7): >>42785456 #>>42786232 #>>42788219 #>>42789260 #>>42792152 #>>42794103 #>>42796598 #
1. janalsncm ◴[21 Jan 25 22:56 UTC] No.42786232[source]▶
Generate a list of 5000 possible topics you’d like it to talk about. Randomly pick one and inject that into your prompt.