←back to thread

Ask HN: Is anyone doing anything cool with tiny language models?

684 points prettyblocks | 2 comments | 21 Jan 25 19:39 UTC | HN request time: 0.417s | source

I mean anything in the 0.5B-3B range that's available on Ollama (for example). Have you built any cool tooling that uses these models as part of your work flow?
Show context
behohippy ◴[21 Jan 25 20:57 UTC] No.42785105[source]▶
I have a mini PC with an n100 CPU connected to a small 7" monitor sitting on my desk, under the regular PC. I have llama 3b (q4) generating endless stories in different genres and styles. It's fun to glance over at it and read whatever it's in the middle of making. I gave llama.cpp one CPU core and it generates slow enough to just read at a normal pace, and the CPU fans don't go nuts. Totally not productive or really useful but I like it.
replies(6): >>42785192 #>>42785253 #>>42785325 #>>42786081 #>>42786114 #>>42787856 #
Uehreka ◴[21 Jan 25 21:20 UTC] No.42785325[source]▶
Do you find that it actually generates varied and diverse stories? Or does it just fall into the same 3 grooves?

Last week I tried to get an LLM (one of the recent Llama models running through Groq, it was 70B I believe) to produce randomly generated prompts in a variety of styles and it kept producing cyberpunk scifi stuff. When I told it to stop doing cyberpunk scifi stuff it went completely to wild west.

replies(7): >>42785456 #>>42786232 #>>42788219 #>>42789260 #>>42792152 #>>42794103 #>>42796598 #
1. coder543 ◴[22 Jan 25 02:33 UTC] No.42788219[source]▶
Someone mentioned generating millions of (very short) stories with an LLM a few weeks ago: https://news.ycombinator.com/item?id=42577644

They linked to an interactive explorer that nicely shows the diversity of the dataset, and the HF repo links to the GitHub repo that has the code that generated the stories: https://github.com/lennart-finke/simple_stories_generate

So, it seems there are ways to get varied stories.

replies(1): >>42841237 #
2. fi-le ◴[27 Jan 25 14:03 UTC] No.42841237[source]▶
I was wondering where the traffic came from, thanks for mentioning it!