←back to thread

Ask HN: Is anyone doing anything cool with tiny language models?

684 points prettyblocks | 3 comments | 21 Jan 25 19:39 UTC | HN request time: 0.238s | source

I mean anything in the 0.5B-3B range that's available on Ollama (for example). Have you built any cool tooling that uses these models as part of your work flow?
Show context
kaspermarstal ◴[22 Jan 25 07:48 UTC] No.42790190[source]▶
I built an Excel Add-In that allows my girlfriend to quickly filter 7000 paper titles and abstracts for a review paper that she is writing [1]. It uses Gemma 2 2b which is a wonderful little model that can run on her laptop CPU. It works surprisingly well for this kind of binary classification task.

The nice thing is that she can copy/paste the titles and abstracts in to two columns and write e.g. "=PROMPT(A1:B1, "If the paper studies diabetic neuropathy and stroke, return 'Include', otherwise return 'Exclude'")" and then drag down the formula across 7000 rows to bulk process the data on her own because it's just Excel. There is a gif on the readme on the Github repo that shows it.

[1] https://github.com/getcellm/cellm

replies(12): >>42790265 #>>42790359 #>>42790494 #>>42790901 #>>42791645 #>>42791646 #>>42793924 #>>42795545 #>>42796501 #>>42805657 #>>42812155 #>>42813125 #
1. TeamDman ◴[23 Jan 25 16:56 UTC] No.42805657[source]▶
Tried it out, very cool! Fun to see it chugging on a bunch of rows. Had a weird issue where it would recompute values endlessly when I used it in a table, but I had another table it worked with so not sure what that was about
replies(2): >>42812306 #>>42857389 #
2. kaspermarstal ◴[24 Jan 25 12:02 UTC] No.42812306[source]▶
Glad you tried it out! Excel triggers recalculation when a referenced cell updates, just like with any other formula. This is also why responses are not streamed, as every update would trigger recalculation. But if the async behavior of responses messes with the recalculation logic I am very interested in looking into it and you are most welcome to open an issue in the repo with steps to reproduce.
3. gorkish ◴[28 Jan 25 20:15 UTC] No.42857389[source]▶
Probably would want to run with Manual calculation set on your sheet if using this.