←back to thread

Ask HN: Is anyone doing anything cool with tiny language models?

684 points prettyblocks | 5 comments | 21 Jan 25 19:39 UTC | HN request time: 1.113s | source

I mean anything in the 0.5B-3B range that's available on Ollama (for example). Have you built any cool tooling that uses these models as part of your work flow?
Show context
kaspermarstal ◴[22 Jan 25 07:48 UTC] No.42790190[source]▶
I built an Excel Add-In that allows my girlfriend to quickly filter 7000 paper titles and abstracts for a review paper that she is writing [1]. It uses Gemma 2 2b which is a wonderful little model that can run on her laptop CPU. It works surprisingly well for this kind of binary classification task.

The nice thing is that she can copy/paste the titles and abstracts in to two columns and write e.g. "=PROMPT(A1:B1, "If the paper studies diabetic neuropathy and stroke, return 'Include', otherwise return 'Exclude'")" and then drag down the formula across 7000 rows to bulk process the data on her own because it's just Excel. There is a gif on the readme on the Github repo that shows it.

[1] https://github.com/getcellm/cellm

replies(12): >>42790265 #>>42790359 #>>42790494 #>>42790901 #>>42791645 #>>42791646 #>>42793924 #>>42795545 #>>42796501 #>>42805657 #>>42812155 #>>42813125 #
afro88 ◴[22 Jan 25 08:38 UTC] No.42790494[source]▶
How accurate are the classifications?
replies(1): >>42790670 #
kaspermarstal ◴[22 Jan 25 09:02 UTC] No.42790670[source]▶
I don't know. This paper [1] reports accuracies in the 97-98% range on a similar task with more powerful models. With Gemma 2 2b the accuracy will certainly be lower.

[1] https://www.medrxiv.org/content/10.1101/2024.10.01.24314702v...

replies(2): >>42791570 #>>42792014 #
beernet ◴[22 Jan 25 12:26 UTC] No.42792014[source]▶
> I don't know.

HN in a nutshell: I've built some cool tech but have no idea if it is helpful or even counter productive...

replies(4): >>42792550 #>>42793068 #>>42794731 #>>42794935 #
1. corobo ◴[22 Jan 25 14:16 UTC] No.42793068[source]▶
Real HN in a nutshell: People who don't build stuff telling people who do build stuff that the thing they built is useless :P

It's a hacker forum, let people hack!

If anything have a dig at OP for posting the thread too soon before the parent commenter has had the chance to gather any data, haha

replies(2): >>42794072 #>>42815728 #
2. greenavocado ◴[22 Jan 25 15:51 UTC] No.42794072[source]▶
Just because you can, doesn't mean you should
replies(1): >>42794179 #
3. corobo ◴[22 Jan 25 16:03 UTC] No.42794179[source]▶
If you're building a dinosaur sanctuary sure
replies(1): >>42795535 #
4. stackghost ◴[22 Jan 25 17:53 UTC] No.42795535{3}[source]▶
Or an Internet surveillance-capitalism panopticon.
5. Breza ◴[24 Jan 25 18:15 UTC] No.42815728[source]▶
Great attitude! I recently built a tool for my wife that uses an LLM to automate a task. Is it production ready? Definitely not. But it saves her time even in its current state.