Ask HN: Is anyone doing anything cool with tiny language models?

I built an Excel Add-In that allows my girlfriend to quickly filter 7000 paper titles and abstracts for a review paper that she is writing [1]. It uses Gemma 2 2b which is a wonderful little model that can run on her laptop CPU. It works surprisingly well for this kind of binary classification task.

The nice thing is that she can copy/paste the titles and abstracts in to two columns and write e.g. "=PROMPT(A1:B1, "If the paper studies diabetic neuropathy and stroke, return 'Include', otherwise return 'Exclude'")" and then drag down the formula across 7000 rows to bulk process the data on her own because it's just Excel. There is a gif on the readme on the Github repo that shows it.

[1] https://github.com/getcellm/cellm

How accurate are the classifications?

I don't know. This paper [1] reports accuracies in the 97-98% range on a similar task with more powerful models. With Gemma 2 2b the accuracy will certainly be lower.

[1] https://www.medrxiv.org/content/10.1101/2024.10.01.24314702v...

> I don't know.

HN in a nutshell: I've built some cool tech but have no idea if it is helpful or even counter productive...

I am not going to claim or report any kind of accuracy, especially with such a small model and such a specific, context dependent use case. It is the user’s responsibility to cross validate if it’s accurate enough for their use case and upgrade model or use another approach if not.