684 points prettyblocks | 1 comments | 21 Jan 25 19:39 UTC | HN request time: 0.325s | source

I mean anything in the 0.5B-3B range that's available on Ollama (for example). Have you built any cool tooling that uses these models as part of your work flow?

Show context

azhenley ◴[21 Jan 25 20:50 UTC] No.42785041[source]▶

>>42784365 (OP) #

Microsoft published a paper on their FLAME model (60M parameters) for Excel formula repair/completion which outperformed much larger models (>100B parameters).

https://arxiv.org/abs/2301.13779

replies(4): >>42785270 #>>42785415 #>>42785673 #>>42788633 #

1. andai ◴[21 Jan 25 21:30 UTC] No.42785415[source]▶

>>42785041 #

This is wild. They claim it was trained exclusively on Excel formulas, but then they mention retrieval? Is it understanding the connection between English and formulas? Or am I misunderstanding retrieval in this context?

Edit: No, the retrieval is Formula-Formula, the model (nor I believe tokenizer) does not handle English.

↑

Ask HN: Is anyone doing anything cool with tiny language models?