Ask HN: Is anyone doing anything cool with tiny language models?

684 points prettyblocks | 1 comments | 21 Jan 25 19:39 UTC | HN request time: 0.204s | source

I mean anything in the 0.5B-3B range that's available on Ollama (for example). Have you built any cool tooling that uses these models as part of your work flow?

Show context

ahrjay ◴[22 Jan 25 11:56 UTC] No.42791788[source]▶

>>42784365 (OP) #

I built https://ffprompt.ryanseddon.com using the chrome ai (Gemini nano). Allows you to do ffmpeg operations on videos using natural language all client side.

replies(1): >>42793833 #

fauigerzigerk ◴[22 Jan 25 15:30 UTC] No.42793833[source]▶

>>42791788 #

What are the prerequisites for this? I keep getting "Bummer, looks like your device doesn't support Chrome AI" on macOS 15.2 Chrome 132.0.6834.84 (Official Build) (arm64)

[Edit] Found it. I had to enable chrome://flags/#prompt-api-for-gemini-nano

replies(1): >>42884184 #

1. ahrjay ◴[31 Jan 25 02:22 UTC] No.42884184[source]▶

>>42793833 #

Yeah the instructions are not clear. They're on the github repo[1] linked in the header.

1. Install Chrome Dev: Ensure you have version 127. [Download Chrome Dev](https://google.com/chrome/dev/).

2. Check that you’re on 127.0.6512.0 or above

3. Enable two flags: chrome://flags/#optimization-guide-on-device-model - BypassPerfRequirement chrome://flags/#prompt-api-for-gemini-nano - Enabled

4. Relaunch Chrome

5. Navigate to chrome://components

6. Check that Optimization Guide On Device Model is downloading or force download if not Might take a few minutes for this component to even appear

7. Open dev tools and type (await ai.languageModel.capabilities()).available, should return "readily" when all good

[1]: https://github.com/ryanseddon/FFprompt

↑