←back to thread

Ask HN: Is anyone doing anything cool with tiny language models?

684 points prettyblocks | 3 comments | 21 Jan 25 19:39 UTC | HN request time: 0.618s | source

I mean anything in the 0.5B-3B range that's available on Ollama (for example). Have you built any cool tooling that uses these models as part of your work flow?
Show context
ata_aman ◴[21 Jan 25 23:08 UTC] No.42786354[source]▶
I have it running on a Raspberry Pi 5 for offline chat and RAG. I wrote this open-source code for it: https://github.com/persys-ai/persys

It also does RAG on apps there, like the music player, contacts app and to-do app. I can ask it to recommend similar artists to listen to based on my music library for example or ask it to quiz me on my PDF papers.

replies(1): >>42788228 #
1. nejsjsjsbsb ◴[22 Jan 25 02:34 UTC] No.42788228[source]▶
Does https://github.com/persys-ai/persys-server run on the rpi?

Is that design 3d printable? Or is that for paid users only.

replies(1): >>42788718 #
2. ata_aman ◴[22 Jan 25 03:41 UTC] No.42788718[source]▶
I can publish it no problem. I’ll create a new repo with instructions for the hardware with CAD files.

Designing a new one for the NVIDIA Orin Nano Super so it might take a few days.

replies(1): >>42791320 #
3. nejsjsjsbsb ◴[22 Jan 25 10:38 UTC] No.42791320[source]▶
Up to you! Totally understand if you want to hold something back for a paid option!