←back to thread

Ask HN: Is anyone doing anything cool with tiny language models?

684 points prettyblocks | 3 comments | 21 Jan 25 19:39 UTC | HN request time: 0s | source

I mean anything in the 0.5B-3B range that's available on Ollama (for example). Have you built any cool tooling that uses these models as part of your work flow?
Show context
Evidlo ◴[22 Jan 25 00:00 UTC] No.42786869[source]▶
I have ollama responding to SMS spam texts. I told it to feign interest in whatever the spammer is selling/buying. Each number gets its own persona, like a millennial gymbro or 19th century British gentleman.

http://files.widloski.com/image10%20(1).png

http://files.widloski.com/image11.png

replies(11): >>42786904 #>>42786974 #>>42787151 #>>42787231 #>>42787781 #>>42789419 #>>42789860 #>>42794672 #>>42795730 #>>42795824 #>>42796084 #
RVuRnvbM2e ◴[22 Jan 25 00:04 UTC] No.42786904[source]▶
This is fantastic. How have your hooked up a mobile number to the llm?
replies(2): >>42786967 #>>42786993 #
Evidlo ◴[22 Jan 25 00:15 UTC] No.42786993[source]▶
Android app that forwards to a Python service on remote workstation over MQTT. I can make a Show HN if people are interested.
replies(5): >>42787153 #>>42787356 #>>42791613 #>>42791742 #>>42793477 #
1. potamic ◴[22 Jan 25 11:32 UTC] No.42791613{3}[source]▶
Why MQTT over HTTP for a low volume, small scale integration?
replies(2): >>42792901 #>>42802009 #
2. c0wb0yc0d3r ◴[22 Jan 25 14:00 UTC] No.42792901[source]▶
I’m not OP, but I would hazard a guess that those are the tools that OP has at hand.
3. 0xedd ◴[23 Jan 25 08:38 UTC] No.42802009[source]▶
Good, cheap design that takes care of dead letters vs implementing a failover endpoint that would require extra hardware.

MQTT is plug and play in Python. No more costly than a HTTP server.