Most active commenters
  • antonok(3)

←back to thread

Ask HN: Is anyone doing anything cool with tiny language models?

684 points prettyblocks | 17 comments | 21 Jan 25 19:39 UTC | HN request time: 0.001s | source | bottom

I mean anything in the 0.5B-3B range that's available on Ollama (for example). Have you built any cool tooling that uses these models as part of your work flow?
Show context
antonok ◴[21 Jan 25 23:57 UTC] No.42786841[source]▶
I've been using Llama models to identify cookie notices on websites, for the purpose of adding filter rules to block them in EasyList Cookie. Otherwise, this is normally done by, essentially, manual volunteer reporting.

Most cookie notices turn out to be pretty similar, HTML/CSS-wise, and then you can grab their `innerText` and filter out false positives with a small LLM. I've found the 3B models have decent performance on this task, given enough prompt engineering. They do fall apart slightly around edge cases like less common languages or combined cookie notice + age restriction banners. 7B has a negligible false-positive rate without much extra cost. Either way these things are really fast and it's amazing to see reports streaming in during a crawl with no human effort required.

Code is at https://github.com/brave/cookiemonster. You can see the prompt at https://github.com/brave/cookiemonster/blob/main/src/text-cl....

replies(4): >>42786891 #>>42786896 #>>42793119 #>>42793157 #
1. binarysneaker ◴[22 Jan 25 00:03 UTC] No.42786891[source]▶
Maybe it could also send automated petitions to the EU to undo cookie consent legislation, and reverse some of the enshitification.
replies(3): >>42786953 #>>42787244 #>>42788894 #
2. antonok ◴[22 Jan 25 00:09 UTC] No.42786953[source]▶
Ha, I'm not sure the EU is prepared to handle the deluge of petitions that would ensue.

On a more serious note, this must be the first time we can quantitatively measure the impact of cookie consent legislation across the web, so maybe there's something to be explored there.

replies(1): >>42790710 #
3. K0balt ◴[22 Jan 25 00:43 UTC] No.42787244[source]▶
I think there is real potential here, for smart browsing. Have the llm get the page, replace all the ads with kittens, find non-paywall versions if possible and needed, spoof fingerprint data, detect and highlight AI generated drivel, etc. The site would have no way of knowing that it wasn’t touching eyeballs. We might be able to rake back a bit of the web this way.
replies(1): >>42787340 #
4. antonok ◴[22 Jan 25 00:58 UTC] No.42787340[source]▶
You probably wouldn't want to run this in real-time on every site as it'll significantly increase the load on your browser, but as long as it's possible to generate adblock filter rules, the fixes can scale to a pretty large audience.
replies(2): >>42788192 #>>42794640 #
5. K0balt ◴[22 Jan 25 02:30 UTC] No.42788192{3}[source]▶
I was thinking running it in my home lab server as a proxy, but yeah, scaling it to the browser would require some pretty strong hardware. Still, maybe in a couple of years it could be mainstream.
6. sebastiennight ◴[22 Jan 25 04:08 UTC] No.42788894[source]▶
To me this take is like smokers complaining that the evil government is forcing the good tobacco companies to degrade the experience by adding pictures of cancer patients on cigarette packs.
replies(1): >>42790227 #
7. kortilla ◴[22 Jan 25 07:56 UTC] No.42790227[source]▶
Those don’t really work: https://jamanetwork.com/journals/jamanetworkopen/fullarticle...
replies(1): >>42792167 #
8. pk-protect-ai ◴[22 Jan 25 09:09 UTC] No.42790710[source]▶
why don't you spam the companies who want your data instead? The sites can simply stop gathering your data, then they will not require to ask for consent ...
replies(2): >>42791064 #>>42791197 #
9. frail_figure ◴[22 Jan 25 09:58 UTC] No.42791064{3}[source]▶
It’s the same comments on HN as always. They think EU setting up rules is somehow worse than companies breaking them. We see how the US is turning out without pesky EU restrictions :)
replies(1): >>42793142 #
10. whywhywhywhy ◴[22 Jan 25 10:19 UTC] No.42791197{3}[source]▶
Because they have no reason to care about what you think or feel or they wouldn't be doing it in the first place.

Cookie notices just gave them another weapon in the end.

11. shiftingleft ◴[22 Jan 25 12:41 UTC] No.42792167{3}[source]▶
Do they help deter people from becoming smokers in the first place?
replies(1): >>42800812 #
12. GardenLetter27 ◴[22 Jan 25 14:22 UTC] No.42793142{4}[source]▶
The US has 3x higher salaries, larger houses and a much higher quality of life?

I work as a senior engineer in Europe and make barely $4k net per month... and that's considered a "good" salary!

replies(2): >>42793619 #>>42803540 #
13. Lutger ◴[22 Jan 25 15:09 UTC] No.42793619{5}[source]▶
It has higher salaries for privileged people like senior engineers. Try making ends meet in a lower class job.

And you have (almost) free and universal healthcare in Europa, good food available everywhere, drinking water that doesn't poison you, walkable cities, good public transport, somewhat decent police and a functioning legal system. The list goes on. Does this not impact your quality of life? Do you not care about these things?

How can you have a higher quality of life as a society with higher murders, much lower life-expectancy, so many people in jail, in debt, etc.

replies(1): >>42793866 #
14. macinjosh ◴[22 Jan 25 15:33 UTC] No.42793866{6}[source]▶
Touch grass. The US is a big place and is nothing like you seem to think it is.

Europe on the other hand can't even manage to defend itself and relies on the US for their sheer existence.

replies(2): >>42794951 #>>42803533 #
15. Tepix ◴[22 Jan 25 16:39 UTC] No.42794640{3}[source]▶
Depends on your machine and on the LLM. Could be doable.
16. pona-a ◴[22 Jan 25 17:05 UTC] No.42794951{7}[source]▶
Can you enlighten me of a state where none of parent's points apply? I'd be glad to be educated.
17. kortilla ◴[23 Jan 25 04:50 UTC] No.42800812{4}[source]▶
Not sure if much serious research has been put into it. I would be suspicious of it deterring them because a lot of initial smoking happens in social situations where friends pass out individual cigarettes.

By the time someone buys their own pack they are probably hooked.

I suspect the obscene taxes blocking out young folks is one of the most effective strategies