Most active commenters
  • spiritplumber(5)

←back to thread

Ask HN: Is anyone doing anything cool with tiny language models?

684 points prettyblocks | 16 comments | 21 Jan 25 19:39 UTC | HN request time: 0.213s | source | bottom

I mean anything in the 0.5B-3B range that's available on Ollama (for example). Have you built any cool tooling that uses these models as part of your work flow?
1. spiritplumber ◴[22 Jan 25 00:10 UTC] No.42786957[source]▶
My husband and me made a stock market analysis thing that gets it right about 55% of the time, so better than a coin toss. The problem is that it keeps making unethical suggestions, so we're not using it to trade stock. Does anyone have any idea what we can do with that?
replies(6): >>42787033 #>>42787185 #>>42787294 #>>42787375 #>>42787659 #>>42789438 #
2. bobbygoodlatte ◴[22 Jan 25 00:19 UTC] No.42787033[source]▶
I'm curious what sort of unethical suggestions it's coming up with haha
replies(1): >>42788357 #
3. Etheryte ◴[22 Jan 25 00:35 UTC] No.42787185[source]▶
Have you backtested this in times when markets were not constantly green? Nearly any strategy is good in the good times.
replies(1): >>42788359 #
4. ◴[22 Jan 25 00:52 UTC] No.42787294[source]▶
5. dkga ◴[22 Jan 25 01:03 UTC] No.42787375[source]▶
Suggestion: calculate the out-of-sample Sharpe ratio[0] of the suggestions over a reasonable period to gauge how good the model would actually perform in terms of return compared to risks. It is better than vanilla accuracy or related metrics. Source: I'm a financial economist.

[0]: https://en.wikipedia.org/wiki/Sharpe_ratio

replies(1): >>42788360 #
6. bongodongobob ◴[22 Jan 25 01:33 UTC] No.42787659[source]▶
You can literally flip coins and get better than 50% success in a bull market. Just buy index funds and spend your time on something that isn't trying to beat entropy. You won't be able to.
replies(1): >>42788940 #
7. spiritplumber ◴[22 Jan 25 02:50 UTC] No.42788357[source]▶
so far, mostly buying companies owned/ran by horrible people.
replies(2): >>42790298 #>>42794032 #
8. spiritplumber ◴[22 Jan 25 02:50 UTC] No.42788359[source]▶
yep. the 55% is over a few years.
replies(1): >>42790290 #
9. spiritplumber ◴[22 Jan 25 02:50 UTC] No.42788360[source]▶
thank you! that's exactly the sort of thing I don't know.
10. spiritplumber ◴[22 Jan 25 04:15 UTC] No.42788940[source]▶
INSUFFICIENT DATA FOR A MEANINGFUL ANSWER.
11. febed ◴[22 Jan 25 05:44 UTC] No.42789438[source]▶
What data do you analyze?
12. kortilla ◴[22 Jan 25 08:05 UTC] No.42790290{3}[source]▶
Right, but if 55% is avg over the last few years, “buy stock” is going to be correct more than not.

https://www.crestmontresearch.com/docs/Stock-Yo-Yo.pdf

replies(1): >>42791373 #
13. kortilla ◴[22 Jan 25 08:07 UTC] No.42790298{3}[source]▶
So if you filter out the Republican owned ones or whatever your bugbear is, does the 55% persist?
14. Etheryte ◴[22 Jan 25 10:48 UTC] No.42791373{4}[source]▶
I think this is a good highlight of why context and reality checks are incredibly important when doing work like this. At first glance, it might look like 55% is a really good result, but in the previous year, a flat buy every day strategy would've been right 56.7% of the time.
replies(1): >>42798262 #
15. GordonS ◴[22 Jan 25 15:48 UTC] No.42794032{3}[source]▶
Can't you adjust the prompt to filter out companies that fund genocide etc?
16. dutchbookmaker ◴[22 Jan 25 22:37 UTC] No.42798262{5}[source]▶
55% means basically nothing in this context if even money. Long 45% to 55% is most likely completely random because it is symmetric with shorting 45% to 55%

Exactly what you would expect from a language model making random stock picks.