←back to thread

175 points koch | 2 comments | | HN request time: 1.398s | source
Show context
briandw ◴[] No.44489892[source]
I feel much more confident that I can take on a project in a domain that im not very familiar with. Ive been digging into llvm ir and I had not prior experience with it. ChatGPT is a much better guide to getting started than the documentation, which is very low quality.
replies(2): >>44490046 #>>44490234 #
javcasas ◴[] No.44490234[source]
Good luck with that.

I have been exploring local AI tools for coding (ollama + aider) with a small stock market simulator (~200 lines of python).

First I tried making the AI extract the dataclasses representing events to a separated file. It decided to extract some extra classes, leave behind some others, and delete parts of the code.

Then I tried to make it explain one of the actors called LongVol_player_v1, around 15 lines of code. It successfully concluded it does options delta hedging, but it jumped to the conclusion that it calculates the implied volatility. I set it as a constant, because I'm simulating specific interactions between volatility players and option dealers. It hasn't caught yet the bug where the vol player buys 3000 options but accounts only for 2000.

When asking for improvements, it is obsessed with splitting the initialization and the execution.

So far I wasted half of Saturday trying to make the machine do simple refactors. Refactors I could do myself in half of an hour.

I'm yet to see the wonders of AI.

replies(3): >>44490537 #>>44491176 #>>44491707 #
simonw ◴[] No.44491176[source]
If you are using Ollama that suggests you are using local models - which ones?

My experience is that the hosted frontier models (o3, Gemini 2.5, Claude 4) would handle those problems with ease.

Local models that fit on a laptop are a lot less capable, sadly.

replies(1): >>44493480 #
1. javcasas ◴[] No.44493480[source]
I have tried with qwen2.5-coder:3b, deepseek-coder:6.7b, deepseek-r1:8b, and llama3:latest.

All of them local, yes.

replies(1): >>44493753 #
2. simonw ◴[] No.44493753[source]
That explains your results. 3B and 8B models are tiny - it's remarkable when they produce code that's even vaguely usable, but it's a stretch to expect them to usefully perform an operation as complex as "extract the dataclasses representing events".

You might start to get useful results if you bump up to the 20B range - Mistral 3/3.1/3.2 Small or one of the ~20B range Gemma 3 models. Even those are way off the capabilities of the hosted frontier models though.