Thesis: Interesting work is less amenable to the use of AI

I feel much more confident that I can take on a project in a domain that im not very familiar with. Ive been digging into llvm ir and I had not prior experience with it. ChatGPT is a much better guide to getting started than the documentation, which is very low quality.

Good luck with that.

I have been exploring local AI tools for coding (ollama + aider) with a small stock market simulator (~200 lines of python).

First I tried making the AI extract the dataclasses representing events to a separated file. It decided to extract some extra classes, leave behind some others, and delete parts of the code.

Then I tried to make it explain one of the actors called LongVol_player_v1, around 15 lines of code. It successfully concluded it does options delta hedging, but it jumped to the conclusion that it calculates the implied volatility. I set it as a constant, because I'm simulating specific interactions between volatility players and option dealers. It hasn't caught yet the bug where the vol player buys 3000 options but accounts only for 2000.

When asking for improvements, it is obsessed with splitting the initialization and the execution.

So far I wasted half of Saturday trying to make the machine do simple refactors. Refactors I could do myself in half of an hour.

I'm yet to see the wonders of AI.

For what it's worth, commercial models are in a completely different league to locally runnable models. If you are really interested in seeing state of the art right now at least give it a whack with opus/gemini/o3 or something of that calibre.

You might still be disappointed but at least you won't have shot your leg off out of the gates!

I'm trying to use local models for privacy reasons. Also, at some point, one of the employers out there will start suing people because those people shared code with a commercial model, and the commercial model decided to train on that code and output learned code to someone else. I'm not interested in anything of such a situation.