←back to thread

1257 points adrianh | 3 comments | | HN request time: 0s | source
Show context
kragen ◴[] No.44491713[source]
I've found this to be one of the most useful ways to use (at least) GPT-4 for programming. Instead of telling it how an API works, I make it guess, maybe starting with some example code to which a feature needs to be added. Sometimes it comes up with a better approach than I had thought of. Then I change the API so that its code works.

Conversely, I sometimes present it with some existing code and ask it what it does. If it gets it wrong, that's a good sign my API is confusing, and how.

These are ways to harness what neural networks are best at: not providing accurate information but making shit up that is highly plausible, "hallucination". Creativity, not logic.

(The best thing about this is that I don't have to spend my time carefully tracking down the bugs GPT-4 has cunningly concealed in its code, which often takes longer than just writing the code the usual way.)

There are multiple ways that an interface can be bad, and being unintuitive is the only one that this will fix. It could also be inherently inefficient or unreliable, for example, or lack composability. The AI won't help with those. But it can make sure your API is guessable and understandable, and that's very valuable.

Unfortunately, this only works with APIs that aren't already super popular.

replies(23): >>44491842 #>>44492001 #>>44492077 #>>44492120 #>>44492212 #>>44492216 #>>44492420 #>>44492435 #>>44493092 #>>44493354 #>>44493865 #>>44493965 #>>44494167 #>>44494305 #>>44494851 #>>44495199 #>>44495821 #>>44496361 #>>44496998 #>>44497042 #>>44497475 #>>44498144 #>>44498656 #
bryanlarsen ◴[] No.44492420[source]
I used this to great success just this morning. I told the AI to write me some unit tests. It flailed and failed badly at that task. But how it failed was instructive, and uncovered a bug in the code I wanted to test.
replies(2): >>44492537 #>>44499794 #
1. kragen ◴[] No.44492537[source]
Haha, that's awesome! Are you going to change the interface? What was the bug?
replies(1): >>44492745 #
2. bryanlarsen ◴[] No.44492745[source]
It used nonsensical parameters to the API in way that I didn't realize was possible (though obvious in hindsight). The AI got confused; it didn't think the parameters were nonsensical. It also didn't quite use them in the way that triggered the error. However it was close enough for me to realize that "hey, I never though of that possibility". I needed to fix the function to return a proper error response for the nonsense.

It also taught me to be more careful about checkpointing my work in git before letting an agent go wild on my codebase. It left a mess trying to fix its problems.

replies(1): >>44492830 #
3. kragen ◴[] No.44492830[source]
Yeah, that's a perfect example of what I'm talking about!