←back to thread

Adding a feature because ChatGPT incorrectly thinks it exists

(www.holovaty.com)
1245 points adrianh | 2 comments | 07 Jul 25 14:58 UTC | HN request time: 0s | source
Show context
kragen ◴[07 Jul 25 16:01 UTC] No.44491713[source]▶
I've found this to be one of the most useful ways to use (at least) GPT-4 for programming. Instead of telling it how an API works, I make it guess, maybe starting with some example code to which a feature needs to be added. Sometimes it comes up with a better approach than I had thought of. Then I change the API so that its code works.

Conversely, I sometimes present it with some existing code and ask it what it does. If it gets it wrong, that's a good sign my API is confusing, and how.

These are ways to harness what neural networks are best at: not providing accurate information but making shit up that is highly plausible, "hallucination". Creativity, not logic.

(The best thing about this is that I don't have to spend my time carefully tracking down the bugs GPT-4 has cunningly concealed in its code, which often takes longer than just writing the code the usual way.)

There are multiple ways that an interface can be bad, and being unintuitive is the only one that this will fix. It could also be inherently inefficient or unreliable, for example, or lack composability. The AI won't help with those. But it can make sure your API is guessable and understandable, and that's very valuable.

Unfortunately, this only works with APIs that aren't already super popular.

replies(23): >>44491842 #>>44492001 #>>44492077 #>>44492120 #>>44492212 #>>44492216 #>>44492420 #>>44492435 #>>44493092 #>>44493354 #>>44493865 #>>44493965 #>>44494167 #>>44494305 #>>44494851 #>>44495199 #>>44495821 #>>44496361 #>>44496998 #>>44497042 #>>44497475 #>>44498144 #>>44498656 #
1. djsavvy ◴[07 Jul 25 22:18 UTC] No.44495199[source]▶
how do prompt it to make it guess about the API for a library? I'm confused how you would structure that in a useful way.
replies(1): >>44495305 #
2. kragen ◴[07 Jul 25 22:38 UTC] No.44495305[source]▶
Often I've started with some example code that invokes part of the API, but not all of it. Or in C I can give it the .h file, maybe without comments.

Sometimes I can just say, "How do I use the <made-up name> API in Python to do <task>?" Unfortunately the safeguards against hallucinations in more recent models can make this more difficult, because it's more likely to tell me it's never heard of it. You can usually coax it into suspension of disbelief, but I think the results aren't as good.