←back to thread

2025 AI Index Report

(hai.stanford.edu)
166 points INGELRII | 2 comments | 10 Apr 25 15:13 UTC | HN request time: 0.001s | source
Show context
mrdependable ◴[10 Apr 25 17:09 UTC] No.43645990[source]▶
I always see these reports about how much better AI is than humans now, but I can't even get it to help me with pretty mundane problem solving. Yesterday I gave Claude a file with a few hundred lines of code, what the input should be, and told it where the problem was. I tried until I ran out of credits and it still could not work backwards to tell me where things were going wrong. In the end I just did it myself and it turned out to be a pretty obvious problem.

The strange part with these LLMs is that they get weirdly hung up on things. I try to direct them away from a certain type of output and somehow they keep going back to it. It's like the same problem I have with Google where if I try to modify my search to be more specific, it just ignores what it doesn't like about my query and gives me the same output.

replies(4): >>43646008 #>>43646119 #>>43646496 #>>43647128 #
simonw ◴[10 Apr 25 17:11 UTC] No.43646008[source]▶
LLMs are difficult to use. Anyone who tells you otherwise is being misleading.
replies(2): >>43646190 #>>43666132 #
1. zamadatix ◴[12 Apr 25 17:02 UTC] No.43666132[source]▶
I also think LLMs are more difficult to use for most tasks than is often flouted myself but I don't really jive with statements like "Anyone who tells you otherwise is being misleading". Most of the time I find they are just using them in a very different capacity.
replies(1): >>43666396 #
2. simonw ◴[12 Apr 25 17:33 UTC] No.43666396[source]▶
I intended those words to imply "being misleading even if they don't know they are being misleading" - I made a better version of that point here: https://simonwillison.net/2025/Mar/11/using-llms-for-code/

> If someone tells you that coding with LLMs is easy they are (probably unintentionally) misleading you. They may well have stumbled on to patterns that work, but those patterns do not come naturally to everyone.