←back to thread

2025 AI Index Report

(hai.stanford.edu)
166 points INGELRII | 10 comments | 10 Apr 25 15:13 UTC | HN request time: 0s | source | bottom
Show context
mrdependable ◴[10 Apr 25 17:09 UTC] No.43645990[source]▶
I always see these reports about how much better AI is than humans now, but I can't even get it to help me with pretty mundane problem solving. Yesterday I gave Claude a file with a few hundred lines of code, what the input should be, and told it where the problem was. I tried until I ran out of credits and it still could not work backwards to tell me where things were going wrong. In the end I just did it myself and it turned out to be a pretty obvious problem.

The strange part with these LLMs is that they get weirdly hung up on things. I try to direct them away from a certain type of output and somehow they keep going back to it. It's like the same problem I have with Google where if I try to modify my search to be more specific, it just ignores what it doesn't like about my query and gives me the same output.

replies(4): >>43646008 #>>43646119 #>>43646496 #>>43647128 #
simonw ◴[10 Apr 25 17:11 UTC] No.43646008[source]▶
LLMs are difficult to use. Anyone who tells you otherwise is being misleading.
replies(2): >>43646190 #>>43666132 #
__loam ◴[10 Apr 25 17:30 UTC] No.43646190[source]▶
"Hey these tools are kind of disappointing"

"You just need to learn to use them right"

Ad infinitum as we continue to get middling results from the most overhyped piece of technology of all time.

replies(6): >>43646640 #>>43646655 #>>43646908 #>>43647257 #>>43652095 #>>43663510 #
1. pants2 ◴[10 Apr 25 19:26 UTC] No.43647257{3}[source]▶
In my experience, most people who say "Hey these tools are kind of disappointing" either refuse to provide a reproducible example of how it falls short, or if they do, it's clear that they're not using the tool correctly.
replies(4): >>43647369 #>>43654440 #>>43654510 #>>43655733 #
2. __loam ◴[10 Apr 25 19:40 UTC] No.43647369[source]▶
Ad infinitum
3. sksxihve ◴[11 Apr 25 14:51 UTC] No.43654440[source]▶
I'd love to see a reproducible example of these tools producing something that is exceptional. Or a clear reproducible example of using them the right way.

I've used them some (sorry I didn't make detailed notes about my usage, probably used them wrong) but pretty much there are always subtle bugs that if I didn't know better I would have overlooked.

I don't doubt people find them useful, personally I'd rather spend my time learning about things that interest me instead of spending money learning how to prompt a machine to do something I can do myself that I also enjoy doing.

I think a lot of the disagreements on hn about this tech is that both sides are mostly on the extremes of either "it doesn't work and at and is pointless" or "it's amazing and makes me 100x more productive" and not much discussion about the mid-ground of it works for some stuff and knowing what stuff it works well on makes it useful but it won't solve all your problems.

replies(3): >>43656928 #>>43663543 #>>43664027 #
4. mickael-kerjean ◴[11 Apr 25 14:56 UTC] No.43654510[source]▶
The latest example for me was trying to generate a thumbnail of a PSD in plain C and figure out the layers in there as I was lazy to read the specs, with the objective to bundle it as a wasm and execute it on a browser, it never got to extract a thumbnail from a given PSD, it's very confident at making stuff but it never got anywhere despite spending a couple hours on it which would have been better spend reading specs and existing code on that topic
5. input_sh ◴[11 Apr 25 16:35 UTC] No.43655733[source]▶
How are we supposed to give a reproducible example with a non-deterministic tool?
6. doug_durham ◴[11 Apr 25 18:34 UTC] No.43656928[source]▶
Why are you setting the bar at "exceptional". If it means that you can write your git commit messages more quickly and with fewer errors then that's all the payoff most orgs need to make them worthwhile.
replies(1): >>43661377 #
7. bluefirebrand ◴[12 Apr 25 04:35 UTC] No.43661377{3}[source]▶
> Why are you setting the bar at "exceptional"

Because that is how they are being sold to us and hyped

> If it means that you can write your git commit messages more quickly and with fewer errors then that's all the payoff most orgs need to make them worthwhile.

This is so trivial that it wouldn't even be worth looking into, it's basically zero value

8. KronisLV ◴[12 Apr 25 11:41 UTC] No.43663543[source]▶
> I'd love to see a reproducible example of these tools producing something that is exceptional.

I’m happy that my standards are somewhat low, because the other day I used Claude Sonnet 3.7 to make me refactor around 70 source files and it worked out really nicely - with a bit of guidance along the way it got me a bunch of correctly architected interfaces and base/abstract classes and made the otherwise tedious task take much less time and effort, with a bit of cleanup and improvements along the way. It all also works okay, after the needed amount of testing.

I don’t need exceptional, I need meaningful productivity improvements that make the career less stressful and frustrating.

Historically, that meant using a good IDE. Along the way, that also started to mean IaC and containers. Now that means LLMs.

replies(1): >>43664482 #
9. xrraptr ◴[12 Apr 25 13:04 UTC] No.43664027[source]▶
I honestly think the problem is you are just a lot smarter than I am.

I find these tools wonderful but I am a lazy, college drop out of the most average intelligence, a very shitty programmer who would never get paid to write code.

I am intellectually curious though and these tools help me level up closer to someone like you.

Of course, if I had 30 more IQ points I wouldn't need these tools but I don't have 30 more IQ points.

10. ◴[12 Apr 25 14:04 UTC] No.43664482{3}[source]▶