←back to thread

Jagged AGI: o3, Gemini 2.5, and everything after

(www.oneusefulthing.org)
265 points ctoth | 3 comments | 20 Apr 25 14:55 UTC | HN request time: 1.458s | source
Show context
sejje ◴[20 Apr 25 17:04 UTC] No.43744995[source]▶
In the last example (the riddle)--I generally assume the AI isn't misreading, rather that it assumes you couldn't give it the riddle correctly, but it has seen it already.

I would do the same thing, I think. It's too well-known.

The variation doesn't read like a riddle at all, so it's confusing even to me as a human. I can't find the riddle part. Maybe the AI is confused, too. I think it makes an okay assumption.

I guess it would be nice if the AI asked a follow up question like "are you sure you wrote down the riddle correctly?", and I think it could if instructed to, but right now they don't generally do that on their own.

replies(5): >>43745113 #>>43746264 #>>43747336 #>>43747621 #>>43751793 #
Jensson ◴[20 Apr 25 17:25 UTC] No.43745113[source]▶
> generally assume the AI isn't misreading, rather that it assumes you couldn't give it the riddle correctly, but it has seen it already.

LLMs doesn't assume, its a text completer. It sees something that looks almost like a well known problem and it will complete with that well known problem, its a problem specific to being a text completer that is hard to get around.

replies(6): >>43745166 #>>43745289 #>>43745300 #>>43745301 #>>43745340 #>>43754148 #
og_kalu ◴[20 Apr 25 18:01 UTC] No.43745340[source]▶
Text Completion is just the objective function. It's not descriptive and says nothing about how the models complete text. Why people hang on this word, I'll never understand. When you wrote your comment, you were completing text.

The problem you've just described is a problem with humans as well. LLMs are assuming all the time. Maybe you would like to call it another word, but it is happening.

replies(2): >>43745745 #>>43746034 #
codr7 ◴[20 Apr 25 18:58 UTC] No.43745745[source]▶
With a plan, aiming for something, that's the difference.
replies(2): >>43745781 #>>43746301 #
1. losvedir ◴[20 Apr 25 20:26 UTC] No.43746301[source]▶
So do LLMs. "In the United States, someone whose job is to go to space is called ____" it will say "an" not because that's the most likely next word, but because it's "aiming" (to use your terminology) for "astronaut" in the future.
replies(2): >>43746549 #>>43756579 #
2. codr7 ◴[20 Apr 25 21:09 UTC] No.43746549[source]▶
I don't know about you, but I tend to make more elaborate plans than the next word. I have a purpose, an idea I'm trying to communicate. These things don't have ideas, they're not creative.
3. yahoozoo ◴[21 Apr 25 21:15 UTC] No.43756579[source]▶
Are we sure “an astronaut” is not the token?