←back to thread

Jagged AGI: o3, Gemini 2.5, and everything after

(www.oneusefulthing.org)
265 points ctoth | 1 comments | 20 Apr 25 14:55 UTC | HN request time: 0s | source
Show context
sejje ◴[20 Apr 25 17:04 UTC] No.43744995[source]▶
In the last example (the riddle)--I generally assume the AI isn't misreading, rather that it assumes you couldn't give it the riddle correctly, but it has seen it already.

I would do the same thing, I think. It's too well-known.

The variation doesn't read like a riddle at all, so it's confusing even to me as a human. I can't find the riddle part. Maybe the AI is confused, too. I think it makes an okay assumption.

I guess it would be nice if the AI asked a follow up question like "are you sure you wrote down the riddle correctly?", and I think it could if instructed to, but right now they don't generally do that on their own.

replies(5): >>43745113 #>>43746264 #>>43747336 #>>43747621 #>>43751793 #
1. Skunkleton ◴[21 Apr 25 00:32 UTC] No.43747621[source]▶
https://kagi.com/assistant/3752c5f9-bf5c-4a43-bada-b3eccbe94...

You should be able to click left right on the prompt to see different responses. Sonnet 3.7 with extended thinking notices the issue, and then chooses to totally ignore it with no explanation.

From Claude for those who don’t want to click:

Wait, I notice a difference from the traditional riddle. In this version, the surgeon says "I can operate on this boy" (affirmative) rather than "I can't operate on this boy" (negative).

This changes the nature of the puzzle somewhat. If the surgeon is saying they CAN operate, then we need to explain why this is surprising or seemingly impossible, but actually possible.

The traditional answer would still apply: the surgeon is the boy's mother.