←back to thread

Getting 50% (SoTA) on Arc-AGI with GPT-4o

(redwoodresearch.substack.com)
394 points tomduncalf | 1 comments | 17 Jun 24 21:51 UTC | HN request time: 0.237s | source
Show context
traject_ ◴[17 Jun 24 22:42 UTC] No.40712005[source]▶
We don't actually know if it is SOTA, the previous SOTA solution also got around the same on the evaluation set.
replies(1): >>40712275 #
1. cma ◴[17 Jun 24 23:13 UTC] No.40712275[source]▶
Yeah and GPT4o was potentially trained on this test set and if the tried to hold it out it was still likely trained on discussions of the problems.