←back to thread

Getting 50% (SoTA) on Arc-AGI with GPT-4o

(redwoodresearch.substack.com)
394 points tomduncalf | 2 comments | 17 Jun 24 21:51 UTC | HN request time: 0s | source
Show context
bjornsing ◴[18 Jun 24 02:58 UTC] No.40713656[source]▶
Can we be sure GPT-4o hasn’t been trained on the public test set?
replies(1): >>40713748 #
1. ivalm ◴[18 Jun 24 03:13 UTC] No.40713748[source]▶
It has been trained on public test set as it’s on github
replies(1): >>40714301 #
2. free_bip ◴[18 Jun 24 05:22 UTC] No.40714301[source]▶
This isn't necessarily true, OpenAI isn't open about how their data cleaning process works.