Getting 50% (SoTA) on Arc-AGI with GPT-4o

The article jumps to the conclusion that "Given that current LLMs can perform decently well on ARC-AGI" after having used multiple hand-crafted tricks to get to these results, including "I also did a small amount of iteration on a 100 problem subset of the public test set" which is hidden in the middle of the article and not mentioned in the bullet list at the top.

Adding the close-to ad-hominem attack on Francois Chollet with the comics at the beginning (Francois never claimed to be a neuro-symbolic believer), this work does a significant disservice to the community.

>Francois never claimed to be a neuro-symbolic believer

His response:

"This has been the most promising branch of approaches so far -- leveraging a LLM to help with discrete program search, by using the LLM as a way to sample programs or branching decisions. This is exactly what neurosymbolic AI is, for the record..."

"Deep learning-guided discrete search over program space is the approach I've been advocating, yes... there are many different flavors it could take though. This is one of them (perhaps the simplest one)."

https://x.com/fchollet/status/1802773156341641480