> Surprisingly, Horace He pointed out that GPT-4 solved 10/10 pre-2021 problems and 0/10 recent problems in the easy category. The training data cutoff for GPT-4 is September 2021. This strongly suggests that the model is able to memorize solutions from its training set — or at least partly memorize them, enough that it can fill in what it can’t recall.
Probably the latency of introducing fresh content into the model training will quickly decrease, as these models develop commercial success.
It will be really nice when the AI can provide the most cutting edge solutions and up-to-date information relevant to your query. That will be really valuable, even if the AI cannot yet synthesize novel cutting-edge solutions to new problems.