Maybe AGI is impossible not for any technical reasons, but because AI is a self-limiting technology? As AI improves, it destroys the educational system that creates the researchers that develop it.
They are working well now because they live off the pile of human-created online content. Give it a few years. When all the AI has to feed on is the output of other AI bots, they will likely hit a reality wall and halucinate themselves into insanity. Forget new physics. It will probably be hard to get an AI to admit the earth isnt flat.
I expect they'll keep training on the pre-2022 corpus and supplement with new curated material. Of course they'll use AI to curate so the same process will eventually happen but probably can be slowed significantly.
Web 2.0 is already doing a great job at "human model collapse". No generative content needed, people are consuming poor quality output of other humans and leading to a decline in discourse. Tik-Tokification.