Thoughts on thinking

I don't agree directly with this, but a variant of it does bother me: will the auto-regressive nature of AI eventually limit the novelty of the ideas humanity can come up with?

So many breakthroughs come from people who work either in ignorance or defiance of existing established ideas. Almost by definition, in fact - to a large extent, everything obvious has already been thought. So to some extent, all the real progress happens in places that violate norms and pre-established logic.

So what's going to happen now if every idea has to run the gauntlet of a supremely intelligent but fully regressive AI? It feels like we could lose a tremendous amount of the potential for original thought from humanity. A good counter argument would be that this has already happened and we're still making progress. I just wonder however if it's a question of degree and that degree matters.

Indeed. Major advancements throughout history often happened because someone looked at a problem differently than traditional approaches.

The AI will have been trained predominantly on the traditional approaches.

I feel AI will be fundamentally limited to regurgitating past ideas and intelligence.

It may at least use a breadth of knowledge to save some people time by helping them avoid repeating work already done.

I’d love to see an AI trained only on knowledge up to 1800 come up with a single invention of the past 200 years. (It won’t happen)