I fail to see how LLMs are anything beyond a lookup function retrieving information from a huge database (containing, in theory, all known human information), and then summarizing the results using language algorithms.
While incredibly powerful and transformative, it is not 'intelligence'. LLMs are forever knowledgebase bound. They are encyclopedias with a fancy way of presenting information looked up in the encyclopedia.
The 'presentation' has no concept, awareness, or understanding of the information being presented - and never will. And this is the critical line. Without comprehension, a LLM is incapable of being creative. Of coming up with new ideas. It cannot ponder. Wonder. Think.