However, witnessing the rapid evolution of AI with just a few hundred GPUs, enough data, and power, I no longer wonder what a billion years of feedback loops and randomness can achieve.
LLM Algorithms don't "evolve" when trained, they just fit data in a pre-existing and hardwired "DNA". More GPU, data, energy consumption etc. simply means different weights (parameters) for the same fixed algorithm. Training involves no feedback loop on the algorithm design itself. The biological analogy is like what happens when you starve or overfeed somebody: They become skinny or obese (but they will not pass on that attribute to their offspring).
The algorithm's DNA is explicitly designed and put in place by human intelligence. When thinking of the observed "evolution" of algorithms we need to include the sum total of the people involved in algorithmic design and deployment, their cognitive toolkit, incentives etc. Now that part is definitely "evolving" (various mathematical, technical or economic breakthroughs), not biologically of course, but culturally.
So-called AI Winters (and other AI Seasons) are indeed evidence of this collective cultural movement. You could say that the invention / adoption of the multilayered neural net pattern has led to a sort of Cambrian explosion similar to going from single cell to multicellular organisms.