However, witnessing the rapid evolution of AI with just a few hundred GPUs, enough data, and power, I no longer wonder what a billion years of feedback loops and randomness can achieve.
Take humans, for example. You can block your trachea and die through the simple act of eating. An intelligent (and omniscient) designer could have avoided that by better designing our overall our overall structure.
Or take the fact or ear bones are modified jaw bones. Or if you believe in intelligent design, ask why our intelligent designer thought it wise to link our jaw to our ears so that it’s hard to hear things when you’re chewing.
The examples you quote do look like a problem today, but I think they must have worked to some benefit to bring us here.
No, the other consequence of evolution is that whatever we are today is not enough to kill us. Some of the things in our bodies and behavior are just useless, but not bad enough that they endanger our life and cut the genetic deviation responsible for it, so it just keeps on being there.
There's no good reason we still have 5 fingers on feet, but they're also not bad enough that having them all would endanger us as as species, so they just stay there.