←back to thread

Why birds do not fall while sleeping

(news.cnrs.fr)
230 points mdp2021 | 2 comments | 15 Oct 24 08:56 UTC | HN request time: 0s | source
Show context
Crazyontap ◴[17 Oct 24 03:05 UTC] No.41866060[source]▶
When I was younger, I was fascinated by evolution, especially the intricacies of how things just work. This fascination also explains why many people believe in the intelligent design theory.

However, witnessing the rapid evolution of AI with just a few hundred GPUs, enough data, and power, I no longer wonder what a billion years of feedback loops and randomness can achieve.

replies(18): >>41866202 #>>41866478 #>>41866660 #>>41866806 #>>41866826 #>>41867595 #>>41867652 #>>41867789 #>>41867813 #>>41867833 #>>41867834 #>>41867913 #>>41868264 #>>41868344 #>>41868565 #>>41868579 #>>41869785 #>>41909242 #
1. eru ◴[17 Oct 24 09:14 UTC] No.41867789[source]▶
The key insight for both evolution and contemporary AI is that hill-climbing (either completely random a la evolution, or guided locally like in back-propagation) can work really well, if you have enough dimensions to play with.

Hill climbing obviously gets stuck easily in 2 or 3 dimensions. So our intuition ain't reliable.

replies(1): >>41867934 #
2. carlmr ◴[17 Oct 24 09:41 UTC] No.41867934[source]▶
>Hill climbing obviously gets stuck easily in 2 or 3 dimensions. So our intuition ain't reliable.

Also our intuition about overfitting from lower-dimensional representations seems to be less of an issue at high dimensions.