Why do random forests work? They are self-regularizing adaptive smoothers

The idea that you can take hundreds of bad models that over fit (the individual decision trees), add even more randomness by randomly picking training data and features*, and averaging them together - it's frankly amazing that this leads to consistently OK models. Often not the best but rarely the worst. There's a reason they're such a common baseline to compare against.

*Unless you're using Sklearn, whose implementation of RandomForestRegressor is not a random forest. It's actually bagged trees because they don't randomly select features. Why they kept the misleading classname is beyond me.