←back to thread

291 points sebg | 1 comments | | HN request time: 0.274s | source
1. oli5679 ◴[] No.41897730[source]
My understanding of why bagging works well is because it’s a variance reduction technique.

If you have a particular algorithm, the bias will not increase if you train n versions in ensemble, but the variance will decrease as more anomalous observations won’t persistently be identified in submodel random samples and so won’t the persist in the bagging process.

You can test this. The difference between train and test auc will not increase dramatically as you increase number of trees in sklearn random forest for same data and hyperparameters.