The past few projects I've done, I've made some effort to just not polyfill anything, it's always too easy to wind up with fills for things long supported in every major browser for the past 3-5 years.
On the flip side... if you're running something bespoke relying on an older JS engine, it's becoming harder to find pure polyfills/shims that aren't a tangled mess relying on DOM specific features.
While there are some newer features I don't mind seeing early, I've mostly just avoided touching some of the more recent features for now, only to avoid the mess that are the current state of fills.