Caching is an abstraction, not an optimization

"I think now caching is probably best understood as a tool for making software simpler" - that's cute. Caching might be beneficial for many cases, but if it doesn't do one thing then this is simplifying software. There's that famous quote "There are only two hard things in Computer Science: cache invalidation and naming things.", and, sure, it's a bit ironical, but there's some truth in there.

(You forgot off-by-1 errors.)

All software has to name things, and count. Caching (including invalidation) is best understood as a liability. If you can foist it off on your CPU and OS and DB, good for you. Programming whatever you're actually trying to get done is already hard enough.

Off by 1-errors is not part of the original quote, but is just a later addon to make it funny.

They also tend not to be very hard.

They're not hard but I will say that when I was writing an app that was using both JavaScript and Julia, I kept getting off-by-one errors because Julia starts at 1 instead of 0.

Really the only time in my entire professional career that off-by-one errors have actually given me headaches.

I think that is from a time when popular programming languages varied in their behavior and also when people were writing for loops with incrementation all the time.

A lot of languages have just settled on zero indexing, and many now have some variation of for/each or for/of that would eliminate a lot of potential ways to encounter this error.