Caching is an abstraction, not an optimization

Ousterhout's grad students did work on ramcloud with some research at facebook and Amazon on cache use at scale in complex organizations.

One bit of interesting trivia say for facebook (from memory): if you add all the RAM caches in redis/memcached/disk + db caches to make the thing work at scale, then for about 20-30% more memory you could've had the whole thing in memory 100% of the time.

The problem there is /the whole thing/ grows - often faster than your memory.