←back to thread

Caching is an abstraction, not an optimization

(buttondown.com)
134 points samuel246 | 3 comments | 01 Jul 25 10:42 UTC | HN request time: 0.608s | source
Show context
ckdot2 ◴[03 Jul 25 19:03 UTC] No.44458190[source]▶
"I think now caching is probably best understood as a tool for making software simpler" - that's cute. Caching might be beneficial for many cases, but if it doesn't do one thing then this is simplifying software. There's that famous quote "There are only two hard things in Computer Science: cache invalidation and naming things.", and, sure, it's a bit ironical, but there's some truth in there.
replies(11): >>44458265 #>>44458365 #>>44458502 #>>44459091 #>>44459123 #>>44459372 #>>44459490 #>>44459654 #>>44459905 #>>44460039 #>>44460321 #
whateveracct ◴[03 Jul 25 19:24 UTC] No.44458365[source]▶
caching often does simplify software though when done well

and - as the OP suggests - it works best when the cache is a well-defined abstraction with properties and rules about how it works

just because "caching" is mentioned in a meme doesn't mean it can't be true that it can simplify software

replies(7): >>44458520 #>>44458529 #>>44458670 #>>44458723 #>>44458778 #>>44458835 #>>44460773 #
jameshart ◴[03 Jul 25 20:00 UTC] No.44458670[source]▶
If you hide caching away as an implementation detail behind an abstraction, it comes back and bites you as a leaky abstraction later.

Look at how CPU cache line behaviors radically change the performance of superficially similar algorithms.

Look at how query performance for a database server drops off a cliff the moment the working cache no longer fits in memory.

Hiding complexity can be a simplification, until you exceed the bounds of the simplification and the complexity you hid demands your attention anyway.

replies(1): >>44459308 #
1. atq2119 ◴[03 Jul 25 21:21 UTC] No.44459308[source]▶
CPUs are still a great example for how caching simplifies things.

There's a long history in computer architecture of cores and accelerators that don't have a cache but instead rely on explicitly programmed local scratchpads. They are universally more difficult to program than general purpose CPUs because of that.

replies(1): >>44459559 #
2. hmottestad ◴[03 Jul 25 22:06 UTC] No.44459559[source]▶
I’m sure the CPU designers would love it if they didn’t need several different layers of cache. Or no cache at all. Imagine if memory IOPS were as fast as L1 cache, no need for all that dedicated SRAM on the chip or worry about side channel attacks.
replies(1): >>44466730 #
3. atq2119 ◴[04 Jul 25 18:16 UTC] No.44466730[source]▶
Sure, but we were talking about the perspective of software developers. The hardware designers take on complexity so that the software developer's work can be simpler.