SICP: The only computer science book worth reading twice? (2010)

I don’t know if distributed systems is consider part of “Computer Science” but it is a much more common problem that I see needs to be solved.

I try to write systems in the simplest way possible and then use observability tools to figure out where the design is deficient and then maybe I will pull out a data structure or some other “computer sciency” thing to solve that problem. It turns out that big O notation and runtime complexity doesn’t matter the majority of the time and you can solve most problems with arrays and fast CPUs. And even when you have runtime problems you should profile the program to find the hot spots.

What computer science doesn’t teach you is how memory caching works in CPUs. Your fancy graph algorithm may have good runtime complexity but it completely hoses the CPU cache and you may have been able to go faster with an array with good cache usage.

The much more common problems I have is how to deal with fault tolerance, correctness in distributed locks and queues, and system scalability.

Maybe I am just biased because I have a computer/electrical engineering background.

It is hard to design systems if you don't have the perspective of implementing them. Yes, you move up the value chain to designing things, no, but no, you don't get to skip gaining experience lower down the value chain.

> What computer science doesn’t teach you is how memory caching works in CPUs.

That was literally my first quarter in my CS undergrad 30 years ago, the old Hennessy and Patterson book, which I believe is still used today. Are things so different now?

> The much more common problems I have is how to deal with fault tolerance, correctness in distributed locks and queues, and system scalability.

All of that was covered in my CS undergrad, I wasn't even in a fancy computer engineering/EE background.

At my uni 10 years ago the CS program didn’t touch anything related to hardware, hell the CS program didn’t even need to take multivariable calculus. In my computer engineering program we covered solid state physics, electromagnetism, digital electronics design, digital signals processing, CPU architecture, compiler design, OS design, algorithms, software engineering, distributed systems design.

The computer engineering program took you from solid state physics and transistor design to PAXOS.

The CS program was much more focused on logic proofs and more formalism and they never touched anything hardware adjacent.

I realize this is different between programs, but from what I read and hear many CS programs these days start at Java and never go down abstraction levels.

I do agree with you that learning the fundamentals is important, but I would argue that a SICP type course is not fundamental — physics is fundamental. And once you learn how we use physics to build CPUs you learn that fancy algorithms and complex solutions are not necessary most of the time given how fast computers are today. If you can get your CPU pipelined properly with high cache hits, branch prediction hits, prefetch hits, and SIMD you can easily brute force many problems.

And for those 10% of problems which cannot be brute forced, 90% of those problems can be solved with profiling and memoization, and for the 10% of those problems you cannot solve with memoization you can solve 90% of them with b-trees.