Essential Coding Theory [pdf]

Claude Shannon's "The Mathematical Theory of Communication" (not mentioned by name, but referenced in the PDF) is a really pleasant little read. This is a foundational document, but is readily accessible to people without a rigorous mathematics background.

https://openlibrary.org/works/OL2296213W/The_mathematical_th...

Shannon is also a great way to get started understanding mathematical reasoning. Largely because he derives information entropy from basic desiderata regarding how a mathematical model of information should behave. That is, information entropy doesn’t initially have any meaning, it just fits Shannon’s requirements.

What’s brilliant about this is that Shannon accidentally arrives at a definition that is essentially identical to thermodynamic entropy (it was actually Von Neumann who pointed this out and gave it the name).

My experience is that many people fail to understand mathematics because mathematics often follows from “what do such and such rules imply” rather than building an intuitive model of the world (which is closer to where Physics traditionally falls).

Interestingly enough though, Shannon only establishes the framework which makes coding theory possible. He doesn’t actually implement any of these examples in that paper.

I used to run a book club many years ago at a startup that worked through the book version of the paper. A great way for anyone to understand both information theory specifically and mathematics in general.