Backpropagation is a leaky abstraction (2016)

I took a course in my Master's (URV.cat) where we had to do exactly this, implementing backpropagation (fwd and backward passes) from a paper explaining it, using just basic math operations in a language of our choice.

I told everyone this was the best single exercise of the whole year for me. It aligns with the kind of activity that I benefit immensely but won't do by myself, so this push was just perfect.

If you are teaching, please consider this kind of assignments.

P.S. Just checked now and it's still in the syllabus :)

I did this in highschool from some online textbook in plain Java. I recall implementing matrix multiplication myself being the hardest part.

I made a UI that showed how the weights and biases changed throughout the training iterations.

The difference in understanding (for me and how my brain works) between reading the paper in what appears to be a future or past alien language & doing a minimal paper / code example is massive.

same here, even more if I'm doing it over few days and different angles

Is that paper publicly available?

I had a whole course just about how computers do maths. Matrix multiplication, linear fit, finding eigenvectors, multiplication and division, square root, solving linear systems, numerically calculating differential equations, spline interpolation, FEM analysis.

"Computers are good at maths" is normally a pretty obvious statement... but many things we take for granted from analytical mathematics, is quite difficult to actually implement in a computer. So there is a mountain of clever algorithms hiding behind some of the seemingly most obvious library operations.

One of the best courses I've ever had.

Would you mind sharing which course it was? Is it available online by any chance?

Unfortunately it was a course at my university, and in Swedish. But it wouldn't surprise me if there are similar courses online.

Hi! It's not public, it's part of the https://www.urv.cat/en/studies/master/courses/computer-secur... and I've not found it online

just found it! but it's private, I can send it to you if interested but not to publish it