←back to thread

Backpropagation is a leaky abstraction (2016)

(karpathy.medium.com)
346 points swatson741 | 1 comments | 02 Nov 25 05:20 UTC | HN request time: 0.203s | source
Show context
joshdavham ◴[02 Nov 25 07:14 UTC] No.45788432[source]▶
Given that we're now in the year 2025 and AI has become ubiquitous, I'd be curious to estimate what percentage of developers now actually understand backprop.

It's a bit snarky of me, but whenever I see some web developer or product person with a strong opinion about AI and its future, I like to ask "but can you at least tell me how gradient descent works?"

I'd like to see a future where more developers have a basic understanding of ML even if they never go on to do much of it. I think we would all benefit from being a bit more ML-literate.

replies(5): >>45788456 #>>45788499 #>>45788793 #>>45788867 #>>45798902 #
confirmmesenpai ◴[02 Nov 25 08:40 UTC] No.45788793[source]▶
so if you want to have a strong opinion on electric cars you need to be able to explain how an electric engine works right?
replies(3): >>45789972 #>>45791816 #>>45798921 #
1. chermi ◴[02 Nov 25 17:15 UTC] No.45791816[source]▶
Depends on what exactly the opinion is, but generally I'd say yes. If it's about their looks, maybe not.. but even then understanding the basics that determine things like not needing air intake, exhaust, the placement of batteries, etc. can be helpful.

If it's about the supply chain, understanding at least the requirements for magnets is helpful.

On way to make sure you understand all of these things is to understand the electric motor. But you could learn the separate pieces of knowledge on the fly too.

The more you understand the fundamentals of what you're talking about, the more likely you are to have genuine insight because you can connect more aspects of the problem and understand more of the "why".

TL;DR it depends, but it almost always helps.