(karpathy.medium.com)

346 points swatson741 | 1 comments | 02 Nov 25 05:20 UTC | HN request time: 0s | source

Show context

jamesblonde ◴[02 Nov 25 08:20 UTC] No.45788699[source]▶

I have to be contrarian here. The students were right. You didn't need to learn to implement backprop in NumPy. Any leakiness in BackProp is addressed by researchers who introduce new optimizers. As a developer, you just pick the best one and find good hparams for it.

replies(5): >>45788770 #>>45788820 #>>45788864 #>>45788882 #>>45790922 #

1. PeterStuer ◴[02 Nov 25 08:48 UTC] No.45788820[source]▶

>>45788699 #

The problem with your reasoning is you never tackle your "unknown unknowns". You just assume they are "known unknowns".

Diving through the abstraction reveals some of those.

↑

Backpropagation is a leaky abstraction (2016)