←back to thread

Differentiable Programming from Scratch

(thenumb.at)
108 points sksxihve | 1 comments | 17 Apr 25 04:30 UTC | HN request time: 0.25s | source
Show context
weinzierl ◴[17 Apr 25 09:27 UTC] No.43714602[source]▶
"Now that we understand differentiation, let’s move on to programming. So far, we’ve only considered mathematical functions, but we can easily translate our perspective to programs. For simplicity, we’ll only consider pure functions, i.e. functions whose output depends solely on its parameters (no state)."

I think I've seen this notion that the constraint is pureness also in documentation of autodiff libraries, but this cannot be strong enough, right?

It easy enough to come up with functions that are nowhere differentiable. So my question is, what are the actual requirements a state of the art autodiff library has for the input function and why do people focus on the pureness aspect if that is probably the least of the problems.

replies(3): >>43714671 #>>43716612 #>>43717272 #
1. yorwba ◴[17 Apr 25 09:40 UTC] No.43714671[source]▶
If the function is pure, autodiff can produce a result. If the function is not differentiable for a given input, the result produced by autodiff of course cannot be the derivative (since none exists) but it could be another number (e.g. a subderivative) that is useful in some of the ways a derivative would have been useful.