←back to thread

Show HN: I wrote an autodiff in C++ and implemented LeNet with it

(gitlab.com)
36 points mebassett | 1 comments | 18 Oct 24 00:52 UTC | HN request time: 0s | source
Show context
tightbookkeeper ◴[18 Oct 24 09:43 UTC] No.41877766[source]▶
using new for each node and value, combined with virtual dispatch tends to be a c++ anti-pattern. It looks like you are writing other languages in C++ syntax, motivated by promises of speed.

The actual benefits of C++ come when you approach problems differently. This is a case where more exposure to C helps you avoid all the Java isms.

Things to consider:

- can you allocate memory for the whole system? - can you make types homogenous so they can fit in tight arrays (unions are common for nodes) - can you batch similar types - specially for auto diff/math can you represent operations as a stack instead of a tree?

I am only bringing this up because you said your goal was to learn C++.

replies(3): >>41877847 #>>41877967 #>>41888674 #
1. mebassett ◴[18 Oct 24 10:24 UTC] No.41877967[source]▶
> - can you allocate memory for the whole system? - can you make types homogenous so they can fit in tight arrays (unions are common for nodes) - can you batch similar types - specially for auto diff/math can you represent operations as a stack instead of a tree?

these are good questions, thank you! I'm "learning" c++ in a completely different way - looking at books rather than looking at existing code. Appreciate this comment!

Some of these questions I had thought about, but was learning more towards "describe the function in some higher level representation and then 'compile' it down to something computable and autodiff-able". This is exactly where my mind goes when I think about allocating memory for the whole system. that felt more like a racket/lisp way of looking at the problem.