←back to thread

Show HN: I wrote an autodiff in C++ and implemented LeNet with it

(gitlab.com)
36 points mebassett | 5 comments | 18 Oct 24 00:52 UTC | HN request time: 1.011s | source
Show context
tightbookkeeper ◴[18 Oct 24 09:43 UTC] No.41877766[source]▶
using new for each node and value, combined with virtual dispatch tends to be a c++ anti-pattern. It looks like you are writing other languages in C++ syntax, motivated by promises of speed.

The actual benefits of C++ come when you approach problems differently. This is a case where more exposure to C helps you avoid all the Java isms.

Things to consider:

- can you allocate memory for the whole system? - can you make types homogenous so they can fit in tight arrays (unions are common for nodes) - can you batch similar types - specially for auto diff/math can you represent operations as a stack instead of a tree?

I am only bringing this up because you said your goal was to learn C++.

replies(3): >>41877847 #>>41877967 #>>41888674 #
pjmlp ◴[18 Oct 24 10:04 UTC] No.41877847[source]▶
The Java isms were C++ best practices in C++ GUI frameworks before Java was even an idea, there is even a famous book about them.
replies(1): >>41877940 #
1. tightbookkeeper ◴[18 Oct 24 10:19 UTC] No.41877940[source]▶
- GUI frameworks work with an order of 10 or 100 long lived widgets. Number crunchers work on the order of millions of values that exist for a few cycles

- would anybody recommend you should reach for C++ to write Java style OOP in 2024?

- Best practice according to who?

replies(1): >>41878411 #
2. pjmlp ◴[18 Oct 24 11:35 UTC] No.41878411[source]▶
First of all, it is C++ OOP style, Java cloned C++, not the other way around.

Secondly, all major surviving C++ GUI frameworks are still using the same style.

Best practice according to well known folks in the computing industry, with more impact than any of us will ever have on our lifetimes.

replies(1): >>41878675 #
3. mebassett ◴[18 Oct 24 12:15 UTC] No.41878675[source]▶
I think the main point here is that while this style might make sense for a GUI framework it's not so great for a numerical library like this.

For example: my library is really slow.

replies(1): >>41878728 #
4. pjmlp ◴[18 Oct 24 12:21 UTC] No.41878728{3}[source]▶
And instead of blaming OOP, have you actually used a profiler?
replies(1): >>41879812 #
5. tightbookkeeper ◴[18 Oct 24 14:31 UTC] No.41879812{4}[source]▶
Are you suggesting that many individual allocations and virtual dispatch are not impactful, or just advocating use of a profiler?

Once again, the context of this thread is someone taking an interest in C++ to write fast autodiff code.