Hand Tracking for Mouse Input (2023)

It unsettled me a lot about just how much work was put into making the JavaScript version of this work instead of a purely Python version, due to how OpenCV works. I wonder how universal the laggy OpenCV thing is, because my friend faced it too when working on an OpenCV application. Is it so unavoidable that the only option is to not use Python? I really hope that there is another way of going about this.

Anyways, I am very glad that you put in all that effort to make the JavaScript version work well. Working under limitations is sometimes cool. I remember having to figure out how PyTorch evaluated neural networks, and having to convert the PyTorch neural network into Java code that could evaluate the model without any external libraries (it was very inefficient) for a Java code competition. Although there may have been a better way, what I did was good enough.

Creating a faster python implementation can definitely be done. OpenCV is a thin wrapper over the C++ API so it's not due to some intrinsic python slowness. It is not easy to resolve though and I suspect the way python code is typically written lends itself to an accidentally blocking operation more often than JS code. It's hard to know without seeing the code.

author here, sorry you have to see my janky JavaScript solution XD but one good thing of going with Tauri is that developing the UI is pretty easy, since it's basically just some web pages, but with access to the system, through the JS <-> Rust communication.

also, rewriting neural network from PyTorch to Java sounds like a big task, I wonder if people are doing ML in Java