←back to thread

Show HN: Automated smooth Nth order derivatives of noisy data

(github.com)
146 points hugohadfield | 2 comments | 16 Oct 24 20:17 UTC | HN request time: 0s | source

This little project came about because I kept running into the same problem: cleanly differentiating sensor data before doing analysis. There are a ton of ways to solve this problem, I've always personally been a fan of using kalman filters for the job as its easy to get the double whammy of resampling/upsampling to a fixed consistent rate and also smoothing/outlier rejection. I wrote a little numpy only bayesian filtering/smoothing library recently (https://github.com/hugohadfield/bayesfilter/) so this felt like a fun and very useful first thing to try it out on! If people find kalmangrad useful I would be more than happy to add a few more features etc. and I would be very grateful if people sent in any bugs they spot.. Thanks!
Show context
youoy ◴[17 Oct 24 11:39 UTC] No.41868662[source]▶
Nice work! Just one quick question (maybe it's clear but I have not looked at it in depth). It says it computes the derivative for non-uniformly sampled time series data and the example image shows this. Is this also well behaved if the sampled measurements have noise (it is not the case of the example)? Or should one use a different approach for that? Thanks!
replies(1): >>41868758 #
hugohadfield ◴[17 Oct 24 11:52 UTC] No.41868758[source]▶
Thanks! So the example image is actually with both non-uniformly sampled measurements and noise :) works great for both/either
replies(1): >>41868772 #
hugohadfield ◴[17 Oct 24 11:54 UTC] No.41868772[source]▶
Noise is added here: ``` # Generate noisy sinusoidal data with random time points np.random.seed(0) t = sorted(np.random.uniform(0.0, 10.0, 100)) noise_std = 0.01 y = np.sin(t) + noise_std * np.random.randn(len(t)) true_first_derivative = np.cos(t) true_second_derivative = -np.sin(t) ``` changing noise_std will change the magnitude of the noise added, hope that helps!
replies(1): >>41868814 #
1. youoy ◴[17 Oct 24 12:00 UTC] No.41868814{3}[source]▶
Ah true! Now I see it on the image too, I was looking at it on the phone and it did not look like it. I will definitely give it a try! Thank you very much
replies(1): >>41868852 #
2. hugohadfield ◴[17 Oct 24 12:06 UTC] No.41868852[source]▶
no problem!