What Is the Fourier Transform?

Always blew my mind that every signal can be recreated simply by adding different sine waves together.

Taking the concept to a 2d path: https://www.myfourierepicycles.com

And, with your own drawing: https://gofigure.impara.ai

Only if it is band-limited.

Back in my uni days I did not get why that works. Why are sine waves special?

Turns out... they are not! You can do the same thing using a different set of functions, like Legendre polynomials, or wavelets.

Another place where functions are approximated is in machine learning which use a variety of non-linear functions for activations, for example the ReLU f(x)= max(0,x)

No, even functions with non-finite frequency representation. You just need a non-finite number of sines. Nyquist speaks only to a finite number of samples.

And of infinite duration, if you want to split hairs.

Same thing! :-) In the purest sense, finite bandwidth requires infinite duration and finite duration requires infinite duration.

The real world is somewhere in between. It must involve quantum mechanics (in a way I don't really understand), as maximum bandwidth/minimum wavelength bump up against limits such as the Planck length and virtual particles in a vacuum.

>Turns out... they are not! You can do the same thing using a different set of functions, like Legendre polynomials, or wavelets.

Yup, any set of orthogonal functions! The special thing about sines is that they form an exceptionally easy-to-understand orthogonal basis, with a bunch of other nice properties to boot.

It makes more sense when you approach it from linear algebra.

Like you can make any vector in R^3 `<x,y,z>` by adding together a linear combination of ` <1,0,0> `, ` <0,1,0> `, ` <0,0,1> `, turns out you can also do it using `<exp(j2pi0/30), exp(j2pi0/31), exp(j2pi0/32)>`, `<exp(j2pi1/30), exp(j2pi1/31), exp(j2pi1/32)>`, and `<exp(j2pi2/30), exp(j2pi2/31), exp(j2pi2/32)>`.

You can actually do it with a lot of different bases. You just need them to be linearly independent.

For the continuous case, it isn't all that different from how you can use a linear combination of polynomials 1,x,x^2,x^3,... to approximate functions (like Taylor series).

To be maximally pedantic, sine waves (or complex exponentials through Euler's formula), ARE special because they're the eigenfunctions of linear time-invariant systems. For anybody reading this without a linear algebra background, this just means using sine waves often makes your math a lot less disgusting when representing a broad class of useful mathematical models.

Which to your point: You're absolutely correct that you can use a bunch of different sets of functions for your decomposition. Linear algebra just says that you might as well use the most convenient one!

You are being confused with #samples needed for perfect reconstruction, i.e. Nyquist sampling frequency. Fourier series/transforms work regardless of the bandwidth of the signal, as long as the integral exists, i.e. it must vanish at infinity.

Essentially it's just projection in infinite-dimensional vector spaces.

Are the dirac or kronecker delta functions infinite duration? I guess it depends on the proof whether you can shorten them or not.

>They're eigenfunctions of linear time-invariant systems

For someone reading this with only a calculus background, an example of this is that you get back a sine (times a constant) if you differentiate it twice, i.e. d^2/dt^2 sin(nt) = -n^2 sin(nt). Put technically, sines/cosines are eigenfunctions of the second derivative operator. This turns out to be really convenient for a lot of physical problems (e.g. wave/diffusion equations).

That's what is commonly understood by reconstruction: perfect reconstruction. And for that you need a band-limited signal. Otherwise he would have said approximate- or lossy reconstruction.

I swear I read in a text book once that Fourier discovered this while a boat. He looked out at the waves on the ocean and saw how there were many different sized waves combining to make up the larger waves. I could never find it again, but that visual helped me understand how the Fourier transform works.

The Heisenberg uncertainty principle in quantum mechanics comes about precisely because position and momentum are a Fourier transform pair.

Related, but not quite the same thing. Band-limiting is needed to avoid aliasing. The infinite-duration part (or perfect phase continuity at the boundaries, or a window function...) is needed to avoid Gibbs ringing.

An interesting anecdote from Lanczos[1] claims that Michelson (of interferometer fame) observed Gibbs ringing when he tried to reconstruct a square wave on what amounted to a steampunk Fourier analyzer [2]. He reportedly blamed the hardware for lacking the necessary precision.

1: https://math.univ-lyon1.fr/wikis/rouge/lib/exe/fetch.php?med...

2: https://engineerguy.com/fourier/pdfs/albert-michelsons-harmo...

Can we agree that a fun thing about the Fourier Transform is the many different ways such a simple idea (a liner combination of basis functions) can be viewed and the many subtle implications it has?

For example, one viewpoint is that "Gibbs ringing" is always present if the bandwidth is limited, just that in the "non-aliased" case the sampling points have been chosen to coincide with the zero-crossings of the Gibbs ringing.

I find that my brain explodes each time I pick up the Fourier Transform, and it takes a few days of exposure to simultaneously get all the subtle details back into my head.

For sure. As I understand it, though, the Gibbs phenomenon arises due to the sinc kernel's infinite support (sinc in Fourier domain = rectangular window in time domain, equivalent to no window at all.)

No amount of precision, no number of coefficients, no degree of lowpass filtering can get around the fact that sin(x)/x never decays all the way to zero. So if you don't have an infinitely-long (or seamlessly repeating) input signal, you must apply something besides a rectangular window to it or you will get Gibbs ringing.

There is always more than one way to look at these phenomena, of course. But I don't think the case can be made that bandlimiting has anything to do with Gibbs.

In that vein, I've always wondered whether the fact that we live in a fundamentally quantum universe is just a mathematically consistent side effect of the Fourier transform and the Universe's limited extent in space-time. Is the Planck time just the point at which we can't determine the difference in frequency of two signals because the wavelength of their beat frequency has to fit inside the Universe? It's fun to think about.

And you can essentially "observe" the Heisenberg principle when looking at a moving object. If you are observing a plane for example to more accurately know its velocity you will need to observe over a longer time, but if you do this you loose accuracy about its position. This does affect radar systems, who can either send short pulses to accurately pin down the position or long pulses to measure the speed.

> And for that you need a band-limited signal.

Luckily, we live in a physical universe, where such mathematical oddities, like infinite bandwidth signals, cannot exist, so this isn't an actual issue. Any signal that that contains infinite bandwidths only exists because it has sampling artifacts. You would, necessarily, be attempting to reconstruct errors. There are many "tricks" around dealing with such flawed signals. But yes, you can't fully reconstruct impossible signals with FFT.