What Is the Fourier Transform?

Back in my uni days I did not get why that works. Why are sine waves special?

Turns out... they are not! You can do the same thing using a different set of functions, like Legendre polynomials, or wavelets.

Another place where functions are approximated is in machine learning which use a variety of non-linear functions for activations, for example the ReLU f(x)= max(0,x)

>Turns out... they are not! You can do the same thing using a different set of functions, like Legendre polynomials, or wavelets.

Yup, any set of orthogonal functions! The special thing about sines is that they form an exceptionally easy-to-understand orthogonal basis, with a bunch of other nice properties to boot.

To be maximally pedantic, sine waves (or complex exponentials through Euler's formula), ARE special because they're the eigenfunctions of linear time-invariant systems. For anybody reading this without a linear algebra background, this just means using sine waves often makes your math a lot less disgusting when representing a broad class of useful mathematical models.

Which to your point: You're absolutely correct that you can use a bunch of different sets of functions for your decomposition. Linear algebra just says that you might as well use the most convenient one!

>They're eigenfunctions of linear time-invariant systems

For someone reading this with only a calculus background, an example of this is that you get back a sine (times a constant) if you differentiate it twice, i.e. d^2/dt^2 sin(nt) = -n^2 sin(nt). Put technically, sines/cosines are eigenfunctions of the second derivative operator. This turns out to be really convenient for a lot of physical problems (e.g. wave/diffusion equations).