I always assumed it was because FM station bandwidths (200kHz) are much wider than AM (10kHz). AM's 10 kHz chops off a lot of human-hearable frequencies.
AM does use the frequency, it just doesn't need as much and uses it differently than FM. If it was all at a single frequency, there just be a single tone getting louder and softer.
Once you change the amplitude of a sine wave (modulate it) it's no longer a side wave. It spreads in the frequency domain. Take the fourier transform of that and you can see the frequency components.