I've noticed the same thing with other fields as well, not just art. Cooking is this way, for example. The food that fancy chefs at fancy restaurants make is so ridiculous that it feels like a joke sometimes. And as far as I can tell, it's the same thing. Those chefs are bored of normal food, are trying super hard to make something creative that has never been done before, and have lost sight of the fact that it's just not going to appeal to people who aren't as bored with food as they are. Maybe it's the inevitable result of being steeped in a craft and spending all your time on it, IDK.
Taking the example of free jazz, the artists are trying to free themselves from what they see as restraints on expression. However, the human mind and heart are themselves governed by pattern and organization, which is why most music took the forms that it did. Departing from those typical structures is an artistic choice, but the artists can't be surprised when most listeners don't respond well to those choices. Perhaps they don't care much about the listeners anyway.
Even when I make art with other people in mind I still give preference to my own personal aesthetic impulses. Art isn't always a product seeking product market fit.