No, it's not crazy at all. Back in the early noughts everyone was associating themselves with Internet and WorldWideWeb (yes, they used to spell that out). Same thing happening today with AI. It is irritating ...
You would be surprised how many people have no idea what "machine learning" means (not even technical definition but just as in the field). I'm working on a PhD in an adjacent field and basically have to tell people I work in AI for them to have any semblance of what I do.
I don't think telling people that have no idea what machine learning is that you work with 'AI' is giving them any semblance of understanding what you do.
I used to be in the same camp, but have since abandoned this position. The world has changed. Very serious people use the term "AI" nowadays. I don't find that it's a high-precision signal for identifying charlatans anymore (though it is high-recall).