Can you explain why? What makes them categorically different or at the very least why is "piracy" quantitatively worse than 'just' copyright violation?
Saying that piracy isn't copyright violation is an RMS talking point. It's not worth trying to ask why because the answer will be RMS said so and will not be backed by the common usage of the word.
You legitimately have it completely backwards. The word "piracy" was coopted to put a more severe spin on copyright violation. As a result, it became "the common usage of the word". But that was by design. And it's worth pushing back on.
Sweden has a political party called "The Pirate Party"(1), and "The Pirate Bay" is Swedish so I think a couple of Swedes memeing before it was cool has a significant impact on making the name stick but also taking the seriousness out of it.