There is no such "thing" as "a dairy" that would or wouldn't care about something. It's all people making decisions and why wouldn't we strive to reduce suffering of other animals?!
Vegans also argue that the entire dairy industry, which necessarily requires keeping cows continually pregnant and separating them from their calves soon after birth, in itself creates immense suffering.
Animal products for better or worse are used everywhere, and by arguing against their use you can be accused of prioritising the welfare of horses over children if you support vaccines. My house was built on forest land that likely displaced animals when it was cleared too, and caused their suffering.
Or, I could say that my presence on the planet has an impact at every level, and I will do my best to try and be conscious of that impact.