Wouldn't someone talking to someone "lower" on the "oppression hierarchy" just be what we basically have today? That sounds like "privilege," or an "imbalanced power dynamic." I think you'll only be able to talk to equals, whatever that is, and by whatever metric is en vogue for that day.
In that circumstance, I think it is clear that my sexual orientation is the basis by which they are judging the authoritativeness I have to speak on the topic. Never mind the formal qualifications, or the logic or veracity of what I am actually saying. Like, I know we all have little unconscious checklists like that for judging whether someone is credible, but it is uncomfortable to see the effect live.
This is not a dig at you, btw, it seems clear that you’re making the best of the situation.