The problem is that to outsiders, the initial set of gatekeepers who arose naturally in the early community as "the people that knew what it was about" will themselves appear to be "the toxic assholes", so every community will naively eventually cut out its gatekeepers to be more inviting to newcomers.
Only to have the actual toxic assholes flood in, become the new gatekeepers, and dominate the discussion, and suddenly your Faces of Evil speedrunning Discord must have a stance on the war on Gaza and the US election because we clearly need to keep out the neo-Nazis according to our CoCs, right?
And no, I don't have an answer to this other than to largely disconnect from online platforms and start engaging in your local community. Something I myself am not guilty of doing.