If you look at the history of the NAACP trying to fight legal discrimination, one of the things they were quite focused on was ensuring that the cases involved people who are as clean as possible. The messenger matters as much as the message does; the more imperfect the messenger it is, the more it allows those against the message to turn the conversation from the message to the messenger.
Stallman, even without the stuff mentioned here, was already a pretty poor choice for messenger. His tendency to focus on semantic nitpicking gave his arguments a kind of tedious quality to them (GNU/Linux, anyone?). The outspoken political views can turn off people who are not politically aligned. And he's always given off this sort of skeevy vibe to me personally--and that sentiment has seemed to be shared by lots of other people. Long before any of the stuff being complained about here happened!
The negative influence of Stallman isn't purely theoretical anymore. We now have a couple of stories of people saying that he made them personally uncomfortable with unwanted seemingly sexual advances. Apparently, these have continued after being explicitly told by several people to knock it off. Several organizations have suspended their funding and involvement with the FSF over his reinstatement. Whatever you think of the accuracy of the accusations against him, the general perception of him is clearly negative, and to be frank, it doesn't seem like he brings any positive qualities that would make letting him go be a tough call (consider Elon Musk, who is obviously a pretty effective salesman for the future and props up Tesla's share price even as his outspoken political views are causing real problems for his companies).