There is a massive difference between other people reacting to your speech and choosing not to associate with you or propagate what you say, and the government banning your speech and prosecuting you for it. You can exercise your freedom of speech, and other people can exercise their freedom of association to want nothing to do with you.