"The primary weakness in the record of past restrictions is the lack of specific causation findings with respect to any discrete instance of content moderation."
What is a boycott but cancel culture? The idea of the free market is that good behaviors and products emerge because consumers "vote" with their wallet. If a company has bad values I don't support then I don't shop there. Enough people do that and the company collapses. So, what remains is an economy where every company acts virtuously.
Theoretically. Then enters propaganda and the GOP. They tell you this invisible hand is bad, and companies should be able to do anything. At a glance this appears to be free speech, but it's not - it's the exact opposite.
You see, they can say anything they want, but we can't. We may not criticize them. Our opinions are not valid, they're "Cancel Culture".
Furthermore, spend a little time on BlueSky and you will find huge support for the hate speech laws found in other countries.
Finally, the distinction between government regulation of speech and private regulation of speech is key in the court of law, but it is almost irrelevant from the point of view of a philosophy that values open inquiry, debate, and dissent as indispensible to human dignity and progress.
"We have a right, also, in various ways, to act upon our unfavourable opinion of any one, not to the oppression of his individuality, but in the exercise of ours. We are not bound, for example, to seek his society; we have a right to avoid it (though not to parade the avoidance), for we have a right to choose the society most acceptable to us. We have a right, and it may be our duty, to caution others against him, if we think his example or conversation likely to have a pernicious effect on those with whom he associates. We may give others a preference over him in optional good offices, except those which tend to his improvement."
That's a boycott.
Mill distinguishes between natural penalties (which are just results of others distancing from the person) and artificial punishment:
"It makes a vast difference both in our feelings and in our conduct towards him, whether he displeases us in things in which we think we have a right to control him, or in things in which we know that we have not. If he displeases us, we may express our distaste, and we may stand aloof from a person as well as from a thing that displeases us; but we shall not therefore feel called on to make his life uncomfortable."
I think it is clear that actions like urging an employer to fire such person are more on level "make his life uncomfortable" than "we may express our distaste and we distance".
For actions "which concerns others" Mill writes:
"The evil consequences of his acts do not then fall on himself, but on others; and society, as the protector of all its members, must retaliate on him; must inflict pain on him for the express purpose of punishment, and must take care that it be sufficiently severe. In the one case, he is an offender at our bar, and we are called on not only to sit in judgment on him, but, in one shape or another, to execute our own sentence: in the other case, it is not our part to inflict any suffering on him, except what may incidentally follow from our using the same liberty in the regulation of our own affairs, which we allow to him in his."
"But there is a sphere of action in which society, as distinguished from the individual, has, if any, only an indirect interest; comprehending all that portion of a person’s life and conduct which affects only himself, or if it also affects others, only with their free, voluntary, and undeceived consent and participation. When I say only himself, I mean directly, and in the first instance: for whatever affects himself, may affect others through himself; and the objection which may be grounded on this contingency, will receive consideration in the sequel. This, then, is the appropriate region of human liberty."
"It comprises, first, the inward domain of consciousness; demanding liberty of conscience, in the most comprehensive sense; liberty of thought and feeling; absolute freedom of opinion and sentiment on all subjects, practical or speculative, scientific, moral, or theological."
"The liberty of expressing and publishing opinions may seem to fall under a different principle, since it belongs to that part of the conduct of an individual which concerns other people; but, being almost of as much importance as the liberty of thought itself, and resting in great part on the same reasons, is practically inseparable from it."
(Also note that your original citation was also related to judgements based on actions that concern only the judged person: "Though doing no wrong to anyone, a person may so act as to compel us to judge him, and feel to him, as a fool, or as a being of an inferior order")