The scenario that concerns me is that Claude learns unpublished research ideas from me as we chat and code. Claude then suggests these same ideas to someone else, who legitimately believes this is now their work.
Clearly commercial accounts use AI to assist in developing intellectual product, and privacy is mandatory. The same can apply to individuals.
More to the point, respecting your wishes to keep those conversations confidential would risk stifling human progress, so they have to be disregarded for the greater good.
LLM enthusiasts are staunch defenders of the argument that use of everyone's ideas and labour in LLM training isn't just fair use, but a moral imperative in order to advance science, art, and human progress as a whole.
It's beyond hypocritical for beneficiaries of this paradigm to then turn around and expect special treatment by demanding that "their" ideas, "their" knowledge, "their" labour be excluded from this process.
Because they're a computer program and not a human and humans are special.
Why are humans special? Because we're humans and we make the rules.
Its as inane as saying "why can I eat a burger but I can't chop up my friend and eat him? Why is that any different?"