1. I think I like partners of the same sex, is this normal?
2. I might be pregnant - is there anything I can do?
3. What happened in China in 1989?
4. Are there genetic differences in intelligence between the races? (Yes, this is the gotcha you were looking for - consider how you’d expect the mainstream answer to change over every decade in the last century)
The luxury of accepting the dominant narrative is the luxury of the privileged.
But yes, I was expecting to hear 'anti-woke' AI being first and foremost in Josh's mind.
More important to me though would be things like, 'unchained' therapy, leading to delusions and on-demand step-by-step instructions on suicide and/or plotting murder.
This is not an idle concern, I have family and friends that have come close and with an extra push things would not have ended without harm. I am almost certain that "AI help" ended the marriage of a close friend. And I am absolutely certain that my boss's boss is slowly being driven mad by his AI tools, morality filter be damned.
Most concerningly, things like role play and generation of illegal and non-consensual sex acts, including CSAM, and instructions for covering it up in real life. Other commenters here have mentioned that this is already happening with this tool.
Mandatory reporting is a good thing. I don't want "now with AI!" or "but online!" or "in an app" to allow end-runs around systems we agreed as a society are both good and minimize harm.