In my opinion, training models on user data without their real consent (real consent = e.g. the user must sign a contract or so, so he's definitely aware), should be considered a serious criminal offense.
replies(5):
AI companies will get bailed out like the auto industry was - they won't be hurt at all.