Indeed bizarre as the statement doesn't say much about data collection or retention.
More generally, I'm conflicted here -- I'm big on personal privacy but the power & convenience that AI will bring will probably be too great to overcome. I'm hoping that powerful, locally-run AI models will become a mainstream alternative.
What is that power? Honest question...
It's hard to predict exactly though. I remember thinking in 2001 that nobody except the busiest businessmen would need a cell phone. A landline at home is perfectly enough and in special cases there are phone booths. And in 2011 I thought the same about smartphones. Why would I need email while walking in the street? Can't it wait until I'm home at the desktop? If I need computer stuff on the go, I can take a laptop. Similarly, I'm not quite sure how exactly it will go but probably in 10 years you'll need to have an AI agent to function in society. The legacy infrastructure decays if nobody uses it even if you'd prefer not to jump on the bandwagon. Today you often MUST have an app downloaded to do things, e.g. some museums require it, sometimes government services are much more tedious otherwise. Some restaurants only have a QR code and no physical menu. Often news items (from school, or local municipality) are only shared in social media. Etc. etc. I can easily imagine that there will be things you can't manually do in 2035, only by asking your AI agent to do it for you. And it will scan all your data to make sure that what you're doing is impeccable in intent and safety and permissibility (like an inverse captcha: you must be Gemini or another approved bot to do the action. As a human you have to jump a million hoops that maybe takes days of providing various details etc. And Gemini will be easy to spook and will be opinionated about whether you should really get to do that action or not.). And it will communicate behind your back with the AI of the other party to decide everything. Or who knows what. But it will be necessary to use.
Frankly, this sounds like a potential nightmare to me. Almost certainly, the big techCos that today use algorithmic "customer support" to randomly flag, ban and screw with users for completely opaque reasons and little recourse will try running all of the nice things you describe. It's very plausible that sooner or later it will become harder and harder to do any of those things by more conventional, atomic means, making you rely ever more on integrated connections between product/service providers and these ordering systems (it's "convenient", you see?) only to suddenly find yourself flagged, blocked, or banned for any number of idiotic, blandly brainless reasons and shut out of the most basic elements of your daily life.
That people would readily agree to sign up for such dependence on these opaque things is a mystery to me, and I hope a huge pushback against it happens at some point.
If you think I exaggerate, bear in mind how often such things already happen on a lesser scale in a world where having social media, a smartphone and accounts with services like Google's becomes necessary in some contexts. Also note how often someone or another finds themselves fucked when these existing dependencies suddenly get shut down because some bullshit algorithm supposedly said so.
It's also bad enough that payment systems and banking can be cut off to people who (having broken no law at all) hold some politically controversial, publicly activist discourse, or that banking and certain basic services can be withheld in some bizarre way because you live a life in which your residency or other life choices are outside the average. To have the same risks apply for ever more minor reasons across a huge swathe of just living your life is a monstrously insidious way of marginalizing and homogenizing social divergence.
I honestly thought tapping screen to pick a pizza and all toppings you want etc is the convenient option but OK. Maybe it's a difference between people like me who like to browse menu and people who like to inquire with the waiter.
Booking a flight is a challenge (starting with captchas and hidden options they require opt out) for a living person already and expecting an LLM to do it is very far fetched. Planning a road trip, that's just robbing yourself of fun.
But because I asked & you answered I should say "thanks not for me". Maybe if I am Elon Msuk at some point and don't care about it falling for some dark pattern than minuses me $1k by accident I will be fine using assistants but then do I choose bots over living people in that scenario? Doubtful...