Indeed bizarre as the statement doesn't say much about data collection or retention.
More generally, I'm conflicted here -- I'm big on personal privacy but the power & convenience that AI will bring will probably be too great to overcome. I'm hoping that powerful, locally-run AI models will become a mainstream alternative.
It kind of reminds me of how the internet used to be. Back then, you had to go to a specific room to use the family computer. The internet was something you visited. Now, tech is everywhere, from our pockets to our bathrooms. I’m not sure I want AI following that same path.
At some point, some reverse engineer will publish a writeup either confirming or denying how local these models are, how much data (and maybe even what data) is being sent up to the mothership, and how these integrations appear to be implemented.
It's not perfect, and it only offers a point-in-time view of the situation, but it's the best we can do in an intensely closed-source world. I'd be happier if these companies published the code (regardless of the license) and allowed users to test for build parity.
I can understand why: you’re only using locally-run AI models every so often (maybe a few times a day), but when you use it, you still want it to be fast.
So it will need to be a pretty heavy AI chip in your phone to be able to deliver that, which spends most of the time idling.
Since compute costs are insane for AI, it only makes sense to optimize this and do the inference in the cloud.
Maybe at some point local AI will be possible, but they’ll always be able to run much more powerful models in the cloud, because it makes much more sense from an economics point of view.
It's not clear to me why certain classes of things still end up farmed out to the cloud (such as this, or is it?). Maybe their LLM hasn't been built in a very pluggable fashion.
What is that power? Honest question...
The fundamental catch here is that 80%+ of the future benefit will likely come from the very thing that erodes privacy: deep integration and context. Imagine if a Gemini had your entire life in its context (haha scary I know!), prompting would be so much more powerful.
That's the core, uncomfortable trade-off we're all facing now.
It's hard to predict exactly though. I remember thinking in 2001 that nobody except the busiest businessmen would need a cell phone. A landline at home is perfectly enough and in special cases there are phone booths. And in 2011 I thought the same about smartphones. Why would I need email while walking in the street? Can't it wait until I'm home at the desktop? If I need computer stuff on the go, I can take a laptop. Similarly, I'm not quite sure how exactly it will go but probably in 10 years you'll need to have an AI agent to function in society. The legacy infrastructure decays if nobody uses it even if you'd prefer not to jump on the bandwagon. Today you often MUST have an app downloaded to do things, e.g. some museums require it, sometimes government services are much more tedious otherwise. Some restaurants only have a QR code and no physical menu. Often news items (from school, or local municipality) are only shared in social media. Etc. etc. I can easily imagine that there will be things you can't manually do in 2035, only by asking your AI agent to do it for you. And it will scan all your data to make sure that what you're doing is impeccable in intent and safety and permissibility (like an inverse captcha: you must be Gemini or another approved bot to do the action. As a human you have to jump a million hoops that maybe takes days of providing various details etc. And Gemini will be easy to spook and will be opinionated about whether you should really get to do that action or not.). And it will communicate behind your back with the AI of the other party to decide everything. Or who knows what. But it will be necessary to use.
... although, to be fair, they're negotiating with OpenAI to run the models in "secure enclaves", which should, assuming everything works right which is a huge assumption, keep Apple or anybody else from reaching inside and seeing what the model is "thinking about".
Windows Recall [1] is this for your PC activities (not yet fed to AI, but I see no reason to think it will stay this way). Meta is working on glasses to record the IRL part. But your phone is probably enough for most of it. Joining Zoom meetings with AI note takers is getting popular [2]. Not long until in-person meetings will have AI listening in from the phone mics, of course just to increase productivity and to summarize and remind you later. Convenience!
[1] https://support.microsoft.com/en-us/windows/retrace-your-ste... [2] https://news.ycombinator.com/item?id=44446916
Frankly, this sounds like a potential nightmare to me. Almost certainly, the big techCos that today use algorithmic "customer support" to randomly flag, ban and screw with users for completely opaque reasons and little recourse will try running all of the nice things you describe. It's very plausible that sooner or later it will become harder and harder to do any of those things by more conventional, atomic means, making you rely ever more on integrated connections between product/service providers and these ordering systems (it's "convenient", you see?) only to suddenly find yourself flagged, blocked, or banned for any number of idiotic, blandly brainless reasons and shut out of the most basic elements of your daily life.
That people would readily agree to sign up for such dependence on these opaque things is a mystery to me, and I hope a huge pushback against it happens at some point.
If you think I exaggerate, bear in mind how often such things already happen on a lesser scale in a world where having social media, a smartphone and accounts with services like Google's becomes necessary in some contexts. Also note how often someone or another finds themselves fucked when these existing dependencies suddenly get shut down because some bullshit algorithm supposedly said so.
It's also bad enough that payment systems and banking can be cut off to people who (having broken no law at all) hold some politically controversial, publicly activist discourse, or that banking and certain basic services can be withheld in some bizarre way because you live a life in which your residency or other life choices are outside the average. To have the same risks apply for ever more minor reasons across a huge swathe of just living your life is a monstrously insidious way of marginalizing and homogenizing social divergence.
I honestly thought tapping screen to pick a pizza and all toppings you want etc is the convenient option but OK. Maybe it's a difference between people like me who like to browse menu and people who like to inquire with the waiter.
Booking a flight is a challenge (starting with captchas and hidden options they require opt out) for a living person already and expecting an LLM to do it is very far fetched. Planning a road trip, that's just robbing yourself of fun.
But because I asked & you answered I should say "thanks not for me". Maybe if I am Elon Msuk at some point and don't care about it falling for some dark pattern than minuses me $1k by accident I will be fine using assistants but then do I choose bots over living people in that scenario? Doubtful...