1. No network latency, you do not have to send anything across the atlantic.
2. Your get privacy.
3. Its free, you do not need to pay any SaaS business.
An additional would be, the scale being built-in. Every person has their own setup. One central agency doesn't have to take care of all.
1. A lot of good models require an amount of VRAM that is only present in data center GPUs.
2. For models which can run locally (Flux, etc.), you get dramatically different performance between top of line cards and older GPUs. Then you have to serve different models with different sampling techniques to different hardware classes.
3. GPU hardware is expensive and most consumers don't have GPUs. You'll severely limit your TAM if you require a GPU.
4. Mac support is horrible, which alienates half of your potential customers.
It's best to follow the Cursor model where the data center is a necessary evil and the local software is an adapter and visualizer of the local file system.
These are two entirely separate paradigms. In many instances it is quite literally impossible to depend on models reachable by RF like in an ultra-low power forest mesh scenario for example.
And related, if there is no service, then the service cannot fail to secure your data.
No possibility of a billing or login error, where you lose access to your stuff because they just think your're not current or valid when you are.
No possibility of losing access because the internet is down or your current location is geo blocked etc.
No possibility of your account being killed because they scanned the data and decided it contained child porn or pirated movies or software or cad files to make guns or hate speech etc.
Those overlap with free and privacy but the seperate point is not the money or the privacy but the fact that someone else can kill your stuff at any time without warning or recourse.
And someone else can lose your stuff, either directly by having their own servers broken into, or indirectly, by your login credentials getting leaked on your end.
Then sure.
Some people have been running Office 2003 on everything from Windows XP to Windows 10. Assuming they bought the license 22 years ago, that's pretty cheap, probably $15 per year. As a bonus, they've never had their workflow disrupted.
But for consumer software that can be internet connected, data center GPU is dominating local edge compute. That's simply because the models are being designed to utilize a lot of VRAM.
Your TV likely has a good enough CPU to run a decent model for home automation. And a game console most definitely does.
I'd love to see a protocol that would allow devices to upload a model to a computer and then let it sleep until a command it received. Current AI models are really self-contained, they don't need complicated infrastructure to run them.
The fact that a hard drive can break and you can fail to have a backup is not remotely in the same class of problem of living at the whim of a service provider.
While the modern world of mobile devices and near-permanent fast and reliable connectivity has brought some real advantages, it has also brought the ability for software developers to ruthlessly exploit their users in ways no-one would have dreamt of 20 or 30 years ago. Often these are pitched as if they are for the user’s benefit — a UI “enhancement” here, an “improved” feature there, a bit of casual spying “to help us improve our software and share only with carefully selected partners”, a subscription model that “avoids the big up-front cost everyone used to pay” (or some questionable logic about “CAPEX vs OPEX” for business software), our startup has been bought by a competitor but all the customers who chose our product specifically to avoid that competitor’s inferior alternative have nothing to worry about because they have no ulterior motive and will continue developing it just the way we have so far.
The truth we all know but don’t want to talk about is that many of the modern trends in software have been widely adopted because they make things easier and/or more profitable for software developers at the direct expense of the user’s experience and/or bank account.
"Not on the trunk" isn't a sufficient threshold for me nor is "not the cloud" a good filter.