1. No network latency, you do not have to send anything across the atlantic.
2. Your get privacy.
3. Its free, you do not need to pay any SaaS business.
An additional would be, the scale being built-in. Every person has their own setup. One central agency doesn't have to take care of all.
1. A lot of good models require an amount of VRAM that is only present in data center GPUs.
2. For models which can run locally (Flux, etc.), you get dramatically different performance between top of line cards and older GPUs. Then you have to serve different models with different sampling techniques to different hardware classes.
3. GPU hardware is expensive and most consumers don't have GPUs. You'll severely limit your TAM if you require a GPU.
4. Mac support is horrible, which alienates half of your potential customers.
It's best to follow the Cursor model where the data center is a necessary evil and the local software is an adapter and visualizer of the local file system.
Your TV likely has a good enough CPU to run a decent model for home automation. And a game console most definitely does.
I'd love to see a protocol that would allow devices to upload a model to a computer and then let it sleep until a command it received. Current AI models are really self-contained, they don't need complicated infrastructure to run them.