The era of open voice assistants

> explain why there's not a clear path to replace Alexa or Google home?

There is. I've used HA with their default assist pipeline (Cloud HA STT, Cloud HA LLM, Cloud HA TTS) and I've also plugged in different providers at each step (both remote and local for each part: STT/LLM/TTS) and it's super cool. Their default LLM isn't great but it works, plugging in OpenAI made it work way better. My local models weren't great in speed but I don't have hardware dedicated for this purpose (currently), seeing an entire local pipeline was amazing for the promise of it in the future. It's too slow (on my hardware) but we are so close to local models (SST/TTS could be improved as well but they are much easier to do already locally).

If this new HA hardware comes even close to performing as well as the Echo's in my house (low bar) I'll replace them all.

What does it use LLMs for?

Taking the text of what you said and figuring out what you want to do. It sends what you said plus a list of devices/states and a list of functions (to turn off/on, set temp, etc of devices). The LLM takes "Turn off basement lights" and turns that into "{function: "call_service", args: ['lights.on', 'entity-id-123']}" (<- Completely made up but it's something like that) that it passes back to HA along with what to say back to the user ("Lights turned off" or whatever) and HA will run the function and then do TTS to respond to you.