Although this is of course ridiculously wasteful right now, I can see this being the optimal solution for many things if a technology like thermodynamic well based neural networks get to the point of viability.
A thermodynamic well based model could have a trillion parameters in the size of an so card at a few milliwatts of power.
In case like that it’s easy to imagine that mass produced implementations could be a one size fits all solution for all but the most trivial or advanced computing tasks. For perhaps less than a dollar for a 100b sized chip, you get the ability to “imagine” video, sound, etc and a strong general purpose “reasoning” capability imbedded into everything right down to children’s toys and toasters.
Kinda makes me think of Rick and Morty with the butter passing robot. A lot of pointless capabilities, but still cheaper than a purpose built deterministic computing device. OTOH having embedded knowledge as an ambient part of everyday life would be kinda neat, even if it would almost surely mean the end of human civilization lol.