One big cost I can see right away is environmental issues and water shortages in areas where AI data centers would run. No one is bothering about it now.
Getting better models at the edge would help in this to some extent as it will decentralise the runtime of AI (still model training would happen in data centers)