I replaced Animal Crossing's dialogue with a live LLM by hacking GameCube memory

It would make single player game so much more alive, even railroaded one like red dead redemption because the npc could adapt more easily to what you just did, how you behave in game and so on.

Games are already demanding on gpu, running a very tiny local LLM and only when people interact with npc wouldn’t require so much power.

I’m sure there are issues which explain why it’s not already being used but the first one to do it will be remembered.

The biggest problem I faced at the time (during ChatGPT 3 era) was that, without a good context, LLMs are the most vanilla roleplayers you’ve ever seen. By themselves, LLMs are just not interesting enough for a player to choose to talk to in-game.

If you want them to be “interesting” to talk to, you must provide (or generate and keep track of): a backstory, chat history, the scene, NPC inventory, the NPC’s current emotional state, the weather, literally everything needs to be given to the model before it generates messages for the player.

At which point you’ve got a big task. You need a way to automatically get the relevant data to the model for the specific conversation you’re having. There might be tools to pick appropriate text documents from a db given a conversation topic, but I didn’t/don’t know how to make that work for games.

I’m sure there’s a way to accomplish this with more modern tools and models. (Maybe instead of providing all that data up front, you would now give the model tools to call to retrieve that data on-demand?) But that’s what made me give up in 2022.