This is awesome. LLM-powered NPCs is one thing I'm most excited about in the future of gaming. Characters repeating the same scripted dialog over and over again is one of the biggest immersion breakers.
replies(13):
I wouldn't ever want a game to use it for the core story writing, because it's pretty important that it is consistent and unable to be derailed. But for less serious NPC interactions or like an RPG scenario it is such a great fit.
I also wouldn't want a single player game to rely on remote inference, because that will get turned off eventually and then your game doesn't work.
(Yes, this is a Paradox callout. Give me less fancy particle effects in Vic3 and use the GPU for computing pop updates faster!)
(Probably the biggest barrier to this is the lack of a convenient C++/C#-level cross-manufacturer compute API. Vulkan is a bit too low-level for game devs to work with, OpenCL kind of sucks, and CUDA is NVIDIA-only.)