Calypso: LLMs as Dungeon Masters' Assistants [pdf]

I would really like to see what a reasoning model with access to player character information, resources available in the location, and the monster manual could do. One of the hardest things as a DM, in my experience, is creating a balanced encounter without fudging. This has always made it hard for me to justify presenting a truly deadly encounter which I feel has lowered the stakes of the game. It seems like it should be possible to create a system that knows the strengths/weaknesses of a party and that could create a challenging but not overwhelming encounter most of the time.

Maybe it'd be useful the other way around: as a DM, let an AI play the PCs to test out your campaigns and dungeons. I wonder how good it would be at finding the weak/missing points that players do, and whether it would give you an idea whether the challenges are too hard or easy.