AI should only run as fast as we can catch up

Make it do TDD. That'll slow it down.

Make it do scrum with sprint planning, retrospectives and sprint demos. A then another AI as product owner and scrum master. Ideally this AI has only a vague idea of what the product needs to or the technology but still has decision power. That should really slow it down.

This was done already, they're called "Agent networks" or whatever buzzword the dev decided to give their abomination :)