I am getting more and more convinced that CI and CD should be completely separate environments. Compromise of CI should not lead to token leaks related to CD.
This is essentially how separation of duties(and concerns) looks like. And this is how some of the good examples of projects work. Specific techniques and tooling and specific boundaries of CI and CD vary depending on the nature of the end product but conceptually you are absolutely right.