Even though there are only two sex chromosomes, issues with them can cause people who are physically intersex.
Then there's all the complexity of the sense of self. Some people could feel they have the 'wrong' body and be fine with that. Some can't.
-Heterosexual male tired of oversimplifications
It is completely different if it is a physical disorder, but this is not the case in vast majority of cases.
Also: if a identity or an emotion is real to me, does it not become real through that act itself?