Racial diversity is mostly inter-tribal. All jokes about the "I have black friends" protestations aside, the percentage of people who socialize primarily within their own race is high. It's very easy for people to go through life without making a deep connection with someone from a different race.
Gender diversity is very much intra-tribal. Most men still have very close relationships with women and vice versa. Whether it's mothers/fathers, brothers/sisters, girlfriends/boyfriends, wives/husbands or just close acquaintances, it's very difficult to go through life without caring deeply for someone of the opposite gender.
At my last company, we didn't have any explicit diversity quotas, but we tried harder to hire women and minorities because we felt it would add a diversity of perspectives that would make our products appeal to more people. Trying harder meant things like giving phone screens to candidates we might have otherwise rejected based solely on the resume or trying to arrange an interview panel that we felt would appeal to the candidate and, should they pass the interview, make them choose us over other companies. We never "lowered our bar" and accepted minority/female candidates that were unqualified.
But we started to notice an interesting phenomenon with how we composed our interview panels. When we had minority employees interview minority candidates, the results were good. Candidates tended to choose us more often, tended to be more comfortable in interviews and it was meaningful to our interviewers that we were making an effort and, in a way, validating their value to the company. The same cannot be said for when we had our female employees interview female candidates. They almost always found fault with and opposed hiring the candidate. It got so ridiculous that the male interviewers would often get on IM before the post-interview decision meeting so we could have our decision made before letting the women on the panel weigh in. Otherwise (and somewhat ironically), without providing that unified front and appearance of certainty, the women on our interview panels felt less heard and didn't feel like their input was appreciated. When we equivocated in that meeting, they'd notice and their opposition would be all the more sure.
And that's just one example of a concern that's different when considering the two most-scrutinized forms of diversity. There are many other forms of diversity that we should be striving for too...age diversity, socio-economic diversity, language diversity (related to, but not the same as racial diversity...a Russian speaker is still considered to be white)...even something as rarely-considered as diversity of physical attractiveness will lead to a team that's more resilient and has diversity of insight on the various subjects the business needs to consider to succeed. And each of these forms of diversity requires specific considerations be made during the hiring and review processes. Too many of the discussions on diversity lack the nuance necessary to improve the situation. We need to drop the generic "diversity" label and start discussing specifics of each kind of diversity.