The community reflects the larger society, which is divided on social issues. Don't forget that users come from many countries and regions. That's a hidden source of conflict, because people frequently misinterpret a conventional comment coming from a different region for an extreme comment coming from nearby.
The biggest factor, though, is that HN is a non-siloed site (https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sor...), meaning that everyone is in everyone's presence. This is uncommon in internet communities and it leads to a lot of misunderstanding.
(Edit: I mean internet communities of HN's size and scope, or larger. The problems are different at smaller size or narrower scope, but those aren't the problems we have.)
People on opposite sides of political/ideological/cultural/national divides tend to self-segregate on the internet, exchanging support with like-minded peers. When they get into conflicts with opponents, it's usually in a context where conflict is expected, e.g. a disagreeable tweet that one of their friends has already responded to. The HN community isn't like that—here we're all in the same boat, whether we like it or not. People frequently experience unwelcome shocks when they realize that other HN users—probably a lot of other users, if the topic is divisive—hold views hostile to their own. Suddenly a person whose views on (say) C++ you might enjoy reading and find knowledgeable, turns out to be a foe about something else—something more important.
This shock is in a way traumatic, if one can speak of trauma on the internet. Many readers bond with HN, come here every day and feel like it's 'their' community—their home, almost—and suddenly it turns out that their home has been invaded by hostile forces, spewing rhetoric that they're mostly insulated from in other places in their life. If they try to reply and defend the home front, they get nasty, forceful pushback that can be just as intelligent as the technical discussions, but now it feels like that intelligence is being used for evil. I know that sounds dramatic, but this really is how it feels, and it's a shock. We get emails from users who have been wounded by this and basically want to cry out: why is HN not what I thought it was?
Different internet communities grow from different initial conditions. Each one replicates in self-similar ways as it grows—Reddit factored into subreddits, Twitter and Facebook have their social graphs, and so on. HN's initial condition was to be a single community that is the same for everybody. That has its wonderful side and its horrible side. The horrible side is that there's no escaping each other: when it comes to divisive topics, we're a bunch of scorpions trapped in a single bottle.
This "non-siloed" nature of HN causes a deep misunderstanding. Because of the shock I mentioned—the shock of discovering that your neighbor is an enemy, someone whose views are hostile when you thought you were surrounded by peers—it can feel like HN is a worse community than the others. When I read what people write about HN on other sites, I frequently encounter narration of this experience. It isn't always framed that way, but if you understand the dynamic you will recognize it unmistakeably, and this is one key to understanding what people say about HN. If you read the profile the New Yorker published about HN last year, you'll find the author's own shock experience of HN encoded into that article. It's something of a miracle of openness and intelligence that she was able to get past that—the shock experience is that bad.
But this is a misunderstanding—it misses a more important truth. The remarkable thing about HN, when it comes to social issues, is not that ugly and offensive comments appear here, though they certainly do. Rather, it's that we're all able to stay in one room without destroying it. Because no other site is even trying to do this, HN seems unusually conflictual, when in reality it's unusually coexistent. Every other place broke into fragments long ago and would never dream of putting everyone together [1].
It's easy to miss, but the important thing about HN is that it remains a single community—one which somehow has managed to withstand the forces that blow the rest of the internet apart. I think that is a genuine social achievement. The conflicts are inevitable—they govern the internet. Just look at how people talk about, and to, each other on Twitter: it's vicious and emotionally violent. I spend my days on HN, and when I look into arguments on Twitter I feel sucker-punched and have to remember to breathe. What's not inevitable is people staying in the same room and somehow still managing to relate to each other, however partially. That actually happens on HN—probably because the site is focused on having other interesting things to talk about.
Unfortunately this social achievement of the HN community, that we manage to coexist in one room and still function despite vehemently disagreeing, ends up feeling like the opposite. Internet users are so unused to being in one big space together that we don't even notice when we are, and so it feels like the orange site sucks.
I'd like to reflect a more accurate picture of this community back to itself. What's actually happening on HN is the opposite of how it feels: what's happening is a rare opportunity to work out how to coexist despite divisions. Other places on the internet don't offer that opportunity because the silos prevent it. On HN we have no silos, so the only options are to modulate the pressure or explode.
HN, fractious and frustrating as it is, turns out to be an experiment in the practice of peace. The word 'peace' may sound like John Lennon's 'Imagine', but in reality peace is uncomfortable. Peace is managing to coexist despite provocation. It is the ability to bear the unpleasant manifestations of others, including on the internet. Peace is not so far from war. Because a non-siloed community brings warring parties together, it gives us an opportunity to become different.
I know it sounds strange and is grandiose to say, but if the above is true, then HN is a step closer to real peace than elsewhere on the internet that I'm aware of—which is the very thing that can make it seem like the opposite. The task facing this community is to move further into coexistence. Becoming conscious of this dynamic is probably a key, which is why I say it's time to reflect a more accurate picture of the HN community back to itself.
[1] Is there another internet community of HN's size (millions of users, 10-20k posts a day), where divisive topics routinely appear, that has managed to stay one whole community instead of ripping itself apart? If so, I'd love to know about it.