We need to teach our students that the employment they take doesn't exist in a vacuum. Your choice of employee can impact not only yourself but the wider world. There's more to life than intellectual satisfaction.
Grey areas. By this I mean I mean ‘involve morally thorny, difficult decisions’: examples include health insurance, immigration enforcement, oil companies, the military, spy agencies, police/crime, and so on.
Every engineer faces a choice: you can work on things like Google search or the Facebook news feed, all of which seem like marginally good things and basically fall into category 1. You can also go work on category 2 things like GiveDirectly or OpenPhilanthropy or whatever.
The critical case against Palantir seemed to be something like “you shouldn’t work on category 3 things, because sometimes this involves making morally bad decisions”. An example was immigration enforcement during 2016-2020, aspects of which many people were uncomfortable with.
But it seems to me that ignoring category 3 entirely, and just disengaging with it, is also an abdication of responsibility. Institutions in category 3 need to exist. The USA is defended by people with guns. The police have to enforce crime, and - in my experience - even people who are morally uncomfortable with some aspects of policing are quick to call the police if their own home has been robbed. Oil companies have to provide energy. Health insurers have to make difficult decisions all the time. Yes, there are unsavory aspects to all of these things. But do we just disengage from all of these institutions entirely, and let them sort themselves out?
I don’t believe there is a clear answer to whether you should work with category 3 customers; it’s a case by case thing. Palantir’s answer to this is something like “we will work with most category 3 organizations, unless they’re clearly bad, and we’ll trust the democratic process to get them trending in a good direction over time”. Thus:
On the ICE question, they disengaged from ERO (Enforcement and Removal Operations) during the Trump era, while continuing to work with HSI (Homeland Security Investigations).
They did work with most other category 3 organizations, on the argument that they’re mostly doing good in the world, even though it’s easy to point to bad things they did as well.
I can’t speak to specific details here, but Palantir software is partly responsible for stopping multiple terror attacks. I believe this fact alone vindicates this stance.
This is an uncomfortable stance for many, precisely because you’re not guaranteed to be doing 100% good at all times. You’re at the mercy of history, in some ways, and you’re betting that (a) more good is being done than bad (b) being in the room is better than not. This was good enough for me. Others preferred to go elsewhere.
The danger of this stance, of course, is that it becomes a fully general argument for doing whatever the power structure wants. You are just amplifying existing processes. This is where the ‘case by case’ comes in: there’s no general answer, you have to be specific. For my own part, I spent most of my time there working on healthcare and bio stuff, and I feel good about my contributions. I’m betting the people who stopped the terror attacks feel good about theirs, too. Or the people who distributed medicines during the pandemic.
Even though the tide has shifted and working on these ‘thorny’ areas is now trendy, these remain relevant questions for technologists. AI is a good example – many people are uncomfortable with some of the consequences of deploying AI. Maybe AI gets used for hacking; maybe deepfakes make the world worse in all these ways; maybe it causes job losses. But there are also major benefits to AI (Dario Amodei articulates some of these well in a recent essay).
As with Palantir, working on AI probably isn’t 100% morally good, nor is it 100% evil. Not engaging with it – or calling for a pause/stop, which is a fantasy – is unlikely to be the best stance. Even if you don’t work at OpenAI or Anthropic, if you’re someone who could plausibly work in AI-related issues, you probably want to do so in some way. There are easy cases: build evals, work on alignment, work on societal resilience. But my claim here is that the grey area is worth engaging in too: work on government AI policy. Deploy AI into areas like healthcare. Sure, it’ll be difficult. Plunge in.8
When I think about the most influential people in AI today, they are almost all people in the room - whether at an AI lab, in government, or at an influential think tank. I’d rather be one of those than one of the pontificators. Sure, it’ll involve difficult decisions. But it’s better to be in the room when things happen, even if you later have to leave and sound the alarm.
However, I cannot more strongly disagree with your implicit assumption of innocence for "category 1." Facebook alone is unquestionably more harmful than Palatir, and any purely for profit entity is by necessity intentionally unanchored to any ethical foundation at all. Facebook is known for explicitly supporting genocidal regimes abroad, and for intentionally ignoring white supremacy, child abuse and domestic terrorism here in the US, all while being very explicit about not cooperating with the government agencies responsible for combatting these issues.
To that end, I would extend your thesis to the effect that people who eschew category 3 for category 1 aren't simply abdicating social responsibility, but are hypocritically engaged in substantially more socially harmful behaviors.
Sure, Palatir leads to people dying, and sometimes those people are innocent bystanders, but those actions are the result of any engagement with the public sector. Facebook is a direct progenitor of genocide abroad and fascism stateside, and is wholly untethered from either conscience or consequence. Category 1 is worse.