A typical workflow for a Palantir customer was that Palantir would come in and dump a ton of data out of old crufty databases and into Palantir's datastore. Then, they'd establish connections between that data. This is all sounds kind of hand-wavy, but the gist of it is that a lot of government agencies have data that lives in separate databases and they can't easily correlate data between those two databases. Once the data was in Palantir's system, they could do queries against all their data, and make connections and correlations that they wouldn't otherwise be able to find when the data was previously siloed.
One of the sample use cases was identifying people filling prescriptions for schedule II drugs multiple times on the same day, and correlating that with pharmacies run by people connected to known drug traffickers. Previously, this was hard to do because the database of prescription purchases was disconnected from the database of drug convictions.
In my experience, internal employees outside Data have a funny relationship with Data. They hate to manage it but they love to blame it, especially in analytical / decision-making scenarios. Teams that "own" the data usually get the blame, on top of having to deal with a mass of rotting pipes and noncompliant teams, while also losing out on credit when non-Data teams report big wins.
Based on what the GP says, it sounds like Palantir knows how to exploit common internal politics around Data. They build up technical & social expertise in ETL'ing disparate data sources, and they can avoid blame by being hired by executives as an external third party.
> Why is data integration so hard? The data is often in different formats that aren’t easily analyzed by computers – PDFs, notebooks, Excel files (my god, so many Excel files) and so on. But often what really gets in the way is organizational politics: a team, or group, controls a key data source, the reason for their existence is that they are the gatekeepers to that data source, and they typically justify their existence in a corporation by being the gatekeepers of that data source (and, often, providing analyses of that data). [3] This politics can be a formidable obstacle to overcome, and in some cases led to hilarious outcomes – you’d have a company buying an 8-12 week pilot, and we’d spend all 8-12 weeks just getting data access, and the final week scrambling to have something to demo.
I think he's seen more companies without talented Data experts than companies with that talent.
Because the ostensible product, at least in the ‘pilot’, produced in just a single week, seems like it is pretty much guaranteed to be bad.