Foursquare Open Source Places: A new foundational dataset

I used to work for Nokia Maps (before it was renamed Here Maps) working on their places registry, which was a proprietary data set compiled from various sources of data with a similar amount of places. It was really messy data. Lots of duplicates, lots of stale data, lots of misleading data (e.g. hotels claiming to be right on the beach that in reality were a few kilometers away), etc. Part of my job was cleaning that data up and making sense of it. The registry was used to power Nokia's address and places search and became hugely popular when Nokia started bundling free maps and navigation with all their phones. Up until then you had to pay for in car navigation.

Foursquare at the time was an app for checking into places and becoming the mayor. Which was hugely popular with several of my colleagues that would obsessively checking to places to claim mayorship of places they liked. To ensure Foursquare had good content, they came up with a clever solution: just ask users that are there. This community driven curation was pretty smart. They got users to categorize and correct information.

Later when doing my first startup, I tried to work with open place data. It was a mess. There were bits and pieces available but nothing great. I actually crawled Foursquare and a few other websites in HTML form at some point to get at their ~30M places (at the time). In the end, we never launched anything and there were obvious IP issues with that data set that would have prevented us from using that commercially.

Anyway, Foursquare got all corporate and the whole checking in thing was spun off into a separate app and the main app became a glorified restaurant recommendation thingy which never really managed to stand out from the field of other recommendation thingies. And a data set + API you could license. Because that was still pretty good.

More recently, other companies partnered up to form Overture, which is a competing open data set that Bing, Amazon, and others are contributing to. Overture is kind of solving the same problem.

I'm guessing that's a big contributing reason for Foursquare to be open sourcing their data. It would be interesting to see if their data is going to be contributed to Overture or whether they are somehow against that. Last I checked, Overture still had lots of data issues. Combining the two data sets would probably help.

Dennis crowley who led that effort had a difficult time getting his marauder map like vision of the places+people going, not for lack of trying. He was at a NYC event describing his experience. First he tried dogeball which google bought and did not do much with it. He lefy and started FSQ. He mentioned that eventually FSQ pivoted to enterprise as the swarm/checkin like features were not making business sense, it was also copied by FB, Yelp which is when traffic moved away from FSQ's app to those. But FSQ always had a higher committment to location privacy and still does(remember cambridge analytica and all those NYT articles on location tracking)

Dennis is trying something new now, which is true(to him) and to the original vision of Marauder's map and FSQ. He has been on the dogeball train since a long time and hats off to his commitment.