Making the most detailed tweet map ever

By Eric Fischer on December 03 2014 (TOTH:

I’ve been tracking geotagged tweets from Twitter’s public API for the last three and a half years. There are about 10 million public geotagged tweets every day, which is about 120 per second, up from about 3 million a day when I first started watching. The accumulated history adds up to nearly three terabytes of compressed JSON and is growing by four gigabytes a day. And here is what those 6,341,973,478 tweets look like on a map, at any scale you want.

© Mapbox © OpenStreetMap Improve this map. Data from the Twitter Streaming API
© Mapbox © OpenStreetMap Improve this map. Data from the Twitter Streaming API

I’ve open sourced the tools I used to manipulate the data and did all the design work in Mapbox Studio. Here’s how you can make one like it yourself.

You can follow Twitter’s stream of geotagged public tweets using the “statuses/filter” API to request tweets from a particular bounding box or the whole world. Before you can connect, you have toregister a Twitter API key and authenticate using it. I couldn’t find a simple library last year to generate the OAuth header for Twitter authentication, so I wrote this one. Once you have authenticated and connected to the filter API, you receive a steady stream of tweets in JSON format. They include a lot more metadata than you necessarily need to make a dot map, so I’ve been using this program to parse the JSON and pull out just each tweet’s username, date, time, location, client, and text.  Click to continue reading.

Leave a Reply

Your email address will not be published. Required fields are marked *