Like all great ideas, it started with a tweet.
Every morning I wake up and check if Trump is polling higher or lower than the movie Suicide Squad. It’s getting close. pic.twitter.com/f9O9NcuUPz— PJ Vogt (@PJVogt) August 11, 2017
In an era where everything is rated, from movies to restaurants to fidget spinners, there was something really amusing about comparing apples and oranges, as it were: this brunch spot is about as popular as that pair of headphones; this album is about as popular as that Airbnb host, etc. So, I set out to find a movie that is doing about as well as the president.
First, I had to figure out how well the president is actually doing. Picking a single one from the tens (maybe hundreds) of different polls, indexes, and surveys turned out to be an impossible task, so I chose the calculations of FiveThirtyEight, which seem to be well reasoned and attempt to account for all other major polls, with each of those scores weighted based on historical accuracy. FiveThirtyEight also updates their rating multiple times a day, which allows for more varied results, which in turn make for a more interesting web experience (since if you check the page at multiple times of the day, the results might differ).
Next, I tackled the problem of building up a database of movies with their corresponding Rotten Tomatoes ratings. The only problem is, there isn’t a publicly accessible API for this information. There is a private API which developers can apply to access (and I did), but almost 4 months after submitting the application, I still haven’t heard anything back. With further googling, I found two API’s that I combined to do the job: The Movie DB, which offers a free API with a
/discover endpoint that yields lists of popular movies, but doesn’t include the RT rating, and The Open Movie Database (OMDb), which doesn’t provide any sort of
/discover feature, but does include the RT rating for specific movies.
While FiveThirtyEight gives a rating to the tenth (e.g. 38.7%), RT deals in integers, so I knew I’d have to gather enough movies to cover each possible rating, 0% - 100. (In reality, there’s never going to be anywhere near total consensus, so it’s unlikely that some movies will be used, but hey, with this President you never know.) I also wanted to have enough for each rating that the results were varied, and you could come back to the site and get different results each time. I ended up building a database of over 1,800 movies, each with a title, year, RT rating, number of IMDB ratings, and poster image. I ignore movies from this year, since their rating may not be stable yet. Then, for each rating, I select the 20 movies with the most ratings on IMDB (even if a movie is ‘bad’, movies with the most reviews are more likely to be recognizable), and provide one at random that matches the president’s current rating (which is updated every hour).
While not publicly accessible, I created a way to manage movies in the DB which allowed me to remove movies from consideration, as well as merely to examine the huge list of movies I created. I decided to remove some movies from the DB that were either unknown, in poor taste, or just weren’t funny. Since my database was a huge JSON file, it was nearly impossible to view in a text editor, but viewing the top results via this interface turned out to be very useful.
I also made an interface for scraping, which allows you to specify the specific pages of the TMDb
/discover endpoint to scrape (useful if the app crashes, which of course it never did). Using
socket.io I streamed the server results in realtime to the client, so that you can follow along and see progress as the scraper runs, which is helpful as I let it run over the course of 10+ hours.
I learned a lot working on this project. I combined several disparate API’s (at least 4, by my count), as well as brushed up my scraping skills (most of it done with Google’s newish Puppeteer, which turned out to be a blast to use). After more than a month of working evenings and weekends, I launched the site (via tweet, of course), which, to my surprise, was retweeted by PJ, the originator of the initial idea.
I've spent the last few weekends working on a thing and I'm ready to be done with it, so here you go: https://t.co/XiRh1HsdaU 🎈— Preston Richey (@prestonrichey) September 25, 2017
All art direction for Rotten Trumpatoes was done by Tanya Karpitskiy.