September 2013

This past spring I was in a data mining class and I spent some time working with the IMDB database. It is freely available, and there are some nifty tools for working with it. The goal of my data mining project was to develop a decision tree or a rule based classifier that could be used to predict a movies average user rating based on its other attributes. So ideally, we would have a decision tree that said something like “if a movie is made by Quentin Tarantino and stars Brad Pitt, then it will have an average user rating of 9/10.” Well the decision tree turned out to be quite a bit more complicated and less insightful than we were hoping, but in the process of building it we did come across some interesting findings that this post is intended to summarize.

Note: the data set was filtered to only include movies that meet the following constraints:

• Produced in the US
• Genre isn’t ‘Short’
• Has at least 10 user ratings
• Released before 2013

Genres

When building a rule-based classifier, rules similar to this kept showing up:

if genre = 'Drama' then user_rating > 7
if genre = 'Horror' then user_rating < 5

This got us thinking that perhaps movies of certain genres just tend to be better. The following table seems to support that idea.

So it looks like either there are a lot of crappy horror movies out there, or people just don’t like horror movies as much as they like documentaries. It also so happens that documentaries are on average the cheapest genre of movie to produce, as the chart below shows.

Animation movies are also very popular, but their high cost is prohibitive. They make up only 1% of the US movies in the IMDB database:

MPAA ratings

One of the data attributes that lent itself really well to data mining was the MPAA rating attribute. This attribute looked something like:

Rated R for strong violence, language and some sexual content/nudity.

The reason these ratings worked so well for data mining was because the MPAA likes to use certain keywords in their ratings: language, drugs, sex, violence, and nudity. So the rating above could be broken down into to a series of booleans indicating that the movie contained violence, language, sex and nudity, but didn’t contain any drugs. I was kinda expecting to find some dark statistic like “movies with violence and drugs tend to be rated higher than those without,” but it turned out that MPAA ratings weren’t a very good predictor of user ratings. Actually, the presence of any of those keywords had a slight negative correlation with the user rating attribute! Maybe there’s hope yet for humanity.

So the MPAA ratings weren’t very useful for predicting user ratings, but they are still useful for looking at what kind of obscenities are filling our movies:

We can also look at rating distributions:

And how rating distributions have changed over the past 20 years:

I was surprised that the percentage of ‘R’ rated movies being produced has declined over the past 20 years. I wonder if the MPAA is just getting less sensitive or if Hollywood is really becoming more PG as the chart above suggests.

Top Directors, Actors, and Actresses

While MPAA ratings and genres are interesting to look at, the real predictors of how good a movie will be should be the people who made it. Here are some people who consistently make great movies:

Note: Only includes directors with at least 3 movies, and actors or actresses with at least 20 movies.

Well that gives a quick snapshot of some of our findings. It would be cool to be able to pinpoint the exact ingredients that make a great movie, but if that were possible then making movies would be more of a science than an art.