Why Tweeting Beats Tracking

Twitter researcher says his figures trump the more seasoned services on weekend grosses

Last Updated: July 27, 2010 @ 9:23 AM

Moviegoers tweeting reviews to friends may be more than just an annoyance to fellow theatergoers.

If researchers at HP Labs are correct, studios may soon be able to more accurately predict the weekend grosses of their upcoming films by combing through tweets.

Looking at more than 24 films over four months, HP's Palo Alto, Calif.-based techies used the frequency that a movie was mentioned in some 3 million tweets from nearly 2 million users to accurately project box office takes.

From blockbusters such as "Avatar" to bombs such as "Did You Hear About the Morgans?" the team at HP actually beat the predictions of more seasoned tracking systems like the Hollywood Stock Exchange.

To get the inside scoop on what these findings could mean for Hollywood, TheWrap checked in with HP researcher Bernardo Huberman (pictured below left), who co-authored the study with Sitaram Asur.

Why did you decide to focus on movies?
For the past three years we've been working on the economics of social attention. These days information is generated at a phenomenal rate, and consequently it's collapsed in value. We've become really interested in how social media like Twitter documents the public's attention. You could look at any number of things, such as the release of the iPad or the earthquakes in Haiti, to see how meaningful social networks form on Twitter.

We started to realize that we could measure people's attention along the lines of box office revenue, in terms of how many people prefer one movie to another. We could have done anything, but we chose movies. It was a way to suck data out of a thin straw.

How did your study compare to traditional box-office tracking?
In terms of our top predictions, we did better on average than the Hollywood Stock Exchange, and they're the gold standard for movie tracking. Our predictions were 97 percent accurate, and they were about 96 percent accurate.

Does the box-office disappointment of movies such as "Kick-Ass," which had great internet buzz, show the limitations of relying heavily on Twitter?
Everyone is asking us that. It would have been great to do that one, but unfortunately we weren't doing the tracking needed to make the predictions. I bet by looking at the sentiment analysis, and seeing what people were saying after opening night, we could have seen a steep drop-off in box office. If you looked at what happened with "New Moon" and "The Blind Side," you might see the same trend. There was a lot of activity on Twitter leading up to "New Moon," but the amount of negative feedback increased after the opening weekend when people saw the film. The opposite was the case with "Blind Side," which saw its second-weekend numbers increase.

How did you set up your study?
Basically we set up a computer program that measured the rate and speed with which an upcoming movie was discussed. We generally looked at movies a few weeks prior to their release. We were mining data from about 3 million tweets. We also measured the positive, negative or neutral reactions of people who saw a movie, which often translates into revenues for the film's second weekend.

Have studios reached out to you?
Yes, a lot of studios have reached out to us. I'm beginning to learn Hollywood speak. They all say how "awesome" the study and everything is. I know that HP plans to monetize our research, so they are discussing ways to turn this research into a product. I'm in the labs, so I don't get involved on the business end.

Are there certain kinds of movies that Twitter wouldn't be good to use for tracking?
Well, it's probably not good for measuring artsy movies. If you wanted to track people's reactions to something like modern art, you'd be better off going to a forum devoted to that than Twitter.

Does Twitter over-sample from men or women or older or younger users, and does that skew the data?
In terms of the demographics of Twitter, the majority of people are between 18 and 34 and it's pretty evenly split between men and women. That makes it a good place for tracking certain kinds of consumer behavior, if not philosophical ideas.

Is it effective for measuring international box office?
Twitter is an almost exclusively American phenomenon. Its popularity is spreading, though. In India it is a huge thing, and the Chinese have their own version, so in time that could happen. We have no idea where people in our study came from geographically.

What's next?
We want to build on this. We want to continue to examine how opinions form. It's amazing how Twitter can show how the public agenda is shaped. It used to be that the media determined the agenda for us — everybody saw the same four stories on the evening news. That's changing with more and more places creating content that aren't part of the establishment, such as YouTube, Digg and the Huffington Post.

When big institutions like studios see that Twitter can actually be predictive, they're recognizing that social media has real value.

These studies are important because we know 30 million people are tweeting, but evidence that it's important has largely been anecdotal.