Netflix had a really good idea: to reach out and invite anyone to improve its movie recommendation process – with a $1 million prize at the other end.
But, whoops, turns out that wasn’t such a hot idea after all. The contest drew an investigation by the Federal Trade Commission and a lawsuit.
The newest round of the contest just got cancelled.
In a blog post on Friday, Netflix chief product officer Neil Hunt wrote:
"In the past few months, the Federal Trade Commission (FTC) asked us how a Netflix Prize sequel might affect Netflix members’ privacy, and a lawsuit was filed by KamberLaw LLC pertaining to the sequel. With both the FTC and the plaintiffs’ lawyers, we’ve had very productive discussions centered on our commitment to protecting our members’ privacy.
"We have reached an understanding with the FTC and have settled the lawsuit with plaintiffs. The resolution to both matters involves certain parameters for how we use Netflix data in any future research programs.
"In light of all this, we have decided to not pursue the Netflix Prize sequel that we announced on August 6, 2009."
It turns out that in releasing data to contestants that Netflix thought had been made anonymous, two clever computer scientists from the University of Texas were able to identify Netflix customers. (Steve Lohr at the NYT found this one.)
They wrote an entire report about it – showing how they cross-correlated the Netflix list with IMDB.com and were able to identify Netflix users, and their political preferences. From the abstract:
"We apply our de-anonymization methodology to the Netflix Prize dataset, which contains anonymous movie ratings of 500,000 subscribers of Netflix, the world’s largest online movie rental service. We demonstrate that an adversary who knows only a little bit about an individual subscriber can easily identify this subscriber’s record in the dataset. Using the Internet Movie Database as the source of background knowledge, we successfully identified the Netflix records of known users, uncovering their apparent political preferences and other potentially sensitive information." (emphasis mine)
One question I have is that the report by Arvind Narayanan and Vitaly Shmatikov (who Netflix really ought to hire) is two years old – dated to February 5, 2008. One wonders why it took Netflix this long to respond to the privacy issues, given that their customer base is the core of their entire business.
Meantime, the commenters on Hunt’s post seem unanimously bummed at the loss of the prize, potential improvement by open-sourcing the recommendation process and general government interference.
Let’s hope the first and last winners, a team from the University of Texas named last September, enjoy their prize money.
