Since we are rather analytical types, we wanted to see exactly how effective our personalization algorithm was. Last year Netflix made available the ratings data for their movies,  so we decided to see how good Thoof’s personalization algorithm was at anticipating what movies people would like.

Like any good experiment, we wanted some controls, so we also tried this with some simpler algorithms to see how our approach compared.

For each algorithm, we would use it to select movies to show to a user, and see what proportion of those movies the user actually liked.

The first algorithm was a naive approach of randomly selecting movies. In this case, the average user liked 10% of the movies they saw.

Next we tried a “most popular” approach, taking the most popular movies, and showing them to everyone. Amazingly, the average user only liked 15% of the movies that were deemed globally popular! Bear in mind that this is essentially the story selection algorithm used by today’s biggest user-generated news websites.

Finally, we tested our algorithm, and were very pleased to find that users liked 40% of the movies it selected. A pretty clear improvement over the other approaches.

Thoof will succeed or fail based on a user’s first impression when they see stories on the front page. Our hope is that using our personalization algorithm, combined with collaborative editing, the quality and relevance of these stories will immediately set us apart from other news and information sources.

- Ian