Abstract

Social Media {eg Twitter, Blogs, Forums, FaceBook} has exploded over the last few years. FaceBook is now the most visited site on the Web, with Blogger being the 7th and Twitter the 13th. These sites contain the aggregated beliefs and opinions of millions of people on an epic range of topics, and in a large number of languages. Twitter in particular is an example of a massive message stream and finding events embedded in it poses hard engineering challenges. I will explain how we use a variant of Locality Sensitive Hashing to find new stories as they break. The approach scales well, easily dealing with the more than 1 million Tweets a day we process and only needing a single processor. For June 2009, the fastest growing stories all concerned deaths of one kind or another.

Bio: Miles Osborne is a Reader in Informatics, Edinburgh. His long-standing reseach interests include machine translation and algorithms for dealing with massive datasets. More recently his interests have extended to Social Media.