Twitter users will be familiar with the term “Trending Topics”. The site displays phrases or hashtags that are being used in a high number of posts. While many may think that trending topics appear quickly, and often disappear just as quickly, this many not be the case. At the Interdisciplinary Workshop on Information and Decision in Social Networks at MIT in November, Associate Professor Devavrat Shah and his student, Stanislav Nikolov plan to present an algorithm that predict topics that will trend. Not only is this algorithm 95% accurate, it can also make predictions an average of one and a half hours before the topics trend – and sometimes up to five.
The algorithm starts by being “trained” – combing through sets of data in an attempt to find meaningful patterns. It then compares changes in the number of tweets about topics in the data set. The algorithm is built to split up it’s execution over several servers to adapt to the modern computational framework, so while the sample data sets have been small, it can easily be scaled up to examine more data.
While this algorithm could help Twitter charge more for targeted ads that revolve around trending topics, its potential use in other categories – ticket sales, or maybe even stock prices, is much more interesting.
What do you think this algorithm could be used to accurately predict?