Tweetbeat Filters Spam From World Cup Tweets



Twitter‘s growing very, very fast. The number of ways of using tweets is rising fast, too, and with the explosion of messages, a new problem: Tweet spam. Kosmix’s Tweetbeat may be the answer, and it’s being tested with World Cup statuses.

Twitter’s very nature of an open one-to-many information broadcasting system is what makes it so very powerful–individual items of interest can very quickly be shared and spread around significant numbers. But it’s becoming tricky to “farm” Twitter to find out interesting new stuff being shared in status updates. Searching for a particular text phrase can often turn up all sorts of trashy or merely irrelevant results, and rampant abuse of the hashtag meme isn’t helping with the problem.

Hence Kosmix’s new Tweetbeat system is potentially fabulous. It uses an algorithmic approach to filtering Twitter’s firehose of 65 million daily tweets for you, assigning complex scores to individual tweets to determine if they are part of a real trending topic or not. The scores are based on a semantic analysis of a tweet’s contents, the influence value of the person tweeting it, and things like the velocity of a particular trend (how quickly its popularity is growing). The upshot is that Tweetbeat will let you very quickly access what may be the most interesting, or relevant tweets about hot topics.

To test it out prior to a full roll-out, Kosmix has built a special World Cup front-end, much more sophisticated than Twitter’s own effort, that demonstrates the tech based on the overall World Cup tweet stream, or on an individual nation basis. Compared to merely searching using Twitter’s own search engine, or clicking on Twitter’s trending “World Cup” topic links, it’s a more considered and interesting stream of statuses.

Whether or not the technology has equally as impressive results when it’s used against more esoteric tweet topics on a day-to-day basis after its proper roll-out we can’t tell yet. But as Twitter’s just reporting that it now has 125 million registered users, up 25% from just two months ago, and that 2 billion tweets are being sent every month, it’s obvious that technologies like Tweetbeat are going to become ever more useful for people who use Twitter in a more-than-casual way. And that raises one final thought: With Twitter’s increasing trend at taking third-party tech like this inside the company, will we see Tweetbeat bought by Biz Stone‘s company, or will it merely develop a competing system in-house?

To keep up with this news follow me, Kit Eaton, on Twitter. That QR code on the left will take your smartphone to my Twitter feed too.

About the author

I'm covering the science/tech/generally-exciting-and-innovative beat for Fast Company. Follow me on Twitter, or Google+ and you'll hear tons of interesting stuff, I promise.