Twitter has some very interesting plans for its newly-unveiled live search function: soon it will activate crawlers that will index the links users include in their Tweets. In one fell swoop that turns Twitter into an even more powerful news and opinion aggregator. Look and learn, Google.
For example: If you want to learn about swine flu, you’d search Twitter and turn up not only a list of opinions, facts, and news that people are Tweeting, but also information from the sites they linked to in their tweets–in this case it may well be other news sources, medical reference texts and so on. Similarly if someone tweeted a link to their blog post, you’d find data from that post appear in Twitter’s search results also.
One thing Jayaran didn’t disclose was exactly which content will be covered by the crawlers. If images or movies files were sampled and then appeared in the search fields alongside plain text, that would totally transform Twitter’s search–and it’s popular support sites like Twitpic–and in some cases making it way more potent than Google.
The danger of searching Twitter, of course, is that Twitter is as much a source of meaningless, trivial or incorrect data as interesting, valuable, real information–the inclusion of the extra info from external sites should help moderate that problem somewhat, helping you judge the “quality” of results turning up in the searches.
Jayaran has further plans on that front as well, and it involves a sort of filtering by reputation of Twitterers. Though this has a hint of centrally-biased censorship, the company is still researching exactly how to quantify a Twitterer’s reputation–it certainly won’t be based solely on the number of followers they have. In the future, the reputation-measuring algorithm is going to have to be pretty carefully monitored though–it’s likely to have a big impact on how Twitter’s search evolves, and Twitter’s status as a live data source is probably the most valuable asset the company has.