At one level, Twitter is just another communications platform. At another, it’s something more profound: a window into our collective health and well-being.
We’ve seen before how social networks can be used to predict flu outbreaks. Now comes a study from the University of Pennsylvania showing how Twitter can be used to forecast more serious problems, like heart attacks.
Researchers took a sample of 826 million tweets from June 2009 to March 2010 and analyzed the language for positive and negative words. They were trying to understand the psychological state of different neighborhoods, as hostility and depression have been associated with higher risks of certain heart diseases. Words like “fuck,” “asshole,” or “bitch” indicated a negative tone to a community, while words like “friends,” “opportunity,” or “dreams” were considered positive.
Remarkably, when the researchers compared the maps with actual results from the Centers for Disease Control and Prevention, the technique proved more predictive than more traditional methods, like analyzing smoking or socio-economic data.
“It’s fast, cheap, and high-resolution,” says Johannes Eichstaedt, the graduate student who led the work. “These other methods take months and they cost millions of dollars to complete.”
Twitter offers a level of psychological insight you don’t get from looking at pure statistics. “It measures a layer of community anger that classical predictors can’t get at,” Eichstaedt says.
The 826 million tweets were actually only about 10% of the total from that period, and the volume has since doubled. Eichstaedt says if he had access to everything, he could make even more accurate and granular predictions–perhaps down to individual ZIP codes.
Unfortunately, Twitter was more generous in releasing its data four years ago than it is now. Today, data scientists need to pay many thousands of dollars for access, which makes the network less useful as a public health tool.
Still, Eichstaedt says Twitter is a vital resource and he’s planning to test its predictive power for a host of other diseases including cancers and depression.
“You can’t learn much from a single individual. But when you have thousands and thousands of Twitter streams, you can triangulate the psychological condition of a community and begin to make meaningful inferences,” he says.