Blogger-statistician Nate Silver has become a bit of a crush object over the past year, what with his sterling electoral prediction record, his debunking prowess, and his encouragement of gambling-trained decision making.
So it's no wonder that when he gave a talk at SXSW, he had a roomful of admirers, to whom he gave a "somewhat uninspiring, but realistic, perspective on big data," as Erin Griffith reports for PandoDaily. The center of his argument: If you're looking for data-enabled successes, try a space with little currently going for it in the way of number-crunching.
Silver says he nabbed the low-hanging statisical fruit of baseball and politics because the competition wasn't great.
“Look for fields that have not been thought about in analytic way before and where you have data available,” he said. (Like, say, papal elections, but not so much something like finance, which has boatloads of quantitative analysis).
As the dating site boom of the past 14 years has shown, you can (somewhat) turn the qualitative parts of life into quantitative analysis—like all that startups trying to do the same for hiring, or any time "Moneyball for X" is excruciatingly invoked.
Overall, Silver took a skeptical tone. Humans take a long time to learn what to do with all the data we make, he says, noting that the printing press was around for three centuries before it produced mass benefits for society.
And now, with the web, the issue's magnified, as 90 percent of the data known to man was created in the past 10 years. That's a lot of information, but we still have little knowledge of how to use it. To get started with that task, read our interview with Silver from last fall.
[Image: Flickr user Caden Crawford]