Time To Build Your Big-Data Muscles

As data grows fast in volume, variety, and velocity, our capacity to consume, analyze, make meaning and act on data lags far behind.

Time To Build Your Big-Data Muscles

What if I told you that for every 100 open Big Data jobs, there are only two qualified candidates? I’d like to tell you that. I heard it from a senior leader grumbling about finding adequate prospects. When I asked for the data, he couldn’t find the source and didn’t have quite that many open reqs (though he had many). 


In this data deluge era, we’ve grown accustomed to people almost having data to support their assertions. Telling you that the 100:2 ratio feels right doesn’t hold the same weight as evidence in hand. If you had to make a sound decision, you might search for something more.

McKinsey Global Institute gauges that by 2018, the United States will create 290,000 to 340,000 new big data jobs and 140,000 to 190,000 (more than half) could go unfilled because skilled candidates are in short supply. Add to that a need for at least 4 million managers across all fields with the know-how to decipher and act on the patterns in big data to make effective choices.

Every senior leader dreams of the day he or she can analyze a pile of data so big and harnessed so well that it becomes possible to make substantially better predictions.

In IBM’s 2011 Chief Information Officer survey of 3,000 global companies, more than 83% of respondents identified business analytics from big data as a top priority and a way in which they plan to enhance their competitiveness.

The data comes from everywhere: transaction records of online purchases, physical plant sensors, rules and regulations, posts conveying sentiment across social networks, uploaded digital pictures and videos, company and industry research reports…the list goes on. To put that in context, 15 out of 17 industry sectors in the U.S. (such as manufacturing, insurance, healthcare, utilities) have more data stored per company than the entire U.S. Library of Congress.

With data growing in volume, variety and velocity at massive rates (by some estimates, increasing another 44x by 2020), technology advances aim to extract deeper insights. Beyond the 3 Vs, our human capacity to consume, analyze, make meaning, and act on the data lags far behind. 


Big data creates new professional categories, job opportunities, and careers. Yet few universities are paying serious attention to the analytic skills needed by graduates in all fields. Academic programs need to provide opportunities for college and graduate students to practice and hone their analytics abilities and understand what big data means across disciplines. 

Terri Griffith, professor of management at Santa Clara University and author of The Plugged-In Manager, sees an opportunity for students to gain a market advantage when starting their careers. For every call Griffith gets from recruiters looking for the school’s best and brightest, she hears from three specifically asking about analytics capabilities.

Introducing big data approaches proves challenging because colleges are as siloed as most companies. Analysis and application usually reside in different schools and programs. Computer science students might learn about predictive analytics and Hadoop, while marketing students may never even hear the terms. “What is marketing today if not an opportunity to apply linear regression on the big data that comes out of experimental design? You learn from that data which source signals the type of attention you seek,” says Griffith.

Tim Sae Koo, cofounder of Hypemarks, a new social discovery aggregator, was challenged in his USC Management of New Enterprises course to think big. He took that to mean crossing fields of study. Sae Koo visited the computer science department and asked what trend the professors thought would eclipse all others in years ahead. Several said big data. As a result, Sae Koo made what he calls a “Big Data play, capturing what people are interested in by aggregating what they content consume and collect on the web. From the data, we can gauge what type of content has your attention and what you’re committing time to learn more about.” Quoting Wayne Gretzky, who said he was so good because he skated to where the puck would be, not where it had been, Tim believes that’s big data and people that it touches. 

Yale’s Ravi Dhar, Professor of Management and Marketing, and director of the Yale School of Management’s Center for Customer Insights, arrives someplace similar. When speaking with a senior executive about the leaders of tomorrow, they have always expected communication and curiosity. Now they add, “handling data.” They don’t have people in traditional functional areas who are good at all three. Dhar points out, “In marketing, people are great at crafting tag lines, branding, and launching campaigns. Now they need to have a deep understanding of customer too. That requires connecting all the dots.” 

Likewise, the people who have historically looked at data weren’t good in traditional lines of business–or conveying that to other people in actionable ways. “There is a disconnect between the ability to collect data and the ability to base decisions on them,” says Eric Bradlow, professor of marketing at the University of Pennsylvania’s Wharton School and co-director of the Wharton Customer Analytics Initiative. “People need to take a deep breath. They need to be more thoughtful about it.” Data will not answer questions by itself. People need to be able to communicate effectively about the findings, linking analytics to key decisions and the bottom line. 


As part of IBM’s academic initiative, Dhar’s Center for Customer Insights runs a 10-15 hour course spanning 12 weeks where students work closely with corporate management teams, answering hard questions. Students get exposure to real-world clients and the ability to experiment and apply new technologies to business challenges. IBM is working with more than 200 academic organizations around the world specifically in the area of expanding and strengthening analytics curricula, merging IT and business skills.

This approach is far more useful for students’ long-term success than market intelligence or market research courses that look at only quantitative 7-point scales and “how much more do you like” type questions, says Dhar. The vast amounts of sentiment data available today capture more naturally how people talk about their preferences, whether it’s on an Amazon review or through their personal activity feed. Facebook and Twitter alone generate 17 terabytes of data every day. The analytics skills of today and tomorrow can focus on the right methods for analyzing people’s priorities, not just those easiest to gather.

Griffith encourages students majoring in math, engineering, or computer science to answer strategic questions. She reminds them, “You’re an expert in analysis itself. Now get your head around the social science. Ask yourself, ‘How might this predict human behavior?’ “I don’t want to teach red ball, blue ball,” she says, referring to a probabilities question often asked in business schools. “I want to talk about the types of real problems business leaders face.” She encourages students to flex their muscles. No matter their major, they ought to consider how available data can serve evidence-based management.  

Think you’re too old or too busy to return to school to build and flex big data muscles? Look at the strengths you have, born from study and experience. Consider how they should adapt to what business needs. 

If the idea of big data and analytics scare you, read Moneyball by Michael Lewis this summer. While the Brad Pitt movie gives you the gist of how numbers can change something fundamentally, the book’s a real treat for even people who were never liked math. 

If you’re not scared by the technology, rather you’re short on time or money, consider Dataversity or Big Data University, which offer introductory and technical programs. 


And if you understand the numbers, but struggle with reframing your skills for other people to use: Read any of Ed Tufte’s books on enlivening information, Beautiful Evidence being the most recent. Watch Hans Rosling’s TED talk on turning statistics into inspiration. Ask the people around you, struggling to find veracity amid the other three Vs, what would help them succeed.

The deluge will continue until more of us become comfortable with the data science, know what questions to ask ourselves and those we work, then stand strong.

Marcia Conner consults with the world’s largest organizations on getting better at getting better. Learn more about her work at Follow for insights on business culture, organizational health, and a big data approach.

[Image: Flickr user Matthias Weinberger]

About the author

Marcia Conner works with ordinary people doing ingenious work and mediocre organizations realizing their employees can work in inspiring ways. She features stories of both in her upcoming book on ingenuity