Amid A Crush Of Data, What Managers Can Learn From Doctors About Making Decisions

Whether you measure it by the gigabyte or the teraflop, data is cheap, and it’s everywhere. How to take a page from evidence-based medicine and make data work in your favor.

I once asked a client what she thought would happen to her sales if a customer were to post a negative product review on her company’s web site (when we had this conversation the site didn’t allow customer reviews).  She said sales would decline, of course, and for many of us this is such an obvious conclusion that my question might have seemed pointless.  The fact is, however, that in many cases a negative customer review has a higher statistical correlation with closing a product sale than a positive review does.  Let me say that again, with feeling: When someone comments negatively on your product, this negative review itself can actually boost your sales!   

The reason, probably, has to do with credibility.  Suppose you go to a web site to evaluate the customer reviews for, say, a vacation destination, and all the reviews are 5-star wonderful.  Aren’t you a tad suspicious?  On the other hand, if most of the reviews are great, but there’s one review that tars and feathers the hotel concierge and check-in staff, then you’re much more likely to believe that the good reviews are actually authentic.  

The point is, data speaks the truth, and you should be listening to it. But most of us just aren’t ready for the oodles and oodles of perfectly truthful data now becoming available.  Data being created and stored today includes not just business documents, government statistics, and scientific research, but also blog posts and web sites, status updates, photos, videos, comments, product reviews, text messages, podcasts, "likes" and "shares," and location check-ins. Information is generated through the individual efforts of millions of people, interacting in billions of ways, independently. Every 24 hours Google processes more than a billion individual search requests and Facebook users generate some three terabytes of new data. In the lingo of the industry, this is "big data." You don’t have to look far to find some authorities predicting the volume of "technical information" available to the human race may soon be doubling every few hours.

From a data-processing perspective, most of the new information flood now beginning to envelop businesses consists of unstructured data—paragraphs of text, images, videos, voice mail messages and recordings, emails, tweets—and increasingly sophisticated analytical tools are required just to make sense of it. More than tools, however, businesses will have to begin embracing a new mindset, as well.

We can look to the field of medicine for guidance. The discipline of "evidence-based medicine" involves using statistical studies to inform medical decisions. Before making their judgments or formal diagnoses, doctors are encouraged to examine the actual data from epidemiological studies and quantified research. In this way, whatever judgment a doctor comes to will be based not only on the doctor’s perspective, but on the best available prior evidence as well. The reason doctors are encouraged to look at the statistical evidence before rendering their own judgments is because of the confirmation bias, our natural human tendency to place more credence in whatever facts or numbers confirm the point of view we already have. And even today, evidence-based medicine is not universally applied, because not all medical professionals are equally capable of objectively balancing judgment and facts.

"Evidence-based management" would impose the same kind of discipline on businesses, given that they are now enveloped in a rapidly growing ocean of unstructured data. To deal with this rush of data, managers need to put aside their pre-conceived notions and rely more on what the numbers actually say. This doesn’t mean judgment and intuition aren’t important, nor should statistical and analytical studies ever be considered flawless, no matter how sophisticated they are.

But in dealing with a large and rapidly increasing volume of data, managers will have to pay closer attention to actual numbers, and they’ll have to become more comfortable with the language and best practices of statistical and mathematical analysis—control groups, correlation vs. causation, standard deviations, confidence intervals, statistical significance, testing the null hypothesis, and so forth. Analytics can help managers shape their own judgments more objectively, with better results.

Yale economist Ian Ayres wrote in his book SuperCrunchers that the best way to employ an expert opinion is to supplement and enhance whatever conclusions or implications we see in our analysis of the data. Analysis first, in other words, and then apply intuition, hunches, or judgments to improve our understanding. And Stanford’s Robert I. Sutton suggests that evidence-based management requires "a mind-set with two critical components: first, willingness to put aside belief and conventional wisdom…and instead hear and act on the facts; second, an unrelenting commitment to gather the facts and information necessary to make more informed and intelligent decisions, and to keep pace with new evidence and use the new facts to update practices."

Evidence-based management, in other words, involves starting with the facts, the numbers, and the analysis—the data—and then developing an assessment. So the next time someone asks you what might appear to be a slow-ball question, think first about what data might illuminate the answer, and reserve your judgment. As a manager, you should use data not to supplement a decision you've already made, but to improve your decision as you are making it.

Unfortunately, our natural bias is to do this the other way around. It is human nature to prefer making judgments first, and then searching out the evidence to support them. It's just the way our psychologies are structured. But for businesses, the economic consequences of allowing such a flawed decision-making process to continue are about to get more severe, because the higher and higher volume of data available makes it a virtual certainty that a competent manager will be able to find some facts to support nearly any decision, retroactively.

[Image: Flickr user shirleypickford]

Add New Comment

2 Comments

  • Statsguy

    Interesting post, thanks. Two thoughts: 1) hypothesis testing is really not appropriate for anything outside of lab chemistry, and better analytical tools are now widespread. (Plus what managers really want can only be achieved through Bayesian stats, i.e., "What's the probability of earning $x over baseline with innovation y?"; 2) I would love to learn HOW to convince managers to follow your advice, when they've spent a career working from the gut but think they are making decisions based on evidence. That's the hard nut to crack, I'd say.

  • Collectual

    Great post. You're right about the ever-growing volume of unstructured data; it's going to be very interesting to see which companies are able to organize and analyze this type of data for insights. Another point to consider is not only the initial analysis but how the findings are integrated with existing and more traditional metrics and how that information will be shared across an organization. So many teams (marketing, customer service, finance, product development) may benefit from the intelligence that is surfaced from unstructured data analysis that organizations will need to come up with a process that ensures the right team receives the right information.

    Thanks for sharing!