Anyone who's been in marketing long enough can relate to the following scenario: It's the monthly leadership meeting for your company and the agenda includes a review of last month’s performance. April sales were up 5% versus March, and media spend was up by 10%. The VP of marketing excitedly proclaims that “our marketing is working, the increase in media spend was responsible for our increase in sales—we need to spend more in media.”
The VP of sales strongly disagrees: “No, it was our sales incentives that caused the increase, and it would have happened without any incremental media spend.” Hence the longstanding feud between sales and marketing is alive and well. But who is right and how can you prove it?
Lawyers, philosophers and mathematicians love to pull out their Latin dictionaries. This particular phrase translates to "with this, therefore because of this." In other words, just because sales followed an increase in media spend doesn’t guarantee that it was the media spend that was responsible.
Two data sets can be correlated, they can even be highly correlated, but that doesn’t mean one caused the other. While it is tempting to jump to the conclusion of causality, due to our personal bias or long-standing experience, it is easy to misdiagnose the situation. And in today’s rapidly evolving media landscape such errors can be costly.
Back to our scenario. Five different possibilities can explain what we observed. In these cases we will look at metrics beyond just media spend and incentives to determine how different variables could have caused sales.
- Media spend caused sales: We will return to this scenario shortly, but to be clear we have witnessed examples of this really being the case. And we will discuss how to statistically prove it and quantify its impact.
- Sales caused media spending: Someone with no knowledge of how a thermometer works may conclude that every time the line on the thermometer goes up it gets warmer outside. Anyone who understands how a thermometer works knows it is the other way around. Going back to our question, were sales already steadily increasing before the media spend increased? Higher sales frequently mean higher media budgets, and historically, the largest media spends occur during peak sales months. Perhaps sales had been steadily growing for the last six months and increased media spend followed this trend. Could it be that it was an ongoing increase in sales that really led to and justified the increased media spend and not the other way around?
- An unknown third factor caused sales to occur: In our example, was the media spend the only budget that increased? Did the budgets for incentives or price cuts also increase? Did the company launch a new product? Or were there external factors? Did a major competitor have bad publicity this month? Did unemployment drop, housing starts increase, and other leading economic indicators suddenly shift so that the economy as a whole picked up significantly this month? Do sales typically follow a seasonal pattern? Any combination of these events could be the largest contributor to the increase in sales and not the increased media spend.
- Marketing and sales give each other “positive feedback”: An engineer may observe that as engine temperature increases, so does oil pressure. But at the same time, the increased oil pressure causes the oil to lubricate less effectively, leading to higher engine temperature. This is also sometimes referred to as a “self-reinforcing” system, or “bidirectional causation.” Back to our original question: maybe previous increases in sales led to an increase in media spend, which did contribute some increase in additional sales, which could lead to more media spend, and so on—but it was not the increased media spend alone that increased sales.
- Last month’s sales were a coincidence: Any observed statistic (such as sales) is subject to random variation. A flipped coin has to come up either heads or tails. If you flip a coin once, and it comes up heads, you’ve really gleaned no information. But if you flip the coin ten times and it comes up heads all ten times, then there are only two possible conclusions: either you’ve just observed a very rare event (with a probability less than 1 in 1,000), or the coin is weighted towards heads. Back to the question at hand; just because we observed one time that an increase in media spend led to an increase in sales, are we really ready to conclude it will happen again? Are we really convinced it will happen every time? Or would you like to gather more data before you’re ready to stake you career on that claim?
So how can we ever make a conclusion about whether media caused sales? All hope is not lost. There are statistical methods that provide insights.
The Nobel Prize-winning economist Clive Granger came up with a method known as Granger Causality. In our example, to prove G-Causality we create a statistical model that determines how well prior months of sales data predict the current month’s sales. Then we would create a second model that used prior sales data and prior media spends to predict the current month’s sales. If a test comparing the two models shows that the inclusion of media spend is a better model, statistically speaking, we conclude that marketing spend G-Caused sales.
However, while this does show a cause between media spend and sales, it does not enumerate the relationship. In other words, it would not necessarily be correct to conclude that an additional 10 percent increase in media spend would always result in an additional 5 percent increase in sales. To gain this level of understanding, an ROI model, which has been discussed in detail, would be needed to enumerate how effective each individual type of media channel is on increasing sales.
Let’s return to our five possible explanations of the marketing-sales relationship and understand what Granger Causality would tell us in each case.
- Media spend caused sales: This is the desired outcome. The tests for Granger Causality would show a statistically significant improvement when adding media spend into the sales model. It is always good form to test the inverse relationship (Sales -> Media) to see if there is a feedback system in play. In the case where marketing is the driver of sales there may be a significant inverse relationship but to a lesser extent. This is often the case for brands that have both effective sales operations and media plans.
- Sales caused media spend: G-Causality would show no improvement when adding media spend into the model. When testing the inverse relationship we would see that adding sales into the media model increases predictability. This would indicate that media budgets are driven by prior sales levels.
- An unknown third factor caused sales to occur: G-Causality would show no improvement when adding media spend into the model and the inverse relationship would be equally as fruitless. We would need to expand our scope to determine what other changes occurred that would have influenced sales levels.
- Media and sales give each other “positive feedback”: The tests for Granger causality would show a statistically significant improvement when adding media into the sales model and the test of the inverse relationship would show an equally significant improvement. This would indicate an interconnected relationship that can be thought of as the yin and yang of the marketing-sales Tao.
- Last month’s sales were a coincidence: G-Causality would show no improvement when adding media into the model, and the inverse relationship would be equally as fruitless. Sometimes coincidences happen.
I’ve discussed the possible root causes to the observed relationship between media and sales but an example my help: A few years back, my team analyzed the relationship between automotive media spend and sales. We found that well-planned media spend has a significant G-Causal relationship on sales. Interestingly, we also found that sales have a weaker G-Causal relationship with media spend.
When you stop and think about it these relationships make perfect sense. In the auto industry, regional and dealer marketing efforts tend to be very price and incentive focused which has an impact on near term sales, hence the G-Causality between media and sales. In the longer term, media budgets tend to be cut based on low sales performance, hence the weaker G-Causality between sales and media.
To be clear, good marketing does cause sales. But it is often a complex relationship, which must account for several factors such as media lag, incentives, and changes in the broader economy and competitive activities. Good marketers are constantly at work to discern which marketing activities are not merely correlated with sales but actually have a causal relationship.
In part two on of our discussion on causality and correlations, I will discuss the types of questions you encounter in the day-to-day marketing world, when you need causality, what to do if you only have correlations—and what happens when you don’t have the time to check.
[Image: Flickr user Andre Roberto Doreto Santos]