What happens when you run an experiment and the outcome is not what you had planned?
One of the traps of experimentation is expecting the testing to validate your thinking. It is natural to want the experiment to say yes—even to think that if it doesn’t confirm your idea, it is a failure.
You may even fall into the trap of designing experiments, so that they can affirm your idea. If you do this, you will have committed another sin: ignoring the original problem and focusing on yourself instead. When it comes to experiments, it’s not about success or failure; it’s about outcome and learning. If you engineer your experiments correctly, you may get outcomes that surprise you, or the results may show you that what you are working on is wrong.
In my career, I have gone through this pain many times. I once ran a two-year experiment, spending millions of dollars, and the experiment was wrong. My team and I believed that we had a product that was going to revolutionize the semiconductor industry at that time. There were 68 different scientists in four different countries working on the project. We ran over 200 different experiments. Looking back, 60 of them were not needed. They were instead created for self-comfort, to convince ourselves that we were working on a relevant problem. Those 60 said what we wanted them to say, but the real problem-focused experiments resulted in different outcomes: our product was not going to work as envisioned. We were wrong. At some point, we had to make a decision to stop, but in the interim, we lost marketplace and the first-move advantage that we used to enjoy.
We learned many lessons from that instance (not limited to):
- Groupthink can lead to complacency and blurred vision
- Focus on the problem, not people’s preferences for power or position
- Fail faster to save money and time
- Don’t let ego get in the way when it comes to experimentation–it is okay to admit the outcomes are different and cut your losses
You run experiments so that you can make a decision. If the outcome is not what you expect, you have to make a choice either to continue, to stay the course but in a different way, or to try something altogether different—to pivot.
When and how to pivot
It’s not a small shift or an iteration. It’s a systematic search of the various configurations of a problem-opportunity, combined with solutions and business models that might prove valuable. It’s about finding what works when the solution you originally planned is not panning out.
It’s asking, based on what you have learned, what else you can do to create better or different outcomes. There are several ways you can pivot: around the problem, around the solution, or around the business model. You can even pivot around the market. If you understand your customers and see that they have a different need, this may inspire a pivot.
Learning how to pivot is a critical skill for leaders and innovators who will undoubtedly experience uncontrollable circumstances and must change course in order to survive. Many of today’s most successful companies have at one time executed a pivot. Twitter, Groupon, Nintendo, and Starbucks all made course corrections based on different factors. In fact, there is no single major thriving brand that has not experienced some form of pivot driven by internal or external factors.
PayPal is one interesting pivot success story. The company started in security software, but they were not as successful as they had hoped, so they saw an opportunity and pivoted. They developed an electronic wallet, and in fact still own the patent for it. But they were a little early for the market, so they pivoted again into financial transactions and became the PayPal that you see today. This is an example of pivoting around a solution. They saw the future and made a course correction; then they recognized customer behavior and made another one. They became a billion-dollar company in fewer than two years. And now of course you have the mobile version in Venmo.
Blackberry, on the other hand, only made incremental changes and never decided to make a major solution pivot. Interestingly, they had enough data to support a pivot, but their leaders were concerned about asking for more money and ego got in the way, so they made just enough changes to appease the board and investors. And we all know what happened: they lost a market that they once had dominated.
If you are considering a pivot, you must make sure that you are not simply chasing symptoms. Pivoting is a major decision; the type and size of pivot dictates the decisions supported by specific experimental outcomes. So how do you know when it’s time? Here are some indicators that it might be time to consider a pivot in your innovation:
The customers tell you so
Once customers tell you that what you’ve created is not what they need, it’s time to pivot and create a different product or find a different market. You will begin to see this in multiple ways: direct or indirect feedback, the different ways they use the product or service, a change of context, or even total apathy. Always listen (and watch) for the voice of the customer.
The cost of experimentation and validation is too high
If you keep experimenting and you are not hitting the mark, this is another indication that a pivot is necessary to find a different solution or a different market. There is such thing as too much experimentation, definitely if you are chasing a solitary outcome. Don’t get caught up in a cycle of testing and validating because you can’t bear to be wrong. Set perimeters around experimentation, and then decide if it’s time to do something different or be something different.
The value has been diluted
If you originally intended to provide a specific value for specific context at a specific volume, but this is no longer attainable, it’s time to pivot to find a different problem to solve, create a different solution, or find a different market. For example, if your experiments tell you that your $20 million business dream idea is really more of a $200,000 business idea based on market demand and customer adoption, then a course correction is necessary. You can take everything you’ve learned, either about the product or about the market, and pivot so you can operate under the right parameters and expectations.
We run experiments to make decisions, and when it comes to innovation and business leadership, you must make decisions; course-changing decisions. Pivoting is a critical skill and experience for any business that wants to have a future. If you’re not courageous enough to pivot when the outcomes of your operations are not favorable, you will become obsolete.
Dr. Evans Baiya is a technology and innovation strategist, author, and speaker who works with companies around the globe to help them identify and develop new value. He is an adviser with Price Associates and the coauthor of The Innovator’s Advantage.