Before Succeeding, You Have To Fully Confront Failure. But How?

We’re wired not to fix our own worst mistakes: Our brains often convince us that failure isn’t our fault or simply an aberration. But we can overcome our wiring, as this remarkable example shows.

Before Succeeding, You Have To Fully Confront Failure. But How?

[The Broadway dance show Movin’ Out is a stunning, exceedingly rare, turnaround story: On July 12, 2002, after its premier at the Shubert Theater in Chicago, a reviewer at the Chicago Sun-Times called it “stupefyingly clichéd and embarrassingly naïve.” But just three months later, when it finally opened in New York, the New York Times called a “shimmering portrait of an American generation.” Soon after, it picked up a couple of Tony Awards.


The turnaround was engineered by the production’s legendary choreographer Twyla Tharp, who faced down her initial failure with remarkable insight. In this chapter from Adapt: Why Success Always Starts with Failure, Tim Harford writes about what Tharp’s example can teach everyone else.–Ed.]

Twyla Tharp’s revamp of Movin’ Out was widely acknowledged to be the most rapid and total transformation of a Broadway show in many years. Michael Phillips, the Chicago Tribune reviewer whose stern review had been so controversially picked up by Newsday, also applauded, but added a question whose answer should interest us all: “How did this happen?”

Part of the answer lies is the very institution of the out-of-town tryout, the show business equivalent of the corporate “skunk works” idea: creating a space to experiment in which failures can be instructive and recoverable. As Tharp writes in her book The Creative Habit, “The best failures are the private ones you commit in the confines of your room, alone, with no strangers watching. Private failures are great.” Quite so: you can learn from them without embarrassing yourself. But the next-best kind is in front of a limited audience. If your new show is going to fail, better that it does so away from Broadway, giving you a shot at recovering before it hits the big stage.

Being willing to fail is the essential first step to applying the ideas of Adapt in everyday life. Twyla Tharp makes a point of failing in private every day. She rises at 5.30 am to work out, improvising alone or–increasingly, as she danced on into her fifties and sixties –with a younger dancer, “scratching,” looking for ideas. She films three hours of improvisation and is pleased enough if she can find 30 seconds that she can use. “Like a jazz musician jamming for an hour to find a few interesting notes, a choreographer looks for interesting movement . . . inspiration comes in molecules of movement, sometimes in nanoseconds.”

[A medley of numbers from Movin’ Out, performed at the 2003 Tony Awards]

The next step is finding, whenever possible, relatively safe spaces in which to fail: when the time came to unveil her new creative work to the public, she did so not directly on Broadway–from where an initial panning would have been even harder to recover–but in a way that allowed for the possibility that the show might not be as good as she must have thought it was. In a radically different context, Tharp’s approach follows Peter Palchinsky’s principles: First, try new things; second, try them in a context where failure is survivable. But the third and final essential step is how to react to failure, and Tharp avoided several oddities of the human brain that often prevent us from learning from our failures and becoming more successful.

The first of those quirks leads to denial. It’s why Sir James Crosby sacked Paul Moore rather than accept his valid critique of the bank, why Joseph Stalin ordered Peter Palchinsky to be killed for his correct analysis of the great Soviet engineering projects, and why Donald Rumsfeld forbade his senior general to use the accurate word “insurgency.” It seems to be the hardest thing in the world to admit that we have made a mistake and to try to put it right. Twyla Tharp herself has the perfect explanation of why: because ‘it requires you to challenge a status quo of your own making.’


Tharp, who was 61 at the time Movin’ Out flopped in Chicago, had an unimpeachable reputation and had worked with everyone: Philip Glass, David Byrne, Milos Forman, Mikhail Baryshnikov. It would have been easy for someone of her stature to reject outright the critics’ views, refuse to change the show, lose her investors’ money, set back the careers of her young dancers, and go to the grave convinced that the world had misunderstood her masterpiece.

Why is denial such a natural tendency? Psychologists have a name for the root cause that has become famous enough that many non-psychologists will recognize the term: cognitive dissonance. Cognitive dissonance describes the mind’s difficulty in holding two apparently contradictory thoughts simultaneously: in Tharp’s case, “I am a capable, experienced, and respected choreographer” and “My latest creation is stupefyingly clichéd.” This odd phenomenon was first pinned down in an ingenious laboratory experiment half a century ago. Leon Festinger and James Carlsmith asked their experimental subjects to perform a tedious task–emptying and refilling a tray with spools, using one hand–for half an hour. On some plausible-sounding pretext they then offered a third of their subjects $1–a small sum even in 1959, about an hour’s wage–to tell the next experimental subject (actually an actress) what a great time they’d had stacking spools onto trays for half an hour. They offered another third of their subjects a much more substantial sum, $20, half a week’s typical wages, to do the same thing. The remaining third went straight to the questionnaire that all the subjects finally filled in, asking if they had enjoyed themselves.

Unsurprisingly, most people said they hadn’t. Yet there was a very odd exception: the students who’d been asked to reassure this stranger about what fun they’d been having, and who’d been paid only one dollar to do so, were much more likely to tell the experimenters that they’d enjoyed themselves. The unconscious cognitive process seems to be: “With very little incentive, I told this girl I enjoyed myself. That’s a contradiction to the idea that I didn’t enjoy myself. So, I guess I must have enjoyed myself, right?” By contrast the ones who’d been paid $20 seemed more able to separate the events in their minds: “Hey, if the pay is good, who wouldn’t tell a white lie?”

The remarkable power of denial is nowhere better illustrated than by the reactions of some lawyers when DNA evidence became admissible in courts and many apparently sound convictions were overturned. Consider the response of Michael McDougal, a prosecutor in Texas, when faced with the evidence that Roy Criner, a man convicted of raping and murdering a young woman, was not the man whose semen was found in the victim’s body. McDougal accepted the evidence but, incredibly, rejected the implication. “It means that the sperm found in her body was not his. It doesn’t mean he didn’t rape her, doesn’t mean he didn’t kill her.” The chief judge of the Texas Court of Appeals, Sharon Keller, pointed out that Criner might have committed the murderous rape while wearing a condom.

Such denial is far from unique. For a prosecutor, the idea that you convicted the wrong man is upsetting. As Richard Ofshe, a social psychologist, comments, it’s ‘one of the worst professional errors you can make–like a physician amputating the wrong arm’. Of course the correct way to resolve the apparent contradiction is to believe, ‘I am a good person and an experienced prosecutor but nevertheless I made a mistake.’ For a human mind that is apparently unable to grasp “I was fibbing when I said I enjoyed stacking spools,” this may be too much to ask. For Tharp, who said of her debut, “I didn’t promote myself as a star. I had always seen myself as a star: I wanted to be a galaxy,” the tension between “I am a star” and “My new work is risible” must have been particularly tempting to repress.

The second trap our minds set for us is that we chase our losses in an attempt to make them go away. Recall Frank, the luckless contestant on Deal or no Deal: having discarded the box containing half a million euros, he proceeded to reject ever more reasonable offers from the Banker until he ended up with next to nothing. All because, to quote the psychologists Kahneman and Tversky, he had not “made peace with his losses.”


Making peace with our losses can be unbearably difficult to do–even for Twyla Tharp. In 1965, she was in a relationship with the artist Bob Huot. He wanted marriage and babies, she wanted to concentrate on her dancing. W hen she became pregnant anyway, she endured a horrific backstreet abortion without anaesthetic before being abandoned by the abortionists, bleeding heavily, at an ice-cream parlour in New Jersey. As she wrote in her autobiography, “That experience remains intensely painful, one of the few that make me wonder whether my professional and artistic aspirations were really worth the price.”

Now comes the moment of chasing the loss: Tharp went on to marry Bob Huot. Only with hindsight did she identify her motivation at the time: “Bob and I had lost a baby; marriage would prove our love and confirm us once again.” The marriage lasted just four years.

Three decades later, Tharp did not chase her losses. It must have been tempting to stick to her original vision for Movin’ Out, deluding herself that the New York critics might prove more discerning, or that New York audiences would like it more. Instead, she made peace with her losses and immediately set about the hard work of winning back both the critics and the audiences.

The final danger Tharp avoided is one we might call “hedonic editing,” borrowing a term coined by Richard Thaler, the behavioral economist behind the book Nudge. While denial is the process of refusing to acknowledge a mistake, and loss-chasing is the process of causing more damage while trying to hastily erase the mistake, hedonic editing is a subtler process of convincing ourselves that the mistake doesn’t matter.

One way we do this is by bundling together losses with gains, like a child trying to eat some disliked healthy foodstuff by mashing it up with something tasty until the whole mess is palatable but unrecognizable. Think of that reliable tool of office life–indeed, of life in general–the “praise sandwich.” The praise sandwich is a criticism sandwiched between two delicious slices of praise: “I think this is excellent work. It would be great if you could [important feedback here]. But overall, as I say, it’s excellent work.” It’s a good way to avoid alienating everyone who works with you, but the criticism sandwiched between praise may be lost in the larger whole. You say, “It’s excellent, but you need to fix . . . “I hear, “It is broadly excellent.” I feel better, but I will not become better.

A different psychological process, but with a similar effect on our ability to learn from our mistakes, is simply to reinterpret our failures as successes. We persuade ourselves that what we did was not that bad; in fact, everything worked out for the best. Twyla Tharp could have decided that what she’d actually set out to achieve was something artistically radical rather than commercially mass-market, so the incomprehension of the critics was, in a way, validation; she could have found a few audience members who liked it, and convinced herself that the views of this discerning clientele should be given greater weight.


How profoundly this tendency runs in the human brain was demonstrated by a team of researchers including the psychologist Daniel Gilbert. The researchers showed their experimental subjects an array of six prints of paintings by Claude Monet–the lilies, the Houses of Parliament at sunset, the haystacks, and others–and asked them to rank the images in order from the one they liked most to the one they liked the least. The researchers then offered the experimental subjects a choice of two spare prints they “just happened” to have, and the spares were always the pair ranked in the middle–number three and number four. Naturally the subject usually chose number three, having just declared it to be preferable to number four.

The researchers came back on a later occasion with the same set of six prints and asked their subjects to re-rank them from one to six. The ranking changed: the print that the subjects had chosen earlier was now ranked one or two; more surprisingly, the print that the subject had previously rejected was demoted to rank five or six. As Gilbert jokes, this is “Happiness being synthesised . . . ‘The one I got is really better than I thought! That other one I didn’t get, suuuucks!'” We systematically reinterpret our past decision as being better than it really was.

That might sound surprising enough, but psychologists have in fact been observing and measuring this tendency for the last half-century. What is truly astonishing is that the experimental subjects in this case were severe anterograde amnesiacs, people completely unable to form new memories. Gilbert and his colleagues didn’t return weeks or months later, but after just thirty minutes, by which time their unfortunate subjects had forgotten everything. They had absolutely no recollection of having ever seen any Monet prints, and yet they strongly preferred the print they had previously chosen even though they had no conscious knowledge that they had chosen it. Our capacity to reinterpret our past decisions as having worked out brilliantly is a very deep one.

These, then, are the three obstacles to heeding that old advice, “learn from your mistakes”: denial, because we cannot separate our error from sense of self-worth; self-destructive behavior, because like the game-show contestant Frank, or Twyla Tharp when marrying Bob Huot, we compound our losses by trying to compensate for them; and the rose-tinted processes outlined by Daniel Gilbert and Richard Thaler, whereby we remember past mistakes as though they were triumphs, or mash together our failures with our successes. How can we overcome them?

Click here to purchase the book at Amazon, for $15.

[This is the third in a three-part series of excerpts from Adapt: Why Success Always Starts with Failure by Tim Harford, published by Farrar, Straus and Giroux, LLC. Copyright © 2011 by Tim Harford. All rights reserved. Top image by LoveFusionPhoto]