In Sunday’s NY Times Magazine, Joe Nocera wrote about Value at Risk (VaR) and the role it might have played in the current financial crisis. Was that tool an aid to dealing with the situations facing its users or did it contribute to their problems? How should we understand its impact? Several important questions are raised by the Nocera interviews. How should responsibility be apportioned between human behavior and measures like VaR? What do measures like VaR mean? And, what dangers must be addressed in the design of such measures?
Many of those Nocera spoke with saw the problem as rooted in human behavior, rather than financial measures or tools. Consider this assertion made by Greg Berman, one of the founding partners of RiskMetrics. “But I do think that this was much more a failure of management than of risk management.”
What does Mr. Berman take the “management” part of “risk management” to mean? We get a clue as we read on. “I think that blaming models for this would be very unfortunate because you are placing blame on a mathematical equation. You can’t blame math.” He seems to think of risk management as the application of mathematical tools to assessing risk, rather than the actual activity of managing it.
But of course you can blame math, just as you can blame the design of a clock that does not show the proper time or an automobile that performs poorly. Financial instruments, including VaR, are designs. They are human artifacts intended to serve a purpose. If you need convincing that mathematics is, like clocks and automobiles, the product of human design, I recommend George Lackoff and Rafael Nunez’s Where Mathematics Comes From: How the Embodied Mind Brings Mathematics into Being.
Others subscribed to similar dichotomies between models (or specifically VaR) and the people who use them. In speaking about the failure of Long Term Capital Management, Nocera noted “…firms took to rationalizing away the fall of L.T.C.M.; they viewed it as a human failure rather than a failure of risk modeling.” This is a common strategy for diverting blame. Rather than question the design of the instrument or system its advocates assert “the user is at fault.” But part of the activity of designing something involves insuring that it is not likely to be widely misused. Power tools have safety guards.
Assumptions play a critical role in thinking about models and many of those using VaR overlooked them. “Indeed, so sure were the firm’s partners that the market would revert to “normal”—which is what their model insisted would happen— that they continued to take on exposures that would destroy the firm as the crisis worsened.” Pay particular attention to that phrase between the hyphens “what their model insisted would happen.” This suggests that a return to normal was considered an output from the model. But the model did not say that things would return to normal; rather it assumed that they do.
An important, indeed the defining, characteristic of VaR is that it is a scaler; that is, a single measure, a number. As the article points out it is a number that got a lot of legitimacy by being marketed, standardized, and regulated. It came to have “meaning.”
Every semester, I ask MBA students to consider why we might want to beware of single measures. What kinds of problems do they bring with them? And every semester, bright students propose that measures are often incomplete, that any particular one might not capture all that we really want to know, that each will have particular assumptions embedded within it. And usually after they have been thinking about it for oh, ten minutes or so, one of them says something like “people will game any simple number.” So, listen now to this, from Nocera’s narrative:
“Guildimann, the great VaR proselytizer, sounded almost mournful when he talked about what he saw as another of VaR’s shortcomings. To him, the big problem was that it turned out that VaR could be gamed. That is what happened when banks began reporting their VaRs. To motivate managers, the banks began to compensate them not just for making big profits but also for making profits with low risks. That sounds good in principle, but managers began to manipulate the VaR by loading up on what Guildimann calls ‘asymmetric risk positions.’ These are products or contracts that, in general, generate small gains and very rarely have losses. But when they do have losses, they are huge. The positions made a manager’s VaR look good because VaR ignored the slim likelihood of giant losses, which could only come about in the event of a true catastrophe. A good example was a credit-default swap, which is essentially insurance that a company won’t default. The gains made from selling credit-default swaps are small and steady—and the chance of ever having to pay off that insurance are assumed to be miniscule.”
Create a single measure of anything, and clever people will find ways to work it. When I was a student, Professor Jim Emery told us about a time when he and his colleagues at Proctor & Gamble created a measure to reward workers for factory floor safety. The measure “days since an accident” was displayed over the door. And when a man cut his thumb one day he was encouraged by his co-workers to have it treated on his way home from work, since bonuses were linked to the measure and a visit to the in-house clinic would reset the counter. As Jim pointed out, no one wanted to induce that behavior, but they did nonetheless. Single measures will be gamed. Keep that in mind all you financial instrument and incentives designers.
I don’t expect that advice to be widely followed, though. Commenting on the widespread use of VaR, Christopher Donohue, who manages research at the Global Association of Risk Professionals, said that because it relates so directly to money people “attach a meaning to it.” And a former risk manager that Nocera spoke with considers it part of the human condition that “People like to have one number they can believe in.”
With all of this there was a heroic story in Nocera’s narrative. It came relatively early in the financial crisis when people at Goldman Sachs began to notice irregularities in the VaR. David Viniar, Goldman’s chief financial officer brought together about 15 people who met for three hours during which they “poured over everything. They examined their VaR numbers, and their other risk models. They talked about how the mortgage-backed securities market ‘felt’ [italics added].” This heroic episode is the story of people using irregularities in one measure as a clue guiding them to look at others, of using feeling to complement thinking, of using talking to make meaning within a complex situation they had never faced before. It is the story of a leader calling on his people to stop and think (“But who has time to stop in the middle of a crisis?” I hear many of the executives I have taught over the years crying out.). And it is the story of tragedy averted, at least for a bit. Goldman Sachs acted on what they learned in those three hours and avoided much of the pain suffered by Bear Stearns, Merrill Lynch, Lehman Brothers, and others.
There are several lessons for the design of financial instruments that follow from the stories Nocera has collected together. First, our financial systems involve humans and as such are complicated. No single measure can address all of the complexities that will be encountered in such complex systems. Second, meaning in complex social systems is not a matter of universal laws (as it is in physics, say); rather meanings are socially negotiated. And finally, because tools such as VaRs are the product of design, the extent to which design methods and attitudes are understood and employed will impact their utility and resilience.