×
FROM LEFT MIT profs Abhijit Banerjee, Esther Duflo, and Rachel Glennerster | Photograph by Doug Menuez

Which Poverty-Fighting Policies Work? J-PAL Has the Answer

A global league of economists called J-PAL is deploying its experimental methods and one all-powerful asset — data — to explain human behavior, change how we help the poor, and try to save the world.

Every year, wealthy countries and donors ship billions upon billions of dollars in aid to the developing world. The money has not bought prosperity: Diarrhea still kills 1.5 million children annually. More than 210 million kids work when they should be in a classroom. Polio, which had once been eradicated in all but four countries, is spreading across

Africa again. Some 2.6 billion people have no access to modern toilets. And more than 1 billion people don't have enough to eat in 2009, setting a new record.

Why? From an unremarkable old building overlooking the Charles River in Cambridge, Massachusetts, a trio of MIT economists is leading a remarkable global movement that's working to find out. "A lot of money gets spent by well-meaning people with no idea what they're doing," says Abhijit Banerjee, who has spent much of his career studying corruption. "It's wasted."

Obviously, much of the money that goes to help the developing world is not being used effectively. But what's not so obvious is how all that money should be spent. That's where the Jameel Poverty Action Lab (J-PAL) comes in. Led by the Indian-born Banerjee, Briton Rachel Glennerster, and Esther Duflo — a Frenchwoman who just won a MacArthur genius grant for her work on poverty — this global network of researchers has one joint ambition: to transform the living conditions of 100 million poverty-stricken people in Africa, Asia, and Latin America. Within the next five years. On a budget of barely $10 million. While building development models that could eventually effect change for billions more people.

The word "lab" is key to the work: The academics are succeeding, in large part, by using the world's poorest citizens — and their villages and nations — as subjects in real-life experiments. Sometimes controversially, they have applied the kinds of clinical-trial techniques that have long been standard in medicine but until now have been rarely used in economics. The data they've gathered have changed the way leaders of some of the world's largest charities and governments give to the poor.

J-PAL's save-the-world goal would be laugh-out-loud ridiculous if not for the research that has come out of the six-year-old group. The results have been impressive — and not just in the head-nodding way that academic papers can be. The wonks who belong to the lab have collectively helped 30 million people, from small farmers in Kenya who now use fertilizer to mothers in rural India who immunized their infants to poor Peruvians who opened savings accounts for the first time in their lives.

The hopes of the J-PAL academics remain as huge as the problem of poverty itself. "We're not interested in making a few villages wonderful," says Glennerster. "We're interested in changing millions of people's lives."

______

Traditional economics, for all its complexity, boils down to two steps: One, observe events and actions. Two, create a theory explaining them. Practitioners of the dismal science have always been good at coming up with more theories, but in development economics, they have always been particularly dismal at translating those theories into successful, real-world methods for fixing what's broken. During the 1960s and 1970s, they created models for fighting poverty that were backed up with inadequate data, and by the 1980s, development was relegated to the dark corners of economics departments. That changed in the 1990s, when databases like the World Bank's World Tables made testing macroeconomic theories easier, and laptops and globalization made fieldwork in far-flung locales incredibly cheap.

Around that time, randomized controlled trials, the experimental method long used in medicine, began coming into vogue in economics. Inspired by researchers including Paul Gertler of the University of California, Berkeley, and Harvard's Michael Kremer (Glennerster's husband and now a J-PAL member), a new generation of economists has adopted the technique over the last decade. In 2003, Banerjee, Duflo, and Harvard's Sendhil Mullainathan formed the Poverty Action Laboratory to coordinate the efforts of like-minded academics and spread the gospel of experimentation. (After a gift from MIT alumnus Mohammed Abdul Latif Jameel, scion of a Saudi family that owns the world's largest chain of Toyota dealerships, the lab was renamed in honor of Jameel's late father.)

J-PAL is a loose club of professors, and membership — limited to those who use experimentation to explore poverty-related issues — is by invitation only. The lab plays matchmaker, teaming its members with one another and with outside organizations seeking academic collaborators. It also acts as a clearinghouse for economics research done via randomized trial, aggregating its members' work and turning it into more than the sum of its parts. "Individually, I could raise money and do projects," Duflo says, "but I could not have the power of all the projects together."

J-PAL's experiments offer a powerful method for answering difficult questions about poverty, especially those that don't fit the tidy models of traditional economics. How would you prove, for example, that a person has a well-paying job because he works hard, not because his parents paid for elite private schooling or because he's blessed with lucky genes? Traditionally, an economist would resort to some statistical wizardry using, say, regression analysis — work that is almost entirely done on computers and in offices. Proponents of randomized controlled trials try to test hypotheses by orchestrating events in the field and on the fly. The thinking, explains Banerjee, is, "Why don't we try it? If we're wrong, we're wrong. That's the beauty of the experimental method — we'll find out in a year."

Here's an example: A Banerjee-led team in India found that in rural Udaipur — a spectacular tropical district of shimmering lakes and devastating poverty — only 5% of children were immunized. Anecdotal evidence suggested two potential causes: Nurses don't show up regularly at government clinics and the region's illiterate mothers have a long-standing traditional belief that vaccination will sicken their children. Banerjee didn't buy the second explanation. Drawing on economic theory about incentives, he speculated that if mothers were given a small gift, they might bring kids in for shots.

Working with the Indian charity Seva Mandir, his team randomly placed mothers into two groups: One received free delivery of immunizations, while the other received delivery plus 1 kilogram of lentils per visit — and, after her child completed all vaccinations, a set of bowls. Immunization rates barely improved in the delivery-only group, but in the lentils group, they jumped to 37%. "A bag of lentils is not going to do anything," says Banerjee, "if people really have those traditional beliefs."

So far, J-PAL's 31 affiliated researchers have completed some 70 projects in 30 nations, and often their findings have upended conventional development wisdom. For instance, African nations — and the Western governments that fund them — have spent millions of dollars to send workers door-to-door, notifying citizens of their HIV status. The thinking is that if people are aware that they're HIV-positive, they'll be more careful, and if they're negative, a test will remind them that HIV from sexual activity is a risk. But experiments performed by J-PAL member and University of Michigan economist Rebecca Thornton in Malawi proved that those who learn they're positive barely change their sexual behavior. The strategy is a big waste of money. Thornton now has studies under way to determine more cost-effective alternatives.

J-PAL's leaders don't believe that they are the smartest people in the room — it's more that they have better data at a time when everyone is craving it. "The fact that we have so many people who want to talk to us says something, not so much about us as about the world recognizing that it is functioning on a very weak knowledge base," says Banerjee.

Glennerster, who serves as executive director, used to help translate research into policy for the IMF, and in her years working with governments, she'd always been frustrated at the dearth of solid, data-driven research. "I always had a belief," she says, "that if you put good evidence out there on questions that people really wanted the answers to, they would respond."

Under her guidance, J-PAL has rapidly grown into a global power center of development. Today, it collaborates with four of the six largest U.S. foundations that work internationally, including the Bill & Melinda Gates Foundation; 7 of the 10 most prominent multilateral organizations, including the Inter-American Development Bank; and governments, from developing giants like India to rich nations like France. J-PAL members have also trained dozens of NGOs, such as Oxfam, to run their own experiments. J-PAL's research has been so influential that the recent Congressional bill funding the World Bank encourages the bank to use randomized controlled trials in evaluating projects.

The J-PAL researchers are not the only economists doing such trials. But they've developed a reputation for top-notch research, and J-PAL is now a global brand. "They will bring technical rigor," says Carola Álvarez, chief of the strategy development division at the Inter-American Development Bank. Rob Yates, a senior health-policy adviser to the U.K. Department for International Development, says, "J-PAL's work has indeed had a profound impact" — so much so that Prime Minister Gordon Brown has repeatedly cited the group's ideas in articles and speeches on poverty, health care, and aid.

______

The notion of experimenting on the poor can be unsettling, evoking shades of past scientific scandals like the Tuskegee Study. Duflo, who is working on projects in five countries, counters that the billions of dollars being spent by governments and donors to fight poverty are tantamount to experiments anyway, so better to have experts watching over them, implementing scientific methodology and gauging whether they actually work. "With precautions in mind, we believe that not experimenting is what is unethical," she says. "If you scale up a policy that does no good, or even some unintended harm, or if you do not adopt a policy that could make a great amount of good, this is when you are carrying out gigantic experiments on people's lives. And that should not be acceptable."

The more common — and more valid — criticisms cut to the heart of the way J-PAL researchers study and interpret their trials. Several well-known economists have raised questions about the randomized controlled approach, citing long-standing criticisms of clinical medical trials by the Nobel-winning University of Chicago economist James Heckman. Though the arguments are math-heavy, the concepts are straightforward: It's difficult to prove that a small-scale experiment in one village applies to another town with another culture in another country. And while trials describe the effects of a program on a population, they may fail to explain why some people are affected more than others or are impacted negatively. Overly focused experiments may be dismissed as little more than program evaluations, salves that address poverty's symptoms without aiding understanding of its root causes.

Most of the critical economists, including Harvard's Dani Rodrik and Princeton's Angus Deaton, do see value in randomized trials. Yet they worry about statements by J-PAL apologists — Banerjee has said "a new economics [is] being born" — that seem to venerate randomized experimentation as the one true path of poverty research. "The thing I think is silly is this idea that there's some lexicographic preference for randomized controlled trials, that their results are always better and other ones are always wrong," says Deaton, who coined the disparaging term "randomista" to describe those with seemingly undying faith in such methods.

Glennerster asserts that Deaton's concerns are best addressed "by doing more of this stuff" — in other words, practicing until the methodology is made perfect, refining the work so that each generation of experiments is more thoughtfully designed and comprehensive than the last.

J-PAL's more recent studies do make good on one thing skeptics like Deaton and Rodrik have been calling for: a stronger marriage of experimentation and theory. Take, for example, a National Bureau of Economic Research study on teacher bonus payments by J-PAL member Karthik Muralidharan, an assistant professor of economics at the University of California, San Diego, and Venkatesh Sundararaman of the World Bank. To find out whether incentives would inspire teachers to work harder, they persuaded the Indian state of Andhra Pradesh to promise bonuses to teachers based on improvement of their pupils' test scores. The massive experiment, covering 300 primary schools, found that even small amounts of performance-based pay led to substantial improvement on tests. Educators weren't just teaching to the test: Students scored higher on both mechanical questions and those measuring their understanding of broader concepts.

The Andhra Pradesh study makes a persuasive argument in a key poverty-policy area: how best to spend the charitable dollars streaming toward education. It also informs underlying economic theory on how incentives alter teacher behavior. Conclusions like these allow Glennerster to share with governments and donors concrete examples of what works. When Kenyan prime minister Raila Odinga visited Boston recently, she had a long list of specific policy recommendations waiting for him. "We've got enough results that we can start saying something about what works, not just about one study but across a sector," says Glennerster.

Most J-PAL studies do have a narrow focus; as UCSD's Muralidharan puts it: "A lot of us now feel it's more satisfying to answer small questions well than big questions badly." So instead of trying to find out whether education spending trumps health spending in curing poverty, they're examining granular issues like "Does this textbook work?" But in aggregate, answering these kinds of questions can produce big effects. Indeed, J-PAL can compound the effect of its members' studies, combining them to show multiple proofs of a hypothesis, not just one. It is now seeing patterns it can translate into practical, evidence-based theories.

Among the most crucial and potentially most influential findings is that small costs and small incentives have huge effects on people's actions, especially in preventive health. The long-standing practice has been for charities to charge small fees for items such as insecticide-treated bed nets, which prevent the spread of malaria, on the grounds that paying for something causes people to value it more. But raising the price of bed nets from zero to 60 cents reduced use by 60 percentage points in a study done in Kenya by J-PAL member Pascaline Dupas of UCLA and Harvard's Jessica Cohen, and people who receive bed nets for free use them no differently from those who pay. A series of deworming studies by Harvard's Michael Kremer and Berkeley's Edward Miguel found that raising the price from free to 30 cents reduced the percentage of kids taking the drug from 75% to 19%. Other J-PAL studies showed the same dynamic in chlorinating water and HIV testing.

Together, this body of work explodes the conventional wisdom. Says Glennerster: "Charging small amounts is the exact wrong thing to do." In fact, if anything, tiny subsidies can greatly increase uptake, as shown by Banerjee's immunization-and-lentils study and others that demonstrate that effect in everything from picking up HIV-test results to buying fertilizer.

Such conclusions have enormous ramifications for the way we spend money in the developing world. Still, J-PAL's leaders are cautious not to make bombastic claims about ending poverty in, say, 10 years. Change, they believe, will be painstakingly constructed much the way their body of evidence has been: piece by tiny piece. The vast majority of the poor kids in Africa "are not going to be Bill Gates. They're not even going to be rich. They may not even be above the poverty line," Glennerster says. "But moving up toward the poverty line is an important improvement."

And nothing about that will be random.

Ryan Blitstein, a contributing editor at the public-policy magazine Miller-McCune, has written for Time, Salon, and San Jose's Mercury News.

FROM LEFT MIT profs Abhijit Banerjee, Esther Duflo, and Rachel Glennerster | Photograph by Doug Menuez

Add New Comment

8 Comments

  • Chris Reich

    For all it's complexity....it boils down to ethics. Democratic governments aren't always ethical. We (United States) create poverty in other countries and have serious endemic poverty here. If we can't solve poverty in the U.S., we can't solve it in countries like Bolivia.

    As our own ethical base has slipped, so poverty increased. When our corporations profit from 'cheap', that is slave, labor in China, we create poverty at home and pretend like we're 'lifting' the Chinese.

    If we insisted on dealing with ethical parties, governments and corporations abroad, and paid a fair price for resources we consume, poverty would be lessened.

    Consider Venezuela. Hugo Chavez gets pretty bad press here in the U.S. But he was elected by a wide margin and the base of his support is the poor. He started by doubling what Venezuela gets for her own oil that U.S. companies extract. That made Chavez the bad guy. He's a rotten socialist for protecting the resources of his country for his people. No, he's no saint, don't misunderstand me. But much poverty has been created by democracy---the key is ethics. And, an ethical government would be democratic. Just don't confuse democratic with capitalistic.

    China created poverty and genocide in Darfur to get their oil. Ethical?

    Chris Reich
    www.TeachU.com

  • Richard Geller

    If one desires positive global change, I can think of nothing quite so valuable as knowing what problems most need solving and building up a reliable data base of what works and what doesn't on the ground. It certainly sounds to me that J-PAL is a step in the right direction. Imagine the benefits here at home if our much needed healthcare reform was being designed by people who actually know something about what works and what doesn't.... Such a sensible idea fairly takes my breath away!

    --
    Richard Geller
    http://www.aSiteAboutSomething...

  • Aly-Khan Satchu

    This Report apparently misses the Following;

    1.What is the amount of Tranmission at the Ground level for each $1.00 Given? About 3 US Cents.

    2. What is the amount of Aid that is snaffled by an Overarching Government and NGO Architecture, whose very raison D'Etre seems to be to subvert.

    3. What about Grameen which seems to me a Preeminent Example of assisting People to raise themselves via their Bootstraps?

    4. Today Most Folk have a Mobile Phone. What about a System that sidesteps everybody and drills right down to Individual. Both in terms of Information and help?

    Aly-Khan Satchu
    www.rich.co.ke

  • Richard Geller

    If one desires positive global change, I can think of nothing quite so valuable as knowing what problems most need solving and building up a reliable data base of what works and what doesn't on the ground. It certainly sounds to me that J-PAL is a step in the right direction. Imagine the benefits here at home if our much needed healthcare reform was being designed by people who actually know something about what works and what doesn't.... Such a sensible idea fairly takes my breath away!

    --
    Richard Geller
    http://www.aSiteAboutSomething...

  • Adam Smith

    I do not think this is particularly innovative . Researchers have been using randomized trials to test development programs at least since the 1970s (Head Start, Perry School, etc). Mexico has been using these trials to assess its poverty fighting initiatives for at least 20 years. Political scientists have been promoting these "experiments" as the best thing ever for almost a decade or so.
    Do these trials work? It depends what is being measured and how you define success. Health trials focus on relatively simple indicators (immunization, mortality, etc). One runs into trouble when you try to measure the impact of more complex interventions like education.
    I think what these economist are doing is OK, but evidence from the past indicates that their success will be modest and limited.
    I recommend sociologist Herbert Gans' work "The Positive Functions of Poverty" (1972/1994), in which he explains how the affluent classes benefit from all the initiatives to fight poverty. This J-Pal example fits so perfectly.

  • Adam Smith

    I do not think this is particularly innovative or new.
    Experts have been using randomized trials to test public policies at least since the 1970s (Head Start, Abecedarian, Perry School). Political scientists have been enamored with "experiments" for at least ten years. Mexico has been testing poverty programs with randomized trials since the early 1990.

    Do these trials work? It depends on your definition of success. Health programs are much easier to test because the indicators that are being measured are relatively simple (immunization, infant mortality, etc). Usually one runs into trouble trying to measure the effects of something more complex, like education for example.
    Evidence from the past indicates that this project is not unlike others and it will certainly perform below expectations. What these economist need to read is Herbert Gans' articles and books about poverty.

  • Richard Leff

    An important question is whether you can really eliminate corruption before you have started to correct poverty. The presence of a stable middle class seems to be very important to eliminating the corruption that plagues the developing world.
    --
    Richard Leff

  • Dave Cearley

    The single most effective way out of poverty is a democratic government somewhat free of corruption. Dictators, totalitarian states, and tinhorn dictators are responsible for most of the misery in this world. While the J PAL work is useful, it ignores the elephant in the room.