Why Social Impact Investing Is A Crock

Over the last decade the world of do-gooding has seemingly been taken over by MBAs. Social entrepreneurship, a field encompassing both mission-driven businesses and entrepreneurial nonprofits, professes to bring the efficiency, rigor, and cold, hard metrics of business to the most important causes on the planet. Does it really?

Not so much, says Dean Karlan, author of the recent book More Than Good Intentions. "The social entrepreneurship world is in a weird spot, to be honest with you. It’s a world full of rhetoric about impact investing, yet I have very rarely seen an investor actually take that seriously. When you look at the actual analysis it lacks rigor." He distinguishes between the type of scientific research done by his lab, Innovations for Poverty Action, with trials complete with control groups, and the type of data collection done in the vast majority of the nonprofit world, which is nothing more than a "monitoring exercise."

"Take microcredit. What they do is they track how many borrowers, what their repayment was, and whether the businesses grew over time. That is not telling you your impact at all because you don’t know what would have happened if you didn’t lend."

Though he doesn't single them out by name, Karlan's critique points to behemoths such as the Bill and Melinda Gates Foundation, the largest grantmaking institution in the U.S. "A lot of money comes from groups that have used quantitative tools in the dotcom world, and they push toward collecting data that doesn’t necessarily answer the question. Now you’ve burdened your organization with data collection so you can feel rigorous." At the same time, he says, the cost of doing the kind of randomized, controlled impact studies that his organization conducts can often be prohibitive, particularly for social enterprises that are tacking for growth and "sustainability" (meaning profitability). "Put on your investor hat. If you want to put a million dollars into a business, are you going to put 33% of that into an impact study? That blows up your return. As a result, I’ve seen very few businesses that are being pushed forward as social entrepreneurship that undergo simultaneous rigorous evaluation."

Still, Karlan does see some bright spots on the horizon as his work and that of Esther Duflo's Poverty Action Lab--which also does scientific assessments of poverty interventions--grows in popular acclaim. "There’s serious excitement and enthusiasm about impact evaulation among the biggies like Gates, Hewlett, Ford, the World Bank. It’s not unanimous--we still get pushback and some are more enthused than others. But many are starting to use impact evaluation to figure out how we should spend the next $100 billion over the next 10 years."

Read more about Esther Duflo's assessment of social impact in this month's Life in Beta column.

[Image: Flickr user blprnt_van]

Add New Comment

14 Comments

  • Jo Reynolds

    This  article was not very in-depth, but has a catching title.   It shows that the author lacks an understanding that the industry is nascent with young models and infrastructure. Take a look at the CAF Ventursome paper, "Impact Investors's Handbook: Lessons form the World of Microfinance" CAF Venturesome provides some excellent recommendations for those of us who are growing this field.  Also the often referenced paper on Impact Investing from the Monitor Group describes the stages of industry development.          

  • Sasha Dichter

    Hmmm - too bad the great content of this article got lost under the (misleading) headline.  I don't get the sense that either Dean or Anya is saying that social impact investing actually IS a crock...just seems like a way to drive traffic, which can seem harmless until there's actual damage done in terms of good work being done.

    More complete thoughts in response here:

    http://sashadichter.wordpress....

    Here's the start of the post:

    In case you missed it (thanks @beckystraw for sharing it), Anya Kamenetz at FastCompany recently wrote a piece titled “Why Social Investing is a Crock.”

    It’s a pity that in the effort to grab attention, Ms. Kamenetz
    crossed the line from an attention-grabbing headline to misrepresenting
    the position of her story’s protagonist, Dean Karlan, author of More than Good Intentions.  Dean’s not saying it’s a crock, he’s saying we haven’t done rigorous analysis yet...

  • manitaray

    It would be a shame to question the integrity of investing in the social sector simply because evaluation tools are questionable or not understood.

    First - lets try and move away from  the term - 'social entrepreneurs' - it is becoming a real buzz word rather than seeing it for what it is : investing in people to help themselves makes good business sense. When people are able to help themselves, they become less reliant on aid, charity and government welfare. As such, there is less reliance on philanthropy and tax-payers money to support these causes - doesn't this just make sense? Moreover, many investments in businesses that build capacity and employ otherwise marginalised individuals, will generate income through these individuals - for the community and the government (as most employed people will pay tax) - again good business sense.Second - those who work in the social/welfare/development sector understand that evaluation is a completely different ball game here. It is critical to measure impact - that I agree. But impact must be seen as both qualitative and quantitative. Providing a micro-credit loan not only provides income - but creates better mental health (people are happier feeling empowered), creates better physical health (people can afford health care), promotes participation and social inclusion, increases access to education - lets not forget these. When you work with the people that you help - when you see the person's face and hear their story about how a $150 microcredit loan helped them to buy work boots so they can now earn an income - it is very hard to walk away and say "sorry - I can't help you as I don't fully understand the impact that the microcredit loan will have for you and everyone else".

    Finally - where there are tools out there to measure to social return on investment (SROI) as ways of measuring impact - I don't necessarily agree with these. As quantitative thinkers - we can put a number against anything and justify it with evidence if we choose. But lets not forget the ethics of helping people.   That helping the poor, marginalised, disadvantaged - is the right thing to do. 
     
    Yes some large and small philanthropic foundations mis-allocate or mis-spend some money. But which private corporation has not at some stage? Isn't that what 'contingency' amounts are about? At least they are investing in the future. I take my hat of to them. 

  • lyle hurst

    This article and the comments reflect well the awkward ambiguity of "hybrid" social ventures. I applaud the trend toward greater accountability and rigor in the social-purpose sector, but agree with Amelia's comment that we don't apply anything like RCT to for-profit enterprises, government initiatives, or many other activities (not to say we shouldn't, however). Investor/philanthropists like Dr. Charly Kleissner measure the performance of "investments" the old-fashioned way (ROI), and measure social impact with a variety of quantitative and qualitative assessments to qualify and prioritize potential investments. To lead with impact metrics seems to imply that these are not really investments at all, but charity with only a veneer of market economics.

    I also don't see fundamental disagreement here - I think the biggest issue is an unfortunate headline. All the best to those who are developing better evaluation methods, to those who are investing and granting money to social ventures, and to those who are creating and running the ventures themselves!

  • Lauren Cochran

    Does anyone else take issue with the fact that our friend Dean here attacks investing and goes on to use institutions that only make grants as examples?  Perhaps we should talk about many social entrepreneurship related businesses as self-subsidized non profits, but don't attack impact investing as a whole. There are some of us who are actually working pretty hard to make sure market rate for profit investments are being made. 

  • Lauren Burnhill

    The search for causality with respect to social impact
    investment increases costs without providing a corresponding benefit.
    Organizations like Innovation for Poverty Action (IPA) are indeed conducting
    scientifically rigorous randomized control trials (RCT), however many of these
    are based on entirely mistaken assumptions (and for many of us involve questionable ethics with regard to control groups). Let’s take poverty alleviation and
    microcredit. When we look at studies of immigrant communities in the US, most
    suggest that it takes a generation for a family to sustainably move up the
    economic ladder. Immigrant parents find (often) low wage or small business
    opportunities that, by dint of hard work, enable their children to acquire the
    language skills and education necessary to move ahead. Perhaps poverty
    alleviation through microcredit wouldn’t require a generation, but results are
    unlikely to be visible within 18-24 months. We know from existing research that
    perhaps 10-20% of microenterprises will grow into small or medium enterprises
    and we know that the another 10-20% will be harmed by overindebtedness that
    accompanies access to finance without economic opportunity. That leaves 60-80%
    of microenterprise borrowers somewhat better off, but still tremendously
    vulnerable to externalities. Given this reality, wouldn’t we be better served
    by studying how to screen out the 10-20% who would be harmed, or how we can
    improve outcomes for the 60-80% stuck in the middle?

     

    At the recent MicroNed conference in the Netherlands, I
    argued that I don’t believe microfinance alleviates poverty – it is a necessary
    but insufficient element for achieving this goal. Nathaneal Goldberg of IPA
    argued that he has clearly seen the benefit of microfinance at 24 months out,
    which serves to highlight another flaw in RCT methodology, namely that it only shows
    a snapshot in time. Four loan cycles in, a microenterprise will typically be
    more stable than before it had access to credit, but vulnerability remains
    high. A family illness, increase in the price of subsidized commodities or
    general economic downturn can all return the microentrepreneur to a precarious
    subsistence state. Over the medium term, this is sadly all too often the case.
    The poor need more than access to finance, they need access to life: health
    care, education, safe housing with clean water and sanitation – all of which
    requires thoughtful social impact investment.

     

    When it comes to complex multi-faceted problems like
    poverty, establishing causality of specific interventions is futile.
    Longitudinal data is critically important, but by its nature cannot serve as a
    short-term driver of social impact investment policy. Instead we should look
    for correlation. In other words, are those receiving goods or services
    better off than they were in their absence?

     

    As an example, let’s look at reducing childhood
    malnutrition, which is a goal of many social impact investors. To establish
    causality, we’d need multi-year data on both the communities we serve and those
    were no similar product or service is offered. Even partnering with government
    agencies that collect this data, it will be expensive to produce and have
    limited relevance for day-to-day investment decision-making. Since we know that
    a child’s height and weight percentile are correlated with overall health, we
    could use a simple SMS self-reporting tool to collect monthly height and weight
    information from the parents buying our product or service for their children. Improving
    height/weight percentiles over time would indicate lesser incidence and likelihood
    of childhood malnutrition. You might not be able to prove definitively that you
    have reduced childhood malnutrition, but you are de facto achieving your goal.

     

    Social impact investment is critical to achieving improved
    quality of life at the base of the pyramid and by extension, peace and economic
    development (happy people rarely spend their time rioting or in other acts of
    civil disobedience). Admittedly much more work needs to be done to understand
    and measure non-financial impact, but to dismiss the value of social impact
    investment entirely is premature and irresponsible.

  • Dean Karlan

    @Kevin: gosh no, that would be far too much research and evaluation. it is odd (to me) that many think proponents of RCTs argue to *always* do an RCT, since i've never heard a single proponent of RCTs propose doing an RCT everywhere.  Quite the opposite.  We all walk away from many situations where we do not think one should be done.  But there are many situations where they are appropriate, where we do not have good evidence on the impact of something, and an RCT is viable for learning better what that impact is. when that is the case, then we should do them, so that future donors and practitioners/policymakers can use that to help decide how to allocate their resources.

  • Andy

    Looks like we're still using financial measures to define success. Let's be less hung up about returns and boosts to the economy and just do what is right.

    http://www.kiva.org/ enables individuals to use their money to help individuals and small groups around the world to develop their ideas through microfinance. I only give through the organisations that charge no interest.

  • Anya Kamenetz

    Amelia, Mirm-- you measure what *didn't* happen by doing a controlled randomized trial. When you match up the beneficiaries with a group that is similar in every way but did not receive the microcredit loans, then you can have a true idea of the impact of microcredit. 

  • Kevin Fanfoni

    Interesting article that hints at the challenges of the impact measurement field (cost, systems, moving beyond outputs to outcomes/impact).  However, I think you should reconsider the title of the article (pretty sensationalist for FC, imo).  Do you really think it is necessary/feasible to do RCTs for every impact investment? 

  • Jacob Wells

    If you don't think Impact investing is working or that Social Entrepreneurs are not rigorous enough, you need to check out GoodCompany Ventures. Based in Philadelphia, they incubate for-profit start-ups that have a social cause. It is a rigorous program that helps the do-gooder entrepreneurs envision their plans from a very business side of things and prepares them for early stage investment.

    Check it out http://www.goodcompanyventures...

  • Amelia

    Um... I have an issue with: ""Take microcredit. What they do is they track how many borrowers, what their repayment was, and whether the businesses grew over time. That is not telling you your impact at all because you don’t know what would have happened if you didn’t lend."

    In what business anywhere can you measure what didn't happen? That's like saying oh, the world would be better if XYZ wasn't elected. We don't know that, because it didn't happen. To hold any enterprise or charity to that fire is absurd. 

    However, what you can measure is did that person or that family or that community have more wealth or create more value post-loan than pre-loan. I think that IS impact. Perhaps this just wasn't a very thorough explanation of his theories, but it just comes off as some high and mighty academic looking down his nose at anyone who actually tries to DO anything.