Yesterday, Katie Hill, India Portfolio Manager of the Acumen Fund explored the importance of measuring performance for social enterprises. Today, she explores the challenges.
This blog entry is part of Beyond Profit’s media partnership with the Khemka Forum on Social Entrepreneurship in Hyderabad, India on December 8-9, 2009. As part of our media partnership, we have invited the leaders of three of the Forum’s tracks – Financial Instruments, Building Alternative Talent Pools, and Performance Metrics – to contribute to our blog.
Clearing the Biggest Hurdles
The challenges of impact measurement may seem infinite. I am just going to name a few biggies.
Getting the Data
The sheer logistical challenge of data collection is often overlooked. Bootstrapped social enterprises often don’t have big budgets or extra staff to complete wide customer surveys. So, we need to use what we’ve got. GEWP, for example, can use their 1-year warranty cards to collect impact data. LifeSpring hospital, a maternity hospital chain targeting low-income families, can use its registration process for new mothers to collect pertinent data, such as education backgrounds or household income levels.
From the investor seat, Acumen Fund experienced data scarcity challenges in our early days, and we’re still making improvements here. Simply collecting 5-7 meaningful data points every quarter, let alone the 30 we strive for today, at first seemed like a Herculean feat. Now, we get monthly data from our investees.
Inconsistency of Metrics in the Field
It seems like every new organization is reinventing the measurement wheel and, therefore, each investor is using slightly different metrics. One solution is IRIS. Evolving out of the Global Impact Investor Network (GIIN), a small group of stakeholders came together to think about a taxonomy of consistent metrics for social and environmental impact. The result has been Impact Reporting and Investment Standards (IRIS), which is an open source of common terms and indicators for financial, operational and social metrics on initiatives ranging from agriculture to education to healthcare. This common language is the first step towards truly building an industry.
Analyzing the Data
There is no point in collecting numbers unless you’re learning something from them. So, how do we go about calculating meaningful indicators from a slew of data? It doesn’t have to be fancy. Use a pencil and paper, Google spreadsheets; use whatever works. As companies scale up, with high volumes of data and multiple locations, more sophisticated tools might be necessary. These could be standard MIS systems like Tally or Salesforce.
As investors, we faced similar challenges of moving from bulky spreadsheets to something more functional; we could not find an appropriate off-the-shelf software tool. So, we co-developed Pulse with Google.org. Pulse is a web-based platform for tracking and managing social investments. Pulse has already been used by a number of peer investors and is now being taken to the next level with the support of Skoll Foundation, Lodestar Foundation, and Salesforce.com.
Correlation vs. Causality
One of the trickiest points on impact measurement is whether we can ever have confidence that our initiative caused the intended outcome. For example, if patients from LifeSpring Hospital demonstrate improved health indicators in mothers and babies, is it because of their visits to LifeSpring (causation) or is it because of another aligned factor—the family cares about their health, so they both visit LifeSpring and engage in a number of other healthy activities, like drinking safe water (correlation).
Randomized control trials, through players like MIT’s Poverty Action Lab, are the “gold standard” in proving causality. In India, IFMR is also making great strides. These evaluations can be costly and are very time-intensive. They should be used strategically and will require additional resources, as most social enterprises with thin margins can’t afford this line item.
Cost-Effectiveness of Measurement
To the point above, there is a general sentiment that social and environmental impact measurement is expensive and social enterprises don’t have an endless budget. It may be helpful here to distinguish between social outputs and social outcomes. For example, if SKS Microfinance distributes HUL’s Purit water filter to its borrowers, the “output” is X number of homes with access to clean drinking water. The “outcome” is the demonstrable impact—reduced incidence of waterborne diseases that result in healthcare cost-savings, higher school attendance, etc.
So, until the cost of doing these rigorous assessments falls, we think it is our responsibility to count the outputs as consistently as possible. The conclusions you can draw from these outputs may not be made with scientific rigor, but can inform businesslike decisions and raise important questions on impact.
Start Somewhere
We would argue that the lack of precision in social impact measurement is no excuse for not trying. Accurate impact measurement will only emerge if we start making imperfect calculations today and constantly improve upon them. There are a handful of impact measurement tools/methodologies out there. Each one in isolation may be problematic, but combined and cross-checked, the effort could lead to collective accuracy. And we need this collective accuracy if we hope to ever demonstrate that the social enterprise hypothesis holds water.
