Why social enterprises struggle to measure impact – and what impact investors can do about it

This article was originally published on Next Billion.

As impact investors seek investable opportunities worldwide, their focus includes enterprises and other organizations that both improve their customers' livelihoods and are financially sustainable. To direct their capital effectively and track its impact, these investors typically rely on output measurements, such as total capital invested by theme, or number of people served. This focus on outputs can help inform their investment decisions and reveal the impact of their funding to some degree, but more must be done to ensure that their investee organizations are truly creating positive change for customers. And unfortunately, social enterprises and other organizations often struggle to measure this outcome-focused impact in a sustainable way.

Kiva has worked with more than 560 microfinance institutions (MFIs) and social enterprises globally since 2005 – many of which are the types of organizations that impact investors typically aim to support. To better understand the results of these partnerships, we conduct an impact measurement survey annually – both for our current 130+ partners and for potential partners that apply for funding through Kiva.org. This article shares what we learned this year from this unique dataset, highlighting current impact measurement practices in the financial inclusion sector, gaps in these practices, and what the impact investing community can do to better design its measurement practices, products and services.

What organizations are measuring

In the past year, we received over 200 unique survey responses from partner MFIs, social enterprises and new applicants located in more than 65 countries. The results provide fascinating insights into the impact measurement practices of these social impact-focused organizations.

The majority of these organizations (over 70%) are conducting impact studies, which is a promising sign of the growing awareness of the importance of measuring impact. However, as we look closer at the types of studies being conducted, we see that the percentage of organizations conducting customer needs assessments or formal quantitative outcome surveys is lower. This limits organizations’ ability to respond to issues affecting customers, understand their needs, or provide clear evidence that they are creating change for customers.

Nearly eight in 10 organizations state that they do collect some outcome data, formally or informally, on how their products and services are changing the livelihoods of their customers. And in the data below we see which social outcomes are the top priorities for these organizations. Not surprisingly, their primary focus for achieving social goals is related to increasing the household or business income of their customers, followed by increasing household spending or consumption, increasing the level of household assets, and decreasing the likelihood of living under the national poverty line.

N = 206 organizations, Approximately 65% are MFIs, and 35% are social enterprise companies. The same organization can cite more than one outcome, so percentages will not add to 100%.

Barriers in impact measurement

In order for impact investors to better understand how their investee organizations impact customers, they can either request data from the investees, conduct impact studies themselves, or rely on proxy indicators and external evidence. The data from our survey shows that trying to increase impact evidence by relying on investees to provide data themselves is difficult. The fact that investees are facing impact measurement challenges is receiving growing attention, for example through 60 Decibels’ opinion piece in Stanford Social Innovation Review last year, and this year’s report from the European Microfinance Platform.

Kiva takes a collaborative and data-driven approach to address and learn more about this issue. This year, we asked our portfolio organizations about the barriers they face in impact measurement. The data they shared provides clear insights, with the top four barriers discussed below.

N = 109 organizations, Approximately 65% are MFIs and 35% are social enterprise companies. The same organization can cite more than one barrier, so percentages will not add to 100%.

1. Impact Surveys Are Perceived As Too Costly: The cost is the top barrier to conducting impact measurements. Nearly one in three organizations say they require more funding in order to be able to complete impact measurement activities. As one off-grid solar company writes: “The main barrier is the additional cost to do an investigation of the social (and environmental) impacts. Our company only holds the funds we receive from customers. It's not something we want to charge clients, so we would have to get a donation or a contract to do a study.”
2. Difficulty Reaching Customers: Nearly one in four organizations cite difficulty in reaching their customers to conduct impact surveys. This is due to clients living in rural or remote areas, having limited phone access, or being difficult to find because they move frequently. As one MFI in Colombia writes: “We serve entrepreneurs in scattered rural areas that complicate access and communication and prevent constant monitoring, in addition to the fact that despite the digital transformation that the country has gone through, there are still territories with connectivity difficulties.”
3. Questionable Credibility of Responses: About one in four organizations cite the difficulty in getting credible, unbiased responses. As one microfinance institution in West Africa writes: “​[A top barrier is] ​the reluctance of clients to answer the questionnaire because they find that we are digging into their lives”
4. The Need for Specialized Training for Staff: We also see that nearly one in five organizations cite the challenge of a lack of qualified staff, which is also often connected to a lack of funding for impact activities. As one microfinance NGO in sub-Saharan Africa states: “Conducting a study of such scope requires financial means but also people who are well-trained and qualified to carry out these types of studies.”

Looking further into that fourth challenge, we found that 40% of organizations lack a dedicated staff person focused on impact measurement and management - another sign that with limited funds available, impact-focused organizations are not hiring impact measurement personnel. We also see that seven in 10 organizations still use paper surveys. However, it is encouraging to see that some organizations are using advanced analysis tools to streamline their ability to capture insights from the social impact data they collect. Almost half leverage integrated data analysis systems such as Salesforce, Tableau or Taroworks, and 35% are using advanced statistical programs such as Python or SQL.

N = 199 organizations, Approximately 65% are MFIs, and 35% are social enterprise companies. The same organization can cite more than one data method, so percentages will not add to 100%.

The need for assistance with impact measurement

When asked if organizations would be interested in receiving assistance from Kiva, researchers or other impact measurement organizations, we find that 94% are interested in receiving additional impact resources, with the top needs being assistance in:

• conducting needs assessments of their customers
• measuring the social impact of their products and services
• researching new product design

N = 95 organizations, Approximately 65% are MFIs and 35% are social enterprise companies. The same organization can cite more than one research need, so percentages will not add to 100%.

We can do better as an industry

It is exciting to see more alignment in the industry around the importance of impact measurement, with more than 160 leading impact investors endorsing the IFC’s Impact Principles. These principles specifically require that impact investors have processes in place to monitor their progress towards achieving positive impact, which includes a recommendation for investors to measure how their investments are affecting stakeholder outcomes. Importantly, those investors that have endorsed these principles are then required to have their processes verified by an external party. We see similar investor alignment around the Impact Management Project’s five dimensions of impact, of which one dimension, “How Much,” is focused on getting investors to measure how much change in outcomes stakeholders have experienced.

However, we still see that many impact investors continue to rely on output measurements, while assuming positive impact for all their investments within a sector. We have a long way to go to change these practices in many sectors, particularly microfinance, where we still see great variability in the outcomes experienced by borrowers. When there is great impact variability within a sector, it is essential to have impact data for individual investees. For instance, the recent 60 Decibels Microfinance Index shows that the average percentage of microfinance customers who say their quality of life has “very much” improved after receiving a loan is 35% – but the report includes one MFI where just 5% of customers reported this improvement, and another MFI where 72% of customers reported it.

We see similar variability in impact evidence in the academic literature. Evidence from Banerjee et al., (2019) shows that entrepreneurs with previous business experience generated double the business revenues of those in control groups, after a microfinance institution arrived in their community. And a 2015 study in Mexico found limited evidence that microfinance improved the household income of borrowers, while in contrast, a 2020 study in China found that microfinance significantly increased household income and reduced poverty.

Given these heterogeneous impacts among regions, MFIs and borrowers, impact investors and microfinance organizations must do more to invest in measuring the effects of their products and services on the lives of their customers.

What impact investors can do

Kiva is focused on increasing the investment and attention we dedicate to impact measurement, enabling us to better learn how our loans are affecting borrowers’ financial health, including improvements in income and financial resilience.

Here are some ways that other impact investors can support efforts to achieve and measure positive customer outcomes:

1. Provide technical assistance to organizations to help them build out their impact measurement practices in-house:
   • Problems Addressed: Impact Measurement Too Costly; Lack of Trained Staff; Lack of Efficient Survey Technology
   • Examples: Impact investors can help cover costs for new technology, such as tablets, staff training and database setup with a mobile data collection system like TaroWorks. For example, Kiva recently supported iDE Mozambique in their adoption of a digital data collection system. The responsibility of measuring impact should not fall entirely on the organizations themselves.
2. Help co-finance third party impact evaluation studies:
   • Problems Addressed: Impact Measurement Too Costly; Questionable Credibility of Responses
   • Examples: As impact research becomes more affordable, such as through the lean data methodology of 60 Decibels, it becomes easier for investors to share the cost of these third party evaluations with investee organizations and other investors. Both organizations and investors greatly benefit from understanding more about the customers' needs and the impact they experience. For example, Kiva is a founding partner of the 60 Decibels Microfinance Index, and we have helped several of our investees conduct outcome studies.
3. Connect organizations with existing funding opportunities in the sector:
   • Problems Addressed: Impact Measurement Too Costly; Difficulty to Attribute Impact to the Organization
   • Examples: Impact investors can provide consistent information to investee organizations about funding opportunities like technical assistance grants from philanthropic organizations and companies, as well as information about how to apply for research grants, such as Innovation for Poverty Action’s open call for proposals that address consumer protection. Thanks to a research grant from UN ESCAP, Kiva was able to partner with UN ESCAP and three investees to conduct impact assessments and publish findings in a public report.
4. Reach out to academic and research organizations:
   • Problems Addressed: Impact Measurement Too Costly; Difficulty to Attribute Impact to the Organization
   • Examples: Impact investors can leverage existing academic and research organizations with financial resources and trained staff that can partner with their investees to measure impact and contribute to the broader learning agenda. For example, Kiva partner Together Association for Development and Environment partnered with researchers from the World Bank and found that their program increased participants’ monthly income by 66%. Similarly, Kiva introduced our partner, Fundación Santo Domingo, to Innovations for Poverty Action researchers, to enable research into the effect of flexible repayments on borrowers.

These measures can provide MFIs and other social enterprises and development organizations with solutions to some of the core challenges that are limiting their impact measurement efforts. We believe these findings, along with the other insights we’ve learned from our surveys of Kiva partners, meaningfully contribute to the discussions of how impact investors and others can support more effective measurement practices and increase accountability for improving the livelihoods of investee organizations’ customers.