Now Available: Impact Evaluation in Practice handbook (2nd Edition)

The second edition of the ​I​mpact Evaluation in Practice handbook is now available. The handbook is a comprehensive introduction to impact evaluation for policymakers and development practitioners. The updated version covers the latest techniques for evaluating programs, with expanded case studies.

---

New Publications
see publications page for full list

How Debit Cards Enable the Poor to Save More Debit Cards Enable the Poor to Save More (Journal of Finance, forthcoming 2021) We study an at-scale natural experiment in which debit cards are given to cash transfer recipients who already have a bank account. We find that beneficiaries accumulate a savings stock equal to 2 percent of annual income after two years with the card. The increase in formal savings represents an increase in overall savings, financed by a reduction in current consumption.  

Air Conditioning and Inequality (Forthcoming Global Environmental Change) We use household-level microdata from 16 countries to characterize empirically the relationship between climate, income, and residential air conditioning. Not only do richer countries have much more air conditioning than poorer countries, but within countries adoption is highly concentrated among high-income households. The pattern of adoption is particularly stark in relatively low-income countries like India, where we show that the vast majority of adoption between now and 2050 will be concentrated among the upper income tercile. We use our model to forecast future adoption, show how patterns vary across countries and income levels, and then discuss what these patterns mean for health, productivity, and educational inequality.

Aspiration Adaptation in Resource-Constrained Environments (Forthcoming in Journal of Urban Economics)
We use a multi-country field experiment to test the effect of a slum-housing intervention on the evolution of housing aspirations. Initially after the intervention, we observe a large housing gap in aspirations to upgrade their dwelling relative to the treatment group, echoing an aspiration to “keep-up” with the treated Joneses’. However, after 2 years of treatment exposure, the aspirational effect completely disappears and no effects are found on housing investment. Estimates based on a structural model of aspiration adaptation show that the decay rate is 38% per month, implying that housing aspirations return to baseline levels in just 28 months. Our evidence suggests that simply fostering housing aspirations may be insufficient to encourage housing investment in poor neighborhoods, and thus slum-upgrading policies designed to indirectly stimulate housing expansion may not be as effective as they promise to be.

Living Up to Expectations: How Vocational Education Made Women Better Off But Left Men Behind (Labour Economics 2020) We study an at-scale natural experiment in which debit cards are given to cash transfer recipients who already have a bank account. We find that beneficiaries accumulate a savings stock equal to 2 percent of annual income after two years with the card. The increase in formal savings represents an increase in overall savings, financed by a reduction in current consumption.  

New Working Papers
See working papers page for full list

Making Entrepreneurs: The Returns to Training Youth in Hard versus Soft Business Skills (2021) We study the medium-term impacts of an innovative mini-MBA program for high school students in Ugandan context. The program featured two separate treatments: (i) hard-skills featuring a mix of approximately 75% hard skills and 25% soft skills; (ii) soft skills featuring the reverse mix. Using data from a nationally representative RCT 3.5 year later demonstrates that training was effective in improving both hard and soft skills, but only soft skills were directly linked to improvements in self-ecacy, persuasion, and negotiation. The skill upgrade was rewarded in substantially higher earnings; 32.1% and 29.8% increases in earnings, most of which, was generated through self-
employment. Furthermore, youth in both groups were more likely to start enterprises and more successful in ensuring their businesses' survival. The program led to significantly larger profits (24.2% and 27.2% for hard- and soft- treatment arms respectively) and larger business capital investments (38.4% and 32.6% for SEED hard and SEED soft, respectively). Both SEED curricula were very cost-effective; two months worth of the extra earnings alone would exceed the cost of the program. These benefits abstract from the job- and business-creation benefits of the program, which were substantial: relative to the control group, SEED entrepreneurs created 985 additional jobs and 550 new businesses.

Digital Collateral (2021) A new form of secured lending utilizing “digital collateral” has recently emerged, most prominently in low- and middle-income countries. Digital collateral relies on “lockout” technology, which allows the lender to temporarily disable the flow value of the collateral to the borrower without physically repossessing it. We explore this new form of credit both in a model and in a field experiment using school-fee loans digitally secured with a solar home system. We find that securing a loan with digital collateral drastically reduces default rates (by 19 pp) and increases the lender’s rate of return (by 38 pp). We decompose the total effect and find that roughly one-third is attributable to adverse selection and two-thirds is attributable to moral hazard. Access to a school-fee loan significantly increases school enrollment and school-related expenditures without detrimental effects to households’ balance sheet.

Managerial Practices and Altruism in Health Care Delivery (2021) We conduct a field experiment of a  comprehensive management consulting intervention fort the Kenyan private health care sector. We find large improvements in management practices and structural quality that translated into better business performance in terms of increased investment, output, higher prices, increased revenue, lower unit costs and higher profits. However, this did not translate into improved process (clinical) quality.  Surprisingly we find the program significantly reduced correct clinical case management of patient care. We also find that the fall in quality did not affect demand, consistent with demand being quality inelastic. Hence, it was optimal for profit-maximizing firms to lower quality and raise prices. We examine this further by measuring provider-specific preferences with a modified dictator game: we find that the least altruistic (most profit maximizing) providers in the program were the ones that reduced correct clinical case management. Altruistic providers in the program did not lower quality or raise prices.

Vulnerability and Clientelism (2021). This study argues that economic vulnerability causes citizens to participate in clientelism, a phenomenon with various pernicious consequences. We employ a RCT that reduced household vulnerability through constructing residential water cisterns in drought-prone areas of Brazil. This intervention significantly decreased requests for private goods from politicians, especially among citizens likely to be in clientelist relationships. We also show the intervention decreased votes for incumbent mayors. 

Increasing Financial Inclusion and Attracting Deposits through Price Linked Savings (2021).  In a large field experiment in Mexico, we provide a temporary incentive to both open and use a savings account: specifically we offer prize-linked savings accounts with cash-prize lotteries, where lottery tickets are awarded as a function of savings balances. We find that 41% more accounts are opened in treatment branches than in control branches on average. Although the incentive to save is temporary as lotteries are only offered for two months, the new accounts continue to be used over time. After five years, clients who opened accounts in response to the lottery continue saving and making transactions at the same rates as those who opened accounts in control branches during the same months.

Another Brick in the Wall: The Effect of Non-contributory Pensions on the Material and Psychological Well-Being of Older Adults (2021)  We explore the effect of non-contributory pensions on the well-being of the beneficiary population with a field experiment in Paraguay. Households with a beneficiary increased their level of consumption by 44 percent and older adults increased their leisure by reducing labor supply. The program also improved a psychological well-being index by 0.48 standard deviations. The well-being index is composed of a depression symptom scale (which itself decreased 7 percentage points), reported satisfaction with quality of life, empowerment, feeling of contribution to the expenditures common to all household members, and perception of self-worth. The results are consistent with the findings of observational studies; Bando, Galiani and Gertler (2020) in Peru and Galiani, and Gertler and Bando (2016) in Mexico. Thus, the effects of non-contributory pensions on well-being in Paraguay are comparable to those found for Peru and Mexico and add to the construction of external validity.

How Spillovers from Appointment Reminders Improve Health Clinic Efficiency (2020). Missed clinic appointments present a significant burden to health care through disruption of care, inefficient use of staff time and wasted clinical resources. Short message service (SMS) appointment reminders show promise to improve clinics’ management through timely appointment cancellations and efficient rescheduling, but evidence from large-scale interventions is missing. We study a nationwide SMS appointment reminder program in Chile for chronic disease patients at public primary care clinics. Using longitudinal clinic-level data we find that after two years the program increased clinics’ total number of visits per by 5.1% on average. The program did not change the number of visits by chronic patients eligible to receive the reminder, but it instead increased visits by other patients, ineligible to receive reminders in clinics that adopted the program by 7.4% on average. These results suggest that the appointment reminder systems increased clinics’ ability to care for more patients through timely cancellations and re-scheduling.

Promoting Parental Involvement in Schools: Evidence From Two Randomized Experiments (2020). Parental involvement programs aim to increase school-and-parent communication and support children’s overall learning environment. This paper examines the effects of low-cost, group-based parental involvement interventions in Mexico using data from two randomized controlled trials. The first experiment provided financial resources to parent associations. The second experiment provided information to parents about how to support their children’s learning. Overall, the interventions induced different types of parental engagement in schools. The information intervention changed parenting behavior at home – with large effects among indigenous parents who have historically been discriminated and socially excluded – and improved student behavior in school. The grants did not impact parent or student behaviors. Notably, we do not find impacts of either intervention on educational achievement. To understand these null effects, we explore how social ties between parents and teachers evolved over the course of the two interventions. Parental involvement interventions led to significant changes in perceived trustworthiness between teachers and parents. The results suggest that parental involvement interventions can backfire if institutional rules are unclear about the expectations of parents and teachers as parents increase their involvement in schools.