Now Available: Impact Evaluation in Practice handbook (2nd Edition)

The second edition of the ​I​mpact Evaluation in Practice handbook is now available. The handbook is a comprehensive introduction to impact evaluation for policymakers and development practitioners. The updated version covers the latest techniques for evaluating programs, with expanded case studies.

---

New Publications
see publications page for full list

Vulnerability and Clientelism (2022). 
(Forthcoming in the American Economic Review)
(working Paper)
Abstract: This study argues that economic vulnerability causes citizens to participate in clientelism, a phenomenon with various pernicious consequences. To examine how reduced vulnerability affects citizens’ participation in clientelism, we employ two ex-
ogenous shocks to vulnerability. First, we designed a randomized control trial to reduce household vulnerability: our development intervention constructed residential water cisterns in drought-prone areas of Brazil. Second, we exploit rainfall shocks. We find that reducing vulnerability significantly decreases requests for private goods from politicians, especially among citizens likely to be in clientelist relationships. Moreover, reducing vulnerability decreases votes for incumbent mayors, who typically have more resources for clientelism.

Another Brick in the Wall: Effect of Non-contributory Pensions on Material and Psychological Well-Being
(Journal of Behavioral and Economic Behavior, 2022).
(Publication, Working Paper)
Abstract: We explore the effect of non-contributory pensions on the well-being with a field experiment in Paraguay. Households with a beneficiary increased consumption by 44 percent and older adults increased their leisure by reducing labor supply. The program also improved a psychological well-being index by 0.48 standard deviations.

How scheduling systems with automated appointment reminders improve health clinic efficiency                           (Journal of Health Economics, 2022).
(Publication, Working Paper)
Abstract: Missed clinic appointments burden health care systems through inefficient use of staff time and resources. Scheduling software with automatic appointment reminders shows promise to improve clinics’ management through timely cancellations and re-scheduling, but at-scale evidence is missing. We study a nationwide text message appointment reminder program in Chile implemented at primary care clinics for patients with chronic disease. We find that visits by 5.0% in the first year and 7.4% in the second.

Do private providers give patients what they demand, even if it is inappropriate? A randomized study utilizing unannounced standardized patients in Kenya
(British Medical Journal - Open, 2022). 
(Publication)
Abstract: Some quality improvement strategies encourage patient engagement; however, patients demanding inappropriate medicines can favor the selection of resistant microbial strains. This study examines the effects when patients demand different types of inappropriate medicines. We conducted an experiment where unannounced standardized patients (SPs), locally recruited individuals trained to simulate a standardized case, present at private clinics. SPs portraying caretakers of a watery diarrhea childhood case scenario (in absentia) conducted visits at 200 private, primary care clinics. Half of the clinics were randomly assigned to receive an SP demanding amoxicillin (an antibiotic); the other half, an SP demanding albendazole (an antiparasitic drug often used for deworming). Providers significantly increased the dispensing rate for those who demand albendazole, but did for amoxicillin.

How Debit Cards Enable the Poor to Save More
(Journal of Finance, 2021) 
(Publication, Working Paper)
Abstract: We study an at-scale natural experiment in which debit cards are given to cash transfer recipients who already have a bank account. We find that beneficiaries accumulate a savings stock equal to 2 percent of annual income after two years with the card. The increase in formal savings represents an increase in overall savings, financed by a reduction in current consumption.

Air Conditioning and Inequality
(Global Environmental Change 2021) 
(publication, working paper)
Abstract: We use household-level microdata from 16 countries to characterize empirically the relationship between climate, income, and residential air conditioning. Not only do richer countries have much more air conditioning than poorer countries, but within countries adoption is highly concentrated among high-income households. The pattern of adoption is particularly stark in relatively low-income countries like India, where the vast majority of adoption will be concentrated among the upper income tercile. We use our model to forecast future adoption, show how patterns vary across countries and income levels, and then discuss what these patterns mean for health, productivity, and educational inequality.

Aspiration Adaptation in Resource-Constrained Environments  (Journal of Urban Economics, 2021).
(Publication, Working Paper)
Abstract: We use a multi-country field experiment to test the effect of a slum-housing intervention on the evolution of housing aspirations. Initially after the intervention, we observe a large housing gap in aspirations to upgrade their dwelling relative to the treatment group, echoing an aspiration to “keep-up” with the treated Joneses’. However, after 2 years of treatment exposure, the aspirational gap completely disappears and no effects are found on housing investment. Our evidence suggests that simply fostering housing aspirations may be insufficient to encourage housing investment in poor neighborhoods, and thus slum-upgrading policies designed to indirectly stimulate housing expansion may not be as effective as they promise to be.

New Working Papers
See working papers page for full list

Mortality from Nestlé’s Marketing of Infant Formula in
Low and Middle-Income Countries (2022)
(working paper)
Abstract: Infant formula use has been implicated in millions of infant deaths in low and middle-income countries over the past several decades, but causal evidence of formula’s link with mortality remains elusive. We combine data on over 2.6 million infant births and deaths from the Demographic and Health Surveys (DHS) across 38 countries with data on the timing of Nestlé entrance into country infant formula markets taken from annual investor reports.
Consistent with the hypothesis that formula mixed with unclean water acts as a disease vector, we find that infant mortality increased in households with unclean water sources by 19.5 per thousand births following Nestlé market entrance, but had no effect among other households. This rate is equivalent to a 27% increase in mortality and amounts to about 212,000 thousand excess deaths per year at the peak of the Nestle controversy in 1981

Road Maintenance and Local Economic Development: Evidence from Indonesia's Highways (2022)
(working paper)
Abstract: This paper estimates the local welfare impacts of road maintenance investments. We instrumentroad quality exploiting Indonesia’s two-step budgeting process for allocating funding to local roadauthorities. Using comprehensive data on road quality from 1990-2007, we find that better roadshelp manufacturers create new jobs, enabling worker transitions out of informal employment, andincreasing wages. In terms of cost of living, road quality reduces perishable food prices but also raiseshousing prices. We estimate the elasticity of household welfare with respect to road quality to be 0.16and the benefit/cost ratio for road maintenance investments to be 2.8.

Management and Altruism in Health Care Delivery (2022)
(working paper)
Abstract: We conduct a field experiment of a  management consulting intervention fort Kenyan private health care clinics. We find large improvements in management practices and a shift in preferences to be more profit oriented and less patient altruistic.  We find the program significantly reduced correct clinical case management of patient care (quality) explained mostly by the shift in preferences and quality inelastic demand. The intervention also
translated into better business performance in terms of scale, profits, and efficiency.

Trust and Savings in Financial Institutions by the Poor (2022) (working paper)
Abstract: We randomly assigned beneficiaries of a conditional cash transfer program in Peru to attend a 3 hour training session designed to build their trust in financial institutions. We find that the intervention: (a) significantly increased trust in banks, but had no effect on financial literacy; (b) significantly increased savings over a ten month period, and (c) had no effect of the use of accounts for transactions. The increase in savings is a 1.4 percentage point increase in the savings rate out of the cash transfer deposits, and a 0.4 percentage point increase in the savings rate out of household income.

Promoting Parental Involvement in Schools: Evidence from Two Randomized Experiments (2022) (working paper)
Abstract: We examine the effects of a parental involvement program in Mexico, which provides parent associations with grants and information. Grants to parent associations did not improve educational outcomes. Information to parent associations reduced disciplinary actions in schools, mainly by increasing parental involvement in schools and changing parenting behavior at home. The divergent results from grants and information are partly explained by significant changes in perceptions of trust between parents and teachers.

Effect of the Jamaica Early Childhood Stimulation Intervention on Labor Market Outcomes at Age 31 (2022)
(working paper, appendicies)
Abstract: We report the labor market effects of the Jamaica Early Childhood Stimulation intervention at age 31. Implemented in 1987-1989, treatment consisted of a two-year home-based intervention designed to improve nutrition and the quality of mother-child interactions to foster cognitive, language and psycho-social skills. The original sample is 127 stunted children between 9 and 24 months old. Our study is able to track and interview 75% of the original sample 30 years after the intervention. We find large and statistically significant effects on income and schooling; the treatment group had 43% higher hourly wages and 37% higher earnings than the control group.

Financing Municipal Water and Sanitation Services in Nairobi's Informal Settlements (2022)
(working paper)
Abstract: We estimate the impacts of two interventions by Nairobi’s water and sanitation utility to improve revenue collection efficiency: (1) engagement with customers to encourage payment and (2) contract enforcement for service disconnection due to nonpayment. We find no effect of engagement but do find large effects of enforcement on payment. We also find no effect on access to water, perceptions of utility fairness or quality of service delivery, relationship between tenants and property owners, or tenant mental well-being. Together these results suggest that contract enforcement was effective at improving revenue collection efficiency without incurring large social or political costs.

Making Entrepreneurs: The Returns to Training Youth in Hard versus Soft Business Skills (2021) (working paper)
Abstract: We study an innovative mini-MBA program for high school students in Ugandan context. The program featured two treatments: (i) hard-skills - a mix of 75% hard skills and 25% soft skills; (ii) soft skills - the reverse mix. We find that both arms were effective in improving both hard and soft skills, and were rewarded with substantially higher earnings of about 30%, most of which, was generated through self-employment. Youth in both treatment groups were more likely to start enterprises, investment larger amounts of capital, and create substantially more employment. Both curricula were very cost-effective; two months worth of the extra earnings alone would exceed the cost of the program. These benefits abstract from the job- and business-creation benefits of the program, which were substantial.

Digital Collateral (2021)  (working paper)
Abstract: A new form of secured lending utilizing “digital collateral” has recently emerged, most prominently in low- and middle-income countries. Digital collateral relies on “lockout” technology, which allows the lender to temporarily disable the flow value of the collateral to the borrower without physically repossessing it. We explore this both in a model and in a field experiment using school-fee loans digitally secured with a solar home system. We find that securing a loan with digital collateral drastically reduces default rates (by 19 pp) and increases the lender’s rate of return (by 38 pp). We decompose the total effect and find that roughly one-third is attributable to adverse selection and two-thirds is attributable to moral hazard. Access to a school-fee loan significantly increases school enrollment and school-related expenditures without detrimental effects to households’ balance sheet.

Increasing Financial Inclusion and Attracting Deposits through Price Linked Savings (2021). (working paper)
Abstract: In a large field experiment in Mexico, we provide a temporary incentive to both open and use a savings account: specifically we offer prize-linked savings accounts with cash-prize lotteries, where lottery tickets are awarded as a function of savings balances. We find that 41% more accounts are opened in treatment branches than in control branches on average. Although the incentive to save is temporary as lotteries are only offered for two months, the new accounts continue to be used over time. After five years, clients who opened accounts in response to the lottery continue saving and making transactions at the same rates as those who opened accounts in control branches during the same months.