GLOBAL POVERTY & IMPACT EVALUATION: LEARNING WHAT WORKS FOR THE WORLD’S POOR
Fall 2008 Syllabus - UC Berkeley DeCal Course
Student Facilitators: Clair Null, Kenny Ajayi, Garret Christensen
Instructor of Record: Ted Miguel, David Levine
Sponsored by the Center of Evaluation for Global Action
Course Content: The course will cover impact evaluation theory (causal inference, experimental design and basic statistics) as well as methods (randomization, difference-in-difference, regression discontinuity, and propensity score matching). The curriculum will be very applied, with weekly case studies of field research drawn from the international development literature. Discussions of methods will include issues related to research ethics and the protection of human subjects. At the end of the course, students will have the opportunity to present their own impact evaluation research projects and get feedback from CEGA faculty members.
Learning Outcomes: The primary objective of the course is to provide participants with the ability to 1) distinguish research-based “best practices” from those that have not been rigorously evaluated; and 2) understand the value and practice of impact evaluation within the development community. For more motivated students, who are considering conducting an impact evaluation of a program, facilitators will provide references to technical resources (e.g. textbooks on sample design and software for power calculations) and guidelines for developing a rigorous study.
Methods of Instruction: During class, facilitators will present the main concepts in short lectures structured around case studies (suggested Reading: s from the literature), which will also serve as the basis for class discussion and small group activities. Lectures will discuss the strongest (most
rigorous) evaluation methods and the shortcomings of weak evaluation methods. Case studies will highlight research from Africa, Asia, and South America as well as the U.S. and will cover programs related to health, governance, education, and agriculture. Group work will provide hands-on experience with research design and data analysis.
Audience: Both undergraduate and graduate students may participate in the course. It is expected that students will have basic knowledge of statistics and quantitative analysis. Again, the curriculum is very applied and will be useful for students engaged in international development
field projects or social entrepreneurship.
Anticipated Schedule:
September 8: Introduction to impact evaluation in international development
Reading:
Case Study: conditional cash transfers in Mexico
Reading:
Case Studies: housing vouchers in the US, microfinance in South Africa, and agriculture in Kenya
Reading:
Case Study: deworming drugs in Kenya
Reading:
Case Study: HIV prevention educational programs in Kenya
Reading:
Case Studies: scholarship program for girls in Kenya, educational finance in Chile
Reading:
Case Studies: anti-corruption programs in Indonesia and Brazil, & community-based monitoring of health clinics in Uganda
Reading:
Case studies: water infrastructure and children’s health in India & workfare in Argentina
Reading:
Reading:
Case studies: malaria eradiation in the Americas and land reform in India
Reading:
Materials:
Fall 2008 Syllabus - UC Berkeley DeCal Course
Student Facilitators: Clair Null, Kenny Ajayi, Garret Christensen
Instructor of Record: Ted Miguel, David Levine
Sponsored by the Center of Evaluation for Global Action
Course Content: The course will cover impact evaluation theory (causal inference, experimental design and basic statistics) as well as methods (randomization, difference-in-difference, regression discontinuity, and propensity score matching). The curriculum will be very applied, with weekly case studies of field research drawn from the international development literature. Discussions of methods will include issues related to research ethics and the protection of human subjects. At the end of the course, students will have the opportunity to present their own impact evaluation research projects and get feedback from CEGA faculty members.
Learning Outcomes: The primary objective of the course is to provide participants with the ability to 1) distinguish research-based “best practices” from those that have not been rigorously evaluated; and 2) understand the value and practice of impact evaluation within the development community. For more motivated students, who are considering conducting an impact evaluation of a program, facilitators will provide references to technical resources (e.g. textbooks on sample design and software for power calculations) and guidelines for developing a rigorous study.
Methods of Instruction: During class, facilitators will present the main concepts in short lectures structured around case studies (suggested Reading: s from the literature), which will also serve as the basis for class discussion and small group activities. Lectures will discuss the strongest (most
rigorous) evaluation methods and the shortcomings of weak evaluation methods. Case studies will highlight research from Africa, Asia, and South America as well as the U.S. and will cover programs related to health, governance, education, and agriculture. Group work will provide hands-on experience with research design and data analysis.
Audience: Both undergraduate and graduate students may participate in the course. It is expected that students will have basic knowledge of statistics and quantitative analysis. Again, the curriculum is very applied and will be useful for students engaged in international development
field projects or social entrepreneurship.
Anticipated Schedule:
September 8: Introduction to impact evaluation in international development
Reading:
Banerjee, Abhijit et al. Making Aid Work. The MIT Press. 2007.September 15: Randomized Evaluations 1: Introduction, methodology, and the basic econometrics | Lecture Slides
Duflo, Esther. Scaling Up and Evaluation. Annual World Bank Conference on Development Economics, 2004.
Case Study: conditional cash transfers in Mexico
Reading:
Duflo, Esther, Rachel Glennerster, and Michael Kremer. Using Randomization in Development Economics Research: A Toolkit. Poverty Action Lab White Paper, MIT.September 22: Randomized Evaluations II: Applications | Lecture Slides
Schultz, T. Paul. School Subsidies for the Poor: Evaluating the Mexican Progresa Poverty Program. Journal of Development Economics. June 2004, 199-250.
Case Studies: housing vouchers in the US, microfinance in South Africa, and agriculture in Kenya
Reading:
Kling, Jeffrey, Jeffrey Liebman, and Lawrence Katz. Experimental Analysis of Neighborhood Effects. Econometrica, January 2007, 83-119.September 29: Randomized Evaluations III: Complications, Externalities | Lecture Outline
Karlan, Dean and Jonathan Zinman. Credit Elasticities in Less Developed Countries: Implications for Microfinance. American Economic Review, forthcoming.
Duflo, Esther, Michael Kremer and John Robinson. How high are rates of return to fertilizer? Evidence from field experiments in Kenya. American Economic Review, May 2008, 482-488.
Case Study: deworming drugs in Kenya
Reading:
Kremer, Michael and Edward Miguel. Worms: Identifying Impacts on Education and Health in the Presence of Treatment Externalities. Econometrica. January 2004, 159-217.October 6: Research Ethics | Lecture Slides | Group Worksheet
Kremer, Michael and Edward Miguel. The Illusion of Sustainability. Quarterly Journal of Economics. August 2007, 1007-1065.
Case Study: HIV prevention educational programs in Kenya
Reading:
Dupas, Pascaline. Relative Risks and the Market for Sex: Teenage Pregnancy, HIV, and Partner Selection in Kenya. Working paper.October 13: Regression Discontinuity | Intro Worksheet | Lecture Slides
Case Studies: scholarship program for girls in Kenya, educational finance in Chile
Reading:
Unpublished results from follow-up on a girl’s merit scholarship program. For a description of the intervention, see Kremer, Michael et al. Incentives to Learn. NBER Working Paper #10971. 2004.October 20: External Validity | Lecture Notes | Case Studies
Chay, Ken et al. The Central Role of Noise in Evaluating Interventions that Use Test Scores to Rank Schools. American Economic Review. September 2005, 1237-1258.
Case Studies: anti-corruption programs in Indonesia and Brazil, & community-based monitoring of health clinics in Uganda
Reading:
Olken, Benjamin. Monitoring Corruption: Evidence from a Field Experiment in Indonesia. Journal of Political Economy. April 2007: 200-249.October 27: Matching, Propensity Score | Lecture Slides | Lecture Exercise
Ferraz, Claudio and Frederico Finan. Exposing Corrupt Politicians: The Effects of Brazil’s Publicly Released Audits on Electoral Outcomes. Quarterly Journal of Economics, May 2008, 703-745.
Bjorkman, Martina and Jakob Svensson. Power to the People: Evidence from a Randomized Field Experiment of a Community-Based Monitoring Project in Uganda. Community-Based Monitoring of Primary Health Care PCEPR Working Paper # 6344. June 2007.
Case studies: water infrastructure and children’s health in India & workfare in Argentina
Reading:
Jalan, Jyotsna and Martin Ravallion. Does Piped Water Reduce Diarrhea for Children in Rural India? Journal of Econometrics. January 2003, 153-173.November 3: Data Quality, Logistics | Worksheet | Lecture Slides
Jalan, Jyotsna and Martin Ravallion. Estimating the Benefit Incidence of an Antipoverty Program by Propensity Score Matching. Journal of Business and Economic Statistics. January 2003, 19-30.
Reading:
Baird, Sarah, Joan Hamory, and Edward Miguel. Tracking, Attrition and Data Quality in the Kenya Life Panel Survey Round 1 Working paper.November 10: Differences in Differences | Worksheet | Lecture Slides
Case studies: malaria eradiation in the Americas and land reform in India
Reading:
Bleakley, Hoyt. Malaria Eradication in the Americas: A Retrospective Analysis of Childhood Exposure. Working paper.November 17: Power Calculations | Lecture Slides | Instructions
Besley, Timothy and Robin Burgess. Land Reform, Poverty Reduction, and Growth: Evidence from India. Quarterly Journal of Economics. May 2000, 389-430.
Data exercise based on the Kenya Rural Water Project:November 24: Summmary / Group Presentations
Excel sheet with calculations yet to be completed
Excel sheet with calculations completed
Materials:
Methods Summary SlidesDecember 1: Group Presentations
Implementation Issues Slides
Guidance Questions
Summary Table
Glossary
David Levine: List of Questions for an NGOFor more information regarding research or employment opportunities, please visit CEGA's website, Innovations for Poverty Action's job listings, or the Poverty Action Lab's job listings. Or see Impact Evaluation at the World Bank, the International Initiative for Impact Evalutaion (3ie), or the Network of Networks on Impact Evaluation (Nonie).
Martha Saavedra: Football for Hope
Miguel Almunia: Health Education for Microfinance Clients
Peter Kelly: Philosophy of Impact Evaluations
Willa Friedman: Education and HIV Prevention