GLOBAL POVERTY & IMPACT EVALUATION: LEARNING WHAT WORKS FOR THE WORLD’S POOR
Fall 2009 Syllabus - UC Berkeley DeCal Course
Student Facilitators: Erick Gong, Garret Christensen
Instructor of Record: Ted Miguel
Class meets Tuesdays 5-7 pm. Location TBD. Sponsored by the Center of Evaluation for Global Action
Course Content: The course will cover impact evaluation theory (causal inference, experimental design and basic statistics) as well as methods (randomization, difference-in-difference, regression discontinuity, and propensity score matching). The curriculum will be applied, with weekly case studies of field research drawn from the international development literature. Discussions of methods will include issues related to research ethics and the protection of human subjects. At the end of the course, students will have the opportunity to present their own impact evaluation research projects and get feedback from CEGA faculty members.
Audience: The course is ideally suited for graduate and advanced undergraduate students with an interested in impact evaluation. Graduate students in Public Policy, Public Health, Education, Political Science, ERG, and Sociology, and undergraduates who have taken statistics courses may benefit the most from this course. The curriculum is very applied and will be useful for students engaged in international development field projects, social entrepreneurship, and policy analysis. Please email one of the student facilitators if you have questions about whether this course is the right fit given your interests and background.
Learning Outcomes: Students who complete this course will be prepared to: 1) distinguish research-based "best practices" from those that have not been rigorously evaluated; 2) design an impact evaluation of a policy or intervention, and 3) evaluate data using a statistical software package. For students who are considering conducting an impact evaluation of a program, facilitators will provide references to technical resources (e.g. textbooks on sample design and software for power calculations) and guidelines for developing a rigorous study.
Methods of Instruction: During class, facilitators will present the main concepts in short lectures structured around case studies (suggested readings from the literature), which will also serve as the basis for class discussion and small group activities. Lectures will discuss the strongest (most rigorous) evaluation methods and the shortcomings of weak evaluation methods. Case studies will highlight research from Africa, Asia, and South America as well as the U.S. and will cover programs related to health, governance, education, and agriculture. Group work will provide hands-on experience with research design and data analysis.
Grading: Students will be graded on the following: 1) attendance, 2) participation in discussion, 3) 4 short problem sets (approx 1 hour of work each) and 4) a group presentation. Students who miss two days of lecture (not including the first week's introduction) will be in danger of failing the course. For every lecture that a student misses, the student will need to submit a one-page summary/reaction to the lecture slides or referenced papers (posted below). Depending on time availability, class size, and students' interests, group presentations will take place in the final two weeks of class.
Assignments: The problem sets are designed to teach students how to apply the four methods (randomization, difference-in-difference, regression discontinuity, and propensity score matching) using statistical software (STATA) with actual data. An example of STATA code will be provided for each problem set. Listed below are the four problem sets with the emphasis in parenthesis and the due dates.
Anticipated Schedule:
September 1: First class meeting
September 8: Introduction to impact evaluation in international development
Reading:
Case Study: conditional cash transfers in Mexico
Reading:
Case Studies: housing vouchers in the US and microfinance in South Africa
Reading:
Case Study: deworming drugs in Kenya
Reading:
Case Study: HIV prevention educational programs in Kenya
Reading:
Case Studies: scholarship program for girls in Kenya, educational finance in Chile
Reading:
Case Studies: anti-corruption programs in Indonesia and Brazil, & community-based monitoring of health clinics in Uganda
Reading:
Case studies: water infrastructure and children’s health in India & workfare in Argentina
Reading:
Reading:
Case studies: malaria eradiation in the Americas and land reform in India
Reading:
December 1: Group Presentations
December 8: Outside Speakers
Fall 2009 Syllabus - UC Berkeley DeCal Course
Student Facilitators: Erick Gong, Garret Christensen
Instructor of Record: Ted Miguel
Class meets Tuesdays 5-7 pm. Location TBD. Sponsored by the Center of Evaluation for Global Action
Course Content: The course will cover impact evaluation theory (causal inference, experimental design and basic statistics) as well as methods (randomization, difference-in-difference, regression discontinuity, and propensity score matching). The curriculum will be applied, with weekly case studies of field research drawn from the international development literature. Discussions of methods will include issues related to research ethics and the protection of human subjects. At the end of the course, students will have the opportunity to present their own impact evaluation research projects and get feedback from CEGA faculty members.
Audience: The course is ideally suited for graduate and advanced undergraduate students with an interested in impact evaluation. Graduate students in Public Policy, Public Health, Education, Political Science, ERG, and Sociology, and undergraduates who have taken statistics courses may benefit the most from this course. The curriculum is very applied and will be useful for students engaged in international development field projects, social entrepreneurship, and policy analysis. Please email one of the student facilitators if you have questions about whether this course is the right fit given your interests and background.
Learning Outcomes: Students who complete this course will be prepared to: 1) distinguish research-based "best practices" from those that have not been rigorously evaluated; 2) design an impact evaluation of a policy or intervention, and 3) evaluate data using a statistical software package. For students who are considering conducting an impact evaluation of a program, facilitators will provide references to technical resources (e.g. textbooks on sample design and software for power calculations) and guidelines for developing a rigorous study.
Methods of Instruction: During class, facilitators will present the main concepts in short lectures structured around case studies (suggested readings from the literature), which will also serve as the basis for class discussion and small group activities. Lectures will discuss the strongest (most rigorous) evaluation methods and the shortcomings of weak evaluation methods. Case studies will highlight research from Africa, Asia, and South America as well as the U.S. and will cover programs related to health, governance, education, and agriculture. Group work will provide hands-on experience with research design and data analysis.
Grading: Students will be graded on the following: 1) attendance, 2) participation in discussion, 3) 4 short problem sets (approx 1 hour of work each) and 4) a group presentation. Students who miss two days of lecture (not including the first week's introduction) will be in danger of failing the course. For every lecture that a student misses, the student will need to submit a one-page summary/reaction to the lecture slides or referenced papers (posted below). Depending on time availability, class size, and students' interests, group presentations will take place in the final two weeks of class.
Assignments: The problem sets are designed to teach students how to apply the four methods (randomization, difference-in-difference, regression discontinuity, and propensity score matching) using statistical software (STATA) with actual data. An example of STATA code will be provided for each problem set. Listed below are the four problem sets with the emphasis in parenthesis and the due dates.
Problem Set 1 (Randomized Evaluations): Handed Out Sept 22nd, Due Oct 3rd.The group presentation will involve a brief research proposal. Additional details of this will be presented in class.
Problem Set 2 (Regression Discontinuity): Handed Out Oct 13, Due Oct 27th.
Problem Set 3 (Matching / Propensity Score): Handed Out Oct 27th, Due Nov 10th.
Problem Set 4 (Difference-in-Difference): Handed out Nov 10th, Due Nov 24th.
Anticipated Schedule:
September 1: First class meeting
September 8: Introduction to impact evaluation in international development
Reading:
Banerjee, Abhijit et al. Making Aid Work. The MIT Press. 2007.September 15: Randomized Evaluations 1: Introduction, methodology, and the basic econometrics
Duflo, Esther. Scaling Up and Evaluation. Annual World Bank Conference on Development Economics, 2004.
Easterly, William. Can the West Save Africa? Journal of Economic Literature, 2009.
Case Study: conditional cash transfers in Mexico
Reading:
Duflo, Esther, Rachel Glennerster, and Michael Kremer. Using Randomization in Development Economics Research: A Toolkit. Poverty Action Lab White Paper, MIT.September 22: Randomized Evaluations II: Applications
Schultz, T. Paul. School Subsidies for the Poor: Evaluating the Mexican Progresa Poverty Program. Journal of Development Economics. June 2004, 199-250.
Fisman, Raymond and Edward Miguel. Chapter 8. In Economic Gangsters. Princeton, New Jersey: Princeton University Press, 2008.
Case Studies: housing vouchers in the US and microfinance in South Africa
Reading:
Kling, Jeffrey, Jeffrey Liebman, and Lawrence Katz. Experimental Analysis of Neighborhood Effects. Econometrica, January 2007, 83-119.September 29: Randomized Evaluations III: Complications, Externalities
Karlan, Dean and Jonathan Zinman. Credit Elasticities in Less Developed Countries: Implications for Microfinance. American Economic Review, forthcoming.
Case Study: deworming drugs in Kenya
Reading:
Kremer, Michael and Edward Miguel. Worms: Identifying Impacts on Education and Health in the Presence of Treatment Externalities. Econometrica. January 2004, 159-217.October 6: Research Ethics & Data Sources
Kremer, Michael and Edward Miguel. The Illusion of Sustainability. Quarterly Journal of Economics. August 2007, 1007-1065.
Case Study: HIV prevention educational programs in Kenya
Reading:
Dupas, Pascaline. Relative Risks and the Market for Sex: Teenage Pregnancy, HIV, and Partner Selection in Kenya. Working paper.October 13: Regression Discontinuity
Case Studies: scholarship program for girls in Kenya, educational finance in Chile
Reading:
Unpublished results from follow-up on a girl’s merit scholarship program. For a description of the intervention, see Kremer, Michael et al. Incentives to Learn. NBER Working Paper #10971. 2004.October 20: External Validity
Chay, Ken et al. The Central Role of Noise in Evaluating Interventions that Use Test Scores to Rank Schools. American Economic Review. September 2005, 1237-1258.
Case Studies: anti-corruption programs in Indonesia and Brazil, & community-based monitoring of health clinics in Uganda
Reading:
Olken, Benjamin. Monitoring Corruption: Evidence from a Field Experiment in Indonesia. Journal of Political Economy. April 2007: 200-249.October 27: Matching, Propensity Score
Ferraz, Claudio and Frederico Finan. Exposing Corrupt Politicians: The Effects of Brazil’s Publicly Released Audits on Electoral Outcomes. Quarterly Journal of Economics, May 2008, 703-745.
Bjorkman, Martina and Jakob Svensson. Power to the People: Evidence from a Randomized Field Experiment of a Community-Based Monitoring Project in Uganda. Community-Based Monitoring of Primary Health Care PCEPR Working Paper # 6344. June 2007.
Case studies: water infrastructure and children’s health in India & workfare in Argentina
Reading:
Jalan, Jyotsna and Martin Ravallion. Does Piped Water Reduce Diarrhea for Children in Rural India? Journal of Econometrics. January 2003, 153-173.November 3: Data Quality, Logistics
Jalan, Jyotsna and Martin Ravallion. Estimating the Benefit Incidence of an Antipoverty Program by Propensity Score Matching. Journal of Business and Economic Statistics. January 2003, 19-30.
Reading:
Baird, Sarah, Joan Hamory, and Edward Miguel. Tracking, Attrition and Data Quality in the Kenya Life Panel Survey Round 1 Working paper.November 10: Differences in Differences
Case studies: malaria eradiation in the Americas and land reform in India
Reading:
Bleakley, Hoyt. Malaria Eradication in the Americas: A Retrospective Analysis of Childhood Exposure. Working paper.November 17: Power Calculations
Besley, Timothy and Robin Burgess. Land Reform, Poverty Reduction, and Growth: Evidence from India. Quarterly Journal of Economics. May 2000, 389-430.
Data exercise based on the Kenya Rural Water ProjectNovember 24: Review / Further Topics
December 1: Group Presentations
December 8: Outside Speakers
For more information regarding research or employment opportunities, please visit CEGA's website, Innovations for Poverty Action's job listings, or the Poverty Action Lab's job listings. Or see Impact Evaluation at the World Bank, the International Initiative for Impact
Evalutaion (3ie), or the Network of Networks on Impact Evaluation (Nonie).