High false positive and false negative reporting probabilities (FPRP and FNRP) reduce the accuracy of the available research in a particular field, undermining the value of evidence to inform policy. In this study, Eva Vivalt and Aidan Coville examined why and how a positive result can be overstated.
The study leveraged AidGrade’s database of impact evaluation results, gathered in the course of meta-analyses of 20 different types of interventions in development economics, covering 635 studies. With this data, the authors gathered up to five predictions from 125 experts covering 130 different results across typical interventions in development economics and generated estimates of the likelihood of false reporting and exaggeration of the effects of significant results. They found that the majority of the reviewed studies are generally credible, particularly in comparison to other disciplines. The study’s broader contribution shows how analysis of study power and the systematic collection of priors can help assess the quality of research.
Copyright 2020. All Rights Reserved
Design & Dev by Wonderland Collective