Support Us

Open Science in Development Engineering: A New Model for Replication

Technology

Illustrations by David Parkins – https://www.nature.com/articles/d41586-023-02997-5

Replication is a critical component of scientific credibility, as it increases confidence in the reliability of knowledge generated by original research. Few replications are completed in economics, and even fewer are published, as in the last 6 years, only 11 replication studies were published in top 11 (empirical) Economics Journals.
To understand the incentive structure for replication in the discipline, Paul Gertler, Sebastian Galiani and Mauricio Romero surveyed the editors of 11 top-tier economics journals. The survey revealed a substantial “overturn bias”, as all of the responding editors stated they would publish a replication study in principle, however less than a third would publish consider publishing a replication study that confirmed the original results. Furthermore, poor compliance with journal data transparency in economics provides another challenge for replicators, as only 14% of papers published in the top 11 economics journals shared code and data for full, “push-button” replication.
To incentivize replication, the authors propose that journals replicate papers after acceptance, and before publication. This process would require authors to submit their data and code once the paper is accepted. Journals would then perform a “push-button” replication to verify that the code executes and reproduces the tables and figures in the paper. If the code does not execute or reported results are different, editors can either ask authors to correct their errors, choose to re‐review the paper, or reject it. Full results forthcoming.

Researchers
Timeline

2015 — ongoing

Share Now

Copyright 2024. All Rights Reserved

Design & Dev by Wonderland Collective