Publication Bias and Editorial Statement on Negative Findings

In February 2015, the editors of eight health economics journals sent out an editorial statement which aims to reduce the incentives to engage in specification searching and reminds referees to accept studies that: “have potential scientific and publication merit regardless of whether such studies’ empirical findings do or do not reject null hypotheses that may be specified.” Abel Brodeur and Cristina Blanco-Perez collected z-statistics from the Journal of Health Economics, Health Economics and a control journal and compared the distribution of tests before and after the editorial statement. The editorial statement decreased by about 20% the number of test statistics that are statistically significant at the 5% level. In other words, the editorial statement shifted the distribution of tests to the left and thus decreased the extent of publication bias. This impact “intensified” over the studied period, as the decrease in the share of significant tests was larger for papers submitted after the editorial statement, than for papers submitted before the statement, but accepted after. This result provides evidence that low-cost interventions like editorial statements can induce a change in the behavior of referees and/or authors.