Randomized controlled trials (RCTs) are an extremely valuable and increasingly popular tool for causal inference. As researchers embark on RCTs, they face many challenges in designing experiments: they must choose a sampling frame and sample size, design an intervention, and collect data, all subject to budget constraints.
Fiona Burlig, Louis Preonas, and Matt Woerman used a Monte Carlo simulation to demonstrate that, with correlated errors, traditional methods for experimental design result in experiments that are incorrectly powered with proper inference. In particular, failing to account for serial correlation yields overpowered experiments in short panels, and underpowered experiments in long panels. The authors developed a new power calculation formula to address this problem, and demonstrated its feasibility in real-world settings using datasets from a randomized experiment in China, and a high-frequency dataset of U.S. electricity consumption.
This work highlights the need to carefully consider the assumptions that will enter ex post analysis when calibrating the design of experiments ex ante. Using the tool developed by Burlig et al., researchers can properly account for serial correlation in panel data settings, and ultimately design well-powered experiments with multiple waves of data.
Copyright 2021. All Rights Reserved
Design & Dev by Wonderland Collective