RCTs in Practice: What Happens When a Study Ends, and What Do We Owe Those Who Participate?

Melissa Graboyes, the Principal Investigator of a National Science Foundation’s Ethical and Responsible Research grant, and colleagues Elyse St. Sauver and Mira Cross share what motivated the study that seeks to establish norms and best practices for returning research results to participants. If you are interested in sharing your views about this topic, either in person or on Zoom, please sign up HERE.

In 2019, the Nobel prize in economics was awarded to the development economists Abhijit Banerjee, Esther Duflo, and Michael Kremer for their experimental approach to alleviating global poverty, which has transformed the field of development economics. To answer questions about the best way to fight global poverty, in the 1990s and early 2000s, the Laureates introduced a new approach: carefully designed field experiments. Those field experiments were typically randomized controlled trials (RCTs), in which researchers tried to establish causality by controlling all elements except that being investigated. 

Initially adapted from the medical field, the field has evolved rapidly: in just under two decades, RCTs became a commonly accepted tool in economics. By 2014, it was estimated that hundreds of RCTs had been carried out in the prior decade (Miguel et al. 2014). At the start of 2023, the American Economic Association’s registry of RCTs listed more than 7,400 studies in 164 countries. 

Despite the widespread adoption of RCTs by economists, there has been limited discussion about the ethical questions they raise, particularly in the context of low-income and vulnerable populations. The discussions that have occurred have focused on balancing research needs, informed consent, demarcations between researchers and implementers, and enumerator wellbeing.

This has been particularly true about whether — and to what extent — there is an ethical or practical need to return results to study participants, in addition to traditional forms of dissemination in academia. From an ethics perspective, returning results may provide a valuable benefit to participants, show respect for persons, and aligns with many participants’ expectations of a respectful research encounter. Practically, returning results offers a clear conclusion to the research process, contributes to good relations between researchers and local populations, and may offer an avenue to generate local feedback on those results.

Our Approach

Our new, multi-year project, funded by the National Science Foundation’s Ethical and Responsible Research program area, will take a mixed-methods approach, collecting both quantitative survey data and qualitative interview data from East African study participants and development economists to understand what’s involved in returning results to participants. From development economists, we’ll seek to learn about their related experiences and perceptions: whether they ever returned (or tried to return) results in their projects, how they perceive barriers, and what conditions might support their implementation of returning results.

Among East African participants, we will collect information about whether they have received results from past research projects, why they might be interested in receiving results, what form those results should take, and how such knowledge might be useful. In keeping with our ethos, we hope to regularly share findings publicly, including back with all respondents. 

These results will inform a draft guide that will compile what we have learned about how to productively, effectively, and efficiently return research results to participants. We expect that document to be a starting point for an ongoing conversation about practical, cost-conscious ways of returning results to participants and about the ethics and obligations of the research encounter. Through this process of learning from researchers and participants, regular sharing of our findings with economists and research participants, and creating a guide for how to return results, we hope to encourage new norms around local dissemination of research results. We believe that the current lack of sharing results is an important issue of research ethics, relating to core principles of respect for persons, beneficence, and justice. 

Is change possible?

CEGA and study co-PI Edward Miguel have demonstrated how research practices can change positively. The field of economics is already changing, including through rapid improvements in research transparency and reproducibility of RCTs through the adoption of study registration and pre-analysis plans (Olken 2015; Duflo et al. 2020; Miguel 2021). The CEGA-incubated BITSS program (Berkeley Initiative for Transparency in the Social Sciences), initiated a discipline-wide dialogue on open science, which was inclusive and iterative, and ultimately led to the creation and normalization of transparency norms. The first two papers with pre-analysis plans were published in 2012 (Finkelstein et al. 2012; Casey et al. 2012); a decade later, there are hundreds of papers that have adopted this practice (Miguel 2021). While pre-analysis plans remain controversial in the wider field of economics, a recent survey indicates that 80% of development economists view them favorably (Swanson et al. 2020).  

In late 2024, our team began conducting hundreds of surveys and interviews with development economists and East African research participants. We want to hear about their expectations, preferences, and experiences in relation to returning research results. If you are interested in sharing your views about this topic in a short online survey or would like to be interviewed, either in person or on zoom, please sign up HERE.