Improved Site Selection Can Strengthen the Real-World Value of Research Findings

A new study published in the Journal of Research on Educational Effectiveness offers practical guidance for designing randomized controlled trials (RCTs) that produce findings more likely to reflect real-world populations, helping policymakers and practitioners make better-informed decisions. Westat’s Emily Diaz, PhD, a Senior Research Associate for Education and Social Policy, is a coauthor.

Using simulations based on the Head Start program, researchers examined how different methods of selecting study sites affect the extent to which trial results can be generalized beyond the participating sites. The analysis found that purposive site selection consistently introduces bias, whereas simple random selection generally reduces bias. Stratified random site selection produces the least external validity bias under the conditions examined, although its performance depends on access to information about key factors before sites are selected.

The findings provide valuable insights for researchers planning future RCTs and highlight approaches that can improve the applicability of research evidence to broader populations.

Learn More

External Validity Bias of Purposive and Random Site Selection When Sites Can Opt Out: Evidence from the Head Start Program

Authors: Robert B. Olsen, Stephen H. Bell, and Emily Diaz