This website uses cookies so that we can provide you with the best user experience possible. Cookie information is stored in your browser and performs functions such as recognizing you when you return to our website and helping our team to understand which sections of the website you find most interesting and useful.
How accurately can policymakers predict the impact of their decisions in their local content? In a new article, “Using a Multi-Site RCT to Predict Impacts for a Single Site: Do Better Data and Methods Yield More Accurate Predictions?” published by the Journal of Research on Educational Effectiveness, researchers suggest better data and methods do not always predict impact accurately. Westat researchers Elizabeth Petraglia, PhD, and Atsushi Miyaoka, MA, co-authored this article with researchers from Johns Hopkins University and George Washington University.
The focus of their research includes the analysis of 6 multisite random control trials (RCTs) plus testing of modern prediction methods—lasso regression and Bayesian Additive Regression Trees (BART).
“We saw a key factor influencing the accuracy of these predictions is the extent to which the impact varies across sites. We suggest all Multi-site RCTs should report evidence on the cross-site impact variation to help readers assess the likelihood that the average impact reported by the RCT provides reasonably accurate impact predictions for individual sites” notes Atsushi Miyaoka.