Abstract

U.S. research funding in education today encourages comparative designs for causal inference, especially randomized field trials (RFT’s), quasiexperiments, regression discontinuity designs, etc. As in medicine, there are ethical concerns about withholding or providing differential interventions; as well as practical political concerns about maintaining experimental control (treatment fidelity, compliance with randomization, etc.) in social experiments. The need for adequate sample sizes to detect educational treatment effects often requires researchers to accumulate data across several different levels: students, teachers, coaches, principals, central office, etc.) in an educational system. Maintaining mutually reinforcing partnerships that support experimental control at each level raises special challenges. We explore these challenges in our ongoing Institute of Educational Sciences (IES) funded RFT designed to evaluate a coach training intervention in terms of student outcomes. Some of the challenges can be met by fairly straightforward design, management and statistical adjustment strategies. Others raise issues about what can be learned from controlled comparative studies designed to evaluate system-changing interventions.

The Psychometrics Centre is grateful to the MRC Biostatistics Unit for co-hosting this event.