Balancing Rigor, Replication, and Relevance: A Case for Multiple-Cohort, Longitudinal Experiments
Citation:
Blazar DL, Kraft MA. Balancing Rigor, Replication, and Relevance: A Case for Multiple-Cohort, Longitudinal Experiments. AERA Open. 2019;5 (3) :1-18.
Abstract:
Over the past 15 years, the education research community has advocated for the application of more rigorous research designs that support causal inferences, for research that provides more generalizable results across settings, and for the value of research-practice partnerships that inform the design of local programs and policies. However, these goals are often in tension with each other. We propose a research design – the multi-cohort, longitudinal experimental (MCLE) design – as one approach to balancing these competing goals of high-quality research. We illustrate the uses and benefits of MCLEs with an example from a research-practice partnership aimed at evaluating the effect of a teacher coaching program. We find that the coaching program failed to replicate its effectiveness with an initial cohort, likely due to changes in personnel, duration, and content. Our analyses can help researchers weigh the tradeoffs of different design features of MCLEs.
