The design of the New York school choice scholarships program evaluation

INTRODUCTION One of Don Campbell's many influential contributions was to the design of studies to estimate causal effects (e.g., Campbell & Stanley, 1966). He had particular interest in the trade-offs between matching and covariance adjustments (e.g., Campbell & Erlebacher, 1970; Cook & Campbell, 1979). One of the authors (Rubin), in fact, had his first conversation with Don on the topic, more than a quarter of a century ago, having recently completed his Ph.D. thesis under the direction of W. G. Cochran on the potential benefits of matching in observational studies. That author believes that the topic of this chapter, using matching in randomized experiments, would have been of great interest to Don and that this chapter would have benefited from his insightful comments. Moreover, we hope that he would have been pleased to see our example of an educational evaluation that did not have to rely on quasi-experimental techniques. Randomized designs have been recognized since the ground-breaking work of R. A. Fisher in the early part of the 20th century as the most principled way to identify empirically causal relationships between treatments and outcomes. The strength of the randomized design lies in its ability to create treatment groups that have similar background characteristics on average. Randomization balances not only the observed characteristics but also the unobserved characteristics of the experimental units.