TY - JOUR
T1 - Prevention science and child/youth development
T2 - Randomized explanatory trials for integrating theory, method, and analysis in program evaluation
AU - Jaccard, James
AU - Bo, Ai
N1 - Publisher Copyright:
© 2018 by the Society for Social Work and Research. All rights reserved.
PY - 2018/12/1
Y1 - 2018/12/1
N2 - We describe randomized explanatory trials (RETs), a framework for evaluating interventions to prevent adolescent problem behaviors. The approach maps intervention components onto hypothesized mediators of program effects and then uses structural equation modeling to evaluate whether the program changed those mediators and if assumptions of mediator relevance are viable. We review and explain key concepts related to theoretical issues of choosing and conceptualizing mediators, mapping structural relations among mediators, considering cascading effects, and incorporating theories of treatment duration and decay. We also explore methodological issues, including the choice of time intervals between assessments, the use of instrumental variables, methods for dealing with measurement error, and sample-size planning. In addition, we discuss analytic issues including use of structural equation modeling rather than ANOVA, ANCOVA and logistic/OLS regression; how to handle large numbers of mediators; incorporating covariates; use of per-protocol vs. intent-to-treat analyses; quantifying and testing mediation; and use of causal inference approaches to mediation. Because RETs can identify program components that are successful in bringing about change and why— providing more useful evaluations than outcome-only designs—we argue that RETs rather than randomized controlled trials should be the gold standard for program evaluation.
AB - We describe randomized explanatory trials (RETs), a framework for evaluating interventions to prevent adolescent problem behaviors. The approach maps intervention components onto hypothesized mediators of program effects and then uses structural equation modeling to evaluate whether the program changed those mediators and if assumptions of mediator relevance are viable. We review and explain key concepts related to theoretical issues of choosing and conceptualizing mediators, mapping structural relations among mediators, considering cascading effects, and incorporating theories of treatment duration and decay. We also explore methodological issues, including the choice of time intervals between assessments, the use of instrumental variables, methods for dealing with measurement error, and sample-size planning. In addition, we discuss analytic issues including use of structural equation modeling rather than ANOVA, ANCOVA and logistic/OLS regression; how to handle large numbers of mediators; incorporating covariates; use of per-protocol vs. intent-to-treat analyses; quantifying and testing mediation; and use of causal inference approaches to mediation. Because RETs can identify program components that are successful in bringing about change and why— providing more useful evaluations than outcome-only designs—we argue that RETs rather than randomized controlled trials should be the gold standard for program evaluation.
KW - Adolescent problem behavior
KW - Randomized trials
KW - Youth development
UR - http://www.scopus.com/inward/record.url?scp=85057039758&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85057039758&partnerID=8YFLogxK
U2 - 10.1086/701388
DO - 10.1086/701388
M3 - Article
AN - SCOPUS:85057039758
SN - 2334-2315
VL - 9
SP - 651
EP - 687
JO - Journal of the Society for Social Work and Research
JF - Journal of the Society for Social Work and Research
IS - 4
ER -