External Validity and Meta-Analysis

Tara Slough, Scott A. Tyson

    Research output: Contribution to journalArticlepeer-review

    Abstract

    Meta-analysis is a method that combines estimates from studies conducted on different samples, in different contexts, or at different times. Social scientists increasingly use meta-analyses to aggregate evidence and learn about general substantive phenomena. We develop a framework to examine the theoretical foundations of meta-analysis, with emphasis on clarifying the role of external validity. We identify the conditions under which multiple studies are target-equivalent, meaning they identify the same empirical target. Our main result shows that external validity and harmonization, in comparisons made and how outcomes are measured, are necessary and sufficient for target-equivalence. We examine common formulations of meta-analysis—fixed- and random-effects models—developing the theoretical assumptions that underpin them and providing design-based identification results for these models. We then provide practical guidance based on our framework and results. Our results reveal limits to agnostic approaches to the combination of causal evidence from multiple studies.

    Original languageEnglish (US)
    JournalAmerican Journal of Political Science
    DOIs
    StateAccepted/In press - 2022

    ASJC Scopus subject areas

    • Sociology and Political Science
    • Political Science and International Relations

    Fingerprint

    Dive into the research topics of 'External Validity and Meta-Analysis'. Together they form a unique fingerprint.

    Cite this