Bias amplification and bias unmasking

Joel A. Middleton, Marc A. Scott, Ronli Diakow, Jennifer L. Hill

Research output: Contribution to journalArticle

Abstract

In the analysis of causal effects in non-experimental studies, conditioning on observable covariates is one way to try to reduce unobserved confounder bias. However, a developing literature has shown that conditioning on certain covariates may increase bias, and the mechanisms underlying this phenomenon have not been fully explored. We add to the literature on bias-increasing covariates by first introducing a way to decompose omitted variable bias into three constituent parts: bias due to an unobserved confounder, bias due to excluding observed covariates, and bias due to amplification. This leads to two important findings. Although instruments have been the primary focus of the bias amplification literature to date, we identify the fact that the popular approach of adding group fixed effects can lead to bias amplification as well. This is an important finding because many practitioners think that fixed effects are a convenient way to account for any and all group-level confounding and are at worst harmless. The second finding introduces the concept of bias unmasking and shows how it can be even more insidious than bias amplification in some cases. After introducing these new results analytically, we use constructed observational placebo studies to illustrate bias amplification and bias unmasking with real data. Finally, we propose a way to add bias decomposition information to graphical displays for sensitivity analysis to help practitioners think through the potential for bias amplification and bias unmasking in actual applications.

Original languageEnglish (US)
Pages (from-to)307-323
Number of pages17
JournalPolitical Analysis
Volume24
Issue number3
DOIs
StatePublished - 2016

Fingerprint

trend
conditioning
Group
literature

ASJC Scopus subject areas

  • Sociology and Political Science

Cite this

Bias amplification and bias unmasking. / Middleton, Joel A.; Scott, Marc A.; Diakow, Ronli; Hill, Jennifer L.

In: Political Analysis, Vol. 24, No. 3, 2016, p. 307-323.

Research output: Contribution to journalArticle

Middleton, Joel A. ; Scott, Marc A. ; Diakow, Ronli ; Hill, Jennifer L. / Bias amplification and bias unmasking. In: Political Analysis. 2016 ; Vol. 24, No. 3. pp. 307-323.
@article{db14351a254241abba2eda1e2a843973,
title = "Bias amplification and bias unmasking",
abstract = "In the analysis of causal effects in non-experimental studies, conditioning on observable covariates is one way to try to reduce unobserved confounder bias. However, a developing literature has shown that conditioning on certain covariates may increase bias, and the mechanisms underlying this phenomenon have not been fully explored. We add to the literature on bias-increasing covariates by first introducing a way to decompose omitted variable bias into three constituent parts: bias due to an unobserved confounder, bias due to excluding observed covariates, and bias due to amplification. This leads to two important findings. Although instruments have been the primary focus of the bias amplification literature to date, we identify the fact that the popular approach of adding group fixed effects can lead to bias amplification as well. This is an important finding because many practitioners think that fixed effects are a convenient way to account for any and all group-level confounding and are at worst harmless. The second finding introduces the concept of bias unmasking and shows how it can be even more insidious than bias amplification in some cases. After introducing these new results analytically, we use constructed observational placebo studies to illustrate bias amplification and bias unmasking with real data. Finally, we propose a way to add bias decomposition information to graphical displays for sensitivity analysis to help practitioners think through the potential for bias amplification and bias unmasking in actual applications.",
author = "Middleton, {Joel A.} and Scott, {Marc A.} and Ronli Diakow and Hill, {Jennifer L.}",
year = "2016",
doi = "10.1093/pan/mpw015",
language = "English (US)",
volume = "24",
pages = "307--323",
journal = "Political Analysis",
issn = "1047-1987",
publisher = "Oxford University Press",
number = "3",

}

TY - JOUR

T1 - Bias amplification and bias unmasking

AU - Middleton, Joel A.

AU - Scott, Marc A.

AU - Diakow, Ronli

AU - Hill, Jennifer L.

PY - 2016

Y1 - 2016

N2 - In the analysis of causal effects in non-experimental studies, conditioning on observable covariates is one way to try to reduce unobserved confounder bias. However, a developing literature has shown that conditioning on certain covariates may increase bias, and the mechanisms underlying this phenomenon have not been fully explored. We add to the literature on bias-increasing covariates by first introducing a way to decompose omitted variable bias into three constituent parts: bias due to an unobserved confounder, bias due to excluding observed covariates, and bias due to amplification. This leads to two important findings. Although instruments have been the primary focus of the bias amplification literature to date, we identify the fact that the popular approach of adding group fixed effects can lead to bias amplification as well. This is an important finding because many practitioners think that fixed effects are a convenient way to account for any and all group-level confounding and are at worst harmless. The second finding introduces the concept of bias unmasking and shows how it can be even more insidious than bias amplification in some cases. After introducing these new results analytically, we use constructed observational placebo studies to illustrate bias amplification and bias unmasking with real data. Finally, we propose a way to add bias decomposition information to graphical displays for sensitivity analysis to help practitioners think through the potential for bias amplification and bias unmasking in actual applications.

AB - In the analysis of causal effects in non-experimental studies, conditioning on observable covariates is one way to try to reduce unobserved confounder bias. However, a developing literature has shown that conditioning on certain covariates may increase bias, and the mechanisms underlying this phenomenon have not been fully explored. We add to the literature on bias-increasing covariates by first introducing a way to decompose omitted variable bias into three constituent parts: bias due to an unobserved confounder, bias due to excluding observed covariates, and bias due to amplification. This leads to two important findings. Although instruments have been the primary focus of the bias amplification literature to date, we identify the fact that the popular approach of adding group fixed effects can lead to bias amplification as well. This is an important finding because many practitioners think that fixed effects are a convenient way to account for any and all group-level confounding and are at worst harmless. The second finding introduces the concept of bias unmasking and shows how it can be even more insidious than bias amplification in some cases. After introducing these new results analytically, we use constructed observational placebo studies to illustrate bias amplification and bias unmasking with real data. Finally, we propose a way to add bias decomposition information to graphical displays for sensitivity analysis to help practitioners think through the potential for bias amplification and bias unmasking in actual applications.

UR - http://www.scopus.com/inward/record.url?scp=84989244309&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84989244309&partnerID=8YFLogxK

U2 - 10.1093/pan/mpw015

DO - 10.1093/pan/mpw015

M3 - Article

AN - SCOPUS:84989244309

VL - 24

SP - 307

EP - 323

JO - Political Analysis

JF - Political Analysis

SN - 1047-1987

IS - 3

ER -