Explanation of Variability and Removal of Confounding Factors from Data through Optimal Transport

Esteban G. Tabak, Giulio Trigila

Research output: Contribution to journalArticle

Abstract

A methodology based on the theory of optimal transport is developed to attribute variability in data sets to known and unknown factors and to remove such attributable components of the variability from the data. Denoting by x the quantities of interest and by z the explanatory factors, the procedure transforms x into filtered variables y through a z-dependent map, so that the conditional probability distributions ρ(x|z) are pushed forward into a target distribution μ(y), independent of z. Among all maps and target distributions that achieve this goal, the procedure selects the one that minimally distorts the original data: the barycenter of the ρ(x|z). Connections are found to unsupervised learning and to fundamental problems in statistics such as conditional density estimation and sampling. Particularly simple instances of the methodology are shown to be equivalent to k-means and principal component analysis. An application is shown to a time series of ground temperature hourly data across the United States.

Original languageEnglish (US)
Pages (from-to)163-199
Number of pages37
JournalCommunications on Pure and Applied Mathematics
Volume71
Issue number1
DOIs
StatePublished - Jan 2018

ASJC Scopus subject areas

  • Mathematics(all)
  • Applied Mathematics

Fingerprint Dive into the research topics of 'Explanation of Variability and Removal of Confounding Factors from Data through Optimal Transport'. Together they form a unique fingerprint.

  • Cite this