Reproducibility in machine learning for health

Matthew B.A. McDermott, Shirly Wang, Nikki Marinsek, Rajesh Ranganath, Marzyeh Ghassemi, Luca Foschini

Research output: Contribution to conferencePaperpeer-review


Machine learning algorithms designed to characterize, monitor, and intervene on human health (ML4H) are expected to perform safely and reliably when operating at scale, potentially outside strict human supervision. This requirement warrants a stricter attention to issues of reproducibility than other fields of machine learning. In this work, we conduct a systematic evaluation of over 100 recently published ML4H research papers along several dimensions related to reproducibility. We find that the field of ML4H compares poorly to more established machine learning fields, particularly concerning data and code accessibility. Finally, drawing from success in other fields of science, we propose recommendations to data providers, academic publishers, and the ML4H research community in order to promote reproducible research moving forward.

Original languageEnglish (US)
StatePublished - Jan 1 2019
Event2019 Reproducibility in Machine Learning, RML@ICLR 2019 Workshop - New Orleans, United States
Duration: May 6 2019 → …


Conference2019 Reproducibility in Machine Learning, RML@ICLR 2019 Workshop
Country/TerritoryUnited States
CityNew Orleans
Period5/6/19 → …

ASJC Scopus subject areas

  • Education
  • Computer Science Applications
  • Linguistics and Language
  • Language and Linguistics


Dive into the research topics of 'Reproducibility in machine learning for health'. Together they form a unique fingerprint.

Cite this