A large-scale study about quality and reproducibility of jupyter notebooks

Joao Felipe Pimentel, Leonardo Murta, Vanessa Braganholo, Juliana Freire

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Jupyter Notebooks have been widely adopted by many different communities, both in science and industry. They support the creation of literate programming documents that combine code, text, and execution results with visualizations and all sorts of rich media. The self-documenting aspects and the ability to reproduce results have been touted as significant benefits of notebooks. At the same time, there has been growing criticism that the way notebooks are being used leads to unexpected behavior, encourage poor coding practices, and that their results can be hard to reproduce. To understand good and bad practices used in the development of real notebooks, we studied 1.4 million notebooks from GitHub. We present a detailed analysis of their characteristics that impact reproducibility. We also propose a set of best practices that can improve the rate of reproducibility and discuss open challenges that require further research and development.

Original languageEnglish (US)
Title of host publicationProceedings - 2019 IEEE/ACM 16th International Conference on Mining Software Repositories, MSR 2019
PublisherIEEE Computer Society
Pages507-517
Number of pages11
ISBN (Electronic)9781728134123
DOIs
StatePublished - May 2019
Event16th IEEE/ACM International Conference on Mining Software Repositories, MSR 2019 - Montreal, Canada
Duration: May 26 2019May 27 2019

Publication series

NameIEEE International Working Conference on Mining Software Repositories
Volume2019-May
ISSN (Print)2160-1852
ISSN (Electronic)2160-1860

Conference

Conference16th IEEE/ACM International Conference on Mining Software Repositories, MSR 2019
CountryCanada
CityMontreal
Period5/26/195/27/19

Keywords

  • Github
  • Jupyter notebook
  • Reproducibility

ASJC Scopus subject areas

  • Computer Science Applications
  • Software

Fingerprint Dive into the research topics of 'A large-scale study about quality and reproducibility of jupyter notebooks'. Together they form a unique fingerprint.

Cite this