Thinking aloud about confusing code: A qualitative investigation of program comprehension and atoms of confusion

Dan Gopstein, Anne Laure Fayard, Sven Apel, Justin Cappos

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Abstract

    Atoms of confusion are small patterns of code that have been empirically validated to be difficult to hand-evaluate by programmers. Previous research focused on defining and quantifying this phenomenon, but not on explaining or critiquing it. In this work, we address core omissions to the body of work on atoms of confusion, focusing on the 'how' and 'why' of programmer misunderstanding. We performed a think-aloud study in which we observed programmers, both professionals and students, as they hand-evaluated confusing code. We performed a qualitative analysis of the data and found several surprising results, which explain previous results, outline avenues of further research, and suggest improvements of the research methodology. A notable observation is that correct hand-evaluations do not imply understanding, and incorrect evaluations not misunderstanding. We believe this and other observations may be used to improve future studies and models of program comprehension. We argue that thinking of confusion as an atomic construct may pose challenges to formulating new candidates for atoms of confusion. Ultimately, we question whether hand-evaluation correctness is, itself, a sufficient instrument to study program comprehension.

    Original languageEnglish (US)
    Title of host publicationESEC/FSE 2020 - Proceedings of the 28th ACM Joint Meeting European Software Engineering Conference and Symposium on the Foundations of Software Engineering
    EditorsPrem Devanbu, Myra Cohen, Thomas Zimmermann
    PublisherAssociation for Computing Machinery, Inc
    Pages605-616
    Number of pages12
    ISBN (Electronic)9781450370431
    DOIs
    StatePublished - Nov 8 2020
    Event28th ACM Joint Meeting European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2020 - Virtual, Online, United States
    Duration: Nov 8 2020Nov 13 2020

    Publication series

    NameESEC/FSE 2020 - Proceedings of the 28th ACM Joint Meeting European Software Engineering Conference and Symposium on the Foundations of Software Engineering

    Conference

    Conference28th ACM Joint Meeting European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2020
    CountryUnited States
    CityVirtual, Online
    Period11/8/2011/13/20

    Keywords

    • Atoms of Confusion
    • Program Understanding
    • Think-Aloud Study

    ASJC Scopus subject areas

    • Software

    Fingerprint Dive into the research topics of 'Thinking aloud about confusing code: A qualitative investigation of program comprehension and atoms of confusion'. Together they form a unique fingerprint.

    Cite this