A survey of domain knowledge elicitation in applied machine learning

Daniel Kerrigan, Jessica Hullman, Enrico Bertini

    Research output: Contribution to journalArticlepeer-review


    Eliciting knowledge from domain experts can play an important role throughout the machine learning process, from correctly specifying the task to evaluating model results. However, knowledge elicitation is also fraught with challenges. In this work, we consider why and how machine learning researchers elicit knowledge from experts in the model development process. We develop a taxonomy to characterize elicitation approaches according to the elicitation goal, elicitation target, elicitation process, and use of elicited knowledge. We analyze the elicitation trends observed in 28 papers with this taxonomy and identify opportunities for adding rigor to these elicitation approaches. We suggest future directions for research in elicitation for machine learning by highlighting avenues for further exploration and drawing on what we can learn from elicitation research in other fields.

    Original languageEnglish (US)
    Article number73
    JournalMultimodal Technologies and Interaction
    Issue number12
    StatePublished - Dec 2021


    • Domain expert
    • Domain knowledge
    • Elicitation
    • Expert knowledge
    • Machine learning

    ASJC Scopus subject areas

    • Neuroscience (miscellaneous)
    • Human-Computer Interaction
    • Computer Science Applications
    • Computer Networks and Communications


    Dive into the research topics of 'A survey of domain knowledge elicitation in applied machine learning'. Together they form a unique fingerprint.

    Cite this