Automatic Context-Aware Inference of Engagement in HMI: A Survey

Hanan Salam, Oya Celiktutan, Hatice Gunes, Mohamed Chetouani

Research output: Contribution to journalArticlepeer-review

Abstract

Engagement is the process by which participants establish, maintain, and end their perceived connection. Automatic engagement inference is one of the tasks required to develop successful human-centered HMI applications. Engagement is a multi-faceted multimodal construct requiring high accuracy in interpretating contextual, verbal and non-verbal cues, making the development of an intelligent automated engagement inference system challenging. Existing surveys concentrate on specific application settings, and a comprehensive survey covering the different engagement facets, definition and inference across various contexts is lacking. Moreover, despite the importance of context-aware modeling, the literature lacks a systematic context-aware overview on the topic. This paper presents a comprehensive survey on previous work in engagement for HMI, entailing interdisciplinary definition, engagement components, publicly available datasets, ground truth assessment, and commonly used features and methods, serving as a guide for the development of future HMI interfaces with reliable context-aware engagement inference capability. An in-depth review across embodied and disembodied interaction modes, and an emphasis on the interaction context of which engagement is studied sets apart this survey from existing ones. Our findings suggest four important directions for future research: (1) context-aware computational modeling, (2) temporal dynamics, (3) personalised computing, and (4) bias and fairness of engagement inference systems.

Original languageEnglish (US)
Pages (from-to)1-20
Number of pages20
JournalIEEE Transactions on Affective Computing
Volume15
Issue number2
DOIs
StatePublished - Apr 1 2024

Keywords

  • Computational modeling
  • Context modeling
  • Context-Aware computing
  • Human computer interaction
  • Human-robot interaction
  • Surveys
  • Task analysis
  • User experience
  • engagement detection
  • human-machine interaction
  • socially intelligent systems

ASJC Scopus subject areas

  • Software
  • Human-Computer Interaction

Cite this