Distinguishing Engagement Facets: An Essential Component for AI-based Interactive Healthcare

Hanan Salam

Distinguishing Engagement Facets: An Essential Component for AI-based Interactive Healthcare

Hanan Salam

Computer Science

Research output: Contribution to journal › Conference article › peer-review

Abstract

Engagement in Human-Machine Interaction is the process by which entities participating in the interaction establish, maintain, and end their perceived connection. It is essential to monitor the engagement state of patients in various AI-based interactive healthcare paradigms. This includes medical conditions that alter social behavior such as Autism Spectrum Disorder (ASD) or Attention-Deficit/Hyperactivity Disorder (ADHD). Engagement is a multi-faceted construct which is composed of behavioral, emotional, and mental components. Previous research has neglected this multi-faceted nature of engagement and focused on the detection of engagement level or binary engagement label. In this paper, a system is presented to distinguish these facets using contextual and relational features. This can facilitate further fine-grained analysis. Several machine learning classifiers including traditional and deep learning models are compared for this task. An F-Score of 0.74 was obtained on a balanced dataset of 22242 instances with neural network-based classification. The proposed framework shall serve as a baseline for further research on engagement facets recognition, and its integration is socially assistive robotic applications.

Original language	English (US)
Pages (from-to)	37-47
Number of pages	11
Journal	CEUR Workshop Proceedings
Volume	3359
State	Published - 2023
Event	Joint Workshops of IUI 2023: HAI-GEN - Human-AI Co-Creation with Generative Models, ITAH - Workshop on Interactive Technologies for AI in Healthcare, MILC - Workshop on Intelligent Music Interfaces for Listening and Creation, SHAI - Workshop on Designing for Safety in Human-AI Interactions, SketchRec - Workshop on Sketch Recognition and SOCIALIZE - Social and Cultural Integration with Personalized Interfaces - Sydney, Australia Duration: Mar 27 2023 → Mar 31 2023

Keywords

Affective Computing
Engagement Recognition
Human-Robot Interaction
Interactive Healthcare

ASJC Scopus subject areas

General Computer Science

Cite this

@article{90be3f8eae3441f9b135a4c1f9554628,

title = "Distinguishing Engagement Facets: An Essential Component for AI-based Interactive Healthcare",

abstract = "Engagement in Human-Machine Interaction is the process by which entities participating in the interaction establish, maintain, and end their perceived connection. It is essential to monitor the engagement state of patients in various AI-based interactive healthcare paradigms. This includes medical conditions that alter social behavior such as Autism Spectrum Disorder (ASD) or Attention-Deficit/Hyperactivity Disorder (ADHD). Engagement is a multi-faceted construct which is composed of behavioral, emotional, and mental components. Previous research has neglected this multi-faceted nature of engagement and focused on the detection of engagement level or binary engagement label. In this paper, a system is presented to distinguish these facets using contextual and relational features. This can facilitate further fine-grained analysis. Several machine learning classifiers including traditional and deep learning models are compared for this task. An F-Score of 0.74 was obtained on a balanced dataset of 22242 instances with neural network-based classification. The proposed framework shall serve as a baseline for further research on engagement facets recognition, and its integration is socially assistive robotic applications.",

keywords = "Affective Computing, Engagement Recognition, Human-Robot Interaction, Interactive Healthcare",

author = "Hanan Salam",

note = "Funding Information: This work is supported in part by the NYUAD Center for Artificial Intelligence and Robotics, funded by Tamkeen under the NYUAD Research Institute Award CG010. Publisher Copyright: {\textcopyright} 2023 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Workshop Proceedings (CEUR-WS.org); Joint Workshops of IUI 2023: HAI-GEN - Human-AI Co-Creation with Generative Models, ITAH - Workshop on Interactive Technologies for AI in Healthcare, MILC - Workshop on Intelligent Music Interfaces for Listening and Creation, SHAI - Workshop on Designing for Safety in Human-AI Interactions, SketchRec - Workshop on Sketch Recognition and SOCIALIZE - Social and Cultural Integration with Personalized Interfaces ; Conference date: 27-03-2023 Through 31-03-2023",

year = "2023",

language = "English (US)",

volume = "3359",

pages = "37--47",

journal = "CEUR Workshop Proceedings",

issn = "1613-0073",

publisher = "CEUR-WS",

}

TY - JOUR

T1 - Distinguishing Engagement Facets

T2 - Joint Workshops of IUI 2023: HAI-GEN - Human-AI Co-Creation with Generative Models, ITAH - Workshop on Interactive Technologies for AI in Healthcare, MILC - Workshop on Intelligent Music Interfaces for Listening and Creation, SHAI - Workshop on Designing for Safety in Human-AI Interactions, SketchRec - Workshop on Sketch Recognition and SOCIALIZE - Social and Cultural Integration with Personalized Interfaces

AU - Salam, Hanan

N1 - Funding Information: This work is supported in part by the NYUAD Center for Artificial Intelligence and Robotics, funded by Tamkeen under the NYUAD Research Institute Award CG010. Publisher Copyright: © 2023 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Workshop Proceedings (CEUR-WS.org)

PY - 2023

Y1 - 2023

N2 - Engagement in Human-Machine Interaction is the process by which entities participating in the interaction establish, maintain, and end their perceived connection. It is essential to monitor the engagement state of patients in various AI-based interactive healthcare paradigms. This includes medical conditions that alter social behavior such as Autism Spectrum Disorder (ASD) or Attention-Deficit/Hyperactivity Disorder (ADHD). Engagement is a multi-faceted construct which is composed of behavioral, emotional, and mental components. Previous research has neglected this multi-faceted nature of engagement and focused on the detection of engagement level or binary engagement label. In this paper, a system is presented to distinguish these facets using contextual and relational features. This can facilitate further fine-grained analysis. Several machine learning classifiers including traditional and deep learning models are compared for this task. An F-Score of 0.74 was obtained on a balanced dataset of 22242 instances with neural network-based classification. The proposed framework shall serve as a baseline for further research on engagement facets recognition, and its integration is socially assistive robotic applications.

AB - Engagement in Human-Machine Interaction is the process by which entities participating in the interaction establish, maintain, and end their perceived connection. It is essential to monitor the engagement state of patients in various AI-based interactive healthcare paradigms. This includes medical conditions that alter social behavior such as Autism Spectrum Disorder (ASD) or Attention-Deficit/Hyperactivity Disorder (ADHD). Engagement is a multi-faceted construct which is composed of behavioral, emotional, and mental components. Previous research has neglected this multi-faceted nature of engagement and focused on the detection of engagement level or binary engagement label. In this paper, a system is presented to distinguish these facets using contextual and relational features. This can facilitate further fine-grained analysis. Several machine learning classifiers including traditional and deep learning models are compared for this task. An F-Score of 0.74 was obtained on a balanced dataset of 22242 instances with neural network-based classification. The proposed framework shall serve as a baseline for further research on engagement facets recognition, and its integration is socially assistive robotic applications.

KW - Affective Computing

KW - Engagement Recognition

KW - Human-Robot Interaction

KW - Interactive Healthcare

UR - http://www.scopus.com/inward/record.url?scp=85150855590&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85150855590&partnerID=8YFLogxK

M3 - Conference article

AN - SCOPUS:85150855590

SN - 1613-0073

VL - 3359

SP - 37

EP - 47

JO - CEUR Workshop Proceedings

JF - CEUR Workshop Proceedings

Y2 - 27 March 2023 through 31 March 2023

ER -

Distinguishing Engagement Facets: An Essential Component for AI-based Interactive Healthcare

Abstract

Keywords

ASJC Scopus subject areas

Other files and links

Fingerprint

Cite this