Towards Automatic Narrative Coherence Prediction

Filip Bendevski, Jumana Ibrahim, Tina Krulec, Theodore Waters, Nizar Habash, Hanan Salam, Himadri Mukherjee, Christin Camia

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Research in Psychology has shown that stories people tell about themselves, and how they recall their experiences, reveal a lot about their individual characteristics and mental well-being. The Narrative Coherence Coding Scheme (NaCCS) is a set of guidelines established in psychology research for annotating the "coherence"of a narrative along three dimensions: context, chronology and theme. A significant correlation was found between a narrative's coherence score and independently collected mental health markers of the narrator. Currently, all coherence annotations are done manually; a time consuming task which drains vital resources. In this paper, we propose an Artificial Intelligence based approach involving Natural Language Processing (NLP) to predict a narrative's coherence score (4-class classification problem). We explore a number of techniques, ranging from traditional machine learning models such as Support Vector Machines (SVM) to pre-trained language models such as BERT (Bidirectional Encoder Representations from Transformers). BERT produced the best results for all dimensions in terms of accuracy: 53.7% (context), 71.8% (chronology), and 69.6% (theme). The location of information in the narratives (beginning, end, throughout) was helpful in improving predictions.

Original languageEnglish (US)
Title of host publicationICMI 2021 - Proceedings of the 2021 International Conference on Multimodal Interaction
PublisherAssociation for Computing Machinery, Inc
Number of pages9
ISBN (Electronic)9781450384810
StatePublished - Oct 18 2021
Event23rd ACM International Conference on Multimodal Interaction, ICMI 2021 - Virtual, Online, Canada
Duration: Oct 18 2021Oct 22 2021

Publication series

NameICMI 2021 - Proceedings of the 2021 International Conference on Multimodal Interaction


Conference23rd ACM International Conference on Multimodal Interaction, ICMI 2021
CityVirtual, Online


  • (Un)supervised learning
  • AI for mental health
  • BERT.
  • Machine Learning (ML)
  • NLP
  • Narrative Coherence Coding Scheme (NaCCS)
  • Narrative text analysis
  • Word embedding

ASJC Scopus subject areas

  • Computer Science Applications
  • Computer Vision and Pattern Recognition
  • Hardware and Architecture
  • Human-Computer Interaction


Dive into the research topics of 'Towards Automatic Narrative Coherence Prediction'. Together they form a unique fingerprint.

Cite this