How and how well do students reflect? Multi-dimensional automated reflection assessment in health professions education

Yeonji Jung, Alyssa Friend Wise

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Reflection assessment is a critical component of health professions education that can be used for personalized learning support. However, reflection assessment at scale remains a challenge due to the demanding nature of tasks and the common use of simplified criteria of quality. Tis study addressed this issue by developing a multi-dimensional automated assessment that uses linguistic models to classify reflections by overall quality (depth) and the presence of six constituent elements denoting quality (description, analysis, feeling, perspective, evaluation, and outcome). 1500 reflections from 369 dental students were manually coded to establish ground truth. Classifiers for each of the six elements were trained and tested based on linguistic features extracted using the LIWC tool applying both single-label and multi-label classification approaches. Classifiers for depth were built both directly from linguistic features and based on the presence of the six elements. Results showed that linguistic modeling can be used to reliably detect the presence of reflection elements and the level of depth. However, the depth classifier showed a heavy reliance on cognitive elements (description, analysis, and evaluation) rather than the others. Tese findings indicate the feasibility of implementing multidimensional automated assessment in health professions education and the need to reconsider how quality of reflection is conceptualized.

Original languageEnglish (US)
Title of host publicationLAK 2020 Conference Proceedings - Celebrating 10 years of LAK
Subtitle of host publicationShaping the Future of the Field - 10th International Conference on Learning Analytics and Knowledge
PublisherAssociation for Computing Machinery
Pages595-604
Number of pages10
ISBN (Electronic)9781450377126
DOIs
StatePublished - Mar 23 2020
Event10th International Conference on Learning Analytics and Knowledge: Shaping the Future of the Field, LAK 2020 - Frankfurt, Germany
Duration: Mar 23 2020Mar 27 2020

Publication series

NameACM International Conference Proceeding Series

Conference

Conference10th International Conference on Learning Analytics and Knowledge: Shaping the Future of the Field, LAK 2020
CountryGermany
CityFrankfurt
Period3/23/203/27/20

Keywords

  • Classification
  • Content Analysis
  • Health Professions Education
  • Natural Language Processing
  • Reflection
  • Reflection Assessment

ASJC Scopus subject areas

  • Software
  • Human-Computer Interaction
  • Computer Vision and Pattern Recognition
  • Computer Networks and Communications

Fingerprint Dive into the research topics of 'How and how well do students reflect? Multi-dimensional automated reflection assessment in health professions education'. Together they form a unique fingerprint.

  • Cite this

    Jung, Y., & Wise, A. F. (2020). How and how well do students reflect? Multi-dimensional automated reflection assessment in health professions education. In LAK 2020 Conference Proceedings - Celebrating 10 years of LAK: Shaping the Future of the Field - 10th International Conference on Learning Analytics and Knowledge (pp. 595-604). (ACM International Conference Proceeding Series). Association for Computing Machinery. https://doi.org/10.1145/3375462.3375528