TY - GEN
T1 - How and how well do students reflect?
T2 - 10th International Conference on Learning Analytics and Knowledge: Shaping the Future of the Field, LAK 2020
AU - Jung, Yeonji
AU - Wise, Alyssa Friend
N1 - Publisher Copyright:
© 2020 Copyright is held by the owner/author(s).
PY - 2020/3/23
Y1 - 2020/3/23
N2 - Reflection assessment is a critical component of health professions education that can be used for personalized learning support. However, reflection assessment at scale remains a challenge due to the demanding nature of tasks and the common use of simplified criteria of quality. Tis study addressed this issue by developing a multi-dimensional automated assessment that uses linguistic models to classify reflections by overall quality (depth) and the presence of six constituent elements denoting quality (description, analysis, feeling, perspective, evaluation, and outcome). 1500 reflections from 369 dental students were manually coded to establish ground truth. Classifiers for each of the six elements were trained and tested based on linguistic features extracted using the LIWC tool applying both single-label and multi-label classification approaches. Classifiers for depth were built both directly from linguistic features and based on the presence of the six elements. Results showed that linguistic modeling can be used to reliably detect the presence of reflection elements and the level of depth. However, the depth classifier showed a heavy reliance on cognitive elements (description, analysis, and evaluation) rather than the others. Tese findings indicate the feasibility of implementing multidimensional automated assessment in health professions education and the need to reconsider how quality of reflection is conceptualized.
AB - Reflection assessment is a critical component of health professions education that can be used for personalized learning support. However, reflection assessment at scale remains a challenge due to the demanding nature of tasks and the common use of simplified criteria of quality. Tis study addressed this issue by developing a multi-dimensional automated assessment that uses linguistic models to classify reflections by overall quality (depth) and the presence of six constituent elements denoting quality (description, analysis, feeling, perspective, evaluation, and outcome). 1500 reflections from 369 dental students were manually coded to establish ground truth. Classifiers for each of the six elements were trained and tested based on linguistic features extracted using the LIWC tool applying both single-label and multi-label classification approaches. Classifiers for depth were built both directly from linguistic features and based on the presence of the six elements. Results showed that linguistic modeling can be used to reliably detect the presence of reflection elements and the level of depth. However, the depth classifier showed a heavy reliance on cognitive elements (description, analysis, and evaluation) rather than the others. Tese findings indicate the feasibility of implementing multidimensional automated assessment in health professions education and the need to reconsider how quality of reflection is conceptualized.
KW - Classification
KW - Content Analysis
KW - Health Professions Education
KW - Natural Language Processing
KW - Reflection
KW - Reflection Assessment
UR - http://www.scopus.com/inward/record.url?scp=85082383146&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85082383146&partnerID=8YFLogxK
U2 - 10.1145/3375462.3375528
DO - 10.1145/3375462.3375528
M3 - Conference contribution
AN - SCOPUS:85082383146
T3 - ACM International Conference Proceeding Series
SP - 595
EP - 604
BT - LAK 2020 Conference Proceedings - Celebrating 10 years of LAK
PB - Association for Computing Machinery
Y2 - 23 March 2020 through 27 March 2020
ER -