Modeling the interaction between resilience and ability in assessments with allowances for multiple attempts

Susu Zhang, Yoav Bergner, Jack DiTrapani, Minjeong Jeon

Research output: Contribution to journalArticlepeer-review


Many computer-based assessments, in particular those administered in learning settings, provide immediate item-level feedback and permit respondents to retry questions following incorrect responses. Doing so supports learning and facilitates the measurement of partial mastery. However, persistence through attempts may introduce new factors, such as resilience, affecting a student's test score. The current study proposes a tree-based approach to jointly model item responses and reattempt decisions in multiple-attempt assessments, allowing for the separate measurement of latent ability and reattempt propensity, or resilience. The proposed model is implemented on two datasets collected from online homework assignments, one from a university for-credit course and the other from a massive open online course. The results shed light on the relationship between resilience and ability both at the individual level and at the item level, where threshold parameters can be interpreted as difficulty and repeatability. Importantly, resilience was found to affect certain commonly used performance scores, such as eventual correctness proportion, posing a validity threat for summative assessment use-cases. The tree-based model not only resolves the problem but also opens the door to ecologically valid measures of resilience that are not reliant on self-reports.

Original languageEnglish (US)
Article number106847
JournalComputers in Human Behavior
StatePublished - Sep 2021


  • Computer-based assessment
  • Multiple attempt
  • Persistence
  • Resilience
  • Summative assessment
  • Tree-based item response models

ASJC Scopus subject areas

  • Arts and Humanities (miscellaneous)
  • Human-Computer Interaction
  • General Psychology


Dive into the research topics of 'Modeling the interaction between resilience and ability in assessments with allowances for multiple attempts'. Together they form a unique fingerprint.

Cite this