Estimation of ability from homework items when there are missing and/or multiple attempts

Yoav Bergner, Kimberly Colvin, David E. Pritchard

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Scoring of student item response data from online courses and especially massively open online courses (MOOCs) is complicated by two challenges, potentially large amounts of missing data and allowances for multiple attempts to answer. Approaches to ability estimation with respect to both of these issues are considered using data from a large-enrollment electrical engineering MOOC. The allowance of unlimited multiple attempts sets up a range of observed score and latent-variable approaches to scoring the constructed response homework. With respect to missing data, two classical approaches are discussed, treating omitted items as incorrect or missing at random (MAR). These treatments turn out to have slightly different interpretations depending on the scoring model. In all, twelve different homework scores are proposed based on combinations of scoring model and missing data handling. The scores are computed and correlations between each score and the final exam score are compared, with attention to different populations of course participants.

Original languageEnglish (US)
Title of host publicationProceedings of the 5th International Conference on Learning Analytics and Knowledge, LAK 2015
PublisherAssociation for Computing Machinery
Pages118-125
Number of pages8
ISBN (Electronic)9781450334174
DOIs
StatePublished - Mar 16 2015
Event5th International Conference on Learning Analytics and Knowledge, LAK 2015 - Poughkeepsie, United States
Duration: Mar 16 2015Mar 20 2015

Publication series

NameACM International Conference Proceeding Series
Volume16-20-March-2015

Other

Other5th International Conference on Learning Analytics and Knowledge, LAK 2015
CountryUnited States
CityPoughkeepsie
Period3/16/153/20/15

Keywords

  • Ability Estimation
  • MOOCs
  • Missing Data
  • Psychometrics

ASJC Scopus subject areas

  • Software
  • Human-Computer Interaction
  • Computer Vision and Pattern Recognition
  • Computer Networks and Communications

Fingerprint Dive into the research topics of 'Estimation of ability from homework items when there are missing and/or multiple attempts'. Together they form a unique fingerprint.

  • Cite this

    Bergner, Y., Colvin, K., & Pritchard, D. E. (2015). Estimation of ability from homework items when there are missing and/or multiple attempts. In Proceedings of the 5th International Conference on Learning Analytics and Knowledge, LAK 2015 (pp. 118-125). (ACM International Conference Proceeding Series; Vol. 16-20-March-2015). Association for Computing Machinery. https://doi.org/10.1145/2723576.2723582