A Probabilistic Filtering Approach to Non-Effortful Responding

Esther Ulitzsch, Benjamin W. Domingue, Radhika Kapoor, Klint Kanopka, Joseph A. Rios

Research output: Contribution to journalArticlepeer-review


Common response-time-based approaches for non-effortful response behavior (NRB) in educational achievement tests filter responses that are associated with response times below some threshold. These approaches are, however, limited in that they require a binary decision on whether a response is classified as stemming from NRB; thus ignoring potential classification uncertainty in resulting parameter estimates. We developed a response-time-based probabilistic filtering procedure that overcomes this limitation. The procedure is rooted in the principles of multiple imputation. Instead of creating multiple plausible replacements of missing data, however, multiple data sets are created that represent plausible filtered response data. We propose two different approaches to filtering models, originating in different research traditions and conceptualizations of response-time-based identification of NRB. The first approach uses Gaussian mixture modeling to identify a response time subcomponent stemming from NRB. Plausible filtered data sets are created based on examinees' posterior probabilities of belonging to the NRB subcomponent. The second approach defines a plausible range of response time thresholds and creates plausible filtered data sets by drawing multiple response time thresholds from the defined range. We illustrate the workings of the proposed procedure as well as differences between the proposed filtering models based on both simulated data and empirical data from PISA 2018.

Original languageEnglish (US)
Pages (from-to)50-64
Number of pages15
JournalEducational Measurement: Issues and Practice
Issue number3
StatePublished - Sep 1 2023


  • motivation filtering
  • non-effortful responding
  • response times

ASJC Scopus subject areas

  • Education


Dive into the research topics of 'A Probabilistic Filtering Approach to Non-Effortful Responding'. Together they form a unique fingerprint.

Cite this