Developing well-calibrated illness severity scores for decision support in the critically ill

Christopher V Cosgriff, Leo Anthony Celi, Stephanie Ko, Tejas Sundaresan, Miguel Ángel Armengol de la Hoz, Aaron Russell Kaufman, David J Stone, Omar Badawi, Rodrigo Octavio Deliberato

Research output: Contribution to journalArticlepeer-review


Illness severity scores are regularly employed for quality improvement and benchmarking in the intensive care unit, but poor generalization performance, particularly with respect to probability calibration, has limited their use for decision support. These models tend to perform worse in patients at a high risk for mortality. We hypothesized that a sequential modeling approach wherein an initial regression model assigns risk and all patients deemed high risk then have their risk quantified by a second, high-risk-specific, regression model would result in a model with superior calibration across the risk spectrum. We compared this approach to a logistic regression model and a sophisticated machine learning approach, the gradient boosting machine. The sequential approach did not have an effect on the receiver operating characteristic curve or the precision-recall curve but resulted in improved reliability curves. The gradient boosting machine achieved a small improvement in discrimination performance and was similarly calibrated to the sequential models.

Original languageEnglish (US)
Article number76
Pages (from-to)76
Journalnpj Digital Medicine
Issue number1
StatePublished - Dec 1 2019


Dive into the research topics of 'Developing well-calibrated illness severity scores for decision support in the critically ill'. Together they form a unique fingerprint.

Cite this