TY - JOUR
T1 - Addressing practical issues of predictive models translation into everyday practice and public health management
T2 - A combined model to predict the risk of type 2 diabetes improves incidence prediction and reduces the prevalence of missing risk predictions
AU - Vettoretti, Martina
AU - Longato, Enrico
AU - Zandonà, Alessandro
AU - Li, Yan
AU - Pagán, José Antonio
AU - Siscovick, David
AU - Carnethon, Mercedes R.
AU - Bertoni, Alain G.
AU - Facchinetti, Andrea
AU - Di Camillo, Barbara
N1 - Publisher Copyright:
© 2020 Journal for ImmunoTherapy of Cancer.
PY - 2020/8/2
Y1 - 2020/8/2
N2 - Introduction Many predictive models for incident type 2 diabetes (T2D) exist, but these models are not used frequently for public health management. Barriers to their application include (1) the problem of model choice (some models are applicable only to certain ethnic groups), (2) missing input variables, and (3) the lack of calibration. While (1) and (2) drives to missing predictions, (3) causes inaccurate incidence predictions. In this paper, a combined T2D risk model for public health management that addresses these three issues is developed. Research design and methods The combined T2D risk model combines eight existing predictive models by weighted average to overcome the problem of missing incidence predictions. Moreover, the combined model implements a simple recalibration strategy in which the risk scores are rescaled based on the T2D incidence in the target population. The performance of the combined model was compared with that of the eight existing models using data from two test datasets extracted from the Multi-Ethnic Study of Atherosclerosis (MESA; n=1031) and the English Longitudinal Study of Ageing (ELSA; n=4820). Metrics of discrimination, calibration, and missing incidence predictions were used for the assessment. Results The combined T2D model performed well in terms of both discrimination (concordance index: 0.83 on MESA; 0.77 on ELSA) and calibration (expected to observed event ratio: 1.00 on MESA; 1.17 on ELSA), similarly to the best-performing existing models. However, while the existing models yielded a large percentage of missing predictions (17%-45% on MESA; 63%-64% on ELSA), this was negligible with the combined model (0% on MESA, 4% on ELSA). Conclusions Leveraging on existing literature T2D predictive models, a simple approach based on risk score rescaling and averaging was shown to provide accurate and robust incidence predictions, overcoming the problem of recalibration and missing predictions in practical application of predictive models.
AB - Introduction Many predictive models for incident type 2 diabetes (T2D) exist, but these models are not used frequently for public health management. Barriers to their application include (1) the problem of model choice (some models are applicable only to certain ethnic groups), (2) missing input variables, and (3) the lack of calibration. While (1) and (2) drives to missing predictions, (3) causes inaccurate incidence predictions. In this paper, a combined T2D risk model for public health management that addresses these three issues is developed. Research design and methods The combined T2D risk model combines eight existing predictive models by weighted average to overcome the problem of missing incidence predictions. Moreover, the combined model implements a simple recalibration strategy in which the risk scores are rescaled based on the T2D incidence in the target population. The performance of the combined model was compared with that of the eight existing models using data from two test datasets extracted from the Multi-Ethnic Study of Atherosclerosis (MESA; n=1031) and the English Longitudinal Study of Ageing (ELSA; n=4820). Metrics of discrimination, calibration, and missing incidence predictions were used for the assessment. Results The combined T2D model performed well in terms of both discrimination (concordance index: 0.83 on MESA; 0.77 on ELSA) and calibration (expected to observed event ratio: 1.00 on MESA; 1.17 on ELSA), similarly to the best-performing existing models. However, while the existing models yielded a large percentage of missing predictions (17%-45% on MESA; 63%-64% on ELSA), this was negligible with the combined model (0% on MESA, 4% on ELSA). Conclusions Leveraging on existing literature T2D predictive models, a simple approach based on risk score rescaling and averaging was shown to provide accurate and robust incidence predictions, overcoming the problem of recalibration and missing predictions in practical application of predictive models.
KW - modeling
KW - prevention
KW - risk factor modeling
KW - type 2 diabetes
UR - http://www.scopus.com/inward/record.url?scp=85089044159&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85089044159&partnerID=8YFLogxK
U2 - 10.1136/bmjdrc-2020-001223
DO - 10.1136/bmjdrc-2020-001223
M3 - Article
C2 - 32747386
AN - SCOPUS:85089044159
SN - 2052-4897
VL - 8
JO - BMJ Open Diabetes Research and Care
JF - BMJ Open Diabetes Research and Care
IS - 1
M1 - e001223
ER -