TY - JOUR
T1 - Addressing practical issues of predictive models translation into everyday practice and public health management
T2 - A combined model to predict the risk of type 2 diabetes improves incidence prediction and reduces the prevalence of missing risk predictions
AU - Vettoretti, Martina
AU - Longato, Enrico
AU - Zandonà, Alessandro
AU - Li, Yan
AU - Pagán, José Antonio
AU - Siscovick, David
AU - Carnethon, Mercedes R.
AU - Bertoni, Alain G.
AU - Facchinetti, Andrea
AU - Di Camillo, Barbara
N1 - Funding Information:
The MESA study was supported by contracts HHSN268201500003I, N01-HC-95159, N01-HC-95160, N01-HC-95161, N01-HC-95162, N01-HC-95163, N01-HC-95164, N01-HC-95165, N01-HC-95166, N01-HC-95167, N01-HC-95168, and N01-HC-95169 from the National Heart, Lung, and Blood Institute, and by grants UL1-TR-000040, UL1-TR-001079, and UL1-TR-001420 from the National Center for Advancing Translational Sciences (NCATS). The ELSA study is funded by the National Institute on Aging (R01AG017644), and a consortium of UK government departments coordinated by the National Institute for Health Research. This research was funded by: the European Commission within the Horizon 2020 PULSE project, ID 727816, from January 11, 2016 to October 31, 2019; MIUR (Italian Minister for Education) under the initiative “Departments of Excellence†(Law 232/2016)
Publisher Copyright:
© 2020 Journal for ImmunoTherapy of Cancer.
PY - 2020/8/2
Y1 - 2020/8/2
N2 - Introduction Many predictive models for incident type 2 diabetes (T2D) exist, but these models are not used frequently for public health management. Barriers to their application include (1) the problem of model choice (some models are applicable only to certain ethnic groups), (2) missing input variables, and (3) the lack of calibration. While (1) and (2) drives to missing predictions, (3) causes inaccurate incidence predictions. In this paper, a combined T2D risk model for public health management that addresses these three issues is developed. Research design and methods The combined T2D risk model combines eight existing predictive models by weighted average to overcome the problem of missing incidence predictions. Moreover, the combined model implements a simple recalibration strategy in which the risk scores are rescaled based on the T2D incidence in the target population. The performance of the combined model was compared with that of the eight existing models using data from two test datasets extracted from the Multi-Ethnic Study of Atherosclerosis (MESA; n=1031) and the English Longitudinal Study of Ageing (ELSA; n=4820). Metrics of discrimination, calibration, and missing incidence predictions were used for the assessment. Results The combined T2D model performed well in terms of both discrimination (concordance index: 0.83 on MESA; 0.77 on ELSA) and calibration (expected to observed event ratio: 1.00 on MESA; 1.17 on ELSA), similarly to the best-performing existing models. However, while the existing models yielded a large percentage of missing predictions (17%-45% on MESA; 63%-64% on ELSA), this was negligible with the combined model (0% on MESA, 4% on ELSA). Conclusions Leveraging on existing literature T2D predictive models, a simple approach based on risk score rescaling and averaging was shown to provide accurate and robust incidence predictions, overcoming the problem of recalibration and missing predictions in practical application of predictive models.
AB - Introduction Many predictive models for incident type 2 diabetes (T2D) exist, but these models are not used frequently for public health management. Barriers to their application include (1) the problem of model choice (some models are applicable only to certain ethnic groups), (2) missing input variables, and (3) the lack of calibration. While (1) and (2) drives to missing predictions, (3) causes inaccurate incidence predictions. In this paper, a combined T2D risk model for public health management that addresses these three issues is developed. Research design and methods The combined T2D risk model combines eight existing predictive models by weighted average to overcome the problem of missing incidence predictions. Moreover, the combined model implements a simple recalibration strategy in which the risk scores are rescaled based on the T2D incidence in the target population. The performance of the combined model was compared with that of the eight existing models using data from two test datasets extracted from the Multi-Ethnic Study of Atherosclerosis (MESA; n=1031) and the English Longitudinal Study of Ageing (ELSA; n=4820). Metrics of discrimination, calibration, and missing incidence predictions were used for the assessment. Results The combined T2D model performed well in terms of both discrimination (concordance index: 0.83 on MESA; 0.77 on ELSA) and calibration (expected to observed event ratio: 1.00 on MESA; 1.17 on ELSA), similarly to the best-performing existing models. However, while the existing models yielded a large percentage of missing predictions (17%-45% on MESA; 63%-64% on ELSA), this was negligible with the combined model (0% on MESA, 4% on ELSA). Conclusions Leveraging on existing literature T2D predictive models, a simple approach based on risk score rescaling and averaging was shown to provide accurate and robust incidence predictions, overcoming the problem of recalibration and missing predictions in practical application of predictive models.
KW - modeling
KW - prevention
KW - risk factor modeling
KW - type 2 diabetes
UR - http://www.scopus.com/inward/record.url?scp=85089044159&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85089044159&partnerID=8YFLogxK
U2 - 10.1136/bmjdrc-2020-001223
DO - 10.1136/bmjdrc-2020-001223
M3 - Article
C2 - 32747386
AN - SCOPUS:85089044159
SN - 2052-4897
VL - 8
JO - BMJ Open Diabetes Research and Care
JF - BMJ Open Diabetes Research and Care
IS - 1
M1 - e001223
ER -