Comparing models for early warning systems of neglected tropical diseases

Background: Early warning systems (EWS) are management tools to predict the occurrence of epidemics of infections diseases. While climate-based EWS have been developed for malaria, no standard protocol to evaluate and compare EWS has been proposed. Additionally, there are several neglected tropical disease whose transmission is sensitive to environmental conditions, for which no EWS have been proposed, through they represent a large burden for the affected populations. Methodology/Principal Findings: In the present paper, an overview of the available linear and non-linear tools to predict seasonal time series of disease is presented. Also, a general methodology to compare and evaluate models for prediction is presented and illustrated using American cutaneous leishmaniasis, a neglected tropical disease, as an example. The comparison of the different models using the predictive R² for forecasts of "out-of-fit" data (data that has not been used to fit the models) shows that for the several linear and non-linear models tested, the best results were obtained for seasonal autoregressive (SAR) models that incorporated climatic covariates. An additional bootstrapping experiment shows that the relationship of the disease time series with the dimatic covariates is strong and consistent for the SAR modeling approach. While the autoregressive part of the model is not significant, the exogenous forcinf due to climate is always statistically significant. Prediction accuracy can vary from 50% to over 80% for disease burden at time scales of one year or shorter. Conclusions/Significance: This study illustrates a protocol for the development of EWS that includes three main steps:(i) the fitting of different models using several methodologies, (ii) the comparison of models based on the predictability of "out-of-fit" data, and (iii) the assessment of the robustness of the relationship between the disease and the variables in the model selected as best with an objective criterion.