Information theory, model error, and predictive skill of stochastic models for complex nonlinear systems

Dimitrios Giannakis, Andrew J. Majda, Illia Horenko

Research output: Contribution to journalArticlepeer-review


Many problems in complex dynamical systems involve metastable regimes despite nearly Gaussian statistics with underlying dynamics that is very different from the more familiar flows of molecular dynamics. There is significant theoretical and applied interest in developing systematic coarse-grained descriptions of the dynamics, as well as assessing their skill for both short- and long-range prediction. Clustering algorithms, combined with finite-state processes for the regime transitions, are a natural way to build such models objectively from data generated by either the true model or an imperfect model. The main theme of this paper is the development of new practical criteria to assess the predictability of regimes and the predictive skill of such coarse-grained approximations through empirical information theory in stationary and periodically-forced environments. These criteria are tested on instructive idealized stochastic models utilizing K-means clustering in conjunction with running-average smoothing of the training and initial data for forecasts. A perspective on these clustering algorithms is explored here with independent interest, where improvement in the information content of finite-state partitions of phase space is a natural outcome of low-pass filtering through running averages. In applications with time-periodic equilibrium statistics, recently developed finite-element, bounded-variation algorithms for nonstationary autoregressive models are shown to substantially improve predictive skill beyond standard autoregressive models.

Original languageEnglish (US)
Pages (from-to)1735-1752
Number of pages18
JournalPhysica D: Nonlinear Phenomena
Issue number20
StatePublished - Oct 15 2012


  • Autoregressive models
  • Clustering algorithms
  • Information theory
  • Model error
  • Predictability
  • Stochastic models

ASJC Scopus subject areas

  • Statistical and Nonlinear Physics
  • Mathematical Physics
  • Condensed Matter Physics
  • Applied Mathematics


Dive into the research topics of 'Information theory, model error, and predictive skill of stochastic models for complex nonlinear systems'. Together they form a unique fingerprint.

Cite this