TY - JOUR

T1 - Quantifying predictability through information theory

T2 - Small sample estimation in a non-Gaussian framework

AU - Haven, Kyle

AU - Majda, Andrew

AU - Abramov, Rafail

N1 - Funding Information:
The authors thank Richard Kleeman for his advice and help on many of the issues discussed in this paper. Kyle Haven is partially supported as an NSF-VIGRE postdoctoral fellow at the Courant Institute. Andrew Majda is partially supported by the NSF Grant DMS-9972865, the ONR Grant N00014-96-1-0043, and the NSF CMG Grant DMS-0222133. Rafail Abramov is supported as a postdoctoral fellow through these research grants.

PY - 2005/6/10

Y1 - 2005/6/10

N2 - Many situations in complex systems require quantitative estimates of the lack of information in one probability distribution relative to another. In short term climate and weather prediction, examples of these issues might involve the lack of information in the historical climate record compared with an ensemble prediction, or the lack of information in a particular Gaussian ensemble prediction strategy involving the first and second moments compared with the non-Gaussian ensemble itself. The relative entropy is a natural way to quantify the predictive utility in this information, and recently a systematic computationally feasible hierarchical framework has been developed. In practical systems with many degrees of freedom, computational overhead limits ensemble predictions to relatively small sample sizes. Here the notion of predictive utility, in a relative entropy framework, is extended to small random samples by the definition of a sample utility, a measure of the unlikeliness that a random sample was produced by a given prediction strategy. The sample utility is the minimum predictability, with a statistical level of confidence, which is implied by the data. Two practical algorithms for measuring such a sample utility are developed here. The first technique is based on the statistical method of null-hypothesis testing, while the second is based upon a central limit theorem for the relative entropy of moment-based probability densities. These techniques are tested on known probability densities with parameterized bimodality and skewness, and then applied to the Lorenz '96 model, a recently developed "toy" climate model with chaotic dynamics mimicking the atmosphere. The results show a detection of non-Gaussian tendencies of prediction densities at small ensemble sizes with between 50 and 100 members, with a 95% confidence level.

AB - Many situations in complex systems require quantitative estimates of the lack of information in one probability distribution relative to another. In short term climate and weather prediction, examples of these issues might involve the lack of information in the historical climate record compared with an ensemble prediction, or the lack of information in a particular Gaussian ensemble prediction strategy involving the first and second moments compared with the non-Gaussian ensemble itself. The relative entropy is a natural way to quantify the predictive utility in this information, and recently a systematic computationally feasible hierarchical framework has been developed. In practical systems with many degrees of freedom, computational overhead limits ensemble predictions to relatively small sample sizes. Here the notion of predictive utility, in a relative entropy framework, is extended to small random samples by the definition of a sample utility, a measure of the unlikeliness that a random sample was produced by a given prediction strategy. The sample utility is the minimum predictability, with a statistical level of confidence, which is implied by the data. Two practical algorithms for measuring such a sample utility are developed here. The first technique is based on the statistical method of null-hypothesis testing, while the second is based upon a central limit theorem for the relative entropy of moment-based probability densities. These techniques are tested on known probability densities with parameterized bimodality and skewness, and then applied to the Lorenz '96 model, a recently developed "toy" climate model with chaotic dynamics mimicking the atmosphere. The results show a detection of non-Gaussian tendencies of prediction densities at small ensemble sizes with between 50 and 100 members, with a 95% confidence level.

UR - http://www.scopus.com/inward/record.url?scp=25444532012&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=25444532012&partnerID=8YFLogxK

U2 - 10.1016/j.jcp.2004.12.008

DO - 10.1016/j.jcp.2004.12.008

M3 - Article

AN - SCOPUS:25444532012

VL - 206

SP - 334

EP - 362

JO - Journal of Computational Physics

JF - Journal of Computational Physics

SN - 0021-9991

IS - 1

ER -