Optimal prediction of Markov chains with and without spectral gap conditions

Yanjun Han, Soham Jana, Yihong Wu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We study the following learning problem with dependent data: Observing a trajectory of length n from a stationary Markov chain with k states, the goal is to predict the next state. For 3 ≤ k ≤ O(√n), using techniques from universal compression, the optimal prediction risk in Kullback-Leibler divergence is shown to be Θ(kn2 log kn2 ), in contrast to the optimal rate of Θ(log lognn ) for k = 2 previously shown in [FOPS16]. These rates, slower than the parametric rate of O(kn2 ), can be attributed to the memory in the data, as the spectral gap of the Markov chain can be arbitrarily small. To quantify the memory effect, we study irreducible reversible chains with a prescribed spectral gap. In addition to characterizing the optimal prediction risk for two states, we show that, as long as the spectral gap is not excessively small, the prediction risk in the Markov model is O(kn2 ), which coincides with that of an iid model with the same number of parameters.

Original languageEnglish (US)
Title of host publicationAdvances in Neural Information Processing Systems 34 - 35th Conference on Neural Information Processing Systems, NeurIPS 2021
EditorsMarc'Aurelio Ranzato, Alina Beygelzimer, Yann Dauphin, Percy S. Liang, Jenn Wortman Vaughan
PublisherNeural information processing systems foundation
Pages11233-11246
Number of pages14
ISBN (Electronic)9781713845393
StatePublished - 2021
Event35th Conference on Neural Information Processing Systems, NeurIPS 2021 - Virtual, Online
Duration: Dec 6 2021Dec 14 2021

Publication series

NameAdvances in Neural Information Processing Systems
Volume14
ISSN (Print)1049-5258

Conference

Conference35th Conference on Neural Information Processing Systems, NeurIPS 2021
CityVirtual, Online
Period12/6/2112/14/21

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Information Systems
  • Signal Processing

Fingerprint

Dive into the research topics of 'Optimal prediction of Markov chains with and without spectral gap conditions'. Together they form a unique fingerprint.

Cite this