Generalized autoregressive linear models for discrete high-dimensional data

Parthe Pandit, Mojtaba Sahraee-Ardakan, Arash A. Amini, Sundeep Rangan, Alyson K. Fletcher

Research output: Contribution to journalArticlepeer-review


Fitting multivariate autoregressive (AR) models is fundamental for time-series data analysis in a wide range of applications. This article considers the problem of learning a p-lag multivariate AR model where each time step involves a linear combination of the past p states followed by a probabilistic, possibly nonlinear, mapping to the next state. The problem is to learn the linear connectivity tensor from observations of the states. We focus on the sparse setting, which arises in applications with a limited number of direct connections between variables. For such problems, _1-regularized maximum likelihood estimation (or M-estimation more generally) is often straightforward to apply and works well in practice. However, the analysis of such methods is difficult due to the feedback in the state dynamic and the presence of nonlinearities, especially when the underlying process is non-Gaussian. Our main result provides a bound on the mean-squared error of the estimated connectivity tensor as a function of the sparsity and the number of samples, for a class of discrete multivariate AR models, in the high-dimensional regime. Importantly, the bound indicates that, with sufficient sparsity, consistent estimation is possible in cases where the number of samples is significantly less than the total number of parameters.

Original languageEnglish (US)
Article number3041714
Pages (from-to)884-896
Number of pages13
JournalIEEE Journal on Selected Areas in Information Theory
Issue number3
StatePublished - Nov 2020


  • Autoregressive processes
  • Compressed sensing
  • High-dimensional time series analysis
  • Long-termmemory
  • Maximum likelihood estimation
  • Nonlinear dynamical systems

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Media Technology
  • Artificial Intelligence
  • Applied Mathematics


Dive into the research topics of 'Generalized autoregressive linear models for discrete high-dimensional data'. Together they form a unique fingerprint.

Cite this