The nearest neighbor information estimator is adaptively near minimax rate-optimal

Jiantao Jiao, Weihao Gao, Yanjun Han

Research output: Contribution to journalConference articlepeer-review


We analyze the Kozachenko-Leonenko (KL) fixed k-nearest neighbor estimator for the differential entropy. We obtain the first uniform upper bound on its performance for any fixed k over Hölder balls on a torus without assuming any conditions on how close the density could be from zero. Accompanying a recent minimax lower bound over the Hölder ball, we show that the KL estimator for any fixed k is achieving the minimax rates up to logarithmic factors without cognizance of the smoothness parameter s of the Hölder ball for s ∈ (0, 2] and arbitrary dimension d, rendering it the first estimator that provably satisfies this property.

Original languageEnglish (US)
Pages (from-to)3156-3167
Number of pages12
JournalAdvances in Neural Information Processing Systems
StatePublished - 2018
Event32nd Conference on Neural Information Processing Systems, NeurIPS 2018 - Montreal, Canada
Duration: Dec 2 2018Dec 8 2018

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Information Systems
  • Signal Processing


Dive into the research topics of 'The nearest neighbor information estimator is adaptively near minimax rate-optimal'. Together they form a unique fingerprint.

Cite this