### Abstract

This paper establishes the optimality of the plugin estimator for the problem of differential entropy estimation under Gaussian convolutions. Specifically, we consider the estimation of the differential entropy h(X + Z), where X and Z are independent d-dimensional random variables with Z{\sim}\mathcal{N}( {0,{σ ^2}{{\text{I}}-d}} ). The distribution of X is unknown and belongs to some nonparametric class, but n independently and identically distributed samples from it are available. We first show that despite the regularizing effect of noise, any good estimator (within an additive gap) for this problem must have an exponential in d sample complexity. We then analyze the absolute-error risk of the plug-in estimator and show that it converges as frac{{{c^d}}}{{n }}, thus attaining the parametric estimation rate. This implies the optimality of the plug-in estimator for the considered problem. We provide numerical results comparing the performance of the plug-in estimator to general-purpose (unstructured) differential entropy estimators (based on kernel density estimation (KDE) or k nearest neighbors (kNN) techniques) applied to samples of X + Z. These results reveal a significant empirical superiority of the plug-in to state-of-the-art KDE- and kNN-based methods.

Original language | English (US) |
---|---|

Title of host publication | 2019 IEEE International Symposium on Information Theory, ISIT 2019 - Proceedings |

Publisher | Institute of Electrical and Electronics Engineers Inc. |

Pages | 892-896 |

Number of pages | 5 |

ISBN (Electronic) | 9781538692912 |

DOIs | |

State | Published - Jul 2019 |

Event | 2019 IEEE International Symposium on Information Theory, ISIT 2019 - Paris, France Duration: Jul 7 2019 → Jul 12 2019 |

### Publication series

Name | IEEE International Symposium on Information Theory - Proceedings |
---|---|

Volume | 2019-July |

ISSN (Print) | 2157-8095 |

### Conference

Conference | 2019 IEEE International Symposium on Information Theory, ISIT 2019 |
---|---|

Country | France |

City | Paris |

Period | 7/7/19 → 7/12/19 |

### ASJC Scopus subject areas

- Theoretical Computer Science
- Information Systems
- Modeling and Simulation
- Applied Mathematics

## Fingerprint Dive into the research topics of 'Optimality of the Plug-in Estimator for Differential Entropy Estimation under Gaussian Convolutions'. Together they form a unique fingerprint.

## Cite this

*2019 IEEE International Symposium on Information Theory, ISIT 2019 - Proceedings*(pp. 892-896). [8849414] (IEEE International Symposium on Information Theory - Proceedings; Vol. 2019-July). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ISIT.2019.8849414