A Probabilistic Upper Bound on Differential Entropy
Publication Date
2008
Journal or Book Title
IEEE TRANSACTIONS ON INFORMATION THEORY
Abstract
A novel probabilistic upper bound on the entropy of an unknown one-dimensional distribution, given the support of the distribution and a sample from that distribution, is presented. No knowledge beyond the support of the unknown distribution is required. Previous distribution-free bounds on the cumulative distribution function of a random variable given a sample of that variable are used to construct the bound. A simple, fast, and intuitive algorithm for computing the entropy bound from a sample is provided.
DOI
https://doi.org/10.1109/TIT.2008.929937
Pages
5223-5230
Volume
54
Issue
11
Recommended Citation
Learned-Miller, E and DeStefano, J, "A Probabilistic Upper Bound on Differential Entropy" (2008). IEEE TRANSACTIONS ON INFORMATION THEORY. 732.
https://doi.org/10.1109/TIT.2008.929937