A Near-Optimal Algorithm for Estimating the Entropy of a Stream

Publication Date

2010

Journal or Book Title

ACM TRANSACTIONS ON ALGORITHMS

Abstract

We describe a simple algorithm for approximating the empirical entropy of a stream of m values up to a multiplicative factor of (1+ϵ) using a single pass, O(ϵ−2 log (δ−1) log m) words of space, and O(log ϵ−1 + log log δ−1 + log log m) processing time per item in the stream. Our algorithm is based upon a novel extension of a method introduced by Alon et al. [1999]. This improves over previous work on this problem. We show a space lower bound of Ω(ϵ−2/log2 (ϵ−1)), demonstrating that our algorithm is near-optimal in terms of its dependency on ϵ.

We show that generalizing to multiplicative-approximation of the kth-order entropy requires close to linear space for k≥1. In contrast we show that additive-approximation is possible in a single pass using only poly-logarithmic space. Lastly, we show how to compute a multiplicative approximation to the entropy of a random walk on an undirected graph.

DOI

https://doi.org/10.1145/1798596.1798604

Pages

-

Volume

6

Issue

3

This document is currently not available here.

Share

COinS