A Resource-minimalist Flow Size Histogram Estimator

Publication Date

2008

Journal or Book Title

IMC'08: PROCEEDINGS OF THE 2008 ACM SIGCOMM INTERNET MEASUREMENT CONFERENCE

Abstract

The histogram of network flow sizes is an important yet difficult metric to estimate in network monitoring. It is important because it characterizes traffic compositions and is a crucial component of anomaly detection methods. It is difficult to estimate because of its high memory and computational requirements. Existing algorithms compute fine grained estimates for each flow size, i.e. 1, 2,... up to the maximum number observed over a finite time interval. Our approach instead relies on the insight that, while many applications require fine grained estimates of small flow sizes, i.e. {1,2,...,k} with a small k, network operators are often only interested in coarse grained estimates of larger flow sizes. Thus, we propose an estimator that outputs a binned histogram of size distributions. Our estimator computes this histogram in O(k3 + log W) operations, where W is the largest flow size of interest to the network operator, while requiring only a few bits of memory per measured flow. This translates into more than 4 fold memory savings and an exponential speedup in the estimator as compared to previous works, greatly increasing the possibility of performing on-line estimation inside a router.

DOI

http://dx.doi.org/10.1145/1452520.1452553

Pages

285-290

This document is currently not available here.

Share

COinS