TY - GEN
T1 - Histogramming data streams with fast per-item processing
AU - Guha, Sudipto
AU - Indyk, Piotr
AU - Muthukrishnan, S.
AU - Strauss, Martin J.
PY - 2002
Y1 - 2002
N2 - A vector A of length N can be approximately represented by a histogram H, by writing [0,N) as the non-overlapping union of B intervals Ij, assigning a value bj to Ij, and approximating A i by Hi = bj for i ∈ Ij. An optimal histogram representation Hopt consists of the choices of Ij and bj that minimize the sum-square-error ∥A - H∥22 = ∑i|Ai-H i|2. Numerous applications in statistics, signal processing and databases rely on histograms; typically B is (significantly) smaller than N and, hence, representing A by H yields substantial compression. We give a deterministic algorithm that approximates Hopt and outputs a histogram H such that ∥A -H∥22≤ (1 + ε) ∥A -Hopt∥22 Our algorithm considers the data items A0,A1,.. in order, i.e., in one pass, spends processing time O(1) per item, uses total space B poly(log(N), log ∥A∥, 1/ε ), and determines the histogram in time poly((B, log(N), log ∥A∥, 1/ε ). Our algorithm is suitable to emerging applications where signal is presented in a stream, size of the signal is very large, and one must construct the histogram using significantly smaller space than the signal size. In particular, our algorithm is suited to high performance needs where the per-item processing time must be minimized. Previous algorithms either used large space, i.e., Ω(N), or worked longer, i.e., N log Ω(1)(N) total time over the N data items. Our algorithm is the first that simultaneously uses small space as well as runs fast, taking O(1) worst case time for per-item processing. In addition, our algorithm is quite simple.
AB - A vector A of length N can be approximately represented by a histogram H, by writing [0,N) as the non-overlapping union of B intervals Ij, assigning a value bj to Ij, and approximating A i by Hi = bj for i ∈ Ij. An optimal histogram representation Hopt consists of the choices of Ij and bj that minimize the sum-square-error ∥A - H∥22 = ∑i|Ai-H i|2. Numerous applications in statistics, signal processing and databases rely on histograms; typically B is (significantly) smaller than N and, hence, representing A by H yields substantial compression. We give a deterministic algorithm that approximates Hopt and outputs a histogram H such that ∥A -H∥22≤ (1 + ε) ∥A -Hopt∥22 Our algorithm considers the data items A0,A1,.. in order, i.e., in one pass, spends processing time O(1) per item, uses total space B poly(log(N), log ∥A∥, 1/ε ), and determines the histogram in time poly((B, log(N), log ∥A∥, 1/ε ). Our algorithm is suitable to emerging applications where signal is presented in a stream, size of the signal is very large, and one must construct the histogram using significantly smaller space than the signal size. In particular, our algorithm is suited to high performance needs where the per-item processing time must be minimized. Previous algorithms either used large space, i.e., Ω(N), or worked longer, i.e., N log Ω(1)(N) total time over the N data items. Our algorithm is the first that simultaneously uses small space as well as runs fast, taking O(1) worst case time for per-item processing. In addition, our algorithm is quite simple.
KW - Histograms
KW - Streaming algorithms
UR - http://www.scopus.com/inward/record.url?scp=84869198292&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84869198292&partnerID=8YFLogxK
U2 - 10.1007/3-540-45465-9_58
DO - 10.1007/3-540-45465-9_58
M3 - Conference contribution
AN - SCOPUS:84869198292
SN - 3540438645
SN - 9783540438649
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 681
EP - 692
BT - Automata, Languages and Programming - 29th International Colloquium, ICALP 2002, Proceedings
A2 - Widmayer, Peter
A2 - Eidenbenz, Stephan
A2 - Triguero, Francisco
A2 - Morales, Rafael
A2 - Conejo, Ricardo
A2 - Hennessy, Matthew
PB - Springer Verlag
T2 - 29th International Colloquium on Automata, Languages, and Programming, ICALP 2002
Y2 - 8 July 2002 through 13 July 2002
ER -