Engineering the compression of massive tables: An experimental approach

Adam L. Buchsbaum; Donald F. Caldwell; Kenneth W. Church; Glenn S. Fowler; S. Muthukrishnan

Engineering the compression of massive tables: An experimental approach

Adam L. Buchsbaum, Donald F. Caldwell, Kenneth W. Church, Glenn S. Fowler, S. Muthukrishnan

Research output: Contribution to conference › Paper › peer-review

Abstract

We study the problem of compressing massive tables. We devise a novel compression paradigm - training for lossless compression - which assumes that the data exhibit dependencies that can be learned by examining a small amount of training material. We develop an experimental methodology to test the approach. Our result is a system, pzip, which outperforms gzip by factors of two in compression size and both compression and uncompression time for various tabular data. Pzip is now in production use in an AT&T network traffic data warehouse.

Original language	English (US)
Pages	175-184
Number of pages	10
State	Published - 2000
Event	11th Annual ACM-SIAM Symposium on Discrete Algorithms - San Francisco, CA, USA Duration: Jan 9 2000 → Jan 11 2000

Other

Other	11th Annual ACM-SIAM Symposium on Discrete Algorithms
City	San Francisco, CA, USA
Period	1/9/00 → 1/11/00

ASJC Scopus subject areas

Software
General Mathematics

Cite this

@conference{a88a6d84a6254230a9fba0d351983a21,

title = "Engineering the compression of massive tables: An experimental approach",

abstract = "We study the problem of compressing massive tables. We devise a novel compression paradigm - training for lossless compression - which assumes that the data exhibit dependencies that can be learned by examining a small amount of training material. We develop an experimental methodology to test the approach. Our result is a system, pzip, which outperforms gzip by factors of two in compression size and both compression and uncompression time for various tabular data. Pzip is now in production use in an AT&T network traffic data warehouse.",

author = "Buchsbaum, {Adam L.} and Caldwell, {Donald F.} and Church, {Kenneth W.} and Fowler, {Glenn S.} and S. Muthukrishnan",

year = "2000",

language = "English (US)",

pages = "175--184",

note = "11th Annual ACM-SIAM Symposium on Discrete Algorithms ; Conference date: 09-01-2000 Through 11-01-2000",

}

TY - CONF

T1 - Engineering the compression of massive tables

T2 - 11th Annual ACM-SIAM Symposium on Discrete Algorithms

AU - Buchsbaum, Adam L.

AU - Caldwell, Donald F.

AU - Church, Kenneth W.

AU - Fowler, Glenn S.

AU - Muthukrishnan, S.

PY - 2000

Y1 - 2000

N2 - We study the problem of compressing massive tables. We devise a novel compression paradigm - training for lossless compression - which assumes that the data exhibit dependencies that can be learned by examining a small amount of training material. We develop an experimental methodology to test the approach. Our result is a system, pzip, which outperforms gzip by factors of two in compression size and both compression and uncompression time for various tabular data. Pzip is now in production use in an AT&T network traffic data warehouse.

AB - We study the problem of compressing massive tables. We devise a novel compression paradigm - training for lossless compression - which assumes that the data exhibit dependencies that can be learned by examining a small amount of training material. We develop an experimental methodology to test the approach. Our result is a system, pzip, which outperforms gzip by factors of two in compression size and both compression and uncompression time for various tabular data. Pzip is now in production use in an AT&T network traffic data warehouse.

UR - http://www.scopus.com/inward/record.url?scp=0033906346&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0033906346&partnerID=8YFLogxK

M3 - Paper

AN - SCOPUS:0033906346

SP - 175

EP - 184

Y2 - 9 January 2000 through 11 January 2000

ER -

Engineering the compression of massive tables: An experimental approach

Abstract

Other

ASJC Scopus subject areas

Other files and links

Fingerprint

Cite this