What's new: Finding significant differences in network data streams

Graham Cormode, S. Muthukrishnan

Research output: Contribution to journalArticle

Abstract

Monitoring and analyzing network traffic usage patterns is vital for managing IP Networks. An important problem is to provide network managers with information about changes in traffic, informing them about "what's new." Specifically, we focus on the challenge of finding significantly large differences in traffic: over time, between interfaces and between routers. We introduce the idea of a deltoid: an item that has a large difference, whether the difference is absolute, relative or variational. We present novel algorithms for finding the mo st significant deltoids in high-speed traffic data, and prove that they use small space, very small time per update, and are guaranteed to find significant deltoids with pre-specified accuracy. In experimental evaluation with real network traffic, our algorithms perform well and recover almost all deltoids. This is the first work to provide solutions capable of working over the data with one pass, at network traffic speeds.

Original languageEnglish (US)
Pages (from-to)1219-1232
Number of pages14
JournalIEEE/ACM Transactions on Networking
Volume13
Issue number6
DOIs
StatePublished - Dec 1 2005
Externally publishedYes

Keywords

  • Change detection
  • Data streams
  • Deltoids
  • Network data analysis

ASJC Scopus subject areas

  • Software
  • Computer Science Applications
  • Computer Networks and Communications
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'What's new: Finding significant differences in network data streams'. Together they form a unique fingerprint.

  • Cite this