Machine Learning for NetFlow Anomaly Detection with Human-Readable Annotations

Prashanth Krishnamurthy, Farshad Khorrami, Steve Schmidt, Kevin Wright

Research output: Contribution to journalArticlepeer-review

Abstract

We propose a framework for anomaly detection in communication network logs along with automated extraction of human-readable annotations that explain the decision logic underlying each anomaly detection. For this purpose, we develop a machine learning methodology formulated in terms of a model comprised of an OR-combination of multiple Boolean logic based sentences. Each sentence is an empirically learned set of inequality conditions involving subsets of features. The feature set, which comprises the “alphabet” for human-readable annotations, is constructed using dynamic graph based spatio-temporal aggregation to extract human-understandable aggregates of network activity. These aggregates are constructed both in terms of computers (nodes in dynamic graph) and communications between computers (edges in dynamic graph). From the alphabet, the learned model identifies subsets of features that relate to each anomaly type and the combinations of conditions in terms of the feature subsets for detection of the specific anomaly type. Given a data point that the learned model detects as anomalous, the model identifies the specific features and their combinations related to the anomaly detection. These human-readable annotations provide a cyber-security analyst a transparent view into the decision logic underlying an anomaly detection.

Original languageEnglish (US)
JournalIEEE Transactions on Network and Service Management
DOIs
StateAccepted/In press - 2021

Keywords

  • Annotations
  • Anomaly detection
  • Anomaly detection
  • Data models
  • Feature extraction
  • Genetic algorithms
  • Human-readable annotations.
  • Intrusion detection
  • Networks
  • Neural networks
  • Security
  • Support vector machines

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'Machine Learning for NetFlow Anomaly Detection with Human-Readable Annotations'. Together they form a unique fingerprint.

Cite this