Reusing Hardware Performance Counters to Detect and Identify Kernel Control-Flow Modifying Rootkits

Xueyang Wang, Ramesh Karri

Research output: Contribution to journalArticle

Abstract

Kernel rootkits are formidable threats to computer systems. They are stealthy and can have unrestricted access to system resources. This paper presents NumChecker, a new virtual machine (VM) monitor based framework to detect and identify control-flow modifying kernel rootkits in a guest VM. NumChecker detects and identifies malicious modifications to a system call in the guest VM by measuring the number of certain hardware events that occur during the system call's execution. To automatically count these events, NumChecker leverages the hardware performance counters (HPCs), which exist in modern processors. By using HPCs, the checking cost is significantly reduced and the tamper-resistance is enhanced. We implement a prototype of NumChecker on Linux with the kernel-based VM. An HPC-based two-phase kernel rootkit detection and identification technique is presented and evaluated on a number of real-world kernel rootkits. The results demonstrate its practicality and effectiveness.

Original languageEnglish (US)
Article number7229276
Pages (from-to)485-498
Number of pages14
JournalIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
Volume35
Issue number3
DOIs
StatePublished - Mar 1 2016

Keywords

  • Controlflow Modifying Kernel Rootkits
  • Hardware Performance Counters
  • Rootkit Detection and Identification
  • Virtualization

ASJC Scopus subject areas

  • Software
  • Computer Graphics and Computer-Aided Design
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'Reusing Hardware Performance Counters to Detect and Identify Kernel Control-Flow Modifying Rootkits'. Together they form a unique fingerprint.

  • Cite this