Human-in-the-Loop Cyber Intrusion Detection Using Active Learning

— Timely detection of cyber attacks is essential for minimizing attack impact, but it requires accurate real-time situational awareness (SA). In practice, SA is hampered by frequent false alerts from anomaly-based intrusion detection systems (IDS), causing alarm fatigue. Investigating alerts by humans can enhance SA, but it is resource-intensive and it is often unclear which alerts to prioritize. In this paper, we propose a framework for optimizing human-in-the-loop attack detection, consisting of three key components: 1) dynamic alert prioritization, which ranks alerts based on previous alerts and investigations, 2) human alert investigation, referring to the manual analysis of alerts, and 3) sequential hypothesis testing, a method that confirms a hypothesis based on incoming alerts, with pruned hidden Markov models (HMMs). We formulate the problem as that of active learning in an HMM, and we propose two alert prioritization policies, namely Max Ratio and Max KL. The proposed policies aim to select the most informative alerts based on historical data and prior investigations, thereby minimizing the detection time. Simulation results show that our proposed policies reduce the time to detection by up to 79% compared to a static baseline policy, while maintaining a target mean time between false detections (MTBFD).