Aggregated bloom filters for intrusion detection and prevention hardware

Bloom Filters (BFs) are fundamental building blocks in various network security applications, where packets from high-speed links are processed using state-of-the-art hardware-based systems. In this paper, we propose Aggregated Bloom Filters (ABFs) to increase the throughput and scalability of BFs. The proposed ABF has two methods to improve average speed and scalability. The first method leverages the query mechanism for hardware BFs. We ptimize queries by removing redundant hash calculations and memory accesses. First, to remove redundancy, the hash functions for each query are calculated sequentially. As soon as we have a no match in any of the hash results, the query is immediately abandoned. We then aggregate multiple queries and query a BF with all of these queries in parallel, which maximizes the throughput of the BF. The second method addresses scalability issues regarding the on-chip memory resources. In most applications multiple BFs are required to store many sets with different numbers of elements. These sets may also be too small for the unit memory on-chip. So, most of the memory is left unused, causing low memory utilization. The second method aggregates small distributed BFs to a single BF allowing better on-chip memory utilization. For the application of Network Intrusion Detection and Prevention Systems (NIDPSs), our proposed ABF shows seven-fold improvement in the average query throughput and four times less memory usage.