RAPID: Real-Time Alert Investigation with Context-aware Prioritization for Efficient Threat Discovery

Abstract

Alerts reported by intrusion detection systems (IDSes) are often the starting points for attack campaign discovery and response procedures. However, the sheer number of alerts compared to the number of real attacks, along with the complexity of alert investigations, poses a challenge to achieving effective alert triage with limited computational resources. Automated procedures and human analysts could suffer from the burden of analyzing floods of alerts, and fail to respond to critical alerts promptly. To scale out the alert processing capability in enterprises, we present RAPID, a real-time alert investigation system to aid analysts perform provenance analysis tasks around alerts in an efficient and collaborative manner. RAPID is built based on two key insights: 1) space and time efficiency of alert investigations can be improved by avoiding the significant overlap between alert triage tasks; 2) prioritization of alert triage tasks should be dynamic to adapt to the newly discovered context. In doing so, RAPID maximizes the utilization of limited computation resources and time, and reacts to the most critical reasoning steps in a timely manner. More specifically, RAPID employs an interruptible tracking algorithm that efficiently uncovers the causal connections between alerts and propagates priorities based on the connections. Unlike prior work, RAPID does not rely on knowledge of existing threat ontologies and focuses on providing a general concurrent alert investigation platform with provenance analysis capabilities. We evaluate RAPID on a 1TB dataset from DARPA Transparent Computing (TC) program with 411 million events, including three attack campaigns. The results show that RAPID is able to improve space efficiency by up to three orders of magnitude and reduce the time of alert provenance analysis to discover all the major attack traces by up to 99%.