The thrill of cyber threat hunting with Kestrel Threat Hunting Language
IBM’s new open-source threat-hunting programming language, which we debuted today at the RSA Conference, offers threat hunters a means to perform cyber reasoning and threat discovery much faster and easier than current threat-hunting practices.
IBM’s new open-source threat-hunting programming language, which we debuted today at the RSA Conference, offers threat hunters a means to perform cyber reasoning and threat discovery much faster and easier than current threat-hunting practices.
In cybersecurity, time is of an essence. Every hour it takes to detect and respond to a threat leaves attackers more time to do damage.
Enter IBM's Kestrel Threat Hunting Language—our new open-source threat-hunting programming language, launched at the 2021 RSA Conference. It offers cyber threat hunters a means to perform cyber reasoning and threat discovery much faster and easier than ever before.
Cyber threat hunting has been around for some time—a way to proactively chase increasingly sophisticated and pervasive cyberattacks. Hunters proactively sift through security logs and develop threat-discovery procedures with information retrieval, logical inference, machine learning, and other steps to create and verify threat hypotheses against advanced threats that evade existing security solutions.
But despite the success of cyber threat hunting efforts—which follow the scientific processes of observation, hypothesis, experimentation and verification—existing approaches rely on repeated, tedious, and time-consuming knowledge encoding on specific data platforms. Rather than benefiting from the threat-hunting community’s collective knowledge and sharing code, cybersecurity professionals end up working in relative isolation and rewriting the same programs over and over with each new attack.
Kestrel Threat Hunting Language offers threat hunters a means to perform cyber reasoning and threat discovery much faster and easier than current threat-hunting practices. Developed jointly by IBM Research and IBM Security and based on years of experimentation in DARPA’s Transparent Computing program’s adversarial engagements, Kestrel offers Security Operations Center (SOC) analysts and threat hunters a domain-specific language that lets them devote more time to figuring out what to hunt, as opposed to how to hunt.
Cyber threat hunting is intended to serve as a countermeasure against today’s threats, many of which are built dynamically and customized to attack specific targets. These stealthy attacks—also refered to as advanced persistent threats (APT), a category that includes ransomware of the kind used to disrupt Colonial Pipeline in the US—can circumvent existing security measures at various levels and cannot be fully captured by pre-programmed detection systems and analytics.
Even with the influx of numerous open-source and commercial investigation tools, cyber threat hunting lacks the efficiency needed to be truly effective. Time and intelligence that should be focused on reasoning about sophisticated threats is instead spent on mundane tasks, such as crafting queries for different data sources, interpreting their results, and recording those results in spreadsheets.
IBM designed Kestrel to codify different steps and flows involved in threat hunting. That includes understanding the security measures in the target environment, thinking about potential threats that escape existing defenses, obtaining useful observations from system and network activities, developing threat hypotheses, revising threat hypotheses iteratively with the last two steps, and confirming new threats.
As a language for threat hunters to express what they are hunting, Kestrel helps hunters organize their thoughts about threat hypotheses around entities, which include numerous identifiable systems or network objects. Under the hood, Kestrel runtime automatically reassembles an entity using pieces of information from different records or logs that describe different aspects of that entity. Kestrel also proactively asks data sources for additional information about different entities in order to ensure threat hunters have all of the information available to track down root causes and impacts of suspicious activity, as well as create and revise threat hypotheses as needed.
Kestrel uses Structured Threat Information Expression (STIX), an open-standard for expressing and exchanging cyber threat data and intelligence. Kestrel runs on top of STIX-Shifter—another open source project by IBM Security—to automatically compile threat-hunting steps down to the native languages that the different data sources speak and execute. Beyond patterns, Kestrel abstracts hunting knowledge codified in analytics and hunting flows. Shareable, composable and reusable threat-hunting flows are critical—look no further than the SolarWinds attack to understand the importance of sharing threat intelligence and analysis across multiple organizations.
Ultimately, we hope Kestrel grows with the community and gives threat hunters the ability to focus on the most interesting aspect of their work—the thrill of the hunt. By alleviating the drudgery of repetitive, mundane tasks, we want to make the field more exciting than ever, a place where cybersecurity professionals can spend their time and energy untangling sophisticated threats and raising the level of skill and effort required to launch successful attacks in the future.
And of course, make it a place to share your best hunting notebooks in a growing repository so the entire community can tackle the latest threats together. Please give us your feedback on how we can improve Kestrel, and make it the go-to-tool for threat hunters, everywhere.
Threat Management: We’re building the next wave of tools and methodologies to help security and operations teams detect, understand, and respond to advanced cybersecurity threats and attacks on their infrastructure and in the cloud.