Tuning complex event processing rules using the prediction-correction paradigm

Abstract

There is a growing need for the use of active systems, systems that act automatically based on events. In many cases, providing such active functionality requires materializing (inferring) the occurrence of relevant events. A widespread paradigm for enabling such materialization is Complex Event Processing (CEP), a rule based paradigm, which currently relies on domain experts to fully define the relevant rules. These experts need to provide the set of basic events which serves as input to the rule, their inter-relationships, and the parameters of the events for determining a new event materialization. While it is reasonable to expect that domain experts will be able to provide a partial rules specification, providing all the required details is a hard task, even for domain experts. Moreover, in many active systems, rules may change over time, due to the dynamic nature of the domain. Such changes complicate even further the specification task, as the expert must constantly update the rules. As a result, we seek additional support to the definition of rules, beyond expert opinion. This work presents a mechanism for automating both the initial definition of rules and the update of rules over time. This mechanism combines partial information provided by the domain expert with machine learning techniques, and is aimed at improving the accuracy of event specification and materialization. The proposed mechanism consists of two main repetitive stages, namely rule parameter prediction and rule parameter correction. The former is performed by updating the parameters using an available expert knowledge regarding the future changes of parameters. The latter stage utilizes expert feedback regarding the actual past occurrence of events and the events materialized by the CEP framework to tune rule parameters. We also include possible implementations for both stages, based on a statistical estimator and evaluate our outcome using a case study from the intrusion detection domain.