RTA: Real time actionable events detection as a service

Abstract

Nowadays vast amounts of data are being producedin continuous ways. They may come from sensors,smart meters, application logs, monitoring software etc. Thedata need to be processed in realtime to gain actionableinsights. Services like smart grid load balancing, cloud platformmaintenance, can be carried out in an efficient way. Streamprocessing is the programming paradigm that answers suchdemand.When talking about stream processing, we can easily recallseveral famous open-source software frameworks such as SparkStreaming, Samza, Flink and Storm. Although they providedistributed, robust, low-latency stream processing engines, it'sstill difficult for an end user to set up a usable stream processingapplication from scratch. Firstly, users are required to writecode to define their business related stream processing logic.Secondly, the submission and update of the stream processinglogic require service restart, therefore it may lead to serviceunavailability for minutes. Thirdly, extra operation effort arerequired for handling scaling and failover issues.In this paper, we present RTA, a released research serviceon realtime data processing. The RTA service fills the gapbetween the stream processing requester and the existingsoftware stacks. It offers a SQL-like stream query languagefor defining stream processing logic definition over streamingdata. It allows users easily define their stream processinglogic without programming. In RTA service, stream processinglogic is also treated as a type of input, which enables onlinelogic update without service downtime. The RTA service alsoprovides scalability, high availability and resource isolationfor serving multiple tenants. In this paper, we also provide acomprehensive evaluation of our service through a case study.