A model-based framework for building extensible, high performance stream processing middleware and programming language for IBM InfoSphere Streams

Abstract

This work presents an extensive case study on the model-based design of a commercial-grade stream processing middleware (IBM's InfoSphere Streams) its runtime and language (SPL) compiler. The model-based underpinnings are pervasive throughout the whole environment, from describing inter-process communication interfaces and objects to the design of the extensibility mechanism in the runtime and language. In addition to many software engineering advantages such as consistent, uniform, and self-documented integration among the different parts of the system, we show intrinsic performance benefits to the platform derived from this design approach. First, we demonstrate how an incremental compilation strategy employed by the SPL compiler and rooted on the model description of the application, extracted by the compiler as part of the application building process, leads to better compile-time performance. Second, we discuss how the model-based code generation strategy employed by the SPL compiler also leads to increased runtime performance, by specializing the generated code to particular characteristics of the runtime environment. Finally, we show how the extensibility strategy used in the SPL language leads to automatic syntactic and semantic checks at compile time, while enabling behavioral reasoning and specific optimizations at runtime. Copyright © 2011 John Wiley & Sons, Ltd. Copyright © 2011 John Wiley & Sons, Ltd.