NBWGuard: Realizing network QoS for kubernetes

Abstract

Kubernetes is a very popular and fast-growing container orchestration platform that automates the process of deploying and managing multi-container applications at scale. Users can specify required and maximum values of resources they need for their containers and Kubernetes realizes them by interfacing with lower levels (container runtime which in turn can use OS capabilities) of the stack for enforcing them. Kubernetes supports differentiated QoS classes - Guaranteed, Burstable, and Best-effort - in order of decreasing priority based on the resource size specifications for CPU and memory capacity. This allows many applications to obtain a desired level of QoS (performance isolation and throughput) when CPU or memory capacity management can provide them. However, when workloads may be critically dependent for their performance on another resource, namely network bandwidth, Kubernetes has no means to meet their QoS needs. Networking between pods in Kubernetes is supported with plug-ins and the network resource is not managed directly. In this work, we propose NBWGuard, a design for network bandwidth management and evaluate its implementation. NBWGuard lets Kubernetes manage network bandwidth as a resource (like CPU or memory capacity) while still using plug-ins for realizing the network specification desired by users. Consistent with Kubernetes approach to application QoS based on resource allocation NBWGuard also supports the 3 QoS classes: Guaranteed, Burstable, and Best-effort with respect to network bandwidth. NBWGuard is evaluated with iperf benchmark on real cloud environment, and the evaluation results demonstrate that it is able to provide network bandwidth isolation without impact on overall throughput.