Free the Turtles: Removing Nested Virtualization for Performance and Confidentiality in the Cloud

Abstract

With the growing popularity of containers running in cloud environments, one might believe techniques like virtual machines (VMs) and nested virtualization are no longer essentialf or cloud computing, but this is not true. First, they are foundational for the Infrastructure as a Service (IaaS) base that most clouds use for their Kubernetes (K8s) offerings - nodes are deployed in VMs instead of on bare-metal machines to improve flexibility and resource utilization. Second, state-of-the-art technology like Kata Containers or KubeVirt deploys VMs inside K8s, to improve container isolation or to offer a cloud-native way to deploy VMs. When such techniques are coupled with VM-based K8s worker node deployments, this requires the use of nested virtualization. However, nested virtualization introduces a larger trusted computing base (TCB) and therefore raises security concerns. In addition, it introduces an important performance overhead and cannot be easily applied to confidential computing, an emerging technology to allow the execution of sensitive workloads in the cloud. In this paper, we propose the secondary-VM (secVM) framework, an alternative to nested virtualization which addresses these challenges by flattening the nested hierarchy, but maintains the resource isolation features of nested virtualization. By removing the emulation layer required for nested virtualization, the secVM framework reduces the TCB, is compatible with any confidential computing techniques,and removes the overhead associated with nested virtualization.Therefore, in reference to the Turtles Project that introduced the concept of nested virtualization in the open-source virtualization technology - Kernel-based Virtual Machine (KVM), we argue it is time to free the “turtles”.