Contention-resilient overcommitment for serverless deployments

Abstract

Serverless computing, or Function-as-a-Service (FaaS), simplifies application deployment by allowing developers to focus on functionality while the service provider manages the underlying resources. This model charges users based on actual resource usage, making it ideal for intermittent workloads, unlike Infrastructure-as-a-Service (IaaS), where users pay for idle virtual machines. Thus, serverless computing shifts the cost of idle resources to the service provider, who must manage the underlying server capacity. In container-based serverless platforms, idle functions are often kept active to avoid the costs associated with initialising containers, or "cold starts", which exacerbates resource under-utilisation. Overcommitting resources is a strategy to enhance resource utilisation by leveraging statistical multiplexing, where multiple workloads are unlikely to reach peak usage simultaneously. This approach allows for greater workload colocation and reduces server costs by pooling resources rather than over-allocating resources per function. However, service providers often adopt conservative overcommitment policies to avoid performance degradation and workload disruption, as higher levels of colocation increase the risk of contention. In container-based serverless systems, the high density of function containers—due to their small resource footprint—limits the benefits of overcommitment because it increases kernel scheduling overhead when managing large number of colocated containers. This dissertation proposes a host-focused approach to addressing the workload colocation problem in the context of serverless-style Linux container workloads and CPU resources. CPUs can be the bottleneck for many workloads and the efficiency of CPU multiplexing can limit the degree of workload colocation that can be achieved. The central thesis of this dissertation is that mitigating the CPU scheduling overhead associated with workload colocation allows to unlock additional capacity in a serverless cluster. The primary contribution is an enhanced Linux CPU scheduler that enables larger number of containers to share the CPU resources of a single server while robustly managing the increased risk of contention. This contribution is evaluated within a cluster resource allocation framework, demonstrating that higher levels of workload colocation are achievable thanks to the proposed scheduler.