Soroban: Attributing Latency in Virtualized Environments

Abstract

Applications running in the cloud have highly-variable response times due to the lack of perfect performance isolation from other services served by common infrastructure. In particular, response latency when executing on a loaded hypervisor or in a container is substantially higher than uncontested bare-metal performance. Whilst efforts to increase performance isolation continue, we present Soroban, a framework for attributing latency to either the cloud provider or their customer. Soroban allows cloud providers to instrument commonly used programs, such as a web server to determine, for each request, how much of the latency is due to the cloud provider, or the consumer. We apply Soroban to a HTTP server and show that it identifies when the cause of latency is due to a provider-induced activity, such as underprovisioning a host, or due to the software run by the customer.