Erlang: Application-Aware Autoscaling for Cloud Microservices

Vighnesh Sachidananda, Anirudh Sivaraman

Research output: Chapter in Book/Report/Conference proceedingConference contribution


As cloud applications shift from monoliths to loosely coupled microservices, application developers must decide how many compute resources (e.g., number of replicated containers) to assign to each microservice within an application. This decision affects both (1) the dollar cost to the application developer and (2) the end-to-end latency perceived by the application user. Today, individual microservices are autoscaled independently by adding VMs whenever per-microservice CPU or memory utilization crosses a configurable threshold. However, an application user's end-to-end latency consists of time spent on multiple microservices and each microservice might need a different number of VMs to achieve an overall end-to-end latency. We present Erlang, an autoscaler for microservice-based applications, which collectively allocates VMs to microservices with a global goal of minimizing dollar cost while keeping end-to-end application latency under a given target. Using 5 open-source applications, we compared Erlang to several utilization and machine learning based autoscalers. We evaluate Erlang across different compute settings on Google Kubernetes Engine (GKE) in which users manage compute resources, GKE standard, and a new mode of operation in which the cloud provider manages compute infrastructure, GKE Autopilot. Erlang meets a desired median or tail latency target on 53 of 63 workloads where it provides a cost reduction of 19.3%, on average, over the next cheapest autoscaler. Erlang is the most cost effective autoscaling policy for 48 of these 53 workloads. The cost savings from managing a cluster with Erlang result in Erlang paying for its training cost in a few days. On smaller applications, for which we can exhaustively search microservice configurations, we find that Erlang is optimal for 90% of cases and near optimal otherwise. Code for Erlang is available at

Original languageEnglish (US)
Title of host publicationEuroSys 2024 - Proceedings of the 2024 European Conference on Computer Systems
PublisherAssociation for Computing Machinery, Inc
Number of pages36
ISBN (Electronic)9798400704376
StatePublished - Apr 22 2024
Event19th European Conference on Computer Systems, EuroSys 2024 - Athens, Greece
Duration: Apr 22 2024Apr 25 2024

Publication series

NameEuroSys 2024 - Proceedings of the 2024 European Conference on Computer Systems


Conference19th European Conference on Computer Systems, EuroSys 2024

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Control and Systems Engineering
  • Hardware and Architecture


Dive into the research topics of 'Erlang: Application-Aware Autoscaling for Cloud Microservices'. Together they form a unique fingerprint.

Cite this