Cloud cost optimization research

These research papers explain the main ideas behind reducing cloud cost. They cover cluster scheduling, resource sizing, cloud pricing, block storage and live migration.

Cluster scheduling, bin packing and autoscaling

How large clusters place work on machines, pack it tightly, and grow or shrink with demand.

Large-scale cluster management at Google with BorgAbhishek Verma et al. · ACM EuroSys, 2015Google’s cluster manager runs a huge mix of jobs on shared machines and packs them tightly so the machines stay busy. Many of its ideas carried over into Kubernetes.

Borg, Omega, and KubernetesBrendan Burns et al. · ACM Queue, 2016Google’s lessons from more than a decade of running containers at scale, across three systems. It explains why Kubernetes is built the way it is, from pods to labels.

Multi-resource packing for cluster schedulersRobert Grandl et al. · ACM SIGCOMM, 2014Most schedulers share out only CPU and memory, and leave scattered gaps of unused capacity. Tetris packs tasks by CPU, memory, disk and network together, and jobs finish faster on the same cluster.

Protean: VM Allocation Service at ScaleOri Hadary et al. · USENIX OSDI, 2020How Microsoft Azure picks the server for each new VM across very large zones. Its rule-based placement keeps servers busy while still meeting each customer’s needs.

Kubernetes Scheduling: Taxonomy, Ongoing Issues and ChallengesCarmen Carrión · ACM Computing Surveys, 2023A survey of research on the Kubernetes scheduler, sorted into clear groups. It is a good map of what has been tried for better resource use and where the gaps are.

Horizontal Pod Autoscaling in Kubernetes for Elastic Container OrchestrationThanh-Tung Nguyen et al. · Sensors (MDPI), 2020Experiments that show how the Kubernetes Horizontal Pod Autoscaler really behaves. It compares built-in resource metrics with custom metrics and gives practical tuning lessons.

Resource prediction and sizing

How much CPU and memory a workload really needs, and how to predict it from past use.

Resource Central: Understanding and Predicting Workloads for Improved Resource Management in Large Cloud PlatformsEli Cortez et al. · ACM SOSP, 2017A study of Microsoft Azure’s VM workload, which finds that a VM’s past behavior predicts its future well. A scheduler that uses these predictions can safely fit more VMs on each server.

Autopilot: workload autoscaling at GoogleKrzysztof Rzadca et al. · ACM EuroSys, 2020People set CPU and memory limits well above real use, to be safe. Google’s Autopilot sets them from history, and its jobs kept 23% unused headroom against 46% for hand-tuned jobs.

Borg: the next generationMuhammad Tirmazi et al. · ACM EuroSys, 2020Analyzes a month of 2019 data from eight Google clusters and compares it with the 2011 trace. It shows how Borg and its workload changed, and the trace is public for anyone to study.

Quasar: resource-efficient and QoS-aware cluster managementChristina Delimitrou and Christos Kozyrakis · ACM ASPLOS, 2014Most cloud clusters leave much of their capacity idle, partly because users guess their resource needs. Quasar asks for a performance target instead and picks the resources itself.

Take it to the limit: peak prediction-driven resource overcommitment in datacentersNoman Bashir et al. · ACM EuroSys, 2021Asks how far a machine can be safely overbooked if you can predict its peak use. In Google’s data centers, these policies gave machines 10 to 16% more usable CPU than no overbooking.

CherryPick: Adaptively Unearthing the Best Cloud Configurations for Big Data AnalyticsOmid Alipourfard et al. · USENIX NSDI, 2017The wrong VM type and cluster size can make a recurring job cost 2 to 3 times more on average. CherryPick finds the best or a near-best setup in a few test runs.

Cloud pricing, spot and reserved capacity

How cloud prices work, and how to mix on-demand, reserved and spot capacity to reduce cloud cost.

A view of cloud computingMichael Armbrust et al. · Communications of the ACM, 2010The Berkeley paper on cloud economics: no up-front commitment, pay for what you use, and capacity that seems endless. It also lists the top obstacles and opportunities for cloud computing.

Deconstructing Amazon EC2 Spot Instance PricingOrna Agmon Ben-Yehuda et al. · ACM Transactions on Economics and Computation, 2013The authors worked out how EC2 spot prices were set, using only the public price history. Their model suggests that, in that period, supply and demand rarely drove the price. Most of the time it looks like a random pick from a narrow band.

SpotCheck: designing a derivative IaaS cloud on the spot marketPrateek Sharma et al. · ACM EuroSys, 2015Spot servers are cheap but can be taken back at any time. SpotCheck carries that risk for its users. It runs their VMs on spot servers and moves them to on-demand servers when a spot server is taken back.

To Reserve or Not to Reserve: Optimal Online Multi-Instance Acquisition in IaaS CloudsWei Wang, Baochun Li and Ben Liang · USENIX ICAC, 2013Reserved instances cost less per hour, but only pay off if you use them. This paper decides when to reserve and when to stay on demand, without any forecast.

Providing SLOs for Resource-Harvesting VMs in Cloud PlatformsPradeep Ambati et al. · USENIX OSDI, 2020Azure has plenty of unused capacity, but the amount changes over time. Harvest VMs grow and shrink with it. With the paper’s scheduling framework, they matched the computing power of regular VMs and reduced cost by 91%.

Snape: Reliable and Low-Cost Computing with Mixture of Spot and On-Demand VMsFangkai Yang et al. · ACM ASPLOS, 2023Production traces show that spot VM evictions are partly predictable. Snape mixes spot and on-demand VMs and saved 44% against on-demand only, while keeping availability at 99.96%.

Block storage, file systems and space allocation

How file systems and cloud storage allocate space, keep data safe and avoid paying for empty capacity.

BTRFS: The Linux B-Tree FilesystemOhad Rodeh, Josef Bacik and Chris Mason · ACM Transactions on Storage, 2013The design of the Btrfs file system: copy-on-write, B-trees, checksums, snapshots and built-in RAID. It also covers spreading one file system over disks of different sizes.

Scalability in the XFS File SystemAdam Sweeney et al. · USENIX ATC, 1996How XFS was built to handle very large file systems, huge files and big directories. It explains how XFS allocates space, and why it uses B+ trees where older file systems used flat lists.

A five-year study of file-system metadataNitin Agrawal et al. · ACM Transactions on Storage, 2007Yearly snapshots of more than 60,000 Windows file systems in a large company, over five years. It tracks how file sizes, disk capacity and space used changed, a rare long view of how storage fills.

A Study of Practical DeduplicationDutch T. Meyer and William J. Bolosky · USENIX FAST, 2011Measures duplicate data on 857 desktop computers at Microsoft over four weeks. On live file systems, removing duplicate whole files saved about three quarters as much space as the most thorough block-level method.

Minerva: An automated resource provisioning tool for large-scale storage systemsGuillermo A. Alvarez et al. · ACM Transactions on Computer Systems, 2001Storage designed by hand often ends up over-provisioned, slow, or both. Minerva designed storage automatically, and in one test matched a 30-disk expert design with 16 disks.

Erasure Coding in Windows Azure StorageCheng Huang et al. · USENIX ATC, 2012Introduces the Local Reconstruction Codes that Azure uses to keep data durable with less extra space than full copies. Repairs stay fast because they read fewer pieces of data.

An In-Depth Analysis of Cloud Block Storage Workloads in Large-Scale ProductionJinhong Li et al. · IEEE IISWC, 2020Studies billions of block-level I/O requests from Alibaba Cloud’s production block storage and compares them with Microsoft Research’s public traces. It gives 15 findings on load and access patterns and releases the traces.

Live migration of VMs and containers

How a running VM or container moves to another machine with only a short pause.

Live Migration of Virtual MachinesChristopher Clark et al. · USENIX NSDI, 2005Shows how to move a running VM to another host while it keeps working. Most memory is copied while the VM runs, so the final pause is very short.

Post-copy live migration of virtual machinesMichael R. Hines, Umesh Deshpande and Kartik Gopalan · ACM SIGOPS Operating Systems Review, 2009Flips the usual order: move the VM’s CPU state first, then fetch memory pages as they are needed. Compared with pre-copy, this reduces total migration time and network traffic.

Live wide-area migration of virtual machines including local persistent stateRobert Bradford et al. · ACM VEE, 2007Moves a running VM across a wide-area network together with its local disk, not just its memory. Open network connections keep working while the VM moves.

Black-box and Gray-box Strategies for Virtual Machine MigrationTimothy Wood et al. · USENIX NSDI, 2007Sandpiper watches servers for hotspots and moves VMs off overloaded machines with live migration. It compares a black-box method with one that also reads OS and application statistics.

Optimal online deterministic algorithms and adaptive heuristics for energy and performance efficient dynamic consolidation of virtual machines in Cloud data centersAnton Beloglazov and Rajkumar Buyya · Concurrency and Computation: Practice and Experience (Wiley), 2012Uses live migration to pack VMs onto fewer servers and put idle servers to sleep. Its algorithms reduce energy use while keeping service levels, using each VM’s usage history.

VM Live Migration At ScaleAdam Ruprecht et al. · ACM VEE, 2018How Google’s cloud moves VMs off hosts for updates and repairs without stopping them. It runs over a million migrations a month, with a median pause of 50 ms.

Voyager: Complete Container State MigrationShripad Nadgowda et al. · IEEE ICDCS, 2017Moves a container with both its memory and its disk data, using CRIU and a union mount. Because both hosts share one view of the data, the container runs again right away and its disk data follows later.

Efficient Live Migration of Linux ContainersRadostin Stoyanov and Martin J. Kollingbaum · ISC High Performance 2018 Workshops (Springer LNCS), 2018Uses CRIU’s image cache and proxy to send a container’s checkpoint straight to the new host, skipping the disk. This shortens both total migration time and downtime.

Other researchers wrote every paper here. Each link goes to the publisher’s page. The short note under each paper is in our own words.

Find your own number

One line in your cloud console. One PDF report with your estimated saving at list price.

Estimate your saving

Talk to the builders

Thirty minutes with the people who build Coral and Amoeba.

Book a demo

Or write to founders@zoorik.com