How Poor Infrastructure Architecture Quietly Increases Your Cloud Bill

Cloud providers make it easy to provision infrastructure. You can spin up a VM in minutes, increase its CPU and memory, and deploy your application.

The problem is that provisioning infrastructure is easy. Designing it efficiently is much harder.

Consider a typical application deployed on virtual machines. You provision a VM with 8 vCPUs and 32 GB of RAM to handle peak traffic. However, during most of the day, the application uses only a fraction of those resources. You continue paying for the provisioned capacity even when the workload does not fully use it.

Now imagine several applications, each running on its own VM with individually allocated resources and limited flexibility to scale.

Over time, unused capacity accumulates. Development and testing environments remain running. Scaling becomes a manual exercise. Teams provision additional infrastructure to handle traffic spikes but rarely revisit their original allocations.

Individually, these decisions might appear reasonable. Collectively, they can become a significant source of cloud waste.

The issue is not necessarily the VM itself. It is the architecture and operating model surrounding it.

A VM runs its own operating system environment on virtualized infrastructure. A container packages an application and its dependencies while sharing the host operating system’s kernel. This difference creates opportunities for more efficient resource utilization.

With a VM-centric architecture, teams often allocate resources to individual application environments. Depending on the design, that capacity may remain reserved even when the application is idle.

With a containerized architecture, multiple application containers can run on a shared pool of compute resources. An orchestrator such as Kubernetes can schedule workloads according to their resource requests, constraints and availability requirements.

This can enable:

• Better resource utilization: multiple workloads can share the underlying compute capacity.
• More flexible scaling: applications can scale independently based on demand.
• Improved workload placement: an orchestrator can schedule containers across available nodes.
• More consistent deployments: standardized packaging simplifies application delivery across environments.
• Greater visibility: resource requests, limits and workload metrics help teams identify overprovisioning.

However, containers do not automatically reduce cloud costs.

A poorly configured Kubernetes cluster with oversized nodes, excessive replicas, unrealistic resource requests and idle environments can cost more than a well-designed VM-based architecture. The most common form of this waste is resource requests: pods reserving more CPU and memory than they ever use, so the cluster keeps buying nodes for capacity nobody consumes.

We see this in our own environment too. On our own Kubernetes development cluster, the applications were reserving almost twice the CPU they actually needed. Right-sizing their requests, removing an idle node and rebalancing onto a cheaper machine cut the cluster’s machine bill by 58% in eight days.

The goal is not to replace every VM with a container. The goal is to select an architecture that matches the workload and uses provisioned resources efficiently.

1. Overprovisioning compute and storage

When applications are sized for their theoretical peak requirements instead of measured demand, organizations often pay for capacity that remains unused.

For example, an application might require substantial resources during a short processing window but consume very little for the rest of the day. A permanently oversized VM can be an inefficient solution for that workload. Depending on the application’s characteristics, alternatives could include smaller, right-sized VMs, scheduled scaling, containers or serverless compute.

The same pattern shows up in storage. Disks are often sized for the largest dataset a team expects, and Azure managed disks and AWS EBS volumes can grow but cannot be shrunk in place. Capacity bought for a one-time peak can stay on the bill for years.

The right answer comes from workload analysis, not assumptions.

2. Running workloads in isolation when they could share capacity

Deploying every application on a separate VM can make sense when isolation, security, licensing or operational requirements demand it.

But when workloads are compatible and have complementary resource patterns, dedicated infrastructure may introduce unnecessary fragmentation. Containerization can allow multiple workloads to share a compute pool while retaining application-level separation.

The potential savings come from reducing unused capacity and improving packing efficiency, not simply from using containers.

3. Scaling infrastructure instead of optimizing applications

Sometimes the easiest response to performance issues is to increase CPU and memory.

But what if the real problem is inefficient database queries, memory leaks, excessive logging, unnecessary background processing or poor caching?

Adding infrastructure may temporarily improve performance, but it can also hide the underlying issue and increase recurring costs. An engineering-led cost review should examine application performance alongside infrastructure utilization.

4. Ignoring non-production environments

Development, QA, staging and temporary testing environments can quietly consume resources around the clock.

If those environments do not need to run continuously, scheduled shutdowns, automated environment provisioning and cleanup policies can deliver savings without changing the production architecture. This is often one of the simplest opportunities to investigate before undertaking a major migration.

5. Designing for peak traffic without a scaling strategy

Production systems must handle traffic spikes, but keeping enough infrastructure running for the absolute peak at all times may be unnecessarily expensive.

Depending on the workload, horizontal scaling, autoscaling, queue-based processing and asynchronous execution can help align capacity with actual demand. The architecture must still preserve availability, acceptable latency and resilience during scaling events.

Cost optimization should never mean compromising reliability.

When investigating an infrastructure bill, the first step should not be to start changing VM sizes or migrating applications. It should be to understand the relationship between architecture, workload behavior and spending.

Our engineering approach follows a few fundamental steps.

1. Establish a baseline

Review cloud expenditure, compute utilization, CPU and memory metrics, traffic patterns, storage, network charges and environment-level spending. Identify which workloads contribute most to the bill and when their resource consumption actually occurs.

2. Understand the workload

Determine whether the application is CPU-intensive, memory-intensive, traffic-driven, batch-oriented or continuously running. A stateless web service, a stateful database and a nightly batch-processing job should not automatically receive the same infrastructure design.

3. Compare realistic architecture options

Evaluate whether the workload is best suited to VMs, containers, managed container platforms, serverless services or a hybrid approach. Include migration effort, platform operating costs, licensing, observability, networking, security and operational complexity in the comparison.

4. Model the expected savings

Estimate the difference between the existing setup and the proposed architecture. Account for actual resource demand, high availability, autoscaling headroom, idle capacity and any additional platform costs.

Avoid comparing only the price of one VM with the price of a container. They are not equivalent units of infrastructure.

5. Validate through measurement

Run a representative workload test or controlled pilot where possible. Compare cost per request, cost per transaction, resource utilization, latency, error rates and availability.

A lower monthly bill is useful. A lower cost while maintaining the required service level is a much stronger result.

6. Prevent the same waste from returning

Introduce resource tagging, budgets, utilization alerts, right-sizing reviews, autoscaling policies and infrastructure standards. Cost optimization should become part of the engineering lifecycle rather than a one-time exercise.

The client was running 12 applications on 18 VMs: six customer-facing services on two VMs each, four internal services on one VM each, and two databases. The services ran on 4 vCPU, 16 GB general-purpose machines and the databases on 8 vCPU, 32 GB ones. Over 30 days, their average CPU utilization was 11% and memory utilization 31%.

We moved the 16 service VMs onto one managed Kubernetes cluster of four 8 vCPU, 32 GB nodes spread across availability zones, with each service sized from its measured usage and the customer-facing ones scaling out on demand. The two databases stayed on VMs, moving to memory-optimized 4 vCPU machines with the same 32 GB of RAM.

The monthly compute bill, at on-demand prices, went from about $2,800 to about $1,560, a 44% reduction, while p95 latency, error rates and availability stayed within the client’s targets through a two-week side-by-side pilot.

Figures are rounded, and some details are changed to keep the client anonymous.

Cloud cost optimization is frequently treated as a finance exercise: review the bill, find expensive services, and negotiate or purchase discounts.

Those activities can help, but they do not address every source of waste. Discounts can reduce the price of infrastructure. They cannot fix an application that has been overprovisioned for years. Monitoring can expose unused capacity. It cannot, by itself, redesign the system that created that capacity.

The more sustainable approach is to bring engineering, architecture and FinOps together. Engineers understand how workloads behave. Architects determine how systems should be structured. FinOps helps connect technical decisions to business outcomes.

When these perspectives work together, cloud cost optimization becomes an ongoing engineering discipline.

The lesson from our client engagement was not that VMs are bad or that containers are always better. It was that infrastructure decisions deserve to be challenged continuously.

Why does this application need this much compute? Could workloads share resources safely? Does capacity need to run all day? Is the system scaling because demand has increased, or because the application itself is inefficient?

These questions can reveal opportunities that are easy to miss when cloud cost optimization begins and ends with the billing dashboard.

The most effective cloud cost optimization often starts before the next resource is provisioned, with a better architectural decision.

What has your experience been? Have you seen cloud costs increase because of architectural decisions that initially seemed perfectly reasonable? We would love to hear your perspective.

Vineet is an infrastructure engineer at Zoorik and led the client engagement described here. Zoorik builds Kubernetes and block storage cost optimization for Azure and AWS. To see where your own cloud spend is going, try the free assessment at zoorik.com/assessment.