LATEST 🚀 Welcome to Upgradix by Shrash Studio – Your destination for free technology learning and career growth.
🕘 Updated:
Kubernetes

Resource Limits & Horizontal Pod Autoscaling (HPA)

Running applications in Kubernetes without resource guardrails is a recipe for a production outage. One single container with a memory leak or an infinite loop can eat up all the CPU and RAM on a server node, crashing every other healthy microservice sharing that machine.

Kubernetes – Resource Limits & Horizontal Pod Autoscaling (HPA)

Running applications in Kubernetes without resource guardrails is a recipe for a production outage. One single container with a memory leak or an infinite loop can eat up all the CPU and RAM on a server node, crashing every other healthy microservice sharing that machine.

At the same time, user traffic is never flat. A flash sale or a marketing campaign can multiply your API requests by 10x in seconds. By combining Resource Requests & Limits with the Horizontal Pod Autoscaler (HPA), you protect your cluster from exhaustion and let Kubernetes automatically add or remove Pods to match real-time demand.

What you will learn
  • The exact difference between CPU/Memory requests and limits.
  • What happens under the hood when a container hits its CPU limit vs. its Memory limit.
  • How to write production-ready Deployment and HPA YAML manifests using autoscaling/v2.
  • How the HPA control loop calculates the exact number of Pods to scale.
  • Common production mistakes, scaling trade-offs, and senior-level interview questions.

1. What Are Resource Requests and Limits?

In Kubernetes, multiple Pods share the physical CPU and RAM of a worker node. To keep things fair and predictable, you tell Kubernetes how much power each container needs using two settings: Requests and Limits.

  • Requests (The Guarantee): This is the minimum amount of CPU and memory a container is guaranteed to get. The Kubernetes Scheduler looks at requests to decide which worker node has enough free space to place the Pod.
  • Limits (The Ceiling): This is the maximum amount of CPU and memory a container is allowed to use. Even if the node has extra free capacity, the container cannot cross this hard ceiling.
Resource Measurement Unit If Request Is Exceeded If Limit Is Exceeded
CPU Millicores (500m = 0.5 core, 1 = 1 full core) Allowed to burst if the node has spare CPU. Throttled: The container is slowed down, not killed.
Memory Bytes (256Mi = 256 Mebibytes, 1Gi = 1 Gibibyte) Allowed to burst if the node has spare RAM. OOMKilled: Linux kernel immediately kills the container (Exit Code 137).
Important: CPU is a compressible resource (it can be slowed down), while Memory is incompressible (you cannot take RAM back once allocated). That is why exceeding CPU causes latency, while exceeding Memory causes an instant crash.

2. Practical Implementation: Defining Requests, Limits & HPA

To make autoscaling work, you need two YAML files: a Deployment that explicitly defines resource requests and limits, and a HorizontalPodAutoscaler that watches that Deployment.

Step A: Deployment with Resource Requests and Limits

Example 1: deployment.yaml (Setting CPU & Memory guardrails)

apiVersion: apps/v1
kind: Deployment
metadata:
  name: order-api
spec:
  replicas: 2
  selector:
    matchLabels:
      app: order-api
  template:
    metadata:
      labels:
        app: order-api
    spec:
      containers:
      - name: order-api-container
        image: myregistry/order-api:1.0
        ports:
        - containerPort: 8080
        resources:
          requests:
            cpu: "250m"      # Guaranteed 0.25 CPU core
            memory: "256Mi"  # Guaranteed 256 MiB RAM
          limits:
            cpu: "500m"      # Hard ceiling at 0.5 CPU core
            memory: "512Mi"  # OOMKilled if it crosses 512 MiB

Step B: Configuring the Horizontal Pod Autoscaler (HPA)

Now we attach an HPA to the order-api Deployment. When the average CPU usage across all running Pods crosses 70% of their requested CPU (250m), Kubernetes will automatically spin up more Pods, up to a maximum of 10.

Example 2: hpa.yaml (Scaling automatically on traffic spikes)

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: order-api-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: order-api
  minReplicas: 2
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300

Apply both files and check HPA status

kubectl apply -f deployment.yaml
kubectl apply -f hpa.yaml
kubectl get hpa

Expected output (During a traffic spike)

NAME            REFERENCE              TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
order-api-hpa   Deployment/order-api   88%/70%   2         10        4          3m12s
Why stabilizationWindowSeconds matters: Setting stabilizationWindowSeconds: 300 tells Kubernetes to wait 5 minutes after a traffic spike drops before removing Pods. This prevents "thrashing" (rapidly creating and deleting Pods when traffic fluctuates).

3. Architecture & Workflow Visualized

The Horizontal Pod Autoscaler is not magic—it is a control loop inside the Kubernetes Control Plane that runs every 15 seconds by default. It relies on the Metrics Server to collect live CPU and memory numbers from every worker node.

How Kubernetes Horizontal Pod Autoscaling (HPA) Works

Traffic Spike High API Requests Running Pods (2) CPU hits 88% (> 70%) Scrapes Metrics Server Collects CPU/RAM via Kubelet Queries every 15s HPA Controller ceil(2 * 88 / 70) = 3 Pods Updates Deployment Scales replicas: 2 → 3 Creates Pod HPA Formula: Desired Replicas = ceil[ Current Replicas × ( Current Metric / Target Metric ) ]

Every 15 seconds, the HPA Controller reads live Pod metrics from the Metrics Server, calculates the required replica count, and updates the Deployment automatically.

4. Step-by-Step Execution Breakdown

Here is exactly what happens inside your cluster when traffic suddenly jumps:

Step 1: Users send a burst of requests to your order-api Pods. Each Pod's CPU usage climbs from 20% to 88% of its requested 250m CPU.
Step 2: The Metrics Server scrapes live resource usage from the kubelet running on each worker node and exposes it through the Kubernetes Metrics API.
Step 3: The HPA Controller wakes up (every 15 seconds) and compares the current average CPU (88%) against your target threshold (70%).
Step 4: The controller runs the scaling formula: ceil(2 * (88 / 70)) = ceil(2.51) = 3. It updates the Deployment's spec.replicas field from 2 to 3.
Step 5: The Deployment's ReplicaSet spins up a new Pod. Once the new Pod passes its readiness probe, Kubernetes routes traffic to it, bringing the average CPU back down below 70%.
Watch out: If you forget to define resources.requests.cpu in your Deployment YAML, the HPA will fail and display <unknown>/70% in the TARGETS column. Without a request value, Kubernetes has no baseline to calculate a percentage!

5. Hands-On Practice

Test your understanding by running these practical scenarios in a local cluster like Minikube, Kind, or Docker Desktop.

Beginner Exercise 1: Trigger an OOMKill on purpose

Deploy a container with a very low memory limit (such as 32Mi) and run a small script that allocates 50MB of memory. Watch the Pod status change to OOMKilled.

Hint: Run kubectl describe pod <pod-name> and look for Exit Code: 137.

Intermediate Exercise 2: Simulate a CPU traffic spike

Deploy the order-api Deployment and HPA from this guide. Use a load-generator Pod (like busybox running a while true; do wget ...; done loop) to hammer your service.

Expected result: Watch kubectl get hpa -w and verify that the REPLICAS count automatically climbs from 2 to 4+ within two minutes.

Advanced Exercise 3: Scale on both CPU and Memory

Update your hpa.yaml to include a second metric under spec.metrics that triggers scaling when average Memory utilization exceeds 80%.

Hint: When multiple metrics are defined, HPA calculates the desired replicas for each metric separately and picks the largest number.

6. Interview Questions

Resource management and autoscaling are favorite topics in senior backend and DevOps interviews. Here is how to answer them with confidence:

Q1. What is the difference between a Pod hitting its CPU limit vs. its Memory limit?

Answer: When a container hits its CPU limit, the Linux kernel throttles it—meaning the app slows down and latency increases, but the container stays alive. When a container hits its Memory limit, the Linux OOM (Out of Memory) killer immediately terminates the container with Exit Code 137, causing a restart.

Q2. Why does kubectl get hpa sometimes show <unknown> in the TARGETS column?

Answer: There are two main causes: either the Metrics Server is not installed or unhealthy in the cluster, or the target Deployment's containers do not have resources.requests defined. HPA needs the request value as the denominator to calculate utilization percentage.

Q3. What is the difference between HPA, VPA, and the Cluster Autoscaler?

Answer: HPA (Horizontal Pod Autoscaler) adds or removes more Pods (scaling out). VPA (Vertical Pod Autoscaler) increases or decreases the CPU/RAM of existing Pods (scaling up). Cluster Autoscaler adds or removes worker nodes (VMs/EC2 instances) when Pods are stuck in a Pending state because the cluster ran out of physical capacity.

Conclusion

Setting accurate CPU and memory requests and limits is the foundation of a stable Kubernetes cluster. It stops noisy neighbors from crashing your nodes and gives the scheduler the data it needs to pack workloads efficiently. Pairing those guardrails with a well-tuned Horizontal Pod Autoscaler ensures your applications stay fast and resilient during unexpected traffic spikes—without waking you up at 3 AM.

Official Documentation

Upgradix by Shrash Studio – Learn Technology. Build Your Future.

Comments & Ratings (0)

Log in or create a student account to comment and rate this article.

Be the first to comment on this article.