Horizontal Pod Autoscaler (HPA)

The Problem with Manual Scaling

In dynamic cloud-native environments, manual scaling via kubectl scale deployment --replicas=N is insufficient. Real-world traffic spikes (e.g., flash sales, batch reports, media coverage) require real-time, automated reaction.

The Horizontal Pod Autoscaler (HPA) implements a closed-loop feedback controller to scale Pods horizontally based on resource metrics or custom metrics.

HPA Control Loop Architecture

The HPA controller runs inside the kube-controller-manager as a periodic loop (configured by the --horizontal-pod-autoscaler-sync-period flag, defaulting to 15 seconds).

Autoscaling Metrics Pipeline

┌──────────────────────────────────────────────────────────────────┐

│ WORKER NODES │

│ ┌─────────────────────────┐ ┌─────────────────────────┐ │

│ │ Node 1 │ │ Node 2 │ │

│ │ [Pod A] [Pod B] │ │ [Pod C] [Pod D] │ │

│ │ (cgroups v2 stats) │ │ (cgroups v2 stats) │ │

│ │ │ │ │ │ │ │

│ │ ▼ │ │ ▼ │ │

│ │ [ Kubelet / cAdvisor ]│ │ [ Kubelet / cAdvisor ]│ │

│ └──────────┬──────────────┘ └──────────┬──────────────┘ │

└─────────────┼──────────────────────────────────┼─────────────────┘

│ (/stats/summary API) │ (/stats/summary API)

▼ ▼

┌────────────────────────────────────────────────┐

│ METRICS SERVER │

│ (Exposes metrics.k8s.io via API Aggregator) │

└───────────────────────┬────────────────────────┘

│ gRPC / HTTPS Poll

▼

┌────────────────────────────────────────────────┐

│ HPA CONTROLLER LOOP │

│ (Calculates: desiredReplicas) │

└───────────────────────┬────────────────────────┘

│ Scale Subresource Write (spec.replicas)

▼

┌────────────────────────────────────────────────┐

│ DEPLOYMENT CONTROLLER │

│ (Creates/Terminates Pod replicas) │

└────────────────────────────────────────────────┘

1. Metric Sourcing: The local kubelet daemon queries resource stats from the host OS's cgroups (v1 or v2) using its internal cAdvisor library. These stats are exposed on the /stats/summary endpoint.
2. Metrics Collection: The metrics-server scrapes these endpoints across all nodes, aggregates the data, and exposes it via the API Aggregator under the metrics.k8s.io API group.
3. Evaluation: The HPA controller queries the API server for these metrics and computes the desired replica count.

The Scaling Algorithm

The replica calculation uses the following formula:

desiredReplicas = ceil(currentReplicas × (currentMetricValue / targetMetricValue))

To prevent rapid oscillation (flapping) and scale-down instability:

• Tolerance: If the ratio of current to target metric is within a tolerance threshold (default: 0.1 or 10%), the controller skips scaling.
• Unready/Missing Pods: If a Pod is unready (e.g., starting up) or has missing metrics, the HPA controller excludes it or makes conservative estimates to prevent prematurely shrinking or growing the workload.

Requirements and Best Practices

• Mandatory Resource Requests: You must define resource requests (spec.containers[].resources.requests) for HPA. The utilization target is calculated as a percentage of the requested CPU/memory, not the node's total physical capacity.
• Cool-Down / Stabilization Window: Scale-up is typically fast to handle traffic spikes. Scale-down uses a stabilization window (default is 5 minutes / 300s) to wait for temporary traffic drops to pass before evicting pods.

Advanced Custom and External Metrics (HPA v2)

The autoscaling/v2 API version adds support for complex scaling conditions:

• Multiple Metrics: Evaluate CPU, memory, and custom metrics simultaneously (the HPA uses the largest calculated replica count).
• Custom Metrics: Scale based on application metrics (e.g., HTTP requests/sec from Prometheus via the custom.metrics.k8s.io API).
• External Metrics: Scale based on external queue sizes (e.g., AWS SQS queue depth, GCP Pub/Sub backlog via external.metrics.k8s.io).
• Scale Behavior Policies: Explicitly tune the scale-up and scale-down rates via spec.behavior.

Vertical Pod Autoscaler (VPA)

While HPA scales the number of replicas, VPA scales the resource requests on existing Pods — it adjusts how much CPU/memory each individual Pod is allocated.

How VPA Works

VPA watches actual resource usage over time and computes recommended requests. It then optionally applies them by evicting Pods so they restart with the new requests.

VPA Modes

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: my-app-vpa
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: my-app
  updatePolicy:
    updateMode: "Off"    # Start with Off — inspect recommendations first

VPA vs HPA

⚠️ Do not enable both HPA (CPU-based) and VPA (Auto) on the same Deployment — they will fight each other. Use VPA Off mode for recommendations only, or use VPA for memory-only recommendations alongside CPU-based HPA.

Install VPA

VPA is not built into Kubernetes. Install from the kubernetes/autoscaler repository:

kubectl apply -f https://github.com/kubernetes/autoscaler/releases/latest/download/vertical-pod-autoscaler.yaml

Verify the VPA components are running:

kubectl get pods -n kube-system | grep vpa
# vpa-admission-controller-xxx   Running
# vpa-recommender-xxx            Running
# vpa-updater-xxx                Running