Upgrading a Cluster with kubeadm

Execute a safe, one-minor-version-at-a-time cluster upgrade: control plane first, then workers — with zero application downtime.

advanced5 hoursRead → Lab → Quiz → Practice

🧠 Brain Warm-Up

🧠 Brain Warm-Up: Kubernetes components tolerate limited version skew between each other. The kubelet on a node may be one minor version behind the API server, but not two. What would happen if you upgraded a 5-node cluster's control plane from 1.28 to 1.30 in one jump, skipping 1.29? Think about what breaks before reading.

Kubernetes Upgrade Strategy

Kubernetes follows a strict N-1 version skew policy between components. You must upgrade one minor version at a time:

Allowed:   v1.28 → v1.29 → v1.30
Forbidden: v1.28 → v1.30   (skips v1.29)

Why One Minor Version at a Time?

The kube-apiserver and kubelet may differ by at most one minor version. If you skip a version:

The kubeadm upgrade apply logic may not have migration steps for the skipped version
API deprecations that were removed in the skipped version cause silent failures
etcd schema changes may be incompatible across two minor versions

Upgrade Sequence Overview

Control Plane Node:
  ① apt-mark unhold kubeadm && apt-get install kubeadm=1.30.0-00
  ② kubeadm upgrade plan              ← shows available upgrade targets
  ③ kubeadm upgrade apply v1.30.0    ← upgrades: apiserver, controller-manager,
  │                                      scheduler, etcd manifests, and kubeadm addons
  ④ apt-get install kubelet=1.30.0-00 kubectl=1.30.0-00
  ⑤ systemctl daemon-reload && systemctl restart kubelet
  ⑥ kubectl get nodes                 ← control plane node shows v1.30.0

Worker Nodes (repeat for each):
  ① kubectl drain <node> --ignore-daemonsets --delete-emptydir-data
  ② On the worker: apt-get install kubelet=1.30.0-00 kubectl=1.30.0-00
  ③ systemctl daemon-reload && systemctl restart kubelet
  ④ kubectl uncordon <node>           ← scheduling resumes

What kubeadm upgrade apply Does

kubeadm upgrade apply updates the control plane static pod manifests and the cluster ConfigMaps (kubeadm-config, kubelet-config) but does not upgrade the kubelet binary. The kubelet must be upgraded separately on every node.

Node Drain Deep Dive

kubectl drain node-2 --ignore-daemonsets --delete-emptydir-data
    │
    ├─ Cordon the node (SchedulingDisabled) — no new pods land here
    ├─ Evict all regular pods (graceful termination, PodDisruptionBudgets respected)
    ├─ --ignore-daemonsets: skip DaemonSet pods (they can't be moved anyway)
    └─ --delete-emptydir-data: force-evict pods using emptyDir (data is lost)

Result: node appears "Ready,SchedulingDisabled" in kubectl get nodes

Comparison: Control Plane vs Worker Upgrade

Step	Control Plane	Worker Node
Update kubeadm	Yes (required first)	No (kubeadm upgrade node, not apply)
Run upgrade command	kubeadm upgrade apply	kubeadm upgrade node
Drain before upgrade	Optional (taint manually)	Required
Upgrade kubelet	Yes	Yes
Uncordon after	N/A if not drained	Yes

←

etcd — Backup, Restore & Health

PKI & Certificate Management

→