HA Control Plane — High Availability with kubeadm

Why HA Control Plane Matters

With a single control plane node:

• kube-apiserver unavailable → kubectl stops working; no new pods can be scheduled
• kube-controller-manager stops → Deployments stop self-healing
• kube-scheduler stops → no new pods get assigned to nodes
• etcd crashes → cluster state is lost (or recoverable only from backup)

Running workloads continue — kubelet on worker nodes keeps pods alive. But you cannot change anything until the control plane recovers.

etcd Quorum

etcd uses the Raft consensus algorithm. To commit a write, a majority (quorum) of members must acknowledge it:

Always use odd numbers — even numbers provide no additional fault tolerance but add more nodes to maintain quorum.

HA Topologies

Stacked etcd (recommended for most clusters)

  LB (HAProxy / cloud LB)
  ├── control-plane-1  [apiserver + etcd]
  ├── control-plane-2  [apiserver + etcd]
  └── control-plane-3  [apiserver + etcd]

• etcd runs on the same nodes as the control plane
• Minimum: 3 control plane nodes (etcd quorum of 3)
• Simpler: fewer machines, one kind of node to manage
• Risk: losing a control plane node also loses an etcd member

External etcd

  LB
  ├── control-plane-1  [apiserver only]
  ├── control-plane-2  [apiserver only]
  └── control-plane-3  [apiserver only]

  etcd-1  etcd-2  etcd-3  (separate nodes)

• etcd cluster is completely separate from control plane nodes
• More expensive (6+ machines), but control plane and etcd fail independently
• Required when you need etcd to survive total control plane loss

kubeadm HA Setup

First control plane node

kubeadm init \
  --control-plane-endpoint "lb.example.com:6443" \  # VIP or DNS of the load balancer
  --upload-certs \                                    # encrypt and store certs in Secret
  --pod-network-cidr "10.244.0.0/16"

Output includes two join commands:

1. For additional control plane nodes: kubeadm join lb.example.com:6443 --control-plane --certificate-key
2. For worker nodes: kubeadm join lb.example.com:6443 --token --discovery-token-ca-cert-hash sha256:

--upload-certs TTL

The --certificate-key is valid for 2 hours. After that, run kubeadm init phase upload-certs --upload-certs on the first control plane to generate a new key.

Leader Election

Only one kube-controller-manager and one kube-scheduler are active at a time, even in an HA setup. Leader election uses Lease objects in kube-system:

kubectl get lease -n kube-system
# NAME                      HOLDER                            AGE
# kube-controller-manager   control-plane-1_abc-uuid          10d
# kube-scheduler            control-plane-2_def-uuid          10d

The holder changes automatically within seconds if the current leader's pod crashes.