Resource Requests & Limits

Requests vs Limits: Linux Kernel Implementation

Kubernetes manages container resources using the Linux kernel's Control Groups (cgroups) mechanism. When a container runtime spins up a container, it translates resource fields into cgroups configurations on the host. Modern systems run cgroups v2, which simplifies directory structures and controller coordination compared to cgroups v1.

Visualizing CPU Throttling vs Memory OOMKills

       +---------------------------------------------+
       |             CONTAINER RUNTIME               |
       |     (Applies limits to cgroups v2)          |
       +---------------------------------------------+
              |                               |
    (CPU: Compressible)             (Memory: Incompressible)
              |                               |
              v                               v
    +-------------------+           +-------------------+
    | cpu.max (cgroup)  |           | memory.max (cg)   |
    | (CFS Bandwidth)   |           |                   |
    +-------------------+           +-------------------+
              |                               |
   Container uses > limit          Container uses > limit
              |                               |
              v                               v
    +-------------------+           +-------------------+
    |  Proactive        |           |  Reactive         |
    |  Throttling       |           |  OOM Killer       |
    |                   |           |  (SIGKILL sent)   |
    +-------------------+           +-------------------+
 (Slows down, continues)             (Process terminated)

CPU (Compressible Resource)

• Requests: Map to cpu.weight (in cgroups v2) or cpu.shares (in cgroups v1). This is a relative weight that determines how much CPU time a container gets under contention. If a node's CPU is idle, a container can consume 100% of the CPU regardless of its request. If multiple containers compete, the kernel allocates CPU cycles proportionally based on their weight.
• Limits: Map to cpu.max (cgroups v2) or cpu.cfs_quota_us and cpu.cfs_period_us (cgroups v1). This is enforced via the Completely Fair Scheduler (CFS) bandwidth controller. If a limit is set to 200m (0.2 cores) with a default period of 100,000 microseconds (100ms), the container is allocated a quota of 20,000 microseconds of CPU time per period. If it exhausts this quota in the first 20ms, it is throttled (suspended) for the remaining 80ms. It is not killed.

Memory (Incompressible Resource)

• Requests: Used primarily by the kube-scheduler to find a node with sufficient allocatable memory capacity. Kubelet also uses it to calculate QoS class eviction priority. It does not enforce a hard limit on startup.
• Limits: Map to memory.max (cgroups v2) or memory.limit_in_bytes (cgroups v1). Unlike CPU, memory cannot be throttled. If a container's processes attempt to allocate memory beyond this limit, the kernel triggers the Out-Of-Memory (OOM) Killer. The OOM killer selects and terminates the process within the container using SIGKILL, resulting in an exit code 137 and an OOMKilled status.

Quality of Service (QoS) Classes and Eviction

Kubernetes groups Pods into three QoS classes based on their resource settings, which determine their eviction priority when the node is under resource pressure:

1. Guaranteed: Every container in the Pod has both CPU and memory requests and limits set, and the request value exactly equals the limit value.
2. Burstable: The Pod does not meet the Guaranteed criteria, but at least one container has a CPU or memory request or limit set.
3. BestEffort: No containers have any requests or limits defined.

Eviction and OOM Score Adjustment

When a node experiences memory pressure, the kubelet eviction manager monitors memory metrics. If the node falls below the hard eviction threshold (e.g., memory.available < 100Mi), it evicts pods in the following order: BestEffort -> Burstable -> Guaranteed.

At the OS kernel level, when the system runs out of physical memory, the kernel OOM killer uses the process's oom_score to decide what to kill. The kubelet configures the kernel's oom_score_adj for each container process based on its QoS class:

• Guaranteed: Gets an oom_score_adj of -997. This makes these processes highly resistant to being killed by host OOM events.
• BestEffort: Gets an oom_score_adj of 1000. These processes are targeted first.
• Burstable: Gets an oom_score_adj calculated dynamically:

oom_score_adj = 1000 - (10 * memoryRequest / nodeCapacity)

This ensures that Burstable pods that request a larger percentage of the node's memory have a lower oom_score_adj and are safer than those that request less but consume more.

LimitRange and ResourceQuota

LimitRange — sets default requests/limits and min/max bounds for a namespace. Pods without resource declarations get the LimitRange defaults applied automatically.

ResourceQuota — caps the total resources (CPU, memory, pod count) that an entire namespace can consume. Used to prevent one team from monopolising cluster resources.

Common Mistakes

• No limits → noisy neighbor problem: one misbehaving pod can consume all node memory and trigger mass evictions
• No requests → scheduler is blind; it may over-schedule pods onto a node that appears to have capacity but actually doesn't
• Limits too low → app is constantly throttled or OOMKilled; always benchmark before setting limits