Kubernetes Resource Limits: CPU, Memory, and Quality of Service

Kubernetes clusters have finite resources. Nodes have CPU and memory that pods consume. Without resource constraints, one pod can starve others, degrade cluster stability, and make scheduling unpredictable. Kubernetes provides mechanisms to declare resource requirements, set hard limits, and classify pods by their importance.

This post explains requests vs limits, Quality of Service classes, and the namespace-level constraints that keep clusters healthy.

If you need Kubernetes fundamentals first, see the Kubernetes fundamentals post. For advanced scheduling patterns, see the Advanced Kubernetes post.

Introduction

Nodes have finite CPU and memory. Pods consume both. Without explicit resource declarations, the scheduler picks nodes based purely on capacity, and running containers can grab as much as they want. The result is noisy neighbors starving other pods, or node-level OOM events taking down services you did not even know were sharing the machine.

Resource requests and limits fix this. Requests tell the scheduler what a container needs to run. Limits cap what it can consume before Kubernetes steps in. Those declarations also assign your pods a Quality of Service class that determines eviction order when resources run short. Getting these values right is one of the most direct things you can do for cluster stability.

This post walks through requests and limits, the three QoS classes, and the namespace constraints (LimitRange, ResourceQuota) that keep one team from eating the entire cluster. You will also learn how to read actual resource usage with kubectl top and Vertical Pod Autoscaler so you are not guessing in the dark.

Requests vs Limits Explained

Every container in a pod can specify resource requests and resource limits:

apiVersion: v1
kind: Pod
metadata:
  name: web-app
  namespace: production
spec:
  containers:
    - name: web-app
      image: nginx:1.25
      resources:
        requests:
          memory: "256Mi"
          cpu: "250m"
        limits:
          memory: "512Mi"
          cpu: "500m"

Requests define what a container needs. The scheduler uses requests to decide which node to place the pod on. A node must have at least as much allocatable resources as the pod’s requests.

Limits define the maximum resources a container can use. When a container hits its memory limit, Kubernetes terminates and restarts it. When it hits its CPU limit, Kubernetes throttles the container.

CPU representation

CPU is measured in cores. You can express it as a whole number (1 CPU = 1 core) or millicores (1000m = 1 CPU). Common values:

• 100m = 0.1 CPU (one-tenth of a core)
• 250m = 0.25 CPU
• 500m = 0.5 CPU
• 1000m = 1 CPU

CPU is compressible. If a container exceeds its CPU limit, Kubernetes throttles it. The container does not get killed for CPU alone.

Memory representation

Memory is measured in bytes. You can use suffixes: Ki (kibibytes), Mi (mebibytes), Gi (gibibytes).

• 128Mi = 128 mebibytes (~134 MB)
• 256Mi = 256 mebibytes (~268 MB)
• 1Gi = 1 gibibyte (~1.07 GB)

Memory is not compressible. If a container exceeds its memory limit, Kubernetes terminates it with an OOMKilled status.

What happens when limits are exceeded

State: Terminated
Reason: OOMKilled
Exit Code: 137

Exit code 137 indicates the container was killed by the OOM (Out of Memory) killer. Frequent OOMKilled pods indicate you need to increase memory limits or optimize the application’s memory usage.

QoS Classes

Kubernetes assigns pods to Quality of Service classes based on their resource requests and limits:

QoS Class	Criteria	Behavior
Guaranteed	requests == limits for all containers	Last to be evicted
Burstable	requests < limits (or some limits not set)	Evicted after Guaranteed
BestEffort	No requests or limits set	First to be evicted

When to Use Each QoS Class

QoS Class	When to Use	When NOT to Use
Guaranteed	Critical infrastructure pods, databases, licensed software with strict resource needs	When you do not need eviction priority guarantees
Burstable	Most application workloads that benefit from burst capacity	When every pod needs identical resource guarantees
BestEffort	Batch jobs, environments where pods are truly disposable	Production workloads, anything that matters

Rule of thumb: Most workloads should be Burstable. Set Guaranteed for workloads that must not be evicted under any circumstances. Never run production workloads as BestEffort.

QoS Decision Flow

flowchart TD
    A[Pod submitted] --> B{Requests == Limits\nfor all containers?}
    B -->|Yes| G[Guaranteed QoS]
    B -->|No| C{Any requests\nor limits set?}
    C -->|Yes| Bu[Burstable QoS]
    C -->|No| BE[BestEffort QoS]
    Bu --> D{Node under\nmemory pressure?}
    BE --> D
    G --> D
    D -->|Guaranteed| L1[Last to evict]
    D -->|Burstable| L2[Middle eviction]
    D -->|BestEffort| L3[First to evict]

Guaranteed pods

containers:
  - name: database
    image: postgres:15
    resources:
      limits:
        memory: "2Gi"
        cpu: "2000m"
      requests:
        memory: "2Gi"
        cpu: "2000m"

Pods with identical requests and limits get the highest QoS. Use this for critical workloads that should not be evicted.

BestEffort pods

containers:
  - name: batch-job
    image: my-batch-job
    resources: {}

No resource specifications means BestEffort. These pods are first in line for eviction when the node runs low on resources.

Burstable pods

containers:
  - name: web-app
    image: nginx:1.25
    resources:
      requests:
        memory: "128Mi"
        cpu: "100m"
      limits:
        memory: "512Mi"
        cpu: "500m"

Most pods fall into Burstable. They have some guaranteed resources but can burst above their requests when available.

LimitRange for Namespace Quotas

A LimitRange sets default, minimum, and maximum resource limits for pods and containers in a namespace. Without it, pods without resource specs become BestEffort.

apiVersion: v1
kind: LimitRange
metadata:
  name: default-limits
  namespace: production
spec:
  limits:
    - max:
        memory: "4Gi"
        cpu: "2000m"
      min:
        memory: "64Mi"
        cpu: "50m"
      default:
        memory: "256Mi"
        cpu: "250m"
      defaultRequest:
        memory: "128Mi"
        cpu: "100m"
      type: Container

This LimitRange:

• Sets maximum memory and CPU per container
• Sets minimum memory and CPU per container
• Applies default limits when containers specify no limits
• Applies default requests when containers specify no requests

Without this LimitRange, a container with no resource specs gets no guaranteed resources and can be evicted first.

Applying LimitRange

kubectl apply -f limitrange.yaml
kubectl describe limitrange default-limits -n production

The output shows the actual limits applied:

Type        Resource  Min   Max   Default Request  Default Limit
Container    cpu       50m   2     100m             250m
Container    memory    64Mi  4Gi   128Mi            256Mi

ResourceQuota for Cluster-Wide Limits

A ResourceQuota limits total resource consumption in a namespace. Use it to prevent any single namespace from consuming all cluster resources.

apiVersion: v1
kind: ResourceQuota
metadata:
  name: production-quota
  namespace: production
spec:
  hard:
    requests.cpu: "10"
    requests.memory: "20Gi"
    limits.cpu: "20"
    limits.memory: "40Gi"
    pods: "50"
    persistentvolumeclaims: "10"

This quota limits the entire production namespace to 10 CPU requests, 20Gi memory requests, 20 CPU limits, 40Gi memory limits, 50 pods, and 10 persistent volume claims.

Viewing quota usage

kubectl describe resourcequota production-quota -n production

The output shows current usage against the hard limits. When a quota is exhausted, Kubernetes rejects new resource creation in that namespace.

Pod Resource Testing and Tuning

Finding the right requests and limits takes measurement. Kubernetes lets you profile pod behavior before setting values in production.

kubectl run with resource specs

kubectl run -it --rm load-generator \
  --image=busybox \
  --restart=Never \
  -- requests.cpu=500m \
  -- requests.memory=128Mi \
  --limits.cpu=1000m \
  --limits.memory=256Mi \
  -- sh

Use this to run temporary pods and observe resource consumption with monitoring tools.

Vertical Pod Autoscaler

The Vertical Pod Autoscaler (VPA) analyzes historical resource usage and recommends or automatically applies better resource requests:

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: web-app-vpa
  namespace: production
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web-app
  updatePolicy:
    updateMode: "Auto"

In Auto mode, VPA evicts and reschedules pods with updated resource specs. In Off mode, it only provides recommendations.

VPA helps you find baseline resource requirements without manual profiling.

Horizontal Pod Autoscaler

The Horizontal Pod Autoscaler (HPA) scales pod replicas based on CPU, memory, or custom metrics:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: web-app-hpa
  namespace: production
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web-app
  minReplicas: 3
  maxReplicas: 10
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70

The HPA scales between 3 and 10 replicas to maintain 70% average CPU utilization. For memory-based scaling:

metrics:
  - type: Resource
    resource:
      name: memory
      target:
        type: Utilization
        averageUtilization: 80

Trade-off Analysis

Requests vs Limits Trade-offs

Aspect	Requests Only	Requests + Limits
Scheduling	Predictable node placement	Predictable scheduling + resource capping
QoS class	Burstable (best available)	Guaranteed if equal, otherwise Burstable
Memory protection	None	Prevents OOMKill from affecting node
CPU behavior	No throttle, unlimited burst	CPU throttled at limit
Production readiness	Insufficient	Required for production

QoS Class Trade-offs

QoS Class	Scheduling guarantee	Eviction priority	Resource efficiency	Use when
Guaranteed	Highest	Last evicted	Low (reserved full)	Critical infrastructure, licensed software
Burstable	Medium	Middle	High (flexible)	Most application workloads
BestEffort	None	First evicted	Highest	Batch jobs only

Trade-off reality: Guaranteed pods reserve their full resource request even when idle. BestEffort pods get whatever is left, making them efficient but risky for production.

LimitRange vs ResourceQuota

Aspect	LimitRange	ResourceQuota
Scope	Per namespace, per container	Per namespace, total namespace
What it controls	Min/max/default requests/limits	Total CPU, memory, pod count
Enforcement	Applied when pod is created	Checked when resources are created
Use case	Prevent BestEffort pods, set defaults	Prevent namespace from monopolizing cluster

Vertical Pod Autoscaler vs Manual Resource Tuning

Aspect	VPA (Automated)	Manual Tuning
Effort	Low	High (profiling required)
Precision	Based on historical data	Based on engineered judgment
Disruption	Pod evictions in Auto mode	None (no changes unless you apply)
Best for	Initial baseline discovery	Production fine-tuning

CPU Throttling Trade-offs

Aspect	With CPU Limits	Without CPU Limits (Guaranteed)
Latency	p99 spikes from throttling	Consistent latency
Resource cost	More efficient node utilization	May over-provision requests
Fairness	Prevents one pod dominating CPU	Pod can use all allocated CPU
Recommendation	Avoid for latency-sensitive svcs	Use for critical services

For latency-sensitive services where consistent p99 latency matters, set CPU limits equal to requests for Guaranteed QoS, or remove CPU limits entirely and rely on requests for scheduling.

Memory Limit Trade-offs

Aspect	Low memory limit	High memory limit
OOM risk	Frequent OOMKilled pods	Rare OOMKilled pods
Node efficiency	Higher (more pods per node)	Lower (reserved memory)
Debugging	Easier to spot (frequent crashes)	May hide memory leaks
Recommendation	Set ~20-30% above expected peak	Set for expected peak + headroom

Production Failure Scenarios

BestEffort Pods Evicted Under Memory Pressure

Pods without resource requests (BestEffort QoS) are the first to be evicted when a node runs low on memory. In a cluster with many BestEffort pods, eviction events can be frequent.

Symptoms: Frequent pod evictions in kubectl get events, pods restart unexpectedly.

Mitigation: Set resource requests for all production pods. Use LimitRange to enforce minimum resource requests per namespace.

OOMKilled Pods from Memory Limit Too Low

When a container exceeds its memory limit, Kubernetes kills it with OOMKilled status. This is one of the most common production issues I see. The fix is almost always raising the memory limit.

Check actual memory usage first:

kubectl describe pod <pod-name>  # Look for OOMKilled in state
kubectl top pod <pod-name>  # Check actual memory usage

Set limits about 20-30% above what you see in staging to account for traffic spikes.

CPU Throttling Impacting Latency

CPU limits throttle containers even when the node has free CPU. For latency-sensitive services, this creates annoying tail latency spikes.

High p99 latency with low average CPU usage usually means CPU throttling. For these workloads, either remove CPU limits or set them equal to requests for a Guaranteed QoS pod.

Anti-Patterns

Setting Identical Requests and Limits for All Pods

Treating every pod the same wastes resources. A web server handling 1000 req/s has different needs than a batch job processing queues.

Profile each application type separately and set appropriate resource specs.

Not Setting Memory Limits

Memory limits prevent runaway processes from consuming all node memory and causing node-level OOM events. Always set memory limits, especially for applications that can experience memory leaks.

Over-Provisioning CPU Limits

Setting CPU limits very high (like 4 cores for a simple web server) defeats the purpose of limits. The scheduler uses requests for node allocation decisions, not limits.

Set CPU limits based on actual expected peak load, not theoretical maximum.

Interview Questions

Further Reading

• Kubernetes Official Documentation - Managing Resources
• QoS Classes
• LimitRange
• ResourceQuota
• Vertical Pod Autoscaler
• Horizontal Pod Autoscaler

Conclusion

Use this checklist when configuring Kubernetes resource limits:

• Set resource requests for all production containers (scheduler uses these for placement)
• Set memory limits to prevent runaway processes from impacting node stability
• Set CPU limits based on actual application needs, not maximum theoretical values
• Used LimitRange to enforce default requests/limits per namespace
• Used ResourceQuota to cap total namespace resource consumption
• Profile applications with VPA or monitoring tools before setting production values
• Monitor OOMKilled events and adjust memory limits accordingly
• Monitor CPU throttling metrics and adjust CPU limits if latency issues appear
• Set Guaranteed QoS (requests == limits) for critical infrastructure pods
• Avoid BestEffort for any production workload