Kubernetes Storage: PersistentVolumes, Claims, and StorageClasses

Stateless applications are straightforward to run in Kubernetes. Pods get scheduled, they do their work, and if they die, Kubernetes replaces them. Stateful applications are different. A database needs its data to survive pod restarts. A message queue requires persistent storage for messages in transit. For these cases, Kubernetes provides PersistentVolumes, PersistentVolumeClaims, and StorageClasses.

This post covers how persistent storage works in Kubernetes, from basic volume attachment to dynamic provisioning across cloud providers.

If you are new to Kubernetes, start with the Kubernetes fundamentals post. For StatefulSet configuration, see the Kubernetes Workload Resources post.

PV and PVC Lifecycle

A PersistentVolume (PV) is a piece of storage in the cluster. It is a cluster-wide resource, not tied to any specific namespace. A PersistentVolumeClaim (PVC) is a request for storage by a user or pod.

The lifecycle of a PV follows these phases:

1. Provisioning: PV is created statically or dynamically
2. Binding: PVC binds to a PV that satisfies its requirements
3. Using: Pod uses the mounted volume
4. Releasing: Pod releases the claim (PVC deleted)
5. Reclaiming: PV is retained, recycled, or deleted based on its reclaim policy

PersistentVolume example

apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv-fast-storage
spec:
  capacity:
    storage: 100Gi
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: fast-ssd
  awsElasticBlockStore:
    volumeID: vol-0abc123def456
    fsType: ext4

PersistentVolumeClaim example

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: database-storage
  namespace: production
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 50Gi
  storageClassName: fast-ssd

Using PVC in a Pod

apiVersion: v1
kind: Pod
metadata:
  name: database
  namespace: production
spec:
  containers:
    - name: postgres
      image: postgres:15
      volumeMounts:
        - name: data
          mountPath: /var/lib/postgresql/data
  volumes:
    - name: data
      persistentVolumeClaim:
        claimName: database-storage

The PVC must exist in the same namespace as the pod. The PV does not need to be in the same namespace (PVs are cluster-wide).

Access modes

Mode	Abbreviation	Description
ReadWriteOnce	RWO	Single node can mount as read-write
ReadOnlyMany	ROX	Multiple nodes can mount as read-only
ReadWriteMany	RWX	Multiple nodes can mount as read-write

Not all storage providers support all modes. AWS EBS supports RWO only. NFS supports RWX. Azure Files supports RWO, ROX, RWX.

When to Use Each Access Mode

Access Mode	Use Case	Limitations
ReadWriteOnce	Single-pod databases, app data	No multi-node read-write
ReadOnlyMany	Shared config files, read-only data	No writes at all
ReadWriteMany	Multi-node file sharing, shared data volumes	Not supported by cloud block storage

Rule of thumb: Default to ReadWriteOnce for databases and stateful apps. Use ReadWriteMany only when multiple pods across nodes genuinely need write access (NFS, shared file systems).

PV Lifecycle Flow

flowchart LR
    A[Provision] --> B[Bind: PVC matches PV]
    B --> C[Pod uses volume]
    C --> D[Release: PVC deleted]
    D --> E{Reclaim Policy}
    E -->|Retain| F[Manual cleanup<br/>or restore]
    E -->|Delete| G[Volume deleted]
    E -->|Recycle| A

Static vs Dynamic Provisioning

Static provisioning means you create PVs manually before any PVC requests come in. You know the exact storage you have and you allocate it to workloads manually.

Dynamic provisioning creates PVs automatically when a PVC requests them. Kubernetes uses a StorageClass to determine what kind of storage to provision.

Static provisioning

# Admin creates the PV
apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv-manual-001
spec:
  capacity:
    storage: 200Gi
  accessModes:
    - ReadWriteOnce
  storageClassName: manual
  hostPath:
    path: /data/pv-manual-001

# User creates PVC
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: my-app-storage
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 50Gi
  storageClassName: manual

Static provisioning gives administrators precise control. It also requires manual tracking of available storage capacity.

Dynamic provisioning

Dynamic provisioning kicks in when a PVC specifies a StorageClass and no matching PV exists. The StorageClass provisioner creates a new PV automatically.

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: fast-ssd
provisioner: kubernetes.io/aws-ebs
parameters:
  type: gp3
  fsType: ext4
  iops: "3000"
  throughput: "125"
volumeBindingMode: WaitForFirstConsumer

The WaitForFirstConsumer binding mode delays PV creation until a pod actually uses the PVC. This lets the scheduler place the pod in the same availability zone as the storage, avoiding cross-zone traffic costs.

StorageClass Providers

Different cloud providers and storage systems expose different provisioners:

Provider	Provisioner	Notes
AWS	kubernetes.io/aws-ebs	gp3, gp2, io1, st1, sc1
GCP	kubernetes.io/gce-pd	pd-standard, pd-ssd
Azure	kubernetes.io/azure-disk	Standard_LRS, Premium_LRS
NFS	nfs.subvol.io (external)	Requires NFS server
Local	kubernetes.io/no-provisioner	For local disks

AWS EBS StorageClass

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: gp3-storage
provisioner: kubernetes.io/aws-ebs
parameters:
  type: gp3
  fsType: ext4
  iops: "3000"
  throughput: "125"
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: true

The allowVolumeExpansion: true field lets you expand volumes without recreating the PVC. You still need to update the PVC spec to request more storage.

NFS StorageClass

NFS works across multiple availability zones and supports ReadWriteMany access mode:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: nfs-storage
provisioner: nfs.subvol.io
parameters:
  server: nfs-server.example.com
  share: /exports
mountOptions:
  - nfsvers=4.1
volumeBindingMode: Immediate

StatefulSet Volume Claim Templates

StatefulSets use volumeClaimTemplates to provision storage for each replica automatically:

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: postgres-cluster
  namespace: database
spec:
  serviceName: postgres-cluster
  replicas: 3
  selector:
    matchLabels:
      app: postgres
  template:
    metadata:
      labels:
        app: postgres
    spec:
      containers:
        - name: postgres
          image: postgres:15
          volumeMounts:
            - name: data
              mountPath: /var/lib/postgresql/data
  volumeClaimTemplates:
    - metadata:
        name: data
      spec:
        accessModes: ["ReadWriteOnce"]
        storageClassName: fast-ssd
        resources:
          requests:
            storage: 100Gi

For StatefulSet with 3 replicas, Kubernetes creates 3 PVCs: data-postgres-cluster-0, data-postgres-cluster-1, data-postgres-cluster-2. Each PVC binds to its own PV with independent lifecycle.

When you scale down the StatefulSet, the PVCs for removed pods remain. You need to manually clean them up or configure automatic deletion (not default behavior).

CSI Drivers and Abstraction

The Container Storage Interface (CSI) is a standard for storage plugins. CSI drivers replace the in-tree volume plugins (like aws-ebs, gce-pd) and provide a cleaner abstraction.

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: ebs-sc
provisioner: ebs.csi.aws.com
parameters:
  type: gp3
  csi.storage.k8s.io/fstype: ext4
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: true

CSI drivers get installed as pods and communicate with the storage provider. AWS EBS CSI driver, GCP Persistent Disk CSI driver, and others are maintained by cloud providers and the community.

Benefits of CSI:

• Vendors can release storage plugins without modifying Kubernetes core
• CSI drivers run as pods, not in-tree components
• Standardized interface across storage backends

Data Backup Considerations

PersistentVolumes do not get backed up automatically. You need explicit backup strategies:

Volume snapshots

apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshot
metadata:
  name: postgres-snapshot
  namespace: database
spec:
  volumeSnapshotClassName: ebs-snapshot-class
  source:
    persistentVolumeClaimName: data-postgres-cluster-0

Backup tools

Tool	Description
Velero	Backup and restore K8s resources and PVs
Kasten K10	Policy-driven data protection
aws-cli / gcloud	Cloud provider snapshot tools

Velero backs up PV snapshots to object storage and can restore entire applications:

velero backup create database-backup --include-namespaces database
velero restore create --from-backup database-backup

Conclusion

Persistent storage in Kubernetes involves three layers: PersistentVolumes (actual storage), PersistentVolumeClaims (requests), and StorageClasses (provisioning logic). Static provisioning gives you manual control. Dynamic provisioning scales automatically using StorageClass drivers.

StatefulSets use volumeClaimTemplates to create per-replica PVCs with independent lifecycle. CSI drivers provide a standardized interface for storage backends across cloud providers.

Remember that PVs do not get backed up automatically. Plan for snapshots and backup tools like Velero to protect your data. For more on running stateful workloads, see the Kubernetes Workload Resources post and the Advanced Kubernetes post.

Production Failure Scenarios

Volume Remains Stuck in “Pending” State

A PVC stays in Pending when no PV satisfies its requirements. This happens when the StorageClass does not exist, the provisioner is not running, or no PV matches the requested size/access mode.

Symptoms: Pending PVC, no events showing provisioning.

Diagnosis:

kubectl describe pvc <name> -n <namespace>
kubectl get storageclass
kubectl get pods -n kube-system  # Check provisioner pods

Mitigation: Verify the StorageClass exists and the provisioner is running. Check that the requested size and accessMode are supported by the storage backend.

Pod Fails to Start Due to Volume Mount Timeout

If a volume is mounted from a remote storage system and the network path is slow or unavailable, the pod can fail to start within the default mount timeout.

Symptoms: ContainerCreating state, MountVolume.SetUp timed out in events.

Mitigation: Increase the mount timeout with the mountOptions field in the StorageClass. Use local storage when possible for latency-sensitive workloads.

Volume Capacity Exhaustion

When a PersistentVolume fills up, the application writing to it fails. Database workloads are particularly vulnerable.

Symptoms: No space left on device errors in application logs, pod enters CrashLoopBackOff.

Mitigation: Set allowVolumeExpansion: true on the StorageClass to expand volumes without recreation. Monitor volume capacity and set up alerts at 80% usage. Implement cleanup policies for old data.

Anti-Patterns

Using ReadWriteMany for Single-Pod Databases

AWS EBS supports ReadWriteOnce only. If you see ReadWriteMany in a cloud database manifest, something is wrong. Most cloud block storage does not support concurrent mounts.

Not Setting volumeBindingMode: WaitForFirstConsumer

WaitForFirstConsumer is the right default for most use cases. Without it, your PV might get provisioned in a different availability zone than your pod, which means cross-zone traffic and unexpected costs.

Manually Creating PVs for StatefulSets

StatefulSets should use volumeClaimTemplates for dynamic volume provisioning. Manually creating PVs for each replica defeats the purpose of dynamic provisioning and makes scaling painful.

Quick Recap Checklist

Use this checklist when working with Kubernetes storage:

• Chose the correct access mode (RWO for databases, RWX for shared file storage)
• Used StorageClass with volumeBindingMode: WaitForFirstConsumer to avoid cross-zone costs
• Enabled allowVolumeExpansion: true on the StorageClass for production
• Used volumeClaimTemplates with StatefulSets, not manually created PVs
• Set up VolumeSnapshots for critical PVs before any major changes
• Monitored volume capacity and set alerts at 80% usage
• Used CSI drivers instead of in-tree provisioners for cloud storage
• Configured Retain reclaim policy for PVs that must survive PVC deletion
• Tested backup and restore procedures in staging
• Used local storage (HostPath or Local PV) for latency-critical workloads where possible