--- title: "Use persistent volumes" description: "Keep data on a TIR Kubernetes cluster with the default StorageClass, claim volumes for pods and StatefulSets, and see what ReadWriteOnce allows." content_type: "how-to" product: "tir-kubernetes" platform: "TIR" url: "/tir/tir-kubernetes/how-to/use-persistent-volumes/" topic: ["workloads"] --- # Use persistent volumes Keep the data your pods write on a persistent volume, which is storage that stays when a pod is restarted or moved. A pod's own filesystem is lost when the pod is rescheduled. A node's local disk goes with the node when its pool is scaled down. Every TIR Kubernetes cluster has a default StorageClass, so you can create a PersistentVolumeClaim right away. ## Before you begin You need a cluster in the **Ready** state and its kubeconfig set up for `kubectl`. See [Connect to a Kubernetes cluster](/tir/tir-kubernetes/how-to/connect-to-a-cluster/). ## The default StorageClass Every cluster you create in the portal comes with Ceph RBD block storage behind a default StorageClass, which supports ReadWriteOnce volumes. The create form shows it under **Cluster storage** for information only. - Each claim creates its own volume, so a claim is all you write. - The access mode is ReadWriteOnce. - The create form shows cluster storage as a usage-based estimate per GB per month. See [E2E's pricing page](https://www.e2enetworks.com/pricing). - The volumes are deleted with the cluster. Check that the class is there, and which one is the default: ```bash kubectl get storageclass ``` The default class is marked `(default)`. A claim that skips the StorageClass field uses it. ## What ReadWriteOnce allows ReadWriteOnce means that one node at a time mounts the volume for reading and writing. Several pods can share it only while they run on the same node. | You want | Works? | What to do | | -------------------------------------------------- | ------ | ------------------------------------------------------------------------------------------ | | One pod writing to one volume | Yes | Create one claim and mount it in the pod | | Several replicas, each with its own volume | Yes | Use a StatefulSet with volume claim templates | | Several pods on one node sharing a volume | Yes | Schedule them onto the same node with pod affinity | | Several pods on different nodes sharing one volume | No | Have one pod write and the others read through an API, or keep the files in object storage | The built-in storage offers ReadWriteOnce volumes only. A claim that asks for ReadWriteMany stays `Pending`, and a second pod on another node that mounts the same ReadWriteOnce volume stays in ContainerCreating. TIR [Shared File System](/tir/tir-sfs/) and [Parallel File System](/tir/tir-pfs/) are separate products, and they are not mounted into Kubernetes clusters. For files that many pods read, use [TIR Object Storage](/tir/tir-object-storage/). ## Claim a volume A PersistentVolumeClaim asks for storage, and the StorageClass creates a volume for it. This claim asks for 20 GiB: ```yaml apiVersion: v1 kind: PersistentVolumeClaim metadata: name: app-data spec: accessModes: - ReadWriteOnce resources: requests: storage: 20Gi ``` ```bash kubectl apply -f pvc.yaml kubectl get pvc app-data ``` The claim reads `Bound` once its volume exists. A class whose binding mode is WaitForFirstConsumer waits for a pod to use the claim, so the claim reads `Pending` until then. The `VOLUMEBINDINGMODE` column of `kubectl get storageclass` shows which mode the default class uses. ### Mount the volume in a pod Name the claim under `volumes` and mount it in the container: ```yaml apiVersion: apps/v1 kind: Deployment metadata: name: app spec: replicas: 1 selector: matchLabels: app: app template: metadata: labels: app: app spec: containers: - name: app image: nginx volumeMounts: - name: data mountPath: /var/lib/app volumes: - name: data persistentVolumeClaim: claimName: app-data ``` Check that the data stays after a restart: ```bash kubectl exec deploy/app -- sh -c 'echo hello > /var/lib/app/test.txt' kubectl rollout restart deploy/app kubectl exec deploy/app -- cat /var/lib/app/test.txt ``` The last command prints `hello`. Keep a Deployment that shares one ReadWriteOnce claim at one replica. With several replicas, it works only while every replica lands on the same node. When each replica needs storage of its own, use a StatefulSet. ## Give each replica its own volume with a StatefulSet For databases, and anything else where each replica needs its own disk, let a StatefulSet create one claim per replica. The Secret `db-secret` holds the database password under the key `password`. Create it first, with your own password in place of ``: ```bash kubectl create secret generic db-secret --from-literal=password= ``` Then apply the StatefulSet: ```yaml apiVersion: apps/v1 kind: StatefulSet metadata: name: db spec: serviceName: db replicas: 3 selector: matchLabels: app: db template: metadata: labels: app: db spec: containers: - name: db image: postgres:16 env: - name: POSTGRES_PASSWORD valueFrom: secretKeyRef: name: db-secret key: password volumeMounts: - name: data mountPath: /var/lib/postgresql/data volumeClaimTemplates: - metadata: name: data spec: accessModes: - ReadWriteOnce resources: requests: storage: 50Gi ``` This creates the claims `data-db-0`, `data-db-1`, and `data-db-2`, each bound to its own pod. Each is ReadWriteOnce, which suits replicas that each keep a disk of their own. To list them, run: ```bash kubectl get pvc -l app=db ``` ## Make a volume larger A volume can grow if its StorageClass allows expansion. Check the class first: ```bash kubectl get storageclass -o custom-columns=NAME:.metadata.name,EXPANSION:.allowVolumeExpansion ``` If `EXPANSION` reads `true`, raise the claim's requested size and watch it change: ```bash kubectl patch pvc app-data -p '{"spec":{"resources":{"requests":{"storage":"40Gi"}}}}' kubectl get pvc app-data -w ``` Volumes can only grow. To make one smaller, create a smaller volume and copy the data across. ## Delete a volume Whether deleting a claim also deletes its volume depends on the StorageClass's reclaim policy. Check it before you delete: ```bash kubectl get storageclass -o custom-columns=NAME:.metadata.name,RECLAIM:.reclaimPolicy ``` `Delete` removes the volume and its data with the claim. `Retain` keeps the volume, and you clean it up yourself. ```bash kubectl delete pvc app-data ``` ## Keep data safe when nodes change Before you scale a worker pool down, make sure that your workloads have replicas on other nodes and keep their data on persistent volumes. The nodes removed from the pool take their local disks with them. See [Resize or autoscale a worker pool](/tir/tir-kubernetes/how-to/resize-or-autoscale-a-worker-pool/). Deleting the cluster permanently deletes its storage and every volume on it. Copy anything you need out of the cluster first. See [Delete a Kubernetes cluster](/tir/tir-kubernetes/how-to/delete-a-cluster/). Your pods now keep their data across restarts. For a claim that stays `Pending` or a pod that cannot mount its volume, see [Troubleshooting TIR Kubernetes](/tir/tir-kubernetes/troubleshooting/tir-kubernetes/#a-persistentvolumeclaim-stays-pending).