Kubernetes With Object Storage (EOS)

Mount an E2E Object Storage (EOS) bucket into your workloads using Datashim. Datashim's Dataset custom resource turns an S3-compatible bucket into a PersistentVolumeClaim that pods can mount like any other volume.

Prerequisites

• An active E2E Kubernetes cluster with kubectl configured.
• An E2E Object Storage bucket, plus its access key, secret key, and endpoint URL (from the E2E Cloud Console under Object Storage).

Step 1: Install Datashim

Datashim runs in its own dlf namespace. Create the namespace first, then apply the Datashim manifest:

kubectl create namespace dlf
kubectl apply -f https://raw.githubusercontent.com/datashim-io/datashim/master/release-tools/manifests/dlf.yaml

warning

Create the dlf namespace before applying the manifest. If it does not exist, the apply fails with namespaces "dlf" not found.

Wait for the Datashim pods to be ready, then enable the dataset admission webhook on the namespace where your workloads run (here, default):

kubectl wait --for=condition=ready pods -l app.kubernetes.io/name=datashim -n dlf --timeout=180s
kubectl label namespace default monitor-pods-datasets=enabled

Step 2: Create a Dataset

A Dataset describes the bucket and credentials. Save the following as dataset.yaml, replacing the placeholders with your EOS values:

apiVersion: datashim.io/v1alpha1
kind: Dataset
metadata:
  name: example-dataset
spec:
  local:
    type: "COS"
    accessKeyID: "<EOS_ACCESS_KEY_ID>"
    secretAccessKey: "<EOS_SECRET_ACCESS_KEY>"
    endpoint: "<EOS_ENDPOINT_URL>"
    bucket: "<BUCKET_NAME>"
    readonly: "true" # OPTIONAL, default is false
    region: "" # OPTIONAL, e.g. a region label such as "Delhi"

Apply it:

kubectl apply -f dataset.yaml

Datashim creates a PVC and a ConfigMap that share the Dataset's name. Confirm they exist:

kubectl get pvc,configmap

You should see a PVC and a ConfigMap named example-dataset.

Use a Secret for credentials

Embedding the access and secret keys directly in the Dataset is convenient for a quick test, but for anything beyond that, pass them through a Kubernetes Secret instead. See the Datashim S3 provisioning example.

Step 3: Mount the Dataset in a Pod

Reference the Dataset from a pod using labels - Datashim's webhook injects the volume automatically. Save as pod.yaml:

apiVersion: v1
kind: Pod
metadata:
  name: nginx
  labels:
    dataset.0.id: "example-dataset"
    dataset.0.useas: "mount"
spec:
  containers:
    - name: nginx
      image: nginx

kubectl apply -f pod.yaml

By convention, the Dataset is mounted at /mnt/datasets/example-dataset inside the container. Verify:

kubectl exec nginx -- ls /mnt/datasets/example-dataset

Refer to the Datashim documentation for advanced configuration.

---