Dynamic provisioning using FSx for Lustre

Now that we understand the FSx for Lustre storage class for Kubernetes, let's modify the UI component to mount the FSx for Lustre volume.

We'll update the UI component to reference the FSx for Lustre PVC:

Kustomize Patch

~/environment/eks-workshop/modules/fundamentals/storage/fsxl/deployment/deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: ui
spec:
  replicas: 2
  template:
    spec:
      initContainers:
        - name: fix-permissions
          image: public.ecr.aws/docker/library/busybox:1.37
          command: ["sh", "-c", "chown 1000:1000 /fsxl"]
          volumeMounts:
            - name: fsxlvolume
              mountPath: /fsxl
          securityContext:
            runAsUser: 0
      containers:
        - name: ui
          volumeMounts:
            - name: fsxlvolume
              mountPath: /fsxl
          env:
            - name: RETAIL_UI_PRODUCT_IMAGES_PATH
              value: /fsxl
      volumes:
        - name: fsxlvolume
          persistentVolumeClaim:
            claimName: fsxl-claim

Deployment/ui

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app.kubernetes.io/created-by: eks-workshop
    app.kubernetes.io/type: app
  name: ui
  namespace: ui
spec:
  replicas: 2
  selector:
    matchLabels:
      app.kubernetes.io/component: service
      app.kubernetes.io/instance: ui
      app.kubernetes.io/name: ui
  template:
    metadata:
      annotations:
        prometheus.io/path: /actuator/prometheus
        prometheus.io/port: "8080"
        prometheus.io/scrape: "true"
      labels:
        app.kubernetes.io/component: service
        app.kubernetes.io/created-by: eks-workshop
        app.kubernetes.io/instance: ui
        app.kubernetes.io/name: ui
    spec:
      containers:
        - env:
            - name: RETAIL_UI_PRODUCT_IMAGES_PATH
              value: /fsxl
            - name: JAVA_OPTS
              value: -XX:MaxRAMPercentage=75.0 -Djava.security.egd=file:/dev/urandom
            - name: METADATA_KUBERNETES_POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: METADATA_KUBERNETES_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
            - name: METADATA_KUBERNETES_NODE_NAME
              valueFrom:
                fieldRef:
                  fieldPath: spec.nodeName
          envFrom:
            - configMapRef:
                name: ui
          image: public.ecr.aws/aws-containers/retail-store-sample-ui:1.2.1
          imagePullPolicy: IfNotPresent
          livenessProbe:
            httpGet:
              path: /actuator/health/liveness
              port: 8080
            initialDelaySeconds: 45
            periodSeconds: 20
          name: ui
          ports:
            - containerPort: 8080
              name: http
              protocol: TCP
          resources:
            limits:
              memory: 1.5Gi
            requests:
              cpu: 250m
              memory: 1.5Gi
          securityContext:
            capabilities:
              add:
                - NET_BIND_SERVICE
              drop:
                - ALL
            readOnlyRootFilesystem: true
            runAsNonRoot: true
            runAsUser: 1000
          volumeMounts:
            - mountPath: /fsxl
              name: fsxlvolume
            - mountPath: /tmp
              name: tmp-volume
      initContainers:
        - command:
            - sh
            - -c
            - chown 1000:1000 /fsxl
          image: public.ecr.aws/docker/library/busybox:1.37
          name: fix-permissions
          securityContext:
            runAsUser: 0
          volumeMounts:
            - mountPath: /fsxl
              name: fsxlvolume
      securityContext:
        fsGroup: 1000
      serviceAccountName: ui
      volumes:
        - name: fsxlvolume
          persistentVolumeClaim:
            claimName: fsxl-claim
        - emptyDir:
            medium: Memory
          name: tmp-volume

Diff

     app.kubernetes.io/type: app
   name: ui
   namespace: ui
 spec:
-  replicas: 1
+  replicas: 2
   selector:
     matchLabels:
       app.kubernetes.io/component: service
       app.kubernetes.io/instance: ui
[...]
         app.kubernetes.io/name: ui
     spec:
       containers:
         - env:
+            - name: RETAIL_UI_PRODUCT_IMAGES_PATH
+              value: /fsxl
             - name: JAVA_OPTS
               value: -XX:MaxRAMPercentage=75.0 -Djava.security.egd=file:/dev/urandom
             - name: METADATA_KUBERNETES_POD_NAME
               valueFrom:
[...]
             readOnlyRootFilesystem: true
             runAsNonRoot: true
             runAsUser: 1000
           volumeMounts:
+            - mountPath: /fsxl
+              name: fsxlvolume
             - mountPath: /tmp
               name: tmp-volume
+      initContainers:
+        - command:
+            - sh
+            - -c
+            - chown 1000:1000 /fsxl
+          image: public.ecr.aws/docker/library/busybox:1.37
+          name: fix-permissions
+          securityContext:
+            runAsUser: 0
+          volumeMounts:
+            - mountPath: /fsxl
+              name: fsxlvolume
       securityContext:
         fsGroup: 1000
       serviceAccountName: ui
       volumes:
+        - name: fsxlvolume
+          persistentVolumeClaim:
+            claimName: fsxl-claim
         - emptyDir:
             medium: Memory
           name: tmp-volume

Apply these changes with the following command:

~$kubectl apply -k ~/environment/eks-workshop/modules/fundamentals/storage/fsxl/deployment

namespace/ui unchanged

serviceaccount/ui unchanged

configmap/ui unchanged

service/ui unchanged

deployment.apps/ui configured

~$kubectl rollout status --timeout=130s deployment/ui -n ui

Let's examine the volumeMounts in the deployment. Notice that our new volume named fsxlvolume is mounted at /fsxl:

~$kubectl get deployment -n ui \

-o yaml | yq '.items[].spec.template.spec.containers[].volumeMounts'

- mountPath: /fsxl

  name: fsxlvolume

- mountPath: /tmp

  name: tmp-volume

A PersistentVolume (PV) has been statically provisioned and bound to our PersistentVolumeClaim (PVC):

~$kubectl get pv

NAME      CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM            STORAGECLASS    AGE

fsxl-pv   1200Gi     RWX            Retain           Bound    ui/fsxl-claim    fsx-lustre-sc   5m

Let's examine the details of our PersistentVolumeClaim (PVC):

~$kubectl describe pvc -n ui fsxl-claim

Name:          fsxl-claim

Namespace:     ui

StorageClass:  fsx-lustre-sc

Status:        Bound

Volume:        fsxl-pv

Labels:        <none>

Annotations:   pv.kubernetes.io/bind-completed: yes

Finalizers:    [kubernetes.io/pvc-protection]

Capacity:      1200Gi

Access Modes:  RWX

VolumeMode:    Filesystem

Used By:       ui-5d4687cf64-phs2w

               ui-5d4687cf64-rc29s

Events:        <none>

At this point, the FSx for Lustre file system is successfully mounted but currently empty:

~$POD_1=$(kubectl -n ui get pods -l app.kubernetes.io/instance=ui -o jsonpath='{.items[0].metadata.name}')

~$kubectl exec --stdin $POD_1 -n ui -- bash -c 'ls /fsxl/'

Let's use a Kubernetes Job to populate the FSx for Lustre volume with images:

~$export PVC_NAME="fsxl-claim"

~$cat ~/environment/eks-workshop/modules/fundamentals/storage/populate-images-job.yaml | envsubst | kubectl apply -f -

~$kubectl wait --for=condition=complete -n ui \

job/populate-images --timeout=300s

Now let's demonstrate the shared storage functionality by listing the current files in /fsxl through one of the UI component Pods:

~$POD_1=$(kubectl -n ui get pods -l app.kubernetes.io/instance=ui -o jsonpath='{.items[0].metadata.name}')

~$kubectl exec --stdin $POD_1 -n ui -- bash -c 'ls /fsxl/'

1ca35e86-4b4c-4124-b6b5-076ba4134d0d.jpg

4f18544b-70a5-4352-8e19-0d070f46745d.jpg

631a3db5-ac07-492c-a994-8cd56923c112.jpg

79bce3f3-935f-4912-8c62-0d2f3e059405.jpg

8757729a-c518-4356-8694-9e795a9b3237.jpg

87e89b11-d319-446d-b9be-50adcca5224a.jpg

a1258cd2-176c-4507-ade6-746dab5ad625.jpg

cc789f85-1476-452a-8100-9e74502198e0.jpg

d27cf49f-b689-4a75-a249-d373e0330bb5.jpg

d3104128-1d14-4465-99d3-8ab9267c687b.jpg

d4edfedb-dbe9-4dd9-aae8-009489394955.jpg

d77f9ae6-e9a8-4a3e-86bd-b72af75cbc49.jpg

To further demonstrate the shared storage capabilities, let's create a new image called placeholder.jpg and add it to the FSx for Lustre volume through the first Pod:

~$POD_1=$(kubectl -n ui get pods -l app.kubernetes.io/instance=ui -o jsonpath='{.items[0].metadata.name}')

~$kubectl exec --stdin $POD_1 -n ui -- bash -c 'curl -sS -o /fsxl/placeholder.jpg https://placehold.co/600x400/jpg?text=EKS+Workshop\\nPlaceholder'

Now we'll verify that the second UI Pod can access this newly created file, demonstrating the shared nature of our FSx for Lustre storage:

~$POD_2=$(kubectl -n ui get pods -o jsonpath='{.items[1].metadata.name}')

~$kubectl exec --stdin $POD_2 -n ui -- bash -c 'ls /fsxl/'

1ca35e86-4b4c-4124-b6b5-076ba4134d0d.jpg

4f18544b-70a5-4352-8e19-0d070f46745d.jpg

631a3db5-ac07-492c-a994-8cd56923c112.jpg

79bce3f3-935f-4912-8c62-0d2f3e059405.jpg

8757729a-c518-4356-8694-9e795a9b3237.jpg

87e89b11-d319-446d-b9be-50adcca5224a.jpg

a1258cd2-176c-4507-ade6-746dab5ad625.jpg

cc789f85-1476-452a-8100-9e74502198e0.jpg

d27cf49f-b689-4a75-a249-d373e0330bb5.jpg

d3104128-1d14-4465-99d3-8ab9267c687b.jpg

d4edfedb-dbe9-4dd9-aae8-009489394955.jpg

d77f9ae6-e9a8-4a3e-86bd-b72af75cbc49.jpg

placeholder.jpg      <----------------

As you can see, even though we created the file through the first Pod, the second Pod has immediate access to it because they're both accessing the same shared FSx for Lustre file system.

Finally, let's confirm that the image is accessible through the UI service:

~$LB_HOSTNAME=$(kubectl -n ui get service ui-nlb -o jsonpath='{.status.loadBalancer.ingress[*].hostname}{"\n"}')

~$echo "http://$LB_HOSTNAME/assets/img/products/placeholder.jpg"

http://k8s-ui-uinlb-647e781087-6717c5049aa96bd9.elb.us-west-2.amazonaws.com/assets/img/products/placeholder.jpg

Visit the URL in your browser:

http://k8s-ui-uinlb-647e781087-6717c5049aa96b...

We've successfully demonstrated how Amazon FSx for Lustre provides high-performance shared parallel storage for workloads running on Amazon EKS. This solution allows multiple pods to read from and write to the same storage volume simultaneously, making it ideal for machine learning training data, HPC workloads, media processing, and other use cases requiring high-throughput parallel file system access.