Using CIFS volumes in Kubernetes

Kubernetes with CIFS

Even though it is clearly stated in the Kubernetes official documentation that hostPath volumes pose security risks and their use should be avoided when possible, you would be surprised by how many companies I have seen using hostPath in their production applications. I recently got to work on a use case where the client had an on-prem Kubernetes cluster and was currently using HostPath volumes, and now wanted to use a CIFS/SMB drive to persist the data.

CIFS is a network file-sharing protocol that allows users to access files on remote servers as if they were on their local computers. It is a dialect of the Server Message Block (SMB) protocol, which was originally developed by IBM and Microsoft in the 1980s. Even though CIFS is outdated and has been replaced by SMB3, people still use CIFS and SMB interchangeably. However, if you go through the supported volumes in the Kubernetes documentation — you will find that CIFS is missing. This is because CIFS is not natively supported by Kubernetes.

Kubernetes supports CSI(Container Storage Interface) which allows storage vendors to develop a plugin once and have it work across several container orchestration (CO) systems such as Kubernetes, Mesos, Docker Swarm, etc. Taking advantage of this, someone has developed an excellent driver for CIFS volumes for Kubernetes — https://github.com/fstab/cifs

The installation and usage of this plugin is straightforward. The plugin first needs to be installed onto the master node and all of the worker nodes.

1. Identify the plugin directory for your Kubernetes setup. Execute the commandps aux|grep kubelet on the node and check the --volume-plugin-dir parameter . In my case, it is/var/lib/kubernetes/plugins

2. Then, navigate to this directory and download the plugin inside this directory in its folder and make sure it has the appropriate permissions:

cd /var/lib/kubernetes/plugins
mkdir fstab-cifs
cd fstab-cifs
curl -L -O https://raw.githubusercontent.com/fstab/cifs/master/cifs
chmod 755 cifs

3. Verify if the plugin is installed properly. Go inside the plugin directory and execute the init command

cd cifs init

It should output a JSON string containing "status": "Success". This command is also run by Kubernetes itself when the CIFS plugin is detected on the file system.

4. Now, that the plugin is installed, the next step would be to use it. Gather the details of your CIFS mount. The username and password will go into a Kubernetes Secret, while the network path will go into the PersistentVolume configuration. We will need the following details:

username=test
password=superstrongPassword
networkPath="//NFSSERVER/APP1"

5. Create a Kubernetes Secret for your credentials in the same namespace as your application:

apiVersion: v1
kind: Secret
metadata:
  name: app1-cifs-creds
  namespace: example
type: fstab/cifs
data:
  username: 'dGVzdA=='
  password: 'c3VwZXJTdHJvbmdQYXNzd29yZA=='

6. Create a PV and PVC for the app

apiVersion: v1
kind: PersistentVolume
metadata:
  name: app1-pv
spec:
  capacity:
    storage: 10Gi
  accessModes:
    - ReadWriteMany
  persistentVolumeReclaimPolicy: Retain
  csi:
    driver: smb.csi.k8s.io
    volumeAttributes:
      source: //NFS_SERVER/APP 
    volumeHandle: app1-cifs-volume
    nodeStageSecretRef:
      name: app1-cifs-creds
    namespace: example
   mountOptions:
     - dir_mode=0775
     - file_mode=0775
---
apiVersion: v1
kind: PersistentVolumeClaim 
metadata: 
  name: app1-pvc
namespace: example
spec: 
  accessModes:
    - ReadWriteMany 
resources:    
  requests:      
    storage: 10Gi
  volumeName: app1-pv

7. Use the created PVC in a pod to persist app data in a CIFS drive

apiVersion: v1
kind: Pod
metadata:
  name: app1-pod
spec:
  containers:
    - name: task-pv-container
      image: nginx
      ports:
        - containerPort: 80
          name: "http-server"
      volumeMounts:
        - mountPath: "/usr/share/nginx/html"
          name: app1-pv-storage
  volumes:
    - name: app1-cifs-volume
      persistentVolumeClaim:
        claimName: app1-pvc #link PVC created earlier

BINGO!! and this is how you can use CIFS volumes in Kubernetes. I hope this helps.

References:

1. https://kubernetes.io/docs/concepts/storage/volumes
2. https://learn.microsoft.com/en-us/windows/win32/fileio/microsoft-smb-protocol-and-cifs-protocol-overview
3. https://github.com/fstab/cifs
4. https://kubernetes.io/docs/tasks/configure-pod-container/configure-persistent-volume-storage/