P Peter Lee - Blog
cloud

Deploy NFS StorageClass for Kubernetes

Use an NFS provisioner to provide persistent storage for a private Kubernetes cluster.

peterlee
peterlee
Aug 27, 2019
English繁中
Deploy NFS StorageClass for Kubernetes

This 2019 lab used an NFS-backed StorageClass and the old nfs-client-provisioner. The manifests below record that setup, including its extensions/v1beta1 Deployment. For a new cluster, use the maintained NFS subdirectory external provisioner and its installation instructions.

Related post: Start a Kubernetes cluster with kubeadm

In the private Kubernetes cluster from the previous post, Pods may be recreated on different nodes, so application data needs persistent storage. I chose NFS as a straightforward storage option for this lab and began by setting up an NFS server.

Install an NFS server on CentOS

The NFS server is another Linux machine on the same local network, at 192.168.2.104.

$ yum install nfs-utils

# share /opt/nfs to 192.168.2.1/24 with read/write access.
# modify mount point
# /etc/exports
/opt/nfs 192.168.2.1/24(rw,sync,no_root_squash,no_all_squash)

$ sudo systemctl restart nfs-server
$ sudo systemctl enable nfs-server

Test that the NFS server is ready

$ showmount -e 192.168.2.104
Export list for 192.168.2.104:
/opt/nfs 192.168.2.1/24

This configuration exposes NFS to the specified internal subnet. Limit access to the nodes that need it, and review whether clients should retain root privileges through no_root_squash.

Define the provisioner and StorageClass

Save the following as nfs-storage.yml. The PROVISIONER_NAME must match the StorageClass provisioner, and NFS_SERVER and NFS_PATH must match the server.

# nfs-storage.yml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: nfs-client-provisioner

---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
  name: nfs-storage
spec:
  replicas: 1
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: nfs-client-provisioner
    spec:
      serviceAccountName: nfs-client-provisioner
      containers:
        - name: nfs-client-provisioner
          image: quay.io/external_storage/nfs-client-provisioner:latest
          volumeMounts:
            - name: nfs-client-root
              mountPath: /persistentvolumes
          env:
            - name: PROVISIONER_NAME
              value: nfs-storage
            - name: NFS_SERVER
              value: 192.168.2.104
            - name: NFS_PATH
              value: /opt/nfs
      volumes:
        - name: nfs-client-root
          nfs:
            server: 192.168.2.104
            path: /opt/nfs
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: managed-nfs-storage
provisioner: nfs-storage
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: default
provisioner: nfs-storage

Add RBAC for nfs-client-provisioner

# nfs-rbac.yaml
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: nfs-client-provisioner-runner
rules:
  - apiGroups: [""]
    resources: ["persistentvolumes"]
    verbs: ["get", "list", "watch", "create", "delete"]
  - apiGroups: [""]
    resources: ["persistentvolumeclaims"]
    verbs: ["get", "list", "watch", "update"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["storageclasses"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["create", "update", "patch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: run-nfs-client-provisioner
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
    namespace: default
roleRef:
  kind: ClusterRole
  name: nfs-client-provisioner-runner
  apiGroup: rbac.authorization.k8s.io
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
rules:
  - apiGroups: [""]
    resources: ["endpoints"]
    verbs: ["get", "list", "watch", "create", "update", "patch"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
    # replace with namespace where provisioner is deployed
    namespace: default
roleRef:
  kind: Role
  name: leader-locking-nfs-client-provisioner
  apiGroup: rbac.authorization.k8s.io

Apply both files, using the same namespace for the provisioner and its RBAC bindings:

kubectl apply -f nfs-rbac.yaml
kubectl apply -f nfs-storage.yml

Set the default StorageClass

To use NFS for new PVCs that do not specify a StorageClass, mark the selected class as the default:

kubectl patch storageclass managed-nfs-storage \
  -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'

See changing the default StorageClass for the full rules.

Create a test PVC and confirm it becomes Bound, a PV is created, and the corresponding directory appears on the NFS share.

#kubernetes#cloud