NFS因为本身的问题，不罕用于生产环境，这里仅作为demo展现动静存储的应用。生产环境能够应用ceph，rook-ceph来治理ceph存储。

假如已部署好NFS Server，这里演示如何在集群中部署动静存储与创立storageclass/pvc/pv。

1.创立独立的namespace

# kubectl create ns storage

2.创立rbac给serviceAccount赋权

创立一个serviceAccount:

apiVersion: v1kind: ServiceAccountmetadata:  name: nfs-client-provisioner  namespace: storage

为serviceAccount赋权：

kind: ClusterRoleapiVersion: rbac.authorization.k8s.io/v1metadata:  name: nfs-client-provisioner-runnerrules:  - apiGroups: [""]    resources: ["persistentvolumes"]    verbs: ["get", "list", "watch", "create", "delete"]  - apiGroups: [""]    resources: ["persistentvolumeclaims"]    verbs: ["get", "list", "watch", "update"]  - apiGroups: ["storage.k8s.io"]    resources: ["storageclasses"]    verbs: ["get", "list", "watch"]  - apiGroups: [""]    resources: ["events"]    verbs: ["list", "watch", "create", "update", "patch"]  - apiGroups: [""]    resources: ["endpoints"]    verbs: ["get", "list", "watch", "create", "update", "patch"]
kind: ClusterRoleBindingapiVersion: rbac.authorization.k8s.io/v1metadata:  name: run-nfs-client-provisionersubjects:  - kind: ServiceAccount    name: nfs-client-provisioner    namespace: storageroleRef:  kind: ClusterRole  name: nfs-client-provisioner-runner  apiGroup: rbac.authorization.k8s.io

3.部署Provisioner

privisioner能够了解为底层存储的驱动，由privisioner治理底层存储。
privisioner以deploy形式部署了1个pod，pod内container指定了nfs的环境信息(包含name/ip/path等)，serviceAccountName=上一步创立的serviceAccount名称；

apiVersion: apps/v1kind: Deploymentmetadata:  labels:    app: nfs-provisioner  name: nfs-provisioner  namespace: storagespec:  replicas: 1  strategy:    type: Recreate  selector:    matchLabels:      app: nfs-provisioner  template:    metadata:      labels:        app: nfs-provisioner    spec:      serviceAccountName: nfs-client-provisioner      containers:        - name: nfs-client-provisioner          image: quay.io/external_storage/nfs-client-provisioner:latest          volumeMounts:            - name: nfs-volume              mountPath: /persistentvolumes          env:            - name: PROVISIONER_NAME              value: fuseim.pri/ifs            - name: NFS_SERVER              value: 178.104.163.63            - name: NFS_PATH              value: /var/nfs      volumes:        - name: nfs-volume          nfs:            server: 178.104.163.63            path: /var/nfs

查看部署的pod:

# kubectl get pod -n storageNAME                               READY   STATUS    RESTARTS   AGEnfs-provisioner-778c655cbd-4twcz   1/1     Running   0          19s

provisioner部署结束后，察看pod的log；若有报错，需及时排查：

# kubectl logs nfs-provisioner-778c655cbd-4twcz -n storageI0330 08:08:27.981652       1 leaderelection.go:185] attempting to acquire leader lease  storage/fuseim.pri-ifs...I0330 08:08:27.992969       1 leaderelection.go:194] successfully acquired lease storage/fuseim.pri-ifsI0330 08:08:27.993028       1 controller.go:631] Starting provisioner controller fuseim.pri/ifs_nfs-provisioner-778c655cbd-4twcz_1bbd49e5-912f-11eb-a36b-fef168fb5776!I0330 08:08:27.993504       1 event.go:221] Event(v1.ObjectReference{Kind:"Endpoints", Namespace:"storage", Name:"fuseim.pri-ifs", UID:"35414cb4-5e1c-45e4-9d0e-9b031e0c3df2", APIVersion:"v1", ResourceVersion:"2323405", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' nfs-provisioner-778c655cbd-4twcz_1bbd49e5-912f-11eb-a36b-fef168fb5776 became leaderI0330 08:08:28.093343       1 controller.go:680] Started provisioner controller fuseim.pri/ifs_nfs-provisioner-778c655cbd-4twcz_1bbd49e5-912f-11eb-a36b-fef168fb5776!

4.创立storageclass

apiVersion: storage.k8s.io/v1kind: StorageClassmetadata:  name: nfs-sc  namespace: storageprovisioner: fuseim.pri/ifs    # 这里的provisioner==下面env.PROVISIONER_NAME

查看storageclass:

# kubectl get sc -n storageNAME     PROVISIONER      RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGEnfs-sc   fuseim.pri/ifs   Delete          Immediate           false                  19s

5.storageclass创立pvc给pod应用

创立pvc：

apiVersion: v1kind: PersistentVolumeClaimmetadata:  name: myclaim  namespace: storagespec:  accessModes:    - ReadWriteMany  volumeMode: Filesystem  resources:    requests:      storage: 1Gi  storageClassName: nfs-sc    //应用storageClass

查看主动创立的pv:

# kubectl get sc,pv,pvc -n storageNAME                                 PROVISIONER      RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGEstorageclass.storage.k8s.io/nfs-sc   fuseim.pri/ifs   Delete          Immediate           false                  13mNAME                                                        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM             STORAGECLASS   REASON   AGEpersistentvolume/pvc-2fa56db1-fe02-4722-a9e9-d0dfad565934   1Gi        RWX            Delete           Bound    storage/myclaim   nfs-sc                  3m12sNAME                            STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGEpersistentvolumeclaim/myclaim   Bound    pvc-2fa56db1-fe02-4722-a9e9-d0dfad565934   1Gi        RWX            nfs-sc         3m12s

将pvc提供给pod应用：

kind: PodapiVersion: v1metadata:  name: test-pod  namespace: storagespec:  containers:  - name: test-pod    image: nginx:1.15.2    volumeMounts:      - name: nfs-pvc        mountPath: /mnt/nginx  volumes:    - name: nfs-pvc      persistentVolumeClaim:        claimName: myclaim        ## claimName==下面创立的pvc的名称

pod创立结束，到容器中查看挂载的目录：

# kubectl exec -it test-pod -n storage -- bash# df -hFilesystem                                                                        Size  Used Avail Use% Mounted onoverlay                                                                           100G  6.2G   94G   7% /tmpfs                                                                              64M     0   64M   0% /devtmpfs                                                                             3.0G     0  3.0G   0% /sys/fs/cgroup/dev/vda1                                                                         100G  6.2G   94G   7% /etc/hosts178.104.163.63:/var/nfs/storage-myclaim-pvc-2fa56db1-fe02-4722-a9e9-d0dfad565934  100G  1.4G   99G   2% /mnt/nginx

6.storageclass为statefulset创立pvc/pv

statefulset.spec指定storageClass，部署时会主动为其创立pvc、pv：

apiVersion: apps/v1kind: StatefulSetmetadata:  labels:    app: nginx  name: nginx  namespace: storagespec:  serviceName: "nginx"        # 等于headless service的名称  replicas: 2  selector:    matchLabels:      app: nginx  template:    metadata:      labels:        app: nginx    spec:      containers:      - image: nginx:1.15.2        imagePullPolicy: IfNotPresent        name: nginx        terminationMessagePath: /dev/termination-log        terminationMessagePolicy: File        volumeMounts:        - mountPath: /mnt/nginx          name: nfs-sc-volume      restartPolicy: Always      schedulerName: default-scheduler      terminationGracePeriodSeconds: 30  volumeClaimTemplates:                ## 指定连贯到sc去申请pvc  - metadata:      name: nfs-sc-volume    spec:      accessModes:      - ReadWriteOnce      resources:        limits:          storage: 2Gi        requests:          storage: 1Gi      storageClassName: nfs-sc      volumeMode: Filesystem

这里的statefulset创立了2个pod，为每个pod都绑定了一个pvc，2个pod应用独立的存储卷:

# kubectl get sc,pvc,pv -n storageNAME                                 PROVISIONER      RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGEstorageclass.storage.k8s.io/nfs-sc   fuseim.pri/ifs   Delete          Immediate           false                  4m58sNAME                                          STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGEpersistentvolumeclaim/nfs-sc-volume-nginx-0   Bound    pvc-63389a42-a00d-4b34-bd51-4542cebb42aa   1Gi        RWO            nfs-sc         75spersistentvolumeclaim/nfs-sc-volume-nginx-1   Bound    pvc-e01d4f5c-004b-4618-94d6-9556612cd198   1Gi        RWO            nfs-sc         70sNAME                                                        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                           STORAGECLASS   REASON   AGEpersistentvolume/pvc-63389a42-a00d-4b34-bd51-4542cebb42aa   1Gi        RWO            Delete           Bound    storage/nfs-sc-volume-nginx-0   nfs-sc                  75spersistentvolume/pvc-e01d4f5c-004b-4618-94d6-9556612cd198   1Gi        RWO            Delete           Bound    storage/nfs-sc-volume-nginx-1   nfs-sc                  70s