关于kubernetes:Kubelet运行在容器下使用LocalPV磁盘地址不存在问题

2次阅读

共计 2086 个字符，预计需要花费 6 分钟才能阅读完成。

明天在通过 kubernetes 部署 elasticsearch 的过程中呈现一个问题，特此记录一下。因为 elasticsearch 是有状态利用，须要应用长久化存储，手头既没有云存储，也没有 nfs，cephfs 之类的存储。kubernetes 给的 elasticsearch 的 yaml 资源文件默认的是 emptydir 形式，并且文档强调：

Storage

The Elasticsearch StatefulSet will use the EmptyDir volume to store data. EmptyDir is erased when the pod terminates, here it is used only for testing purposes. Important: please change the storage to persistent volume claim before actually using this StatefulSet in your setup!

说白了，这块须要自由发挥，抉择适宜本人的长久化存储。所以理所应当应用本地磁盘，Local PV 的形式，PV 的资源文件如下：

apiVersion: v1
kind: PersistentVolume
metadata:
  name: es-data-holder-01
spec:
  capacity:
    storage: 100Gi
  volumeMode: Filesystem
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Delete
  storageClassName: local-storage
  local:
    path: /home/k8s/localpv  # 节点上的目录
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - kubernetes-node03

其中指定了 /home/k8s/localpv 目录作为存储目录，这些都不是关键点，关键点在于，最初 elasticsearch 的 pod 启动时会报错：

Error: MountVolume.NewMounter initialization failed for volume“local-pv-xxxxx”: path“/home/k8s/localpv”does not exist

找不到 /home/k8s/localpv 门路。

因为我的 Kubernetes 集群是通过 Rancher 装置的 RKE 集群，并不是原生通过 Kubeadm 装置的，所以集群组件都运行在 Docker 容器内，所以 kubelet 运行在容器外部，无奈读到 Node 节点上的门路。

官网 FAQ 也阐明了这个问题

Volume does not exist with containerized kubelet

If your kubelet is running in a container, it may not be able to access the path on the host.

In order to allow the kubelet to access the path on the host, you must prefix hostDir with the prefix of the host filesystem in the kubelet container or mount the directory of the local volumes into the kubelet container at the same path.

For example, if the root filesystem of the host is mounted at /rootfs in the kubelet container, you need to prefix the hostDir with /rootfs. This requires recreating the local PV objects. You can delete them all and wait for them to be discovered again.

Another solution is to add a bind in the kubelet deployment configuration to mount the parent directory of local volumes into the kubelet container at the same path. This requires restarting the kubelet container.

For Rancher clusters, users must configure additional local volumes (or the parent directory) via Extra Binds.

解决办法如下：

services:
    kubelet:
      extra_binds:
        - "/host/dev:/dev"
        - "/usr/libexec/kubernetes/kubelet-plugins:/usr/libexec/kubernetes/kubelet-plugins:z"

正文完