关于云计算:用-KubeKey-快速离线部署-K8s-与-KubeSphere

作者：尹珉，KubeSphere Ambassador，KubeSphere 社区用户委员会杭州站站长

一、KubeKey 介绍

KubeKey（以下简称 KK) 是一个用于部署 Kubernetes 集群的开源轻量级工具。它提供了一种灵便、疾速、便捷的形式来仅装置 Kubernetes/K3s，或同时装置 Kubernetes/K3s 和 KubeSphere，以及其余云原生插件。除此之外，它也是扩大和降级集群的无效工具。

KubeKey v2.0.0 版本新增了清单（manifest）和制品（artifact）的概念，为用户离线部署 Kubernetes 集群提供了一种解决方案。在过来，用户须要筹备部署工具，镜像 tar 包和其余相干的二进制文件，每位用户须要部署的 Kubernetes 版本和须要部署的镜像都是不同的。当初应用 kk，用户只需应用清单 manifest 文件来定义将要离线部署的集群环境须要的内容，再通过该 manifest 来导出制品 artifact 文件即可实现筹备工作。离线部署时只须要 kk 和 artifact 就可疾速、简略的在环境中部署镜像仓库和 Kubernetes 集群。

二、部署筹备

1. 资源清单

2. 源集群中下载解压 KK2.0.0-rc-3

阐明：因为 KK 版本不断更新请依照 github 上最新 Releases 版本为准

$ wget https://github.com/kubesphere/kubekey/releases/download/v2.0.0-rc.3/kubekey-v2.0.0-rc.3-linux-amd64.tar.gz

$ tar -zxvf kubekey-v2.0.0-rc.3-linux-amd64.tar.gz 

3. 源集群中应用 KK 创立 manifest

阐明：manifest 就是一个形容以后 Kubernetes 集群信息和定义 artifact 制品中须要蕴含哪些内容的文本文件。目前有两种形式来生成该文件：

依据模版手动创立并编写该文件。
应用 kk 命令依据已存在的集群生成该文件。

$ ./kk create manifest

4. 源集群中批改 manifest 配置

阐明：

1.reppostiory 局部须要指定服务器零碎的依赖 iso 包，能够间接在 url 中填入对应下载地址或者提前下载 iso 包到本地在 localPath 里填写本地寄存门路并删除 url 配置项即可

2. 开启 harbor、docker-compose 配置项，为前面通过 KK 自建 harbor 仓库推送镜像应用
3. 默认创立的 manifest 外面的镜像列表从 docker.io 获取，倡议批改以下示例中的青云仓库中获取镜像
4. 可依据理论状况批改 manifest-sample.yaml 文件的内容，用以之后导出冀望的 artifact 文件

$ vim manifest.yaml

---
apiVersion: kubekey.kubesphere.io/v1alpha2
kind: Manifest
metadata:
  name: sample
spec:
  arches:
  - amd64
  operatingSystems:
  - arch: amd64
    type: linux
    id: centos
    version: "7"
    repository:
      iso:
        localPath: /mnt/sdb/kk2.0-rc/kubekey/centos-7-amd64-rpms.iso
        url: #这里填写下载地址也能够
  kubernetesDistributions:
  - type: kubernetes
    version: v1.21.5
  components:
    helm:
      version: v3.6.3
    cni:
      version: v0.9.1
    etcd:
      version: v3.4.13
    ## For now, if your cluster container runtime is containerd, KubeKey will add a docker 20.10.8 container runtime in the below list.
    ## The reason is KubeKey creates a cluster with containerd by installing a docker first and making kubelet connect the socket file of containerd which docker contained.
    containerRuntimes:
    - type: docker
      version: 20.10.8
    crictl:
      version: v1.22.0
    ##
    # docker-registry:
    #   version: "2"
    harbor:
      version: v2.4.1
    docker-compose:
      version: v2.2.2
  images:
  - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-apiserver:v1.22.1
  - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-controller-manager:v1.22.1
  - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-proxy:v1.22.1
  - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-scheduler:v1.22.1
  - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-apiserver:v1.21.5
  - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-controller-manager:v1.21.5
  - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-proxy:v1.21.5
  - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-scheduler:v1.21.5
  - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-apiserver:v1.20.10
  - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-controller-manager:v1.20.10
  - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-proxy:v1.20.10
  - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-scheduler:v1.20.10
  - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-apiserver:v1.19.9
  - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-controller-manager:v1.19.9
  - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-proxy:v1.19.9
  - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-scheduler:v1.19.9
  - registry.cn-beijing.aliyuncs.com/kubesphereio/pause:3.5
  - registry.cn-beijing.aliyuncs.com/kubesphereio/pause:3.4.1
  - registry.cn-beijing.aliyuncs.com/kubesphereio/coredns:1.8.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/cni:v3.20.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-controllers:v3.20.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/node:v3.20.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/pod2daemon-flexvol:v3.20.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/typha:v3.20.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/flannel:v0.12.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/provisioner-localpv:2.10.1
  - registry.cn-beijing.aliyuncs.com/kubesphereio/linux-utils:2.10.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/haproxy:2.3
  - registry.cn-beijing.aliyuncs.com/kubesphereio/nfs-subdir-external-provisioner:v4.0.2
  - registry.cn-beijing.aliyuncs.com/kubesphereio/k8s-dns-node-cache:1.15.12
  - registry.cn-beijing.aliyuncs.com/kubesphereio/ks-installer:v3.2.1
  - registry.cn-beijing.aliyuncs.com/kubesphereio/ks-apiserver:v3.2.1
  - registry.cn-beijing.aliyuncs.com/kubesphereio/ks-console:v3.2.1
  - registry.cn-beijing.aliyuncs.com/kubesphereio/ks-controller-manager:v3.2.1
  - registry.cn-beijing.aliyuncs.com/kubesphereio/kubectl:v1.21.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/kubectl:v1.20.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/kubefed:v0.8.1
  - registry.cn-beijing.aliyuncs.com/kubesphereio/tower:v0.2.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/minio:RELEASE.2019-08-07T01-59-21Z
  - registry.cn-beijing.aliyuncs.com/kubesphereio/mc:RELEASE.2019-08-07T23-14-43Z
  - registry.cn-beijing.aliyuncs.com/kubesphereio/snapshot-controller:v4.0.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/nginx-ingress-controller:v0.48.1
  - registry.cn-beijing.aliyuncs.com/kubesphereio/defaultbackend-amd64:1.4
  - registry.cn-beijing.aliyuncs.com/kubesphereio/metrics-server:v0.4.2
  - registry.cn-beijing.aliyuncs.com/kubesphereio/redis:5.0.14-alpine
  - registry.cn-beijing.aliyuncs.com/kubesphereio/haproxy:2.0.25-alpine
  - registry.cn-beijing.aliyuncs.com/kubesphereio/alpine:3.14
  - registry.cn-beijing.aliyuncs.com/kubesphereio/openldap:1.3.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/netshoot:v1.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/cloudcore:v1.7.2
  - registry.cn-beijing.aliyuncs.com/kubesphereio/edge-watcher:v0.1.1
  - registry.cn-beijing.aliyuncs.com/kubesphereio/edge-watcher-agent:v0.1.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/gatekeeper:v3.5.2
  - registry.cn-beijing.aliyuncs.com/kubesphereio/openpitrix-jobs:v3.2.1
  - registry.cn-beijing.aliyuncs.com/kubesphereio/devops-apiserver:v3.2.1
  - registry.cn-beijing.aliyuncs.com/kubesphereio/devops-controller:v3.2.1
  - registry.cn-beijing.aliyuncs.com/kubesphereio/devops-tools:v3.2.1
  - registry.cn-beijing.aliyuncs.com/kubesphereio/ks-jenkins:v3.2.0-2.249.1
  - registry.cn-beijing.aliyuncs.com/kubesphereio/jnlp-slave:3.27-1
  - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-base:v3.2.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-nodejs:v3.2.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-maven:v3.2.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-python:v3.2.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-go:v3.2.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-go:v3.2.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-base:v3.2.0-podman
  - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-nodejs:v3.2.0-podman
  - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-maven:v3.2.0-podman
  - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-python:v3.2.0-podman
  - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-go:v3.2.0-podman
  - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-go:v3.2.0-podman
  - registry.cn-beijing.aliyuncs.com/kubesphereio/s2ioperator:v3.2.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/s2irun:v3.2.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/s2i-binary:v3.2.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/tomcat85-java11-centos7:v3.2.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/tomcat85-java11-runtime:v3.2.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/tomcat85-java8-centos7:v3.2.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/tomcat85-java8-runtime:v3.2.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/java-11-centos7:v3.2.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/java-8-centos7:v3.2.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/java-8-runtime:v3.2.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/java-11-runtime:v3.2.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/nodejs-8-centos7:v3.2.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/nodejs-6-centos7:v3.2.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/nodejs-4-centos7:v3.2.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/python-36-centos7:v3.2.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/python-35-centos7:v3.2.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/python-34-centos7:v3.2.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/python-27-centos7:v3.2.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/configmap-reload:v0.3.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/prometheus:v2.26.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/prometheus-config-reloader:v0.43.2
  - registry.cn-beijing.aliyuncs.com/kubesphereio/prometheus-operator:v0.43.2
  - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-rbac-proxy:v0.8.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-state-metrics:v1.9.7
  - registry.cn-beijing.aliyuncs.com/kubesphereio/node-exporter:v0.18.1
  - registry.cn-beijing.aliyuncs.com/kubesphereio/k8s-prometheus-adapter-amd64:v0.6.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/alertmanager:v0.21.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/thanos:v0.18.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/grafana:7.4.3
  - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-rbac-proxy:v0.8.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/notification-manager-operator:v1.4.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/notification-manager:v1.4.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/notification-tenant-sidecar:v3.2.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/elasticsearch-curator:v5.7.6
  - registry.cn-beijing.aliyuncs.com/kubesphereio/elasticsearch-oss:6.7.0-1
  - registry.cn-beijing.aliyuncs.com/kubesphereio/fluentbit-operator:v0.11.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/docker:19.03
  - registry.cn-beijing.aliyuncs.com/kubesphereio/fluent-bit:v1.8.3
  - registry.cn-beijing.aliyuncs.com/kubesphereio/log-sidecar-injector:1.1
  - registry.cn-beijing.aliyuncs.com/kubesphereio/filebeat:6.7.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-events-operator:v0.3.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-events-exporter:v0.3.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-events-ruler:v0.3.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-auditing-operator:v0.2.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-auditing-webhook:v0.2.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/pilot:1.11.1
  - registry.cn-beijing.aliyuncs.com/kubesphereio/proxyv2:1.11.1
  - registry.cn-beijing.aliyuncs.com/kubesphereio/jaeger-operator:1.27
  - registry.cn-beijing.aliyuncs.com/kubesphereio/jaeger-agent:1.27
  - registry.cn-beijing.aliyuncs.com/kubesphereio/jaeger-collector:1.27
  - registry.cn-beijing.aliyuncs.com/kubesphereio/jaeger-query:1.27
  - registry.cn-beijing.aliyuncs.com/kubesphereio/jaeger-es-index-cleaner:1.27
  - registry.cn-beijing.aliyuncs.com/kubesphereio/kiali-operator:v1.38.1
  - registry.cn-beijing.aliyuncs.com/kubesphereio/kiali:v1.38
  - registry.cn-beijing.aliyuncs.com/kubesphereio/busybox:1.31.1
  - registry.cn-beijing.aliyuncs.com/kubesphereio/nginx:1.14-alpine
  - registry.cn-beijing.aliyuncs.com/kubesphereio/wget:1.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/hello:plain-text
  - registry.cn-beijing.aliyuncs.com/kubesphereio/wordpress:4.8-apache
  - registry.cn-beijing.aliyuncs.com/kubesphereio/hpa-example:latest
  - registry.cn-beijing.aliyuncs.com/kubesphereio/java:openjdk-8-jre-alpine
  - registry.cn-beijing.aliyuncs.com/kubesphereio/fluentd:v1.4.2-2.0
  - registry.cn-beijing.aliyuncs.com/kubesphereio/perl:latest
  - registry.cn-beijing.aliyuncs.com/kubesphereio/examples-bookinfo-productpage-v1:1.16.2
  - registry.cn-beijing.aliyuncs.com/kubesphereio/examples-bookinfo-reviews-v1:1.16.2
  - registry.cn-beijing.aliyuncs.com/kubesphereio/examples-bookinfo-reviews-v2:1.16.2
  - registry.cn-beijing.aliyuncs.com/kubesphereio/examples-bookinfo-details-v1:1.16.2
  - registry.cn-beijing.aliyuncs.com/kubesphereio/examples-bookinfo-ratings-v1:1.16.3
  registry:
    auths: {}

5. 源集群中导出制品 artifact

阐明：

制品就是一个依据指定的 manifest 文件内容导出的蕴含镜像 tar 包和相干二进制文件的 tgz 包。在 kk 初始化镜像仓库、创立集群、增加节点和降级集群的命令中均可指定一个 artifact，kk 将主动解包该 artifact 并将在执行命令时间接应用解包进去的文件。

留神：

1. 导出命令会从互联网中下载相应的二进制文件，请确保网络连接失常。
2. 导出命令会依据 manifest 文件中的镜像列表一一拉取镜像，请确保 kk 的工作节点已装置 containerd 或最低版本为 18.09 的 docker。

3.kk 会解析镜像列表中的镜像名，若镜像名中的镜像仓库须要鉴权信息，可在 manifest 文件中的 .registry.auths 字段中进行配置。

4. 若须要导出的 artifact 文件中蕴含操作系统依赖文件（如：conntarck、chrony 等），可在 operationSystem 元素中的 .repostiory.iso.url 中配置相应的 ISO 依赖文件下载地址。

$ export KKZONE=cn
$ ./kk artifact export -m manifest-sample.yaml -o kubesphere.tar.gz
#默认 tar 包的名字是 kubekey-artifact.tar.gz，可通过 - o 参数自定义包名 

三、离线环境装置集群

1. 离线环境下载 KK

$ wget https://github.com/kubesphere/kubekey/releases/download/v2.0.0-rc.3/kubekey-v2.0.0-rc.3-linux-amd64.tar.gz

2. 创立离线集群配置文件

$./kk create config --with-kubesphere v3.2.1 --with-kubernetes v1.21.5 -f config-sample.yaml

3. 批改配置文件

$ vim config-sample.yaml

阐明：

1. 依照理论离线环境配置批改节点信息
2. 必须指定 registry 仓库部署节点（因为 KK 部署自建 harbor 仓库须要应用）
   3.registry 里必须指定 type 类型为 harbor，不配 harbor 的话默认是装的 docker registry

apiVersion: kubekey.kubesphere.io/v1alpha2
kind: Cluster
metadata:
  name: sample
spec:
  hosts:
  - {name: master, address: 192.168.149.133, internalAddress: 192.168.149.133, user: root, password: "Supaur@2022"}
  - {name: node1, address: 192.168.149.134, internalAddress: 192.168.149.134, user: root, password: "Supaur@2022"}

  roleGroups:
    etcd:
    - master
    control-plane:
    - master
    worker:
    - node1
    # 如需应用 kk 主动部署镜像仓库，请设置该主机组（倡议仓库与集群拆散部署，缩小相互影响）registry:
    - node1
  controlPlaneEndpoint:
    ## Internal loadbalancer for apiservers 
    # internalLoadbalancer: haproxy

    domain: lb.kubesphere.local
    address: ""
    port: 6443
  kubernetes:
    version: v1.21.5
    clusterName: cluster.local
  network:
    plugin: calico
    kubePodsCIDR: 10.233.64.0/18
    kubeServiceCIDR: 10.233.0.0/18
    ## multus support. https://github.com/k8snetworkplumbingwg/multus-cni
    multusCNI:
      enabled: false
  registry:
    # 如需应用 kk 部署 harbor, 可将该参数设置为 harbor，不设置该参数且需应用 kk 创立容器镜像仓库，将默认应用 docker registry。type: harbor 
    # 如应用 kk 部署的 harbor 或其余须要登录的仓库，可设置对应仓库的 auths，如应用 kk 创立的 docker registry 仓库，则无需配置该参数。# 留神：如应用 kk 部署 harbor，该参数请于 harbor 启动后设置。#auths:
    #  "dockerhub.kubekey.local":
    #    username: admin
    #    password: Harbor12345
    plainHTTP: false
    # 设置集群部署时应用的公有仓库
    privateRegistry: "dockerhub.kubekey.local"
    namespaceOverride: ""
    registryMirrors: []
    insecureRegistries: []
  addons: []

4. 形式一：执行脚本创立 harbor 我的项目

4.1 下载指定脚本初始化 harbor 仓库

$ curl https://github.com/kubesphere/ks-installer/blob/master/scripts/create_project_harbor.sh

4.2 批改脚本配置文件

阐明：

1. 批改 url 的值为 https://dockerhub.kubekey.local
2. 须要指定仓库项目名称和镜像列表的项目名称保持一致

3. 脚本开端 curl 命令开端加上 -k

   $ vim create_project_harbor.sh

   #!/usr/bin/env bash
   
   # Copyright 2018 The KubeSphere Authors.
   #
   # Licensed under the Apache License, Version 2.0 (the "License");
   # you may not use this file except in compliance with the License.
   # You may obtain a copy of the License at
   #
   #     http://www.apache.org/licenses/LICENSE-2.0
   #
   # Unless required by applicable law or agreed to in writing, software
   # distributed under the License is distributed on an "AS IS" BASIS,
   # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   # See the License for the specific language governing permissions and
   # limitations under the License.
   
   url="https://dockerhub.kubekey.local"  #批改 url 的值为 https://dockerhub.kubekey.local
   user="admin"
   passwd="Harbor12345"
   
   harbor_projects=(library
    kubesphereio  #须要指定仓库项目名称和镜像列表的项目名称保持一致
   )
   
   for project in "${harbor_projects[@]}"; do
    echo "creating $project"
    curl -u "${user}:${passwd}" -X POST -H "Content-Type: application/json" "${url}/api/v2.0/projects" -d "{\"project_name\": \"${project}\", \"public\": true}" -k #curl 命令开端加上 -k
   done

   $ chmod +x create_project_harbor.sh

   $ ./create_project_harbor.sh

4.3 形式二：登录 harbor 仓库创立我的项目

5. 应用 KK 装置镜像仓库

阐明：
1.config-sample.yaml(离线环境集群的配置文件)
2.kubesphere.tar.gz（源集群打包进去的 tar 包镜像）
3.harbor 安装文件在 /opt/harbor , 如需运维 harbor，可至该目 录下。

$ ./kk init registry -f config-sample.yaml -a kubesphere.tar.gz

6. 再次批改集群配置文件

阐明：

1. 新增 auths 配置减少 dockerhub.kubekey.local、账号密码

2.privateRegistry 减少 dockerhub.kubekey.local

3.namespaceOverride 减少 kubesphereio（对应仓库里新建的我的项目）

$ vim config-sample.yaml

  ...
  registry:
    type: harbor  
    auths: 
      "dockerhub.kubekey.local":
        username: admin
        password: Harbor12345
    plainHTTP: false
    privateRegistry: "dockerhub.kubekey.local"
    namespaceOverride: "kubesphereio"
    registryMirrors: []
    insecureRegistries: []
  addons: []

7. 装置 kubesphere 集群

阐明 :
1.config-sample.yaml(离线环境集群的配置文件)
2.kubesphere.tar.gz（源集群打包进去的 tar 包镜像）

3. 指定 k8s 版本、kubepshere 版本
   4.–with-packages（必须增加否则 ISO 依赖装置失败）

$ ./kk create cluster -f config-sample1.yaml -a kubesphere.tar.gz --with-kubernetes v1.21.5 --with-kubesphere v3.2.1 --with-packages

8. 查看集群集群状态

$ kubectl logs -n kubesphere-system $(kubectl get pod -n kubesphere-system -l app=ks-install -o jsonpath='{.items[0].metadata.name}') -f

**************************************************
#####################################################
###              Welcome to KubeSphere!           ###
#####################################################

Console: http://192.168.149.133:30880
Account: admin
Password: P@88w0rd

NOTES：1. After you log into the console, please check the
monitoring status of service components in
the "Cluster Management". If any service is not
ready, please wait patiently until all components
are up and running.
2. Please change the default password after login.

#####################################################
https://kubesphere.io             2022-02-28 23:30:06
#####################################################

9. 登录 kubesphere 控制台

四、结尾

本教程应用 KK 2.0.0 作为部署工具来实现 kubesphere 集群在离线环境中的部署，当然 KK 也反对 kubernetes 的部署。心愿 KK 能帮忙大家实现离线闪电交付的目标。如果大家有好的想法和倡议能够到 Kubekey 仓库中提交 issue 帮忙解决。

本文由博客一文多发平台 OpenWrite 公布！