背景
应用sealos+NFS部署一个带有长久化存储的k8s1.18.17，并在其之上部署kubesphere。
sealos 简介
https://github.com/fanux/sealos
一条命令离线装置kubernetes，超全版本，反对国产化，生产环境中稳如老狗，99年证书，0依赖，去haproxy keepalived，v1.20反对containerd！
NFS 简介
没啥好说的
kubesphere 简介
https://kubesphere.com.cn/
KubeSphere 是在 Kubernetes 之上构建的面向云原生利用的分布式操作系统，齐全开源，反对多云与多集群治理，提供全栈的 IT 自动化运维能力，简化企业的 DevOps 工作流。它的架构能够十分不便地使第三方利用与云原生生态组件进行即插即用 (plug-and-play) 的集成。
作为全栈的多租户容器平台，KubeSphere 提供了运维敌对的向导式操作界面，帮忙企业疾速构建一个弱小和功能丰富的容器云平台。KubeSphere 为用户提供构建企业级 Kubernetes 环境所需的多项性能，例如多云与多集群治理、Kubernetes 资源管理、DevOps、利用生命周期治理、微服务治理（服务网格）、日志查问与收集、服务与网络、多租户治理、监控告警、事件与审计查问、存储管理、拜访权限管制、GPU 反对、网络策略、镜像仓库治理以及平安治理等。
总之是kubernetes图形界面就是了。
应用sealos部署k8s1.18.17
筹备3个master节点，N个node节点。零碎初始化不讲了，工夫要同步，/var/lib/docker倡议独自挂盘，主机名记得批改，上面简略介绍下怎么批量批改主机名。
Ansible批改主机名
首先编辑 hosts，将你想要设置的主机名写在ip前面
[m]172.26.140.151 hostname=k8sm1172.26.140.145 hostname=k8sm2172.26.140.216 hostname=k8sm3[n]172.26.140.202 hostname=k8sn1172.26.140.185 hostname=k8sn2172.26.140.156 hostname=k8sn3剧本如下
---- hosts: n gather_facts: no tasks: - name: change name raw: "echo {{hostname|quote}} > /etc/hostname" - name: shell: hostname {{hostname|quote}}运行
ansible-playbook main.yaml装置k8s
sealos init --passwd '123456' --master 172.26.140.151 --master 172.26.140.145 --master 172.26.140.216 --node 172.26.140.202 --node 172.26.140.185 --user root --pkg-url /root/kube1.18.17.tar.gz --version v1.18.17过个几分钟就会输入：
[root@k8sm1 ks]# kubectl get nodesNAME STATUS ROLES AGE VERSIONk8sm1 Ready master 22m v1.18.17k8sm2 Ready master 22m v1.18.17k8sm3 Ready master 22m v1.18.17k8sn1 Ready <none> 21m v1.18.17k8sn2 Ready <none> 21m v1.18.17要减少node节点
sealos join --node 172.26.140.156 --node 172.26.140.139helm & NFS
NFS服务器怎么搭建不讲了。上面将如何将NFS设置为k8s的默认存储，须要用到Helm。
Helm
到这下载二进制文件，https://github.com/helm/helm/...
wget https://get.helm.sh/helm-v3.4.2-linux-amd64.tar.gztar zxvf helm-v3.4.2-linux-amd64.tar.gzmv linux-amd64/helm /usr/bin/helmNFS
docker pull heegor/nfs-subdir-external-provisioner:v4.0.0docker tag heegor/nfs-subdir-external-provisioner:v4.0.0 gcr.io/k8s-staging-sig-storage/nfs-subdir-external-provisioner:v4.0.0helm repo add nfs-subdir-external-provisioner https://kubernetes-sigs.github.io/nfs-subdir-external-provisioner/helm install nfs-subdir-external-provisioner nfs-subdir-external-provisioner/nfs-subdir-external-provisioner --set nfs.server=1.2.3.4 --set nfs.path=/xx/k8s设置默认StorageClass
[root@k8sm1 ~]# kubectl get storageclassNAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGEnfs-client cluster.local/nfs-subdir-external-provisioner Delete Immediate true 114s[root@k8sm1 ~]# kubectl patch storageclass nfs-client -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'storageclass.storage.k8s.io/nfs-client patched[root@k8sm1 ~]# kubectl get storageclassNAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGEnfs-client (default) cluster.local/nfs-subdir-external-provisioner Delete Immediate true 2m5skubesphere
批改配置文件
kubesphere-installer.yaml不必改变
cluster-configuration.yaml 次要批改两项
应用外置的elasticsearch，须要新增
externalElasticsearchUrl: 192.168.1.1 externalElasticsearchPort: 9200还须要设置etcd集群的地址
endpointIps: 172.26.140.151,172.26.140.145,172.26.140.216 # etcd cluster EndpointIps, it can be a bunch of IPs here.残缺配置如下：
---apiVersion: installer.kubesphere.io/v1alpha1kind: ClusterConfigurationmetadata: name: ks-installer namespace: kubesphere-system labels: version: v3.0.0spec: persistence: storageClass: "" # If there is not a default StorageClass in your cluster, you need to specify an existing StorageClass here. authentication: jwtSecret: "" # Keep the jwtSecret consistent with the host cluster. Retrive the jwtSecret by executing "kubectl -n kubesphere-system get cm kubesphere-config -o yaml | grep -v "apiVersion" | grep jwtSecret" on the host cluster. etcd: monitoring: true # Whether to enable etcd monitoring dashboard installation. You have to create a secret for etcd before you enable it. endpointIps: 172.26.140.151,172.26.140.145,172.26.140.216 # etcd cluster EndpointIps, it can be a bunch of IPs here. port: 2379 # etcd port tlsEnable: true common: mysqlVolumeSize: 20Gi # MySQL PVC size. minioVolumeSize: 20Gi # Minio PVC size. etcdVolumeSize: 20Gi # etcd PVC size. openldapVolumeSize: 2Gi # openldap PVC size. redisVolumSize: 2Gi # Redis PVC size. es: # Storage backend for logging, events and auditing. elasticsearchMasterReplicas: 1 # total number of master nodes, it's not allowed to use even number elasticsearchDataReplicas: 1 # total number of data nodes. elasticsearchMasterVolumeSize: 4Gi # Volume size of Elasticsearch master nodes. elasticsearchDataVolumeSize: 20Gi # Volume size of Elasticsearch data nodes. logMaxAge: 7 # Log retention time in built-in Elasticsearch, it is 7 days by default. elkPrefix: logstash # The string making up index names. The index name will be formatted as ks-<elk_prefix>-log. externalElasticsearchUrl: 192.168.1.1 externalElasticsearchPort: 9200 console: enableMultiLogin: true # enable/disable multiple sing on, it allows an account can be used by different users at the same time. port: 30880 alerting: # (CPU: 0.3 Core, Memory: 300 MiB) Whether to install KubeSphere alerting system. It enables Users to customize alerting policies to send messages to receivers in time with different time intervals and alerting levels to choose from. enabled: false auditing: # Whether to install KubeSphere audit log system. It provides a security-relevant chronological set of records，recording the sequence of activities happened in platform, initiated by different tenants. enabled: false devops: # (CPU: 0.47 Core, Memory: 8.6 G) Whether to install KubeSphere DevOps System. It provides out-of-box CI/CD system based on Jenkins, and automated workflow tools including Source-to-Image & Binary-to-Image. enabled: false jenkinsMemoryLim: 2Gi # Jenkins memory limit. jenkinsMemoryReq: 1500Mi # Jenkins memory request. jenkinsVolumeSize: 8Gi # Jenkins volume size. jenkinsJavaOpts_Xms: 512m # The following three fields are JVM parameters. jenkinsJavaOpts_Xmx: 512m jenkinsJavaOpts_MaxRAM: 2g events: # Whether to install KubeSphere events system. It provides a graphical web console for Kubernetes Events exporting, filtering and alerting in multi-tenant Kubernetes clusters. enabled: false ruler: enabled: true replicas: 2 logging: # (CPU: 57 m, Memory: 2.76 G) Whether to install KubeSphere logging system. Flexible logging functions are provided for log query, collection and management in a unified console. Additional log collectors can be added, such as Elasticsearch, Kafka and Fluentd. enabled: true logsidecarReplicas: 2 metrics_server: # (CPU: 56 m, Memory: 44.35 MiB) Whether to install metrics-server. IT enables HPA (Horizontal Pod Autoscaler). enabled: false monitoring: # prometheusReplicas: 1 # Prometheus replicas are responsible for monitoring different segments of data source and provide high availability as well. prometheusMemoryRequest: 400Mi # Prometheus request memory. prometheusVolumeSize: 20Gi # Prometheus PVC size. # alertmanagerReplicas: 1 # AlertManager Replicas. multicluster: clusterRole: none # host | member | none # You can install a solo cluster, or specify it as the role of host or member cluster. networkpolicy: # Network policies allow network isolation within the same cluster, which means firewalls can be set up between certain instances (Pods). # Make sure that the CNI network plugin used by the cluster supports NetworkPolicy. There are a number of CNI network plugins that support NetworkPolicy, including Calico, Cilium, Kube-router, Romana and Weave Net. enabled: false notification: # Email Notification support for the legacy alerting system, should be enabled/disabled together with the above alerting option. enabled: false openpitrix: # (2 Core, 3.6 G) Whether to install KubeSphere Application Store. It provides an application store for Helm-based applications, and offer application lifecycle management. enabled: false servicemesh: # (0.3 Core, 300 MiB) Whether to install KubeSphere Service Mesh (Istio-based). It provides fine-grained traffic management, observability and tracing, and offer visualization for traffic topology. enabled: false装置
kubectl apply -f kubesphere-installer.yaml kubectl apply -f cluster-configuration.yaml 而后过几分钟就装置etcd证书，不然Prometheus-0、Prometheus-1会装置不上
etcd证书
kubectl -n kubesphere-monitoring-system create secret generic kube-etcd-client-certs \--from-file=etcd-client-ca.crt=/etc/kubernetes/pki/etcd/ca.crt \--from-file=etcd-client.crt=/etc/kubernetes/pki/etcd/healthcheck-client.crt \--from-file=etcd-client.key=/etc/kubernetes/pki/etcd/healthcheck-client.key验证
kubectl logs -n kubesphere-system $(kubectl get pod -n kubesphere-system -l app=ks-install -o jsonpath='{.items[0].metadata.name}') -f如果日志卡住了，就去看pod有啥问题
kubectl get pod -A看到pod不是running状态的，比方crashloopbackoff、init 0/1之类的就desc或看日志
在依据状况去解决
最初会输入：
**************************************************######################################################## Welcome to KubeSphere! ########################################################Console: http://172.26.140.151:30880Account: adminPassword: P@88w0rdNOTES： 1. After logging into the console, please check the monitoring status of service components in the "Cluster Management". If any service is not ready, please wait patiently until all components are ready. 2. Please modify the default password after login.#####################################################https://kubesphere.io 2021-03-27 09:35:17#####################################################