共计 4819 个字符，预计需要花费 13 分钟才能阅读完成。
Containerd 作为 Runtime
所有节点装置 docker-ce-20.10：
yum install docker-ce-20.10.* docker-ce-cli-20.10.* -y配置 Containerd 所需的模块（所有节点）：
cat <<EOF | sudo tee /etc/modules-load.d/containerd.conf
overlay
br_netfilter
EOF所有节点加载模块：
modprobe -- overlay
modprobe -- br_netfilter所有节点，配置 Containerd 所需的内核：
cat <<EOF | sudo tee /etc/sysctl.d/99-kubernetes-cri.conf
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-ip6tables = 1
EOF所有节点加载内核：
sysctl --system所有节点配置 Containerd 的配置文件：
mkdir -p /etc/containerd
containerd config default | tee /etc/containerd/config.toml所有节点将 Containerd 的 Cgroup 改为 Systemd：
vim /etc/containerd/config.toml找到 containerd.runtimes.runc.options，增加 SystemdCgroup = true
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
SystemdCgroup = true所有节点将 sandbox_image 的 Pause 镜像改成合乎本人版本的地址
sandbox_image = "registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.6"所有节点启动 Containerd，并配置开机自启动：
systemctl daemon-reload
systemctl enable --now containerd所有节点配置 crictl 客户端连贯的运行时地位：
cat > /etc/crictl.yaml <<EOF
runtime-endpoint: unix:///run/containerd/containerd.sock
image-endpoint: unix:///run/containerd/containerd.sock
timeout: 10
debug: false
EOFDocker 作为 Runtime（版本小于 1.24）
所有节点装置 docker-ce 20.10：
yum install docker-ce-20.10.* docker-ce-cli-20.10.* -yCgroupDriver 改成 systemd:
mkdir /etc/docker
cat > /etc/docker/daemon.json <<EOF
{"exec-opts": ["native.cgroupdriver=systemd"]
}
EOF所有节点设置开机自启动 Docker：
systemctl daemon-reload && systemctl enable --now docker装置 Kubenetes 组件
所有节点装置 1.23 最新版本 kubeadm、kubelet 和 kubectl：
yum install kubeadm-1.23* kubelet-1.23* kubectl-1.23* -y更改 Kubelet 的配置应用 Containerd 作为 Runtime：
cat >/etc/sysconfig/kubelet<<EOF
KUBELET_KUBEADM_ARGS="--container-runtime=remote --runtime-request-timeout=15m --container-runtime-endpoint=unix:///run/containerd/containerd.sock"
EOF所有节点设置 Kubelet 开机自启动
systemctl daemon-reload
systemctl enable --now kubelet高可用组件
所有 Master 节点通过 yum 装置 HAProxy 和 KeepAlived：
yum install keepalived haproxy -y所有 Master 节点配置 HAProxy（所有 Master 节点的 HAProxy 配置雷同）：
mkdir /etc/haproxy
vim /etc/haproxy/haproxy.cfg global
maxconn 2000
ulimit-n 16384
log 127.0.0.1 local0 err
stats timeout 30s
defaults
log global
mode http
option httplog
timeout connect 5000
timeout client 50000
timeout server 50000
timeout http-request 15s
timeout http-keep-alive 15s
frontend monitor-in
bind *:33305
mode http
option httplog
monitor-uri /monitor
frontend k8s-master
bind 0.0.0.0:16443
bind 127.0.0.1:16443
mode tcp
option tcplog
tcp-request inspect-delay 5s
default_backend k8s-master
backend k8s-master
mode tcp
option tcplog
option tcp-check
balance roundrobin
default-server inter 10s downinter 5s rise 2 fall 2 slowstart 60s maxconn 250 maxqueue 256 weight 100
server k8s-master01 192.168.2.201:6443 check
server k8s-master02 192.168.2.202:6443 check
server k8s-master03 192.168.2.203:6443 check查看端口是否被监听：
netstat -lntp所有 Master 节点配置 KeepAlived（配置不一样，留神辨别）：
mkdir /etc/keepalived
vim /etc/keepalived/keepalived.conf查看网卡信息（替换 keepalived.conf 中的 interface）
ip aMaster01 节点的配置：
! Configuration File for keepalived
global_defs {
router_id LVS_DEVEL
script_user root
enable_script_security
}
vrrp_script chk_apiserver {
script "/etc/keepalived/check_apiserver.sh"
interval 5
weight -5
fall 2
rise 1
}
vrrp_instance VI_1 {
state MASTER
interface ens33
mcast_src_ip 192.168.2.201
virtual_router_id 51
priority 101
advert_int 2
authentication {
auth_type PASS
auth_pass K8SHA_KA_AUTH
}
virtual_ipaddress {192.168.2.236}
track_script {chk_apiserver}
}Master02 节点的配置：
! Configuration File for keepalived
global_defs {
router_id LVS_DEVEL
script_user root
enable_script_security
}
vrrp_script chk_apiserver {
script "/etc/keepalived/check_apiserver.sh"
interval 5
weight -5
fall 2
rise 1
}
vrrp_instance VI_1 {
state BACKUP
interface ens33
mcast_src_ip 192.168.2.202
virtual_router_id 51
priority 100
advert_int 2
authentication {
auth_type PASS
auth_pass K8SHA_KA_AUTH
}
virtual_ipaddress {192.168.2.236}
track_script {chk_apiserver}
}Master03 节点的配置：
! Configuration File for keepalived
global_defs {
router_id LVS_DEVEL
script_user root
enable_script_security
}
vrrp_script chk_apiserver {
script "/etc/keepalived/check_apiserver.sh"
interval 5
weight -5
fall 2
rise 1
}
vrrp_instance VI_1 {
state BACKUP
interface ens33
mcast_src_ip 192.168.2.203
virtual_router_id 51
priority 100
advert_int 2
authentication {
auth_type PASS
auth_pass K8SHA_KA_AUTH
}
virtual_ipaddress {192.168.2.236}
track_script {chk_apiserver}
}所有 master 节点配置 KeepAlived 健康检查文件：
vim /etc/keepalived/check_apiserver.sh#!/bin/bash
err=0
for k in $(seq 1 3)
do
check_code=$(pgrep haproxy)
if [[$check_code == ""]]; then
err=$(expr $err + 1)
sleep 1
continue
else
err=0
break
fi
done
if [[$err != "0"]]; then
echo "systemctl stop keepalived"
/usr/bin/systemctl stop keepalived
exit 1
else
exit 0
fichmod +x /etc/keepalived/check_apiserver.sh启动 haproxy 和 keepalived
systemctl daemon-reload
systemctl enable --now haproxy
systemctl enable --now keepalived测试 keepalived 是否是失常
ping 192.168.2.236 -c 4
telnet 192.168.2.236 16443如果 ping 不通且 telnet 没有呈现 ]，则认为 VIP 不能够，不可在持续往下执行，须要排查 keepalived 的问题，比方防火墙和 selinux，haproxy 和 keepalived 的状态，监听端口等:
- 所有节点查看防火墙状态必须为 disable 和 inactive：systemctl status firewalld
- 所有节点查看 selinux 状态，必须为 disable：getenforce
- master 节点查看 haproxy 和 keepalived 状态：systemctl status keepalived haproxy
- master 节点查看监听端口：netstat -lntp
正文完
