kubespray211安装kubernetes115

共计 11371 个字符，预计需要花费 29 分钟才能阅读完成。

Kubespray 是开源的 kubernetes 部署工具，整合了 ansible，可以方便的部署高可用集群环境，官网地址：https://github.com/kubernetes…，本文是用 kubespray2.11 版本部署 kubernetes1.15 版本的实战；

本次实战采用官方推荐的在线安装，因此会去谷歌镜像仓库下载镜像， 需要您的网络可以访问谷歌服务 ；

本次实战共计四台机器，它们的主机名、IP 地址和作用描述如下：

主机名	IP 地址	作用
ansible	192.168.133.134	ansible 主机
a001	192.168.133.139	k8s 集群的一号工作节点
a002	192.168.133.140	k8s 集群的二号工作节点
a003	192.168.133.141	k8s 集群的三号工作节点

本次实战的所有机器都要做以下设置：

操作系统：CentOS Linux release 7.7.1908
所以操作都是 root 账号执行的
关闭防火墙：

systemctl stop firewalld && systemctl disable firewalld

关闭 selinux：

setenforce 0
sed -i --follow-symlinks 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/sysconfig/selinux

ipv4 网络设置：

modprobe br_netfilter
echo '1' > /proc/sys/net/bridge/bridge-nf-call-iptables
sysctl -w net.ipv4.ip_forward=1

ssh 登录 ansible 主机；
生成 ssh 公私钥，输入命令 ssh-keygen，然后连续四次回车：

[root@ansible ~]# ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): 
Created directory '/root/.ssh'.
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:Empen3/RfLndRkS8mKfkq6a2IXtSdqwK7TqKNoHkNEU root@ansible
The key's randomart image is:
+---[RSA 2048]----+
|  .E           . |
|   .            o|
|  .   .       o..|
| +   . .     + o.|
|= . o o S . ooo..|
|.o o ..o + o.oo.o|
|  . .. o=.o  ..o+|
| o.  .o.o=.... .+|
|......o+=o=o.  . |
+----[SHA256]-----+

输入命令 ssh-copy-id root@192.168.133.139，将 ansible 的 ssh 分发给 a001 主机，会要求输入 yes 和 a001 主机的 root 账号的密码，完成输入后，以后 ansible 就可以免密码 ssh 登录 a001 主机了：

[root@ansible ~]# ssh-copy-id root@192.168.133.139
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub"
The authenticity of host '192.168.133.139 (192.168.133.139)' can't be established.
ECDSA key fingerprint is SHA256:DPE2nldWHiOhC4DB9doy7jPWNZVup6XFZ+sR2i1gqz8.
ECDSA key fingerprint is MD5:fc:21:f7:7f:e8:cd:1a:76:d7:fb:cc:d4:28:91:f3:5a.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@192.168.133.139's password: 

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh'root@192.168.133.139'"
and check to make sure that only the key(s) you wanted were added.

继续输入命令 ssh-copy-id root@192.168.133.140 和 ssh-copy-id root@192.168.133.141，使得 ansible 主机可以免密码登录 a002 和 a003；
至此，ansible 主机可以用命令 ssh root@192.168.133.139、ssh root@192.168.133.140、ssh root@192.168.133.141 免密码登录 a001、a002、a003 了；

ssh 登录 ansible 主机；
安装 ansible 应用：

yum install -y epel-release ansible

安装 pip：

easy_install pip

通过 pip 安装 jinja2：

pip2 install jinja2 --upgrade

安装 python36：

yum install python36 -y

mkdir /usr/local/kubespray && cd /usr/local/kubespray/

下载 kubespray，我这里下载的是 v2.11.0 版本：

wget https://github.com/kubernetes-sigs/kubespray/archive/v2.11.0.tar.gz

解压：

tar -zxvf v2.11.0.tar.gz

cd kubespray-2.11.0/

安装 kubespray 所需的应用 (注意是 pip3)：

pip3 install -r requirements.txt

复制一份 demo 配置信息到目录 inventory/mycluster：

cp -rfp inventory/sample inventory/mycluster

进去看一下，可见 mycluster 目录下复制了很多文件：

[root@ansible kubespray-2.11.0]# tree inventory/
inventory/
├── local
│   ├── group_vars -> ../sample/group_vars
│   └── hosts.ini
├── mycluster
│   ├── group_vars
│   │   ├── all
│   │   │   ├── all.yml
│   │   │   ├── azure.yml
│   │   │   ├── coreos.yml
│   │   │   ├── docker.yml
│   │   │   ├── oci.yml
│   │   │   └── openstack.yml
│   │   ├── etcd.yml
│   │   └── k8s-cluster
│   │       ├── addons.yml
│   │       ├── k8s-cluster.yml
│   │       ├── k8s-net-calico.yml
│   │       ├── k8s-net-canal.yml
│   │       ├── k8s-net-cilium.yml
│   │       ├── k8s-net-contiv.yml
│   │       ├── k8s-net-flannel.yml
│   │       ├── k8s-net-kube-router.yml
│   │       ├── k8s-net-macvlan.yml
│   │       └── k8s-net-weave.yml
│   └── inventory.ini

设置集群信息 (当前目录仍旧是 kubespray-2.11.0)：

declare -a IPS=(192.168.133.139 192.168.133.140 192.168.133.141)

配置 ansible：

CONFIG_FILE=inventory/mycluster/hosts.yml python3 contrib/inventory_builder/inventory.py ${IPS[@]}

此时 kubespray 的脚本根据输入的 IP 信息做好了集群规划，具体信息可见 inventory/mycluster/hosts.yml，如下所示，您也可以自行修改此文件：

all:
  hosts:
    node1:
      ansible_host: 192.168.133.139
      ip: 192.168.133.139
      access_ip: 192.168.133.139
    node2:
      ansible_host: 192.168.133.140
      ip: 192.168.133.140
      access_ip: 192.168.133.140
    node3:
      ansible_host: 192.168.133.141
      ip: 192.168.133.141
      access_ip: 192.168.133.141
  children:
    kube-master:
      hosts:
        node1:
        node2:
    kube-node:
      hosts:
        node1:
        node2:
        node3:
    etcd:
      hosts:
        node1:
        node2:
        node3:
    k8s-cluster:
      children:
        kube-master:
        kube-node:
    calico-rr:
      hosts: {}

执行以下命令即可开始安装，在线安装比较耗时请耐心等待：

ansible-playbook -i inventory/mycluster/hosts.yml --become --become-user=root cluster.yml

安装完成时控制台输出类似如下的信息：

PLAY RECAP ********************************************************************************************************************************************************************************
localhost                  : ok=1    changed=0    unreachable=0    failed=0   
node1                      : ok=658  changed=95   unreachable=0    failed=0   
node2                      : ok=566  changed=77   unreachable=0    failed=0   
node3                      : ok=475  changed=66   unreachable=0    failed=0   

Sunday 17 November 2019  17:31:19 +0800 (0:00:00.064)       0:09:56.193 ******* 
=============================================================================== 
kubernetes/master : kubeadm | Init other uninitialized masters -------------------------------------------------------------------------------------------------------------------- 94.91s
kubernetes/master : kubeadm | Initialize first master ----------------------------------------------------------------------------------------------------------------------------- 42.95s
etcd : Install | Copy etcdctl binary from docker container ------------------------------------------------------------------------------------------------------------------------ 14.26s
download : download_container | Download image if required ------------------------------------------------------------------------------------------------------------------------ 12.87s
download : download_container | Download image if required ------------------------------------------------------------------------------------------------------------------------ 12.28s
download : download_container | Download image if required ------------------------------------------------------------------------------------------------------------------------ 10.79s
etcd : reload etcd ---------------------------------------------------------------------------------------------------------------------------------------------------------------- 10.71s
download : download_container | Download image if required ------------------------------------------------------------------------------------------------------------------------- 9.71s
download : download_container | Download image if required ------------------------------------------------------------------------------------------------------------------------- 9.48s
download : download_container | Download image if required ------------------------------------------------------------------------------------------------------------------------- 8.02s
download : download_container | Download image if required ------------------------------------------------------------------------------------------------------------------------- 7.88s
etcd : wait for etcd up ------------------------------------------------------------------------------------------------------------------------------------------------------------ 7.16s
etcd : Gen_certs | Write etcd master certs ----------------------------------------------------------------------------------------------------------------------------------------- 6.39s
download : download_container | Download image if required ------------------------------------------------------------------------------------------------------------------------- 5.75s
download : download_container | Download image if required ------------------------------------------------------------------------------------------------------------------------- 5.53s
download : download_container | Download image if required ------------------------------------------------------------------------------------------------------------------------- 5.42s
download : download_container | Download image if required ------------------------------------------------------------------------------------------------------------------------- 5.41s
download : download_container | Download image if required ------------------------------------------------------------------------------------------------------------------------- 5.06s
download : download_container | Download image if required ------------------------------------------------------------------------------------------------------------------------- 4.87s
kubernetes-apps/ansible : Kubernetes Apps | Start Resources ------------------------------------------------------------------------------------------------------------------------ 4.78s

至此，kubernetes 集群环境部署完成，接下来简单验证一下环境是否可用；

ssh 登录 a001 机器；
查看节点、service、pod：

[root@node1 ~]# kubectl get nodes
NAME    STATUS   ROLES    AGE   VERSION
node1   Ready    master   25m   v1.15.3
node2   Ready    master   23m   v1.15.3
node3   Ready    <none>   23m   v1.15.3
[root@node1 ~]# kubectl get services --all-namespaces
NAMESPACE     NAME                   TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)                  AGE
default       kubernetes             ClusterIP   10.233.0.1    <none>        443/TCP                  25m
kube-system   coredns                ClusterIP   10.233.0.3    <none>        53/UDP,53/TCP,9153/TCP   22m
kube-system   kubernetes-dashboard   ClusterIP   10.233.35.1   <none>        443/TCP                  22m
[root@node1 ~]# kubectl get pods --all-namespaces
NAMESPACE     NAME                                      READY   STATUS    RESTARTS   AGE
kube-system   calico-kube-controllers-c6fb79b8b-v24nq   1/1     Running   0          22m
kube-system   calico-node-46s8t                         1/1     Running   0          23m
kube-system   calico-node-mcjfs                         1/1     Running   0          23m
kube-system   calico-node-q989m                         1/1     Running   1          23m
kube-system   coredns-74c9d4d795-4xz6s                  1/1     Running   0          22m
kube-system   coredns-74c9d4d795-kh6vl                  1/1     Running   0          22m
kube-system   dns-autoscaler-7d95989447-gmcrl           1/1     Running   0          22m
kube-system   kube-apiserver-node1                      1/1     Running   0          24m
kube-system   kube-apiserver-node2                      1/1     Running   0          23m
kube-system   kube-controller-manager-node1             1/1     Running   0          24m
kube-system   kube-controller-manager-node2             1/1     Running   0          23m
kube-system   kube-proxy-2zhwn                          1/1     Running   0          23m
kube-system   kube-proxy-59qx8                          1/1     Running   0          23m
kube-system   kube-proxy-fgpx6                          1/1     Running   0          23m
kube-system   kube-scheduler-node1                      1/1     Running   0          24m
kube-system   kube-scheduler-node2                      1/1     Running   0          23m
kube-system   kubernetes-dashboard-7c547b4c64-x7nfq     1/1     Running   0          22m
kube-system   nginx-proxy-node3                         1/1     Running   0          23m
kube-system   nodelocaldns-8khfq                        1/1     Running   0          22m
kube-system   nodelocaldns-pzx2p                        1/1     Running   0          22m
kube-system   nodelocaldns-s5kcd                        1/1     Running   0          22m

dashboard 可以查看 kubernetes 系统的整体情况，为了访问 dashboard 页面，需要增加 RBAC：

ssh 登录 a001 机器；
执行以下命令，创建文件 admin-user.yaml：

tee admin-user.yaml <<-'EOF'
apiVersion: v1
kind: ServiceAccount
metadata:
  name: admin-user
  namespace: kube-system
EOF

执行以下命令，创建文件 admin-user-role.yaml：

tee admin-user-role.yaml <<-'EOF'
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: admin-user
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: admin-user
  namespace: kube-system
EOF

创建 ServiceAccount 和 ClusterRoleBinding：

kubectl create -f admin-user.yaml && kubectl create -f admin-user-role.yaml

获取 token 看，用于登录 dashboard 页面：

kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep admin-user | awk '{print $1}')

下图红框中就是 token 的内容：

现在通过浏览器访问 dashboard 页面了，地址是：https://192.168.133.139:6443/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/，其中 192.168.133.139 是 a001 机器的 IP 地址，也可以换成 a002IP 地址；
由于不是 https 协议，因此浏览器可能弹出安全提示，如下图，选择 继续前往 ：

此时页面会让您选择登录方式，选择 令牌 并输入前面得到的 token，即可登录：

登录成功后可以见到系统信息，如下图：

至此，kubespray2.11 安装 kubernetes1.15 完成，希望本文能给您一些参考。

关于 kubespray

重要前提

机器信息

标准化设置

ansible 主机免密码 ssh 登录 a001、a002、a003

ansible 主机操作

检查环境

访问 dashboard

欢迎关注公众号：程序员欣宸

Just My Socks（注册教程内含优惠码）