乐趣区

kubespray211安装kubernetes115

jiezi

关于 kubespray

Kubespray 是开源的 kubernetes 部署工具，整合了 ansible，可以方便的部署高可用集群环境，官网地址：https://github.com/kubernetes…，本文是用 kubespray2.11 版本部署 kubernetes1.15 版本的实战；

重要前提

本次实战采用官方推荐的在线安装，因此会去谷歌镜像仓库下载镜像，<font color=”red”> 需要您的网络可以访问谷歌服务 </font>；

机器信息

本次实战共计四台机器，它们的主机名、IP 地址和作用描述如下：

主机名 IP 地址 作用
ansible 192.168.133.134 ansible 主机
a001 192.168.133.139 k8s 集群的一号工作节点
a002 192.168.133.140 k8s 集群的二号工作节点
a003 192.168.133.141 k8s 集群的三号工作节点

标准化设置

本次实战的所有机器都要做以下设置：

  1. 操作系统：CentOS Linux release 7.7.1908
  2. 所以操作都是 root 账号执行的
  3. 关闭防火墙：
systemctl stop firewalld && systemctl disable firewalld
  1. 关闭 selinux：
setenforce 0
sed -i --follow-symlinks 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/sysconfig/selinux
  1. ipv4 网络设置：
modprobe br_netfilter
echo '1' > /proc/sys/net/bridge/bridge-nf-call-iptables
sysctl -w net.ipv4.ip_forward=1

ansible 主机免密码 ssh 登录 a001、a002、a003

  1. ssh 登录 ansible 主机；
  2. 生成 ssh 公私钥，输入命令 <font color=”blue”>ssh-keygen</font>，然后连续四次回车：
[root@ansible ~]# ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): 
Created directory '/root/.ssh'.
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:Empen3/RfLndRkS8mKfkq6a2IXtSdqwK7TqKNoHkNEU root@ansible
The key's randomart image is:
+---[RSA 2048]----+
|  .E           . |
|   .            o|
|  .   .       o..|
| +   . .     + o.|
|= . o o S . ooo..|
|.o o ..o + o.oo.o|
|  . .. o=.o  ..o+|
| o.  .o.o=.... .+|
|......o+=o=o.  . |
+----[SHA256]-----+
  1. 输入命令 <font color=”blue”>ssh-copy-id root@192.168.133.139</font>，将 ansible 的 ssh 分发给 a001 主机，会要求输入 <font color=”blue”>yes</font> 和 a001 主机的 root 账号的密码，完成输入后，以后 ansible 就可以免密码 ssh 登录 a001 主机了：
[root@ansible ~]# ssh-copy-id root@192.168.133.139
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub"
The authenticity of host '192.168.133.139 (192.168.133.139)' can't be established.
ECDSA key fingerprint is SHA256:DPE2nldWHiOhC4DB9doy7jPWNZVup6XFZ+sR2i1gqz8.
ECDSA key fingerprint is MD5:fc:21:f7:7f:e8:cd:1a:76:d7:fb:cc:d4:28:91:f3:5a.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@192.168.133.139's password: 

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh'root@192.168.133.139'"
and check to make sure that only the key(s) you wanted were added.
  1. 继续输入命令 <font color=”blue”>ssh-copy-id root@192.168.133.140</font> 和 <font color=”blue”>ssh-copy-id root@192.168.133.141</font>，使得 ansible 主机可以免密码登录 a002 和 a003；
  2. 至此，ansible 主机可以用命令 <font color=”blue”>ssh root@192.168.133.139</font>、<font color=”blue”>ssh root@192.168.133.140</font>、<font color=”blue”>ssh root@192.168.133.141</font> 免密码登录 a001、a002、a003 了；

ansible 主机操作

  1. ssh 登录 ansible 主机；
  2. 安装 ansible 应用：
yum install -y epel-release ansible
  1. 安装 pip：
easy_install pip
  1. 通过 pip 安装 jinja2：
pip2 install jinja2 --upgrade
  1. 安装 python36：
yum install python36 -y
  1. 创建工作目录，进入工作目录：
mkdir /usr/local/kubespray && cd /usr/local/kubespray/
  1. 下载 kubespray，我这里下载的是 <font color=”blue”>v2.11.0</font> 版本：
wget https://github.com/kubernetes-sigs/kubespray/archive/v2.11.0.tar.gz
  1. 解压：
tar -zxvf v2.11.0.tar.gz
  1. 进入解压后的目录：
cd kubespray-2.11.0/
  1. 安装 kubespray 所需的应用 (注意是 <font color=”red”>pip3</font>)：
pip3 install -r requirements.txt
  1. 复制一份 demo 配置信息到目录 <font color=”blue”>inventory/mycluster</font>：
cp -rfp inventory/sample inventory/mycluster
  1. 进去看一下，可见 mycluster 目录下复制了很多文件：
[root@ansible kubespray-2.11.0]# tree inventory/
inventory/
├── local
│   ├── group_vars -> ../sample/group_vars
│   └── hosts.ini
├── mycluster
│   ├── group_vars
│   │   ├── all
│   │   │   ├── all.yml
│   │   │   ├── azure.yml
│   │   │   ├── coreos.yml
│   │   │   ├── docker.yml
│   │   │   ├── oci.yml
│   │   │   └── openstack.yml
│   │   ├── etcd.yml
│   │   └── k8s-cluster
│   │       ├── addons.yml
│   │       ├── k8s-cluster.yml
│   │       ├── k8s-net-calico.yml
│   │       ├── k8s-net-canal.yml
│   │       ├── k8s-net-cilium.yml
│   │       ├── k8s-net-contiv.yml
│   │       ├── k8s-net-flannel.yml
│   │       ├── k8s-net-kube-router.yml
│   │       ├── k8s-net-macvlan.yml
│   │       └── k8s-net-weave.yml
│   └── inventory.ini
  1. 设置集群信息 (当前目录仍旧是 kubespray-2.11.0)：
declare -a IPS=(192.168.133.139 192.168.133.140 192.168.133.141)
  1. 配置 ansible：
CONFIG_FILE=inventory/mycluster/hosts.yml python3 contrib/inventory_builder/inventory.py ${IPS[@]}

此时 kubespray 的脚本根据输入的 IP 信息做好了集群规划，具体信息可见 <font color=”blue”>inventory/mycluster/hosts.yml</font>，如下所示，您也可以自行修改此文件：

all:
  hosts:
    node1:
      ansible_host: 192.168.133.139
      ip: 192.168.133.139
      access_ip: 192.168.133.139
    node2:
      ansible_host: 192.168.133.140
      ip: 192.168.133.140
      access_ip: 192.168.133.140
    node3:
      ansible_host: 192.168.133.141
      ip: 192.168.133.141
      access_ip: 192.168.133.141
  children:
    kube-master:
      hosts:
        node1:
        node2:
    kube-node:
      hosts:
        node1:
        node2:
        node3:
    etcd:
      hosts:
        node1:
        node2:
        node3:
    k8s-cluster:
      children:
        kube-master:
        kube-node:
    calico-rr:
      hosts: {}
  1. 执行以下命令即可开始安装，在线安装比较耗时请耐心等待：
ansible-playbook -i inventory/mycluster/hosts.yml --become --become-user=root cluster.yml

安装完成时控制台输出类似如下的信息：

PLAY RECAP ********************************************************************************************************************************************************************************
localhost                  : ok=1    changed=0    unreachable=0    failed=0   
node1                      : ok=658  changed=95   unreachable=0    failed=0   
node2                      : ok=566  changed=77   unreachable=0    failed=0   
node3                      : ok=475  changed=66   unreachable=0    failed=0   

Sunday 17 November 2019  17:31:19 +0800 (0:00:00.064)       0:09:56.193 ******* 
=============================================================================== 
kubernetes/master : kubeadm | Init other uninitialized masters -------------------------------------------------------------------------------------------------------------------- 94.91s
kubernetes/master : kubeadm | Initialize first master ----------------------------------------------------------------------------------------------------------------------------- 42.95s
etcd : Install | Copy etcdctl binary from docker container ------------------------------------------------------------------------------------------------------------------------ 14.26s
download : download_container | Download image if required ------------------------------------------------------------------------------------------------------------------------ 12.87s
download : download_container | Download image if required ------------------------------------------------------------------------------------------------------------------------ 12.28s
download : download_container | Download image if required ------------------------------------------------------------------------------------------------------------------------ 10.79s
etcd : reload etcd ---------------------------------------------------------------------------------------------------------------------------------------------------------------- 10.71s
download : download_container | Download image if required ------------------------------------------------------------------------------------------------------------------------- 9.71s
download : download_container | Download image if required ------------------------------------------------------------------------------------------------------------------------- 9.48s
download : download_container | Download image if required ------------------------------------------------------------------------------------------------------------------------- 8.02s
download : download_container | Download image if required ------------------------------------------------------------------------------------------------------------------------- 7.88s
etcd : wait for etcd up ------------------------------------------------------------------------------------------------------------------------------------------------------------ 7.16s
etcd : Gen_certs | Write etcd master certs ----------------------------------------------------------------------------------------------------------------------------------------- 6.39s
download : download_container | Download image if required ------------------------------------------------------------------------------------------------------------------------- 5.75s
download : download_container | Download image if required ------------------------------------------------------------------------------------------------------------------------- 5.53s
download : download_container | Download image if required ------------------------------------------------------------------------------------------------------------------------- 5.42s
download : download_container | Download image if required ------------------------------------------------------------------------------------------------------------------------- 5.41s
download : download_container | Download image if required ------------------------------------------------------------------------------------------------------------------------- 5.06s
download : download_container | Download image if required ------------------------------------------------------------------------------------------------------------------------- 4.87s
kubernetes-apps/ansible : Kubernetes Apps | Start Resources ------------------------------------------------------------------------------------------------------------------------ 4.78s

至此，kubernetes 集群环境部署完成，接下来简单验证一下环境是否可用；

检查环境

  1. ssh 登录 a001 机器；
  2. 查看节点、service、pod：
[root@node1 ~]# kubectl get nodes
NAME    STATUS   ROLES    AGE   VERSION
node1   Ready    master   25m   v1.15.3
node2   Ready    master   23m   v1.15.3
node3   Ready    <none>   23m   v1.15.3
[root@node1 ~]# kubectl get services --all-namespaces
NAMESPACE     NAME                   TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)                  AGE
default       kubernetes             ClusterIP   10.233.0.1    <none>        443/TCP                  25m
kube-system   coredns                ClusterIP   10.233.0.3    <none>        53/UDP,53/TCP,9153/TCP   22m
kube-system   kubernetes-dashboard   ClusterIP   10.233.35.1   <none>        443/TCP                  22m
[root@node1 ~]# kubectl get pods --all-namespaces
NAMESPACE     NAME                                      READY   STATUS    RESTARTS   AGE
kube-system   calico-kube-controllers-c6fb79b8b-v24nq   1/1     Running   0          22m
kube-system   calico-node-46s8t                         1/1     Running   0          23m
kube-system   calico-node-mcjfs                         1/1     Running   0          23m
kube-system   calico-node-q989m                         1/1     Running   1          23m
kube-system   coredns-74c9d4d795-4xz6s                  1/1     Running   0          22m
kube-system   coredns-74c9d4d795-kh6vl                  1/1     Running   0          22m
kube-system   dns-autoscaler-7d95989447-gmcrl           1/1     Running   0          22m
kube-system   kube-apiserver-node1                      1/1     Running   0          24m
kube-system   kube-apiserver-node2                      1/1     Running   0          23m
kube-system   kube-controller-manager-node1             1/1     Running   0          24m
kube-system   kube-controller-manager-node2             1/1     Running   0          23m
kube-system   kube-proxy-2zhwn                          1/1     Running   0          23m
kube-system   kube-proxy-59qx8                          1/1     Running   0          23m
kube-system   kube-proxy-fgpx6                          1/1     Running   0          23m
kube-system   kube-scheduler-node1                      1/1     Running   0          24m
kube-system   kube-scheduler-node2                      1/1     Running   0          23m
kube-system   kubernetes-dashboard-7c547b4c64-x7nfq     1/1     Running   0          22m
kube-system   nginx-proxy-node3                         1/1     Running   0          23m
kube-system   nodelocaldns-8khfq                        1/1     Running   0          22m
kube-system   nodelocaldns-pzx2p                        1/1     Running   0          22m
kube-system   nodelocaldns-s5kcd                        1/1     Running   0          22m

访问 dashboard

dashboard 可以查看 kubernetes 系统的整体情况，为了访问 dashboard 页面，需要增加 RBAC：

  1. ssh 登录 a001 机器；
  2. 执行以下命令，创建文件 <font color=”blue”>admin-user.yaml</font>：
tee admin-user.yaml <<-'EOF'
apiVersion: v1
kind: ServiceAccount
metadata:
  name: admin-user
  namespace: kube-system
EOF
  1. 执行以下命令，创建文件 <font color=”blue”>admin-user-role.yaml</font>：
tee admin-user-role.yaml <<-'EOF'
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: admin-user
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: admin-user
  namespace: kube-system
EOF
  1. 创建 ServiceAccount 和 ClusterRoleBinding：
kubectl create -f admin-user.yaml && kubectl create -f admin-user-role.yaml
  1. 获取 token 看，用于登录 dashboard 页面：
kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep admin-user | awk '{print $1}')

下图红框中就是 token 的内容：

  1. 现在通过浏览器访问 dashboard 页面了，地址是：https://192.168.133.139:6443/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/，其中 <font color=”blue”>192.168.133.139</font> 是 a001 机器的 IP 地址，也可以换成 a002IP 地址；
  2. 由于不是 https 协议，因此浏览器可能弹出安全提示，如下图，选择 <font color=”blue”> 继续前往 </font>：

  1. 此时页面会让您选择登录方式，选择 <font color=”blue”> 令牌 </font> 并输入前面得到的 token，即可登录：

  1. 登录成功后可以见到系统信息，如下图：


至此，kubespray2.11 安装 kubernetes1.15 完成，希望本文能给您一些参考。

欢迎关注公众号：程序员欣宸

退出移动版