一、环境介绍
默认状况下，Harbor 不附带证书。能够在没有平安爱护的状况下部署 Harbor，以便您能够通过 HTTP 连贯到它。在生产环境中，举荐始终应用 HTTPS。要配置 HTTPS，必须创立 SSL 证书。能够应用由受信赖的第三方 CA 签名的证书，也能够应用自签名证书。本文以自签名证书为例。
应用到的各个软件版本
操作系统版本：ubuntu 20.04harbor版本：v2.5.3-797c3536docker版本：20.10.8kubernetets版本：1.22.2harbor地址：192.168.10.112 域名：harbor.snow.commaster01地址：192.168.10.100 二、仓库部署
配置主机名与hosts文件
root@harbor:~# cat /etc/hosts127.0.0.1 localhost192.168.10.112 harbor.snow.com批改主机名root@barbor:~# hostnamectl set-hostname harborroot@harbor:~# bashroot@harbor:~# hostnameharbor下载harbor安装包
root@harbor:~#wget https://github.com/goharbor/harbor/releases/download/v2.5.3/harbor-offline-installer-v2.5.3.tgz解压harbor安装包root@harbor:~# tar xf harbor-offline-installer-v2.5.3.tgz -C /usr/local/src/装置docker-composeroot@harbor:~# curl -SL https://github.com/docker/compose/releases/download/v2.7.0/docker-compose-linux-x86_64 -o /usr/local/bin/docker-compose生成证书颁发机构证书及私钥
root@harbor:/usr/local/src/harbor/certs# openssl genrsa -out ca.key 4096Generating RSA private key, 4096 bit long modulus (2 primes)............................++++............................................++++e is 65537 (0x010001)root@harbor:/usr/local/src/harbor/certs# openssl req -x509 -new -nodes -sha512 -days 3650 \> -subj "/C=CN/ST=Shanghai/L=Shanghai/O=SmartX/OU=Lab/CN=harbor.snow.com" \> -key ca.key \> -out ca.crtroot@harbor:/usr/local/src/harbor/certs# lsca.crt ca.key生成服务器私钥及证书签名申请（CSR）
root@harbor:/usr/local/src/harbor/certs# openssl genrsa -out harbor.snow.com.key 4096Generating RSA private key, 4096 bit long modulus (2 primes)...........++++.....................................................................++++e is 65537 (0x010001)生成证书签名申请root@harbor:/usr/local/src/harbor/certs# openssl req -sha512 -new \> -subj "/C=CN/ST=Shanghai/L=Shanghai/O=SmartX/OU=Lab/CN=harbor.snow.com" \> -key harbor.snow.com.key \> -out harbor.snow.com.csrroot@harbor:/usr/local/src/harbor/certs# lsca.crt ca.key harbor.snow.com.csr harbor.snow.com.key生成 x509 v3 扩大文件。
无论应用 FQDN 还是 IP 地址连贯到Harbor服务器，都必须创立此文件，以便keyi 为的 Harbor 主机生成合乎主题备用名称 (SAN) 和 x509 v3 的证书扩大要求。
root@harbor:/usr/local/src/harbor/certs# cat v3.extauthorityKeyIdentifier=keyid,issuerbasicConstraints=CA:FALSEkeyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEnciphermentextendedKeyUsage = serverAuthsubjectAltName = @alt_names[alt_names]DNS.1=harbor.snow.topDNS.2=snow.topDNS.3=harbor应用该v3.ext文件为 Harbor 服务器生成证书。
root@harbor:/usr/local/src/harbor/certs# openssl x509 -req -sha512 -days 3650 \> -extfile v3.ext \> -CA ca.crt -CAkey ca.key -CAcreateserial \> -in harbor.snow.com.csr \> -out harbor.snow.com.crtSignature oksubject=C = CN, ST = Shanghai, L = Shanghai, O = SmartX, OU = Lab, CN = harbor.snow.comGetting CA Private Key将 harbor.snow.com.crt 转换为 harbor.snow.com.cert , 供 Docker 应用。Docker 守护过程将.crt文件解释为 CA 证书，.cert将文件解释为客户端证书。
root@harbor:/usr/local/src/harbor/certs# openssl x509 -inform PEM -in harbor.snow.com.crt -out harbor.snow.com.cert配置 harbor yml文件，留神须要批改hostname与指定证书地位。
root@harbor:/usr/local/src/harbor# cat harbor.yml |more# Configuration file of Harbor# The IP address or hostname to access admin UI and registry service.# DO NOT use localhost or 127.0.0.1, because Harbor needs to be accessed by external clients.hostname: harbor.snow.com# http related confighttp: # port for http, default is 80. If https enabled, this port will redirect to https port port: 80# https related confighttps: # https port for harbor, default is 443 port: 443 # The path of cert and key files for nginx certificate: /usr/local/src/harbor/certs/harbor.snow.com.cert private_key: /usr/local/src/harbor/certs/harbor.snow.com.key装置harbor
root@harbor:/usr/local/src/harbor# ./install.sh [Step 0]: checking if docker is installed ...Note: docker version: 20.10.12[Step 1]: checking docker-compose is installed ...Note: docker-compose version: 1.25.0[Step 2]: loading Harbor images ...Loaded image: goharbor/harbor-portal:v2.5.3Loaded image: goharbor/harbor-core:v2.5.3Loaded image: goharbor/redis-photon:v2.5.3Loaded image: goharbor/prepare:v2.5.3Loaded image: goharbor/harbor-db:v2.5.3Loaded image: goharbor/chartmuseum-photon:v2.5.3Loaded image: goharbor/harbor-jobservice:v2.5.3Loaded image: goharbor/harbor-registryctl:v2.5.3Loaded image: goharbor/nginx-photon:v2.5.3Loaded image: goharbor/notary-signer-photon:v2.5.3Loaded image: goharbor/harbor-log:v2.5.3Loaded image: goharbor/harbor-exporter:v2.5.3Loaded image: goharbor/registry-photon:v2.5.3Loaded image: goharbor/notary-server-photon:v2.5.3Loaded image: goharbor/trivy-adapter-photon:v2.5.3[Step 3]: preparing environment ...[Step 4]: preparing harbor configs ...prepare base dir is set to /usr/local/src/harborClearing the configuration file: /config/registryctl/envClearing the configuration file: /config/registryctl/config.ymlClearing the configuration file: /config/jobservice/envClearing the configuration file: /config/jobservice/config.ymlClearing the configuration file: /config/portal/nginx.confClearing the configuration file: /config/nginx/nginx.confClearing the configuration file: /config/log/rsyslog_docker.confClearing the configuration file: /config/log/logrotate.confClearing the configuration file: /config/registry/passwdClearing the configuration file: /config/registry/config.ymlClearing the configuration file: /config/core/envClearing the configuration file: /config/core/app.confClearing the configuration file: /config/db/envGenerated configuration file: /config/portal/nginx.confGenerated configuration file: /config/log/logrotate.confGenerated configuration file: /config/log/rsyslog_docker.confGenerated configuration file: /config/nginx/nginx.confGenerated configuration file: /config/core/envGenerated configuration file: /config/core/app.confGenerated configuration file: /config/registry/config.ymlGenerated configuration file: /config/registryctl/envGenerated configuration file: /config/registryctl/config.ymlGenerated configuration file: /config/db/envGenerated configuration file: /config/jobservice/envGenerated configuration file: /config/jobservice/config.ymlloaded secret from file: /data/secret/keys/secretkeyGenerated configuration file: /compose_location/docker-compose.ymlClean up the input dir[Step 5]: starting Harbor ...Creating network "harbor_harbor" with the default driverCreating harbor-log ... doneCreating harbor-portal ... doneCreating registryctl ... doneCreating registry ... doneCreating redis ... doneCreating harbor-db ... doneCreating harbor-core ... doneCreating nginx ... doneCreating harbor-jobservice ... done✔ ----Harbor has been installed and started successfully.----装置完harbor后验证
root@harbor:/usr/local/src/harbor# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES0304ebf0cddc goharbor/nginx-photon:v2.5.3 "nginx -g 'daemon of…" 38 seconds ago Up 35 seconds (healthy) 0.0.0.0:80->8080/tcp, :::80->8080/tcp, 0.0.0.0:443->8443/tcp, :::443->8443/tcp nginx10d1ff19120b goharbor/harbor-jobservice:v2.5.3 "/harbor/entrypoint.…" 38 seconds ago Up 36 seconds (healthy) harbor-jobservice2d5e3d8877f8 goharbor/harbor-core:v2.5.3 "/harbor/entrypoint.…" 39 seconds ago Up 38 seconds (healthy) harbor-coreee6c12da89d9 goharbor/harbor-db:v2.5.3 "/docker-entrypoint.…" 43 seconds ago Up 39 seconds (healthy) harbor-db5b7455e56733 goharbor/redis-photon:v2.5.3 "redis-server /etc/r…" 43 seconds ago Up 39 seconds (healthy) redisa0c279ca0206 goharbor/harbor-registryctl:v2.5.3 "/home/harbor/start.…" 43 seconds ago Up 41 seconds (healthy) registryctl478f35433f69 goharbor/registry-photon:v2.5.3 "/home/harbor/entryp…" 43 seconds ago Up 39 seconds (healthy) registryeb33121f4126 goharbor/harbor-portal:v2.5.3 "nginx -g 'daemon of…" 43 seconds ago Up 39 seconds (healthy) harbor-portalb0c7ab3a7133 goharbor/harbor-log:v2.5.3 "/bin/sh -c /usr/loc…" 44 seconds ago Up 43 seconds (healthy) 127.0.0.1:1514->10514/tcp harbor-log批改宿主机hosts文件，做好harbor主机的域名解析，在浏览器应用域名拜访harbor服务器发现提醒（疏忽域名，域名是第一次测试的时候用的）
在以后页面用键盘输入thisisunsafe，不是在地址栏输出，就间接敲键盘就行了，页面即会主动刷新进入网页
应用默认用户名admin，明码Harbor12345即可登录。
三、配置docker客户端
将证书发送到master01服务器,留神在操作之前要先在master01创立好寄存证书的文件夹/etc/docker/certs.d/harbor.snow.com
root@master01:~# mkdir -p /etc/docker/certs.d/harbor.snow.comroot@harbor:/usr/local/src/harbor/certs# scp harbor.snow.com.cert 192.168.10.100:/etc/docker/certs.d/harbor.snow.com/root@192.168.10.100's password: harbor.snow.com.cert 100% 2102 2.3MB/s 00:00 root@harbor:/usr/local/src/harbor/certs# scp harbor.snow.com.key 192.168.10.100:/etc/docker/certs.d/harbor.snow.com/root@192.168.10.100's password: harbor.snow.com.key 100% 2045 2.0MB/s 00:00 root@harbor:/usr/local/src/harbor/certs# scp ca.crt 192.168.10.100:/etc/docker/certs.d/harbor.snow.comroot@192.168.10.100's password: ca.crt 100% 2045 2.0MB/s 00:00 在master01节点看到证书曾经胜利发送过去
root@master01:/etc/docker/certs.d/harbor.snow.com# lsca.crt harbor.snow.com.cert harbor.snow.com.key批改docker的daemon.json文件，使其信赖harbor.snow.com仓库
root@master01:/etc/docker# cat daemon.json { "exec-opts": ["native.cgroupdriver=systemd"], "insecure-registries":["harbor.snow.com"], "registry-mirrors": [ "https://docker.mirrors.ustc.edu.cn", "http://hub-mirror.c.163.com" ], "max-concurrent-downloads": 10, "log-driver": "json-file", "log-level": "warn", "log-opts": { "max-size": "10m", "max-file": "3" }, "data-root": "/var/lib/docker"}重启docker服务并批改其hosts文件使master01节点可能通过域名找到对应的harbor仓库
root@master01:~# systemctl restart dockerroot@master01:~# cat /etc/hosts127.0.0.1 localhost192.168.10.112 harbor.snow.com登录仓库
root@master01:~# docker login harbor.snow.comUsername: adminPassword: WARNING! Your password will be stored unencrypted in /root/.docker/config.json.Configure a credential helper to remove this warning. Seehttps://docs.docker.com/engine/reference/commandline/login/#credentials-storeLogin Succeeded上传镜像进行测试，留神要提前在web页面创立好新的我的项目。
root@master01:~# docker imagesREPOSITORY TAG IMAGE ID CREATED SIZEcalico/cni v3.19.2 05bf027c9836 17 months ago 146MB为镜像改名字root@master01:~# docker tag calico/cni:v3.19.2 harbor.snow.com/calico/cni:v3.19.2推送到镜像仓库root@master01:~# docker push harbor.snow.com/calico/cni:v3.19.2The push refers to repository [harbor.snow.com/calico/cni]758850cabe72: Pushed e88fd392bb0c: Pushed fb44224b4823: Pushed v3.19.2: digest: sha256:00a619424a20d1b31c0f65d1194484aebe9f90dfc1d4d6396ce706941004860b size: 946推送到镜像仓库后即可在web页面看到刚刚推送下来镜像
镜像拉取测试
root@master01:~# docker imagesREPOSITORY TAG IMAGE ID CREATED SIZEkubernetesui/dashboard v2.3.1 e1482a24335a 18 months ago 220MBroot@master01:~# docker pull harbor.snow.com/calico/cni:v3.19.2v3.19.2: Pulling from calico/cniDigest: sha256:00a619424a20d1b31c0f65d1194484aebe9f90dfc1d4d6396ce706941004860bStatus: Downloaded newer image for harbor.snow.com/calico/cni:v3.19.2harbor.snow.com/calico/cni:v3.19.2root@master01:~# docker imagesREPOSITORY TAG IMAGE ID CREATED SIZEkubernetesui/dashboard v2.3.1 e1482a24335a 18 months ago 220MBharbor.snow.com/calico/cni v3.19.2 05bf027c9836 17 months ago 146MB4、K8S集群配置应用公有镜像仓库Harbor
在配置k8s应用harbor之前须要先实现本文的第三步，先让docker能够失常应用harbor。
在docker胜利登录harbor后会在用户家目录生成一个秘钥文件
root@master01:~# cat /root/.docker/config.json{ "auths": { "harbor.snow.com": { "auth": "YWRtaW46SGFyYm9yMTIzNDU=" } }}将秘钥进行base64加密
root@master01:~# cat /root/.docker/config.json | base64 -w 0ewoJImF1dGhzIjogewoJCSJoYXJib3Iuc25vdy5jb20iOiB7CgkJCSJhdXRoIjogIllXUnRhVzQ2U0dGeVltOXlNVEl6TkRVPSIKCQl9Cgl9Cn0=创立用户harbor认证的secret的yaml文件
root@master01:~/secret# cat registry-pull-secret.yaml apiVersion: v1kind: Secretmetadata: name: registry-pull-secret namespace: kube-systemdata: .dockerconfigjson: ewoJImF1dGhzIjogewoJCSJoYXJib3Iuc25vdy5jb20iOiB7CgkJCSJhdXRoIjogIllXUnRhVzQ2U0dGeVltOXlNVEl6TkRVPSIKCQl9Cgl9Cn0=type: kubernetes.io/dockerconfigjson创立secret，留神secret为名称空间级别的资源，调用这个secret的yaml文件要与此secret在同一个名称空间下。
root@master01:~/secret# kubectl delete -f registry-pull-secret.yaml secret "registry-pull-secret" deletedroot@master01:~/secret# kubectl apply -f registry-pull-secret.yaml secret/registry-pull-secret createdroot@master01:~/secret# kubectl get secretNAME TYPE DATA AGEdefault-token-rxd89 kubernetes.io/service-account-token 3 5h2mregistry-pull-secret kubernetes.io/dockerconfigjson 1 3h最初只须要在须要去harbor拉取镜像的pod的yaml文件中增加如下配置即可（只展现出了实现yaml文件的局部内容）留神其实与containers语句是平级的。
template: metadata: labels: k8s-app: metrics-server spec: imagePullSecrets: - name: registry-pull-secret containers: image: harbor.snow.com/base/metrics-server:v0.5.0 imagePullPolicy: Always配置实现后k8s集群即可胜利拉取harbor中的镜像。
ps：以上内容在自己实现环境中已试验胜利，如发现有问题或表述不清的中央欢送斧正。