关于kubernetes:Kubernetes搭建spinnaker服务

99次阅读

共计 37912 个字符，预计需要花费 95 分钟才能阅读完成。

背景：

2017-2018 年左右的吧，不记得看什么了看到了 spinnaker，然而过后真的装置不起来。各种被墙裂。2020 年底学习了泽阳大佬的 spinnaker 实际课程。通过 Halyard 形式搭建了 spinnaker 的集群，并与 jenkins gitlab harbor k8s 实现了集成。2021 年初略微玩了一下，就去整别的事件去了，没有能利用于线上环境。下半年了，jenkins k8s 这些的流程当初根本都是清晰了。想把 cd 从 jenkins 中剥离进去教给 spinnaker 了，就从新复习一下 spinnaker 吧！

对于 spinnaker

spinnaker 是 Netfix 公司开源的一款继续部署工具，采纳 java 语言编写，遵循微服务的设计思维，指标是为团队提供灵便的继续部署流水线并提供软件的部署效率

spinnaker 的劣势

  • 反对多云部署
  • 主动公布

  • 内置部署最佳实际

    spinnaker 架构

    对于 spinnaker 的架构阐明

  • deck-基于浏览器的 UI
  • gate 微服务 api 网关，Spinnaker UI 和所有 api 调用者通过 Gate 与 Spinnaker 通信
  • orca 流水线阶段编排引擎。它解决所有长期操作和管道。浏览无关 Orca 服务概述的更多信息
  • clouddriver 负责对云提供商的所有变异调用以及索引 / 缓存所有部署的资源。
  • front50 用于长久化应用程序、管道、我的项目和告诉的元数据

  • rosco 为各种云提供商生成不可变的 VM 映像（或映像模板）

    它用于生成机器映像（例如 GCE 映像、AWS AMI、Azure VM 映像）。它目前包装了 packer，但将 被扩大以反对用于生成图像的其余机制。

  • igor 用于通过 Jenkins 和 Travis CI 等零碎中的继续集成作业触发管道，它容许在管道中应用 Jenkins/Travis 阶段
  • echo 事件总线 它反对发送告诉（例如 Slack、电子邮件、SMS），并对来自 Github 等服务的传入 webhook 采取行动。
  • fiat 认证受权核心 它用于查问用户对帐户、应用程序和服务帐户的拜访权限
  • kayenta 主动金丝雀剖析

  • Keel 为治理交付提供能源

    注：这个还没有用过

  • halyard 配置服务 治理上述每项服务的生命周期。它仅在 Spinnaker 启动、更新和回滚期间与这些服务交互。

    服务依赖调用关系：


    重要的事件：这些货色去看官网文档很是具体，比其余的比拟具体多了：https://spinnaker.io/docs/reference/architecture/microservices-overview/

Kubernetes 搭建 spinnaker 服务

注：spinnaker 的装置形式有 helm 和 halyard 的本地部署形式 这里采纳了 halyard 的形式！。根本过程参照泽阳大佬的 spinnaker 课程！
自己集群环境为 kubernetes1.20.6 rutime 应用了 containerd 并没有采纳 docker。两头过程尝试了很屡次各种失败，先基于 docker 的形式做一次装置部署。前面分析一下 containerd 形式！

根本环境

腾讯云同一 vpc 内服务器，内网互通，ip 为内网地址

主机名 ip 零碎 内核 k8s 版本
k8s-master-01 10.0.0.41 CentOS Linux 8 5.4.134-1.el8.elrepo.x86_64 v1.21.3 containerd
k8s-master-02 10.0.0.34 CentOS Linux 8 5.4.134-1.el8.elrepo.x86_64 v1.21.3 containerd
k8s-master-03 10.0.0.26 CentOS Linux 8 5.4.134-1.el8.elrepo.x86_64 v1.21.3 containerd
k8s-node-01 10.0.4.49 CentOS Linux 8 5.4.134-1.el8.elrepo.x86_64 v1.21.3 containerd
k8s-node-02 10.0.4.48 CentOS Linux 8 5.4.134-1.el8.elrepo.x86_64 v1.21.3 containerd
k8s-node-03 10.0.4.23 CentOS Linux 8 5.4.134-1.el8.elrepo.x86_64 v1.21.3 containerd
k8s-node-04 10.0.4.47 CentOS Linux 8 5.4.134-1.el8.elrepo.x86_64 v1.21.3 containerd
k8s-node-05 10.0.4.32 CentOS Linux 8 5.4.134-1.el8.elrepo.x86_64 v1.21.3 containerd
k8s-node-06 10.0.4.18 CentOS Linux 8 5.4.134-1.el8.elrepo.x86_64 v1.21.3 docker
k8s-01 10.0.2.17 CentOS Linux 8 4.18.0-305.12.1.el8_4.x86_64 不在集群内（然而也是一个测试的 k8s 集群，故下面的其余 pod 疏忽） docker(集群外一台运行 docker 的服务器)

注：集体尝试 containerd 运行 halyard 未能胜利，最终应用 docker 形式运行 halyard

基于 docker runtime 形式部署 halyard 的形式部署 spinnaker

注：对于 halyard 的操作都在 k8s-01 节点操作。另外申明一下 k8s-01 原主机名为 k8s-02 应用了 hostnamectl set-hostname 批改主机名。有些截图或者命令都仍然为 k8-02，理论为同一个台服务器。xshell 早些时候关上 10.0.2.17 的窗口 ……

下载镜像，挂载本地配置文件目录，并启动容器

[root@k8s-01 ~]# docker pull registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0
#### 创立.hall 文件夹前面长久化存储 spinnaker 生成文件
[root@k8s-01 ~]# mkdir -p /home/spinnaker/.hal
### 创立.kube 文件夹并将集群中的 config 文件上传到此目录
[root@k8s-01 ~]# mkdir -p /home/spinnaker/.kube
[root@k8s-01 ~]# ls  /home/spinnaker/.kube
config
#### 启动 halyard 容器
[root@k8s-01 ~]# docker run -itd --name halyard   -v /home/spinnaker/.hal:/home/spinnaker/.hal   -v /home/spinnaker/.kube:/home/spinnaker/.kube   registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0

特权身份进入容器敞开 gcs

## 以 root 身份进入容器，批改配置文件
[root@k8s-01 .kube]# docker exec -it -u root halyard bash
bash-5.0# 
## 批改 spinnaker.config.input.gcs.enabled = false。vi /opt/halyard/config/halyard.yml
 
spinnaker:
  artifacts:
    debian: https://dl.bintray.com/spinnaker-releases/debians
    docker: gcr.io/spinnaker-marketplace
  config:
    input:
      gcs:
        enabled: false
      writerEnabled: false
      bucket: halconfig

重新启动 halyard 容器

## 须要重启容器（如果此命令未重启，则须要退出容器而后 docker restart halyard）bash-5.0# hal shutdown
Halyard Daemon Response: Shutting down, bye...
## 重启容器
[root@k8s-01 .kube]# docker start halyard
halyard

上传 boms 文件到服务器

参照 https://github.com/zeyangli/spinnaker-cd-install, 这里应用的是 https://github.com/zeyangli/spinnaker-cd-install/actions/runs/1368350526 1.26.6 的制品：

### 通过 rz 命令上传制品库到运行 halyard 的服务器，并解压压缩包
[root@k8s-01 work]# ls
1.26.6-Install-Scripts.zip
[root@k8s-01 work]# unzip 1.26.6-Install-Scripts.zip


嗯看到了这个.boms 的文件夹，将其 copy 到 /home/spinnaker/.hal/ 目录下！

[root@k8s-01 1.26.6]# ls .boms/
bom  clouddriver  deck  echo  fiat  front50  gate  igor  kayenta  monitoring-daemon  orca  rosco
[root@k8s-01 1.26.6]# cp -Ra .boms/ /home/spinnaker/.hal/
[root@k8s-01 1.26.6]# ls /home/spinnaker/.hal/.boms/
bom  clouddriver  deck  echo  fiat  front50  gate  igor  kayenta  monitoring-daemon  orca  rosco

对于镜像的下载

镜像下载泽阳大佬的制品库下载中有下载镜像的脚本：

#!/bin/bash

S_REGISTRY="gcr.io/spinnaker-marketplace"
#T_REGISTRY="registry.cn-beijing.aliyuncs.com/spinnaker-cd"
T_REGISTRY="docker.io/spinnakercd"
NODES="node01.zy.com node02.zy.com"

## 下载镜像
function GetImages(){
    echo -e "\033[43;34m =====GetImg===== \033[0m"

    IMAGES=$(cat tagfile.txt)

    for image in ${IMAGES}
    do
        for node in ${NODES}
        do 
           echo  -e "\033[32m ${node} ---> pull ---> ${image} \033[0m"
           ssh ${node} "docker pull ${T_REGISTRY}/${image}"
           echo  -e "\033[32m ${node} ---> tag ---> ${image} \033[0m"
           ssh ${node} "docker tag ${T_REGISTRY}/${image} ${S_REGISTRY}/${image}"
        done
    done
    for node in ${NODES}
    do
       echo -e "\033[43;34m =====${node}=== 镜像信息 ===== \033[0m"
       ssh ${node} "docker images | grep'spinnaker-marketplace' "
    done
    
}

GetImages

But 我的集群的运行时是containerd。ctr crictl 两个命令的区别有必要从新复习一下。crictl 也没法批改标签啊？

#!/bin/bash

S_REGISTRY="gcr.io/spinnaker-marketplace"
#T_REGISTRY="registry.cn-beijing.aliyuncs.com/spinnaker-cd"
T_REGISTRY="docker.io/spinnakercd"
NODES="10.0.4.18 10.0.4.49 10.0.4.48 10.0.4.23 10.0.4.47 10.0.4.32"

## 下载镜像
function GetImages(){
    echo -e "\033[43;34m =====GetImg===== \033[0m"

    IMAGES=$(cat tagfile.txt)

    for image in ${IMAGES}
    do
        for node in ${NODES}
        do 
           echo  -e "\033[32m ${node} ---> pull ---> ${image} \033[0m"
           ssh -p 36000 ${node} "crictl pull  ${T_REGISTRY}/${image}"
           echo  -e "\033[32m ${node} ---> tag ---> ${image} \033[0m"
           ssh -p 36000 ${node} "crictl images ${T_REGISTRY}/${image} ${S_REGISTRY}/${image}"
        done
    done
    for node in ${NODES}
    do
       echo -e "\033[43;34m =====${node}=== 镜像信息 ===== \033[0m"
       ssh -p 36000 ${node} "crictl images ls| grep'spinnaker-marketplace' "
    done
    
}

GetImages

所以这个形式就行不通了, 而后偶尔搜到 csdn 的 — 装置篇——用 halyard 装置 Spinnaker。通过在.hall 目录下 default/service-settings/ 目录创立对应配置文件。并设置 artifactId！
至于 service-settings 目录为什么在 default 目录下我也不求甚解泽阳大佬的课程中批改 redis 为内部 redis 的时候有这个目录

[root@k8s-2 .hal]# mkdir -p /home/spinnaker/.hal/default/service-settings
[root@k8s-2 .hal]# cd /home/spinnaker/.hal/default/service-settings
[root@k8s-2 service-settings]# pwd
/home/spinnaker/.hal/default/service-settings
[root@k8s-2 service-settings]# ls
clouddriver.yml  deck.yml  echo.yml  fiat.yml  front50.yml  gate.yml  igor.yml  kayenta.yml  orca.yml  rosco.yml
[root@k8s-2 service-settings]# cat *
artifactId: docker.io/spinnakercd/clouddriver:8.0.4-20210625060028
artifactId: docker.io/spinnakercd/deck:3.7.2-20210614020020 
artifactId: docker.io/spinnakercd/echo:2.17.1-20210429125836 
artifactId: docker.io/spinnakercd/fiat:1.16.0-20210422230020
artifactId: docker.io/spinnakercd/front50:0.27.1-20210625161956
artifactId: docker.io/spinnakercd/gate:1.22.1-20210603020019
artifactId: docker.io/spinnakercd/igor:1.16.0-20210422230020
artifactId: docker.io/spinnakercd/kayenta:0.21.0-20210322140019 
artifactId: docker.io/spinnakercd/orca:2.20.3-20210630022216
artifactId: docker.io/spinnakercd/rosco:0.25.0-20210422230020



就不批改标签间接应用泽阳大佬 docker 的镜像仓库外面的镜像了免去下载镜像批改标签的步骤

Halyard 配置管理

注：halyard 的配置都在 k8s-01 节点执行默认在 halyard 容器内

设置 Spinnaker 版本，–version 指定版本

[root@k8s-01 .kube]# docker exec -it -u root halyard bash
bash-5.0$ hal config version edit --version local:1.26.6
+ Get current deployment
  Success
- Edit Spinnaker version
  Failure
Validation in Global:
! ERROR Failure writing your halconfig to path
  "/home/spinnaker/.hal/config": /home/spinnaker/.hal/config

- Failed to update version.


嗯强调一下 .hall 目录要有读写权限啊

[root@k8s-01 1.26.6]# chmod 777 -R /home/spinnaker/.hal/
[root@k8s-01 1.26.6]#

持续指定 spinnaker 版本并生成配置文件

bash-5.0$ hal config version edit --version local:1.26.6
+ Get current deployment
  Success
+ Edit Spinnaker version
  Success
+ Spinnaker has been configured to update/install version
  "local:1.26.6". Deploy this version of Spinnaker with `hal deploy apply`.
bash-5.0$ ls
config   default
bash-5.0$ cat config 
currentDeployment: default
deploymentConfigurations:
- name: default
  version: local:1.26.6
  providers:
    appengine:
      enabled: false
      accounts: []
    aws:
      enabled: false
      accounts: []
      bakeryDefaults:
        baseImages: []
      defaultKeyPairTemplate: '{{name}}-keypair'
      defaultRegions:
      - name: us-west-2
      defaults:
        iamRole: BaseIAMRole
    ecs:
      enabled: false
      accounts: []
    azure:
      enabled: false
      accounts: []
      bakeryDefaults:
        templateFile: azure-linux.json
        baseImages: []
    dcos:
      enabled: false
      accounts: []
      clusters: []
    dockerRegistry:
      enabled: false
      accounts: []
    google:
      enabled: false
      accounts: []
      bakeryDefaults:
        templateFile: gce.json
        baseImages: []
        zone: us-central1-f
        network: default
        useInternalIp: false
    huaweicloud:
      enabled: false
      accounts: []
      bakeryDefaults:
        baseImages: []
    kubernetes:
      enabled: false
      accounts: []
    tencentcloud:
      enabled: false
      accounts: []
      bakeryDefaults:
        baseImages: []
    oracle:
      enabled: false
      accounts: []
      bakeryDefaults:
        templateFile: oci.json
        baseImages: []
    cloudfoundry:
      enabled: false
      accounts: []
  deploymentEnvironment:
    size: SMALL
    type: LocalDebian
    imageVariant: SLIM
    updateVersions: true
    consul:
      enabled: false
    vault:
      enabled: false
    customSizing: {}
    sidecars: {}
    initContainers: {}
    hostAliases: {}
    affinity: {}
    tolerations: {}
    nodeSelectors: {}
    gitConfig:
      upstreamUser: spinnaker
    livenessProbeConfig:
      enabled: false
    haServices:
      clouddriver:
        enabled: false
        disableClouddriverRoDeck: false
      echo:
        enabled: false
  persistentStorage:
    azs: {}
    gcs:
      rootFolder: front50
    redis: {}
    s3:
      rootFolder: front50
    oracle: {}
  features:
    auth: false
    fiat: false
    chaos: false
    entityTags: false
  metricStores:
    datadog:
      enabled: false
      tags: []
    prometheus:
      enabled: false
      add_source_metalabels: true
    stackdriver:
      enabled: false
    newrelic:
      enabled: false
      tags: []
    period: 30
    enabled: false
  notifications:
    slack:
      enabled: false
    twilio:
      enabled: false
      baseUrl: https://api.twilio.com/
    github-status:
      enabled: false
  timezone: America/Los_Angeles
  ci:
    jenkins:
      enabled: false
      masters: []
    travis:
      enabled: false
      masters: []
    wercker:
      enabled: false
      masters: []
    concourse:
      enabled: false
      masters: []
    gcb:
      enabled: false
      accounts: []
    codebuild:
      enabled: false
      accounts: []
  repository:
    artifactory:
      enabled: false
      searches: []
  security:
    apiSecurity:
      ssl:
        enabled: false
    uiSecurity:
      ssl:
        enabled: false
    authn:
      oauth2:
        enabled: false
        client: {}
        resource: {}
        userInfoMapping: {}
      saml:
        enabled: false
        userAttributeMapping: {}
      ldap:
        enabled: false
      x509:
        enabled: false
      iap:
        enabled: false
      enabled: false
    authz:
      groupMembership:
        service: EXTERNAL
        google:
          roleProviderType: GOOGLE
        github:
          roleProviderType: GITHUB
        file:
          roleProviderType: FILE
        ldap:
          roleProviderType: LDAP
      enabled: false
  artifacts:
    bitbucket:
      enabled: false
      accounts: []
    gcs:
      enabled: false
      accounts: []
    oracle:
      enabled: false
      accounts: []
    github:
      enabled: false
      accounts: []
    gitlab:
      enabled: false
      accounts: []
    gitrepo:
      enabled: false
      accounts: []
    http:
      enabled: false
      accounts: []
    helm:
      enabled: false
      accounts: []
    s3:
      enabled: false
      accounts: []
    maven:
      enabled: false
      accounts: []
    templates: []
  pubsub:
    enabled: false
    google:
      enabled: false
      pubsubType: GOOGLE
      subscriptions: []
      publishers: []
  canary:
    enabled: false
    serviceIntegrations:
    - name: google
      enabled: false
      accounts: []
      gcsEnabled: false
      stackdriverEnabled: false
    - name: prometheus
      enabled: false
      accounts: []
    - name: datadog
      enabled: false
      accounts: []
    - name: signalfx
      enabled: false
      accounts: []
    - name: aws
      enabled: false
      accounts: []
      s3Enabled: false
    - name: newrelic
      enabled: false
      accounts: []
    reduxLoggerEnabled: true
    defaultJudge: NetflixACAJudge-v1.0
    stagesEnabled: true
    templatesEnabled: true
    showAllConfigsEnabled: true
  spinnaker:
    extensibility:
      plugins: {}
      repositories: {}
  webhook:
    trust:
      enabled: false
  stats:
    enabled: true
    endpoint: https://stats.spinnaker.io
    instanceId: 01FKDR1B3P8PF35RRC93XTE9AS
    deploymentMethod: {}
    connectionTimeoutMillis: 3000
    readTimeoutMillis: 5000
bash-5.0$

设置时区

# 设置时区
hal config edit --timezone Asia/Shanghai

S3–no-validate

# 设置存储为 s3(前面不必，然而必须配置 bug)
hal config storage edit --type s3  --no-validate

拜访形式, 设置 deck 与 gate 的域名

# 拜访形式：设置 deck 与 gate 的域名
hal config security ui edit --override-base-url http://spinnaker.xxxx.com
hal config security api edit --override-base-url http://spin-gate.xxxx.com


来比照一下执行以上命令后 config 文件的变动：


做这些比照是为了不便当前本人手动更改配置文件。大佬的能够疏忽这些截图步骤。

增加镜像仓库（harbor）和 k8s 集群账户

开启镜像仓库配置并增加 account

bash-5.0$ hal config provider docker-registry enable --no-validate
+ Get current deployment
  Success
+ Edit the dockerRegistry provider
  Success
+ Successfully enabled dockerRegistry
bash-5.0$ hal config provider docker-registry account add my-harbor-registry \
>     --address https://harbor.xxxx.com \
>     --username xxxx \
>     --password xxxx
+ Get current deployment
  Success
+ Add the my-harbor-registry account
  Success
Validation in
  default.provider.dockerRegistry.my-harbor-registry:
- WARNING Your docker registry has no repositories specified, and
  the registry's catalog is empty. Spinnaker will not be able to deploy any images
  until some are pushed to this registry.
? Manually specify some repositories for this docker registry to
  index.

+ Successfully added account my-harbor-registry for provider
  dockerRegistry.

开启 kubernetes 配置并增加 account

bash-5.0$ hal config provider kubernetes enable
+ Get current deployment
  Success
+ Edit the kubernetes provider
  Success
Validation in default.provider.kubernetes:
- WARNING Provider kubernetes is enabled, but no accounts have been
  configured.

+ Successfully enabled kubernetes
bash-5.0$ hal config provider kubernetes account add default \
>     --docker-registries my-harbor-registry \
>     --context $(kubectl config current-context) \
>     --service-account true \
>     --omit-namespaces=kube-system,kube-public \
>     --provider-version v2 \
>     --no-validate
+ Get current deployment
  Success
+ Add the default account
  Success
+ Successfully added account default for provider kubernetes.


再瞄一眼配置文件 config:

指定部署应用 account 和命名空间，部署形式 distributed（分布式）

​

bash-5.0$ hal config deploy edit \
>     --account-name default \
>     --type distributed \
>     --location spinnaker


看了一眼配置文件应该对应的是 deploymentEnvironment 上面的配置：

开启一些次要的性能（前期能够再追加）

bash-5.0$ hal config features edit --pipeline-templates true
bash-5.0$ hal config features edit --artifacts true
bash-5.0$ hal config features edit --managed-pipeline-templates-v2-ui true

查看 config 配置文件对应的为 features 下开关：

配置与 jenkins CI 集成

# 配置 Jenkins
hal config ci jenkins enable
### JenkinsServer 须要用到账号和明码
hal config ci jenkins master add my-jenkins-master-01 \
    --address https://jenkins.xxxx.com \
    --username zhangpeng \
    --password xxxx
### 启用 csrf
hal config ci jenkins master edit my-jenkins-master-01 --csrf true


cat config 对应如下：当然了也能够开启 travis wercker consourse gcb 等 ci 工具？

配置 GitHub/GitLab 集成

github 的是泽阳大佬的。我这里就只集成了 gitlab。github 仅供参考在配置文件中也生成一下。不便比照配置文件。token 的生成就不必做过多的赘述了！

# GitHub
## 参考：https://spinnaker.io/setup/artifacts/github/
## 创立 token https://github.com/settings/tokens

hal config artifact github enable

hal config artifact github account add my-github-account \
    --token xxxxxxxxxxxxxxxxxxxxxxx  \
    --username zeyangli

# GitLab
## https://spinnaker.io/setup/artifacts/gitlab/
## 创立一个集体的 token（admin）hal config artifact gitlab enable
hal config artifact gitlab account add my-gitlab-account \
    --token xxxxxxxxxxxxxx


artifacts 下找到相干配置

应用内部 redis 集群

对于 redis 我是应用的腾讯云的云 redis。失常该搞一个明码的。然而没有去认真看下官网文档，就间接应用了免密的形式！

## service-settings
bash-5.0$ pwd
/home/spinnaker/.hal/default/service-settings
vi .hal/default/service-settings/redis.yml

overrideBaseUrl: redis://10.0.0.31:6379
skipLifeCycleManagement: true



## profiles
## /home/spinnaker/.hal/default/profiless
bash-5.0$ pwd
/home/spinnaker/.hal/default
bash-5.0$ mkdir /home/spinnaker/.hal/default/profiles
bash-5.0$ cd profiles/
bash-5.0$ vi gate-local.yml

redis:
    configuration:
         secure:
              true


应用 SQL 数据库

mysql 我是间接开启了腾讯云的 TDSQL-C

Clouddriver 服务

创立数据库：

CREATE DATABASE `clouddriver` DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;

GRANT
  SELECT, INSERT, UPDATE, DELETE, CREATE, EXECUTE, SHOW VIEW
ON `clouddriver`.*
TO 'clouddriver_service'@'%' IDENTIFIED BY 'clouddriver@spinnaker.com';


GRANT
  SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, REFERENCES, INDEX, ALTER, LOCK TABLES, EXECUTE, SHOW VIEW
ON `clouddriver`.*
TO 'clouddriver_migrate'@'%' IDENTIFIED BY 'clouddriver@spinnaker.com';

批改配置文件：

bash-5.0$ pwd
/home/spinnaker/.hal/default/profiles
bash-5.0$ vi clouddriver-local.yml
sql:
  enabled: true
  # read-only boolean toggles `SELECT` or `DELETE` health checks for all pools.
  # Especially relevant for clouddriver-ro and clouddriver-ro-deck which can
  # target a SQL read replica in their default pools.
  read-only: false
  taskRepository:
    enabled: true
  cache:
    enabled: true
    # These parameters were determined to be optimal via benchmark comparisons
    # in the Netflix production environment with Aurora. Setting these too low
    # or high may negatively impact performance. These values may be sub-optimal
    # in some environments.
    readBatchSize: 500
    writeBatchSize: 300
  scheduler:
    enabled: true

  # Enable clouddriver-caching's clean up agent to periodically purge old
  # clusters and accounts. Set to true when using the Kubernetes provider.
  unknown-agent-cleanup-agent:
    enabled: false

  connectionPools:
    default:
      # additional connection pool parameters are available here,
      # for more detail and to view defaults, see:
      # https://github.com/spinnaker/kork/blob/master/kork-sql/src/main/kotlin/com/netflix/spinnaker/kork/sql/config/ConnectionPoolProperties.kt
      default: true
      jdbcUrl: jdbc:mysql://10.0.4.22:3306/clouddriver
      user: clouddriver_service
      password: clouddriver@spinnaker.com
    # The following tasks connection pool is optional. At Netflix, clouddriver
    # instances pointed to Aurora read replicas have a tasks pool pointed at the
    # master. Instances where the default pool is pointed to the master omit a
    # separate tasks pool.
    tasks:
      user: clouddriver_service
      jdbcUrl: jdbc:mysql://10.0.4.22:3306/clouddriver
      password: clouddriver@spinnaker.com
  migration:
    user: clouddriver_migrate
    jdbcUrl: jdbc:mysql://10.0.4.22:3306/clouddriver
    password: clouddriver@spinnaker.com

redis:
  enabled: false
  cache:
    enabled: false
  scheduler:
    enabled: false
  taskRepository:
    enabled: false

Front50 服务

创立数据库

CREATE DATABASE `front50` DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;

GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, EXECUTE, SHOW VIEW ON `front50`.*  TO 'front50_service'@'%' IDENTIFIED BY "front50@spinnaker.com";

GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, REFERENCES, INDEX, ALTER, LOCK TABLES, EXECUTE, SHOW VIEW ON `front50`.* TO 'front50_migrate'@'%' IDENTIFIED BY "front50@spinnaker.com";

批改配置文件

bash-5.0$ pwd
/home/spinnaker/.hal/default/profiles
bash-5.0$ vi front50-local.yml
spinnaker:
  s3:
    enabled: false
sql:
  enabled: true
  connectionPools:
    default:
      # additional connection pool parameters are available here,
      # for more detail and to view defaults, see:
      # https://github.com/spinnaker/kork/blob/master/kork-sql/src/main/kotlin/com/netflix/spinnaker/kork/sql/config/ConnectionPoolProperties.kt
      default: true
      jdbcUrl: jdbc:mysql://10.0.4.22:3306/front50
      user: front50_service
      password: front50@spinnaker.com
  migration:
    user: front50_migrate
    jdbcUrl: jdbc:mysql://10.0.4.22:3306/front50
    password: front50@spinnaker.com

Orca 服务

创立数据库

set tx_isolation = 'REPEATABLE-READ';

CREATE SCHEMA `orca` DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;

GRANT 
SELECT, INSERT, UPDATE, DELETE, CREATE, EXECUTE, SHOW VIEW
ON `orca`.* 
TO 'orca_service'@'%' IDENTIFIED BY "orca@spinnaker.com" ;

GRANT 
SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, REFERENCES, INDEX, ALTER, LOCK TABLES, EXECUTE, SHOW VIEW 
ON `orca`.* 
TO 'orca_migrate'@'%'  IDENTIFIED BY "orca@spinnaker.com" ;

批改配置文件

bash-5.0$ pwd
/home/spinnaker/.hal/default/profiles
bash-5.0$ vi front50-local.yml
bash-5.0$ pwd
/home/spinnaker/.hal/default/profiles
bash-5.0$ vi orca-local.yml

tasks:
  useManagedServiceAccounts: true
sql:
  enabled: true
  connectionPool:
    jdbcUrl: jdbc:mysql://10.0.4.22:3306/orca
    user: orca_service
    password: orca@spinnaker.com
    connectionTimeout: 5000
    maxLifetime: 30000
    # MariaDB-specific:
    maxPoolSize: 50
  migration:
    jdbcUrl: jdbc:mysql://10.0.4.22:3306/orca
    user: orca_migrate
    password: orca@spinnaker.com
# Ensure we're only using SQL for accessing execution state
executionRepository:
  sql:
    enabled: true
  redis:
    enabled: false
 
# Reporting on active execution metrics will be handled by SQL
monitor:
  activeExecutions:
    redis: false
 
# Use SQL for Orca's work queue
# Settings from Netflix and may require adjustment for your environment
# Only validated with AWS Aurora MySQL 5.7
# Please PR if you have success with other databases
keiko:
  queue:
    sql:
      enabled: true
    redis:
      enabled: false
 
queue:
  zombieCheck:
    enabled: true
  pendingExecutionService:
    sql:
      enabled: true
    redis:
      enabled: false

部署服务

bash-5.0$ hal deploy apply --no-validate


创立 Ingress 拜访 web 测试

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: spinnaker-service
  namespace: spinnaker
  annotations:
    kubernetes.io/ingress.class: traefik  
    traefik.ingress.kubernetes.io/router.entrypoints: web
spec:
  rules:
  - host: spinnaker.xxxx.com
    http:
     paths:
     - pathType: Prefix
       path: /
       backend:
          service:
            name:  spin-deck
            port:
              number: 9000
  - host: spin-gate.xxxx.com
    http:
      paths:
      - pathType: Prefix
        path: /
        backend:
          service:
            name: spin-gate
            port: 
              number: 8084


通过 web 浏览器拜访 https://spinnaker.layame.com/ 如下：

注：至于为什么拜访 https 呢？因为我的代理是 traefik slb 下面做了跳转。当然了这里应该依据本人理论的环境登程！

集成 ldap:

至于为什么集成 ldap 呢？账号平安方面思考了当然是基于，还有其余的各种形式：Google Groups, GitHub Teams, SAML Roles, or LDAP groups。参照：https://spinnaker.io/docs/setup/other_config/security/。
对于 ldap 的装置能够参考 Kuberneters 搭建 openLDAP
首先登陆 web 治理页面登陆用户：

创立 ou-devops



创立 inetOrgPerson-zhangpeng



Password 设置用户 zhangpeng 的明码

Commit 确认

最终如下：

halyard 容器中操作. 可能复制命令时候出现异常：Was passed main parameter ‘    –user-search-base’ but no main parameter was defined in your arg class。把代码复制到编辑器解决一下

hal config security authn ldap edit \
--user-search-base 'ou=devops,dc=zy,dc=com' \
--url 'ldap://192.168.1.200:389' \
--user-search-filter 'cn={0}' \
--manager-dn 'cn=admin,dc=zy,dc=com' \
--manager-password '12345678'
hal config security authn ldap enable

bash-5.0$ cd /home/spinnaker/.hal/
bash-5.0$ pwd
/home/spinnaker/.hal
bash-5.0$ cat config


web 拜访如下：狐疑我 traefik 强跳搞的

bash-5.0$ hal deploy apply --no-validate


[root@k8s-master-01 ~]# kubectl get pods -n spinnaker


期待 pod 起来


进入首页

对于受权

首先登陆 ldap web 治理页面两个用户组 groupOfUniqueNames yunwenzu devops 两个组, 依据 ldap 中组进行受权。

ldap 创立用户组与用户

yunweizu- 用户 zhangpeng


将 zhangpeng 用户增加到组中：

devop 用户组 - 用户 huozhonghao

同理将 huozhonghao 退出 devops 组

halyard 中配置：

开启 ldap security 配置。并减少相干配置：

hal config security authz ldap edit \
    --url 'ldap://172.19.252.28:389/dc=xxxx,dc=com' \
    --manager-dn 'cn=admin,dc=xxxx,dc=com' \
    --manager-password 'xxxxxx' \
    --user-dn-pattern 'cn={0}' \
    --group-search-base 'ou=devops' \
    --group-search-filter 'uniqueMember={0}' \
    --group-role-attributes 'cn' \
    --user-search-filter 'cn={0}'
hal config security authz edit --type ldap
hal config security authz enable


设置那些用户能够拜访集群账户、镜像仓库、应用程序

## 配置 yunweizu 和 group02 角色的用户能够应用 default 这个集群账户
hal config provider kubernetes account edit default \
--add-read-permission yunweizu,group02  \
--add-write-permission yunweizu
  
## 配置 yunweizu 角色的用户能够应用 my-harbor-registry 账户
hal config provider docker-registry account edit my-harbor-registry \
    --read-permissions yunweizu \
    --write-permissions yunweizu
## 更新部署    
hal deploy apply

注：group2 copy 自泽阳大佬的课程笔记。保留了没有什么实际意义。当然了也能够去掉的 ……

登陆 spinnaker web 尝试：

注：用 zhangpeng 用户建了一个空白的
devops 的用户 huozhonghao 创立一个空白的 applications 做下测试

​



就先只看到这里的权限，正告提醒通知你 read 会所有用户锁定在此应用程序之外。
具体的权限是跟 ldap 绑定的那么应该是这样的：
1. 在 ldap 治理页面中，将用户 zhangpeng 退出 devops 组

2.spinnaker 登陆 zhangpeng 用户新建一个利用，yunweizu 读写可执行，devops 组仅仅可读。
​

  1. 创立一个新的用户组 platform 将 huozhonghao 用户退出

  1. spinnaker web 登陆 huozhonghao 用户


嗯 这里也能够看到 platform 组了 批改一下权限试试，删除一下 devops 的试试：


减少 platform 组权限也是失败因为只有 read 权限，没有 writer 权限

开启管道权限

halyard 容器中操作：

bash-5.0$ pwd
/home/spinnaker/.hal/default/profiles
bash-5.0$ cat /home/spinnaker/.hal/default/profiles/orca-local.yml
tasks: 
  useManagedServiceAccounts: true

bash-5.0$ cat ~/.hal/default/profiles/settings-local.js
window.spinnakerSettings.feature.managedServiceAccounts = true;
bash-5.0$ hal deploy apply --no-validate


留神：orca-local.yml 中的开启。我其实在 orca 服务中早配置上了！

权限的一些测试

测试一下权限。登陆 zhangpeng 用户新建一个 pipeline zhangpeng


能够发现默认的 kubernetes 的 default account 并能够保留 pipeline

huozhonghao 用户批改 zhangpeng pipeline 中的 Manifest. 嗯没有操作权限

嗯给 devops 组增加一个 read kubernetes account 的权限是不是要？否则连 account 都没有！

bash-5.0$ hal deploy apply --no-validate

[root@k8s-master-01 develop]# kubectl get pods -n spinnaker

期待 clouddriver running！

[root@k8s-master-01 develop]#kubectl get svc -n spinnaker
[root@k8s-master-01 develop]# curl -X POST http://172.19.254.33:7003/roles/sync
[root@k8s-master-01 develop]#curl 172.19.254.33:7003/authorize/huozhonghao


read 权限仍然无奈看到 accout!

kubernetes default account 增加 devops 组 writer 权限：

bash-5.0$ vi config 
bash-5.0$ hal deploy apply --no-validate

持续期待 clouddriver crunning

嗯再次刷新 web 登陆 huozhonghao 用户能够看到 kubernetes default account 了然而批改 Manifest 无奈 writer。验证通过！

装置环境根本实现。其余的步骤后续操作

一些失败的尝试（还是没有胜利）

1. 下载 Halyard 镜像并启动容器 —ctr 各种命令的温习

ctr pull


[root@k8s-master-01 ~]# ctr image pull registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0
[root@k8s-master-01 ~]# mkdir /root/.hal


参考一下 docker 时代的启动形式：

docker run -itd --name halyard \
  -v /root/.hal:/home/spinnaker/.hal \
  -v /root/.kube:/home/spinnaker/.kube \
  registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0

ctr run

依着葫芦画瓢一下？

ctr run -itd --name halyard \
  -v /root/.hal:/home/spinnaker/.hal \
  -v /root/.kube:/home/spinnaker/.kube \
  registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0

两头尝试了很屡次各种 ctr 命令的确没有搞明确 …… 参考了应用 ctr 命令治理 Containerd 容器
我感觉应用 containerd 装置 spinnaker 这真的是能够温习 ctr critical 命令了

ctr create


[root@k8s-master-01 1.26.6]# ctr c create registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0 halyard   --mount type=bind,src=/root/.hal,dst=/home/spinnaker/.hal,options=rbind:row   --mount type=bind,src=/root/.kube,dst=/home/spinnaker/.kube,options=rbind:ro --mount type=bind,src=/opt/halyard/config/,dst=/opt/halyard/config/,options=rbind:rw 
[root@k8s-master-01 1.26.6]# ctr c ls
CONTAINER    IMAGE                                                           RUNTIME                  
halyard      registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0    io.containerd.runc.v2

ctr t start

[root@k8s-master-01 1.26.6]# ctr t start -d  halyard
[root@k8s-master-01 1.26.6]# ctr t ls
TASK       PID        STATUS    
halyard    1729924    RUNNING


当初问题来了 如何进入容器呢？

ctr tasks exec -t –exec-id

[root@k8s-master-01 1.26.6]# ctr tasks list
TASK       PID        STATUS    
halyard    1729924    RUNNING
[root@k8s-master-01 1.26.6]# ctr tasks exec -t --exec-id 1729924 halyard sh
/ $


ctr c rm ctr c kill—- 读写权限没有搞明确 只能采纳挂载本地文件的形式从新搞一波了

嗯哼没有权限？docker 的时候能够用 root 的特权模式进入，这里的 ctr 也没有找到相干命令。而后就偷懒吧 halyard.yml 文件 copy 进去：
true 批改为 false!

而后挂载文件夹的形式去执行！删除容器从新走一遍流程，走一遍 ctr 命令
要删除容器应该是先进行？stop？后果不出意外我想错了是 kill…… 当然了 ctr t kill –signal 9 halyard 强制也很重要
​

[root@k8s-master-01 1.26.6]# ctr t ls
TASK       PID        STATUS    
halyard    4184764    RUNNING
[root@k8s-master-01 1.26.6]# ctr t kill halyard
[root@k8s-master-01 1.26.6]# ctr t ls
TASK       PID        STATUS    
halyard    4184764    STOPPED
[root@k8s-master-01 1.26.6]# ctr t ls
TASK       PID        STATUS    
halyard    4184764    STOPPED
[root@k8s-master-01 1.26.6]# ctr c rm halyard
[root@k8s-master-01 1.26.6]# ctr t ls
TASK    PID    STATUS 

[root@k8s-master-01 1.26.6]# ctr c create registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0 halyard   --mount type=bind,src=/root/.hal,dst=/home/spinnaker/.hal,options=rbind:row   --mount type=bind,src=/root/.kube,dst=/home/spinnaker/.kube,options=rbind:ro --mount type=bind,src=/opt/halyard/config/,dst=/opt/halyard/config/,options=rbind:rw 
[root@k8s-master-01 1.26.6]# ctr c ls
CONTAINER    IMAGE                                                           RUNTIME                  
halyard      registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0    io.containerd.runc.v2    
[root@k8s-master-01 1.26.6] # ctr t start -d  halyard
[root@k8s-master-01 1.26.6] # ctr t ls
TASK       PID        STATUS    
halyard    1729924    RUNNING
[root@k8s-master-01 1.26.6] # ctr tasks exec -t --exec-id 1729924 halyard sh

下载镜像的尝试：

小伙伴们感觉下载镜像应该用上面哪个脚本？用 ctr or crictl 呢？最终应用镜像的是要 kubernetes…. 应该是用 crictl 的。ctr 搞了 kubernetes 集群利用是发现不了镜像的！

#!/bin/bash

S_REGISTRY="gcr.io/spinnaker-marketplace"
#T_REGISTRY="registry.cn-beijing.aliyuncs.com/spinnaker-cd"
T_REGISTRY="docker.io/spinnakercd"
NODES="10.0.4.18 10.0.4.49 10.0.4.48 10.0.4.23 10.0.4.47 10.0.4.32"

## 下载镜像
function GetImages(){
    echo -e "\033[43;34m =====GetImg===== \033[0m"

    IMAGES=$(cat tagfile.txt)

    for image in ${IMAGES}
    do
        for node in ${NODES}
        do 
           echo  -e "\033[32m ${node} ---> pull ---> ${image} \033[0m"
           ssh -p 36000 ${node} "crictl pull ${T_REGISTRY}/${image}"
           echo  -e "\033[32m ${node} ---> tag ---> ${image} \033[0m"
           ssh -p 36000 ${node} "ctr image tag ${T_REGISTRY}/${image} ${S_REGISTRY}/${image}"
        done
    done
    for node in ${NODES}
    do
       echo -e "\033[43;34m =====${node}=== 镜像信息 ===== \033[0m"
       ssh -p 36000 ${node} "ctr image ls | grep'spinnaker-marketplace' "
    done
    
}

GetImages

#!/bin/bash

S_REGISTRY="gcr.io/spinnaker-marketplace"
#T_REGISTRY="registry.cn-beijing.aliyuncs.com/spinnaker-cd"
T_REGISTRY="docker.io/spinnakercd"
NODES="10.0.4.18 10.0.4.49 10.0.4.48 10.0.4.23 10.0.4.47 10.0.4.32"

## 下载镜像
function GetImages(){
    echo -e "\033[43;34m =====GetImg===== \033[0m"

    IMAGES=$(cat tagfile.txt)

    for image in ${IMAGES}
    do
        for node in ${NODES}
        do 
           echo  -e "\033[32m ${node} ---> pull ---> ${image} \033[0m"
           ssh -p 36000 ${node} "crictl pull  ${T_REGISTRY}/${image}"
           echo  -e "\033[32m ${node} ---> tag ---> ${image} \033[0m"
           ssh -p 36000 ${node} "crictl images ${T_REGISTRY}/${image} ${S_REGISTRY}/${image}"
        done
    done
    for node in ${NODES}
    do
       echo -e "\033[43;34m =====${node}=== 镜像信息 ===== \033[0m"
       ssh -p 36000 ${node} "crictl images ls| grep'spinnaker-marketplace' "
    done
    
}

GetImages

当然了还有一个问题就是 crictl 能够更改镜像名字吗？貌似是不能够的 … 而后此形式就失败了。

各种失败的尝试 -containerd 下：

[root@k8s-master-01 .boms]# ctr c create registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0 halyard   --mount type=bind,src=/root/.hal,dst=/home/spinnaker/.hal,options=rbind:row   --mount type=bind,src=/root/.kube,dst=/home/spinnaker/.kube,options=rbind:ro --mount type=bind,src=/opt/halyard/config/,dst=/opt/halyard/config/,options=rbind:rw
[root@k8s-master-01 .boms]# ctr c ls
CONTAINER    IMAGE                                                           RUNTIME                  
halyard      registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0    io.containerd.runc.v2    
[root@k8s-master-01 .boms]# ctr t start -d  halyard
[root@k8s-master-01 .boms]# ctr t ls
TASK       PID        STATUS    
halyard    1775521    RUNNING
[root@k8s-master-01 .boms]# ctr tasks exec -t --exec-id 1729924 halyard sh
/ $ hal config version edit --version local:1.26.6
~ $ cd /home/spinnaker/.hal/
vi config

timezone: America/Los_Angeles  
timezone: Asia/Shanghai

hal config storage edit --type s3  --no-validate

hal config security ui edit --override-base-url http://spinnaker.layame.com
hal config security api edit --override-base-url http://spin-gate.layame.com

这都 tmd 怎么会事件 ….. 要疯了

[root@k8s-master-01 .boms]#  ctr t kill --signal 9  halyard
[root@k8s-master-01 .boms]#  ctr c rm halyard



[root@k8s-master-01 .boms]# ctr c create registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0 halyard   --mount type=bind,src=/root/.hal,dst=/home/spinnaker/.hal,options=rbind:row   --mount type=bind,src=/root/.kube,dst=/home/spinnaker/.kube,options=rbind:ro --mount type=bind,src=/opt/halyard/config/,dst=/opt/halyard/config/,options=rbind:rw
[root@k8s-master-01 .boms]# ctr c ls
CONTAINER    IMAGE                                                           RUNTIME                  
halyard      registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0    io.containerd.runc.v2    
[root@k8s-master-01 .boms]# ctr t start -d  halyard
[root@k8s-master-01 .boms]# ctr t ls
TASK       PID        STATUS    
halyard    1832934   RUNNING
[root@k8s-master-01 .boms]# ctr tasks exec -t --exec-id 1832934 halyard sh
~ $ cd /home/spinnaker/.hal/
~/.hal $ cat config |grep time
  timezone: Asia/Shanghai
  ~/.hal $ cat config |grep s3
    persistentStoreType: s3
    s3:
    s3:
      s3Enabled: true
      
~/.hal $ cat config |grep com
      baseUrl: https://api.twilio.com/
      overrideBaseUrl: http://spin-gate.layame.com
      overrideBaseUrl: http://spinnaker.layame.com
~/.hal $ hal config provider kubernetes enable
~/.hal $ hal config provider kubernetes account add default \
    --docker-registries my-harbor-registry \
    --context $(kubectl config current-context) \
    --service-account true \
    --omit-namespaces=kube-system,kube-public \
    --provider-version v2 \
    --no-validate

至于这个中央的报错 他还是须要 w 宿主机 chmod 了一下

hal config deploy edit \
    --account-name default \
    --type distributed \
    --location spinnaker 

hal config features edit --pipeline-templates true
hal config features edit --artifacts true
hal config features edit --managed-pipeline-templates-v2-ui true


尼玛又疯了！。。。。。。。。。。。。。。。。。分隔符吧 我筹备全副都批改好了这些文件了

我又开始狐疑了 一下人生：是不是我的服务器资源不够了？因为我这是 kubernetes 的 master 节点，而后呢资源只有 4 外围 8g，我找一个资源多的 server 测试一下？
先 copy 一下 .kube 下的 config

[root@k8s-node-01 home]# mkdir -p /home/spinnaker/.hal
[root@k8s-node-01 home]# mkdir -p /opt/halyard/config
[root@k8s-node-01 home]# mkdir -p /home/spinnaker/.kube
[root@k8s-node-01 home]# crictl pull registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0
Image is up to date for sha256:8673f1670b8768138cd8349b7d9843eb4fd451658227d2e9f02d5fbe454c500d
[root@k8s-node-01 home]# cd /home/spinnaker/.kube
[root@k8s-node-01 .kube]# rz

[root@k8s-node-01 .kube]# ls
config
[root@k8s-node-01 .kube]# ctr image pull registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0
[root@k8s-node-01 .kube]# ctr c create registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0 halyard   --mount type=bind,src=/home/spinnaker/.hal,dst=/home/spinnaker/.hal,options=rbind:row   --mount type=bind,src=/home/spinnaker/.kube,dst=/home/spinnaker/.kube,options=rbind:ro --mount type=bind,src=/opt/halyard/config/,dst=/opt/halyard/config/,options=rbind:rw


[root@k8s-node-01 .boms]# pwd
/home/spinnaker/.hal/.boms
[root@k8s-node-01 .boms]# ls
bom  clouddriver  deck  echo  fiat  front50  gate  igor  kayenta  monitoring-daemon  orca  rosco
[root@k8s-node-01 .boms]# cd /opt/halyard/config/
[root@k8s-node-01 config]# cat halyard.yaml

[root@k8s-node-01 ~]# ctr t ls
TASK    PID    STATUS    
[root@k8s-node-01 ~]# ctr t start -d  halyard
[root@k8s-node-01 ~]# ctr t ls
TASK       PID        STATUS    
halyard    3910255    RUNNING
[root@k8s-node-01 ~]# ctr tasks exec -t --exec-id 3910255 halyard sh
/ $ hal config version edit --version local:1.26.6
+ Get current deployment
  Success
- Edit Spinnaker version
  Failure
Validation in Global:
! ERROR Failure writing your halconfig to path
  "/home/spinnaker/.hal/config": /home/spinnaker/.hal/config

- Failed to update version.
/ $ hal config version edit --version local:1.26.6
+ Get current deployment
  Success
+ Edit Spinnaker version
  Success
+ Spinnaker has been configured to update/install version
  "local:1.26.6". Deploy this version of Spinnaker with `hal deploy apply`.
/ $ hal config edit --timezone Asia/Shanghai
******** 又 tmd  sb 了 不晓得怎么回事不试了。间接改好配置文件间接启动了！

总结以上失败 执行啥也不行 … 最初决定间接把 docker 环境面 config 文件以及其余制品搞过来试试！

my config 文件：

currentDeployment: default
deploymentConfigurations:
- name: default
  version: local:1.26.6
  providers:
    appengine:
      enabled: false
      accounts: []
    aws:
      enabled: false
      accounts: []
      bakeryDefaults:
        baseImages: []
      defaultKeyPairTemplate: '{{name}}-keypair'
      defaultRegions:
      - name: us-west-2
      defaults:
        iamRole: BaseIAMRole
    ecs:
      enabled: false
      accounts: []
    azure:
      enabled: false
      accounts: []
      bakeryDefaults:
        templateFile: azure-linux.json
        baseImages: []
    dcos:
      enabled: false
      accounts: []
      clusters: []
    dockerRegistry:
      enabled: true
      accounts:
      - name: my-harbor-registry
        requiredGroupMembership: []
        providerVersion: V1
        permissions:
          READ:
          - yunweizu
          WRITE:
          - yunweizu
        address: https://harbor.layame.com
        username: zhangpeng
        password: xxxx
        email: fake.email@spinnaker.io
        cacheIntervalSeconds: 30
        clientTimeoutMillis: 60000
        cacheThreads: 1
        paginateSize: 100
        sortTagsByDate: false
        trackDigests: false
        insecureRegistry: false
        repositories: []
      primaryAccount: my-harbor-registry
    google:
      enabled: false
      accounts: []
      bakeryDefaults:
        templateFile: gce.json
        baseImages: []
        zone: us-central1-f
        network: default
        useInternalIp: false
    huaweicloud:
      enabled: false
      accounts: []
      bakeryDefaults:
        baseImages: []
    kubernetes:
      enabled: true
      accounts:
      - name: default
        requiredGroupMembership: []
        providerVersion: V2
        permissions:
          READ:
          - yunweizu,group02 
          - devops
          WRITE:
          - yunweizu
          - devops
        dockerRegistries:
        - accountName: my-harbor-registry
          namespaces: []
        context: kubernetes-admin@kubernetes
        configureImagePullSecrets: true
        serviceAccount: true
        cacheThreads: 1
        namespaces: []
        omitNamespaces:
        - kube-system
        - kube-public
        kinds: []
        omitKinds: []
        customResources: []
        cachingPolicies: []
        oAuthScopes: []
        onlySpinnakerManaged: false
      primaryAccount: default
    tencentcloud:
      enabled: false
      accounts: []
      bakeryDefaults:
        baseImages: []
    oracle:
      enabled: false
      accounts: []
      bakeryDefaults:
        templateFile: oci.json
        baseImages: []
    cloudfoundry:
      enabled: false
      accounts: []
  deploymentEnvironment:
    size: SMALL
    type: Distributed
    accountName: default
    imageVariant: SLIM
    updateVersions: true
    consul:
      enabled: false
    vault:
      enabled: false
    location: spinnaker
    customSizing: {}
    sidecars: {}
    initContainers: {}
    hostAliases: {}
    affinity: {}
    tolerations: {}
    nodeSelectors: {}
    gitConfig:
      upstreamUser: spinnaker
    livenessProbeConfig:
      enabled: false
    haServices:
      clouddriver:
        enabled: false
        disableClouddriverRoDeck: false
      echo:
        enabled: false
  persistentStorage:
    persistentStoreType: s3
    azs: {}
    gcs:
      rootFolder: front50
    redis: {}
    s3:
      rootFolder: front50
    oracle: {}
  features:
    auth: false
    fiat: false
    chaos: false
    entityTags: false
    pipelineTemplates: true
    artifacts: true
    managedPipelineTemplatesV2UI: true
  metricStores:
    datadog:
      enabled: false
      tags: []
    prometheus:
      enabled: false
      add_source_metalabels: true
    stackdriver:
      enabled: false
    newrelic:
      enabled: false
      tags: []
    period: 30
    enabled: false
  notifications:
    slack:
      enabled: false
    twilio:
      enabled: false
      baseUrl: https://api.twilio.com/
    github-status:
      enabled: false
  timezone: Asia/Shanghai
  ci:
    jenkins:
      enabled: true
      masters:
      - name: my-jenkins-master-01
        permissions: {}
        address: https://jenkins.xxxx.com
        username: zhangpeng
        password: xxxxx
        csrf: true
    travis:
      enabled: false
      masters: []
    wercker:
      enabled: false
      masters: []
    concourse:
      enabled: false
      masters: []
    gcb:
      enabled: false
      accounts: []
    codebuild:
      enabled: false
      accounts: []
  repository:
    artifactory:
      enabled: false
      searches: []
  security:
    apiSecurity:
      ssl:
        enabled: false
      overrideBaseUrl: https://spin-gate.xxxx.com
    uiSecurity:
      ssl:
        enabled: false
      overrideBaseUrl: https://spinnaker.xxxx.com
    authn:
      oauth2:
        enabled: false
        client: {}
        resource: {}
        userInfoMapping: {}
      saml:
        enabled: false
        userAttributeMapping: {}
      ldap:
        enabled: true
        url: ldap://172.19.252.28:389
        userSearchBase: ou=devops,dc=xxxx,dc=com
        userSearchFilter: cn={0}
        managerDn: cn=admin,dc=xxxx,dc=com
        managerPassword: xxxx
      x509:
        enabled: false
      iap:
        enabled: false
      enabled: true
    authz:
      groupMembership:
        service: LDAP
        google:
          roleProviderType: GOOGLE
        github:
          roleProviderType: GITHUB
        file:
          roleProviderType: FILE
          path: /home/spinnaker/.hal/userrole.yml
        ldap:
          roleProviderType: LDAP
          url: ldap://172.19.252.28:389/dc=xxxx,dc=com
          managerDn: cn=admin,dc=xxxx,dc=com
          managerPassword: xxxx
          userDnPattern: cn={0}
          groupSearchBase: ou=devops
          userSearchFilter: cn={0}
          groupSearchFilter: uniqueMember={0}
          groupRoleAttributes: cn
      enabled: true
  artifacts:
    bitbucket:
      enabled: false
      accounts: []
    gcs:
      enabled: false
      accounts: []
    oracle:
      enabled: false
      accounts: []
    github:
      enabled: true
      accounts:
      - name: my-github-account
        username: zeyangli
        token: xxxx
    gitlab:
      enabled: true
      accounts:
      - name: my-gitlab-account
        token: xxxx
    gitrepo:
      enabled: false
      accounts: []
    http:
      enabled: false
      accounts: []
    helm:
      enabled: false
      accounts: []
    s3:
      enabled: false
      accounts: []
    maven:
      enabled: false
      accounts: []
    templates: []
  pubsub:
    enabled: false
    google:
      enabled: false
      pubsubType: GOOGLE
      subscriptions: []
      publishers: []
  canary:
    enabled: false
    serviceIntegrations:
    - name: google
      enabled: false
      accounts: []
      gcsEnabled: false
      stackdriverEnabled: false
    - name: prometheus
      enabled: false
      accounts: []
    - name: datadog
      enabled: false
      accounts: []
    - name: signalfx
      enabled: false
      accounts: []
    - name: aws
      enabled: false
      accounts: []
      s3Enabled: false
    - name: newrelic
      enabled: false
      accounts: []
    reduxLoggerEnabled: true
    defaultJudge: NetflixACAJudge-v1.0
    stagesEnabled: true
    templatesEnabled: true
    showAllConfigsEnabled: true
  spinnaker:
    extensibility:
      plugins: {}
      repositories: {}
  webhook:
    trust:
      enabled: false
  stats:
    enabled: true
    endpoint: https://stats.spinnaker.io
    instanceId: 01FKDR1B3P8PF35RRC93XTE9AS
    deploymentMethod: {}
    connectionTimeoutMillis: 3000
    readTimeoutMillis: 5000

间接搞过来试一波

上传文件并解压到 k8s-master-01 节点 home 目录下

持续

[root@k8s-master-01 .kube]# ctr c create registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0 halyard   --mount type=bind,src=/root/.hal,dst=/home/spinnaker/.hal,options=rbind:row   --mount type=bind,src=/home/spinnaker/.kube,dst=/home/spinnaker/.kube,options=rbind:ro --mount type=bind,src=/opt/halyard/config/,dst=/opt/halyard/config/,options=rbind:rw 
[root@k8s-master-01 .kube]#  ctr t start -d  halyard
[root@k8s-master-01 .kube]# ctr t ls
TASK       PID        STATUS    
halyard    3073271    RUNNING
[root@k8s-master-01 .kube]# ctr tasks exec -t --exec-id 3073271 halyard sh
bash-5.0$  hal deploy apply --no-validate

​


从新来一遍

[root@k8s-master-01 .kube]# ctr t kill --signal 9 halyard
[root@k8s-master-01 .kube]# ctr c rm halyard


[root@k8s-master-01 .hal]# ctr c create registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0 halyard   --mount type=bind,src=/root/.hal,dst=/home/spinnaker/.hal,options=rbind:row   --mount type=bind,src=/home/spinnaker/.kube,dst=/home/spinnaker/.kube,options=rbind:ro --mount type=bind,src=/opt/halyard/config/,dst=/opt/halyard/config/,options=rbind:rw
[root@k8s-master-01 .hal]# ctr t start -d  halyard
[root@k8s-master-01 .hal]# ctr t ls
TASK       PID        STATUS    
halyard    3085723    RUNNING
[root@k8s-master-01 .hal]# ctr tasks exec -t --exec-id 3085723 halyard bash
bash-5.0$

算了我放弃了 ……，containerd 的装置形式

总结一下失败以及教训：

  1. containerd or docker 的运行时中都能够在文件夹 /home/spinnaker/.hal/default/service-settings 本地写文件的件形式指定 image tag，docker 环境下还好，containerd 形式下 crictl 批改镜像标签本人把握的不是很好！
  2. containerd 命令跟 docker 还是不一样。启动 halyard 的形式还是很不好弄，最好的形式还是在一台装置 docker 的机器下面运行 halyard。
  3. halyard 执行脚本复制命令的空格格局问题
  4. 部署过程中呈现数据库地址写错问题 … 写成了 TDSQL- C 中的读地址 ….

​

正文完
kubernetes
发表至： kubernetes
2021-11-06
 0
关于kubernetes:不背锅运维一文搞清楚应用发布到k8s集群的基本流程
关于kubernetes:kubeadm方式搭建k8s环境
关于kubernetes:基于-Kubernetes-的企业级大数据平台EMR-on-ACK-技术初探
StatefulSet认识headless-service的作用sts