乐趣区

关于etcd:etcd单点迁移到高可用集群

jiezi

因为一个业务 etcd 存在单点，所以独自搭建一个集群，替换掉原来的单点，在数据同步的时候还折腾了一下，好忘性比方烂笔头!!!

一、部署一个全新的 etcd 集群

OLD
etcd=172.0.254.66

NEW
etcd1=172.0.254.5
etcd2=172.0.254.6
etcd3=172.0.254.7

在脚本处填写 3 台 ETCD 集群的 IP，并在每台服务器执行

#!/bin/bash

# 下载二进制 etcd 并装置
version=v3.1.11
downloadUrl=https://github.com/etcd-io/etcd/releases/download

etcd1=172.0.254.5
etcd2=172.0.254.6
etcd3=172.0.254.7

localIp=$(ip a show eth0|grep -o -P '(\d*\.){3}\d*'|head -1)

if ["$localIp"  == "$etcd1"];then
    etcdNum="etcd-1"
elif ["$localIp" == "$etcd2"];then
    etcdNum="etcd-2"
elif ["$localIp" == "$etcd3"];then
    etcdNum="etcd-3"
else
    echo "local server ip is not etcd server:${localIp}"; exit
fi

mkdir -p /soft

curl -L ${downloadUrl}/${version}/etcd-${version}-linux-amd64.tar.gz -o /soft/etcd-${version}-linux-amd64.tar.gz

cd /soft && tar -xf /soft/etcd-${version}-linux-amd64.tar.gz

mv /soft/etcd-${version}-linux-amd64 /soft/etcd

/soft/etcd/etcd --version
/soft/etcd/etcdctl version

# etcd 配置文件
mkdir -p /soft/etcd/conf

cat >/soft/etcd/conf/etcd.yml <<EOF
name: $etcdNum
data-dir: /data/etcd
listen-client-urls: http://${localIp}:2379,http://127.0.0.1:2379
advertise-client-urls: http://${localIp}:2379,http://127.0.0.1:2379
listen-peer-urls: http://${localIp}:2380
initial-advertise-peer-urls: http://${localIp}:2380
initial-cluster: etcd-1=http://${etcd1}:2380,etcd-2=http://${etcd2}:2380,etcd-3=http://${etcd3}:2380
initial-cluster-token: etcd-cluster-token
initial-cluster-state: new
EOF

nohup /soft/etcd/etcd --config-file=/soft/etcd/conf/etcd.yml >>/soft/etcd/stdout.out 2>&1 &

ps -ef|grep etcd

二、过程治理

通过一个 shell 脚本治理过程
vim /etc/init.d/etcd

#!/bin/bash
# chkconfig: - 00 00
# description: etcd manager
# date=2020.11.05
# 用于治理过程启动敞开查看


# 启动程序文件
command=/soft/etcd/etcd


function func_getpid()
{pid=$(ps -ef | grep "$command"|grep -v "grep"|awk '{print $2}')
}

function func_start(){
    func_getpid
    [-n "$pid"] && {echo "[start] $command is already unning,exit";exit; }
    
    nohup /soft/etcd/etcd --config-file=/soft/etcd/conf/etcd.yml >>/soft/etcd/stdout.out 2>&1 &

    if [$? == 0];then
        echo "[start] suscess"
    else
        echo "[start] error"
    fi
}

function func_stop(){
    func_getpid
    for i in ${pid[@]}
    do
        kill -9 $i || echo "[stop] error"
        sleep 1 && echo "[stop] $command pid:$i stoped"
    done
}

function func_status(){
    func_getpid
    if [! -n "$pid"];then
        echo "[check] $command is already stoped"
    else
        for i in ${pid[@]}
        do
            echo "[check] $command is running,pid is $i"
        done
    fi
}

function func_manager()
{
    case "$1" in
    start)
        func_start
        func_status
        ;;
    stop)
        func_stop
        func_status
        ;;
    status)
        func_status
        ;;
    restart)
        func_status
        func_stop
        func_start
        func_status
        ;;
    *)
        echo "Arguments use start|status|stop|restart"
        ;;
    esac
}

if ["$#" -ne "1"];then
    echo "Arguments number need eq 1"
    exit 1
fi

func_manager $1

三、迁徙

make-mirror 简介
参考：https://www.mankier.com/1/etc…

make-mirror 命令的字面意思是：制作一个指标 etcd 集群的镜像，次要用来多个 etcd 集群合并，但要求集群间的 key 不能反复，咱们是一个现有的单点迁徙到空集群，因而刚好满足

查看 etcdctl make-mirror 命令解释

ETCDCTL_API=3 etcdctl make-mirror [options] <destination> [flags]
make-mirror        Makes a mirror at the destination etcd cluster
$ etcdctl make-mirror --help
NAME:
    make-mirror - Makes a mirror at the destination etcd cluster

USAGE:
    etcdctl make-mirror [options] <destination>

OPTIONS:
      --dest-cacert=""            Verify certificates of TLS enabled secure servers using this CA bundle
      --dest-cert=""            Identify secure client using this TLS certificate file for the destination cluster
      --dest-insecure-transport[=true]    Disable transport security for client connections
      --dest-key=""            Identify secure client using this TLS key file
      --dest-prefix=""            destination prefix to mirror a prefix to a different prefix in the destination cluster
      --no-dest-prefix[=false]        mirror key-values to the root of the destination cluster
      --prefix=""            Key-value prefix to mirror

GLOBAL OPTIONS:
      --cacert=""                verify certificates of TLS-enabled secure servers using this CA bundle
      --cert=""                    identify secure client using this TLS certificate file
      --command-timeout=5s            timeout for short running command (excluding dial timeout)
      --dial-timeout=2s                dial timeout for client connections
      --endpoints=[127.0.0.1:2379]        gRPC endpoints
      --hex[=false]                print byte strings as hex encoded strings
      --insecure-skip-tls-verify[=false]    skip server certificate verification
      --insecure-transport[=true]        disable transport security for client connections
      --key=""                    identify secure client using this TLS key file
      --user=""                    username[:password] for authentication (prompt if password is not supplied)
  -w, --write-out="simple"            set the output format (fields, json, protobuf, simple, table)

同步 etcd 数据
在其中一台新的 etcd 服务器执行, 例如在 etcd1:172.0.254.5 下面执行

ETCDCTL_API=3 etcdctl make-mirror --endpoints=http://172.0.253.66:2379 http://172.0.254.5:2379

四、验证

export ETCDCTL_API=3
etcdctl get  --prefix --keys-only "" #查看所有 key
# 导出所有 key 的值
for i in `etcdctl get  --prefix --keys-only "" |sort`;do etcdctl get $i >>/tmp/etcdcheck.log;done

将新老 etcd 集群的数据导出来比照一下键值，看看是否统一，统一则通过
注：这里只适宜小数据量比照

退出移动版