乐趣区

关于k8s:K8S搭建Nacos集群服务

jiezi

前言

最近在学习如何应用 k8s 搭建 nacos 服务以及如何应用，记录下来分享给大家。

筹备工作

  • K8S：我应用的是阿里云 ACK(阿里的 k8s 服务)，

  • Nacos：因为我应用的是阿里云 RDS 的 mysql，所以 用的是 nacos-group/nacos-k8s 的 nacos-no-pvc-ingress.yaml 文件

    开始搭建

    首先咱们来查看 nacos-no-pvc-ingress.yaml 文件

    ### 应用自建数据库；应用 Ingress 公布配置后盾 ###
---
apiVersion: v1
kind: Service
metadata:
name: nacos-headless
labels:
  app: nacos-headless
spec:
type: ClusterIP
clusterIP: None
ports:
  - port: 8848
    name: server
    targetPort: 8848
  - port: 9848
    name: client-rpc
    targetPort: 9848
  - port: 9849
    name: raft-rpc
    targetPort: 9849
    ## 兼容 1.4.x 版本的选举端口
  - port: 7848
    name: old-raft-rpc
    targetPort: 7848
selector:
  app: nacos
---
apiVersion: v1
kind: ConfigMap
metadata:
name: nacos-cm
data:
mysql.host: "10.127.1.12"
mysql.db.name: "nacos_devtest"
mysql.port: "3306"
mysql.user: "nacos"
mysql.password: "passwd"
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: nacos
spec:
serviceName: nacos-headless
replicas: 3
template:
  metadata:
    labels:
      app: nacos
    annotations:
      pod.alpha.kubernetes.io/initialized: "true"
  spec:
    affinity:
      podAntiAffinity:
        requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchExpressions:
                - key: "app"
                  operator: In
                  values:
                    - nacos
            topologyKey: "kubernetes.io/hostname"
    containers:
      - name: k8snacos
        imagePullPolicy: Always
        image: nacos/nacos-server:latest
        resources:
          requests:
            memory: "2Gi"
            cpu: "500m"
        ports:
          - containerPort: 8848
            name: client
          - containerPort: 9848
            name: client-rpc
          - containerPort: 9849
            name: raft-rpc
          - containerPort: 7848
            name: old-raft-rpc
        env:
          - name: NACOS_REPLICAS
            value: "3"
          - name: MYSQL_SERVICE_HOST
            valueFrom:
              configMapKeyRef:
                name: nacos-cm
                key: mysql.host
          - name: MYSQL_SERVICE_DB_NAME
            valueFrom:
              configMapKeyRef:
                name: nacos-cm
                key: mysql.db.name
          - name: MYSQL_SERVICE_PORT
            valueFrom:
              configMapKeyRef:
                name: nacos-cm
                key: mysql.port
          - name: MYSQL_SERVICE_USER
            valueFrom:
              configMapKeyRef:
                name: nacos-cm
                key: mysql.user
          - name: MYSQL_SERVICE_PASSWORD
            valueFrom:
              configMapKeyRef:
                name: nacos-cm
                key: mysql.password
          - name: MODE
            value: "cluster"
          - name: NACOS_SERVER_PORT
            value: "8848"
          - name: PREFER_HOST_MODE
            value: "hostname"
          - name: NACOS_SERVERS
            value: "nacos-0.nacos-headless.default.svc.cluster.local:8848 nacos-1.nacos-headless.default.svc.cluster.local:8848 nacos-2.nacos-headless.default.svc.cluster.local:8848"
selector:
  matchLabels:
    app: nacos
---
# ------------------- App Ingress ------------------- #
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: nacos-headless
namespace: default

spec:
rules:
- host: nacos-web.nacos-demo.com
  http:
    paths:
    - path: /
      backend:
        serviceName: nacos-headless
        servicePort: server

接着咱们改改下面的配置文件变成咱们本人的。

  • 1、ingress 反对 https，这里有篇应用 cert-manager 申请收费的 HTTPS 证书具体过程就不叙述了，过程为：
  1. 部署 cert-manager
  2. 创立 ClusterIssuer
  3. 创立 Ingress 资源对象
  • 2、ingress 配置 http 主动跳转到 https，应用 nginx.ingress.kubernetes.io/force-ssl-redirect: 'true' 注解
  • 3、ConfigMap 配置本人的 Mysql 地址和明码
  • 4、StatefulSet 集群模式下配置正本数 replicas 至多为 2，否则不起作用
  • 5、StatefulSet 设置内存、CPU 和模式

内存、CPU：

- name: k8snacos
  imagePullPolicy: Always
  image: nacos/nacos-server:latest
  resources:
    requests:
      memory: "256Mi"
      cpu: "250m"

模式：

- name: MODE
  # 单机部署，value: "standalone" 
  # 集群部署，value: "cluster"    
  value: "cluster"

其余的都不必变，如下所示：

# 参考：/Users/zhangwei/Development/com.github/nacos-k8s/deploy/nacos/nacos-no-pvc-ingress.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: nacos-headless
  namespace: default
  annotations:
    kubernetes.io/ingress.class: nginx
    # 增加 cert-manager.io/cluster-issuer 注解
    cert-manager.io/cluster-issuer: "letsencrypt-prod-http01"
    nginx.ingress.kubernetes.io/service-weight: ''nginx.ingress.kubernetes.io/force-ssl-redirect:'true'
spec:
  # 增加 ssl 证书
  tls:
    - hosts:
        # 替换为您的域名
        - baidu.com        
      secretName: server-seaurl-tls
  rules:
    - host: baidu.com
      http:
        paths:
          - path: /nacos
            backend:
              serviceName: nacos-headless
              servicePort: server
---
apiVersion: v1
kind: Service
metadata:
  name: nacos-headless
  labels:
    app: nacos-headless
spec:
  type: ClusterIP
  # headless service
  clusterIP: None
  ports:
    - port: 8848
      name: server
      targetPort: 8848
    - port: 9848
      name: client-rpc
      targetPort: 9848
    - port: 9849
      name: raft-rpc
      targetPort: 9849
    ## 兼容 1.4.x 版本的选举端口
    - port: 7848
      name: old-raft-rpc
      targetPort: 7848
  selector:
    app: nacos
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: nacos-cm
data:
  mysql.host: "your-aliyun-rds-host"
  mysql.db.name: "nacos"
  mysql.port: "3306"
  mysql.user: "username"
  mysql.password: "password"
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: nacos
spec:
  serviceName: nacos-headless
  # 单机模式设置成 1，集群模式设置成 3
  replicas: 3
  template:
    metadata:
      labels:
        app: nacos
      annotations:
        pod.alpha.kubernetes.io/initialized: "true"
    spec:
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchExpressions:
                  - key: "app"
                    operator: In
                    values:
                      - nacos
              topologyKey: "kubernetes.io/hostname"
      containers:
        - name: k8snacos
          imagePullPolicy: Always
          image: nacos/nacos-server:latest
          resources:
            requests:
              memory: "256Mi"
              cpu: "250m"
          ports:
            - containerPort: 8848
              name: client
            - containerPort: 9848
              name: client-rpc
            - containerPort: 9849
              name: raft-rpc
            - containerPort: 7848
              name: old-raft-rpc
          env:
            - name: NACOS_REPLICAS
              # 单机模式设置成 1，集群模式设置成 3
              value: "3"
            - name: MYSQL_SERVICE_HOST # 增加 mysql 拜访地址的环境变量
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: mysql.host
            - name: MYSQL_SERVICE_DB_NAME
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: mysql.db.name
            - name: MYSQL_SERVICE_PORT
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: mysql.port
            - name: MYSQL_SERVICE_USER
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: mysql.user
            - name: MYSQL_SERVICE_PASSWORD
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: mysql.password
            - name: NACOS_SERVER_PORT
              value: "8848"
            - name: NACOS_APPLICATION_PORT
              value: "8848"
            - name: PREFER_HOST_MODE
              value: "hostname"
            - name: NACOS_SERVERS
              # 单机模式设置成 nacos-0.xxxx, 集群模式要增加：nacos-1.xxxx 和 nacos-2.xxxxx
              value: "nacos-0.nacos-headless.default.svc.cluster.local:8848 nacos-1.nacos-headless.default.svc.cluster.local:8848 nacos-2.nacos-headless.default.svc.cluster.local:8848"
            - name: MODE
              # 单机部署，value: "standalone" 
              # 集群部署，value: "cluster"    
              value: "cluster"            
  selector:
    matchLabels:
      app: nacos

而后执行命令来创立 Nacos 服务：

kubectl apply -f nacos.yaml

通过命令查看是否胜利

kubectl get StatefulSet
kubectl get ingress
kubectl get svc
kubectl get pod


从图中能够看出部署 nacos 服务胜利，咱们拜访试试：

总结

1、首先咱们理解下什么是 Service headless，就是 type: ClusterIP 且 clusterIP: None 的 Service，所以只能通过 dns 对外去拜访你的服务 nacos-headless，
2、单机模式没有应用过，大家能够试试

注意事项

1、如果集群模式下只有一个正本会呈现问题，至多两个正本

援用

应用 cert-manager 申请收费的 HTTPS 证书
K8S 部署 Nacos 微服务
k8s 部署单节点 nacos 报错 server is DOWN now, please try again later! 解决
在 Kubernetes 中应用 DNS 和 Headless Service 发现运行中的 Pod
K8S 容器编排之 Headless 浅谈

退出移动版