背景:

团队要公布一组利用,springboot开发的ws利用。而后须要对外。反对ws wss协定。jenkins写完pipeline公布工作。记得过来没有上容器的时候都是用的腾讯云的cls 挂证书映射cvm端口。我当初的网络环境是这样的:Kubernetes 1.20.5 装置traefik在腾讯云下的实际(当然了本次的环境是跑在tke1.20.6下面的,都是依照下面实例搭建的---除了我新建了一个namespace traefik,并将traefik利用都装置在了这个命名空间内!这样做的起因是tke的kebe-system下的pod太多了!我有强迫症)

部署与剖析过程:

1. 对于我的利用:

利用的部署形式是statefulset,如下:

cat <<EOF >  xxx-gateway.yamlapiVersion: apps/v1kind: StatefulSetmetadata:  name: xxx-gatewayspec:  serviceName: xxx-gateway  replicas: 1  selector:    matchLabels:      app: xxx-gateway  template:    metadata:      labels:        app: xxx-gateway    spec:      containers:        - name: xxx-gateway          image: ccr.ccs.tencentyun.com/xxx-master/xxx-gateway:202107151002          env:          - name: SPRING_PROFILES_ACTIVE            value: "official"          - name: SPRING_APPLICATION_JSON            valueFrom:             configMapKeyRef:              name: spring-config              key: dev-config.json          ports:            - containerPort: 8443          resources:            requests:              memory: "512M"              cpu: "500m"            limits:              memory: "512M"              cpu: "500m"       imagePullSecrets:                                                      - name: tencent---apiVersion: v1kind: Servicemetadata:  name: xxx-gateway  labels:    app: xxx-gatewayspec:  ports:  - port: 8443  selector:    app: xxx-gateway  clusterIP: NoneEOF
kubectl apply -f xxx-gateway.yaml -n official

偷个懒间接copy了一个其余利用的 ingress yaml批改了一下,如下:

cat <<EOF >  gateway-0-ingress.yamlapiVersion: networking.k8s.io/v1kind: Ingressmetadata:  name: layaverse-gateway-0-http  namespace: official  annotations:    kubernetes.io/ingress.class: traefik      traefik.ingress.kubernetes.io/router.entrypoints: webspec:  rules:  - host: xxx-gateway-0.xxx.com    http:      paths:      - pathType: Prefix        path: /        backend:          service:            name: xxx-gateway             port:               number: 8443EOF

部署ingress

kubectl apply -f gateway-0-ingress.yaml

查看ingress部署情况

kubectl get ingress -n official


嗯 而后测试一下wss(wss我间接用443端口了。证书挂载slb层的--这是我了解的!具体的参照我traefik的配置),这里强调一下wscat这个工具。反正看了下咱们的后端小伙伴测试ws利用都是用的在线的ws工具:

就这样的。而后刚巧看到wscat就装置了一下:

sudo apt install npmsudo npm install -g wscat wscat -c wss://xxx-gateway-0.xxx.com:443/ws

嗯哼 基本上能够确认是利用对外胜利了?

当然了以上只是我顺利的假想!
实际上是代理后连贯后端的ws服务仍然有各种问题(开始我狐疑是traefik的问题),还是连不上!我粗犷的把xxx-gateway 的裸露形式批改为NodePort 而后挂载到了slb层(在scl间接增加了ssl证书),测试了一下是能够的就间接用了。先让利用跑起来,而后再钻研一下怎么解决。

2. 对于ws和http:

先不去管那么多,先整明确实现我的traefik如何实现代理ws呢?

图中内容摘自:https://blog.csdn.net/fmm_sunshine/article/details/77918477

3. 排查下是谁的锅

1. 搭建一个简略的ws利用

后端的代码既然搞不懂,那我就找一个简略的ws的服务而后用traefik代理测试一下!
dockerhub搜得一个nodejs 的websocket镜像:https://hub.docker.com/r/ksdn117/web-socket-test
部署一下:

cat <<EOF >  web-socket-test.yamlapiVersion: apps/v1kind: StatefulSetmetadata:  name: web-socket-testspec:  serviceName: web-socket-test  replicas: 1  selector:    matchLabels:      app: web-socket-test  template:    metadata:      labels:        app: web-socket-test    spec:      containers:        - name: web-socket-test          image: ksdn117/web-socket-test          ports:            - containerPort: 8010              name: web            - containerPort: 8443              name: ssl          resources:            requests:              memory: "512M"              cpu: "500m"            limits:              memory: "512M"              cpu: "500m"---apiVersion: v1kind: Servicemetadata:  name: web-socket-test  labels:    app: web-socket-testspec:  type: NodePort  ports:  - port: 8010    targetPort: 8010    protocol: TCP    name: web  - port: 8443    targetPort: 8443    name: ssl    protocol: TCP  selector:    app: web-socket-testEOF

注: 我这里的配置文件加了type:NodePort

kubectl  apply -f web-socket-test.yamlkubectl get pods kubectl get svc 

2.外部wscat测试ws服务是否联通

先外部连贯container pod ip测试一下服务:

wscat --connect ws://172.22.0.230:8010
kubectl logs -f web-socket-test-0

3.traefik对外代理ws利用并测试

traefik失常的对外裸露服务能够用ingress的形式还有ingressroute我都去尝试一下:

1. ingressroute形式

cat <<EOF >  web-socket-ingressroute.yamlapiVersion: traefik.containo.us/v1alpha1kind: IngressRoutemetadata:  name: web-socket-test-http  namespaces: defaultspec:  entryPoints:  - web  routes:  - match: Host(`web-socket-test.xxx.com`)    kind: Rule    services:      - name: web-socket-test        port: 8010 EOF kubectl apply -f web-socket-ingressroute.yaml


wscat连贯测试一下:

这样测来是没有问题的?
删除ingress

 kubectl delete -f web-socket-ingressroute.yaml

2. ingress形式

整一下ingress的形式:

cat <<EOF >  web-socket-ingress.yamlapiVersion: networking.k8s.io/v1kind: Ingressmetadata:  name: web-socket-test  namespace: default  annotations:    kubernetes.io/ingress.class: traefik      traefik.ingress.kubernetes.io/router.entrypoints: webspec:  rules:  - host: web-socket-test.layame.com    http:      paths:      - pathType: Prefix        path: /        backend:          service:            name: web-socket-test             port:               number: 8010 EOF kubectl apply -f web-socket-ingress.yaml

wscat --connect wss://web-socket-test.xxx.com:443


甩锅根本实现起码不是我的基础设施应的问题.....让后端小伙伴测试一下看下是哪里有问题了。从我代理层来说是没有问题的!

对于其余:

当然了看一些博客还有要加passHostHeader: true的配置的

1. ingressroute:

2. ingress

ingress:traefik.ingress.kubernetes.io/service.passhostheader: "true"

如果有问题 能够尝试一下下面的形式!