关于kubernetes:kubeeventer事件监控

文章链接

下载 deployment

我这里保留成 kube-event.yaml

# cat kube-event.yaml
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    name: kube-eventer
  name: kube-eventer
  namespace: kube-system
spec:
  replicas: 1
  selector:
    matchLabels:
      app: kube-eventer
  template:
    metadata:
      labels:
        app: kube-eventer
      annotations:
        scheduler.alpha.kubernetes.io/critical-pod: ''
    spec:
      dnsPolicy: ClusterFirstWithHostNet
      serviceAccount: kube-eventer
      containers:
        - image: registry.aliyuncs.com/acs/kube-eventer-amd64:v1.2.0-484d9cd-aliyun
          name: kube-eventer
          command:
            - "/kube-eventer"
            - "--source=kubernetes:https://kubernetes.default"
            ## .e.g,dingtalk sink demo
            #- --sink=dingtalk:[your_webhook_url]&label=[your_cluster_id]&level=[Normal or Warning(default)]
            - --sink=dingtalk:https://oapi.dingtalk.com/robot/send?access_token=355cf0156xxxxxxxxxxxxxxxxxx&level=Warning
          env:
          # If TZ is assigned, set the TZ value as the time zone
          - name: TZ
            value: "Asia/Shanghai"
          volumeMounts:
            - name: localtime
              mountPath: /etc/localtime
              readOnly: true
            - name: zoneinfo
              mountPath: /usr/share/zoneinfo
              readOnly: true
          resources:
            requests:
              cpu: 100m
              memory: 100Mi
            limits:
              cpu: 500m
              memory: 250Mi
      volumes:
        - name: localtime
          hostPath:
            path: /etc/localtime
        - name: zoneinfo
          hostPath:
            path: /usr/share/zoneinfo
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: kube-eventer
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - events
    verbs:
      - get
      - list
      - watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: kube-eventer
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: kube-eventer
subjects:
  - kind: ServiceAccount
    name: kube-eventer
    namespace: kube-system
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: kube-eventer
  namespace: kube-system

钉钉群里创立自定义 webhook

设置 – 智能群助手 – 增加机器人 – 抉择 WeebHook。定义机器人名称和平安设置

平安设置这里我定义了关键字，Waring。创立后复制 webhook 地址。而后更改下面 deployment 中的 sink 处。

我把下面的 label 删掉了，只留下了 level=Waring，刚好对应了我关键字的 Waring。只有带有关键字的才会触发告警。

测试告警

而后创立一个测试的 Tomcat 的 deployment，成心把 image 镜像的 tag 写错，让他无奈拉取镜像

[root@master allenjol]# kubectl apply -f deploy-tomcat-test.yaml
deployment.apps/tomcat-deployment-allenjol created
service/tomcat-service-allenjol created

[root@master allenjol]# kubectl get po
NAME                                        READY   STATUS             RESTARTS   AGE
tomcat-deployment-allenjol-b6687f99-l5vj9   0/1     ImagePullBackOff   0          45s

部署 kube-event.yaml 并查看日志。能够看到隔 30s 去检测一次

]# kubectl apply -f kube-event.yaml
]# kubectl get po -n kube-system | grep kube-event


[root@master allenjol]# kubectl logs -f kube-eventer-648f64c985-zfkkg -n kube-system
I0708 09:26:36.409034       1 eventer.go:67] /kube-eventer --source=kubernetes:https://kubernetes.default --sink=dingtalk:https://oapi.dingtalk.com/robot/send?access_token=355cf01569aef206dc6c05681aaf3ed0ea19ed3597db4c26c565dbeb69ce1303&level=Warning
I0708 09:26:36.409191       1 eventer.go:68] kube-eventer version: v1.2.0 commit: 484d9cd
I0708 09:26:36.411557       1 eventer.go:94] Starting with DingTalkSink sink
I0708 09:26:36.411596       1 eventer.go:108] Starting eventer
I0708 09:26:36.411678       1 eventer.go:116] Starting eventer http service
I0708 09:27:00.000163       1 manager.go:102] Exporting 5 events
I0708 09:27:30.000130       1 manager.go:102] Exporting 9 events
I0708 09:28:00.000147       1 manager.go:102] Exporting 1 events
I0708 09:28:30.000150       1 manager.go:102] Exporting 4 events
I0708 09:29:00.000138       1 manager.go:102] Exporting 1 events
...

能够看到这里曾经看到了钉钉的 webhook 地址了，并且还收集到了 events。
查看钉钉群，就会看到曾经呈现了告警了。

其实这个告警以后还存在点问题。集体认为不应该这么频繁发送，应该像 prometheus 一样能够配置克制和静默。而后监控工夫能够更改。当然相熟 go 语言能够本人改源码而后构建成镜像。
文章链接