关于kubernetes:kubernetes-的TCP-数据包可视化

kubernetes 的TCP 数据包可视化

介绍

k8spacket是用 Golang 编写的工具，它应用gopacket第三方库来嗅探工作负载（传入和传出）上的 TCP 数据包。它在运行的容器网络接口上创立 TCP 侦听器。当 Kubernetes 创立一个新容器时，CNI 插件负责提供与其余容器进行通信的可能性。最常见的办法是用linux namespace隔离网络并用veth pair连贯隔离的 namespace 与网桥。除了bridge 类型，CNI 插件还能够应用其余类型（vlan, ipvlan,macvlan)，但都为容器创立了一个网络接口，它是k8spacket嗅探器的次要句柄。

k8spacket有助于理解 Kubernetes 集群中的 TCP 数据包流量：

显示集群中工作负载之间的流量
告诉流量在集群外路由到哪里
显示无关连贯敞开套接字的信息
显示工作负载发送/接管的字节数
计算建设连贯的工夫
显示整个集群中工作负载之间的网络连接拓扑

k8spacket是一个 Kubernetes API 客户端，能够将嗅探到的工作负载解析为可视化上可见的集群资源名称（Pods和Services）。它作为DaemonSet Pod启动，应用 hostNetwork，并监听节点上的网络接口。

k8spacket 收集 TCP 流、解决数据，应用 Node Graph API Grafana 数据源插件（详情请查看 Node Graph API 插件），通过 API 展现在Grafana面板。

要装置k8spacket，须要同时装置 Grafana。上面将在Kind装置的 k8s 集群上做演示。

增加 k8spacket 的helm源

[[email protected] ~]# helm repo add k8spacket https://k8spacket.github.io/k8spacket-helm-chart"k8spacket" has been added to your repositories[[email protected] ~]#[[email protected] ~]#[[email protected] ~]#[[email protected] ~]# helm install k8spacket --namespace k8spacket k8spacket/k8spacket --create-namespaceNAME: k8spacketLAST DEPLOYED: Thu Oct 27 18:48:30 2022NAMESPACE: k8spacketSTATUS: deployedREVISION: 1TEST SUITE: NoneNOTES:1. Get the application URL by running these commands:  export NODE_PORT=$(kubectl get --namespace k8spacket -o jsonpath="{.spec.ports[0].nodePort}" services k8spacket)  export NODE_IP=$(kubectl get nodes --namespace k8spacket -o jsonpath="{.items[0].status.addresses[0].address}")  echo http://$NODE_IP:$NODE_PORT[[email protected] ~]#

查看 pod 信息

[email protected]:~# kubectl  get pod -n k8spacketNAME              READY   STATUS             RESTARTS      AGEk8spacket-46587   0/1     CrashLoopBackOff   2 (23s ago)   2m24sk8spacket-9wb5q   0/1     CrashLoopBackOff   1 (6s ago)    2m24sk8spacket-grh7k   0/1     ImagePullBackOff   0             2m24sk8spacket-hcgg4   0/1     CrashLoopBackOff   1 (4s ago)    2m24sk8spacket-ng99p   0/1     CrashLoopBackOff   1 (3s ago)    2m24sk8spacket-p7hgb   0/1     CrashLoopBackOff   1 (4s ago)    2m24sk8spacket-pk4zt   0/1     CrashLoopBackOff   1 (4s ago)    2m24sk8spacket-tcksl   0/1     CrashLoopBackOff   1 (6s ago)    2m24sk8spacket-tkzcc   0/1     CrashLoopBackOff   1 (8s ago)    2m24sk8spacket-w8r5r   0/1     CrashLoopBackOff   3 (11s ago)   2m24s[email protected]:~#

查看报错为 tunl0 问题

[[email protected] ~]# kubectl logs -n k8spacket k8spacket-465872022/10/27 13:35:36 Serving requests on port 66762022/10/27 13:35:36 Refreshing interfaces for capturing...2022/10/27 13:35:36 Starting capture on interface "cilium_host"2022/10/27 13:35:36 Starting capture on interface "tunl0"2022/10/27 13:35:36 Starting capture on interface "lxc_health"2022/10/27 13:35:36 Starting capture on interface "cilium_net"2022/10/27 13:35:36 Starting capture on interface "lxcaaf84592af2d"2022/10/27 13:35:36 Starting capture on interface "lxcc06519232b44"2022/10/27 13:35:36 reading in packets2022/10/27 13:35:36 reading in packets2022/10/27 13:35:36 error opening pcap handle: tunl0: That device is not up[[email protected] ~]#

批改配置

# 将 charts 包拉取到本地 在进行批改信息[[email protected] ~]# cd /tmp/[[email protected] tmp]# helm fetch k8spacket/k8spacket[[email protected] tmp]# tar -zxf k8spacket-0.1.3.tgz [[email protected] tmp]# cd k8spacket# 设置配置为  command: "ip address | grep @  | grep -v tunl0 | sed -E 's/.* (\\w+)@.*/\\1/' | tr '\\n' ',' | sed 's/.$//'"# 残缺配置如下[[email protected] k8spacket]# vim values.yaml[[email protected] k8spacket]# cat values.yamlreplicaCount: 1affinity: {}image:  repository: docker.io/k8spacket/k8spacket  pullPolicy: IfNotPresentserviceAccount:  create: true  # Annotations to add to the service account  annotations: {}clusterRole:  create: truenodeSelector: {}podAnnotations: {}priorityClassName: ""podSecurityContext:  runAsUser: 1000securityContext:  allowPrivilegeEscalation: true  capabilities:    add: [ "NET_ADMIN", "NET_RAW" ]service:  type: ClusterIP  port: 8080  nodePort:resources:  requests:    memory: "1000Mi"    cpu: "250m"  limits:    memory: "1500Mi"    cpu: "500m"tolerations: []k8sPacket:  metrics:    ## Hide source port when 'true' (set to string value 'dynamic' instead of decimal real source port) for Prometheus metrics cardinality reasons    hideSourcePort: true    reverseLookup:      ## Reverse lookup db file based on GeoLite2 Free Geolocation Data      ## See: https://dev.maxmind.com/geoip/geolite2-free-geolocation-data?lang=en      geoipDBPath: "/home/k8spacket/GeoLite2-City.mmdb"      ## Whois result match regexp      whoisRegexp: "(?:OrgName:|org-name:)\\s*(.*)"  tcp:    listener:      port: 6676      interfaces:        ## Command to achieve containers network interfaces        command: "ip address | grep @  | grep -v tunl0 | sed -E 's/.* (\\w+)@.*/\\1/' | tr '\\n' ',' | sed 's/.$//'"        ## How often refresh the list of network interfaces to listen        refreshPeriod: "10s"    assembler:      ## See: https://pkg.go.dev/github.com/google/gopacket/tcpassembly#AssemblerOptions      maxPagesPerConnection: 50      maxPagesTotal: 50      ## Every (periodDuration) seconds, flush connections that haven't seen activity in the past (closeOlderThanDuration) seconds.      flushing:        periodDuration: "10s"        closeOlderThanDuration: "20s"[[email protected] k8spacket]#

重新安装 k8spacket

[[email protected] k8spacket]# helm uninstall k8spacket -n k8spacket[[email protected] k8spacket]# helm install k8spacket --namespace k8spacket . --create-namespaceNAME: k8spacketLAST DEPLOYED: Thu Oct 27 21:47:38 2022NAMESPACE: k8spacketSTATUS: deployedREVISION: 1TEST SUITE: NoneNOTES:1. Get the application URL by running these commands:  export NODE_PORT=$(kubectl get --namespace k8spacket -o jsonpath="{.spec.ports[0].nodePort}" services k8spacket)  export NODE_IP=$(kubectl get nodes --namespace k8spacket -o jsonpath="{.items[0].status.addresses[0].address}")  echo http://$NODE_IP:$NODE_PORT[[email protected] k8spacket]#

查看验证

[[email protected] ~]# kubectl  get pod -n k8spacket  -o wideNAME              READY   STATUS    RESTARTS   AGE     IP             NODE           NOMINATED NODE   READINESS GATESk8spacket-8kxnx   1/1     Running   0          4m27s   192.168.1.66   k8s-node-3     <none>           <none>k8spacket-cqpks   1/1     Running   0          4m27s   192.168.1.70   k8s-node-6     <none>           <none>k8spacket-h72fc   1/1     Running   0          4m27s   192.168.1.67   k8s-node-4     <none>           <none>k8spacket-jkxg9   1/1     Running   0          4m27s   192.168.1.75   k8s-node-7     <none>           <none>k8spacket-kgpql   1/1     Running   0          4m27s   192.168.1.62   k8s-master-2   <none>           <none>k8spacket-lf9br   1/1     Running   0          4m27s   192.168.1.61   k8s-master-1   <none>           <none>k8spacket-mcbv5   1/1     Running   0          4m27s   192.168.1.68   k8s-node-5     <none>           <none>k8spacket-ndlzt   1/1     Running   0          4m27s   192.168.1.64   k8s-node-1     <none>           <none>k8spacket-vfg2x   1/1     Running   0          4m27s   192.168.1.63   k8s-master-3   <none>           <none>k8spacket-vvwtr   1/1     Running   0          4m27s   192.168.1.65   k8s-node-2     <none>           <none>[[email protected] ~]#[[email protected] ~]# kubectl  get svc -n k8spacket  -o wideNAME        TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)    AGE     SELECTORk8spacket   ClusterIP   10.110.30.53   <none>        8080/TCP   4m31s   app.kubernetes.io/instance=k8spacket,app.kubernetes.io/name=k8spacket[[email protected] ~]#拜访验证[[email protected] ~]# curl 10.110.30.53:8080/metrics

装置 dashboards 配置

[[email protected] ~]# cd /tmp/[[email protected] tmp]#[[email protected] tmp]# wget https://github.com/k8spacket/k8spacket/archive/refs/heads/master.zip[[email protected] tmp]# unzip master.zip[[email protected] tmp]#[[email protected] tmp]# cd k8spacket-master[[email protected] k8spacket-master]# [[email protected] k8spacket-master]# kubectl apply -f ./dashboards/configmap/k8spacket-logs-dashboard createdconfigmap/k8spacket-metrics-dashboard createdconfigmap/k8spacket-node-graph-dashboard created[[email protected] k8spacket-master]#

装置 Grafana

[[email protected] tmp]# helm repo add grafana https://grafana.github.io/helm-charts"grafana" has been added to your repositories[[email protected] tmp]# helm fetch grafana/grafana[[email protected] tmp]#[[email protected] tmp]# tar -zxf grafana-6.43.1.tgz

批改Grafana配置内容

[[email protected] tmp]# cd grafana/[[email protected] grafana]#[[email protected] grafana]# vim values.yaml批改以下配置内容persistence:  type: pvc  enabled: true env:  GF_INSTALL_PLUGINS: hamedkarbasi93-nodegraphapi-datasource dashboardProviders:  dashboardproviders.yaml:    apiVersion: 1    providers:    - name: 'default'      orgId: 1      folder: ''      type: file      disableDeletion: false      editable: true      options:        path: /var/lib/grafana/dashboards/default dashboardsConfigMaps:  default: k8spacket-node-graph-dashboard datasources:  nodegraphapi-plugin-datasource.yaml:      apiVersion: 1      datasources:      - name: "Node Graph API"        jsonData:          url: "http://k8spacket.k8spacket.svc.cluster.local:8080"        access: "proxy"        basicAuth: false        isDefault: false        readOnly: false        type: "hamedkarbasi93-nodegraphapi-datasource"        typeLogoUrl: "public/plugins/hamedkarbasi93-nodegraphapi-datasource/img/logo.svg"        typeName: "node-graph-plugin"        orgId: 1        version: 1

装置Grafana

[[email protected] grafana]# helm install grafana -f values.yaml  ./NAME: grafanaLAST DEPLOYED: Thu Oct 27 22:11:27 2022NAMESPACE: defaultSTATUS: deployedREVISION: 1NOTES:1. Get your 'admin' user password by running:   kubectl get secret --namespace default grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo2. The Grafana server can be accessed via port 80 on the following DNS name from within your cluster:   grafana.default.svc.cluster.local   Get the Grafana URL to visit by running these commands in the same shell:     export POD_NAME=$(kubectl get pods --namespace default -l "app.kubernetes.io/name=grafana,app.kubernetes.io/instance=grafana" -o jsonpath="{.items[0].metadata.name}")     kubectl --namespace default port-forward $POD_NAME 30003. Login with the password from step 1 and the username: admin[[email protected] grafana]#

批改为NodePort

[[email protected] grafana]# kubectl  get svcNAME                   TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGEecho-a                 ClusterIP   10.108.160.226   <none>        8080/TCP         6d9hecho-b                 NodePort    10.108.200.169   <none>        8080:31414/TCP   6d9hecho-b-headless        ClusterIP   None             <none>        8080/TCP         6d9hecho-b-host-headless   ClusterIP   None             <none>        <none>           6d9hgrafana                ClusterIP   10.101.109.183   <none>        80/TCP           4mkubernetes             ClusterIP   10.96.0.1        <none>        443/TCP          6d9h[[email protected] grafana]#[[email protected] grafana]# kubectl  edit svc grafanaservice/grafana edited[[email protected] grafana]#[[email protected] grafana]# kubectl  get svcNAME                   TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGEecho-a                 ClusterIP   10.108.160.226   <none>        8080/TCP         6d9hecho-b                 NodePort    10.108.200.169   <none>        8080:31414/TCP   6d9hecho-b-headless        ClusterIP   None             <none>        8080/TCP         6d9hecho-b-host-headless   ClusterIP   None             <none>        <none>           6d9hgrafana                NodePort    10.101.109.183   <none>        80:30973/TCP     4m37skubernetes             ClusterIP   10.96.0.1        <none>        443/TCP          6d9h[[email protected] grafana]#

查看Grafana明码

[[email protected] grafana]# kubectl get secret --namespace default grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo9O1Hd9LOqJ6LKUjZTlEWAGeXRitr0CZd4p6fr00J[[email protected] grafana]#

拜访地址

拜访http://192.168.1.61:30973/增加 Node Graph API 插件http://192.168.1.61:30973/plugins查看 Node Graph API 数据收集源http://192.168.1.61:30973/datasources

对于
https://www.oiox.cn/
https://www.oiox.cn/index.php...
CSDN、GitHub、51CTO、知乎、开源中国、思否、掘金、简书、华为云、阿里云、腾讯云、哔哩哔哩、今日头条、新浪微博、集体博客
全网可搜《小陈运维》
文章次要公布于微信公众号