乐趣区

关于python:python-kubernetes-获取-pod-的-cpu-占用率

jiezi

这个教程是应用 kubernetes 的 python client sdk 获取 pod 的 cpu 占用率，而不是通过 kubectl 命令！

kubernetes python client sdk

动机

我要做什么？

最近有一个 pod，是 rabbitmq 的消费者，然而会呈现频繁卡死的状况，所以我须要判断 pod 是不是卡死了，而后重启。

这个判断没有方法通过个别的健康检查发现

判断根据：CPU 应用配额低于 20m 就认定为卡死，就删除 pod（删除 pod 之后，k8s 会从新一个新的）

技术计划

计划一：应用 shell+kubectl。然而我不喜爱 shell，也不喜爱解析非结构化的输入，所以这个计划就淘汰了

计划二：应用 python + kubernetes sdk。我喜爱 python，而且这样能够输入结构化的数据结构，比方 json，不便我解析，good

所以，我才用计划二！

获取一个『命名空间』下的所有 pod

首先，咱们要列出一个 namespace 上面所有的 pod

相似 kubectl get pod -n vddb

vddb 是 namespace 的 name

from kubernetes.client.models.v1_pod import V1Pod
from kubernetes.client.models.v1_pod_list import V1PodList
from kubernetes.client.models.v1_object_meta import V1ObjectMeta
from kubernetes import client, config

from kubernetes.client import ApiClient
from kubernetes.client.rest import RESTResponse
from loguru import logger

config.load_kube_config()

v1 = client.CoreV1Api()


namespaced_name = 'vddb'


pod_list: V1PodList = v1.list_namespaced_pod(namespaced_name)

for pod in pod_list.items:
    pod: V1Pod
    metadata: V1ObjectMeta = pod.metadata
    pod_name = metadata.name

获取一个 pod 的 metrics

列出了 pod name 之后，咱们就是获取 pod 的对应的 metrics，比方应用的 CPU、内存配额 

import json
from kubernetes.client.models.v1_pod import V1Pod
from kubernetes.client.models.v1_pod_list import V1PodList
from kubernetes.client.models.v1_object_meta import V1ObjectMeta
from kubernetes import client, config

from kubernetes.client import ApiClient
from kubernetes.client.rest import RESTResponse
from loguru import logger

config.load_kube_config()

v1 = client.CoreV1Api()
api_client = ApiClient()

namespaced_name = 'vddb'


pod_list: V1PodList = v1.list_namespaced_pod(namespaced_name)

for pod in pod_list.items:
    pod: V1Pod
    metadata: V1ObjectMeta = pod.metadata
    pod_name = metadata.name

    rest_response: RESTResponse = api_client.request(
        url=api_client.configuration.host +
        f'/apis/metrics.k8s.io/v1beta1/namespaces/{namespaced_name}/pods/{pod_name}',
        method='GET'
    )
    _data: str = rest_response.data
    data: dict = json.loads(_data)

    _cpu: str = data['containers'][0]['usage']['cpu']

    cpu = int(int(_cpu.removesuffix('n'))/1000/1000)

响应体的格局如下所示：

{
  "kind": "PodMetrics",
  "apiVersion": "metrics.k8s.io/v1beta1",
  "metadata": {
    "name": "svddb-servixxxxxxxxxxxxxxx4b4-bzs84",
    "namespace": "vddb",
    "selfLink": "/apis/metrics.k8s.io/v1beta1/namespaces/vddb/pods/svddbxxxxxxxxxxxxxxxx-bzs84",
    "creationTimestamp": "2022-12-16T14:40:46Z"
  },
  "timestamp": "2022-12-16T14:40:09Z",
  "window": "30s",
  "containers": [
    {
      "name": "svdxxxxxxxxrators",
      "usage": {"cpu": "2575748239n", "memory": "1257180Ki"}
    }
  ]
}

留神，这里的 containers 是一个列表

删除 pod

import json
from kubernetes.client.models.v1_pod import V1Pod
from kubernetes.client.models.v1_pod_list import V1PodList
from kubernetes.client.models.v1_object_meta import V1ObjectMeta
from kubernetes import client, config

from kubernetes.client import ApiClient
from kubernetes.client.rest import RESTResponse
from loguru import logger

config.load_kube_config()

v1 = client.CoreV1Api()
api_client = ApiClient()

namespaced_name = 'vddb'


pod_list: V1PodList = v1.list_namespaced_pod(namespaced_name)

for pod in pod_list.items:
    pod: V1Pod
    metadata: V1ObjectMeta = pod.metadata
    pod_name = metadata.name

    rest_response: RESTResponse = api_client.request(
        url=api_client.configuration.host +
        f'/apis/metrics.k8s.io/v1beta1/namespaces/{namespaced_name}/pods/{pod_name}',
        method='GET'
    )
    _data: str = rest_response.data
    data: dict = json.loads(_data)

    _cpu: str = data['containers'][0]['usage']['cpu']

    cpu = int(int(_cpu.removesuffix('n'))/1000/1000)

    if 'svddb-service-generators-server-prod' in pod_name and cpu < 20:
        v1.delete_namespaced_pod(pod_name, namespaced_name)

参考教程：
Get cpu and memory usage through in cluster config
Does the library support “kubectl top pod” api?
https://kubernetes.io/docs/tasks/debug/debug-cluster/resource-metrics-pipeline/

退出移动版