概述

通常咱们要 silence 某个 AlertManager 的 alert 时，须要通过 UI 界面操作，如下图：

效率有点低，而且不够自动化，那么是否能够有一种方法疾速创立 AlertManager silence 呢？

-- 有的，通过 API.

API Payload

v1

如下：

{    "matchers": [        {            "name": "alername1",            "value": ".*",            "isRegex": true        }    ],    "startsAt": "2022-04-29T22:12:33.533Z",    "endsAt": "2022-04-29T23:11:44.603Z",    "createdBy": "api",    "comment": "Silence",    "status": {        "state": "active"    }}

v2

{    "matchers": [        {            "name": "service",            "value": "rancher",            "isRegex": false,            "isEqual": true        },        {            "name": "alertname",            "value": "TargetDown",            "isRegex": false,            "isEqual": true        }    ],    "startsAt": "2022-04-29T10:11:35.656Z",    "endsAt": "2022-04-29T12:11:35.656Z",    "createdBy": "Casey Cui",    "comment": "配置谬误导致的误报",    "id": null}

具体实现

curl 实现

Notes:

以 v1 api 为例

如下：

curl https://alertmanager.ewhisper.cn/api/v1/silences -d '{      "matchers": [        {          "name": "alername1",          "value": ".*",          "isRegex": true        }      ],      "startsAt": "2022-04-29T22:12:33.533Z",      "endsAt": "2022-04-29T23:11:44.603Z",      "createdBy": "api",      "comment": "Silence",      "status": {        "state": "active"      }}'
Notes:
在 K8S 集群内，地址个别为：http://alertmanager:9093

python 实现

如下：

#!/usr/bin/python3import requestsimport socketimport datetimeimport timeres = requests.post("http://alertmanager:9093/api/v2/silences", json={    "matchers": [        {"name": "job", "value": "myjob", "isRegex": False},        {"name": "instance", "value": "{}:1234".format(socket.gethostname()), "isRegex": False},        ],    "startsAt": datetime.datetime.utcfromtimestamp(time.time()).isoformat(),    "endsAt": datetime.datetime.utcfromtimestamp(time.time() + 4*3600).isoformat(),    "comment": "Backups on {}".format(socket.gethostname()),    "createdBy": "My backup script",    },    )res.raise_for_status()silenceId = res.json()["silenceID"]

移除 silence 的脚本：

res = requests.delete("http://alertmanager:9093/api/v2/silence/{}".format(silenceId))res.raise_for_status()

EOF

本文由东风微鸣技术博客 EWhisper.cn 编写！