乐趣区

关于python:如何使用-loguru-接管程序的所有日志输出

jiezi

背景和痛点——日志的要求：输入格局对立

loguru 是如许的优良就不必我介绍了，咱们本人的业务代码，能够轻松的 import loguru 来打印日志，纳闷的一个对立的输入

然而很多第三方库集成的日志模块是规范库中的 logging，其 format 多种多样

我心愿能够实现：应用 loguru 接管所有库的 logging，对立应用 loguru 的格局输入

起因？因为不便日志采集和解析存储呀！比方对接 loki、sls、elk 等等。

最佳实际——我现实的日志输入格局：

我心愿每条日志输入都长这样 

{"text": "2022-04-25 18:12:40.179 | ERROR    | __main__:<module>:18 - An error has been caught in function'<module>', process'MainProcess'(60788), thread'MainThread'(139849590506112):\nTraceback (most recent call last):\n\n> File \"/home/bot/Desktop/ideaboom/test_logger/loguru_contextualize.py\", line 18, in <module>\n    div(1,0)\n    └ <function div at 0x7f3143edf490>\n\n  File \"/home/bot/Desktop/ideaboom/test_logger/loguru_contextualize.py\", line 8, in div\n    return a/b\n           │ └ 0\n           └ 1\n\nZeroDivisionError: division by zero\n",
    "record": {"elapsed": { "repr": "0:00:00.007519", "seconds": 0.007519},
        "exception": {
            "type": "ZeroDivisionError",
            "value": "division by zero",
            "traceback": true
        },
        "extra": {
            "context_id": "fd77b1d159224d939c1cc0d7a566d09b",
            "message_id": "8ad2c351ef3240998fc10a5015c591c1"
        },
        "file": {
            "name": "loguru_contextualize.py",
            "path": "/home/bot/Desktop/ideaboom/test_logger/loguru_contextualize.py"
        },
        "function": "<module>",
        "level": {"icon": "❌", "name": "DEUBG", "no": 40},
        "line": 18,
        "message": "An error has been caught in function'<module>', process'MainProcess'(60788), thread'MainThread'(139849590506112):",
        "module": "loguru_contextualize",
        "name": "__main__",
        "process": {"id": 60788, "name": "MainProcess"},
        "thread": {"id": 139849590506112, "name": "MainThread"},
        "time": {
            "repr": "2022-04-25 18:12:40.179070+08:00",
            "timestamp": 1650881560.17907
        }
    }
}

对应的代码就是这样：

from loguru import logger
from mark import BASE_DIR
import logging

logger.add(BASE_DIR/'run.log', serialize='json')

我心愿 loguru 的日志输入到文件，而后日志收集程序收集日志文件。而不是规范输入，我心愿的规范输入是给人类看的，不是给日志收集程序的。

也就是说日志输入两份：

  • 一个输入到 stdout、stderror，能够通过 tail、docker-compose logs、kubectl logs 等等命令间接查看，给人类看（格局不须要对立，反正不对立人类也看的懂，并且这个就是简略的 text，不是 josn 之类的）；
  • 另一份输入到 run.log 文件，logtail、promtail 等等日志收集程序就收集 run.log 中的日志记录。（领有对立的日志格局，并且一行就是一个 json）

以 json 输入日志的长处：

  • 不便采集，不存在因为格局问题导致的采集谬误！也不必焦虑堆栈换行而须要应用行首正则匹配表达式等等
  • 不便解析，对接 es、mongodb 很不便

毛病呢？

  • 体积收缩，json 相比间接的 text，体积大了 n 倍

代码实现—— loguru 偷天换日，滥竽充数

建设一个 loggers.py 文件，内容如下：

from loguru import logger
from mark import BASE_DIR
import logging

logger.add(BASE_DIR/'run.log', serialize='json')


class InterceptHandler(logging.Handler):
    def emit(self, record):
        # Retrieve context where the logging call occurred, this happens to be in the 6th frame upward
        logger_opt = logger.opt(depth=6, exception=record.exc_info)
        logger_opt.log(record.levelno, record.getMessage())


# logging.getLogger("uvicorn").setLevel(10)
# logging.getLogger("uvicorn").handlers = []
# logging.getLogger("uvicorn").addHandler(InterceptHandler())

# logging.getLogger("uvicorn.error").setLevel(10)
# logging.getLogger("uvicorn.error").handlers = []
# logging.getLogger("uvicorn.error").addHandler(InterceptHandler())

# logging.getLogger("fawn.error").setLevel(10)
# logging.getLogger("fawn.error").handlers = []
# logging.getLogger("fawn.error").addHandler(InterceptHandler())

logger_name_list = [name for name in logging.root.manager.loggerDict]
# logger_name_list = [name for name in logging.root.manager.loggerDict if '.' not in name]

for logger_name in logger_name_list:
    logging.getLogger(logger_name).setLevel(10)
    logging.getLogger(logger_name).handlers = []
    if '.' not in logger_name:
        logging.getLogger(logger_name).addHandler(InterceptHandler())


print(logger_name_list)
print(logging.root.manager.loggerDict)

事实中，比方 uvicorn、peewee 都有本人的输入格局，如下：

─➤  python api.py
INFO:     Started server process [2382519]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)

应用下面的代码之后，就变成了：

─➤  uvicorn api:app                                                                                                   1 ↵
2022-04-30 23:09:11.283 | INFO     | uvicorn.server:serve:75 - Started server process [2395776]
2022-04-30 23:09:11.283 | INFO     | uvicorn.lifespan.on:startup:45 - Waiting for application startup.
2022-04-30 23:09:11.283 | INFO     | uvicorn.lifespan.on:startup:59 - Application startup complete.
2022-04-30 23:09:11.284 | INFO     | uvicorn.server:_log_started_message:206 - Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)

解释一下代码

下面这段代码你晓得是什么意思吗？

todo

参考文章：
过滤器对象
how can i logging peewee with loguru
How To Override Uvicorn Logger in FastAPI using Loguru

退出移动版