背景和痛点——日志的要求:输入格局对立

loguru 是如许的优良就不必我介绍了,咱们本人的业务代码,能够轻松的 import loguru 来打印日志,纳闷的一个对立的输入

然而很多第三方库集成的日志模块是规范库中的 logging,其 format 多种多样

我心愿能够实现:应用 loguru 接管所有库的 logging,对立应用 loguru 的格局输入

起因?因为不便日志采集和解析存储呀!比方对接 lokislselk 等等。

最佳实际——我现实的日志输入格局:

我心愿每条日志输入都长这样

{    "text": "2022-04-25 18:12:40.179 | ERROR    | __main__:<module>:18 - An error has been caught in function '<module>', process 'MainProcess' (60788), thread 'MainThread' (139849590506112):\nTraceback (most recent call last):\n\n> File \"/home/bot/Desktop/ideaboom/test_logger/loguru_contextualize.py\", line 18, in <module>\n    div(1,0)\n    └ <function div at 0x7f3143edf490>\n\n  File \"/home/bot/Desktop/ideaboom/test_logger/loguru_contextualize.py\", line 8, in div\n    return a/b\n           │ └ 0\n           └ 1\n\nZeroDivisionError: division by zero\n",    "record": {        "elapsed": { "repr": "0:00:00.007519", "seconds": 0.007519 },        "exception": {            "type": "ZeroDivisionError",            "value": "division by zero",            "traceback": true        },        "extra": {            "context_id": "fd77b1d159224d939c1cc0d7a566d09b",            "message_id": "8ad2c351ef3240998fc10a5015c591c1"        },        "file": {            "name": "loguru_contextualize.py",            "path": "/home/bot/Desktop/ideaboom/test_logger/loguru_contextualize.py"        },        "function": "<module>",        "level": { "icon": "❌", "name": "DEUBG", "no": 40 },        "line": 18,        "message": "An error has been caught in function '<module>', process 'MainProcess' (60788), thread 'MainThread' (139849590506112):",        "module": "loguru_contextualize",        "name": "__main__",        "process": { "id": 60788, "name": "MainProcess" },        "thread": { "id": 139849590506112, "name": "MainThread" },        "time": {            "repr": "2022-04-25 18:12:40.179070+08:00",            "timestamp": 1650881560.17907        }    }}

对应的代码就是这样:

from loguru import loggerfrom mark import BASE_DIRimport logginglogger.add(BASE_DIR/'run.log', serialize='json')

我心愿 loguru 的日志输入到文件,而后日志收集程序收集日志文件。而不是规范输入,我心愿的规范输入是给人类看的,不是给日志收集程序的。

也就是说日志输入两份:

  • 一个输入到 stdoutstderror,能够通过 taildocker-compose logskubectl logs 等等命令间接查看,给人类看(格局不须要对立,反正不对立人类也看的懂,并且这个就是简略的 text,不是 josn 之类的);
  • 另一份输入到 run.log 文件,logtailpromtail 等等日志收集程序就收集 run.log 中的日志记录。(领有对立的日志格局,并且一行就是一个 json)

以 json 输入日志的长处:

  • 不便采集,不存在因为格局问题导致的采集谬误!也不必焦虑堆栈换行而须要应用行首正则匹配表达式等等
  • 不便解析,对接 esmongodb 很不便

毛病呢?

  • 体积收缩,json 相比间接的 text,体积大了 n 倍

代码实现—— loguru 偷天换日,滥竽充数

建设一个 loggers.py 文件,内容如下:

from loguru import loggerfrom mark import BASE_DIRimport logginglogger.add(BASE_DIR/'run.log', serialize='json')class InterceptHandler(logging.Handler):    def emit(self, record):        # Retrieve context where the logging call occurred, this happens to be in the 6th frame upward        logger_opt = logger.opt(depth=6, exception=record.exc_info)        logger_opt.log(record.levelno, record.getMessage())# logging.getLogger("uvicorn").setLevel(10)# logging.getLogger("uvicorn").handlers = []# logging.getLogger("uvicorn").addHandler(InterceptHandler())# logging.getLogger("uvicorn.error").setLevel(10)# logging.getLogger("uvicorn.error").handlers = []# logging.getLogger("uvicorn.error").addHandler(InterceptHandler())# logging.getLogger("fawn.error").setLevel(10)# logging.getLogger("fawn.error").handlers = []# logging.getLogger("fawn.error").addHandler(InterceptHandler())logger_name_list = [name for name in logging.root.manager.loggerDict]# logger_name_list = [name for name in logging.root.manager.loggerDict if '.' not in name]for logger_name in logger_name_list:    logging.getLogger(logger_name).setLevel(10)    logging.getLogger(logger_name).handlers = []    if '.' not in logger_name:        logging.getLogger(logger_name).addHandler(InterceptHandler())print(logger_name_list)print(logging.root.manager.loggerDict)

事实中,比方 uvicornpeewee 都有本人的输入格局,如下:

─➤  python api.pyINFO:     Started server process [2382519]INFO:     Waiting for application startup.INFO:     Application startup complete.INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)

应用下面的代码之后,就变成了:

─➤  uvicorn api:app                                                                                                   1 ↵2022-04-30 23:09:11.283 | INFO     | uvicorn.server:serve:75 - Started server process [2395776]2022-04-30 23:09:11.283 | INFO     | uvicorn.lifespan.on:startup:45 - Waiting for application startup.2022-04-30 23:09:11.283 | INFO     | uvicorn.lifespan.on:startup:59 - Application startup complete.2022-04-30 23:09:11.284 | INFO     | uvicorn.server:_log_started_message:206 - Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)

解释一下代码

下面这段代码你晓得是什么意思吗?

todo

参考文章:
过滤器对象
how can i logging peewee with loguru
How To Override Uvicorn Logger in FastAPI using Loguru