Python Logging 日志输出到文件

Posted:   May 03, 2020

Edited:   May 03, 2020

Status:   Writing

Tags :   Python

Categories :   Python

Were equations, pictures or diagrams not properly rendered, please refresh the page. If the problem persists, you can contact me.

背景

了解如何使用 python logging, 输出日志到文件中.

并初步结合 flask 框架以及多模块的环境配置, 尚不太成熟, 待改进

至于为什么要用 logging, 而不是之前的 print, 原因也简单, logging 的信息量更大, 如果输出到文件中也更方便.

Python Logging 碎碎念之配置文件

python 有自带的 logging 模块, 足够用了, 也挺好用.

用前先想想:

  • 要能兼顾所有模块
  • 要有时间戳
  • 要方便配置, 比如更改 log文件名, log level 等
    • 文件名不写死在代码中…

自然需要外置 conf 配置文件.

当然, 外置 conf 文件虽然配置起来方便, 但要在代码中操控它就有点麻烦, 不过从用户角度看, 这个还是首选. 具体的 pro 和 cons 请参考Logging — The Hitchhiker’s Guide to Python

那:

  • conf 文件格式有什么要求?
  • conf 文件内容怎么写?
  • conf 文件怎么读?

文件格式一般有几种:

  • ini 文件: 类似 key=value
  • yml 文件
  • json 文件

对他们的解析不一样, 读取的时候用到的模块:

  • ini 文件: logging.config.fileConfg
  • yml/json 文件: logging.config.dictConfig

示例见下节

文件内容怎么写?

一般比较重要的得包含:

  • level: debug/info/warn…
  • handlers
    • file handler
      • file name: 比如带时间戳等
    • console handler
  • formatters
    • file formatter
    • console formatter

简单的示例如下, 详细示例请见文末的附录

[logger_root]
level=DEBUG
handlers=fileHandler

[handler_fileHandler]
class=FileHandler
level=DEBUG
formatter=logFormatter
args=("_log/" + time.strftime("%%Y%%m%%d%%H%%M%%S")+'.log', 'a')

[formatter_logFormatter]
format=%(asctime)s | %(levelname)-8s | %(lineno)04d | %(message)s

在代码中加载 log 配置文件

加载 yml/json 文件:

import logging
import logging.config.dictConfig(yaml.load(open('logging.yml', Loader=yaml.FullLoader))))

加载 .ini 文件

import logging
logging.config.fileConfig("logging.ini")

在模块中使用就很简单了,

logger = logging.getLogger(__name__)
logger.debug("log %s", it)

当然这只是很简单的场景, 实际应用中会需要用到 filter, 更改文件名, 获取 log 文件名等等

比如, 我想在程序运行结束的时候把 log 文件地址也输出到 log 文件中, 然而却发现有点难度, 试了几种方法, 只成功了下面这个:

LOG_FILE = logging.getLoggerClass().root.handlers[0].baseFilename
logger = logging.getLogger(__name__)
logger.debug(f"LOG_FILE: , {LOG_FILE}")

未成功之一: 我拿到的 handlers list 总是为空…

import logging
dir(logging.FileHandler)
>>> handler = job_logger.handlers[0]
>>> filename = handler.baseFilename
>>> print(filename)
'/tmp/test_logging_file'

还想输出 exception details

先在需要处理 exception 的地方 raise exception

raise Exception(msg)

然后, catch 的时候输出到 log 中

import traceback
except Exception as e:
    msg = traceback.from_exec()
    logger.debug(msg)

注意, 这里的 form_exec() 是不带参数的

我之前忽略了, 想当然的加了 e 作为参数, 结果错误: '>=' not supported between instances of 'Exception' and 'int'

Flask Logging

如果代码中用到了 flask 框架, 如何输出 log 呢?

同样, 重用前面的 log 配置, 在 app.py 中直接用 app.logger 即可, 底层用的还是 python logging 模块

app = create_app()
logger = app.logger

Reference

Python logging

flask logging

Logging — Flask Documentation (1.1.x)

附录: 大致的目录结构

log 配置文件的读取是在 module2/init.py 中;

import os
import logging
from logging.config import fileConfig

if not os.path.exists("_log"):
    os.mkdir("_log")
logging.config.fileConfig("logging.conf"))
LOG_FILE = logging.getLoggerClass().root.handlers[0].baseFilename
logger = logging.getLogger(__name__)
logger.debug(f"LOG_FILE: , {LOG_FILE}")

main.py 启动 flask app

.root
├── logging.conf
├── logging.yml
├── main.py
├── README.md
├── _log
            └── 20200503204102.log
├── module1
            ├── __init__.py
            ├── requirements.txt
            ├── src
            │   ├── __init__.py
            │   └── app.py
            └── test
└── module2
            ├── __init__.py
            ├── src
                        ├── __init__.py
            └── test
                        ├── __init__.py
            ├── requirements.txt
            ├── README.md
            ├── data
            ├── docs
            ├── invoke.yml

附录: logging.ini sample (摘自网络),

因为带了注释, 所以摘录在此供参考:

如需 yml 文件, 可参考python - Flask logging - Cannot get it to write to a file - Stack Overflow

#These are the loggers that are available from the code
#Each logger requires a handler, but can have more than one
[loggers]
keys=root,Admin_Client

#Each handler requires a single formatter
[handlers]
keys=fileHandler, consoleHandler

[formatters]
keys=logFormatter, consoleFormatter

[logger_root]
level=DEBUG
handlers=fileHandler

[logger_Admin_Client]
level=DEBUG
handlers=fileHandler, consoleHandler
qualname=Admin_Client
#propagate=0 Does not pass messages to ancestor loggers(root)
propagate=0

# Do not use a console logger when running scripts from a bat file without a console
# because it hangs!
[handler_consoleHandler]
class=StreamHandler
level=DEBUG
formatter=consoleFormatter
args=(sys.stdout,)# The comma is correct, because the parser is looking for args

[handler_fileHandler]
class=FileHandler
level=DEBUG
formatter=logFormatter
# This causes a new file to be created for each script
# Change time.strftime("%Y%m%d%H%M%S") to time.strftime("%Y%m%d")
# And only one log per day will be created. All messages will be amended to it.
args=("_log/" + time.strftime("%%Y%%m%%d%%H%%M%%S")+'.log', 'a')

[formatter_logFormatter]
#name is the name of the logger root or Admin_Client
#levelname is the log message level debug, warn, ect 
#lineno is the line number from where the call to log is made
#04d is simple formatting to ensure there are four numeric places with leading zeros
#4s would work as well, but would simply pad the string with leading spaces, right justify
#-4s would work as well, but would simply pad the string with trailing spaces, left justify
#filename is the file name from where the call to log is made
#funcName is the method name from where the call to log is made
#format=%(asctime)s | %(lineno)d | %(message)s
#format=%(asctime)s | %(name)s | %(levelname)s | %(message)s
#format=%(asctime)s | %(name)s | %(module)s-%(lineno) | %(levelname)s | %(message)s
#format=%(asctime)s | %(name)s | %(module)s-%(lineno)04d | %(levelname)s | %(message)s
#format=%(asctime)s | %(name)s | %(module)s-%(lineno)4s | %(levelname)-8s | %(message)s
format=%(asctime)s | %(levelname)-8s | %(lineno)04d | %(message)s

#Use a separate formatter for the console if you want
[formatter_consoleFormatter]
format=%(asctime)s | %(levelname)-8s | %(filename)s-%(funcName)s-%(lineno)04d | %(message)s

ChangeLog

  • 2020-05-03 init

Comments


Be the first one to comment on this page!
You can use extended GitHub flavored markdown in your comment. Commenting FAQs & Guidelines