Improve logger setup in devtool/recipetool/scriptutils
cover Improve logger setup in devtool/recipetool/scriptutils
From: Christopher Larson <[email protected]>
This improves logger setup in devtool and recipetool to send ERRORs to stderr, the way bitbake itself does, and also sends WARNINGs there, as those are essentially out of band as well. This makes it easier to script with devtool and recipetool, as any *real* subcommand output can be more easily captured without having to manually filter out the cruft.
The following changes since commit b11725db2d5549dc45d8ae36fbf94a5c8e342d69:
xserver-xorg: Fix build errors with clang (2019-03-04 14:26:41 +0000)
are available in the Git repository at:
[email protected]:kergoth/openembedded-core scriptutils-log-improvements
for you to fetch changes up to c4f37568236d2e3d8c40d57399d0b275ad5b7c8b:
oe.scriptutils: also send WARNING to stderr (2019-03-04 21:31:00 +0000)
----------------------------------------------------------------
Christopher Larson (4):
oe.scriptutils: enable color in a more flexible way oe.scriptutils: add logger_setup_filters recipetool, devtool: set up the logging filters oe.scriptutils: also send WARNING to stderr
scripts/devtool | 2 ++ scripts/lib/scriptutils.py | 30 ++++++++++++++++++++++++------ scripts/recipetool | 2 ++ 3 files changed, 28 insertions(+), 6 deletions(-)
Signature · 1 line
2.20.1
PATCH 1/4 oe.scriptutils: enable color in a more flexible way
From: Christopher Larson <[email protected]>
Rather than recreating handlers and forcing them, iterate over the handlers and enable color on ones we can handle. This makes it easier to handle color properly when we introduce the bb.msg default log filters.
--- scripts/lib/scriptutils.py | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-)
scripts/lib/scriptutils.py +7 -6
@@ -39,12 +39,12 @@ def logger_create(name, stream=None): def logger_setup_color(logger, color='auto'): from bb.msg import BBLogFormatter - console = logging.StreamHandler(sys.stdout) - formatter = BBLogFormatter("%(levelname)s: %(message)s") - console.setFormatter(formatter) - logger.handlers = [console] - if color == 'always' or (color=='auto' and console.stream.isatty()): - formatter.enable_color() + + for handler in logger.handlers: + if (isinstance(handler, logging.StreamHandler) and + isinstance(handler.formatter, BBLogFormatter)): + if color == 'always' or (color == 'auto' and handler.stream.isatty()): + handler.formatter.enable_color() def load_plugins(logger, plugins, pluginpath):
@@ -69,6 +69,7 @@ def load_plugins(logger, plugins, pluginpath): plugin.plugin_init(plugins) plugins.append(plugin) + def git_convert_standalone_clone(repodir): """If specified directory is a git repository, ensure it's a standalone clone""" import bb.process
Signature · 1 line
2.20.1
PATCH 2/4 oe.scriptutils: add logger_setup_filters
In reply to Christopher LarsonFrom: Christopher Larson <[email protected]>
This function sets up the default bb.msg formatter and log filters, which by default sends ERROR messages to stderr rather than stdout.
--- scripts/lib/scriptutils.py | 15 +++++++++++++++ 1 file changed, 15 insertions(+)
scripts/lib/scriptutils.py +15 -0
@@ -29,6 +29,7 @@ import tempfile import importlib from importlib import machinery + def logger_create(name, stream=None): logger = logging.getLogger(name) loggerhandler = logging.StreamHandler(stream=stream)
@@ -37,6 +38,20 @@ def logger_create(name, stream=None): logger.setLevel(logging.INFO) return logger + +def logger_setup_filters(logger): + import bb.msg + + console = logging.StreamHandler(sys.stdout) + errconsole = logging.StreamHandler(sys.stderr) + bb.msg.addDefaultlogFilter(console, bb.msg.BBLogFilterStdOut) + bb.msg.addDefaultlogFilter(errconsole, bb.msg.BBLogFilterStdErr) + format_str = "%(levelname)s: %(message)s" + console.setFormatter(bb.msg.BBLogFormatter(format_str)) + errconsole.setFormatter(bb.msg.BBLogFormatter(format_str)) + logger.handlers = [console, errconsole] + + def logger_setup_color(logger, color='auto'): from bb.msg import BBLogFormatter
Signature · 1 line
2.20.1
PATCH 3/4 recipetool, devtool: set up the logging filters
In reply to Christopher LarsonFrom: Christopher Larson <[email protected]>
We want errors going to stderr, the way they do in bitbake itself.
--- scripts/devtool | 2 ++ scripts/recipetool | 2 ++ 2 files changed, 4 insertions(+)
scripts/devtool +2 -0
@@ -282,11 +282,13 @@ def main(): logger.debug('Using standard bitbake path %s' % bitbakepath) scriptpath.add_oe_lib_path() + scriptutils.logger_setup_filters(logger) scriptutils.logger_setup_color(logger, global_args.color) if global_args.bbpath is None: try: tinfoil = setup_tinfoil(config_only=True, basepath=basepath) + tinfoil.logger.handlers = logger.handlers try: global_args.bbpath = tinfoil.config_data.getVar('BBPATH') finally:
scripts/recipetool +2 -0
@@ -75,9 +75,11 @@ def main(): logger.debug('Found bitbake path: %s' % bitbakepath) scriptpath.add_oe_lib_path() + scriptutils.logger_setup_filters(logger) scriptutils.logger_setup_color(logger, global_args.color) tinfoil = tinfoil_init(False) + tinfoil.logger.handlers = logger.handlers try: for path in (tinfoil.config_data.getVar('BBPATH').split(':') + [scripts_path]):
Signature · 1 line
2.20.1
PATCH 4/4 oe.scriptutils: also send WARNING to stderr
In reply to Christopher LarsonFrom: Christopher Larson <[email protected]>
stderr isn't just for errors, but anything out of band. It's highly unlikely that warning messages are anything but out of band, which makes scripting our tools more troublesome, since the script calling it has to filter out the warnings from stdout to get the real output.
--- scripts/lib/scriptutils.py | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
scripts/lib/scriptutils.py +4 -2
@@ -43,9 +43,11 @@ def logger_setup_filters(logger): import bb.msg console = logging.StreamHandler(sys.stdout) + bb.msg.addDefaultlogFilter(console) errconsole = logging.StreamHandler(sys.stderr) - bb.msg.addDefaultlogFilter(console, bb.msg.BBLogFilterStdOut) - bb.msg.addDefaultlogFilter(errconsole, bb.msg.BBLogFilterStdErr) + bb.msg.addDefaultlogFilter(errconsole) + console.addFilter(lambda r: r.levelno < logging.WARNING) + errconsole.addFilter(lambda r: r.levelno >= logging.WARNING) format_str = "%(levelname)s: %(message)s" console.setFormatter(bb.msg.BBLogFormatter(format_str)) errconsole.setFormatter(bb.msg.BBLogFormatter(format_str))
Signature · 1 line
2.20.1
Quoted 3 lines · NZDT Christopher Larson
From: Christopher Larson <[email protected]>
This improves logger setup in devtool and recipetool to send ERRORs to
stderr,
Quoted 15 lines
the way bitbake itself does, and also sends WARNINGs there, as those are essentially out of band as well. This makes it easier to script with devtool and recipetool, as any *real* subcommand output can be more easily captured without having to manually filter out the cruft.
The following changes since commit
b11725db2d5549dc45d8ae36fbf94a5c8e342d69:xserver-xorg: Fix build errors with clang (2019-03-04 14:26:41 +0000)
are available in the Git repository at:
[email protected]:kergoth/openembedded-core scriptutils-log-improvements
for you to fetch changes up to
c4f37568236d2e3d8c40d57399d0b275ad5b7c8b:
Quoted 9 lines
oe.scriptutils: also send WARNING to stderr (2019-03-04 21:31:00 +0000)
----------------------------------------------------------------
Christopher Larson (4):
oe.scriptutils: enable color in a more flexible way oe.scriptutils: addlogger_setup_filtersrecipetool, devtool: set up the logging filters oe.scriptutils: also send WARNING to stderr
All of these look good to me.
Cheers
Paul
Signature · 2 lines
Paul Eggleton Intel Open Source Technology Centre
Quoted 39 lines · Paul Eggleton
Quoted 4 lines · NZDT Christopher Larson
From: Christopher Larson <[email protected]>
This improves logger setup in devtool and recipetool to send ERRORs to
stderr,
Quoted 25 lines
the way bitbake itself does, and also sends WARNINGs there, as those are
essentially out of band as well. This makes it easier to script with devtool
and recipetool, as any *real* subcommand output can be more easily captured
without having to manually filter out the cruft.The following changes since commit
b11725db2d5549dc45d8ae36fbf94a5c8e342d69:xserver-xorg: Fix build errors with clang (2019-03-04 14:26:41
+0000)are available in the Git repository at:
[email protected]:kergoth/openembedded-core scriptutils-log-
improvementsfor you to fetch changes up to
c4f37568236d2e3d8c40d57399d0b275ad5b7c8b:oe.scriptutils: also send WARNING to stderr (2019-03-04 21:31:00
+0000)Quoted 7 lines
----------------------------------------------------------------
Christopher Larson (4):
oe.scriptutils: enable color in a more flexible way oe.scriptutils: addlogger_setup_filtersrecipetool, devtool: set up the logging filters oe.scriptutils: also send WARNING to stderr
Quoted 3 lines
All of these look good to me.
Acked-by Paul Eggleton <[email protected]>
Unfortunately there were a couple of issues on the autobuilder in the devtool selftests with these applied:
https://autobuilder.yoctoproject.org/typhoon/#/builders/56/builds/297
Cheers,
Richard