recipetool/devtool: send errors to stderr

From: Christopher Larson <[email protected]>

This aligns with the behavior of bitbake proper. There is a bit of code duplication between knotty and scriptutils, but that's not new -- that was the case before as well. We should consider moving these logger configuration functions into bb.msg.

The following changes since commit 2ad7308ee7166641eff99f3b9fe6794de143f6bc:

oeqa/utils/qemurunner.py: Remove duplicate message on LoggingThread start (2015-09-22 18:13:02 +0100)

are available in the git repository at:

git://github.com/kergoth/openembedded-core scriptutils-log-improvements
https://github.com/kergoth/openembedded-core/tree/scriptutils-log-improvements

Christopher Larson (3):
oe.scriptutils: enable color in a more flexible way oe.scriptutils: add logger_setup_filters recipetool, devtool: set up the logging filters

 scripts/devtool            |  1 +
 scripts/lib/scriptutils.py | 29 +++++++++++++++++++++++------
 scripts/recipetool         |  1 +
 3 files changed, 25 insertions(+), 6 deletions(-)

2.2.1

From: Christopher Larson <[email protected]>

Rather than recreating handlers and forcing them, iterate over the handlers and enable color on ones we can handle. This makes it easier to handle color properly when we introduce the bb.msg default log filters.

---
 scripts/lib/scriptutils.py | 14 ++++++++------
 1 file changed, 8 insertions(+), 6 deletions(-)

index 3366882..3c165eb 100644
@@ -30,12 +30,12 @@ def logger_create(name):
 
 def logger_setup_color(logger, color='auto'):
     from bb.msg import BBLogFormatter
-    console = logging.StreamHandler(sys.stdout)
-    formatter = BBLogFormatter("%(levelname)s: %(message)s")
-    console.setFormatter(formatter)
-    logger.handlers = [console]
-    if color == 'always' or (color=='auto' and console.stream.isatty()):
-        formatter.enable_color()
+
+    for handler in logger.handlers:
+        if (isinstance(handler, logging.StreamHandler) and
+            isinstance(handler.formatter, BBLogFormatter)):
+            if color == 'always' or (color == 'auto' and handler.stream.isatty()):
+                handler.formatter.enable_color()
 
 
 def load_plugins(logger, plugins, pluginpath):

@@ -59,6 +59,7 @@ def load_plugins(logger, plugins, pluginpath):
                 plugin.plugin_init(plugins)
             plugins.append(plugin)
 
+
 def git_convert_standalone_clone(repodir):
     """If specified directory is a git repository, ensure it's a standalone clone"""
     import bb.process

@@ -70,6 +71,7 @@ def git_convert_standalone_clone(repodir):
             bb.process.run('git repack -a', cwd=repodir)
             os.remove(alternatesfile)
 
+
 def fetch_uri(d, uri, destdir, srcrev=None):
     """Fetch a URI to a local directory"""
     import bb.data

2.2.1

From: Christopher Larson <[email protected]>

This function sets up the default bb.msg formatter and log filters, which by default sends ERROR messages to stderr rather than stdout.

---
 scripts/lib/scriptutils.py | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

index 3c165eb..db0394a 100644
@@ -20,6 +20,7 @@ import os
 import logging
 import glob
 
+
 def logger_create(name):
     logger = logging.getLogger(name)
     loggerhandler = logging.StreamHandler()

@@ -28,6 +29,20 @@ def logger_create(name):
     logger.setLevel(logging.INFO)
     return logger
 
+
+def logger_setup_filters(logger):
+    import bb.msg
+
+    console = logging.StreamHandler(sys.stdout)
+    errconsole = logging.StreamHandler(sys.stderr)
+    bb.msg.addDefaultlogFilter(console, bb.msg.BBLogFilterStdOut)
+    bb.msg.addDefaultlogFilter(errconsole, bb.msg.BBLogFilterStdErr)
+    format_str = "%(levelname)s: %(message)s"
+    console.setFormatter(bb.msg.BBLogFormatter(format_str))
+    errconsole.setFormatter(bb.msg.BBLogFormatter(format_str))
+    logger.handlers = [console, errconsole]
+
+
 def logger_setup_color(logger, color='auto'):
     from bb.msg import BBLogFormatter
 

2.2.1

From: Christopher Larson <[email protected]>

We want errors going to stderr, the way they do in bitbake itself.

---
 scripts/devtool    | 1 +
 scripts/recipetool | 1 +
 2 files changed, 2 insertions(+)

index 87df951..5146538 100755
@@ -241,6 +241,7 @@ def main():
         logger.debug('Using standard bitbake path %s' % bitbakepath)
         scriptpath.add_oe_lib_path()
 
+    scriptutils.logger_setup_filters(logger)
     scriptutils.logger_setup_color(logger, global_args.color)
 
     if global_args.bbpath is None:

index 87fb35e..a1fc6e6 100755
@@ -72,6 +72,7 @@ def main():
         sys.exit(1)
     logger.debug('Found bitbake path: %s' % bitbakepath)
 
+    scriptutils.logger_setup_filters(logger)
     scriptutils.logger_setup_color(logger, global_args.color)
 
     tinfoil = tinfoil_init(False)

2.2.1

Hmm, might be worth making this even more generic and moving it to bb.msg. If one was to run this against a logger which isn't configured to use BBLogFormatter, this would do nothing. It should probably raise an exception if that occurs, or automatically replace the stock formatter instance with a BBLogFormatter instance using the same format string.

Christopher Larson
kergoth at gmail dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

Does anyone happen to know if there any objection to this? I just noticed this hasn't been merged yet. This makes the behavior of recipetool and devtool more consistent with bitbake itself by sending errors to stderr. Long term I'd like to see warnings go there as well, but that could be more controversial and is a later task, I think this should go in as is. Eventually we should consolidate some of the logger setup and configuration logic between scriptutils and bitbake as well, but that was the case with or without this series

Does anyone happen to know if there any objection to this? I just noticed this hasn't been merged yet. This makes the behavior of recipetool and devtool more consistent with bitbake itself by sending errors to stderr. Long term I'd like to see warnings go there as well, but that could be more controversial and is a later task, I think this should go in as is. Eventually we should consolidate some of the logger setup and configuration logic between scriptutils and bitbake as well, but that was the case with or without this series

I certainly don't - I did see this hadn't been merged a few weeks back, I should have acked it.

Cheers,
Paul

Paul Eggleton
Intel Open Source Technology Centre

Hi Chris,

There's also a similar scriptutils-log-improvements-2 branch, what branch is the one you want merged?

Ross

The scriptutils-log-improvements branch, please. scriptutils-log-improvements-2 includes bits which aren't yet ready. Thanks.

Christopher Larson
kergoth at gmail dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics