Skip to contentSkip to conversation

BitBake signature tool improvements

PATCH In review Paul Eggleton 1 participant 6 messages Oct 4, 2013

lore
Share & tools
lore
#0 lore raw

cover BitBake signature tool improvements

Make bitbake-diffsigs and bitbake-dumpsig a little easier to use.

The following changes (against poky, but apply cleanly against bitbake master with -p2) are available in the git repository at:

git://git.yoctoproject.org/poky-contrib paule/bb-sig-tools
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=paule/bb-sig-tools

Paul Eggleton (4):
bitbake-diffsigs: handle if task name is specified without do_ prefix
bitbake-diffsigs: refactor argument parsing slightly
bitbake-diffsigs: improve error handling
bitbake-dumpsig: introduce command line and error handling

 bitbake/bin/bitbake-diffsigs | 57 ++++++++++++++++++++++++++++-------------
 bitbake/bin/bitbake-dumpsig  | 60 +++++++++++++++++++++++++++++++++++++++++---
 2 files changed, 96 insertions(+), 21 deletions(-)
Signature · 1 line
1.8.1.2
#1 lore raw

PATCH 1/4 bitbake-diffsigs: handle if task name is specified without do_ prefix

Warn the user and adjust the task name automatically if the -t option is specified with a task name that doesn't start with do_ (e.g. "configure" instead of "do_configure").

---
 bitbake/bin/bitbake-diffsigs | 4 ++++
 1 file changed, 4 insertions(+)
bitbake/bin/bitbake-diffsigs +4 -0
index 3ce70da..6e4c44a 100755
@@ -39,6 +39,10 @@ def find_compare_task(bbhandler, pn, taskname):
         logger.error('Metadata does not support finding signature data files')
         sys.exit(1)
 
+    if not taskname.startswith('do_'):
+        logger.warn('Invalid task name "%s" - assuming you meant "do_%s"' % (taskname, taskname))
+        taskname = 'do_%s' % taskname
+
     filedates = bb.siggen.find_siginfo(pn, taskname, None, bbhandler.config_data)
     latestfiles = sorted(filedates.keys(), key=lambda f: filedates[f])[-2:]
     if not latestfiles:
Signature · 1 line
1.8.1.2
#3 lore raw

PATCH 2/4 bitbake-diffsigs: refactor argument parsing slightly

In reply to Paul Eggleton

* Use OptionParser to parse the two options to -t rather than trying to pick them out ourselves.
* Add a description shown with --help output

---
 bitbake/bin/bitbake-diffsigs | 29 +++++++++++++----------------
 1 file changed, 13 insertions(+), 16 deletions(-)
bitbake/bin/bitbake-diffsigs +13 -16
index 6e4c44a..6f24981 100755
@@ -75,6 +75,7 @@ def find_compare_task(bbhandler, pn, taskname):
 
 
 parser = optparse.OptionParser(
+    description = "Compares siginfo/sigdata files written out by BitBake",
     usage = """
   %prog -t recipename taskname
   %prog sigdatafile1 sigdatafile2
@@ -82,25 +83,21 @@ parser = optparse.OptionParser(
 
 parser.add_option("-t", "--task",
         help = "find the signature data files for last two runs of the specified task and compare them",
-        action="store_true", dest="taskmode")
+        action="store", dest="taskargs", nargs=2, metavar='recipename taskname')
 
 options, args = parser.parse_args(sys.argv)
 
-if len(args) == 1:
-    parser.print_help()
+if options.taskargs:
+    tinfoil = bb.tinfoil.Tinfoil()
+    tinfoil.prepare(config_only = True)
+    find_compare_task(tinfoil, options.taskargs[0], options.taskargs[1])
 else:
-    if options.taskmode:
-        tinfoil = bb.tinfoil.Tinfoil()
-        if len(args) < 3:
-            logger.error("Please specify a recipe and task name")
-            sys.exit(1)
-        tinfoil.prepare(config_only = True)
-        find_compare_task(tinfoil, args[1], args[2])
+    if len(args) == 1:
+        parser.print_help()
+    elif len(args) == 2:
+        output = bb.siggen.dump_sigfile(sys.argv[1])
     else:
-        if len(args) == 2:
-            output = bb.siggen.dump_sigfile(sys.argv[1])
-        else:
-            output = bb.siggen.compare_sigfiles(sys.argv[1], sys.argv[2])
+        output = bb.siggen.compare_sigfiles(sys.argv[1], sys.argv[2])
 
-        if output:
-            print '\n'.join(output)
+    if output:
+        print '\n'.join(output)
Signature · 1 line
1.8.1.2
#4 lore raw

PATCH 3/4 bitbake-diffsigs: improve error handling

In reply to Paul Eggleton

* Set up a logger independent of BitBake so we can log errors ourselves
* Handle common errors without printing a traceback

---
 bitbake/bin/bitbake-diffsigs | 30 +++++++++++++++++++++++++-----
 1 file changed, 25 insertions(+), 5 deletions(-)
bitbake/bin/bitbake-diffsigs +25 -5
index 6f24981..53b47e0 100755
@@ -3,7 +3,7 @@
 # bitbake-diffsigs
 # BitBake task signature data comparison utility
 #
-# Copyright (C) 2012 Intel Corporation
+# Copyright (C) 2013 Intel Corporation
 #
 # This program is free software; you can redistribute it and/or modify
 # it under the terms of the GNU General Public License version 2 as
@@ -30,7 +30,18 @@ sys.path.insert(0, os.path.join(os.path.dirname(os.path.dirname(sys.argv[0])), '
 import bb.tinfoil
 import bb.siggen
 
-logger = logging.getLogger('BitBake')
+def logger_create(name, output=sys.stderr):
+    logger = logging.getLogger(name)
+    console = logging.StreamHandler(output)
+    format = bb.msg.BBLogFormatter("%(levelname)s: %(message)s")
+    if output.isatty():
+        format.enable_color()
+    console.setFormatter(format)
+    logger.addHandler(console)
+    logger.setLevel(logging.INFO)
+    return logger
+
+logger = logger_create('bitbake-diffsigs')
 
 def find_compare_task(bbhandler, pn, taskname):
     """ Find the most recent signature files for the specified PN/task and compare them """
@@ -94,10 +105,19 @@ if options.taskargs:
 else:
     if len(args) == 1:
         parser.print_help()
-    elif len(args) == 2:
-        output = bb.siggen.dump_sigfile(sys.argv[1])
     else:
-        output = bb.siggen.compare_sigfiles(sys.argv[1], sys.argv[2])
+        import cPickle
+        try:
+            if len(args) == 2:
+                output = bb.siggen.dump_sigfile(sys.argv[1])
+            else:
+                output = bb.siggen.compare_sigfiles(sys.argv[1], sys.argv[2])
+        except IOError as e:
+            logger.error(str(e))
+            sys.exit(1)
+        except cPickle.UnpicklingError, EOFError:
+            logger.error('Invalid signature data - ensure you are specifying sigdata/siginfo files')
+            sys.exit(1)
 
     if output:
         print '\n'.join(output)
Signature · 1 line
1.8.1.2
#5 lore raw

PATCH 4/4 bitbake-dumpsig: introduce command line and error handling

In reply to Paul Eggleton

This utility doesn't take any special arguments, but it's nice if it at least knows how to deal with no arguments, --help and errors properly.

---
 bitbake/bin/bitbake-dumpsig | 60 ++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 57 insertions(+), 3 deletions(-)
bitbake/bin/bitbake-dumpsig +57 -3
index ccbc412..656d93a 100755
@@ -1,11 +1,65 @@
 #!/usr/bin/env python
+
+# bitbake-dumpsig
+# BitBake task signature dump utility
+#
+# Copyright (C) 2013 Intel Corporation
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
 import os
 import sys
 import warnings
+import optparse
+import logging
+
 sys.path.insert(0, os.path.join(os.path.dirname(os.path.dirname(sys.argv[0])), 'lib'))
 
 import bb.siggen
 
-output = bb.siggen.dump_sigfile(sys.argv[1])
-if output:
-    print '\n'.join(output)
+def logger_create(name, output=sys.stderr):
+    logger = logging.getLogger(name)
+    console = logging.StreamHandler(output)
+    format = bb.msg.BBLogFormatter("%(levelname)s: %(message)s")
+    if output.isatty():
+        format.enable_color()
+    console.setFormatter(format)
+    logger.addHandler(console)
+    logger.setLevel(logging.INFO)
+    return logger
+
+logger = logger_create('bitbake-dumpsig')
+
+parser = optparse.OptionParser(
+    description = "Dumps siginfo/sigdata files written out by BitBake",
+    usage = """
+  %prog sigdatafile""")
+
+options, args = parser.parse_args(sys.argv)
+
+if len(args) == 1:
+    parser.print_help()
+else:
+    import cPickle
+    try:
+        output = bb.siggen.dump_sigfile(args[1])
+    except IOError as e:
+        logger.error(str(e))
+        sys.exit(1)
+    except cPickle.UnpicklingError, EOFError:
+        logger.error('Invalid signature data - ensure you are specifying a sigdata/siginfo file')
+        sys.exit(1)
+
+    if output:
+        print '\n'.join(output)
Signature · 1 line
1.8.1.2
#2 lore raw

Re: [PATCH 1/4] bitbake-diffsigs: handle if task name is specified without do_ prefix

In reply to Paul Eggleton
Quoted 19 lines · On Friday 04 October 2013 17:44:25 Paul Eggleton wrote:

Warn the user and adjust the task name automatically if the -t option is specified with a task name that doesn't start with do_ (e.g. "configure" instead of "do_configure").

---
 bitbake/bin/bitbake-diffsigs | 4 ++++
 1 file changed, 4 insertions(+)
bitbake/bin/bitbake-diffsigs +2 -0
index 3ce70da..6e4c44a 100755
@@ -39,6 +39,10 @@ def find_compare_task(bbhandler, pn, taskname):
         logger.error('Metadata does not support finding signature data
 files') sys.exit(1)
 
+    if not taskname.startswith('do_'):
+        logger.warn('Invalid task name "%s" - assuming you meant "do_%s"' %
Quoted 5 lines

(taskname, taskname)) + taskname = 'do_%s' % taskname
+
filedates = bb.siggen.find_siginfo(pn, taskname, None,
bbhandler.config_data) latestfiles = sorted(filedates.keys(), key=lambda f:
filedates[f])[-2:] if not latestfiles:

I've dropped the warning message on the branch after discussion with Richard - the do_ prefix is really an internal detail and users are used to specifying these without the prefix when using bitbake -c.

Cheers,
Paul

Signature · 2 lines
Paul Eggleton
Intel Open Source Technology Centre