Skip to contentSkip to conversation

A couple of fixes to error handling v2

PATCH In review Paul Eggleton 2 participants 4 messages Feb 23, 2012

lore
Share & tools
lore
#0 lore raw

cover A couple of fixes to error handling v2

A couple of patches to fix/improve error handling during parsing.

Since v1, remove changes to general exception handling that weren't necessary and bypassed the ability for the UI to format the exception.

The patches (against poky, but apply cleanly with -p2 against bitbake master) are available at:
git://git.yoctoproject.org/poky-contrib paule/bitbake-errors2
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=paule/bitbake-errors2

Paul Eggleton (2):
bitbake: fix parse errors not being reported
bitbake: add file and line number to ParseError

 bitbake/lib/bb/cooker.py                     |   12 +++++-------
 bitbake/lib/bb/parse/__init__.py             |   15 +++++++++++++--
 bitbake/lib/bb/parse/ast.py                  |    8 ++++----
 bitbake/lib/bb/parse/parse_py/BBHandler.py   |    6 ++----
 bitbake/lib/bb/parse/parse_py/ConfHandler.py |   11 ++++++-----
 5 files changed, 30 insertions(+), 22 deletions(-)
Signature · 1 line
1.7.5.4
#1 lore raw

PATCH 1/2 bitbake: fix parse errors not being reported

Fixes a regression introduced in BitBake rev c9f58ef6b897d3fa5b0d23734b5f2cb3dabb057a which prevents errors during parsing from being fully reported because BitBake shuts down before it can print them. Move the error printing to before the shutdown in order to fix it.

Also remove a couple of superfluous calls to sys.exit() since these will never be called (as self.shutdown(clean=False) does this itself).

---
 bitbake/lib/bb/cooker.py |   10 ++++------
 1 files changed, 4 insertions(+), 6 deletions(-)
bitbake/lib/bb/cooker.py +4 -6
index f0778e5..308805a 100644
@@ -1605,22 +1605,20 @@ class CookerParser(object):
             self.shutdown()
             return False
         except ParsingFailure as exc:
-            self.shutdown(clean=False)
-            bb.fatal('Unable to parse %s: %s' %
+            logger.error('Unable to parse %s: %s' %
                      (exc.recipe, bb.exceptions.to_string(exc.realexception)))
+            self.shutdown(clean=False)
         except (bb.parse.ParseError, bb.data_smart.ExpansionError) as exc:
+            logger.error(str(exc))
             self.shutdown(clean=False)
-            bb.fatal(str(exc))
         except SyntaxError as exc:
-            self.shutdown(clean=False)
             logger.error('Unable to parse %s', exc.recipe)
-            sys.exit(1)
+            self.shutdown(clean=False)
         except Exception as exc:
             etype, value, tb = sys.exc_info()
             logger.error('Unable to parse %s', value.recipe,
                          exc_info=(etype, value, exc.traceback))
             self.shutdown(clean=False)
-            sys.exit(1)
 
         self.current += 1
         self.virtuals += len(result)
Signature · 1 line
1.7.5.4
#2 lore raw

PATCH 2/2 bitbake: add file and line number to ParseError

In reply to Paul Eggleton

Ensure that a file and line number are reported for ParseError where possible. This helps particularly in the case of inherit and require which previously did not report either of these upon failure.

---
 bitbake/lib/bb/cooker.py                     |    2 +-
 bitbake/lib/bb/parse/__init__.py             |   15 +++++++++++++--
 bitbake/lib/bb/parse/ast.py                  |    8 ++++----
 bitbake/lib/bb/parse/parse_py/BBHandler.py   |    6 ++----
 bitbake/lib/bb/parse/parse_py/ConfHandler.py |   11 ++++++-----
 5 files changed, 26 insertions(+), 16 deletions(-)
bitbake/lib/bb/cooker.py +1 -1
index 308805a..bf25a83 100644
@@ -1398,7 +1398,7 @@ def _parse(fn, data, include=True):
 
 @catch_parse_error
 def _inherit(bbclass, data):
-    bb.parse.BBHandler.inherit([bbclass], data)
+    bb.parse.BBHandler.inherit([bbclass], "configuration INHERITs", 0, data)
     return data
 
 class ParsingFailure(Exception):
bitbake/lib/bb/parse/__init__.py +13 -2
index c5005ae..8b7ec73 100644
@@ -37,6 +37,17 @@ logger = logging.getLogger("BitBake.Parsing")
 
 class ParseError(Exception):
     """Exception raised when parsing fails"""
+    def __init__(self, msg, filename, lineno=0):
+        self.msg = msg
+        self.filename = filename
+        self.lineno = lineno
+        Exception.__init__(self, msg, filename, lineno)
+
+    def __str__(self):
+        if self.lineno:
+            return "ParseError at %s:%d: %s" % (self.filename, self.lineno, self.msg)
+        else:
+            return "ParseError in %s: %s" % (self.filename, self.msg)
 
 class SkipPackage(Exception):
     """Exception raised to skip this package"""
@@ -78,7 +89,7 @@ def handle(fn, data, include = 0):
     for h in handlers:
         if h['supports'](fn, data):
             return h['handle'](fn, data, include)
-    raise ParseError("%s is not a BitBake file" % fn)
+    raise ParseError("not a BitBake file", fn)
 
 def init(fn, data):
     for h in handlers:
@@ -111,7 +122,7 @@ def vars_from_file(mypkg, d):
     parts = myfile[0].split('_')
     __pkgsplit_cache__[mypkg] = parts
     if len(parts) > 3:
-        raise ParseError("Unable to generate default variables from the filename: %s (too many underscores)" % mypkg)
+        raise ParseError("Unable to generate default variables from filename (too many underscores)", mypkg)
     exp = 3 - len(parts)
     tmplist = []
     while exp != 0:
bitbake/lib/bb/parse/ast.py +4 -4
index 31c930d..94fa175 100644
@@ -59,9 +59,9 @@ class IncludeNode(AstNode):
 
         # TODO: Cache those includes... maybe not here though
         if self.force:
-            bb.parse.ConfHandler.include(self.filename, s, data, "include required")
+            bb.parse.ConfHandler.include(self.filename, s, self.lineno, data, "include required")
         else:
-            bb.parse.ConfHandler.include(self.filename, s, data, False)
+            bb.parse.ConfHandler.include(self.filename, s, self.lineno, data, False)
 
 class ExportNode(AstNode):
     def __init__(self, filename, lineno, var):
@@ -267,7 +267,7 @@ class InheritNode(AstNode):
         self.classes = classes
 
     def eval(self, data):
-        bb.parse.BBHandler.inherit(self.classes, data)
+        bb.parse.BBHandler.inherit(self.classes, self.filename, self.lineno, data)
 
 def handleInclude(statements, filename, lineno, m, force):
     statements.append(IncludeNode(filename, lineno, m.group(1), force))
@@ -450,7 +450,7 @@ def multi_finalize(fn, d):
                 d.setVar("BBEXTENDVARIANT", variantmap[name])
             else:
                 d.setVar("PN", "%s-%s" % (pn, name))
-            bb.parse.BBHandler.inherit([extendedmap[name]], d)
+            bb.parse.BBHandler.inherit([extendedmap[name]], fn, 0, d)
 
         safe_d.setVar("BBCLASSEXTEND", extended)
         _create_variants(datastores, extendedmap.keys(), extendfunc)
bitbake/lib/bb/parse/parse_py/BBHandler.py +2 -4
index 2d6e331..125f458 100644
@@ -68,10 +68,8 @@ def supports(fn, d):
     """Return True if fn has a supported extension"""
     return os.path.splitext(fn)[-1] in [".bb", ".bbclass", ".inc"]
 
-def inherit(files, d):
+def inherit(files, fn, lineno, d):
     __inherit_cache = data.getVar('__inherit_cache', d) or []
-    fn = ""
-    lineno = 0
     for file in files:
         file = data.expand(file, d)
         if not os.path.isabs(file) and not file.endswith(".bbclass"):
@@ -81,7 +79,7 @@ def inherit(files, d):
             logger.log(logging.DEBUG -1, "BB %s:%d: inheriting %s", fn, lineno, file)
             __inherit_cache.append( file )
             data.setVar('__inherit_cache', __inherit_cache, d)
-            include(fn, file, d, "inherit")
+            include(fn, file, lineno, d, "inherit")
             __inherit_cache = data.getVar('__inherit_cache', d) or []
 
 def get_statements(filename, absolute_filename, base_name):
bitbake/lib/bb/parse/parse_py/ConfHandler.py +6 -5
index 6ae9d97..9242632 100644
@@ -44,10 +44,11 @@ def init(data):
 def supports(fn, d):
     return fn[-5:] == ".conf"
 
-def include(oldfn, fn, data, error_out):
+def include(oldfn, fn, lineno, data, error_out):
     """
-    error_out If True a ParseError will be raised if the to be included
-    config-files could not be included.
+    error_out: A string indicating the verb (e.g. "include", "inherit") to be
+    used in a ParseError that will be raised if the file to be included could
+    not be included. Specify False to avoid raising an error in this case.
     """
     if oldfn == fn: # prevent infinite recursion
         return None
@@ -68,7 +69,7 @@ def include(oldfn, fn, data, error_out):
         ret = handle(fn, data, True)
     except IOError:
         if error_out:
-            raise ParseError("Could not %(error_out)s file %(fn)s" % vars() )
+            raise ParseError("Could not %(error_out)s file %(fn)s" % vars(), oldfn, lineno)
         logger.debug(2, "CONF file '%s' not found", fn)
 
 def handle(fn, data, include):
@@ -131,7 +132,7 @@ def feeder(lineno, s, fn, statements):
         ast.handleExport(statements, fn, lineno, m)
         return
 
-    raise ParseError("%s:%d: unparsed line: '%s'" % (fn, lineno, s));
+    raise ParseError("unparsed line: '%s'" % s, fn, lineno);
 
 # Add us to the handlers list
 from bb.parse import handlers
Signature · 1 line
1.7.5.4
#3 lore raw
In reply to Paul Eggleton
Quoted 5 lines · Paul Eggleton

A couple of patches to fix/improve error handling during parsing.

Since v1, remove changes to general exception handling that weren't necessary and bypassed the ability for the UI to format the exception.

Quoted 8 lines

The patches (against poky, but apply cleanly with -p2 against bitbake master) are available at:
git://git.yoctoproject.org/poky-contrib paule/bitbake-errors2
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=paule/bitbake-errors2

Paul Eggleton (2):
bitbake: fix parse errors not being reported
bitbake: add file and line number to ParseError

These look like they improve the current situation a lot, merged to master, thanks.

Richard