Skip to contentSkip to conversation

utils: Remove double compile from better_compile

PATCH In review Richard Purdie 2 participants 3 messages Jan 4, 2016

lore
Share & tools
lore

Richard Purdie

author
#0 lore raw

PATCH 1/1 utils: Remove double compile from better_compile

Poking around the ast to correct linenumbers works well for runtime failures but not for parsing ones. We can use blank linefeeds to correct the line numbers instead, with the advantage that we don't need to double compile.

Signed-off-by Richard Purdie <[email protected]>
bitbake/lib/bb/utils.py +4 -5
index cd5fced..9a3efb2 100644
@@ -292,7 +292,7 @@ def _print_trace(body, line):
             error.append('     %.4d:%s' % (i, body[i-1].rstrip()))
     return error
 
-def better_compile(text, file, realfile, mode = "exec", lineno = None):
+def better_compile(text, file, realfile, mode = "exec", lineno = 0):
     """
     A better compile method. This method
     will print the offending lines.

@@ -301,10 +301,9 @@ def better_compile(text, file, realfile, mode = "exec", lineno = None):
         cache = bb.methodpool.compile_cache(text)
         if cache:
             return cache
-        code = compile(text, realfile, mode, ast.PyCF_ONLY_AST)
-        if lineno is not None:
-            ast.increment_lineno(code, lineno)
-        code = compile(code, realfile, mode)
+        # We can't add to the linenumbers for compile, we can pad to the correct number of blank lines though
+        text2 = "\n" * int(lineno) + text
+        code = compile(text2, realfile, mode)
         bb.methodpool.compile_cache_add(text, code)
         return code
     except Exception as e:
#1 lore raw
Quoted 32 lines · Richard Purdie

Poking around the ast to correct linenumbers works well for runtime failures
but not for parsing ones. We can use blank linefeeds to correct the line numbers instead, with the advantage that we don't need to double compile.

Signed-off-by Richard Purdie <[email protected]>
bitbake/lib/bb/utils.py +1 -1
index cd5fced..9a3efb2 100644
@@ -292,7 +292,7 @@ def _print_trace(body, line):
             error.append('     %.4d:%s' % (i, body[i-1].rstrip()))
     return error
 
-def better_compile(text, file, realfile, mode = "exec", lineno = None):
+def better_compile(text, file, realfile, mode = "exec", lineno = 0):
     """
     A better compile method. This method
     will print the offending lines.

@@ -301,10 +301,9 @@ def better_compile(text, file, realfile, mode =

"exec", lineno = None):
cache = bb.methodpool.compile_cache(text) if cache:
return cache

bitbake/lib/bb/utils.py +1 -4
-        code = compile(text, realfile, mode, ast.PyCF_ONLY_AST)
-        if lineno is not None:
-            ast.increment_lineno(code, lineno)
-        code = compile(code, realfile, mode)
+        # We can't add to the linenumbers for compile, we can pad to the

correct number of blank lines though

bitbake/lib/bb/utils.py def better_compile(text, file, realfile, mode = "exec", lineno = None): · line 306

Quoted 2 lines
bitbake/lib/bb/utils.py +2 -0
+        text2 = "\n" * int(lineno) + text
+        code = compile(text2, realfile, mode)

A SyntaxError can have its line numbers adjusted fairly easily, if that's your concern. Afaik that's the usual case which ast.increment_lineno isn't sufficient to handle. See
https://gist.github.com/kergoth/743677#file-compile-py-L67-L86.

Signature · 5 lines
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics
author
#2 lore raw
Quoted 37 lines · Christopher Larson
Quoted 34 lines · Richard Purdie

Poking around the ast to correct linenumbers works well for runtime failures
but not for parsing ones. We can use blank linefeeds to correct the line
numbers instead, with the advantage that we don't need to double compile.

Signed-off-by Richard Purdie <[email protected]>
bitbake/lib/bb/utils.py +0 -1
index cd5fced..9a3efb2 100644
@@ -292,7 +292,7 @@ def _print_trace(body, line):
             error.append('     %.4d:%s' % (i, body[i-1].rstrip()))
     return error
 
-def better_compile(text, file, realfile, mode = "exec", lineno =

None):
+def better_compile(text, file, realfile, mode = "exec", lineno = 0):
"""
A better compile method. This method
will print the offending lines.

bitbake/lib/bb/utils.py +0 -0
@@ -301,10 +301,9 @@ def better_compile(text, file, realfile, mode

= "exec", lineno = None):
cache = bb.methodpool.compile_cache(text) if cache:
return cache

bitbake/lib/bb/utils.py +1 -4
-        code = compile(text, realfile, mode, ast.PyCF_ONLY_AST)
-        if lineno is not None:
-            ast.increment_lineno(code, lineno)
-        code = compile(code, realfile, mode)
+        # We can't add to the linenumbers for compile, we can pad
Quoted 8 lines

bitbake/lib/bb/utils.py def better_compile(text, file, realfile, mode = "exec", lineno = None): · line 306

Quoted 4 lines

to the correct number of blank lines though

bitbake/lib/bb/utils.py +2 -0
+        text2 = "\n" * int(lineno) + text
+        code = compile(text2, realfile, mode)

A SyntaxError can have its line numbers adjusted fairly easily, if that's your concern. Afaik that's the usual case which ast.increment_lineno isn't sufficient to handle. See
https://gist.github.com/kergoth/743677#file-compile-py-L67-L86.

I did start to wonder about catching specific exceptions and then adjusting the line numbers. It seemed unlikely that we'd catch all the right cases and adjust them correctly though and when there is a much easier workaround with a few blank lines that likely performs better than the existing double compile, I concluded that whilst not that architecturally pleasing, it is simple and effective...

Cheers,

Richard