Variable/include tracking (rebased)
cover Variable/include tracking (rebased)
This is a set of changes so that "bitbake -e" will report a list of all the files processed (and from what they were included), and prefix the variable values with their assignment history.
The following changes since commit 23bd5300b4a99218a15f4f6b0ab4091d63a602a5:
Richard Purdie (1):
data_smart: Fix unanchored regexp causing strange parsing issue
are available in the git repository at:
git://git.yoctoproject.org/poky-contrib seebs/ntracking
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=seebs/ntracking
Peter Seebach (2):
data_smart.py: Provide (optional) logging of variable modifications data_smart.py: Track configuration file inclusions
lib/bb/cooker.py | 2 + lib/bb/data.py | 65 +++++++++++++++++----- lib/bb/data_smart.py | 140 +++++++++++++++++++++++++++++++++++++-------- lib/bb/parse/__init__.py | 6 ++- lib/bb/parse/ast.py | 65 ++++++++++++---------- 5 files changed, 209 insertions(+), 69 deletions(-)
PATCH 1/2 data_smart.py: Provide (optional) logging of variable modifications
This is very preliminary, and probably not very good, but it provides a much-wanted feature: An explanation of how variables got their current values.
How it works:
1. We create a history dict in the data_smart object, the members of which are lists of tuples. It's three, count them, THREE unrelated data types in a single object!
2. There is an eventLog(...) function which allows recording things which happen to variables.
3. All over the place, tons of setVar and related functions are updated with two optional arguments, filename and lineno.
4. We log pretty much everything if logging is enabled.
5. You can query the history of a thing with data.getHistory(var)
Tracking must be specifically turned on. If it's not, the history object is a single empty table (cheap) and the calls to eventLog() and extra argument passing are also probably quite cheap. As a proof of concept, the showEnvironment function used by bitbake -e now uses this if available.
--- lib/bb/cooker.py | 2 + lib/bb/data.py | 49 +++++++++++++++++------- lib/bb/data_smart.py | 103 ++++++++++++++++++++++++++++++++++++++------------ lib/bb/parse/ast.py | 65 +++++++++++++++++-------------- 4 files changed, 152 insertions(+), 67 deletions(-)
lib/bb/cooker.py +2 -0
@@ -830,6 +830,8 @@ class BBCooker: def parseConfigurationFiles(self, prefiles, postfiles): data = self.configuration.data + if self.configuration.show_environment: + data.enableTracking() bb.parse.init_parser(data) # Parse files for loading *before* bitbake.conf and any includes
lib/bb/data.py +35 -14
97 lines not on this page · raw message
lib/bb/data_smart.py +79 -24
253 lines not on this page · raw message
lib/bb/parse/ast.py +36 -29
163 lines not on this page · raw message
Signature · 1 line
1.7.0.4
PATCH 2/2 data_smart.py: Track configuration file inclusions
In reply to Peter SeebachThis is a preliminary attempt to create a structured representation of the processing of include files. It ignores line numbers because they're meaningless. It could probably be updated to, say, distinguish between includes and requires. Output is added to bitbake -e.
--- lib/bb/data.py | 16 ++++++++++++++++ lib/bb/data_smart.py | 41 ++++++++++++++++++++++++++++++++++++++--- lib/bb/parse/__init__.py | 6 +++++- 3 files changed, 59 insertions(+), 4 deletions(-)
lib/bb/data.py +16 -0
@@ -253,10 +253,26 @@ def emit_env(o=sys.__stdout__, d = init(), all=False): isfunc = lambda key: bool(d.getVarFlag(key, "func")) keys = sorted((key for key in d.keys() if not key.startswith("__")), key=isfunc) grouped = groupby(keys, isfunc) + # Include history! + if d.tracking(): + o.write('#\n# INCLUDE HISTORY:\n#\n') + emit_history(o, d.getIncludeHistory()) + for isfunc, keys in grouped: for key in keys: emit_var(key, o, d, all and not isfunc) and o.write('\n') +def emit_history(o, h, depth = 0): + if not h: + return + for event in h: + o.write("# %*s%s" % (depth * 2, "", event[0])) + if event[1]: + o.write(" includes:\n") + emit_history(o, event[1], depth + 1) + else: + o.write("\n") + def exported_keys(d): return (key for key in d.keys() if not key.startswith('__') and d.getVarFlag(key, 'export') and
lib/bb/data_smart.py +38 -3
65 lines not on this page · raw message
lib/bb/parse/__init__.py +5 -1
@@ -88,7 +88,11 @@ def handle(fn, data, include = 0): """Call the handler that is appropriate for this file""" for h in handlers: if h['supports'](fn, data): - return h['handle'](fn, data, include) + data.includeLog(fn) + try: + return h['handle'](fn, data, include) + finally: + data.includeLogDone(fn) raise ParseError("not a BitBake file", fn) def init(fn, data):
Signature · 1 line
1.7.0.4