RFC data_smart.py (mostly): Variable and include tracking!

This is a first pass at allowing us to answer that vexing question:

Where the heck did that value come from?

The complexity of bitbake's variable assignment means that even if you had a complete list of the lines being processed in order, you might need a few tries to figure out why a variable is set to what it is. When debugging, you might want to know what it would have been otherwise, too; knowing that a given ?= is processed after a +=, for instance, could save you a great deal of trouble. To solve this, we add two pieces of functionality to "bitbake -e".

The first is a list of the .conf/.inc files that were parsed, in order; the list shows included files indented under the thing that included them, making it easier to see not just what got processed, but why it got processed.

The second is a list of all the operations performed on each value, in order, citing to the file and line number where they happened (whenever possible). So, for instance, say I want to know how SSTATETASKS got its value. The output of "bitbake -e" now says:

# SSTATETASKS [4]
# append /home/seebs/poky/meta/classes/staging.bbclass:105:
# < do_populate_sysroot>
# append /home/seebs/poky/meta/classes/package.bbclass:1750:
# < do_populate_sysroot do_package>
# append /home/seebs/poky/meta/classes/package_rpm.bbclass:1112:
# < do_populate_sysroot do_package do_package_write_rpm>
# append /home/seebs/poky/meta/classes/license.bbclass:380:
# < do_populate_sysroot do_package do_package_write_rpm do_populate_lic>
#
# SSTATETASKS= do_populate_sysroot do_package do_package_write_rpm do_populate_lic

Wait, staging.bbclass? I don't remember asking for that. Let's go look at the include history:

# /home/seebs/poky/meta/classes/base.bbclass includes:
#   /home/seebs/poky/meta/classes/patch.bbclass includes:
#     /home/seebs/poky/meta/classes/terminal.bbclass
#   /home/seebs/poky/meta/classes/staging.bbclass

So this tells me that staging.bbclass was included from base.bbclass. Yay! Mystery is solved. Everything is working because it's doing what it is supposed to do. :)

The following changes since commit f8bf4499549f978ce7c93bc088a66f74cd751e2c:
Martin Jansa (1):
setup.py: fix path for bitbake-selftest

are available in the git repository at:

git://git.yoctoproject.org/poky-contrib seebs/tracking
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=seebs/tracking

Peter Seebach (2):
data_smart.py: Provide (optional) logging of variable modifications data_smart.py: Track configuration file inclusions

 lib/bb/cooker.py                   |    4 +-
 lib/bb/data.py                     |   46 +++++++++++--
 lib/bb/data_smart.py               |  128 +++++++++++++++++++++++++++--------
 lib/bb/parse/__init__.py           |    8 ++-
 lib/bb/parse/ast.py                |   89 +++++++++++++------------
 lib/bb/parse/parse_py/BBHandler.py |    8 +-
 lib/bb/siggen.py                   |    2 +-
 7 files changed, 198 insertions(+), 87 deletions(-)