update_data() removal patchset

Removing the need for finalization or update_data is something we've idly thought about for a long time. Basically, the whole idea of a specific finalization phase to the data store causes a lof of issues and horrible corner cases. For example, should bitbake call update_data, then expandKeys, or the other way around? Or should we keep calling update_data until nothing changes?

Through careful thought of some operations and working around the corner case problems, we've avoided most of the issues however there are some things we keep observing:

* We keep needing "expanded" data stores in different parts of the system alongside their unexpanded versions
* Few people understand why/when exactly they should call update_data()
* Its near impossible to set PREFERRED_PROVIDERS of recipes which are dynamically named (gcc-cross-${TARGET_ARCH}) along with overrides. This is something meta-darwin/mingw/baremetal could really use.

The alternative is a datastore which dynamically adjusts when anything changes so that the work is done at getVar time rather than precaching things at update_data time.

What is missing from the branch as yet is removal of the "expanded" data stores from use within bitbake and starting to remove the update_data calls (which are now just empty calls). This can happen once the code is merged/stablising.

The following changes since commit f883cf240266ee7be2cbd8971a8164cf4df9e372:

cooker: Improve DATE/TIME handling (2015-07-08 00:06:01 +0100)

are available in the git repository at:

git://git.openembedded.org/bitbake noupdatedata
http://cgit.openembedded.org/cgit.cgi/bitbake/log/?h=noupdatedata

Richard Purdie (16):
data_smart: Defer append/prepend handling
data_smart: Remove need for update_data calls
data_smart: Improve override handling
data_smart: Fix cache clearance problems
data_smart: Cache overrides and fix data store
data_smart: Move override handling to getVar parse/ast/data_smart: Add parsing flag to getVar/setVar
data_smart: VariableHistory: Ignore override duplicates and overlap
with CoW functions
data_smart: Seperate out the override cache
data_smart: Tweak OVERRIDES value cache for performance
data_smart: Improve override recursion handling
data_smart: Cache set(self.overrides)
data_smart: Fix keys() missing overrides
data_smart: Fix data expansion cache issues
data_smart: Fix appendVar/prependVar
tests/data: Add new data tests

 lib/bb/data_smart.py  | 337 ++++++++++++++++++++++++++++++--------------------
 lib/bb/parse/ast.py   |  14 +--
 lib/bb/tests/data.py  |  47 +++++++
 lib/bb/tests/parse.py |  46 +++++++
 4 files changed, 306 insertions(+), 138 deletions(-)

2.1.0