reworked environment variable whitelisting fixes
cover reworked environment variable whitelisting fixes
Adjustments to variable whitelisting to fix sstate package hashing. This supersedes the paule/sstate contrib branch.
The bitbake changes have been discussed briefly with Richard Purdie & Chris Larson.
Pull URL: git://git.pokylinux.org/poky-contrib.git
Branch: paule/sstate2
Browse: http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=paule/sstate2
Thanks,
Paul Eggleton <[email protected]>
---
Paul Eggleton (3):
bitbake: allow excluding dependencies of variables
bitbake: add optional expansion to getVarFlag() poky.conf: adjust variable whitelisting to fix sstate checksums
bitbake/lib/bb/data.py | 3 ++- bitbake/lib/bb/data_smart.py | 9 ++++++--- meta/conf/distro/poky.conf | 3 ++- 3 files changed, 10 insertions(+), 5 deletions(-)
PATCH 1/3 bitbake: allow excluding dependencies of variables
Adds a vardepsexclude flag that can be used to exclude a dependency of a variable (the opposite of vardeps). This will allow the exclusion of variables from the hash generation much more selectively than blanket whitelisting using BB_HASHBASE_WHITELIST.
--- bitbake/lib/bb/data.py | 1 + 1 files changed, 1 insertions(+), 0 deletions(-)
bitbake/lib/bb/data.py +1 -0
@@ -297,6 +297,7 @@ def build_dependencies(key, keys, shelldeps, d): deps |= parser.references deps = deps | (keys & parser.execs) deps |= set((d.getVarFlag(key, "vardeps") or "").split()) + deps -= set((d.getVarFlag(key, "vardepsexclude") or "").split()) except: bb.note("Error expanding variable %s" % key) raise
Signature · 1 line
1.7.1
PATCH 2/3 bitbake: add optional expansion to getVarFlag()
In reply to Paul EggletonAdd a parameter to getVarFlag() to auto-expand the value of the flag. This makes getVarFlag() more consistent with getVar(), and allows expansion of vardeps and vardepsexclude (which has been done in this commit).
--- bitbake/lib/bb/data.py | 4 ++-- bitbake/lib/bb/data_smart.py | 9 ++++++--- 2 files changed, 8 insertions(+), 5 deletions(-)
bitbake/lib/bb/data.py +2 -2
@@ -296,8 +296,8 @@ def build_dependencies(key, keys, shelldeps, d): parser = d.expandWithRefs(d.getVar(key, False), key) deps |= parser.references deps = deps | (keys & parser.execs) - deps |= set((d.getVarFlag(key, "vardeps") or "").split()) - deps -= set((d.getVarFlag(key, "vardepsexclude") or "").split()) + deps |= set((d.getVarFlag(key, "vardeps", True) or "").split()) + deps -= set((d.getVarFlag(key, "vardepsexclude", True) or "").split()) except: bb.note("Error expanding variable %s" % key) raise
bitbake/lib/bb/data_smart.py +6 -3
@@ -277,12 +277,15 @@ class DataSmart: self._makeShadowCopy(var) self.dict[var][flag] = flagvalue - def getVarFlag(self, var, flag): + def getVarFlag(self, var, flag, exp = False): local_var = self._findVar(var) + value = None if local_var: if flag in local_var: - return copy.copy(local_var[flag]) - return None + value = copy.copy(local_var[flag]) + if exp and value: + value = self.expand(value, None) + return value def delVarFlag(self, var, flag): local_var = self._findVar(var)
Signature · 1 line
1.7.1
PATCH 3/3 poky.conf: adjust variable whitelisting to fix sstate checksums
In reply to Paul EggletonAfter some analysis of task dependencies I have adjusted the global whitelist used to filter out variables that would otherwise cause unwanted changes in task checksums for sstate packages.
The following changes have been made to BB_HASHBASE_WHITELIST:
* Fix typo: FILESEXTRAPATHS not FILESEXTRPATHS
* Add variables FILE_DIRNAME HOME LOGNAME SHELL TERM USER
* Remove variables DATE and TIME (these are referred to by a number of tasks where they should affect the checksum, e.g. because they influence PV)
* Remove variable _ (not found in dependencies)
Additionally DATE and SRCDATE are excluded but only explicitly for patch_do_patch, which uses these only to provide a warning if patches are out of date.
--- meta/conf/distro/poky.conf | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-)
meta/conf/distro/poky.conf +2 -1
@@ -143,4 +143,5 @@ require conf/distro/include/world-broken.inc # Setup our hash policy BB_SIGNATURE_HANDLER = "basic" BB_HASHTASK_WHITELIST = "(.*-cross$|.*-native$|.*-cross-initial$|.*-cross-intermediate$|^virtual:native:.*|^virtual:nativesdk:.*)" -BB_HASHBASE_WHITELIST = "TMPDIR FILE PATH PWD BB_TASKHASH TIME DATE BBPATH DL_DIR SSTATE_DIR THISDIR _ FILESEXTRPATHS" +BB_HASHBASE_WHITELIST = "TMPDIR FILE PATH PWD BB_TASKHASH BBPATH DL_DIR SSTATE_DIR THISDIR FILESEXTRAPATHS FILE_DIRNAME HOME LOGNAME SHELL TERM USER" +patch_do_patch[vardepsexclude] = "DATE SRCDATE"
Signature · 1 line
1.7.1
Richard Purdie
AppliedQuoted 11 lines · Paul Eggleton
Adjustments to variable whitelisting to fix sstate package hashing. This supersedes the paule/sstate contrib branch.
The bitbake changes have been discussed briefly with Richard Purdie & Chris Larson.
Pull URL:
git://git.pokylinux.org/poky-contrib.git
Branch: paule/sstate2
Browse: http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=paule/sstate2Thanks,
Paul Eggleton <[email protected]>
Merged into master, thanks!
Cheers,
Richard