Skip to contentSkip to conversation

reworked environment variable whitelisting fixes

PATCH Applied Paul Eggleton 2 participants 5 messages Dec 14, 2010 – Dec 15, 2010

lore
Share & tools
lore
#0 lore raw

cover reworked environment variable whitelisting fixes

Adjustments to variable whitelisting to fix sstate package hashing. This supersedes the paule/sstate contrib branch.

The bitbake changes have been discussed briefly with Richard Purdie & Chris Larson.

Pull URL: git://git.pokylinux.org/poky-contrib.git
Branch: paule/sstate2
Browse: http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=paule/sstate2

Thanks,
Paul Eggleton <[email protected]>
---

Paul Eggleton (3):
bitbake: allow excluding dependencies of variables
bitbake: add optional expansion to getVarFlag() poky.conf: adjust variable whitelisting to fix sstate checksums

 bitbake/lib/bb/data.py       |    3 ++-
 bitbake/lib/bb/data_smart.py |    9 ++++++---
 meta/conf/distro/poky.conf   |    3 ++-
 3 files changed, 10 insertions(+), 5 deletions(-)
#1 lore raw

PATCH 1/3 bitbake: allow excluding dependencies of variables

Adds a vardepsexclude flag that can be used to exclude a dependency of a variable (the opposite of vardeps). This will allow the exclusion of variables from the hash generation much more selectively than blanket whitelisting using BB_HASHBASE_WHITELIST.

---
 bitbake/lib/bb/data.py |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)
bitbake/lib/bb/data.py +1 -0
index d4d43fd..0c95ebb 100644
@@ -297,6 +297,7 @@ def build_dependencies(key, keys, shelldeps, d):
             deps |= parser.references
             deps = deps | (keys & parser.execs)
         deps |= set((d.getVarFlag(key, "vardeps") or "").split())
+        deps -= set((d.getVarFlag(key, "vardepsexclude") or "").split())
     except:
         bb.note("Error expanding variable %s" % key) 
         raise
Signature · 1 line
1.7.1
#2 lore raw

PATCH 2/3 bitbake: add optional expansion to getVarFlag()

In reply to Paul Eggleton

Add a parameter to getVarFlag() to auto-expand the value of the flag. This makes getVarFlag() more consistent with getVar(), and allows expansion of vardeps and vardepsexclude (which has been done in this commit).

---
 bitbake/lib/bb/data.py       |    4 ++--
 bitbake/lib/bb/data_smart.py |    9 ++++++---
 2 files changed, 8 insertions(+), 5 deletions(-)
bitbake/lib/bb/data.py +2 -2
index 0c95ebb..0aa8b40 100644
@@ -296,8 +296,8 @@ def build_dependencies(key, keys, shelldeps, d):
             parser = d.expandWithRefs(d.getVar(key, False), key)
             deps |= parser.references
             deps = deps | (keys & parser.execs)
-        deps |= set((d.getVarFlag(key, "vardeps") or "").split())
-        deps -= set((d.getVarFlag(key, "vardepsexclude") or "").split())
+        deps |= set((d.getVarFlag(key, "vardeps", True) or "").split())
+        deps -= set((d.getVarFlag(key, "vardepsexclude", True) or "").split())
     except:
         bb.note("Error expanding variable %s" % key) 
         raise
bitbake/lib/bb/data_smart.py +6 -3
index c8cd8f8..30f9cbc 100644
@@ -277,12 +277,15 @@ class DataSmart:
             self._makeShadowCopy(var)
         self.dict[var][flag] = flagvalue
 
-    def getVarFlag(self, var, flag):
+    def getVarFlag(self, var, flag, exp = False):
         local_var = self._findVar(var)
+        value = None
         if local_var:
             if flag in local_var:
-                return copy.copy(local_var[flag])
-        return None
+                value = copy.copy(local_var[flag])
+        if exp and value:
+            value = self.expand(value, None)
+        return value
 
     def delVarFlag(self, var, flag):
         local_var = self._findVar(var)
Signature · 1 line
1.7.1
#3 lore raw

PATCH 3/3 poky.conf: adjust variable whitelisting to fix sstate checksums

In reply to Paul Eggleton

After some analysis of task dependencies I have adjusted the global whitelist used to filter out variables that would otherwise cause unwanted changes in task checksums for sstate packages.

The following changes have been made to BB_HASHBASE_WHITELIST:

* Fix typo: FILESEXTRAPATHS not FILESEXTRPATHS
* Add variables FILE_DIRNAME HOME LOGNAME SHELL TERM USER
* Remove variables DATE and TIME (these are referred to by a number of tasks where they should affect the checksum, e.g. because they influence PV)
* Remove variable _ (not found in dependencies)

Additionally DATE and SRCDATE are excluded but only explicitly for patch_do_patch, which uses these only to provide a warning if patches are out of date.

---
 meta/conf/distro/poky.conf |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)
meta/conf/distro/poky.conf +2 -1
index 10b3165..95ca097 100644
@@ -143,4 +143,5 @@ require conf/distro/include/world-broken.inc
 # Setup our hash policy
 BB_SIGNATURE_HANDLER = "basic"
 BB_HASHTASK_WHITELIST = "(.*-cross$|.*-native$|.*-cross-initial$|.*-cross-intermediate$|^virtual:native:.*|^virtual:nativesdk:.*)"
-BB_HASHBASE_WHITELIST = "TMPDIR FILE PATH PWD BB_TASKHASH TIME DATE BBPATH DL_DIR SSTATE_DIR THISDIR _ FILESEXTRPATHS"
+BB_HASHBASE_WHITELIST = "TMPDIR FILE PATH PWD BB_TASKHASH BBPATH DL_DIR SSTATE_DIR THISDIR FILESEXTRAPATHS FILE_DIRNAME HOME LOGNAME SHELL TERM USER"
+patch_do_patch[vardepsexclude] = "DATE SRCDATE"
Signature · 1 line
1.7.1
#4 lore raw
In reply to Paul Eggleton
Quoted 11 lines · Paul Eggleton

Adjustments to variable whitelisting to fix sstate package hashing. This supersedes the paule/sstate contrib branch.

The bitbake changes have been discussed briefly with Richard Purdie & Chris Larson.

Pull URL: git://git.pokylinux.org/poky-contrib.git
Branch: paule/sstate2
Browse: http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=paule/sstate2

Thanks,
Paul Eggleton <[email protected]>

Merged into master, thanks!

Cheers,

Richard