utils.bbclass: make FILESEXTRAPATHS colon delimited

PATCH In review Darren Hart 3 participants 6 messages May 25, 2011 – May 27, 2011

Share & tools

Darren Hart

author

2011-05-25 23:08 UTC #0 lore raw

cover utils.bbclass: make FILESEXTRAPATHS colon delimited

In trying to fix the broken FILESEXTRAPATHS breakage in meta-intel, Phil Blundell noted that the space delimiting of FILESEXTRAPATHS was sub-optimal. I didn't want to fix meta-intel twice, so this patch converts FILESEXTRAPATHS to being colon delimited.

This patch applies cleanly to oe-core and poky.

The following changes since commit 1607d7b6809eb3f0aa8d09713a4e467a1f4585a2:

IMAGE_ROOTFS_SIZE: Cleanup machine conf files (2011-05-25 14:07:37 -0700)

are available in the git repository at:
git://git.pokylinux.org/poky-contrib dvhart/filesextrapaths
http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=dvhart/filesextrapaths

Darren Hart (1):
utils.bbclass: make FILESEXTRAPATHS colon delimited

 meta/classes/utils.bbclass |    6 ++++--
 1 files changed, 4 insertions(+), 2 deletions(-)

Darren Hart

author

2011-05-25 23:08 UTC #1 lore raw

PATCH 1/1 utils.bbclass: make FILESEXTRAPATHS colon delimited

Fixes [YOCTO 1102]

Path variables are typically : delimited. White space is allowed in paths, so is not a good choice for separating paths. Currently utils.bbclass performs the following:

extrapaths = (bb.data.getVar("FILESEXTRAPATHS", d, True) or "").split()

This splits FILESEXTRAPATHS on whitespace. It later splits overrides on : and reassembles them all together as : delimited.

There is only one user of FILESEXTRAPATHS in oe-core (qt4-tools-native, which uses : anyway) and none in oe.

Change the split() in utils.bbclass to split on : instead of whitespace. When splitting on a defined string (":") we must be careful to handle the empty string case which returns [''] instead of [].

Tested building qt4-tools-native and core-image-minimal for surgarbay from meta-intel with a couple extra layers with FILESEXTRAPATHS modifications added.

Signed-off-by Darren Hart <[email protected]>

Cc Phil Blundell <[email protected]>

---
 meta/classes/utils.bbclass |    6 ++++--
 1 files changed, 4 insertions(+), 2 deletions(-)

meta/classes/utils.bbclass +4 -2

index e103351..9930a24 100644
@@ -331,8 +331,10 @@ def explode_deps(s):
 
 def base_set_filespath(path, d):
 	filespath = []
-	extrapaths = (bb.data.getVar("FILESEXTRAPATHS", d, True) or "").split()
-	path = extrapaths + path
+	extrapaths = (bb.data.getVar("FILESEXTRAPATHS", d, True) or "")
+	# Don't prepend empty strings to the path list
+	if extrapaths != "":
+		path = extrapaths.split(":") + path
 	# The ":" ensures we have an 'empty' override
 	overrides = (bb.data.getVar("OVERRIDES", d, 1) or "") + ":"
 	for p in path:

Signature · 1 line

1.7.1

Richard Purdie

2011-05-27 00:17 UTC #2 lore raw

Quoted 34 lines · Darren Hart

Fixes [YOCTO 1102]
Path variables are typically : delimited. White space is allowed in paths, so is not a good choice for separating paths. Currently utils.bbclass performs the following:
extrapaths = (bb.data.getVar("FILESEXTRAPATHS", d, True) or "").split()
This splits FILESEXTRAPATHS on whitespace. It later splits overrides on : and reassembles them all together as : delimited.
There is only one user of FILESEXTRAPATHS in oe-core (qt4-tools-native, which uses : anyway) and none in oe.
Change the split() in utils.bbclass to split on : instead of whitespace. When splitting on a defined string (":") we must be careful to handle the empty string case which returns [''] instead of [].
Tested building qt4-tools-native and core-image-minimal for surgarbay from meta-intel with a couple extra layers with FILESEXTRAPATHS modifications added.
Signed-off-by Darren Hart <[email protected]>
Cc Phil Blundell <[email protected]>
---
 meta/classes/utils.bbclass |    6 ++++--
 1 files changed, 4 insertions(+), 2 deletions(-)
meta/classes/utils.bbclass +0 -0
index e103351..9930a24 100644
@@ -331,8 +331,10 @@ def explode_deps(s):
 
 def base_set_filespath(path, d):

meta/classes/utils.bbclass def explode_deps(s): · line 340

Quoted 10 lines

meta/classes/utils.bbclass +4 -2

 	filespath = []
-	extrapaths = (bb.data.getVar("FILESEXTRAPATHS", d, True) or "").split()
-	path = extrapaths + path
+	extrapaths = (bb.data.getVar("FILESEXTRAPATHS", d, True) or "")
+	# Don't prepend empty strings to the path list
+	if extrapaths != "":
+		path = extrapaths.split(":") + path
 	# The ":" ensures we have an 'empty' override
 	overrides = (bb.data.getVar("OVERRIDES", d, 1) or "") + ":"
 	for p in path:

This is being a little picky but I find the above hard to read and can't we just do:

- extrapaths = (bb.data.getVar("FILESEXTRAPATHS", d, True) or "").split()
+ extrapaths = (bb.data.getVar("FILESEXTRAPATHS", d, True) or "").split(":")

Cheers,

Richard

Darren Hart

author

2011-05-27 03:20 UTC #3 lore raw

Quoted 49 lines · Richard Purdie

Quoted 34 lines · Darren Hart
Fixes [YOCTO 1102]
Path variables are typically : delimited. White space is allowed in paths, so is not a good choice for separating paths. Currently utils.bbclass performs the following:
extrapaths = (bb.data.getVar("FILESEXTRAPATHS", d, True) or "").split()
This splits FILESEXTRAPATHS on whitespace. It later splits overrides on : and reassembles them all together as : delimited.
There is only one user of FILESEXTRAPATHS in oe-core (qt4-tools-native, which uses : anyway) and none in oe.
Change the split() in utils.bbclass to split on : instead of whitespace. When splitting on a defined string (":") we must be careful to handle the empty string case which returns [''] instead of [].
Tested building qt4-tools-native and core-image-minimal for surgarbay from meta-intel with a couple extra layers with FILESEXTRAPATHS modifications added.
Signed-off-by Darren Hart <[email protected]>
Cc Phil Blundell <[email protected]>
---
 meta/classes/utils.bbclass |    6 ++++--
 1 files changed, 4 insertions(+), 2 deletions(-)
meta/classes/utils.bbclass +0 -0
index e103351..9930a24 100644
@@ -331,8 +331,10 @@ def explode_deps(s):
 
 def base_set_filespath(path, d):
meta/classes/utils.bbclass def explode_deps(s): · line 340
Quoted 10 lines
meta/classes/utils.bbclass +4 -2
 	filespath = []
-	extrapaths = (bb.data.getVar("FILESEXTRAPATHS", d, True) or "").split()
-	path = extrapaths + path
+	extrapaths = (bb.data.getVar("FILESEXTRAPATHS", d, True) or "")
+	# Don't prepend empty strings to the path list
+	if extrapaths != "":
+		path = extrapaths.split(":") + path
 	# The ":" ensures we have an 'empty' override
 	overrides = (bb.data.getVar("OVERRIDES", d, 1) or "") + ":"
 	for p in path:
This is being a little picky but I find the above hard to read and can't we just do:

Review

Quoted 2 lines

(quoted diff) +1 -1

-	extrapaths = (bb.data.getVar("FILESEXTRAPATHS", d, True) or "").split()
+	extrapaths = (bb.data.getVar("FILESEXTRAPATHS", d, True) or "").split(":")

Unfortunately not (this is what I tried initially), python split() has a rather annoying "feature", see:
http://docs.python.org/library/string.html

Where it states:
"
The behavior of split on an empty string depends on the value of sep. If sep is not specified, or specified as None, the result will be an empty list. If sep is specified as any string, the result will be a list containing one element which is an empty string.
"

ie:
"".split() -> []
"".split(":") -> [""]

So by changing from split() to split(":") we change the behavior of split when operating on empty strings, requiring us to special case the output. Rather obnoxious wouldn't you say?

Signature · 3 lines

Darren Hart
Intel Open Source Technology Center
Yocto Project - Linux Kernel

Phil Blundell

2011-05-27 09:40 UTC #4 lore raw

Quoted 3 lines · Darren Hart

So by changing from split() to split(":") we change the behavior of split when operating on empty strings, requiring us to special case the output. Rather obnoxious wouldn't you say?

Yes, that is an annoying misfeature. The other related thing to watch out for is this:

Quoted 1 line

Quoted 1 line
Quoted 1 line
print " a b ".split()

['a', 'b']

Quoted 1 line

Quoted 1 line
Quoted 1 line
print ":a:b:".split(":")

['', 'a', 'b', '']

Quoted 1 line

Quoted 1 line
Quoted 1 line

... in other words, if you make the separator be anything other than a space, you need to take special care to strip off any null elements that might have been accidentally generated at the start and end.

Richard Purdie

2011-05-27 15:50 UTC #5 lore raw

In reply to Darren Hart

Quoted 66 lines · Darren Hart

Quoted 49 lines · Richard Purdie
Quoted 34 lines · Darren Hart
Fixes [YOCTO 1102]
Path variables are typically : delimited. White space is allowed in paths, so is not a good choice for separating paths. Currently utils.bbclass performs the following:
extrapaths = (bb.data.getVar("FILESEXTRAPATHS", d, True) or "").split()
This splits FILESEXTRAPATHS on whitespace. It later splits overrides on : and reassembles them all together as : delimited.
There is only one user of FILESEXTRAPATHS in oe-core (qt4-tools-native, which uses : anyway) and none in oe.
Change the split() in utils.bbclass to split on : instead of whitespace. When splitting on a defined string (":") we must be careful to handle the empty string case which returns [''] instead of [].
Tested building qt4-tools-native and core-image-minimal for surgarbay from meta-intel with a couple extra layers with FILESEXTRAPATHS modifications added.
Signed-off-by Darren Hart <[email protected]>
Cc Phil Blundell <[email protected]>
---
 meta/classes/utils.bbclass |    6 ++++--
 1 files changed, 4 insertions(+), 2 deletions(-)
meta/classes/utils.bbclass +0 -0
index e103351..9930a24 100644
@@ -331,8 +331,10 @@ def explode_deps(s):
 
 def base_set_filespath(path, d):
meta/classes/utils.bbclass def explode_deps(s): · line 340
Quoted 10 lines
meta/classes/utils.bbclass +4 -2
 	filespath = []
-	extrapaths = (bb.data.getVar("FILESEXTRAPATHS", d, True) or "").split()
-	path = extrapaths + path
+	extrapaths = (bb.data.getVar("FILESEXTRAPATHS", d, True) or "")
+	# Don't prepend empty strings to the path list
+	if extrapaths != "":
+		path = extrapaths.split(":") + path
 	# The ":" ensures we have an 'empty' override
 	overrides = (bb.data.getVar("OVERRIDES", d, 1) or "") + ":"
 	for p in path:
This is being a little picky but I find the above hard to read and can't we just do:
Review
Quoted 2 lines
(quoted diff) +1 -1
-	extrapaths = (bb.data.getVar("FILESEXTRAPATHS", d, True) or "").split()
+	extrapaths = (bb.data.getVar("FILESEXTRAPATHS", d, True) or "").split(":")
Unfortunately not (this is what I tried initially), python split() has a rather annoying "feature", see:
http://docs.python.org/library/string.html
Where it states:
"
The behavior of split on an empty string depends on the value of sep. If sep is not specified, or specified as None, the result will be an empty list. If sep is specified as any string, the result will be a list containing one element which is an empty string.
"

Quoted 7 lines

ie:
"".split() -> []
"".split(":") -> [""]
So by changing from split() to split(":") we change the behavior of split when operating on empty strings, requiring us to special case the output. Rather obnoxious wouldn't you say?

Horrible but merged to master ;-).

Cheers,

Richard