Allow environment variables and ~ in bbconf paths

PATCH In review Ed Bartosh 5 participants 10 messages Apr 14, 2015 – Apr 18, 2015

lore

Share & tools

lore

Ed Bartosh

author

2015-04-14 20:46 UTC #0 lore raw

PATCH 1/1 Allow environment variables and ~ in bbconf paths

This should allow to use ~, $HOME and other environment variables in bbconf paths.

Signed-off-by Ed Bartosh <[email protected]>

---
 bitbake/lib/bb/cookerdata.py | 1 +
 1 file changed, 1 insertion(+)

bitbake/lib/bb/cookerdata.py +1 -0

index 6c11a60..ec5c9ea 100644
@@ -271,6 +271,7 @@ class CookerDataBuilder(object):
             data = bb.data.createCopy(data)
             for layer in layers:
                 parselog.debug(2, "Adding layer %s", layer)
+                layer = os.path.expanduser(os.path.expandvars(layer))
                 data.setVar('LAYERDIR', layer)
                 data = parse_config_file(os.path.join(layer, "conf", "layer.conf"), data)
                 data.expandVarref('LAYERDIR')

Signature · 1 line

2.1.4

Rifenbark, Scott M

2015-04-14 21:19 UTC #1 lore raw

Ed,

This worked for me.... Thanks.

Scott

Quoted 30 lines

-----Original Message-----
From: Ed Bartosh [mailto:[email protected]]
Sent: Tuesday, April 14, 2015 1:46 PM
To: [email protected]
Cc: Rifenbark, Scott M; Ed Bartosh
Subject: [PATCH] Allow environment variables and ~ in bbconf paths
This should allow to use ~, $HOME and other environment variables in bbconf paths.
Signed-off-by Ed Bartosh <[email protected]>
---
bitbake/lib/bb/cookerdata.py | 1 +
1 file changed, 1 insertion(+)
bitbake/lib/bb/cookerdata.py +1 -0
index 6c11a60..ec5c9ea 100644
@@ -271,6 +271,7 @@ class CookerDataBuilder(object):
            data = bb.data.createCopy(data)
            for layer in layers:
                parselog.debug(2, "Adding layer %s", layer)
+                layer = os.path.expanduser(os.path.expandvars(layer))
                data.setVar('LAYERDIR', layer)
                data = parse_config_file(os.path.join(layer, "conf", "layer.conf"),
 data)
data.expandVarref('LAYERDIR')
--
2.1.4

Christopher Larson

2015-04-14 21:34 UTC #2 lore raw

Quoted 13 lines · Scott M

Ed,
This worked for me.... Thanks.
Scott
Quoted 7 lines
-----Original Message-----
From: Ed Bartosh [mailto:[email protected]]
Sent: Tuesday, April 14, 2015 1:46 PM
To: [email protected]
Cc: Rifenbark, Scott M; Ed Bartosh
Subject: [PATCH] Allow environment variables and ~ in bbconf paths

Quoted 6 lines

Quoted 1 line
This should allow to use ~, $HOME and other environment variables in
bbconf
Quoted 3 lines
paths.
Signed-off-by Ed Bartosh <[email protected]>

We already have a mechanism by which environment variables flow into the metadata. Is there something wrong with using BB_ENV_EXTRAWHITE to use ${HOME} in this context?

Signature · 5 lines

Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

Ed Bartosh

author

2015-04-15 11:48 UTC #3 lore raw

Quoted 18 lines · Christopher Larson

Quoted 15 lines · Scott M
Ed,
This worked for me.... Thanks.
Scott
Quoted 8 lines
-----Original Message-----
From: Ed Bartosh [mailto:[email protected]]
Sent: Tuesday, April 14, 2015 1:46 PM
To: [email protected]
Cc: Rifenbark, Scott M; Ed Bartosh
Subject: [PATCH] Allow environment variables and ~ in bbconf paths
This should allow to use ~, $HOME and other environment variables in
bbconf

Quoted 8 lines

Quoted 4 lines
Quoted 3 lines
paths.
Signed-off-by Ed Bartosh <[email protected]>
We already have a mechanism by which environment variables flow into the metadata. Is there something wrong with using BB_ENV_EXTRAWHITE to use ${HOME} in this context?

Thank you for pointing out to this. However, I couldn't make this mechanism
to work for bbconf. I tried to set BB_PRESERVE_ENV, BB_ENV_WHITELIST and BB_ENV_EXTRAWHITE without any luck. bitbake still crashes if HOME is used in bbconf.

--
Regards,
Ed

Ed Bartosh

author

2015-04-15 14:43 UTC #4 lore raw

PATCH 1/1 bitbake: Allow environment variables and ~ in bblayers

In reply to Christopher Larson

Enabled processing of environment variables in bblayers paths using bitbake apporved_variables API.

Enabled processing of ~ if HOME environment variable is approved.

Signed-off-by Ed Bartosh <[email protected]>

---
 bitbake/lib/bb/cookerdata.py | 6 ++++++
 1 file changed, 6 insertions(+)

bitbake/lib/bb/cookerdata.py +6 -0

index 6c11a60..05b3986 100644
@@ -25,6 +25,7 @@
 import os, sys
 from functools import wraps
 import logging
+import string
 import bb
 from bb import data
 import bb.parse

@@ -269,8 +270,13 @@ class CookerDataBuilder(object):
             layers = (data.getVar('BBLAYERS', True) or "").split()
 
             data = bb.data.createCopy(data)
+            approved = dict((var, data.getVar(var, True)) \
+                                  for var in bb.utils.approved_variables())
             for layer in layers:
                 parselog.debug(2, "Adding layer %s", layer)
+                layer = string.Template(layer).safe_substitute(approved)
+                if 'HOME' in approved:
+                    layer = os.path.expanduser(layer)
                 data.setVar('LAYERDIR', layer)
                 data = parse_config_file(os.path.join(layer, "conf", "layer.conf"), data)
                 data.expandVarref('LAYERDIR')

Signature · 1 line

2.1.4

Christopher Larson

2015-04-15 19:39 UTC #5 lore raw

Re: [PATCH] bitbake: Allow environment variables and ~ in bblayers

Quoted 36 lines · Ed Bartosh

Enabled processing of environment variables in bblayers paths using bitbake apporved_variables API.

Enabled processing of ~ if HOME environment variable is approved.

Signed-off-by Ed Bartosh <[email protected]>

---
 bitbake/lib/bb/cookerdata.py | 6 ++++++
 1 file changed, 6 insertions(+)

bitbake/lib/bb/cookerdata.py +6 -0

index 6c11a60..05b3986 100644
@@ -25,6 +25,7 @@
 import os, sys
 from functools import wraps
 import logging
+import string
 import bb
 from bb import data
 import bb.parse

@@ -269,8 +270,13 @@ class CookerDataBuilder(object):
             layers = (data.getVar('BBLAYERS', True) or "").split()
 
             data = bb.data.createCopy(data)
+            approved = dict((var, data.getVar(var, True)) \
+                                  for var in bb
 .utils.approved_variables())
             for layer in layers:
                 parselog.debug(2, "Adding layer %s", layer)
+                layer = string.Template(layer).safe_substitute(approved)
+                if 'HOME' in approved:
+                    layer = os.path.expanduser(layer)
                 data.setVar('LAYERDIR', layer)
                 data = parse_config_file(os.path.join(layer, "conf",

Quoted 3 lines

"layer.conf"), data)
data.expandVarref('LAYERDIR')

Why the use of string.Template here? We already have a variable expansion mechanism.

Signature · 5 lines

Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

Ed Bartosh

author

2015-04-16 18:29 UTC #6 lore raw

PATCH 1/1 bitbake: Allow ~ in bblayers

Implemented processing of ~ in bblayer's paths if HOME environment variable is approved.

Signed-off-by Ed Bartosh <[email protected]>

---
 bitbake/lib/bb/cookerdata.py | 3 +++
 1 file changed, 3 insertions(+)

bitbake/lib/bb/cookerdata.py +3 -0

index 6c11a60..bf08c63 100644
@@ -269,8 +269,11 @@ class CookerDataBuilder(object):
             layers = (data.getVar('BBLAYERS', True) or "").split()
 
             data = bb.data.createCopy(data)
+            approved = bb.utils.approved_variables()
             for layer in layers:
                 parselog.debug(2, "Adding layer %s", layer)
+                if 'HOME' in approved:
+                    layer = os.path.expanduser(layer)
                 data.setVar('LAYERDIR', layer)
                 data = parse_config_file(os.path.join(layer, "conf", "layer.conf"), data)
                 data.expandVarref('LAYERDIR')

Signature · 1 line

2.1.4

Richard Purdie

2015-04-17 10:42 UTC #7 lore raw

Re: [PATCH] bitbake: Allow ~ in bblayers

Quoted 13 lines · Ed Bartosh

Implemented processing of ~ in bblayer's paths if HOME environment variable is approved.
Signed-off-by Ed Bartosh <[email protected]>
---
 bitbake/lib/bb/cookerdata.py | 3 +++
 1 file changed, 3 insertions(+)
bitbake/lib/bb/cookerdata.py +0 -0
index 6c11a60..bf08c63 100644

bitbake/lib/bb/cookerdata.py class CookerDataBuilder(object): · line 275

Quoted 7 lines

bitbake/lib/bb/cookerdata.py +2 -0

@@ -269,8 +269,11 @@ class CookerDataBuilder(object):
             layers = (data.getVar('BBLAYERS', True) or "").split()
 
             data = bb.data.createCopy(data)
+            approved = bb.utils.approved_variables()
             for layer in layers:
                 parselog.debug(2, "Adding layer %s", layer)
+                if 'HOME' in approved:

can we change this to

if 'HOME' in approved and '~' in layer:

simply so that we can more easily remember what this code is doing in the future?

Cheers,

Richard

Quoted 7 lines

bitbake/lib/bb/cookerdata.py +1 -0

+                    layer = os.path.expanduser(layer)
                 data.setVar('LAYERDIR', layer)
                 data = parse_config_file(os.path.join(layer, "conf", "layer.conf"), data)
                 data.expandVarref('LAYERDIR')

Signature · 1 line

2.1.4

Gary Thomas

2015-04-17 11:27 UTC #8 lore raw

Re: [PATCH] bitbake: Allow ~ in bblayers

Quoted 24 lines · Richard Purdie

Quoted 13 lines · Ed Bartosh
Implemented processing of ~ in bblayer's paths if HOME environment variable is approved.
Signed-off-by Ed Bartosh <[email protected]>
---
  bitbake/lib/bb/cookerdata.py | 3 +++
  1 file changed, 3 insertions(+)
bitbake/lib/bb/cookerdata.py +0 -0
index 6c11a60..bf08c63 100644
bitbake/lib/bb/cookerdata.py class CookerDataBuilder(object): · line 275
Quoted 7 lines
bitbake/lib/bb/cookerdata.py +2 -0
@@ -269,8 +269,11 @@ class CookerDataBuilder(object):
              layers = (data.getVar('BBLAYERS', True) or "").split()
 
              data = bb.data.createCopy(data)
+            approved = bb.utils.approved_variables()
              for layer in layers:
                  parselog.debug(2, "Adding layer %s", layer)
+                if 'HOME' in approved:
can we change this to

Quoted 4 lines

if 'HOME' in approved and '~' in layer:
simply so that we can more easily remember what this code is doing in the future?

This change also has the side effect of not causing an expansion of all layers if HOME is in the approved list which would seem a useful optimization.

Quoted 14 lines

Cheers,

Richard

Quoted 7 lines

bitbake/lib/bb/cookerdata.py +1 -1

+                    layer = os.path.expanduser(layer)
                  data.setVar('LAYERDIR', layer)
                  data = parse_config_file(os.path.join(layer, "conf", "layer.conf"), data)
                  data.expandVarref('LAYERDIR')
--

2.1.4

Signature · 4 lines

------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Ed Bartosh

author

2015-04-18 15:10 UTC #9 lore raw

PATCH 1/1 bitbake: Allow ~ in bblayers

In reply to Richard Purdie

Implemented processing of ~ in bblayer's paths if HOME environment variable is approved.

Signed-off-by Ed Bartosh <[email protected]>

---
 bitbake/lib/bb/cookerdata.py | 3 +++
 1 file changed, 3 insertions(+)

bitbake/lib/bb/cookerdata.py +3 -0

index 6c11a60..630ee27 100644
@@ -269,8 +269,11 @@ class CookerDataBuilder(object):
             layers = (data.getVar('BBLAYERS', True) or "").split()
 
             data = bb.data.createCopy(data)
+            approved = bb.utils.approved_variables()
             for layer in layers:
                 parselog.debug(2, "Adding layer %s", layer)
+                if 'HOME' in approved and '~' in layer:
+                    layer = os.path.expanduser(layer)
                 data.setVar('LAYERDIR', layer)
                 data = parse_config_file(os.path.join(layer, "conf", "layer.conf"), data)
                 data.expandVarref('LAYERDIR')

Signature · 1 line

2.1.4