Skip to contentSkip to conversation

Extensible SDK: 3 fixes

PATCH Superseded Chen Qi 4 participants 10 messages May 27, 2015 – Jun 9, 2015

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Chen Qi

author
#0 lore raw

cover Extensible SDK: 3 fixes

The first two have been sent before.

The following changes since commit 7ffe10df73cc20d10fcd41b121074445273bd60e:

license_class: license_create_manifest improvment (2015-05-09 22:26:02 +0100)

are available in the git repository at:

git://git.openembedded.org/openembedded-core-contrib ChenQi/ext-sdk-3-fixes
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=ChenQi/ext-sdk-3-fixes

Chen Qi (3):
populate_sdk_ext: install the latest buildtools-tarball
populate_sdk_ext: consider custom configuration in local.conf
copy_buildsystem: make sure bitbake directory is copied

 meta/classes/populate_sdk_ext.bbclass | 27 ++++++++++++++++++++++++++-
 meta/lib/oe/copy_buildsystem.py       |  7 +++----
 2 files changed, 29 insertions(+), 5 deletions(-)
Signature · 1 line
1.9.1
author
#1 lore raw

PATCH 1/3 populate_sdk_ext: install the latest buildtools-tarball

If we do bitbake buildtools-tarball' and then after one day do bitbake core-image-minimal -c populate_sdk_ext', we would meet errors like below.

| install: cannot stat '/buildarea2/chenqi/poky/build-systemd/tmp/deploy/sdk/ poky-glibc-x86_64-buildtools-tarball-core2-64-buildtools-nativesdk-standalone -1.8+snapshot-20150429.sh': No such file or directory

The problem is that the output name for buildtools-tarball has ${DATE} in it. So if populate_sdk_ext task is executed but buildtools-tarball is not rebuilt, the above error appears.

Instead of hardcoding ${DISTRO_VERSION} which consists of ${DATE} in the install_tools() function, we should find the latest buildtools-tarball based on the modification time and install it.

[YOCTO #7674]

Signed-off-by Chen Qi <[email protected]>
---
 meta/classes/populate_sdk_ext.bbclass | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
meta/classes/populate_sdk_ext.bbclass +3 -1
index dc2c58e..2fc4c11 100644
@@ -168,7 +168,9 @@ install_tools() {
 	ln -sr ${SDK_OUTPUT}/${SDKPATH}/${scriptrelpath}/recipetool ${SDK_OUTPUT}/${SDKPATHNATIVE}${bindir_nativesdk}/recipetool
 	touch ${SDK_OUTPUT}/${SDKPATH}/.devtoolbase
 
-	install ${SDK_DEPLOY}/${DISTRO}-${TCLIBC}-${SDK_ARCH}-buildtools-tarball-${TUNE_PKGARCH}-buildtools-nativesdk-standalone-${DISTRO_VERSION}.sh ${SDK_OUTPUT}/${SDKPATH}
+	# find latest buildtools-tarball and install it
+	buildtools_path=`ls -t1 ${SDK_DEPLOY}/${DISTRO}-${TCLIBC}-${SDK_ARCH}-buildtools-tarball-${TUNE_PKGARCH}-buildtools-nativesdk-standalone-*.sh | head -n1`
+	install $buildtools_path ${SDK_OUTPUT}/${SDKPATH}
 
 	install ${SDK_DEPLOY}/${BUILD_ARCH}-nativesdk-libc.tar.bz2 ${SDK_OUTPUT}/${SDKPATH}
 }
Signature · 1 line
1.9.1
author
#5 lore raw

PATCH 2/3 populate_sdk_ext: consider custom configuration in local.conf

In reply to Chen Qi

Copy the contents of local.conf under TOPDIR into the final generated local.conf. In this way, custom settings are also made into the final local.conf like IMAGE_INSTALL, DISTRO_FEATURES, VIRTUAL-RUNTIME_xxx, etc.

Before this change, installing extensible SDK would usually report failure when preparing the build system if the user has custom configuration for DISTRO_FEATURES in local.conf. Also, items in IMAGE_INSTALL_append in local.conf also don't get built correctly.

This patch solves the above problem.

A blacklist mechanism is also introduced so that we can blacklist variables that should not be copied into the final local.conf file. Currently, the blacklist contains 'TMPDIR', 'SSTATE_DIR', 'DL_DIR', 'STAMPS_DIR', 'BASE_WORKDIR' and 'DEPLOY_DIR'. What these variables have in common is that they are set in bitbake.conf using '?=' or '??='.

In addition, we check to avoid any setting that might lead to host path bleeding into SDK's configuration.

[YOCTO #7616]

Signed-off-by Chen Qi <[email protected]>
---
 meta/classes/populate_sdk_ext.bbclass | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)
meta/classes/populate_sdk_ext.bbclass +23 -0
index 2fc4c11..49ba26b 100644
@@ -16,6 +16,7 @@ SDK_RDEPENDS_append_task-populate-sdk-ext = " ${SDK_TARGETS}"
 SDK_RELOCATE_AFTER_INSTALL_task-populate-sdk-ext = "0"
 
 SDK_META_CONF_WHITELIST ?= "MACHINE DISTRO PACKAGE_CLASSES"
+SDK_META_CONF_BLACKLIST ?= "TMPDIR DL_DIR SSTATE_DIR STAMPS_DIR BASE_WORKDIR DEPLOY_DIR"
 
 SDK_TARGETS ?= "${PN}"
 OE_INIT_ENV_SCRIPT ?= "oe-init-build-env"

@@ -114,6 +115,28 @@ python copy_buildsystem () {
         f.write('# this configuration provides, it is strongly suggested that you set\n')
         f.write('# up a proper instance of the full build system and use that instead.\n\n')
 
+        # Copy configurations from the current local.conf
+        builddir = d.getVar('TOPDIR', True)
+        with open(builddir + '/conf/local.conf', 'r') as lf:
+            varblacklist = d.getVar('SDK_META_CONF_BLACKLIST', True).split()
+            skip = False
+            for line in lf:
+                line = line.lstrip()
+                if line.startswith('#'):
+                    continue
+                # avoid host path bleeding into SDK configuration
+                if line.find('"/') != -1:
+                    continue
+                for varname in varblacklist:
+                    if line.startswith(varname):
+                        skip = True
+                        break
+                if not skip:
+                    f.write(line)
+                skip = False
+        f.write('\n')
+
+        # Configurations in local.conf which are specific for extensible SDK
         f.write('INHERIT += "%s"\n\n' % 'uninative')
         f.write('CONF_VERSION = "%s"\n\n' % d.getVar('CONF_VERSION'))
Signature · 1 line
1.9.1
author
#9 lore raw

PATCH 3/3 copy_buildsystem: make sure bitbake directory is copied

In reply to Chen Qi

The previous code assumes that bitbake/ directory is under the core layer. This is the case for Yocto project. But users might clone oe-core and bitbake separately. So we use bb.__file__ to locate the bitbake directory to make sure it's copied into the extensible SDK.

Signed-off-by Chen Qi <[email protected]>
---
 meta/lib/oe/copy_buildsystem.py | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)
meta/lib/oe/copy_buildsystem.py +3 -4
index cf7fada..979578c 100644
@@ -28,11 +28,10 @@ class BuildSystem(object):
         layers.append(corebase)
 
         corebase_files = self.d.getVar('COREBASE_FILES', True).split()
-
-        # bitbake belongs in corebase so make sure it goes there
-        if "bitbake" not in corebase_files:
-            corebase_files.append("bitbake")
         corebase_files = [corebase + '/' +x for x in corebase_files]
+        # Make sure bitbake goes in
+        bitbake_dir = bb.__file__.rsplit('/', 3)[0]
+        corebase_files.append(bitbake_dir)
 
         for layer in layers:
             layerconf = os.path.join(layer, 'conf', 'layer.conf')
Signature · 1 line
1.9.1
#2 lore raw

Re: [PATCH 1/3] populate_sdk_ext: install the latest buildtools-tarball

In reply to Chen Qi
Quoted 15 lines · Chen Qi

If we do bitbake buildtools-tarball' and then after one day do bitbake core-image-minimal -c populate_sdk_ext', we would meet errors like below.

| install: cannot stat '/buildarea2/chenqi/poky/build-systemd/tmp/deploy/sdk/

poky-glibc-x86_64-buildtools-tarball-core2-64-buildtools-nativesdk-standalone -1.8+snapshot-20150429.sh': No such file or directory

The problem is that the output name for buildtools-tarball has ${DATE} in it.
So if populate_sdk_ext task is executed but buildtools-tarball is not rebuilt,
the above error appears.

Quoted 9 lines

Instead of hardcoding ${DISTRO_VERSION} which consists of ${DATE} in the install_tools() function, we should find the latest buildtools-tarball based
on the modification time and install it.

[YOCTO #7674]

Signed-off-by Chen Qi <[email protected]>

Hmm, if buildtools-tarball is required, why not just pull it in via dependencies and build a current one, rather than assuming/forcing the user to have done so themselves?

Signature · 5 lines
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics
#3 lore raw

Re: [PATCH 1/3] populate_sdk_ext: install the latest buildtools-tarball

Quoted 22 lines · Christopher Larson
Quoted 20 lines · Chen Qi

If we do bitbake buildtools-tarball' and then after one day do bitbake core-image-minimal -c populate_sdk_ext', we would meet errors like below.

| install: cannot stat '/buildarea2/chenqi/poky/build-systemd/tmp/deploy/sdk/

poky-glibc-x86_64-buildtools-tarball-core2-64-buildtools-nativesdk-standalone -1.8+snapshot-20150429.sh': No such file or directory

The problem is that the output name for buildtools-tarball has ${DATE} in it.
So if populate_sdk_ext task is executed but buildtools-tarball is not rebuilt,
the above error appears.

Instead of hardcoding ${DISTRO_VERSION} which consists of ${DATE} in the install_tools() function, we should find the latest buildtools-tarball based
on the modification time and install it.

Quoted 8 lines
Quoted 4 lines

[YOCTO #7674]

Signed-off-by Chen Qi <[email protected]>

Hmm, if buildtools-tarball is required, why not just pull it in via dependencies and build a current one, rather than assuming/forcing the user to have done so themselves?

It is a dependency and will be built if it hasn't been built. But since

TOOLCHAIN_OUTPUTNAME ?= "${SDK_NAME}-buildtools-nativesdk-standalone-${DISTRO_VERSION}"

in buildtools-tarball.bb the filename will be different each time we try to copy it, unless we force buildtools-tarball to be rebuilt each time the sdk is constructed.

${DATE} being used in DISTRO_VERSION is misleading in my opinion because different dates don't necessarily mean different contents. So we could change DISTRO_VERSION to not use date, or buildtools-tarball.bb to not use DISTRO_VERSION.

Quoted 2 lines
#4 lore raw

Re: [PATCH 1/3] populate_sdk_ext: install the latest buildtools-tarball

Quoted 41 lines · Randy Witt
Quoted 25 lines · Christopher Larson

On Tue, May 26, 2015 at 11:09 PM, Chen Qi <[email protected]> wrote:

If we do bitbake buildtools-tarball' and then after one day do bitbake

Quoted 22 lines

core-image-minimal -c populate_sdk_ext', we would meet errors like below.

| install: cannot stat '/buildarea2/chenqi/poky/build-systemd/tmp/deploy/sdk/

poky-glibc-x86_64-buildtools-tarball-core2-64-buildtools-nativesdk-standalone -1.8+snapshot-20150429.sh': No such file or directory

The problem is that the output name for buildtools-tarball has ${DATE} in it.
So if populate_sdk_ext task is executed but buildtools-tarball is not rebuilt,
the above error appears.

Instead of hardcoding ${DISTRO_VERSION} which consists of ${DATE} in the install_tools() function, we should find the latest buildtools-tarball based
on the modification time and install it.

[YOCTO #7674]

Quoted 8 lines
Quoted 3 lines
Signed-off-by Chen Qi <[email protected]>

Hmm, if buildtools-tarball is required, why not just pull it in via dependencies and build a current one, rather than assuming/forcing the user
to have done so themselves?

It is a dependency and will be built if it hasn't been built. But since

TOOLCHAIN_OUTPUTNAME ?= "${SDK_NAME}-buildtools-nativesdk-standalone-${DISTRO_VERSION}"

Quoted 8 lines

in buildtools-tarball.bb the filename will be different each time we try to copy it, unless we force buildtools-tarball to be rebuilt each time the sdk is constructed.

${DATE} being used in DISTRO_VERSION is misleading in my opinion because different dates don't necessarily mean different contents. So we could change DISTRO_VERSION to not use date, or buildtools-tarball.bb to not use DISTRO_VERSION.

I expect it’d be good if in this case DATE wasn’t whitelisted, so recipes which use a DISTRO_VERSION which includes a date get rebuilt for a new day, and if you don’t want that, you can remove DATE from DISTRO_VERSION.

Actually, it’d be interesting to try to rig some form of auto-incrementing distro version based on distro metadata checksum changes or something.. Hmm.

Signature · 5 lines
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics
#6 lore raw

Re: [PATCH 2/3] populate_sdk_ext: consider custom configuration in local.conf

In reply to Chen Qi

Hi Qi,

Quoted 68 lines · On Wednesday 27 May 2015 14:09:39 Chen Qi wrote:

Copy the contents of local.conf under TOPDIR into the final generated local.conf. In this way, custom settings are also made into the final local.conf like IMAGE_INSTALL, DISTRO_FEATURES, VIRTUAL-RUNTIME_xxx, etc.

Before this change, installing extensible SDK would usually report failure when preparing the build system if the user has custom configuration for DISTRO_FEATURES in local.conf. Also, items in IMAGE_INSTALL_append in local.conf also don't get built correctly.

This patch solves the above problem.

A blacklist mechanism is also introduced so that we can blacklist variables that should not be copied into the final local.conf file. Currently, the blacklist contains 'TMPDIR', 'SSTATE_DIR', 'DL_DIR', 'STAMPS_DIR', 'BASE_WORKDIR' and 'DEPLOY_DIR'. What these variables have in common is that they are set in bitbake.conf using '?=' or '??='.

In addition, we check to avoid any setting that might lead to host path bleeding into SDK's configuration.

[YOCTO #7616]

Signed-off-by Chen Qi <[email protected]>
---
 meta/classes/populate_sdk_ext.bbclass | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)
(quoted diff) +0 -0

b/meta/classes/populate_sdk_ext.bbclass index 2fc4c11..49ba26b 100644

meta/classes/populate_sdk_ext.bbclass +0 -0
@@ -16,6 +16,7 @@ SDK_RDEPENDS_append_task-populate-sdk-ext = "

${SDK_TARGETS}" SDK_RELOCATE_AFTER_INSTALL_task-populate-sdk-ext = "0"

SDK_META_CONF_WHITELIST ?= "MACHINE DISTRO PACKAGE_CLASSES"
+SDK_META_CONF_BLACKLIST ?= "TMPDIR DL_DIR SSTATE_DIR STAMPS_DIR BASE_WORKDIR DEPLOY_DIR"

SDK_TARGETS ?= "${PN}"
OE_INIT_ENV_SCRIPT ?= "oe-init-build-env"

meta/classes/populate_sdk_ext.bbclass +0 -0
@@ -114,6 +115,28 @@ python copy_buildsystem () {
         f.write('# this configuration provides, it is strongly suggested

that you set\n') f.write('# up a proper instance of the full build system and use that instead.\n\n')

meta/classes/populate_sdk_ext.bbclass +4 -0
+        # Copy configurations from the current local.conf
+        builddir = d.getVar('TOPDIR', True)
+        with open(builddir + '/conf/local.conf', 'r') as lf:
+            varblacklist = d.getVar('SDK_META_CONF_BLACKLIST',

True).split() + skip = False

meta/classes/populate_sdk_ext.bbclass +17 -0
+            for line in lf:
+                line = line.lstrip()
+                if line.startswith('#'):
+                    continue
+                # avoid host path bleeding into SDK configuration
+                if line.find('"/') != -1:
+                    continue
+                for varname in varblacklist:
+                    if line.startswith(varname):
+                        skip = True
+                        break
+                if not skip:
+                    f.write(line)
+                skip = False
+        f.write('\n')
+
+        # Configurations in local.conf which are specific for extensible

SDK f.write('INHERIT += "%s"\n\n' % 'uninative')

Quoted 1 line
         f.write('CONF_VERSION = "%s"\n\n' % d.getVar('CONF_VERSION'))

I've been thinking about this; since I've recently added edit_metadata() in bitbake's lib/bb/utils.py, it would make sense to use that rather than adding another piece of code that effectively parses bitbake configuration files (though I know this isn't adding much). I know this series has been going back and forth for a while now and the delays have largely been my fault, so I'd be happy to make this change and resend the series - it's up to you.

Cheers,
Paul

Signature · 2 lines
Paul Eggleton
Intel Open Source Technology Centre
author
#7 lore raw

Re: [PATCH 2/3] populate_sdk_ext: consider custom configuration in local.conf

Quoted 79 lines · Paul Eggleton

Hi Qi,

Quoted 68 lines · On Wednesday 27 May 2015 14:09:39 Chen Qi wrote:

Copy the contents of local.conf under TOPDIR into the final generated local.conf. In this way, custom settings are also made into the final local.conf like IMAGE_INSTALL, DISTRO_FEATURES, VIRTUAL-RUNTIME_xxx, etc.

Before this change, installing extensible SDK would usually report failure when preparing the build system if the user has custom configuration for DISTRO_FEATURES in local.conf. Also, items in IMAGE_INSTALL_append in local.conf also don't get built correctly.

This patch solves the above problem.

A blacklist mechanism is also introduced so that we can blacklist variables that should not be copied into the final local.conf file. Currently, the blacklist contains 'TMPDIR', 'SSTATE_DIR', 'DL_DIR', 'STAMPS_DIR', 'BASE_WORKDIR' and 'DEPLOY_DIR'. What these variables have in common is that they are set in bitbake.conf using '?=' or '??='.

In addition, we check to avoid any setting that might lead to host path bleeding into SDK's configuration.

[YOCTO #7616]

Signed-off-by Chen Qi <[email protected]>
---
  meta/classes/populate_sdk_ext.bbclass | 23 +++++++++++++++++++++++
  1 file changed, 23 insertions(+)
(quoted diff) +0 -0

b/meta/classes/populate_sdk_ext.bbclass index 2fc4c11..49ba26b 100644

meta/classes/populate_sdk_ext.bbclass +0 -0
@@ -16,6 +16,7 @@ SDK_RDEPENDS_append_task-populate-sdk-ext = "

${SDK_TARGETS}" SDK_RELOCATE_AFTER_INSTALL_task-populate-sdk-ext = "0"

SDK_META_CONF_WHITELIST ?= "MACHINE DISTRO PACKAGE_CLASSES"
+SDK_META_CONF_BLACKLIST ?= "TMPDIR DL_DIR SSTATE_DIR STAMPS_DIR BASE_WORKDIR DEPLOY_DIR"

  SDK_TARGETS ?= "${PN}"
  OE_INIT_ENV_SCRIPT ?= "oe-init-build-env"
meta/classes/populate_sdk_ext.bbclass +0 -0
@@ -114,6 +115,28 @@ python copy_buildsystem () {
          f.write('# this configuration provides, it is strongly suggested

that you set\n') f.write('# up a proper instance of the full build system and use that instead.\n\n')

meta/classes/populate_sdk_ext.bbclass +4 -0
+        # Copy configurations from the current local.conf
+        builddir = d.getVar('TOPDIR', True)
+        with open(builddir + '/conf/local.conf', 'r') as lf:
+            varblacklist = d.getVar('SDK_META_CONF_BLACKLIST',

True).split() + skip = False

meta/classes/populate_sdk_ext.bbclass +17 -0
+            for line in lf:
+                line = line.lstrip()
+                if line.startswith('#'):
+                    continue
+                # avoid host path bleeding into SDK configuration
+                if line.find('"/') != -1:
+                    continue
+                for varname in varblacklist:
+                    if line.startswith(varname):
+                        skip = True
+                        break
+                if not skip:
+                    f.write(line)
+                skip = False
+        f.write('\n')
+
+        # Configurations in local.conf which are specific for extensible

SDK f.write('INHERIT += "%s"\n\n' % 'uninative')

Quoted 1 line
          f.write('CONF_VERSION = "%s"\n\n' % d.getVar('CONF_VERSION'))

I've been thinking about this; since I've recently added edit_metadata() in bitbake's lib/bb/utils.py, it would make sense to use that rather than adding another piece of code that effectively parses bitbake configuration files (though I know this isn't adding much). I know this series has been going back and forth for a while now and the delays have largely been my fault, so I'd be happy to make this change and resend the series - it's up to you.

Quoted 3 lines

Cheers,
Paul

Hi Paul,

Thanks for your suggestion.
I've been looking into edit_metadata(). But it seems that it's used to modify *specific* variables.
In this case, we don't know which variables we are going to skip in local.conf. We just know that they start with '/'. So I don't see how to do it. I'm not sure if I understand things wrong. Please point it out directly if I've missed something.

Best Regards,
Chen Qi

#8 lore raw

Re: [PATCH 2/3] populate_sdk_ext: consider custom configuration in local.conf

Quoted 93 lines · On Tuesday 09 June 2015 14:41:53 ChenQi wrote:
Quoted 89 lines · Paul Eggleton
Quoted 77 lines · On Wednesday 27 May 2015 14:09:39 Chen Qi wrote:

Copy the contents of local.conf under TOPDIR into the final generated local.conf. In this way, custom settings are also made into the final local.conf like IMAGE_INSTALL, DISTRO_FEATURES, VIRTUAL-RUNTIME_xxx, etc.

Before this change, installing extensible SDK would usually report failure
when preparing the build system if the user has custom configuration for DISTRO_FEATURES in local.conf. Also, items in IMAGE_INSTALL_append in local.conf also don't get built correctly.

This patch solves the above problem.

A blacklist mechanism is also introduced so that we can blacklist variables
that should not be copied into the final local.conf file. Currently, the blacklist contains 'TMPDIR', 'SSTATE_DIR', 'DL_DIR', 'STAMPS_DIR', 'BASE_WORKDIR' and 'DEPLOY_DIR'. What these variables have in common is that they are set in bitbake.conf using '?=' or '??='.

In addition, we check to avoid any setting that might lead to host path bleeding into SDK's configuration.

[YOCTO #7616]

Signed-off-by Chen Qi <[email protected]>

---

  meta/classes/populate_sdk_ext.bbclass | 23 +++++++++++++++++++++++
  1 file changed, 23 insertions(+)
(quoted diff) +0 -0

b/meta/classes/populate_sdk_ext.bbclass index 2fc4c11..49ba26b 100644

meta/classes/populate_sdk_ext.bbclass +0 -0
@@ -16,6 +16,7 @@ SDK_RDEPENDS_append_task-populate-sdk-ext = "

${SDK_TARGETS}" SDK_RELOCATE_AFTER_INSTALL_task-populate-sdk-ext = "0"

SDK_META_CONF_WHITELIST ?= "MACHINE DISTRO PACKAGE_CLASSES"

+SDK_META_CONF_BLACKLIST ?= "TMPDIR DL_DIR SSTATE_DIR STAMPS_DIR BASE_WORKDIR DEPLOY_DIR"

  SDK_TARGETS ?= "${PN}"
  OE_INIT_ENV_SCRIPT ?= "oe-init-build-env"
meta/classes/populate_sdk_ext.bbclass +0 -0
@@ -114,6 +115,28 @@ python copy_buildsystem () {
 
          f.write('# this configuration provides, it is strongly
          suggested

that you set\n') f.write('# up a proper instance of the full build system and use that instead.\n\n')

meta/classes/populate_sdk_ext.bbclass +4 -0
+        # Copy configurations from the current local.conf
+        builddir = d.getVar('TOPDIR', True)
+        with open(builddir + '/conf/local.conf', 'r') as lf:
+            varblacklist = d.getVar('SDK_META_CONF_BLACKLIST',

True).split() + skip = False

meta/classes/populate_sdk_ext.bbclass +17 -0
+            for line in lf:
+                line = line.lstrip()
+                if line.startswith('#'):
+                    continue
+                # avoid host path bleeding into SDK configuration
+                if line.find('"/') != -1:
+                    continue
+                for varname in varblacklist:
+                    if line.startswith(varname):
+                        skip = True
+                        break
+                if not skip:
+                    f.write(line)
+                skip = False
+        f.write('\n')
+
+        # Configurations in local.conf which are specific for extensible

SDK f.write('INHERIT += "%s"\n\n' % 'uninative')

Quoted 1 line
          f.write('CONF_VERSION = "%s"\n\n' % d.getVar('CONF_VERSION'))

I've been thinking about this; since I've recently added edit_metadata() in
bitbake's lib/bb/utils.py, it would make sense to use that rather than adding another piece of code that effectively parses bitbake configuration files (though I know this isn't adding much). I know this series has been going back and forth for a while now and the delays have largely been my fault, so I'd be happy to make this change and resend the series - it's up to you.

Quoted 2 lines

Cheers,
Paul

Quoted 8 lines

Hi Paul,

Thanks for your suggestion.
I've been looking into edit_metadata(). But it seems that it's used to modify *specific* variables.
In this case, we don't know which variables we are going to skip in local.conf. We just know that they start with '/'. So I don't see how to do it.

Well, what you pass in actually goes into a regex, so you can just match any variable with it - you just need to take care that it doesn't match characters it shouldn't. Here's an example I just tested - you can place it in scripts/ in order to try it out:

------------- snip ------------
import sys
import os

scripts_path = os.path.dirname(os.path.realpath(__file__)) lib_path = scripts_path + '/lib'
sys.path = sys.path + [lib_path]

import scriptpath
bitbakepath = scriptpath.add_bitbake_lib_path()

import bb.utils

whitelist = ['BBMASK']
def handle_var(varname, origvalue, op, newlines):
print '%s "%s"' % (varname, origvalue)
if '/' in origvalue and (not ':/' in value or varname in whitelist):
newlines.append('# Removed original setting of %s\n' % varname) return None, op, 0, True
else:
return origvalue, op, 0, True

varlist = ['[^#=+ ]*']
with open('conf/local.conf', 'r') as f:
oldlines = f.readlines()
(updated, newlines) = bb.utils.edit_metadata(oldlines, varlist, handle_var)

with open('conf/local.conf.newedit', 'w') as f:
for line in newlines:
f.write(line)

------------- snip ------------

We really ought not to have to care about matching #, that's probably a bug in edit_metadata(). While looking at my own local.conf I noticed that BBMASK might have / in it and we'd probably want to allow it through; I don't know if there are other reasonable examples. I suspect we ought to have a variable for this to allow for some flexibility. As in the example we probably also want to default to allowing variable values that contain URLs as well.

Cheers,
Paul

Signature · 2 lines
Paul Eggleton
Intel Open Source Technology Centre