Extensible SDK fixes
cover Extensible SDK fixes
The following changes since commit e3fc8ef152494e9b7cff8de110a784150295f17e:
recipes-support/rng-tools: Change runlevel start from S to 2, 3, 4, 5. (2016-03-30 21:32:11 +0100)
are available in the git repository at:
git://git.openembedded.org/openembedded-core-contrib paule/extsdkfixes10-oe
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=paule/extsdkfixes10-oe
Paul Eggleton (5):
classes/populate_sdk_ext: support setting vars from environment at build time
oe-publish-sdk: prevent specifying a directory for the SDK argument
oe-publish-sdk: exclude sstate-cache if publishing minimal SDK toolchain-shar-extract.sh: ensure all_proxy is allowed through
devtool: modify: call shutdown on tinfoil when done
meta/classes/populate_sdk_ext.bbclass | 26 ++++++++++++++++++++++++++ meta/files/toolchain-shar-extract.sh | 13 ++++++++++--- scripts/lib/devtool/standard.py | 2 ++ scripts/oe-publish-sdk | 13 +++++++------ 4 files changed, 45 insertions(+), 9 deletions(-)
Signature · 1 line
2.5.5
PATCH 1/5 classes/populate_sdk_ext: support setting vars from environment at build time
When running bitbake you may pass in values of variables from the external environment (making use of BB_ENV_EXTRAWHITE), and you may choose to do this when building the extensible SDK, for example:
MACHINE=qemuarm bitbake -c populate_sdk_ext core-image-minimal
You would naturally expect those settings to be reflected in the extensible SDK itself; however they were not, since we were only considering local.conf and auto.conf. Check the variables mentioned in BB_ENV_EXTRAWHITE to see if any are different than the values set in local.conf/auto.conf and add lines setting them in the SDK's local.conf if so.
Fixes [YOCTO #9339].
--- meta/classes/populate_sdk_ext.bbclass | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+)
meta/classes/populate_sdk_ext.bbclass +20 -0
@@ -164,6 +164,9 @@ python copy_buildsystem () { f.write(' $' + '{SDKBASEMETAPATH}/workspace \\\n') f.write(' "\n') + env_whitelist = (d.getVar('BB_ENV_EXTRAWHITE', True) or '').split() + env_whitelist_values = {} + # Create local.conf builddir = d.getVar('TOPDIR', True) if derivative:
@@ -176,6 +179,8 @@ python copy_buildsystem () { newlines.append('# Removed original setting of %s\n' % varname) return None, op, 0, True else: + if varname in env_whitelist: + env_whitelist_values[varname] = origvalue return origvalue, op, 0, True varlist = ['[^#=+ ]*'] with open(builddir + '/conf/local.conf', 'r') as f:
@@ -241,6 +246,21 @@ python copy_buildsystem () { if line.strip() and not line.startswith('#'): f.write(line) + # Ensure any variables set from the external environment (by way of + # BB_ENV_EXTRAWHITE) are set in the SDK's configuration + extralines = [] + for name, value in env_whitelist_values.iteritems(): + actualvalue = d.getVar(name, True) or '' + if value != actualvalue: + extralines.append('%s = "%s"\n' % (name, actualvalue)) + if extralines: + with open(baseoutpath + '/conf/local.conf', 'a') as f: + f.write('\n') + f.write('# Extra settings from environment:\n') + for line in extralines: + f.write(line) + f.write('\n') + # Filter the locked signatures file to just the sstate tasks we are interested in excluded_targets = d.getVar('SDK_TARGETS', True) sigfile = d.getVar('WORKDIR', True) + '/locked-sigs.inc'
Signature · 1 line
2.5.5
PATCH 2/5 oe-publish-sdk: prevent specifying a directory for the SDK argument
In reply to Paul EggletonThe SDK argument is expected to be an installer .sh file; if a directory is specified we can get an ugly failure later on; best to check up front.
Fixes [YOCTO #9065].
--- scripts/oe-publish-sdk | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
scripts/oe-publish-sdk +4 -1
@@ -50,7 +50,10 @@ def publish(args): # Ensure the SDK exists if not os.path.exists(target_sdk): - logger.error("%s doesn't exist" % target_sdk) + logger.error("Specified SDK %s doesn't exist" % target_sdk) + return -1 + if os.path.isdir(target_sdk): + logger.error("%s is a directory - expected path to SDK installer file" % target_sdk) return -1 if ':' in destination:
Signature · 1 line
2.5.5
PATCH 3/5 oe-publish-sdk: exclude sstate-cache if publishing minimal SDK
In reply to Paul EggletonIf SDK_EXT_TYPE is set to "minimal" then the SDK won't contain many sstate artifacts, and you're required to set up an sstate mirror in this case anyway so there's no point publishing the "stub" sstate-cache directory from within the SDK since it won't be useful for update purposes and may be confused with the real sstate-cache.
There is however a possibility that people might publish the real sstate-cache directory under the same output directory provided to oe-publish-sdk, thus deleting it after extracting (as we were doing with other files we wanted to clean up at the end) would be problematic, besides which extracting it and then deleting it is wasteful. Thus, introduce a "-p" command line option to the SDK installer that we can use to tell tar not to extract the items we don't want when publishing. This has the added benefit of mostly keeping references to these in the place they belong i.e. in populate_sdk_ext.bbclass.
--- meta/classes/populate_sdk_ext.bbclass | 6 ++++++ meta/files/toolchain-shar-extract.sh | 11 +++++++++-- scripts/oe-publish-sdk | 8 +++----- 3 files changed, 18 insertions(+), 7 deletions(-)
meta/classes/populate_sdk_ext.bbclass +6 -0
@@ -365,6 +365,12 @@ sdk_ext_preinst() { exit 1 fi SDK_EXTENSIBLE="1" + if [ "$publish" = "1" ] ; then + EXTRA_TAR_OPTIONS="$EXTRA_TAR_OPTIONS --exclude=ext-sdk-prepare.py" + if [ "${SDK_EXT_TYPE}" = "minimal" ] ; then + EXTRA_TAR_OPTIONS="$EXTRA_TAR_OPTIONS --exclude=sstate-cache" + fi + fi } SDK_PRE_INSTALL_COMMAND_task-populate-sdk-ext = "${sdk_ext_preinst}"
meta/files/toolchain-shar-extract.sh +9 -2
@@ -38,12 +38,14 @@ fi DEFAULT_INSTALL_DIR="@SDKPATH@" SUDO_EXEC="" +EXTRA_TAR_OPTIONS="" target_sdk_dir="" answer="" relocate=1 savescripts=0 verbose=0 -while getopts ":yd:nDRS" OPT; do +publish=0 +while getopts ":yd:npDRS" OPT; do case $OPT in y) answer="Y"
@@ -54,6 +56,10 @@ while getopts ":yd:nDRS" OPT; do n) prepare_buildsystem="no" ;; + p) + prepare_buildsystem="no" + publish=1 + ;; D) verbose=1 ;;
@@ -70,6 +76,7 @@ while getopts ":yd:nDRS" OPT; do echo " -d <dir> Install the SDK to <dir>" echo "======== Extensible SDK only options ============" echo " -n Do not prepare the build system" + echo " -p Publish mode (implies -n)" echo "======== Advanced DEBUGGING ONLY OPTIONS ========" echo " -S Save relocation scripts" echo " -R Do not relocate executables"
@@ -181,7 +188,7 @@ fi payload_offset=$(($(grep -na -m1 "^MARKER:$" $0|cut -d':' -f1) + 1)) printf "Extracting SDK..." -tail -n +$payload_offset $0| $SUDO_EXEC tar xJ -C $target_sdk_dir --checkpoint=.2500 || exit 1 +tail -n +$payload_offset $0| $SUDO_EXEC tar xJ -C $target_sdk_dir --checkpoint=.2500 $EXTRA_TAR_OPTIONS || exit 1 echo "done" printf "Setting it up..."
scripts/oe-publish-sdk +3 -5
@@ -94,19 +94,17 @@ def publish(args): # Unpack the SDK logger.info("Unpacking SDK") - cleanupfiles = [dest_sdk, os.path.join(destdir, 'ext-sdk-prepare.py')] if not is_remote: - cmd = "sh %s -n -y -d %s" % (dest_sdk, destination) + cmd = "sh %s -p -y -d %s" % (dest_sdk, destination) ret = subprocess.call(cmd, shell=True) if ret == 0: logger.info('Successfully unpacked %s to %s' % (dest_sdk, destination)) - for cleanupfile in cleanupfiles: - os.remove(cleanupfile) + os.remove(dest_sdk) else: logger.error('Failed to unpack %s to %s' % (dest_sdk, destination)) return ret else: - cmd = "ssh %s 'sh %s -n -y -d %s && rm -f %s'" % (host, dest_sdk, destdir, ' '.join(cleanupfiles)) + cmd = "ssh %s 'sh %s -p -y -d %s && rm -f %s'" % (host, dest_sdk, destdir, dest_sdk) ret = subprocess.call(cmd, shell=True) if ret == 0: logger.info('Successfully unpacked %s to %s' % (dest_sdk, destdir))
Signature · 1 line
2.5.5
PATCH 4/5 toolchain-shar-extract.sh: ensure all_proxy is allowed through
In reply to Paul Eggletonall_proxy is referred to by oe-git-proxy so ensure it is allowed through into the installer environment in case the extensible SDK install process needs to query a remote git repository.
--- meta/files/toolchain-shar-extract.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/files/toolchain-shar-extract.sh +1 -1
@@ -2,7 +2,7 @@ [ -z "$ENVCLEANED" ] && exec /usr/bin/env -i ENVCLEANED=1 HOME="$HOME" \ http_proxy="$http_proxy" https_proxy="$https_proxy" ftp_proxy="$ftp_proxy" \ - no_proxy="$no_proxy" GIT_PROXY_COMMAND="$GIT_PROXY_COMMAND" "$0" "$@" + no_proxy="$no_proxy" all_proxy="$all_proxy" GIT_PROXY_COMMAND="$GIT_PROXY_COMMAND" "$0" "$@" [ -f /etc/environment ] && . /etc/environment export PATH=`echo "$PATH" | sed -e 's/:\.//' -e 's/::/:/'`
Signature · 1 line
2.5.5
PATCH 5/5 devtool: modify: call shutdown on tinfoil when done
In reply to Paul EggletonStrictly speaking we ought to explicitly shut down a tinfoil instance when we're done with it. This doesn't affect modify's operation but is important if you want to be able to call into modify() from another plugin (though anyone doing so should be advised that the function is by no means a stable API and is subject to change in future releases).
--- scripts/lib/devtool/standard.py | 2 ++ 1 file changed, 2 insertions(+)
scripts/lib/devtool/standard.py +2 -0
@@ -805,6 +805,8 @@ def modify(args, config, basepath, workspace): logger.info('Recipe %s now set up to build from %s' % (pn, srctree)) + tinfoil.shutdown() + return 0 def _get_patchset_revs(args, srctree, recipe_path):
Signature · 1 line
2.5.5