Extensible SDK fixes
cover Extensible SDK fixes
Fixes for the extensible SDK and related items.
The following changes since commit 6f98c39418c60b7c0b25b30983d2e5257158a6a4:
gcc: Drop 4.8 (2015-11-16 14:59:07 +0000)
are available in the git repository at:
git://git.openembedded.org/openembedded-core-contrib paule/extsdkfixes2
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=paule/extsdkfixes2
Paul Eggleton (7):
scripts/gen-lockedsig-cache: improve output nativesdk-buildtools-perl-dummy: fix rebuilding when SDKMACHINE changes toolchain-shar-extract.sh: do not allow $ in paths for ext SDK classes/populate_sdk_ext: tidy up preparation log file writing classes/populate_sdk_ext: make it clear when SDK installation has failed classes/populate_sdk_ext: tweak reporting of workspace exclusion classes/populate_sdk_ext: fail if SDK_ARCH != BUILD_ARCH
meta/classes/populate_sdk_ext.bbclass | 10 ++++++++-- meta/files/toolchain-shar-extract.sh | 4 ++-- meta/lib/oe/copy_buildsystem.py | 5 +++-- meta/recipes-core/meta/nativesdk-buildtools-perl-dummy.bb | 11 +++++++++-- scripts/gen-lockedsig-cache | 13 ++++++++++--- 5 files changed, 32 insertions(+), 11 deletions(-)
Signature · 1 line
2.1.0
PATCH 1/7 scripts/gen-lockedsig-cache: improve output
* Print some status when running
* When incorrect number of arguments specified, print usage text
--- scripts/gen-lockedsig-cache | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-)
scripts/gen-lockedsig-cache +10 -3
@@ -1,7 +1,4 @@ #!/usr/bin/env python -# -# gen-lockedsig-cache <locked-sigs.inc> <input-cachedir> <output-cachedir> -# import os import sys
@@ -18,14 +15,17 @@ def mkdir(d): if len(sys.argv) < 3: print("Incorrect number of arguments specified") + print("syntax: gen-lockedsig-cache <locked-sigs.inc> <input-cachedir> <output-cachedir>") sys.exit(1) +print('Reading %s' % sys.argv[1]) sigs = [] with open(sys.argv[1]) as f: for l in f.readlines(): if ":" in l: sigs.append(l.split(":")[2].split()[0]) +print('Gathering file list') files = set() for s in sigs: p = sys.argv[2] + "/" + s[:2] + "/*" + s + "*"
@@ -33,10 +33,13 @@ for s in sigs: p = sys.argv[2] + "/*/" + s[:2] + "/*" + s + "*" files |= set(glob.glob(p)) +print('Processing files') for f in files: + sys.stdout.write('Processing %s... ' % f) _, ext = os.path.splitext(f) if not ext in ['.tgz', '.siginfo', '.sig']: # Most likely a temp file, skip it + print('skipping') continue dst = f.replace(sys.argv[2], sys.argv[3]) destdir = os.path.dirname(dst)
@@ -45,6 +48,10 @@ for f in files: if os.path.exists(dst): os.remove(dst) if (os.stat(f).st_dev == os.stat(destdir).st_dev): + print('linking') os.link(f, dst) else: + print('copying') shutil.copyfile(f, dst) + +print('Done!')
Signature · 1 line
2.1.0
PATCH 2/7 nativesdk-buildtools-perl-dummy: fix rebuilding when SDKMACHINE changes
In reply to Paul EggletonThis recipe produces an empty dummy package (in order to satisfy dependencies on perl so we don't have perl within buildtools-tarball). Because we were inheriting nativesdk here the recipe was being rebuilt, but having forced PACKAGE_ARCH to a particular value the packages for each architecture were stepping on eachother. Since the packages are empty they can in fact be allarch (even though they won't actually go into the "all" package feed). It turns out that nheriting nativesdk wasn't actually necessary either, so drop that.
Fixes [YOCTO #8509].
--- meta/recipes-core/meta/nativesdk-buildtools-perl-dummy.bb | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-)
meta/recipes-core/meta/nativesdk-buildtools-perl-dummy.bb +9 -2
@@ -2,9 +2,16 @@ SUMMARY = "Dummy package which ensures perl is excluded from buildtools" LICENSE = "MIT" LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302" -inherit nativesdk +inherit allarch + +python() { + # Put the package somewhere separate to ensure it's never used except + # when we want it + # (note that we have to do this in anonymous python here to avoid + # allarch.bbclass disabling itself) + d.setVar('PACKAGE_ARCH', 'buildtools-dummy-${SDKPKGSUFFIX}') +} -# Put it somewhere separate to ensure it's never used except when we want it PACKAGE_ARCH = "buildtools-dummy-${SDKPKGSUFFIX}" PERLPACKAGES = "nativesdk-perl \
Signature · 1 line
2.1.0
PATCH 3/7 toolchain-shar-extract.sh: do not allow $ in paths for ext SDK
In reply to Paul EggletonIf you put an $ character in the path, SDK installation fails during the preparation stage, so add this to the disallowed characters.
Fixes [YOCTO #8625].
--- meta/files/toolchain-shar-extract.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
meta/files/toolchain-shar-extract.sh +2 -2
@@ -101,9 +101,9 @@ fi if [ "$SDK_EXTENSIBLE" = "1" ]; then # We're going to be running the build system, additional restrictions apply - if echo "$target_sdk_dir" | grep -q '[+\ @]'; then + if echo "$target_sdk_dir" | grep -q '[+\ @$]'; then echo "The target directory path ($target_sdk_dir) contains illegal" \ - "characters such as spaces, @ or +. Abort!" + "characters such as spaces, @, \$ or +. Abort!" exit 1 fi else
Signature · 1 line
2.1.0
PATCH 4/7 classes/populate_sdk_ext: tidy up preparation log file writing
In reply to Paul EggletonUse a variable for the log file which includes the full path; this is not only neater but avoids us writing the first part (the output of oe-init-build-env) to a file in another directory since we are changing directory as part of this subshell.
--- meta/classes/populate_sdk_ext.bbclass | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
meta/classes/populate_sdk_ext.bbclass +2 -1
@@ -249,7 +249,8 @@ sdk_ext_postinst() { # dash which is /bin/sh on Ubuntu will not preserve the # current working directory when first ran, nor will it set $1 when # sourcing a script. That is why this has to look so ugly. - sh -c ". buildtools/environment-setup* > preparing_build_system.log && cd $target_sdk_dir/`dirname ${oe_init_build_env_path}` && set $target_sdk_dir && . $target_sdk_dir/${oe_init_build_env_path} $target_sdk_dir >> preparing_build_system.log && $target_sdk_dir/ext-sdk-prepare.sh $target_sdk_dir '${SDK_TARGETS}' >> preparing_build_system.log 2>&1" || { echo "SDK preparation failed: see `pwd`/preparing_build_system.log" ; exit 1 ; } + LOGFILE="$target_sdk_dir/preparing_build_system.log" + sh -c ". buildtools/environment-setup* > $LOGFILE && cd $target_sdk_dir/`dirname ${oe_init_build_env_path}` && set $target_sdk_dir && . $target_sdk_dir/${oe_init_build_env_path} $target_sdk_dir >> $LOGFILE && $target_sdk_dir/ext-sdk-prepare.sh $target_sdk_dir '${SDK_TARGETS}' >> $LOGFILE 2>&1" || { echo "SDK preparation failed: see $LOGFILE" ; exit 1 ; } fi rm -f $target_sdk_dir/ext-sdk-prepare.sh echo done
Signature · 1 line
2.1.0
PATCH 5/7 classes/populate_sdk_ext: make it clear when SDK installation has failed
In reply to Paul EggletonWhen SDK preparation fails:
* Insert an ERROR: in front of the error message
* Add an error message to the environment setup script
Hopefully this should make it more obvious when this happens.
Fixes [YOCTO #8658].
--- meta/classes/populate_sdk_ext.bbclass | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes/populate_sdk_ext.bbclass +1 -1
@@ -250,7 +250,7 @@ sdk_ext_postinst() { # current working directory when first ran, nor will it set $1 when # sourcing a script. That is why this has to look so ugly. LOGFILE="$target_sdk_dir/preparing_build_system.log" - sh -c ". buildtools/environment-setup* > $LOGFILE && cd $target_sdk_dir/`dirname ${oe_init_build_env_path}` && set $target_sdk_dir && . $target_sdk_dir/${oe_init_build_env_path} $target_sdk_dir >> $LOGFILE && $target_sdk_dir/ext-sdk-prepare.sh $target_sdk_dir '${SDK_TARGETS}' >> $LOGFILE 2>&1" || { echo "SDK preparation failed: see $LOGFILE" ; exit 1 ; } + sh -c ". buildtools/environment-setup* > $LOGFILE && cd $target_sdk_dir/`dirname ${oe_init_build_env_path}` && set $target_sdk_dir && . $target_sdk_dir/${oe_init_build_env_path} $target_sdk_dir >> $LOGFILE && $target_sdk_dir/ext-sdk-prepare.sh $target_sdk_dir '${SDK_TARGETS}' >> $LOGFILE 2>&1" || { echo "ERROR: SDK preparation failed: see $LOGFILE"; echo "printf 'ERROR: this SDK wasn't fully installed and needs reinstalling\n'" >> $env_setup_script ; exit 1 ; } fi rm -f $target_sdk_dir/ext-sdk-prepare.sh echo done
Signature · 1 line
2.1.0
PATCH 6/7 classes/populate_sdk_ext: tweak reporting of workspace exclusion
In reply to Paul EggletonIf you have a local workspace layer enabled when building the extensible SDK, we explicitly exclude that from the SDK (mostly because the SDK has its own for the user to use). Adjust the message we print notifying the user of this so it's clear that we're excluding it from the SDK, and scale it back from a warning to a note printed with bb.plain().
--- meta/classes/populate_sdk_ext.bbclass | 2 +- meta/lib/oe/copy_buildsystem.py | 5 +++-- 2 files changed, 4 insertions(+), 3 deletions(-)
meta/classes/populate_sdk_ext.bbclass +1 -1
@@ -51,7 +51,7 @@ python copy_buildsystem () { core_meta_subdir = '' # Copy in all metadata layers + bitbake (as repositories) - buildsystem = oe.copy_buildsystem.BuildSystem(d) + buildsystem = oe.copy_buildsystem.BuildSystem('extensible SDK', d) baseoutpath = d.getVar('SDK_OUTPUT', True) + '/' + d.getVar('SDKPATH', True) layers_copied = buildsystem.copy_bitbake_and_layers(baseoutpath + '/layers')
meta/lib/oe/copy_buildsystem.py +3 -2
@@ -14,8 +14,9 @@ def _smart_copy(src, dest): shutil.copymode(src, dest) class BuildSystem(object): - def __init__(self, d): + def __init__(self, context, d): self.d = d + self.context = context self.layerdirs = d.getVar('BBLAYERS', True).split() def copy_bitbake_and_layers(self, destdir):
@@ -38,7 +39,7 @@ class BuildSystem(object): if os.path.exists(layerconf): with open(layerconf, 'r') as f: if f.readline().startswith("# ### workspace layer auto-generated by devtool ###"): - bb.warn("Skipping local workspace layer %s" % layer) + bb.plain("NOTE: Excluding local workspace layer %s from %s" % (layer, self.context)) continue # If the layer was already under corebase, leave it there
Signature · 1 line
2.1.0
PATCH 7/7 classes/populate_sdk_ext: fail if SDK_ARCH != BUILD_ARCH
In reply to Paul EggletonThe extensible SDK relies upon uninative, and with the way that uninative works, the build system architecture must be the same as the SDK architecture or the extensible SDK won't be usable. At some point in future hopefully we can remove this limitation, but until then it's disingenuous to allow this to build, so add a check to ensure SDK_ARCH == BUILD_ARCH and fail if it isn't.
--- meta/classes/populate_sdk_ext.bbclass | 5 +++++ 1 file changed, 5 insertions(+)
meta/classes/populate_sdk_ext.bbclass +5 -0
@@ -261,6 +261,11 @@ SDK_POST_INSTALL_COMMAND_task-populate-sdk-ext = "${sdk_ext_postinst}" SDK_POSTPROCESS_COMMAND_prepend_task-populate-sdk-ext = "copy_buildsystem; install_tools; " fakeroot python do_populate_sdk_ext() { + # FIXME hopefully we can remove this restriction at some point, but uninative + # currently forces this upon us + if d.getVar('SDK_ARCH', True) != d.getVar('BUILD_ARCH', True): + bb.fatal('The extensible SDK can currently only be built for the same architecture as the machine being built on - SDK_ARCH is set to %s (likely via setting SDKMACHINE) which is different from the architecture of the build machine (%s). Unable to continue.' % (d.getVar('SDK_ARCH', True), d.getVar('BUILD_ARCH', True))) + bb.build.exec_func("do_populate_sdk", d) }
Signature · 1 line
2.1.0