Skip to contentSkip to conversation

Extensible SDK fixes

PATCH New Paul Eggleton 1 participant 8 messages Nov 23, 2015

lore
Share & tools
lore
#0 lore raw

cover Extensible SDK fixes

Fixes for the extensible SDK and related items.

The following changes since commit 6f98c39418c60b7c0b25b30983d2e5257158a6a4:

gcc: Drop 4.8 (2015-11-16 14:59:07 +0000)

are available in the git repository at:

git://git.openembedded.org/openembedded-core-contrib paule/extsdkfixes2
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=paule/extsdkfixes2

Paul Eggleton (7):
scripts/gen-lockedsig-cache: improve output nativesdk-buildtools-perl-dummy: fix rebuilding when SDKMACHINE changes toolchain-shar-extract.sh: do not allow $ in paths for ext SDK classes/populate_sdk_ext: tidy up preparation log file writing classes/populate_sdk_ext: make it clear when SDK installation has failed classes/populate_sdk_ext: tweak reporting of workspace exclusion classes/populate_sdk_ext: fail if SDK_ARCH != BUILD_ARCH

 meta/classes/populate_sdk_ext.bbclass                     | 10 ++++++++--
 meta/files/toolchain-shar-extract.sh                      |  4 ++--
 meta/lib/oe/copy_buildsystem.py                           |  5 +++--
 meta/recipes-core/meta/nativesdk-buildtools-perl-dummy.bb | 11 +++++++++--
 scripts/gen-lockedsig-cache                               | 13 ++++++++++---
 5 files changed, 32 insertions(+), 11 deletions(-)
Signature · 1 line
2.1.0
#1 lore raw

PATCH 1/7 scripts/gen-lockedsig-cache: improve output

* Print some status when running
* When incorrect number of arguments specified, print usage text

---
 scripts/gen-lockedsig-cache | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)
scripts/gen-lockedsig-cache +10 -3
index 806c1e4..9c16506 100755
@@ -1,7 +1,4 @@
 #!/usr/bin/env python
-#
-# gen-lockedsig-cache <locked-sigs.inc> <input-cachedir> <output-cachedir>
-#
 
 import os
 import sys
@@ -18,14 +15,17 @@ def mkdir(d):
 
 if len(sys.argv) < 3:
     print("Incorrect number of arguments specified")
+    print("syntax: gen-lockedsig-cache <locked-sigs.inc> <input-cachedir> <output-cachedir>")
     sys.exit(1)
 
+print('Reading %s' % sys.argv[1])
 sigs = []
 with open(sys.argv[1]) as f:
     for l in f.readlines():
         if ":" in l:
             sigs.append(l.split(":")[2].split()[0])
 
+print('Gathering file list')
 files = set()
 for s in sigs:
     p = sys.argv[2] + "/" + s[:2] + "/*" + s + "*"
@@ -33,10 +33,13 @@ for s in sigs:
     p = sys.argv[2] + "/*/" + s[:2] + "/*" + s + "*"
     files |= set(glob.glob(p))
 
+print('Processing files')
 for f in files:
+    sys.stdout.write('Processing %s... ' % f)
     _, ext = os.path.splitext(f)
     if not ext in ['.tgz', '.siginfo', '.sig']:
         # Most likely a temp file, skip it
+        print('skipping')
         continue
     dst = f.replace(sys.argv[2], sys.argv[3])
     destdir = os.path.dirname(dst)
@@ -45,6 +48,10 @@ for f in files:
     if os.path.exists(dst):
         os.remove(dst)
     if (os.stat(f).st_dev == os.stat(destdir).st_dev):
+        print('linking')
         os.link(f, dst)
     else:
+        print('copying')
         shutil.copyfile(f, dst)
+
+print('Done!')
Signature · 1 line
2.1.0
#2 lore raw

PATCH 2/7 nativesdk-buildtools-perl-dummy: fix rebuilding when SDKMACHINE changes

In reply to Paul Eggleton

This recipe produces an empty dummy package (in order to satisfy dependencies on perl so we don't have perl within buildtools-tarball). Because we were inheriting nativesdk here the recipe was being rebuilt, but having forced PACKAGE_ARCH to a particular value the packages for each architecture were stepping on eachother. Since the packages are empty they can in fact be allarch (even though they won't actually go into the "all" package feed). It turns out that nheriting nativesdk wasn't actually necessary either, so drop that.

Fixes [YOCTO #8509].

---
 meta/recipes-core/meta/nativesdk-buildtools-perl-dummy.bb | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)
meta/recipes-core/meta/nativesdk-buildtools-perl-dummy.bb +9 -2
index d971c3c..6fb2b64 100644
@@ -2,9 +2,16 @@ SUMMARY = "Dummy package which ensures perl is excluded from buildtools"
 LICENSE = "MIT"
 LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302"
 
-inherit nativesdk
+inherit allarch
+
+python() {
+    # Put the package somewhere separate to ensure it's never used except
+    # when we want it
+    # (note that we have to do this in anonymous python here to avoid
+    # allarch.bbclass disabling itself)
+    d.setVar('PACKAGE_ARCH', 'buildtools-dummy-${SDKPKGSUFFIX}')
+}
 
-# Put it somewhere separate to ensure it's never used except when we want it
 PACKAGE_ARCH = "buildtools-dummy-${SDKPKGSUFFIX}"
 
 PERLPACKAGES = "nativesdk-perl \
Signature · 1 line
2.1.0
#3 lore raw

PATCH 3/7 toolchain-shar-extract.sh: do not allow $ in paths for ext SDK

In reply to Paul Eggleton

If you put an $ character in the path, SDK installation fails during the preparation stage, so add this to the disallowed characters.

Fixes [YOCTO #8625].

---
 meta/files/toolchain-shar-extract.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
meta/files/toolchain-shar-extract.sh +2 -2
index 98b9f1c..954b6b3 100644
@@ -101,9 +101,9 @@ fi
 
 if [ "$SDK_EXTENSIBLE" = "1" ]; then
 	# We're going to be running the build system, additional restrictions apply
-	if echo "$target_sdk_dir" | grep -q '[+\ @]'; then
+	if echo "$target_sdk_dir" | grep -q '[+\ @$]'; then
 		echo "The target directory path ($target_sdk_dir) contains illegal" \
-		     "characters such as spaces, @ or +. Abort!"
+		     "characters such as spaces, @, \$ or +. Abort!"
 		exit 1
 	fi
 else
Signature · 1 line
2.1.0
#4 lore raw

PATCH 4/7 classes/populate_sdk_ext: tidy up preparation log file writing

In reply to Paul Eggleton

Use a variable for the log file which includes the full path; this is not only neater but avoids us writing the first part (the output of oe-init-build-env) to a file in another directory since we are changing directory as part of this subshell.

---
 meta/classes/populate_sdk_ext.bbclass | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
meta/classes/populate_sdk_ext.bbclass +2 -1
index 249ec36..d368d4f 100644
@@ -249,7 +249,8 @@ sdk_ext_postinst() {
 		# dash which is /bin/sh on Ubuntu will not preserve the
 		# current working directory when first ran, nor will it set $1 when
 		# sourcing a script. That is why this has to look so ugly.
-		sh -c ". buildtools/environment-setup* > preparing_build_system.log && cd $target_sdk_dir/`dirname ${oe_init_build_env_path}` && set $target_sdk_dir && . $target_sdk_dir/${oe_init_build_env_path} $target_sdk_dir >> preparing_build_system.log && $target_sdk_dir/ext-sdk-prepare.sh $target_sdk_dir '${SDK_TARGETS}' >> preparing_build_system.log 2>&1" || { echo "SDK preparation failed: see `pwd`/preparing_build_system.log" ; exit 1 ; }
+		LOGFILE="$target_sdk_dir/preparing_build_system.log"
+		sh -c ". buildtools/environment-setup* > $LOGFILE && cd $target_sdk_dir/`dirname ${oe_init_build_env_path}` && set $target_sdk_dir && . $target_sdk_dir/${oe_init_build_env_path} $target_sdk_dir >> $LOGFILE && $target_sdk_dir/ext-sdk-prepare.sh $target_sdk_dir '${SDK_TARGETS}' >> $LOGFILE 2>&1" || { echo "SDK preparation failed: see $LOGFILE" ; exit 1 ; }
 	fi
 	rm -f $target_sdk_dir/ext-sdk-prepare.sh
 	echo done
Signature · 1 line
2.1.0
#5 lore raw

PATCH 5/7 classes/populate_sdk_ext: make it clear when SDK installation has failed

In reply to Paul Eggleton

When SDK preparation fails:

* Insert an ERROR: in front of the error message
* Add an error message to the environment setup script

Hopefully this should make it more obvious when this happens.

Fixes [YOCTO #8658].

---
 meta/classes/populate_sdk_ext.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes/populate_sdk_ext.bbclass +1 -1
index d368d4f..e806d32 100644
@@ -250,7 +250,7 @@ sdk_ext_postinst() {
 		# current working directory when first ran, nor will it set $1 when
 		# sourcing a script. That is why this has to look so ugly.
 		LOGFILE="$target_sdk_dir/preparing_build_system.log"
-		sh -c ". buildtools/environment-setup* > $LOGFILE && cd $target_sdk_dir/`dirname ${oe_init_build_env_path}` && set $target_sdk_dir && . $target_sdk_dir/${oe_init_build_env_path} $target_sdk_dir >> $LOGFILE && $target_sdk_dir/ext-sdk-prepare.sh $target_sdk_dir '${SDK_TARGETS}' >> $LOGFILE 2>&1" || { echo "SDK preparation failed: see $LOGFILE" ; exit 1 ; }
+		sh -c ". buildtools/environment-setup* > $LOGFILE && cd $target_sdk_dir/`dirname ${oe_init_build_env_path}` && set $target_sdk_dir && . $target_sdk_dir/${oe_init_build_env_path} $target_sdk_dir >> $LOGFILE && $target_sdk_dir/ext-sdk-prepare.sh $target_sdk_dir '${SDK_TARGETS}' >> $LOGFILE 2>&1" || { echo "ERROR: SDK preparation failed: see $LOGFILE"; echo "printf 'ERROR: this SDK wasn't fully installed and needs reinstalling\n'" >> $env_setup_script ; exit 1 ; }
 	fi
 	rm -f $target_sdk_dir/ext-sdk-prepare.sh
 	echo done
Signature · 1 line
2.1.0
#6 lore raw

PATCH 6/7 classes/populate_sdk_ext: tweak reporting of workspace exclusion

In reply to Paul Eggleton

If you have a local workspace layer enabled when building the extensible SDK, we explicitly exclude that from the SDK (mostly because the SDK has its own for the user to use). Adjust the message we print notifying the user of this so it's clear that we're excluding it from the SDK, and scale it back from a warning to a note printed with bb.plain().

---
 meta/classes/populate_sdk_ext.bbclass | 2 +-
 meta/lib/oe/copy_buildsystem.py       | 5 +++--
 2 files changed, 4 insertions(+), 3 deletions(-)
meta/classes/populate_sdk_ext.bbclass +1 -1
index e806d32..736f17f 100644
@@ -51,7 +51,7 @@ python copy_buildsystem () {
     core_meta_subdir = ''
 
     # Copy in all metadata layers + bitbake (as repositories)
-    buildsystem = oe.copy_buildsystem.BuildSystem(d)
+    buildsystem = oe.copy_buildsystem.BuildSystem('extensible SDK', d)
     baseoutpath = d.getVar('SDK_OUTPUT', True) + '/' + d.getVar('SDKPATH', True)
     layers_copied = buildsystem.copy_bitbake_and_layers(baseoutpath + '/layers')
 
meta/lib/oe/copy_buildsystem.py +3 -2
index 979578c..c0e7541 100644
@@ -14,8 +14,9 @@ def _smart_copy(src, dest):
         shutil.copymode(src, dest)
 
 class BuildSystem(object):
-    def __init__(self, d):
+    def __init__(self, context, d):
         self.d = d
+        self.context = context
         self.layerdirs = d.getVar('BBLAYERS', True).split()
 
     def copy_bitbake_and_layers(self, destdir):
@@ -38,7 +39,7 @@ class BuildSystem(object):
             if os.path.exists(layerconf):
                 with open(layerconf, 'r') as f:
                     if f.readline().startswith("# ### workspace layer auto-generated by devtool ###"):
-                        bb.warn("Skipping local workspace layer %s" % layer)
+                        bb.plain("NOTE: Excluding local workspace layer %s from %s" % (layer, self.context))
                         continue
 
             # If the layer was already under corebase, leave it there
Signature · 1 line
2.1.0
#7 lore raw

PATCH 7/7 classes/populate_sdk_ext: fail if SDK_ARCH != BUILD_ARCH

In reply to Paul Eggleton

The extensible SDK relies upon uninative, and with the way that uninative works, the build system architecture must be the same as the SDK architecture or the extensible SDK won't be usable. At some point in future hopefully we can remove this limitation, but until then it's disingenuous to allow this to build, so add a check to ensure SDK_ARCH == BUILD_ARCH and fail if it isn't.

---
 meta/classes/populate_sdk_ext.bbclass | 5 +++++
 1 file changed, 5 insertions(+)
meta/classes/populate_sdk_ext.bbclass +5 -0
index 736f17f..dc05261 100644
@@ -261,6 +261,11 @@ SDK_POST_INSTALL_COMMAND_task-populate-sdk-ext = "${sdk_ext_postinst}"
 SDK_POSTPROCESS_COMMAND_prepend_task-populate-sdk-ext = "copy_buildsystem; install_tools; "
 
 fakeroot python do_populate_sdk_ext() {
+    # FIXME hopefully we can remove this restriction at some point, but uninative
+    # currently forces this upon us
+    if d.getVar('SDK_ARCH', True) != d.getVar('BUILD_ARCH', True):
+        bb.fatal('The extensible SDK can currently only be built for the same architecture as the machine being built on - SDK_ARCH is set to %s (likely via setting SDKMACHINE) which is different from the architecture of the build machine (%s). Unable to continue.' % (d.getVar('SDK_ARCH', True), d.getVar('BUILD_ARCH', True)))
+
     bb.build.exec_func("do_populate_sdk", d)
 }
 
Signature · 1 line
2.1.0