Skip to contentSkip to conversation

Extensible SDK

PATCH In review Paul Eggleton 3 participants 17 messages Feb 23, 2015

lore
Share & tools
lore
#0 lore raw

cover Extensible SDK

At long last, here is the first mergeable version of what I'm now calling the "extensible SDK". The initial proposal for this was posted last year:

https://lists.yoctoproject.org/pipermail/yocto/2014-August/021044.html

Other than some names there hasn't been all that much change since then, although things have changed quite a lot behind the scenes. In any case, the basic idea is (example from poky):

1) Create an extensible SDK:

bitbake -c populate_sdk_ext core-image-minimal

2) Install the SDK:

./poky-glibc-x86_64-core-image-minimal-core2-64-toolchain-ext-1.7.sh

(usual SDK installation procedure; however do not run the installer
as root or under sudo - it should be installed as the user you intend to run it with)

3) Run the environment script:

    . ./environment-setup-core2-64-poky-linux

4) Add your application to the build:

devtool add myapp /path/to/myapp/source

5) Build it:

devtool build myapp

6) Deploy the application to your target:

    devtool deploy-target myapp [email protected]

Now your application is ready to test on the target. "devtool build" puts items into the SDK's sysroot as well so you can also add new libraries to the installed SDK as needed.

There are still some rough edges to be worked out, and there is no capability to update the SDK without reinstalling yet, but the basic functionality is there.

The following changes since commit c4de42aadd4c8a4a8f16c25e7dcdefef79daf030:

pulseaudio: upgrade to 6.0 (2015-02-23 08:08:05 +0000)

are available in the git repository at:

git://git.openembedded.org/openembedded-core-contrib paule/extensible-sdk
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=paule/extensible-sdk

Paul Eggleton (1):
sstatesig: Add ability to filter list of tasks for locked-sigs.inc

Randy Witt (8):
gen-lockedsig-cache: Allow cross-filesystem copies. copy_buildsystem.py: Add a way to copy buildsystem to a directory. copy_buildsystem.py: Add methods to copy shared state.
toolchain-scripts: Add parameters to toolchain_create_sdk_env_script. toolchain-shar-template.sh: Make relocation optional.
uninative-tarball: Actually use bzip2 for compression. scripts/oe-buildenv-internal: add means of skipping SDK check during
setup
populate_sdk_ext: add extensible SDK

 meta/classes/image.bbclass                         |   2 +-
 meta/classes/populate_sdk_base.bbclass             |  10 +-
 meta/classes/populate_sdk_ext.bbclass              | 217 +++++++++++++++++++++
 meta/classes/toolchain-scripts.bbclass             |  15 +-
 ...-shar-template.sh => toolchain-shar-extract.sh} |  53 +----
 meta/files/toolchain-shar-relocate.sh              |  50 +++++
 meta/lib/oe/copy_buildsystem.py                    | 102 ++++++++++
 meta/lib/oe/sstatesig.py                           |   5 +-
 meta/recipes-core/meta/meta-environment-extsdk.bb  |  12 ++
 meta/recipes-core/meta/uninative-tarball.bb        |   2 +-
 scripts/gen-lockedsig-cache                        |  10 +-
 scripts/oe-buildenv-internal                       |   2 +-
 12 files changed, 414 insertions(+), 66 deletions(-)
 create mode 100644 meta/classes/populate_sdk_ext.bbclass
 rename meta/files/{toolchain-shar-template.sh => toolchain-shar-extract.sh} (64%)
 create mode 100644 meta/files/toolchain-shar-relocate.sh
 create mode 100644 meta/lib/oe/copy_buildsystem.py
 create mode 100644 meta/recipes-core/meta/meta-environment-extsdk.bb
Signature · 1 line
1.9.3
#1 lore raw

PATCH 1/9 gen-lockedsig-cache: Allow cross-filesystem copies.

From: Randy Witt <[email protected]>

Since this previously always tried to use hardlinks you couldn't have the source and destination be on different devices. This change allows for that and also prevents failure in situations where the files already existed.

---
 scripts/gen-lockedsig-cache | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)
scripts/gen-lockedsig-cache +8 -2
index dfb282e..c93b2c0 100755
@@ -35,6 +35,12 @@ for s in sigs:
 
 for f in files:
     dst = f.replace(sys.argv[2], sys.argv[3])
-    mkdir(os.path.dirname(dst))
-    os.link(f, dst)
+    destdir = os.path.dirname(dst)
+    mkdir(destdir)
 
+    if os.path.exists(dst):
+        os.remove(dst)
+    if (os.stat(f).st_dev == os.stat(destdir).st_dev):
+        os.link(f, dst)
+    else:
+        shutil.copyfile(f, dst)
Signature · 1 line
1.9.3
#2 lore raw

PATCH 2/9 sstatesig: Add ability to filter list of tasks for locked-sigs.inc

In reply to Paul Eggleton
---
 meta/lib/oe/sstatesig.py | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
meta/lib/oe/sstatesig.py +4 -1
index af7617e..62e75c2 100644
@@ -137,13 +137,16 @@ class SignatureGeneratorOEBasicHash(bb.siggen.SignatureGeneratorBasicHash):
             return
         super(bb.siggen.SignatureGeneratorBasicHash, self).dump_sigtask(fn, task, stampbase, runtime)
 
-    def dump_lockedsigs(self, sigfile=None):
+    def dump_lockedsigs(self, sigfile=None, taskfilter=None):
         if not sigfile:
             sigfile = os.getcwd() + "/locked-sigs.inc"
 
         bb.plain("Writing locked sigs to %s" % sigfile)
         types = {}
         for k in self.runtaskdeps:
+            if taskfilter:
+                if not k in taskfilter:
+                    continue
             fn = k.rsplit(".",1)[0]
             t = self.lockedhashfn[fn].split(" ")[1].split(":")[5]
             t = 't-' + t.replace('_', '-')
Signature · 1 line
1.9.3
#3 lore raw

PATCH 3/9 copy_buildsystem.py: Add a way to copy buildsystem to a directory.

In reply to Paul Eggleton

From: Randy Witt <[email protected]>

This file provides a way to take bitbake and the layers in the current build and copy them to a target specified.

---
 meta/lib/oe/copy_buildsystem.py | 70 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 70 insertions(+)
 create mode 100644 meta/lib/oe/copy_buildsystem.py
meta/lib/oe/copy_buildsystem.py new file +70 -0
new file mode 100644
index 0000000..5a4d8c3
@@ -0,0 +1,70 @@
+# This class should provide easy access to the different aspects of the
+# buildsystem such as layers, bitbake location, etc.
+import stat
+import shutil
+
+def _smart_copy(src, dest):
+    # smart_copy will choose the correct function depending on whether the
+    # source is a file or a directory.
+    mode = os.stat(src).st_mode
+    if stat.S_ISDIR(mode):
+        shutil.copytree(src, dest, symlinks=True)
+    else:
+        shutil.copyfile(src, dest)
+        shutil.copymode(src, dest)
+
+class BuildSystem(object):
+    def __init__(self, d):
+        self.d = d
+        self.layerdirs = d.getVar('BBLAYERS', True).split()
+
+    def copy_bitbake_and_layers(self, destdir):
+        # Copy in all metadata layers + bitbake (as repositories)
+        layers_copied = []
+        bb.utils.mkdirhier(destdir)
+        layers = list(self.layerdirs)
+
+        corebase = self.d.getVar('COREBASE', True)
+        layers.append(corebase)
+
+        corebase_files = self.d.getVar('COREBASE_FILES', True).split()
+
+        # bitbake belongs in corebase so make sure it goes there
+        if "bitbake" not in corebase_files:
+            corebase_files.append("bitbake")
+        corebase_files = [corebase + '/' +x for x in corebase_files]
+
+        for layer in layers:
+            layerconf = os.path.join(layer, 'conf', 'layer.conf')
+            if os.path.exists(layerconf):
+                with open(layerconf, 'r') as f:
+                    if f.readline().startswith("# ### workspace layer auto-generated by devtool ###"):
+                        bb.warn("Skipping local workspace layer %s" % layer)
+                        continue
+
+            # If the layer was already under corebase, leave it there
+            # since layers such as meta have issues when moved.
+            layerdestpath = destdir
+            if corebase == os.path.dirname(layer):
+                layerdestpath += '/' + os.path.basename(corebase)
+            layerdestpath += '/' + os.path.basename(layer)
+
+            layer_relative = os.path.relpath(layerdestpath,
+                                             destdir)
+            layers_copied.append(layer_relative)
+
+            # Treat corebase as special since it typically will contain
+            # build directories or other custom items.
+            if corebase == layer:
+                bb.utils.mkdirhier(layerdestpath)
+                for f in corebase_files:
+                    f_basename = os.path.basename(f)
+                    destname = os.path.join(layerdestpath, f_basename)
+                    _smart_copy(f, destname)
+            else:
+                if os.path.exists(layerdestpath):
+                    bb.note("Skipping layer %s, already handled" % layer)
+                else:
+                    _smart_copy(layer, layerdestpath)
+
+        return layers_copied
Signature · 1 line
1.9.3
#4 lore raw

PATCH 4/9 copy_buildsystem.py: Add methods to copy shared state.

In reply to Paul Eggleton

From: Randy Witt <[email protected]>

Added the helper functions necessary to copy the sstate from the current build, and generate the file to "lock" it.

---
 meta/lib/oe/copy_buildsystem.py | 32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)
meta/lib/oe/copy_buildsystem.py +32 -0
index 5a4d8c3..cf7fada 100644
@@ -68,3 +68,35 @@ class BuildSystem(object):
                     _smart_copy(layer, layerdestpath)
 
         return layers_copied
+
+def generate_locked_sigs(sigfile, d):
+    bb.utils.mkdirhier(os.path.dirname(sigfile))
+    depd = d.getVar('BB_TASKDEPDATA', True)
+    tasks = ['%s.%s' % (v[2], v[1]) for v in depd.itervalues()]
+    bb.parse.siggen.dump_lockedsigs(sigfile, tasks)
+
+def prune_lockedsigs(allowed_tasks, excluded_targets, lockedsigs, pruned_output):
+    with open(lockedsigs, 'r') as infile:
+        bb.utils.mkdirhier(os.path.dirname(pruned_output))
+        with open(pruned_output, 'w') as f:
+            invalue = False
+            for line in infile:
+                if invalue:
+                    if line.endswith('\\\n'):
+                        splitval = line.strip().split(':')
+                        if splitval[1] in allowed_tasks and not splitval[0] in excluded_targets:
+                            f.write(line)
+                    else:
+                        f.write(line)
+                        invalue = False
+                elif line.startswith('SIGGEN_LOCKEDSIGS'):
+                    invalue = True
+                    f.write(line)
+
+def create_locked_sstate_cache(lockedsigs, input_sstate_cache, output_sstate_cache, d, fixedlsbstring=""):
+    bb.note('Generating sstate-cache...')
+
+    bb.process.run("gen-lockedsig-cache %s %s %s" % (lockedsigs, input_sstate_cache, output_sstate_cache))
+    if fixedlsbstring:
+        os.rename(output_sstate_cache + '/' + d.getVar('NATIVELSBSTRING', True),
+        output_sstate_cache + '/' + fixedlsbstring)
Signature · 1 line
1.9.3
#5 lore raw

PATCH 5/9 toolchain-scripts: Add parameters to toolchain_create_sdk_env_script.

In reply to Paul Eggleton

From: Randy Witt <[email protected]>

To add some flexibility to setting up the paths for the toolchain, add some parameters. This initial use will be in order to point at the buildsystem toolchain copied in by copy_buildsystem.py.

---
 meta/classes/toolchain-scripts.bbclass | 15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)
meta/classes/toolchain-scripts.bbclass +9 -6
index 9836db4..ea6aff0 100644
@@ -7,6 +7,9 @@ REAL_MULTIMACH_TARGET_SYS ?= "${MULTIMACH_TARGET_SYS}"
 # This function creates an environment-setup-script for use in a deployable SDK
 toolchain_create_sdk_env_script () {
 	# Create environment setup script
+	sdkpathnative=${7:-${SDKPATHNATIVE}}
+	prefix=${6:-${prefix_nativesdk}}
+	bindir=${5:-${bindir_nativesdk}}
 	libdir=${4:-${libdir}}
 	sysroot=${3:-${SDKTARGETSYSROOT}}
 	multimach_target_sys=${2:-${REAL_MULTIMACH_TARGET_SYS}}
@@ -16,17 +19,17 @@ toolchain_create_sdk_env_script () {
 	echo 'export SDKTARGETSYSROOT='"$sysroot" >> $script
 	EXTRAPATH=""
 	for i in ${CANADIANEXTRAOS}; do
-		EXTRAPATH="$EXTRAPATH:${SDKPATHNATIVE}${bindir_nativesdk}/${TARGET_ARCH}${TARGET_VENDOR}-$i"
+		EXTRAPATH="$EXTRAPATH:$sdkpathnative$bindir/${TARGET_ARCH}${TARGET_VENDOR}-$i"
 	done
-	echo 'export PATH=${SDKPATHNATIVE}${bindir_nativesdk}:${SDKPATHNATIVE}${bindir_nativesdk}/${TARGET_SYS}'$EXTRAPATH':$PATH' >> $script
-	echo 'export CCACHE_PATH=${SDKPATHNATIVE}${bindir_nativesdk}:${SDKPATHNATIVE}${bindir_nativesdk}/${TARGET_SYS}'$EXTRAPATH':$CCACHE_PATH' >> $script
+	echo "export PATH=$sdkpathnative$bindir:$sdkpathnative$bindir/${TARGET_SYS}"$EXTRAPATH':$PATH' >> $script
+	echo 'export CCACHE_PATH=$sdkpathnative$bindir:$sdkpathnative$bindir/${TARGET_SYS}'$EXTRAPATH':$CCACHE_PATH' >> $script
 	echo 'export PKG_CONFIG_SYSROOT_DIR=$SDKTARGETSYSROOT' >> $script
 	echo 'export PKG_CONFIG_PATH=$SDKTARGETSYSROOT'"$libdir"'/pkgconfig' >> $script
 	echo 'export CONFIG_SITE=${SDKPATH}/site-config-'"${multimach_target_sys}" >> $script
-	echo 'export OECORE_NATIVE_SYSROOT="${SDKPATHNATIVE}"' >> $script
+	echo "export OECORE_NATIVE_SYSROOT=\"$sdkpathnative\"" >> $script
 	echo 'export OECORE_TARGET_SYSROOT="$SDKTARGETSYSROOT"' >> $script
-	echo 'export OECORE_ACLOCAL_OPTS="-I ${SDKPATHNATIVE}/usr/share/aclocal"' >> $script
-	echo 'export PYTHONHOME=${SDKPATHNATIVE}${prefix_nativesdk}' >> $script
+	echo "export OECORE_ACLOCAL_OPTS=\"-I $sdkpathnative/usr/share/aclocal\"" >> $script
+	echo "export PYTHONHOME=$sdkpathnative$prefix" >> $script
 
 	toolchain_shared_env_script
 }
Signature · 1 line
1.9.3
#6 lore raw

PATCH 6/9 toolchain-shar-template.sh: Make relocation optional.

In reply to Paul Eggleton

From: Randy Witt <[email protected]>

If the buildsystem is copied into the sdk and its toolchain is to be used, then the relocation provided in toolchain-shar-template.sh isn't needed and will actually fail.

So break the relocation aspect out and essentially make it another SDK_POST_INSTALL_COMMAND script.

---
 meta/classes/populate_sdk_base.bbclass             | 10 +++-
 ...-shar-template.sh => toolchain-shar-extract.sh} | 53 +---------------------
 meta/files/toolchain-shar-relocate.sh              | 50 ++++++++++++++++++++
 3 files changed, 59 insertions(+), 54 deletions(-)
 rename meta/files/{toolchain-shar-template.sh => toolchain-shar-extract.sh} (64%)
 create mode 100644 meta/files/toolchain-shar-relocate.sh
meta/classes/populate_sdk_base.bbclass +8 -2
index e3adacb..5c07693 100644
@@ -52,6 +52,7 @@ EXCLUDE_FROM_WORLD = "1"
 
 SDK_PACKAGING_FUNC ?= "create_shar"
 SDK_POST_INSTALL_COMMAND ?= ""
+SDK_RELOCATE_AFTER_INSTALL ?= "1"
 
 SDK_MANIFEST = "${SDK_DEPLOY}/${TOOLCHAIN_OUTPUTNAME}.manifest"
 python write_target_sdk_manifest () {
@@ -116,9 +117,14 @@ fakeroot tar_sdk() {
 
 fakeroot create_shar() {
 	# copy in the template shar extractor script
-	cp ${COREBASE}/meta/files/toolchain-shar-template.sh ${SDK_DEPLOY}/${TOOLCHAIN_OUTPUTNAME}.sh
+	cp ${COREBASE}/meta/files/toolchain-shar-extract.sh ${SDK_DEPLOY}/${TOOLCHAIN_OUTPUTNAME}.sh
 
-	cat << "EOF" > ${T}/post_install_command
+	rm -f ${T}/post_install_command
+
+	if [ ${SDK_RELOCATE_AFTER_INSTALL} -eq 1 ] ; then
+		cp ${COREBASE}/meta/files/toolchain-shar-relocate.sh ${T}/post_install_command
+	fi
+	cat << "EOF" >> ${T}/post_install_command
 ${SDK_POST_INSTALL_COMMAND}
 EOF
 	sed -i -e '/@SDK_POST_INSTALL_COMMAND@/r ${T}/post_install_command' ${SDK_DEPLOY}/${TOOLCHAIN_OUTPUTNAME}.sh
meta/files/toolchain-shar-extract.sh renamed from meta/files/toolchain-shar-template.sh +1 -52

64 lines not on this page · raw message

meta/files/toolchain-shar-relocate.sh new file +50 -0

50 lines not on this page · raw message

Signature · 1 line
1.9.3
#7 lore raw

PATCH 7/9 uninative-tarball: Actually use bzip2 for compression.

In reply to Paul Eggleton

From: Randy Witt <[email protected]>

uninative.bbclass uses -xjf for decompression so actually run the data through bzip2.

---
 meta/recipes-core/meta/uninative-tarball.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/recipes-core/meta/uninative-tarball.bb +1 -1
index e17685a..53435f2 100644
@@ -44,5 +44,5 @@ fakeroot tar_sdk() {
 	rm sysroots -rf
 	patchelf --set-interpreter ${@''.join('a' for n in xrange(1024))} ./${BUILD_SYS}/usr/bin/patchelf
 	mv ./${BUILD_SYS}/usr/bin/patchelf ./${BUILD_SYS}/usr/bin/patchelf-uninative
-	tar ${SDKTAROPTS} -c --file=${SDK_DEPLOY}/${TOOLCHAIN_OUTPUTNAME}.tar.bz2 .
+	tar ${SDKTAROPTS} -c -j --file=${SDK_DEPLOY}/${TOOLCHAIN_OUTPUTNAME}.tar.bz2 .
 }
Signature · 1 line
1.9.3
#8 lore raw

PATCH 8/9 scripts/oe-buildenv-internal: add means of skipping SDK check during setup

In reply to Paul Eggleton

From: Randy Witt <[email protected]>

The oe-buildenv-internal script checks if the user is already in an sdk environment and errors if true. Add a way to skip this check.

---
 scripts/oe-buildenv-internal | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
scripts/oe-buildenv-internal +1 -1
index bba6f8f..9ed2721 100755
@@ -24,7 +24,7 @@ if [ -z "$OEROOT" ]; then
     return 1
 fi
 
-if [ ! -z "$OECORE_SDK_VERSION" ]; then
+if [ -z "$OE_SKIP_SDK_CHECK" -a ! -z "$OECORE_SDK_VERSION" ]; then
     echo >&2 "Error: The OE SDK/ADT was detected as already being present in this shell environment. Please use a clean shell when sourcing this environment script."
     return 1
 fi
Signature · 1 line
1.9.3
#9 lore raw

PATCH 9/9 populate_sdk_ext: add extensible SDK

In reply to Paul Eggleton

From: Randy Witt <[email protected]>

This bbclass will create an SDK with a copy of bitbake and the metadata and sstate for the target specified for the task. The idea is to let "system" developers both work on applications and then test adding them to an image without having to switch between workspaces or having to download separate items.

Rather than running bitbake directly however, the primary way of running builds within the extensible SDK is to use the "devtool" command. The rest of the build system is fixed via locked shared state signatures, and thus only the recipes you have added get built.

---
 meta/classes/image.bbclass                        |   2 +-
 meta/classes/populate_sdk_ext.bbclass             | 217 ++++++++++++++++++++++
 meta/recipes-core/meta/meta-environment-extsdk.bb |  12 ++
 3 files changed, 230 insertions(+), 1 deletion(-)
 create mode 100644 meta/classes/populate_sdk_ext.bbclass
 create mode 100644 meta/recipes-core/meta/meta-environment-extsdk.bb
meta/classes/image.bbclass +1 -1
index 893eb40..89eb5f3 100644
@@ -1,6 +1,6 @@
 inherit rootfs_${IMAGE_PKGTYPE}
 
-inherit populate_sdk_base
+inherit populate_sdk_ext
 
 TOOLCHAIN_TARGET_TASK += "${PACKAGE_INSTALL}"
 TOOLCHAIN_TARGET_TASK_ATTEMPTONLY += "${PACKAGE_INSTALL_ATTEMPTONLY}"
meta/classes/populate_sdk_ext.bbclass new file +217 -0

217 lines not on this page · raw message

meta/recipes-core/meta/meta-environment-extsdk.bb new file +12 -0
new file mode 100644
index 0000000..d9e5961
@@ -0,0 +1,12 @@
+# meta-environment for extensible SDK
+
+require meta-environment.bb
+
+PN = "meta-environment-extsdk-${MACHINE}"
+
+create_sdk_files_append() {
+	local sysroot=${SDKPATH}/${@os.path.relpath(d.getVar('STAGING_DIR_TARGET', True), d.getVar('TOPDIR', True))}
+	local sdkpathnative=${SDKPATH}/${@os.path.relpath(d.getVar('STAGING_DIR_NATIVE',True), d.getVar('TOPDIR', True))}
+
+	toolchain_create_sdk_env_script '' '' $sysroot '' ${bindir_native} ${prefix_native} $sdkpathnative
+}
Signature · 1 line
1.9.3
#10 lore raw

Re: [PATCH 9/9] populate_sdk_ext: add extensible SDK

Quoted 8 lines · Paul Eggleton

From: Randy Witt <[email protected]>

This bbclass will create an SDK with a copy of bitbake and the metadata and sstate for the target specified for the task. The idea is to let "system" developers both work on applications and then test adding them to an image without having to switch between workspaces or having to download separate items.

Quoted 8 lines

Rather than running bitbake directly however, the primary way of running builds within the extensible SDK is to use the "devtool" command. The rest of the build system is fixed via locked shared state signatures, and thus only the recipes you have added get built.

Why another class? it could be tuned using SDKIMAGE_FEATURES

Signature · 3 lines
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750
#11 lore raw

Re: [PATCH 9/9] populate_sdk_ext: add extensible SDK

Quoted 15 lines · On Monday 23 February 2015 14:41:34 Otavio Salvador wrote:
Quoted 13 lines · Paul Eggleton

From: Randy Witt <[email protected]>

This bbclass will create an SDK with a copy of bitbake and the metadata and sstate for the target specified for the task. The idea is to let "system" developers both work on applications and then test adding them to an image without having to switch between workspaces or having to download separate items.

Rather than running bitbake directly however, the primary way of running builds within the extensible SDK is to use the "devtool" command. The rest of the build system is fixed via locked shared state signatures, and thus only the recipes you have added get built.

Quoted 5 lines
Quoted 3 lines

Why another class? it could be tuned using SDKIMAGE_FEATURES

If you look at what the class does it would be a bit messy to do it that way - I wanted to get something working with minimal impact. I don't doubt that it could be implemented as an SDKIMAGE_FEATURES item though with extra work.

Cheers,
Paul

Signature · 2 lines
Paul Eggleton
Intel Open Source Technology Centre
#12 lore raw

Re: [PATCH 9/9] populate_sdk_ext: add extensible SDK

Quoted 20 lines · Paul Eggleton
Quoted 19 lines · On Monday 23 February 2015 14:41:34 Otavio Salvador wrote:
Quoted 16 lines · Paul Eggleton

From: Randy Witt <[email protected]>

This bbclass will create an SDK with a copy of bitbake and the metadata and sstate for the target specified for the task. The idea is to let "system" developers both work on applications and then test adding them to an image without having to switch between workspaces or having to download separate items.

Rather than running bitbake directly however, the primary way of running builds within the extensible SDK is to use the "devtool" command. The rest of the build system is fixed via locked shared state signatures, and thus only the recipes you have added get built.

Quoted 6 lines
Quoted 1 line

Why another class? it could be tuned using SDKIMAGE_FEATURES

If you look at what the class does it would be a bit messy to do it that way - I wanted to get something working with minimal impact. I don't doubt that it could be implemented as an SDKIMAGE_FEATURES item though with extra work.

As far this does not gets merged I am fine with that. This is clearly a WIP code and shouldn't be merged as is.

Signature · 3 lines
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750
#13 lore raw

Re: [PATCH 9/9] populate_sdk_ext: add extensible SDK

Quoted 24 lines · Otavio Salvador
Quoted 22 lines · Paul Eggleton
Quoted 20 lines · On Monday 23 February 2015 14:41:34 Otavio Salvador wrote:
Quoted 8 lines · Paul Eggleton

From: Randy Witt <[email protected]>

This bbclass will create an SDK with a copy of bitbake and the metadata and sstate for the target specified for the task. The idea is to let "system" developers both work on applications and then test adding them to an image without having to switch between workspaces or having to download separate items.

Quoted 8 lines

Rather than running bitbake directly however, the primary way of running builds within the extensible SDK is to use the "devtool" command. The rest of the build system is fixed via locked shared state signatures, and thus only the recipes you have added get built.

Why another class? it could be tuned using SDKIMAGE_FEATURES

Quoted 7 lines
Quoted 4 lines

If you look at what the class does it would be a bit messy to do it that way - I wanted to get something working with minimal impact. I don't doubt that it could be implemented as an SDKIMAGE_FEATURES item though with extra work.

As far this does not gets merged I am fine with that. This is clearly a WIP code and shouldn't be merged as is.

This code adds in an alternative SDK format and it drives that using a different task target. Right now its hard for people to see where things are going, this puts it in easy reach whilst allowing the current SDK to continue to work too.

I think a choice of two different task targets here makes sense to drive this configuration rather than SDKIMAGE_FEATURES and I agree with Paul that adding it the other way would be complex, error prone and confusing.

Cheers,

Richard

#14 lore raw

Re: [PATCH 9/9] populate_sdk_ext: add extensible SDK

Quoted 38 lines · Richard Purdie
Quoted 24 lines · Otavio Salvador
Quoted 22 lines · Paul Eggleton
Quoted 20 lines · On Monday 23 February 2015 14:41:34 Otavio Salvador wrote:
Quoted 8 lines · Paul Eggleton

From: Randy Witt <[email protected]>

This bbclass will create an SDK with a copy of bitbake and the metadata and sstate for the target specified for the task. The idea is to let "system" developers both work on applications and then test adding them to an image without having to switch between workspaces or having to download separate items.

Quoted 8 lines

Rather than running bitbake directly however, the primary way of running builds within the extensible SDK is to use the "devtool" command. The rest of the build system is fixed via locked shared state signatures, and thus only the recipes you have added get built.

Why another class? it could be tuned using SDKIMAGE_FEATURES

Quoted 7 lines
Quoted 4 lines

If you look at what the class does it would be a bit messy to do it that way - I wanted to get something working with minimal impact. I don't doubt that it could be implemented as an SDKIMAGE_FEATURES item though with extra work.

As far this does not gets merged I am fine with that. This is clearly a WIP code and shouldn't be merged as is.

This code adds in an alternative SDK format and it drives that using a different task target. Right now its hard for people to see where things are going, this puts it in easy reach whilst allowing the current SDK to continue to work too.

Quoted 4 lines

I think a choice of two different task targets here makes sense to drive this configuration rather than SDKIMAGE_FEATURES and I agree with Paul that adding it the other way would be complex, error prone and confusing.

Well so why we don't add populate_sdk+dev-pkgs+dbg-pkgs.

Sorry ... it is clear that people wish this feature (and I also do) and appreciate the hard work Paul has put on this (yes this is indeed a good amount of work) but the extra task does not seem to fit how the system is designed.

The SDKIMAGE_FEATURES[1] is described as:

Equivalent to IMAGE_FEATURES. However, this variable applies to the SDK generated from an image using the following command:

     $ bitbake -c populate_sdk imagename

and it makes a lot of sense to have a 'extended' or 'devtool' or another name for this feature.

1. http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#var-SDKIMAGE_FEATURES

Signature · 3 lines
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750
#15 lore raw

Re: [PATCH 9/9] populate_sdk_ext: add extensible SDK

Quoted 65 lines · On Monday 23 February 2015 15:11:38 Otavio Salvador wrote:
Quoted 46 lines · Richard Purdie
Quoted 31 lines · Otavio Salvador

On Mon, Feb 23, 2015 at 2:54 PM, Paul Eggleton

Quoted 28 lines · <[email protected]> wrote:
Quoted 26 lines · On Monday 23 February 2015 14:41:34 Otavio Salvador wrote:

On Mon, Feb 23, 2015 at 2:00 PM, Paul Eggleton

Quoted 18 lines · <[email protected]> wrote:

From: Randy Witt <[email protected]>

This bbclass will create an SDK with a copy of bitbake and the metadata
and sstate for the target specified for the task. The idea is to let "system" developers both work on applications and then test adding them
to an image without having to switch between workspaces or having to download separate items.

Rather than running bitbake directly however, the primary way of running
builds within the extensible SDK is to use the "devtool" command. The
rest of the build system is fixed via locked shared state signatures,
and thus only the recipes you have added get built.

Quoted 3 lines

Why another class? it could be tuned using SDKIMAGE_FEATURES

Quoted 8 lines
Quoted 5 lines

If you look at what the class does it would be a bit messy to do it that way - I wanted to get something working with minimal impact. I don't doubt that it could be implemented as an SDKIMAGE_FEATURES item though
with extra work.

As far this does not gets merged I am fine with that. This is clearly a WIP code and shouldn't be merged as is.

This code adds in an alternative SDK format and it drives that using a different task target. Right now its hard for people to see where things are going, this puts it in easy reach whilst allowing the current SDK to continue to work too.

Quoted 4 lines

I think a choice of two different task targets here makes sense to drive this configuration rather than SDKIMAGE_FEATURES and I agree with Paul that adding it the other way would be complex, error prone and confusing.

Well so why we don't add populate_sdk+dev-pkgs+dbg-pkgs.

Sorry ... it is clear that people wish this feature (and I also do) and appreciate the hard work Paul has put on this (yes this is indeed a good amount of work) but the extra task does not seem to fit how the system is designed.

The SDKIMAGE_FEATURES[1] is described as:

Equivalent to IMAGE_FEATURES. However, this variable applies to the SDK generated from an image using the following command:

Quoted 8 lines
     $ bitbake -c populate_sdk imagename

and it makes a lot of sense to have a 'extended' or 'devtool' or another name for this feature.

1.
http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#var-SDK
IMAGE_FEATURES

I see where you are coming from, but again if we look at what the class does, it's not just adding some packages. It fundamentally changes how the SDK works
- you don't actually have a nativesdk toolchain (in fact you don't have any nativesdk packages at all) and you don't have anything in the target sysroot either - because the build system sysroots pretty much contain everything already. It's not simply an add-on or a post-processing of the SDK.

With the way it's been implemented, you get a separate way of building it and a separate SDK installer, and therefore it's a little clearer as to what you are building/installing. At the moment, with this being very new functionality, that seems to me to be the right way to do things for now. If this turns out to work really well for everyone then great, we can look at integrating it more tightly.

Cheers,
Paul

Signature · 2 lines
Paul Eggleton
Intel Open Source Technology Centre
#16 lore raw

Re: [PATCH 9/9] populate_sdk_ext: add extensible SDK

In reply to Otavio Salvador
Quoted 24 lines · Otavio Salvador
Quoted 11 lines · Richard Purdie
Quoted 1 line · Otavio Salvador

On Mon, Feb 23, 2015 at 2:54 PM, Paul Eggleton

This code adds in an alternative SDK format and it drives that using a different task target. Right now its hard for people to see where things are going, this puts it in easy reach whilst allowing the current SDK to continue to work too.

I think a choice of two different task targets here makes sense to drive this configuration rather than SDKIMAGE_FEATURES and I agree with Paul that adding it the other way would be complex, error prone and confusing.

Well so why we don't add populate_sdk+dev-pkgs+dbg-pkgs.

Sorry ... it is clear that people wish this feature (and I also do) and appreciate the hard work Paul has put on this (yes this is indeed a good amount of work) but the extra task does not seem to fit how the system is designed.

The SDKIMAGE_FEATURES[1] is described as:

Quoted 7 lines

Equivalent to IMAGE_FEATURES. However, this variable applies to the SDK generated from an image using the following command:

     $ bitbake -c populate_sdk imagename

and it makes a lot of sense to have a 'extended' or 'devtool' or another name for this feature.

I see this as that SDKIMAGE_FEATURES controls the content of the SDK, not the actual construction of it so I'm afraid I don't agree.

Cheers,

Richard