Extensible SDK fixesExtensible SDK fixes
v3 cover Extensible SDK fixesExtensible SDK fixes
Changes since V2:
Remove buildtools from the ext SDK.
The following changes since commit c38acd720b3f6ffbeb544063692eb471dada8593:
binconfig-disabled: write an message to stderr to help confused developers (2015-08-19 17:57:58 +0100)
are available in the git repository at:
git://git.openembedded.org/openembedded-core-contrib ChenQi/sdktool
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=ChenQi/sdktool
Chen Qi (8):
copy_buildsystem: make sure bitbake directory is copied
populate_sdk_ext: consider custom configuration in local.conf
populate_sdk_ext: don't remove the native qemu dependencies Extensible SDK: allow for installation without preparing build system
oe-publish-sdk: add script
populate_sdk_ext: record SDK_TARGETS in devtool.conf
devtool: add sdk.py plugin
populate_sdk_ext: get rid of buildtools
meta/classes/populate_sdk_ext.bbclass | 69 +++++++++---- meta/files/toolchain-shar-extract.sh | 7 +- meta/lib/oe/copy_buildsystem.py | 7 +- scripts/lib/devtool/sdk.py | 188 ++++++++++++++++++++++++++++++++++ scripts/oe-publish-sdk | 143 ++++++++++++++++++++++++++ 5 files changed, 387 insertions(+), 27 deletions(-) create mode 100644 scripts/lib/devtool/sdk.py create mode 100755 scripts/oe-publish-sdk
Signature · 1 line
1.9.1
PATCH v3 1/8 copy_buildsystem: make sure bitbake directory is copied
The previous code assumes that bitbake/ directory is under the core layer. This is the case for Yocto project. But users might clone oe-core and bitbake separately. So we use bb.__file__ to locate the bitbake directory to make sure it's copied into the extensible SDK.
--- meta/lib/oe/copy_buildsystem.py | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-)
meta/lib/oe/copy_buildsystem.py +3 -4
@@ -28,11 +28,10 @@ class BuildSystem(object): layers.append(corebase) corebase_files = self.d.getVar('COREBASE_FILES', True).split() - - # bitbake belongs in corebase so make sure it goes there - if "bitbake" not in corebase_files: - corebase_files.append("bitbake") corebase_files = [corebase + '/' +x for x in corebase_files] + # Make sure bitbake goes in + bitbake_dir = bb.__file__.rsplit('/', 3)[0] + corebase_files.append(bitbake_dir) for layer in layers: layerconf = os.path.join(layer, 'conf', 'layer.conf')
Signature · 1 line
1.9.1
PATCH v3 2/8 populate_sdk_ext: consider custom configuration in local.conf
In reply to Chen QiCopy the contents of local.conf under TOPDIR into the final generated local.conf. In this way, custom settings are also made into the final local.conf like IMAGE_INSTALL, DISTRO_FEATURES, VIRTUAL-RUNTIME_xxx, etc.
Before this change, installing extensible SDK would usually report failure when preparing the build system if the user has custom configuration for DISTRO_FEATURES in local.conf. Also, items in IMAGE_INSTALL_append in local.conf also don't get built correctly.
This patch solves the above problem by making use of the bb.utils.edit_metadata.
In addition, we check to avoid any setting that might lead to host path bleeding into SDK's configuration. Basically, variables with values starting with '/' are removed. A whitelist mechanism is introduced so that users could specify variables that should not be ignored. The name of the whitelist is SDK_LOCAL_CONF_WHITELIST.
The SDK_META_CONF_WHITELIST is removed as it's of no use after this change.
Add SDK_INHERIT_BLACKLIST to forbit local.conf in SDK to herit certain classes like 'buildhistory' or 'icecc'.
[YOCTO #7616]
--- meta/classes/populate_sdk_ext.bbclass | 23 ++++++++++++++++++++--- 1 file changed, 20 insertions(+), 3 deletions(-)
meta/classes/populate_sdk_ext.bbclass +20 -3
@@ -15,7 +15,8 @@ SDK_RDEPENDS_append_task-populate-sdk-ext = " ${SDK_TARGETS}" SDK_RELOCATE_AFTER_INSTALL_task-populate-sdk-ext = "0" -SDK_META_CONF_WHITELIST ?= "MACHINE DISTRO PACKAGE_CLASSES" +SDK_LOCAL_CONF_WHITELIST ?= "" +SDK_INHERIT_BLACKLIST ?= "buildhistory icecc" SDK_TARGETS ?= "${PN}" OE_INIT_ENV_SCRIPT ?= "oe-init-build-env"
@@ -109,15 +110,33 @@ python copy_buildsystem () { f.write(' "\n') # Create local.conf + local_conf_whitelist = d.getVar('SDK_LOCAL_CONF_WHITELIST', True).split() + def handle_var(varname, origvalue, op, newlines): + if origvalue.strip().startswith('/') and not varname in local_conf_whitelist: + newlines.append('# Removed original setting of %s\n' % varname) + return None, op, 0, True + else: + return origvalue, op, 0, True + varlist = ['[^#=+ ]*'] + builddir = d.getVar('TOPDIR', True) + with open(builddir + '/conf/local.conf', 'r') as f: + oldlines = f.readlines() + (updated, newlines) = bb.utils.edit_metadata(oldlines, varlist, handle_var) + with open(baseoutpath + '/conf/local.conf', 'w') as f: f.write('# WARNING: this configuration has been automatically generated and in\n') f.write('# most cases should not be edited. If you need more flexibility than\n') f.write('# this configuration provides, it is strongly suggested that you set\n') f.write('# up a proper instance of the full build system and use that instead.\n\n') + for line in newlines: + f.write(line) f.write('INHERIT += "%s"\n\n' % 'uninative') f.write('CONF_VERSION = "%s"\n\n' % d.getVar('CONF_VERSION', False)) + # Some classes are not suitable for SDK, remove them from INHERIT + f.write('INHERIT_remove = "%s"\n' % d.getVar('SDK_INHERIT_BLACKLIST')) + # This is a bit of a hack, but we really don't want these dependencies # (we're including them in the SDK as nativesdk- versions instead) f.write('POKYQEMUDEPS_forcevariable = ""\n\n')
@@ -134,8 +153,6 @@ python copy_buildsystem () { # Ensure locked sstate cache objects are re-used without error f.write('SIGGEN_LOCKEDSIGS_CHECK_LEVEL = "warn"\n\n') - for varname in d.getVar('SDK_META_CONF_WHITELIST', True).split(): - f.write('%s = "%s"\n' % (varname, d.getVar(varname, True))) f.write('require conf/locked-sigs.inc\n') f.write('require conf/work-config.inc\n')
Signature · 1 line
1.9.1
PATCH v3 3/8 populate_sdk_ext: don't remove the native qemu dependencies
In reply to Chen QiThese dependencies were deliberately removed because it was assumed that they were provided by nativesdk packages. On the one hand, nativesdk packages in extensible SDK don't have these packages; on the other hand, even if we add these nativesdk packages, they are still not useful because we we need runqemu to run correctly.
So we don't remove these native qemu dependencies.
--- meta/classes/populate_sdk_ext.bbclass | 5 ----- 1 file changed, 5 deletions(-)
meta/classes/populate_sdk_ext.bbclass +0 -5
@@ -137,11 +137,6 @@ python copy_buildsystem () { # Some classes are not suitable for SDK, remove them from INHERIT f.write('INHERIT_remove = "%s"\n' % d.getVar('SDK_INHERIT_BLACKLIST')) - # This is a bit of a hack, but we really don't want these dependencies - # (we're including them in the SDK as nativesdk- versions instead) - f.write('POKYQEMUDEPS_forcevariable = ""\n\n') - f.write('EXTRA_IMAGEDEPENDS_remove = "qemu-native qemu-helper-native"\n\n') - # Bypass the default connectivity check if any f.write('CONNECTIVITY_CHECK_URIS = ""\n\n')
Signature · 1 line
1.9.1
PATCH v3 4/8 Extensible SDK: allow for installation without preparing build system
In reply to Chen QiWhen publishing SDK, what we want is basically its metadata and sstate cache objects. We don't want the SDK to be prepared with running bitbake as it takes time which reproduces meaningless output for the published SDK.
So this patch adds an option to allow for SDK to be extracted without preparing the build system.
--- meta/classes/populate_sdk_ext.bbclass | 12 +++++++----- meta/files/toolchain-shar-extract.sh | 7 ++++++- 2 files changed, 13 insertions(+), 6 deletions(-)
meta/classes/populate_sdk_ext.bbclass +7 -5
@@ -217,11 +217,13 @@ sdk_ext_postinst() { # For now this is where uninative.bbclass expects the tarball mv *-nativesdk-libc.tar.* $target_sdk_dir/`dirname ${oe_init_build_env_path}` - printf "Preparing build system...\n" - # dash which is /bin/sh on Ubuntu will not preserve the - # current working directory when first ran, nor will it set $1 when - # sourcing a script. That is why this has to look so ugly. - sh -c ". buildtools/environment-setup* > preparing_build_system.log && cd $target_sdk_dir/`dirname ${oe_init_build_env_path}` && set $target_sdk_dir && . $target_sdk_dir/${oe_init_build_env_path} $target_sdk_dir >> preparing_build_system.log && bitbake ${SDK_TARGETS} >> preparing_build_system.log" || { echo "SDK preparation failed: see `pwd`/preparing_build_system.log" ; exit 1 ; } + if [ "$prepare_buildsystem" != "no" ]; then + printf "Preparing build system...\n" + # dash which is /bin/sh on Ubuntu will not preserve the + # current working directory when first ran, nor will it set $1 when + # sourcing a script. That is why this has to look so ugly. + sh -c ". buildtools/environment-setup* > preparing_build_system.log && cd $target_sdk_dir/`dirname ${oe_init_build_env_path}` && set $target_sdk_dir && . $target_sdk_dir/${oe_init_build_env_path} $target_sdk_dir >> preparing_build_system.log && bitbake ${SDK_TARGETS} >> preparing_build_system.log" || { echo "SDK preparation failed: see `pwd`/preparing_build_system.log" ; exit 1 ; } + fi echo done }
meta/files/toolchain-shar-extract.sh +6 -1
@@ -32,7 +32,7 @@ answer="" relocate=1 savescripts=0 verbose=0 -while getopts ":yd:DRS" OPT; do +while getopts ":yd:nDRS" OPT; do case $OPT in y) answer="Y"
@@ -41,6 +41,9 @@ while getopts ":yd:DRS" OPT; do d) target_sdk_dir=$OPTARG ;; + n) + prepare_buildsystem="no" + ;; D) verbose=1 ;;
@@ -55,6 +58,8 @@ while getopts ":yd:DRS" OPT; do echo "Usage: $(basename $0) [-y] [-d <dir>]" echo " -y Automatic yes to all prompts" echo " -d <dir> Install the SDK to <dir>" + echo "======== Extensible SDK only options ============" + echo " -n Do not prepare the build system" echo "======== Advanced DEBUGGING ONLY OPTIONS ========" echo " -S Save relocation scripts" echo " -R Do not relocate executables"
Signature · 1 line
1.9.1
PATCH v3 5/8 oe-publish-sdk: add script
In reply to Chen QiAdd a script to publish extensible SDK to a specified destination.
e.g.
oe-publish-sdk <ext-sdk> <destination>
--- scripts/oe-publish-sdk | 143 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 143 insertions(+) create mode 100755 scripts/oe-publish-sdk
scripts/oe-publish-sdk new file +143 -0
@@ -0,0 +1,143 @@ +#!/usr/bin/env python + +# OpenEmbedded SDK Tool + +# oe-publish-sdk publish <ext-sdk> <destination> +# <ext-sdk>: extensible SDK to publish +# <destination>: local or remote location which servers as an SDK update server +# e.g. +# oe-publish-sdk publish sdk-ext.sh /mnt/poky/sdk-ext +# oe-publish-sdk publish sdk-ext.sh user@host:/opt/poky/sdk-ext +# + +import sys +import os +import argparse +import glob +import re +import subprocess +import logging +import shutil +import errno + +scripts_path = os.path.dirname(os.path.realpath(__file__)) +lib_path = scripts_path + '/lib' +sys.path = sys.path + [lib_path] +import scriptutils +logger = scriptutils.logger_create('sdktool') + +def mkdir(d): + try: + os.makedirs(d) + except OSError as e: + if e.errno != errno.EEXIST: + raise e + +def publish(args): + logger.debug("In publish function") + target_sdk = args.sdk + destination = args.dest + logger.debug("target_sdk = %s, update_server = %s" % (target_sdk, destination)) + sdk_basename = os.path.basename(target_sdk) + + # Ensure the SDK exists + if not os.path.exists(target_sdk): + logger.error("%s doesn't exist" % target_sdk) + return -1 + + if ':' in destination: + is_remote = True + host, destdir = destination.split(':') + dest_sdk = os.path.join(destdir, sdk_basename) + else: + is_remote = False + dest_sdk = os.path.join(destination, sdk_basename) + + # Making sure the directory exists + logger.debug("Making sure the destination directory exists") + if not is_remote: + mkdir(destination) + else: + cmd = "ssh %s 'mkdir -p %s'" % (host, destdir) + ret = subprocess.call(cmd, shell=True) + if ret != 0: + logger.error("Making directory %s on %s failed" % (destdir, host)) + return ret + + # Copying the SDK to the destination + logger.debug("Copying the SDK to destination") + if not is_remote: + if os.path.exists(dest_sdk): + os.remove(dest_sdk) + if (os.stat(target_sdk).st_dev == os.stat(destination).st_dev): + os.link(target_sdk, dest_sdk) + else: + shutil.copy(target_sdk, dest_sdk) + else: + cmd = "scp %s %s" % (target_sdk, destination) + ret = subprocess.call(cmd, shell=True) + if ret != 0: + logger.error("scp %s %s failed" % (target_sdk, destination)) + return ret + + # Unpack the SDK + logger.debug("Unpacking SDK") + if not is_remote: + cmd = "sh %s -n -y -d %s" % (dest_sdk, destination) + ret = subprocess.call(cmd, shell=True) + if ret == 0: + logger.info('Successfully unpacked %s to %s' % (dest_sdk, destination)) + else: + logger.error('Failed to unpack %s to %s' % (dest_sdk, destination)) + return ret + else: + cmd = "ssh %s 'sh %s -n -y -d %s'" % (host, dest_sdk, destdir) + ret = subprocess.call(cmd, shell=True) + if ret == 0: + logger.info('Successfully unpacked %s to %s' % (dest_sdk, destdir)) + else: + logger.error('Failed to unpack %s to %s' % (dest_sdk, destdir)) + return ret + + # Setting up the git repo + if not is_remote: + cmd = 'cd %s/layers; if [ ! -e .git ]; then git init .; mv .git/hooks/post-update.sample .git/hooks/post-update; fi; git add .; git commit -m "init repo" || true;' % destination + else: + cmd = "ssh %s 'cd %s/layers; if [ ! -e .git ]; then git init .; mv .git/hooks/post-update.sample .git/hooks/post-update; fi; git add .; git commit -m \"init repo\" || true;'" % (host, destdir) + ret = subprocess.call(cmd, shell=True) + if ret == 0: + logger.info('Git repo set up successfully') + else: + logger.error('Failed to set up git repo') + return ret + + +def main(): + parser = argparse.ArgumentParser(description="OpenEmbedded development tool", + epilog="Use %(prog)s <subcommand> --help to get help on a specific command") + parser.add_argument('-d', '--debug', help='Enable debug output', action='store_true') + parser.add_argument('-q', '--quiet', help='Print only errors', action='store_true') + + parser.add_argument('sdk', help='Extensible SDK to publish') + parser.add_argument('dest', help='Destination to publish SDK to') + + parser.set_defaults(func=publish) + + args = parser.parse_args() + + if args.debug: + logger.setLevel(logging.DEBUG) + elif args.quiet: + logger.setLevel(logging.ERROR) + + ret = args.func(args) + return ret + +if __name__ == "__main__": + try: + ret = main() + except Exception: + ret = 1 + import traceback + traceback.print_exc(5) + sys.exit(ret)
Signature · 1 line
1.9.1
PATCH v3 6/8 populate_sdk_ext: record SDK_TARGETS in devtool.conf
In reply to Chen QiRecord the value of SDK_TARGETS for later use of devtool's sdk plugin for updating SDK.
--- meta/classes/populate_sdk_ext.bbclass | 2 ++ 1 file changed, 2 insertions(+)
meta/classes/populate_sdk_ext.bbclass +2 -0
@@ -84,6 +84,8 @@ python copy_buildsystem () { config.set('General', 'bitbake_subdir', conf_bbpath) config.set('General', 'init_path', conf_initpath) config.set('General', 'core_meta_subdir', core_meta_subdir) + config.add_section('SDK') + config.set('SDK', 'sdk_targets', d.getVar('SDK_TARGETS', True)) bb.utils.mkdirhier(os.path.join(baseoutpath, 'conf')) with open(os.path.join(baseoutpath, 'conf', 'devtool.conf'), 'w') as f: config.write(f)
Signature · 1 line
1.9.1
PATCH v3 7/8 devtool: add sdk.py plugin
In reply to Chen QiThis plugin is only enabled when devtool is used within the extensible SDK. It is used to update the current SDK from a local or remote server.
E.g.
devtool sdk-update /mnt/sdk-repo/
devtool sdk-update http://mysdkhost/sdk
--- scripts/lib/devtool/sdk.py | 188 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 188 insertions(+) create mode 100644 scripts/lib/devtool/sdk.py
scripts/lib/devtool/sdk.py new file +188 -0
@@ -0,0 +1,188 @@ +# Development tool - sdk-update command plugin + +import os +import subprocess +import logging +import glob +import shutil +import errno +import sys +from devtool import exec_build_env_command, setup_tinfoil + +logger = logging.getLogger('devtool') + +def plugin_init(pluginlist): + """Plugin initialization""" + pass + +def parse_locked_sigs(sigfile_path): + """Return <pn:task>:<hash> dictionary""" + sig_dict = {} + with open(sigfile_path) as f: + lines = f.readlines() + for line in lines: + if ':' in line: + taskkey, _, hashval = line.rpartition(':') + sig_dict[taskkey.strip()] = hashval.split()[0] + return sig_dict + +def generate_update_dict(sigfile_new, sigfile_old): + """Return a dict containing <pn:task>:<hash> which indicates what need to be updated""" + update_dict = {} + sigdict_new = parse_locked_sigs(sigfile_new) + sigdict_old = parse_locked_sigs(sigfile_old) + for k in sigdict_new: + if k not in sigdict_old: + update_dict[k] = sigdict_new[k] + continue + if sigdict_new[k] != sigdict_old[k]: + update_dict[k] = sigdict_new[k] + continue + return update_dict + +def get_sstate_objects(update_dict, newsdk_path): + """Return a list containing sstate objects which are to be installed""" + sstate_objects = [] + # Ensure newsdk_path points to an extensible SDK + sstate_dir = os.path.join(newsdk_path, 'sstate-cache') + if not os.path.exists(sstate_dir): + logger.error("sstate-cache directory not found under %s" % newsdk_path) + raise + for k in update_dict: + files = set() + hashval = update_dict[k] + p = sstate_dir + '/' + hashval[:2] + '/*' + hashval + '*.tgz' + files |= set(glob.glob(p)) + p = sstate_dir + '/*/' + hashval[:2] + '/*' + hashval + '*.tgz' + files |= set(glob.glob(p)) + files = list(files) + if len(files) == 1: + sstate_objects.extend(files) + elif len(files) > 1: + logger.error("More than one matching sstate object found for %s" % hashval) + + return sstate_objects + +def mkdir(d): + try: + os.makedirs(d) + except OSError as e: + if e.errno != errno.EEXIST: + raise e + +def install_sstate_objects(sstate_objects, src_sdk, dest_sdk): + """Install sstate objects into destination SDK""" + sstate_dir = os.path.join(dest_sdk, 'sstate-cache') + if not os.path.exists(sstate_dir): + logger.error("Missing sstate-cache directory in %s, it might not be an extensible SDK." % dest_sdk) + raise + for sb in sstate_objects: + dst = sb.replace(src_sdk, dest_sdk) + destdir = os.path.dirname(dst) + mkdir(destdir) + logger.debug("Copying %s to %s" % (sb, dst)) + shutil.copy(sb, dst) + +def sdk_update(args, config, basepath, workspace): + # Fetch locked-sigs.inc file from remote/local destination + logger.debug("updateserver: %s" % args.updateserver) + + # Make sure we are using sdk-update from within SDK + logger.debug("basepath = %s" % basepath) + old_locked_sig_file_path = os.path.join(basepath, 'conf/locked-sigs.inc') + if not os.path.exists(old_locked_sig_file_path): + logger.error("Not using devtool's sdk-update command from within an extensible SDK. Please specify correct basepath via --basepath option") + return -1 + else: + logger.debug("Found conf/locked-sigs.inc in %s" % basepath) + + if ':' in args.updateserver: + is_remote = True + else: + is_remote = False + + if not is_remote: + # devtool sdk-update /local/path/to/latest/sdk + new_locked_sig_file_path = os.path.join(args.updateserver, 'conf/locked-sigs.inc') + if not os.path.exists(new_locked_sig_file_path): + logger.error("%s doesn't exist or is not an extensible SDK" % args.updateserver) + return -1 + else: + logger.debug("Found conf/locked-sigs.inc in %s" % args.updateserver) + update_dict = generate_update_dict(new_locked_sig_file_path, old_locked_sig_file_path) + logger.debug("update_dict = %s" % update_dict) + sstate_objects = get_sstate_objects(update_dict, args.updateserver) + logger.debug("sstate_objects = %s" % sstate_objects) + if len(sstate_objects) == 0: + logger.info("No need to update.") + return 0 + logger.info("Installing sstate objects into %s", basepath) + install_sstate_objects(sstate_objects, args.updateserver.rstrip('/'), basepath) + logger.info("Updating configuration files") + new_conf_dir = os.path.join(args.updateserver, 'conf') + old_conf_dir = os.path.join(basepath, 'conf') + shutil.rmtree(old_conf_dir) + shutil.copytree(new_conf_dir, old_conf_dir) + logger.info("Updating layers") + new_layers_dir = os.path.join(args.updateserver, 'layers') + old_layers_dir = os.path.join(basepath, 'layers') + shutil.rmtree(old_layers_dir) + shutil.copytree(new_layers_dir, old_layers_dir) + else: + # devtool sdk-update http://myhost/sdk + tmpsdk_dir = '/tmp/sdk-ext' + if os.path.exists(tmpsdk_dir): + shutil.rmtree(tmpsdk_dir) + os.makedirs(tmpsdk_dir) + os.makedirs(os.path.join(tmpsdk_dir, 'conf')) + # Fetch locked-sigs.inc from update server + ret = subprocess.call("wget -q -O - %s/conf/locked-sigs.inc > %s/locked-sigs.inc" % (args.updateserver, os.path.join(tmpsdk_dir, 'conf')), shell=True) + if ret != 0: + logger.error("Fetching conf/locked-sigs.inc from %s to %s/locked-sigs.inc failed" % (args.updateserver, os.path.join(tmpsdk_dir, 'conf'))) + return ret + else: + logger.info("Fetching conf/locked-sigs.inc from %s to %s/locked-sigs.inc succeeded" % (args.updateserver, os.path.join(tmpsdk_dir, 'conf'))) + new_locked_sig_file_path = os.path.join(tmpsdk_dir, 'conf/locked-sigs.inc') + update_dict = generate_update_dict(new_locked_sig_file_path, old_locked_sig_file_path) + logger.debug("update_dict = %s" % update_dict) + if len(update_dict) == 0: + logger.info("No need to update.") + return 0 + # Update metadata + logger.debug("Updating meta data via git ...") + # Try using 'git pull', if failed, use 'git clone' + if os.path.exists(os.path.join(basepath, 'layers/.git')): + ret = subprocess.call("cd layers && git pull", shell=True) + else: + ret = -1 + if ret != 0: + ret = subprocess.call("rm -rf layers && git clone %s/layers" % args.updateserver, shell=True) + if ret != 0: + logger.error("Updating meta data via git failed") + return ret + logger.debug("Updating conf files ...") + conf_files = ['local.conf', 'bblayers.conf', 'devtool.conf', 'work-config.inc', 'locked-sigs.inc'] + for conf in conf_files: + ret = subprocess.call("wget -q -O - %s/conf/%s > conf/%s" % (args.updateserver, conf, conf), shell=True) + if ret != 0: + logger.error("Update %s failed" % conf) + return ret + with open(os.path.join(basepath, 'conf/local.conf'), 'a') as f: + f.write('SSTATE_MIRRORS_append = " file://.* %s/sstate-cache/PATH \\n "\n' % args.updateserver) + + # Run bitbake command for the whole SDK + sdk_targets = config.get('SDK', 'sdk_targets') + logger.info("Executing 'bitbake %s' ... (This may take some time.)" % sdk_targets) + try: + exec_build_env_command(config.init_path, basepath, 'bitbake %s' % sdk_targets) + except: + logger.error('bitbake %s failed' % sdk_targets) + return -1 + return 0 + +def register_commands(subparsers, context): + """Register devtool subcommands from the sdk plugin""" + if context.fixed_setup: + parser_sdk = subparsers.add_parser('sdk-update', help='Update SDK components from a nominated destination') + parser_sdk.add_argument('updateserver', help='The update server to fetch latest SDK components') + parser_sdk.set_defaults(func=sdk_update)
Signature · 1 line
1.9.1
PATCH v3 8/8 populate_sdk_ext: get rid of buildtools
In reply to Chen QiIf we do bitbake buildtools-tarball' and then after one day do bitbake core-image-minimal -c populate_sdk_ext', we would meet errors like below.
| install: cannot stat '/buildarea2/chenqi/poky/build-systemd/tmp/deploy/sdk/ poky-glibc-x86_64-buildtools-tarball-core2-64-buildtools-nativesdk-standalone -1.8+snapshot-20150429.sh': No such file or directory
The problem is that the output name for buildtools-tarball has ${DATE} in it. So if populate_sdk_ext task is executed but buildtools-tarball is not rebuilt, the above error appears.
In fact, ext SDK does not have to depend on buildtools. This patch removes the buildtools from the ext SDK. After this patch, the ext SDK still works and we can no longer see the buildtools error.
[YOCTO #7674]
--- meta/classes/populate_sdk_ext.bbclass | 29 +++++++++++++++++++---------- 1 file changed, 19 insertions(+), 10 deletions(-)
meta/classes/populate_sdk_ext.bbclass +19 -10
@@ -7,8 +7,25 @@ inherit populate_sdk_base TOOLCHAIN_HOST_TASK_task-populate-sdk-ext = " \ meta-environment-extsdk-${MACHINE} \ + nativesdk-python-core \ + nativesdk-python-modules \ + nativesdk-python-misc \ + nativesdk-python-git \ + nativesdk-python-pexpect \ + nativesdk-ncurses-terminfo-base \ + nativesdk-chrpath \ + nativesdk-tar \ + nativesdk-buildtools-perl-dummy \ + nativesdk-git \ + nativesdk-git-perltools \ + nativesdk-pigz \ + nativesdk-make \ + nativesdk-wget \ + nativesdk-ca-certificates \ " +SDK_PACKAGE_ARCHS =+ "buildtools-dummy-${SDKPKGSUFFIX}" + TOOLCHAIN_TARGET_TASK_task-populate-sdk-ext = "" SDK_RDEPENDS_append_task-populate-sdk-ext = " ${SDK_TARGETS}"
@@ -183,8 +200,6 @@ install_tools() { lnr ${SDK_OUTPUT}/${SDKPATH}/${scriptrelpath}/recipetool ${SDK_OUTPUT}/${SDKPATHNATIVE}${bindir_nativesdk}/recipetool touch ${SDK_OUTPUT}/${SDKPATH}/.devtoolbase - install ${SDK_DEPLOY}/${DISTRO}-${TCLIBC}-${SDK_ARCH}-buildtools-tarball-${TUNE_PKGARCH}-buildtools-nativesdk-standalone-${DISTRO_VERSION}.sh ${SDK_OUTPUT}/${SDKPATH} - install ${SDK_DEPLOY}/${BUILD_ARCH}-nativesdk-libc.tar.bz2 ${SDK_OUTPUT}/${SDKPATH} }
@@ -201,13 +216,7 @@ SDK_PRE_INSTALL_COMMAND_task-populate-sdk-ext = "${sdk_ext_preinst}" # FIXME this preparation should be done as part of the SDK construction sdk_ext_postinst() { - printf "\nExtracting buildtools...\n" cd $target_sdk_dir - printf "buildtools\ny" | ./*buildtools-tarball* > /dev/null - - # Make sure when the user sets up the environment, they also get - # the buildtools-tarball tools in their path. - echo ". $target_sdk_dir/buildtools/environment-setup*" >> $target_sdk_dir/environment-setup* # Allow bitbake environment setup to be ran as part of this sdk. echo "export OE_SKIP_SDK_CHECK=1" >> $target_sdk_dir/environment-setup*
@@ -224,7 +233,7 @@ sdk_ext_postinst() { # dash which is /bin/sh on Ubuntu will not preserve the # current working directory when first ran, nor will it set $1 when # sourcing a script. That is why this has to look so ugly. - sh -c ". buildtools/environment-setup* > preparing_build_system.log && cd $target_sdk_dir/`dirname ${oe_init_build_env_path}` && set $target_sdk_dir && . $target_sdk_dir/${oe_init_build_env_path} $target_sdk_dir >> preparing_build_system.log && bitbake ${SDK_TARGETS} >> preparing_build_system.log" || { echo "SDK preparation failed: see `pwd`/preparing_build_system.log" ; exit 1 ; } + sh -c "cd $target_sdk_dir/`dirname ${oe_init_build_env_path}` && set $target_sdk_dir && . $target_sdk_dir/${oe_init_build_env_path} $target_sdk_dir >> preparing_build_system.log && bitbake ${SDK_TARGETS} >> preparing_build_system.log" || { echo "SDK preparation failed: see `pwd`/preparing_build_system.log" ; exit 1 ; } fi echo done }
@@ -249,7 +258,7 @@ do_populate_sdk_ext[rdepends] = "${@get_sdk_ext_rdepends(d)}" do_populate_sdk_ext[recrdeptask] += "${@d.getVarFlag('do_populate_sdk', 'recrdeptask', False)}" -do_populate_sdk_ext[depends] += "buildtools-tarball:do_populate_sdk uninative-tarball:do_populate_sdk" +do_populate_sdk_ext[depends] += "uninative-tarball:do_populate_sdk" do_populate_sdk_ext[rdepends] += "${@' '.join([x + ':do_build' for x in d.getVar('SDK_TARGETS', True).split()])}" do_populate_sdk_ext[recrdeptask] += "do_populate_lic do_package_qa do_populate_sysroot do_deploy"
Signature · 1 line
1.9.1
Re: [PATCH V3 8/8] populate_sdk_ext: get rid of buildtools
Hi Qi,
Quoted 83 lines · On Monday 24 August 2015 15:46:56 Chen Qi wrote:
If we do
bitbake buildtools-tarball' and then after one day dobitbake core-image-minimal -cpopulate_sdk_ext', we would meet errors like below.| install: cannot stat
| '/buildarea2/chenqi/poky/build-systemd/tmp/deploy/sdk/poky-glibc-x86_64-buildtools-tarball-core2-64-buildtools-nativesdk-standalon e
-1.8+snapshot-20150429.sh': No such file or directoryThe problem is that the output name for buildtools-tarball has ${DATE} in it. So if
populate_sdk_exttask is executed but buildtools-tarball is not rebuilt, the above error appears.In fact, ext SDK does not have to depend on buildtools. This patch removes the buildtools from the ext SDK. After this patch, the ext SDK still works and we can no longer see the buildtools error.
[YOCTO #7674]
Signed-off-by Chen Qi <[email protected]>--- meta/classes/populate_sdk_ext.bbclass | 29 +++++++++++++++++++---------- 1 file changed, 19 insertions(+), 10 deletions(-)(quoted diff) +0
-0b/meta/classes/populate_sdk_ext.bbclass index 151ae1d..1dcd982 100644
meta/classes/populate_sdk_ext.bbclass +17
-0@@ -7,8 +7,25 @@ inherit populate_sdk_base TOOLCHAIN_HOST_TASK_task-populate-sdk-ext = " \ meta-environment-extsdk-${MACHINE} \ + nativesdk-python-core \ + nativesdk-python-modules \ + nativesdk-python-misc \ + nativesdk-python-git \ + nativesdk-python-pexpect \ + nativesdk-ncurses-terminfo-base \ + nativesdk-chrpath \ + nativesdk-tar \ + nativesdk-buildtools-perl-dummy \ + nativesdk-git \ + nativesdk-git-perltools \ + nativesdk-pigz \ + nativesdk-make \ + nativesdk-wget \ + nativesdk-ca-certificates \ " +SDK_PACKAGE_ARCHS =+ "buildtools-dummy-${SDKPKGSUFFIX}" + TOOLCHAIN_TARGET_TASK_task-populate-sdk-ext = "" SDK_RDEPENDS_append_task-populate-sdk-ext = " ${SDK_TARGETS}"@@ -183,8 +200,6 @@ install_tools() { lnr ${SDK_OUTPUT}/${SDKPATH}/${scriptrelpath}/recipetool${SDK_OUTPUT}/${SDKPATHNATIVE}${bindir_nativesdk}/recipetool touch ${SDK_OUTPUT}/${SDKPATH}/.devtoolbase
meta/classes/populate_sdk_ext.bbclass +0
-1- install${SDK_DEPLOY}/${DISTRO}-${TCLIBC}-${SDK_ARCH}-buildtools-tarball-${TUNE_PKG ARCH}-buildtools-nativesdk-standalone-${DISTRO_VERSION}.sh ${SDK_OUTPUT}/${SDKPATH} -
install ${SDK_DEPLOY}/${BUILD_ARCH}-nativesdk-libc.tar.bz2
${SDK_OUTPUT}/${SDKPATH} }meta/classes/populate_sdk_ext.bbclass +0
-0@@ -201,13 +216,7 @@ SDK_PRE_INSTALL_COMMAND_task-populate-sdk-ext ="${sdk_ext_preinst}"
# FIXME this preparation should be done as part of the SDK construction
sdk_ext_postinst(){meta/classes/populate_sdk_ext.bbclass +0
-6- printf "\nExtracting buildtools...\n" cd $target_sdk_dir - printf "buildtools\ny" | ./*buildtools-tarball* > /dev/null - - # Make sure when the user sets up the environment, they also get - # the buildtools-tarball tools in their path. - echo ". $target_sdk_dir/buildtools/environment-setup*" >>$target_sdk_dir/environment-setup*
meta/classes/populate_sdk_ext.bbclass +0
-0# Allow bitbake environment setup to be ran as part of this sdk. echo "export OE_SKIP_SDK_CHECK=1" >> $target_sdk_dir/environment-setup*
meta/classes/populate_sdk_ext.bbclass sdk_ext_postinst() { · line 227
Quoted 4 lines
meta/classes/populate_sdk_ext.bbclass +0
-1@@ -224,7 +233,7 @@ sdk_ext_postinst() { # dash which is /bin/sh on Ubuntu will not preserve the # current working directory when first ran, nor will it set $1 when # sourcing a script. That is why this has to look so ugly. - sh -c ". buildtools/environment-setup* > preparing_build_system.log
&&
Quoted 17 lines
cd $target_sdk_dir/
dirname ${oe_init_build_env_path}&& set $target_sdk_dir && . $target_sdk_dir/${oe_init_build_env_path} $target_sdk_dir >> preparing_build_system.log && bitbake ${SDK_TARGETS} >> preparing_build_system.log" || { echo "SDK preparation failed: seepwd/preparing_build_system.log" ; exit 1 ; } + sh -c "cd
$target_sdk_dir/dirname ${oe_init_build_env_path}&& set $target_sdk_dir && . $target_sdk_dir/${oe_init_build_env_path} $target_sdk_dir >> preparing_build_system.log && bitbake ${SDK_TARGETS} >> preparing_build_system.log" || { echo "SDK preparation failed: seepwd/preparing_build_system.log" ; exit 1 ; } fimeta/classes/populate_sdk_ext.bbclass +0
-0echo done }@@ -249,7 +258,7 @@ do_populate_sdk_ext[rdepends] ="${@get_sdk_ext_rdepends(d)}" do_populate_sdk_ext[recrdeptask] += "${@d.getVarFlag('do_populate_sdk', 'recrdeptask', False)}"
Quoted 7 lines
-do_populate_sdk_ext[depends] += "buildtools-tarball:do_populate_sdk uninative-tarball:do_populate_sdk" +do_populate_sdk_ext[depends] += "uninative-tarball:do_populate_sdk"
do_populate_sdk_ext[rdepends] += "${@' '.join([x + ':do_build' for x in
d.getVar('SDK_TARGETS', True).split()])}" do_populate_sdk_ext[recrdeptask] += "do_populate_licdo_package_qado_populate_sysrootdo_deploy"
This doesn't seem to set up the environment correctly such that the components we're installing into the host part of the SDK are able to be used by the build system - prior to this change we were running the buildtools environment setup script in order to do that. (The best way to test this is try to install it on a machine that doesn't have the required git/tar/python versions e.g. CentOS 6).
Cheers,
Paul
Signature · 2 lines
Paul Eggleton Intel Open Source Technology Centre
Re: [PATCH V3 8/8] populate_sdk_ext: get rid of buildtools
Quoted 97 lines · On Tuesday 01 September 2015 17:13:49 Paul Eggleton wrote:
Hi Qi,
Quoted 94 lines · On Monday 24 August 2015 15:46:56 Chen Qi wrote:
If we do
bitbake buildtools-tarball' and then after one day dobitbake core-image-minimal -cpopulate_sdk_ext', we would meet errors like below.| install: cannot stat
| '/buildarea2/chenqi/poky/build-systemd/tmp/deploy/sdk/poky-glibc-x86_64-buildtools-tarball-core2-64-buildtools-nativesdk-standal on e
-1.8+snapshot-20150429.sh': No such file or directoryThe problem is that the output name for buildtools-tarball has ${DATE} in it. So if
populate_sdk_exttask is executed but buildtools-tarball is not rebuilt, the above error appears.In fact, ext SDK does not have to depend on buildtools. This patch removes the buildtools from the ext SDK. After this patch, the ext SDK still works and we can no longer see the buildtools error.
[YOCTO #7674]
Signed-off-by Chen Qi <[email protected]>---
meta/classes/populate_sdk_ext.bbclass | 29 +++++++++++++++++++---------- 1 file changed, 19 insertions(+), 10 deletions(-)(quoted diff) +0
-0b/meta/classes/populate_sdk_ext.bbclass index 151ae1d..1dcd982 100644
meta/classes/populate_sdk_ext.bbclass +17
-0@@ -7,8 +7,25 @@ inherit populate_sdk_base TOOLCHAIN_HOST_TASK_task-populate-sdk-ext = " \ meta-environment-extsdk-${MACHINE} \ + nativesdk-python-core \ + nativesdk-python-modules \ + nativesdk-python-misc \ + nativesdk-python-git \ + nativesdk-python-pexpect \ + nativesdk-ncurses-terminfo-base \ + nativesdk-chrpath \ + nativesdk-tar \ + nativesdk-buildtools-perl-dummy \ + nativesdk-git \ + nativesdk-git-perltools \ + nativesdk-pigz \ + nativesdk-make \ + nativesdk-wget \ + nativesdk-ca-certificates \ " +SDK_PACKAGE_ARCHS =+ "buildtools-dummy-${SDKPKGSUFFIX}" +TOOLCHAIN_TARGET_TASK_task-populate-sdk-ext = ""
SDK_RDEPENDS_append_task-populate-sdk-ext = " ${SDK_TARGETS}"
meta/classes/populate_sdk_ext.bbclass +0
-0@@ -183,8 +200,6 @@ install_tools() { lnr ${SDK_OUTPUT}/${SDKPATH}/${scriptrelpath}/recipetool${SDK_OUTPUT}/${SDKPATHNATIVE}${bindir_nativesdk}/recipetool touch ${SDK_OUTPUT}/${SDKPATH}/.devtoolbase
meta/classes/populate_sdk_ext.bbclass +0
-1- install${SDK_DEPLOY}/${DISTRO}-${TCLIBC}-${SDK_ARCH}-buildtools-tarball-${TUNE_PK G
ARCH}-buildtools-nativesdk-standalone-${DISTRO_VERSION}.sh ${SDK_OUTPUT}/${SDKPATH} -install ${SDK_DEPLOY}/${BUILD_ARCH}-nativesdk-libc.tar.bz2
${SDK_OUTPUT}/${SDKPATH} }
meta/classes/populate_sdk_ext.bbclass +0
-0@@ -201,13 +216,7 @@ SDK_PRE_INSTALL_COMMAND_task-populate-sdk-ext ="${sdk_ext_preinst}"
# FIXME this preparation should be done as part of the SDK construction
sdk_ext_postinst(){meta/classes/populate_sdk_ext.bbclass +0
-6- printf "\nExtracting buildtools...\n" cd $target_sdk_dir - printf "buildtools\ny" | ./*buildtools-tarball* > /dev/null - - # Make sure when the user sets up the environment, they also get - # the buildtools-tarball tools in their path. - echo ". $target_sdk_dir/buildtools/environment-setup*" >>$target_sdk_dir/environment-setup*
Quoted 2 lines
meta/classes/populate_sdk_ext.bbclass
Quoted 2 lines
meta/classes/populate_sdk_ext.bbclass +0
-0# Allow bitbake environment setup to be ran as part of this sdk. echo "export OE_SKIP_SDK_CHECK=1" >> $target_sdk_dir/environment-
setup*
Quoted 5 lines
meta/classes/populate_sdk_ext.bbclass sdk_ext_postinst() { · line 235
Quoted 5 lines
meta/classes/populate_sdk_ext.bbclass +0
-0@@ -224,7 +233,7 @@ sdk_ext_postinst() { # dash which is /bin/sh on Ubuntu will not preserve the # current working directory when first ran, nor will it set $1
when
Quoted 3 lines
meta/classes/populate_sdk_ext.bbclass sdk_ext_postinst() { · line 229
Quoted 3 lines
meta/classes/populate_sdk_ext.bbclass +0
-1# sourcing a script. That is why this has to look so ugly. - sh -c ". buildtools/environment-setup* >
preparing_build_system.log
Quoted 29 lines
&&
Quoted 26 lines
cd $target_sdk_dir/
dirname ${oe_init_build_env_path}&& set $target_sdk_dir && . $target_sdk_dir/${oe_init_build_env_path} $target_sdk_dir >> preparing_build_system.log && bitbake ${SDK_TARGETS} >> preparing_build_system.log" || { echo "SDK preparation failed: seepwd/preparing_build_system.log" ; exit 1 ; } + sh -c "cd
$target_sdk_dir/dirname ${oe_init_build_env_path}&& set $target_sdk_dir && . $target_sdk_dir/${oe_init_build_env_path} $target_sdk_dir >> preparing_build_system.log && bitbake ${SDK_TARGETS} >> preparing_build_system.log" || { echo "SDK preparation failed: seepwd/preparing_build_system.log" ; exit 1 ; } fimeta/classes/populate_sdk_ext.bbclass +0
-0echo done }@@ -249,7 +258,7 @@ do_populate_sdk_ext[rdepends] ="${@get_sdk_ext_rdepends(d)}" do_populate_sdk_ext[recrdeptask] += "${@d.getVarFlag('do_populate_sdk', 'recrdeptask', False)}"
-do_populate_sdk_ext[depends] += "buildtools-tarball:do_populate_sdk uninative-tarball:do_populate_sdk" +do_populate_sdk_ext[depends] += "uninative-tarball:do_populate_sdk"
do_populate_sdk_ext[rdepends] += "${@' '.join([x + ':do_build' for x in
Quoted 9 lines
Quoted 2 lines
d.getVar('SDK_TARGETS', True).split()])}" do_populate_sdk_ext[recrdeptask] += "do_populate_licdo_package_qado_populate_sysrootdo_deploy"This doesn't seem to set up the environment correctly such that the components we're installing into the host part of the SDK are able to be used by the build system - prior to this change we were running the buildtools environment setup script in order to do that. (The best way to test this is try to install it on a machine that doesn't have the required git/tar/python versions e.g. CentOS 6).
Unfortunately there's another problem - the relocation isn't happening for binaries in the host part of the SDK. We can enable that by dropping the SDK_RELOCATE_AFTER_INSTALL_task-populate-sdk-ext line but then meta- environment-extsdk is setting up the paths for uninative to work and that means that the relocation script is looking in the wrong place for the native sysroot.
Ugly as it is we may be better off sticking with the buildtools solution for now and working around the problems there.
Cheers,
Paul
Signature · 2 lines
Paul Eggleton Intel Open Source Technology Centre
Re: [PATCH V3 8/8] populate_sdk_ext: get rid of buildtools
Quoted 172 lines · On Tuesday 01 September 2015 18:35:53 Paul Eggleton wrote:
Quoted 105 lines · On Tuesday 01 September 2015 17:13:49 Paul Eggleton wrote:
Hi Qi,
Quoted 102 lines · On Monday 24 August 2015 15:46:56 Chen Qi wrote:
If we do
bitbake buildtools-tarball' and then after one day dobitbake core-image-minimal -cpopulate_sdk_ext', we would meet errors like below.| install: cannot stat
| '/buildarea2/chenqi/poky/build-systemd/tmp/deploy/sdk/poky-glibc-x86_64-buildtools-tarball-core2-64-buildtools-nativesdk-stand al
on e-1.8+snapshot-20150429.sh': No such file or directoryThe problem is that the output name for buildtools-tarball has ${DATE} in
it. So ifpopulate_sdk_exttask is executed but buildtools-tarball is not
rebuilt, the above error appears.In fact, ext SDK does not have to depend on buildtools. This patch removes
the buildtools from the ext SDK. After this patch, the ext SDK still works
and we can no longer see the buildtools error.[YOCTO #7674]
Signed-off-by Chen Qi <[email protected]>---
meta/classes/populate_sdk_ext.bbclass | 29 +++++++++++++++++++---------- 1 file changed, 19 insertions(+), 10 deletions(-)(quoted diff) +0
-0b/meta/classes/populate_sdk_ext.bbclass index 151ae1d..1dcd982 100644
meta/classes/populate_sdk_ext.bbclass +17
-0@@ -7,8 +7,25 @@ inherit populate_sdk_base TOOLCHAIN_HOST_TASK_task-populate-sdk-ext = " \ meta-environment-extsdk-${MACHINE} \ + nativesdk-python-core \ + nativesdk-python-modules \ + nativesdk-python-misc \ + nativesdk-python-git \ + nativesdk-python-pexpect \ + nativesdk-ncurses-terminfo-base \ + nativesdk-chrpath \ + nativesdk-tar \ + nativesdk-buildtools-perl-dummy \ + nativesdk-git \ + nativesdk-git-perltools \ + nativesdk-pigz \ + nativesdk-make \ + nativesdk-wget \ + nativesdk-ca-certificates \ " +SDK_PACKAGE_ARCHS =+ "buildtools-dummy-${SDKPKGSUFFIX}" +TOOLCHAIN_TARGET_TASK_task-populate-sdk-ext = ""
SDK_RDEPENDS_append_task-populate-sdk-ext = " ${SDK_TARGETS}"
meta/classes/populate_sdk_ext.bbclass +0
-0@@ -183,8 +200,6 @@ install_tools() { lnr ${SDK_OUTPUT}/${SDKPATH}/${scriptrelpath}/recipetool${SDK_OUTPUT}/${SDKPATHNATIVE}${bindir_nativesdk}/recipetool touch ${SDK_OUTPUT}/${SDKPATH}/.devtoolbase
meta/classes/populate_sdk_ext.bbclass +0
-1- install${SDK_DEPLOY}/${DISTRO}-${TCLIBC}-${SDK_ARCH}-buildtools-tarball-${TUNE_ PK
G
ARCH}-buildtools-nativesdk-standalone-${DISTRO_VERSION}.sh ${SDK_OUTPUT}/${SDKPATH} -install ${SDK_DEPLOY}/${BUILD_ARCH}-nativesdk-libc.tar.bz2
${SDK_OUTPUT}/${SDKPATH} }
meta/classes/populate_sdk_ext.bbclass +0
-0@@ -201,13 +216,7 @@ SDK_PRE_INSTALL_COMMAND_task-populate-sdk-ext ="${sdk_ext_preinst}"
# FIXME this preparation should be done as part of the SDK construction
sdk_ext_postinst(){meta/classes/populate_sdk_ext.bbclass +0
-6- printf "\nExtracting buildtools...\n" cd $target_sdk_dir - printf "buildtools\ny" | ./*buildtools-tarball* > /dev/null - - # Make sure when the user sets up the environment, they also get - # the buildtools-tarball tools in their path. - echo ". $target_sdk_dir/buildtools/environment-setup*" >>$target_sdk_dir/environment-setup*
Quoted 2 lines
meta/classes/populate_sdk_ext.bbclass
Quoted 2 lines
meta/classes/populate_sdk_ext.bbclass +0
-0# Allow bitbake environment setup to be ran as part of this sdk. echo "export OE_SKIP_SDK_CHECK=1" >> $target_sdk_dir/environment-setup*
Quoted 4 lines
meta/classes/populate_sdk_ext.bbclass sdk_ext_postinst() { · line 235
Quoted 4 lines
meta/classes/populate_sdk_ext.bbclass +0
-0@@ -224,7 +233,7 @@ sdk_ext_postinst() { # dash which is /bin/sh on Ubuntu will not preserve the # current working directory when first ran, nor will it set $1when
Quoted 3 lines
meta/classes/populate_sdk_ext.bbclass sdk_ext_postinst() { · line 229
Quoted 3 lines
meta/classes/populate_sdk_ext.bbclass +0
-1# sourcing a script. That is why this has to look so ugly. - sh -c ". buildtools/environment-setup* >preparing_build_system.log
Quoted 30 lines
&&
Quoted 28 lines
cd $target_sdk_dir/
dirname ${oe_init_build_env_path}&& set $target_sdk_dir && . $target_sdk_dir/${oe_init_build_env_path} $target_sdk_dir >> preparing_build_system.log && bitbake ${SDK_TARGETS}Quoted 1 line
Quoted 1 line
preparing_build_system.log" || { echo "SDK preparation failed: see
pwd/preparing_build_system.log" ; exit 1 ; } + sh -c "cd
$target_sdk_dir/dirname ${oe_init_build_env_path}&& set $target_sdk_dir
&& . $target_sdk_dir/${oe_init_build_env_path} $target_sdk_dir >> preparing_build_system.log && bitbake ${SDK_TARGETS} >> preparing_build_system.log" || { echo "SDK preparation failed: seepwd/preparing_build_system.log" ; exit 1 ; } fimeta/classes/populate_sdk_ext.bbclass +0
-0echo done }@@ -249,7 +258,7 @@ do_populate_sdk_ext[rdepends] ="${@get_sdk_ext_rdepends(d)}" do_populate_sdk_ext[recrdeptask] += "${@d.getVarFlag('do_populate_sdk', 'recrdeptask', False)}"
-do_populate_sdk_ext[depends] += "buildtools-tarball:do_populate_sdk uninative-tarball:do_populate_sdk" +do_populate_sdk_ext[depends] += "uninative-tarball:do_populate_sdk"
do_populate_sdk_ext[rdepends] += "${@' '.join([x + ':do_build' for x in
Quoted 10 lines
Quoted 3 lines
d.getVar('SDK_TARGETS', True).split()])}" do_populate_sdk_ext[recrdeptask]
+= "do_populate_licdo_package_qado_populate_sysrootdo_deploy"This doesn't seem to set up the environment correctly such that the components we're installing into the host part of the SDK are able to be used by the build system - prior to this change we were running the buildtools environment setup script in order to do that. (The best way to test this is try to install it on a machine that doesn't have the required git/tar/python versions e.g. CentOS 6).
Unfortunately there's another problem - the relocation isn't happening for binaries in the host part of the SDK. We can enable that by dropping the SDK_RELOCATE_AFTER_INSTALL_task-populate-sdk-ext line but then meta- environment-extsdk is setting up the paths for uninative to work and that means that the relocation script is looking in the wrong place for the native sysroot.
Quoted 2 lines
Ugly as it is we may be better off sticking with the buildtools solution for now and working around the problems there.
FYI I have slightly reworked your earlier fix for the buildtools filename matching and will be sending it in a new series that includes the rest of this series plus Brendan's set of fixes.
Cheers,
Paul
Signature · 2 lines
Paul Eggleton Intel Open Source Technology Centre
Re: [PATCH V3 8/8] populate_sdk_ext: get rid of buildtools
Quoted 167 lines · Paul Eggleton
Quoted 166 lines · On Tuesday 01 September 2015 18:35:53 Paul Eggleton wrote:
Quoted 105 lines · On Tuesday 01 September 2015 17:13:49 Paul Eggleton wrote:
Hi Qi,
Quoted 102 lines · On Monday 24 August 2015 15:46:56 Chen Qi wrote:
If we do
bitbake buildtools-tarball' and then after one day dobitbake core-image-minimal -cpopulate_sdk_ext', we would meet errors like below.| install: cannot stat
| '/buildarea2/chenqi/poky/build-systemd/tmp/deploy/sdk/poky-glibc-x86_64-buildtools-tarball-core2-64-buildtools-nativesdk-stand al
on e-1.8+snapshot-20150429.sh': No such file or directoryThe problem is that the output name for buildtools-tarball has ${DATE} in
it. So ifpopulate_sdk_exttask is executed but buildtools-tarball is not
rebuilt, the above error appears.In fact, ext SDK does not have to depend on buildtools. This patch removes
the buildtools from the ext SDK. After this patch, the ext SDK still works
and we can no longer see the buildtools error.[YOCTO #7674]
Signed-off-by Chen Qi <[email protected]>---
meta/classes/populate_sdk_ext.bbclass | 29 +++++++++++++++++++---------- 1 file changed, 19 insertions(+), 10 deletions(-)(quoted diff) +0
-0b/meta/classes/populate_sdk_ext.bbclass index 151ae1d..1dcd982 100644
meta/classes/populate_sdk_ext.bbclass +17
-0@@ -7,8 +7,25 @@ inherit populate_sdk_base TOOLCHAIN_HOST_TASK_task-populate-sdk-ext = " \ meta-environment-extsdk-${MACHINE} \ + nativesdk-python-core \ + nativesdk-python-modules \ + nativesdk-python-misc \ + nativesdk-python-git \ + nativesdk-python-pexpect \ + nativesdk-ncurses-terminfo-base \ + nativesdk-chrpath \ + nativesdk-tar \ + nativesdk-buildtools-perl-dummy \ + nativesdk-git \ + nativesdk-git-perltools \ + nativesdk-pigz \ + nativesdk-make \ + nativesdk-wget \ + nativesdk-ca-certificates \ " +SDK_PACKAGE_ARCHS =+ "buildtools-dummy-${SDKPKGSUFFIX}" +TOOLCHAIN_TARGET_TASK_task-populate-sdk-ext = ""
SDK_RDEPENDS_append_task-populate-sdk-ext = " ${SDK_TARGETS}"meta/classes/populate_sdk_ext.bbclass +0
-0@@ -183,8 +200,6 @@ install_tools() { lnr ${SDK_OUTPUT}/${SDKPATH}/${scriptrelpath}/recipetool${SDK_OUTPUT}/${SDKPATHNATIVE}${bindir_nativesdk}/recipetool touch ${SDK_OUTPUT}/${SDKPATH}/.devtoolbase
meta/classes/populate_sdk_ext.bbclass +0
-1- install${SDK_DEPLOY}/${DISTRO}-${TCLIBC}-${SDK_ARCH}-buildtools-tarball-${TUNE_ PK
G
ARCH}-buildtools-nativesdk-standalone-${DISTRO_VERSION}.sh ${SDK_OUTPUT}/${SDKPATH} -install ${SDK_DEPLOY}/${BUILD_ARCH}-nativesdk-libc.tar.bz2${SDK_OUTPUT}/${SDKPATH} }
meta/classes/populate_sdk_ext.bbclass +0
-0@@ -201,13 +216,7 @@ SDK_PRE_INSTALL_COMMAND_task-populate-sdk-ext ="${sdk_ext_preinst}"
# FIXME this preparation should be done as part of the SDK construction sdk_ext_postinst() {meta/classes/populate_sdk_ext.bbclass +0
-6- printf "\nExtracting buildtools...\n" cd $target_sdk_dir - printf "buildtools\ny" | ./*buildtools-tarball* > /dev/null - - # Make sure when the user sets up the environment, they also get - # the buildtools-tarball tools in their path. - echo ". $target_sdk_dir/buildtools/environment-setup*" >>$target_sdk_dir/environment-setup*
Quoted 2 lines
Quoted 2 lines
# Allow bitbake environment setup to be ran as part of this sdk. echo "export OE_SKIP_SDK_CHECK=1" >> $target_sdk_dir/environment-
setup*
Quoted 4 lines
meta/classes/populate_sdk_ext.bbclass sdk_ext_postinst() { · line 235
Quoted 4 lines
meta/classes/populate_sdk_ext.bbclass +0
-0@@ -224,7 +233,7 @@ sdk_ext_postinst() { # dash which is /bin/sh on Ubuntu will not preserve the # current working directory when first ran, nor will it set $1when
Quoted 3 lines
meta/classes/populate_sdk_ext.bbclass sdk_ext_postinst() { · line 229
Quoted 3 lines
meta/classes/populate_sdk_ext.bbclass +0
-1# sourcing a script. That is why this has to look so ugly. - sh -c ". buildtools/environment-setup* >preparing_build_system.log
Quoted 29 lines
&&
Quoted 27 lines
cd $target_sdk_dir/
dirname ${oe_init_build_env_path}&& set $target_sdk_dir && . $target_sdk_dir/${oe_init_build_env_path} $target_sdk_dir >> preparing_build_system.log && bitbake ${SDK_TARGETS} preparing_build_system.log" || { echo "SDK preparation failed: seepwd/preparing_build_system.log" ; exit 1 ; } + sh -c "cd
$target_sdk_dir/dirname ${oe_init_build_env_path}&& set $target_sdk_dir
&& . $target_sdk_dir/${oe_init_build_env_path} $target_sdk_dir >> preparing_build_system.log && bitbake ${SDK_TARGETS} >> preparing_build_system.log" || { echo "SDK preparation failed: seepwd/preparing_build_system.log" ; exit 1 ; } fiecho done
}meta/classes/populate_sdk_ext.bbclass +0
-0@@ -249,7 +258,7 @@ do_populate_sdk_ext[rdepends] ="${@get_sdk_ext_rdepends(d)}" do_populate_sdk_ext[recrdeptask] += "${@d.getVarFlag('do_populate_sdk', 'recrdeptask', False)}"
-do_populate_sdk_ext[depends] += "buildtools-tarball:do_populate_sdk uninative-tarball:do_populate_sdk" +do_populate_sdk_ext[depends] += "uninative-tarball:do_populate_sdk"
do_populate_sdk_ext[rdepends] += "${@' '.join([x + ':do_build' for x inQuoted 9 lines
Quoted 3 lines
d.getVar('SDK_TARGETS', True).split()])}" do_populate_sdk_ext[recrdeptask]
+= "do_populate_licdo_package_qado_populate_sysrootdo_deploy"This doesn't seem to set up the environment correctly such that the components we're installing into the host part of the SDK are able to be used by the build system - prior to this change we were running the buildtools environment setup script in order to do that. (The best way to test this is try to install it on a machine that doesn't have the required git/tar/python versions e.g. CentOS 6).
Unfortunately there's another problem - the relocation isn't happening for binaries in the host part of the SDK. We can enable that by dropping the SDK_RELOCATE_AFTER_INSTALL_task-populate-sdk-ext line but then meta- environment-extsdk is setting up the paths for uninative to work and that means that the relocation script is looking in the wrong place for the native sysroot.
Quoted 9 lines
Quoted 2 lines
Ugly as it is we may be better off sticking with the buildtools solution for now and working around the problems there.
FYI I have slightly reworked your earlier fix for the buildtools filename matching and will be sending it in a new series that includes the rest of this series plus Brendan's set of fixes.
Cheers,
Paul
Thanks a lot, Paul.
Best Regards,
Chen Qi