devtool: Support multiconfig workflows and fix kernel extraction
cover devtool: Support multiconfig workflows and fix kernel extraction
This series adds multiconfig support to devtool workflows, isolates workspace changes between configurations, and fixes kernel configure failure handling and source revision baselines.
It also adds selftests for multiconfig workflows and isolation.
This series addresses [YOCTO #13933].
Anusmita Dutta Mazumder (5):
devtool: Support multiconfig recipe workflows
devtool: Check kernel configure result
devtool: Isolate multiconfig workspace entries
devtool: Record kernel baseline after revision validation
oe-selftest: devtool: Add multiconfig workflow tests
meta/classes/devtool-source.bbclass | 3 +- meta/lib/oeqa/selftest/cases/devtool.py | 133 ++++++++++++++++++++++++ scripts/devtool | 24 ++++- scripts/lib/devtool/__init__.py | 8 +- scripts/lib/devtool/standard.py | 88 ++++++++++++---- 5 files changed, 230 insertions(+), 26 deletions(-)
base-commit: ae5cafacec8e99c5a27fa5b3b378eefc595910ef
PATCH 1/5 devtool: Support multiconfig recipe workflows
Preserve multiconfig-qualified targets when extracting and configuring sources.
Use the recipe name without the multiconfig prefix for workspace lookups. Keep the prefix when calling BitBake from reset, update-recipe, and finish.
Fixes [YOCTO #13933]
--- scripts/lib/devtool/standard.py | 78 +++++++++++++++++++++++++-------- 1 file changed, 60 insertions(+), 18 deletions(-)
scripts/lib/devtool/standard.py +60 -18
@@ -23,6 +23,7 @@ from collections import OrderedDict from devtool import exec_build_env_command, setup_tinfoil, check_workspace_recipe, use_external_build, setup_git_repo, recipe_to_append, get_bbclassextend_targets, update_unlockedsigs, check_prerelease_version, check_git_repo_dirty, check_git_repo_op, DevtoolError from devtool import parse_recipe +import bb.runqueue import bb.utils logger = logging.getLogger('devtool')
@@ -30,6 +31,21 @@ logger = logging.getLogger('devtool') override_branch_prefix = 'devtool-override-' +def _prefix_multiconfig_targets(mc, targets): + """Add a multiconfig prefix to build targets.""" + if not mc: + return targets + + return ' '.join(target if target.startswith('mc:') else + 'mc:%s:%s' % (mc, target) + for target in targets.split()) + + +def _get_multiconfig_targets(d, targets): + """Add the recipe datastore's multiconfig prefix to build targets.""" + return _prefix_multiconfig_targets(d.getVar('BB_CURRENT_MC'), targets) + + def add(args, config, basepath, workspace): """Entry point for the devtool 'add' subcommand""" import bb.data
@@ -596,7 +612,7 @@ def _extract_source(srctree, keep_temp, devbranch, sync, config, basepath, works task = None # Run the fetch + unpack tasks - res = tinfoil.build_targets(pn, + res = tinfoil.build_targets(_get_multiconfig_targets(d, pn), task, handle_events=True) finally:
@@ -829,7 +845,8 @@ def modify(args, config, basepath, workspace): f.write('%s\n' % fname) task = 'do_configure' - res = tinfoil.build_targets(pn, task, handle_events=True) + target = _get_multiconfig_targets(rd, pn) + res = tinfoil.build_targets(target, task, handle_events=True) # Copy .config to workspace kconfpath = rd.getVar('B')
@@ -1919,7 +1936,7 @@ def _update_recipe(recipename, workspace, rd, mode, appendlayerdir, wildcard_ver def update_recipe(args, config, basepath, workspace): """Entry point for the devtool 'update-recipe' subcommand""" - check_workspace_recipe(workspace, args.recipename) + workspacepn = check_workspace_recipe(workspace, args.recipename) if args.append: if not os.path.exists(args.append):
@@ -1941,7 +1958,11 @@ def update_recipe(args, config, basepath, workspace): if args.dry_run: dry_run_output = tempfile.TemporaryDirectory(prefix='devtool') dry_run_outdir = dry_run_output.name - updated, _, _ = _update_recipe(args.recipename, workspace, rd, args.mode, args.append, args.wildcard_version, args.no_remove, args.initial_rev, dry_run_outdir=dry_run_outdir, no_overrides=args.no_overrides, force_patch_refresh=args.force_patch_refresh) + updated, _, _ = _update_recipe( + workspacepn, workspace, rd, args.mode, args.append, + args.wildcard_version, args.no_remove, args.initial_rev, + dry_run_outdir=dry_run_outdir, no_overrides=args.no_overrides, + force_patch_refresh=args.force_patch_refresh) if updated: rf = rd.getVar('FILE')
@@ -1968,11 +1989,17 @@ def status(args, config, basepath, workspace): return 0 -def _reset(recipes, no_clean, remove_work, config, basepath, workspace): +def _reset(recipes, no_clean, remove_work, config, basepath, workspace, + bitbake_targets=None): """Reset one or more recipes""" import bb.process import oe.path + if bitbake_targets is None: + bitbake_targets = recipes + if len(bitbake_targets) != len(recipes): + raise DevtoolError('Reset recipes and BitBake targets do not match') + def clean_preferred_provider(pn, layerconf_path): """Remove PREFERRED_PROVIDER from layer.conf'""" import re
@@ -2002,11 +2029,15 @@ def _reset(recipes, no_clean, remove_work, config, basepath, workspace): # it uses BBCLASSEXTEND, then we need to also clean the other # variants targets = [] - for recipe in recipes: - targets.append(recipe) + for recipe, bitbake_target in zip(recipes, bitbake_targets): + targets.append(bitbake_target) recipefile = workspace[recipe]['recipefile'] if recipefile and os.path.exists(recipefile): - targets.extend(get_bbclassextend_targets(recipefile, recipe)) + variants = get_bbclassextend_targets(recipefile, recipe) + mc, _ = bb.runqueue.split_mc(bitbake_target) + variants = _prefix_multiconfig_targets(mc, + ' '.join(variants)) + targets.extend(variants.split()) try: exec_build_env_command(config.init_path, basepath, 'bitbake -c clean %s' % ' '.join(targets)) except bb.process.ExecutionError as e:
@@ -2086,23 +2117,27 @@ def _reset(recipes, no_clean, remove_work, config, basepath, workspace): def reset(args, config, basepath, workspace): """Entry point for the devtool 'reset' subcommand""" - recipes = "" + recipes = [] + bitbake_targets = [] if args.recipename: if args.all: raise DevtoolError("Recipe cannot be specified if -a/--all is used") else: for recipe in args.recipename: - check_workspace_recipe(workspace, recipe, checksrc=False) + workspacepn = check_workspace_recipe(workspace, recipe, + checksrc=False) + recipes.append(workspacepn) + bitbake_targets.append(recipe) elif not args.all: raise DevtoolError("Recipe must be specified, or specify -a/--all to " "reset all recipes") if args.all: recipes = list(workspace.keys()) - else: - recipes = args.recipename + bitbake_targets = recipes - _reset(recipes, args.no_clean, args.remove_work, config, basepath, workspace) + _reset(recipes, args.no_clean, args.remove_work, config, basepath, workspace, + bitbake_targets=bitbake_targets) return 0
@@ -2137,14 +2172,14 @@ def finish(args, config, basepath, workspace): import bb import oe.recipeutils - check_workspace_recipe(workspace, args.recipename) + workspacepn = check_workspace_recipe(workspace, args.recipename) dry_run_suffix = ' (dry-run)' if args.dry_run else '' # Grab the equivalent of COREBASE without having to initialise tinfoil corebasedir = os.path.abspath(os.path.join(os.path.dirname(__file__), '..', '..', '..')) - srctree = workspace[args.recipename]['srctree'] + srctree = workspace[workspacepn]['srctree'] check_git_repo_op(srctree, [corebasedir]) dirty = check_git_repo_dirty(srctree) if dirty:
@@ -2175,7 +2210,7 @@ def finish(args, config, basepath, workspace): # If it's an upgrade, grab the original path origpath = None origfilelist = None - append = workspace[args.recipename]['bbappend'] + append = workspace[workspacepn]['bbappend'] with open(append, 'r') as f: for line in f: if line.startswith('# original_path:'):
@@ -2217,7 +2252,12 @@ def finish(args, config, basepath, workspace): if args.dry_run: dry_run_output = tempfile.TemporaryDirectory(prefix='devtool') dry_run_outdir = dry_run_output.name - updated, appendfile, removed = _update_recipe(args.recipename, workspace, rd, args.mode, appendlayerdir, wildcard_version=True, no_remove=False, no_report_remove=removing_original, initial_rev=args.initial_rev, dry_run_outdir=dry_run_outdir, no_overrides=args.no_overrides, force_patch_refresh=args.force_patch_refresh) + updated, appendfile, removed = _update_recipe( + workspacepn, workspace, rd, args.mode, appendlayerdir, + wildcard_version=True, no_remove=False, + no_report_remove=removing_original, initial_rev=args.initial_rev, + dry_run_outdir=dry_run_outdir, no_overrides=args.no_overrides, + force_patch_refresh=args.force_patch_refresh) removed = [os.path.relpath(pth, recipedir) for pth in removed] # Remove any old files in the case of an upgrade
@@ -2306,7 +2346,9 @@ def finish(args, config, basepath, workspace): if args.dry_run: logger.info('Resetting recipe (dry-run)') else: - _reset([args.recipename], no_clean=no_clean, remove_work=remove_work, config=config, basepath=basepath, workspace=workspace) + _reset([workspacepn], no_clean=no_clean, remove_work=remove_work, + config=config, basepath=basepath, workspace=workspace, + bitbake_targets=[args.recipename]) return 0
PATCH 2/5 devtool: Check kernel configure result
In reply to Anusmita Dutta MazumderStop source modification when the kernel do_configure task fails instead of continuing with a potentially stale configuration.
--- scripts/lib/devtool/standard.py | 3 +++ 1 file changed, 3 insertions(+)
scripts/lib/devtool/standard.py +3 -0
@@ -847,6 +847,9 @@ def modify(args, config, basepath, workspace): task = 'do_configure' target = _get_multiconfig_targets(rd, pn) res = tinfoil.build_targets(target, task, handle_events=True) + if not res: + raise DevtoolError('Configuring source for %s failed' % + target) # Copy .config to workspace kconfpath = rd.getVar('B')
PATCH 3/5 devtool: Isolate multiconfig workspace entries
In reply to Anusmita Dutta MazumderStore workspace appends in multiconfig-specific directories and select them using BB_CURRENT_MC to prevent changes from affecting same-named recipes in other configurations.
Fixes [YOCTO #13933]
--- scripts/devtool | 24 ++++++++++++++++++++++-- scripts/lib/devtool/__init__.py | 8 ++++++-- scripts/lib/devtool/standard.py | 7 ++++--- 3 files changed, 32 insertions(+), 7 deletions(-)
scripts/devtool +22 -2
@@ -28,6 +28,11 @@ import scriptutils import argparse_oe logger = scriptutils.logger_create('devtool') +# Only expose workspace appends belonging to the current configuration. +workspace_append_pattern = ("${LAYERDIR}/appends/" + "${@'multiconfig/%s/' % d.getVar('BB_CURRENT_MC') " + "if d.getVar('BB_CURRENT_MC') else ''}*.bbappend") + class ConfigHandler: basepath = None
@@ -99,9 +104,21 @@ def read_workspace(basepath, context): if not context.fixed_setup: _enable_workspace_layer(context.config.workspace_path, context.config, basepath) + # Upgrade existing workspace layers as well as newly created ones. + layerconf = os.path.join(context.config.workspace_path, 'conf', 'layer.conf') + with open(layerconf) as f: + contents = f.read() + updated = contents.replace('${LAYERDIR}/appends/*.bbappend', workspace_append_pattern) + if updated != contents: + with open(layerconf, 'w') as f: + f.write(updated) + logger.debug('Reading workspace in %s' % context.config.workspace_path) externalsrc_re = re.compile(r'^EXTERNALSRC(:pn-([^ =]+))? *= *"([^"]*)"$') - for fn in glob.glob(os.path.join(context.config.workspace_path, 'appends', '*.bbappend')): + appenddir = os.path.join(context.config.workspace_path, 'appends') + appendfiles = glob.glob(os.path.join(appenddir, '*.bbappend')) + appendfiles += glob.glob(os.path.join(appenddir, 'multiconfig', '*', '*.bbappend')) + for fn in appendfiles: with open(fn, 'r') as f: pnvalues = {} pn = None
@@ -130,6 +147,9 @@ def read_workspace(basepath, context): if not pnvalues.get('srctreebase', None): pnvalues['srctreebase'] = pnvalues['srctree'] logger.debug('Found recipe %s' % pnvalues) + if pn in workspace: + raise DevtoolError("Multiple workspace appends found for recipe '%s'; " + "only one configuration can be modified at a time" % pn) workspace[pn] = pnvalues return workspace
@@ -166,7 +186,7 @@ def _create_workspace(workspacedir, basepath, layerseries=None): f.write('# ### workspace layer auto-generated by devtool ###\n') f.write('BBPATH =. "$' + '{LAYERDIR}:"\n') f.write('BBFILES += "$' + '{LAYERDIR}/recipes/*/*.bb \\\n') - f.write(' $' + '{LAYERDIR}/appends/*.bbappend"\n') + f.write(' %s"\n' % workspace_append_pattern) f.write('BBFILE_COLLECTIONS += "workspacelayer"\n') f.write('BBFILE_PATTERN_workspacelayer = "^$' + '{LAYERDIR}/"\n') f.write('BBFILE_PATTERN_IGNORE_EMPTY_workspacelayer = "1"\n')
scripts/lib/devtool/__init__.py +6 -2
@@ -133,13 +133,15 @@ def setup_tinfoil(config_only=False, basepath=None, tracking=False): def parse_recipe(config, tinfoil, pn, appends, filter_workspace=True): """Parse the specified recipe""" import bb.providers + import bb.runqueue try: recipefile = tinfoil.get_recipe_file(pn) except bb.providers.NoProvider as e: logger.error(str(e)) return None if appends: - append_files = tinfoil.get_file_appends(recipefile) + mc, _ = bb.runqueue.split_mc(pn) + append_files = tinfoil.get_file_appends(recipefile, mc) if filter_workspace: # Filter out appends from the workspace append_files = [path for path in append_files if
@@ -276,7 +278,7 @@ def setup_git_repo(repodir, version, devbranch, basetag='devtool-base', d=None): if os.path.exists(os.path.join(repodir, '.gitmodules')): bb.process.run('git submodule foreach --recursive "git tag -f --no-sign %s"' % basetag, cwd=repodir) -def recipe_to_append(recipefile, config, wildcard=False): +def recipe_to_append(recipefile, config, wildcard=False, mc=''): """ Convert a recipe file to a bbappend file path within the workspace. NOTE: if the bbappend already exists, you should be using
@@ -287,6 +289,8 @@ def recipe_to_append(recipefile, config, wildcard=False): if wildcard: appendname = re.sub(r'_.*', '_%', appendname) appendpath = os.path.join(config.workspace_path, 'appends') + if mc: + appendpath = os.path.join(appendpath, 'multiconfig', mc) appendfile = os.path.join(appendpath, appendname + '.bbappend') return appendfile
scripts/lib/devtool/standard.py +4 -3
@@ -540,7 +540,7 @@ def _extract_source(srctree, keep_temp, devbranch, sync, config, basepath, works initial_rev = None recipefile = d.getVar('FILE') - appendfile = recipe_to_append(recipefile, config) + appendfile = recipe_to_append(recipefile, config, mc=d.getVar('BB_CURRENT_MC')) is_kernel_yocto = bb.data.inherits_class('kernel-yocto', d) # We need to redirect WORKDIR, STAMPS_DIR etc. under a temporary
@@ -804,7 +804,8 @@ def modify(args, config, basepath, workspace): srctree) recipefile = rd.getVar('FILE') - appendfile = recipe_to_append(recipefile, config, args.wildcard) + appendfile = recipe_to_append(recipefile, config, args.wildcard, + mc=rd.getVar('BB_CURRENT_MC')) if os.path.exists(appendfile): raise DevtoolError("Another variant of recipe %s is already in your " "workspace (only one variant of a recipe can "
@@ -2076,7 +2077,7 @@ def _reset(recipes, no_clean, remove_work, config, basepath, workspace, # This should always be true if recipefile is set, but just in case preservedir(os.path.dirname(recipefile)) # We don't automatically create this dir next to appends, but the user can - preservedir(os.path.join(config.workspace_path, 'appends', pn)) + preservedir(os.path.join(os.path.dirname(appendfile), pn)) srctreebase = workspace[pn]['srctreebase'] if os.path.isdir(srctreebase):
PATCH 4/5 devtool: Record kernel baseline after revision validation
In reply to Anusmita Dutta MazumderDevtool creates its kernel branch before the source is reset to the commit specified by the recipe. This can leave the workspace on the wrong kernel version and fail the build.
Create the devtool branch after revision validation so it uses the commit specified by the recipe.
--- meta/classes/devtool-source.bbclass | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
meta/classes/devtool-source.bbclass +2 -1
@@ -52,7 +52,8 @@ python() { # Hook in pre/postfuncs is_kernel_yocto = bb.data.inherits_class('kernel-yocto', d) if is_kernel_yocto: - unpacktask = 'do_kernel_checkout' + # Record the devtool baseline after the kernel SRCREV is enforced. + unpacktask = 'do_validate_branches' d.appendVarFlag('do_configure', 'postfuncs', ' devtool_post_configure') else: unpacktask = 'do_unpack'
PATCH 5/5 oe-selftest: devtool: Add multiconfig workflow tests
In reply to Anusmita Dutta MazumderCover modify, build, reset, update-recipe and finish with a recipe available only in the selected multiconfig. Check workspace isolation between configurations.
[YOCTO #13933]
--- meta/lib/oeqa/selftest/cases/devtool.py | 133 ++++++++++++++++++++++++ 1 file changed, 133 insertions(+)
meta/lib/oeqa/selftest/cases/devtool.py +133 -0
@@ -1327,6 +1327,139 @@ class DevtoolModifyTests(DevtoolBase): self.assertExists(os.path.join(source_repo_gitsm_gitmodules, 'bitbake'), 'Submodule not found') self.assertExists(os.path.join(source_repo_gitsm_gitmodules, 'bitbake-gitsm-test1'), 'Submodule not found') +class DevtoolMulticonfigTests(DevtoolBase): + + def setUp(self): + super(DevtoolMulticonfigTests, self).setUp() + self.recipe = 'devtool-multiconfig-test' + self.target = 'mc:devtooltest:%s' % self.recipe + self.tempdir = tempfile.mkdtemp(prefix='devtoolqa') + self.track_for_cleanup(self.tempdir) + self.track_for_cleanup(self.workspacedir) + self.srctree = os.path.join(self.tempdir, 'source') + self.layerdir = os.path.join(self.tempdir, 'meta-devtool-multiconfig') + + self.append_config('BBMULTICONFIG = "devtooltest"\n') + # Only this configuration satisfies the test recipe's compatibility + # restriction, regardless of the machine used by the selftest runner. + self.write_config('MACHINEOVERRIDES .= ":devtool-mc"\n' + 'TMPDIR = "${TOPDIR}/tmp-devtool-mc"\n', 'devtooltest') + self.add_command_to_tearDown('bitbake -c clean %s' % self.target) + self.add_command_to_tearDown('bitbake-layers remove-layer */workspace') + self._create_temp_layer(self.layerdir, True, 'devtoolmulticonfig') + + self.recipedir = os.path.join(self.layerdir, 'recipes-test', self.recipe) + filesdir = os.path.join(self.recipedir, self.recipe) + os.makedirs(filesdir) + archive = 'devtool-test-subdir.tar.gz' + shutil.copyfile(os.path.join(get_test_layer(), 'recipes-test', 'devtool', + 'devtool-test-subdir', archive), + os.path.join(filesdir, archive)) + with open(os.path.join(self.recipedir, self.recipe + '_1.0.bb'), 'w') as f: + f.write('LICENSE = "MIT"\n' + 'LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302"\n' + 'inherit allarch\n' + 'INHIBIT_DEFAULT_DEPS = "1"\n' + 'COMPATIBLE_MACHINE = "devtool-mc"\n' + 'SRC_URI = "file://devtool-test-subdir.tar.gz"\n' + 'S = "${UNPACKDIR}/devtool-test-subdir"\n' + 'B = "${WORKDIR}/build"\n' + 'do_compile() {\n' + ' cp ${S}/testfile ${B}/testfile\n' + '}\n' + 'do_install() {\n' + ' install -Dm0644 ${B}/testfile ${D}${datadir}/${BPN}/testfile\n' + '}\n' + 'FILES:${PN} = "${datadir}/${BPN}"\n') + + def _modify(self, wildcard=False): + runCmd('devtool modify %s %s %s' % + ('--wildcard' if wildcard else '', self.target, self.srctree)) + self._check_src_repo(self.srctree) + append = os.path.join(self.workspacedir, 'appends', 'multiconfig', + 'devtooltest', self.recipe + ('_%.bbappend' if wildcard else '_1.0.bbappend')) + self.assertExists(append) + self.assertEqual(get_bb_var('EXTERNALSRC', self.target), self.srctree) + self.assertIn(self.recipe, runCmd('devtool status').output) + return append + + def test_devtool_multiconfig_modify_build_reset(self): + """Extract, build and clean a recipe unavailable in the default config.""" + result = bitbake('%s -n' % self.recipe, ignore_status=True) + self.assertNotEqual(result.status, 0) + self.assertIn('incompatible with machine', result.output) + + append = self._modify() + with open(os.path.join(self.srctree, 'testfile'), 'a') as f: + f.write('\nmulticonfig workspace change\n') + runCmd('devtool build %s' % self.target) + builddir = get_bb_var('B', self.target) + with open(os.path.join(builddir, 'testfile')) as f: + self.assertIn('multiconfig workspace change', f.read()) + + # Do not use --no-clean: cleaning must retain the multiconfig prefix. + runCmd('devtool reset %s' % self.target) + self.assertNotExists(append) + self.assertNotIn(self.recipe, runCmd('devtool status').output) + self.assertFalse(get_bb_var('EXTERNALSRC', self.target)) + + def test_devtool_multiconfig_update_finish(self): + """Update and finish using qualified targets and bare workspace keys.""" + append = self._modify() + for command, marker in [('update-recipe', 'update'), ('finish', 'finish')]: + with open(os.path.join(self.srctree, 'testfile'), 'a') as f: + f.write('\nmulticonfig %s change\n' % marker) + runCmd('git commit -am "Multiconfig %s change" ' + '-m "Upstream-Status: Inappropriate [selftest]"' % marker, + cwd=self.srctree) + destination = self.layerdir if command == 'finish' else '' + runCmd('devtool %s %s %s' % (command, self.target, destination)) + patches = glob.glob(os.path.join(self.recipedir, self.recipe, '*.patch')) + self.assertTrue(patches, 'No patches written by %s' % command) + contents = '' + for patch in patches: + with open(patch) as f: + contents += f.read() + self.assertIn('multiconfig %s change' % marker, contents) + if command == 'update-recipe': + self.assertExists(append) + self.assertIn(self.recipe, runCmd('devtool status').output) + + self.assertNotExists(append) + self.assertNotIn(self.recipe, runCmd('devtool status').output) + self.assertFalse(get_bb_var('EXTERNALSRC', self.target)) + bitbake('%s -c patch' % self.target) + with open(os.path.join(get_bb_var('S', self.target), 'testfile')) as f: + contents = f.read() + self.assertIn('multiconfig update change', contents) + self.assertIn('multiconfig finish change', contents) + + def test_devtool_multiconfig_workspace_isolation(self): + """A wildcard workspace append must affect only its configuration.""" + self.append_config('COMPATIBLE_MACHINE:pn-%s = ".*"\n' + 'BBMULTICONFIG += "devtoolother"\n' % self.recipe) + self.write_config('TMPDIR = "${TOPDIR}/tmp-devtool-other"\n', 'devtoolother') + other_target = 'mc:devtoolother:%s' % self.recipe + append = self._modify(wildcard=True) + self.assertFalse(get_bb_var('EXTERNALSRC', self.recipe)) + self.assertFalse(get_bb_var('EXTERNALSRC', other_target)) + + result = runCmd('devtool modify -n %s %s' % (self.recipe, self.srctree), + ignore_status=True) + self.assertNotEqual(result.status, 0) + self.assertIn('already in your workspace', result.output) + self.assertExists(append) + runCmd('devtool reset %s' % self.target) + self.assertNotExists(append) + + # The default configuration's append must likewise stay out of MCs. + runCmd('devtool modify -n %s %s' % (self.recipe, self.srctree)) + self.assertEqual(get_bb_var('EXTERNALSRC', self.recipe), self.srctree) + self.assertFalse(get_bb_var('EXTERNALSRC', self.target)) + self.assertFalse(get_bb_var('EXTERNALSRC', other_target)) + runCmd('devtool reset %s' % self.recipe) + + class DevtoolUpdateTests(DevtoolBase): def test_devtool_update_recipe(self):
Hi Anusmita,
Thanks for working on this, multiconfig support in devtool is long overdue (YOCTO #13933). Handling the mc: prefix when devtool calls bitbake is clearly needed. Before going further, I'd like to agree on the long term design of the workspace, as my series works on the same problem from another angle.
https://git.openembedded.org/openembedded-core-contrib/log/?h=adrianf/ide-sdk-improvements
In most cases there is one recipe and one source tree, and what differs is only what that source is built for: another multiconfig, -native, nativesdk or a multilib variant such as lib32. Build output is already separate for all of them: each multiconfig has its own TMPDIR, and the BBCLASSEXTEND variants have their own WORKDIR. So the question is mostly how the workspace records which variants are modified, and which source tree they build.
Your series puts the appends in a directory per multiconfig (appends/multiconfig/<mc>/) and selects them through BBFILES with BB_CURRENT_MC. This is small and makes it explicit which configuration is modified. The workspace is still keyed by the bare PN though, so a recipe can only be modified in one configuration at a time. This matches the current restriction to one BBCLASSEXTEND variant per recipe.
My series keeps one bbappend per recipe file and scopes all settings with pn-<pn>, so that e.g. foo, foo-native and lib32-foo can be in the workspace at the same time. By default they share one source tree. A variant gets its own tree only if its SRC_URI, SRCREV or S differ, or if a variant builds in the source tree (B = S). The oe-workdir and oe-logs symlinks in the shared tree are named per variant, e.g. oe-workdir-native or oe-workdir-mc1. This does not cover BBMULTICONFIG
yet: PN is the same in every configuration, so pn- cannot tell them apart, and something like a multiconfig specific override would be needed in the shared bbappend.
The two approaches overlap but don't fit together well. With both merged, devtool would have one mechanism for multiconfigs (separate append files) and another one for BBCLASSEXTEND variants (scoped lines in a shared append). read_workspace, reset, update-recipe/finish and ide-sdk would have to handle both. Modifying e.g. foo-native and mc:mc1:foo at the same time would combine both mechanisms.
So my question: what is the better long term design, also with -native, nativesdk and multilib in mind? Separate workspace appends per configuration or variant, or one append per recipe file with the modified variants scoped inside it and one shared source tree?
Regards,
Adrian
Hi Adrian,
Thank you for the review. I agree that a common workspace mechanism for multiconfig and BBCLASSEXTEND variants makes sense. The shared-bbappend approach looks sensible to me, with settings scoped by both recipe variant and multiconfig.
I’d be happy to work on the multiconfig extension, building on the relevant shared-workspace changes already in your series and keeping the changes separately reviewable.
Would that be a suitable way forward? If so, which patches or version of your series would you recommend using as the base?
Regards,
Anusmita
Hi all,
I’m following up on the workspace design discussion for this devtool multiconfig series. Adrian raised the overlap with his work on supporting multiple recipe variants, and I’ve proposed building the multiconfig support on the relevant shared-workspace changes from his series.
Could the maintainers share their thoughts on this direction and how best to coordinate the two series? That would help me prepare the next revision.
Thanks and regards,
Anusmita