devtool ide-sdk: support booting from NFS, VSCode clangd
cover devtool ide-sdk: support booting from NFS, VSCode clangd
From: Adrian Freihofer <[email protected]>
devtool ide-sdk: support booting the debug rootfs over NFS, plus VSCode clangd integration (IntelliSense, non-clang toolchain support, and .clang-format-aware formatting).
This series includes some commits already sent before, now reworked with independent ordering: first extract_sdk_rootfs() is implemented standalone, then runqemu-extract-sdk is rewritten in Python, and finally the new pseudo_rootfs_utils Python module is shared between runqemu-extract-sdk and devtool ide-sdk. This step-by-step split makes the Python rewrite optionally pickable on its own, independent of the NFS features.
Adrian Freihofer (15):
devtool: deploy: split ssh deployment into a separate function
devtool: deploy: allow deploying directly into a local rootfs
directory
oe-selftest: devtool deploy-target: test deploying into a local rootfs
path
devtool: ide-sdk: support NFS rootfs
oe-selftest: devtool ide-sdk: test NFS debug rootfs
runqemu-extract-sdk: refactor in Python
devtool: ide-sdk: reuse pseudo_rootfs_utils for SDK rootfs extraction
cpp-example: update formatting and code style
cpp-example: fix clang-tidy warnings
devtool: ide-sdk: add clangd support for VSCode IntelliSense
devtool: ide-sdk: support clangd for non-clang toolchain recipes
devtool: ide-sdk: format C/C++ with clangd when .clang-format is
present
devtool: ide-sdk: support LLDB ATTACH mode
devtool: ide-sdk: pin BBPATH in generated do_install script
buildstats: close proc/disk log files via weakref.finalize safety net
.../recipes-test/cpp/cpp-example.inc | 2 + .../recipes-test/cpp/files/.clang-format | 42 +++ .../recipes-test/cpp/files/.clang-tidy | 33 ++ .../cpp/files/cpp-example-lib.hpp | 3 +- .../recipes-test/cpp/files/cpp-example.cpp | 5 +- .../recipes-test/cpp/files/daemonize.cpp | 2 +- .../recipes-test/cpp/files/daemonize.hpp | 2 +- .../cpp/files/test-cpp-example.cpp | 13 +- meta/lib/oe/buildstats.py | 15 +- meta/lib/oeqa/selftest/cases/devtool.py | 320 +++++++++++++++++- .../qemu/nativesdk-qemu-helper_1.0.bb | 3 + scripts/lib/devtool/__init__.py | 22 +- scripts/lib/devtool/deploy.py | 220 ++++++++++-- scripts/lib/devtool/ide_plugins/__init__.py | 79 +++-- scripts/lib/devtool/ide_plugins/ide_code.py | 108 ++++-- scripts/lib/devtool/ide_plugins/ide_none.py | 17 +- scripts/lib/devtool/ide_sdk.py | 275 ++++++++++++++- scripts/lib/pseudo_rootfs_utils.py | 113 +++++++ scripts/runqemu-extract-sdk | 116 ++----- 19 files changed, 1156 insertions(+), 234 deletions(-) create mode 100644 meta-selftest/recipes-test/cpp/files/.clang-format create mode 100644 meta-selftest/recipes-test/cpp/files/.clang-tidy create mode 100644 scripts/lib/pseudo_rootfs_utils.py
Signature · 1 line
2.55.0
PATCH 1/15 devtool: deploy: split ssh deployment into a separate function
From: Adrian Freihofer <[email protected]>
This is a refactoring preparation for adding a new local deployment method that will be used for NFS rootfs deployments.
--- scripts/lib/devtool/deploy.py | 10 ++++++++++ 1 file changed, 10 insertions(+)
scripts/lib/devtool/deploy.py +10 -0
@@ -353,6 +353,12 @@ def deploy_no_d(srcdir, workdir, path, strip_cmd, libdir, base_libdir, max_proce print(' %s' % item) return 0 + return _deploy_ssh(args, destdir, filelist, ftotalsize, tar_relpaths, allowed_files, + fakerootcmd, fakerootenv, path, recipe_outdir) + +def _deploy_ssh(args, destdir, filelist, ftotalsize, tar_relpaths, allowed_files, + fakerootcmd, fakerootenv, path, recipe_outdir): + """Copy files to target_dir over ssh/scp (user@hostname[:destdir]).""" extraoptions = '' if args.no_host_check: extraoptions += '-o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no'
@@ -440,6 +446,10 @@ def undeploy(args, config, basepath, workspace): elif not args.recipename and not args.all: raise argparse_oe.ArgumentUsageError('If you don\'t specify a recipe, you must specify -a/--all', 'undeploy-target') + return _undeploy_ssh(args) + +def _undeploy_ssh(args): + """Run the undeploy script on the target over ssh/scp (user@hostname[:destdir]).""" extraoptions = '' if args.no_host_check: extraoptions += '-o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no'
Signature · 1 line
2.55.0
PATCH 2/15 devtool: deploy: allow deploying directly into a local rootfs directory
In reply to AdrianFFrom: Adrian Freihofer <[email protected]>
Add support for deploy-target/undeploy-target to write into a local pseudo-managed rootfs directory (e.g. one extracted by runqemu-extract-sdk for NFS booting) instead of requiring an ssh connection to a live target.
If the 'target' argument is an absolute path, _deploy_local and _undeploy_local run the same tar/manifest based copy/removal as the ssh path, but locally: the file transfer is piped between two pseudo instances (one scoped to the recipe's own PSEUDO_INCLUDE_PATHS, one scoped to the target rootfs's own PSEUDO_LOCALSTATEDIR/PSEUDO_INCLUDE_PATHS) instead of over ssh/scp.
exec_fakeroot_no_d()'s environment-building code is factored out into build_fakeroot_env_no_d() so _deploy_local can assemble its own pseudo environment without exec_fakeroot_no_d's implicit single-command prefixing, which doesn't compose with the dual-pseudo pipeline.
This is a prerequisite for adding NFS export support to devtool ide-sdk.
--- scripts/lib/devtool/__init__.py | 22 ++-- scripts/lib/devtool/deploy.py | 210 ++++++++++++++++++++++++++++---- 2 files changed, 201 insertions(+), 31 deletions(-)
scripts/lib/devtool/__init__.py +13 -9
@@ -77,6 +77,18 @@ def exec_watch(cmd, **options): return buf, None +def build_fakeroot_env_no_d(fakerootenv, path, env_overrides=None): + """Build the environment for running a command under pseudo (see exec_fakeroot_no_d).""" + newenv = dict(os.environ) + newenv['PATH'] = path + for varvalue in fakerootenv.split(): + if '=' in varvalue: + splitval = varvalue.split('=', 1) + newenv[splitval[0]] = splitval[1] + if env_overrides: + newenv.update(env_overrides) + return newenv + def exec_fakeroot_no_d(fakerootcmd, fakerootenv, path, cmd, env_overrides=None, **kwargs): """Run cmd under pseudo using a recipe's own FAKEROOTCMD/FAKEROOTENV.
@@ -88,15 +100,7 @@ def exec_fakeroot_no_d(fakerootcmd, fakerootenv, path, cmd, env_overrides=None, if not os.path.exists(fakerootcmd): logger.error('pseudo executable %s could not be found - have you run a build yet? pseudo-native should install this and if you have run any build then that should have been built') return 2 - # Set up the appropriate environment - newenv = dict(os.environ) - newenv['PATH'] = path - for varvalue in fakerootenv.split(): - if '=' in varvalue: - splitval = varvalue.split('=', 1) - newenv[splitval[0]] = splitval[1] - if env_overrides: - newenv.update(env_overrides) + newenv = build_fakeroot_env_no_d(fakerootenv, path, env_overrides) return subprocess.call("%s %s" % (fakerootcmd, cmd), env=newenv, **kwargs) def setup_tinfoil(config_only=False, basepath=None, tracking=False):
scripts/lib/devtool/deploy.py +188 -22
270 lines not on this page · raw message
Signature · 1 line
2.55.0
PATCH 3/15 oe-selftest: devtool deploy-target: test deploying into a local rootfs path
In reply to AdrianFFrom: Adrian Freihofer <[email protected]>
Extract an image's rootfs tarball with runqemu-extract-sdk, boot it via NFS with runqemu, then deploy a devtool-modified recipe directly into that directory with plain 'devtool deploy-target <recipe> <path>' (no ssh) while the target has it mounted live, and confirm the target immediately sees the change. Deploying a second time covers replacing an existing deployment, a --file-glob deploy covers the filtered variant which packs an explicit file list, and 'devtool undeploy-target <recipe> <path>' covers removing the files again, all while the rootfs stays mounted.
Also deploy/undeploy the same recipe over ssh into the very same live directory and compare the resulting file ownership/permissions (captured via the same pseudo find/ls trick test_devtool_deploy_target uses) against the local-path deploy's result, to confirm the two independent code paths (_deploy_local's pseudo pipe vs _deploy_ssh's tar-over-ssh) produce identical output.
--- meta/lib/oeqa/selftest/cases/devtool.py | 161 ++++++++++++++++++++++++ 1 file changed, 161 insertions(+)
meta/lib/oeqa/selftest/cases/devtool.py +161 -0
@@ -2171,6 +2171,167 @@ class DevtoolDeployTargetTests(DevtoolBase): extra_args = ' '.join(a for a in (strip_opt, filter_args) if a) _deploy_and_check(extra_args, check_full_filelist=not filter_args, expected_files=expected_files) + @OETestTag("runqemu") + def test_devtool_deploy_target_path(self): + """Verify 'devtool deploy-target/undeploy-target <recipe> <path>' deploy/remove straight into + a local pseudo-managed rootfs directory (no ssh), and that a target + booting that same directory via NFS immediately sees the change. + """ + self._check_runqemu_prerequisites() + self.assertTrue(not os.path.exists(self.workspacedir), 'This test cannot be run with a workspace directory under the build directory') + testrecipe = 'mdadm' + testfile = '/sbin/mdadm' + # mdmon is installed by the same do_install, used to check --file-glob filtering + otherfile = '/sbin/mdmon' + if "usrmerge" in get_bb_var('DISTRO_FEATURES'): + testfile = '/usr/sbin/mdadm' + otherfile = '/usr/sbin/mdmon' + testcommand = '/sbin/mdadm --help' + testimage = 'oe-selftest-image' + # Use the mdadm-doc package to check --package filtering excludes it + mandir = get_bb_var('mandir', testrecipe) + docfile = os.path.join(mandir, 'man8', '%s.8' % testrecipe) + + # A tar rootfs is needed both to extract a local copy of it (below) + # and for runqemu to NFS-boot straight from that extracted directory. + self.append_config('IMAGE_FSTYPES:append = " tar"\n') + bitbake("%s qemu-native qemu-helper-native" % testimage) + bb_vars = get_bb_vars(['DEPLOY_DIR_IMAGE', 'IMAGE_LINK_NAME'], testimage) + deploy_dir_image = bb_vars['DEPLOY_DIR_IMAGE'] + image_link_name = bb_vars['IMAGE_LINK_NAME'] + self.add_command_to_tearDown('bitbake -c clean %s' % testimage) + self.add_command_to_tearDown('rm -f %s/%s*' % (deploy_dir_image, testimage)) + + tempdir = tempfile.mkdtemp(prefix='devtoolqa') + self.track_for_cleanup(tempdir) + self.track_for_cleanup(self.workspacedir) + self.add_command_to_tearDown('bitbake -c clean %s' % testrecipe) + self.add_command_to_tearDown('bitbake-layers remove-layer */workspace') + runCmd('devtool modify %s -x %s' % (testrecipe, tempdir)) + runCmd('devtool build %s' % testrecipe) + + # Extract a local pseudo-managed rootfs the same way + rootfs_tarball = os.path.join(deploy_dir_image, image_link_name + '.tar') + self.assertExists(rootfs_tarball) + extractdir = tempfile.mkdtemp(prefix='devtoolqa') + self.track_for_cleanup(extractdir) + nfs_rootfs = os.path.join(extractdir, 'rootfs') + runCmd('runqemu-extract-sdk %s %s' % (rootfs_tarball, nfs_rootfs)) + self.assertExists(nfs_rootfs) + self.assertExists(nfs_rootfs + '.pseudo_state') + + # oe-selftest-image does not install mdadm by default, so the target must not see it yet. + self.assertNotExists(os.path.join(nfs_rootfs, testfile.lstrip('/'))) + + qemuboot = os.path.join(deploy_dir_image, image_link_name + '.qemuboot.conf') + self.assertExists(qemuboot) + launch_cmd = 'runqemu %s %s nographic' % (shlex.quote(qemuboot), shlex.quote(nfs_rootfs)) + with runqemu(testimage, launch_cmd=launch_cmd) as qemu: + status, output = qemu.run("awk '$2 == \"/\" {print $3}' /proc/mounts") + self.assertEqual(status, 0) + self.assertEqual(output.strip(), 'nfs') + + status, _ = qemu.run(testcommand) + self.assertNotEqual(status, 0, '%s should not be deployed yet' % testfile) + + # Deploy directly into the local rootfs path (no ssh) while the target has it NFS-mounted live + deploy_cmd = 'devtool deploy-target %s %s' % (testrecipe, nfs_rootfs) + if self.logger.isEnabledFor(logging.DEBUG): + deploy_cmd += ' -s' + result = runCmd(deploy_cmd) + self.assertEqual(result.status, 0) + self.assertExists(os.path.join(nfs_rootfs, testfile.lstrip('/'))) + self.assertExists(os.path.join(nfs_rootfs, otherfile.lstrip('/'))) + + status, _ = qemu.run(testcommand) + self.assertEqual(status, 0, '%s was not deployed' % testfile) + + # Deploying again while the target still has this directory NFS-mounted live must still succeed. + result = runCmd(deploy_cmd) + self.assertEqual(result.status, 0) + + # Undeploy directly from the local rootfs path (no ssh) while the target still has it NFS-mounted live + undeploy_cmd = 'devtool undeploy-target %s %s' % (testrecipe, nfs_rootfs) + if self.logger.isEnabledFor(logging.DEBUG): + undeploy_cmd += ' -s' + result = runCmd(undeploy_cmd) + self.assertEqual(result.status, 0) + self.assertNotExists(os.path.join(nfs_rootfs, testfile.lstrip('/'))) + + status, _ = qemu.run(testcommand) + self.assertNotEqual(status, 0, 'undeploy-target did not remove %s as it should have' % testfile) + + # Confirm the local-path deploy (dual-pseudo pipe) and the ssh deploy + # (tar over ssh) land identical files with identical ownership/perms + # into the very same live NFS-exported directory, for both an + # unfiltered deploy and a --package filtered one. + bb_vars = get_bb_vars(['FAKEROOTENV', 'FAKEROOTCMD', 'PATH'], testrecipe) + fakerootenv = bb_vars['FAKEROOTENV'] + fakerootcmd = bb_vars['FAKEROOTCMD'] + path = bb_vars['PATH'] + state_dir = nfs_rootfs + '.pseudo_state' + + def _get_rootfs_filelist(): + # nfs_rootfs has its own pseudo database, distinct from the one + # FAKEROOTENV points at, so override it for this inspection. + cmd = 'PATH="%s" %s PSEUDO_LOCALSTATEDIR=%s PSEUDO_INCLUDE_PATHS=%s %s find . -type f -exec ls -l {} \\;' % ( + path, fakerootenv, shlex.quote(state_dir), shlex.quote(nfs_rootfs), fakerootcmd) + result = runCmd(cmd, cwd=nfs_rootfs) + filelist = self._process_ls_output(result.output) + filelist.sort(key=lambda item: item.split()[-1]) + return filelist + + def _compare_local_and_ssh_deploy(extra_args, expected_files): + """Deploy extra_args once locally and once over ssh into the same + live directory, and assert both leave identical files with + identical ownership/permissions (expected_files checks each path + was/wasn't deployed, on both sides).""" + local_cmd = ('%s %s' % (deploy_cmd, extra_args)).strip() + result = runCmd(local_cmd) + self.assertEqual(result.status, 0) + for filepath, expected in expected_files.items(): + exists = os.path.exists(os.path.join(nfs_rootfs, filepath.lstrip('/'))) + self.assertEqual(exists, expected, '%s: %s exists=%s after local deploy' % (extra_args, filepath, exists)) + local_filelist = _get_rootfs_filelist() + + result = runCmd(undeploy_cmd) + self.assertEqual(result.status, 0) + self.assertNotExists(os.path.join(nfs_rootfs, testfile.lstrip('/'))) + + ssh_cmd = ('devtool deploy-target -c %s root@%s %s' % (testrecipe, qemu.ip, extra_args)).strip() + result = runCmd(ssh_cmd) + self.assertEqual(result.status, 0) + for filepath, expected in expected_files.items(): + exists = os.path.exists(os.path.join(nfs_rootfs, filepath.lstrip('/'))) + self.assertEqual(exists, expected, '%s: %s exists=%s after ssh deploy' % (extra_args, filepath, exists)) + ssh_filelist = _get_rootfs_filelist() + + self.assertEqual(local_filelist, ssh_filelist, + '%s: local-path deploy and ssh deploy produced different file ownership/permissions' % extra_args) + + result = runCmd('devtool undeploy-target -c %s root@%s' % (testrecipe, qemu.ip)) + self.assertEqual(result.status, 0) + self.assertNotExists(os.path.join(nfs_rootfs, testfile.lstrip('/'))) + + _compare_local_and_ssh_deploy('', {testfile: True, otherfile: True, docfile: True}) + _compare_local_and_ssh_deploy('--package %s' % testrecipe, {testfile: True, otherfile: True, docfile: False}) + + # A --package/--file-glob filtered deploy hands tar an explicit file + # list instead of packing the whole tree, so cover that path as well. + filtered_deploy_cmd = deploy_cmd + ' --file-glob %s' % testfile + result = runCmd(filtered_deploy_cmd) + self.assertEqual(result.status, 0) + self.assertExists(os.path.join(nfs_rootfs, testfile.lstrip('/'))) + self.assertNotExists(os.path.join(nfs_rootfs, otherfile.lstrip('/'))) + + status, _ = qemu.run(testcommand) + self.assertEqual(status, 0, '%s was not deployed' % testfile) + + result = runCmd(undeploy_cmd) + self.assertEqual(result.status, 0) + self.assertNotExists(os.path.join(nfs_rootfs, testfile.lstrip('/'))) + + class DevtoolBuildImageTests(DevtoolBase): def test_devtool_build_image(self):
Signature · 1 line
2.55.0
PATCH 4/15 devtool: ide-sdk: support NFS rootfs
In reply to AdrianFFrom: Adrian Freihofer <[email protected]>
Add a new command line option --nfs to devtool ide-sdk. This allows running the full ide-sdk workflow against a target device booted with an NFS rootfs, instead of deploying over ssh.
devtool ide-sdk extracts the image's tar archive, a bootable rootfs, below <workspace>/nfs-exports/<image-PN>/ (optionally relocatable via --nfs-extract-dir), and generates a small helper script next to it that starts QEMU booting from that rootfs, which in turn serves it via NFS using runqemu-export-rootfs.
Deploying the modified recipe writes directly into this directory instead of ssh, using devtool deploy-target's local rootfs support.
--nfs=rootfs boots the plain rootfs, same as a non-NFS ssh setup, so debugging still relies on the locally built rootfs-dbg/D debug symbols. --nfs=rootfs-dbg instead extracts and boots the combined debug rootfs. The generated debugging configurations still read those debug symbols on the host side, but from the extracted directory, not as usual from the image's WORKDIR. Since the target also boots that same directory over NFS, the symbols are available target-side too, e.g. for perf or gcov.
--- scripts/lib/devtool/ide_plugins/__init__.py | 19 +- scripts/lib/devtool/ide_plugins/ide_code.py | 29 +-- scripts/lib/devtool/ide_plugins/ide_none.py | 2 +- scripts/lib/devtool/ide_sdk.py | 217 +++++++++++++++++++- 4 files changed, 240 insertions(+), 27 deletions(-)
scripts/lib/devtool/ide_plugins/__init__.py +15 -4
@@ -411,11 +411,22 @@ class IdeBase: raise err -def get_devtool_deploy_opts(args): +def resolve_deploy_target(args, target_override=None): + """Resolve the effective devtool deploy-target destination. + + target_override, e.g. the local NFS rootfs directory when --nfs was used + (see devtool.deploy for how a directory target is handled without ssh), + always takes precedence over the ssh target from -t/--target. + """ + target = target_override or args.target + if not target: + raise DevtoolError('No deploy target, pass -t/--target or --nfs') + return target + + +def get_devtool_deploy_opts(args, target_override=None): """Filter args for devtool deploy-target args""" - if not args.target: - return None - devtool_deploy_opts = [args.target] + devtool_deploy_opts = [resolve_deploy_target(args, target_override)] if args.no_host_check: devtool_deploy_opts += ["-c"] if args.show_status:
scripts/lib/devtool/ide_plugins/ide_code.py +15 -14
@@ -9,7 +9,7 @@ import json import logging import os import shutil -from devtool.ide_plugins import BuildTool, IdeBase, GdbCrossConfig, DebuggerServerModes, LldbServerConfig, get_devtool_deploy_opts +from devtool.ide_plugins import BuildTool, IdeBase, GdbCrossConfig, DebuggerServerModes, LldbServerConfig, get_devtool_deploy_opts, resolve_deploy_target logger = logging.getLogger('devtool')
@@ -594,10 +594,10 @@ class IdeVSCode(IdeBase): IdeBase.update_json_file( self.dot_code_dir(modified_recipe), launch_file, launch_dict) - def vscode_tasks_cpp(self, args, modified_recipe): - run_install_deploy = modified_recipe.gen_install_deploy_script(args) + def vscode_tasks_cpp(self, args, image_recipe, modified_recipe): + run_install_deploy = modified_recipe.gen_install_deploy_script(args, image_recipe.nfs_deploy_dir) install_task_name = "install && deploy-target %s" % modified_recipe.recipe_id_pretty - deploy_args = ["--target", args.target] + deploy_args = ["--target", resolve_deploy_target(args, image_recipe.nfs_deploy_dir)] if args.port: deploy_args += ["--port", args.port] for package in args.package or []:
@@ -694,7 +694,7 @@ class IdeVSCode(IdeBase): args += [target_device.target, remote_cmd] return args - def vscode_tasks_kernel_module(self, args, modified_recipe): + def vscode_tasks_kernel_module(self, args, image_recipe, modified_recipe): """Generate tasks.json for kernel module recipes. Three tasks are generated and chained in sequence:
@@ -717,8 +717,9 @@ class IdeVSCode(IdeBase): install_task_name = "install && deploy-target %s" % modified_recipe.recipe_id_pretty reload_task_name = "reload module %s" % modified_recipe.recipe_id_pretty verify_task_name = "verify module %s" % modified_recipe.recipe_id_pretty - run_install_deploy = modified_recipe.gen_install_deploy_script(args) - deploy_args = ["--target", args.target] + run_install_deploy = modified_recipe.gen_install_deploy_script(args, image_recipe.nfs_deploy_dir) + # Redundant with the default baked into the script, but keeps the task self-contained. + deploy_args = ["--target", resolve_deploy_target(args, image_recipe.nfs_deploy_dir)] if args.port: deploy_args += ["--port", args.port] for package in args.package or []:
@@ -760,7 +761,7 @@ class IdeVSCode(IdeBase): IdeBase.update_json_file( self.dot_code_dir(modified_recipe), tasks_file, tasks_dict) - def vscode_tasks_fallback(self, args, modified_recipe): + def vscode_tasks_fallback(self, args, image_recipe, modified_recipe): oe_init_dir = modified_recipe.oe_init_dir oe_init = ". %s %s > /dev/null && " % (modified_recipe.oe_init_build_env, modified_recipe.topdir) dt_build = "devtool build "
@@ -773,7 +774,7 @@ class IdeVSCode(IdeBase): dt_deploy_label = dt_deploy + modified_recipe.recipe_id_pretty dt_deploy_cmd = dt_deploy + modified_recipe.bpn dt_build_deploy_label = "devtool build & deploy-target %s" % modified_recipe.recipe_id_pretty - deploy_opts = ' '.join(get_devtool_deploy_opts(args)) + deploy_opts = ' '.join(get_devtool_deploy_opts(args, image_recipe.nfs_deploy_dir)) tasks_dict = { "version": "2.0.0", "tasks": [
@@ -893,13 +894,13 @@ class IdeVSCode(IdeBase): IdeBase.update_json_file( self.dot_code_dir(modified_recipe), tasks_file, tasks_dict) - def vscode_tasks(self, args, modified_recipe): + def vscode_tasks(self, args, image_recipe, modified_recipe): if modified_recipe.build_tool.is_c_cpp: - self.vscode_tasks_cpp(args, modified_recipe) + self.vscode_tasks_cpp(args, image_recipe, modified_recipe) elif modified_recipe.build_tool == BuildTool.KERNEL_MODULE: - self.vscode_tasks_kernel_module(args, modified_recipe) + self.vscode_tasks_kernel_module(args, image_recipe, modified_recipe) else: - self.vscode_tasks_fallback(args, modified_recipe) + self.vscode_tasks_fallback(args, image_recipe, modified_recipe) def setup_modified_recipe(self, args, image_recipe, modified_recipe): self.vscode_settings(modified_recipe, image_recipe)
@@ -913,7 +914,7 @@ class IdeVSCode(IdeBase): self.initialize_cross_debug_configs( image_recipe, modified_recipe, GdbCrossConfigVSCode) self.vscode_launch(args, modified_recipe) - self.vscode_tasks(args, modified_recipe) + self.vscode_tasks(args, image_recipe, modified_recipe) def register_ide_plugin(ide_plugins):
scripts/lib/devtool/ide_plugins/ide_none.py +1 -1
@@ -312,7 +312,7 @@ class IdeNone(IdeBase): - Generate a gdbinit file per executable - Generate the oe-scripts sym-link """ - script_path = modified_recipe.gen_install_deploy_script(args) + script_path = modified_recipe.gen_install_deploy_script(args, image_recipe.nfs_deploy_dir) logger.info("Created: %s" % script_path) if modified_recipe.toolchain == 'clang':
scripts/lib/devtool/ide_sdk.py +209 -8
339 lines not on this page · raw message
Signature · 1 line
2.55.0
PATCH 5/15 oe-selftest: devtool ide-sdk: test NFS debug rootfs
In reply to AdrianFFrom: Adrian Freihofer <[email protected]>
Exercise --nfs=rootfs-dbg with a real image build and verify the extracted rootfs, pseudo state, runqemu helper, image bbappend and the generated VS Code debug paths. Two further tests boot that rootfs over NFS with runqemu, once with tap and once with slirp networking, re-run ide-sdk against the actual target address and drive a full remote GDB debug session.
Booting from NFS also exposed that the gdbserver pid file written by the start script can become readable over the next SSH connection only a moment later, so poll for it instead of reading it once.
--- meta/lib/oeqa/selftest/cases/devtool.py | 120 ++++++++++++++++++++++++ 1 file changed, 120 insertions(+)
meta/lib/oeqa/selftest/cases/devtool.py +120 -0
@@ -3627,8 +3627,17 @@ class DevtoolIdeSdkGccTests(DevtoolIdeSdkTests): r = runCmd(gdbserver_script, output_log=self._cmd_logger) self.assertEqual(r.status, 0) + # The start script waits for gdbserver's port before returning, but on + # an NFS-root target (--nfs=rootfs-dbg) the pid file it wrote has been + # observed to become readable over the next SSH connection only a + # moment later, so poll for it. pid_file = '/tmp/gdbserver_1234_usr-bin-%s_multi/gdbserver.pid' % example_exe status, output = qemu.run('cat %s' % pid_file) + for _ in range(10): + if status == 0: + break + time.sleep(1) + status, output = qemu.run('cat %s' % pid_file) self.assertEqual(status, 0) gdbserver_pid = output.strip() self.assertRegex(gdbserver_pid, r'^\d+$')
@@ -4068,6 +4077,117 @@ class DevtoolIdeSdkGccTests(DevtoolIdeSdkTests): self.assertEqual(r.status, 0) debug_check_func(r.output, DevtoolIdeSdkTests.MAGIC_STRING_ORIG) + def test_devtool_ide_sdk_code_nfs_debug_rootfs(self): + """Verify ide-sdk extracts an NFS debug rootfs for VS Code debugging.""" + recipe_name = "cmake-example" + build_file = "CMakeLists.txt" + testimage = "oe-selftest-image" + + self._check_workspace() + self._write_bb_config() + tempdir = self._devtool_ide_sdk_recipe( + recipe_name, build_file, testimage) + runCmd('devtool ide-sdk %s %s -c --ide=code --nfs=rootfs-dbg' % + (recipe_name, testimage), output_log=self._cmd_logger) + + nfs_rootfs = os.path.join( + self.workspacedir, 'nfs-exports', testimage, 'rootfs-dbg') + self.assertExists(nfs_rootfs) + self.assertExists(nfs_rootfs + '.pseudo_state') + self.assertExists(os.path.join(nfs_rootfs, 'usr', 'bin', recipe_name)) + runqemu_helper = os.path.join( + self.workspacedir, 'nfs-exports', testimage, + 'runqemu-rootfs-dbg') + self.assertExists(runqemu_helper) + self.assertTrue(os.access(runqemu_helper, os.X_OK)) + with open(runqemu_helper) as helper_file: + helper = helper_file.read() + self.assertIn('exec runqemu ', helper) + self.assertIn(nfs_rootfs, helper) + + bbappend = os.path.join( + self.workspacedir, 'appends', testimage + '.bbappend') + with open(bbappend) as append_file: + self.assertRegex( + append_file.read(), + r'IMAGE_FSTYPES_DEBUGFS(?::append)? = " ?tar"') + + with open(os.path.join(tempdir, '.vscode', 'launch.json')) as launch_file: + launch_configurations = json.load(launch_file)['configurations'] + for configuration in launch_configurations: + self.assertIn( + nfs_rootfs, configuration['additionalSOLibSearchPath']) + self.assertEqual( + os.path.join(nfs_rootfs, 'usr', 'src', 'debug'), + configuration['sourceFileMap']['/usr/src/debug']) + + def _test_devtool_ide_sdk_nfs_debug_rootfs_qemu(self, slirp=False): + """Boot the extracted NFS debug rootfs and exercise remote GDB.""" + recipe_name = "cmake-example" + build_file = "CMakeLists.txt" + testimage = "oe-selftest-image" + + self._check_workspace() + self._write_bb_config() + if not slirp: + self._check_runqemu_prerequisites() + + tempdir = self._devtool_ide_sdk_recipe( + recipe_name, build_file, testimage) + runCmd('devtool ide-sdk %s %s -c --ide=none --nfs=rootfs-dbg' % + (recipe_name, testimage), + output_log=self._cmd_logger) + + nfs_rootfs = os.path.join( + self.workspacedir, 'nfs-exports', testimage, 'rootfs-dbg') + runqemu_helper = os.path.join( + self.workspacedir, 'nfs-exports', testimage, + 'runqemu-rootfs-dbg') + self.assertExists(nfs_rootfs) + self.assertExists(runqemu_helper) + + launch_cmd = '%s nographic' % shlex.quote(runqemu_helper) + runqemuparams = '' + if slirp: + launch_cmd += ' slirp' + # QemuTarget uses this only to select its localhost SSH endpoint; + # launch_cmd supplies the actual runqemu option. + runqemuparams = 'slirp' + + with runqemu(testimage, runqemuparams=runqemuparams, + launch_cmd=launch_cmd) as qemu: + status, output = qemu.run( + "awk '$2 == \"/\" {print $3}' /proc/mounts") + self.assertEqual(status, 0) + self.assertEqual(output.strip(), 'nfs') + + # Re-run with the real QEMU target address, instead of 192.168.7.2 IP. + # --skip-bitbake also skips the rootfs extraction, which would + # otherwise wipe the directory the target has mounted right now. + if slirp: + self.assertIsNotNone(qemu.port, 'No SSH port for the slirp target') + target_options = '-t root@%s -P %s ' % (qemu.ip, qemu.port) + else: + target_options = '-t root@%s -c ' % qemu.ip + runCmd('devtool ide-sdk %s %s %s--skip-bitbake --ide=none --nfs=rootfs-dbg' % + (recipe_name, testimage, target_options), + output_log=self._cmd_logger) + + self._gdb_cross() + compile_cmd = self._verify_cmake_preset(tempdir) + self._devtool_ide_sdk_qemu( + tempdir, qemu, recipe_name, recipe_name, compile_cmd) + + @OETestTag("runqemu") + def test_devtool_ide_sdk_none_nfs_qemu(self): + """Verify remote GDB debugging through an NFS-root QEMU target.""" + self._test_devtool_ide_sdk_nfs_debug_rootfs_qemu() + + @OETestTag("runqemu") + def test_devtool_ide_sdk_none_nfs_qemu_slirp(self): + """Verify remote GDB debugging through an NFS-root slirp target.""" + self._test_devtool_ide_sdk_nfs_debug_rootfs_qemu(slirp=True) + @OETestTag("runqemu") def test_devtool_ide_sdk_code_cmake(self): """Verify a cmake recipe works with ide=code mode"""
Signature · 1 line
2.55.0
PATCH 6/15 runqemu-extract-sdk: refactor in Python
In reply to AdrianFFrom: Adrian Freihofer <[email protected]>
Move the pseudo-backed rootfs extraction logic into a reusable Python module while preserving the existing command interface. Install the module with qemu-helper so the wrapper also works from an SDK.
--- .../qemu/nativesdk-qemu-helper_1.0.bb | 3 + scripts/lib/pseudo_rootfs_utils.py | 113 +++++++++++++++++ scripts/runqemu-extract-sdk | 116 ++++-------------- 3 files changed, 137 insertions(+), 95 deletions(-) create mode 100644 scripts/lib/pseudo_rootfs_utils.py
meta/recipes-devtools/qemu/nativesdk-qemu-helper_1.0.bb +3 -0
@@ -9,6 +9,7 @@ RDEPENDS:${PN} = "nativesdk-qemu nativesdk-unfs3 nativesdk-pseudo \ LIC_FILES_CHKSUM = "file://${COREBASE}/scripts/runqemu;beginline=5;endline=10;md5=ac2b489a58739c7628a2604698db5e7f" SRC_URI = "file://${COREBASE}/scripts/runqemu \ + file://${COREBASE}/scripts/lib/pseudo_rootfs_utils.py \ file://${COREBASE}/scripts/runqemu-addptable2image \ file://${COREBASE}/scripts/runqemu-gen-tapdevs \ file://${COREBASE}/scripts/runqemu-ifup \
@@ -30,4 +31,6 @@ do_install() { install -d ${D}${bindir} install -m 0755 ${S}${COREBASE}/scripts/oe-* ${D}${bindir}/ install -m 0755 ${S}${COREBASE}/scripts/runqemu* ${D}${bindir}/ + # The runqemu-* wrappers import this module from their own directory. + install -m 0644 ${S}${COREBASE}/scripts/lib/pseudo_rootfs_utils.py ${D}${bindir}/ }
scripts/lib/pseudo_rootfs_utils.py new file +113 -0
113 lines not on this page · raw message
scripts/runqemu-extract-sdk +21 -95
125 lines not on this page · raw message
Signature · 1 line
2.55.0
Re: [OE-core] [PATCH 06/15] runqemu-extract-sdk: refactor in Python
Quoted 87 lines · Adrian Freihofer via lists.openembedded.org
From: Adrian Freihofer <[email protected]>
Move the pseudo-backed rootfs extraction logic into a reusable Python module while preserving the existing command interface. Install the module with qemu-helper so the wrapper also works from an SDK.
Signed-off-by Adrian Freihofer <[email protected]>---
.../qemu/nativesdk-qemu-helper_1.0.bb | 3 + scripts/lib/pseudo_rootfs_utils.py | 113 +++++++++++++++++ scripts/runqemu-extract-sdk | 116 ++++--------------3 files changed, 137 insertions(+), 95 deletions(-)create mode 100644
scripts/lib/pseudo_rootfs_utils.pymeta/recipes-devtools/qemu/nativesdk-qemu-helper_1.0.bb +0
-0@@ -9,6 +9,7 @@ RDEPENDS:${PN} = "nativesdk-qemu nativesdk-unfs3 nativesdk-pseudo \LIC_FILES_CHKSUM = "file://${COREBASE}/scripts/runqemu;beginline=5;endline=10;md5=ac2b489a58739c7628a2604698db5e7f"
SRC_URI = "file://${COREBASE}/scripts/runqemu \
+ file://${COREBASE}/scripts/lib/pseudo_rootfs_utils.py \
file://${COREBASE}/scripts/runqemu-addptable2image \
file://${COREBASE}/scripts/runqemu-gen-tapdevs \
file://${COREBASE}/scripts/runqemu-ifup \meta/recipes-devtools/qemu/nativesdk-qemu-helper_1.0.bb +0
-0@@ -30,4 +31,6 @@ do_install() { install -d ${D}${bindir}install -m 0755 ${S}${COREBASE}/scripts/oe-* ${D}${bindir}/ install -m 0755 ${S}${COREBASE}/scripts/runqemu* ${D}${bindir}/
meta/recipes-devtools/qemu/nativesdk-qemu-helper_1.0.bb +2
-0+ # The runqemu-* wrappers import this module from their own directory. + install -m 0644 ${S}${COREBASE}/scripts/lib/pseudo_rootfs_utils.py ${D}${bindir}/}
scripts/lib/pseudo_rootfs_utils.py new file +48
-0@@ -0,0 +1,113 @@ +#!/usr/bin/env python3 +# +# Helpers for preparing pseudo-managed rootfs trees (NFS booting, SDK/rootfs +# extraction, target deploy). +# +# SPDX-License-Identifier: GPL-2.0-only + +"""Extract rootfs tarballs and locate their pseudo state for NFS booting.""" + +import os +import subprocess +from pathlib import Path + + +class PseudoRootfsError(Exception): + """Raised when a pseudo-managed rootfs cannot be prepared or exported.""" + + +def pseudo_native_environment(): + """Return the pseudo native environment (PSEUDO and OECORE_NATIVE_SYSROOT), from the + qemu-helper-native recipe which provides a pseudo binary usable outside a recipe sysroot.""" + native_sysroot = os.environ.get('OECORE_NATIVE_SYSROOT') + if not native_sysroot: + try: + import bb.tinfoil + except ImportError as exc: + raise PseudoRootfsError( + 'Unable to import bitbake.\n' + 'Did you forget to source your build system environment setup script?') from exc + try: + with bb.tinfoil.Tinfoil() as tinfoil: + tinfoil.prepare(quiet=2) + native_sysroot = tinfoil.parse_recipe('qemu-helper-native').getVar('STAGING_DIR_NATIVE') + except Exception as exc: + raise PseudoRootfsError('Unable to set up the qemu-helper-native sysroot') from exc + + if not native_sysroot or not os.path.exists(native_sysroot): + raise PseudoRootfsError("%s doesn't exist" % native_sysroot) + + environment = {'OECORE_NATIVE_SYSROOT': native_sysroot} + environment['PSEUDO'] = os.path.join(native_sysroot, 'usr', 'bin', 'pseudo') + return environment + + +def _tar_options(rootfs_tarball): + tar_extract_options = { + '.tar.xz': '-xJf', + '.tar.bz2': '-xjf',
scripts/lib/pseudo_rootfs_utils.py line 58
Quoted 10 lines
scripts/lib/pseudo_rootfs_utils.py new file +10
-0+ '.tar.gz': '-xzf', + '.tar.zst': '--zstd -xf', + '.tar': '-xf', + } + for extension, option in tar_extract_options.items(): + if rootfs_tarball.endswith(extension): + return ['--numeric-owner', *option.split()] + raise PseudoRootfsError( + 'Unable to determine sdk tarball format\n' + 'Accepted types: .tar / .tar.gz / .tar.bz2 / .tar.xz / .tar.zst')
This is breaking on RHEL8 era hosts, e.g.:
https://autobuilder.yoctoproject.org/valkyrie/#/builders/42/builds/4591
"tar: unrecognized option '--zstd'"
:(
Cheers,
Richard
PATCH 7/15 devtool: ide-sdk: reuse pseudo_rootfs_utils for SDK rootfs extraction
In reply to AdrianFFrom: Adrian Freihofer <[email protected]>
RecipeImage duplicated the tar/pseudo extraction logic now available in scripts/lib/pseudo_rootfs_utils.py (shared with runqemu-extract-sdk). Drop the local copy in favor of the shared extract_sdk_rootfs()/ pseudo_state_dir().
extract_sdk_rootfs() now takes the pseudo invocation and environment as parameters instead of resolving them itself via qemu-helper-native:
devtool ide-sdk keeps using the image recipe's own FAKEROOTCMD/FAKEROOTENV, which is already guaranteed to be built as part of the image, rather than gaining a new, otherwise unused build dependency on qemu-helper-native. runqemu-extract-sdk resolves pseudo via pseudo_native_environment() itself and passes it in, as it has no recipe of its own to take FAKEROOTCMD from.
--- scripts/lib/devtool/ide_sdk.py | 78 +++++++--------------------------- 1 file changed, 16 insertions(+), 62 deletions(-)
scripts/lib/devtool/ide_sdk.py +16 -62
@@ -27,6 +27,7 @@ from devtool.standard import get_real_srctree from devtool.deploy import parse_packages_arg from devtool.ide_plugins import BuildTool, DebuggerCrossConfig from oe.kernel_module import kernel_module_os_env +from pseudo_rootfs_utils import PseudoRootfsError, extract_sdk_rootfs, pseudo_state_dir logger = logging.getLogger('devtool')
@@ -441,66 +442,6 @@ class RecipeImage: slirp_changed = self.update_qb_slirp_opt() return image_changed or slirp_changed - @staticmethod - def _tar_options(rootfs_tarball): - tar_extract_options = { - '.tar.xz': '-xJf', - '.tar.bz2': '-xjf', - '.tar.gz': '-xzf', - '.tar.zst': '--zstd -xf', - '.tar': '-xf', - } - for extension, option in tar_extract_options.items(): - if rootfs_tarball.endswith(extension): - return ['--numeric-owner', *option.split()] - raise DevtoolError( - 'Unable to determine sdk tarball format\n' - 'Accepted types: .tar / .tar.gz / .tar.bz2 / .tar.xz / .tar.zst') - - @staticmethod - def pseudo_state_dir(rootfs_dir): - """Return the pseudo database location associated with an extracted rootfs.""" - return os.path.realpath(rootfs_dir) + '.pseudo_state' - - def extract_sdk_rootfs(self, rootfs_tarball, rootfs_dir): - """Extract a rootfs tarball under pseudo and return its absolute directory.""" - if not os.path.exists(rootfs_tarball): - raise DevtoolError("sdk tarball '%s' does not exist" % rootfs_tarball) - if not os.path.exists(self.fakerootcmd): - raise DevtoolError('%s does not exist' % self.fakerootcmd) - - rootfs_tarball = os.path.realpath(rootfs_tarball) - rootfs_dir = os.path.realpath(rootfs_dir) - tar_options = self._tar_options(rootfs_tarball) - state_dir = self.pseudo_state_dir(rootfs_dir) - - os.makedirs(rootfs_dir, exist_ok=True) - os.makedirs(state_dir, exist_ok=True) - Path(state_dir, 'pseudo.pid').touch() - - environment = dict(os.environ) - for varvalue in (self.fakerootenv or '').split(): - if '=' in varvalue: - key, value = varvalue.split('=', 1) - environment[key] = value - command = [self.fakerootcmd, 'tar', '-C', rootfs_dir] - - environment['PSEUDO_LOCALSTATEDIR'] = state_dir - environment['PSEUDO_INCLUDE_PATHS'] = rootfs_dir - command.extend(tar_options) - command.append(rootfs_tarball) - logger.info('Extracting rootfs tarball using pseudo: %s', ' '.join(command)) - try: - subprocess.run(command, env=environment, check=True) - except subprocess.CalledProcessError as exc: - raise DevtoolError('Failed to extract %s' % rootfs_tarball) from exc - - if len(os.listdir(rootfs_dir)) < 4: - logger.warning( - "Only few files in %s, please double-check the extraction " - "worked as intended", rootfs_dir) - return rootfs_dir - def extract_nfs_rootfs(self, nfs_export_base_dir, nfs, target): """Refresh the selected rootfs below nfs_export_base_dir.""" if not self.image_link_name:
@@ -511,7 +452,7 @@ class RecipeImage: rootfs_tarball = os.path.join( self.deploy_dir_image, self.image_link_name + suffix + '.tar') rootfs_dir = self.nfs_rootfs_dir(nfs_export_base_dir, nfs) - state_dir = self.pseudo_state_dir(rootfs_dir) + state_dir = pseudo_state_dir(rootfs_dir) if os.path.exists(rootfs_dir): logger.warning(
@@ -521,7 +462,20 @@ class RecipeImage: if os.path.exists(stale_dir): shutil.rmtree(stale_dir) - self.extract_sdk_rootfs(rootfs_tarball, rootfs_dir) + if not os.path.exists(self.fakerootcmd): + raise DevtoolError('%s does not exist' % self.fakerootcmd) + # Reuse the image's own pseudo instead of qemu-helper-native's, so + # extraction does not depend on a recipe devtool ide-sdk never builds. + pseudo_cmd = [self.fakerootcmd] + environment = dict(os.environ) + for varvalue in (self.fakerootenv or '').split(): + if '=' in varvalue: + key, value = varvalue.split('=', 1) + environment[key] = value + try: + extract_sdk_rootfs(rootfs_tarball, rootfs_dir, pseudo_cmd, environment) + except PseudoRootfsError as exc: + raise DevtoolError('Unable to prepare NFS rootfs: %s' % exc) from exc logger.info('NFS rootfs extracted to %s', rootfs_dir) helper = self.nfs_runqemu_helper(nfs_export_base_dir, nfs)
Signature · 1 line
2.55.0
PATCH 8/15 cpp-example: update formatting and code style
In reply to AdrianFFrom: Adrian Freihofer <[email protected]>
Just run clang-format on the source files and add the .clang-format configuration file.
--- .../recipes-test/cpp/cpp-example.inc | 1 + .../recipes-test/cpp/files/.clang-format | 42 +++++++++++++++++++ .../recipes-test/cpp/files/cpp-example.cpp | 4 +- .../recipes-test/cpp/files/daemonize.cpp | 2 +- .../recipes-test/cpp/files/daemonize.hpp | 2 +- .../cpp/files/test-cpp-example.cpp | 13 +++--- 6 files changed, 54 insertions(+), 10 deletions(-) create mode 100644 meta-selftest/recipes-test/cpp/files/.clang-format
meta-selftest/recipes-test/cpp/cpp-example.inc +1 -0
@@ -22,6 +22,7 @@ SRC_URI = "\ file://config.h.in \ file://cpp-example.service \ file://cpp-example.init \ + file://.clang-format \ file://run-ptest \ "
meta-selftest/recipes-test/cpp/files/.clang-format new file +42 -0
42 lines not on this page · raw message
meta-selftest/recipes-test/cpp/files/cpp-example.cpp +2 -2
@@ -28,7 +28,7 @@ bool g_use_syslog = false; // Prints an informational message. Under systemd, stdout is already captured // by the journal. Otherwise, use stdout until daemonized (stdio is then // redirected to /dev/null), and syslog afterwards so messages aren't lost. -void log_info(const std::string& msg) +void log_info(const std::string &msg) { #ifndef WITH_SYSTEMD if (g_use_syslog) {
@@ -41,7 +41,7 @@ void log_info(const std::string& msg) } // namespace -int main(int argc, char* argv[]) +int main(int argc, char *argv[]) { bool endless_mode = false; #ifndef WITH_SYSTEMD
meta-selftest/recipes-test/cpp/files/daemonize.cpp +1 -1
@@ -38,7 +38,7 @@ void daemonize() } } -void write_pidfile(const std::string& path) +void write_pidfile(const std::string &path) { std::ofstream pidfile(path, std::ios::trunc); pidfile << getpid() << std::endl;
meta-selftest/recipes-test/cpp/files/daemonize.hpp +1 -1
@@ -20,7 +20,7 @@ void daemonize(); // Writes the current process's pid to the given path. -void write_pidfile(const std::string& path); +void write_pidfile(const std::string &path); // Permanently drops from root to the given group/user. Must run gid before // uid: once uid is dropped, the process no longer has permission to setgid.
meta-selftest/recipes-test/cpp/files/test-cpp-example.cpp +7 -6
@@ -1,8 +1,8 @@ /* -* Copyright OpenEmbedded Contributors -* -* SPDX-License-Identifier: MIT -*/ + * Copyright OpenEmbedded Contributors + * + * SPDX-License-Identifier: MIT + */ #include "cpp-example-lib.hpp"
@@ -13,10 +13,11 @@ #define FAIL_COMPARISON_STR "" #endif -int main() { +int main() +{ auto cpp_example = CppExample(); auto ret_string = cpp_example.get_string(); - if(0 == ret_string.compare(CppExample::test_string + FAIL_COMPARISON_STR)) { + if (0 == ret_string.compare(CppExample::test_string + FAIL_COMPARISON_STR)) { std::cout << "PASS: " << ret_string << " = " << CppExample::test_string << std::endl; } else { std::cout << "FAIL: " << ret_string << " != " << CppExample::test_string << std::endl;
Signature · 1 line
2.55.0
PATCH 9/15 cpp-example: fix clang-tidy warnings
In reply to AdrianFFrom: Adrian Freihofer <[email protected]>
Just run clang-tidy, fixing warnings and applying the configured checks. Add also the .clang-tidy configuration file.
cmake-example: add .clang-tidy configuration file
--- .../recipes-test/cpp/cpp-example.inc | 1 + .../recipes-test/cpp/files/.clang-tidy | 33 +++++++++++++++++++ .../cpp/files/cpp-example-lib.hpp | 3 +- .../recipes-test/cpp/files/cpp-example.cpp | 1 + .../cpp/files/test-cpp-example.cpp | 2 +- meta/lib/oeqa/selftest/cases/devtool.py | 20 +++++------ 6 files changed, 48 insertions(+), 12 deletions(-) create mode 100644 meta-selftest/recipes-test/cpp/files/.clang-tidy
meta-selftest/recipes-test/cpp/cpp-example.inc +1 -0
@@ -23,6 +23,7 @@ SRC_URI = "\ file://cpp-example.service \ file://cpp-example.init \ file://.clang-format \ + file://.clang-tidy \ file://run-ptest \ "
meta-selftest/recipes-test/cpp/files/.clang-tidy new file +33 -0
@@ -0,0 +1,33 @@ +--- +# Copyright OpenEmbedded Contributors +# +# SPDX-License-Identifier: MIT +Checks: > + -*, + bugprone-*, + -bugprone-easily-swappable-parameters, + clang-analyzer-*, + modernize-*, + -modernize-use-trailing-return-type, + performance-*, + -performance-avoid-endl, + readability-*, + -readability-magic-numbers, + -readability-identifier-length, + -readability-implicit-bool-conversion, + -readability-convert-member-functions-to-static, + -readability-isolate-declaration + +WarningsAsErrors: '' +HeaderFilterRegex: '.*' +FormatStyle: file + +CheckOptions: + - key: readability-identifier-naming.VariableCase + value: lower_case + - key: readability-identifier-naming.FunctionCase + value: lower_case + - key: readability-identifier-naming.StructCase + value: CamelCase + - key: readability-identifier-naming.ClassCase + value: CamelCase
meta-selftest/recipes-test/cpp/files/cpp-example-lib.hpp +2 -1
@@ -12,13 +12,14 @@ struct CppExample { + // NOLINTNEXTLINE(bugprone-throwing-static-initialization) -- exercises debugger handling of a lazy-init static member, see devtool ide-sdk tests inline static const std::string test_string = "cpp-example-lib Magic: 123456789"; /* Header-only function, to exercise breakpoint resolution against * header-only debug info. volatile prevents compiler optimization from * eliminating the function body, ensuring a concrete code location exists * for debugger breakpoints. */ - inline static int scale_number(int n) + static int scale_number(int n) { volatile int scaled = n * 7; std::cout << "scale_number(" << n << ") = " << scaled << std::endl;
meta-selftest/recipes-test/cpp/files/cpp-example.cpp +1 -0
@@ -113,6 +113,7 @@ int main(int argc, char *argv[]) log_info(std::string("Linking json-c version ") + cpp_example.get_json_c_version()); cpp_example.print_json(); + // NOLINTNEXTLINE(bugprone-infinite-loop) -- intentionally endless when requested; stopped externally (e.g. systemd stop/SIGTERM) do { // Read and print message from config file std::string config_message = cpp_example.read_config_message();
meta-selftest/recipes-test/cpp/files/test-cpp-example.cpp +1 -1
@@ -17,7 +17,7 @@ int main() { auto cpp_example = CppExample(); auto ret_string = cpp_example.get_string(); - if (0 == ret_string.compare(CppExample::test_string + FAIL_COMPARISON_STR)) { + if (ret_string == CppExample::test_string + FAIL_COMPARISON_STR) { std::cout << "PASS: " << ret_string << " = " << CppExample::test_string << std::endl; } else { std::cout << "FAIL: " << ret_string << " != " << CppExample::test_string << std::endl;
meta/lib/oeqa/selftest/cases/devtool.py +10 -10
@@ -3312,8 +3312,8 @@ class DevtoolIdeSdkTests(DevtoolBase): cpp_code = file.read() cpp_code = cpp_code.replace(DevtoolIdeSdkTests.MAGIC_STRING_ORIG, MAGIC_STRING_NEW) cpp_code = cpp_code.replace( - " inline static int scale_number(int n)", - extra_lines + " inline static int scale_number(int n)") + " static int scale_number(int n)", + extra_lines + " static int scale_number(int n)") with open(cpp_example_lib_hpp, 'w') as file: file.write(cpp_code)
@@ -3362,8 +3362,8 @@ class DevtoolIdeSdkTests(DevtoolBase): # the first _gdb_cross_debugging_multi call above. self._gdb_cross_debugging_multi( qemu, recipe_name, example_exe, MAGIC_STRING_NEW, - exe_break_line=136 + LINE_SHIFT, exe_list_line=128 + LINE_SHIFT, - hpp_break_line=21 + LINE_SHIFT, lib_break_line=31 + LINE_SHIFT) + exe_break_line=136 + LINE_SHIFT, exe_list_line=129 + LINE_SHIFT, + hpp_break_line=24 + LINE_SHIFT, lib_break_line=31 + LINE_SHIFT) def _verify_cmake_preset(self, tempdir): """Verify the generated cmake preset works as expected
@@ -3532,14 +3532,14 @@ class DevtoolIdeSdkGccTests(DevtoolIdeSdkTests): self.assertIn("GNU gdb", r.output) def _gdb_debug_cpp_example(self, magic_string, gdb_start_cmd="run", - exe_break_line=136, exe_list_line=128, hpp_break_line=21, + exe_break_line=136, exe_list_line=129, hpp_break_line=24, lib_break_line=31): """Get a series of gdb commands to debug the cpp-example-lib example""" gdb_batch_cmd = " -ex 'break main' -ex '%s'" % gdb_start_cmd gdb_batch_cmd += " -ex 'break CppExample::print_json()' -ex 'continue'" gdb_batch_cmd += " -ex 'print CppExample::test_string.compare(\"cpp-example-lib %s\")'" % magic_string gdb_batch_cmd += " -ex 'print CppExample::test_string.compare(\"cpp-example-lib %saaa\")'" % magic_string - gdb_batch_cmd += " -ex 'list cpp-example-lib.hpp:15,15'" + gdb_batch_cmd += " -ex 'list cpp-example-lib.hpp:16,16'" # Break inside the library's own .cpp file by file:line (not by # symbol), while still inside the print_json() call reached above.
@@ -3578,7 +3578,7 @@ class DevtoolIdeSdkGccTests(DevtoolIdeSdkTests): gdb_batch_cmd += " -ex 'continue'" return gdb_batch_cmd - def _gdb_debug_cpp_example_check(self, gdb_output, magic_string, exe_list_line=128, lib_break_line=31): + def _gdb_debug_cpp_example_check(self, gdb_output, magic_string, exe_list_line=129, lib_break_line=31): self.assertIn("Breakpoint 1, main", gdb_output) self.assertIn("$1 = 0", gdb_output) # test.string.compare equal self.assertIn("$2 = -3", gdb_output) # test.string.compare longer
@@ -3606,7 +3606,7 @@ class DevtoolIdeSdkGccTests(DevtoolIdeSdkTests): self.assertIn("exited normally", gdb_output) def _gdb_cross_debugging_multi(self, qemu, recipe_name, example_exe, magic_string, - exe_break_line=136, exe_list_line=128, hpp_break_line=21, + exe_break_line=136, exe_list_line=129, hpp_break_line=24, lib_break_line=31): """Verify gdb-cross is working
@@ -4814,8 +4814,8 @@ class DevtoolIdeSdkClangTests(DevtoolIdeSdkTests): cpp_code = file.read() cpp_code = cpp_code.replace(DevtoolIdeSdkTests.MAGIC_STRING_ORIG, magic_string_new) cpp_code = cpp_code.replace( - " inline static int scale_number(int n)", - extra_lines + " inline static int scale_number(int n)") + " static int scale_number(int n)", + extra_lines + " static int scale_number(int n)") with open(cpp_example_lib_hpp, 'w') as file: file.write(cpp_code)
Signature · 1 line
2.55.0
PATCH 10/15 devtool: ide-sdk: add clangd support for VSCode IntelliSense
In reply to AdrianFFrom: Adrian Freihofer <[email protected]>
Add an llvm-vs-code-extensions.vscode-clangd based IntelliSense provider as an alternative to cpptools for ide=code. clangd is matched to the recipe's real cross compiler via compile_commands.json and --query-driver, which works for gcc cross-compilers as well as clang. As a fisrst step clangd is supported if TOOLCHAIN is clang.
--- scripts/lib/devtool/ide_plugins/ide_code.py | 41 ++++++++++++++++++--- scripts/lib/devtool/ide_sdk.py | 2 + 2 files changed, 38 insertions(+), 5 deletions(-)
scripts/lib/devtool/ide_plugins/ide_code.py +36 -5
@@ -228,6 +228,26 @@ class IdeVSCode(IdeBase): } ] + def __vscode_settings_clangd(self, settings_dict, modified_recipe): + if not (modified_recipe.ide_sdk_intellisense == 'clangd' and modified_recipe.build_tool.is_c_cpp): + return + # Use the recipe's real cross compiler (gcc or clang, whichever TOOLCHAIN + # actually builds it), not a hardcoded 'clang'/'clang++' name. + cross_cc = os.path.join(modified_recipe.staging_bindir_toolchain, modified_recipe.cc.split()[0]) + cross_cxx = os.path.join(modified_recipe.staging_bindir_toolchain, modified_recipe.cxx.split()[0]) + settings_dict["clangd.path"] = os.path.join( + modified_recipe.recipe_sysroot_native, 'usr', 'bin', 'clangd') + settings_dict["clangd.arguments"] = [ + # Allow-list the cross compiler so clangd queries it (not its own + # bundled clang) for target-specific builtin includes/defines. + # Works for gcc cross-compilers too, not just clang. + "--query-driver=%s,%s" % (cross_cc, cross_cxx), + "--compile-commands-dir=%s" % modified_recipe.b, + "--background-index" + ] + # Avoid cpptools (if also installed) fighting clangd over IntelliSense. + settings_dict["C_Cpp.intelliSenseEngine"] = "disabled" + def vscode_settings(self, modified_recipe, image_recipe): files_hide = { "**/.git/**": True,
@@ -261,6 +281,7 @@ class IdeVSCode(IdeBase): self.__vscode_settings_cmake(settings_dict, modified_recipe) self.__vscode_settings_meson(settings_dict, modified_recipe) self.__vscode_settings_kernel_module(settings_dict, modified_recipe) + self.__vscode_settings_clangd(settings_dict, modified_recipe) settings_file = 'settings.json' IdeBase.update_json_file(
@@ -268,16 +289,22 @@ class IdeVSCode(IdeBase): def vscode_extensions(self, modified_recipe): recommendations = [] - if modified_recipe.build_tool.is_c_cpp_kernel: + # clangd (matched to the recipe's real cross compiler via + # compile_commands.json) replaces cpptools as the IntelliSense provider + # if toolchain is clang. CodeLLDB/GDB debugging is unaffected either way. + use_clangd = (modified_recipe.ide_sdk_intellisense == 'clangd' + and modified_recipe.build_tool.is_c_cpp) + if use_clangd: + recommendations += [ + "llvm-vs-code-extensions.vscode-clangd", + "vadimcn.vscode-lldb" + ] + elif modified_recipe.build_tool.is_c_cpp_kernel: recommendations += [ "ms-vscode.cpptools", "ms-vscode.cpptools-extension-pack", "ms-vscode.cpptools-themes" ] - # For clang toolchain, CodeLLDB provides native LLDB debugging in VSCode - if (modified_recipe.toolchain == 'clang' - and modified_recipe.build_tool.is_c_cpp): - recommendations.append("vadimcn.vscode-lldb") if modified_recipe.build_tool is BuildTool.CMAKE: recommendations.append("ms-vscode.cmake-tools") if modified_recipe.build_tool is BuildTool.MESON:
@@ -290,6 +317,10 @@ class IdeVSCode(IdeBase): self.dot_code_dir(modified_recipe), extensions_file, {"recommendations": recommendations}) def vscode_c_cpp_properties(self, modified_recipe, image_recipe): + # cpptools' IntelliSense is disabled in settings.json when clangd is + # the active provider, so this file would be inert clutter. + if modified_recipe.build_tool.is_c_cpp and modified_recipe.ide_sdk_intellisense == 'clangd': + return properties_dict = { "name": modified_recipe.recipe_id_pretty, }
scripts/lib/devtool/ide_sdk.py +2 -0
@@ -789,6 +789,7 @@ class RecipeModified: self.target_arch = None self.tmpdir = None self.toolchain = None + self.ide_sdk_intellisense = None self.topdir = None self.workdir = None # Maps each package name (from PACKAGES) to the glob patterns from its FILES variable
@@ -884,6 +885,7 @@ class RecipeModified: self.target_arch = recipe_d.getVar('TARGET_ARCH') self.tmpdir = os.path.realpath(recipe_d.getVar('TMPDIR')) self.toolchain = recipe_d.getVar('TOOLCHAIN') + self.ide_sdk_intellisense = 'clangd' if self.toolchain == 'clang' else 'cpptools' self.topdir = recipe_d.getVar('TOPDIR') self.workdir = os.path.realpath(recipe_d.getVar('WORKDIR'))
Signature · 1 line
2.55.0
PATCH 11/15 devtool: ide-sdk: support clangd for non-clang toolchain recipes
In reply to AdrianFFrom: Adrian Freihofer <[email protected]>
Make the IntelliSense provider choice independent of TOOLCHAIN.
clangd ships in the clang recipe, whose "clang-native" is only pulled into RECIPE_SYSROOT_NATIVE via clang-cross-* when TOOLCHAIN == "clang". Introduce a new bitbake variable IDE_SDK_INTELLISENSE which can be set to "clangd" on a gcc (or other non-clang) recipe.
This requires adding the clang-native recipe to the sysroots in addition to the recipes TOOLCHAIN. Add RecipeModified.update_bbappend(), mirroring RecipeImage's marker-section bbappend management: which adds DEPENDS:append = " clang-native" to the recipe's own workspace bbappend.
--- scripts/lib/devtool/ide_plugins/ide_code.py | 3 +- scripts/lib/devtool/ide_sdk.py | 99 ++++++++++++++++++++- 2 files changed, 98 insertions(+), 4 deletions(-)
scripts/lib/devtool/ide_plugins/ide_code.py +2 -1
@@ -291,7 +291,8 @@ class IdeVSCode(IdeBase): recommendations = [] # clangd (matched to the recipe's real cross compiler via # compile_commands.json) replaces cpptools as the IntelliSense provider - # if toolchain is clang. CodeLLDB/GDB debugging is unaffected either way. + # when selected via IDE_SDK_INTELLISENSE (defaults to clangd for the + # clang toolchain). CodeLLDB/GDB debugging is unaffected either way. use_clangd = (modified_recipe.ide_sdk_intellisense == 'clangd' and modified_recipe.build_tool.is_c_cpp) if use_clangd:
scripts/lib/devtool/ide_sdk.py +96 -3
138 lines not on this page · raw message
Signature · 1 line
2.55.0
PATCH 12/15 devtool: ide-sdk: format C/C++ with clangd when .clang-format is present
In reply to AdrianFFrom: Adrian Freihofer <[email protected]>
Add "[cpp]"/"[c]" defaultFormatter and editor.formatOnSave settings pointing at the clangd VSCode extension, so devtool ide-sdk-generated workspaces auto-format on save using the recipe's own .clang-format style.
Only enabled when clangd is the active IntelliSense engine (IDE_SDK_INTELLISENSE == "clangd", which also gates whether the clangd extension is recommended/installed) and a .clang-format file exists in the recipe source tree, so gcc/cpptools recipes and recipes without a .clang-format are unaffected.
--- scripts/lib/devtool/ide_plugins/ide_code.py | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
scripts/lib/devtool/ide_plugins/ide_code.py +14 -0
@@ -248,6 +248,19 @@ class IdeVSCode(IdeBase): # Avoid cpptools (if also installed) fighting clangd over IntelliSense. settings_dict["C_Cpp.intelliSenseEngine"] = "disabled" + def __vscode_settings_format(self, settings_dict, modified_recipe): + # clangd is the only formatter wired up here, so only enable it when + # clangd is actually the active IntelliSense engine (its extension is + # recommended/available). + if not (modified_recipe.ide_sdk_intellisense == 'clangd' and modified_recipe.build_tool.is_c_cpp): + return + # Respect the project's own formatting style only if it opted in. + if not os.path.isfile(os.path.join(modified_recipe.real_srctree, '.clang-format')): + return + settings_dict["[cpp]"] = {"editor.defaultFormatter": "llvm-vs-code-extensions.vscode-clangd"} + settings_dict["[c]"] = {"editor.defaultFormatter": "llvm-vs-code-extensions.vscode-clangd"} + settings_dict["editor.formatOnSave"] = True + def vscode_settings(self, modified_recipe, image_recipe): files_hide = { "**/.git/**": True,
@@ -282,6 +295,7 @@ class IdeVSCode(IdeBase): self.__vscode_settings_meson(settings_dict, modified_recipe) self.__vscode_settings_kernel_module(settings_dict, modified_recipe) self.__vscode_settings_clangd(settings_dict, modified_recipe) + self.__vscode_settings_format(settings_dict, modified_recipe) settings_file = 'settings.json' IdeBase.update_json_file(
Signature · 1 line
2.55.0
PATCH 13/15 devtool: ide-sdk: support LLDB ATTACH mode
In reply to AdrianFFrom: Adrian Freihofer <[email protected]>
Bring support for LLDB ATTACH mode to the IDE SDK as it is already supported for GDB.
lldb-server platform mode has no launch/attach distinction on the
target: it just offers a platform connection, and the CodeLLDB client decides client-side whether to launch or attach. ATTACH was previously rejected outright because gdbserver's PID-based attach model doesn't apply, but lldb-server can serve both modes identically.
Give each server_modes() entry (MULTI and ATTACH, for service binaries) its own gdbserver_ports entry instead of the single gdbserver_port, so both get their own slirp-forwarded port. _vscode_launch_bin_dbg_lldb() now emits "request": "attach" with a postDebugTask to stop the lldb-server instance afterwards, and drops the launch-only stopOnEntry/cwd keys, which are invalid in the attach schema. ide_none.py's lldbinit generation and LldbServerConfigNone.initialize() are updated the same way to also generate the ATTACH-mode start script for service binaries.
Also update the tests accordingly to verify the new ATTACH mode configurations.
--- meta/lib/oeqa/selftest/cases/devtool.py | 19 +++++-- scripts/lib/devtool/ide_plugins/__init__.py | 60 ++++++++++----------- scripts/lib/devtool/ide_plugins/ide_code.py | 23 +++++--- scripts/lib/devtool/ide_plugins/ide_none.py | 15 +++--- 4 files changed, 66 insertions(+), 51 deletions(-)
meta/lib/oeqa/selftest/cases/devtool.py +15 -4
@@ -4640,10 +4640,21 @@ class DevtoolIdeSdkClangTests(DevtoolIdeSdkTests): f"Configuration '{config_name}' should not have MIMode (CodeLLDB)") self.assertNotIn("miDebuggerPath", config, f"Configuration '{config_name}' should not have miDebuggerPath") - self.assertEqual(config["request"], "launch", - f"Configuration '{config_name}' should be launch type") - self.assertEqual(config["cwd"], "/tmp", - f"Configuration '{config_name}' cwd should be /tmp (writable on target)") + + is_attach = "_attach" in config_name + if is_attach: + self.assertEqual(config["request"], "attach", + f"Configuration '{config_name}' should be attach type") + self.assertNotIn("cwd", config, + f"Configuration '{config_name}' should not set cwd in attach mode") + self.assertIn("postDebugTask", config, + f"attach configuration '{config_name}' should have postDebugTask " + "to stop the lldb-server platform instance afterwards") + else: + self.assertEqual(config["request"], "launch", + f"Configuration '{config_name}' should be launch type") + self.assertEqual(config["cwd"], "/tmp", + f"Configuration '{config_name}' cwd should be /tmp (writable on target)") # Verify initCommands contain the platform connect sequence init_commands = config.get("initCommands", [])
scripts/lib/devtool/ide_plugins/__init__.py +29 -31
93 lines not on this page · raw message
scripts/lib/devtool/ide_plugins/ide_code.py +16 -7
@@ -508,11 +508,12 @@ class IdeVSCode(IdeBase): return launch_config def _vscode_launch_bin_dbg_lldb(self, lldb_config, server_mode): - """Generate a CodeLLDB (type: lldb) launch configuration entry for launch.json. + """Generate a CodeLLDB (type: lldb) launch/attach configuration entry for launch.json. CodeLLDB connects to lldb-server via the LLDB platform protocol. The initCommands select the remote platform and open the connection before - the process is launched, so CodeLLDB can inspect and control it. + the process is launched or attached to, so CodeLLDB can inspect and + control it. Using targetCreateCommands instead of "program" so we can pass both the local host binary (for debug symbols) and the remote target path (where
@@ -520,14 +521,16 @@ class IdeVSCode(IdeBase): "target create --remote-file". This prevents LLDB from uploading the binary from its module cache to a temporary directory and ensures the process starts from its installed location where the dynamic linker can - find shared libraries via the standard search paths. + find shared libraries via the standard search paths. In ATTACH mode the + same lldb-server platform connection is used. """ modified_recipe = lldb_config.modified_recipe debugger_cross = modified_recipe.debugger_cross + is_attach = server_mode == DebuggerServerModes.ATTACH init_commands = [ "platform select remote-linux", - "platform connect connect://%s:%d" % (debugger_cross.host, lldb_config.debug_server_port), + "platform connect connect://%s:%d" % (debugger_cross.host, lldb_config.port(server_mode)), # Clear the default step-avoid-regexp so std:: and other library # namespaces are not silently skipped on step-in. (default is "std::" in LLDB 15+) "settings set target.process.thread.step-avoid-regexp \"\"",
@@ -592,15 +595,21 @@ class IdeVSCode(IdeBase): launch_config = { "name": lldb_config.id_pretty_mode(server_mode), "type": "lldb", - "request": "launch", + "request": "attach" if is_attach else "launch", # Use targetCreateCommands instead of "program" to control both # the local binary (for debug symbols) and the remote path. "targetCreateCommands": [target_create_cmd], - "stopOnEntry": False, - "cwd": "/tmp", "preLaunchTask": lldb_config.id_pretty_mode(server_mode), "initCommands": init_commands, } + if is_attach: + launch_config["postDebugTask"] = self._stop_task_label( + lldb_config, server_mode) + else: + # cwd configures the process the debugger launches, it is not + # part of the attach schema. + launch_config["stopOnEntry"] = False + launch_config["cwd"] = "/tmp" if source_map: launch_config["sourceMap"] = source_map if modified_recipe.b:
scripts/lib/devtool/ide_plugins/ide_none.py +6 -9
@@ -205,20 +205,15 @@ class LldbServerConfigNone(LldbServerConfig): if server_mode is None: server_mode = self.default_mode lines = ['# This file is generated by devtool ide-sdk'] - if server_mode == DebuggerServerModes.MULTI: - lines.append('# On the remote target:') - lines.append('# lldb-server platform --server --listen *:%d --gdbserver-port %d' % ( - self.debug_server_port, self.gdbserver_port)) - else: - lines.append('# On the remote target:') - lines.append('# lldb-server platform --one-shot --server --listen *:%d --gdbserver-port %d' % ( - self.debug_server_port, self.gdbserver_port)) + lines.append('# On the remote target:') + lines.append('# lldb-server platform --server --listen *:%d --gdbserver-port %d' % ( + self.port(server_mode), self.gdbserver_ports[server_mode])) lines.append('# On the build machine:') lines.append('# cd ' + self.modified_recipe.real_srctree) lines.append('# ' + self.debugger_cross.lldb + ' -s ' + self.lldbinit) lines.append('platform select remote-linux') lines.append('platform connect connect://%s:%d' % ( - self.debugger_cross.host, self.debug_server_port)) + self.debugger_cross.host, self.port(server_mode))) lines.append('settings set target.process.thread.step-avoid-regexp ""') if self.image_recipe.rootfs_dbg:
@@ -267,6 +262,8 @@ class LldbServerConfigNone(LldbServerConfig): def initialize(self): self._gen_lldb_server_start_script() + if self.binary.runs_as_service and self.default_mode != DebuggerServerModes.ATTACH: + self._gen_lldb_server_start_script(DebuggerServerModes.ATTACH) self._gen_lldbinit_config() self._gen_lldb_start_script()
Signature · 1 line
2.55.0
PATCH 14/15 devtool: ide-sdk: pin BBPATH in generated do_install script
In reply to AdrianFFrom: Adrian Freihofer <[email protected]>
The generated bb_run_do_install script already os.chdir()s into the recipe's build topdir before starting tinfoil, but that only covers the fallback case: bb.cookerdata.findConfigFile() checks BBPATH entries for conf/bblayers.conf before ever falling back to cwd. If BBPATH happens to still be set in the caller's environment and points at another valid build directory (e.g. a leftover from a sibling checkout), bitbake would pick up that build's config instead of the intended one.
Set os.environ["BBPATH"] to the recipe's topdir alongside the chdir so the generated script doesn't depend on the caller's environment being clean.
--- scripts/lib/devtool/ide_sdk.py | 5 +++++ 1 file changed, 5 insertions(+)
scripts/lib/devtool/ide_sdk.py +5 -0
@@ -1633,6 +1633,11 @@ class RecipeModified: os.path.join(self.bitbakepath, '..', 'lib')), 'import bb.tinfoil', 'os.chdir(%r)' % self.topdir, + # A stale BBPATH from the caller's environment (e.g. a + # different, still-valid build dir) would otherwise take + # precedence over cwd when bitbake looks for + # conf/bblayers.conf (see bb.cookerdata.findConfigFile). + 'os.environ["BBPATH"] = %r' % self.topdir, 'tinfoil = bb.tinfoil.Tinfoil()', 'try:', ' tinfoil.prepare(config_only=False, quiet=2)',
Signature · 1 line
2.55.0
PATCH 15/15 buildstats: close proc/disk log files via weakref.finalize safety net
In reply to AdrianFFrom: Adrian Freihofer <[email protected]>
SystemStats.close() is normally invoked by the buildstats.bbclass handler on bb.event.BuildCompleted, but that event is skipped for internal, event-less task runs (e.g. tinfoil's run_prepared_task(), used by devtool ide-sdk). Without a BuildCompleted event, the open proc/disk log files were only closed implicitly at GC/interpreter-exit time, causing a ResourceWarning for leaked file handles.
Register a weakref.finalize() callback at construction time as a safety net. close() now just triggers the finalizer, which is idempotent and keeps no reference back to SystemStats itself.
AI-Generated: Uses GitHub Copilot
--- meta/lib/oe/buildstats.py | 15 +++++++++++++-- 1 file changed, 13 insertions(+), 2 deletions(-)
meta/lib/oe/buildstats.py +13 -2
@@ -10,6 +10,7 @@ import os import time import re +import weakref import bb.event from collections import deque
@@ -56,6 +57,12 @@ class SystemStats: destfile = os.path.join(bsdir, '%sproc_%s.log' % ('reduced_' if handler else '', filename)) self.proc_files.append((filename, open(destfile, 'ab'), handler)) self.monitor_disk = open(os.path.join(bsdir, 'monitor_disk.log'), 'ab') + # Safety net: normally close() is called when bb.event.BuildCompleted + # fires, but that event is skipped for internal, event-less task runs + # (e.g. tinfoil's run_prepared_task()). Without this, the open files + # would only get closed implicitly at GC/interpreter-exit time, + # triggering a ResourceWarning. + self._finalizer = weakref.finalize(self, self._close_files, self.proc_files, self.monitor_disk) # Last time that we sampled /proc data resp. recorded disk monitoring data. self.last_proc = 0 self.last_disk_monitor = 0
@@ -81,8 +88,12 @@ class SystemStats: self.net_stats = {} def close(self): - self.monitor_disk.close() - for _, output, _ in self.proc_files: + self._finalizer() + + @staticmethod + def _close_files(proc_files, monitor_disk): + monitor_disk.close() + for _, output, _ in proc_files: output.close() def _reduce_meminfo(self, time, data, filename):
Signature · 1 line
2.55.0