oeqa/sdk: run the SDK's own copy of a host tool
PATCH 1/3 oeqa/sdk: run the SDK's own copy of a host tool
ensure_host_package() decides whether a test can run by looking for a package name in the SDK's host manifest, which establishes neither that the package is present nor that the command the test runs is the SDK's own. On an extensible SDK the manifest lists what the shipped sstate could provide, so a case can pass the check and then run a command that is not there, or one belonging to the build host.
Ask the SDK for the command instead. An eSDK ships sstate for more than it installs, so when the command is missing devtool installs the recipe and the lookup runs again. A command that resolves outside the SDK now fails rather than reporting a result about the build host.
AI-Generated: codex/claude-opus 5 (xhigh)
--- meta/lib/oeqa/sdk/case.py | 96 ++++++++++++++++++++++++++++----------- 1 file changed, 69 insertions(+), 27 deletions(-)
meta/lib/oeqa/sdk/case.py +69 -27
@@ -18,38 +18,80 @@ class OESDKTestCase(OETestCase): (self.tc.sdk_env, cmd), shell=True, executable="/bin/bash", stderr=subprocess.STDOUT, universal_newlines=True) - def ensure_host_package(self, *packages, recipe=None): + def find_sdk_binary(self, binaries): """ - Check that the host variation of one of the packages listed is available - in the SDK (nativesdk-foo for SDK, foo-native for eSDK). The package is - a list for the case where debian-renaming may have occured, and the - manifest could contain 'foo' or 'libfoo'. + (name, outside) for the first of binaries that resolves. - If testing an eSDK and the package is not found, then try to install the - specified recipe to install it from sstate. + name is the command to run when it came from the SDK, and outside + lists what resolved beyond it otherwise. Callers run the name + returned: an eSDK stages native recipes, which need not name a + command the way their nativesdk counterparts do. perl-native keeps + its interpreter at bindir/perl-native/perl, reached as nativeperl. + """ + if isinstance(binaries, str): + binaries = (binaries,) + sdk_root = os.path.dirname(self.tc.sdk_env) + + outside = [] + for binary in binaries: + try: + path = self._run("command -v %s" % binary).strip() + except subprocess.CalledProcessError: + continue + if path.startswith(sdk_root + os.sep): + return binary, [] + outside.append("%s is %s" % (binary, path)) + return None, outside + + def ensure_sdk_binary(self, binaries, packages=()): """ + Return the first of binaries that resolves inside the SDK. - # In a SDK the manifest is correct. In an eSDK the manifest may be - # correct (type=full) or not include packages that exist in sstate but - # not installed yet (minimal) so we should try to install the recipe. - for package in packages: - if isinstance(self.tc, OESDKExtTestContext): - package = package + "-native" - else: - package = "nativesdk-" + package + Fail when one resolves outside it, because a result from the build + host describes the host and not the SDK. Skip when none resolves. + """ + if isinstance(binaries, str): + binaries = (binaries,) + if isinstance(packages, str): + packages = (packages,) + name, outside = self.find_sdk_binary(binaries) + if name: + return name + need = ", ".join(packages) or ", ".join(binaries) + if outside: + self.fail("Test %s needs one of %s from the SDK, but found %s" + % (self.id(), need, "; ".join(outside))) + raise unittest.SkipTest("Test %s needs one of %s: the SDK has no %s" + % (self.id(), need, " or ".join(binaries))) + + def ensure_host_package(self, *packages, recipe=None, binary=None): + """ + Return the command the test should run, having confirmed it comes + from the SDK rather than the build host. - if self.tc.hasHostPackage(package): - break - else: - if isinstance(self.tc, OESDKExtTestContext): - recipe = (recipe or packages[0]) + "-native" - print("Trying to install %s..." % recipe) - try: - self._run('devtool sdk-install %s' % recipe) - except subprocess.CalledProcessError: - raise unittest.SkipTest("Test %s needs one of %s" % (self.id(), ", ".join(packages))) - else: - raise unittest.SkipTest("Test %s needs one of %s" % (self.id(), ", ".join(packages))) + An eSDK ships sstate for more than it installs, so a tool can be + available without being present. When the command is missing, ask + devtool to install the recipe and look again. + + binary names the commands the packages provide, and is needed only + where a command is not named after its package: pkgconf provides + pkg-config, and an eSDK reaches perl as nativeperl. + """ + binaries = binary or packages + + name, _ = self.find_sdk_binary(binaries) + if name: + return name + + if isinstance(self.tc, OESDKExtTestContext): + install = (recipe or packages[0]) + "-native" + print("Trying to install %s..." % install) + try: + self._run('devtool sdk-install %s' % install) + except subprocess.CalledProcessError: + pass + + return self.ensure_sdk_binary(binaries, packages) def ensure_target_package(self, *packages, multilib=False, recipe=None): """
PATCH 2/3 oeqa/sdk: name the commands the host tool cases need
These cases identify a tool only by the package carrying it, so they run whatever the build host provides. gtk3 shows this is not confined to
eSDKs: its glib-2.0-utils check passes on a standard SDK while glib-compile-resources and glib-mkenums both resolve to /usr/bin.
perl needs two names because the recipe installs it twice:
nativesdk-perl creates ${bindir}/perl, and perl-native creates ${bindir}/perl-native/perl, kept off PATH so recipes running "env perl" do not find it.
AI-Generated: codex/claude-opus 5 (xhigh)
--- meta/lib/oeqa/sdk/cases/gtk3.py | 3 ++- meta/lib/oeqa/sdk/cases/maturin.py | 8 +++++--- meta/lib/oeqa/sdk/cases/meson.py | 2 +- meta/lib/oeqa/sdk/cases/perl.py | 4 ++-- meta/lib/oeqa/sdk/cases/python.py | 2 +- meta/lib/oeqa/sdk/cases/rust.py | 2 ++ 6 files changed, 13 insertions(+), 8 deletions(-)
meta/lib/oeqa/sdk/cases/gtk3.py +2 -1
@@ -18,7 +18,8 @@ class GTK3Test(MesonTestBase): def setUp(self): super().setUp() self.ensure_target_package("gtk+3", "libgtk-3.0", recipe="gtk+3") - self.ensure_host_package("glib-2.0-utils", "libglib-2.0-utils", recipe="glib-2.0") + self.ensure_host_package("glib-2.0-utils", "libglib-2.0-utils", + recipe="glib-2.0", binary="glib-compile-resources") """ Test that autotools and GTK+ 3 compiles correctly.
meta/lib/oeqa/sdk/cases/maturin.py +5 -3
@@ -16,10 +16,12 @@ errors_have_output() class MaturinTest(OESDKTestCase): def setUp(self): - self.ensure_host_package("python3-maturin") + self.ensure_host_package("python3-maturin", binary="maturin") + self.python3 = self.ensure_host_package("python3-core", recipe="python3", + binary="python3") def test_maturin_list_python(self): - out = self._run(r"""python3 -c 'import sys; print(f"{sys.executable}\n{sys.version_info.major}.{sys.version_info.minor}")'""") + out = self._run(r"""%s -c 'import sys; print(f"{sys.executable}\n{sys.version_info.major}.{sys.version_info.minor}")'""" % self.python3) executable, version = out.splitlines() output = self._run("maturin list-python")
@@ -32,7 +34,7 @@ class MaturinTest(OESDKTestCase): class MaturinDevelopTest(OESDKTestCase): def setUp(self): machine = self.td.get("MACHINE") - self.ensure_host_package("python3-maturin") + self.ensure_host_package("python3-maturin", binary="maturin") if not ( self.tc.hasHostPackage("packagegroup-rust-cross-canadian-%s" % machine)
meta/lib/oeqa/sdk/cases/meson.py +1 -1
@@ -25,7 +25,7 @@ class MesonTestBase(OESDKTestCase): self.skipTest(f"{self.id()} does not support eSDK (https://bugzilla.yoctoproject.org/show_bug.cgi?id=15854)") self.ensure_host_package("meson") - self.ensure_host_package("pkgconf") + self.ensure_host_package("pkgconf", binary="pkg-config") def build_meson(self, sourcedir, builddir, installdir=None, options=""): """
meta/lib/oeqa/sdk/cases/perl.py +2 -2
@@ -11,9 +11,9 @@ errors_have_output() class PerlTest(OESDKTestCase): def setUp(self): - self.ensure_host_package("perl") + self.perl = self.ensure_host_package("perl", binary=("perl", "nativeperl")) def test_perl(self): - cmd = "perl -e '$_=\"Uryyb, jbeyq\"; tr/a-zA-Z/n-za-mN-ZA-M/;print'" + cmd = "%s -e '$_=\"Uryyb, jbeyq\"; tr/a-zA-Z/n-za-mN-ZA-M/;print'" % self.perl output = self._run(cmd) self.assertEqual(output, "Hello, world")
meta/lib/oeqa/sdk/cases/python.py +1 -1
@@ -11,7 +11,7 @@ errors_have_output() class Python3Test(OESDKTestCase): def setUp(self): - self.ensure_host_package("python3-core", recipe="python3") + self.ensure_host_package("python3-core", recipe="python3", binary="python3") def test_python3(self): cmd = "python3 -c \"import codecs; print(codecs.encode('Uryyb, jbeyq', 'rot13'))\""
meta/lib/oeqa/sdk/cases/rust.py +2 -0
@@ -29,6 +29,7 @@ class RustCompileTest(OESDKTestCase): machine = self.td.get("MACHINE") if not self.tc.hasHostPackage("packagegroup-rust-cross-canadian-%s" % machine): raise unittest.SkipTest("RustCompileTest class: SDK doesn't contain a Rust cross-canadian toolchain") + self.ensure_sdk_binary("cargo") def test_cargo_build(self): self._run('cd %s/hello; cargo add zstd' % (self.tc.sdk_dir))
@@ -50,6 +51,7 @@ class RustHostCompileTest(OESDKTestCase): machine = self.td.get("MACHINE") if not self.tc.hasHostPackage("packagegroup-rust-cross-canadian-%s" % machine): raise unittest.SkipTest("RustCompileTest class: SDK doesn't contain a Rust cross-canadian toolchain") + self.ensure_sdk_binary("cargo") def test_cargo_build(self): sdksys = self.td.get("SDK_SYS")
PATCH 3/3 oeqa/sdk: run the Go driver the SDK installs
In reply to Trevor WoernerThe Go cases confirm go-cross-canadian is in the SDK and then guard on "which go", which cannot succeed: the recipe installs ${TARGET_PREFIX}go and leaves the unprefixed binary in its own libdir, off PATH. So the cases skip on an SDK that does contain Go. Ask for either name and run what the SDK provides.
Running them for the first time shows the module cases also need -buildvcs=false, as they build inside a git tree and Go fails stamping VCS information into the binary.
AI-Generated: codex/claude-opus 5 (xhigh)
--- meta/lib/oeqa/sdk/cases/go.py | 24 ++++++++---------------- 1 file changed, 8 insertions(+), 16 deletions(-)
meta/lib/oeqa/sdk/cases/go.py +8 -16
@@ -30,21 +30,17 @@ class GoCompileTest(OESDKTestCase): if not self.tc.hasHostPackage("go-cross-canadian-%s" % translated_target_arch): raise unittest.SkipTest("GoCompileTest class: SDK doesn't contain a Go cross-canadian toolchain") - # Additional runtime check for go command availability - try: - self._run('which go') - except Exception as e: - raise unittest.SkipTest("GoCompileTest class: go command not available: %s" % str(e)) + self.go = self.ensure_sdk_binary(("go", "${TARGET_PREFIX}go")) def test_go_build(self): """Test Go build command (native compilation)""" - self._run('cd %s; go build -o test test.go' % self.tc.sdk_dir) + self._run('cd %s; %s build -o test test.go' % (self.tc.sdk_dir, self.go)) def test_go_module(self): """Test Go module creation and building""" # Create a simple Go module - self._run('cd %s; go mod init hello-go' % self.tc.sdk_dir) - self._run('cd %s; go build -o hello-go' % self.tc.sdk_dir) + self._run('cd %s; %s mod init hello-go' % (self.tc.sdk_dir, self.go)) + self._run('cd %s; %s build -buildvcs=false -o hello-go' % (self.tc.sdk_dir, self.go)) @classmethod def tearDownClass(self):
@@ -68,11 +64,7 @@ class GoHostCompileTest(OESDKTestCase): if not self.tc.hasHostPackage("go-cross-canadian-%s" % translated_target_arch): raise unittest.SkipTest("GoHostCompileTest class: SDK doesn't contain a Go cross-canadian toolchain") - # Additional runtime check for go command availability - try: - self._run('which go') - except Exception as e: - raise unittest.SkipTest("GoHostCompileTest class: go command not available: %s" % str(e)) + self.go = self.ensure_sdk_binary(("go", "${TARGET_PREFIX}go")) def _get_go_arch(self): """Get Go architecture from SDK_SYS"""
@@ -85,13 +77,13 @@ class GoHostCompileTest(OESDKTestCase): def test_go_cross_compile(self): """Test Go cross-compilation for target""" goarch = self._get_go_arch() - self._run('cd %s; GOOS=linux GOARCH=%s go build -o test-%s test.go' % (self.tc.sdk_dir, goarch, goarch)) + self._run('cd %s; GOOS=linux GOARCH=%s %s build -o test-%s test.go' % (self.tc.sdk_dir, goarch, self.go, goarch)) def test_go_module_cross_compile(self): """Test Go module cross-compilation""" goarch = self._get_go_arch() - self._run('cd %s; go mod init hello-go' % self.tc.sdk_dir) - self._run('cd %s; GOOS=linux GOARCH=%s go build -o hello-go-%s' % (self.tc.sdk_dir, goarch, goarch)) + self._run('cd %s; %s mod init hello-go' % (self.tc.sdk_dir, self.go)) + self._run('cd %s; GOOS=linux GOARCH=%s %s build -buildvcs=false -o hello-go-%s' % (self.tc.sdk_dir, goarch, self.go, goarch)) @classmethod def tearDownClass(self):
Quoted 7 lines · Trevor Woerner via lists.openembedded.org
ensure_host_package()decides whether a test can run by looking for a package name in the SDK's host manifest, which establishes neither that the package is present nor that the command the test runs is the SDK's own. On an extensible SDK the manifest lists what the shipped sstate could provide, so a case can pass the check and then run a command that is not there, or one belonging to the build host.
Quoted 8 lines
Ask the SDK for the command instead. An eSDK ships sstate for more than it installs, so when the command is missing devtool installs the recipe and the lookup runs again. A command that resolves outside the SDK now fails rather than reporting a result about the build host.
AI-Generated: codex/claude-opus 5 (xhigh)
Signed-off-by Trevor Woerner <[email protected]>---
Hi Trevor,
Thanks for your patch.
It looks like this is breaking the meta_ide.MetaIDE.test_meta_ide_can_run_sdk_tests test:
2026-08-31 12:29:54,708 - oe-selftest - INFO - meta_ide.MetaIDE.test_meta_ide_can_run_sdk_tests (subunit.RemotedTestCase)
2026-08-31 12:29:54,709 - oe-selftest - INFO - ... FAIL
...
2026-08-31 12:29:54,709 - oe-selftest - INFO - 12: 3/4 9/21 (127.32s) (0 failed) (meta_ide.MetaIDE.test_meta_ide_can_run_sdk_tests)
2026-08-31 12:29:54,709 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last):
File "/srv/pokybuild/yocto-worker/qemux86-tc/build/layers/openembedded-core/meta/lib/oeqa/selftest/cases/meta_ide.py", line 60, in test_meta_ide_can_run_sdk_tests
bitbake('-c testsdk meta-ide-support')
~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/srv/pokybuild/yocto-worker/qemux86-tc/build/layers/openembedded-core/meta/lib/oeqa/utils/commands.py", line 236, in bitbake
return runCmd(cmd, ignore_status, timeout, output_log=output_log, **options)
File "/srv/pokybuild/yocto-worker/qemux86-tc/build/layers/openembedded-core/meta/lib/oeqa/utils/commands.py", line 214, in runCmd
raise AssertionError("Command '%s' returned non-zero exit status %d:\n%s" % (command, result.status, exc_output))AssertionError: Command 'bitbake -c testsdk meta-ide-support' returned non-zero exit status 1:
...
https://autobuilder.yoctoproject.org/valkyrie/#/builders/28/builds/4509
https://autobuilder.yoctoproject.org/valkyrie/#/builders/5/builds/4500
https://autobuilder.yoctoproject.org/valkyrie/#/builders/66/builds/4614
Can you have a look at the issue?
Thanks,
Mathieu
Signature · 3 lines
Mathieu Dubois-Briand, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
Quoted 12 lines · Trevor Woerner via lists.openembedded.org
ensure_host_package()decides whether a test can run by looking for a package name in the SDK's host manifest, which establishes neither that the package is present nor that the command the test runs is the SDK's own. On an extensible SDK the manifest lists what the shipped sstate could provide, so a case can pass the check and then run a command that is not there, or one belonging to the build host.Ask the SDK for the command instead. An eSDK ships sstate for more than it installs, so when the command is missing devtool installs the recipe and the lookup runs again. A command that resolves outside the SDK now fails rather than reporting a result about the build host.
Quoted 5 lines
AI-Generated: codex/claude-opus 5 (xhigh)
Signed-off-by Trevor Woerner <[email protected]>---
meta/lib/oeqa/sdk/case.py | 96 ++++++++++++++++++++++++++++-----------1 file changed, 69 insertions(+), 27 deletions(-)
I'll be honest, this feels/looks wrong. In order to say why, I need to spend a lot of time looking into the problem more, trying to work out the right way to solve it and so on.
If I keep doing this, people will just keep feeding me patches like this and expecting me to give that level of feedback which I simply can't do, it isn't easy.
So no, this isn't right and no, I don't feel it appropriate to merge it, sorry.
Cheers,
Richard
Quoted 23 lines · Richard Purdie
Quoted 12 lines · Trevor Woerner via lists.openembedded.org
ensure_host_package()decides whether a test can run by looking for a package name in the SDK's host manifest, which establishes neither that the package is present nor that the command the test runs is the SDK's own. On an extensible SDK the manifest lists what the shipped sstate could provide, so a case can pass the check and then run a command that is not there, or one belonging to the build host.Ask the SDK for the command instead. An eSDK ships sstate for more than it installs, so when the command is missing devtool installs the recipe and the lookup runs again. A command that resolves outside the SDK now fails rather than reporting a result about the build host.
Quoted 5 lines
AI-Generated: codex/claude-opus 5 (xhigh)
Signed-off-by Trevor Woerner <[email protected]>---
meta/lib/oeqa/sdk/case.py | 96 ++++++++++++++++++++++++++++-----------1 file changed, 69 insertions(+), 27 deletions(-)I'll be honest, this feels/looks wrong. In order to say why, I need to spend a lot of time looking into the problem more, trying to work out the right way to solve it and so on.
Quoted 6 lines
If I keep doing this, people will just keep feeding me patches like this and expecting me to give that level of feedback which I simply can't do, it isn't easy.
So no, this isn't right and no, I don't feel it appropriate to merge it, sorry.
Understood, thank you for the feedback.
Quoted 3 lines
Cheers,
Richard