image-artifact-names: add IMAGE_MACHINE_SUFFIX variable
PATCH 1/14 git-submodule-test: disable upstream version check
In reply to a message not in the archive* this is one of the failures from distrodata.Distrodata.test_checkpkg:
2023-03-11 14:26:21,482 - oe-selftest - INFO - ======================================================================
2023-03-11 14:26:21,482 - oe-selftest - INFO - FAIL: test_checkpkg (distrodata.Distrodata.test_checkpkg)
2023-03-11 14:26:21,482 - oe-selftest - INFO - ----------------------------------------------------------------------
2023-03-11 14:26:21,482 - oe-selftest - INFO - Traceback (most recent call last):
File "/OE/build/poky/meta/lib/oeqa/selftest/cases/distrodata.py", line 40, in test_checkpkg
self.assertTrue(len(regressed_failures) == 0 and len(regressed_successes) == 0, msg)
AssertionError: False is not true :
The following packages failed upstream version checks. Please fix them using UPSTREAM_CHECK_URI/UPSTREAM_CHECK_REGEX
(when using tarballs) or UPSTREAM_CHECK_GITTAGREGEX (when using git). If an upstream version check cannot be performed
(for example, if upstream does not use git tags), you can set UPSTREAM_VERSION_UNKNOWN to '1' in the recipe to acknowledge
that the check cannot be performed.
git-submodule-test
binutils Stdout:
Loading cache...done.
Loaded 0 entries from dependency cache.
Parsing recipes...done.
Parsing of 947 .bb files complete (0 cached, 947 parsed). 1764 targets, 46 skipped, 0 masked, 0 errors.
----------------------------------------------------------------------
2023-03-11 14:26:21,482 - oe-selftest - INFO - Ran 1 test in 193.764s 2023-03-11 14:26:21,482 - oe-selftest - INFO - FAILED 2023-03-11 14:26:21,483 - oe-selftest - INFO - (failures=1) 2023-03-11 14:26:26,258 - oe-selftest - INFO - RESULTS:
martin@jama /OE/build/poky/build $ devtool check-upgrade-status git-submodule-test
NOTE: Could not list remote: Fetcher failure for URL: 'gitsm://git.yoctoproject.org/git-submodule-test;branch=master'.
The command git -c gc.autoDetach=false -c core.pager=cat ls-remote git://git.yoctoproject.org/git-submodule-test refs/tags/* gave empty output unexpectedly
INFO: git-submodule-test 1.0 UNKNOWN_BROKEN None
and there are no tags in this test repo:
$ git ls-remote git://git.yoctoproject.org/git-submodule-test a2885dd7d25380d23627e7544b7bbb55014b16ee HEAD d199bbf9ed2216bd1f38aec000d865ae08279119 refs/heads/changed-url a2885dd7d25380d23627e7544b7bbb55014b16ee refs/heads/master 049da4a6cb198d7c0302e9e8b243a1443cb809a7 refs/heads/ssh-gitsm-tests bbe99a1465c5cc52720936d075ddbc5ebe2ed731 refs/meta/cgit
* the binutils failure is strange, when I've added some debug output to see why it's listed the issue disappeared, maybe some temporary network glitch or something
--- .../recipes-test/git-submodule-test/git-submodule-test.bb | 2 ++ 1 file changed, 2 insertions(+)
meta-selftest/recipes-test/git-submodule-test/git-submodule-test.bb +2 -0
@@ -5,6 +5,8 @@ LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda INHIBIT_DEFAULT_DEPS = "1" +UPSTREAM_VERSION_UNKNOWN = "1" + SRC_URI = "gitsm://git.yoctoproject.org/git-submodule-test;branch=master" SRCREV = "a2885dd7d25380d23627e7544b7bbb55014b16ee"
Signature · 1 line
2.39.2
PATCH 2/14 selftest: devtool: set BB_HASHSERVE_UPSTREAM when setting SSTATE_MIRROR
In reply to a message not in the archive* with my build/conf/local.conf:
SSTATE_DIR = "/OE/build/poky/build/sstate-cache"
these devtool tests will first set own SSTATE_DIR and the original one set as SSTATE_MIRROR:
2023-03-11 11:51:46,837 - oe-selftest - INFO - test_devtool_update_recipe_append (devtool.DevtoolUpdateTests.test_devtool_update_recipe_append) 2023-03-11 11:51:46,846 - oe-selftest - DEBUG - Appending to: /OE/build/poky/build/build-st-2023-03-11-patch2/devtool.DevtoolUpdateTests.test_devtool_update_recipe_append/build-st/conf/selftest.inc
SSTATE_DIR = "/OE/build/poky/build/build-st-2023-03-11-patch2/devtool.DevtoolUpdateTests.test_devtool_update_recipe_append/build-st/sstate_devtool" SSTATE_MIRRORS += "file://.* file:////OE/build/poky/build/sstate-cache/PATH"
* but that unfortunately leads to a warning from sanity.bbclass about SSTATE_MIRRORS without matching BB_HASHSERVE, because BB_HASHSERVE is set to "auto" by default
these tests failing with:
2023-03-11 11:55:39,610 - oe-selftest - INFO - ======================================================================
2023-03-11 11:55:39,610 - oe-selftest - INFO - FAIL: test_devtool_update_recipe_append_git (devtool.DevtoolUpdateTests.test_devtool_update_recipe_append_git)
2023-03-11 11:55:39,610 - oe-selftest - INFO - ----------------------------------------------------------------------
2023-03-11 11:55:39,611 - oe-selftest - INFO - Traceback (most recent call last):
File "/OE/build/poky/meta/lib/oeqa/selftest/cases/devtool.py", line 1118, in test_devtool_update_recipe_append_git
self.assertNotIn('WARNING:', result.output)AssertionError: 'WARNING:' unexpectedly found in 'NOTE: Starting bitbake server...\nWARNING: You are using a local hash equivalence server but have configured an sstate mirror. This will likely mean no sstate will match from the mirror. You may wish to disable the hash equivalence use (BB_HASHSERVE), or use a hash equivalence server alongside the sstate mirror.\nLoading cache...done.\nLoaded 0 entries from dependency cache.\nParsing recipes...done.\nParsing of 947 .bb files complete (0 cached, 947 parsed). 1764 targets, 52 skipped, 0 masked, 0 errors.\n\nSummary: There was 1 WARNING message.\nINFO: Updating SRCREV in recipe mtd-utils-selftest_git.bb\nNOTE: Writing append file /tmp/devtoolqa1m2lh02v/layer/recipes-devtools/mtd/mtd-utils-selftest_git.bbappend'
----------------------------------------------------------------------
* just setting BB_HASHSERVE to empty doesn't work, because then we would need to disable OEEquivHash as well as it fails with:
ERROR: OEEquivHash requires BB_HASHSERVE to be set
--- meta/lib/oeqa/selftest/cases/devtool.py | 1 + 1 file changed, 1 insertion(+)
meta/lib/oeqa/selftest/cases/devtool.py +1 -0
@@ -276,6 +276,7 @@ class DevtoolBase(DevtoolTestCase): cls.sstate_conf = 'SSTATE_DIR = "%s"\n' % cls.devtool_sstate cls.sstate_conf += ('SSTATE_MIRRORS += "file://.* file:///%s/PATH"\n' % cls.original_sstate) + cls.sstate_conf += ('BB_HASHSERVE_UPSTREAM = "hashserv.yocto.io:8687"\n') @classmethod def tearDownClass(cls):
Signature · 1 line
2.39.2
PATCH 3/14 selftest: wic: respect IMAGE_LINK_NAME
In reply to a message not in the archive* use IMAGE_LINK_NAME instead of hardcoding
core-image-minimal-${MACHINE} assumption
[YOCTO #12937]
--- meta/lib/oeqa/selftest/cases/wic.py | 38 ++++++++++++++--------------- 1 file changed, 19 insertions(+), 19 deletions(-)
meta/lib/oeqa/selftest/cases/wic.py +19 -19
@@ -805,12 +805,13 @@ class Wic2(WicTestCase): config = 'IMAGE_FSTYPES += "wic"\nWKS_FILE = "wic-image-minimal"\n'\ 'MACHINE_FEATURES:append = " efi"\n' self.append_config(config) - bitbake('wic-image-minimal') + image = 'wic-image-minimal' + bitbake(image) self.remove_config(config) - deploy_dir = get_bb_var('DEPLOY_DIR_IMAGE') - machine = self.td['MACHINE'] - prefix = os.path.join(deploy_dir, 'wic-image-minimal-%s.' % machine) + bb_vars = get_bb_vars(['DEPLOY_DIR_IMAGE', 'IMAGE_LINK_NAME'], image) + prefix = os.path.join(bb_vars['DEPLOY_DIR_IMAGE'], '%s.' % bb_vars['IMAGE_LINK_NAME']) + # check if we have result image and manifests symlinks # pointing to existing files for suffix in ('wic', 'manifest'):
@@ -1049,14 +1050,14 @@ class Wic2(WicTestCase): def _rawcopy_plugin(self, fstype): """Test rawcopy plugin""" - img = 'core-image-minimal' - machine = self.td["MACHINE"] + image = 'core-image-minimal' + bb_vars = get_bb_vars(['DEPLOY_DIR_IMAGE', 'IMAGE_LINK_NAME'], image) params = ',unpack' if fstype.endswith('.gz') else '' with NamedTemporaryFile("w", suffix=".wks") as wks: - wks.write('part / --source rawcopy --sourceparams="file=%s-%s.%s%s"\n'\ - % (img, machine, fstype, params)) + wks.write('part / --source rawcopy --sourceparams="file=%s.%s%s"\n'\ + % (bb_vars['IMAGE_LINK_NAME'], fstype, params)) wks.flush() - cmd = "wic create %s -e %s -o %s" % (wks.name, img, self.resultdir) + cmd = "wic create %s -e %s -o %s" % (wks.name, image, self.resultdir) runCmd(cmd) wksname = os.path.splitext(os.path.basename(wks.name))[0] out = glob(os.path.join(self.resultdir, "%s-*direct" % wksname))
@@ -1077,12 +1078,11 @@ class Wic2(WicTestCase): """Test empty plugin""" config = 'IMAGE_FSTYPES = "wic"\nWKS_FILE = "test_empty_plugin.wks"\n' self.append_config(config) - bitbake('core-image-minimal') + image = 'core-image-minimal' + bitbake(image) self.remove_config(config) - deploy_dir = get_bb_var('DEPLOY_DIR_IMAGE') - machine = self.td['MACHINE'] - - image_path = os.path.join(deploy_dir, 'core-image-minimal-%s.wic' % machine) + bb_vars = get_bb_vars(['DEPLOY_DIR_IMAGE', 'IMAGE_LINK_NAME'], image) + image_path = os.path.join(bb_vars['DEPLOY_DIR_IMAGE'], '%s.wic' % bb_vars['IMAGE_LINK_NAME']) self.assertTrue(os.path.exists(image_path)) sysroot = get_bb_var('RECIPE_SYSROOT_NATIVE', 'wic-tools')
@@ -1297,12 +1297,12 @@ class Wic2(WicTestCase): # build an image config = 'IMAGE_FSTYPES = "wic"\nWKS_FILE = "directdisk.wks"\n' self.append_config(config) - bitbake('core-image-minimal') + image = 'core-image-minimal' + bitbake(image) # get path to the image - deploy_dir = get_bb_var('DEPLOY_DIR_IMAGE') - machine = self.td['MACHINE'] - image_path = os.path.join(deploy_dir, 'core-image-minimal-%s.wic' % machine) + bb_vars = get_bb_vars(['DEPLOY_DIR_IMAGE', 'IMAGE_LINK_NAME'], image) + image_path = os.path.join(bb_vars['DEPLOY_DIR_IMAGE'], '%s.wic' % bb_vars['IMAGE_LINK_NAME']) self.remove_config(config)
@@ -1310,7 +1310,7 @@ class Wic2(WicTestCase): # expand image to 1G new_image_path = None with NamedTemporaryFile(mode='wb', suffix='.wic.exp', - dir=deploy_dir, delete=False) as sparse: + dir=bb_vars['DEPLOY_DIR_IMAGE'], delete=False) as sparse: sparse.truncate(1024 ** 3) new_image_path = sparse.name
Signature · 1 line
2.39.2
PATCH 4/14 selftest: wic: respect IMAGE_LINK_NAME also in test_rawcopy_plugin_qemu
In reply to a message not in the archive* this one is more tricky, because the test_rawcopy_plugin.wks.in file is used while building core-image-minimal-mtdutils, but the image filename inside wks.in is from core-image-minimal, so we cannot just let bitbake expand IMAGE_LINK_NAME, use separate variable set in the same config fragment IMAGE_LINK_NAME_CORE_IMAGE_MINIMAL
[YOCTO #12937]
--- meta-selftest/wic/test_rawcopy_plugin.wks.in | 2 +- meta/lib/oeqa/selftest/cases/wic.py | 6 +++++- 2 files changed, 6 insertions(+), 2 deletions(-)
meta-selftest/wic/test_rawcopy_plugin.wks.in +1 -1
@@ -1,6 +1,6 @@ # short-description: This file is used in oe-selftest wic module to test rawcopy plugin part /boot --active --source bootimg-pcbios -part / --source rawcopy --sourceparams="file=core-image-minimal-${MACHINE}.ext4" --use-uuid +part / --source rawcopy --sourceparams="file=${IMAGE_LINK_NAME_CORE_IMAGE_MINIMAL}.ext4" --use-uuid bootloader --timeout=0 --append="console=ttyS0,115200n8"
meta/lib/oeqa/selftest/cases/wic.py +5 -1
@@ -1034,9 +1034,13 @@ class Wic2(WicTestCase): config = 'IMAGE_FSTYPES = "ext4"\n' self.append_config(config) bitbake('core-image-minimal') + image_link_name = get_bb_var('IMAGE_LINK_NAME', 'core-image-minimal') self.remove_config(config) - config = 'IMAGE_FSTYPES = "wic"\nWKS_FILE = "test_rawcopy_plugin.wks.in"\n' + config = 'IMAGE_FSTYPES = "wic"\n' \ + 'IMAGE_LINK_NAME_CORE_IMAGE_MINIMAL = "%s"\n'\ + 'WKS_FILE = "test_rawcopy_plugin.wks.in"\n'\ + % image_link_name self.append_config(config) bitbake('core-image-minimal-mtdutils') self.remove_config(config)
Signature · 1 line
2.39.2
PATCH 5/14 selftest: runqemu: respect IMAGE_LINK_NAME
In reply to a message not in the archive* don't assume that every built image is named:
<recipe>-<machine>.<suffix>
[YOCTO #12937]
--- meta/lib/oeqa/selftest/cases/runqemu.py | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)
meta/lib/oeqa/selftest/cases/runqemu.py +6 -4
@@ -24,6 +24,7 @@ class RunqemuTests(OESelftestTestCase): super(RunqemuTests, self).setUpLocal() self.recipe = 'core-image-minimal' self.machine = self.td['MACHINE'] + self.image_link_name = get_bb_var('IMAGE_LINK_NAME', self.recipe) self.fstypes = "ext4" if self.td["HOST_ARCH"] in ('i586', 'i686', 'x86_64'):
@@ -127,7 +128,7 @@ SYSLINUX_TIMEOUT = "10" def test_boot_qemu_boot(self): """Test runqemu /path/to/image.qemuboot.conf""" - qemuboot_conf = "%s-%s.qemuboot.conf" % (self.recipe, self.machine) + qemuboot_conf = "%s.qemuboot.conf" % (self.image_link_name) qemuboot_conf = os.path.join(self.deploy_dir_image, qemuboot_conf) if not os.path.exists(qemuboot_conf): self.skipTest("%s not found" % qemuboot_conf)
@@ -138,7 +139,7 @@ SYSLINUX_TIMEOUT = "10" def test_boot_rootfs(self): """Test runqemu /path/to/rootfs.ext4""" - rootfs = "%s-%s.ext4" % (self.recipe, self.machine) + rootfs = "%s.ext4" % (self.image_link_name) rootfs = os.path.join(self.deploy_dir_image, rootfs) if not os.path.exists(rootfs): self.skipTest("%s not found" % rootfs)
@@ -167,8 +168,9 @@ class QemuTest(OESelftestTestCase): cls.recipe = 'core-image-minimal' cls.machine = get_bb_var('MACHINE') cls.deploy_dir_image = get_bb_var('DEPLOY_DIR_IMAGE') + cls.image_link_name = get_bb_var('IMAGE_LINK_NAME', cls.recipe) cls.cmd_common = "runqemu nographic" - cls.qemuboot_conf = "%s-%s.qemuboot.conf" % (cls.recipe, cls.machine) + cls.qemuboot_conf = "%s.qemuboot.conf" % (cls.image_link_name) cls.qemuboot_conf = os.path.join(cls.deploy_dir_image, cls.qemuboot_conf) bitbake(cls.recipe)
@@ -200,7 +202,7 @@ class QemuTest(OESelftestTestCase): self.assertTrue(qemu_shutdown_succeeded, 'Failed: %s does not shutdown within timeout(%s)' % (self.machine, shutdown_timeout)) def test_qemu_can_boot_nfs_and_shutdown(self): - rootfs_tar = "%s-%s.tar.bz2" % (self.recipe, self.machine) + rootfs_tar = "%s.tar.bz2" % (self.image_link_name) rootfs_tar = os.path.join(self.deploy_dir_image, rootfs_tar) self.assertExists(rootfs_tar) cmd = "%s %s" % (self.cmd_common, rootfs_tar)
Signature · 1 line
2.39.2
PATCH 6/14 selftest: multiconfig-image-packager: try to respect IMAGE_LINK_NAME
In reply to a message not in the archive* this still assumes that IMAGE_LINK_NAME will contain IMAGE_BASENAME which will be BPN 'multiconfig-image-packager' and that replacing it with 'core-image-minimal' will match with the actual IMAGE_LINK_NAME from core-image-minimal recipe - there is no good way to query core-image-minimal's context, but this is still closer than assuming:
core-image-minimal-${MCMACHINE}.${MCIMGTYPE}
which works only with the current default:
IMAGE_LINK_NAME ?= "${IMAGE_BASENAME}-${MACHINE}"
[YOCTO #12937]
--- .../multiconfig-image-packager_0.1.bb | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-)
meta-selftest/recipes-test/multiconfig/multiconfig-image-packager_0.1.bb +17 -1
@@ -7,15 +7,19 @@ MCIMGTYPE:virtclass-mcextend-musl = "ext4" MCIMGTYPE:virtclass-mcextend-tiny = "cpio.gz" MC_DEPLOY_DIR_IMAGE = "${TOPDIR}/tmp-mc-${MCNAME}/deploy/images/${MCMACHINE}" +MC_DEPLOY_IMAGE_BASENAME = "core-image-minimal" do_install[mcdepends] += "mc::${MCNAME}:core-image-minimal:do_image_complete mc::${MCNAME}:virtual/kernel:do_deploy" do_install () { install -d ${D}/var/lib/machines/${MCNAME} - install ${MC_DEPLOY_DIR_IMAGE}/core-image-minimal-${MCMACHINE}.${MCIMGTYPE} ${D}/var/lib/machines/${MCNAME}/core-image-minimal.${MCIMGTYPE} + install ${MC_DEPLOY_DIR_IMAGE}/${IMAGE_LINK_NAME_CORE_IMAGE_MINIMAL}.${MCIMGTYPE} ${D}/var/lib/machines/${MCNAME}/${MC_DEPLOY_IMAGE_BASENAME}.${MCIMGTYPE} install ${MC_DEPLOY_DIR_IMAGE}/bzImage ${D}/var/lib/machines/${MCNAME} } +# for IMAGE_LINK_NAME, IMAGE_BASENAME +inherit image-artifact-names + python () { mcname = d.getVar('MCNAME') if not mcname:
@@ -23,6 +27,18 @@ python () { multiconfigs = d.getVar('BBMULTICONFIG') or "" if mcname not in multiconfigs: raise bb.parse.SkipRecipe("multiconfig target %s not enabled" % mcname) + + # these will most likely start with my BPN multiconfig-image-packager, but I want them from core-image-minimal + # as there is no good way to query core-image-minimal's context lets assume that there are no overrides + # and that we can just replace IMAGE_BASENAME + image_link_name = d.getVar('IMAGE_LINK_NAME') + image_basename = d.getVar('IMAGE_BASENAME') + machine = d.getVar('MACHINE') + mcmachine = d.getVar('MCMACHINE') + image_to_deploy = d.getVar('MC_DEPLOY_IMAGE_BASENAME') + image_link_name_to_deploy = image_link_name.replace(image_basename, image_to_deploy).replace(machine, mcmachine) + bb.warn('%s: assuming that "%s" built for "%s" has IMAGE_LINK_NAME "%s"' % (d.getVar('PN'), mcmachine, image_to_deploy, image_link_name_to_deploy)) + d.setVar('IMAGE_LINK_NAME_CORE_IMAGE_MINIMAL', image_link_name_to_deploy) } BBCLASSEXTEND = "mcextend:tiny mcextend:musl"
Signature · 1 line
2.39.2
PATCH 7/14 image-artifact-names.bbclass: add INITRAMFS_IMAGE_NAME from kernel.bbclass
In reply to a message not in the archive* move it from kernel.bbclass, because it needs to stay in sync with IMAGE_LINK_NAME structure
* image-artifact-names.bbclass is also inheritted from kernel-artifact-names.bbclass so every recipe which needs this variable probably already inherits one of these
* fixes kernel-fitimage.bbclass with modified IMAGE_LINK_NAME
[YOCTO #12937]
--- meta/classes-recipe/image-artifact-names.bbclass | 3 +++ meta/classes-recipe/kernel.bbclass | 1 - 2 files changed, 3 insertions(+), 1 deletion(-)
meta/classes-recipe/image-artifact-names.bbclass +3 -0
@@ -14,6 +14,9 @@ IMAGE_VERSION_SUFFIX[vardepsexclude] += "DATETIME SOURCE_DATE_EPOCH" IMAGE_NAME ?= "${IMAGE_BASENAME}-${MACHINE}${IMAGE_VERSION_SUFFIX}" IMAGE_LINK_NAME ?= "${IMAGE_BASENAME}-${MACHINE}" +# This needs to stay in sync with IMAGE_LINK_NAME, but with INITRAMFS_IMAGE instead of IMAGE_BASENAME +INITRAMFS_IMAGE_NAME ?= "${@['${INITRAMFS_IMAGE}-${MACHINE}', ''][d.getVar('INITRAMFS_IMAGE') == '']}" + # IMAGE_NAME is the base name for everything produced when building images. # The actual image that contains the rootfs has an additional suffix (.rootfs # by default) followed by additional suffices which describe the format (.ext4,
meta/classes-recipe/kernel.bbclass +0 -1
@@ -33,7 +33,6 @@ INHIBIT_DEFAULT_DEPS = "1" KERNEL_IMAGETYPE ?= "zImage" INITRAMFS_IMAGE ?= "" -INITRAMFS_IMAGE_NAME ?= "${@['${INITRAMFS_IMAGE}-${MACHINE}', ''][d.getVar('INITRAMFS_IMAGE') == '']}" INITRAMFS_TASK ?= "" INITRAMFS_IMAGE_BUNDLE ?= "" INITRAMFS_DEPLOY_DIR_IMAGE ?= "${DEPLOY_DIR_IMAGE}"
Signature · 1 line
2.39.2
PATCH 8/14 selftest: fitimage.py: respect INITRAMFS_IMAGE_NAME and KERNEL_FIT_LINK_NAME
In reply to a message not in the archive* use these variables instead of assuming:
"fitImage-its-%s-%s-%s" % (image_type, machine, machine) notice that these files have no extension and the -machine suffix is duplicated (once from core-image-minimal and once from kernel-fitimage.bbclass:kernel_do_deploy:append through KERNEL_FIT_LINK_NAME which is MACHINE by default:
KERNEL_FIT_NAME ?= "${KERNEL_ARTIFACT_NAME}"
KERNEL_FIT_LINK_NAME ?= "${KERNEL_ARTIFACT_LINK_NAME}"
KERNEL_ARTIFACT_NAME ?= "${PKGE}-${PKGV}-${PKGR}-${MACHINE}${IMAGE_VERSION_SUFFIX}"
KERNEL_ARTIFACT_LINK_NAME ?= "${MACHINE}"[YOCTO #12937]
--- meta/lib/oeqa/selftest/cases/fitimage.py | 34 +++++++++++------------- 1 file changed, 16 insertions(+), 18 deletions(-)
meta/lib/oeqa/selftest/cases/fitimage.py +16 -18
@@ -5,7 +5,7 @@ # from oeqa.selftest.case import OESelftestTestCase -from oeqa.utils.commands import runCmd, bitbake, get_bb_var +from oeqa.utils.commands import runCmd, bitbake, get_bb_var, get_bb_vars import os import re
@@ -42,15 +42,14 @@ FIT_DESC = "A model description" self.write_config(config) # fitImage is created as part of linux recipe - bitbake("virtual/kernel") + image = "virtual/kernel" + bitbake(image) + bb_vars = get_bb_vars(['DEPLOY_DIR_IMAGE', 'INITRAMFS_IMAGE_NAME', 'KERNEL_FIT_LINK_NAME'], image) - image_type = "core-image-minimal" - deploy_dir_image = get_bb_var('DEPLOY_DIR_IMAGE') - machine = get_bb_var('MACHINE') - fitimage_its_path = os.path.join(deploy_dir_image, - "fitImage-its-%s-%s-%s" % (image_type, machine, machine)) - fitimage_path = os.path.join(deploy_dir_image, - "fitImage-%s-%s-%s" % (image_type, machine, machine)) + fitimage_its_path = os.path.join(bb_vars['DEPLOY_DIR_IMAGE'], + "fitImage-its-%s-%s" % (bb_vars['INITRAMFS_IMAGE_NAME'], bb_vars['KERNEL_FIT_LINK_NAME'])) + fitimage_path = os.path.join(bb_vars['DEPLOY_DIR_IMAGE'], + "fitImage-%s-%s" % (bb_vars['INITRAMFS_IMAGE_NAME'], bb_vars['KERNEL_FIT_LINK_NAME'])) self.assertTrue(os.path.exists(fitimage_its_path), "%s image tree source doesn't exist" % (fitimage_its_path))
@@ -123,15 +122,14 @@ UBOOT_MKIMAGE_SIGN_ARGS = "-c 'a smart comment'" self.write_config(config) # fitImage is created as part of linux recipe - bitbake("virtual/kernel") - - image_type = "core-image-minimal" - deploy_dir_image = get_bb_var('DEPLOY_DIR_IMAGE') - machine = get_bb_var('MACHINE') - fitimage_its_path = os.path.join(deploy_dir_image, - "fitImage-its-%s" % (machine,)) - fitimage_path = os.path.join(deploy_dir_image, - "fitImage-%s.bin" % (machine,)) + image = "virtual/kernel" + bitbake(image) + bb_vars = get_bb_vars(['DEPLOY_DIR_IMAGE', 'KERNEL_FIT_LINK_NAME'], image) + + fitimage_its_path = os.path.join(bb_vars['DEPLOY_DIR_IMAGE'], + "fitImage-its-%s" % (bb_vars['KERNEL_FIT_LINK_NAME'])) + fitimage_path = os.path.join(bb_vars['DEPLOY_DIR_IMAGE'], + "fitImage-%s.bin" % (bb_vars['KERNEL_FIT_LINK_NAME'])) self.assertTrue(os.path.exists(fitimage_its_path), "%s image tree source doesn't exist" % (fitimage_its_path))
Signature · 1 line
2.39.2
PATCH 9/14 image-artifact-names: add IMAGE_MACHINE_SUFFIX variable
In reply to a message not in the archive* to make it easier for projects to avoid default -${MACHINE} suffix if the ${MACHINE} named DEPLOY_DIR_IMAGE works better for them
* also use IMAGE_LINK_NAME in IMAGE_NAME to make it more clear that IMAGE_NAME is the same as IMAGE_LINK_NAME but with version suffix
* adding it as separate variable helps us to catch the cases where we didn't respect ${IMAGE_LINK_NAME} variable and just used the common default ${IMAGE_BASENAME}-${MACHINE}.
[YOCTO #12937]
--- meta/classes-recipe/image-artifact-names.bbclass | 15 ++++++++++++--- meta/classes-recipe/kernel-artifact-names.bbclass | 2 +- 2 files changed, 13 insertions(+), 4 deletions(-)
meta/classes-recipe/image-artifact-names.bbclass +12 -3
@@ -11,11 +11,20 @@ IMAGE_BASENAME ?= "${PN}" IMAGE_VERSION_SUFFIX ?= "-${DATETIME}" IMAGE_VERSION_SUFFIX[vardepsexclude] += "DATETIME SOURCE_DATE_EPOCH" -IMAGE_NAME ?= "${IMAGE_BASENAME}-${MACHINE}${IMAGE_VERSION_SUFFIX}" -IMAGE_LINK_NAME ?= "${IMAGE_BASENAME}-${MACHINE}" +IMAGE_NAME ?= "${IMAGE_LINK_NAME}${IMAGE_VERSION_SUFFIX}" +IMAGE_LINK_NAME ?= "${IMAGE_BASENAME}${IMAGE_MACHINE_SUFFIX}" # This needs to stay in sync with IMAGE_LINK_NAME, but with INITRAMFS_IMAGE instead of IMAGE_BASENAME -INITRAMFS_IMAGE_NAME ?= "${@['${INITRAMFS_IMAGE}-${MACHINE}', ''][d.getVar('INITRAMFS_IMAGE') == '']}" +INITRAMFS_IMAGE_NAME ?= "${@['${INITRAMFS_IMAGE}${IMAGE_MACHINE_SUFFIX}', ''][d.getVar('INITRAMFS_IMAGE') == '']}" + +# The default DEPLOY_DIR_IMAGE is ${MACHINE} directory: +# meta/conf/bitbake.conf:DEPLOY_DIR_IMAGE ?= "${DEPLOY_DIR}/images/${MACHINE}" +# so many people find it unnecessary to include this suffix to every image +# stored there, but other people often fetch various images for different +# MACHINEs to the same downloads directory and then the suffix is very helpful +# add separate variable for projects to decide which scheme works best for them +# without understanding the IMAGE_NAME/IMAGE_LINK_NAME structure. +IMAGE_MACHINE_SUFFIX ??= "-${MACHINE}" # IMAGE_NAME is the base name for everything produced when building images. # The actual image that contains the rootfs has an additional suffix (.rootfs
meta/classes-recipe/kernel-artifact-names.bbclass +1 -1
@@ -12,7 +12,7 @@ inherit image-artifact-names -KERNEL_ARTIFACT_NAME ?= "${PKGE}-${PKGV}-${PKGR}-${MACHINE}${IMAGE_VERSION_SUFFIX}" +KERNEL_ARTIFACT_NAME ?= "${PKGE}-${PKGV}-${PKGR}${IMAGE_MACHINE_SUFFIX}${IMAGE_VERSION_SUFFIX}" KERNEL_ARTIFACT_LINK_NAME ?= "${MACHINE}" KERNEL_ARTIFACT_BIN_EXT ?= ".bin"
Signature · 1 line
2.39.2
PATCH 10/14 selftest: gdbserver.py: respect IMAGE_LINK_NAME
In reply to a message not in the archive* use IMAGE_LINK_NAME instead of hardcoding
core-image-minimal-${MACHINE} assumption
[YOCTO #12937]
--- meta/lib/oeqa/selftest/cases/gdbserver.py | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-)
meta/lib/oeqa/selftest/cases/gdbserver.py +5 -4
@@ -10,13 +10,12 @@ import shutil import concurrent.futures from oeqa.selftest.case import OESelftestTestCase -from oeqa.utils.commands import bitbake, get_bb_var, runqemu, runCmd +from oeqa.utils.commands import bitbake, get_bb_var, get_bb_vars , runqemu, runCmd class GdbServerTest(OESelftestTestCase): def test_gdb_server(self): target_arch = self.td["TARGET_ARCH"] target_sys = self.td["TARGET_SYS"] - deploy_dir = get_bb_var("DEPLOY_DIR_IMAGE") features = """ IMAGE_GEN_DEBUGFS = "1"
@@ -34,11 +33,13 @@ CORE_IMAGE_EXTRA_INSTALL = "gdbserver" r = runCmd("%s --version" % gdb_binary, native_sysroot=native_sysroot, target_sys=target_sys) self.assertEqual(r.status, 0) self.assertIn("GNU gdb", r.output) + image = 'core-image-minimal' + bb_vars = get_bb_vars(['DEPLOY_DIR_IMAGE', 'IMAGE_LINK_NAME'], image) with tempfile.TemporaryDirectory(prefix="debugfs-") as debugfs: - filename = os.path.join(deploy_dir, "core-image-minimal-%s-dbg.tar.bz2" % self.td["MACHINE"]) + filename = os.path.join(bb_vars['DEPLOY_DIR_IMAGE'], "%s-dbg.tar.bz2" % bb_vars['IMAGE_LINK_NAME']) shutil.unpack_archive(filename, debugfs) - filename = os.path.join(deploy_dir, "core-image-minimal-%s.tar.bz2" % self.td["MACHINE"]) + filename = os.path.join(bb_vars['DEPLOY_DIR_IMAGE'], "%s.tar.bz2" % bb_vars['IMAGE_LINK_NAME']) shutil.unpack_archive(filename, debugfs) with runqemu("core-image-minimal", runqemuparams="nographic") as qemu:
Signature · 1 line
2.39.2
PATCH 11/14 selftest: minidebuginfo.py respect IMAGE_LINK_NAME
In reply to a message not in the archive* use IMAGE_LINK_NAME instead of hardcoding
core-image-minimal-${MACHINE} assumption
[YOCTO #12937]
--- meta/lib/oeqa/selftest/cases/minidebuginfo.py | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-)
meta/lib/oeqa/selftest/cases/minidebuginfo.py +8 -7
@@ -9,7 +9,7 @@ import tempfile import shutil from oeqa.selftest.case import OESelftestTestCase -from oeqa.utils.commands import bitbake, get_bb_var, runCmd +from oeqa.utils.commands import bitbake, get_bb_var, get_bb_vars, runCmd class Minidebuginfo(OESelftestTestCase):
@@ -17,27 +17,28 @@ class Minidebuginfo(OESelftestTestCase): target_sys = get_bb_var("TARGET_SYS") binutils = "binutils-cross-{}".format(get_bb_var("TARGET_ARCH")) + image = 'core-image-minimal' + bb_vars = get_bb_vars(['DEPLOY_DIR_IMAGE', 'IMAGE_LINK_NAME', 'READELF'], image) + self.write_config(""" PACKAGE_MINIDEBUGINFO = "1" IMAGE_FSTYPES = "tar.bz2" """) - bitbake("core-image-minimal {}:do_addto_recipe_sysroot".format(binutils)) + bitbake("{} {}:do_addto_recipe_sysroot".format(image, binutils)) - deploy_dir = get_bb_var("DEPLOY_DIR_IMAGE") native_sysroot = get_bb_var("RECIPE_SYSROOT_NATIVE", binutils) - readelf = get_bb_var("READELF", "core-image-minimal") # confirm that executables and shared libraries contain an ELF section # ".gnu_debugdata" which stores minidebuginfo. with tempfile.TemporaryDirectory(prefix = "unpackfs-") as unpackedfs: - filename = os.path.join(deploy_dir, "core-image-minimal-{}.tar.bz2".format(self.td["MACHINE"])) + filename = os.path.join(bb_vars['DEPLOY_DIR_IMAGE'], "{}.tar.bz2".format(bb_vars['IMAGE_LINK_NAME'])) shutil.unpack_archive(filename, unpackedfs) - r = runCmd([readelf, "-W", "-S", os.path.join(unpackedfs, "bin", "busybox")], + r = runCmd([bb_vars['READELF'], "-W", "-S", os.path.join(unpackedfs, "bin", "busybox")], native_sysroot = native_sysroot, target_sys = target_sys) self.assertIn(".gnu_debugdata", r.output) - r = runCmd([readelf, "-W", "-S", os.path.join(unpackedfs, "lib", "libc.so.6")], + r = runCmd([bb_vars['READELF'], "-W", "-S", os.path.join(unpackedfs, "lib", "libc.so.6")], native_sysroot = native_sysroot, target_sys = target_sys) self.assertIn(".gnu_debugdata", r.output)
Signature · 1 line
2.39.2
PATCH 12/14 selftest: efibootpartition.py: fix QEMU_USE_KVM usage
In reply to a message not in the archive* it's not in self.td causing:
2023-03-12 18:06:29,591 - oe-selftest - DEBUG - Checking if qemux86-64 is not this MACHINE
2023-03-12 18:06:29,594 - oe-selftest - INFO - ... ERROR
2023-03-12 18:06:29,594 - oe-selftest - INFO - Traceback (most recent call last):
File "/OE/build/poky/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/OE/build/poky/meta/lib/oeqa/selftest/cases/efibootpartition.py", line 18, in test_boot_efi
if oe.types.qemu_use_kvm(self.td['QEMU_USE_KVM'], self.td["TARGET_ARCH"]):
~~~~~~~^^^^^^^^^^^^^^^^
KeyError: 'QEMU_USE_KVM'[YOCTO #12937]
--- meta/lib/oeqa/selftest/cases/efibootpartition.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
meta/lib/oeqa/selftest/cases/efibootpartition.py +2 -2
@@ -6,7 +6,7 @@ # from oeqa.selftest.case import OESelftestTestCase -from oeqa.utils.commands import bitbake, runqemu +from oeqa.utils.commands import bitbake, get_bb_var, runqemu from oeqa.core.decorator.data import skipIfNotMachine import oe.types
@@ -15,7 +15,7 @@ class GenericEFITest(OESelftestTestCase): @skipIfNotMachine("qemux86-64", "test is qemux86-64 specific currently") def test_boot_efi(self): cmd = "runqemu nographic serial wic ovmf" - if oe.types.qemu_use_kvm(self.td['QEMU_USE_KVM'], self.td["TARGET_ARCH"]): + if oe.types.qemu_use_kvm(get_bb_var('QEMU_USE_KVM'), self.td["TARGET_ARCH"]): cmd += " kvm" image = "core-image-minimal"
Signature · 1 line
2.39.2
PATCH 13/14 runqemu: get_first_file() rename cmd* to glob*
In reply to a message not in the archive* to better indicate how it's used in get_first_file
* cmd* is used in other places for actual shell commands to execute
* RunQemuError('KERNEL not found: %s, %s or %s' % cmds) also looked weird to me, but that works (to my python-noob surprise)
[YOCTO #12937]
--- scripts/runqemu | 36 ++++++++++++++++++------------------ 1 file changed, 18 insertions(+), 18 deletions(-)
scripts/runqemu +18 -18
@@ -116,10 +116,10 @@ def check_tun(): if not os.access(dev_tun, os.W_OK): raise RunQemuError("TUN control device %s is not writable, please fix (e.g. sudo chmod 666 %s)" % (dev_tun, dev_tun)) -def get_first_file(cmds): - """Return first file found in wildcard cmds""" - for cmd in cmds: - all_files = glob.glob(cmd) +def get_first_file(globs): + """Return first file found in wildcard globs""" + for g in globs: + all_files = glob.glob(g) if all_files: for f in all_files: if not os.path.isdir(f):
@@ -676,12 +676,12 @@ class BaseConfig(object): self.rootfs, self.get('MACHINE'), self.fstype) elif not self.rootfs: - cmd_name = '%s/%s*.%s' % (self.get('DEPLOY_DIR_IMAGE'), self.get('IMAGE_NAME'), self.fstype) - cmd_link = '%s/%s*.%s' % (self.get('DEPLOY_DIR_IMAGE'), self.get('IMAGE_LINK_NAME'), self.fstype) - cmds = (cmd_name, cmd_link) - self.rootfs = get_first_file(cmds) + glob_name = '%s/%s*.%s' % (self.get('DEPLOY_DIR_IMAGE'), self.get('IMAGE_NAME'), self.fstype) + glob_link = '%s/%s*.%s' % (self.get('DEPLOY_DIR_IMAGE'), self.get('IMAGE_LINK_NAME'), self.fstype) + globs = (glob_name, glob_link) + self.rootfs = get_first_file(globs) if not self.rootfs: - raise RunQemuError("Failed to find rootfs: %s or %s" % cmds) + raise RunQemuError("Failed to find rootfs: %s or %s" % globs) if not os.path.exists(self.rootfs): raise RunQemuError("Can't find rootfs: %s" % self.rootfs)
@@ -741,10 +741,10 @@ class BaseConfig(object): kernel_match_name = "%s/%s" % (deploy_dir_image, kernel_name) kernel_match_link = "%s/%s" % (deploy_dir_image, self.get('KERNEL_IMAGETYPE')) kernel_startswith = "%s/%s*" % (deploy_dir_image, self.get('KERNEL_IMAGETYPE')) - cmds = (kernel_match_name, kernel_match_link, kernel_startswith) - self.kernel = get_first_file(cmds) + globs = (kernel_match_name, kernel_match_link, kernel_startswith) + self.kernel = get_first_file(globs) if not self.kernel: - raise RunQemuError('KERNEL not found: %s, %s or %s' % cmds) + raise RunQemuError('KERNEL not found: %s, %s or %s' % globs) if not os.path.exists(self.kernel): raise RunQemuError("KERNEL %s not found" % self.kernel)
@@ -761,13 +761,13 @@ class BaseConfig(object): dtb = self.get('QB_DTB') if dtb: deploy_dir_image = self.get('DEPLOY_DIR_IMAGE') - cmd_match = "%s/%s" % (deploy_dir_image, dtb) - cmd_startswith = "%s/%s*" % (deploy_dir_image, dtb) - cmd_wild = "%s/*.dtb" % deploy_dir_image - cmds = (cmd_match, cmd_startswith, cmd_wild) - self.dtb = get_first_file(cmds) + glob_match = "%s/%s" % (deploy_dir_image, dtb) + glob_startswith = "%s/%s*" % (deploy_dir_image, dtb) + glob_wild = "%s/*.dtb" % deploy_dir_image + globs = (glob_match, glob_startswith, glob_wild) + self.dtb = get_first_file(globs) if not os.path.exists(self.dtb): - raise RunQemuError('DTB not found: %s, %s or %s' % cmds) + raise RunQemuError('DTB not found: %s, %s or %s' % globs) def check_bios(self): """Check and set bios"""
Signature · 1 line
2.39.2
PATCH 14/14 selftest: imagefeatures.py: respect IMAGE_LINK_NAME for debugfs and manifest as well
In reply to a message not in the archive* these cases were correctly respecting IMAGE_LINK_NAME in most tests
* the only exception was relatively wide glob for manifest:
"test-empty-image-*.manifest"
* and even wider glob for -dbg:
"*-dbg.rootfs.tar.bz2"
* replace them with the exact filename we expect for given image
* be aware that gzip won't accept the symlink in IMAGE_LINK_NAME causing:
2023-03-13 08:58:23,845 - oe-selftest - INFO - ... FAIL
2023-03-13 08:58:23,845 - oe-selftest - INFO - Traceback (most recent call last):
File "/OE/build/poky/meta/lib/oeqa/selftest/cases/imagefeatures.py", line 124, in test_bmap
self.assertTrue(runCmd('gzip -t %s' % gzip_path))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/OE/build/poky/meta/lib/oeqa/utils/commands.py", line 214, in runCmd
raise AssertionError("Command '%s' returned non-zero exit status %d:\n%s" % (command, result.status, exc_output))
AssertionError: Command 'gzip -t /OE/build/poky/build/build-st-2023-03-12-todo-patch2/imagefeatures.ImageFeatures.test_bmap/build-st/tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.rootfs.ext4.bmap.gz' returned non-zero exit status 1:
gzip: skipping: /OE/build/poky/build/build-st-2023-03-12-todo-patch2/imagefeatures.ImageFeatures.test_bmap/build-st/tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.rootfs.ext4.bmap.gz is a symbolic link* and "*-dbg.rootfs.tar.bz2" doesn't work if IMAGE_NAME_SUFFIX is changed to anything else than ".rootfs" or moved into IMAGE_LINK_NAME (like I plan in future changes where this will need to be updated again).
Luckily we can use the symlink which currently doesn't have IMAGE_NAME_SUFFIX nor the DATETIME, so we don't need to search for it with glob, e.g. currently:
core-image-minimal-qemux86-64-dbg-20230313112546-dbg.rootfs.tar.bz2 core-image-minimal-qemux86-64-dbg.tar.bz2 -> core-image-minimal-qemux86-64-dbg-20230313112546-dbg.rootfs.tar.bz2
[YOCTO #12937]
--- meta/lib/oeqa/selftest/cases/imagefeatures.py | 82 +++++++++---------- 1 file changed, 40 insertions(+), 42 deletions(-)
meta/lib/oeqa/selftest/cases/imagefeatures.py +40 -42
@@ -6,7 +6,7 @@ from oeqa.selftest.case import OESelftestTestCase from oeqa.core.decorator import OETestTag -from oeqa.utils.commands import runCmd, bitbake, get_bb_var, runqemu +from oeqa.utils.commands import runCmd, bitbake, get_bb_var, get_bb_vars, runqemu from oeqa.utils.sshcontrol import SSHControl import glob import os
@@ -102,12 +102,11 @@ class ImageFeatures(OESelftestTestCase): features = 'IMAGE_FSTYPES += " ext4 ext4.bmap ext4.bmap.gz"' self.write_config(features) - image_name = 'core-image-minimal' - bitbake(image_name) + image = 'core-image-minimal' + bitbake(image) + bb_vars = get_bb_vars(['DEPLOY_DIR_IMAGE', 'IMAGE_LINK_NAME'], image) - deploy_dir_image = get_bb_var('DEPLOY_DIR_IMAGE') - link_name = get_bb_var('IMAGE_LINK_NAME', image_name) - image_path = os.path.join(deploy_dir_image, "%s.ext4" % link_name) + image_path = os.path.join(bb_vars['DEPLOY_DIR_IMAGE'], "%s.ext4" % bb_vars['IMAGE_LINK_NAME']) bmap_path = "%s.bmap" % image_path gzip_path = "%s.gz" % bmap_path
@@ -120,8 +119,8 @@ class ImageFeatures(OESelftestTestCase): image_stat = os.stat(image_path) self.assertGreater(image_stat.st_size, image_stat.st_blocks * 512) - # check if the resulting gzip is valid - self.assertTrue(runCmd('gzip -t %s' % gzip_path)) + # check if the resulting gzip is valid, --force is needed in case gzip_path is a symlink + self.assertTrue(runCmd('gzip --test --force %s' % gzip_path)) def test_hypervisor_fmts(self): """
@@ -139,14 +138,13 @@ class ImageFeatures(OESelftestTestCase): features += 'IMAGE_FSTYPES += "ext4.%s"\n' % itype self.write_config(features) - image_name = 'core-image-minimal' - bitbake(image_name) + image = 'core-image-minimal' + bitbake(image) + bb_vars = get_bb_vars(['DEPLOY_DIR_IMAGE', 'IMAGE_LINK_NAME'], image) - deploy_dir_image = get_bb_var('DEPLOY_DIR_IMAGE') - link_name = get_bb_var('IMAGE_LINK_NAME', image_name) for itype in img_types: - image_path = os.path.join(deploy_dir_image, "%s.ext4.%s" % - (link_name, itype)) + image_path = os.path.join(bb_vars['DEPLOY_DIR_IMAGE'], "%s.ext4.%s" % + (bb_vars['IMAGE_LINK_NAME'], itype)) # check if result image file is in deploy directory self.assertTrue(os.path.exists(image_path))
@@ -177,13 +175,11 @@ class ImageFeatures(OESelftestTestCase): features = 'IMAGE_FSTYPES += "%s %s.sha256sum"' % (conv, conv) self.write_config(features) - image_name = 'core-image-minimal' - bitbake(image_name) - - deploy_dir_image = get_bb_var('DEPLOY_DIR_IMAGE') - link_name = get_bb_var('IMAGE_LINK_NAME', image_name) - image_path = os.path.join(deploy_dir_image, "%s.%s" % - (link_name, conv)) + image = 'core-image-minimal' + bitbake(image) + bb_vars = get_bb_vars(['DEPLOY_DIR_IMAGE', 'IMAGE_LINK_NAME'], image) + image_path = os.path.join(bb_vars['DEPLOY_DIR_IMAGE'], "%s.%s" % + (bb_vars['IMAGE_LINK_NAME'], conv)) # check if resulting image is in the deploy directory self.assertTrue(os.path.exists(image_path))
@@ -191,7 +187,7 @@ class ImageFeatures(OESelftestTestCase): # check if the resulting sha256sum agrees self.assertTrue(runCmd('cd %s;sha256sum -c %s.%s.sha256sum' % - (deploy_dir_image, link_name, conv))) + (bb_vars['DEPLOY_DIR_IMAGE'], bb_vars['IMAGE_LINK_NAME'], conv))) def test_image_fstypes(self): """
@@ -200,9 +196,9 @@ class ImageFeatures(OESelftestTestCase): Product: oe-core Author: Ed Bartosh <[email protected]> """ - image_name = 'core-image-minimal' + image = 'core-image-minimal' - all_image_types = set(get_bb_var("IMAGE_TYPES", image_name).split()) + all_image_types = set(get_bb_var("IMAGE_TYPES", image).split()) skip_image_types = set(('container', 'elf', 'f2fs', 'multiubi', 'tar.zst', 'wic.zst', 'squashfs-lzo')) img_types = all_image_types - skip_image_types
@@ -214,12 +210,11 @@ UBINIZE_ARGS ?= "-m 2048 -p 128KiB -s 512" """ % ' '.join(img_types) self.write_config(config) - bitbake(image_name) + bitbake(image) + bb_vars = get_bb_vars(['DEPLOY_DIR_IMAGE', 'IMAGE_LINK_NAME'], image) - deploy_dir_image = get_bb_var('DEPLOY_DIR_IMAGE') - link_name = get_bb_var('IMAGE_LINK_NAME', image_name) for itype in img_types: - image_path = os.path.join(deploy_dir_image, "%s.%s" % (link_name, itype)) + image_path = os.path.join(bb_vars['DEPLOY_DIR_IMAGE'], "%s.%s" % (bb_vars['IMAGE_LINK_NAME'], itype)) # check if result image is in deploy directory self.assertTrue(os.path.exists(image_path), "%s image %s doesn't exist" % (itype, image_path))
@@ -271,19 +266,20 @@ SKIP_RECIPE[busybox] = "Don't build this" Yeoh Ee Peng <[email protected]> """ - image_name = 'core-image-minimal' + image = 'core-image-minimal' + image_fstypes_debugfs = 'tar.bz2' features = 'IMAGE_GEN_DEBUGFS = "1"\n' - features += 'IMAGE_FSTYPES_DEBUGFS = "tar.bz2"\n' + features += 'IMAGE_FSTYPES_DEBUGFS = "%s"\n' % image_fstypes_debugfs self.write_config(features) - bitbake(image_name) - deploy_dir_image = get_bb_var('DEPLOY_DIR_IMAGE') - dbg_tar_file = os.path.join(deploy_dir_image, "*-dbg.rootfs.tar.bz2") - debug_files = glob.glob(dbg_tar_file) - self.assertNotEqual(len(debug_files), 0, 'debug filesystem not generated at %s' % dbg_tar_file) - result = runCmd('cd %s; tar xvf %s' % (deploy_dir_image, dbg_tar_file)) + bitbake(image) + bb_vars = get_bb_vars(['DEPLOY_DIR_IMAGE', 'IMAGE_LINK_NAME'], image) + + dbg_tar_file = os.path.join(bb_vars['DEPLOY_DIR_IMAGE'], "%s-dbg.%s" % (bb_vars['IMAGE_LINK_NAME'], image_fstypes_debugfs)) + self.assertTrue(os.path.exists(dbg_tar_file), 'debug filesystem not generated at %s' % dbg_tar_file) + result = runCmd('cd %s; tar xvf %s' % (bb_vars['DEPLOY_DIR_IMAGE'], dbg_tar_file)) self.assertEqual(result.status, 0, msg='Failed to extract %s: %s' % (dbg_tar_file, result.output)) - result = runCmd('find %s -name %s' % (deploy_dir_image, "udevadm")) + result = runCmd('find %s -name %s' % (bb_vars['DEPLOY_DIR_IMAGE'], "udevadm")) self.assertTrue("udevadm" in result.output, msg='Failed to find udevadm: %s' % result.output) dbg_symbols_targets = result.output.splitlines() self.assertTrue(dbg_symbols_targets, msg='Failed to split udevadm: %s' % dbg_symbols_targets)
@@ -293,11 +289,13 @@ SKIP_RECIPE[busybox] = "Don't build this" def test_empty_image(self): """Test creation of image with no packages""" - bitbake('test-empty-image') - res_dir = get_bb_var('DEPLOY_DIR_IMAGE') - images = os.path.join(res_dir, "test-empty-image-*.manifest") - result = glob.glob(images) - with open(result[1],"r") as f: + image = 'test-empty-image' + bitbake(image) + bb_vars = get_bb_vars(['DEPLOY_DIR_IMAGE', 'IMAGE_LINK_NAME'], image) + manifest = os.path.join(bb_vars['DEPLOY_DIR_IMAGE'], "%s.manifest" % bb_vars['IMAGE_LINK_NAME']) + self.assertTrue(os.path.exists(manifest)) + + with open(manifest, "r") as f: self.assertEqual(len(f.read().strip()),0) def test_mandb(self):
Signature · 1 line
2.39.2
Re: [OE-core] [PATCH 12/14] selftest: efibootpartition.py: fix QEMU_USE_KVM usage
In reply to a message not in the archiveQuoted 51 lines · Martin Jansa via lists.openembedded.org
* it's not in self.td causing:
2023-03-12 18:06:29,591 - oe-selftest - DEBUG - Checking if qemux86-64 is not this MACHINE 2023-03-12 18:06:29,594 - oe-selftest - INFO - ... ERROR 2023-03-12 18:06:29,594 - oe-selftest - INFO - Traceback (most recentcall last):
File "/OE/build/poky/meta/lib/oeqa/core/decorator/__init__.py", line
35, inwrapped_f
returnfunc(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File
"/OE/build/poky/meta/lib/oeqa/selftest/cases/efibootpartition.py", line 18, intest_boot_efi
if oe.types.qemu_use_kvm(self.td['QEMU_USE_KVM'], self.td
["TARGET_ARCH"]):~~~~~~~^^^^^^^^^^^^^^^^KeyError: '
QEMU_USE_KVM'[YOCTO #12937]
Signed-off-by Martin Jansa <[email protected]>--- meta/lib/oeqa/selftest/cases/efibootpartition.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)meta/lib/oeqa/selftest/cases/efibootpartition.py +1
-2@@ -6,7 +6,7 @@ # from oeqa.selftest.case import OESelftestTestCase -from oeqa.utils.commands import bitbake, runqemu +from oeqa.utils.commands import bitbake, get_bb_var, runqemu from oeqa.core.decorator.data import skipIfNotMachine import oe.types@@ -15,7 +15,7 @@ class GenericEFITest(OESelftestTestCase): @skipIfNotMachine("qemux86-64", "test is qemux86-64 specific currently") def test_boot_efi(self): cmd = "runqemu nographic serial wic ovmf" - if oe.types.qemu_use_kvm(self.td['QEMU_USE_KVM'], self.td["
TARGET_ARCH"]):
+ if oe.types.qemu_use_kvm(get_bb_var('QEMU_USE_KVM'), self.td
["TARGET_ARCH"]):
cmd += " kvm"
image = "core-image-minimal"
Quoted 3 lines
--
2.39.2
This is still reproducible with latest poky master:0e5bdb623b0 xdg-utils: Fix CVE number
with just these 5 lines in local.conf:
BB_NUMBER_THREADS = "8"BB_NUMBER_PARSE_THREADS = "64"PARALLEL_MAKE = "-j 70 -l 140"DL_DIR = "/OE/build/downloads"SANITY_TESTED_DISTROS = ""
2023-04-05 17:00:33,452 - oe-selftest - WARNING - meta-selftest layer not found in BBLAYERS, adding it 2023-04-05 17:01:03,686 - oe-selftest - INFO - Adding layer libraries: 2023-04-05 17:01:03,686 - oe-selftest - INFO - /OE/build/poky/meta-poky/lib 2023-04-05 17:01:03,686 - oe-selftest - INFO - /OE/build/poky/meta/lib 2023-04-05 17:01:03,686 - oe-selftest - INFO - /OE/build/poky/meta-yocto-bsp/lib 2023-04-05 17:01:03,687 - oe-selftest - INFO - /OE/build/poky/meta-selftest/lib 2023-04-05 17:01:03,689 - oe-selftest - INFO - Running bitbake -e to test the configuration is valid/parsable 2023-04-05 17:01:18,699 - oe-selftest - INFO - Adding: "include selftest.inc" in /OE/build/poky/build-st/conf/local.conf 2023-04-05 17:01:18,699 - oe-selftest - INFO - Adding: "include bblayers.inc" in bblayers.conf 2023-04-05 17:01:18,699 - oe-selftest - INFO - test_boot_efi (efibootpartition.GenericEFITest.test_boot_efi) 2023-04-05 17:01:18,700 - oe-selftest - DEBUG - Checking if qemux86-64 is not this MACHINE 2023-04-05 17:01:18,707 - oe-selftest - INFO - ... ERROR 2023-04-05 17:01:18,708 - oe-selftest - INFO - Traceback (most recent call
last):
File "/OE/build/poky/meta/lib/oeqa/core/decorator/__init__.py", line 35,
in wrapped_f
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/OE/build/poky/meta/lib/oeqa/selftest/cases/efibootpartition.py",
line 18, in test_boot_efi
if oe.types.qemu_use_kvm(self.td['QEMU_USE_KVM'], self.td
["TARGET_ARCH"]):
~~~~~~~^^^^^^^^^^^^^^^^
KeyError: 'QEMU_USE_KVM'
2023-04-05 17:01:18,709 - oe-selftest - INFO - ====================================================================== 2023-04-05 17:01:18,709 - oe-selftest - INFO - ERROR: test_boot_efi (efibootpartition.GenericEFITest.test_boot_efi) 2023-04-05 17:01:18,709 - oe-selftest - INFO - ---------------------------------------------------------------------- 2023-04-05 17:01:18,709 - oe-selftest - INFO - Traceback (most recent call
last):
File "/OE/build/poky/meta/lib/oeqa/core/decorator/__init__.py", line 35,
in wrapped_f
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/OE/build/poky/meta/lib/oeqa/selftest/cases/efibootpartition.py",
line 18, in test_boot_efi
if oe.types.qemu_use_kvm(self.td['QEMU_USE_KVM'], self.td
["TARGET_ARCH"]):
~~~~~~~^^^^^^^^^^^^^^^^
KeyError: 'QEMU_USE_KVM'
2023-04-05 17:01:18,710 - oe-selftest - INFO - ---------------------------------------------------------------------- 2023-04-05 17:01:18,710 - oe-selftest - INFO - Ran 1 test in 7.873s 2023-04-05 17:01:18,710 - oe-selftest - INFO - FAILED 2023-04-05 17:01:18,710 - oe-selftest - INFO - (errors=1) 2023-04-05 17:01:28,486 - oe-selftest - INFO - RESULTS: 2023-04-05 17:01:28,487 - oe-selftest - INFO - RESULTS - efibootpartition.GenericEFITest.test_boot_efi: ERROR (0.01s) 2023-04-05 17:01:28,488 - oe-selftest - INFO - SUMMARY: 2023-04-05 17:01:28,488 - oe-selftest - INFO - oe-selftest () - Ran 1 test in 7.875s 2023-04-05 17:01:28,488 - oe-selftest - INFO - oe-selftest - FAIL -
Required tests failed (successes=0, skipped=0, failures=0, errors=1)
I was just following what runqemu selftest already does:meta/lib/oeqa/selftest/cases/runqemu.py: kvm =oe.types.qemu_use_kvm(get_bb_var('QEMU_USE_KVM'), self.td["TARGET_ARCH"])
I've discussed this with Ross (who introduced this in
https://git.openembedded.org/openembedded-core-contrib/commit/?h=jansa/pull&id=2bc2ee171f976807053b7da44c1eedbb07c10949)
and RP on IRC and also verified that adding:
QEMU_USE_KVM = "1"
to local.conf is possible work around, but still this fix is better as selftest should work without any extra local.conf modifications.
Cheers,
Hi Martin
Quoted 25 lines · On Tuesday, 14 March 2023 01:15:36 NZST Martin Jansa wrote:
* to make it easier for projects to avoid default -${MACHINE} suffix if the ${MACHINE} named
DEPLOY_DIR_IMAGEworks better for them* also use
IMAGE_LINK_NAMEinIMAGE_NAMEto make it more clear thatIMAGE_NAMEis the same asIMAGE_LINK_NAMEbut with version suffix* adding it as separate variable helps us to catch the cases where we didn't respect ${IMAGE_LINK_NAME} variable and just used the common default ${IMAGE_BASENAME}-${MACHINE}.
[YOCTO #12937]
Signed-off-by Martin Jansa <[email protected]>--- meta/classes-recipe/image-artifact-names.bbclass | 15 ++++++++++++--- meta/classes-recipe/kernel-artifact-names.bbclass | 2 +- 2 files changed, 13 insertions(+), 4 deletions(-)(quoted diff) +0
-0b/meta/classes-recipe/image-artifact-names.bbclass index
9dc25b6dde..ac2376d59a100644
meta/classes-recipe/image-artifact-names.bbclass line 15
Quoted 7 lines
meta/classes-recipe/image-artifact-names.bbclass +2
-2@@ -11,11 +11,20 @@ IMAGE_BASENAME ?= "${PN}" IMAGE_VERSION_SUFFIX ?= "-${DATETIME}" IMAGE_VERSION_SUFFIX[vardepsexclude] += "DATETIME SOURCE_DATE_EPOCH" -IMAGE_NAME ?= "${IMAGE_BASENAME}-${MACHINE}${IMAGE_VERSION_SUFFIX}" -IMAGE_LINK_NAME ?= "${IMAGE_BASENAME}-${MACHINE}" +IMAGE_NAME ?= "${IMAGE_LINK_NAME}${IMAGE_VERSION_SUFFIX}" +IMAGE_LINK_NAME ?= "${IMAGE_BASENAME}${IMAGE_MACHINE_SUFFIX}"
So there's a minor unfortunate side-effect of this in that you can no longer just set IMAGE_LINK_NAME = "" to drop the symlinks - a minority use case, but something I have used. Of course you can just re-set the value of IMAGE_NAME. I am making a note in the manual and migration guide.
Cheers
Paul
Hi Paul,
I'm sorry about that, will make sure this is supported again in final state with all [YOCTO #12937] changes where the *_NAME and *_LINK_NAME variables are reversed:
-IMAGE_NAME ?= "${IMAGE_LINK_NAME}${IMAGE_VERSION_SUFFIX}" -IMAGE_LINK_NAME ?= "${IMAGE_BASENAME}${IMAGE_MACHINE_SUFFIX}${IMAGE_NAME_SUFFIX}" +IMAGE_NAME ?= "${IMAGE_BASENAME}${IMAGE_MACHINE_SUFFIX}${IMAGE_NAME_SUFFIX}" +IMAGE_LINK_NAME ?= "${IMAGE_NAME}${IMAGE_VERSION_SUFFIX}"
and versioned hardlinks (instead of symlinks) *_LINK_NAME are created by separate task. Which will respect *_LINK_NAME variables being set to empty to disable creating them.
I've just added another commit to my [YOCTO #12937] staging branch
https://git.openembedded.org/openembedded-core-contrib/log/?h=jansa/artifacts
to respect empty *_LINK_NAME variables also for kernel artifacts.
Cheers,
Quoted 51 lines · Paul Eggleton
Hi Martin
Quoted 25 lines · On Tuesday, 14 March 2023 01:15:36 NZST Martin Jansa wrote:
* to make it easier for projects to avoid default -${MACHINE} suffix if the ${MACHINE} named
DEPLOY_DIR_IMAGEworks better for them* also use
IMAGE_LINK_NAMEinIMAGE_NAMEto make it more clear thatIMAGE_NAMEis the same asIMAGE_LINK_NAMEbut with version suffix* adding it as separate variable helps us to catch the cases where we didn't respect ${IMAGE_LINK_NAME} variable and just used the common default ${IMAGE_BASENAME}-${MACHINE}.
[YOCTO #12937]
Signed-off-by Martin Jansa <[email protected]>--- meta/classes-recipe/image-artifact-names.bbclass | 15 ++++++++++++--- meta/classes-recipe/kernel-artifact-names.bbclass | 2 +- 2 files changed, 13 insertions(+), 4 deletions(-)(quoted diff) +0
-0b/meta/classes-recipe/image-artifact-names.bbclass index
9dc25b6dde..ac2376d59a100644meta/classes-recipe/image-artifact-names.bbclass line 15
Quoted 7 lines
meta/classes-recipe/image-artifact-names.bbclass +2
-2@@ -11,11 +11,20 @@ IMAGE_BASENAME ?= "${PN}" IMAGE_VERSION_SUFFIX ?= "-${DATETIME}" IMAGE_VERSION_SUFFIX[vardepsexclude] += "DATETIME SOURCE_DATE_EPOCH" -IMAGE_NAME ?= "${IMAGE_BASENAME}-${MACHINE}${IMAGE_VERSION_SUFFIX}" -IMAGE_LINK_NAME ?= "${IMAGE_BASENAME}-${MACHINE}" +IMAGE_NAME ?= "${IMAGE_LINK_NAME}${IMAGE_VERSION_SUFFIX}" +IMAGE_LINK_NAME ?= "${IMAGE_BASENAME}${IMAGE_MACHINE_SUFFIX}"So there's a minor unfortunate side-effect of this in that you can no longer
just setIMAGE_LINK_NAME= "" to drop the symlinks - a minority use case, but
something I have used. Of course you can just re-set the value ofIMAGE_NAME.
I am making a note in the manual and migration guide.Cheers
Paul
Hi Paul,
this was fixed in:
https://patchwork.yoctoproject.org/project/oe-core/list/?series=19541
please check it and share your opinion about these changes, otherwise it won't get merged and https://bugzilla.yoctoproject.org/show_bug.cgi?id=12937
will be closed as "won't fix".
Cheers,
Quoted 77 lines · Martin Jansa
Hi Paul,
I'm sorry about that, will make sure this is supported again in final state with all [YOCTO #12937] changes where the
*_NAMEand*_LINK_NAMEvariables are reversed:(quoted diff) +0
-2-IMAGE_NAME ?= "${IMAGE_LINK_NAME}${IMAGE_VERSION_SUFFIX}" -IMAGE_LINK_NAME ?="${IMAGE_BASENAME}${IMAGE_MACHINE_SUFFIX}${IMAGE_NAME_SUFFIX}" +IMAGE_NAME ?= "${IMAGE_BASENAME}${IMAGE_MACHINE_SUFFIX}${IMAGE_NAME_SUFFIX}" +IMAGE_LINK_NAME ?= "${IMAGE_NAME}${IMAGE_VERSION_SUFFIX}"
and versioned hardlinks (instead of symlinks)
*_LINK_NAMEare created by separate task. Which will respect*_LINK_NAMEvariables being set to empty to disable creating them.I've just added another commit to my [YOCTO #12937] staging branch
https://git.openembedded.org/openembedded-core-contrib/log/?h=jansa/artifacts
to respect empty*_LINK_NAMEvariables also for kernel artifacts.Cheers,
Quoted 51 lines · Paul Eggleton
Hi Martin
Quoted 25 lines · On Tuesday, 14 March 2023 01:15:36 NZST Martin Jansa wrote:
* to make it easier for projects to avoid default -${MACHINE} suffix if the ${MACHINE} named
DEPLOY_DIR_IMAGEworks better for them* also use
IMAGE_LINK_NAMEinIMAGE_NAMEto make it more clear thatIMAGE_NAMEis the same asIMAGE_LINK_NAMEbut with version suffix* adding it as separate variable helps us to catch the cases where we didn't respect ${IMAGE_LINK_NAME} variable and just used the common default ${IMAGE_BASENAME}-${MACHINE}.
[YOCTO #12937]
Signed-off-by Martin Jansa <[email protected]>--- meta/classes-recipe/image-artifact-names.bbclass | 15 ++++++++++++--- meta/classes-recipe/kernel-artifact-names.bbclass | 2 +- 2 files changed, 13 insertions(+), 4 deletions(-)(quoted diff) +0
-0b/meta/classes-recipe/image-artifact-names.bbclass index
9dc25b6dde..ac2376d59a100644meta/classes-recipe/image-artifact-names.bbclass line 15
Quoted 7 lines
meta/classes-recipe/image-artifact-names.bbclass +2
-2@@ -11,11 +11,20 @@ IMAGE_BASENAME ?= "${PN}" IMAGE_VERSION_SUFFIX ?= "-${DATETIME}" IMAGE_VERSION_SUFFIX[vardepsexclude] += "DATETIME SOURCE_DATE_EPOCH" -IMAGE_NAME ?= "${IMAGE_BASENAME}-${MACHINE}${IMAGE_VERSION_SUFFIX}" -IMAGE_LINK_NAME ?= "${IMAGE_BASENAME}-${MACHINE}" +IMAGE_NAME ?= "${IMAGE_LINK_NAME}${IMAGE_VERSION_SUFFIX}" +IMAGE_LINK_NAME ?= "${IMAGE_BASENAME}${IMAGE_MACHINE_SUFFIX}"So there's a minor unfortunate side-effect of this in that you can no longer
just setIMAGE_LINK_NAME= "" to drop the symlinks - a minority use case, but
something I have used. Of course you can just re-set the value ofIMAGE_NAME.
I am making a note in the manual and migration guide.Cheers
Paul