wic: ship the tools it invokes
v6 cover wic: ship the tools it invokes
wic shells out to a range of host tools but does not declare them, so wherever it is installed as a package it works only by chance depending on what the host has installed. The do_image_wic task does not have that
problem: image_types_wic.bbclass makes the image recipe depend on the -native recipe of each tool, so they are all in the image's own native sysroot by the time wic runs. Nothing outside that task gets the same treatment.
Patch 1 narrows wic-tools to staging wic's host tools: the target bootloader firmware (syslinux, grub-efi, systemd-boot) it depended on for the wic oe-selftest is now built by that oe-selftest instead. Which firmware is needed depends on the plugins a .wks uses, so it belongs to whoever supplies the .wks, not to wic-tools. Images already work that way through WKS_FILE_DEPENDS in image_types_wic.bbclass, and now the oe-selftest does too.
Patch 2 removes a dependency that outlived its reason. wic gained virtual/cross-binutils in 2022 because its EFI boot image plugin assembled unified kernel images and needed cross-objcopy. That plugin stopped doing so in 2024 and now points at uki.bbclass, which declares virtual/cross-binutils itself. Nothing wic runs invokes objcopy any more.
Patch 3 gives the tool list a single home and uses it. wic declares the host tools it runs as RDEPENDS, so they are installed alongside it in every variant that packages it; and the two other copies of that list go away in the same patch. wic-tools and image_types_wic.bbclass had written it out separately and drifted apart; both now just depend on wic-native, and native.bbclass's do_populate_sysroot[rdeptask] stages everything wic runtime-depends on. Three things stay written out. cdrtools has only a native recipe in oe-core, so there is no package a target or nativesdk wic could name, even though wic does run mkisofs; the build host gets it from wic-tools and from WKS_FILE_DEPENDS. grub is bootloader territory, which belongs to whatever drives wic. pseudo is wic-tools staging what the oe-selftest needs.
Patches 4 and 5 are follow-on cleanups of the wic oe-selftest: drop a dead in-tree wic lookup, and drop the redundant per-test PATH overrides that duplicated what the test setup already establishes.
Patch 6 fixes a dependency that has been on the wrong axis since 2019:
an image depends on syslinux-native when the build host is x86, but it is the target that decides whether a .wks needs it, so an x86 image built on a non-x86 host ends up with the bootloader but not the installer. That is bug 16383, which carries the full history:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=16383
changes in v6:
- new patch 2: drop the obsolete virtual/cross-binutils dependency from wic-tools and image_types_wic.bbclass. It comes before the tool-list patch so that patch does not have to describe wic-tools as needing binutils only for it to be removed shortly afterwards.
- drop the shared conf/wic-helper-tools.inc that v5 added. Review pointed out that once wic has the RDEPENDS, a consumer can just depend on wic-native and let do_populate_sysroot[rdeptask] do the rest, which is true and removes the need for a list to keep in sync. The tool list now lives only in the wic recipe.
- the tool list is now defined and used in one patch rather than two:
wic gets the RDEPENDS, and image_types_wic.bbclass and wic-tools take the tools from wic-native, together. Images still gain tar-native and util-linux-native relative to master, which was the drift v5 was fixing.
- retitle the syslinux patch with an image_types_wic prefix, and gate WKS_FILE_DEPENDS_DEFAULT with target-arch appends now that the shared variable is gone. wic-tools keeps the arch-gated DEPENDS it already had on master.
changes in v5:
- the SDK_FEATURES half of the v4 series (patches 5 to 10) is not here. Reworking it to resolve features at the class level, as review asked, turned it into a larger piece of work than the wic changes it was travelling with, so it is being finished and sent as its own series.
- new patch: one shared list of the tools wic runs, replacing the three copies that had drifted apart. Images gain tar-native and util-linux-native, which only wic-tools staged before.
- new patch: gate syslinux-native on the target rather than the build host [YOCTO #13276]
- wic RDEPENDS patch: drop the bootloader (grub, syslinux) RDEPENDS entirely; wic declares only the host tools it runs. The bootloader firmware wic copies into an image is the responsibility of whatever drives wic (the oe-selftest, and image_types_wic.bbclass via WKS_FILE_DEPENDS), not of the wic package. This also removes the arch-gating that the v4 grub/syslinux RDEPENDS required.
- drastically trimmed the commit messages and in-tree comments throughout the series
changes in v4:
- new patch 1: move the target bootloader firmware out of wic-tools and stage it from the wic oe-selftest instead, so wic-tools carries only the host tools wic runs
- new selftest cleanups (drop the dead COREBASE/scripts wic lookup and the redundant per-test PATH overrides)
changes in v3:
- list the tools on all variants rather than only the nativesdk variant
changes in v2:
- dropped the wic-tools.inc refactor; folded the tool list into the wic recipe
- reworked the buildtools-extended-tarball change into a plain removal
Trevor Woerner (6):
wic-tools: drop the target bootloader firmware image_types_wic, wic-tools: drop the obsolete cross-binutils
dependency
wic: add runtime dependencies on the tools it invokes oeqa/selftest/wic: drop dead COREBASE/scripts wic lookup oeqa/selftest/wic: drop redundant per-test PATH overrides
image_types_wic: gate syslinux-native on the target, not the build
host
meta/classes-recipe/image_types_wic.bbclass | 10 +- meta/lib/oeqa/selftest/cases/wic.py | 842 +++++++++----------- meta/recipes-core/meta/wic-tools.bb | 13 +- meta/recipes-support/wic/wic_0.3.1.bb | 12 + 4 files changed, 414 insertions(+), 463 deletions(-)
Signature · 1 line
2.50.0.173.g8b6f19ccfc3a
PATCH v6 1/6 wic-tools: drop the target bootloader firmware
wic-tools should stage only wic's host tools, but its arch-specific appends also depended on target bootloader firmware (syslinux, grub-efi, systemd-boot) solely for the wic oe-selftest. Which firmware is needed depends on the plugins a .wks uses, so that firmware is the responsibility of whoever supplies the .wks, not of wic-tools. Have the oe-selftest bitbake those recipes itself instead.
Images already work that way through WKS_FILE_DEPENDS in image_types_wic.bbclass, and the oe-selftest now does the equivalent, adding syslinux to core-image-minimal's DEPENDS where the tests expect it in the recipe sysroot.
AI-Generated: codex/claude-opus 5 (xhigh)
---
changes in v6:
- none
changes in v5:
- trimmed the commit message and in-tree comments
changes in v4:
- new in v4
--- meta/lib/oeqa/selftest/cases/wic.py | 33 +++++++++++++++++++++++------ meta/recipes-core/meta/wic-tools.bb | 7 +++--- 2 files changed, 29 insertions(+), 11 deletions(-)
meta/lib/oeqa/selftest/cases/wic.py +26 -7
@@ -72,12 +72,23 @@ class WicTestCase(OESelftestTestCase): if self.td['USE_NLS'] != 'yes': self.skipTest('wic-tools needs USE_NLS=yes') - bitbake('wic-tools core-image-minimal core-image-minimal-mtdutils') + targets = 'wic-tools core-image-minimal core-image-minimal-mtdutils' + targets += ' ' + ' '.join(self._firmware_recipes()) + bitbake(targets) WicTestCase.image_is_ready = True os.environ['PATH'] = self._get_wic_path() rmtree(self.resultdir, ignore_errors=True) + def _firmware_recipes(self): + arch = self.td['HOST_ARCH'] + recipes = [] + if arch in ('i586', 'i686', 'x86_64', 'x86-64'): + recipes += ['syslinux', 'grub-efi', 'systemd-boot'] + elif arch == 'aarch64': + recipes += ['grub-efi', 'systemd-boot'] + return recipes + def tearDownLocal(self): """Remove resultdir as it may contain images.""" if self._old_path is None:
@@ -391,9 +402,13 @@ class Wic(WicTestCase): @skipIfNotArch(['i586', 'i686', 'x86_64']) def test_build_artifacts(self): """Test wic create directdisk providing all artifacts.""" - bb_vars = get_bb_vars(['STAGING_DATADIR', 'RECIPE_SYSROOT_NATIVE'], - 'wic-tools') - bb_vars.update(get_bb_vars(['DEPLOY_DIR_IMAGE', 'IMAGE_ROOTFS'], + # wic expects syslinux in core-image-minimal's recipe sysroot. + config = 'DEPENDS:pn-core-image-minimal += "syslinux"\n' + self.append_config(config) + bitbake('core-image-minimal') + self.remove_config(config) + bb_vars = get_bb_vars(['RECIPE_SYSROOT_NATIVE'], 'wic-tools') + bb_vars.update(get_bb_vars(['STAGING_DATADIR', 'DEPLOY_DIR_IMAGE', 'IMAGE_ROOTFS'], 'core-image-minimal')) bbvars = {key.lower(): value for key, value in bb_vars.items()} bbvars['resultdir'] = self.resultdir
@@ -494,9 +509,13 @@ class Wic(WicTestCase): @skipIfNotArch(['i586', 'i686', 'x86_64']) def test_rootfs_artifacts(self): """Test usage of rootfs plugin with rootfs paths""" - bb_vars = get_bb_vars(['STAGING_DATADIR', 'RECIPE_SYSROOT_NATIVE'], - 'wic-tools') - bb_vars.update(get_bb_vars(['DEPLOY_DIR_IMAGE', 'IMAGE_ROOTFS'], + # wic expects syslinux in core-image-minimal's recipe sysroot. + config = 'DEPENDS:pn-core-image-minimal += "syslinux"\n' + self.append_config(config) + bitbake('core-image-minimal') + self.remove_config(config) + bb_vars = get_bb_vars(['RECIPE_SYSROOT_NATIVE'], 'wic-tools') + bb_vars.update(get_bb_vars(['STAGING_DATADIR', 'DEPLOY_DIR_IMAGE', 'IMAGE_ROOTFS'], 'core-image-minimal')) bbvars = {key.lower(): value for key, value in bb_vars.items()} bbvars['wks'] = "directdisk-multi-rootfs"
meta/recipes-core/meta/wic-tools.bb +3 -4
@@ -10,10 +10,9 @@ DEPENDS = "\ e2fsprogs-native util-linux-native tar-native erofs-utils-native \ virtual/cross-binutils \ " -DEPENDS:append:x86 = " syslinux-native syslinux grub-efi systemd-boot" -DEPENDS:append:x86-64 = " syslinux-native syslinux grub-efi systemd-boot" -DEPENDS:append:x86-x32 = " syslinux-native syslinux grub-efi" -DEPENDS:append:aarch64 = " grub-efi systemd-boot" +DEPENDS:append:x86 = " syslinux-native" +DEPENDS:append:x86-64 = " syslinux-native" +DEPENDS:append:x86-x32 = " syslinux-native" INHIBIT_DEFAULT_DEPS = "1"
Signature · 1 line
2.50.0.173.g8b6f19ccfc3a
PATCH v6 2/6 image_types_wic, wic-tools: drop the obsolete cross-binutils dependency
In reply to Trevor Woernerwic depends on virtual/cross-binutils because its EFI boot image plugin once assembled unified kernel images and needed cross-objcopy for it. That plugin no longer builds them: it refuses and points at uki.bbclass, which declares virtual/cross-binutils itself.
No code wic runs invokes objcopy any more, so drop the dependency from the two places still carrying it.
AI-Generated: codex/claude-opus 5 (xhigh)
---
changes in v6:
- new patch
--- meta/classes-recipe/image_types_wic.bbclass | 2 -- meta/recipes-core/meta/wic-tools.bb | 1 - 2 files changed, 3 deletions(-)
meta/classes-recipe/image_types_wic.bbclass +0 -2
@@ -120,8 +120,6 @@ do_image_wic[deptask] += "do_image_complete" WKS_FILE_DEPENDS_DEFAULT = '${@bb.utils.contains_any("BUILD_ARCH", [ 'x86_64', 'i686' ], "syslinux-native", "",d)}' WKS_FILE_DEPENDS_DEFAULT += "wic-native bmaptool-native cdrtools-native btrfs-tools-native squashfs-tools-native e2fsprogs-native erofs-utils-native" -# Unified kernel images need objcopy -WKS_FILE_DEPENDS_DEFAULT += "virtual/cross-binutils" WKS_FILE_DEPENDS_BOOTLOADERS = "" WKS_FILE_DEPENDS_BOOTLOADERS:aarch64 = "grub-efi systemd-boot" WKS_FILE_DEPENDS_BOOTLOADERS:arm = "systemd-boot"
meta/recipes-core/meta/wic-tools.bb +0 -1
@@ -8,7 +8,6 @@ DEPENDS = "\ mtools-native bmaptool-native grub-native cdrtools-native \ btrfs-tools-native squashfs-tools-native pseudo-native \ e2fsprogs-native util-linux-native tar-native erofs-utils-native \ - virtual/cross-binutils \ " DEPENDS:append:x86 = " syslinux-native" DEPENDS:append:x86-64 = " syslinux-native"
Signature · 1 line
2.50.0.173.g8b6f19ccfc3a
PATCH v6 3/6 wic: add runtime dependencies on the tools it invokes
In reply to Trevor Woernerwic shells out to a range of host tools (parted, mkfs.*, mcopy, sfdisk, and more) but, since the recipe was created, has declared none of them, so an installed wic works only by chance depending on what the host provides. Declare them as RDEPENDS so they are installed with wic.
The wic image type and wic-tools each carried a copy of that same list, and the copies had drifted: only wic-tools staged tar-native and util-linux-native. Neither needs a list of its own now. native.bbclass makes a native recipe's do_populate_sysroot depend on its RDEPENDS, so depending on wic-native stages the lot.
Three entries stay written out. cdrtools has only a native recipe, so there is no package a target or nativesdk wic could name, even though wic does run mkisofs. grub is bootloader territory and belongs to whatever drives wic. pseudo is wic-tools staging what the oe-selftest needs.
AI-Generated: codex/claude-opus 5 (xhigh)
---
changes in v6:
- carry the tool list here directly; the shared conf/wic-helper-tools.inc that v5 added is gone
- fold the image_types_wic.bbclass and wic-tools change into this patch, so the one list is defined and used in the same place: both now take the tools from wic-native instead of repeating them
changes in v5:
- take the tool list from the shared conf/wic-helper-tools.inc
- drop the bootloader (grub, syslinux) RDEPENDS entirely; declare only the host tools wic runs
- trimmed the commit message; dropped the in-recipe comments
changes in v4:
- gate grub to x86 and aarch64; move syslinux into the same arch-gated appends
- also depend on syslinux-isolinux for isolinux.bin
changes in v3:
- list the tools on all variants rather than only the nativesdk variant
changes in v2:
- merge the tool list into the wic recipe as RDEPENDS; drop the separate .inc
--- meta/classes-recipe/image_types_wic.bbclass | 4 ++-- meta/recipes-core/meta/wic-tools.bb | 5 +---- meta/recipes-support/wic/wic_0.3.1.bb | 12 ++++++++++++ 3 files changed, 15 insertions(+), 6 deletions(-)
meta/classes-recipe/image_types_wic.bbclass +2 -2
@@ -112,14 +112,14 @@ do_image_wic[cleandirs] = "${WORKDIR}/build-wic" USING_WIC = "${@bb.utils.contains_any('IMAGE_FSTYPES', 'wic ' + ' '.join('wic.%s' % c for c in '${CONVERSIONTYPES}'.split()), '1', '', d)}" WKS_FILE_CHECKSUM = "${@wks_checksums(d.getVar('WKS_FILES').split(), d.getVar('WKS_SEARCH_PATH')) if '${USING_WIC}' else ''}" do_image_wic[file-checksums] += "${WKS_FILE_CHECKSUM}" -do_image_wic[depends] += "${@' '.join('%s-native:do_populate_sysroot' % r for r in ('wic', 'parted', 'gptfdisk', 'dosfstools', 'mtools'))}" +do_image_wic[depends] += "wic-native:do_populate_sysroot" # We ensure all artfacts are deployed (e.g virtual/bootloader) do_image_wic[recrdeptask] += "do_deploy" do_image_wic[deptask] += "do_image_complete" WKS_FILE_DEPENDS_DEFAULT = '${@bb.utils.contains_any("BUILD_ARCH", [ 'x86_64', 'i686' ], "syslinux-native", "",d)}' -WKS_FILE_DEPENDS_DEFAULT += "wic-native bmaptool-native cdrtools-native btrfs-tools-native squashfs-tools-native e2fsprogs-native erofs-utils-native" +WKS_FILE_DEPENDS_DEFAULT += "cdrtools-native" WKS_FILE_DEPENDS_BOOTLOADERS = "" WKS_FILE_DEPENDS_BOOTLOADERS:aarch64 = "grub-efi systemd-boot" WKS_FILE_DEPENDS_BOOTLOADERS:arm = "systemd-boot"
meta/recipes-core/meta/wic-tools.bb +1 -4
@@ -4,10 +4,7 @@ LICENSE = "MIT" DEPENDS = "\ wic-native \ - parted-native gptfdisk-native dosfstools-native \ - mtools-native bmaptool-native grub-native cdrtools-native \ - btrfs-tools-native squashfs-tools-native pseudo-native \ - e2fsprogs-native util-linux-native tar-native erofs-utils-native \ + cdrtools-native grub-native pseudo-native \ " DEPENDS:append:x86 = " syslinux-native" DEPENDS:append:x86-64 = " syslinux-native"
meta/recipes-support/wic/wic_0.3.1.bb +12 -0
@@ -15,6 +15,18 @@ RDEPENDS:${PN} += " \ python3-json \ python3-logging \ python3-misc \ + parted \ + gptfdisk \ + dosfstools \ + mtools \ + bmaptool \ + btrfs-tools \ + squashfs-tools \ + e2fsprogs \ + e2fsprogs-resize2fs \ + util-linux \ + tar \ + erofs-utils \ " BBCLASSEXTEND = "native nativesdk"
Signature · 1 line
2.50.0.173.g8b6f19ccfc3a
PATCH v6 4/6 oeqa/selftest/wic: drop dead COREBASE/scripts wic lookup
In reply to Trevor Woernerwic moved to a standalone repository and is no longer shipped in scripts/, so that search path can never match. Look for wic only in the wic-tools native sysroot.
AI-Generated: codex/claude-opus 4.8 (xhigh)
---
changes in v6:
- none
changes in v5:
- trimmed the commit message
changes in v4:
- new in v4
--- meta/lib/oeqa/selftest/cases/wic.py | 17 +++++------------ 1 file changed, 5 insertions(+), 12 deletions(-)
meta/lib/oeqa/selftest/cases/wic.py +5 -12
@@ -100,18 +100,11 @@ class WicTestCase(OESelftestTestCase): def _get_wic_path(self): if WicTestCase.wic_bindir is None: - search_paths = [ - os.path.join(self.td['COREBASE'], 'scripts'), - os.path.join(get_bb_var('RECIPE_SYSROOT_NATIVE', 'wic-tools'), 'usr', 'bin'), - ] - - for bindir in search_paths: - if os.path.exists(os.path.join(bindir, 'wic')): - WicTestCase.wic_bindir = bindir - break - - if WicTestCase.wic_bindir is None: - self.fail("Unable to find the wic binary in %s" % ', '.join(search_paths)) + bindir = os.path.join(get_bb_var('RECIPE_SYSROOT_NATIVE', 'wic-tools'), + 'usr', 'bin') + if not os.path.exists(os.path.join(bindir, 'wic')): + self.fail("Unable to find the wic binary in %s" % bindir) + WicTestCase.wic_bindir = bindir path_entries = [] for path_group in (
Signature · 1 line
2.50.0.173.g8b6f19ccfc3a
PATCH v6 5/6 oeqa/selftest/wic: drop redundant per-test PATH overrides
In reply to Trevor WoernersetUpLocal() already puts wic and its tools on PATH for every test, so the per-test "os.environ['PATH'] = get_bb_var('PATH', 'wic-tools')" overrides and their try/finally wrappers add nothing. Drop them; the resulting dedent makes the diff large, so review with git show -w.
AI-Generated: codex/claude-opus 4.8 (xhigh)
---
changes in v6:
- none
changes in v5:
- trimmed the commit message
changes in v4:
- new in v4
--- meta/lib/oeqa/selftest/cases/wic.py | 792 +++++++++++++--------------- 1 file changed, 362 insertions(+), 430 deletions(-)
meta/lib/oeqa/selftest/cases/wic.py +362 -430
@@ -525,104 +525,97 @@ class Wic(WicTestCase): def test_exclude_path(self): """Test --exclude-path wks option.""" - oldpath = os.environ['PATH'] - os.environ['PATH'] = get_bb_var("PATH", "wic-tools") - - try: - wks_file = 'temp.wks' - with open(wks_file, 'w') as wks: - rootfs_dir = get_bb_var('IMAGE_ROOTFS', 'core-image-minimal') - wks.write(""" + wks_file = 'temp.wks' + with open(wks_file, 'w') as wks: + rootfs_dir = get_bb_var('IMAGE_ROOTFS', 'core-image-minimal') + wks.write(""" part / --source rootfs --ondisk mmcblk0 --fstype=ext4 --exclude-path usr part /usr --source rootfs --ondisk mmcblk0 --fstype=ext4 --rootfs-dir %s/usr part /etc --source rootfs --ondisk mmcblk0 --fstype=ext4 --exclude-path bin/ --rootfs-dir %s/usr part /mnt --source rootfs --ondisk mmcblk0 --fstype=ext4 --exclude-path bin/whoami --rootfs-dir %s/usr""" - % (rootfs_dir, rootfs_dir, rootfs_dir)) - runCmd("wic create %s -e core-image-minimal -o %s" \ - % (wks_file, self.resultdir)) - - os.remove(wks_file) - wicout = glob(os.path.join(self.resultdir, "%s-*direct" % 'temp')) - self.assertEqual(1, len(wicout)) + % (rootfs_dir, rootfs_dir, rootfs_dir)) + runCmd("wic create %s -e core-image-minimal -o %s" \ + % (wks_file, self.resultdir)) - wicimg = wicout[0] - - # verify partition size with wic - res = runCmd("parted -m %s unit b p" % wicimg, stderr=subprocess.PIPE) - - # parse parted output which looks like this: - # BYT;\n - # /var/tmp/wic/build/tmpfwvjjkf_-201611101222-hda.direct:200MiB:file:512:512:msdos::;\n - # 1:0.00MiB:200MiB:200MiB:ext4::;\n - partlns = res.output.splitlines()[2:] + os.remove(wks_file) + wicout = glob(os.path.join(self.resultdir, "%s-*direct" % 'temp')) + self.assertEqual(1, len(wicout)) - self.assertEqual(4, len(partlns)) + wicimg = wicout[0] - for part in [1, 2, 3, 4]: - part_file = os.path.join(self.resultdir, "selftest_img.part%d" % part) - partln = partlns[part-1].split(":") - self.assertEqual(7, len(partln)) - start = int(partln[1].rstrip("B")) / 512 - length = int(partln[3].rstrip("B")) / 512 - runCmd("dd if=%s of=%s skip=%d count=%d" % - (wicimg, part_file, start, length)) - - # Test partition 1, should contain the normal root directories, except - # /usr. - res = runCmd("debugfs -R 'ls -p' %s" % \ - os.path.join(self.resultdir, "selftest_img.part1"), stderr=subprocess.PIPE) - files = extract_files(res.output) - self.assertIn("etc", files) - self.assertNotIn("usr", files) - - # Partition 2, should contain common directories for /usr, not root - # directories. - res = runCmd("debugfs -R 'ls -p' %s" % \ - os.path.join(self.resultdir, "selftest_img.part2"), stderr=subprocess.PIPE) - files = extract_files(res.output) - self.assertNotIn("etc", files) - self.assertNotIn("usr", files) - self.assertIn("share", files) - - # Partition 3, should contain the same as partition 2, including the bin - # directory, but not the files inside it. - res = runCmd("debugfs -R 'ls -p' %s" % \ - os.path.join(self.resultdir, "selftest_img.part3"), stderr=subprocess.PIPE) - files = extract_files(res.output) - self.assertNotIn("etc", files) - self.assertNotIn("usr", files) - self.assertIn("share", files) - self.assertIn("bin", files) - res = runCmd("debugfs -R 'ls -p bin' %s" % \ - os.path.join(self.resultdir, "selftest_img.part3"), stderr=subprocess.PIPE) - files = extract_files(res.output) - self.assertIn(".", files) - self.assertIn("..", files) - self.assertEqual(2, len(files)) - - # Partition 4, should contain the same as partition 2, including the bin - # directory, but not whoami (a symlink to busybox.nosuid) inside it. - res = runCmd("debugfs -R 'ls -p' %s" % \ - os.path.join(self.resultdir, "selftest_img.part4"), stderr=subprocess.PIPE) - files = extract_files(res.output) - self.assertNotIn("etc", files) - self.assertNotIn("usr", files) - self.assertIn("share", files) - self.assertIn("bin", files) - res = runCmd("debugfs -R 'ls -p bin' %s" % \ - os.path.join(self.resultdir, "selftest_img.part4"), stderr=subprocess.PIPE) - files = extract_files(res.output) - self.assertIn(".", files) - self.assertIn("..", files) - self.assertIn("who", files) - self.assertNotIn("whoami", files) - - for part in [1, 2, 3, 4]: - part_file = os.path.join(self.resultdir, "selftest_img.part%d" % part) - os.remove(part_file) + # verify partition size with wic + res = runCmd("parted -m %s unit b p" % wicimg, stderr=subprocess.PIPE) - finally: - os.environ['PATH'] = oldpath + # parse parted output which looks like this: + # BYT;\n + # /var/tmp/wic/build/tmpfwvjjkf_-201611101222-hda.direct:200MiB:file:512:512:msdos::;\n + # 1:0.00MiB:200MiB:200MiB:ext4::;\n + partlns = res.output.splitlines()[2:] + + self.assertEqual(4, len(partlns)) + + for part in [1, 2, 3, 4]: + part_file = os.path.join(self.resultdir, "selftest_img.part%d" % part) + partln = partlns[part-1].split(":") + self.assertEqual(7, len(partln)) + start = int(partln[1].rstrip("B")) / 512 + length = int(partln[3].rstrip("B")) / 512 + runCmd("dd if=%s of=%s skip=%d count=%d" % + (wicimg, part_file, start, length)) + + # Test partition 1, should contain the normal root directories, except + # /usr. + res = runCmd("debugfs -R 'ls -p' %s" % \ + os.path.join(self.resultdir, "selftest_img.part1"), stderr=subprocess.PIPE) + files = extract_files(res.output) + self.assertIn("etc", files) + self.assertNotIn("usr", files) + + # Partition 2, should contain common directories for /usr, not root + # directories. + res = runCmd("debugfs -R 'ls -p' %s" % \ + os.path.join(self.resultdir, "selftest_img.part2"), stderr=subprocess.PIPE) + files = extract_files(res.output) + self.assertNotIn("etc", files) + self.assertNotIn("usr", files) + self.assertIn("share", files) + + # Partition 3, should contain the same as partition 2, including the bin + # directory, but not the files inside it. + res = runCmd("debugfs -R 'ls -p' %s" % \ + os.path.join(self.resultdir, "selftest_img.part3"), stderr=subprocess.PIPE) + files = extract_files(res.output) + self.assertNotIn("etc", files) + self.assertNotIn("usr", files) + self.assertIn("share", files) + self.assertIn("bin", files) + res = runCmd("debugfs -R 'ls -p bin' %s" % \ + os.path.join(self.resultdir, "selftest_img.part3"), stderr=subprocess.PIPE) + files = extract_files(res.output) + self.assertIn(".", files) + self.assertIn("..", files) + self.assertEqual(2, len(files)) + + # Partition 4, should contain the same as partition 2, including the bin + # directory, but not whoami (a symlink to busybox.nosuid) inside it. + res = runCmd("debugfs -R 'ls -p' %s" % \ + os.path.join(self.resultdir, "selftest_img.part4"), stderr=subprocess.PIPE) + files = extract_files(res.output) + self.assertNotIn("etc", files) + self.assertNotIn("usr", files) + self.assertIn("share", files) + self.assertIn("bin", files) + res = runCmd("debugfs -R 'ls -p bin' %s" % \ + os.path.join(self.resultdir, "selftest_img.part4"), stderr=subprocess.PIPE) + files = extract_files(res.output) + self.assertIn(".", files) + self.assertIn("..", files) + self.assertIn("who", files) + self.assertNotIn("whoami", files) + + for part in [1, 2, 3, 4]: + part_file = os.path.join(self.resultdir, "selftest_img.part%d" % part) + os.remove(part_file) def test_exclude_path_with_extra_space(self): """Test having --exclude-path with IMAGE_ROOTFS_EXTRA_SPACE. [Yocto #15555]"""
@@ -661,75 +654,61 @@ part /mnt --source rootfs --ondisk mmcblk0 --fstype=ext4 --exclude-path bin/whoa def test_include_path(self): """Test --include-path wks option.""" - oldpath = os.environ['PATH'] - os.environ['PATH'] = get_bb_var("PATH", "wic-tools") - - try: - include_path = os.path.join(self.resultdir, 'test-include') - os.makedirs(include_path) - with open(os.path.join(include_path, 'test-file'), 'w') as t: - t.write("test\n") - wks_file = os.path.join(include_path, 'temp.wks') - with open(wks_file, 'w') as wks: - rootfs_dir = get_bb_var('IMAGE_ROOTFS', 'core-image-minimal') - wks.write(""" + include_path = os.path.join(self.resultdir, 'test-include') + os.makedirs(include_path) + with open(os.path.join(include_path, 'test-file'), 'w') as t: + t.write("test\n") + wks_file = os.path.join(include_path, 'temp.wks') + with open(wks_file, 'w') as wks: + rootfs_dir = get_bb_var('IMAGE_ROOTFS', 'core-image-minimal') + wks.write(""" part /part1 --source rootfs --ondisk mmcblk0 --fstype=ext4 part /part2 --source rootfs --ondisk mmcblk0 --fstype=ext4 --include-path %s""" - % (include_path)) - runCmd("wic create %s -e core-image-minimal -o %s" \ - % (wks_file, self.resultdir)) - - part1 = glob(os.path.join(self.resultdir, 'temp-*.direct.p1'))[0] - part2 = glob(os.path.join(self.resultdir, 'temp-*.direct.p2'))[0] + % (include_path)) + runCmd("wic create %s -e core-image-minimal -o %s" \ + % (wks_file, self.resultdir)) - # Test partition 1, should not contain 'test-file' - res = runCmd("debugfs -R 'ls -p' %s" % (part1), stderr=subprocess.PIPE) - files = extract_files(res.output) - self.assertNotIn('test-file', files) - self.assertEqual(True, files_own_by_root(res.output)) + part1 = glob(os.path.join(self.resultdir, 'temp-*.direct.p1'))[0] + part2 = glob(os.path.join(self.resultdir, 'temp-*.direct.p2'))[0] - # Test partition 2, should contain 'test-file' - res = runCmd("debugfs -R 'ls -p' %s" % (part2), stderr=subprocess.PIPE) - files = extract_files(res.output) - self.assertIn('test-file', files) - self.assertEqual(True, files_own_by_root(res.output)) + # Test partition 1, should not contain 'test-file' + res = runCmd("debugfs -R 'ls -p' %s" % (part1), stderr=subprocess.PIPE) + files = extract_files(res.output) + self.assertNotIn('test-file', files) + self.assertEqual(True, files_own_by_root(res.output)) - finally: - os.environ['PATH'] = oldpath + # Test partition 2, should contain 'test-file' + res = runCmd("debugfs -R 'ls -p' %s" % (part2), stderr=subprocess.PIPE) + files = extract_files(res.output) + self.assertIn('test-file', files) + self.assertEqual(True, files_own_by_root(res.output)) def test_include_path_embeded(self): """Test --include-path wks option.""" - oldpath = os.environ['PATH'] - os.environ['PATH'] = get_bb_var("PATH", "wic-tools") - - try: - include_path = os.path.join(self.resultdir, 'test-include') - os.makedirs(include_path) - with open(os.path.join(include_path, 'test-file'), 'w') as t: - t.write("test\n") - wks_file = os.path.join(include_path, 'temp.wks') - with open(wks_file, 'w') as wks: - wks.write(""" + include_path = os.path.join(self.resultdir, 'test-include') + os.makedirs(include_path) + with open(os.path.join(include_path, 'test-file'), 'w') as t: + t.write("test\n") + wks_file = os.path.join(include_path, 'temp.wks') + with open(wks_file, 'w') as wks: + wks.write(""" part / --source rootfs --fstype=ext4 --include-path %s --include-path core-image-minimal-mtdutils export/""" - % (include_path)) - runCmd("wic create %s -e core-image-minimal -o %s" \ - % (wks_file, self.resultdir)) - - part1 = glob(os.path.join(self.resultdir, 'temp-*.direct.p1'))[0] + % (include_path)) + runCmd("wic create %s -e core-image-minimal -o %s" \ + % (wks_file, self.resultdir)) - res = runCmd("debugfs -R 'ls -p' %s" % (part1), stderr=subprocess.PIPE) - files = extract_files(res.output) - self.assertIn('test-file', files) - self.assertEqual(True, files_own_by_root(res.output)) + part1 = glob(os.path.join(self.resultdir, 'temp-*.direct.p1'))[0] - res = runCmd("debugfs -R 'ls -p /export/etc/' %s" % (part1), stderr=subprocess.PIPE) - files = extract_files(res.output) - self.assertIn('passwd', files) - self.assertEqual(True, files_own_by_root(res.output)) + res = runCmd("debugfs -R 'ls -p' %s" % (part1), stderr=subprocess.PIPE) + files = extract_files(res.output) + self.assertIn('test-file', files) + self.assertEqual(True, files_own_by_root(res.output)) - finally: - os.environ['PATH'] = oldpath + res = runCmd("debugfs -R 'ls -p /export/etc/' %s" % (part1), stderr=subprocess.PIPE) + files = extract_files(res.output) + self.assertIn('passwd', files) + self.assertEqual(True, files_own_by_root(res.output)) def test_include_path_errors(self): """Test --include-path wks option error handling."""
@@ -780,9 +759,6 @@ part / --source rootfs --fstype=ext4 --include-path %s --include-path core-imag # prepare wicenv and rootfs bitbake('core-image-minimal core-image-minimal-mtdutils -c do_rootfs_wicenv') - oldpath = os.environ['PATH'] - os.environ['PATH'] = get_bb_var("PATH", "wic-tools") - t_normal = """ part / --source rootfs --fstype=ext4 """
@@ -799,61 +775,50 @@ part /etc --source rootfs --fstype=ext4 --change-directory=etc """ tests = [t_normal, t_exclude, t_multi, t_change] - try: - for test in tests: - include_path = os.path.join(self.resultdir, 'test-include') - os.makedirs(include_path) - wks_file = os.path.join(include_path, 'temp.wks') - with open(wks_file, 'w') as wks: - wks.write(test) - runCmd("wic create %s -e core-image-minimal -o %s" \ - % (wks_file, self.resultdir)) - - for part in glob(os.path.join(self.resultdir, 'temp-*.direct.p*')): - res = runCmd("debugfs -R 'ls -p' %s" % (part), stderr=subprocess.PIPE) - self.assertEqual(True, files_own_by_root(res.output)) + for test in tests: + include_path = os.path.join(self.resultdir, 'test-include') + os.makedirs(include_path) + wks_file = os.path.join(include_path, 'temp.wks') + with open(wks_file, 'w') as wks: + wks.write(test) + runCmd("wic create %s -e core-image-minimal -o %s" \ + % (wks_file, self.resultdir)) - config = 'IMAGE_FSTYPES += "wic"\nWKS_FILE = "%s"\n' % wks_file - self.append_config(config) - bitbake('core-image-minimal') - tmpdir = os.path.join(get_bb_var('WORKDIR', 'core-image-minimal'),'build-wic') + for part in glob(os.path.join(self.resultdir, 'temp-*.direct.p*')): + res = runCmd("debugfs -R 'ls -p' %s" % (part), stderr=subprocess.PIPE) + self.assertEqual(True, files_own_by_root(res.output)) - # check each partition for permission - for part in glob(os.path.join(tmpdir, 'temp-*.direct.p*')): - res = runCmd("debugfs -R 'ls -p' %s" % (part), stderr=subprocess.PIPE) - self.assertTrue(files_own_by_root(res.output) - ,msg='Files permission incorrect using wks set "%s"' % test) + config = 'IMAGE_FSTYPES += "wic"\nWKS_FILE = "%s"\n' % wks_file + self.append_config(config) + bitbake('core-image-minimal') + tmpdir = os.path.join(get_bb_var('WORKDIR', 'core-image-minimal'),'build-wic') - # clean config and result directory for next cases - self.remove_config(config) - rmtree(self.resultdir, ignore_errors=True) + # check each partition for permission + for part in glob(os.path.join(tmpdir, 'temp-*.direct.p*')): + res = runCmd("debugfs -R 'ls -p' %s" % (part), stderr=subprocess.PIPE) + self.assertTrue(files_own_by_root(res.output) + ,msg='Files permission incorrect using wks set "%s"' % test) - finally: - os.environ['PATH'] = oldpath + # clean config and result directory for next cases + self.remove_config(config) + rmtree(self.resultdir, ignore_errors=True) def test_change_directory(self): """Test --change-directory wks option.""" - oldpath = os.environ['PATH'] - os.environ['PATH'] = get_bb_var("PATH", "wic-tools") - - try: - include_path = os.path.join(self.resultdir, 'test-include') - os.makedirs(include_path) - wks_file = os.path.join(include_path, 'temp.wks') - with open(wks_file, 'w') as wks: - wks.write("part /etc --source rootfs --fstype=ext4 --change-directory=etc") - runCmd("wic create %s -e core-image-minimal -o %s" \ - % (wks_file, self.resultdir)) - - part1 = glob(os.path.join(self.resultdir, 'temp-*.direct.p1'))[0] + include_path = os.path.join(self.resultdir, 'test-include') + os.makedirs(include_path) + wks_file = os.path.join(include_path, 'temp.wks') + with open(wks_file, 'w') as wks: + wks.write("part /etc --source rootfs --fstype=ext4 --change-directory=etc") + runCmd("wic create %s -e core-image-minimal -o %s" \ + % (wks_file, self.resultdir)) - res = runCmd("debugfs -R 'ls -p' %s" % (part1), stderr=subprocess.PIPE) - files = extract_files(res.output) - self.assertIn('passwd', files) + part1 = glob(os.path.join(self.resultdir, 'temp-*.direct.p1'))[0] - finally: - os.environ['PATH'] = oldpath + res = runCmd("debugfs -R 'ls -p' %s" % (part1), stderr=subprocess.PIPE) + files = extract_files(res.output) + self.assertIn('passwd', files) def test_change_directory_errors(self): """Test --change-directory wks option error handling."""
@@ -876,41 +841,34 @@ part /etc --source rootfs --fstype=ext4 --change-directory=etc def test_no_fstab_update(self): """Test --no-fstab-update wks option.""" - oldpath = os.environ['PATH'] - os.environ['PATH'] = get_bb_var("PATH", "wic-tools") - # Get stock fstab from base-files recipe bitbake('base-files -c do_install') bf_fstab = os.path.join(get_bb_var('D', 'base-files'), 'etc', 'fstab') self.assertEqual(True, os.path.exists(bf_fstab)) bf_fstab_md5sum = runCmd('md5sum %s ' % bf_fstab).output.split(" ")[0] - try: - no_fstab_update_path = os.path.join(self.resultdir, 'test-no-fstab-update') - os.makedirs(no_fstab_update_path) - wks_file = os.path.join(no_fstab_update_path, 'temp.wks') - with open(wks_file, 'w') as wks: - wks.writelines(['part / --source rootfs --fstype=ext4 --label rootfs\n', - 'part /mnt/p2 --source rootfs --rootfs-dir=core-image-minimal ', - '--fstype=ext4 --label p2 --no-fstab-update\n']) - runCmd("wic create %s -e core-image-minimal -o %s" \ - % (wks_file, self.resultdir)) - - part_fstab_md5sum = [] - for i in range(1, 3): - part = glob(os.path.join(self.resultdir, 'temp-*.direct.p') + str(i))[0] - part_fstab = runCmd("debugfs -R 'cat etc/fstab' %s" % (part), stderr=subprocess.PIPE) - part_fstab_md5sum.append(hashlib.md5((part_fstab.output + "\n\n").encode('utf-8')).hexdigest()) + no_fstab_update_path = os.path.join(self.resultdir, 'test-no-fstab-update') + os.makedirs(no_fstab_update_path) + wks_file = os.path.join(no_fstab_update_path, 'temp.wks') + with open(wks_file, 'w') as wks: + wks.writelines(['part / --source rootfs --fstype=ext4 --label rootfs\n', + 'part /mnt/p2 --source rootfs --rootfs-dir=core-image-minimal ', + '--fstype=ext4 --label p2 --no-fstab-update\n']) + runCmd("wic create %s -e core-image-minimal -o %s" \ + % (wks_file, self.resultdir)) - # '/etc/fstab' in partition 2 should contain the same stock fstab file - # as the one installed by the base-file recipe. - self.assertEqual(bf_fstab_md5sum, part_fstab_md5sum[1]) + part_fstab_md5sum = [] + for i in range(1, 3): + part = glob(os.path.join(self.resultdir, 'temp-*.direct.p') + str(i))[0] + part_fstab = runCmd("debugfs -R 'cat etc/fstab' %s" % (part), stderr=subprocess.PIPE) + part_fstab_md5sum.append(hashlib.md5((part_fstab.output + "\n\n").encode('utf-8')).hexdigest()) - # '/etc/fstab' in partition 1 should contain an updated fstab file. - self.assertNotEqual(bf_fstab_md5sum, part_fstab_md5sum[0]) + # '/etc/fstab' in partition 2 should contain the same stock fstab file + # as the one installed by the base-file recipe. + self.assertEqual(bf_fstab_md5sum, part_fstab_md5sum[1]) - finally: - os.environ['PATH'] = oldpath + # '/etc/fstab' in partition 1 should contain an updated fstab file. + self.assertNotEqual(bf_fstab_md5sum, part_fstab_md5sum[0]) def test_no_fstab_update_errors(self): """Test --no-fstab-update wks option error handling."""
@@ -984,153 +942,139 @@ bootloader --ptable gpt""") def test_wic_sector_size_env(self): """Test generation image sector size via environment (obsolete)""" - oldpath = os.environ['PATH'] - os.environ['PATH'] = get_bb_var("PATH", "wic-tools") + # Add WIC_SECTOR_SIZE into config + config = 'WIC_SECTOR_SIZE = "4096"\n'\ + 'WICVARS:append = " WIC_SECTOR_SIZE"\n' + self.append_config(config) + bitbake('core-image-minimal') - try: - # Add WIC_SECTOR_SIZE into config - config = 'WIC_SECTOR_SIZE = "4096"\n'\ - 'WICVARS:append = " WIC_SECTOR_SIZE"\n' - self.append_config(config) - bitbake('core-image-minimal') + # Check WIC_SECTOR_SIZE apply to bitbake variable + wic_sector_size_str = get_bb_var('WIC_SECTOR_SIZE', 'core-image-minimal') + wic_sector_size = int(wic_sector_size_str) + self.assertEqual(4096, wic_sector_size) - # Check WIC_SECTOR_SIZE apply to bitbake variable - wic_sector_size_str = get_bb_var('WIC_SECTOR_SIZE', 'core-image-minimal') - wic_sector_size = int(wic_sector_size_str) - self.assertEqual(4096, wic_sector_size) - - self.logger.info("Test wic_sector_size: %d \n" % wic_sector_size) - - with NamedTemporaryFile("w", suffix=".wks") as wks: - wks.writelines( - ['bootloader --ptable gpt\n', - 'part --fstype vfat --fstype vfat --label emptyfat --size 1M --mkfs-extraopts "-S 4096"\n', - 'part --fstype ext4 --source rootfs --label rofs-a --mkfs-extraopts "-b 4096"\n', - 'part --fstype ext4 --source rootfs --use-uuid --mkfs-extraopts "-b 4096"\n']) - wks.flush() - cmd = "wic create %s -e core-image-minimal -o %s" % (wks.name, self.resultdir) - runCmd(cmd) - wksname = os.path.splitext(os.path.basename(wks.name))[0] - images = glob(os.path.join(self.resultdir, "%s-*direct" % wksname)) - self.assertEqual(1, len(images)) + self.logger.info("Test wic_sector_size: %d \n" % wic_sector_size) - sysroot = get_bb_var('RECIPE_SYSROOT_NATIVE', 'wic-tools') - # list partitions - result = runCmd("wic ls %s -n %s" % (images[0], sysroot)) - print(result.output) - # Deprecated message + 4 lines of output: header + 3 partitions - self.assertEqual(5, len(result.output.split('\n'))) + with NamedTemporaryFile("w", suffix=".wks") as wks: + wks.writelines( + ['bootloader --ptable gpt\n', + 'part --fstype vfat --fstype vfat --label emptyfat --size 1M --mkfs-extraopts "-S 4096"\n', + 'part --fstype ext4 --source rootfs --label rofs-a --mkfs-extraopts "-b 4096"\n', + 'part --fstype ext4 --source rootfs --use-uuid --mkfs-extraopts "-b 4096"\n']) + wks.flush() + cmd = "wic create %s -e core-image-minimal -o %s" % (wks.name, self.resultdir) + runCmd(cmd) + wksname = os.path.splitext(os.path.basename(wks.name))[0] + images = glob(os.path.join(self.resultdir, "%s-*direct" % wksname)) + self.assertEqual(1, len(images)) - # verify partition size with wic - res = runCmd("export PARTED_SECTOR_SIZE=%d; parted -m %s unit b p" % (wic_sector_size, images[0]), - stderr=subprocess.PIPE) + sysroot = get_bb_var('RECIPE_SYSROOT_NATIVE', 'wic-tools') + # list partitions + result = runCmd("wic ls %s -n %s" % (images[0], sysroot)) + print(result.output) + # Deprecated message + 4 lines of output: header + 3 partitions + self.assertEqual(5, len(result.output.split('\n'))) - print(res.output) - # parse parted output which looks like this: - # BYT;\n - # /var/tmp/wic/build/tmpgjzzefdd-202410281021-sda.direct:78569472B:file:4096:4096:gpt::;\n - # 1:139264B:1187839B:1048576B::emptyfat:msftdata; - # 2:1187840B:149270527B:148082688B:ext4:rofs-a:; - # 3:149270528B:297353215B:148082688B:ext4:primary:; - disk_info = res.output.splitlines()[1] - # Check sector sizes - sector_size_logical = int(disk_info.split(":")[3]) - sector_size_physical = int(disk_info.split(":")[4]) - self.assertEqual(wic_sector_size, sector_size_logical, "Logical sector size is not %d." % wic_sector_size) - self.assertEqual(wic_sector_size, sector_size_physical, "Physical sector size is not %d." % wic_sector_size) - - # It is a known issue with parsed that a 4K FAT partition does - # not have a recognized filesystem type of *fat. - part_info = res.output.splitlines()[2] - partname = part_info.split(":")[5] - parttype = part_info.split(":")[6] - self.assertEqual('emptyfat', partname) - self.assertEqual('msftdata;', parttype) - - part_info = res.output.splitlines()[3] - parttype = part_info.split(":")[4] - partname = part_info.split(":")[5] - self.assertEqual('ext4', parttype) - self.assertEqual('rofs-a', partname) - - part_info = res.output.splitlines()[4] - parttype = part_info.split(":")[4] - partname = part_info.split(":")[5] - self.assertEqual('ext4', parttype) - self.assertEqual('primary', partname) + # verify partition size with wic + res = runCmd("export PARTED_SECTOR_SIZE=%d; parted -m %s unit b p" % (wic_sector_size, images[0]), + stderr=subprocess.PIPE) - finally: - os.environ['PATH'] = oldpath + print(res.output) + # parse parted output which looks like this: + # BYT;\n + # /var/tmp/wic/build/tmpgjzzefdd-202410281021-sda.direct:78569472B:file:4096:4096:gpt::;\n + # 1:139264B:1187839B:1048576B::emptyfat:msftdata; + # 2:1187840B:149270527B:148082688B:ext4:rofs-a:; + # 3:149270528B:297353215B:148082688B:ext4:primary:; + disk_info = res.output.splitlines()[1] + # Check sector sizes + sector_size_logical = int(disk_info.split(":")[3]) + sector_size_physical = int(disk_info.split(":")[4]) + self.assertEqual(wic_sector_size, sector_size_logical, "Logical sector size is not %d." % wic_sector_size) + self.assertEqual(wic_sector_size, sector_size_physical, "Physical sector size is not %d." % wic_sector_size) + + # It is a known issue with parsed that a 4K FAT partition does + # not have a recognized filesystem type of *fat. + part_info = res.output.splitlines()[2] + partname = part_info.split(":")[5] + parttype = part_info.split(":")[6] + self.assertEqual('emptyfat', partname) + self.assertEqual('msftdata;', parttype) + + part_info = res.output.splitlines()[3] + parttype = part_info.split(":")[4] + partname = part_info.split(":")[5] + self.assertEqual('ext4', parttype) + self.assertEqual('rofs-a', partname) + + part_info = res.output.splitlines()[4] + parttype = part_info.split(":")[4] + partname = part_info.split(":")[5] + self.assertEqual('ext4', parttype) + self.assertEqual('primary', partname) def test_wic_sector_size_cli(self): """Test sector size handling via CLI option.""" - oldpath = os.environ['PATH'] - os.environ['PATH'] = get_bb_var("PATH", "wic-tools") - - try: - bitbake('core-image-minimal') - - with NamedTemporaryFile("w", suffix=".wks") as wks: - wks.writelines( - ['bootloader --ptable gpt\n', - 'part --fstype vfat --fstype vfat --label emptyfat --size 1M\n', - 'part --fstype ext4 --source rootfs --label rofs-a\n', - 'part --fstype ext4 --source rootfs --use-uuid\n']) - wks.flush() - cmd = "wic create %s -e core-image-minimal -o %s --sector-size 4096" % (wks.name, self.resultdir) - runCmd(cmd) - wksname = os.path.splitext(os.path.basename(wks.name))[0] - images = glob(os.path.join(self.resultdir, "%s-*direct" % wksname)) - self.assertEqual(1, len(images)) + bitbake('core-image-minimal') - sysroot = get_bb_var('RECIPE_SYSROOT_NATIVE', 'wic-tools') - # list partitions - result = runCmd("wic ls %s -n %s --sector-size 4096" % (images[0], sysroot)) - print(result.output) - # 4 lines of output: header + 3 partitions - self.assertEqual(4, len(result.output.split('\n'))) + with NamedTemporaryFile("w", suffix=".wks") as wks: + wks.writelines( + ['bootloader --ptable gpt\n', + 'part --fstype vfat --fstype vfat --label emptyfat --size 1M\n', + 'part --fstype ext4 --source rootfs --label rofs-a\n', + 'part --fstype ext4 --source rootfs --use-uuid\n']) + wks.flush() + cmd = "wic create %s -e core-image-minimal -o %s --sector-size 4096" % (wks.name, self.resultdir) + runCmd(cmd) + wksname = os.path.splitext(os.path.basename(wks.name))[0] + images = glob(os.path.join(self.resultdir, "%s-*direct" % wksname)) + self.assertEqual(1, len(images)) - # verify partition size with parted output - res = runCmd("export PARTED_SECTOR_SIZE=%d; parted -m %s unit b p" % (4096, images[0]), - stderr=subprocess.PIPE) + sysroot = get_bb_var('RECIPE_SYSROOT_NATIVE', 'wic-tools') + # list partitions + result = runCmd("wic ls %s -n %s --sector-size 4096" % (images[0], sysroot)) + print(result.output) + # 4 lines of output: header + 3 partitions + self.assertEqual(4, len(result.output.split('\n'))) - print(res.output) - # parse parted output which looks like this: - # BYT;\n - # /var/tmp/wic/build/tmpgjzzefdd-202410281021-sda.direct:78569472B:file:4096:4096:gpt::;\n - # 1:139264B:1187839B:1048576B::emptyfat:msftdata; - # 2:1187840B:149270527B:148082688B:ext4:rofs-a:; - # 3:149270528B:297353215B:148082688B:ext4:primary:; - disk_info = res.output.splitlines()[1] - # Check sector sizes - sector_size_logical = int(disk_info.split(":")[3]) - sector_size_physical = int(disk_info.split(":")[4]) - self.assertEqual(4096, sector_size_logical, "Logical sector size is not 4096.") - self.assertEqual(4096, sector_size_physical, "Physical sector size is not 4096.") - - # It is a known issue with parsed that a 4K FAT partition does - # not have a recognized filesystem type of *fat. - part_info = res.output.splitlines()[2] - partname = part_info.split(":")[5] - parttype = part_info.split(":")[6] - self.assertEqual('emptyfat', partname) - self.assertEqual('msftdata;', parttype) - - part_info = res.output.splitlines()[3] - parttype = part_info.split(":")[4] - partname = part_info.split(":")[5] - self.assertEqual('ext4', parttype) - self.assertEqual('rofs-a', partname) - - part_info = res.output.splitlines()[4] - parttype = part_info.split(":")[4] - partname = part_info.split(":")[5] - self.assertEqual('ext4', parttype) - self.assertEqual('primary', partname) + # verify partition size with parted output + res = runCmd("export PARTED_SECTOR_SIZE=%d; parted -m %s unit b p" % (4096, images[0]), + stderr=subprocess.PIPE) - finally: - os.environ['PATH'] = oldpath + print(res.output) + # parse parted output which looks like this: + # BYT;\n + # /var/tmp/wic/build/tmpgjzzefdd-202410281021-sda.direct:78569472B:file:4096:4096:gpt::;\n + # 1:139264B:1187839B:1048576B::emptyfat:msftdata; + # 2:1187840B:149270527B:148082688B:ext4:rofs-a:; + # 3:149270528B:297353215B:148082688B:ext4:primary:; + disk_info = res.output.splitlines()[1] + # Check sector sizes + sector_size_logical = int(disk_info.split(":")[3]) + sector_size_physical = int(disk_info.split(":")[4]) + self.assertEqual(4096, sector_size_logical, "Logical sector size is not 4096.") + self.assertEqual(4096, sector_size_physical, "Physical sector size is not 4096.") + + # It is a known issue with parsed that a 4K FAT partition does + # not have a recognized filesystem type of *fat. + part_info = res.output.splitlines()[2] + partname = part_info.split(":")[5] + parttype = part_info.split(":")[6] + self.assertEqual('emptyfat', partname) + self.assertEqual('msftdata;', parttype) + + part_info = res.output.splitlines()[3] + parttype = part_info.split(":")[4] + partname = part_info.split(":")[5] + self.assertEqual('ext4', parttype) + self.assertEqual('rofs-a', partname) + + part_info = res.output.splitlines()[4] + parttype = part_info.split(":")[4] + partname = part_info.split(":")[5] + self.assertEqual('ext4', parttype) + self.assertEqual('primary', partname) class Wic2(WicTestCase):
@@ -1472,47 +1416,41 @@ run_wic_cmd() { def test_extra_partition_space(self): native_sysroot = get_bb_var("RECIPE_SYSROOT_NATIVE", "wic-tools") - oldpath = os.environ['PATH'] - os.environ['PATH'] = get_bb_var("PATH", "wic-tools") + with NamedTemporaryFile("w", suffix=".wks") as tempf: + tempf.write("bootloader --ptable gpt\n" \ + "part --ondisk hda --size 10M --extra-partition-space 10M --fstype=ext4\n" \ + "part --ondisk hda --fixed-size 20M --extra-partition-space 10M --fstype=ext4\n" \ + "part --source rootfs --ondisk hda --extra-partition-space 10M --fstype=ext4\n" \ + "part --source rootfs --ondisk hda --fixed-size 200M --extra-partition-space 10M --fstype=ext4\n") + tempf.flush() + + _, wicimg = self._get_wic(tempf.name) - try: - with NamedTemporaryFile("w", suffix=".wks") as tempf: - tempf.write("bootloader --ptable gpt\n" \ - "part --ondisk hda --size 10M --extra-partition-space 10M --fstype=ext4\n" \ - "part --ondisk hda --fixed-size 20M --extra-partition-space 10M --fstype=ext4\n" \ - "part --source rootfs --ondisk hda --extra-partition-space 10M --fstype=ext4\n" \ - "part --source rootfs --ondisk hda --fixed-size 200M --extra-partition-space 10M --fstype=ext4\n") - tempf.flush() - - _, wicimg = self._get_wic(tempf.name) - - res = runCmd("parted -m %s unit b p" % wicimg, - native_sysroot=native_sysroot, stderr=subprocess.PIPE) - - # parse parted output which looks like this: - # BYT;\n - # /var/tmp/wic/build/tmpfwvjjkf_-201611101222-hda.direct:200MiB:file:512:512:msdos::;\n - # 1:0.00MiB:200MiB:200MiB:ext4::;\n - partlns = res.output.splitlines()[2:] - - self.assertEqual(4, len(partlns)) - - # Test for each partitions that the extra part space exists - for part in range(0, len(partlns)): - part_file = os.path.join(self.resultdir, "selftest_img.part%d" % (part + 1)) - partln = partlns[part].split(":") - self.assertEqual(7, len(partln)) - self.assertRegex(partln[3], r'^[0-9]+B$') - part_size = int(partln[3].rstrip("B")) - start = int(partln[1].rstrip("B")) / 512 - length = part_size / 512 - runCmd("dd if=%s of=%s skip=%d count=%d" % - (wicimg, part_file, start, length)) - res = runCmd("dumpe2fs %s -h | grep \"^Block count\"" % part_file) - fs_size = int(res.output.split(":")[1].strip()) * 1024 - self.assertLessEqual(fs_size + 10485760, part_size, "part file: %s" % part_file) - finally: - os.environ['PATH'] = oldpath + res = runCmd("parted -m %s unit b p" % wicimg, + native_sysroot=native_sysroot, stderr=subprocess.PIPE) + + # parse parted output which looks like this: + # BYT;\n + # /var/tmp/wic/build/tmpfwvjjkf_-201611101222-hda.direct:200MiB:file:512:512:msdos::;\n + # 1:0.00MiB:200MiB:200MiB:ext4::;\n + partlns = res.output.splitlines()[2:] + + self.assertEqual(4, len(partlns)) + + # Test for each partitions that the extra part space exists + for part in range(0, len(partlns)): + part_file = os.path.join(self.resultdir, "selftest_img.part%d" % (part + 1)) + partln = partlns[part].split(":") + self.assertEqual(7, len(partln)) + self.assertRegex(partln[3], r'^[0-9]+B$') + part_size = int(partln[3].rstrip("B")) + start = int(partln[1].rstrip("B")) / 512 + length = part_size / 512 + runCmd("dd if=%s of=%s skip=%d count=%d" % + (wicimg, part_file, start, length)) + res = runCmd("dumpe2fs %s -h | grep \"^Block count\"" % part_file) + fs_size = int(res.output.split(":")[1].strip()) * 1024 + self.assertLessEqual(fs_size + 10485760, part_size, "part file: %s" % part_file) # TODO this test could also work on aarch64 @skipIfNotArch(['i586', 'i686', 'x86_64'])
@@ -1825,45 +1763,39 @@ INITRAMFS_IMAGE = "core-image-initramfs-boot" testfile.write("test %s" % testfilename) testfile.close() - oldpath = os.environ['PATH'] - os.environ['PATH'] = get_bb_var("PATH", "wic-tools") + with NamedTemporaryFile("w", suffix=".wks") as wks: + wks.writelines([ + 'part / --source extra_partition --ondisk sda --sourceparams "name=foo" --align 4 --size 5M\n', + 'part / --source extra_partition --ondisk sda --label foo --align 4 --size 5M\n', + 'part / --source extra_partition --ondisk sda --fstype=vfat --uuid e7d0824e-cda3-4bed-9f54-9ef5312d105d --align 4 --size 5M\n', + 'part / --source extra_partition --ondisk sda --fstype=ext4 --label bar --align 4 --size 5M\n', + 'bootloader --ptable gpt\n', + ]) + wks.flush() + _, wicimg = self._get_wic(wks.name) - try: - with NamedTemporaryFile("w", suffix=".wks") as wks: - wks.writelines([ - 'part / --source extra_partition --ondisk sda --sourceparams "name=foo" --align 4 --size 5M\n', - 'part / --source extra_partition --ondisk sda --label foo --align 4 --size 5M\n', - 'part / --source extra_partition --ondisk sda --fstype=vfat --uuid e7d0824e-cda3-4bed-9f54-9ef5312d105d --align 4 --size 5M\n', - 'part / --source extra_partition --ondisk sda --fstype=ext4 --label bar --align 4 --size 5M\n', - 'bootloader --ptable gpt\n', - ]) - wks.flush() - _, wicimg = self._get_wic(wks.name) - - result = runCmd("wic ls %s -n %s" % (wicimg, sysroot)) - partls = result.output.split('\n')[1:] - - # Assert the number of partitions is correct - self.assertEqual(4, len(partls), msg="Expect 4 partitions, not %s" % result.output) - - # Fstype column from 'wic ls' should be fstype as given in the part command - for part_id, part_fs in enumerate(["fat16", "fat16", "fat16", "ext4"]): - self.assertIn(part_fs, partls[part_id]) - - # For each partition, assert expected files exist - for part, part_glob in enumerate([ - ["bar.conf"], - ["foo.conf"], - ["foobar.conf", "foobar2.conf", "bar3.conf", "bar4.conf"], - ["bar.conf", "bar2.conf"], - ]): - for part_file in part_glob: - result = runCmd("wic ls %s:%d/%s -n %s" % (wicimg, part + 1, part_file, sysroot)) - self.assertEqual(0, result.status, msg="File '%s' not found in the partition #%d" % (part_file, part)) + result = runCmd("wic ls %s -n %s" % (wicimg, sysroot)) + partls = result.output.split('\n')[1:] - self.remove_config(config) - finally: - os.environ['PATH'] = oldpath + # Assert the number of partitions is correct + self.assertEqual(4, len(partls), msg="Expect 4 partitions, not %s" % result.output) + + # Fstype column from 'wic ls' should be fstype as given in the part command + for part_id, part_fs in enumerate(["fat16", "fat16", "fat16", "ext4"]): + self.assertIn(part_fs, partls[part_id]) + + # For each partition, assert expected files exist + for part, part_glob in enumerate([ + ["bar.conf"], + ["foo.conf"], + ["foobar.conf", "foobar2.conf", "bar3.conf", "bar4.conf"], + ["bar.conf", "bar2.conf"], + ]): + for part_file in part_glob: + result = runCmd("wic ls %s:%d/%s -n %s" % (wicimg, part + 1, part_file, sysroot)) + self.assertEqual(0, result.status, msg="File '%s' not found in the partition #%d" % (part_file, part)) + + self.remove_config(config) def test_fs_types(self): """Test filesystem types for empty and not empty partitions"""
Signature · 1 line
2.50.0.173.g8b6f19ccfc3a
PATCH v6 6/6 image_types_wic: gate syslinux-native on the target, not the build host
In reply to Trevor WoernerA .wks needs the syslinux installer only when the target is x86. image_types_wic.bbclass selects syslinux-native on the build host architecture instead, so the dependency lands like this:
| x86 target | non-x86 target
-------------+------------+----------------
x86 host | yes | yes
non-x86 host | no | no
Two of those cells are wrong. An x86 image cross-built on a non-x86 host gets the target bootloader without the installer that writes it, and an x86 host building a non-x86 image builds an installer nothing uses.
Gate on the target, as wic-tools does [1]:
| x86 target | non-x86 target
-------------+------------+----------------
x86 host | yes | no
non-x86 host | yes | no
Bug 13276 [2] was an aarch64 host failing to build syslinux-native for an ARM target, at a time when syslinux was x86-only in every variant. It was closed in 2019 by gating on BUILD_ARCH [3]. Since 7273e131bfc7 [4] only syslinux's target code is x86-specific, so a host gate is no longer needed, and gating on the target keeps 13276's case fixed.
[YOCTO #16383]
[1] https://git.openembedded.org/openembedded-core/tree/meta/recipes-core/meta/wic-tools.bb
[2] https://bugzilla.yoctoproject.org/show_bug.cgi?id=13276
[3] https://git.openembedded.org/openembedded-core/commit/?id=7e2ee2b59319
[4] https://git.openembedded.org/openembedded-core/commit/?id=7273e131bfc7
AI-Generated: codex/claude-opus 5 (xhigh)
---
changes in v6:
- retitled with an image_types_wic prefix, which is now the only file it touches
- gate WKS_FILE_DEPENDS_DEFAULT with target-arch appends instead of the removed shared variable; wic-tools keeps the arch-gated DEPENDS it already had
changes in v5:
- new in v5
--- meta/classes-recipe/image_types_wic.bbclass | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
meta/classes-recipe/image_types_wic.bbclass +4 -2
@@ -118,8 +118,10 @@ do_image_wic[depends] += "wic-native:do_populate_sysroot" do_image_wic[recrdeptask] += "do_deploy" do_image_wic[deptask] += "do_image_complete" -WKS_FILE_DEPENDS_DEFAULT = '${@bb.utils.contains_any("BUILD_ARCH", [ 'x86_64', 'i686' ], "syslinux-native", "",d)}' -WKS_FILE_DEPENDS_DEFAULT += "cdrtools-native" +WKS_FILE_DEPENDS_DEFAULT = "cdrtools-native" +WKS_FILE_DEPENDS_DEFAULT:append:x86 = " syslinux-native" +WKS_FILE_DEPENDS_DEFAULT:append:x86-64 = " syslinux-native" +WKS_FILE_DEPENDS_DEFAULT:append:x86-x32 = " syslinux-native" WKS_FILE_DEPENDS_BOOTLOADERS = "" WKS_FILE_DEPENDS_BOOTLOADERS:aarch64 = "grub-efi systemd-boot" WKS_FILE_DEPENDS_BOOTLOADERS:arm = "systemd-boot"
Signature · 1 line
2.50.0.173.g8b6f19ccfc3a