Skip to contentSkip to conversation

wic: ship the tools it needs where it is packaged

PATCH New Trevor Woerner 1 participant 5 messages Jul 1

lore
Share & tools
lore
#0 lore raw

cover wic: ship the tools it needs where it is packaged

wic shells out to a number of host-side tools (parted, mkdosfs, mcopy, the mkfs.* family, mdir, sfdisk, e2fsck, resize2fs, ...) but does not declare them as dependencies. For the bitbake do_image_wic task this is covered by the wic-tools recipe. Wherever else wic is installed as a package it gets none of them, and its offline lookup (wic ls/cp/write) falls back to the host PATH, so wic works only by chance depending on what the host has installed:

wic.WicError: Can't find executable 'mcopy'

This series makes wic carry its own tool dependencies wherever it is packaged, and then takes advantage of that to offer it in the SDK:

1. move the helper-tool list shared with wic-tools into wic-tools.inc
2. have nativesdk-wic RDEPEND that list, so wic is complete wherever it is packaged
3. simplify buildtools-extended-tarball to just nativesdk-wic
4. add nativesdk-wic to the standard SDK host packagegroup

The native and target wic variants are unaffected; the tools are added to the nativesdk variant only. cdrtools (mkisofs) is native-only with no nativesdk variant and so is not included.

Trevor Woerner (4):
wic-tools.inc: add
wic: add runtime dependencies on the tools it invokes buildtools-extended-tarball: use nativesdk-wic nativesdk-packagegroup-sdk-host: add wic

 .../meta/buildtools-extended-tarball.bb       |  4 +---
 meta/recipes-core/meta/wic-tools.bb           | 15 ++++++------
 meta/recipes-core/meta/wic-tools.inc          | 24 +++++++++++++++++++
 .../nativesdk-packagegroup-sdk-host.bb        |  1 +
 meta/recipes-support/wic/wic_0.3.0.bb         | 18 ++++++++++++++
 5 files changed, 52 insertions(+), 10 deletions(-)
 create mode 100644 meta/recipes-core/meta/wic-tools.inc
Signature · 1 line
2.50.0.173.g8b6f19ccfc3a
#1 lore raw

PATCH 1/4 wic-tools.inc: add

wic invokes a range of host-side tools (parted, mkdosfs, mcopy, mkfs.*, grub-mkimage, ...) at image-creation time, depending on the partition types and bootloaders in a .wks file. The set wic could invoke is enumerated in its NATIVE_RECIPES table, and matches the helpers wic-tools lists in its DEPENDS.

Refactor that helper list into a shared wic-tools.inc (WIC_TOOLS) and switch wic-tools to it, so the canonical list lives in one place and can be reused by other recipes that need the same set.

AI-Generated: codex/claude-opus 4.8 (xhigh)

---
 meta/recipes-core/meta/wic-tools.bb  | 15 ++++++++-------
 meta/recipes-core/meta/wic-tools.inc | 24 ++++++++++++++++++++++++
 2 files changed, 32 insertions(+), 7 deletions(-)
 create mode 100644 meta/recipes-core/meta/wic-tools.inc
meta/recipes-core/meta/wic-tools.bb +8 -7
index 45fb873dd6bd..b81729d9ff1b 100644
@@ -2,13 +2,14 @@ SUMMARY = "A meta recipe to build native tools used by wic."
 
 LICENSE = "MIT"
 
-DEPENDS = "\
-           wic-native \
-           parted-native gptfdisk-native dosfstools-native \
-           mtools-native bmaptool-native grub-native cdrtools-native \
-           btrfs-tools-native squashfs-tools-native pseudo-native \
-           e2fsprogs-native util-linux-native tar-native erofs-utils-native \
-           virtual/cross-binutils \
+require wic-tools.inc
+
+# wic itself, the shared WIC_TOOLS helpers as -native build dependencies,
+# plus the extras specific to building the native sysroot: native-only
+# cdrtools, pseudo and the cross binutils.
+DEPENDS = "wic-native \
+           ${@' '.join('%s-native' % t for t in d.getVar('WIC_TOOLS').split())} \
+           cdrtools-native pseudo-native virtual/cross-binutils \
            "
 DEPENDS:append:x86 = " syslinux-native syslinux grub-efi systemd-boot"
 DEPENDS:append:x86-64 = " syslinux-native syslinux grub-efi systemd-boot"
meta/recipes-core/meta/wic-tools.inc new file +24 -0
new file mode 100644
index 000000000000..a1d622e9d345
@@ -0,0 +1,24 @@
+# Recipes providing the host-side tools wic may invoke at image-creation
+# time, as listed in wic's NATIVE_RECIPES table. Names are given without
+# a class suffix so they can be mapped to -native, nativesdk- or any
+# other variant as needed.
+#
+# This list is limited to tools that exist both as -native and as
+# nativesdk- packages. Tools that are not available in every variant
+# (for example native-only cdrtools) and architecture-gated tools (for
+# example syslinux) are intentionally left out and handled separately.
+
+WIC_TOOLS = "\
+    parted \
+    gptfdisk \
+    dosfstools \
+    mtools \
+    bmaptool \
+    grub \
+    btrfs-tools \
+    squashfs-tools \
+    e2fsprogs \
+    util-linux \
+    tar \
+    erofs-utils \
+"
Signature · 1 line
2.50.0.173.g8b6f19ccfc3a
#2 lore raw

PATCH 2/4 wic: add runtime dependencies on the tools it invokes

In reply to Trevor Woerner

wic shells out to a number of host-side tools (parted, mkdosfs, mcopy, the mkfs.* family, mdir, sfdisk, e2fsck, resize2fs, debugfs, blkid, ...) but does not declare them as dependencies. For the bitbake do_image_wic task this is handled by the wic-tools recipe, which builds those tools into the native sysroot. Wherever else wic is installed as a package it gets none of them, and its offline lookup (wic ls/cp/write) falls back to the host PATH, so wic works only by chance depending on what the host has installed:

wic.WicError: Can't find executable 'mcopy'

Make wic carry the tools it may invoke by adding the shared WIC_TOOLS helpers as RDEPENDS. They are added to the nativesdk variant, which is where wic is packaged for use outside a bitbake build; the native and target variants are unaffected.

resize2fs is added explicitly: e2fsprogs splits it into its own e2fsprogs-resize2fs package, so nativesdk-e2fsprogs alone does not provide it, and it is needed by the wic cp/write resize path.

syslinux (with its isohybrid helper) is gated to x86 hosts; cdrtools (mkisofs) is native-only with no nativesdk variant and is only needed for ISO images, so it cannot be included.

AI-Generated: codex/claude-opus 4.8 (xhigh)

---
 meta/recipes-support/wic/wic_0.3.0.bb | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)
meta/recipes-support/wic/wic_0.3.0.bb +18 -0
index 7dbf84b039a6..6efe974e02bd 100644
@@ -10,6 +10,8 @@ CVE_PRODUCT = "yoctoproject:wic"
 
 inherit python_hatchling
 
+require recipes-core/meta/wic-tools.inc
+
 RDEPENDS:${PN} += " \
     python3-core \
     python3-json \
@@ -17,4 +19,20 @@ RDEPENDS:${PN} += " \
     python3-misc \
     "
 
+# wic shells out to the WIC_TOOLS tools but does not declare them, so
+# wherever wic is installed as a package it may not have them and its
+# offline lookup falls back to the host PATH. Pull them in for the
+# nativesdk variant so they are present regardless of the host; native
+# and target wic are unaffected.
+RDEPENDS:${PN}:append:class-nativesdk = "${@' '.join(' nativesdk-%s' % t for t in d.getVar('WIC_TOOLS').split())}"
+
+# resize2fs is split into its own e2fsprogs-resize2fs package, so
+# nativesdk-e2fsprogs alone does not provide it.
+RDEPENDS:${PN}:append:class-nativesdk = " nativesdk-e2fsprogs-resize2fs"
+
+# syslinux (and its isohybrid helper, in the -misc package) only builds
+# for x86 hosts; cdrtools (mkisofs) is native-only with no nativesdk
+# variant and so cannot be included.
+RDEPENDS:${PN}:append:class-nativesdk = "${@' nativesdk-syslinux nativesdk-syslinux-misc' if d.getVar('SDK_ARCH') in ['x86_64', 'i686'] else ''}"
+
 BBCLASSEXTEND = "native nativesdk"
Signature · 1 line
2.50.0.173.g8b6f19ccfc3a
#3 lore raw

PATCH 3/4 buildtools-extended-tarball: use nativesdk-wic

In reply to Trevor Woerner

The extended buildtools tarball carried nativesdk-parted, -dosfstools and -gptfdisk so that "wic ls" and friends could work on an image. That list predated wic becoming a recipe, so it provided some of wic's helper tools but not wic itself, nor the rest of the tools wic could invoke.

nativesdk-wic now pulls in the tools it needs on its own, so replace those three entries with nativesdk-wic. The tarball ships a working wic with its full tool set instead of a partial set of its dependencies.

AI-Generated: codex/claude-opus 4.8 (xhigh)

---
 meta/recipes-core/meta/buildtools-extended-tarball.bb | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)
meta/recipes-core/meta/buildtools-extended-tarball.bb +1 -3
index 633f8e6b99d6..9dddd70a2ce5 100644
@@ -30,9 +30,7 @@ TOOLCHAIN_HOST_TASK += "\
     nativesdk-glibc-utils \
     nativesdk-glibc-gconvs \
     nativesdk-libxcrypt-dev \
-    nativesdk-parted \
-    nativesdk-dosfstools \
-    nativesdk-gptfdisk \
+    nativesdk-wic \
     "
 # gconvs needed for iconv to work in vim builds
 
Signature · 1 line
2.50.0.173.g8b6f19ccfc3a
#4 lore raw

PATCH 4/4 nativesdk-packagegroup-sdk-host: add wic

In reply to Trevor Woerner

wic is useful from an installed SDK for assembling or inspecting a wic image, but it is not part of the host packagegroup, so a user who wants it has to add it themselves. Add nativesdk-wic so it is available by default.

AI-Generated: codex/claude-opus 4.8 (xhigh)

---
 .../packagegroups/nativesdk-packagegroup-sdk-host.bb             | 1 +
 1 file changed, 1 insertion(+)
meta/recipes-core/packagegroups/nativesdk-packagegroup-sdk-host.bb +1 -0
index 51e48f250b19..a8e59de4bca0 100644
@@ -29,6 +29,7 @@ RDEPENDS:${PN} = "\
     nativesdk-bison \
     nativesdk-flex \
     nativesdk-perl-module-integer \
+    nativesdk-wic \
     "
 
 RDEPENDS:${PN}:darwin = "\
Signature · 1 line
2.50.0.173.g8b6f19ccfc3a