Skip to content

Implement 'containers' jobs

Tim Orling yocto-patches 2 versions – New

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v5 Latest12 patches 0 repliesNew
  2. v411 patches 2 repliesSuperseded

Changes in v5 from the cover letter

* containers-library: Drop alpine single-arch container as we do not want to imply that building Yocto Project packages on top of alpine is officially supported. This does mean the single-arch workflow is no longer being utilized, but a more appropriate single-arch container can be added later.
* Remove NO_BUILDTOOLS = "1" from vcontainer-tests job as we need python 3.10+ for pytest 9.x (and buildtools works alongside the vcontainer-tarball).
* Add VCONTAINER_TARBALL_URL as path to the latest built vcontainer- tarball until some future date when we decide if we want to use a release build. This will be used for future run-time container tests or future pushing of single-arch containers.
* The container recipe series has been merged in meta-virtualization:
8e33959d container-nonroot-user: document + assert inherit order vs image-oci 6d464f78 app-container-mosquitto: fix PACKAGECONFIG 'dev' comment 8fe9331f app-container-valkey: drop redundant OCI_IMAGE_RUNTIME_UID faf960ab vcontainer-tarball: fix buildbot-venv shadowing ce767999 container-image-multiarch: add helper recipe
be1eafbf app-container-curl: use multilayer mode; container-nonroot-user 712f9695 vcontainer-bbmask.inc: allow meta-webserver/recipes-httpd/nginx 5af4f1cb recipes-containers/images: add app-container-nginx 24d5e836 recipes-containers/images: add app-container-valkey 0d277995 vcontainer-bbmask.inc: allow 'mosquitto', 'libwebsockets' a3190a33 recipes-containers/images: add app-container-mosquitto b525f916 recipes-containers/images: add app-container-python 09e92d14 classes: add container-dev-mode.bbclass
f03ae95e classes: add container-volatile-fixup.bbclass
a250220d classes: add container-nonroot-user.bbclass
fdbc054d image-oci: set OCI_IMAGE_RUNTIME_UID with ??=

Patches in v5 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/12scripts: add vcontainer-tarball setup, integration, and publishingscripts/publish-artefacts scripts/run-config scripts/utils.py+32 -40 replies
2/12config.json: add vcontainer-tarball build targetconfig.json+25 -00 replies
3/12scripts: add run-vcontainer-tests for meta-virtualizationconfig.json scripts/run-vcontainer-tests+240 -00 replies
4/12scripts: add container registry push, auth, tagging, runtime selectionconfig.json scripts/run-config scripts/run-push-containers+201 -80 replies
5/12config.json: add 'containers-library' build jobconfig.json+59 -00 replies
6/12scripts/run-push-containers: push multiarch containers with skopeo-nativescripts/run-push-containers+77 -230 replies
7/12containers-library: switch to multiarchconfig.json+14 -140 replies
8/12run-push-containers: conditionally sign pushed containers with cosignconfig.json scripts/run-push-containers+113 -30 replies
9/12scripts/run-push-containers: add SPDX SBOM attestationscripts/run-push-containers+104 -20 replies
10/12config.json: add -dev container builds to containers-libraryconfig.json+50 -90 replies
11/12run-push-containers: optionally push -dev tagged containersscripts/run-push-containers+26 -00 replies
12/12config.json: set VCONTAINER_TARBALL_URLconfig.json+2 -00 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v5 yet.

Read the v5 thread

Cover letter [yocto-autobuilder-helper][PATCH v5 00/12] Implement 'containers' jobs

This series builds on top of the patches already merged from v2 and previously submitted as v3 and v4.

This series contains the 'yocto-autobuilder-helper' patches to enable:
* building, testing and publishing of 'vcontainer-tarball'
- vcontainer-tests job
* building and pushing containers from:
- meta-virtualization
* containers are pushed when the push_containers boolean is set in the AutoBuilder UI
- regstries are set with CONTAINER_REGISTRIES variable
* authentication uses local user credentials (e.g. ~/.docker/config.json or .../auth.json for podman)
* containers are tagged with PV_MAJOR, PV_MAJOR.PV_MINOR, DISTRO_CODENAME and yocto-x.y release (yocto-x.y.z for release builds).
* multiarch containers built for x86-64 and aarch64
* signing of containers
* attaching SPDX 3.0 SBOM attestation (and signing it) for pushed containers
* 'dev' mode containers which run as 'root' user and in some cases add a shell. These are tagged as above but with a '-dev' suffix.

Additional features, such as attaching SLSA provenance attestations will come in a future series.

Concerns about the possible infinite loop around the tarball extraction "while True, try, expect OSError" code [1] are addressed separately [2], since the original patch has been merged.

This series relies on the resolution of [3] for the vcontainer-tests to be more reliable.

This series was built on the Yocto Project AB in [4][5][6].

[1] https://lore.kernel.org/yocto-patches/[email protected]/
[2] https://lore.kernel.org/yocto-patches/[email protected]/
[3] https://lore.kernel.org/yocto-meta-virtualization/[email protected]/
[4] vcontainer-tarball: https://autobuilder.yoctoproject.org/valkyrie/#/builders/116/builds/11
[5] vcontainer-tests: https://autobuilder.yoctoproject.org/valkyrie/#/builders/118/builds/12
[6] containers-library: https://autobuilder.yoctoproject.org/valkyrie/#/builders/117/builds/27

Changes in v2:
* Simplify by merging vdkr-tests and vpdmn-tests into vcontainer-tests
* Simplify by building containers from meta-virtualization:
- Depends on resolution of: https://lists.yoctoproject.org/g/meta-virtualization/message/9826
* Workaround for recent (since Friday May 29, 2026) errors:
Error: reading blob sha256:<hash>: file integrity checksum failed for "<file>"
- This is probably related to either sstate changes or recent changes
in vcontainer-common...

Changes in v3:
* vcontainer-tarball is installed in jobs that set "vcontainer" similar to "extratools", instead of the more global "buildtools" behavior.
* The (rather large amount of) code for pushing containers in run-config is broken out into a dedicated run-push-containers script.
* Workaround for pushing containers is moved to run-push-containers. The most recent behavior (on top of meta-virt master-next) showed a different error pattern:
Error: reading blob sha256:<hash>: EOF
The workaround simply removes all container images from the VM's container-registry before importing or pushing new container images. The commit message was reworded accordingly.
* Rather than installing latest versions of pip, setuptools and wheel, silence the pip warning with PIP_DISABLE_PIP_VERSION_CHECK=1
* Rather than installing the latest versions of pytest, pytest-timeout and pexpect, install from meta-virtualization/tests/requirements.txt.
* Refactor run-vcontainer-tests script to use optargs instead of a mix of positional args and env vars.

Changes in v4:
* Adjusted the 'vcontainer' installation of the vcontainer-tarball to only apply to 'push-containers' step, as we currently only need the functionality to 'vimport' and push single-arch containers.
* Added app-container-alpine as a single-arch container example
* Switched other container builds to multi-arch, building both x86-64 and aarch64 flavors
* Added conditional signing of containers in the run-push-containers script, this is gated by CONTAINER_COSIGN_KEY being set to the path to a 'cosign' private key.
* Added conditional attachment of SPDX 3.0 SBOM attestation (when CONTAINER_COSIGN_KEY is defined).
* Added conditional verification of the SPDX 3.0 SBOM attestation, which is gated by CONTAINER_COSIGN_PUB being set to the path to a 'cosign' public key.
* Added building of '-dev' mode containers, with PACKAGECONFIG 'dev', which run as 'root' user instead of the 'nonroot' user and in some cases conditionally add a shell (depends on the container image recipe).
* Added pushing of '-dev'containers adding a '-dev' suffix to the tags.

Changes in v5:
* containers-library: Drop alpine single-arch container as we do not want to imply that building Yocto Project packages on top of alpine is officially supported. This does mean the single-arch workflow is no longer being utilized, but a more appropriate single-arch container can be added later.
* Remove NO_BUILDTOOLS = "1" from vcontainer-tests job as we need python 3.10+ for pytest 9.x (and buildtools works alongside the vcontainer-tarball).
* Add VCONTAINER_TARBALL_URL as path to the latest built vcontainer- tarball until some future date when we decide if we want to use a release build. This will be used for future run-time container tests or future pushing of single-arch containers.
* The container recipe series has been merged in meta-virtualization:
8e33959d container-nonroot-user: document + assert inherit order vs image-oci 6d464f78 app-container-mosquitto: fix PACKAGECONFIG 'dev' comment 8fe9331f app-container-valkey: drop redundant OCI_IMAGE_RUNTIME_UID faf960ab vcontainer-tarball: fix buildbot-venv shadowing ce767999 container-image-multiarch: add helper recipe
be1eafbf app-container-curl: use multilayer mode; container-nonroot-user 712f9695 vcontainer-bbmask.inc: allow meta-webserver/recipes-httpd/nginx 5af4f1cb recipes-containers/images: add app-container-nginx 24d5e836 recipes-containers/images: add app-container-valkey 0d277995 vcontainer-bbmask.inc: allow 'mosquitto', 'libwebsockets' a3190a33 recipes-containers/images: add app-container-mosquitto b525f916 recipes-containers/images: add app-container-python 09e92d14 classes: add container-dev-mode.bbclass
f03ae95e classes: add container-volatile-fixup.bbclass
a250220d classes: add container-nonroot-user.bbclass
fdbc054d image-oci: set OCI_IMAGE_RUNTIME_UID with ??=

Tim Orling (12):
scripts: add vcontainer-tarball setup, integration, and publishing config.json: add vcontainer-tarball build target
scripts: add run-vcontainer-tests for meta-virtualization
scripts: add container registry push, auth, tagging, runtime selection config.json: add 'containers-library' build job scripts/run-push-containers: push multiarch containers with
skopeo-native
containers-library: switch to multiarch
run-push-containers: conditionally sign pushed containers with cosign scripts/run-push-containers: add SPDX SBOM attestation config.json: add -dev container builds to containers-library
run-push-containers: optionally push -dev tagged containers config.json: set VCONTAINER_TARBALL_URL

 config.json                  | 161 ++++++++++++
 scripts/publish-artefacts    |   5 +
 scripts/run-config           |  19 ++
 scripts/run-push-containers  | 471 +++++++++++++++++++++++++++++++++++
 scripts/run-vcontainer-tests | 212 ++++++++++++++++
 scripts/utils.py             |  20 +-
 6 files changed, 884 insertions(+), 4 deletions(-)
 create mode 100755 scripts/run-push-containers
 create mode 100755 scripts/run-vcontainer-tests
Signature · 1 line
2.55.0