Skip to contentSkip to conversation

Add support for OCI rootfs images

PATCH Applied Matthew Weber via buildroot 3 participants 10 messages Aug 27, 2021 – Sep 11, 2021 VFS

lore
Share & tools
lore
#0 lore raw

v3 cover Add support for OCI rootfs images

This patch set adds support for OCI images.

The OCI image format is a specification for container images defined and maintained as an open standard by the Open Container Initiative [1].

To generate the image according to the specification [2], a tool called sloci-image is used.

The OCI image is generated (by default) in a directory called rootfs-oci:

$ ls output/images/rootfs-oci/
blobs  index.json  oci-layout

It can be pushed to a container registry using tools like skopeo:

$ skopeo copy --dest-creds <user>:<pass> oci:rootfs-oci:<tag> \
         docker://<user>/<image>[:tag]

And then we can pull/run the container image with tools like docker:

$ docker run --rm -it <user>/<image>[:tag]

In v3 of this series, a runtime test has been added where the OCI archive is imported into a containerd instance and executed. This verifies the full workflow and use at runtime. ./support/testing/tests/fs/test_oci.py

[1] https://opencontainers.org/
[2] https://github.com/opencontainers/image-spec/blob/master/spec.md

---
Changes
v2 -> v3
[Matt]
- Patched sloci tar archive creation to remove nested folder (merged upstream but not yet in release)
- Added runtime testing of OCI tar archive using containerd direct import

v1 -> v2:
[Sergio]
- bump sloci-image version to 0.1.1 abd drop patch applied upstream
- improve usage comments
- fix check-package warnings

Matthew Weber (1):
support/testing/tests/fs: test OCI format

Sergio Prado (2):
package/sloci-image: new host package
fs: new OCI filesystem type

 DEVELOPERS                                    |  1 +
 board/qemu/aarch64-virt/linux.config          |  6 ++
 fs/Config.in                                  |  1 +
 fs/oci/Config.in                              | 88 +++++++++++++++++
 fs/oci/oci.mk                                 | 99 +++++++++++++++++++
 package/Config.in.host                        |  2 +
 ...-from-the-contents-of-working-folder.patch | 37 +++++++
 package/sloci-image/Config.in.host            |  7 ++
 package/sloci-image/sloci-image.hash          |  5 +
 package/sloci-image/sloci-image.mk            | 19 ++++
 support/testing/tests/fs/test_oci.py          | 56 +++++++++++
 11 files changed, 321 insertions(+)
 create mode 100644 fs/oci/Config.in
 create mode 100644 fs/oci/oci.mk
 create mode 100644 package/sloci-image/0001-create-OCI-tar-from-the-contents-of-working-folder.patch
 create mode 100644 package/sloci-image/Config.in.host
 create mode 100644 package/sloci-image/sloci-image.hash
 create mode 100644 package/sloci-image/sloci-image.mk
 create mode 100644 support/testing/tests/fs/test_oci.py
Signature · 6 lines
2.17.1

_______________________________________________
buildroot mailing list
[email protected]
http://lists.busybox.net/mailman/listinfo/buildroot
#1 lore raw

PATCH v3 1/3 package/sloci-image: new host package

From: Sergio Prado <[email protected]>

sloci-image is a simple CLI tool for packing rootfs into a single-layer OCI image.

---
 DEVELOPERS                                    |  1 +
 package/Config.in.host                        |  2 +
 ...-from-the-contents-of-working-folder.patch | 37 +++++++++++++++++++
 package/sloci-image/Config.in.host            |  7 ++++
 package/sloci-image/sloci-image.hash          |  5 +++
 package/sloci-image/sloci-image.mk            | 19 ++++++++++
 6 files changed, 71 insertions(+)
 create mode 100644 package/sloci-image/0001-create-OCI-tar-from-the-contents-of-working-folder.patch
 create mode 100644 package/sloci-image/Config.in.host
 create mode 100644 package/sloci-image/sloci-image.hash
 create mode 100644 package/sloci-image/sloci-image.mk
DEVELOPERS +1 -0
index 53448dd100..af684021f4 100644
@@ -2425,6 +2425,7 @@ F:	package/curlpp/
 F:	package/daq/
 F:	package/libgdiplus/
 F:	package/pimd/
+F:	package/sloci-image/
 F:	package/snort/
 F:	package/stella/
 F:	package/tio/
package/Config.in.host +2 -0
index 551352482e..209b1ff138 100644
@@ -78,7 +78,9 @@ menu "Host utilities"
 	source "package/rustc/Config.in.host"
 	source "package/s6-rc/Config.in.host"
 	source "package/sam-ba/Config.in.host"
+
 	source "package/sdbusplus/Config.in.host"
+	source "package/sloci-image/Config.in.host"
 	source "package/squashfs/Config.in.host"
 	source "package/sunxi-tools/Config.in.host"
 	source "package/swig/Config.in.host"
package/sloci-image/0001-create-OCI-tar-from-the-contents-of-working-folder.patch new file +37 -0
new file mode 100644
index 0000000000..3d048e3843
@@ -0,0 +1,37 @@
+From 66c73c2ba3057634f19f37d3cfbbe752638c9f9f Mon Sep 17 00:00:00 2001
+From: Matthew Weber <[email protected]>
+Date: Fri, 27 Aug 2021 12:24:19 -0500
+Subject: [PATCH] create OCI tar from the contents of working folder
+
+When testing with containerd, it was found that the tarfile
+created with sloci can't import because the contents are inside
+a folder.  Looking around online, it seems that the an oci tar
+should container the blob/index.json/oci-layout in the . of the
+tarfile.
+
+Upstream: https://github.com/jirutka/sloci-image/pull/3
+
+Refs:
+https://snyk.io/blog/container-image-formats/
+https://github.com/opencontainers/image-spec/blob/main/image-layout.md
+
+Signed-off-by: Matthew Weber <[email protected]>
+---
+ sloci-image | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/sloci-image b/sloci-image
+index 90dba26..7f3c775 100755
+--- a/sloci-image
++++ b/sloci-image
+@@ -425,6 +425,6 @@ if [ "$OUT_TYPE" = tar ]; then
+ 	file_name="$IMAGE_NAME-$CFG_REF_NAME-$CFG_ARCH"
+ 	file_name="$file_name${CFG_ARCH_VARIANT:+"-$CFG_ARCH_VARIANT"}-$CFG_OS.oci-image"
+ 
+-	tar ${DEBUG:+-v} -cf "$file_name.tar" "$IMAGE_NAME"
++	tar ${DEBUG:+-v} -cf "$file_name.tar" -C "$IMAGE_NAME" .
+ 	rm -Rf "$IMAGE_NAME"
+ fi
+-- 
+2.17.1
+
package/sloci-image/Config.in.host new file +7 -0
new file mode 100644
index 0000000000..d809f1f3c7
@@ -0,0 +1,7 @@
+config BR2_PACKAGE_HOST_SLOCI_IMAGE
+	bool "host sloci-image"
+	help
+	  A simple CLI tool for packing rootfs into a single-layer OCI
+	  image.
+
+	  https://github.com/jirutka/sloci-image
package/sloci-image/sloci-image.hash new file +5 -0
new file mode 100644
index 0000000000..117f0ba6e0
@@ -0,0 +1,5 @@
+# Locally computed
+sha256  56fbc335c193eccc3992f0a723c6dabe762f6d4854fc52059be074bdf5060d9e  sloci-image-0.1.1.tar.gz
+
+# Hash for license files:
+sha256  8ea53673b084576813fc40cd63817c5a1619438942b5e6c30dac1d10707c27e4  LICENSE
package/sloci-image/sloci-image.mk new file +19 -0
new file mode 100644
index 0000000000..f6db7e2060
@@ -0,0 +1,19 @@
+################################################################################
+#
+# sloci-image
+#
+################################################################################
+
+SLOCI_IMAGE_VERSION = 0.1.1
+SLOCI_IMAGE_SITE = $(call github,jirutka,sloci-image,v$(SLOCI_IMAGE_VERSION))
+
+SLOCI_IMAGE_LICENSE = MIT
+SLOCI_IMAGE_LICENSE_FILES = LICENSE
+
+HOST_SLOCI_IMAGE_DEPENDENCIES = host-gawk
+
+define HOST_SLOCI_IMAGE_INSTALL_CMDS
+	$(HOST_MAKE_ENV) $(MAKE) DESTDIR=$(HOST_DIR) PREFIX=/usr -C $(@D) install
+endef
+
+$(eval $(host-generic-package))
Signature · 6 lines
2.17.1

_______________________________________________
buildroot mailing list
[email protected]
http://lists.busybox.net/mailman/listinfo/buildroot
#2 lore raw

Re: [Buildroot] [PATCH v3 1/3] package/sloci-image: new host package

Quoted 25 lines · Matthew Weber via buildroot

From: Sergio Prado <[email protected]>

sloci-image is a simple CLI tool for packing rootfs into a single-layer OCI image.

---
 DEVELOPERS                                    |  1 +
 package/Config.in.host                        |  2 +
 ...-from-the-contents-of-working-folder.patch | 37 +++++++++++++++++++
 package/sloci-image/Config.in.host            |  7 ++++
 package/sloci-image/sloci-image.hash          |  5 +++
 package/sloci-image/sloci-image.mk            | 19 ++++++++++
 6 files changed, 71 insertions(+)
 create mode 100644 package/sloci-image/0001-create-OCI-tar-from-the-contents-of-working-folder.patch
 create mode 100644 package/sloci-image/Config.in.host
 create mode 100644 package/sloci-image/sloci-image.hash
 create mode 100644 package/sloci-image/sloci-image.mk
DEVELOPERS +0 -0
index 53448dd100..af684021f4 100644

DEVELOPERS F: package/curlpp/ · line 2428

Quoted 4 lines
DEVELOPERS +1 -0
@@ -2425,6 +2425,7 @@ F:	package/curlpp/
 F:	package/daq/
 F:	package/libgdiplus/
 F:	package/pimd/
+F:	package/sloci-image/

Matt, I took the liberty of adding sloci-image to your entry as well.

package/Config.in.host menu "Host utilities" · line 81

Quoted 12 lines
DEVELOPERS +0 -0
 F:	package/snort/
 F:	package/stella/
 F:	package/tio/
package/Config.in.host +1 -0
index 551352482e..209b1ff138 100644
@@ -78,7 +78,9 @@ menu "Host utilities"
 	source "package/rustc/Config.in.host"
 	source "package/s6-rc/Config.in.host"
 	source "package/sam-ba/Config.in.host"
+

Spurious empty line.

Quoted 13 lines
package/Config.in.host +1 -0
 	source "package/sdbusplus/Config.in.host"
+	source "package/sloci-image/Config.in.host"
 	source "package/squashfs/Config.in.host"
 	source "package/sunxi-tools/Config.in.host"
 	source "package/swig/Config.in.host"
package/sloci-image/0001-create-OCI-tar-from-the-contents-of-working-folder.patch new file +2 -0
new file mode 100644
index 0000000000..3d048e3843
@@ -0,0 +1,37 @@
+From 66c73c2ba3057634f19f37d3cfbbe752638c9f9f Mon Sep 17 00:00:00 2001
+From: Matthew Weber <[email protected]>

package/sloci-image/0001-create-OCI-tar-from-the-contents-of-working-folder.patch line 12

Quoted 10 lines
package/sloci-image/0001-create-OCI-tar-from-the-contents-of-working-folder.patch new file +10 -0
+Date: Fri, 27 Aug 2021 12:24:19 -0500
+Subject: [PATCH] create OCI tar from the contents of working folder
+
+When testing with containerd, it was found that the tarfile
+created with sloci can't import because the contents are inside
+a folder.  Looking around online, it seems that the an oci tar
+should container the blob/index.json/oci-layout in the . of the
+tarfile.
+
+Upstream: https://github.com/jirutka/sloci-image/pull/3

It was accepted, great!

Quoted 55 lines
package/sloci-image/0001-create-OCI-tar-from-the-contents-of-working-folder.patch new file +25 -0
+
+Refs:
+https://snyk.io/blog/container-image-formats/
+https://github.com/opencontainers/image-spec/blob/main/image-layout.md
+
+Signed-off-by: Matthew Weber <[email protected]>
+---
+ sloci-image | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/sloci-image b/sloci-image
+index 90dba26..7f3c775 100755
+--- a/sloci-image
++++ b/sloci-image
+@@ -425,6 +425,6 @@ if [ "$OUT_TYPE" = tar ]; then
+ 	file_name="$IMAGE_NAME-$CFG_REF_NAME-$CFG_ARCH"
+ 	file_name="$file_name${CFG_ARCH_VARIANT:+"-$CFG_ARCH_VARIANT"}-$CFG_OS.oci-image"
+ 
+-	tar ${DEBUG:+-v} -cf "$file_name.tar" "$IMAGE_NAME"
++	tar ${DEBUG:+-v} -cf "$file_name.tar" -C "$IMAGE_NAME" .
+ 	rm -Rf "$IMAGE_NAME"
+ fi
+-- 
+2.17.1
+
package/sloci-image/Config.in.host new file +7 -0
new file mode 100644
index 0000000000..d809f1f3c7
@@ -0,0 +1,7 @@
+config BR2_PACKAGE_HOST_SLOCI_IMAGE
+	bool "host sloci-image"
+	help
+	  A simple CLI tool for packing rootfs into a single-layer OCI
+	  image.
+
+	  https://github.com/jirutka/sloci-image
package/sloci-image/sloci-image.hash new file +5 -0
new file mode 100644
index 0000000000..117f0ba6e0
@@ -0,0 +1,5 @@
+# Locally computed
+sha256  56fbc335c193eccc3992f0a723c6dabe762f6d4854fc52059be074bdf5060d9e  sloci-image-0.1.1.tar.gz
+
+# Hash for license files:
+sha256  8ea53673b084576813fc40cd63817c5a1619438942b5e6c30dac1d10707c27e4  LICENSE
package/sloci-image/sloci-image.mk new file +0 -0
new file mode 100644
index 0000000000..f6db7e2060

package/sloci-image/sloci-image.mk line 7

Quoted 7 lines
package/sloci-image/sloci-image.mk new file +7 -0
@@ -0,0 +1,19 @@
+################################################################################
+#
+# sloci-image
+#
+################################################################################
+
+SLOCI_IMAGE_VERSION = 0.1.1

It doesn't look like upstream is going to make a release any time soon, so you
could just as well update to the most recent sha1 and drop the patch. Oh well, I've applied as is.

package/sloci-image/sloci-image.mk line 16

Quoted 9 lines
package/sloci-image/sloci-image.mk new file +9 -0
+SLOCI_IMAGE_SITE = $(call github,jirutka,sloci-image,v$(SLOCI_IMAGE_VERSION))
+
+SLOCI_IMAGE_LICENSE = MIT
+SLOCI_IMAGE_LICENSE_FILES = LICENSE
+
+HOST_SLOCI_IMAGE_DEPENDENCIES = host-gawk
+
+define HOST_SLOCI_IMAGE_INSTALL_CMDS
+	$(HOST_MAKE_ENV) $(MAKE) DESTDIR=$(HOST_DIR) PREFIX=/usr -C $(@D) install

This is wrong! For host packages, DESTDIR should be empty and PREFIX should be
set to $(HOST_DIR) without /usr. It doesn't make a difference in practice because PREFIX is only used to construct the install path and the usr symlink makes it work, but it's the principle!

I fixed all of the above and applied to master, thanks.

Regards,
Arnout

package/sloci-image/sloci-image.mk line 19

Quoted 4 lines
package/sloci-image/sloci-image.mk new file +3 -0
+endef
+
+$(eval $(host-generic-package))

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#3 lore raw

PATCH v3 2/3 fs: new OCI filesystem type

In reply to Matthew Weber via buildroot

From: Sergio Prado <[email protected]>

Add support to generate OCI (Open Container Initiative) images.

An OCI image consists of a manifest, an image index (optional), a set of filesystem layers, and a configuration. The complete specification is available in the link below:

https://github.com/opencontainers/image-spec/blob/master/spec.md

The image is generated with the host tool sloci-image, and config options can be used to configure image parameters.

By default, the image is generated in a directory called rootfs-oci:

$ cd output/images
$ ls rootfs-oci/
blobs  index.json  oci-layout

Optionally, the image can be packed into a tar archive.

The image can be pushed to a registry using containers tools like skopeo:

$ skopeo copy --dest-creds <user>:<pass> oci:rootfs-oci:<tag> \
	docker://<user>/<image>[:tag]

And then we can pull/run the container image with tools like docker:

$ docker run -it <user>/<image>[:tag]
---
 fs/Config.in     |  1 +
 fs/oci/Config.in | 88 ++++++++++++++++++++++++++++++++++++++++++
 fs/oci/oci.mk    | 99 ++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 188 insertions(+)
 create mode 100644 fs/oci/Config.in
 create mode 100644 fs/oci/oci.mk
fs/Config.in +1 -0
index 37a2aa21f8..eee5e26bb2 100644
@@ -11,6 +11,7 @@ source "fs/f2fs/Config.in"
 source "fs/initramfs/Config.in"
 source "fs/iso9660/Config.in"
 source "fs/jffs2/Config.in"
+source "fs/oci/Config.in"
 source "fs/romfs/Config.in"
 source "fs/squashfs/Config.in"
 source "fs/tar/Config.in"
fs/oci/Config.in new file +88 -0

88 lines not on this page · raw message

fs/oci/oci.mk new file +99 -0

99 lines not on this page · raw message

Signature · 6 lines
2.17.1

_______________________________________________
buildroot mailing list
[email protected]
http://lists.busybox.net/mailman/listinfo/buildroot
#4 lore raw

Re: [Buildroot] [PATCH v3 2/3] fs: new OCI filesystem type

Quoted 33 lines · Matthew Weber via buildroot

From: Sergio Prado <[email protected]>

Add support to generate OCI (Open Container Initiative) images.

An OCI image consists of a manifest, an image index (optional), a set of filesystem layers, and a configuration. The complete specification is available in the link below:

https://github.com/opencontainers/image-spec/blob/master/spec.md

The image is generated with the host tool sloci-image, and config options can be used to configure image parameters.

By default, the image is generated in a directory called rootfs-oci:

$ cd output/images
$ ls rootfs-oci/
blobs  index.json  oci-layout

Optionally, the image can be packed into a tar archive.

The image can be pushed to a registry using containers tools like skopeo:

$ skopeo copy --dest-creds <user>:<pass> oci:rootfs-oci:<tag> \
	docker://<user>/<image>[:tag]

And then we can pull/run the container image with tools like docker:

$ docker run -it <user>/<image>[:tag]

[snip]

Quoted 18 lines
fs/oci/Config.in new file +18 -0
+
+config BR2_TARGET_ROOTFS_OCI_ENTRYPOINT
+	string "entrypoint"
+	default "sh"
+	help
+	  Command to execute when the container starts.
+
+config BR2_TARGET_ROOTFS_OCI_ENTRYPOINT_ARGS
+	string "entrypoint arguments"
+	help
+	  Default arguments to the entrypoint of the container.
+
+config BR2_TARGET_ROOTFS_OCI_WORKDIR
+	string "working directory"
+	help
+	  Working directory of the entrypoint process in the
+	  container.
+

fs/oci/Config.in line 64

Quoted 10 lines
fs/oci/Config.in new file +10 -0
+config BR2_TARGET_ROOTFS_OCI_UID
+	string "username or UID"
+	default "0"
+	help
+	  The username or UID of user the process run as.
+
+config BR2_TARGET_ROOTFS_OCI_ENV_VARS
+	string "environment variables"
+	help
+	  Default environment variables for the container.

I've added here that variable assignments are space separated.

Quoted 10 lines
fs/oci/Config.in new file +10 -0
+
+config BR2_TARGET_ROOTFS_OCI_PORTS
+	string "ports"
+	help
+	  Default set of ports to expose from a container running
+	  this image in the following format:
+
+	  <port>/tcp, <port>/udp, <port> (same as <port>/tcp).
+
+config BR2_TARGET_ROOTFS_OCI_LABELS

fs/oci/Config.in line 84

Quoted 10 lines
fs/oci/Config.in new file +10 -0
+	string "labels"
+	help
+	  Metadata in the format KEY=VALUE for the container compliant
+	  with OCI annotation rules. If KEY starts with a dot, it will
+	  be prefixed with "org.opencontainers.image"
+	  (e.g. .url -> org.opencontainers.image.url).
+
+config BR2_TARGET_ROOTFS_OCI_ARCHIVE
+	bool "pack oci image into a tar archive"
+	default n

default n is not needed.

Quoted 12 lines
fs/oci/Config.in new file +4 -0
+	help
+	  Select whether the image should be packed into a TAR archive.
+
+endif
fs/oci/oci.mk new file +2 -0
new file mode 100644
index 0000000000..09c3e88069
@@ -0,0 +1,99 @@
+################################################################################
+#

fs/oci/oci.mk line 12

Quoted 10 lines
fs/oci/oci.mk new file +10 -0
+# Build the oci image
+#
+################################################################################
+
+ROOTFS_OCI_IMAGE_NAME = rootfs-oci
+
+ROOTFS_OCI_DEPENDENCIES = host-sloci-image
+
+# architecture
+OCI_SLOCI_IMAGE_OPTS = --arch $(BR2_ARCH)

That's unlikely to be correct... For example, the typical 32-bit Intel target
is i686 in Buildroot, not 386 as expected by OCI.

The specification says it has to correspond to GOARCH - and we have that, in
GO_GOARCH. So I replaced it with that.

Ideally, the Config.in option should also depend on
BR2_PACKAGE_HOST_GO_TARGET_ARCH_SUPPORTS. Unfortunately, that symbol includes a dependency on BR2_PACKAGE_HOST_GO_BOOTSTRAP_ARCH_SUPPORTS, which is not needed here. I decided that no sane person would try to build an OCI image for an architecture which doesn't support containers, so I left it as is.

fs/oci/oci.mk line 23

Quoted 10 lines
fs/oci/oci.mk new file +10 -0
+# architecture variant (typically used only for arm)
+ifeq ($(BR2_ARM_CPU_HAS_ARM),y)
+ifeq ($(BR2_ARM_CPU_ARMV5),y)
+OCI_SLOCI_IMAGE_OPTS += --arch-variant v5
+else ifeq ($(BR2_ARM_CPU_ARMV6),y)
+OCI_SLOCI_IMAGE_OPTS += --arch-variant v6
+else ifeq ($(BR2_ARM_CPU_ARMV7A),y)
+OCI_SLOCI_IMAGE_OPTS += --arch-variant v7
+else ifeq ($(BR2_ARM_CPU_ARMV8A),y)
+OCI_SLOCI_IMAGE_OPTS += --arch-variant v8

Same here, I replaced all that with

OCI_SLOCI_IMAGE_OPTS += $(and $(GO_GOARM),--arch-variant $(GO_GOARM))

fs/oci/oci.mk line 30

Quoted 7 lines
fs/oci/oci.mk new file +7 -0
+endif
+endif
+
+# entrypoint
+OCI_ENTRYPOINT = $(call qstrip,$(BR2_TARGET_ROOTFS_OCI_ENTRYPOINT))
+ifneq ($(OCI_ENTRYPOINT),)
+OCI_SLOCI_IMAGE_OPTS += --entrypoint $(OCI_ENTRYPOINT)

For consistency, I've added quotes around all options. It's a string so it may
contain spaces.

Quoted 44 lines
fs/oci/oci.mk new file +44 -0
+endif
+
+# entrypoint arguments
+OCI_ENTRYPOINT_ARGS = $(call qstrip,$(BR2_TARGET_ROOTFS_OCI_ENTRYPOINT_ARGS))
+ifneq ($(OCI_ENTRYPOINT_ARGS),)
+OCI_SLOCI_IMAGE_OPTS += --cmd "$(OCI_ENTRYPOINT_ARGS)"
+endif
+
+# author
+OCI_AUTHOR = $(call qstrip,$(BR2_TARGET_ROOTFS_OCI_AUTHOR))
+ifneq ($(OCI_AUTHOR),)
+OCI_SLOCI_IMAGE_OPTS += --author "$(OCI_AUTHOR)"
+endif
+
+# username or UID
+OCI_UID = $(call qstrip,$(BR2_TARGET_ROOTFS_OCI_UID))
+ifneq ($(OCI_UID),)
+OCI_SLOCI_IMAGE_OPTS += --user $(OCI_UID)
+endif
+
+# labels
+OCI_LABELS = $(call qstrip,$(BR2_TARGET_ROOTFS_OCI_LABELS))
+ifneq ($(OCI_LABELS),)
+OCI_SLOCI_IMAGE_OPTS += \
+	$(foreach label,$(OCI_LABELS),--label $(label))
+endif
+
+# environment variables
+OCI_ENV_VARS = $(call qstrip,$(BR2_TARGET_ROOTFS_OCI_ENV_VARS))
+ifneq ($(OCI_ENV_VARS),)
+OCI_SLOCI_IMAGE_OPTS += \
+	$(foreach var,$(OCI_ENV_VARS),--env $(var))
+endif
+
+# working directory
+OCI_WORKDIR = $(call qstrip,$(BR2_TARGET_ROOTFS_OCI_WORKDIR))
+ifneq ($(OCI_WORKDIR),)
+OCI_SLOCI_IMAGE_OPTS += --working-dir $(OCI_WORKDIR)
+endif
+
+# ports
+OCI_PORTS = $(call qstrip,$(BR2_TARGET_ROOTFS_OCI_PORTS))
+ifneq ($(OCI_PORTS),)
+OCI_SLOCI_IMAGE_OPTS += \

fs/oci/oci.mk line 84

Quoted 10 lines
fs/oci/oci.mk new file +10 -0
+	$(foreach port,$(OCI_PORTS),--port $(port))
+endif
+
+# tag
+OCI_TAG = $(call qstrip,$(BR2_TARGET_ROOTFS_OCI_TAG))
+ifeq ($(OCI_TAG),)
+# we need a tag, so if it is empty, it is safe to override here
+# check-package OverriddenVariable
+OCI_TAG = latest
+endif

This can also be simplified to

OCI_TAG = $(or $(call qstrip,$(BR2_TARGET_ROOTFS_OCI_TAG)),latest)

fs/oci/oci.mk line 96

Quoted 12 lines
fs/oci/oci.mk new file +12 -0
+
+# enable tar archive
+ifeq ($(BR2_TARGET_ROOTFS_OCI_ARCHIVE),y)
+OCI_SLOCI_IMAGE_OPTS += --tar
+endif
+
+define ROOTFS_OCI_CMD
+	(cd $(BINARIES_DIR); \
+		rm -rf $(ROOTFS_OCI_IMAGE_NAME)*
+		$(HOST_DIR)/bin/sloci-image $(OCI_SLOCI_IMAGE_OPTS) $(TARGET_DIR) \
+			$(ROOTFS_OCI_IMAGE_NAME):$(OCI_TAG)
+	)

The iamge name is just a file name, so the cd is not needed. I just added
$(BINARIES_DIR)/ in front and get the same result.

I've also removed the ROOTFS_OCI_IMAGE_NAME, it serves no purpose IMHO.

And i've removed the wildcard after the rm -rf. Only the directory needs to be
removed; if there's a tarball, it will simply be overwritten.

Applied to master with all those fixed.

Regards,
Arnout

fs/oci/oci.mk line 99

Quoted 4 lines
fs/oci/oci.mk new file +3 -0
+endef
+
+$(eval $(rootfs))

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#5 lore raw

Re: [Buildroot] [PATCH v3 2/3] fs: new OCI filesystem type

In reply to Matthew Weber via buildroot
Quoted 8 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Matthew" == Matthew Weber via buildroot <[email protected]> writes:

From: Sergio Prado <[email protected]>
Add support to generate OCI (Open Container Initiative) images.

An OCI image consists of a manifest, an image index (optional), a set of filesystem layers, and a configuration. The complete specification is available in the link below:

Out of interest, what is the functional difference between this and just doing docker import output/build/rootfs.tar <my-tag>?

Signature · 5 lines
Bye, Peter Korsgaard
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#6 lore raw

Re: [Buildroot] [PATCH v3 2/3] fs: new OCI filesystem type

Quoted 11 lines · Peter Korsgaard
Quoted 8 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Matthew" == Matthew Weber via buildroot <[email protected]> writes:

From: Sergio Prado <[email protected]>
Add support to generate OCI (Open Container Initiative) images.

An OCI image consists of a manifest, an image index (optional), a set of filesystem layers, and a configuration. The complete specification is available in the link below:

Out of interest, what is the functional difference between this and just doing docker import output/build/rootfs.tar <my-tag>?

"import" just creates a filesystem. An OCI image also has an entrypoint, an
environment, exposed ports, and a bunch of metadata.

Regards,
Arnout

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#7 lore raw

Re: [Buildroot] [PATCH v3 2/3] fs: new OCI filesystem type

Quoted 12 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Arnout" == Arnout Vandecappelle <[email protected]> writes:

Quoted 9 lines · Peter Korsgaard
Quoted 8 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Matthew" == Matthew Weber via buildroot <[email protected]> writes:

From: Sergio Prado <[email protected]>
Add support to generate OCI (Open Container Initiative) images.

An OCI image consists of a manifest, an image index (optional), a set of filesystem layers, and a configuration. The complete specification is available in the link below:

Quoted 5 lines
Quoted 2 lines

Out of interest, what is the functional difference between this and just doing docker import output/build/rootfs.tar <my-tag>?

"import" just creates a filesystem. An OCI image also has an entrypoint, an
environment, exposed ports, and a bunch of metadata.

Ok. I guess you can do most if not all of those with the --change option to docker import though:

https://docs.docker.com/engine/reference/commandline/import/

Signature · 5 lines
Bye, Peter Korsgaard
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#8 lore raw

PATCH v3 3/3 support/testing/tests/fs: test OCI format

In reply to Matthew Weber via buildroot

This patch adds runtime testing of the OCI archive created by the sloci scripting. It launches a containerd instance, imports, and runs the OCI container.

The existing QEMU AARCH64 kernel config was extended to enable common options used by a container runtime (cgroup and overlayfs).

---
The updating of the QEMU AARCH64 kernel config seemed like a better option then 1) adding to the docker compose test kernel fragment or
2) creating a new fragment for this test.

---
 board/qemu/aarch64-virt/linux.config |  6 +++
 support/testing/tests/fs/test_oci.py | 56 ++++++++++++++++++++++++++++
 2 files changed, 62 insertions(+)
 create mode 100644 support/testing/tests/fs/test_oci.py
board/qemu/aarch64-virt/linux.config +6 -0
index 02e0812ffd..e3a4a92913 100644
@@ -3,6 +3,11 @@ CONFIG_POSIX_MQUEUE=y
 CONFIG_NO_HZ_IDLE=y
 CONFIG_HIGH_RES_TIMERS=y
 CONFIG_TASKSTATS=y
+CONFIG_MEMCG=y
+CONFIG_BLK_CGROUP=y
+CONFIG_CPUSETS=y
+CONFIG_CGROUP_DEVICE=y
+CONFIG_CGROUP_CPUACCT=y
 CONFIG_SCHED_AUTOGROUP=y
 CONFIG_PROFILING=y
 CONFIG_ARCH_VEXPRESS=y
@@ -62,5 +67,6 @@ CONFIG_ARM_SMMU_V3=y
 CONFIG_EXT4_FS=y
 CONFIG_FUSE_FS=y
 CONFIG_VIRTIO_FS=y
+CONFIG_OVERLAY_FS=y
 CONFIG_TMPFS=y
 CONFIG_TMPFS_POSIX_ACL=y
support/testing/tests/fs/test_oci.py new file +56 -0

56 lines not on this page · raw message

Signature · 6 lines
2.17.1

_______________________________________________
buildroot mailing list
[email protected]
http://lists.busybox.net/mailman/listinfo/buildroot
#9 lore raw

Re: [Buildroot] [PATCH v3 3/3] support/testing/tests/fs: test OCI format

Quoted 7 lines · Matthew Weber via buildroot

This patch adds runtime testing of the OCI archive created by the sloci scripting. It launches a containerd instance, imports, and runs the OCI container.

The existing QEMU AARCH64 kernel config was extended to enable common options used by a container runtime (cgroup and overlayfs).

Quoted 6 lines

---
The updating of the QEMU AARCH64 kernel config seemed like a better option then 1) adding to the docker compose test kernel fragment or
2) creating a new fragment for this test.

I think a new fragment would have been better, but potato potato.

Applied to master, thanks.

Regards,
Arnout

Quoted 82 lines
---
 board/qemu/aarch64-virt/linux.config |  6 +++
 support/testing/tests/fs/test_oci.py | 56 ++++++++++++++++++++++++++++
 2 files changed, 62 insertions(+)
 create mode 100644 support/testing/tests/fs/test_oci.py
board/qemu/aarch64-virt/linux.config +6 -0
index 02e0812ffd..e3a4a92913 100644
@@ -3,6 +3,11 @@ CONFIG_POSIX_MQUEUE=y
 CONFIG_NO_HZ_IDLE=y
 CONFIG_HIGH_RES_TIMERS=y
 CONFIG_TASKSTATS=y
+CONFIG_MEMCG=y
+CONFIG_BLK_CGROUP=y
+CONFIG_CPUSETS=y
+CONFIG_CGROUP_DEVICE=y
+CONFIG_CGROUP_CPUACCT=y
 CONFIG_SCHED_AUTOGROUP=y
 CONFIG_PROFILING=y
 CONFIG_ARCH_VEXPRESS=y
@@ -62,5 +67,6 @@ CONFIG_ARM_SMMU_V3=y
 CONFIG_EXT4_FS=y
 CONFIG_FUSE_FS=y
 CONFIG_VIRTIO_FS=y
+CONFIG_OVERLAY_FS=y
 CONFIG_TMPFS=y
 CONFIG_TMPFS_POSIX_ACL=y
support/testing/tests/fs/test_oci.py new file +46 -0
new file mode 100644
index 0000000000..eb5548c510
@@ -0,0 +1,56 @@
+import os
+import shutil
+import infra.basetest
+
+
+class TestOci(infra.basetest.BRTest):
+    config = \
+        """
+        BR2_aarch64=y
+        BR2_TOOLCHAIN_EXTERNAL=y
+        BR2_LINUX_KERNEL=y
+        BR2_LINUX_KERNEL_CUSTOM_VERSION=y
+        BR2_LINUX_KERNEL_CUSTOM_VERSION_VALUE="5.10.61"
+        BR2_LINUX_KERNEL_USE_CUSTOM_CONFIG=y
+        BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE="board/qemu/aarch64-virt/linux.config"
+        BR2_LINUX_KERNEL_NEEDS_HOST_OPENSSL=y
+        BR2_PACKAGE_CGROUPFS_MOUNT=y
+        BR2_PACKAGE_CONTAINERD=y
+        BR2_TARGET_ROOTFS_EXT2=y
+        BR2_TARGET_ROOTFS_EXT2_SIZE="600M"
+        BR2_TARGET_ROOTFS_OCI=y
+        BR2_TARGET_ROOTFS_OCI_ENTRYPOINT="df"
+        BR2_TARGET_ROOTFS_OCI_ENTRYPOINT_ARGS="-h"
+        BR2_TARGET_ROOTFS_OCI_ARCHIVE=y
+        # BR2_TARGET_ROOTFS_TAR is not set
+        """
+
+    def login(self):
+        rootfs = os.path.join(self.builddir, "images", "rootfs.ext2")
+        kern = os.path.join(self.builddir, "images", "Image")
+        self.emulator.boot(arch="aarch64",
+                           kernel=kern,
+                           kernel_cmdline=["root=/dev/vda", "console=ttyAMA0"],
+                           options=["-M", "virt",
+                                    "-cpu", "cortex-a57",
+                                    "-m", "512M",
+                                    "-drive", "file={},format=raw,if=virtio".format(rootfs)])
+        self.emulator.login()
+
+    def place_test_oci(self):
+        shutil.copy(os.path.join(self.builddir, 'images', 'rootfs-oci-latest-aarch64-linux.oci-image.tar'),
+                    os.path.join(self.builddir, 'target', 'oci.tar'))
+        self.b.build()
+
+    def test_run(self):
+        self.place_test_oci()

support/testing/tests/fs/test_oci.py line 56

Quoted 10 lines
support/testing/tests/fs/test_oci.py new file +10 -0
+        self.login()
+
+        cmd = "containerd &"
+        self.assertRunOk(cmd)
+
+        cmd = "ctr image import --base-name buildroot-test /oci.tar"
+        self.assertRunOk(cmd, timeout=120)
+
+        cmd = "ctr run --rm --tty buildroot-test:latest v1"
+        self.assertRunOk(cmd, timeout=120)

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot