Skip to contentSkip to conversation

package/criu: new package

PATCH Superseded Marcus Folkesson 2 participants 5 messages Sep 18, 2023 – Oct 9, 2023

v3 of 6 Superseded Latest is v6 →

lore
Share & tools
lore

Marcus Folkesson

author
#0 lore raw

PATCH v3 1/1 package/criu: new package

Checkpoint/Restore In Userspace (CRIU), is a software tool for the Linux operating system to make it possible to freeze a running application and checkpoint it to persistent storage as a collection of files.

Signed-off-by Marcus Folkesson <[email protected]>

---

Notes:
v2:
- Addressed comments from Thomas.
- Tested on ARM target and on x86_64 with qemu.

v3:
- set CONFIG_CHECKPOINT_RESTORE in kernel config
- Only be available for ARMv6, ARMv7 and ARMv8

 DEVELOPERS             |  1 +
 package/Config.in      |  1 +
 package/criu/Config.in | 48 ++++++++++++++++++++++++++++++++++
 package/criu/criu.hash |  3 +++
 package/criu/criu.mk   | 59 ++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 112 insertions(+)
 create mode 100644 package/criu/Config.in
 create mode 100644 package/criu/criu.hash
 create mode 100644 package/criu/criu.mk
DEVELOPERS +1 -0
index 7fae4841e4..d6e6a98202 100644
@@ -1986,6 +1986,7 @@ F:	support/testing/tests/package/test_python_pytest.py
 F:	support/testing/tests/package/test_python_pytest_asyncio.py
 
 N:	Marcus Folkesson <[email protected]>
+F:	package/criu/
 F:	package/libcamera/
 F:	package/libcamera-apps/
 F:	package/libostree/
package/Config.in +1 -0
index e8dbadadf3..9a754cb072 100644
@@ -2667,6 +2667,7 @@ menu "System tools"
 	source "package/coreutils/Config.in"
 	source "package/cpulimit/Config.in"
 	source "package/cpuload/Config.in"
+	source "package/criu/Config.in"
 	source "package/crun/Config.in"
 	source "package/daemon/Config.in"
 	source "package/dc3dd/Config.in"
package/criu/Config.in new file +48 -0

48 lines not on this page · raw message

package/criu/criu.hash new file +3 -0
new file mode 100644
index 0000000000..2c4a07252b
@@ -0,0 +1,3 @@
+# Locally calculated
+sha256  6a9997981c9fe4730c848ce59346b3a22fad69b803607cb67a3f6ec0557fa474  criu-3.18.tar.gz
+sha256  568a1fa9d90e18a1a1a61ea58ec2eece16b56a5042cc72c1b4f8d4455ae6fcb7  COPYING
package/criu/criu.mk new file +59 -0

59 lines not on this page · raw message

Signature · 6 lines
2.41.0

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#2 lore raw

Hi Marcus,

Quoted 4 lines · Marcus Folkesson

Julien, do you have the posibility to run test-pkg on this patch?

I still get wierd errors (related to host-libzlib (?) ) for the bootlin-armv7-glibc toolchain.

I ran a test-pkg on this v3 patch. I am not able to reproduce this host-libzlib failure (both on my host system and in the reference utils/docker-run Docker image). But I do see a build failure with the bootlin-armv7-glibc toolchain in the criu package:

    cat > criu.config <<EOF
    BR2_PACKAGE_HOST_PYTHON3=y
    BR2_PACKAGE_CRIU=y
    EOF

    utils/test-pkg -c criu.config -p criu
        bootlin-armv5-uclibc [1/6]: SKIPPED
         bootlin-armv7-glibc [2/6]: FAILED
       bootlin-armv7m-uclibc [3/6]: SKIPPED
         bootlin-x86-64-musl [4/6]: OK
          br-arm-full-static [5/6]: SKIPPED
                sourcery-arm [6/6]: SKIPPED
    6 builds, 4 skipped, 1 build failed, 0 legal-info failed, 0

show-info failed

It fails with output:

/home/br-user/br-test-pkg/bootlin-armv7-glibc/host/bin/arm-linux-ld:
./compel/plugins/std.lib.a(string.o): in function `std_vdprintf':

    string.c:(.text+0x114): undefined reference to `__aeabi_idivmod'

Maybe the SUBARCH make variable need to be set for Arm, because the criu Makefile adjust few flags based on its value. See [1].

Quoted 5 lines

bootlin-x86-64-musl is 'OK' though.

Thanks,
Marcus

Best regards,

Julien.

[1] https://github.com/checkpoint-restore/criu/blob/v3.18/Makefile#L35
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

author
#3 lore raw

Hi Julian,

Quoted 28 lines · Julien Olivain

Hi Marcus,

Quoted 4 lines · Marcus Folkesson

Julien, do you have the posibility to run test-pkg on this patch?

I still get wierd errors (related to host-libzlib (?) ) for the bootlin-armv7-glibc toolchain.

I ran a test-pkg on this v3 patch. I am not able to reproduce this host-libzlib failure (both on my host system and in the reference utils/docker-run Docker image). But I do see a build failure with the bootlin-armv7-glibc toolchain in the criu package:

    cat > criu.config <<EOF
    BR2_PACKAGE_HOST_PYTHON3=y
    BR2_PACKAGE_CRIU=y
    EOF

    utils/test-pkg -c criu.config -p criu
        bootlin-armv5-uclibc [1/6]: SKIPPED
         bootlin-armv7-glibc [2/6]: FAILED
       bootlin-armv7m-uclibc [3/6]: SKIPPED
         bootlin-x86-64-musl [4/6]: OK
          br-arm-full-static [5/6]: SKIPPED
                sourcery-arm [6/6]: SKIPPED
    6 builds, 4 skipped, 1 build failed, 0 legal-info failed, 0 show-info

failed

Quoted 8 lines

It fails with output:

/home/br-user/br-test-pkg/bootlin-armv7-glibc/host/bin/arm-linux-ld:
./compel/plugins/std.lib.a(string.o): in function `std_vdprintf':

    string.c:(.text+0x114): undefined reference to `__aeabi_idivmod'

Maybe the SUBARCH make variable need to be set for Arm, because the criu Makefile adjust few flags based on its value. See [1].

Thank you for testing, it is highly appreciated.

I will try to build using the docker image as I'm only getting problem with host-libzlib, even for a clean build.

I will add this for the next patch version

ifeq ($(BR2_ARM_CPU_ARMV6), y)
CRIU_MAKE_ENV += SUBARCH=armv6
else ifeq ($(BR2_ARM_CPU_ARMV7A), y)
CRIU_MAKE_ENV += SUBARCH=armv7
else ifeq ($(BR2_ARM_CPU_ARMV7M), y)
CRIU_MAKE_ENV += SUBARCH=armv7
else ifeq ($(BR2_ARM_CPU_ARMV8A), y)
CRIU_MAKE_ENV += SUBARCH=armv8
else ifeq ($(BR2_ARM_CPU_ARMV8M), y)
CRIU_MAKE_ENV += SUBARCH=armv8
endif

But the Makefile defaults to armv7, so I do not think it make any difference for bootlin-armv7-glibc.

Quoted 4 lines
Quoted 4 lines

bootlin-x86-64-musl is 'OK' though.

Quoted 8 lines
Quoted 2 lines

Thanks,
Marcus

Best regards,

Julien.

[1] https://github.com/checkpoint-restore/criu/blob/v3.18/Makefile#L35

/Marcus

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#4 lore raw
In reply to Marcus Folkesson

Hi Marcus,

While testing this v3 patch, I've seen two small parts that would need to be reformatted. See below:

Quoted 122 lines · Marcus Folkesson

Checkpoint/Restore In Userspace (CRIU), is a software tool for the Linux operating system to make it possible to freeze a running application and checkpoint it to persistent storage as a collection of files.

Signed-off-by Marcus Folkesson <[email protected]>

---

Notes:
v2:
- Addressed comments from Thomas.
- Tested on ARM target and on x86_64 with qemu.

v3:
- set CONFIG_CHECKPOINT_RESTORE in kernel config
- Only be available for ARMv6, ARMv7 and ARMv8

 DEVELOPERS             |  1 +
 package/Config.in      |  1 +
 package/criu/Config.in | 48 ++++++++++++++++++++++++++++++++++
 package/criu/criu.hash |  3 +++
 package/criu/criu.mk   | 59 ++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 112 insertions(+)
 create mode 100644 package/criu/Config.in
 create mode 100644 package/criu/criu.hash
 create mode 100644 package/criu/criu.mk
DEVELOPERS +0 -0
index 7fae4841e4..d6e6a98202 100644
@@ -1986,6 +1986,7 @@

F: support/testing/tests/package/test_python_pytest.py
F: support/testing/tests/package/test_python_pytest_asyncio.py

N: Marcus Folkesson <[email protected]>

DEVELOPERS +1 -0
+F:	package/criu/
 F:	package/libcamera/
 F:	package/libcamera-apps/
 F:	package/libostree/
package/Config.in +1 -0
index e8dbadadf3..9a754cb072 100644
@@ -2667,6 +2667,7 @@ menu "System tools"
 	source "package/coreutils/Config.in"
 	source "package/cpulimit/Config.in"
 	source "package/cpuload/Config.in"
+	source "package/criu/Config.in"
 	source "package/crun/Config.in"
 	source "package/daemon/Config.in"
 	source "package/dc3dd/Config.in"
package/criu/Config.in new file +48 -0
new file mode 100644
index 0000000000..06e809fb83
@@ -0,0 +1,48 @@
+# criu only builds on certain architectures
+config BR2_PACKAGE_CRIU_ARCH_SUPPORTS
+	bool
+	default y if BR2_ARM_CPU_ARMV6
+	default y if BR2_ARM_CPU_ARMV7A
+	default y if BR2_ARM_CPU_ARMV7M
+	default y if BR2_ARM_CPU_ARMV8A
+	default y if BR2_ARM_CPU_ARMV8M
+	default y if BR2_aarch64
+	default y if BR2_i386
+	default y if BR2_mips
+	default y if BR2_x86_64
+	default y if BR2_powerpc64
+	default y if BR2_s390x
+
+menuconfig BR2_PACKAGE_CRIU
+	bool "criu"
+	depends on BR2_PACKAGE_CRIU_ARCH_SUPPORTS
+	depends on BR2_PACKAGE_HOST_PYTHON3 # host-python3-ssl
+	depends on BR2_PACKAGE_HOST_PROTOBUF_ARCH_SUPPORTS # protobuf-c
+	depends on BR2_PACKAGE_PROTOBUF_ARCH_SUPPORTS
+	depends on BR2_PACKAGE_LIBBSD_ARCH_SUPPORTS
+	depends on BR2_INSTALL_LIBSTDCPP # protobuf
+	depends on BR2_TOOLCHAIN_HAS_THREADS # protobuf, libnl
+	depends on BR2_TOOLCHAIN_GCC_AT_LEAST_4_8 # protobuf
+	depends on !BR2_STATIC_LIBS # protobuf, libbsd
+	depends on BR2_USE_WCHAR # libbsd
+	depends on BR2_USE_MMU # libcap
+	select BR2_PACKAGE_HOST_PYTHON3_SSL
+	select BR2_PACKAGE_PROTOBUF
+	select BR2_PACKAGE_PROTOBUF_C
+	select BR2_PACKAGE_LIBAIO
+	select BR2_PACKAGE_LIBBSD
+	select BR2_PACKAGE_LIBCAP
+	select BR2_PACKAGE_LIBNET
+	select BR2_PACKAGE_LIBNL
+	select BR2_PACKAGE_PYTHON3
+	help
+	  Checkpoint/Restore In Userspace (CRIU), is a software
+	  tool for the Linux operating system to make it possible
+	  to freeze a running application and checkpoint it to
+	  persistent storage as a collection of files.
+
+	  https://criu.org/Main_Page
+
+comment "criu needs a toolchain w/ threads, dynamic library, wchar"
+	depends on !BR2_TOOLCHAIN_HAS_THREADS || BR2_STATIC_LIBS \
+		|| !BR2_USE_WCHAR
package/criu/criu.hash new file +2 -0
new file mode 100644
index 0000000000..2c4a07252b
@@ -0,0 +1,3 @@
+# Locally calculated
+sha256

6a9997981c9fe4730c848ce59346b3a22fad69b803607cb67a3f6ec0557fa474 criu-3.18.tar.gz +sha256 568a1fa9d90e18a1a1a61ea58ec2eece16b56a5042cc72c1b4f8d4455ae6fcb7 COPYING

package/criu/criu.mk new file +0 -0
new file mode 100644
index 0000000000..3da7ec90ab

package/criu/criu.mk line 8

Quoted 8 lines
package/criu/criu.mk new file +8 -0
@@ -0,0 +1,59 @@
+################################################################################
+#
+# CRIU
+#
+################################################################################
+
+CRIU_VERSION = 3.18
+CRIU_SOURCE = criu-$(CRIU_VERSION).tar.gz

Running command:

utils/docker-run make check-package

generates the warning:

package/criu/criu.mk:8: remove default value of _SOURCE variable
(http://nightly.buildroot.org/#generic-package-reference)

So this CRIU_SOURCE line can be removed.

Quoted 2 lines
package/criu/criu.mk new file +1 -0
+CRIU_SITE =

https://github.com/checkpoint-restore/criu/archive/refs/tags/v$(CRIU_VERSION)

I suggest to use the Buildroot download helper for github here:

CRIU_SITE = $(call github,checkpoint-restore,criu,v$(CRIU_VERSION))

Quoted 23 lines
package/criu/criu.mk new file +23 -0
+
+CRIU_LICENSE = GPL-2.0
+CRIU_LICENSE_FILES = COPYING
+CRIU_DEPENDENCIES =\
+	host-pkgconf \
+	host-protobuf-c \
+	host-python3 \
+	host-python-pip \
+	libaio \
+	libbsd \
+	libcap \
+	libnet \
+	libnl \
+	protobuf \
+	protobuf-c \
+	python3
+
+CRIU_MAKE_ENV =\
+	$(TARGET_MAKE_ENV) \
+	$(TARGET_CONFIGURE_OPTS) \
+	CROSS_COMPILE=$(TARGET_CROSS) \
+	WERROR=0
+

package/criu/criu.mk line 42

Quoted 10 lines
package/criu/criu.mk new file +10 -0
+#x86_64 is treated as x86 in criu
+#Also, powerpc64 is refered to as ppc64 in criu.
+ifeq ($(BR2_ARCH),"x86_64")
+CRIU_MAKE_ENV += ARCH=x86
+else ifeq ($(BR2_ARCH),"powerpc64")
+CRIU_MAKE_ENV += ARCH=ppc64
+else
+CRIU_MAKE_ENV += ARCH=$(BR2_ARCH)
+endif
+

The following CRIU_LINUX_CONFIG_FIXUPS text blocks has extra spaces at the beginning of the lines that could be removed.

package/criu/criu.mk line 46

Quoted 4 lines

+ # Criu needs Kernel Checkpoint/restore support which is not enabled
+ # by default.
+ define CRIU_LINUX_CONFIG_FIXUPS

package/criu/criu.mk new file +1 -0
+		 $(call KCONFIG_ENABLE_OPT,CONFIG_CHECKPOINT_RESTORE)

A single tab and no space would be better here.

Quoted 18 lines

+ endef

package/criu/criu.mk new file +4 -0
+
+define CRIU_BUILD_CMDS
+	rm -rf $(@D)/images/google/protobuf/descriptor.proto
+	ln -s  $(STAGING_DIR)/usr/include/google/protobuf/descriptor.proto

$(@D)/images/google/protobuf/descriptor.proto

package/criu/criu.mk new file +5 -0
+	$(CRIU_MAKE_ENV) $(MAKE) USERCFLAGS="$(TARGET_CFLAGS)" -C $(@D)
+endef
+
+define CRIU_INSTALL_TARGET_CMDS
+	$(CRIU_MAKE_ENV) $(MAKE) DESTDIR=$(TARGET_DIR) PREFIX=/usr -C $(@D)

install-criu install-lib install-compel

package/criu/criu.mk new file +3 -1
+endef
+
+$(eval $(generic-package))
--

2.41.0

Quoted 4 lines

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

Best regards,

Julien.
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot