package/criu: new package

PATCH Superseded Marcus Folkesson 2 participants 7 messages Oct 20, 2023 – Oct 22, 2023

Share & tools

Marcus Folkesson

author

2023-10-20 05:58 UTC #0 lore raw

PATCH v4 1/1 package/criu: new package

Checkpoint/Restore In Userspace (CRIU), is a software tool for the Linux operating system to make it possible to freeze a running application and checkpoint it to persistent storage as a collection of files.

Signed-off-by Marcus Folkesson <[email protected]>

---

Notes:
v2:
- Addressed comments from Thomas.
- Tested on ARM target and on x86_64 with qemu.

v3:
- set CONFIG_CHECKPOINT_RESTORE in kernel config
- Only be available for ARMv6, ARMv7 and ARMv8

v4:
- set SUBARCH for armv6, armv7 and armv8
- Use github download helper

Result from test-pkg:
bootlin-armv5-uclibc [1/6]: SKIPPED
bootlin-armv7-glibc [2/6]: OK
bootlin-armv7m-uclibc [3/6]: SKIPPED
bootlin-x86-64-musl [4/6]: OK
br-arm-full-static [5/6]: SKIPPED
sourcery-arm [6/6]: SKIPPED
6 builds, 4 skipped, 0 build failed, 0 legal-info failed, 0 show-info failed

 DEVELOPERS             |  1 +
 package/Config.in      |  1 +
 package/criu/Config.in | 48 +++++++++++++++++++++++++++++
 package/criu/criu.hash |  3 ++
 package/criu/criu.mk   | 70 ++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 123 insertions(+)
 create mode 100644 package/criu/Config.in
 create mode 100644 package/criu/criu.hash
 create mode 100644 package/criu/criu.mk

DEVELOPERS +1 -0

index 57015e245e..2047827bd9 100644
@@ -2007,6 +2007,7 @@ F:	support/testing/tests/package/test_python_pytest.py
 F:	support/testing/tests/package/test_python_pytest_asyncio.py
 
 N:	Marcus Folkesson <[email protected]>
+F:	package/criu/
 F:	package/libcamera/
 F:	package/libcamera-apps/
 F:	package/libostree/

package/Config.in +1 -0

index 4e489c4706..9e2099f6a5 100644
@@ -2678,6 +2678,7 @@ menu "System tools"
 	source "package/coreutils/Config.in"
 	source "package/cpulimit/Config.in"
 	source "package/cpuload/Config.in"
+	source "package/criu/Config.in"
 	source "package/crun/Config.in"
 	source "package/daemon/Config.in"
 	source "package/dc3dd/Config.in"

package/criu/Config.in new file +48 -0

48 lines not on this page · raw message

package/criu/criu.hash new file +3 -0

new file mode 100644
index 0000000000..2c4a07252b
@@ -0,0 +1,3 @@
+# Locally calculated
+sha256  6a9997981c9fe4730c848ce59346b3a22fad69b803607cb67a3f6ec0557fa474  criu-3.18.tar.gz
+sha256  568a1fa9d90e18a1a1a61ea58ec2eece16b56a5042cc72c1b4f8d4455ae6fcb7  COPYING

package/criu/criu.mk new file +70 -0

70 lines not on this page · raw message

Signature · 6 lines

2.41.0

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

Julien Olivain

2023-10-21 11:15 UTC #1 lore raw

Hi Marcus,

Thanks for the updated patch!

I confirm the normal test-pkg is passing. Then, I ran a "test-pkg -a" (which test more toolchains/arches) and observed few new minor failures. I'll comment a bit each failures and provide hints for improvements, when possible.

cat >criu.config <<EOF
BR2_PACKAGE_HOST_PYTHON3=y
BR2_PACKAGE_CRIU=y
EOF
utils/test-pkg -a -c criu.config -p criu
arm-aarch64 [ 1/45]: OK
bootlin-aarch64-glibc [ 2/45]: FAILED

criu uses restartable sequences, see:
https://github.com/checkpoint-restore/criu/blob/v3.18/criu/include/linux/rseq.h#L7

Criu seems to have a header detection on "sys/rseq.h" (i.e. on the libc side). But the sys/rseq.h then includes linux/rseq.h (i.e. on the kernel side), which is not checked. Some configurations like bootlin-aarch64-glibc includes the glibc header, but also have older kernel. Restartable sequences were introduced in kernel >= 4.18. So this can be handled by adding a dependency on kernel.

       bootlin-arcle-hs38-uclibc [ 3/45]: SKIPPED
            bootlin-armv5-uclibc [ 4/45]: SKIPPED
             bootlin-armv7-glibc [ 5/45]: OK
           bootlin-armv7m-uclibc [ 6/45]: SKIPPED
              bootlin-armv7-musl [ 7/45]: OK
        bootlin-m68k-5208-uclibc [ 8/45]: SKIPPED
       bootlin-m68k-68040-uclibc [ 9/45]: SKIPPED
     bootlin-microblazeel-uclibc [10/45]: SKIPPED
        bootlin-mipsel32r6-glibc [11/45]: SKIPPED
           bootlin-mipsel-uclibc [12/45]: SKIPPED
             bootlin-nios2-glibc [13/45]: SKIPPED
         bootlin-openrisc-uclibc [14/45]: SKIPPED
bootlin-powerpc64le-power8-glibc [15/45]: SKIPPED
   bootlin-powerpc-e500mc-uclibc [16/45]: SKIPPED
           bootlin-riscv32-glibc [17/45]: SKIPPED
           bootlin-riscv64-glibc [18/45]: SKIPPED
            bootlin-riscv64-musl [19/45]: SKIPPED
         bootlin-s390x-z13-glibc [20/45]: FAILED

Build fails with output:

    Makefile:23: *** "The architecture s390x isn't supported".  Stop.

criu Makefile checks for "s390" arch (but "s390x" is passed here). See:
https://github.com/checkpoint-restore/criu/blob/v3.18/Makefile#L22

              bootlin-sh4-uclibc [21/45]: SKIPPED
           bootlin-sparc64-glibc [22/45]: SKIPPED
            bootlin-sparc-uclibc [23/45]: SKIPPED
            bootlin-x86-64-glibc [24/45]: OK
             bootlin-x86-64-musl [25/45]: OK
           bootlin-x86-64-uclibc [26/45]: FAILED

Build fails with output:

    criu/fsnotify.c:18:10: fatal error: aio.h: No such file or directory
       18 | #include <aio.h>
          |          ^~~~~~~

           bootlin-xtensa-uclibc [27/45]: SKIPPED
                    br-arm-basic [28/45]: SKIPPED
            br-arm-full-nothread [29/45]: SKIPPED
              br-arm-full-static [30/45]: SKIPPED
           br-i386-pentium4-full [31/45]: FAILED

Build fails with output:

Makefile:23: *** "The architecture i686 isn't supported". Stop.

br-i386-pentium-mmx-musl [32/45]: FAILED

Build fails with output:

    Makefile:23: *** "The architecture i586 isn't supported".  Stop.

              br-mips64-n64-full [33/45]: SKIPPED
         br-mips64r6-el-hf-glibc [34/45]: SKIPPED
       br-powerpc-603e-basic-cpp [35/45]: SKIPPED
       br-powerpc64-power7-glibc [36/45]: FAILED

Build fails with output:

In file included from compel/plugins/std/infect.c:14:
compel/include/uapi/compel/asm/sigframe.h:27:2: error: #error Only
supporting ABIv2.

       27 | #error Only supporting ABIv2.
          |  ^~~~~

               linaro-aarch64-be [37/45]: FAILED

Build fails with output:

Makefile:23: *** "The architecture aarch64_be isn't supported".
Stop.

                  linaro-aarch64 [38/45]: OK
                      linaro-arm [39/45]: FAILED

Build fails with output:

arm-linux-gnueabihf-gcc: error: unrecognized argument in option
'-march=armv7-a+fp'
arm-linux-gnueabihf-gcc: note: valid arguments to '-march=' are:
armv2 armv2a armv3 armv3m armv4 armv4t armv5 armv5e armv5t armv5te armv5tej armv6 armv6-m armv6j armv6k armv6kz armv6s-m armv6t2 armv6z armv6zk armv7 armv7-a armv7-m armv7-r armv7e-m armv7ve armv8-a armv8-a+ crc armv8-m.base armv8-m.main armv8-m.main+dsp armv8.1-a armv8.2-a armv8.2-a+dotprod armv8.2-a+fp16 armv8.2-a+fp16+dotprod iwmmxt iwmmxt2 native; did you mean 'armv7-a'?

Gcc option "-march=armv7-a+fp" was introduced in gcc commit
https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=76d7d5334991a5646026e5aa8c3e7d23629f383a
first included in version 8.1.0.
So the package should have a dependency on this. linaro-arm fails because it contains 7.3.1.

             sourcery-arm-armv4t [40/45]: SKIPPED
                    sourcery-arm [41/45]: SKIPPED
             sourcery-arm-thumb2 [42/45]: FAILED

Same story as linaro-arm toolchain.

                 sourcery-mips64 [43/45]: SKIPPED
                   sourcery-mips [44/45]: FAILED

Fails with output:

compel/arch/mips/plugins/std/memcpy.S: Assembler messages:
compel/arch/mips/plugins/std/memcpy.S:7: Error: opcode not supported
on this processor: mips32r2 (mips32r2) 'dadd $2,$0,$4'
compel/arch/mips/plugins/std/memcpy.S:8: Error: opcode not supported
on this processor: mips32r2 (mips32r2) 'daddiu $13,$0,0'
compel/arch/mips/plugins/std/memcpy.S:14: Error: opcode not
supported on this processor: mips32r2 (mips32r2) 'daddiu $13,$13,1'
compel/arch/mips/plugins/std/memcpy.S:15: Error: opcode not
supported on this processor: mips32r2 (mips32r2) 'daddiu $4,$4,1'
compel/arch/mips/plugins/std/memcpy.S:16: Error: opcode not
supported on this processor: mips32r2 (mips32r2) 'daddiu $5,$5,1'

There is maybe few extra condtions missing in the _ARCH_SUPPORTS for mips.

                  sourcery-nios2 [45/45]: SKIPPED

45 builds, 29 skipped, 10 build failed, 0 legal-info failed, 0 show-info failed

To test thoroughly your package with a specific list of toolchains, you can use the following commands:

cp support/config-fragments/autobuild/toolchain-configs.csv
criu-toolchains.csv

    # edit criu-toolchains.csv to keep your toolchains of interest.
    utils/test-pkg -a -t criu-toolchains.csv -c criu.config -p criu

This will retest only the toolchains kept in the csv.

Quoted 75 lines · Marcus Folkesson

Checkpoint/Restore In Userspace (CRIU), is a software tool for the Linux operating system to make it possible to freeze a running application and checkpoint it to persistent storage as a collection of files.
Signed-off-by Marcus Folkesson <[email protected]>
---
Notes:
v2:
- Addressed comments from Thomas.
- Tested on ARM target and on x86_64 with qemu.
v3:
- set CONFIG_CHECKPOINT_RESTORE in kernel config
- Only be available for ARMv6, ARMv7 and ARMv8
v4:
- set SUBARCH for armv6, armv7 and armv8
- Use github download helper
Result from test-pkg:
bootlin-armv5-uclibc [1/6]: SKIPPED
bootlin-armv7-glibc [2/6]: OK
bootlin-armv7m-uclibc [3/6]: SKIPPED
bootlin-x86-64-musl [4/6]: OK
br-arm-full-static [5/6]: SKIPPED
sourcery-arm [6/6]: SKIPPED
6 builds, 4 skipped, 0 build failed, 0 legal-info failed, 0
show-info failed
 DEVELOPERS             |  1 +
 package/Config.in      |  1 +
 package/criu/Config.in | 48 +++++++++++++++++++++++++++++
 package/criu/criu.hash |  3 ++
 package/criu/criu.mk   | 70 ++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 123 insertions(+)
 create mode 100644 package/criu/Config.in
 create mode 100644 package/criu/criu.hash
 create mode 100644 package/criu/criu.mk
DEVELOPERS +0 -0
index 57015e245e..2047827bd9 100644
@@ -2007,6 +2007,7 @@ 
F: support/testing/tests/package/test_python_pytest.py
F: support/testing/tests/package/test_python_pytest_asyncio.py
N: Marcus Folkesson <[email protected]>
DEVELOPERS +1 -0
+F:	package/criu/
 F:	package/libcamera/
 F:	package/libcamera-apps/
 F:	package/libostree/
package/Config.in +1 -0
index 4e489c4706..9e2099f6a5 100644
@@ -2678,6 +2678,7 @@ menu "System tools"
 	source "package/coreutils/Config.in"
 	source "package/cpulimit/Config.in"
 	source "package/cpuload/Config.in"
+	source "package/criu/Config.in"
 	source "package/crun/Config.in"
 	source "package/daemon/Config.in"
 	source "package/dc3dd/Config.in"
package/criu/Config.in new file +4 -0
new file mode 100644
index 0000000000..06e809fb83
@@ -0,0 +1,48 @@
+# criu only builds on certain architectures
+config BR2_PACKAGE_CRIU_ARCH_SUPPORTS
+	bool
+	default y if BR2_ARM_CPU_ARMV6

package/criu/Config.in line 14

Quoted 10 lines

package/criu/Config.in new file +10 -0

+	default y if BR2_ARM_CPU_ARMV7A
+	default y if BR2_ARM_CPU_ARMV7M
+	default y if BR2_ARM_CPU_ARMV8A
+	default y if BR2_ARM_CPU_ARMV8M
+	default y if BR2_aarch64
+	default y if BR2_i386
+	default y if BR2_mips
+	default y if BR2_x86_64
+	default y if BR2_powerpc64
+	default y if BR2_s390x

There is possibly some rework needed here to fix architecture related build failures described earlier.

package/criu/Config.in line 24

Quoted 10 lines

package/criu/Config.in new file +10 -0

+
+menuconfig BR2_PACKAGE_CRIU
+	bool "criu"
+	depends on BR2_PACKAGE_CRIU_ARCH_SUPPORTS
+	depends on BR2_PACKAGE_HOST_PYTHON3 # host-python3-ssl
+	depends on BR2_PACKAGE_HOST_PROTOBUF_ARCH_SUPPORTS # protobuf-c
+	depends on BR2_PACKAGE_PROTOBUF_ARCH_SUPPORTS
+	depends on BR2_PACKAGE_LIBBSD_ARCH_SUPPORTS
+	depends on BR2_INSTALL_LIBSTDCPP # protobuf
+	depends on BR2_TOOLCHAIN_HAS_THREADS # protobuf, libnl

A dependency on kernel header >= 4.18 should be added:

depends on BR2_TOOLCHAIN_HEADERS_AT_LEAST_4_18 # rseq.h

package/criu/Config.in line 25

Quoted 1 line

package/criu/Config.in new file +1 -0
+	depends on BR2_TOOLCHAIN_GCC_AT_LEAST_4_8 # protobuf

The condition can be raised to:

depends on BR2_HOST_GCC_AT_LEAST_8 # -march=armv7-a+fp

If gcc older than 8 for non-arm architecture are really needed, it can also be refined as something like "gcc >= 4.8 && !arm || gcc >= 8 && arm". For reference, gcc 8.1 was release on 2018-05-02.

Quoted 13 lines

package/criu/Config.in new file +13 -0

+	depends on !BR2_STATIC_LIBS # protobuf, libbsd
+	depends on BR2_USE_WCHAR # libbsd
+	depends on BR2_USE_MMU # libcap
+	select BR2_PACKAGE_HOST_PYTHON3_SSL
+	select BR2_PACKAGE_PROTOBUF
+	select BR2_PACKAGE_PROTOBUF_C
+	select BR2_PACKAGE_LIBAIO
+	select BR2_PACKAGE_LIBBSD
+	select BR2_PACKAGE_LIBCAP
+	select BR2_PACKAGE_LIBNET
+	select BR2_PACKAGE_LIBNL
+	select BR2_PACKAGE_PYTHON3
+	help

package/criu/Config.in line 48

Quoted 10 lines

package/criu/Config.in new file +10 -0

+	  Checkpoint/Restore In Userspace (CRIU), is a software
+	  tool for the Linux operating system to make it possible
+	  to freeze a running application and checkpoint it to
+	  persistent storage as a collection of files.
+
+	  https://criu.org/Main_Page
+
+comment "criu needs a toolchain w/ threads, dynamic library, wchar"
+	depends on !BR2_TOOLCHAIN_HAS_THREADS || BR2_STATIC_LIBS \
+		|| !BR2_USE_WCHAR

The new toolchain dependencies you will add will need to be reflected here.

Quoted 40 lines

package/criu/criu.hash new file +2 -0

new file mode 100644
index 0000000000..2c4a07252b
@@ -0,0 +1,3 @@
+# Locally calculated
+sha256

6a9997981c9fe4730c848ce59346b3a22fad69b803607cb67a3f6ec0557fa474 criu-3.18.tar.gz +sha256 568a1fa9d90e18a1a1a61ea58ec2eece16b56a5042cc72c1b4f8d4455ae6fcb7 COPYING

package/criu/criu.mk new file +23 -0

new file mode 100644
index 0000000000..a84c625696
@@ -0,0 +1,70 @@
+################################################################################
+#
+# CRIU
+#
+################################################################################
+
+CRIU_VERSION = 3.18
+CRIU_SITE = $(call github,checkpoint-restore,criu,v$(CRIU_VERSION))
+
+CRIU_LICENSE = GPL-2.0
+CRIU_LICENSE_FILES = COPYING
+CRIU_DEPENDENCIES =\
+	host-pkgconf \
+	host-protobuf-c \
+	host-python3 \
+	host-python-pip \
+	libaio \
+	libbsd \
+	libcap \
+	libnet \
+	libnl \
+	protobuf \
+	protobuf-c \

package/criu/criu.mk line 33

Quoted 10 lines

package/criu/criu.mk new file +10 -0

+	python3
+
+CRIU_MAKE_ENV =\
+	$(TARGET_MAKE_ENV) \
+	$(TARGET_CONFIGURE_OPTS) \
+	CROSS_COMPILE=$(TARGET_CROSS) \
+	WERROR=0
+
+#x86_64 is treated as x86 in criu
+#Also, powerpc64 is refered to as ppc64 in criu.

There is possibly some rework needed here to fix architecture related build failures described earlier.

Quoted 40 lines

package/criu/criu.mk new file +29 -0

+ifeq ($(BR2_ARCH),"x86_64")
+CRIU_MAKE_ENV += ARCH=x86
+else ifeq ($(BR2_ARCH),"powerpc64")
+CRIU_MAKE_ENV += ARCH=ppc64
+else
+CRIU_MAKE_ENV += ARCH=$(BR2_ARCH)
+endif
+
+ifeq ($(BR2_ARM_CPU_ARMV6), y)
+CRIU_MAKE_ENV += SUBARCH=armv6
+else ifeq ($(BR2_ARM_CPU_ARMV7A), y)
+CRIU_MAKE_ENV += SUBARCH=armv7
+else ifeq ($(BR2_ARM_CPU_ARMV7M), y)
+CRIU_MAKE_ENV += SUBARCH=armv7
+else ifeq ($(BR2_ARM_CPU_ARMV8A), y)
+CRIU_MAKE_ENV += SUBARCH=armv8
+else ifeq ($(BR2_ARM_CPU_ARMV8M), y)
+CRIU_MAKE_ENV += SUBARCH=armv8
+endif
+
+# Criu needs Kernel Checkpoint/restore support which is not enabled
+# by default.
+define CRIU_LINUX_CONFIG_FIXUPS
+	$(call KCONFIG_ENABLE_OPT,CONFIG_CHECKPOINT_RESTORE)
+endef
+
+define CRIU_BUILD_CMDS
+	rm -rf $(@D)/images/google/protobuf/descriptor.proto
+	ln -s  $(STAGING_DIR)/usr/include/google/protobuf/descriptor.proto

$(@D)/images/google/protobuf/descriptor.proto

package/criu/criu.mk new file +5 -0

+	$(CRIU_MAKE_ENV) $(MAKE) -C $(@D)
+endef
+
+define CRIU_INSTALL_TARGET_CMDS
+	$(CRIU_MAKE_ENV) $(MAKE) DESTDIR=$(TARGET_DIR) PREFIX=/usr -C $(@D)

install-criu install-lib install-compel

package/criu/criu.mk new file +3 -1

+endef
+
+$(eval $(generic-package))
--

Quoted 1 line

2.41.0

Best regards,

Julien.
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

Marcus Folkesson

author

2023-10-21 15:21 UTC #2 lore raw

Hi Julien,

Thank you for your solid review!

I will apply all changes you suggested.

Quoted 4 lines · Julien Olivain

Hi Marcus,
Thanks for the updated patch!

[...]

Quoted 7 lines

           bootlin-x86-64-uclibc [26/45]: FAILED

Build fails with output:

    criu/fsnotify.c:18:10: fatal error: aio.h: No such file or directory
       18 | #include <aio.h>
          |          ^~~~~~~

Not sure that to do about this. It already depends on libaio.

Quoted 7 lines

           bootlin-xtensa-uclibc [27/45]: SKIPPED
                    br-arm-basic [28/45]: SKIPPED
            br-arm-full-nothread [29/45]: SKIPPED
              br-arm-full-static [30/45]: SKIPPED
           br-i386-pentium4-full [31/45]: FAILED

[...]

Quoted 6 lines

              br-mips64-n64-full [33/45]: SKIPPED
         br-mips64r6-el-hf-glibc [34/45]: SKIPPED
       br-powerpc-603e-basic-cpp [35/45]: SKIPPED
       br-powerpc64-power7-glibc [36/45]: FAILED

Quoted 7 lines

Build fails with output:
In file included from compel/plugins/std/infect.c:14:
compel/include/uapi/compel/asm/sigframe.h:27:2: error: #error Only
supporting ABIv2.
27 | #error Only supporting ABIv2.
| ^~~~~

I will only support PPC64 with LE. The GCC says that:
`
-mabi=elfv1
Change the current ABI to use the ELFv1 ABI. This is the default ABI for big-endian PowerPC 64-bit Linux. Overriding the default ABI requires special system support and is likely to fail in spectacular ways.

-mabi=elfv2
Change the current ABI to use the ELFv2 ABI. This is the default ABI for little-endian PowerPC 64-bit Linux. Overriding the default ABI requires special system support and is likely to fail in spectacular ways. `

[...]

Quoted 16 lines

                   sourcery-mips [44/45]: FAILED
Fails with output:
compel/arch/mips/plugins/std/memcpy.S: Assembler messages:
compel/arch/mips/plugins/std/memcpy.S:7: Error: opcode not supported on
this processor: mips32r2 (mips32r2) 'dadd $2,$0,$4'
compel/arch/mips/plugins/std/memcpy.S:8: Error: opcode not supported on
this processor: mips32r2 (mips32r2) 'daddiu $13,$0,0'
compel/arch/mips/plugins/std/memcpy.S:14: Error: opcode not supported on
this processor: mips32r2 (mips32r2) 'daddiu $13,$13,1'
compel/arch/mips/plugins/std/memcpy.S:15: Error: opcode not supported on
this processor: mips32r2 (mips32r2) 'daddiu $4,$4,1'
compel/arch/mips/plugins/std/memcpy.S:16: Error: opcode not supported on
this processor: mips32r2 (mips32r2) 'daddiu $5,$5,1'

Quoted 1 line

There is maybe few extra condtions missing in the _ARCH_SUPPORTS for mips.

Hrm, will have a look. Thanks.

Quoted 9 lines

                  sourcery-nios2 [45/45]: SKIPPED
45 builds, 29 skipped, 10 build failed, 0 legal-info failed, 0 show-info failed
To test thoroughly your package with a specific list of toolchains, you can use the
following commands:

Quoted 6 lines

cp support/config-fragments/autobuild/toolchain-configs.csv
criu-toolchains.csv
    # edit criu-toolchains.csv to keep your toolchains of interest.
    utils/test-pkg -a -t criu-toolchains.csv -c criu.config -p criu
This will retest only the toolchains kept in the csv.

Thanks for the tip!

[...]

Quoted 13 lines

package/criu/Config.in line 24

Quoted 10 lines

package/criu/Config.in new file +10 -0

+
+menuconfig BR2_PACKAGE_CRIU
+	bool "criu"
+	depends on BR2_PACKAGE_CRIU_ARCH_SUPPORTS
+	depends on BR2_PACKAGE_HOST_PYTHON3 # host-python3-ssl
+	depends on BR2_PACKAGE_HOST_PROTOBUF_ARCH_SUPPORTS # protobuf-c
+	depends on BR2_PACKAGE_PROTOBUF_ARCH_SUPPORTS
+	depends on BR2_PACKAGE_LIBBSD_ARCH_SUPPORTS
+	depends on BR2_INSTALL_LIBSTDCPP # protobuf
+	depends on BR2_TOOLCHAIN_HAS_THREADS # protobuf, libnl

A dependency on kernel header >= 4.18 should be added:

Quoted 1 line

depends on BR2_TOOLCHAIN_HEADERS_AT_LEAST_4_18 # rseq.h

This makes the bootlin-x86-64-musl test to be skipped as it has BR2_TOOLCHAIN_HEADERS_AT_LEAST_4_14 at highest.

The build used to succeed though :-/

[...]

Quoted 28 lines

package/criu/criu.mk new file +28 -0

+################################################################################
+#
+# CRIU
+#
+################################################################################
+
+CRIU_VERSION = 3.18
+CRIU_SITE = $(call github,checkpoint-restore,criu,v$(CRIU_VERSION))
+
+CRIU_LICENSE = GPL-2.0
+CRIU_LICENSE_FILES = COPYING
+CRIU_DEPENDENCIES =\
+	host-pkgconf \
+	host-protobuf-c \
+	host-python3 \
+	host-python-pip \
+	libaio \
+	libbsd \
+	libcap \
+	libnet \
+	libnl \
+	protobuf \
+	protobuf-c \
+	python3
+
+CRIU_MAKE_ENV =\
+	$(TARGET_MAKE_ENV) \
+	$(TARGET_CONFIGURE_OPTS) \

Quoted 9 lines

package/criu/criu.mk line 33
Quoted 5 lines
package/criu/criu.mk new file +5 -0
+	CROSS_COMPILE=$(TARGET_CROSS) \
+	WERROR=0
+
+#x86_64 is treated as x86 in criu
+#Also, powerpc64 is refered to as ppc64 in criu.
There is possibly some rework needed here to fix architecture related build failures described earlier.

I will go for $(BR2_NORMALIZED_ARCH) instead of $(BR2_ARCH), it will solve the most of these type of errors.

Quoted 4 lines

Best regards,
Julien.

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

Julien Olivain

2023-10-22 11:30 UTC #3 lore raw

Hi Marcus,

Quoted 15 lines · Marcus Folkesson

Hi Julien,
Thank you for your solid review!
I will apply all changes you suggested.
Quoted 4 lines · Julien Olivain
Hi Marcus,
Thanks for the updated patch!
[...]
Quoted 2 lines
           bootlin-x86-64-uclibc [26/45]: FAILED

Quoted 8 lines

Quoted 6 lines
Build fails with output:
criu/fsnotify.c:18:10: fatal error: aio.h: No such file or
directory
       18 | #include <aio.h>
          |          ^~~~~~~
Not sure that to do about this. It already depends on libaio.

There is something strange here, in the criu code:

libaio (Linux AIO) normally provides the "libaio.h" header file. See:
https://pagure.io/libaio/blob/master/f/src/libaio.h

This header file is used exactly once in a criu test, here:
https://github.com/checkpoint-restore/criu/blob/v3.18/test/zdtm/static/aio00.c#L1

The rest of the code uses "aio.h" and <aio.h> (Posix AIO).

criu code also includes its own "aio.h" file:
https://github.com/checkpoint-restore/criu/blob/v3.18/criu/include/aio.h

The <aio.h> header file is normally provided by the libc, if implemented. See glibc for example:
https://sourceware.org/git/?p=glibc.git;a=blob;f=include/aio.h

musl does also provide aio.h, see for example:
https://git.musl-libc.org/cgit/musl/tree/include/aio.h?h=v1.2.4

uclibc-ng does not currently provide aio.h.

So maybe this means criu requires a libc with aio.h?
In that case, that would mean this packages need a dependency on:

depends on !BR2_TOOLCHAIN_USES_UCLIBC # no aio.h

Were you ever able to do a successful build with uclibc?

Best regards,

Julien.
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

Julien Olivain

2023-10-22 11:37 UTC #4 lore raw

In reply to Marcus Folkesson

Hi Marcus,

Quoted 2 lines · Marcus Folkesson

Hi Julien,

[...]

Quoted 13 lines

package/criu/Config.in line 24

Quoted 10 lines

package/criu/Config.in new file +10 -0

+
+menuconfig BR2_PACKAGE_CRIU
+	bool "criu"
+	depends on BR2_PACKAGE_CRIU_ARCH_SUPPORTS
+	depends on BR2_PACKAGE_HOST_PYTHON3 # host-python3-ssl
+	depends on BR2_PACKAGE_HOST_PROTOBUF_ARCH_SUPPORTS # protobuf-c
+	depends on BR2_PACKAGE_PROTOBUF_ARCH_SUPPORTS
+	depends on BR2_PACKAGE_LIBBSD_ARCH_SUPPORTS
+	depends on BR2_INSTALL_LIBSTDCPP # protobuf
+	depends on BR2_TOOLCHAIN_HAS_THREADS # protobuf, libnl

A dependency on kernel header >= 4.18 should be added:

Quoted 7 lines

Quoted 1 line
depends on BR2_TOOLCHAIN_HEADERS_AT_LEAST_4_18 # rseq.h
This makes the bootlin-x86-64-musl test to be skipped as it has BR2_TOOLCHAIN_HEADERS_AT_LEAST_4_14 at highest.
The build used to succeed though :-/

Ah, good point. Adding this kernel >= 4.18 requirement would indeed exclude some working cases...

criu has some logic to detect if a libc provides rseq.h. See:
https://github.com/checkpoint-restore/criu/blob/v3.18/scripts/feature-tests.mak#L187
which define CONFIG_HAS_NO_LIBC_RSEQ_DEFS.

In the case of the working bootlin-x86-64-musl toolchain, we have a 4.14 kernel without linux/rseq.h AND a libc also not providing sys/rseq.h.

In the failing bootlin-aarch64-glibc toolchain, we have a kernel headers without linux/rseq.h, but a glibc with sys/rseq.h.

I did not investigated any further.

Maybe there is a corner case in the criu detection logic and it could be patched to support it?

Best regards,

Julien.
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

Marcus Folkesson

author

2023-10-21 16:23 UTC #5 lore raw

In reply to Julien Olivain

Hi again,

Quoted 46 lines · Julien Olivain

Hi Marcus,
Thanks for the updated patch!
I confirm the normal test-pkg is passing. Then, I ran a "test-pkg -a" (which test more toolchains/arches) and observed few new minor failures. I'll comment a bit each failures and provide hints for improvements, when possible.
cat >criu.config <<EOF
BR2_PACKAGE_HOST_PYTHON3=y
BR2_PACKAGE_CRIU=y
EOF
utils/test-pkg -a -c criu.config -p criu
arm-aarch64 [ 1/45]: OK
bootlin-aarch64-glibc [ 2/45]: FAILED
criu uses restartable sequences, see:
https://github.com/checkpoint-restore/criu/blob/v3.18/criu/include/linux/rseq.h#L7
Criu seems to have a header detection on "sys/rseq.h" (i.e. on the libc side). But the sys/rseq.h then includes linux/rseq.h (i.e. on the kernel side), which is not checked. Some configurations like bootlin-aarch64-glibc includes the glibc header, but also have older kernel. Restartable sequences were introduced in kernel >= 4.18. So this can be handled by adding a dependency on kernel.
       bootlin-arcle-hs38-uclibc [ 3/45]: SKIPPED
            bootlin-armv5-uclibc [ 4/45]: SKIPPED
             bootlin-armv7-glibc [ 5/45]: OK
           bootlin-armv7m-uclibc [ 6/45]: SKIPPED
              bootlin-armv7-musl [ 7/45]: OK
        bootlin-m68k-5208-uclibc [ 8/45]: SKIPPED
       bootlin-m68k-68040-uclibc [ 9/45]: SKIPPED
     bootlin-microblazeel-uclibc [10/45]: SKIPPED
        bootlin-mipsel32r6-glibc [11/45]: SKIPPED
           bootlin-mipsel-uclibc [12/45]: SKIPPED
             bootlin-nios2-glibc [13/45]: SKIPPED
         bootlin-openrisc-uclibc [14/45]: SKIPPED
bootlin-powerpc64le-power8-glibc [15/45]: SKIPPED
   bootlin-powerpc-e500mc-uclibc [16/45]: SKIPPED
           bootlin-riscv32-glibc [17/45]: SKIPPED
           bootlin-riscv64-glibc [18/45]: SKIPPED
            bootlin-riscv64-musl [19/45]: SKIPPED
         bootlin-s390x-z13-glibc [20/45]: FAILED

Quoted 3 lines

Build fails with output:

    Makefile:23: *** "The architecture s390x isn't supported".  Stop.

Another problem that occured...
After I use $(BR2_NORMALIZED_ARCH) instead of $(BR2_ARCH), the arch is at least supported. But it fails with

Error (compel/src/lib/handle-elf-host.c:161): Unsupported header detected make[3]: *** [criu/pie/Makefile:58: criu/pie/parasite-blob.h] Error 255

After some more investigation, it ended up here:

static bool is_header_supported(Elf_Ehdr *hdr)

{
	if (!arch_is_machine_supported(hdr->e_machine))
		return false;
	if ((hdr->e_type != ET_REL
#ifdef NO_RELOCS
	     && hdr->e_type != ET_EXEC
#endif
	     ) ||
	    hdr->e_version != EV_CURRENT)
		return false;
	return true;
}

I've added debug trace to see why it fails and got this:
Error (compel/src/lib/handle-elf-host.c:155): type 0x100 machine 0x1600 version 0x1000000
Error (compel/src/lib/handle-elf-host.c:158): ET_REL 0x1 EM_s390 0x16, EV_CURRENT 0x1

It is a tool that runs on host (little endian), but read data of an elf compiled for s390 (big endian), so bytes is read swapped.

Right now I thinking of not list s390 as a compatible architecture for this package.

Do you think it is reasonable?

/Marcus

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

Julien Olivain

2023-10-22 11:46 UTC #6 lore raw

Hi Marcus,

Quoted 80 lines · Marcus Folkesson

Hi again,
Quoted 46 lines · Julien Olivain
Hi Marcus,
Thanks for the updated patch!
I confirm the normal test-pkg is passing. Then, I ran a "test-pkg -a" (which test more toolchains/arches) and observed few new minor failures. I'll comment a bit each failures and provide hints for improvements, when possible.
cat >criu.config <<EOF
BR2_PACKAGE_HOST_PYTHON3=y
BR2_PACKAGE_CRIU=y
EOF
utils/test-pkg -a -c criu.config -p criu
arm-aarch64 [ 1/45]: OK
bootlin-aarch64-glibc [ 2/45]: FAILED
criu uses restartable sequences, see:
https://github.com/checkpoint-restore/criu/blob/v3.18/criu/include/linux/rseq.h#L7
Criu seems to have a header detection on "sys/rseq.h" (i.e. on the libc side). But the sys/rseq.h then includes linux/rseq.h (i.e. on the kernel side), which is not checked. Some configurations like bootlin-aarch64-glibc includes the glibc header, but also have older kernel. Restartable sequences were introduced in kernel >= 4.18. So this can be handled by adding a dependency on kernel.
       bootlin-arcle-hs38-uclibc [ 3/45]: SKIPPED
            bootlin-armv5-uclibc [ 4/45]: SKIPPED
             bootlin-armv7-glibc [ 5/45]: OK
           bootlin-armv7m-uclibc [ 6/45]: SKIPPED
              bootlin-armv7-musl [ 7/45]: OK
        bootlin-m68k-5208-uclibc [ 8/45]: SKIPPED
       bootlin-m68k-68040-uclibc [ 9/45]: SKIPPED
     bootlin-microblazeel-uclibc [10/45]: SKIPPED
        bootlin-mipsel32r6-glibc [11/45]: SKIPPED
           bootlin-mipsel-uclibc [12/45]: SKIPPED
             bootlin-nios2-glibc [13/45]: SKIPPED
         bootlin-openrisc-uclibc [14/45]: SKIPPED
bootlin-powerpc64le-power8-glibc [15/45]: SKIPPED
   bootlin-powerpc-e500mc-uclibc [16/45]: SKIPPED
           bootlin-riscv32-glibc [17/45]: SKIPPED
           bootlin-riscv64-glibc [18/45]: SKIPPED
            bootlin-riscv64-musl [19/45]: SKIPPED
         bootlin-s390x-z13-glibc [20/45]: FAILED
Quoted 3 lines
Build fails with output:
    Makefile:23: *** "The architecture s390x isn't supported".  Stop.
Another problem that occured...
After I use $(BR2_NORMALIZED_ARCH) instead of $(BR2_ARCH), the arch is at least supported. But it fails with
Error (compel/src/lib/handle-elf-host.c:161): Unsupported header detected make[3]: *** [criu/pie/Makefile:58: criu/pie/parasite-blob.h] Error 255
After some more investigation, it ended up here:
static bool is_header_supported(Elf_Ehdr *hdr)
{
	if (!arch_is_machine_supported(hdr->e_machine))
		return false;
	if ((hdr->e_type != ET_REL
#ifdef NO_RELOCS
	     && hdr->e_type != ET_EXEC
#endif
	     ) ||
	    hdr->e_version != EV_CURRENT)
		return false;
	return true;
}
I've added debug trace to see why it fails and got this:
Error (compel/src/lib/handle-elf-host.c:155): type 0x100 machine 0x1600 version 0x1000000
Error (compel/src/lib/handle-elf-host.c:158): ET_REL 0x1 EM_s390 0x16, EV_CURRENT 0x1

Quoted 7 lines

It is a tool that runs on host (little endian), but read data of an elf compiled for s390 (big endian), so bytes is read swapped.
Right now I thinking of not list s390 as a compatible architecture for this package.
Do you think it is reasonable?

Yes, I think it is reasonable to exclude s390 for now.

You can also add a comment about that in the _ARCH_SUPPORTS, that criu does have "some" support for s390 but it is not included due to this endianess issue.

This can be improved later, if there is Buildroot criu users on s390.

Best regards,

Julien.
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot