xz: Add ptest support
PATCH v3 1/1 xz: Add ptest support
There are two types of cases: executables and POSIX shell scripts. All test cases PASS.
Add xz-ptest to PTESTS_FAST because test duration less than 30s on qemux86-64.
root@qemux86-64:~# ptest-runner xz START: ptest-runner 2024-01-26T03:32 BEGIN: /usr/lib/xz/ptest === test_bcj_exact_size.c === PASS: test_exact_size PASS: test_empty_block --- === END === ... === test_vli.c === PASS: test_lzma_vli_size PASS: test_lzma_vli_encode PASS: test_lzma_vli_decode --- === END === PASS: test_files.sh PASS: test_scripts.sh PASS: test_compress.sh DURATION: 18 END: /usr/lib/xz/ptest 2024-01-26T03:33 STOP: ptest-runner TOTAL: 1 FAIL: 0
--- .../distro/include/ptest-packagelists.inc | 1 + meta/recipes-extended/xz/xz/run-ptest | 26 +++++++++++++++++ meta/recipes-extended/xz/xz_5.4.5.bb | 28 +++++++++++++++++-- 3 files changed, 53 insertions(+), 2 deletions(-) create mode 100644 meta/recipes-extended/xz/xz/run-ptest
meta/conf/distro/include/ptest-packagelists.inc +1 -0
@@ -77,6 +77,7 @@ PTESTS_FAST = "\ sed \ slang \ wayland \ + xz \ zlib \ libexif \ "
meta/recipes-extended/xz/xz/run-ptest new file +26 -0
@@ -0,0 +1,26 @@ +#!/bin/bash + +ptestdir=$(dirname "$(readlink -f "$0")") +cd "$ptestdir"/tests || exit + +# executables test cases +tests_exec=$(file $(ls test_*) | grep "ELF" | awk -F: '{print $1}') +for i in ${tests_exec}; do + ./$i +done + +# POSIX shell script test cases +export srcdir=${ptestdir}/tests +test_shell="test_files.sh test_scripts.sh test_compress.sh" +for j in ${test_shell}; do + if [ $j == "test_compress.sh" ]; then + ./$j create_compress_files + else + ./$j + fi + if [ $? -eq 0 ]; then + echo "PASS: $j" + else + echo "FAIL: $j" + fi +done
meta/recipes-extended/xz/xz_5.4.5.bb +26 -2
@@ -24,13 +24,15 @@ LIC_FILES_CHKSUM = "file://COPYING;md5=c8ea84ebe7b93cce676b54355dc6b2c0 \ file://lib/getopt.c;endline=23;md5=2069b0ee710572c03bb3114e4532cd84 \ " -SRC_URI = "https://tukaani.org/xz/xz-${PV}.tar.gz" +SRC_URI = "https://tukaani.org/xz/xz-${PV}.tar.gz \ + ${@bb.utils.contains('PTEST_ENABLED', '1', 'file://run-ptest', '', d)} \ + " SRC_URI[sha256sum] = "135c90b934aee8fbc0d467de87a05cb70d627da36abe518c357a873709e5b7d6" UPSTREAM_CHECK_REGEX = "xz-(?P<pver>\d+(\.\d+)+)\.tar" CACHED_CONFIGUREVARS += "gl_cv_posix_shell=/bin/sh" -inherit autotools gettext +inherit autotools gettext ptest PACKAGES =+ "liblzma"
@@ -42,3 +44,25 @@ ALTERNATIVE:${PN} = "xz xzcat unxz \ lzma lzcat unlzma" BBCLASSEXTEND = "native nativesdk" + +RDEPENDS:${PN}-ptest += "bash file" + +do_compile_ptest() { + oe_runmake check TESTS= +} + +do_install_ptest () { + install -d ${D}${PTEST_PATH}/tests + find ${B}/tests/.libs -type f -executable -exec cp {} ${D}${PTEST_PATH}/tests \; + cp ${B}/config.h ${D}${PTEST_PATH} + for i in files xzgrep_expected_output test_files.sh test_scripts.sh test_compress.sh; do + cp -r ${S}/tests/$i ${D}${PTEST_PATH}/tests + done + mkdir -p ${D}${PTEST_PATH}/src/xz + ln -s ${bindir}/xz ${D}${PTEST_PATH}/src/xz/xz + mkdir -p ${D}${PTEST_PATH}/src/xzdec + ln -s ${bindir}/xzdec ${D}${PTEST_PATH}/src/xzdec/xzdec + mkdir -p ${D}${PTEST_PATH}/src/scripts + ln -s ${bindir}/xzdiff ${D}${PTEST_PATH}/src/scripts/xzdiff + ln -s ${bindir}/xzgrep ${D}${PTEST_PATH}/src/scripts/xzgrep +}
Signature · 1 line
2.25.1
Hello,
I've had a reproducibility failure:
https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20240203-fdpg888c/packages/diff-html/
Quoted 143 lines · Chi Xu
There are two types of cases: executables and POSIX shell scripts. All test cases PASS.
Add xz-ptest toPTESTS_FASTbecause test duration less than 30s on qemux86-64.root@qemux86-64:~# ptest-runner xz START: ptest-runner 2024-01-26T03:32 BEGIN: /usr/lib/xz/ptest === test_bcj_exact_size.c === PASS: test_exact_size PASS: test_empty_block --- === END === ... === test_vli.c === PASS: test_lzma_vli_size PASS: test_lzma_vli_encode PASS: test_lzma_vli_decode --- === END === PASS: test_files.sh PASS: test_scripts.sh PASS: test_compress.sh DURATION: 18 END: /usr/lib/xz/ptest 2024-01-26T03:33 STOP: ptest-runner TOTAL: 1 FAIL: 0Signed-off-by Xu, Chi <[email protected]>--- .../distro/include/ptest-packagelists.inc | 1 + meta/recipes-extended/xz/xz/run-ptest | 26 +++++++++++++++++ meta/recipes-extended/xz/xz_5.4.5.bb | 28 +++++++++++++++++-- 3 files changed, 53 insertions(+), 2 deletions(-) create mode 100644 meta/recipes-extended/xz/xz/run-ptestmeta/conf/distro/include/ptest-packagelists.inc +1
-0@@ -77,6 +77,7 @@ PTESTS_FAST = "\ sed \ slang \ wayland \ + xz \ zlib \ libexif \ "meta/recipes-extended/xz/xz/run-ptest new file +26
-0@@ -0,0 +1,26 @@ +#!/bin/bash + +ptestdir=$(dirname "$(readlink -f "$0")") +cd "$ptestdir"/tests || exit + +# executables test cases +tests_exec=$(file $(ls test_*) | grep "ELF" | awk -F: '{print $1}') +for i in ${tests_exec}; do + ./$i +done + +# POSIX shell script test cases +export srcdir=${ptestdir}/tests +test_shell="test_files.sh test_scripts.sh test_compress.sh" +for j in ${test_shell}; do + if [ $j == "test_compress.sh" ]; then + ./$j create_compress_files + else + ./$j + fi + if [ $? -eq 0 ]; then + echo "PASS: $j" + else + echo "FAIL: $j" + fi +donemeta/recipes-extended/xz/xz_5.4.5.bb +26
-2@@ -24,13 +24,15 @@ LIC_FILES_CHKSUM = "file://COPYING;md5=c8ea84ebe7b93cce676b54355dc6b2c0 \ file://lib/getopt.c;endline=23;md5=2069b0ee710572c03bb3114e4532cd84 \ " -SRC_URI = "https://tukaani.org/xz/xz-${PV}.tar.gz" +SRC_URI = "https://tukaani.org/xz/xz-${PV}.tar.gz \ + ${@bb.utils.contains('PTEST_ENABLED', '1', 'file://run-ptest', '', d)} \ + " SRC_URI[sha256sum] = "135c90b934aee8fbc0d467de87a05cb70d627da36abe518c357a873709e5b7d6" UPSTREAM_CHECK_REGEX = "xz-(?P<pver>\d+(\.\d+)+)\.tar" CACHED_CONFIGUREVARS += "gl_cv_posix_shell=/bin/sh" -inherit autotools gettext +inherit autotools gettext ptest PACKAGES =+ "liblzma"@@ -42,3 +44,25 @@ ALTERNATIVE:${PN} = "xz xzcat unxz \ lzma lzcat unlzma" BBCLASSEXTEND = "native nativesdk" + +RDEPENDS:${PN}-ptest += "bash file" + +do_compile_ptest() { + oe_runmake check TESTS= +} + +do_install_ptest () { + install -d ${D}${PTEST_PATH}/tests + find ${B}/tests/.libs -type f -executable -exec cp {} ${D}${PTEST_PATH}/tests \; + cp ${B}/config.h ${D}${PTEST_PATH} + for i in files xzgrep_expected_output test_files.sh test_scripts.sh test_compress.sh; do + cp -r ${S}/tests/$i ${D}${PTEST_PATH}/tests + done + mkdir -p ${D}${PTEST_PATH}/src/xz + ln -s ${bindir}/xz ${D}${PTEST_PATH}/src/xz/xz + mkdir -p ${D}${PTEST_PATH}/src/xzdec + ln -s ${bindir}/xzdec ${D}${PTEST_PATH}/src/xzdec/xzdec + mkdir -p ${D}${PTEST_PATH}/src/scripts + ln -s ${bindir}/xzdiff ${D}${PTEST_PATH}/src/scripts/xzdiff + ln -s ${bindir}/xzgrep ${D}${PTEST_PATH}/src/scripts/xzgrep +}Signature · 11 lines
2.25.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#194345): https://lists.openembedded.org/g/openembedded-core/message/194345 Mute This Topic: https://lists.openembedded.org/mt/103971280/3617179 Group Owner: [email protected] Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
Signature · 3 lines
Alexandre Belloni, co-owner and COO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
Hello,
I'm checking this issue, but can not reproduce it on my host now. I tried to build and rebuild xz-ptest on ubuntu 20.04 and ubuntu 22.04, xz-ptest-5.4.5-r0.core2_64.rpm are same.
Do you have any suggestions?
Ubuntu 20.04
root@localhost:/root$ git clone git://git.yoctoproject.org/poky root@localhost:/root/poky$ cd poky root@localhost:/root/poky$ source oe-init-build-env root@localhost:/root/poky/build$ bitbake xz root@localhost:/root/poky/build$ ls -l tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm -rw-r--r-- 2 root root 162489 Feb 5 13:27 tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm root@localhost:/root/poky/build$ md5sum tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm b26fd0014d1c23f6e3d9bf8704137b82 tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm root@localhost:/root/poky/build$ bitbake -c cleansstate xz root@localhost:/root/poky/build$ ls -l tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm -rw-r--r-- 2 root root 162489 Feb 5 13:32 tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm root@localhost:/root/poky/build$ md5sum tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm b26fd0014d1c23f6e3d9bf8704137b82 tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm
Ubuntu 22.04
root@localhost:/root/poky/build$ ls -l tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm -rw-r--r-- 2 root root 162489 Feb 5 13:56 tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm root@localhost:/root/poky/build$ md5sum tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm b26fd0014d1c23f6e3d9bf8704137b82 tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm root@localhost:/root/poky/build$ bitbake -c cleansstate xz root@localhost:/root/poky/build$ ls -l tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm -rw-r--r-- 2 root root 162489 Feb 5 13:59 tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm root@localhost:/root/poky/build$ md5sum tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm b26fd0014d1c23f6e3d9bf8704137b82 tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm
Thanks,
Chi
Quoted 179 lines
-----Original Message-----
From: Alexandre Belloni <[email protected]>
Sent: Sunday, February 4, 2024 5:23 AM
To: Xu, Chi <[email protected]>
Cc: [email protected]
Subject: Re: [OE-core] [PATCH v3] xz: Add ptest supportCAUTION: This email comes from a non Wind River email account! Do not click links or open attachments unless you recognize the sender and know the content is safe.
Hello,
I've had a reproducibility failure:
https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20240203-
fdpg888c/packages/diff-html/meta/recipes-extended/xz/xz_5.4.5.bb LIC_FILES_CHKSUM =
Quoted 90 lines · Chi Xu
There are two types of cases: executables and POSIX shell scripts. All test cases PASS.
Add xz-ptest toPTESTS_FASTbecause test duration less than 30s on qemux86-64.root@qemux86-64:~# ptest-runner xz START: ptest-runner 2024-01-26T03:32 BEGIN: /usr/lib/xz/ptest === test_bcj_exact_size.c === PASS: test_exact_size PASS: test_empty_block --- === END === ... === test_vli.c === PASS: test_lzma_vli_size PASS: test_lzma_vli_encode PASS: test_lzma_vli_decode --- === END === PASS: test_files.sh PASS: test_scripts.sh PASS: test_compress.sh DURATION: 18 END: /usr/lib/xz/ptest 2024-01-26T03:33 STOP: ptest-runner TOTAL: 1 FAIL: 0Signed-off-by Xu, Chi <[email protected]>--- .../distro/include/ptest-packagelists.inc | 1 + meta/recipes-extended/xz/xz/run-ptest | 26 +++++++++++++++++ meta/recipes-extended/xz/xz_5.4.5.bb | 28 +++++++++++++++++-- 3 files changed, 53 insertions(+), 2 deletions(-) create mode 100644meta/recipes-extended/xz/xz/run-ptest
meta/conf/distro/include/ptest-packagelists.inc +1
-0@@ -77,6 +77,7 @@ PTESTS_FAST = "\ sed \ slang \ wayland \ + xz \ zlib \ libexif \ "meta/recipes-extended/xz/xz/run-ptest new file +26
-0@@ -0,0 +1,26 @@ +#!/bin/bash + +ptestdir=$(dirname "$(readlink -f "$0")") cd "$ptestdir"/tests || +exit + +# executables test cases +tests_exec=$(file $(ls test_*) | grep "ELF" | awk -F: '{print $1}') +for i in ${tests_exec}; do + ./$i +done + +# POSIX shell script test cases +export srcdir=${ptestdir}/tests +test_shell="test_files.sh test_scripts.sh test_compress.sh" +for j in ${test_shell}; do + if [ $j == "test_compress.sh" ]; then + ./$j create_compress_files + else + ./$j + fi + if [ $? -eq 0 ]; then + echo "PASS: $j" + else + echo "FAIL: $j" + fi +donemeta/recipes-extended/xz/xz_5.4.5.bb +0
-0@@ -24,13 +24,15 @@ LIC_FILES_CHKSUM ="file://COPYING;md5=c8ea84ebe7b93cce676b54355dc6b2c0 \
Quoted 1 line
file://lib/getopt.c;endline=23;md5=2069b0ee710572c03bb3114e4532cd84 \
meta/recipes-extended/xz/xz_5.4.5.bb LIC_FILES_CHKSUM = "file://COPYING;md5=c8ea84ebe7b93cce676b54355dc6b2c0 \
Quoted 7 lines
"meta/recipes-extended/xz/xz_5.4.5.bb +3
-1-SRC_URI = "https://tukaani.org/xz/xz-${PV}.tar.gz" +SRC_URI = "https://tukaani.org/xz/xz-${PV}.tar.gz \ + ${@bb.utils.contains('PTEST_ENABLED', '1', 'file://run-ptest', '', d)} \ + " SRC_URI[sha256sum] ="
135c90b934aee8fbc0d467de87a05cb70d627da36abe518c357a873709e5b7d6"Quoted 13 lines
UPSTREAM_CHECK_REGEX= "xz-(?P<pver>\d+(\.\d+)+)\.tar"
CACHED_CONFIGUREVARS+= "gl_cv_posix_shell=/bin/sh"meta/recipes-extended/xz/xz_5.4.5.bb +1
-1-inherit autotools gettext +inherit autotools gettext ptest PACKAGES =+ "liblzma"@@ -42,3 +44,25 @@ ALTERNATIVE:${PN} = "xz xzcat unxz \ lzma lzcat unlzma" BBCLASSEXTEND = "native nativesdk"meta/recipes-extended/xz/xz_5.4.5.bb ALTERNATIVE:${PN} = "xz xzcat unxz \ · line 56
Quoted 10 lines
meta/recipes-extended/xz/xz_5.4.5.bb +10
-0+ +RDEPENDS:${PN}-ptest += "bash file" + +do_compile_ptest() { + oe_runmake check TESTS= +} + +do_install_ptest () { + install -d ${D}${PTEST_PATH}/tests + find ${B}/tests/.libs -type f -executable -exec cp {}${D}${PTEST_PATH}/tests \;
meta/recipes-extended/xz/xz_5.4.5.bb ALTERNATIVE:${PN} = "xz xzcat unxz \ · line 58
Quoted 2 lines
meta/recipes-extended/xz/xz_5.4.5.bb +2
-0+ cp ${B}/config.h ${D}${PTEST_PATH} + for i in files xzgrep_expected_output test_files.sh test_scripts.sh
test_compress.sh; doQuoted 15 lines
meta/recipes-extended/xz/xz_5.4.5.bb +10
-1+ cp -r ${S}/tests/$i ${D}${PTEST_PATH}/tests + done + mkdir -p ${D}${PTEST_PATH}/src/xz + ln -s ${bindir}/xz ${D}${PTEST_PATH}/src/xz/xz + mkdir -p ${D}${PTEST_PATH}/src/xzdec + ln -s ${bindir}/xzdec ${D}${PTEST_PATH}/src/xzdec/xzdec + mkdir -p ${D}${PTEST_PATH}/src/scripts + ln -s ${bindir}/xzdiff ${D}${PTEST_PATH}/src/scripts/xzdiff + ln -s ${bindir}/xzgrep ${D}${PTEST_PATH}/src/scripts/xzgrep +} --2.25.1
Quoted 7 lines
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#194345):
https://lists.openembedded.org/g/openembedded-core/message/194345
Mute This Topic: https://lists.openembedded.org/mt/103971280/3617179
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
Quoted 3 lines
[[email protected]]
-=-=-=-=-=-=-=-=-=-=-=---
Alexandre Belloni, co-owner and COO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
Hi,
FYI, since the reproducibility problem only impact the RPM package, it might be related to the problem noticed by Richard last night on IRC:
[11:35:32 pm] <RP> kanavin: It looks like it runs file on the files within the archive but for some reason it isn't sorting that in the header, so probably an unsorted directory listing somewhere
[11:58:34 pm] <RP> kanavin: looking at package_rpm and it's os.walk, there is no sorting in there at all. I suspect that leads to non-determinstic ordering of the files sections in the spec files and I suspect that in turn leads rpm to reorder headers
(https://libera.irclog.whitequark.org/yocto/2024-02-04#35724802;)
Quoted 238 lines · Chi Xu
Hello,
I'm checking this issue, but can not reproduce it on my host now. I tried to build and rebuild xz-ptest on ubuntu 20.04 and ubuntu 22.04, xz-ptest-5.4.5-r0.core2_64.rpm are same.
Do you have any suggestions?Ubuntu 20.04
root@localhost:/root$ git clone git://git.yoctoproject.org/poky root@localhost:/root/poky$ cd poky root@localhost:/root/poky$ source oe-init-build-env root@localhost:/root/poky/build$ bitbake xz root@localhost:/root/poky/build$ ls -l tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm -rw-r--r-- 2 root root 162489 Feb 5 13:27 tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm root@localhost:/root/poky/build$ md5sum tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm b26fd0014d1c23f6e3d9bf8704137b82 tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm root@localhost:/root/poky/build$ bitbake -c cleansstate xz root@localhost:/root/poky/build$ ls -l tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm -rw-r--r-- 2 root root 162489 Feb 5 13:32 tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm root@localhost:/root/poky/build$ md5sum tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm b26fd0014d1c23f6e3d9bf8704137b82 tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpmUbuntu 22.04
root@localhost:/root/poky/build$ ls -l tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm -rw-r--r-- 2 root root 162489 Feb 5 13:56 tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm root@localhost:/root/poky/build$ md5sum tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm b26fd0014d1c23f6e3d9bf8704137b82 tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm root@localhost:/root/poky/build$ bitbake -c cleansstate xz root@localhost:/root/poky/build$ ls -l tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm -rw-r--r-- 2 root root 162489 Feb 5 13:59 tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm root@localhost:/root/poky/build$ md5sum tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm b26fd0014d1c23f6e3d9bf8704137b82 tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpmThanks,
ChiQuoted 9 lines
-----Original Message-----
From: Alexandre Belloni <[email protected]>
Sent: Sunday, February 4, 2024 5:23 AM
To: Xu, Chi <[email protected]>
Cc: [email protected]
Subject: Re: [OE-core] [PATCH v3] xz: Add ptest supportCAUTION: This email comes from a non Wind River email account! Do not click links or open attachments unless you recognize the sender
and
Quoted 45 lines
know the content is safe.
Hello,
I've had a reproducibility failure:
https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20240203-
fdpg888c/packages/diff-html/Quoted 35 lines · Chi Xu
There are two types of cases: executables and POSIX shell scripts. All test cases PASS.
Add xz-ptest toPTESTS_FASTbecause test duration less than 30s on qemux86-64.root@qemux86-64:~# ptest-runner xz START: ptest-runner 2024-01-26T03:32 BEGIN: /usr/lib/xz/ptest === test_bcj_exact_size.c === PASS: test_exact_size PASS: test_empty_block --- === END === ... === test_vli.c === PASS: test_lzma_vli_size PASS: test_lzma_vli_encode PASS: test_lzma_vli_decode --- === END === PASS: test_files.sh PASS: test_scripts.sh PASS: test_compress.sh DURATION: 18 END: /usr/lib/xz/ptest 2024-01-26T03:33 STOP: ptest-runner TOTAL: 1 FAIL: 0Signed-off-by Xu, Chi <[email protected]>--- .../distro/include/ptest-packagelists.inc | 1 + meta/recipes-extended/xz/xz/run-ptest | 26 +++++++++++++++++ meta/recipes-extended/xz/xz_5.4.5.bb | 28+++++++++++++++++--
Quoted 57 lines
meta/recipes-extended/xz/xz_5.4.5.bb LIC_FILES_CHKSUM =
Quoted 55 lines
3 files changed, 53 insertions(+), 2 deletions(-) create mode 100644meta/recipes-extended/xz/xz/run-ptest
meta/conf/distro/include/ptest-packagelists.inc +1
-0@@ -77,6 +77,7 @@ PTESTS_FAST = "\ sed \ slang \ wayland \ + xz \ zlib \ libexif \ "meta/recipes-extended/xz/xz/run-ptest new file +26
-0@@ -0,0 +1,26 @@ +#!/bin/bash + +ptestdir=$(dirname "$(readlink -f "$0")") cd "$ptestdir"/tests || +exit + +# executables test cases +tests_exec=$(file $(ls test_*) | grep "ELF" | awk -F: '{print $1}') +for i in ${tests_exec}; do + ./$i +done + +# POSIX shell script test cases +export srcdir=${ptestdir}/tests +test_shell="test_files.sh test_scripts.sh test_compress.sh" +for j in ${test_shell}; do + if [ $j == "test_compress.sh" ]; then + ./$j create_compress_files + else + ./$j + fi + if [ $? -eq 0 ]; then + echo "PASS: $j" + else + echo "FAIL: $j" + fi +donemeta/recipes-extended/xz/xz_5.4.5.bb +0
-0@@ -24,13 +24,15 @@ LIC_FILES_CHKSUM ="file://COPYING;md5=c8ea84ebe7b93cce676b54355dc6b2c0 \
Quoted 1 line
Quoted 6 lines
file://lib/getopt.c;endline=23;md5=2069b0ee710572c03bb3114e4532cd84 \
meta/recipes-extended/xz/xz_5.4.5.bb LIC_FILES_CHKSUM = "file://COPYING;md5=c8ea84ebe7b93cce676b54355dc6b2c0 \
Quoted 5 lines
"meta/recipes-extended/xz/xz_5.4.5.bb +2
-1-SRC_URI = "https://tukaani.org/xz/xz-${PV}.tar.gz" +SRC_URI = "https://tukaani.org/xz/xz-${PV}.tar.gz \ + ${@bb.utils.contains('PTEST_ENABLED', '1','file://run-ptest', '', d)} \
Quoted 16 lines
Quoted 2 lines
+ "
SRC_URI[sha256sum] ="
135c90b934aee8fbc0d467de87a05cb70d627da36abe518c357a873709e5b7d6"Quoted 12 lines
UPSTREAM_CHECK_REGEX= "xz-(?P<pver>\d+(\.\d+)+)\.tar"
CACHED_CONFIGUREVARS+= "gl_cv_posix_shell=/bin/sh"meta/recipes-extended/xz/xz_5.4.5.bb +1
-1-inherit autotools gettext +inherit autotools gettext ptest PACKAGES =+ "liblzma"@@ -42,3 +44,25 @@ ALTERNATIVE:${PN} = "xz xzcat unxz \ lzma lzcat unlzma"Quoted 14 lines
meta/recipes-extended/xz/xz_5.4.5.bb ALTERNATIVE:${PN} = "xz xzcat unxz \
Quoted 11 lines
meta/recipes-extended/xz/xz_5.4.5.bb +10
-0BBCLASSEXTEND = "native nativesdk" + +RDEPENDS:${PN}-ptest += "bash file" + +do_compile_ptest() { + oe_runmake check TESTS= +} + +do_install_ptest () { + install -d ${D}${PTEST_PATH}/tests + find ${B}/tests/.libs -type f -executable -exec cp {}${D}${PTEST_PATH}/tests \;
Quoted 2 lines
meta/recipes-extended/xz/xz_5.4.5.bb +2
-0+ cp ${B}/config.h ${D}${PTEST_PATH} + for i in files xzgrep_expected_output test_files.sh
test_scripts.shQuoted 20 lines
test_compress.sh; doQuoted 11 lines
meta/recipes-extended/xz/xz_5.4.5.bb +10
-1+ cp -r ${S}/tests/$i ${D}${PTEST_PATH}/tests + done + mkdir -p ${D}${PTEST_PATH}/src/xz + ln -s ${bindir}/xz ${D}${PTEST_PATH}/src/xz/xz + mkdir -p ${D}${PTEST_PATH}/src/xzdec + ln -s ${bindir}/xzdec ${D}${PTEST_PATH}/src/xzdec/xzdec + mkdir -p ${D}${PTEST_PATH}/src/scripts + ln -s ${bindir}/xzdiff ${D}${PTEST_PATH}/src/scripts/xzdiff + ln -s ${bindir}/xzgrep ${D}${PTEST_PATH}/src/scripts/xzgrep +} --Quoted 6 lines
2.25.1
Quoted 3 lines
--
Alexandre Belloni, co-owner and COO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#194916):
https://lists.openembedded.org/g/openembedded-core/message/194916
Mute This Topic: https://lists.openembedded.org/mt/103971280/4316185
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ [email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Signature · 2 lines
Yoann Congal Smile ECS - Tech expert
I just sent two patches for this.
Alex
Quoted 233 lines · Yoann Congal
Hi,
FYI, since the reproducibility problem only impact the RPM package, it might be related to the problem noticed by Richard last night on IRC:
[11:35:32 pm] <RP> kanavin: It looks like it runs file on the files within the archive but for some reason it isn't sorting that in the header, so probably an unsorted directory listing somewhere
[11:58:34 pm] <RP> kanavin: looking atpackage_rpmand it's os.walk, there is no sorting in there at all. I suspect that leads to non-determinstic ordering of the files sections in the spec files and I suspect that in turn leads rpm to reorder headers
(https://libera.irclog.whitequark.org/yocto/2024-02-04#35724802;)Quoted 34 lines · Chi Xu
Hello,
I'm checking this issue, but can not reproduce it on my host now. I tried to build and rebuild xz-ptest on ubuntu 20.04 and ubuntu 22.04, xz-ptest-5.4.5-r0.core2_64.rpm are same.
Do you have any suggestions?Ubuntu 20.04
root@localhost:/root$ git clone git://git.yoctoproject.org/poky root@localhost:/root/poky$ cd poky root@localhost:/root/poky$ source oe-init-build-env root@localhost:/root/poky/build$ bitbake xz root@localhost:/root/poky/build$ ls -l tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm -rw-r--r-- 2 root root 162489 Feb 5 13:27 tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm root@localhost:/root/poky/build$ md5sum tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm b26fd0014d1c23f6e3d9bf8704137b82 tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm root@localhost:/root/poky/build$ bitbake -c cleansstate xz root@localhost:/root/poky/build$ ls -l tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm -rw-r--r-- 2 root root 162489 Feb 5 13:32 tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm root@localhost:/root/poky/build$ md5sum tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm b26fd0014d1c23f6e3d9bf8704137b82 tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpmUbuntu 22.04
root@localhost:/root/poky/build$ ls -l tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm -rw-r--r-- 2 root root 162489 Feb 5 13:56 tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm root@localhost:/root/poky/build$ md5sum tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm b26fd0014d1c23f6e3d9bf8704137b82 tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm root@localhost:/root/poky/build$ bitbake -c cleansstate xz root@localhost:/root/poky/build$ ls -l tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm -rw-r--r-- 2 root root 162489 Feb 5 13:59 tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm root@localhost:/root/poky/build$ md5sum tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpm b26fd0014d1c23f6e3d9bf8704137b82 tmp/work/core2-64-poky-linux/xz/5.4.5/deploy-rpms/core2_64/xz-ptest-5.4.5-r0.core2_64.rpmQuoted 176 lines
Thanks,
ChiQuoted 170 lines
-----Original Message-----
From: Alexandre Belloni <[email protected]>
Sent: Sunday, February 4, 2024 5:23 AM
To: Xu, Chi <[email protected]>
Cc: [email protected]
Subject: Re: [OE-core] [PATCH v3] xz: Add ptest supportCAUTION: This email comes from a non Wind River email account! Do not click links or open attachments unless you recognize the sender and know the content is safe.
Hello,
I've had a reproducibility failure:
https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20240203-
fdpg888c/packages/diff-html/meta/recipes-extended/xz/xz_5.4.5.bb LIC_FILES_CHKSUM =
Quoted 90 lines · Chi Xu
There are two types of cases: executables and POSIX shell scripts. All test cases PASS.
Add xz-ptest toPTESTS_FASTbecause test duration less than 30s on qemux86-64.root@qemux86-64:~# ptest-runner xz START: ptest-runner 2024-01-26T03:32 BEGIN: /usr/lib/xz/ptest === test_bcj_exact_size.c === PASS: test_exact_size PASS: test_empty_block --- === END === ... === test_vli.c === PASS: test_lzma_vli_size PASS: test_lzma_vli_encode PASS: test_lzma_vli_decode --- === END === PASS: test_files.sh PASS: test_scripts.sh PASS: test_compress.sh DURATION: 18 END: /usr/lib/xz/ptest 2024-01-26T03:33 STOP: ptest-runner TOTAL: 1 FAIL: 0Signed-off-by Xu, Chi <[email protected]>--- .../distro/include/ptest-packagelists.inc | 1 + meta/recipes-extended/xz/xz/run-ptest | 26 +++++++++++++++++ meta/recipes-extended/xz/xz_5.4.5.bb | 28 +++++++++++++++++-- 3 files changed, 53 insertions(+), 2 deletions(-) create mode 100644meta/recipes-extended/xz/xz/run-ptest
meta/conf/distro/include/ptest-packagelists.inc +1
-0@@ -77,6 +77,7 @@ PTESTS_FAST = "\ sed \ slang \ wayland \ + xz \ zlib \ libexif \ "meta/recipes-extended/xz/xz/run-ptest new file +26
-0@@ -0,0 +1,26 @@ +#!/bin/bash + +ptestdir=$(dirname "$(readlink -f "$0")") cd "$ptestdir"/tests || +exit + +# executables test cases +tests_exec=$(file $(ls test_*) | grep "ELF" | awk -F: '{print $1}') +for i in ${tests_exec}; do + ./$i +done + +# POSIX shell script test cases +export srcdir=${ptestdir}/tests +test_shell="test_files.sh test_scripts.sh test_compress.sh" +for j in ${test_shell}; do + if [ $j == "test_compress.sh" ]; then + ./$j create_compress_files + else + ./$j + fi + if [ $? -eq 0 ]; then + echo "PASS: $j" + else + echo "FAIL: $j" + fi +donemeta/recipes-extended/xz/xz_5.4.5.bb +0
-0@@ -24,13 +24,15 @@ LIC_FILES_CHKSUM ="file://COPYING;md5=c8ea84ebe7b93cce676b54355dc6b2c0 \
Quoted 1 line
file://lib/getopt.c;endline=23;md5=2069b0ee710572c03bb3114e4532cd84 \
meta/recipes-extended/xz/xz_5.4.5.bb LIC_FILES_CHKSUM = "file://COPYING;md5=c8ea84ebe7b93cce676b54355dc6b2c0 \
Quoted 7 lines
"meta/recipes-extended/xz/xz_5.4.5.bb +3
-1-SRC_URI = "https://tukaani.org/xz/xz-${PV}.tar.gz" +SRC_URI = "https://tukaani.org/xz/xz-${PV}.tar.gz \ + ${@bb.utils.contains('PTEST_ENABLED', '1', 'file://run-ptest', '', d)} \ + " SRC_URI[sha256sum] ="
135c90b934aee8fbc0d467de87a05cb70d627da36abe518c357a873709e5b7d6"Quoted 13 lines
UPSTREAM_CHECK_REGEX= "xz-(?P<pver>\d+(\.\d+)+)\.tar"
CACHED_CONFIGUREVARS+= "gl_cv_posix_shell=/bin/sh"meta/recipes-extended/xz/xz_5.4.5.bb +1
-1-inherit autotools gettext +inherit autotools gettext ptest PACKAGES =+ "liblzma"@@ -42,3 +44,25 @@ ALTERNATIVE:${PN} = "xz xzcat unxz \ lzma lzcat unlzma" BBCLASSEXTEND = "native nativesdk"meta/recipes-extended/xz/xz_5.4.5.bb ALTERNATIVE:${PN} = "xz xzcat unxz \ · line 56
Quoted 10 lines
meta/recipes-extended/xz/xz_5.4.5.bb +10
-0+ +RDEPENDS:${PN}-ptest += "bash file" + +do_compile_ptest() { + oe_runmake check TESTS= +} + +do_install_ptest () { + install -d ${D}${PTEST_PATH}/tests + find ${B}/tests/.libs -type f -executable -exec cp {}${D}${PTEST_PATH}/tests \;
meta/recipes-extended/xz/xz_5.4.5.bb ALTERNATIVE:${PN} = "xz xzcat unxz \ · line 58
Quoted 2 lines
meta/recipes-extended/xz/xz_5.4.5.bb +2
-0+ cp ${B}/config.h ${D}${PTEST_PATH} + for i in files xzgrep_expected_output test_files.sh test_scripts.sh
test_compress.sh; doQuoted 11 lines
meta/recipes-extended/xz/xz_5.4.5.bb +10
-1+ cp -r ${S}/tests/$i ${D}${PTEST_PATH}/tests + done + mkdir -p ${D}${PTEST_PATH}/src/xz + ln -s ${bindir}/xz ${D}${PTEST_PATH}/src/xz/xz + mkdir -p ${D}${PTEST_PATH}/src/xzdec + ln -s ${bindir}/xzdec ${D}${PTEST_PATH}/src/xzdec/xzdec + mkdir -p ${D}${PTEST_PATH}/src/scripts + ln -s ${bindir}/xzdiff ${D}${PTEST_PATH}/src/scripts/xzdiff + ln -s ${bindir}/xzgrep ${D}${PTEST_PATH}/src/scripts/xzgrep +} --Quoted 6 lines
2.25.1
--
Alexandre Belloni, co-owner and COO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com--
Yoann Congal
Smile ECS - Tech expert-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#194917): https://lists.openembedded.org/g/openembedded-core/message/194917
Mute This Topic: https://lists.openembedded.org/mt/103971280/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-