curl : Add ptest
PATCH v4 1/1 curl : Add ptest
- curl-ptest is taking around 200 seconds to execute so
added curl-ptest to PTESTS_SLOW
- This patch is rework on an existing patch provided
by Maxin B. John ([email protected])
https://www.openembedded.org/pipermail/openembedded-core/2017-July/139176.html
- Below is the run log of curl-ptest
START: ptest-runner
2022-07-03T15:52
BEGIN: /usr/lib/curl/ptest
********* System characteristics ********
* curl 7.83.1 (x86_64-poky-linux-gnu)
* libcurl/7.83.1 OpenSSL/3.0.3 zlib/1.2.12 libidn2/2.3.2
* Features: alt-svc AsynchDNS Debug HSTS HTTPS-proxy IDN Largefile libz NTLM SSL TLS-SRP UnixSockets
* Disabled: headers-api
* Host: qemux86-64
* System: Linux qemux86-64 5.15.44-yocto-standard #1 SMP PREEMPT Tue May 31 20:28:59 UTC 2022 x86_64 GNU/Linux
* OS: linux
* Servers: HTTP-unix
* Env:
* Seed: 238593
*****************************************
PASS: test 0001 (1 out of 1466, remaining: 25:07, took 1.029s, duration: 00:01)
PASS: test 0002 (2 out of 1466, remaining: 13:21, took 0.065s, duration: 00:01)
...
...
PASS: test 3019 (1460 out of 1466, remaining: 00:00, took 0.012s, duration: 03:16)
PASS: test 3020 (1461 out of 1466, remaining: 00:00, took 0.011s, duration: 03:16)
test 3025...The tool set in the test case for this: 'lib3025' does not exist
TESTDONE: 1280 tests were considered during 197 seconds.
TESTDONE: 783 tests out of PASS: 783 report: 100%
DURATION: 202
END: /usr/lib/curl/ptest
2022-07-03T15:56
STOP: ptest-runner
TOTAL: 1 FAIL: 0
- disable the curl tests that are expected to fail
- remove the generated file configurehelp.pm from curl test beacuse it is causing reproducible build failure. this file is used by some curl tests to scan symbols from curl headers. we are anyway not installing curl headers and already have disabled those tests.
[YOCTO #6707]
---
v4:removed configurehelp.pm from curl test which is causing reproducible build error
.../distro/include/ptest-packagelists.inc | 1 + meta/recipes-support/curl/curl/disable-tests | 28 +++++++++++++++++++ meta/recipes-support/curl/curl/run-ptest | 6 ++++ meta/recipes-support/curl/curl_7.84.0.bb | 21 +++++++++++++- 4 files changed, 55 insertions(+), 1 deletion(-) create mode 100644 meta/recipes-support/curl/curl/disable-tests create mode 100644 meta/recipes-support/curl/curl/run-ptest
meta/conf/distro/include/ptest-packagelists.inc +1 -0
@@ -81,6 +81,7 @@ PTESTS_SLOW = "\ babeltrace2-ptest \ busybox-ptest \ coreutils-ptest \ + curl-ptest \ dbus-ptest \ e2fsprogs-ptest \ findutils-ptest \
meta/recipes-support/curl/curl/disable-tests new file +28 -0
@@ -0,0 +1,28 @@ +# These CRL test (alt-avc) are failing +356 +412 +413 +# These CRL tests are scanning docs +971 +1119 +1132 +1135 +# These CRL tests are scnning headers +1167 +# These CRL tests are scanning man pages +1139 +1140 +1173 +1177 +# This CRL test is looking for m4 files +1165 +# This CRL test is looking for src files +1185 +# These CRL tests need --libcurl option to be enabled +1400 +1401 +1402 +1403 +1404 +1405 +1465
meta/recipes-support/curl/curl/run-ptest new file +6 -0
@@ -0,0 +1,6 @@ +#!/bin/sh +cd tests +./runtests.pl -a -n -s | sed \ + -e 's|\([^ ]* *\) \([^ ]* *\)...OK|PASS: \1 \2|' \ + -e 's|\([^ ]* *\) \([^ ]* *\)...FAILED|FAIL: \1 \2|' \ + -e 's/Warning: test[0-9]\+ not present in tests\/data\/Makefile.inc//'
meta/recipes-support/curl/curl_7.84.0.bb +20 -1
@@ -12,13 +12,15 @@ LIC_FILES_CHKSUM = "file://COPYING;md5=190c514872597083303371684954f238" SRC_URI = " \ https://curl.se/download/${BP}.tar.xz \ file://0001-easy_lock.h-include-sched.h-if-available-to-fix-buil.patch \ + file://run-ptest \ + file://disable-tests \ " SRC_URI[sha256sum] = "2d118b43f547bfe5bae806d8d47b4e596ea5b25a6c1f080aef49fbcd817c5db8" # Curl has used many names over the years... CVE_PRODUCT = "haxx:curl haxx:libcurl curl:curl curl:libcurl libcurl:libcurl daniel_stenberg:curl" -inherit autotools pkgconfig binconfig multilib_header +inherit autotools pkgconfig binconfig multilib_header ptest # Entropy source for random PACKAGECONFIG option RANDOM ?= "/dev/urandom"
@@ -83,6 +85,23 @@ do_install:append:class-target() { ${D}${bindir}/curl-config } +do_compile_ptest() { + oe_runmake test + oe_runmake -C ${B}/tests/server +} + +do_install_ptest() { + cat ${WORKDIR}/disable-tests >> ${S}/tests/data/DISABLED + rm ${B}/tests/configurehelp.pm + cp -rf ${B}/tests ${D}${PTEST_PATH} + cp -rf ${S}/tests ${D}${PTEST_PATH} + install -d ${D}${PTEST_PATH}/src + ln -sf ${bindir}/curl ${D}${PTEST_PATH}/src/curl + cp -rf ${D}${bindir}/curl-config ${D}${PTEST_PATH} +} + +RDEPENDS:${PN}-ptest += "bash perl-modules" + PACKAGES =+ "lib${BPN}" FILES:lib${BPN} = "${libdir}/lib*.so.*"
Signature · 1 line
2.36.1
Hello,
This made it in master and we found a reproducibility issue. I've opened this bug to track that:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=14863
Could you have a look?
I guess that if the tests don't need the Makefiles, we could just drop them.
Thanks!
Quoted 169 lines · Yogesh Tyagi
- curl-ptest is taking around 200 seconds to execute so
added curl-ptest toPTESTS_SLOW
- This patch is rework on an existing patch provided
by Maxin B. John ([email protected])
https://www.openembedded.org/pipermail/openembedded-core/2017-July/139176.html
- Below is the run log of curl-ptest
START: ptest-runner
2022-07-03T15:52
BEGIN:/usr/lib/curl/ptest
********* System characteristics ********
* curl 7.83.1 (x86_64-poky-linux-gnu)
* libcurl/7.83.1 OpenSSL/3.0.3 zlib/1.2.12 libidn2/2.3.2
* Features: alt-svc AsynchDNS Debug HSTS HTTPS-proxy IDN Largefile libz NTLM SSL TLS-SRP UnixSockets
* Disabled: headers-api
* Host: qemux86-64
* System: Linux qemux86-64 5.15.44-yocto-standard #1 SMP PREEMPT Tue May 31 20:28:59 UTC 2022x86_64GNU/Linux
* OS: linux
* Servers: HTTP-unix
* Env:
* Seed: 238593
*****************************************
PASS: test 0001 (1 out of 1466, remaining: 25:07, took 1.029s, duration: 00:01)
PASS: test 0002 (2 out of 1466, remaining: 13:21, took 0.065s, duration: 00:01)
...
...
PASS: test 3019 (1460 out of 1466, remaining: 00:00, took 0.012s, duration: 03:16)
PASS: test 3020 (1461 out of 1466, remaining: 00:00, took 0.011s, duration: 03:16)
test 3025...The tool set in the test case for this: 'lib3025' does not exist
TESTDONE: 1280 tests were considered during 197 seconds.
TESTDONE: 783 tests out of PASS: 783 report: 100%
DURATION: 202
END:/usr/lib/curl/ptest
2022-07-03T15:56
STOP: ptest-runner
TOTAL: 1 FAIL: 0
- disable the curl tests that are expected to fail
- remove the generated file configurehelp.pm from curl test beacuse it is causing reproducible build failure. this file is used by some curl tests to scan symbols from curl headers. we are anyway not installing curl headers and already have disabled those tests.
[YOCTO #6707]Signed-off-by Yogesh Tyagi <[email protected]>---
v4:removed configurehelp.pm from curl test which is causing reproducible build error.../distro/include/ptest-packagelists.inc | 1 + meta/recipes-support/curl/curl/disable-tests | 28 +++++++++++++++++++ meta/recipes-support/curl/curl/run-ptest | 6 ++++ meta/recipes-support/curl/curl_7.84.0.bb | 21 +++++++++++++- 4 files changed, 55 insertions(+), 1 deletion(-) create mode 100644 meta/recipes-support/curl/curl/disable-tests create mode 100644 meta/recipes-support/curl/curl/run-ptestmeta/conf/distro/include/ptest-packagelists.inc +1
-0@@ -81,6 +81,7 @@ PTESTS_SLOW = "\ babeltrace2-ptest \ busybox-ptest \ coreutils-ptest \ + curl-ptest \ dbus-ptest \ e2fsprogs-ptest \ findutils-ptest \meta/recipes-support/curl/curl/disable-tests new file +28
-0@@ -0,0 +1,28 @@ +# These CRL test (alt-avc) are failing +356 +412 +413 +# These CRL tests are scanning docs +971 +1119 +1132 +1135 +# These CRL tests are scnning headers +1167 +# These CRL tests are scanning man pages +1139 +1140 +1173 +1177 +# This CRL test is looking for m4 files +1165 +# This CRL test is looking for src files +1185 +# These CRL tests need --libcurl option to be enabled +1400 +1401 +1402 +1403 +1404 +1405 +1465meta/recipes-support/curl/curl/run-ptest new file +6
-0@@ -0,0 +1,6 @@ +#!/bin/sh +cd tests +./runtests.pl -a -n -s | sed \ + -e 's|\([^ ]* *\) \([^ ]* *\)...OK|PASS: \1 \2|' \ + -e 's|\([^ ]* *\) \([^ ]* *\)...FAILED|FAIL: \1 \2|' \ + -e 's/Warning: test[0-9]\+ not present in tests\/data\/Makefile.inc//'meta/recipes-support/curl/curl_7.84.0.bb +20
-1@@ -12,13 +12,15 @@ LIC_FILES_CHKSUM = "file://COPYING;md5=190c514872597083303371684954f238" SRC_URI = " \ https://curl.se/download/${BP}.tar.xz \ file://0001-easy_lock.h-include-sched.h-if-available-to-fix-buil.patch \ + file://run-ptest \ + file://disable-tests \ " SRC_URI[sha256sum] = "2d118b43f547bfe5bae806d8d47b4e596ea5b25a6c1f080aef49fbcd817c5db8" # Curl has used many names over the years... CVE_PRODUCT = "haxx:curl haxx:libcurl curl:curl curl:libcurl libcurl:libcurl daniel_stenberg:curl" -inherit autotools pkgconfig binconfig multilib_header +inherit autotools pkgconfig binconfig multilib_header ptest # Entropy source for random PACKAGECONFIG option RANDOM ?= "/dev/urandom"@@ -83,6 +85,23 @@ do_install:append:class-target() { ${D}${bindir}/curl-config } +do_compile_ptest() { + oe_runmake test + oe_runmake -C ${B}/tests/server +} + +do_install_ptest() { + cat ${WORKDIR}/disable-tests >> ${S}/tests/data/DISABLED + rm ${B}/tests/configurehelp.pm + cp -rf ${B}/tests ${D}${PTEST_PATH} + cp -rf ${S}/tests ${D}${PTEST_PATH} + install -d ${D}${PTEST_PATH}/src + ln -sf ${bindir}/curl ${D}${PTEST_PATH}/src/curl + cp -rf ${D}${bindir}/curl-config ${D}${PTEST_PATH} +} + +RDEPENDS:${PN}-ptest += "bash perl-modules" + PACKAGES =+ "lib${BPN}" FILES:lib${BPN} = "${libdir}/lib*.so.*"Signature · 11 lines
2.36.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#167716): https://lists.openembedded.org/g/openembedded-core/message/167716 Mute This Topic: https://lists.openembedded.org/mt/92203371/3617179 Group Owner: [email protected] Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
Signature · 3 lines
Alexandre Belloni, co-owner and COO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com