recipes-support/gnutls: disable slow tests for RISCV64
v2 cover recipes-support/gnutls: disable slow tests for RISCV64
Hello,
After Trevor's investigation, I was able to remove only the most time consuming test cases.
Changes since v1 [1]:
- patched the tests instead of entirely disabling gnutls ptests
- updated the title accordingly
Best regards,
[1] https://lists.openembedded.org/g/openembedded-core/message/246201
João Marcos Costa (1):
recipes-support/gnutls: disable slow tests for RISCV64
...m-disable-slow-test-cases-in-RISCV64.patch | 109 ++++++++++++++++++ meta/recipes-support/gnutls/gnutls_3.8.13.bb | 4 + 2 files changed, 113 insertions(+) create mode 100644 meta/recipes-support/gnutls/gnutls/0001-tests-Makefile.am-disable-slow-test-cases-in-RISCV64.patch
Signature · 1 line
2.55.0
PATCH v2 1/1 recipes-support/gnutls: disable slow tests for RISCV64
Timeouts have been observed (very frequently in the past weeks) in the autobuilder, and a bug [1] was filed at Bugzilla.
After some investigation [2] was performed by Trevor Gamblin, we now have a list of the most time consuming test cases:
- mini-record-2 (187880ms)
- record-retvals (145390ms)
- rng-op-key (63820ms)
- rng-op-random (63330ms)
- rng-op-nonce (63260ms)
- dtls-rehandshake-cert (61360ms)
- dtls-rehandshake-cert-2 (61230ms)
- dtls-rehandshake-anon (61030ms)
- mini-loss-time (60130ms)
- mini-dtls-hello-verify-48 (30050ms)
- tls12-ffdhe (27790ms)
- x509sign-verify-rsa (18460ms)
- x509sign-verify-ecdsa (16050ms)
Disable them to avoid so many timeouts in qemuriscv64-ptest runs.
[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=16162
[2] https://bugzilla.yoctoproject.org/show_bug.cgi?id=16162#c19
--- ...m-disable-slow-test-cases-in-RISCV64.patch | 109 ++++++++++++++++++ meta/recipes-support/gnutls/gnutls_3.8.13.bb | 4 + 2 files changed, 113 insertions(+) create mode 100644 meta/recipes-support/gnutls/gnutls/0001-tests-Makefile.am-disable-slow-test-cases-in-RISCV64.patch
meta/recipes-support/gnutls/gnutls/0001-tests-Makefile.am-disable-slow-test-cases-in-RISCV64.patch new file +109 -0
@@ -0,0 +1,109 @@ +From a27faed63722263f5ae8e2b7fac564de3a65fda1 Mon Sep 17 00:00:00 2001 +From: =?UTF-8?q?Jo=C3=A3o=20Marcos=20Costa?= <[email protected]> +Date: Mon, 28 Sep 2026 15:56:37 +0000 +Subject: [PATCH] tests/Makefile.am: disable slow test cases in RISCV64 +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +Timeouts have been observed (very frequently in the past weeks) +in the autobuilder, and a bug [1] was filed at Bugzilla. + +After some investigation [2] was performed by Trevor Gamblin, we now have a +list of the most time consuming test cases: + +- mini-record-2 (187880ms) +- record-retvals (145390ms) +- rng-op-key (63820ms) +- rng-op-random (63330ms) +- rng-op-nonce (63260ms) +- dtls-rehandshake-cert (61360ms) +- dtls-rehandshake-cert-2 (61230ms) +- dtls-rehandshake-anon (61030ms) +- mini-loss-time (60130ms) +- mini-dtls-hello-verify-48 (30050ms) +- tls12-ffdhe (27790ms) +- x509sign-verify-rsa (18460ms) +- x509sign-verify-ecdsa (16050ms) + +Disable them to avoid so many timeouts in qemuriscv64-ptest runs. + +Upstream-Status: Inappropriate [oe-core specific] + +[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=16162 +[2] https://bugzilla.yoctoproject.org/show_bug.cgi?id=16162#c19 + +Signed-off-by: João Marcos Costa <[email protected]> +--- + tests/Makefile.am | 20 ++++++++++---------- + 1 file changed, 10 insertions(+), 10 deletions(-) + +diff --git a/tests/Makefile.am b/tests/Makefile.am +index f84b2f1..b03e570 100644 +--- a/tests/Makefile.am ++++ b/tests/Makefile.am +@@ -161,7 +161,7 @@ ctests += tls13-early-start + + ctests += tls13/no-auto-send-ticket + +-ctests += mini-record-2 simple gnutls_hmac_fast set_pkcs12_cred cert certuniqueid tls-neg-ext-key \ ++ctests += simple gnutls_hmac_fast set_pkcs12_cred cert certuniqueid tls-neg-ext-key \ + mpi certificate_set_x509_crl dn parse_ca x509-dn x509-dn-decode record-sizes \ + hostname-check cve-2008-4989 pkcs12_s2k chainverify missingissuer missingissuer_aia record-sizes-range \ + crq_key_id x509sign-verify sign-verify cve-2009-1415 cve-2009-1416 \ +@@ -172,13 +172,13 @@ ctests += mini-record-2 simple gnutls_hmac_fast set_pkcs12_cred cert certuniquei + rng-fork mini-eagain-dtls resume-dtls empty_retrieve_function \ + tls13-rehandshake-cert gnutls_ext_raw_parse handshake-large-cert \ + x509cert x509cert-tl x509cert-ct infoaccess mini-dtls-hello-verify sign-verify-ed25519-rfc8080 \ +- trustdb-tofu dtls-rehandshake-anon mini-alpn mini-dtls-large \ ++ trustdb-tofu mini-alpn mini-dtls-large \ + mini-termination mini-x509-cas mini-x509-2 pkcs12_simple \ +- mini-emsgsize-dtls chainverify-unsorted mini-overhead tls12-ffdhe \ ++ mini-emsgsize-dtls chainverify-unsorted mini-overhead \ + mini-dtls-heartbeat mini-x509-callbacks key-openssl priorities priorities-groups \ + gnutls_x509_privkey_import gnutls_x509_crt_list_import time x509-server-verify \ + sign-verify-ext4 tls-neg-ext4-key resume-lifetime \ +- mini-dtls-srtp rsa-encrypt-decrypt mini-loss-time gnutls-strcodes \ ++ mini-dtls-srtp rsa-encrypt-decrypt gnutls-strcodes \ + mini-record mini-dtls-record handshake-timeout mini-record-range \ + cert-status rsa-psk global-init sec-params sign-verify-data \ + fips-test fips-override-test mini-global-load name-constraints x509-extensions \ +@@ -192,16 +192,16 @@ ctests += mini-record-2 simple gnutls_hmac_fast set_pkcs12_cred cert certuniquei + rsa-rsa-oaep rsa-rsa-pss \ + mini-dtls-fork mini-key-material x509cert-invalid \ + tls-ext-register tls-supplemental mini-dtls0-9 duplicate-extensions \ +- record-retvals mini-server-name tls-etm tls-force-etm x509-cert-callback alerts \ ++ mini-server-name tls-etm tls-force-etm x509-cert-callback alerts \ + client-sign-md5-rep tls12-invalid-key-exchanges session-rdn-read \ + tls13-cert-key-exchange x509-cert-callback-ocsp gnutls_ocsp_resp_list_import2 \ + server-sign-md5-rep privkey-keygen mini-tls-nonblock no-signal pkcs7-gen dtls-etm \ +- x509sign-verify-rsa x509sign-verify-ecdsa x509sign-verify-gost \ ++ x509sign-verify-gost \ + cipher-alignment oids atfork prf psk-file priority-init2 post-client-hello-change-prio \ + status-request status-request-ok rfc7633-missing sign-verify-ext \ +- fallback-scsv pkcs8-key-decode urls dtls-rehandshake-cert rfc7633-ok \ ++ fallback-scsv pkcs8-key-decode urls rfc7633-ok \ + key-usage-rsa key-usage-ecdhe-rsa mini-session-verify-function auto-verify \ +- record-timeouts mini-dtls-hello-verify-48 set-default-prio \ ++ record-timeouts set-default-prio \ + tls12-anon-upgrade tlsext-decoding rsa-psk-cb gnutls-ids \ + rehandshake-switch-cert rehandshake-switch-cert-allow rehandshake-switch-cert-client \ + rehandshake-switch-cert-client-allow handshake-versions dtls-handshake-versions \ +@@ -214,7 +214,7 @@ ctests += mini-record-2 simple gnutls_hmac_fast set_pkcs12_cred cert certuniquei + set_x509_key_file_der set_x509_pkcs12_key crt_apis tls12-cert-key-exchange \ + tls11-cert-key-exchange tls10-cert-key-exchange ssl30-cert-key-exchange \ + dtls12-cert-key-exchange dtls10-cert-key-exchange x509-cert-callback-legacy \ +- keylog-env ssl2-hello tlsfeature-ext dtls-rehandshake-cert-2 dtls-session-ticket-lost \ ++ keylog-env ssl2-hello tlsfeature-ext dtls-session-ticket-lost \ + tlsfeature-crt dtls-rehandshake-cert-3 resume-with-false-start \ + set_x509_key_file_ocsp client-fastopen rng-sigint srp \ + safe-renegotiation/srn0 safe-renegotiation/srn1 safe-renegotiation/srn2 \ +@@ -230,7 +230,7 @@ ctests += mini-record-2 simple gnutls_hmac_fast set_pkcs12_cred cert certuniquei + rng-no-onload dtls1-2-mtu-check crl_apis cert_verify_inv_utf8 no-extensions \ + no-status-request hostname-check-utf8 pkcs8-key-decode-encrypted priority-mix pkcs7 \ + send-data-before-handshake recv-data-before-handshake crt_inv_write \ +- x509sign-verify-error rng-op-nonce rng-op-random rng-op-key x509-dn-decode-compat \ ++ x509sign-verify-error x509-dn-decode-compat \ + ip-check mini-x509-ipaddr trust-store base64-raw random-art dhex509self \ + dss-sig-val sign-pk-api tls-session-ext-override record-pad \ + tls13-server-kx-neg gnutls_ext_raw_parse_dtls key-export-pkcs8 \
meta/recipes-support/gnutls/gnutls_3.8.13.bb +4 -0
@@ -27,6 +27,10 @@ SRC_URI = "https://www.gnupg.org/ftp/gcrypt/gnutls/v${SHRT_VER}/gnutls-${PV}.tar file://0001-tests-mini-dtls-framents-link-to-gnulib.patch \ " +SRC_URI:append:riscv64 = "\ + file://0001-tests-Makefile.am-disable-slow-test-cases-in-RISCV64.patch \ +" + SRC_URI[sha256sum] = "ffed8ec1bf09c2426d4f14aae377de4753b53e537d685e604e99a8b16ca9c97e" inherit autotools texinfo pkgconfig gettext lib_package gtk-doc ptest
Signature · 1 line
2.55.0
Quoted 20 lines · João Marcos Costa
Timeouts have been observed (very frequently in the past weeks) in the autobuilder, and a bug [1] was filed at Bugzilla.
After some investigation [2] was performed by Trevor Gamblin, we now have a list of the most time consuming test cases:
- mini-record-2 (187880ms)
- record-retvals (145390ms)
- rng-op-key (63820ms)
- rng-op-random (63330ms)
- rng-op-nonce (63260ms)
- dtls-rehandshake-cert (61360ms)
- dtls-rehandshake-cert-2 (61230ms)
- dtls-rehandshake-anon (61030ms)
- mini-loss-time (60130ms)
- mini-dtls-hello-verify-48 (30050ms)
- tls12-ffdhe (27790ms)
- x509sign-verify-rsa (18460ms)
- x509sign-verify-ecdsa (16050ms)
Quoted 1 line
Disable them to avoid so many timeouts in qemuriscv64-ptest runs.
Thanks for submitting this. I'm sorry if I wasn't clear before, but I think we should only disable mini-record-2 and record-retvals to start, since they are significantly longer than all of the others. If gnutls remains a problem afterward then the others should be considered, but for now we want to try retaining the others to maximize test coverage.
Trevor
Quoted 141 lines
[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=16162
[2] https://bugzilla.yoctoproject.org/show_bug.cgi?id=16162#c19Signed-off-by João Marcos Costa <[email protected]>--- ...m-disable-slow-test-cases-in-RISCV64.patch | 109 ++++++++++++++++++ meta/recipes-support/gnutls/gnutls_3.8.13.bb | 4 + 2 files changed, 113 insertions(+) create mode 100644 meta/recipes-support/gnutls/gnutls/0001-tests-Makefile.am-disable-slow-test-cases-in-RISCV64.patchmeta/recipes-support/gnutls/gnutls/0001-tests-Makefile.am-disable-slow-test-cases-in-RISCV64.patch new file +109
-0@@ -0,0 +1,109 @@ +From a27faed63722263f5ae8e2b7fac564de3a65fda1 Mon Sep 17 00:00:00 2001 +From: =?UTF-8?q?Jo=C3=A3o=20Marcos=20Costa?= <[email protected]> +Date: Mon, 28 Sep 2026 15:56:37 +0000 +Subject: [PATCH] tests/Makefile.am: disable slow test cases in RISCV64 +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +Timeouts have been observed (very frequently in the past weeks) +in the autobuilder, and a bug [1] was filed at Bugzilla. + +After some investigation [2] was performed by Trevor Gamblin, we now have a +list of the most time consuming test cases: + +- mini-record-2 (187880ms) +- record-retvals (145390ms) +- rng-op-key (63820ms) +- rng-op-random (63330ms) +- rng-op-nonce (63260ms) +- dtls-rehandshake-cert (61360ms) +- dtls-rehandshake-cert-2 (61230ms) +- dtls-rehandshake-anon (61030ms) +- mini-loss-time (60130ms) +- mini-dtls-hello-verify-48 (30050ms) +- tls12-ffdhe (27790ms) +- x509sign-verify-rsa (18460ms) +- x509sign-verify-ecdsa (16050ms) + +Disable them to avoid so many timeouts in qemuriscv64-ptest runs. + +Upstream-Status: Inappropriate [oe-core specific] + +[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=16162 +[2] https://bugzilla.yoctoproject.org/show_bug.cgi?id=16162#c19 + +Signed-off-by: João Marcos Costa <[email protected]> +--- + tests/Makefile.am | 20 ++++++++++---------- + 1 file changed, 10 insertions(+), 10 deletions(-) + +diff --git a/tests/Makefile.am b/tests/Makefile.am +index f84b2f1..b03e570 100644 +--- a/tests/Makefile.am ++++ b/tests/Makefile.am +@@ -161,7 +161,7 @@ ctests += tls13-early-start + + ctests += tls13/no-auto-send-ticket + +-ctests += mini-record-2 simple gnutls_hmac_fast set_pkcs12_cred cert certuniqueid tls-neg-ext-key \ ++ctests += simple gnutls_hmac_fast set_pkcs12_cred cert certuniqueid tls-neg-ext-key \ + mpi certificate_set_x509_crl dn parse_ca x509-dn x509-dn-decode record-sizes \ + hostname-check cve-2008-4989 pkcs12_s2k chainverify missingissuer missingissuer_aia record-sizes-range \ + crq_key_id x509sign-verify sign-verify cve-2009-1415 cve-2009-1416 \ +@@ -172,13 +172,13 @@ ctests += mini-record-2 simple gnutls_hmac_fast set_pkcs12_cred cert certuniquei + rng-fork mini-eagain-dtls resume-dtls empty_retrieve_function \ + tls13-rehandshake-cert gnutls_ext_raw_parse handshake-large-cert \ + x509cert x509cert-tl x509cert-ct infoaccess mini-dtls-hello-verify sign-verify-ed25519-rfc8080 \ +- trustdb-tofu dtls-rehandshake-anon mini-alpn mini-dtls-large \ ++ trustdb-tofu mini-alpn mini-dtls-large \ + mini-termination mini-x509-cas mini-x509-2 pkcs12_simple \ +- mini-emsgsize-dtls chainverify-unsorted mini-overhead tls12-ffdhe \ ++ mini-emsgsize-dtls chainverify-unsorted mini-overhead \ + mini-dtls-heartbeat mini-x509-callbacks key-openssl priorities priorities-groups \ + gnutls_x509_privkey_import gnutls_x509_crt_list_import time x509-server-verify \ + sign-verify-ext4 tls-neg-ext4-key resume-lifetime \ +- mini-dtls-srtp rsa-encrypt-decrypt mini-loss-time gnutls-strcodes \ ++ mini-dtls-srtp rsa-encrypt-decrypt gnutls-strcodes \ + mini-record mini-dtls-record handshake-timeout mini-record-range \ + cert-status rsa-psk global-init sec-params sign-verify-data \ + fips-test fips-override-test mini-global-load name-constraints x509-extensions \ +@@ -192,16 +192,16 @@ ctests += mini-record-2 simple gnutls_hmac_fast set_pkcs12_cred cert certuniquei + rsa-rsa-oaep rsa-rsa-pss \ + mini-dtls-fork mini-key-material x509cert-invalid \ + tls-ext-register tls-supplemental mini-dtls0-9 duplicate-extensions \ +- record-retvals mini-server-name tls-etm tls-force-etm x509-cert-callback alerts \ ++ mini-server-name tls-etm tls-force-etm x509-cert-callback alerts \ + client-sign-md5-rep tls12-invalid-key-exchanges session-rdn-read \ + tls13-cert-key-exchange x509-cert-callback-ocsp gnutls_ocsp_resp_list_import2 \ + server-sign-md5-rep privkey-keygen mini-tls-nonblock no-signal pkcs7-gen dtls-etm \ +- x509sign-verify-rsa x509sign-verify-ecdsa x509sign-verify-gost \ ++ x509sign-verify-gost \ + cipher-alignment oids atfork prf psk-file priority-init2 post-client-hello-change-prio \ + status-request status-request-ok rfc7633-missing sign-verify-ext \ +- fallback-scsv pkcs8-key-decode urls dtls-rehandshake-cert rfc7633-ok \ ++ fallback-scsv pkcs8-key-decode urls rfc7633-ok \ + key-usage-rsa key-usage-ecdhe-rsa mini-session-verify-function auto-verify \ +- record-timeouts mini-dtls-hello-verify-48 set-default-prio \ ++ record-timeouts set-default-prio \ + tls12-anon-upgrade tlsext-decoding rsa-psk-cb gnutls-ids \ + rehandshake-switch-cert rehandshake-switch-cert-allow rehandshake-switch-cert-client \ + rehandshake-switch-cert-client-allow handshake-versions dtls-handshake-versions \ +@@ -214,7 +214,7 @@ ctests += mini-record-2 simple gnutls_hmac_fast set_pkcs12_cred cert certuniquei + set_x509_key_file_der set_x509_pkcs12_key crt_apis tls12-cert-key-exchange \ + tls11-cert-key-exchange tls10-cert-key-exchange ssl30-cert-key-exchange \ + dtls12-cert-key-exchange dtls10-cert-key-exchange x509-cert-callback-legacy \ +- keylog-env ssl2-hello tlsfeature-ext dtls-rehandshake-cert-2 dtls-session-ticket-lost \ ++ keylog-env ssl2-hello tlsfeature-ext dtls-session-ticket-lost \ + tlsfeature-crt dtls-rehandshake-cert-3 resume-with-false-start \ + set_x509_key_file_ocsp client-fastopen rng-sigint srp \ + safe-renegotiation/srn0 safe-renegotiation/srn1 safe-renegotiation/srn2 \ +@@ -230,7 +230,7 @@ ctests += mini-record-2 simple gnutls_hmac_fast set_pkcs12_cred cert certuniquei + rng-no-onload dtls1-2-mtu-check crl_apis cert_verify_inv_utf8 no-extensions \ + no-status-request hostname-check-utf8 pkcs8-key-decode-encrypted priority-mix pkcs7 \ + send-data-before-handshake recv-data-before-handshake crt_inv_write \ +- x509sign-verify-error rng-op-nonce rng-op-random rng-op-key x509-dn-decode-compat \ ++ x509sign-verify-error x509-dn-decode-compat \ + ip-check mini-x509-ipaddr trust-store base64-raw random-art dhex509self \ + dss-sig-val sign-pk-api tls-session-ext-override record-pad \ + tls13-server-kx-neg gnutls_ext_raw_parse_dtls key-export-pkcs8 \meta/recipes-support/gnutls/gnutls_3.8.13.bb +4
-0@@ -27,6 +27,10 @@ SRC_URI = "https://www.gnupg.org/ftp/gcrypt/gnutls/v${SHRT_VER}/gnutls-${PV}.tar file://0001-tests-mini-dtls-framents-link-to-gnulib.patch \ " +SRC_URI:append:riscv64 = "\ + file://0001-tests-Makefile.am-disable-slow-test-cases-in-RISCV64.patch \ +" + SRC_URI[sha256sum] = "ffed8ec1bf09c2426d4f14aae377de4753b53e537d685e604e99a8b16ca9c97e" inherit autotools texinfo pkgconfig gettext lib_package gtk-doc ptest
Hello,
Quoted 23 lines · Trevor Gamblin
Quoted 20 lines · João Marcos Costa
Timeouts have been observed (very frequently in the past weeks) in the autobuilder, and a bug [1] was filed at Bugzilla.
After some investigation [2] was performed by Trevor Gamblin, we now have a list of the most time consuming test cases:
- mini-record-2 (187880ms)
- record-retvals (145390ms)
- rng-op-key (63820ms)
- rng-op-random (63330ms)
- rng-op-nonce (63260ms)
- dtls-rehandshake-cert (61360ms)
- dtls-rehandshake-cert-2 (61230ms)
- dtls-rehandshake-anon (61030ms)
- mini-loss-time (60130ms)
- mini-dtls-hello-verify-48 (30050ms)
- tls12-ffdhe (27790ms)
- x509sign-verify-rsa (18460ms)
- x509sign-verify-ecdsa (16050ms)Quoted 1 line
Disable them to avoid so many timeouts in qemuriscv64-ptest runs.
Quoted 7 lines
Thanks for submitting this. I'm sorry if I wasn't clear before, but I think we should only disable mini-record-2 and record-retvals to start, since they are significantly longer than all of the others. If gnutls remains a problem afterward then the others should be considered, but for now we want to try retaining the others to maximize test coverage.
Trevor
In that case, I'd rather remove at least the cases that take > 60000ms to run, and avoid getting back to this again when the runners are more loaded than usual. What do you think?
Signature · 2 lines
Best regards, João Marcos Costa
Quoted 32 lines · Joao Marcos Costa
Hello,
Quoted 29 lines · Trevor Gamblin
Quoted 20 lines · João Marcos Costa
Timeouts have been observed (very frequently in the past weeks) in the autobuilder, and a bug [1] was filed at Bugzilla.
After some investigation [2] was performed by Trevor Gamblin, we now have a list of the most time consuming test cases:
- mini-record-2 (187880ms)
- record-retvals (145390ms)
- rng-op-key (63820ms)
- rng-op-random (63330ms)
- rng-op-nonce (63260ms)
- dtls-rehandshake-cert (61360ms)
- dtls-rehandshake-cert-2 (61230ms)
- dtls-rehandshake-anon (61030ms)
- mini-loss-time (60130ms)
- mini-dtls-hello-verify-48 (30050ms)
- tls12-ffdhe (27790ms)
- x509sign-verify-rsa (18460ms)
- x509sign-verify-ecdsa (16050ms)Quoted 1 line
Disable them to avoid so many timeouts in qemuriscv64-ptest runs.
Thanks for submitting this. I'm sorry if I wasn't clear before, but I think we should only disable mini-record-2 and record-retvals to start, since they are significantly longer than all of the others. If gnutls remains a problem afterward then the others should be considered, but for now we want to try retaining the others to maximize test coverage.
Quoted 5 lines
Quoted 1 line
Trevor
In that case, I'd rather remove at least the cases that take > 60000ms to run, and avoid getting back to this again when the runners are more loaded than usual. What do you think?
Sure, let's do that for now. I'll keep a list of tests disabled for load/performance reasons and come back to them at some point when we have faster riscv64 hardware in the AB cluster :)