Skip to contentSkip to conversation

curl: Update from 8.2.1 to 8.3.0

PATCH Superseded Robert Joslyn 3 participants 10 messages Sep 18, 2023 – Sep 23, 2023

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 curl: Update from 8.2.1 to 8.3.0

NSS support was removed, so adjust PACKAGECONFIG options.

The --enable-crypto-auth option was removed and split into separate options for basic-auth, bearer-auth, digest-auth, kerberos-auth, negotiate-auth, and aws. Enable basic, bearer, digest, and negotiate by default as these are probably the most common.

Disable test 1279 since this requires libcurl and hangs the tests.

---
 meta/recipes-support/curl/curl/disable-tests       |  1 +
 .../curl/{curl_8.2.1.bb => curl_8.3.0.bb}          | 14 +++++++++-----
 2 files changed, 10 insertions(+), 5 deletions(-)
 rename meta/recipes-support/curl/{curl_8.2.1.bb => curl_8.3.0.bb} (88%)
meta/recipes-support/curl/curl/disable-tests +1 -0
index b687b2bb76..fdac795662 100644
@@ -21,6 +21,7 @@
 # This test is scanning the source tree
 1222
 # These CRL tests need --libcurl option to be enabled
+1279
 1400
 1401
 1402
meta/recipes-support/curl/curl_8.3.0.bb renamed from meta/recipes-support/curl/curl_8.2.1.bb +9 -5
similarity index 88%
rename from meta/recipes-support/curl/curl_8.2.1.bb
rename to meta/recipes-support/curl/curl_8.3.0.bb
index 068486a352..327c261b1a 100644
@@ -14,7 +14,7 @@ SRC_URI = " \
     file://run-ptest \
     file://disable-tests \
 "
-SRC_URI[sha256sum] = "dd322f6bd0a20e6cebdfd388f69e98c3d183bed792cf4713c8a7ef498cba4894"
+SRC_URI[sha256sum] = "376d627767d6c4f05105ab6d497b0d9aba7111770dd9d995225478209c37ea63"
 
 # Curl has used many names over the years...
 CVE_PRODUCT = "haxx:curl haxx:libcurl curl:curl curl:libcurl libcurl:libcurl daniel_stenberg:curl"
@@ -24,21 +24,26 @@ inherit autotools pkgconfig binconfig multilib_header ptest
 # Entropy source for random PACKAGECONFIG option
 RANDOM ?= "/dev/urandom"
 
-PACKAGECONFIG ??= "${@bb.utils.filter('DISTRO_FEATURES', 'ipv6', d)} libidn openssl proxy random threaded-resolver verbose zlib"
+PACKAGECONFIG ??= "${@bb.utils.filter('DISTRO_FEATURES', 'ipv6', d)} basic-auth bearer-auth digest-auth negotiate-auth libidn openssl proxy random threaded-resolver verbose zlib"
 PACKAGECONFIG:class-native = "ipv6 openssl proxy random threaded-resolver verbose zlib"
 PACKAGECONFIG:class-nativesdk = "ipv6 openssl proxy random threaded-resolver verbose zlib"
 
 # 'ares' and 'threaded-resolver' are mutually exclusive
 PACKAGECONFIG[ares] = "--enable-ares,--disable-ares,c-ares,,,threaded-resolver"
+PACKAGECONFIG[aws] = "--enable-aws,--disable-aws"
+PACKAGECONFIG[basic-auth] = "--enable-basic-auth,--disable-basic-auth"
+PACKAGECONFIG[bearer-auth] = "--enable-bearer-auth,--disable-bearer-auth"
 PACKAGECONFIG[brotli] = "--with-brotli,--without-brotli,brotli"
 PACKAGECONFIG[builtinmanual] = "--enable-manual,--disable-manual"
 # Don't use this in production
 PACKAGECONFIG[debug] = "--enable-debug,--disable-debug"
 PACKAGECONFIG[dict] = "--enable-dict,--disable-dict,"
+PACKAGECONFIG[digest-auth] = "--enable-digest-auth,--disable-digest-auth"
 PACKAGECONFIG[gnutls] = "--with-gnutls,--without-gnutls,gnutls"
 PACKAGECONFIG[gopher] = "--enable-gopher,--disable-gopher,"
 PACKAGECONFIG[imap] = "--enable-imap,--disable-imap,"
 PACKAGECONFIG[ipv6] = "--enable-ipv6,--disable-ipv6,"
+PACKAGECONFIG[kerberos-auth] = "--enable-kerberos-auth,--disable-kerberos-auth"
 PACKAGECONFIG[krb5] = "--with-gssapi,--without-gssapi,krb5"
 PACKAGECONFIG[ldap] = "--enable-ldap,--disable-ldap,openldap"
 PACKAGECONFIG[ldaps] = "--enable-ldaps,--disable-ldaps,openldap"
@@ -47,6 +52,7 @@ PACKAGECONFIG[libidn] = "--with-libidn2,--without-libidn2,libidn2"
 PACKAGECONFIG[libssh2] = "--with-libssh2,--without-libssh2,libssh2"
 PACKAGECONFIG[mbedtls] = "--with-mbedtls=${STAGING_DIR_TARGET},--without-mbedtls,mbedtls"
 PACKAGECONFIG[mqtt] = "--enable-mqtt,--disable-mqtt,"
+PACKAGECONFIG[negotiate-auth] = "--enable-negotiate-auth,--disable-negotiate-auth"
 PACKAGECONFIG[nghttp2] = "--with-nghttp2,--without-nghttp2,nghttp2"
 PACKAGECONFIG[openssl] = "--with-openssl,--without-openssl,openssl"
 PACKAGECONFIG[pop3] = "--enable-pop3,--disable-pop3,"
@@ -56,7 +62,6 @@ PACKAGECONFIG[rtmpdump] = "--with-librtmp,--without-librtmp,rtmpdump"
 PACKAGECONFIG[rtsp] = "--enable-rtsp,--disable-rtsp,"
 PACKAGECONFIG[smb] = "--enable-smb,--disable-smb,"
 PACKAGECONFIG[smtp] = "--enable-smtp,--disable-smtp,"
-PACKAGECONFIG[nss] = "--with-nss,--without-nss,nss"
 PACKAGECONFIG[telnet] = "--enable-telnet,--disable-telnet,"
 PACKAGECONFIG[tftp] = "--enable-tftp,--disable-tftp,"
 PACKAGECONFIG[threaded-resolver] = "--enable-threaded-resolver,--disable-threaded-resolver,,,,ares"
@@ -67,11 +72,10 @@ PACKAGECONFIG[zstd] = "--with-zstd,--without-zstd,zstd"
 EXTRA_OECONF = " \
     --disable-libcurl-option \
     --disable-ntlm-wb \
-    --enable-crypto-auth \
     --with-ca-bundle=${sysconfdir}/ssl/certs/ca-certificates.crt \
     --without-libpsl \
     --enable-optimize \
-    ${@'--without-ssl' if (bb.utils.filter('PACKAGECONFIG', 'gnutls mbedtls nss openssl', d) == '') else ''} \
+    ${@'--without-ssl' if (bb.utils.filter('PACKAGECONFIG', 'gnutls mbedtls openssl', d) == '') else ''} \
 "
 
 do_install:append:class-target() {
Signature · 1 line
2.41.0
#1 lore raw
Quoted 21 lines · Robert Joslyn via lists.openembedded.org

NSS support was removed, so adjust PACKAGECONFIG options.

The --enable-crypto-auth option was removed and split into separate options for basic-auth, bearer-auth, digest-auth, kerberos-auth, negotiate-auth, and aws. Enable basic, bearer, digest, and negotiate by default as these are probably the most common.

Disable test 1279 since this requires libcurl and hangs the tests.

---
 meta/recipes-support/curl/curl/disable-tests       |  1 +
 .../curl/{curl_8.2.1.bb => curl_8.3.0.bb}          | 14 +++++++++-----
 2 files changed, 10 insertions(+), 5 deletions(-)
 rename meta/recipes-support/curl/{curl_8.2.1.bb => curl_8.3.0.bb} (88%)
meta/recipes-support/curl/curl/disable-tests +0 -0
index b687b2bb76..fdac795662 100644

meta/recipes-support/curl/curl/disable-tests line 27

Quoted 7 lines
meta/recipes-support/curl/curl/disable-tests +1 -0
@@ -21,6 +21,7 @@
 # This test is scanning the source tree
 1222
 # These CRL tests need --libcurl option to be enabled
+1279
 1400
 1401
 1402

Thanks for the patch. Unfortunately there were warnings generated on the autobuilder test run for this:

https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/5452
https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/5631

I'd also wondered if you'd mind setting your from address correctly for patches please? For more info see:

https://docs.yoctoproject.org/contributor-guide/submit-changes.html#fixing-your-from-identity

Cheers,

Richard

#2 lore raw
Quoted 33 lines · Richard Purdie
Quoted 21 lines · Robert Joslyn via lists.openembedded.org

NSS support was removed, so adjust PACKAGECONFIG options.

The --enable-crypto-auth option was removed and split into separate options for basic-auth, bearer-auth, digest-auth, kerberos-auth, negotiate-auth, and aws. Enable basic, bearer, digest, and negotiate by default as these are probably the most common.

Disable test 1279 since this requires libcurl and hangs the tests.

---
  meta/recipes-support/curl/curl/disable-tests       |  1 +
  .../curl/{curl_8.2.1.bb => curl_8.3.0.bb}          | 14 +++++++++-----
  2 files changed, 10 insertions(+), 5 deletions(-)

rename meta/recipes-support/curl/{curl_8.2.1.bb => curl_8.3.0.bb} (88%)

meta/recipes-support/curl/curl/disable-tests +0 -0
index b687b2bb76..fdac795662 100644

meta/recipes-support/curl/curl/disable-tests line 27

Quoted 7 lines
meta/recipes-support/curl/curl/disable-tests +1 -0
@@ -21,6 +21,7 @@
  # This test is scanning the source tree
  1222
  # These CRL tests need --libcurl option to be enabled
+1279
  1400
  1401
  1402

Thanks for the patch. Unfortunately there were warnings generated on the autobuilder test run for this:

Quoted 2 lines

https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/5452
https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/5631

I'm not very familiar with the autobuilder infrastructure, but it looks to be running something like "bitbake core-image-ptest-curl -c testimage". When I run this, I do get a warning:

WARNING: core-image-ptest-curl-1.0-r0 do_testimage: There were failing ptests.

Unfortunately, I don't see any failing tests. When running the tests manually (called runqemu, then ptest-runner curl in the VM) there are some tests that print an error about not finding a library and get skipped. I removed those test cases as well, but I still get the warning about failing ptests.

What's the ptest code looking for to trigger a test failure? I should have some more time to look at this later today, I'm probably just missing something obvious.

Quoted 5 lines

I'd also wondered if you'd mind setting your from address correctly for patches please? For more info see:

https://docs.yoctoproject.org/contributor-guide/submit-changes.html#fixing-your-from-identity

Thanks, for this, hopefully it's fixed now. My email provider recently changed, which does have DMARC enabled.

Thanks,

Robert

#3 lore raw
Quoted 37 lines · Robert Joslyn
Quoted 33 lines · Richard Purdie
Quoted 21 lines · Robert Joslyn via lists.openembedded.org

NSS support was removed, so adjust PACKAGECONFIG options.

The --enable-crypto-auth option was removed and split into separate options for basic-auth, bearer-auth, digest-auth, kerberos-auth, negotiate-auth, and aws. Enable basic, bearer, digest, and negotiate by default as these are probably the most common.

Disable test 1279 since this requires libcurl and hangs the tests.

---
  meta/recipes-support/curl/curl/disable-tests       |  1 +
  .../curl/{curl_8.2.1.bb => curl_8.3.0.bb}          | 14 +++++++++-----
  2 files changed, 10 insertions(+), 5 deletions(-)

rename meta/recipes-support/curl/{curl_8.2.1.bb => curl_8.3.0.bb} (88%)

meta/recipes-support/curl/curl/disable-tests +0 -0
index b687b2bb76..fdac795662 100644

meta/recipes-support/curl/curl/disable-tests line 27

Quoted 7 lines
meta/recipes-support/curl/curl/disable-tests +1 -0
@@ -21,6 +21,7 @@
  # This test is scanning the source tree
  1222
  # These CRL tests need --libcurl option to be enabled
+1279
  1400
  1401
  1402

Thanks for the patch. Unfortunately there were warnings generated on the autobuilder test run for this:

Quoted 2 lines

https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/5452
https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/5631

Quoted 6 lines

I'm not very familiar with the autobuilder infrastructure, but it looks to be running something like "bitbake core-image-ptest-curl -c testimage". When I run this, I do get a warning:

WARNING: core-image-ptest-curl-1.0-r0 do_testimage: There were failing ptests.

Running with -v might help show a touch more information.

Quoted 9 lines

Unfortunately, I don't see any failing tests. When running the tests manually (called runqemu, then ptest-runner curl in the VM) there are some tests that print an error about not finding a library and get skipped. I removed those test cases as well, but I still get the warning about failing ptests.

What's the ptest code looking for to trigger a test failure? I should have some more time to look at this later today, I'm probably just missing something obvious.

Commands like:

resulttool report ./tmp/log/oeqa/

resulttool log --dump-ptest ./output-logs ./tmp/log/oeqa/

Might help amd/or have a look at the testimage log in core-image-ptest- curl's WORKDIR/temp/log.do_testimage

Cheers,

Richard

#4 lore raw
Quoted 60 lines · Richard Purdie
Quoted 34 lines · Robert Joslyn
Quoted 33 lines · Richard Purdie
Quoted 21 lines · Robert Joslyn via lists.openembedded.org

NSS support was removed, so adjust PACKAGECONFIG options.

The --enable-crypto-auth option was removed and split into separate options for basic-auth, bearer-auth, digest-auth, kerberos-auth, negotiate-auth, and aws. Enable basic, bearer, digest, and negotiate by default as these are probably the most common.

Disable test 1279 since this requires libcurl and hangs the tests.

---
   meta/recipes-support/curl/curl/disable-tests       |  1 +
   .../curl/{curl_8.2.1.bb => curl_8.3.0.bb}          | 14 +++++++++-----
   2 files changed, 10 insertions(+), 5 deletions(-)

rename meta/recipes-support/curl/{curl_8.2.1.bb => curl_8.3.0.bb} (88%)

meta/recipes-support/curl/curl/disable-tests +0 -0
index b687b2bb76..fdac795662 100644

meta/recipes-support/curl/curl/disable-tests line 27

Quoted 7 lines
meta/recipes-support/curl/curl/disable-tests +1 -0
@@ -21,6 +21,7 @@
   # This test is scanning the source tree
   1222
   # These CRL tests need --libcurl option to be enabled
+1279
   1400
   1401
   1402

Thanks for the patch. Unfortunately there were warnings generated on the autobuilder test run for this:

Quoted 8 lines
Quoted 2 lines

https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/5452
https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/5631

I'm not very familiar with the autobuilder infrastructure, but it looks to be running something like "bitbake core-image-ptest-curl -c testimage". When I run this, I do get a warning:

WARNING: core-image-ptest-curl-1.0-r0 do_testimage: There were failing ptests.

Running with -v might help show a touch more information.

Quoted 9 lines

Unfortunately, I don't see any failing tests. When running the tests manually (called runqemu, then ptest-runner curl in the VM) there are some tests that print an error about not finding a library and get skipped. I removed those test cases as well, but I still get the warning about failing ptests.

What's the ptest code looking for to trigger a test failure? I should have some more time to look at this later today, I'm probably just missing something obvious.

Commands like:

resulttool report ./tmp/log/oeqa/

resulttool log --dump-ptest ./output-logs ./tmp/log/oeqa/

Quoted 8 lines

Might help amd/or have a look at the testimage log in core-image-ptest- curl's WORKDIR/temp/log.do_testimage

Cheers,

Richard

In running this a few times, on my machine it's not curl but other tests that fail. The autobuilder output only showed a curl failure, so I'm not sure what's different in my environment. This is what resulttool shows:

s.robert@hezmana ~/yocto/poky-master/build $ resulttool report ./tmp/log/oeqa/
==============================================================================================================
Test Result Status Summary (Counts/Percentages sorted by testseries, ID)
==============================================================================================================
--------------------------------------------------------------------------------------------------------------
Test Series | ID                                                      | Passed     | Failed | Skipped
--------------------------------------------------------------------------------------------------------------
oeqa | runtime_core-image-ptest-curl_qemux86-64_20230920012443 | 4725 (91%) | 7 (0%) | 448 (9%)
--------------------------------------------------------------------------------------------------------------
Total | 1 | 4725       | 7      | 448
--------------------------------------------------------------------------------------------------------------

==============================================================================================================
qemux86-64 PTest Result Summary
==============================================================================================================
--------------------------------------------------------------------------------------------------------------
Recipe     | Passed     | Failed | Skipped  | Time(s)
--------------------------------------------------------------------------------------------------------------
attr       | 2          | 0      | 0        | 0 bash       | 78         | 5      | 0        | 107 busybox    | 619        | 0      | 88       | 7 curl       | 847        | 0      | 0        | 188 openssh    | 81         | 0      | 1        | 712

openssl    | 0          | 0      | 0        | 0
perl       | 2570       | 1      | 192      | 549

util-linux | 524        | 1      | 166      | 36 zlib       | 1          | 0      | 0        | 0
--------------------------------------------------------------------------------------------------------------

==============================================================================================================
Failed test cases (sorted by testseries, ID)
==============================================================================================================
--------------------------------------------------------------------------------------------------------------
testseries | result_id : oeqa | runtime_core-image-ptest-curl_qemux86-64_20230920012443 ptestresult.bash.run-array
    ptestresult.bash.run-execscript
    ptestresult.bash.run-intl
    ptestresult.bash.run-read
    ptestresult.bash.run-shopt ptestresult.perl.cpan/ExtUtils-Constant/t/Constant ptestresult.util-linux.mount:_complex
--------------------------------------------------------------------------------------------------------------

I posted the output from a manual ptest-runner call here:

https://gist.github.com/bobolopolis/e02401a87d1d6b2f48ca2d70998f13c1

Thanks,

Robert

#5 lore raw
Quoted 125 lines · Robert Joslyn
Quoted 60 lines · Richard Purdie
Quoted 34 lines · Robert Joslyn
Quoted 33 lines · Richard Purdie
Quoted 21 lines · Robert Joslyn via lists.openembedded.org

NSS support was removed, so adjust PACKAGECONFIG options.

The --enable-crypto-auth option was removed and split into separate options for basic-auth, bearer-auth, digest-auth, kerberos-auth, negotiate-auth, and aws. Enable basic, bearer, digest, and negotiate by default as these are probably the most common.

Disable test 1279 since this requires libcurl and hangs the tests.

---
   meta/recipes-support/curl/curl/disable-tests       |  1 +
   .../curl/{curl_8.2.1.bb => curl_8.3.0.bb}          | 14 +++++++++-----
   2 files changed, 10 insertions(+), 5 deletions(-)

rename meta/recipes-support/curl/{curl_8.2.1.bb => curl_8.3.0.bb} (88%)

meta/recipes-support/curl/curl/disable-tests +0 -0
index b687b2bb76..fdac795662 100644

meta/recipes-support/curl/curl/disable-tests line 27

Quoted 7 lines
meta/recipes-support/curl/curl/disable-tests +1 -0
@@ -21,6 +21,7 @@
   # This test is scanning the source tree
   1222
   # These CRL tests need --libcurl option to be enabled
+1279
   1400
   1401
   1402

Thanks for the patch. Unfortunately there were warnings generated on the autobuilder test run for this:

Quoted 8 lines
Quoted 2 lines

https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/5452
https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/5631

I'm not very familiar with the autobuilder infrastructure, but it looks to be running something like "bitbake core-image-ptest-curl -c testimage". When I run this, I do get a warning:

WARNING: core-image-ptest-curl-1.0-r0 do_testimage: There were failing ptests.

Running with -v might help show a touch more information.

Quoted 9 lines

Unfortunately, I don't see any failing tests. When running the tests manually (called runqemu, then ptest-runner curl in the VM) there are some tests that print an error about not finding a library and get skipped. I removed those test cases as well, but I still get the warning about failing ptests.

What's the ptest code looking for to trigger a test failure? I should have some more time to look at this later today, I'm probably just missing something obvious.

Commands like:

resulttool report ./tmp/log/oeqa/

resulttool log --dump-ptest ./output-logs ./tmp/log/oeqa/

Quoted 8 lines

Might help amd/or have a look at the testimage log in core-image-ptest- curl's WORKDIR/temp/log.do_testimage

Cheers,

Richard

In running this a few times, on my machine it's not curl but other tests that fail. The autobuilder output only showed a curl failure, so I'm not sure what's different in my environment. This is what resulttool shows:

s.robert@hezmana ~/yocto/poky-master/build $ resulttool report ./tmp/log/oeqa/
==============================================================================================================
Test Result Status Summary (Counts/Percentages sorted by testseries, ID)
==============================================================================================================
--------------------------------------------------------------------------------------------------------------
Test Series | ID                                                      | Passed     | Failed | Skipped
--------------------------------------------------------------------------------------------------------------
oeqa | runtime_core-image-ptest-curl_qemux86-64_20230920012443 | 4725 (91%) | 7 (0%) | 448 (9%)
--------------------------------------------------------------------------------------------------------------
Total | 1 | 4725       | 7      | 448
--------------------------------------------------------------------------------------------------------------

==============================================================================================================
qemux86-64 PTest Result Summary
==============================================================================================================
--------------------------------------------------------------------------------------------------------------
Recipe     | Passed     | Failed | Skipped  | Time(s)
--------------------------------------------------------------------------------------------------------------
attr       | 2          | 0      | 0        | 0 bash       | 78         | 5      | 0        | 107 busybox    | 619        | 0      | 88       | 7 curl       | 847        | 0      | 0        | 188 openssh    | 81         | 0      | 1        | 712

openssl    | 0          | 0      | 0        | 0
perl       | 2570       | 1      | 192      | 549

util-linux | 524        | 1      | 166      | 36
zlib       | 1          | 0      | 0        | 0
--------------------------------------------------------------------------------------------------------------

==============================================================================================================
Failed test cases (sorted by testseries, ID)
==============================================================================================================
--------------------------------------------------------------------------------------------------------------
testseries | result_id : oeqa | runtime_core-image-ptest-curl_qemux86-64_20230920012443
    ptestresult.bash.run-array ptestresult.bash.run-execscript ptestresult.bash.run-intl ptestresult.bash.run-read ptestresult.bash.run-shopt ptestresult.perl.cpan/ExtUtils-Constant/t/Constant ptestresult.util-linux.mount:_complex
--------------------------------------------------------------------------------------------------------------

Quoted 3 lines

I posted the output from a manual ptest-runner call here:

https://gist.github.com/bobolopolis/e02401a87d1d6b2f48ca2d70998f13c1

If you delete or move tmp/log/oeqa/ out the way, you effectively reset the state so if you do that, then just run the curl ptest the result should be clearer.

I did go back and look up the autobuilder failure and tried to dig up more logs:

http://autobuilder.yocto.io/pub/non-release/20230918-9/testresults/qemux86-64-ptest/curl.log
http://autobuilder.yocto.io/pub/non-release/20230918-9/testresults/qemux86-64-ptest/core-image-ptest-curl/log.do_testimage.4047690.20230918105238

http://autobuilder.yocto.io/pub/non-release/20230918-9/testresults/qemuarm64-ptest/curl.log
http://autobuilder.yocto.io/pub/non-release/20230918-9/testresults/qemuarm64-ptest/core-image-ptest-curl/log.do_testimage.59151.20230918103439

so it was test 1964 that failed in both cases. I know know why though :/

Cheers,

Richard

#6 lore raw
Quoted 142 lines · Richard Purdie
Quoted 125 lines · Robert Joslyn
Quoted 60 lines · Richard Purdie
Quoted 34 lines · Robert Joslyn
Quoted 33 lines · Richard Purdie
Quoted 21 lines · Robert Joslyn via lists.openembedded.org

NSS support was removed, so adjust PACKAGECONFIG options.

The --enable-crypto-auth option was removed and split into separate options for basic-auth, bearer-auth, digest-auth, kerberos-auth, negotiate-auth, and aws. Enable basic, bearer, digest, and negotiate by default as these are probably the most common.

Disable test 1279 since this requires libcurl and hangs the tests.

---
    meta/recipes-support/curl/curl/disable-tests       |  1 +
    .../curl/{curl_8.2.1.bb => curl_8.3.0.bb}          | 14 +++++++++-----
    2 files changed, 10 insertions(+), 5 deletions(-)
    rename meta/recipes-support/curl/{curl_8.2.1.bb => curl_8.3.0.bb} (88%)
meta/recipes-support/curl/curl/disable-tests +0 -0
index b687b2bb76..fdac795662 100644

meta/recipes-support/curl/curl/disable-tests line 27

Quoted 7 lines
meta/recipes-support/curl/curl/disable-tests +1 -0
@@ -21,6 +21,7 @@
    # This test is scanning the source tree
    1222
    # These CRL tests need --libcurl option to be enabled
+1279
    1400
    1401
    1402

Thanks for the patch. Unfortunately there were warnings generated on the autobuilder test run for this:

Quoted 8 lines
Quoted 2 lines

https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/5452
https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/5631

I'm not very familiar with the autobuilder infrastructure, but it looks to be running something like "bitbake core-image-ptest-curl -c testimage". When I run this, I do get a warning:

WARNING: core-image-ptest-curl-1.0-r0 do_testimage: There were failing ptests.

Running with -v might help show a touch more information.

Quoted 9 lines

Unfortunately, I don't see any failing tests. When running the tests manually (called runqemu, then ptest-runner curl in the VM) there are some tests that print an error about not finding a library and get skipped. I removed those test cases as well, but I still get the warning about failing ptests.

What's the ptest code looking for to trigger a test failure? I should have some more time to look at this later today, I'm probably just missing something obvious.

Commands like:

resulttool report ./tmp/log/oeqa/

resulttool log --dump-ptest ./output-logs ./tmp/log/oeqa/

Quoted 8 lines

Might help amd/or have a look at the testimage log in core-image-ptest- curl's WORKDIR/temp/log.do_testimage

Cheers,

Richard

In running this a few times, on my machine it's not curl but other tests that fail. The autobuilder output only showed a curl failure, so I'm not sure what's different in my environment. This is what resulttool shows:

s.robert@hezmana ~/yocto/poky-master/build $ resulttool report ./tmp/log/oeqa/
==============================================================================================================
Test Result Status Summary (Counts/Percentages sorted by testseries, ID)
==============================================================================================================
--------------------------------------------------------------------------------------------------------------
Test Series | ID                                                      | Passed     | Failed | Skipped
--------------------------------------------------------------------------------------------------------------
oeqa | runtime_core-image-ptest-curl_qemux86-64_20230920012443 | 4725 (91%) | 7 (0%) | 448 (9%)
--------------------------------------------------------------------------------------------------------------
Total | 1 | 4725       | 7      | 448
--------------------------------------------------------------------------------------------------------------

==============================================================================================================
qemux86-64 PTest Result Summary
==============================================================================================================
--------------------------------------------------------------------------------------------------------------
Recipe     | Passed     | Failed | Skipped  | Time(s)
--------------------------------------------------------------------------------------------------------------
attr       | 2          | 0      | 0        | 0 bash       | 78         | 5      | 0        | 107 busybox    | 619        | 0      | 88       | 7 curl       | 847        | 0      | 0        | 188 openssh    | 81         | 0      | 1        | 712

openssl    | 0          | 0      | 0        | 0
perl       | 2570       | 1      | 192      | 549

util-linux | 524        | 1      | 166      | 36
zlib       | 1          | 0      | 0        | 0
--------------------------------------------------------------------------------------------------------------

==============================================================================================================
Failed test cases (sorted by testseries, ID)
==============================================================================================================
--------------------------------------------------------------------------------------------------------------
testseries | result_id : oeqa | runtime_core-image-ptest-curl_qemux86-64_20230920012443
    ptestresult.bash.run-array ptestresult.bash.run-execscript ptestresult.bash.run-intl ptestresult.bash.run-read ptestresult.bash.run-shopt ptestresult.perl.cpan/ExtUtils-Constant/t/Constant ptestresult.util-linux.mount:_complex
--------------------------------------------------------------------------------------------------------------

Quoted 3 lines

I posted the output from a manual ptest-runner call here:

https://gist.github.com/bobolopolis/e02401a87d1d6b2f48ca2d70998f13c1

If you delete or move tmp/log/oeqa/ out the way, you effectively reset the state so if you do that, then just run the curl ptest the result should be clearer.

I did go back and look up the autobuilder failure and tried to dig up more logs:

http://autobuilder.yocto.io/pub/non-release/20230918-9/testresults/qemux86-64-ptest/curl.log
http://autobuilder.yocto.io/pub/non-release/20230918-9/testresults/qemux86-64-ptest/core-image-ptest-curl/log.do_testimage.4047690.20230918105238

http://autobuilder.yocto.io/pub/non-release/20230918-9/testresults/qemuarm64-ptest/curl.log
http://autobuilder.yocto.io/pub/non-release/20230918-9/testresults/qemuarm64-ptest/core-image-ptest-curl/log.do_testimage.59151.20230918103439

Quoted 6 lines

so it was test 1964 that failed in both cases. I know know why though :/

Cheers,

Richard

Interestingly, that test gets skipped for me (on both x86-64 and arm64):

test 1964... 1964: IGNORED: The tool set in the test case for this: 'lib1964' does not exist

That test looks related to the aws authentication feature that is disabled via PACKAGECONFIG and is a new flag in this version. I can send an updated patch enabling that to see if it changes anything, although I'm still not sure why I see different behavior than the autobuilder.

Thanks,

Robert

#7 lore raw

I applied the patch on current poky master with default config (a83a16b7939e066b6686b62ae7ec66f12ef34f1e) and ran the tests locally:

root@qemux86-64:/usr/lib/curl/ptest/tests# ./runtests.pl -a -n -j4

The failure I am getting is:

test 1964...[HTTP AWS_SIGV4 with one provider and auth cred via URL, but X-Xxx-Date header set manually]

1964: protocol FAILED!
There was no content at all in the file log/3/server.input. Server glitch? Total curl failure? Returned: 48

So you should be able to reproduce this. Did you rebase on my curl changes that landed in master recently?

Alex

Quoted 174 lines · Robert Joslyn via lists.openembedded.org
Quoted 142 lines · Richard Purdie
Quoted 125 lines · Robert Joslyn
Quoted 60 lines · Richard Purdie
Quoted 34 lines · Robert Joslyn
Quoted 33 lines · Richard Purdie
Quoted 21 lines · Robert Joslyn via lists.openembedded.org

NSS support was removed, so adjust PACKAGECONFIG options.

The --enable-crypto-auth option was removed and split into separate options for basic-auth, bearer-auth, digest-auth, kerberos-auth, negotiate-auth, and aws. Enable basic, bearer, digest, and negotiate by default as these are probably the most common.

Disable test 1279 since this requires libcurl and hangs the tests.

---
    meta/recipes-support/curl/curl/disable-tests       |  1 +
    .../curl/{curl_8.2.1.bb => curl_8.3.0.bb}          | 14 +++++++++-----
    2 files changed, 10 insertions(+), 5 deletions(-)
    rename meta/recipes-support/curl/{curl_8.2.1.bb => curl_8.3.0.bb} (88%)
meta/recipes-support/curl/curl/disable-tests +0 -0
index b687b2bb76..fdac795662 100644

meta/recipes-support/curl/curl/disable-tests line 27

Quoted 7 lines
meta/recipes-support/curl/curl/disable-tests +1 -0
@@ -21,6 +21,7 @@
    # This test is scanning the source tree
    1222
    # These CRL tests need --libcurl option to be enabled
+1279
    1400
    1401
    1402

Thanks for the patch. Unfortunately there were warnings generated on the autobuilder test run for this:

Quoted 8 lines
Quoted 2 lines

https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/5452
https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/5631

I'm not very familiar with the autobuilder infrastructure, but it looks to be running something like "bitbake core-image-ptest-curl -c testimage". When I run this, I do get a warning:

WARNING: core-image-ptest-curl-1.0-r0 do_testimage: There were failing ptests.

Running with -v might help show a touch more information.

Quoted 9 lines

Unfortunately, I don't see any failing tests. When running the tests manually (called runqemu, then ptest-runner curl in the VM) there are some tests that print an error about not finding a library and get skipped. I removed those test cases as well, but I still get the warning about failing ptests.

What's the ptest code looking for to trigger a test failure? I should have some more time to look at this later today, I'm probably just missing something obvious.

Commands like:

resulttool report ./tmp/log/oeqa/

resulttool log --dump-ptest ./output-logs ./tmp/log/oeqa/

Quoted 8 lines

Might help amd/or have a look at the testimage log in core-image-ptest- curl's WORKDIR/temp/log.do_testimage

Cheers,

Richard

In running this a few times, on my machine it's not curl but other tests that fail. The autobuilder output only showed a curl failure, so I'm not sure what's different in my environment. This is what resulttool shows:

s.robert@hezmana ~/yocto/poky-master/build $ resulttool report ./tmp/log/oeqa/
==============================================================================================================
Test Result Status Summary (Counts/Percentages sorted by testseries, ID)
==============================================================================================================
--------------------------------------------------------------------------------------------------------------
Test Series | ID |
Passed | Failed | Skipped
--------------------------------------------------------------------------------------------------------------
oeqa | runtime_core-image-ptest-curl_qemux86-64_20230920012443 | 4725 (91%) | 7 (0%) | 448 (9%)
--------------------------------------------------------------------------------------------------------------
Total | 1 | 4725 | 7 | 448
--------------------------------------------------------------------------------------------------------------

==============================================================================================================
qemux86-64 PTest Result Summary
==============================================================================================================
--------------------------------------------------------------------------------------------------------------
Recipe | Passed | Failed | Skipped | Time(s)
--------------------------------------------------------------------------------------------------------------

attr       | 2          | 0      | 0        | 0
bash       | 78         | 5      | 0        | 107
busybox    | 619        | 0      | 88       | 7
curl       | 847        | 0      | 0        | 188
openssh    | 81         | 0      | 1        | 712
openssl    | 0          | 0      | 0        | 0
perl       | 2570       | 1      | 192      | 549
util-linux | 524        | 1      | 166      | 36
zlib       | 1          | 0      | 0        | 0

--------------------------------------------------------------------------------------------------------------

==============================================================================================================
Failed test cases (sorted by testseries, ID)
==============================================================================================================
--------------------------------------------------------------------------------------------------------------
testseries | result_id : oeqa | runtime_core-image-ptest-curl_qemux86-64_20230920012443
ptestresult.bash.run-array ptestresult.bash.run-execscript ptestresult.bash.run-intl ptestresult.bash.run-read ptestresult.bash.run-shopt ptestresult.perl.cpan/ExtUtils-Constant/t/Constant ptestresult.util-linux.mount:_complex
--------------------------------------------------------------------------------------------------------------

Quoted 3 lines

I posted the output from a manual ptest-runner call here:

https://gist.github.com/bobolopolis/e02401a87d1d6b2f48ca2d70998f13c1

If you delete or move tmp/log/oeqa/ out the way, you effectively reset the state so if you do that, then just run the curl ptest the result should be clearer.

I did go back and look up the autobuilder failure and tried to dig up more logs:

http://autobuilder.yocto.io/pub/non-release/20230918-9/testresults/qemux86-64-ptest/curl.log
http://autobuilder.yocto.io/pub/non-release/20230918-9/testresults/qemux86-64-ptest/core-image-ptest-curl/log.do_testimage.4047690.20230918105238

http://autobuilder.yocto.io/pub/non-release/20230918-9/testresults/qemuarm64-ptest/curl.log
http://autobuilder.yocto.io/pub/non-release/20230918-9/testresults/qemuarm64-ptest/core-image-ptest-curl/log.do_testimage.59151.20230918103439

Quoted 6 lines

so it was test 1964 that failed in both cases. I know know why though :/

Cheers,

Richard

Interestingly, that test gets skipped for me (on both x86-64 and arm64):

test 1964... 1964: IGNORED: The tool set in the test case for this:
'lib1964' does not exist

That test looks related to the aws authentication feature that is disabled via PACKAGECONFIG and is a new flag in this version. I can send an updated patch enabling that to see if it changes anything, although I'm still not sure why I see different behavior than the autobuilder.

Thanks,

Robert

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#187969): https://lists.openembedded.org/g/openembedded-core/message/187969
Mute This Topic: https://lists.openembedded.org/mt/101426058/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

#8 lore raw
Quoted 11 lines · Alexander Kanavin

I applied the patch on current poky master with default config (a83a16b7939e066b6686b62ae7ec66f12ef34f1e) and ran the tests locally:

root@qemux86-64:/usr/lib/curl/ptest/tests# ./runtests.pl -a -n -j4

The failure I am getting is:

test 1964...[HTTP AWS_SIGV4 with one provider and auth cred via URL, but X-Xxx-Date header set manually]

Quoted 9 lines

1964: protocol FAILED!
There was no content at all in the file log/3/server.input. Server glitch? Total curl failure? Returned: 48

So you should be able to reproduce this. Did you rebase on my curl changes that landed in master recently?

Alex

I didn't have your commit on my branch, sorry I should have checked that. I rebased on latest master and am now able to reproduce this. Adding aws to PACKAGECONFIG does fix it for me, so I think the test is just running even when the feature it's testing is disabled. I'm fine enabling aws authentication, upstream enables it by default anyway. I'll send a v2 doing this.

Thanks everyone for the help on this!

Robert

#9 lore raw
In reply to Robert Joslyn

It helps if you copy-paste the complete output of ptest-runner somewhere where we can see it and identify what fails. It might be not a particular test case, but something in the test harness.

Please do not randomly disable tests if they are not actually failing:
quite often tests are safely skipped even though what they print resembles an error.

Alex

Quoted 78 lines · Robert Joslyn via lists.openembedded.org
Quoted 33 lines · Richard Purdie
Quoted 21 lines · Robert Joslyn via lists.openembedded.org

NSS support was removed, so adjust PACKAGECONFIG options.

The --enable-crypto-auth option was removed and split into separate options for basic-auth, bearer-auth, digest-auth, kerberos-auth, negotiate-auth, and aws. Enable basic, bearer, digest, and negotiate by default as these are probably the most common.

Disable test 1279 since this requires libcurl and hangs the tests.

---
  meta/recipes-support/curl/curl/disable-tests       |  1 +
  .../curl/{curl_8.2.1.bb => curl_8.3.0.bb}          | 14 +++++++++-----
  2 files changed, 10 insertions(+), 5 deletions(-)

rename meta/recipes-support/curl/{curl_8.2.1.bb => curl_8.3.0.bb} (88%)

meta/recipes-support/curl/curl/disable-tests +0 -0
index b687b2bb76..fdac795662 100644

meta/recipes-support/curl/curl/disable-tests line 27

Quoted 7 lines
meta/recipes-support/curl/curl/disable-tests +1 -0
@@ -21,6 +21,7 @@
  # This test is scanning the source tree
  1222
  # These CRL tests need --libcurl option to be enabled
+1279
  1400
  1401
  1402

Thanks for the patch. Unfortunately there were warnings generated on the autobuilder test run for this:

Quoted 2 lines

https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/5452
https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/5631

I'm not very familiar with the autobuilder infrastructure, but it looks to be running something like "bitbake core-image-ptest-curl -c testimage". When I run this, I do get a warning:

WARNING: core-image-ptest-curl-1.0-r0 do_testimage: There were failing ptests.

Unfortunately, I don't see any failing tests. When running the tests manually (called runqemu, then ptest-runner curl in the VM) there are some tests that print an error about not finding a library and get skipped. I removed those test cases as well, but I still get the warning about failing ptests.

What's the ptest code looking for to trigger a test failure? I should have some more time to look at this later today, I'm probably just missing something obvious.

Quoted 5 lines

I'd also wondered if you'd mind setting your from address correctly for patches please? For more info see:

https://docs.yoctoproject.org/contributor-guide/submit-changes.html#fixing-your-from-identity

Thanks, for this, hopefully it's fixed now. My email provider recently changed, which does have DMARC enabled.

Thanks,

Robert

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#187861): https://lists.openembedded.org/g/openembedded-core/message/187861
Mute This Topic: https://lists.openembedded.org/mt/101426058/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-