Skip to contentSkip to conversation

curl: Update from 8.2.1 to 8.3.0

PATCH In review [email protected] 2 participants 5 messages Sep 23, 2023 – Sep 27, 2023

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 curl: Update from 8.2.1 to 8.3.0

From: Robert Joslyn <[email protected]>

NSS support was removed, so adjust PACKAGECONFIG options.

The --enable-crypto-auth option was removed and split into separate options for basic-auth, bearer-auth, digest-auth, kerberos-auth, negotiate-auth, and aws. Enable these new options since upstream enables them by default.

Disable test 1279 since this requires libcurl and hangs the tests.

---
Changes from v1:
Enable aws authentication by default. Upstream enables it by default and test 1964 fails with it disabled.

 meta/recipes-support/curl/curl/disable-tests       |  1 +
 .../curl/{curl_8.2.1.bb => curl_8.3.0.bb}          | 14 +++++++++-----
 2 files changed, 10 insertions(+), 5 deletions(-)
 rename meta/recipes-support/curl/{curl_8.2.1.bb => curl_8.3.0.bb} (88%)
meta/recipes-support/curl/curl/disable-tests +1 -0
index b687b2bb76..fdac795662 100644
@@ -21,6 +21,7 @@
 # This test is scanning the source tree
 1222
 # These CRL tests need --libcurl option to be enabled
+1279
 1400
 1401
 1402
meta/recipes-support/curl/curl_8.3.0.bb renamed from meta/recipes-support/curl/curl_8.2.1.bb +9 -5
similarity index 88%
rename from meta/recipes-support/curl/curl_8.2.1.bb
rename to meta/recipes-support/curl/curl_8.3.0.bb
index b86c6f3bba..646ac01914 100644
@@ -14,7 +14,7 @@ SRC_URI = " \
     file://run-ptest \
     file://disable-tests \
 "
-SRC_URI[sha256sum] = "dd322f6bd0a20e6cebdfd388f69e98c3d183bed792cf4713c8a7ef498cba4894"
+SRC_URI[sha256sum] = "376d627767d6c4f05105ab6d497b0d9aba7111770dd9d995225478209c37ea63"
 
 # Curl has used many names over the years...
 CVE_PRODUCT = "haxx:curl haxx:libcurl curl:curl curl:libcurl libcurl:libcurl daniel_stenberg:curl"
@@ -24,21 +24,26 @@ inherit autotools pkgconfig binconfig multilib_header ptest
 # Entropy source for random PACKAGECONFIG option
 RANDOM ?= "/dev/urandom"
 
-PACKAGECONFIG ??= "${@bb.utils.filter('DISTRO_FEATURES', 'ipv6', d)} libidn openssl proxy random threaded-resolver verbose zlib"
+PACKAGECONFIG ??= "${@bb.utils.filter('DISTRO_FEATURES', 'ipv6', d)} aws basic-auth bearer-auth digest-auth negotiate-auth libidn openssl proxy random threaded-resolver verbose zlib"
 PACKAGECONFIG:class-native = "ipv6 openssl proxy random threaded-resolver verbose zlib"
 PACKAGECONFIG:class-nativesdk = "ipv6 openssl proxy random threaded-resolver verbose zlib"
 
 # 'ares' and 'threaded-resolver' are mutually exclusive
 PACKAGECONFIG[ares] = "--enable-ares,--disable-ares,c-ares,,,threaded-resolver"
+PACKAGECONFIG[aws] = "--enable-aws,--disable-aws"
+PACKAGECONFIG[basic-auth] = "--enable-basic-auth,--disable-basic-auth"
+PACKAGECONFIG[bearer-auth] = "--enable-bearer-auth,--disable-bearer-auth"
 PACKAGECONFIG[brotli] = "--with-brotli,--without-brotli,brotli"
 PACKAGECONFIG[builtinmanual] = "--enable-manual,--disable-manual"
 # Don't use this in production
 PACKAGECONFIG[debug] = "--enable-debug,--disable-debug"
 PACKAGECONFIG[dict] = "--enable-dict,--disable-dict,"
+PACKAGECONFIG[digest-auth] = "--enable-digest-auth,--disable-digest-auth"
 PACKAGECONFIG[gnutls] = "--with-gnutls,--without-gnutls,gnutls"
 PACKAGECONFIG[gopher] = "--enable-gopher,--disable-gopher,"
 PACKAGECONFIG[imap] = "--enable-imap,--disable-imap,"
 PACKAGECONFIG[ipv6] = "--enable-ipv6,--disable-ipv6,"
+PACKAGECONFIG[kerberos-auth] = "--enable-kerberos-auth,--disable-kerberos-auth"
 PACKAGECONFIG[krb5] = "--with-gssapi,--without-gssapi,krb5"
 PACKAGECONFIG[ldap] = "--enable-ldap,--disable-ldap,openldap"
 PACKAGECONFIG[ldaps] = "--enable-ldaps,--disable-ldaps,openldap"
@@ -47,6 +52,7 @@ PACKAGECONFIG[libidn] = "--with-libidn2,--without-libidn2,libidn2"
 PACKAGECONFIG[libssh2] = "--with-libssh2,--without-libssh2,libssh2"
 PACKAGECONFIG[mbedtls] = "--with-mbedtls=${STAGING_DIR_TARGET},--without-mbedtls,mbedtls"
 PACKAGECONFIG[mqtt] = "--enable-mqtt,--disable-mqtt,"
+PACKAGECONFIG[negotiate-auth] = "--enable-negotiate-auth,--disable-negotiate-auth"
 PACKAGECONFIG[nghttp2] = "--with-nghttp2,--without-nghttp2,nghttp2"
 PACKAGECONFIG[openssl] = "--with-openssl,--without-openssl,openssl"
 PACKAGECONFIG[pop3] = "--enable-pop3,--disable-pop3,"
@@ -56,7 +62,6 @@ PACKAGECONFIG[rtmpdump] = "--with-librtmp,--without-librtmp,rtmpdump"
 PACKAGECONFIG[rtsp] = "--enable-rtsp,--disable-rtsp,"
 PACKAGECONFIG[smb] = "--enable-smb,--disable-smb,"
 PACKAGECONFIG[smtp] = "--enable-smtp,--disable-smtp,"
-PACKAGECONFIG[nss] = "--with-nss,--without-nss,nss"
 PACKAGECONFIG[telnet] = "--enable-telnet,--disable-telnet,"
 PACKAGECONFIG[tftp] = "--enable-tftp,--disable-tftp,"
 PACKAGECONFIG[threaded-resolver] = "--enable-threaded-resolver,--disable-threaded-resolver,,,,ares"
@@ -67,11 +72,10 @@ PACKAGECONFIG[zstd] = "--with-zstd,--without-zstd,zstd"
 EXTRA_OECONF = " \
     --disable-libcurl-option \
     --disable-ntlm-wb \
-    --enable-crypto-auth \
     --with-ca-bundle=${sysconfdir}/ssl/certs/ca-certificates.crt \
     --without-libpsl \
     --enable-optimize \
-    ${@'--without-ssl' if (bb.utils.filter('PACKAGECONFIG', 'gnutls mbedtls nss openssl', d) == '') else ''} \
+    ${@'--without-ssl' if (bb.utils.filter('PACKAGECONFIG', 'gnutls mbedtls openssl', d) == '') else ''} \
 "
 
 do_install:append:class-target() {
Signature · 1 line
2.41.0
#1 lore raw

I am seeing a ptest failure on qemx86-64/glibc

Failed ptests:
{'curl': ['test_1474', 'curl']}

Quoted 120 lines · Robert Joslyn via lists.openembedded.org

From: Robert Joslyn <[email protected]>

NSS support was removed, so adjust PACKAGECONFIG options.

The --enable-crypto-auth option was removed and split into separate options for basic-auth, bearer-auth, digest-auth, kerberos-auth, negotiate-auth, and aws. Enable these new options since upstream enables them by default.

Disable test 1279 since this requires libcurl and hangs the tests.

---
Changes from v1:
Enable aws authentication by default. Upstream enables it by default and test 1964 fails with it disabled.

 meta/recipes-support/curl/curl/disable-tests       |  1 +
 .../curl/{curl_8.2.1.bb => curl_8.3.0.bb}          | 14 +++++++++-----
 2 files changed, 10 insertions(+), 5 deletions(-)
 rename meta/recipes-support/curl/{curl_8.2.1.bb => curl_8.3.0.bb} (88%)
meta/recipes-support/curl/curl/disable-tests +1 -0
index b687b2bb76..fdac795662 100644
@@ -21,6 +21,7 @@
 # This test is scanning the source tree
 1222
 # These CRL tests need --libcurl option to be enabled
+1279
 1400
 1401
 1402
meta/recipes-support/curl/curl_8.3.0.bb renamed from meta/recipes-support/curl/curl_8.2.1.bb +9 -6
similarity index 88%
rename from meta/recipes-support/curl/curl_8.2.1.bb
rename to meta/recipes-support/curl/curl_8.3.0.bb
index b86c6f3bba..646ac01914 100644
@@ -14,7 +14,7 @@ SRC_URI = " \
     file://run-ptest \
     file://disable-tests \
 "
-SRC_URI[sha256sum] = "dd322f6bd0a20e6cebdfd388f69e98c3d183bed792cf4713c8a7ef498cba4894"
+SRC_URI[sha256sum] = "376d627767d6c4f05105ab6d497b0d9aba7111770dd9d995225478209c37ea63"
 
 # Curl has used many names over the years...
 CVE_PRODUCT = "haxx:curl haxx:libcurl curl:curl curl:libcurl libcurl:libcurl daniel_stenberg:curl"
@@ -24,21 +24,26 @@ inherit autotools pkgconfig binconfig multilib_header ptest
 # Entropy source for random PACKAGECONFIG option
 RANDOM ?= "/dev/urandom"
 
-PACKAGECONFIG ??= "${@bb.utils.filter('DISTRO_FEATURES', 'ipv6', d)} libidn openssl proxy random threaded-resolver verbose zlib"
+PACKAGECONFIG ??= "${@bb.utils.filter('DISTRO_FEATURES', 'ipv6', d)} aws basic-auth bearer-auth digest-auth negotiate-auth libidn openssl proxy random threaded-resolver verbose zlib"
 PACKAGECONFIG:class-native = "ipv6 openssl proxy random threaded-resolver verbose zlib"
 PACKAGECONFIG:class-nativesdk = "ipv6 openssl proxy random threaded-resolver verbose zlib"
 
 # 'ares' and 'threaded-resolver' are mutually exclusive
 PACKAGECONFIG[ares] = "--enable-ares,--disable-ares,c-ares,,,threaded-resolver"
+PACKAGECONFIG[aws] = "--enable-aws,--disable-aws"
+PACKAGECONFIG[basic-auth] = "--enable-basic-auth,--disable-basic-auth"
+PACKAGECONFIG[bearer-auth] = "--enable-bearer-auth,--disable-bearer-auth"
 PACKAGECONFIG[brotli] = "--with-brotli,--without-brotli,brotli"
 PACKAGECONFIG[builtinmanual] = "--enable-manual,--disable-manual"
 # Don't use this in production
 PACKAGECONFIG[debug] = "--enable-debug,--disable-debug"
 PACKAGECONFIG[dict] = "--enable-dict,--disable-dict,"
+PACKAGECONFIG[digest-auth] = "--enable-digest-auth,--disable-digest-auth"
 PACKAGECONFIG[gnutls] = "--with-gnutls,--without-gnutls,gnutls"
 PACKAGECONFIG[gopher] = "--enable-gopher,--disable-gopher,"
 PACKAGECONFIG[imap] = "--enable-imap,--disable-imap,"
 PACKAGECONFIG[ipv6] = "--enable-ipv6,--disable-ipv6,"
+PACKAGECONFIG[kerberos-auth] = "--enable-kerberos-auth,--disable-kerberos-auth"
 PACKAGECONFIG[krb5] = "--with-gssapi,--without-gssapi,krb5"
 PACKAGECONFIG[ldap] = "--enable-ldap,--disable-ldap,openldap"
 PACKAGECONFIG[ldaps] = "--enable-ldaps,--disable-ldaps,openldap"
@@ -47,6 +52,7 @@ PACKAGECONFIG[libidn] = "--with-libidn2,--without-libidn2,libidn2"
 PACKAGECONFIG[libssh2] = "--with-libssh2,--without-libssh2,libssh2"
 PACKAGECONFIG[mbedtls] = "--with-mbedtls=${STAGING_DIR_TARGET},--without-mbedtls,mbedtls"
 PACKAGECONFIG[mqtt] = "--enable-mqtt,--disable-mqtt,"
+PACKAGECONFIG[negotiate-auth] = "--enable-negotiate-auth,--disable-negotiate-auth"
 PACKAGECONFIG[nghttp2] = "--with-nghttp2,--without-nghttp2,nghttp2"
 PACKAGECONFIG[openssl] = "--with-openssl,--without-openssl,openssl"
 PACKAGECONFIG[pop3] = "--enable-pop3,--disable-pop3,"
@@ -56,7 +62,6 @@ PACKAGECONFIG[rtmpdump] = "--with-librtmp,--without-librtmp,rtmpdump"
 PACKAGECONFIG[rtsp] = "--enable-rtsp,--disable-rtsp,"
 PACKAGECONFIG[smb] = "--enable-smb,--disable-smb,"
 PACKAGECONFIG[smtp] = "--enable-smtp,--disable-smtp,"
-PACKAGECONFIG[nss] = "--with-nss,--without-nss,nss"
 PACKAGECONFIG[telnet] = "--enable-telnet,--disable-telnet,"
 PACKAGECONFIG[tftp] = "--enable-tftp,--disable-tftp,"
 PACKAGECONFIG[threaded-resolver] = "--enable-threaded-resolver,--disable-threaded-resolver,,,,ares"
@@ -67,11 +72,10 @@ PACKAGECONFIG[zstd] = "--with-zstd,--without-zstd,zstd"
 EXTRA_OECONF = " \
     --disable-libcurl-option \
     --disable-ntlm-wb \
-    --enable-crypto-auth \
     --with-ca-bundle=${sysconfdir}/ssl/certs/ca-certificates.crt \
     --without-libpsl \
     --enable-optimize \
-    ${@'--without-ssl' if (bb.utils.filter('PACKAGECONFIG', 'gnutls mbedtls nss openssl', d) == '') else ''} \
+    ${@'--without-ssl' if (bb.utils.filter('PACKAGECONFIG', 'gnutls mbedtls openssl', d) == '') else ''} \
 "
 
 do_install:append:class-target() {
--

2.41.0

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#188152): https://lists.openembedded.org/g/openembedded-core/message/188152
Mute This Topic: https://lists.openembedded.org/mt/101543494/1997914
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

#2 lore raw
Quoted 4 lines · Khem Raj

I am seeing a ptest failure on qemx86-64/glibc

   Failed ptests:
   {'curl': ['test_1474', 'curl']}

In looking at the test, it is marked as flaky with the comment:

# Because of the timing sensitivity (scheduling delays of 500 msec can cause

# the test to fail), this test is marked flaky to avoid it being run in the CI
# builds which are often run on overloaded servers.

https://github.com/curl/curl/blob/curl-8_3_0/tests/data/test1474#L21

My server is pretty lightly loaded while running the tests, so that could explain why I haven't seen it fail normally. I was able to induce a failure by running a very heavy load while running that test on my desktop (which is older and slower than my build server). If you'd like, I can send a follow up patch to skip tests marked as flaky.

Thanks,

Robert

#3 lore raw
Quoted 14 lines · Robert Joslyn
Quoted 4 lines · Khem Raj

I am seeing a ptest failure on qemx86-64/glibc

   Failed ptests:
   {'curl': ['test_1474', 'curl']}

In looking at the test, it is marked as flaky with the comment:

# Because of the timing sensitivity (scheduling delays of 500 msec can cause

# the test to fail), this test is marked flaky to avoid it being run in
the CI
# builds which are often run on overloaded servers.
Quoted 8 lines

https://github.com/curl/curl/blob/curl-8_3_0/tests/data/test1474#L21

My server is pretty lightly loaded while running the tests, so that could explain why I haven't seen it fail normally. I was able to induce a failure by running a very heavy load while running that test on my desktop (which is older and slower than my build server). If you'd like, I can send a follow up patch to skip tests marked as flaky.

another run resulted in yet another failure

test 0587...
FAIL: 587: protoc!
There was no content at all in the file log/2/server.input. Server glitch? Total curl failure? Returned: 42

prior to this upgrade my test setup never saw curl failures and it has same amount of load etc.

Quoted 4 lines

Thanks,

Robert

#4 lore raw
Quoted 24 lines · Khem Raj
Quoted 12 lines · Robert Joslyn
Quoted 4 lines · Khem Raj

I am seeing a ptest failure on qemx86-64/glibc

    Failed ptests:
    {'curl': ['test_1474', 'curl']}

In looking at the test, it is marked as flaky with the comment:

# Because of the timing sensitivity (scheduling delays of 500 msec can cause

# the test to fail), this test is marked flaky to avoid it being run in
the CI
# builds which are often run on overloaded servers.
Quoted 8 lines

https://github.com/curl/curl/blob/curl-8_3_0/tests/data/test1474#L21

My server is pretty lightly loaded while running the tests, so that could explain why I haven't seen it fail normally. I was able to induce a failure by running a very heavy load while running that test on my desktop (which is older and slower than my build server). If you'd like, I can send a follow up patch to skip tests marked as flaky.

another run resulted in yet another failure

Quoted 7 lines

test 0587...
FAIL: 587: protoc!
There was no content at all in the file log/2/server.input. Server glitch? Total curl failure? Returned: 42

prior to this upgrade my test setup never saw curl failures and it has same amount of load etc.

Test 587 is also marked flaky by upstream. There are 12 tests marked as flaky total, and since you've just hit two of them, I'm much more inclined to just skip them. Right before this version bump there was also an update to run the curl ptests in parallel, which could be contributing to the change in test behavior.

Thanks,

Robert