ci: work around Debian 12's HTTP/2 authentication failures

From: Johannes Schindelin <[email protected]>

Since 00fa8502354 (ci: bump debian-11 job to debian-12, 2026-09-05), the debian-12 job has intermittently failed t5559's half-auth clone with:

curl 92 Stream error in the HTTP/2 framing layer

Anonymous discovery succeeds, but the upload-pack POST requires authentication. Apache can return an early 401 and close the HTTP/2 stream before libcurl finishes sending the request body. Debian 12's curl 7.88.1 treats that closure as a transport error instead of allowing an authentication retry. Curl fixed this handling in 331b89a319d0 (http2: polish things around POST), included in 8.3.0:
https://github.com/curl/curl/pull/11756

This did not happen before switching to Debian 12 because Debian 11 ships with libcurl 7.74.0-1.3+deb11u16, which does not have that bug.

Replacing the packaged libcurl with a modern build would defeat this job's purpose of testing older supported distributions. So let's simply exclude the flaky t5559.15 and its dependent t5559.16 on Debian 12 until the packaged curl carries the fix (or until the end of time, whichever comes first).

it's just working around a flaky test. In other words: This patch does not need to be fast-tracked into v2.56.0, but it would be good to get it into master pretty soon after that, to reduce developer friction.

Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-2236%2Fdscho%2Fwork-around-debian-curl-stream-error-92-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-2236/dscho/work-around-debian-curl-stream-error-92-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/2236

 ci/lib.sh | 6 ++++++
 1 file changed, 6 insertions(+)

ci/lib.sh +6 -0

index c6ccbf8c17..1cf31b5a2c 100755
@@ -334,6 +334,12 @@ pull_request,*|push,*next*|push,*master*|push,*main*|push,*maint*)
 esac
 
 case "$distro" in
+debian-12)
+	# Debian 12's curl 7.88.1 mishandles early HTTP/2 responses; see
+	# https://github.com/curl/curl/pull/11756. Skip the half-auth
+	# clone and its dependent fetch until Debian has the fix.
+	export GIT_SKIP_TESTS="$GIT_SKIP_TESTS t5559.15 t5559.16"
+	;;
 ubuntu-*)
 	# Python 2 is end of life, and Ubuntu 23.04 and newer don't actually
 	# have it anymore. We thus only test with Python 2 on older LTS

base-commit: 3bc0341126508f78f5869cbfc0005e987efdf0c7

Anonymous discovery succeeds, but the upload-pack POST requires authentication. Apache can return an early 401 and close the HTTP/2 stream before libcurl finishes sending the request body. Debian 12's curl 7.88.1 treats that closure as a transport error instead of allowing an authentication retry. Curl fixed this handling in 331b89a319d0 (http2: polish things around POST), included in 8.3.0:
https://github.com/curl/curl/pull/11756

This did not happen before switching to Debian 12 because Debian 11 ships with libcurl 7.74.0-1.3+deb11u16, which does not have that bug.

Replacing the packaged libcurl with a modern build would defeat this job's purpose of testing older supported distributions. So let's simply exclude the flaky t5559.15 and its dependent t5559.16 on Debian 12 until the packaged curl carries the fix (or until the end of time, whichever comes first).

So an ideal solution to me would be more like t5559 checking for the buggy version itself, setting a prereq, and then marking the tests with !HAVE_CURL_HTTP2_BUG.

That said, I'm not sure how tricky that would be to implement. We give the curl version with "git version --build-options", but we'd have to do some version number comparisons. It might not be worth spending a lot of time on this.

I guess our robot overlords^Whelpers could help with that. I passed your patch and my email to Astra, which came up with this:

[...]

Not too bad. I had envisioned checking the range of versions, since you found the fix (but we'd have to either use 7.88.1 as the start, or find the actual bug introduction). But this covers at least as much as the CI debian-12 specifier would.

t/t5551-http-fetch-smart.sh +10 -0

+test_lazy_prereq HAVE_CURL_HTTP2_BUG '
+	test_have_prereq HTTP2 &&
+	build_option libcurl |
+	awk -F. '\''
+		($1 == 7 && ($2 > 88 || ($2 == 88 && $3 >= 1))) ||
+		($1 == 8 && $2 < 3) { broken = 1 }
+		END { exit !broken }
+	'\''
+'
+
 test_expect_success HTTP2 'enable client-side http/2' '
 	git config --global http.version HTTP/2
 '

which is not _too_ ugly.

-Peff

t/t5551-http-fetch-smart.sh fi · line 24

Quoted 5 lines · Jeff King

t/t5551-http-fetch-smart.sh +5 -0

+# Curl 7.88.1 can fail to retry authentication after an early HTTP/2
+# response. This was fixed in curl 8.3.0; see
+# https://github.com/curl/curl/pull/11756. The first affected version is
+# unknown, so conservatively assume that versions from 7.88.1 up to (but
+# not including) 8.3.0 are broken.

Just nitpicking the wording, but if the first affected version is truly unknown, assuming that versions from 7.88.1 up is *not* a conservative thing to do at all, is it?

If 7.88.1 is from an irrelevantly ancient past, I would say that we should just skip anything older than 8.3.0, but 7.88.1 is from early 2023 and we cannot do such a simplification.

Quoted 12 lines · Junio C Hamano

t/t5551-http-fetch-smart.sh fi · line 24

Quoted 5 lines · Jeff King

t/t5551-http-fetch-smart.sh +5 -0

+# Curl 7.88.1 can fail to retry authentication after an early HTTP/2
+# response. This was fixed in curl 8.3.0; see
+# https://github.com/curl/curl/pull/11756. The first affected version is
+# unknown, so conservatively assume that versions from 7.88.1 up to (but
+# not including) 8.3.0 are broken.

Just nitpicking the wording, but if the first affected version is truly unknown, assuming that versions from 7.88.1 up is *not* a conservative thing to do at all, is it?

Quoted 3 lines

If 7.88.1 is from an irrelevantly ancient past, I would say that we should just skip anything older than 8.3.0, but 7.88.1 is from early 2023 and we cannot do such a simplification.

It depends on what bad outcome we are being conservative against. If the bad outcome is skipping the test on a version for which we could reliably use it, then it is conservative to only select known-bad versions. If the bad outcome is somebody running the test and seeing a flaky fail, then yes, the more conservative thing would be extending to skip older unknown versions (potentially up to "forever").

I think you could argue either way (and I am OK with either, or even just matching 7.88.1).

Quoted 2 lines

I think you could argue either way (and I am OK with either, or even just matching 7.88.1).

I had GPT-6 dig deeper into the issue, since you're right: This should not be a CI/Debian-only gate, at the same time I didn't know what was the first version with the bug, so I suspected the version range to be subtly inaccurate. Turns out that the bug appeared first in cURL v7.88.0. So I adjusted your version range in preparation for the next patch iteration.

I had GPT-6 dig deeper into the issue, since you're right: This should not be a CI/Debian-only gate, at the same time I didn't know what was the first version with the bug, so I suspected the version range to be subtly inaccurate. Turns out that the bug appeared first in cURL v7.88.0. So I adjusted your version range in preparation for the next patch iteration.

2. Single-quote the snippet, then quote interior single-quotes as '\''. Reasonably obvious, but ugly.

3. Use the '<<\EOT' here-doc trick to specify the snippet. This would look nice, but we don't yet support it for prereqs. ;)

This patch uses (2), and we can circle back to (3) to make it look nicer later.

Review

Quoted 12 lines · Johannes Schindelin via GitGitGadget

(quoted diff) +12 -0

+# The cURL version which Debian 12 ships (v7.88.1) can fail to retry
+# authentication after an early HTTP/2 response. This bug was introduced
+# in cURL v7.88.0 (8c762f5998 (http2: minor buffer and error path fixes,
+# 2023-02-08)) and fixed in v8.3.0 (https://github.com/curl/curl/pull/11756).
+test_lazy_prereq HAVE_CURL_HTTP2_BUG "
+	test_have_prereq HTTP2 &&
+	build_option libcurl |
+	awk -F. '
+		($1 == 7 && $2 >= 88) || ($1 == 8 && $2 < 3) { broken = 1 }
+		END { exit !broken }
+	'
+"

Doh, this is totally broken. The prereq snippet is in double-quotes, so the $1, etc in the awk invocation are interpolated before we even eval it.

[...]

Since I know we both used GPT to work on this, I was curious if this slipped past it. Doesn't look like it from what I sent (which used option 2 above). I wonder if your agent flipped it, or if you saw how ugly it was and flipped it yourself. Not blaming, but it's just a funny and interesting data point if a human second-guessing the AI output introduced a bug.

 t/t5551-http-fetch-smart.sh | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

t/t5551-http-fetch-smart.sh +4 -4

index f66d7ce7ac..cb681e644f 100755
@@ -21,14 +21,14 @@ start_httpd
 # authentication after an early HTTP/2 response. This bug was introduced
 # in cURL v7.88.0 (8c762f5998 (http2: minor buffer and error path fixes,
 # 2023-02-08)) and fixed in v8.3.0 (https://github.com/curl/curl/pull/11756).
-test_lazy_prereq HAVE_CURL_HTTP2_BUG "
+test_lazy_prereq HAVE_CURL_HTTP2_BUG '
 	test_have_prereq HTTP2 &&
 	build_option libcurl |
-	awk -F. '
+	awk -F. '\''
 		($1 == 7 && $2 >= 88) || ($1 == 8 && $2 < 3) { broken = 1 }
 		END { exit !broken }
-	'
-"
+	'\''
+'
 
 test_expect_success HTTP2 'enable client-side http/2' '
 	git config --global http.version HTTP/2

Signature · 1 line

2.56.0.399.g9e0ddc9b37

Review

Quoted 12 lines · Johannes Schindelin via GitGitGadget

(quoted diff) +12 -0

+# The cURL version which Debian 12 ships (v7.88.1) can fail to retry
+# authentication after an early HTTP/2 response. This bug was introduced
+# in cURL v7.88.0 (8c762f5998 (http2: minor buffer and error path fixes,
+# 2023-02-08)) and fixed in v8.3.0 (https://github.com/curl/curl/pull/11756).
+test_lazy_prereq HAVE_CURL_HTTP2_BUG "
+	test_have_prereq HTTP2 &&
+	build_option libcurl |
+	awk -F. '
+		($1 == 7 && $2 >= 88) || ($1 == 8 && $2 < 3) { broken = 1 }
+		END { exit !broken }
+	'
+"

Doh, this is totally broken. The prereq snippet is in double-quotes, so the $1, etc in the awk invocation are interpolated before we even eval it. Fix is below.

-- >8 --
Subject: [PATCH] t5551: fix quoting in curl version bug prereq

We have a prereq snippet that invokes awk. The awk script's $1, etc, variables need to be quoted to avoid shell interpolation. We correctly use a single-quote inside the prereq snippet, but the snippet itself is contained in double-quotes. So we interpolate "$1" into whatever value that happens to have in the outer shell, and eval nonsense like:

  awk '(--some-garbage == 7 && --other-garbage >= 88) ...'

As a result, we don't think we have a buggy curl version even when we do, and run the test anyway. But of course it's easy not to notice, since this prereq was protecting us from a racy bug. It only breaks sometimes.

There are a few options for fixing the quoting:

1. Backslash-escaping the dollar signs. This is perhaps the least-ugly version, but it's a minor hassle to remember if somebody touches the code later.

2. Single-quote the snippet, then quote interior single-quotes as '\''. Reasonably obvious, but ugly.

3. Use the '<<\EOT' here-doc trick to specify the snippet. This would look nice, but we don't yet support it for prereqs. ;)

This patch uses (2), and we can circle back to (3) to make it look nicer later.

Signed-off-by Jeff King <[email protected]>

---
This should go on top of js/ci-debian-12-http2-workaround.

Since I know we both used GPT to work on this, I was curious if this slipped past it. Doesn't look like it from what I sent (which used option 2 above). I wonder if your agent flipped it, or if you saw how ugly it was and flipped it yourself. Not blaming, but it's just a funny and interesting data point if a human second-guessing the AI output introduced a bug.

 t/t5551-http-fetch-smart.sh | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

t/t5551-http-fetch-smart.sh +4 -4

index f66d7ce7ac..cb681e644f 100755
@@ -21,14 +21,14 @@ start_httpd
 # authentication after an early HTTP/2 response. This bug was introduced
 # in cURL v7.88.0 (8c762f5998 (http2: minor buffer and error path fixes,
 # 2023-02-08)) and fixed in v8.3.0 (https://github.com/curl/curl/pull/11756).
-test_lazy_prereq HAVE_CURL_HTTP2_BUG "
+test_lazy_prereq HAVE_CURL_HTTP2_BUG '
 	test_have_prereq HTTP2 &&
 	build_option libcurl |
-	awk -F. '
+	awk -F. '\''
 		($1 == 7 && $2 >= 88) || ($1 == 8 && $2 < 3) { broken = 1 }
 		END { exit !broken }
-	'
-"
+	'\''
+'
 
 test_expect_success HTTP2 'enable client-side http/2' '
 	git config --global http.version HTTP/2