t5410: avoid hangs in CI runs in the win+Meson test jobs

From: Johannes Schindelin <[email protected]>

In the GitHub workflow used in Git's CI builds, the vs test jobs use a subset of a specific revision of Git for Windows' SDK to run Git's test suite. This revision is validated by another CI workflow to ensure that said revision _can_ run Git's test suite successfully, skipping buggy updates in Git for Windows' SDK.

The win+Meson test jobs do things differently, quite differently. They use the Bash of the Git for Windows version that is installed on the runners to run Git's test suite.

This difference has consequences.

When 68cb0b5253a0 (builtin/receive-pack: add option to skip connectivity check, 2025-05-20) introduced a test case that uses tee <file> | git receive-pack as --receive-pack parameter (imitating an existing pattern in the same test script), it hit just the sweet spot to trigger a bug in the MSYS2 runtime shipped in Git for Windows v2.49.0. This version is the one currently installed on GitHub's runners.

The problem is that the git receive-pack process finishes while the tee process does not need to write anything anymore and therefore does not receive an EOF. Instead, it should receive a SIGPIPE, but the bug in the MSYS2 runtime prevents that from working as intended. As a consequence, the tee process waits for more input from the git.exe send-pack process but none is coming, and the test script patiently waits until the 6h timeout hits.

Only every once in a while, the git receive-pack process manages to send an EOF to the tee process and no hang occurs. Therefore, the problem can be worked around by cancelling the clearly-hanging job after twenty or so minutes and re-running it, repeating the process about half a dozen times, until the hang was successfully avoided.

This bug in the MSYS2 runtime has been fixed in the meantime, which is the reason why the same test case causes no problems in the win test and the vs test jobs.

This will continue to be the case until the Git for Windows version on the GitHub runners is upgraded to a version that distributes a newer MSYS2 runtime version. However, as of time of writing, this _is_ the latest Git for Windows version, and will be for another 1.5 weeks, until Git v2.50.0 is scheduled to appear (and shortly thereafter Git for Windows v2.50.0). Traditionally it takes a while before the runners pick up the new version.

We could just wait it out, six hours at a time.

Here, I opt for an alternative: Detect the buggy MSYS2 runtime and simply skip the test case. It's not like the receive-pack test cases are specific to Windows, and even then, to my chagrin the CI runs in git-for-windows/git spend around ten hours of compute time each and every time to run the entire test suite on all the platforms, even the tests that cover cross-platform code, and for Windows alone we do that three times: with GCC, with MSVC, and with MSVC via Meson. Therefore, I deem it more than acceptable to skip this test case in one of those matrices.

For good luck, also the preceding test case is skipped in that scenario, as it uses the same --receive-pack=tee <file> | git receive-pack pattern, even though I never observed that test case to hang in practice.

---
t5410: avoid hangs in CI runs in the win+Meson test jobs

I finally had a chance to look more closely at this problem. Here is my alternative to what Patrick proposed in
https://lore.kernel.org/git/[email protected]/.

Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1932%2Fdscho%2Ft5410-hangs-in-win%2BMeson-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1932/dscho/t5410-hangs-in-win+Meson-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/1932

 t/t5410-receive-pack.sh | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)

index f76a22943ef..09d6bfd2a10 100755
@@ -41,7 +41,19 @@ test_expect_success 'with core.alternateRefsPrefixes' '
 	test_cmp expect actual.haves
 '
 
-test_expect_success 'receive-pack missing objects fails connectivity check' '
+# The `tee.exe` shipped in Git for Windows v2.49.0 is known to hang frequently
+# when spawned from `git.exe` and piping its output to `git.exe`. This seems
+# related to MSYS2 runtime bug fixes regarding the signal handling; Let's just
+# skip the tests that need to exercise this when the faulty MSYS2 runtime is
+# detected; The test cases are exercised enough in other matrix jobs of the CI
+# runs.
+test_lazy_prereq TEE_DOES_NOT_HANG '
+	test_have_prereq !MINGW &&
+	case "$(uname -a)" in *3.5.7-463ebcdc.x86_64*) false;; esac
+'
+
+test_expect_success TEE_DOES_NOT_HANG \
+	'receive-pack missing objects fails connectivity check' '
 	test_when_finished rm -rf repo remote.git setup.git &&
 
 	git init repo &&

@@ -62,7 +74,8 @@ test_expect_success 'receive-pack missing objects fails connectivity check' '
 	test_must_fail git -C remote.git cat-file -e $(git -C repo rev-parse HEAD)
 '
 
-test_expect_success 'receive-pack missing objects bypasses connectivity check' '
+test_expect_success TEE_DOES_NOT_HANG \
+	'receive-pack missing objects bypasses connectivity check' '
 	test_when_finished rm -rf repo remote.git setup.git &&
 
 	git init repo &&

base-commit: 0d42fbd9a1f30c63cf0359a1c5aaa77020972f72

gitgitgadget

"Johannes Schindelin via GitGitGadget" <[email protected]>

Superb. It must have taken a truly heroic effort.

Thanks and congratulations for finally solving the puzzle.

I do agree that Patrick's "wrap the same in a script" smelled like shifting a timing issue and not truly a solution.