Skip to contentSkip to conversation

ci(github): restore "print test failures" step name

PATCH In review Phillip Wood via GitGitGadget 2 participants 2 messages Jan 3, 2023 – Jan 4, 2023

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 ci(github): restore "print test failures" step name

From: Phillip Wood <[email protected]>

As well as removing the explicit shell setting d8b21a0fe2 (CI: don't explicitly pick "bash" shell outside of Windows, fix regression, 2022-12-07) also reverted the name of the print test failures step introduced by 5aeb145780f (ci(github): bring back the 'print test failures' step, 2022-06-08). This is unfortunate as 5aeb145780f added a message to direct contributors to the "print test failures" step when a test fails and that step is no-longer known by that name on the non-windows ci jobs.

In principle we could update the message to print the correct name for the step but then we'd have to deal with having two different names for the same step on different jobs. It is simpler for the implementation and contributors to use the same name for this step on all jobs.

---
ci(github): restore "print test failures" step name

Ævar seems to think the name change in 5aeb145780f was unintentional [1] but looking at the original commit I don't think that's the case.

[1]
https://lore.kernel.org/git/[email protected]/

Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1453%2Fphillipwood%2Fci-print-test-failures-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1453/phillipwood/ci-print-test-failures-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/1453

 .github/workflows/main.yml | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
.github/workflows/main.yml +4 -2
index e67847a682c..126eac8239d 100644
@@ -265,8 +265,9 @@ jobs:
     - uses: actions/checkout@v3
     - run: ci/install-dependencies.sh
     - run: ci/run-build-and-tests.sh
-    - run: ci/print-test-failures.sh
+    - name: print test failures
       if: failure() && env.FAILED_TEST_ARTIFACTS != ''
+      run: ci/print-test-failures.sh
     - name: Upload failed tests' directories
       if: failure() && env.FAILED_TEST_ARTIFACTS != ''
       uses: actions/upload-artifact@v3
@@ -298,8 +299,9 @@ jobs:
       if: matrix.vector.jobname == 'linux32'
     - run: ci/install-docker-dependencies.sh
     - run: ci/run-build-and-tests.sh
-    - run: ci/print-test-failures.sh
+    - name: print test failures
       if: failure() && env.FAILED_TEST_ARTIFACTS != ''
+      run: ci/print-test-failures.sh
     - name: Upload failed tests' directories
       if: failure() && env.FAILED_TEST_ARTIFACTS != '' && matrix.vector.jobname != 'linux32'
       uses: actions/upload-artifact@v3

base-commit: 8a4e8f6a67e7fc97048d4666eec38399b88e0e3b

Signature · 1 line
gitgitgadget
#1 lore raw
Quoted 17 lines · Phillip Wood via GitGitGadget

From: Phillip Wood <[email protected]>

As well as removing the explicit shell setting d8b21a0fe2 (CI: don't explicitly pick "bash" shell outside of Windows, fix regression, 2022-12-07) also reverted the name of the print test failures step introduced by 5aeb145780f (ci(github): bring back the 'print test failures' step, 2022-06-08). This is unfortunate as 5aeb145780f added a message to direct contributors to the "print test failures" step when a test fails and that step is no-longer known by that name on the non-windows ci jobs.

In principle we could update the message to print the correct name for the step but then we'd have to deal with having two different names for the same step on different jobs. It is simpler for the implementation and contributors to use the same name for this step on all jobs.

Quoted 8 lines

---
ci(github): restore "print test failures" step name

Ævar seems to think the name change in 5aeb145780f was unintentional [1] but looking at the original commit I don't think that's the case.

[1]
https://lore.kernel.org/git/[email protected]/

Reading it again I think you're right, i.e. that the migration of everything to "bash" was unintentional & a result of copy/pasting, but the initial goal was to mention the "print test failures" step....

Quoted 26 lines
.github/workflows/main.yml +4 -3
index e67847a682c..126eac8239d 100644
--- a/.github/workflows/main.yml
@@ -265,8 +265,9 @@ jobs:
     - uses: actions/checkout@v3
     - run: ci/install-dependencies.sh
     - run: ci/run-build-and-tests.sh
-    - run: ci/print-test-failures.sh
+    - name: print test failures
       if: failure() && env.FAILED_TEST_ARTIFACTS != ''
+      run: ci/print-test-failures.sh
     - name: Upload failed tests' directories
       if: failure() && env.FAILED_TEST_ARTIFACTS != ''
       uses: actions/upload-artifact@v3
@@ -298,8 +299,9 @@ jobs:
       if: matrix.vector.jobname == 'linux32'
     - run: ci/install-docker-dependencies.sh
     - run: ci/run-build-and-tests.sh
-    - run: ci/print-test-failures.sh
+    - name: print test failures
       if: failure() && env.FAILED_TEST_ARTIFACTS != ''
+      run: ci/print-test-failures.sh
     - name: Upload failed tests' directories
       if: failure() && env.FAILED_TEST_ARTIFACTS != '' && matrix.vector.jobname != 'linux32'
       uses: actions/upload-artifact@v3
Quoted 1 line

base-commit: 8a4e8f6a67e7fc97048d4666eec38399b88e0e3b

...but as far as moving forward, why make every other job be inconsistent in naming this one step, rather than just making the Windows ones consistent with rest?

I.e. why not:

	diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
	index e67847a682c..2bfb841206e 100644
	--- a/.github/workflows/main.yml
	+++ b/.github/workflows/main.yml
	@@ -119,8 +119,7 @@ jobs:
	     - name: test
	       shell: bash
	       run: . /etc/profile && ci/run-test-slice.sh ${{matrix.nr}} 10
	-    - name: print test failures
	-      if: failure() && env.FAILED_TEST_ARTIFACTS != ''
	+    - if: failure() && env.FAILED_TEST_ARTIFACTS != ''
	       shell: bash
	       run: ci/print-test-failures.sh
	     - name: Upload failed tests' directories
	diff --git a/ci/lib.sh b/ci/lib.sh
	index db7105e8a8d..d6450fd4957 100755
	--- a/ci/lib.sh
	+++ b/ci/lib.sh
	@@ -183,7 +183,7 @@ then
	 			test_name="${test_exit%.exit}"
	 			test_name="${test_name##*/}"
	 			printf "\\e[33m\\e[1m=== Failed test: ${test_name} ===\\e[m\\n"
	-			echo "The full logs are in the 'print test failures' step below."
	+			echo "The full logs are in the 'ci/print-test-failures.sh' step below."
	 			echo "See also the 'failed-tests-*' artifacts attached to this run."
	 			cat "t/test-results/$test_name.markup"

The implicit argument being made here & in the original 5aeb145780f is that it's a good thing to give explicit "names"'s to all of these steps.

For some of the steps I agree, e.g. "test" is probably better than ". /etc/profile && ci/run-test-slice.sh ${{matrix.nr}} 10" or whatever, and e.g. "generate Visual Studio solution" is definitely better than some multi-line shellscript (I'm not sure how the UX would even turn that into a collapsible title).

But in the case of "ci/print-test-failures.sh", "ci/run-build-and-tests.sh", "ci/install-dependencies.sh" etc. the script name is already self-descriptive, so giving it a name just serves to obscure the connection between the step & the script implementing it, which is otherwise immediately apparent.

So I think it's better to just remove the "name" label in those cases where the Windows jobs can use such a script, which we're already not naming in the *nix jobs.

It's a minor point either way, and your change would also be an improvement. I just think it's better to aim for e.g. this snippet (from the current main.yml):

- run: ci/install-docker-dependencies.sh
- run: ci/run-build-and-tests.sh
- run: ci/print-test-failures.sh

Rather than providing "name" fields for them all, and doubling the line count for no apparent (at least to me) reason.