Skip to contentSkip to conversation

Avoid suppression of git's exit code in tests

PATCH In review Edwin Fernando 3 participants 13 messages Mar 26, 2023 – Apr 3, 2023

lore
Share & tools
lore
#0 lore raw

cover Avoid suppression of git's exit code in tests

This is a GSOC microproject. Address the problem of errors resulting from calling git in the test scripts being suppressed. Suppressions due to pipes and command substitution were encountered in this file (t/t3701). These regexs where used to search for such occurences .*\<git\>.*|.* and .*$(.*git.*).* respectively (in vim). Both types have been resolved and the tests pass as before commit.

Edwin Fernando (1):
t3701: Avoid suppression of exit status of git

 t/t3701-add-interactive.sh | 25 ++++++++++++++++---------
 1 file changed, 16 insertions(+), 9 deletions(-)

base-commit: d15644fe0226af7ffc874572d968598564a230dd

Signature · 1 line
2.40.0
#1 lore raw

PATCH 1/1 t3701: Avoid suppression of exit status of git

Pipes and command substitution in bash suppress the exit codes of the first executed command. In the test scripts errors when running git should be exposed. Remove all such suppressions of git by writing output to files, chaining commands with &&, or other means.

---
 t/t3701-add-interactive.sh | 25 ++++++++++++++++---------
 1 file changed, 16 insertions(+), 9 deletions(-)
t/t3701-add-interactive.sh +16 -9
index 3a99837d9b..80446b311d 100755
@@ -292,8 +292,10 @@ test_expect_success FILEMODE 'patch does not affect mode' '
 	echo content >>file &&
 	chmod +x file &&
 	printf "n\\ny\\n" | git add -p &&
-	git show :file | grep content &&
-	git diff file | grep "new mode"
+	git show :file > show_file &&
+	grep content show_file &&
+	git diff file > diff_file &&
+	grep "new mode" diff_file
 '
 
 test_expect_success FILEMODE 'stage mode but not hunk' '
@@ -301,8 +303,10 @@ test_expect_success FILEMODE 'stage mode but not hunk' '
 	echo content >>file &&
 	chmod +x file &&
 	printf "y\\nn\\n" | git add -p &&
-	git diff --cached file | grep "new mode" &&
-	git diff          file | grep "+content"
+	git diff --cached file > diff_file &&
+	grep "new mode" diff_file &&
+	git diff file > diff_file &&
+	grep "+content" diff_file
 '
 
 
@@ -311,9 +315,12 @@ test_expect_success FILEMODE 'stage mode and hunk' '
 	echo content >>file &&
 	chmod +x file &&
 	printf "y\\ny\\n" | git add -p &&
-	git diff --cached file | grep "new mode" &&
-	git diff --cached file | grep "+content" &&
-	test -z "$(git diff file)"
+	git diff --cached file > diff_file &&
+	grep "new mode" diff_file &&
+	git diff --cached file diff_file &&
+	grep "+content" diff_file &&
+	git diff file > diff_file &&
+	test -z $(cat diff_file)
 '
 
 # end of tests disabled when filemode is not usable
@@ -970,8 +977,8 @@ test_expect_success 'handle submodules' '
 
 	force_color git -C for-submodules add -p dirty-head >output 2>&1 <y &&
 	git -C for-submodules ls-files --stage dirty-head >actual &&
-	rev="$(git -C for-submodules/dirty-head rev-parse HEAD)" &&
-	grep "$rev" actual
+	git -C for-submodules/dirty-head rev-parse HEAD > rev &&
+	grep -f rev actual
 '
 
 test_expect_success 'set up pathological context' '
Signature · 1 line
2.40.0
#2 lore raw

Re: [GSOC][PATCH 1/1] t3701: Avoid suppression of exit status of git

Hello, Edwin.

In subject line "t3701: Avoid suppression of exit status of git" -- the first word after a colon shouldn't be capitalized. By the way, similar existing commits call it:

don't lose "git" exit codes

and

preserve "git" exit codes

Quoted 1 line · Edwin Fernando

Pipes and command substitution in bash suppress the exit codes of the

Bash is not the only Shell used for running Git's test suite, so referring to it like this in the commit message might not be a good idea.

See also similar commit 9fdc79ecba ("tests: don't lose misc "git" exit codes", 2023-02-06) for inspiration in writing the commit message.

Quoted 14 lines

first executed command. In the test scripts errors when running git should be exposed. Remove all such suppressions of git by writing output to files, chaining commands with &&, or other means.

---
  t/t3701-add-interactive.sh | 25 ++++++++++++++++---------
  1 file changed, 16 insertions(+), 9 deletions(-)
t/t3701-add-interactive.sh +0 -0
index 3a99837d9b..80446b311d 100755

t/t3701-add-interactive.sh test_expect_success FILEMODE 'patch does not affect mode' ' · line 295

Quoted 6 lines
t/t3701-add-interactive.sh +1 -2
@@ -292,8 +292,10 @@ test_expect_success FILEMODE 'patch does not affect mode' '
  	echo content >>file &&
  	chmod +x file &&
  	printf "n\\ny\\n" | git add -p &&
-	git show :file | grep content &&
-	git diff file | grep "new mode"
+	git show :file > show_file &&

Code style here and in all output redirections below: lose the space between redirection and the path, like so:

git show :file >show_file

Also, the suffix "_file" is unnecessary. Just

git show :file >show

or

git show :file >out

or

git show :file >actual

might be better.

Quoted 22 lines
t/t3701-add-interactive.sh +7 -2
+	grep content show_file &&
+	git diff file > diff_file &&
+	grep "new mode" diff_file
  '
 
  test_expect_success FILEMODE 'stage mode but not hunk' '
@@ -301,8 +303,10 @@ test_expect_success FILEMODE 'stage mode but not hunk' '
  	echo content >>file &&
  	chmod +x file &&
  	printf "y\\nn\\n" | git add -p &&
-	git diff --cached file | grep "new mode" &&
-	git diff          file | grep "+content"
+	git diff --cached file > diff_file &&
+	grep "new mode" diff_file &&
+	git diff file > diff_file &&
+	grep "+content" diff_file
  '
 
 
@@ -311,9 +315,12 @@ test_expect_success FILEMODE 'stage mode and hunk' '
  	echo content >>file &&
  	chmod +x file &&

t/t3701-add-interactive.sh test_expect_success FILEMODE 'stage mode and hunk' ' · line 323

Quoted 10 lines
t/t3701-add-interactive.sh +6 -3
  	printf "y\\ny\\n" | git add -p &&
-	git diff --cached file | grep "new mode" &&
-	git diff --cached file | grep "+content" &&
-	test -z "$(git diff file)"
+	git diff --cached file > diff_file &&
+	grep "new mode" diff_file &&
+	git diff --cached file diff_file &&
+	grep "+content" diff_file &&
+	git diff file > diff_file &&
+	test -z $(cat diff_file)

Function test_stdout_line_count can be used here instead of "test -z". test_stdout_line_count would produce a more helpful error message in case of test failure.

Quoted 14 lines
t/t3701-add-interactive.sh +2 -2
  '
 
  # end of tests disabled when filemode is not usable
@@ -970,8 +977,8 @@ test_expect_success 'handle submodules' '
 
  	force_color git -C for-submodules add -p dirty-head >output 2>&1 <y &&
  	git -C for-submodules ls-files --stage dirty-head >actual &&
-	rev="$(git -C for-submodules/dirty-head rev-parse HEAD)" &&
-	grep "$rev" actual
+	git -C for-submodules/dirty-head rev-parse HEAD > rev &&
+	grep -f rev actual
  '
 
  test_expect_success 'set up pathological context' '
#3 lore raw

Re: [GSOC][PATCH 1/1] t3701: Avoid suppression of exit status of git

Quoted 7 lines · Andrei Rybak

On 26/03/2023 19:31, Edwin Fernando wrote:
In subject line "t3701: Avoid suppression of exit status of git" -- the first word after a colon shouldn't be capitalized. By the way, similar existing commits call it:
don't lose "git" exit codes
and
preserve "git" exit codes

Thanks, your review comments covered almost everything I was going to say, and I agree with all you said, so I'll just add a couple comments not covered by your review.

Quoted 6 lines
Quoted 1 line

+ git show :file > show_file &&

Code style here and in all output redirections below: lose the space between redirection and the path, like so:
git show :file >show_file

Quoted 7 lines

Also, the suffix "_file" is unnecessary. Just
git show :file >show
or
git show :file >out
or
git show :file >actual
might be better.

Fully agree with either of the latter two suggestions.

Quoted 6 lines
Quoted 6 lines
t/t3701-add-interactive.sh +2 -3
@@ -311,9 +315,12 @@ test_expect_success FILEMODE 'stage mode and hunk' '
-     git diff --cached file | grep "new mode" &&
-     git diff --cached file | grep "+content" &&
-     test -z "$(git diff file)"
+     git diff --cached file > diff_file &&
+     grep "new mode" diff_file &&
Quoted 8 lines

t/t3701-add-interactive.sh test_expect_success FILEMODE 'stage mode and hunk' ' · line 323

Quoted 4 lines
t/t3701-add-interactive.sh +4 -0
+     git diff --cached file diff_file &&
+     grep "+content" diff_file &&
+     git diff file > diff_file &&
+     test -z $(cat diff_file)

Function test_stdout_line_count can be used here instead of "test -z". test_stdout_line_count would produce a more helpful error message in case of test failure.

The other idiomatic possibility would be to use test_must_be_empty() to verify that the file is empty.

Quoted 4 lines

t/t3701-add-interactive.sh test_expect_success 'handle submodules' ' · line 981

Quoted 4 lines
t/t3701-add-interactive.sh +2 -2
-     rev="$(git -C for-submodules/dirty-head rev-parse HEAD)" &&
-     grep "$rev" actual
+     git -C for-submodules/dirty-head rev-parse HEAD > rev &&
+     grep -f rev actual

This change is unnecessary since this is a cases in which the exit code does not get lost; the exit code of the command substitution becomes the exit code of the assignment:

  % x="$(true)"
  % echo $?
  0
  % x="$(false)"
  % echo $?
  1

So let's drop this change from the patch.

#4 lore raw

PATCH v2 1/1 t3701: don't lose "git" exit codes in test scripts

In reply to Edwin Fernando

Exit codes were lost due to piping and command substitution:

- "git ... | <command>"
- "<command> $(git ... )"

Fix these issues using the intermediate step of writing output to file.
---
Changes in response to review:
- addressed code style issues: ">diff" not "> diff_file"
- a more direct alternative to "test -z $(cat ...)"
- commit message similar to previous commits accomplishing same goals
- revert unnecessary change. keep "<var> = $(git ...)"

 t/t3701-add-interactive.sh | 31 +++++++++++++++----------------
 1 file changed, 15 insertions(+), 16 deletions(-)
t/t3701-add-interactive.sh +15 -16
index 80446b311d..77aad9032a 100755
@@ -292,10 +292,10 @@ test_expect_success FILEMODE 'patch does not affect mode' '
 	echo content >>file &&
 	chmod +x file &&
 	printf "n\\ny\\n" | git add -p &&
-	git show :file > show_file &&
-	grep content show_file &&
-	git diff file > diff_file &&
-	grep "new mode" diff_file
+	git show :file >show &&
+	grep content show &&
+	git diff file >diff &&
+	grep "new mode" diff
 '
 
 test_expect_success FILEMODE 'stage mode but not hunk' '
@@ -303,10 +303,10 @@ test_expect_success FILEMODE 'stage mode but not hunk' '
 	echo content >>file &&
 	chmod +x file &&
 	printf "y\\nn\\n" | git add -p &&
-	git diff --cached file > diff_file &&
-	grep "new mode" diff_file &&
-	git diff file > diff_file &&
-	grep "+content" diff_file
+	git diff --cached file >diff &&
+	grep "new mode" diff &&
+	git diff file >diff &&
+	grep "+content" diff
 '
 
 
@@ -315,12 +315,11 @@ test_expect_success FILEMODE 'stage mode and hunk' '
 	echo content >>file &&
 	chmod +x file &&
 	printf "y\\ny\\n" | git add -p &&
-	git diff --cached file > diff_file &&
-	grep "new mode" diff_file &&
-	git diff --cached file diff_file &&
-	grep "+content" diff_file &&
-	git diff file > diff_file &&
-	test -z $(cat diff_file)
+	git diff --cached file >diff &&
+	grep "new mode" diff &&
+	grep "+content" diff &&
+	git diff file >diff &&
+	test_must_be_empty diff
 '
 
 # end of tests disabled when filemode is not usable
@@ -977,8 +976,8 @@ test_expect_success 'handle submodules' '
 
 	force_color git -C for-submodules add -p dirty-head >output 2>&1 <y &&
 	git -C for-submodules ls-files --stage dirty-head >actual &&
-	git -C for-submodules/dirty-head rev-parse HEAD > rev &&
-	grep -f rev actual
+	rev="$(git -C for-submodules/dirty-head rev-parse HEAD)" &&
+	grep "$rev" actual
 '
 
 test_expect_success 'set up pathological context' '
Signature · 1 line
2.40.0
#5 lore raw

Re: [GSOC][PATCH v2] t3701: don't lose "git" exit codes in test scripts

Quoted 1 line · Edwin Fernando

Exit codes were lost due to piping and command substitution:

s/were/are/

See section "[[present-tense]]" in Documentation/SubmittingPatches:

The problem statement that describes the status quo is written in the present tense. Write "The code does X when it is given input Y", instead of "The code used to do Y when given input X". You do not have to say "Currently"---the status quo in the problem statement is about the code _without_ your change, by project convention.

Quoted 68 lines

- "git ... | <command>"
- "<command> $(git ... )"

Fix these issues using the intermediate step of writing output to file.
---
Changes in response to review:
- addressed code style issues: ">diff" not "> diff_file"
- a more direct alternative to "test -z $(cat ...)"
- commit message similar to previous commits accomplishing same goals
- revert unnecessary change. keep "<var> = $(git ...)"

  t/t3701-add-interactive.sh | 31 +++++++++++++++----------------
  1 file changed, 15 insertions(+), 16 deletions(-)
t/t3701-add-interactive.sh +13 -14
index 80446b311d..77aad9032a 100755
@@ -292,10 +292,10 @@ test_expect_success FILEMODE 'patch does not affect mode' '
  	echo content >>file &&
  	chmod +x file &&
  	printf "n\\ny\\n" | git add -p &&
-	git show :file > show_file &&
-	grep content show_file &&
-	git diff file > diff_file &&
-	grep "new mode" diff_file
+	git show :file >show &&
+	grep content show &&
+	git diff file >diff &&
+	grep "new mode" diff
  '
 
  test_expect_success FILEMODE 'stage mode but not hunk' '
@@ -303,10 +303,10 @@ test_expect_success FILEMODE 'stage mode but not hunk' '
  	echo content >>file &&
  	chmod +x file &&
  	printf "y\\nn\\n" | git add -p &&
-	git diff --cached file > diff_file &&
-	grep "new mode" diff_file &&
-	git diff file > diff_file &&
-	grep "+content" diff_file
+	git diff --cached file >diff &&
+	grep "new mode" diff &&
+	git diff file >diff &&
+	grep "+content" diff
  '
 
 
@@ -315,12 +315,11 @@ test_expect_success FILEMODE 'stage mode and hunk' '
  	echo content >>file &&
  	chmod +x file &&
  	printf "y\\ny\\n" | git add -p &&
-	git diff --cached file > diff_file &&
-	grep "new mode" diff_file &&
-	git diff --cached file diff_file &&
-	grep "+content" diff_file &&
-	git diff file > diff_file &&
-	test -z $(cat diff_file)
+	git diff --cached file >diff &&
+	grep "new mode" diff &&
+	grep "+content" diff &&
+	git diff file >diff &&
+	test_must_be_empty diff
  '
 
  # end of tests disabled when filemode is not usable

t/t3701-add-interactive.sh test_expect_success 'handle submodules' ' · line 983

Quoted 10 lines
t/t3701-add-interactive.sh +2 -2
@@ -977,8 +976,8 @@ test_expect_success 'handle submodules' '
 
  	force_color git -C for-submodules add -p dirty-head >output 2>&1 <y &&
  	git -C for-submodules ls-files --stage dirty-head >actual &&
-	git -C for-submodules/dirty-head rev-parse HEAD > rev &&
-	grep -f rev actual
+	rev="$(git -C for-submodules/dirty-head rev-parse HEAD)" &&
+	grep "$rev" actual
  '
 
  test_expect_success 'set up pathological context' '

It seems that you've created a second commit on top of the patch you've originally sent and have only sent the second commit as "v2". That is not what you want to show your reviewers, and more importantly, that is not what we want to record in our history.

Instead, you should squash these two commits together and send it as a single patch, as though the commit from v1 doesn't exist.

#6 lore raw

PATCH v3 1/1 t3701: don't lose "git" exit codes in test scripts

In reply to Edwin Fernando

Exit codes are lost due to piping and command substitution:

- "git ... | <command>"
- "<command> $(git ... )"

Fix these issues using the intermediate step of writing output to file.

---
Changes from v2:
- use git rebase to squash commit history,
and diff with upstream commit for review
- use present tense for code before the commit.

 t/t3701-add-interactive.sh | 20 +++++++++++++-------
 1 file changed, 13 insertions(+), 7 deletions(-)
t/t3701-add-interactive.sh +13 -7
index 3a99837d9b..77aad9032a 100755
@@ -292,8 +292,10 @@ test_expect_success FILEMODE 'patch does not affect mode' '
 	echo content >>file &&
 	chmod +x file &&
 	printf "n\\ny\\n" | git add -p &&
-	git show :file | grep content &&
-	git diff file | grep "new mode"
+	git show :file >show &&
+	grep content show &&
+	git diff file >diff &&
+	grep "new mode" diff
 '
 
 test_expect_success FILEMODE 'stage mode but not hunk' '
@@ -301,8 +303,10 @@ test_expect_success FILEMODE 'stage mode but not hunk' '
 	echo content >>file &&
 	chmod +x file &&
 	printf "y\\nn\\n" | git add -p &&
-	git diff --cached file | grep "new mode" &&
-	git diff          file | grep "+content"
+	git diff --cached file >diff &&
+	grep "new mode" diff &&
+	git diff file >diff &&
+	grep "+content" diff
 '
 
 
@@ -311,9 +315,11 @@ test_expect_success FILEMODE 'stage mode and hunk' '
 	echo content >>file &&
 	chmod +x file &&
 	printf "y\\ny\\n" | git add -p &&
-	git diff --cached file | grep "new mode" &&
-	git diff --cached file | grep "+content" &&
-	test -z "$(git diff file)"
+	git diff --cached file >diff &&
+	grep "new mode" diff &&
+	grep "+content" diff &&
+	git diff file >diff &&
+	test_must_be_empty diff
 '
 
 # end of tests disabled when filemode is not usable
Signature · 1 line
2.40.0
#7 lore raw

Re: [GSOC][PATCH v3] t3701: don't lose "git" exit codes in test scripts

Quoted 5 lines · Edwin Fernando

Exit codes are lost due to piping and command substitution:

- "git ... | <command>"
- "<command> $(git ... )"

Quoted 8 lines

Fix these issues using the intermediate step of writing output to file.

---
Changes from v2:
- use git rebase to squash commit history,
and diff with upstream commit for review
- use present tense for code before the commit.

Thanks. The patch is looking a lot better. One comment, though...

t/t3701-add-interactive.sh test_expect_success FILEMODE 'stage mode and hunk' ' · line 321

Quoted 12 lines
t/t3701-add-interactive.sh +5 -3
@@ -311,9 +315,11 @@ test_expect_success FILEMODE 'stage mode and hunk' '
        printf "y\\ny\\n" | git add -p &&
-       git diff --cached file | grep "new mode" &&
-       git diff --cached file | grep "+content" &&
-       test -z "$(git diff file)"
+       git diff --cached file >diff &&
+       grep "new mode" diff &&
+       grep "+content" diff &&
+       git diff file >diff &&
+       test_must_be_empty diff
 '

The other changes in your patch are worthwhile, but it seems that you have based this patch on an outdated copy of the repository. This particular change had already been made by 9fdc79ecba (tests: don't lose misc "git" exit codes, 2023-02-06) a couple months ago[1].

So, you'll want to update your repository to the latest available from Junio, re-roll using "master" as a base for the patch, and drop this change while keeping the others.

[1]: https://github.com/git/git/commit/9fdc79ecba0f4a3ef885f1409d2db5a1dbabd649

#8 lore raw

PATCH v4 1/1 t3701: don't lose "git" exit codes in test scripts

In reply to Edwin Fernando

Exit codes are lost due to piping: "git ... | <command>"
Fix these issues using the intermediate step of writing output to file.

---
Apply my changes on top of the latest commit in upstream history. Test 76 in t3701 fails. This commit breaks the test:
a9f4a01760 (Merge branch 'jk/add-p-unmerged-fix', 2023-03-19) Identified by checking out commits in:
"git log --oneline -- t/t3701-add-interactive.sh"
I don't know if it is normal to have broken tests in the main repository.

 t/t3701-add-interactive.sh | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)
t/t3701-add-interactive.sh +8 -4
index 3982b6b49d..b8291206a0 100755
@@ -286,8 +286,10 @@ test_expect_success FILEMODE 'patch does not affect mode' '
 	echo content >>file &&
 	chmod +x file &&
 	printf "n\\ny\\n" | git add -p &&
-	git show :file | grep content &&
-	git diff file | grep "new mode"
+	git show :file >show &&
+	grep content show &&
+	git diff file >diff &&
+	grep "new mode" diff
 '
 
 test_expect_success FILEMODE 'stage mode but not hunk' '
@@ -295,8 +297,10 @@ test_expect_success FILEMODE 'stage mode but not hunk' '
 	echo content >>file &&
 	chmod +x file &&
 	printf "y\\nn\\n" | git add -p &&
-	git diff --cached file | grep "new mode" &&
-	git diff          file | grep "+content"
+	git diff --cached file >diff &&
+	grep "new mode" diff &&
+	git diff file >diff &&
+	grep "+content" diff
 '
 
 
Signature · 1 line
2.40.0
#9 lore raw

Re: [GSOC][PATCH v4] t3701: don't lose "git" exit codes in test scripts

Quoted 11 lines · Edwin Fernando

Exit codes are lost due to piping: "git ... | <command>"
Fix these issues using the intermediate step of writing output to file.

---
Apply my changes on top of the latest commit in upstream history. Test 76 in t3701 fails. This commit breaks the test:
a9f4a01760 (Merge branch 'jk/add-p-unmerged-fix', 2023-03-19) Identified by checking out commits in:
"git log --oneline -- t/t3701-add-interactive.sh"
I don't know if it is normal to have broken tests in the main repository.

It's not normal to have broken tests in the repository; they should not exist.

However, it's not clear what breakage you are seeing. In my testing, I don't see any failures, either in a9f4a01760 or after applying your patch to "master". Can you provide more information about the failure you're experiencing?

#10 lore raw

Re: [GSOC][PATCH v4] t3701: don't lose "git" exit codes in test scripts

#11 lore raw

Re: [GSOC][PATCH v4] t3701: don't lose "git" exit codes in test scripts

In reply to Eric Sunshine
Quoted 1 line · Eric Sunshine

It's not normal to have broken tests in the repository; they should not exist.

Sorry the test did not break after all.

Quoted 4 lines

However, it's not clear what breakage you are seeing. In my testing, I don't see any failures, either in a9f4a01760 or after applying your patch to "master". Can you provide more information about the failure you're experiencing?

I cloned git again, ran "make" and all the tests in t3701 passed. The local repository I have been working on so far still gives the same error though. The error message is not very helpful so I haven't included it. I ran "make" on the faulty repository and the tests passed this time. So I think the problem was that whatever "make" does was not up to date. Also please ignore some mails you may have received because of misconfiguration.

#12 lore raw

Re: [GSOC][PATCH v4] t3701: don't lose "git" exit codes in test scripts

Quoted 6 lines · Edwin Fernando
Quoted 4 lines · Eric Sunshine

However, it's not clear what breakage you are seeing. In my testing, I don't see any failures, either in a9f4a01760 or after applying your patch to "master". Can you provide more information about the failure you're experiencing?

Quoted 7 lines

I cloned git again, ran "make" and all the tests in t3701 passed. The local repository I have been working on so far still gives the same error though. The error message is not very helpful so I haven't included it. I ran "make" on the faulty repository and the tests passed this time. So I think the problem was that whatever "make" does was not up to date. Also please ignore some mails you may have received because of misconfiguration.

That makes sense. An outdated build could indeed account for an unexpected test failure.