Skip to contentSkip to conversation

Hash-independent tests, part 5

PATCH Superseded brian m. carlson 8 participants 229 messages Aug 18, 2019 – Apr 16, 2021 Page 2 of 2

Subsystems (2)
lore
Share & tools
lore
#97 lore raw

PATCH 1/1 tests: remove all uses of test_i18cmp

In reply to Ævar Arnfjörð Bjarmason

Finish the removal I started in 1108cea7f8e (tests: remove most uses of test_i18ncmp, 2021-02-11). At that time the function wasn't removed due to disruption with in-flight changes, remove the occurrences that have landed since then.

As of writing this there are no test_i18ncmp uses between "master" and "seen", so let's also remove the function to finally put it to rest.

---
 t/t3437-rebase-fixup-options.sh | 6 +++---
 t/t6300-for-each-ref.sh         | 6 +++---
 t/test-lib-functions.sh         | 7 -------
 3 files changed, 6 insertions(+), 13 deletions(-)
t/t3437-rebase-fixup-options.sh +3 -3
index d0bdc7ed02d..c023fefd681 100755
@@ -157,7 +157,7 @@ test_expect_success 'sequence of fixup, fixup -C & squash --signoff works' '
 		git -c commit.status=false rebase -ik --signoff A &&
 	git diff-tree --exit-code --patch HEAD B3 -- &&
 	test_cmp_rev HEAD^ A &&
-	test_i18ncmp "$TEST_DIRECTORY/t3437/expected-squash-message" \
+	test_cmp "$TEST_DIRECTORY/t3437/expected-squash-message" \
 		actual-squash-message
 '
 
@@ -191,7 +191,7 @@ test_expect_success 'sequence squash, fixup & fixup -c gives combined message' '
 	FAKE_LINES="1 squash 2 fixup 3 fixup_-c 4" \
 		FAKE_MESSAGE_COPY=actual-combined-message \
 		git -c commit.status=false rebase -i A &&
-	test_i18ncmp "$TEST_DIRECTORY/t3437/expected-combined-message" \
+	test_cmp "$TEST_DIRECTORY/t3437/expected-combined-message" \
 		actual-combined-message &&
 	test_cmp_rev HEAD^ A
 '
@@ -204,7 +204,7 @@ test_expect_success 'fixup -C works upon --autosquash with amend!' '
 						--signoff A &&
 	git diff-tree --exit-code --patch HEAD B3 -- &&
 	test_cmp_rev HEAD^ A &&
-	test_i18ncmp "$TEST_DIRECTORY/t3437/expected-squash-message" \
+	test_cmp "$TEST_DIRECTORY/t3437/expected-squash-message" \
 		actual-squash-message
 '
 
t/t6300-for-each-ref.sh +3 -3
index cac7f443d00..7cfa934913b 100755
@@ -945,9 +945,9 @@ test_failing_trailer_option () {
 	test_expect_success "$title" '
 		# error message cannot be checked under i18n
 		test_must_fail git for-each-ref --format="%($option)" refs/heads/main 2>actual &&
-		test_i18ncmp expect actual &&
+		test_cmp expect actual &&
 		test_must_fail git for-each-ref --format="%(contents:$option)" refs/heads/main 2>actual &&
-		test_i18ncmp expect actual
+		test_cmp expect actual
 	'
 }
 
@@ -966,7 +966,7 @@ test_expect_success 'if arguments, %(contents:trailers) shows error if colon is
 	fatal: unrecognized %(contents) argument: trailersonly
 	EOF
 	test_must_fail git for-each-ref --format="%(contents:trailersonly)" 2>actual &&
-	test_i18ncmp expect actual
+	test_cmp expect actual
 '
 
 test_expect_success 'basic atom: head contents:trailers' '
t/test-lib-functions.sh +0 -7
index 6348e8d7339..b823c140271 100644
@@ -1025,13 +1025,6 @@ test_cmp_bin () {
 	cmp "$@"
 }
 
-# Wrapper for test_cmp which used to be used for
-# GIT_TEST_GETTEXT_POISON=false. Only here as a shim for other
-# in-flight changes. Should not be used and will be removed soon.
-test_i18ncmp () {
-	test_cmp "$@"
-}
-
 # Wrapper for grep which used to be used for
 # GIT_TEST_GETTEXT_POISON=false. Only here as a shim for other
 # in-flight changes. Should not be used and will be removed soon.
Signature · 1 line
2.31.1.646.g06d606cb9c7
#98 lore raw

cover diff tests: un-flaky and post-gettext-poison cleanup

In reply to [1] because they're related post-GETTEXT_POISON cleanup, but both apply on top of master.

I had a flaky t4013 test with some local WIP code that I hadn't investigated, as it turns out re-running the test was hiding the failure. 2/2 fixes that, 1/2 is a minor post-GETTEXT_POISON cleanup.

1. https://lore.kernel.org/git/[email protected]

Ævar Arnfjörð Bjarmason (2):
diff tests: remove redundant case statement diff tests: rewrite flakyness-causing test "aid"

 t/t4013-diff-various.sh | 31 +++++++++++++++----------------
 1 file changed, 15 insertions(+), 16 deletions(-)
Signature · 1 line
2.31.1.646.g06d606cb9c7
#99 lore raw

PATCH 1/2 diff tests: remove redundant case statement

Remove a case statement whose two arms executed the same code. This was added for i18n testing in 6dd88832e77 (diffstat summary line varies by locale: miscellany, 2012-03-13), and was changed from test_i18ncmp in my 1108cea7f8e (tests: remove most uses of test_i18ncmp, 2021-02-11).

---
 t/t4013-diff-various.sh | 8 +-------
 1 file changed, 1 insertion(+), 7 deletions(-)
t/t4013-diff-various.sh +1 -7
index 6cca8b84a6b..67f6411aff9 100755
@@ -220,13 +220,7 @@ do
 		then
 			process_diffs "$actual" >actual &&
 			process_diffs "$expect" >expect &&
-			case $cmd in
-			*format-patch* | *-stat*)
-				test_cmp expect actual;;
-			*)
-				test_cmp expect actual;;
-			esac &&
-			rm -f "$actual" actual expect
+			test_cmp expect actual
 		else
 			# this is to help developing new tests.
 			cp "$actual" "$expect"
Signature · 1 line
2.31.1.646.g06d606cb9c7
#100 lore raw

Re: [PATCH 1/2] diff tests: remove redundant case statement

Quoted 6 lines · Ævar Arnfjörð Bjarmason
t/t4013-diff-various.sh +0 -0
index 6cca8b84a6b..67f6411aff9 100755

t/t4013-diff-various.sh do · line 229

Quoted 10 lines
t/t4013-diff-various.sh +0 -7
@@ -220,13 +220,7 @@ do
                then
                        process_diffs "$actual" >actual &&
                        process_diffs "$expect" >expect &&
-                       case $cmd in
-                       *format-patch* | *-stat*)
-                               test_cmp expect actual;;
-                       *)
-                               test_cmp expect actual;;
-                       esac &&
-                       rm -f "$actual" actual expect

Was this line removed by accident? It doesn't seem related to the described change. Shouldn't this line be left after...

t/t4013-diff-various.sh do · line 223

Quoted 1 line
t/t4013-diff-various.sh +1 -0
+                       test_cmp expect actual

...this one?

Quoted 6 lines
t/t4013-diff-various.sh +0 -1
                else
                        # this is to help developing new tests.
                        cp "$actual" "$expect"
--

2.31.1.646.g06d606cb9c7

#101 lore raw

Re: [PATCH 1/2] diff tests: remove redundant case statement

Quoted 22 lines · Matheus Tavares Bernardino
Quoted 6 lines · Ævar Arnfjörð Bjarmason
t/t4013-diff-various.sh +0 -0
index 6cca8b84a6b..67f6411aff9 100755

t/t4013-diff-various.sh do · line 229

Quoted 10 lines
t/t4013-diff-various.sh +0 -7
@@ -220,13 +220,7 @@ do
                then
                        process_diffs "$actual" >actual &&
                        process_diffs "$expect" >expect &&
-                       case $cmd in
-                       *format-patch* | *-stat*)
-                               test_cmp expect actual;;
-                       *)
-                               test_cmp expect actual;;
-                       esac &&
-                       rm -f "$actual" actual expect

Was this line removed by accident? It doesn't seem related to the described change. Shouldn't this line be left after...

Quoted 3 lines

t/t4013-diff-various.sh do · line 223

Quoted 1 line
t/t4013-diff-various.sh +1 -0
+                       test_cmp expect actual

...this one?

Yes, it should be in 2/2 where this gets turned into a test_when_finished "rm [...]", sorry.

#102 lore raw

PATCH 2/2 diff tests: rewrite flakyness-causing test "aid"

In reply to Ævar Arnfjörð Bjarmason

If a new test is added to this "while read magic cmd" test facility added in 3c2f75b590c (t4013: add tests for diff/log family output options., 2006-06-26) but no test file is added it'll fail the first time, but then succeed on subsequent runs as a new file has been added in t4013.

Let's accomplish the same aim in way that doesn't cause subsequent test runs to succeed. If we can't find the file we'll BUG out, and suggest to the developer that they copy our "expect.new" file over, unlike the previous "expect" file this won't be picked up on subsequent runs.

---
 t/t4013-diff-various.sh | 25 +++++++++++++++----------
 1 file changed, 15 insertions(+), 10 deletions(-)
t/t4013-diff-various.sh +15 -10
index 67f6411aff9..228ff100c61 100755
@@ -200,10 +200,12 @@ do
 	esac
 	test=$(echo "$label" | sed -e 's|[/ ][/ ]*|_|g')
 	pfx=$(printf "%04d" $test_count)
-	expect="$TEST_DIRECTORY/t4013/diff.$test"
+	expect_relative="t4013/diff.$test"
+	expect="$TEST_DIRECTORY/$expect_relative"
 	actual="$pfx-diff.$test"
 
 	test_expect_$status "git $cmd # magic is ${magic:-(not used)}" '
+		test_when_finished "rm $actual" &&
 		{
 			echo "$ git $cmd"
 			case "$magic" in
@@ -216,16 +218,19 @@ do
 			    -e "s/^\\(.*mixed; boundary=\"-*\\)$V\\(-*\\)\"\$/\\1g-i-t--v-e-r-s-i-o-n\2\"/"
 			echo "\$"
 		} >"$actual" &&
-		if test -f "$expect"
+
+		if ! test -f "$expect"
 		then
-			process_diffs "$actual" >actual &&
-			process_diffs "$expect" >expect &&
-			test_cmp expect actual
-		else
-			# this is to help developing new tests.
-			cp "$actual" "$expect"
-			false
-		fi
+			expect_new="$expect.new" &&
+			cp "$actual" "$expect_new" &&
+			BUG "Have no \"$expect_relative\", new test? The output is in \"$expect_new\", maybe use that?"
+		fi &&
+
+		test_when_finished "rm actual" &&
+		process_diffs "$actual" >actual &&
+		test_when_finished "rm expect" &&
+		process_diffs "$expect" >expect &&
+		test_cmp expect actual
 	'
 done <<\EOF
 diff-tree initial
Signature · 1 line
2.31.1.646.g06d606cb9c7
#103 lore raw

Re: [PATCH 2/2] diff tests: rewrite flakyness-causing test "aid"

Quoted 8 lines · Ævar Arnfjörð Bjarmason

If a new test is added to this "while read magic cmd" test facility added in 3c2f75b590c (t4013: add tests for diff/log family output options., 2006-06-26) but no test file is added it'll fail the first time, but then succeed on subsequent runs as a new file has been added in t4013.

Let's accomplish the same aim in way that doesn't cause subsequent

s/in way/in a way/ ?

Quoted 15 lines

test runs to succeed. If we can't find the file we'll BUG out, and suggest to the developer that they copy our "expect.new" file over, unlike the previous "expect" file this won't be picked up on subsequent runs.

---
 t/t4013-diff-various.sh | 25 +++++++++++++++----------
 1 file changed, 15 insertions(+), 10 deletions(-)
t/t4013-diff-various.sh +0 -0
index 67f6411aff9..228ff100c61 100755

t/t4013-diff-various.sh do · line 208

Quoted 10 lines
t/t4013-diff-various.sh +3 -1
@@ -200,10 +200,12 @@ do
        esac
        test=$(echo "$label" | sed -e 's|[/ ][/ ]*|_|g')
        pfx=$(printf "%04d" $test_count)
-       expect="$TEST_DIRECTORY/t4013/diff.$test"
+       expect_relative="t4013/diff.$test"
+       expect="$TEST_DIRECTORY/$expect_relative"
        actual="$pfx-diff.$test"
 
        test_expect_$status "git $cmd # magic is ${magic:-(not used)}" '
+               test_when_finished "rm $actual" &&

Nit: before these two patches, "$actual" was only removed when the test succeeded. So, in case of failure, the failed output files would still be there for debugging. It might be interesting to keep this behavior and only remove "$actual" at the end of the test.

Quoted 22 lines
t/t4013-diff-various.sh +5 -9
                {
                        echo "$ git $cmd"
                        case "$magic" in
@@ -216,16 +218,19 @@ do
                            -e "s/^\\(.*mixed; boundary=\"-*\\)$V\\(-*\\)\"\$/\\1g-i-t--v-e-r-s-i-o-n\2\"/"
                        echo "\$"
                } >"$actual" &&
-               if test -f "$expect"
+
+               if ! test -f "$expect"
                then
-                       process_diffs "$actual" >actual &&
-                       process_diffs "$expect" >expect &&
-                       test_cmp expect actual
-               else
-                       # this is to help developing new tests.
-                       cp "$actual" "$expect"
-                       false
-               fi
+                       expect_new="$expect.new" &&
+                       cp "$actual" "$expect_new" &&
+                       BUG "Have no \"$expect_relative\", new test? The output is in \"$expect_new\", maybe use that?"

t/t4013-diff-various.sh do · line 236

Quoted 10 lines
t/t4013-diff-various.sh +7 -0
+               fi &&
+
+               test_when_finished "rm actual" &&
+               process_diffs "$actual" >actual &&
+               test_when_finished "rm expect" &&
+               process_diffs "$expect" >expect &&
+               test_cmp expect actual
        '
 done <<\EOF
 diff-tree initial

The rest LGTM, thanks.

#104 lore raw

Re: [PATCH 2/2] diff tests: rewrite flakyness-causing test "aid"

Quoted 9 lines · Matheus Tavares Bernardino
Quoted 7 lines · Ævar Arnfjörð Bjarmason

If a new test is added to this "while read magic cmd" test facility added in 3c2f75b590c (t4013: add tests for diff/log family output options., 2006-06-26) but no test file is added it'll fail the first time, but then succeed on subsequent runs as a new file has been added in t4013.

Quoted 3 lines
Quoted 1 line

Let's accomplish the same aim in way that doesn't cause subsequent

s/in way/in a way/ ?

*nod*

Quoted 23 lines
Quoted 23 lines

test runs to succeed. If we can't find the file we'll BUG out, and suggest to the developer that they copy our "expect.new" file over, unlike the previous "expect" file this won't be picked up on subsequent runs.

---
 t/t4013-diff-various.sh | 25 +++++++++++++++----------
 1 file changed, 15 insertions(+), 10 deletions(-)
t/t4013-diff-various.sh +2 -1
index 67f6411aff9..228ff100c61 100755
@@ -200,10 +200,12 @@ do
        esac
        test=$(echo "$label" | sed -e 's|[/ ][/ ]*|_|g')
        pfx=$(printf "%04d" $test_count)
-       expect="$TEST_DIRECTORY/t4013/diff.$test"
+       expect_relative="t4013/diff.$test"
+       expect="$TEST_DIRECTORY/$expect_relative"
        actual="$pfx-diff.$test"
 
Quoted 7 lines

t/t4013-diff-various.sh do · line 208

Quoted 2 lines
t/t4013-diff-various.sh +1 -0
        test_expect_$status "git $cmd # magic is ${magic:-(not used)}" '
+               test_when_finished "rm $actual" &&

Nit: before these two patches, "$actual" was only removed when the test succeeded. So, in case of failure, the failed output files would still be there for debugging. It might be interesting to keep this behavior and only remove "$actual" at the end of the test.

Either I'm missing something or you are, that's how test_when_finished works.

It's skipped under e.g. "--immediate --debug". See b586744a864 (test:
skip clean-up when running under --immediate mode, 2011-06-27)

Maybe there's some edge case where we'd like to keep the files that it's not covering, but then we should patch it to do the right thing, not use manual "rm" at the end instead.

Quoted 26 lines
Quoted 26 lines
t/t4013-diff-various.sh +9 -9
                {
                        echo "$ git $cmd"
                        case "$magic" in
@@ -216,16 +218,19 @@ do
                            -e "s/^\\(.*mixed; boundary=\"-*\\)$V\\(-*\\)\"\$/\\1g-i-t--v-e-r-s-i-o-n\2\"/"
                        echo "\$"
                } >"$actual" &&
-               if test -f "$expect"
+
+               if ! test -f "$expect"
                then
-                       process_diffs "$actual" >actual &&
-                       process_diffs "$expect" >expect &&
-                       test_cmp expect actual
-               else
-                       # this is to help developing new tests.
-                       cp "$actual" "$expect"
-                       false
-               fi
+                       expect_new="$expect.new" &&
+                       cp "$actual" "$expect_new" &&
+                       BUG "Have no \"$expect_relative\", new test? The output is in \"$expect_new\", maybe use that?"
+               fi &&
+
+               test_when_finished "rm actual" &&
+               process_diffs "$actual" >actual &&
Quoted 8 lines

t/t4013-diff-various.sh do · line 236

Quoted 6 lines
t/t4013-diff-various.sh +3 -0
+               test_when_finished "rm expect" &&
+               process_diffs "$expect" >expect &&
+               test_cmp expect actual
        '
 done <<\EOF
 diff-tree initial

The rest LGTM, thanks.

Thanks a lot for the review!

#105 lore raw

Re: [PATCH 2/2] diff tests: rewrite flakyness-causing test "aid"

Quoted 30 lines · Ævar Arnfjörð Bjarmason
Quoted 18 lines · Matheus Tavares Bernardino
Quoted 16 lines · Ævar Arnfjörð Bjarmason
 t/t4013-diff-various.sh | 25 +++++++++++++++----------
 1 file changed, 15 insertions(+), 10 deletions(-)
t/t4013-diff-various.sh +2 -1
index 67f6411aff9..228ff100c61 100755
@@ -200,10 +200,12 @@ do
        esac
        test=$(echo "$label" | sed -e 's|[/ ][/ ]*|_|g')
        pfx=$(printf "%04d" $test_count)
-       expect="$TEST_DIRECTORY/t4013/diff.$test"
+       expect_relative="t4013/diff.$test"
+       expect="$TEST_DIRECTORY/$expect_relative"
        actual="$pfx-diff.$test"
 
Quoted 7 lines

t/t4013-diff-various.sh do · line 208

Quoted 2 lines
t/t4013-diff-various.sh +1 -0
        test_expect_$status "git $cmd # magic is ${magic:-(not used)}" '
+               test_when_finished "rm $actual" &&

Nit: before these two patches, "$actual" was only removed when the test succeeded. So, in case of failure, the failed output files would still be there for debugging. It might be interesting to keep this behavior and only remove "$actual" at the end of the test.

Quoted 5 lines

Either I'm missing something or you are, that's how test_when_finished works.

It's skipped under e.g. "--immediate --debug". See b586744a864 (test:
skip clean-up when running under --immediate mode, 2011-06-27)

I was mostly thinking about the artifacts zip we get from our CI when a test fails. I find the final trash dir quite useful for some post-mortem analysis, especially to debug WIP tests that only fail occasionally or test failures on OSes I don't have quick access to.

#106 lore raw

Re: [PATCH 2/2] diff tests: rewrite flakyness-causing test "aid"

Quoted 11 lines · Matheus Tavares Bernardino
Quoted 9 lines · Ævar Arnfjörð Bjarmason
Quoted 5 lines

Nit: before these two patches, "$actual" was only removed when the test succeeded. So, in case of failure, the failed output files would still be there for debugging. It might be interesting to keep this behavior and only remove "$actual" at the end of the test.

Either I'm missing something or you are, that's how test_when_finished works.

Quoted 7 lines
Quoted 2 lines

It's skipped under e.g. "--immediate --debug". See b586744a864 (test:
skip clean-up when running under --immediate mode, 2011-06-27)

I was mostly thinking about the artifacts zip we get from our CI when a test fails. I find the final trash dir quite useful for some post-mortem analysis, especially to debug WIP tests that only fail occasionally or test failures on OSes I don't have quick access to.

Yeah, without "--immediate" we do run the clean-up tasks registered with test_when_finished, so it may be a problem.

#107 lore raw

Re: [PATCH 2/2] diff tests: rewrite flakyness-causing test "aid"

In reply to Matheus Tavares Bernardino
Quoted 35 lines · Matheus Tavares Bernardino
Quoted 33 lines · Ævar Arnfjörð Bjarmason
Quoted 18 lines · Matheus Tavares Bernardino
Quoted 16 lines · Ævar Arnfjörð Bjarmason
 t/t4013-diff-various.sh | 25 +++++++++++++++----------
 1 file changed, 15 insertions(+), 10 deletions(-)
t/t4013-diff-various.sh +2 -1
index 67f6411aff9..228ff100c61 100755
@@ -200,10 +200,12 @@ do
        esac
        test=$(echo "$label" | sed -e 's|[/ ][/ ]*|_|g')
        pfx=$(printf "%04d" $test_count)
-       expect="$TEST_DIRECTORY/t4013/diff.$test"
+       expect_relative="t4013/diff.$test"
+       expect="$TEST_DIRECTORY/$expect_relative"
        actual="$pfx-diff.$test"
 
Quoted 7 lines

t/t4013-diff-various.sh do · line 208

Quoted 2 lines
t/t4013-diff-various.sh +1 -0
        test_expect_$status "git $cmd # magic is ${magic:-(not used)}" '
+               test_when_finished "rm $actual" &&

Nit: before these two patches, "$actual" was only removed when the test succeeded. So, in case of failure, the failed output files would still be there for debugging. It might be interesting to keep this behavior and only remove "$actual" at the end of the test.

Either I'm missing something or you are, that's how test_when_finished works.

Quoted 7 lines
Quoted 2 lines

It's skipped under e.g. "--immediate --debug". See b586744a864 (test:
skip clean-up when running under --immediate mode, 2011-06-27)

I was mostly thinking about the artifacts zip we get from our CI when a test fails. I find the final trash dir quite useful for some post-mortem analysis, especially to debug WIP tests that only fail occasionally or test failures on OSes I don't have quick access to.

Ah, yes that's a problem we should solve, but I think we should not put off migration to test_when_finished because of that.

The whole reason we use it is to clean up the work area for the next test.

Thus if we do:

    git something >expected &&
    test_cmp expected actual &&
    rm expected actual

And "git something" segfaults it's only dumb luck that subsequent tests don't fail in non-obvious ways due to the files being left behind.

If we could rely on this in general we could make everything under test_when_finished a noop, but I think you'll find that'll fail a lot of things in the test suite if we do that.

But the problem you cite is legitimate, but we should solve it with something like:

1. If we fail tests 58 and 67 out of 100 we should/could run these again at the end under some more exhaustive debugging mode where we'd save away all the intermediate files.

2. Or, and test_when_finished helps here, we could set $PATH when it runs to a directory with a custom "rm", which would save away the temporary files, expecting that we'll tar them up if any of the tests whose files we saved failed (we'd delete the other ones).

3. Similarly, wrap "rm" in some "cat and rm" for common cases like actual/expected files, and improve the verbose output to some "verbose if fails".

#108 lore raw

Re: [PATCH 2/2] diff tests: rewrite flakyness-causing test "aid"

Quoted 15 lines · Ævar Arnfjörð Bjarmason

On Tue, Apr 13 2021, Matheus Tavares Bernardino wrote:
...

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 4 lines
        actual="$pfx-diff.$test"

        test_expect_$status "git $cmd # magic is ${magic:-(not used)}" '

+ test_when_finished "rm $actual" &&

Nit: before these two patches, "$actual" was only removed when the test succeeded. So, in case of failure, the failed output files would still be there for debugging. It might be interesting to keep this behavior and only remove "$actual" at the end of the test.

...
Ah, yes that's a problem we should solve, but I think we should not put off migration to test_when_finished because of that.

Quoted 8 lines

The whole reason we use it is to clean up the work area for the next test.

Thus if we do:

    git something >expected &&
    test_cmp expected actual &&
    rm expected actual

Isn't it a poor example to use to argue for your particular change, where $actual in the original is designed to be unique among tests, in order to ensure that $actual files left after test pieces fail would not interfere with the tests that come later? IOW, there is not a reason to remove $actual until the end of the test sequence, is there?

#109 lore raw

Re: [PATCH 2/2] diff tests: rewrite flakyness-causing test "aid"

Quoted 26 lines · Junio C Hamano
Quoted 15 lines · Ævar Arnfjörð Bjarmason

On Tue, Apr 13 2021, Matheus Tavares Bernardino wrote:
...

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 4 lines
        actual="$pfx-diff.$test"

        test_expect_$status "git $cmd # magic is ${magic:-(not used)}" '

+ test_when_finished "rm $actual" &&

Nit: before these two patches, "$actual" was only removed when the test succeeded. So, in case of failure, the failed output files would still be there for debugging. It might be interesting to keep this behavior and only remove "$actual" at the end of the test.

...
Ah, yes that's a problem we should solve, but I think we should not put off migration to test_when_finished because of that.

Quoted 8 lines

The whole reason we use it is to clean up the work area for the next test.

Thus if we do:

    git something >expected &&
    test_cmp expected actual &&
    rm expected actual
Quoted 6 lines

Isn't it a poor example to use to argue for your particular change, where $actual in the original is designed to be unique among tests, in order to ensure that $actual files left after test pieces fail would not interfere with the tests that come later? IOW, there is not a reason to remove $actual until the end of the test sequence, is there?

Not really, because you needed to read the rest of the test file to come to that conclusion.

The point of using a helper that guarantees cleanup such as test_when_finished or test_config over manual "git config" or "git rm" isn't that we can prove that we need it because a later test needs the cleanup, but that anyone can add new tests or functionality without having to worry about cleaning up the existing trash directory.

So yes, it's not needed here, but that's only because we know the rest of the tests don't have e.g. a test that does a:

    for file in t4013/*

Anyway, that's only half of the example I cited for why we should use test_when_finished in general, the other half was noting that because we've semantically annotated cleanup tasks we can smartly handle those for debugging, e.g. stash the "removed" files away, and if the test fails present them to the user.

#110 lore raw

Re: [PATCH 2/2] diff tests: rewrite flakyness-causing test "aid"

Quoted 16 lines · Ævar Arnfjörð Bjarmason
Quoted 6 lines
Quoted 5 lines

Thus if we do:

    git something >expected &&
    test_cmp expected actual &&
    rm expected actual
Quoted 6 lines

Isn't it a poor example to use to argue for your particular change, where $actual in the original is designed to be unique among tests, in order to ensure that $actual files left after test pieces fail would not interfere with the tests that come later? IOW, there is not a reason to remove $actual until the end of the test sequence, is there?

Not really, because you needed to read the rest of the test file to come to that conclusion.

Quoted 8 lines

The point of using a helper that guarantees cleanup such as test_when_finished or test_config over manual "git config" or "git rm" isn't that we can prove that we need it because a later test needs the cleanup, but that anyone can add new tests or functionality without having to worry about cleaning up the existing trash directory.

So yes, it's not needed here, but that's only because we know the rest of the tests don't have e.g. a test that does a:

In this particular case, $actual files are designed to be left behind for failed test pieces, so that the tester can come back and inspect them. I probably should have said it a bit more strongly than "there is not a reason to remove". You SHOULD NOT remove and that is why we had "check and then remove only upon success" there, instead of test_when_finished. We want them left for and only for failing test pieces.

Please do not advocate for and encourage newbies who would be reading the discussion from sidelines to use test_when_finished out of dogmatic principle without thinking. Even though there are valid cases where test_when_finished is the perfect fit, in this particular case, use of it is a clear regression.

Thanks.

#111 lore raw

Re: [PATCH 2/2] diff tests: rewrite flakyness-causing test "aid"

On Wed, Apr 14 2021, Junio C Hamano wrote:

["tl;dr" below]:

Quoted 27 lines
Quoted 16 lines · Ævar Arnfjörð Bjarmason
Quoted 6 lines
Quoted 5 lines

Thus if we do:

    git something >expected &&
    test_cmp expected actual &&
    rm expected actual
Quoted 6 lines

Isn't it a poor example to use to argue for your particular change, where $actual in the original is designed to be unique among tests, in order to ensure that $actual files left after test pieces fail would not interfere with the tests that come later? IOW, there is not a reason to remove $actual until the end of the test sequence, is there?

Not really, because you needed to read the rest of the test file to come to that conclusion.

Quoted 8 lines

The point of using a helper that guarantees cleanup such as test_when_finished or test_config over manual "git config" or "git rm" isn't that we can prove that we need it because a later test needs the cleanup, but that anyone can add new tests or functionality without having to worry about cleaning up the existing trash directory.

So yes, it's not needed here, but that's only because we know the rest of the tests don't have e.g. a test that does a:

Quoted 7 lines

In this particular case, $actual files are designed to be left behind for failed test pieces, so that the tester can come back and inspect them. I probably should have said it a bit more strongly than "there is not a reason to remove". You SHOULD NOT remove and that is why we had "check and then remove only upon success" there, instead of test_when_finished. We want them left for and only for failing test pieces.

Yes, it's clear that it's designed to do that. I'm not disagreeing on the intent of your commit in 2006 to set it up this way.

What I am saying is that it's incompatible to have:

1. Failing tests
2. Not removing scratch files that would otherwise be removed

And:

3. Knowing that the rest of the tests pass in the case of #1 without reading them all.

Hence the suggestion that we should use test_when_finished without exception for such patterns.

Because consistently using the helper would allow us to smartly get #1 and #2 without #3, i.e. some "copy them for later analysis" as suggested upthread in:
http://lore.kernel.org/git/[email protected]

Quoted 3 lines

Please do not advocate for and encourage newbies who would be reading the discussion from sidelines to use test_when_finished out of dogmatic principle without thinking.

Is Matheus the newbie here? I think he's contributed enough to form his own opinion.

I think it's clear to anyone else reading from the sidelines that we're having some informed disagreement about a finer point of patterns in the test suite.

I doubt either one of us is likely to have much of an impact on newbies reading from the sidelines.

In any case, I don't see how you're able to read this thread and come to the conclusion that I'm proposing the use of test_when_finished out of "[some] dogmatic principle".

I'm not. I'm proposing to use it because I think it makes sense, and I think the reasons you've noted for avoiding it in this case have more downsides than upsites, as noted in the #1-3 examples above.

Quoted 2 lines

Even though there are valid cases where test_when_finished is the perfect fit, in this particular case, use of it is a clear regression.

Well, I disagree with you.

I think even if the patch I was proposing was a mere conversion to test_when_finished as would be typical for most newly written tests, i.e. a diff like this:

	diff --git a/t/t4013-diff-various.sh b/t/t4013-diff-various.sh
	index 6cca8b84a6b..d0031aa0f7b 100755
	--- a/t/t4013-diff-various.sh
	+++ b/t/t4013-diff-various.sh
	@@ -204,6 +204,7 @@ do
	 	actual="$pfx-diff.$test"
	 
	 	test_expect_$status "git $cmd # magic is ${magic:-(not used)}" '
	+		test_when_finished "rm \"$actual\"" &&
	 		{
	 			echo "$ git $cmd"
	 			case "$magic" in
	@@ -216,22 +217,11 @@ do
	 			    -e "s/^\\(.*mixed; boundary=\"-*\\)$V\\(-*\\)\"\$/\\1g-i-t--v-e-r-s-i-o-n\2\"/"
	 			echo "\$"
	 		} >"$actual" &&
	-		if test -f "$expect"
	-		then
	-			process_diffs "$actual" >actual &&
	-			process_diffs "$expect" >expect &&
	-			case $cmd in
	-			*format-patch* | *-stat*)
	-				test_cmp expect actual;;
	-			*)
	-				test_cmp expect actual;;
	-			esac &&
	-			rm -f "$actual" actual expect
	-		else
	-			# this is to help developing new tests.
	-			cp "$actual" "$expect"
	-			false
	-		fi
	+		test_when_finished "rm actual" &&
	+		process_diffs "$actual" >actual &&
	+		test_when_finished "rm expect" &&
	+		process_diffs "$expect" >expect &&
	+		test_cmp expect actual
	 	'
	 done <<\EOF
	 diff-tree initial

It's a clear improvement, because, on current master, emulating the case when you add a new test-case:

    $ rm t4013/diff.log_-GF_-p_--pickaxe-all_master

Now run it once:

    $ ./t4013-diff-various.sh
    [...]
    # failed 1 among 199 test(s)

And then to debug it:

    $ ./t4013-diff-various.sh -vixd
    [...]
    # passed all 199 test(s)

So because we copied the file around the only get the failure the first time around.

You might rightly say that we could have narrowly just fixed that particular bug and kept the old "copy and return false", so something like:

	diff --git a/t/t4013-diff-various.sh b/t/t4013-diff-various.sh
	index 6cca8b84a6b..6f0f3c7f53c 100755
	--- a/t/t4013-diff-various.sh
	+++ b/t/t4013-diff-various.sh
	@@ -229,7 +229,7 @@ do
	 			rm -f "$actual" actual expect
	 		else
	 			# this is to help developing new tests.
	-			cp "$actual" "$expect"
	+			cp "$actual" "$expect.MAYBE-USE-THIS"
	 			false
	 		fi
	 	'

But as noted above that gets us into needing to worry about cascading failure. I.e. any new test added later in the file needs to think about "what if the t4013 directory is in this state?".

We also document what we'll do and how we'll do it in t/README in this case:

--immediate::
This causes the test to immediately exit upon the first failed test. Cleanup commands requested with test_when_finished are not executed if the test failed, in order to keep the state for inspection by the tester to diagnose the bug.

So it's much more obvious and consistent to have a failure that ends on the test_cmp and being able to see the test_when_finished for the intermediate files in the trace output.

The hypothetical newbie might have debugged such a pattern already, but not having it end in a special-case used only for this test:

[...]
+ test -f /home/avar/g/git/t/t4013/diff.log_-GF_-p_--pickaxe-all_master
+ cp 0096-diff.log_-GF_-p_--pickaxe-all_master /home/avar/g/git/t/t4013/diff.log_-GF_-p_--pickaxe-all_master.MAYBE-USE-THIS
+ false

In any case, neither one of those is the patch I'm suggesting upthread.

I am suggesting not just to use test_when_finished, but that we should use BUG here. Since not having the file itself is a bug in git's test suite, not a mere failure of the test case. The test will never pass if the file doesn't exist.

So even if I agreed with you that the more narrow migration to test_when_finished would be a regression, I don't see how you think *this* patch could be a regression.

You seem to just be focusing on the test_when_finished part of it, but that's not all it's doing. Since we also use BUG the output is now:

    $ ./t4013-diff-various.sh 
    [...]
    error: bug in the test script: Have no "t4013/diff.log_-GF_-p_--pickaxe-all_master", new test? [...]

I.e. whether we used "rm" or "cp" or "test_when_finished 'rm [...]'" earlier has become irrelevant to achieve the states aims in 3c2f75b590c (t4013: add tests for diff/log family output options., 2006-06-26).

It doesn't matter how we're removing/copying etc. the file anymore, that's no longer how we're helping the user debug the test.

tl;dr:

In any case, I think it's best to just drop this series. I wrote the above more as a synopsis of what I belive we should be doing in general, I do have some patches queued up to address the general debug-ability of our test output (e.g. for CI).

For that end-goal having e.g. test_when_finished (mostly) consistently used would help a lot, but not using it in any one test would be fine. This is just one such case, and something I thought wouldn't be a controversial patch.

I think at this point me trying to gleaming entrails of what you want & you responding back is going to take both of us much more time than replacing this series with a patch you'd be happy with, and this wasn't something I cared about more than a one-off.

#112 lore raw

Re: [PATCH 2/2] diff tests: rewrite flakyness-causing test "aid"

Quoted 5 lines · Ævar Arnfjörð Bjarmason

What I am saying is that it's incompatible to have:

1. Failing tests
2. Not removing scratch files that would otherwise be removed

Quoted 7 lines

And:

3. Knowing that the rest of the tests pass in the case of #1 without reading them all.

Hence the suggestion that we should use test_when_finished without exception for such patterns.

I disagree with the above; t4013's "read magic cmd" part is designed to be independent from each other; I do not think you need to read all of the parts enclosed in << ... EOF to understand that.

In short,

* "test_when_finished 'rm ...'" is a good tool to ensure something gets removed no matter what else happens in the same test. Since it does not run the clean-up under "-i", it can even be used on files that would be useful in diagnosing failures. But under "-d", it does clean-up to avoid affecting the following test.

* $actual was made unique so that even under "-d", a failing test would not negatively affect the subsequent ones. Removing it for failure cases is actively wrong, so use of test_when_finished, which may be an expedient way to remove artifact that would negatively affect later test pieces, does not apply --- existing code is doing better than test_when_finished can offer, and replacing it with test_when_finished is a regression.

* And the most important part: the unnecessary removal of $actual was not even part of the "flakyness-causing" bug you started this topic to fix anyway. We should just remove the regression caused by unnecessary use of test_when_finished and focus on the "don't place the actual output from a brand new test under the file used for the expectation the next time---instead place it under temporary file and call for attention" part, which was the real improvement.

Quoted 6 lines
Quoted 3 lines

Please do not advocate for and encourage newbies who would be reading the discussion from sidelines to use test_when_finished out of dogmatic principle without thinking.

Is Matheus the newbie here? I think he's contributed enough to form his own opinion.

No, those "reading the discussion from sidelines" are not on To: or Cc:, but are eager to learn by reading what is available to git@ subscribers (including the lore archive). I do not want those of us whose name appear often in the list archive to be sending a wrong signal to them.

Quoted 1 line

In any case, I think it's best to just drop this series.

That is a sad and wrong conclusion. We should just realize what we really wanted to fix in the first place and keep the improvement; otherwise all the review time was wasted.

And next time, I'd suggest you not to spend too many bytes on "while at it". Clear and obvious clean-up while the tree and the project is otherwise quiescent is very much no-brainer welcome, but otherwise...

Thanks.

#113 lore raw

Re: [PATCH 2/2] diff tests: rewrite flakyness-causing test "aid"

Quoted 21 lines · Junio C Hamano
Quoted 5 lines · Ævar Arnfjörð Bjarmason

What I am saying is that it's incompatible to have:

1. Failing tests
2. Not removing scratch files that would otherwise be removed

Quoted 7 lines

And:

3. Knowing that the rest of the tests pass in the case of #1 without reading them all.

Hence the suggestion that we should use test_when_finished without exception for such patterns.

I disagree with the above; t4013's "read magic cmd" part is designed to be independent from each other; I do not think you need to read all of the parts enclosed in << ... EOF to understand that.

In short,

Quoted 8 lines

* "test_when_finished 'rm ...'" is a good tool to ensure something gets removed no matter what else happens in the same test. Since it does not run the clean-up under "-i", it can even be used on files that would be useful in diagnosing failures. But under "-d", it does clean-up to avoid affecting the following test.

* $actual was made unique so that even under "-d", a failing test would not negatively affect the subsequent ones. [...]

I don't mind disagreeing with you, but I do feel like we're just talking past each other, either that or we've got a different understanding of "negatively affect the subsequent ones".

I'm saying that yes I agree that *right now* it doesn't negatively affect any of the subsequent ones, but that you can only know that if you need all the tests in the file past the one you're modifying.

I.e. imagine if we added, obviously contrived, but a stand-in for "we expect consistent state":

	diff --git a/t/t4013-diff-various.sh b/t/t4013-diff-various.sh
	index 6cca8b84a6b..a326968e8f3 100755
	--- a/t/t4013-diff-various.sh
	+++ b/t/t4013-diff-various.sh
	@@ -526,4 +526,9 @@ test_expect_success 'diff -I<regex>: detect malformed regex' '
	 	test_i18ngrep "invalid regex given to -I: " error
	 '
	 
	+test_expect_success 'stuff' '
	+	find * -type f >actual &&
	+	test_line_count = 9 actual
	+'
	+
	 test_done

Now run it, and you'll pass them all, now break one of the earlier tests with:

Quoted 1 line

t4013/diff.log_-GF_-p_--pickaxe-all_master

You'll fail two tests (the expected one, and the new one), instead of just one, which is purely an artifact of the earlier test's cleanup / state being different as a result of whether it passes or not.

With my patch you'll fail just the one earlier test, because we do the same cleanup no matter whether we failed midway through it or not.

So yes, while this small series amounts to re-arranging some deck chairs on the Titanic I do think not being able to rely on consistent state is a big anti-pattern in the test suite in general.

It's a big part of the reason for why you can't reliably use --run=N or GIT_SKIP_TESTS (especially in cases of partial failures) to selectively skip things without what amounts to bespoke analysis of each test you'd like to run a subset of.

Now, of course test_when_finished won't get us there by itself, we'd also need consistently use some "test_expect_setup" to clearly delineate "setup" and other "this must run" state changes from "just consumes existing state and resets". But consistently writing new/modified tests as "just consumes existing state and resets" will slowly get us there.

Quoted 6 lines

[...] Removing it for
failure cases is actively wrong, so use of test_when_finished, which may be an expedient way to remove artifact that would negatively affect later test pieces, does not apply --- existing code is doing better than test_when_finished can offer, and replacing it with test_when_finished is a regression.

Aside from whether we agree or disagree on whether test_when_finished should be consistently used to clean up state I genuinely still don't see how the sum total of replacing the current "rm && cp" with "test_when_finished && BUG" leaves us worse off.

Are you thinking of the case where someone wants to add N new lines to the "while read magic cmd" loop?

That's the only case I can think of that would arguably be worse, but I think it's far outweighed by the more obvious failure mode, and a "git blame" on the lines it consumes shows that additions are very occasional (and not in large batches).

Other than that, under e.g. -vixd having the test suite stop on the specific line that failed (the test_cmp, not a "false" after some "cp") is how most other tests works, and I think makes things much more readable and understandable at a glance.

Quoted 8 lines

* And the most important part: the unnecessary removal of $actual was not even part of the "flakyness-causing" bug you started this topic to fix anyway. We should just remove the regression caused by unnecessary use of test_when_finished and focus on the "don't place the actual output from a brand new test under the file used for the expectation the next time---instead place it under temporary file and call for attention" part, which was the real improvement.

I see your point and agree in theory that the "we write into the t/t4013/ directory" and "we inconsistently clear up the trash directory on OK/NOK" are in principle different issues.

But in this case we always removed or copied "$actual" away from its original name before, we do the same now (just more reliably, and in a way that doesn't cause flakyness in the "cp" case).

So I don't really see how to split up those two arguably different changes in a way that would make sense.

The only case where "$actual" was left before was (I think) a clear of leaking state unnecessarily outside of --immediate.

Quoted 4 lines
Quoted 4 lines
Quoted 3 lines

Please do not advocate for and encourage newbies who would be reading the discussion from sidelines to use test_when_finished out of dogmatic principle without thinking.

Quoted 8 lines
Quoted 2 lines

Is Matheus the newbie here? I think he's contributed enough to form his own opinion.

No, those "reading the discussion from sidelines" are not on To: or Cc:, but are eager to learn by reading what is available to git@ subscribers (including the lore archive). I do not want those of us whose name appear often in the list archive to be sending a wrong signal to them.

I'll endeavor to make it clear that my opinion is just my opinion etc. in the future, I thought it was clear from the context :)

Quoted 5 lines
Quoted 1 line

In any case, I think it's best to just drop this series.

That is a sad and wrong conclusion. We should just realize what we really wanted to fix in the first place and keep the improvement; otherwise all the review time was wasted.

To be clear that's not some "I can't have the change I want so I'm taking my toys and going home" tantrum. I don't mind making changes in response to reviews etc.

As noted above I'm genuinely still unable to see how I'd selectively untangle the change I made in a way that both wouldn't either fix bugs in the test, or introduce new bugs while I was at it, and that I'd be comfortable putting my name behind.

Hence the suggestion that since we're already we past the point of just having come up with two independent approaches to this in terms of time investment, that doing so would be a better thing overall.

I do think (and would be willing to work on) that t/README should have some summary of best practices for new/updated tests that we should be striving for. Looking at our "test_when_finished" there's at least tow independent bugs[1] in it, so ther's some room for improvement.

1. It creates a ref with echo, but removes it with update-ref -d, which e.g. couldn't delete a corrupt ref (if rev-parse is buggy); and more importantly if rev-parse dies midway through we're left with a corrupt repo, the "test_when_finished ... update-ref -d" should come *first*.

#114 lore raw

Re: [PATCH 2/2] diff tests: rewrite flakyness-causing test "aid"

Quoted 2 lines · Ævar Arnfjörð Bjarmason

Are you thinking of the case where someone wants to add N new lines to the "while read magic cmd" loop?

No. I think stopping with BUG is fine. Adding a new test is a serious event and I'd prefer to make it stop to give it undistracted attention by the developer, instead of letting many to be handled at once.

Of course, we _could_ instead remember the fact that we deposted newly proposed expectation pattern that the user must audit, still keep going and list the new files at the end, but I do not think it is necessary. Your newly added BUG is perfectly fine.

Quoted 6 lines
Quoted 5 lines
Quoted 1 line

In any case, I think it's best to just drop this series.

That is a sad and wrong conclusion. We should just realize what we really wanted to fix in the first place and keep the improvement; otherwise all the review time was wasted.

Quoted 8 lines

To be clear that's not some "I can't have the change I want so I'm taking my toys and going home" tantrum. I don't mind making changes in response to reviews etc.

As noted above I'm genuinely still unable to see how I'd selectively untangle the change I made in a way that both wouldn't either fix bugs in the test, or introduce new bugs while I was at it, and that I'd be comfortable putting my name behind.

I fail to see why the "rewrite flakyness-causing test aid" that you set out to perform should need more than a two-liner change. There is no reason to touch '&& rm "$actual" actual expect' at the end. It is the success case where the flakyness-causing "aid" does not even kick in.

 t/t4013-diff-various.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
t/t4013-diff-various.sh +2 -2
index 6cca8b84a6..da6da13920 100755
@@ -229,8 +229,8 @@ do
 			rm -f "$actual" actual expect
 		else
 			# this is to help developing new tests.
-			cp "$actual" "$expect"
-			false
+			mv "$actual" "$expect.new"
+			BUG "No $expect; check $expect.new and use it perhaps?"
 		fi
 	'
 done <<\EOF

n

#115 lore raw

Re: [PATCH 2/2] diff tests: rewrite flakyness-causing test "aid"

In reply to Matheus Tavares Bernardino
Quoted 25 lines · Matheus Tavares Bernardino

On Tue, Apr 13, 2021 at 4:01 PM Ævar Arnfjörð Bjarmason

Quoted 24 lines
Quoted 13 lines
Quoted 13 lines
t/t4013-diff-various.sh +2 -1
index 67f6411aff9..228ff100c61 100755
@@ -200,10 +200,12 @@ do
        esac
        test=$(echo "$label" | sed -e 's|[/ ][/ ]*|_|g')
        pfx=$(printf "%04d" $test_count)
-       expect="$TEST_DIRECTORY/t4013/diff.$test"
+       expect_relative="t4013/diff.$test"
+       expect="$TEST_DIRECTORY/$expect_relative"
        actual="$pfx-diff.$test"
 
Quoted 7 lines

t/t4013-diff-various.sh do · line 208

Quoted 2 lines
t/t4013-diff-various.sh +1 -0
        test_expect_$status "git $cmd # magic is ${magic:-(not used)}" '
+               test_when_finished "rm $actual" &&

Nit: before these two patches, "$actual" was only removed when the test succeeded. So, in case of failure, the failed output files would still be there for debugging. It might be interesting to keep this behavior and only remove "$actual" at the end of the test.

Either I'm missing something or you are, that's how test_when_finished works.

Quoted 7 lines
Quoted 2 lines

It's skipped under e.g. "--immediate --debug". See b586744a864 (test:
skip clean-up when running under --immediate mode, 2011-06-27)

I was mostly thinking about the artifacts zip we get from our CI when a test fails. I find the final trash dir quite useful for some post-mortem analysis, especially to debug WIP tests that only fail occasionally or test failures on OSes I don't have quick access to.

On Travis CI we run tests with '--immediate' for exactly this reason; I don't know why it's done differently on other CI systems, and, unfortunately, 'git log --grep=immediate ci/' didn't turn up any insights.

#116 lore raw

Re: [PATCH 0/2] diff tests: un-flaky and post-gettext-poison cleanup

In reply to Ævar Arnfjörð Bjarmason
Quoted 6 lines · Ævar Arnfjörð Bjarmason

In reply to [1] because they're related post-GETTEXT_POISON cleanup, but both apply on top of master.

I had a flaky t4013 test with some local WIP code that I hadn't investigated, as it turns out re-running the test was hiding the failure. 2/2 fixes that, 1/2 is a minor post-GETTEXT_POISON cleanup.

I feel somewhat offended to see t4013's "auto preparation of the expected output" feature (which of course I invented) as "flakey" ;-) but I agree that was a misfeature. Thanks for taking a look.

#117 lore raw

PATCH v2 3/3 tests: remove uses of GIT_TEST_GETTEXT_POISON=false

In reply to Ævar Arnfjörð Bjarmason

As noted in previous commits we are removing the use of GIT_TEST_GETTEXT_POISON=false. These tests all relied on the facility being off, it always is off after an earlier change, but we hadn't removed the redundant assignments to "false" in the tests.

I'm preserving the deletion of "error" lines in 38b9197a76a (t5411:
add basic test cases for proc-receive hook, 2020-08-27), it turns out that's useful even without GIT_TEST_GETTEXT_POISON=true in play. Update a comment added in that commit to note that.

---
 t/t1305-config-include.sh   | 4 +---
 t/t5411/common-functions.sh | 5 ++---
 t/t7201-co.sh               | 2 +-
 t/t9902-completion.sh       | 1 -
 4 files changed, 4 insertions(+), 8 deletions(-)
t/t1305-config-include.sh +1 -3
index f1e1b289f9..f08cfcdfae 100755
@@ -352,9 +352,7 @@ test_expect_success 'include cycles are detected' '
 	git init --bare cycle &&
 	git -C cycle config include.path cycle &&
 	git config -f cycle/cycle include.path config &&
-	test_must_fail \
-		env GIT_TEST_GETTEXT_POISON=false \
-		git -C cycle config --get-all test.value 2>stderr &&
+	test_must_fail git -C cycle config --get-all test.value 2>stderr &&
 	grep "exceeded maximum include depth" stderr
 '
 
t/t5411/common-functions.sh +2 -3
index 344d13f61a..13107fcdb6 100644
@@ -36,9 +36,8 @@ create_commits_in () {
 # without having to worry about future changes of the commit ID and spaces
 # of the output.  Single quotes are replaced with double quotes, because
 # it is boring to prepare unquoted single quotes in expect text.  We also
-# remove some locale error messages, which break test if we turn on
-# `GIT_TEST_GETTEXT_POISON=true` in order to test unintentional translations
-# on plumbing commands.
+# remove some locale error messages. The emitted human-readable errors are
+# redundant to the more machine-readable output the tests already assert.
 make_user_friendly_and_stable_output () {
 	sed \
 		-e "s/  *\$//" \
t/t7201-co.sh +1 -1
index b36a93056f..d10076efd7 100755
@@ -245,7 +245,7 @@ test_expect_success 'checkout to detach HEAD' '
 	rev=$(git rev-parse --short renamer^) &&
 	git checkout -f renamer &&
 	git clean -f &&
-	GIT_TEST_GETTEXT_POISON=false git checkout renamer^ 2>messages &&
+	git checkout renamer^ 2>messages &&
 	grep "HEAD is now at $rev" messages &&
 	test_line_count -gt 1 messages &&
 	H=$(git rev-parse --verify HEAD) &&
t/t9902-completion.sh +0 -1
index c4a7758409..ea669cfaf5 100755
@@ -2363,7 +2363,6 @@ test_expect_success 'sourcing the completion script clears cached commands' '
 '
 
 test_expect_success 'sourcing the completion script clears cached merge strategies' '
-	GIT_TEST_GETTEXT_POISON=false &&
 	__git_compute_merge_strategies &&
 	verbose test -n "$__git_merge_strategies" &&
 	. "$GIT_BUILD_DIR/contrib/completion/git-completion.bash" &&
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#118 lore raw

PATCH 1/6 ci: remove GETTEXT_POISON jobs

In reply to Ævar Arnfjörð Bjarmason

A subsequent commit will remove GETTEXT_POISON entirely, let's start by removing the CI jobs that enable the option.

We cannot just remove the job because the CI is implicitly depending on the "poison" job being a sort of "default" job. Let's instead add a "default" job.

This means we can remove the initial "make test" from the "linux-gcc" job (it does another one after setting a bunch of GIT_TEST_* variables).

I'm not doing that because it would conflict with the in-flight 334afbc76fb (tests: mark tests relying on the current default for init.defaultBranch, 2020-11-18) (currently on the "seen" branch, so the SHA-1 will almost definitely change). It's going to use that "make test" again for different reasons, so let's preserve it for now.

---
 .github/workflows/main.yml | 2 +-
 .travis.yml                | 2 +-
 ci/install-dependencies.sh | 2 +-
 ci/lib.sh                  | 3 +--
 4 files changed, 4 insertions(+), 5 deletions(-)
.github/workflows/main.yml +1 -1
index aef6643648..8b52df200f 100644
@@ -286,7 +286,7 @@ jobs:
           - jobname: osx-gcc
             cc: gcc
             pool: macos-latest
-          - jobname: GETTEXT_POISON
+          - jobname: linux-gcc-default
             cc: gcc
             pool: ubuntu-latest
     env:
.travis.yml +1 -1
index 05f3e3f8d7..908330a0a3 100644
@@ -16,7 +16,7 @@ compiler:
 
 matrix:
   include:
-    - env: jobname=GETTEXT_POISON
+    - env: jobname=linux-gcc-default
       os: linux
       compiler:
       addons:
ci/install-dependencies.sh +1 -1
index 0229a77f7d..79c0633a18 100755
@@ -72,7 +72,7 @@ Documentation)
 	test -n "$ALREADY_HAVE_ASCIIDOCTOR" ||
 	sudo gem install --version 1.5.8 asciidoctor
 	;;
-linux-gcc-4.8|GETTEXT_POISON)
+linux-gcc-default|linux-gcc-4.8)
 	sudo apt-get -q update
 	sudo apt-get -q -y install $UBUNTU_COMMON_PKGS
 	;;
ci/lib.sh +1 -2
index 38c0eac351..d848c036c5 100755
@@ -220,8 +220,7 @@ osx-clang|osx-gcc)
 	# Travis CI OS X
 	export GIT_SKIP_TESTS="t9810 t9816"
 	;;
-GETTEXT_POISON)
-	export GIT_TEST_GETTEXT_POISON=true
+linux-gcc-default)
 	;;
 Linux32)
 	CC=gcc
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#119 lore raw

Re: [PATCH 1/6] ci: remove GETTEXT_POISON jobs

Quoted 5 lines · Ævar Arnfjörð Bjarmason

A subsequent commit will remove GETTEXT_POISON entirely, let's start by removing the CI jobs that enable the option.

We cannot just remove the job because the CI is implicitly depending on the "poison" job being a sort of "default" job.

I don't understand what you mean here with a "default job" that the CI is implicitly depending on. There is certainly no such default job on Travis CI, and I don't think there is one on the GitHub thing.

Quoted 77 lines

Let's instead add a
"default" job.

This means we can remove the initial "make test" from the "linux-gcc" job (it does another one after setting a bunch of GIT_TEST_* variables).

I'm not doing that because it would conflict with the in-flight 334afbc76fb (tests: mark tests relying on the current default for init.defaultBranch, 2020-11-18) (currently on the "seen" branch, so the SHA-1 will almost definitely change). It's going to use that "make test" again for different reasons, so let's preserve it for now.

---
 .github/workflows/main.yml | 2 +-
 .travis.yml                | 2 +-
 ci/install-dependencies.sh | 2 +-
 ci/lib.sh                  | 3 +--
 4 files changed, 4 insertions(+), 5 deletions(-)
.github/workflows/main.yml +1 -1
index aef6643648..8b52df200f 100644
@@ -286,7 +286,7 @@ jobs:
           - jobname: osx-gcc
             cc: gcc
             pool: macos-latest
-          - jobname: GETTEXT_POISON
+          - jobname: linux-gcc-default
             cc: gcc
             pool: ubuntu-latest
     env:
.travis.yml +1 -1
index 05f3e3f8d7..908330a0a3 100644
@@ -16,7 +16,7 @@ compiler:
 
 matrix:
   include:
-    - env: jobname=GETTEXT_POISON
+    - env: jobname=linux-gcc-default
       os: linux
       compiler:
       addons:
ci/install-dependencies.sh +1 -1
index 0229a77f7d..79c0633a18 100755
@@ -72,7 +72,7 @@ Documentation)
 	test -n "$ALREADY_HAVE_ASCIIDOCTOR" ||
 	sudo gem install --version 1.5.8 asciidoctor
 	;;
-linux-gcc-4.8|GETTEXT_POISON)
+linux-gcc-default|linux-gcc-4.8)
 	sudo apt-get -q update
 	sudo apt-get -q -y install $UBUNTU_COMMON_PKGS
 	;;
ci/lib.sh +1 -2
index 38c0eac351..d848c036c5 100755
@@ -220,8 +220,7 @@ osx-clang|osx-gcc)
 	# Travis CI OS X
 	export GIT_SKIP_TESTS="t9810 t9816"
 	;;
-GETTEXT_POISON)
-	export GIT_TEST_GETTEXT_POISON=true
+linux-gcc-default)
 	;;
 Linux32)
 	CC=gcc
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#120 lore raw

Re: [PATCH 1/6] ci: remove GETTEXT_POISON jobs

Quoted 10 lines · SZEDER Gábor
Quoted 5 lines · Ævar Arnfjörð Bjarmason

A subsequent commit will remove GETTEXT_POISON entirely, let's start by removing the CI jobs that enable the option.

We cannot just remove the job because the CI is implicitly depending on the "poison" job being a sort of "default" job.

I don't understand what you mean here with a "default job" that the CI is implicitly depending on. There is certainly no such default job on Travis CI, and I don't think there is one on the GitHub thing.

I'll reword this. I meant that the poison job was using the default compiler etc., whereas the other ones were setting custom values.

I vaguely remember some list traffic about this in the past, i.e. the reliance on the poison job not just for that, but also as "the default OS image" setup.

#121 lore raw

Re: [PATCH 1/6] ci: remove GETTEXT_POISON jobs

Quoted 14 lines · Ævar Arnfjörð Bjarmason
Quoted 10 lines · SZEDER Gábor
Quoted 5 lines · Ævar Arnfjörð Bjarmason

A subsequent commit will remove GETTEXT_POISON entirely, let's start by removing the CI jobs that enable the option.

We cannot just remove the job because the CI is implicitly depending on the "poison" job being a sort of "default" job.

I don't understand what you mean here with a "default job" that the CI is implicitly depending on. There is certainly no such default job on Travis CI, and I don't think there is one on the GitHub thing.

Quoted 6 lines

I'll reword this. I meant that the poison job was using the default compiler etc., whereas the other ones were setting custom values.

I vaguely remember some list traffic about this in the past, i.e. the reliance on the poison job not just for that, but also as "the default OS image" setup.

Oh, I didn't mean that the commit message should be clarified. I meant that the GETTEXT_POISON job can simply be deleted. Sorry for not being clear enough.

#122 lore raw

Re: [PATCH 1/6] ci: remove GETTEXT_POISON jobs

Quoted 18 lines · SZEDER Gábor
Quoted 17 lines · Ævar Arnfjörð Bjarmason
Quoted 10 lines · SZEDER Gábor
Quoted 5 lines · Ævar Arnfjörð Bjarmason

A subsequent commit will remove GETTEXT_POISON entirely, let's start by removing the CI jobs that enable the option.

We cannot just remove the job because the CI is implicitly depending on the "poison" job being a sort of "default" job.

I don't understand what you mean here with a "default job" that the CI is implicitly depending on. There is certainly no such default job on Travis CI, and I don't think there is one on the GitHub thing.

I'll reword this. I meant that the poison job was using the default compiler etc., whereas the other ones were setting custom values.

Quoted 7 lines
Quoted 3 lines

I vaguely remember some list traffic about this in the past, i.e. the reliance on the poison job not just for that, but also as "the default OS image" setup.

Oh, I didn't mean that the commit message should be clarified. I meant that the GETTEXT_POISON job can simply be deleted. Sorry for not being clear enough.

I still don't get it, and my patch still seems fine as it is to me. But I'm probably missing something.

I'm looking at these, both pointing at e6362826a04 (The fourth batch, 2021-01-25):

# Currently says "Please be aware travis-ci.org will be shutting # down in several weeks"
https://www.travis-ci.org/github/git/git/builds/756176098

https://github.com/git/git/actions/runs/510676142

For the GitHub one, there's only two things that are linux && gcc. The linux-gcc job, and GETTEXT_POISON (renamed to linux-gcc-default in this series).

In ci/install-dependencies.sh we install GCC 8 for linux-gcc, but also perforce, apache, git-lfs etc.

Then in ci/lib.sh we set CC=gcc-8 on linux-gcc, but retain the default on GETTEXT_POISON. We also set GIT_TEST_HTTPD=true.

So hence "linux-gcc-default", we're running as close to a default set of packages we can get and build/run git at all. I.e. just "apt install $UBUNTU_COMMON_PKGS".

Then in ci/run-build-and-tests.sh the "linux-gcc" job is also our "let's turn on every GIT_TEST_* option you've heard of" job. Whereas on the GETTEXT_POISON job we just ran "make test", well, with GIT_TEST_GETTEXT_POISON=true before this series, but that was the only non-default, now we've got no non-defaults.

I can see why we'd want this on top:

    diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh
    index 50e0b90073f..3071099eaca 100755
    --- a/ci/run-build-and-tests.sh
    +++ b/ci/run-build-and-tests.sh
    @@ -32,11 +32,6 @@ linux-clang)
            export GIT_TEST_DEFAULT_HASH=sha256
            make test
            ;;
    -linux-gcc-4.8)
    -       # Don't run the tests; we only care about whether Git can be
    -       # built with GCC 4.8, as it errors out on some undesired (C99)
    -       # constructs that newer compilers seem to quietly accept.
    -       ;;
     *)
            make test
            ;;

Because there we're skipping tests on linux-gcc-4.8, but on travis it's the only Ubuntu Trusty image (default is Xenial), so yes we build with gcc 4.8, but we're also missing out in seeing if our tests are working on a slightly older Ubuntu.

Also on GitHub's page the GETTEXT_POISON job finishes in ~10m but all the other ones in ~30m, I see some run "make test" twice, as an aside is there some limit on jobs but not overall CPU? Otherwise those would finish faster if that was split up.

But "make test" twice doesn't account for a ~3x increase, so it's got to be the extra tests we're running, i.e. svn tests, httpd etc.

It seems to me to provide value in itself to run some shorter version of the CI for people using it as a poor man's edit/push/ci/edit loop.

#123 lore raw

Re: [PATCH 1/6] ci: remove GETTEXT_POISON jobs

Quoted 29 lines · Ævar Arnfjörð Bjarmason
Quoted 18 lines · SZEDER Gábor
Quoted 17 lines · Ævar Arnfjörð Bjarmason
Quoted 10 lines · SZEDER Gábor
Quoted 5 lines · Ævar Arnfjörð Bjarmason

A subsequent commit will remove GETTEXT_POISON entirely, let's start by removing the CI jobs that enable the option.

We cannot just remove the job because the CI is implicitly depending on the "poison" job being a sort of "default" job.

I don't understand what you mean here with a "default job" that the CI is implicitly depending on. There is certainly no such default job on Travis CI, and I don't think there is one on the GitHub thing.

I'll reword this. I meant that the poison job was using the default compiler etc., whereas the other ones were setting custom values.

Quoted 7 lines
Quoted 3 lines

I vaguely remember some list traffic about this in the past, i.e. the reliance on the poison job not just for that, but also as "the default OS image" setup.

Oh, I didn't mean that the commit message should be clarified. I meant that the GETTEXT_POISON job can simply be deleted. Sorry for not being clear enough.

Quoted 8 lines

I still don't get it, and my patch still seems fine as it is to me. But I'm probably missing something.

I'm looking at these, both pointing at e6362826a04 (The fourth batch, 2021-01-25):

# Currently says "Please be aware travis-ci.org will be shutting # down in several weeks"

... and continues with:

"with all accounts migrating to travis-ci.com. Please stay tuned
here for more information."

FWIW, I recently migrated manually, and apart from the different TLD haven't noticed anything different (though somehow the jobs in my .org builds sometimes took a while to start lately, but on .com they all started real fast so far, as they should).

Quoted 11 lines
    https://www.travis-ci.org/github/git/git/builds/756176098

    https://github.com/git/git/actions/runs/510676142

For the GitHub one, there's only two things that are linux && gcc. The linux-gcc job, and GETTEXT_POISON (renamed to linux-gcc-default in this series).

In ci/install-dependencies.sh we install GCC 8 for linux-gcc, but also perforce, apache, git-lfs etc.

Quoted 6 lines

Then in ci/lib.sh we set CC=gcc-8 on linux-gcc, but retain the default on GETTEXT_POISON. We also set GIT_TEST_HTTPD=true.

So hence "linux-gcc-default", we're running as close to a default set of packages we can get and build/run git at all. I.e. just "apt install $UBUNTU_COMMON_PKGS".

We started installing newer compilers, because we wanted to benefit from their better error reporting and improved warnings. The reason we did that in existing CI jobs without adding a "default" job was that we don't care enough about the default compiler and configuration to justify its additional runtime, because we run the test suite enough times already.

Considering that the number of CI jobs has grown since then, I would rather see the build complete faster than a default job running the test suite for the N+1th time.

Quoted 25 lines

Then in ci/run-build-and-tests.sh the "linux-gcc" job is also our "let's turn on every GIT_TEST_* option you've heard of" job. Whereas on the GETTEXT_POISON job we just ran "make test", well, with GIT_TEST_GETTEXT_POISON=true before this series, but that was the only non-default, now we've got no non-defaults.

I can see why we'd want this on top:

    diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh
    index 50e0b90073f..3071099eaca 100755
    --- a/ci/run-build-and-tests.sh
    +++ b/ci/run-build-and-tests.sh
    @@ -32,11 +32,6 @@ linux-clang)
            export GIT_TEST_DEFAULT_HASH=sha256
            make test
            ;;
    -linux-gcc-4.8)
    -       # Don't run the tests; we only care about whether Git can be
    -       # built with GCC 4.8, as it errors out on some undesired (C99)
    -       # constructs that newer compilers seem to quietly accept.
    -       ;;
     *)
            make test
            ;;
Quoted 4 lines

Because there we're skipping tests on linux-gcc-4.8, but on travis it's the only Ubuntu Trusty image (default is Xenial), so yes we build with gcc 4.8, but we're also missing out in seeing if our tests are working on a slightly older Ubuntu.

The above comment you propose to remove explains why you should not remove it... And, again, running the test suite doesn't seem to worth the extra runtime.

Quoted 10 lines

Also on GitHub's page the GETTEXT_POISON job finishes in ~10m but all the other ones in ~30m, I see some run "make test" twice, as an aside is there some limit on jobs but not overall CPU? Otherwise those would finish faster if that was split up.

But "make test" twice doesn't account for a ~3x increase, so it's got to be the extra tests we're running, i.e. svn tests, httpd etc.

It seems to me to provide value in itself to run some shorter version of the CI for people using it as a poor man's edit/push/ci/edit loop.

#124 lore raw

Re: [PATCH 1/6] ci: remove GETTEXT_POISON jobs

Quoted 67 lines · SZEDER Gábor
Quoted 29 lines · Ævar Arnfjörð Bjarmason
Quoted 18 lines · SZEDER Gábor
Quoted 17 lines · Ævar Arnfjörð Bjarmason
Quoted 10 lines · SZEDER Gábor
Quoted 5 lines · Ævar Arnfjörð Bjarmason

A subsequent commit will remove GETTEXT_POISON entirely, let's start by removing the CI jobs that enable the option.

We cannot just remove the job because the CI is implicitly depending on the "poison" job being a sort of "default" job.

I don't understand what you mean here with a "default job" that the CI is implicitly depending on. There is certainly no such default job on Travis CI, and I don't think there is one on the GitHub thing.

I'll reword this. I meant that the poison job was using the default compiler etc., whereas the other ones were setting custom values.

Quoted 7 lines
Quoted 3 lines

I vaguely remember some list traffic about this in the past, i.e. the reliance on the poison job not just for that, but also as "the default OS image" setup.

Oh, I didn't mean that the commit message should be clarified. I meant that the GETTEXT_POISON job can simply be deleted. Sorry for not being clear enough.

Quoted 8 lines

I still don't get it, and my patch still seems fine as it is to me. But I'm probably missing something.

I'm looking at these, both pointing at e6362826a04 (The fourth batch, 2021-01-25):

# Currently says "Please be aware travis-ci.org will be shutting # down in several weeks"

... and continues with:

"with all accounts migrating to travis-ci.com. Please stay tuned
here for more information."

FWIW, I recently migrated manually, and apart from the different TLD haven't noticed anything different (though somehow the jobs in my .org builds sometimes took a while to start lately, but on .com they all started real fast so far, as they should).

Quoted 11 lines
    https://www.travis-ci.org/github/git/git/builds/756176098

    https://github.com/git/git/actions/runs/510676142

For the GitHub one, there's only two things that are linux && gcc. The linux-gcc job, and GETTEXT_POISON (renamed to linux-gcc-default in this series).

In ci/install-dependencies.sh we install GCC 8 for linux-gcc, but also perforce, apache, git-lfs etc.

Quoted 6 lines

Then in ci/lib.sh we set CC=gcc-8 on linux-gcc, but retain the default on GETTEXT_POISON. We also set GIT_TEST_HTTPD=true.

So hence "linux-gcc-default", we're running as close to a default set of packages we can get and build/run git at all. I.e. just "apt install $UBUNTU_COMMON_PKGS".

Quoted 6 lines

We started installing newer compilers, because we wanted to benefit from their better error reporting and improved warnings. The reason we did that in existing CI jobs without adding a "default" job was that we don't care enough about the default compiler and configuration to justify its additional runtime, because we run the test suite enough times already.

So in summary, you'd like it removed because the combination of GETTEXT_POISON + old compiler is no longer worth it, but while we were running with GETTEXT_POISON anyway we might as well run it on the old compiler?

Quoted 3 lines

Considering that the number of CI jobs has grown since then, I would rather see the build complete faster than a default job running the test suite for the N+1th time.

[...]

Quoted 25 lines
Quoted 25 lines

Then in ci/run-build-and-tests.sh the "linux-gcc" job is also our "let's turn on every GIT_TEST_* option you've heard of" job. Whereas on the GETTEXT_POISON job we just ran "make test", well, with GIT_TEST_GETTEXT_POISON=true before this series, but that was the only non-default, now we've got no non-defaults.

I can see why we'd want this on top:

    diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh
    index 50e0b90073f..3071099eaca 100755
    --- a/ci/run-build-and-tests.sh
    +++ b/ci/run-build-and-tests.sh
    @@ -32,11 +32,6 @@ linux-clang)
            export GIT_TEST_DEFAULT_HASH=sha256
            make test
            ;;
    -linux-gcc-4.8)
    -       # Don't run the tests; we only care about whether Git can be
    -       # built with GCC 4.8, as it errors out on some undesired (C99)
    -       # constructs that newer compilers seem to quietly accept.
    -       ;;
     *)
            make test
            ;;
Quoted 8 lines
Quoted 4 lines

Because there we're skipping tests on linux-gcc-4.8, but on travis it's the only Ubuntu Trusty image (default is Xenial), so yes we build with gcc 4.8, but we're also missing out in seeing if our tests are working on a slightly older Ubuntu.

The above comment you propose to remove explains why you should not remove it... And, again, running the test suite doesn't seem to worth the extra runtime.

How is it extra runtime in any meaningful sense?

Yes, we spend more CPU time, but as far as I can tell (and I'm looking at an ongoing run on GitHub now) these jobs are run concurrently. Some of that was covered in the quoted paragraph below that you didn't reply to.

I.e. if I push a commit I don't care if 10 minutes of CPU time are being wasted by a machine sitting in some rack somewhere, the important part is that I may be saving 20 minutes of my time, since the faster 10 minute test (running concurrently with the ~30m tests) might fail earlier.

Or are there some job limits involved here that I'm missing?

As I was writing this the CI job for GETTEXT_POISON just finished in 10m16s showing OK=green, and the rest are still churning.

Quoted 10 lines
Quoted 10 lines

Also on GitHub's page the GETTEXT_POISON job finishes in ~10m but all the other ones in ~30m, I see some run "make test" twice, as an aside is there some limit on jobs but not overall CPU? Otherwise those would finish faster if that was split up.

But "make test" twice doesn't account for a ~3x increase, so it's got to be the extra tests we're running, i.e. svn tests, httpd etc.

It seems to me to provide value in itself to run some shorter version of the CI for people using it as a poor man's edit/push/ci/edit loop.

#125 lore raw

Re: [PATCH 1/6] ci: remove GETTEXT_POISON jobs

Quoted 77 lines · Ævar Arnfjörð Bjarmason
Quoted 67 lines · SZEDER Gábor
Quoted 29 lines · Ævar Arnfjörð Bjarmason
Quoted 18 lines · SZEDER Gábor
Quoted 17 lines · Ævar Arnfjörð Bjarmason
Quoted 10 lines · SZEDER Gábor
Quoted 5 lines · Ævar Arnfjörð Bjarmason

A subsequent commit will remove GETTEXT_POISON entirely, let's start by removing the CI jobs that enable the option.

We cannot just remove the job because the CI is implicitly depending on the "poison" job being a sort of "default" job.

I don't understand what you mean here with a "default job" that the CI is implicitly depending on. There is certainly no such default job on Travis CI, and I don't think there is one on the GitHub thing.

I'll reword this. I meant that the poison job was using the default compiler etc., whereas the other ones were setting custom values.

Quoted 7 lines
Quoted 3 lines

I vaguely remember some list traffic about this in the past, i.e. the reliance on the poison job not just for that, but also as "the default OS image" setup.

Oh, I didn't mean that the commit message should be clarified. I meant that the GETTEXT_POISON job can simply be deleted. Sorry for not being clear enough.

Quoted 8 lines

I still don't get it, and my patch still seems fine as it is to me. But I'm probably missing something.

I'm looking at these, both pointing at e6362826a04 (The fourth batch, 2021-01-25):

# Currently says "Please be aware travis-ci.org will be shutting # down in several weeks"

... and continues with:

"with all accounts migrating to travis-ci.com. Please stay tuned
here for more information."

FWIW, I recently migrated manually, and apart from the different TLD haven't noticed anything different (though somehow the jobs in my .org builds sometimes took a while to start lately, but on .com they all started real fast so far, as they should).

Quoted 11 lines
    https://www.travis-ci.org/github/git/git/builds/756176098

    https://github.com/git/git/actions/runs/510676142

For the GitHub one, there's only two things that are linux && gcc. The linux-gcc job, and GETTEXT_POISON (renamed to linux-gcc-default in this series).

In ci/install-dependencies.sh we install GCC 8 for linux-gcc, but also perforce, apache, git-lfs etc.

Quoted 6 lines

Then in ci/lib.sh we set CC=gcc-8 on linux-gcc, but retain the default on GETTEXT_POISON. We also set GIT_TEST_HTTPD=true.

So hence "linux-gcc-default", we're running as close to a default set of packages we can get and build/run git at all. I.e. just "apt install $UBUNTU_COMMON_PKGS".

Quoted 6 lines

We started installing newer compilers, because we wanted to benefit from their better error reporting and improved warnings. The reason we did that in existing CI jobs without adding a "default" job was that we don't care enough about the default compiler and configuration to justify its additional runtime, because we run the test suite enough times already.

Quoted 4 lines

So in summary, you'd like it removed because the combination of GETTEXT_POISON + old compiler is no longer worth it, but while we were running with GETTEXT_POISON anyway we might as well run it on the old compiler?

Yes, though I suspect that "no one gave a damn about the compiler used in the GETTEXT_POISON job" might be closer to the truth. Or even "no one gave a damn about the GETTEXT_POISON job"...

And apart from the occasional translation-related failure (i.e. 'grep "<translated message>" actual') and one flaky test that happened to fail in the GETTEXT_POISON job I can't recall a single case where building and testing with the default compiler discovered an issue that the other jobs didn't.

Quoted 36 lines
Quoted 3 lines

Considering that the number of CI jobs has grown since then, I would rather see the build complete faster than a default job running the test suite for the N+1th time.

[...]

Quoted 30 lines
Quoted 29 lines

Then in ci/run-build-and-tests.sh the "linux-gcc" job is also our "let's turn on every GIT_TEST_* option you've heard of" job. Whereas on the GETTEXT_POISON job we just ran "make test", well, with GIT_TEST_GETTEXT_POISON=true before this series, but that was the only non-default, now we've got no non-defaults.

I can see why we'd want this on top:

    diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh
    index 50e0b90073f..3071099eaca 100755
    --- a/ci/run-build-and-tests.sh
    +++ b/ci/run-build-and-tests.sh
    @@ -32,11 +32,6 @@ linux-clang)
            export GIT_TEST_DEFAULT_HASH=sha256
            make test
            ;;
    -linux-gcc-4.8)
    -       # Don't run the tests; we only care about whether Git can be
    -       # built with GCC 4.8, as it errors out on some undesired (C99)
    -       # constructs that newer compilers seem to quietly accept.
    -       ;;
     *)
            make test
            ;;

Because there we're skipping tests on linux-gcc-4.8, but on travis it's the only Ubuntu Trusty image (default is Xenial), so yes we build with gcc 4.8, but we're also missing out in seeing if our tests are working on a slightly older Ubuntu.

Quoted 8 lines
Quoted 3 lines

The above comment you propose to remove explains why you should not remove it... And, again, running the test suite doesn't seem to worth the extra runtime.

How is it extra runtime in any meaningful sense?

Yes, we spend more CPU time, but as far as I can tell (and I'm looking at an ongoing run on GitHub now)

(Well, I'm looking at eight ongoing builds now...)

Quoted 11 lines

these jobs are run concurrently. Some
of that was covered in the quoted paragraph below that you didn't reply to.

I.e. if I push a commit I don't care if 10 minutes of CPU time are being wasted by a machine sitting in some rack somewhere, the important part is that I may be saving 20 minutes of my time, since the faster 10 minute test (running concurrently with the ~30m tests) might fail earlier.

Or are there some job limits involved here that I'm missing?

Travis CI usually runs 3 Linux and 2 OSX jobs in parallel, and I doubt that GitHub would be foolish enough to run all my jobs parallel if I were to use it.

Quoted 2 lines

As I was writing this the CI job for GETTEXT_POISON just finished in 10m16s showing OK=green, and the rest are still churning.

Indeed, but you should also consider that:

- At 10m16s that job is notably slower than even my tiny old laptop, even though it runs less tests. If you really want to save time, then run 'make test' locally.

- Without the GETTEXT_POISON job the whole build would be finished ~10 minutes earlier. With six builds that's an hour.

Quoted 11 lines
Quoted 10 lines
Quoted 10 lines

Also on GitHub's page the GETTEXT_POISON job finishes in ~10m but all the other ones in ~30m, I see some run "make test" twice, as an aside is there some limit on jobs but not overall CPU? Otherwise those would finish faster if that was split up.

But "make test" twice doesn't account for a ~3x increase, so it's got to be the extra tests we're running, i.e. svn tests, httpd etc.

It seems to me to provide value in itself to run some shorter version of the CI for people using it as a poor man's edit/push/ci/edit loop.

#126 lore raw

PATCH 2/6 tests: remove support for GIT_TEST_GETTEXT_POISON

In reply to Ævar Arnfjörð Bjarmason

This removes the ability to inject "poison" gettext() messages via the GIT_TEST_GETTEXT_POISON special test setup.

I initially added this as a compile-time option in bb946bba761 (i18n:
add GETTEXT_POISON to simulate unfriendly translator, 2011-02-22), and most recently modified to be toggleable at runtime in 6cdccfce1e0 (i18n: make GETTEXT_POISON a runtime option, 2018-11-08)..

The reason for its removal is that the trade-off of maintaining it v.s. what it's getting us has long since flipped. When gettext was integrated in 5e9637c6297 (i18n: add infrastructure for translating Git with gettext, 2011-11-18) there was understandable concern on the Git ML that in marking messages for translation en-masse we'd inadvertently mark plumbing messages. The GETTEXT_POISON facility was a way to smoke those out via our test suite.

Nowadays however we're done (or almost entirely done) with any marking of messages for translation. New messages are usually marked by their authors, who'll know whether it makes sense to translate them or not. If not any errors in marking the messages are much more likely to be spotted in review than in the the initial deluge of i18n patches in the 2011-2012 era.

So let's just remove this. This leaves the test suite in a state where we still have a lot of test_i18n, C_LOCALE_OUTPUT
etc. uses. Subsequent commits will remove those too.

---
 Documentation/MyFirstContribution.txt |  2 +-
 config.c                              |  9 -------
 gettext.c                             | 10 -------
 gettext.h                             |  7 +----
 git-sh-i18n.sh                        | 22 +--------------
 po/README                             | 34 ++++++++++++-----------
 t/README                              |  6 -----
 t/lib-gettext.sh                      |  2 +-
 t/t0017-env-helper.sh                 |  8 +++---
 t/t0205-gettext-poison.sh             | 39 ---------------------------
 t/test-lib-functions.sh               | 23 +++++-----------
 t/test-lib.sh                         | 23 +++-------------
 12 files changed, 38 insertions(+), 147 deletions(-)
 delete mode 100755 t/t0205-gettext-poison.sh
Documentation/MyFirstContribution.txt +1 -1
index 7c9a037cc2..af0a9da62e 100644
@@ -664,7 +664,7 @@ mention the right animal somewhere:
 ----
 test_expect_success 'runs correctly with no args and good output' '
 	git psuh >actual &&
-	test_i18ngrep Pony actual
+	grep Pony actual
 '
 ----
 
config.c +0 -9
index 1137bd73af..a8dec9a247 100644
@@ -996,15 +996,6 @@ static void die_bad_number(const char *name, const char *value)
 	if (!value)
 		value = "";
 
-	if (!strcmp(name, "GIT_TEST_GETTEXT_POISON"))
-		/*
-		 * We explicitly *don't* use _() here since it would
-		 * cause an infinite loop with _() needing to call
-		 * use_gettext_poison(). This is why marked up
-		 * translations with N_() above.
-		 */
-		die(bad_numeric, value, name, error_type);
-
 	if (!(cf && cf->name))
 		die(_(bad_numeric), value, name, _(error_type));
 
gettext.c +0 -10
index 35d2c1218d..308c4b3129 100644
@@ -65,14 +65,6 @@ const char *get_preferred_languages(void)
 	return NULL;
 }
 
-int use_gettext_poison(void)
-{
-	static int poison_requested = -1;
-	if (poison_requested == -1)
-		poison_requested = git_env_bool("GIT_TEST_GETTEXT_POISON", 0);
-	return poison_requested;
-}
-
 #ifndef NO_GETTEXT
 static int test_vsnprintf(const char *fmt, ...)
 {
@@ -181,8 +173,6 @@ void git_setup_gettext(void)
 	if (!podir)
 		podir = p = system_path(GIT_LOCALE_PATH);
 
-	use_gettext_poison(); /* getenv() reentrancy paranoia */
-
 	if (!is_directory(podir)) {
 		free(p);
 		return;
gettext.h +1 -6
index bee52eb113..c8b34fd612 100644
@@ -28,15 +28,12 @@
 
 #define FORMAT_PRESERVING(n) __attribute__((format_arg(n)))
 
-int use_gettext_poison(void);
-
 #ifndef NO_GETTEXT
 void git_setup_gettext(void);
 int gettext_width(const char *s);
 #else
 static inline void git_setup_gettext(void)
 {
-	use_gettext_poison(); /* getenv() reentrancy paranoia */
 }
 static inline int gettext_width(const char *s)
 {
@@ -48,14 +45,12 @@ static inline FORMAT_PRESERVING(1) const char *_(const char *msgid)
 {
 	if (!*msgid)
 		return "";
-	return use_gettext_poison() ? "# GETTEXT POISON #" : gettext(msgid);
+	return gettext(msgid);
 }
 
 static inline FORMAT_PRESERVING(1) FORMAT_PRESERVING(2)
 const char *Q_(const char *msgid, const char *plu, unsigned long n)
 {
-	if (use_gettext_poison())
-		return "# GETTEXT POISON #";
 	return ngettext(msgid, plu, n);
 }
 
git-sh-i18n.sh +1 -21
index 8eef60b43f..e3d9f4836d 100644
@@ -17,12 +17,7 @@ export TEXTDOMAINDIR
 
 # First decide what scheme to use...
 GIT_INTERNAL_GETTEXT_SH_SCHEME=fallthrough
-if test -n "$GIT_TEST_GETTEXT_POISON" &&
-	    git env--helper --type=bool --default=0 --exit-code \
-		GIT_TEST_GETTEXT_POISON
-then
-	GIT_INTERNAL_GETTEXT_SH_SCHEME=poison
-elif test -n "@@USE_GETTEXT_SCHEME@@"
+if test -n "@@USE_GETTEXT_SCHEME@@"
 then
 	GIT_INTERNAL_GETTEXT_SH_SCHEME="@@USE_GETTEXT_SCHEME@@"
 elif test -n "$GIT_INTERNAL_GETTEXT_TEST_FALLBACKS"
@@ -63,21 +58,6 @@ gettext_without_eval_gettext)
 		)
 	}
 	;;
-poison)
-	# Emit garbage so that tests that incorrectly rely on translatable
-	# strings will fail.
-	gettext () {
-		printf "%s" "# GETTEXT POISON #"
-	}
-
-	eval_gettext () {
-		printf "%s" "# GETTEXT POISON #"
-	}
-
-	eval_ngettext () {
-		printf "%s" "# GETTEXT POISON #"
-	}
-	;;
 *)
 	gettext () {
 		printf "%s" "$1"
po/README +19 -15
index 07595d369b..b9b6c9cb24 100644
@@ -284,23 +284,27 @@ Perl:
 Testing marked strings
 ----------------------
 
-Even if you've correctly marked porcelain strings for translation
-something in the test suite might still depend on the US English
-version of the strings, e.g. to grep some error message or other
-output.
+Git's tests are run under LANG=C LC_ALL=C. So the tests do not need be
+changed to account for translations as they're added.
 
-To smoke out issues like these, Git tested with a translation mode that
-emits gibberish on every call to gettext. To use it run the test suite
-with it, e.g.:
+Testing whether marked strings are used by plumbing
+---------------------------------------------------
 
-    cd t && GIT_TEST_GETTEXT_POISON=true prove -j 9 ./t[0-9]*.sh
+It may be useful to use the test suite to smoke out any strings that
+shouldn't be translated, because they are used by plumbing or
+otherwise part of machine-readable output.
 
-If tests break with it you should inspect them manually and see if
-what you're translating is sane, i.e. that you're not translating
+This is most useful in cases where e.g. library function might return
+a string like _("tree"), and the developer marking strings for
+translation wishes to check that they're not inadvertently translating
 plumbing output.
 
-If not you should replace calls to grep with test_i18ngrep, or
-test_cmp calls with test_i18ncmp. If that's not enough you can skip
-the whole test by making it depend on the C_LOCALE_OUTPUT
-prerequisite. See existing test files with this prerequisite for
-examples.
+There used to be a GIT_TEST_GETTEXT_POISON=true test mode to replace
+translated strings with gibberish to facilitate this. Maintaining it
+and annotating tests appropriately was considered too high of a burden
+for its usefulness.
+
+It's still possible to do a one-off test of where a string might be
+used in the tests by simply replacing the "some hardcoded string" with
+something like "POISON ME" locally and running the tests, before
+finally converting it to _("some hardcoded string").
t/README +0 -6
index c730a70770..9a9dded033 100644
@@ -358,12 +358,6 @@ whether this mode is active, and e.g. skip some tests that are hard to
 refactor to deal with it. The "SYMLINKS" prerequisite is currently
 excluded as so much relies on it, but this might change in the future.
 
-GIT_TEST_GETTEXT_POISON=<boolean> turns all strings marked for
-translation into gibberish if true. Used for spotting those tests that
-need to be marked with a C_LOCALE_OUTPUT prerequisite when adding more
-strings for translation. See "Testing marked strings" in po/README for
-details.
-
 GIT_TEST_SPLIT_INDEX=<boolean> forces split-index mode on the whole
 test suite. Accept any boolean values that are accepted by git-config.
 
t/lib-gettext.sh +1 -1
index 2139b427ca..cc6bb2cdea 100644
@@ -17,7 +17,7 @@ else
 	. "$GIT_BUILD_DIR"/git-sh-i18n
 fi
 
-if test_have_prereq GETTEXT && test_have_prereq C_LOCALE_OUTPUT
+if test_have_prereq GETTEXT
 then
 	# is_IS.UTF-8 on Solaris and FreeBSD, is_IS.utf8 on Debian
 	is_IS_locale=$(locale -a 2>/dev/null |
t/t0017-env-helper.sh +4 -4
index c1ecf6aeac..4a159f99e4 100755
@@ -86,14 +86,14 @@ test_expect_success 'env--helper reads config thanks to trace2' '
 	git config -f home/cycle include.path .gitconfig &&
 
 	test_must_fail \
-		env HOME="$(pwd)/home" GIT_TEST_GETTEXT_POISON=false \
+		env HOME="$(pwd)/home" \
 		git config -l 2>err &&
 	grep "exceeded maximum include depth" err &&
 
 	test_must_fail \
-		env HOME="$(pwd)/home" GIT_TEST_GETTEXT_POISON=true \
-		git -C cycle env--helper --type=bool --default=0 --exit-code GIT_TEST_GETTEXT_POISON 2>err &&
-	grep "# GETTEXT POISON #" err
+		env HOME="$(pwd)/home" GIT_TEST_ENV_HELPER=true \
+		git -C cycle env--helper --type=bool --default=0 --exit-code GIT_TEST_ENV_HELPER 2>err &&
+	grep "exceeded maximum include depth" err
 '
 
 test_done
t/t0205-gettext-poison.sh deleted +0 -39
deleted file mode 100755
index f9fa16ad83..0000000000
@@ -1,39 +0,0 @@
-#!/bin/sh
-#
-# Copyright (c) 2010 Ævar Arnfjörð Bjarmason
-#
-
-test_description='Gettext Shell poison'
-
-GIT_TEST_GETTEXT_POISON=true
-export GIT_TEST_GETTEXT_POISON
-. ./lib-gettext.sh
-
-test_expect_success 'sanity: $GIT_INTERNAL_GETTEXT_SH_SCHEME" is poison' '
-    test "$GIT_INTERNAL_GETTEXT_SH_SCHEME" = "poison"
-'
-
-test_expect_success 'gettext: our gettext() fallback has poison semantics' '
-    printf "# GETTEXT POISON #" >expect &&
-    gettext "test" >actual &&
-    test_cmp expect actual &&
-    printf "# GETTEXT POISON #" >expect &&
-    gettext "test more words" >actual &&
-    test_cmp expect actual
-'
-
-test_expect_success 'eval_gettext: our eval_gettext() fallback has poison semantics' '
-    printf "# GETTEXT POISON #" >expect &&
-    eval_gettext "test" >actual &&
-    test_cmp expect actual &&
-    printf "# GETTEXT POISON #" >expect &&
-    eval_gettext "test more words" >actual &&
-    test_cmp expect actual
-'
-
-test_expect_success "gettext: invalid GIT_TEST_GETTEXT_POISON value doesn't infinitely loop" "
-	test_must_fail env GIT_TEST_GETTEXT_POISON=xyz git version 2>error &&
-	grep \"fatal: bad numeric config value 'xyz' for 'GIT_TEST_GETTEXT_POISON': invalid unit\" error
-"
-
-test_done
t/test-lib-functions.sh +7 -16
index 999982fe4a..c7b1d857c4 100644
@@ -988,19 +988,16 @@ test_cmp_bin () {
 	cmp "$@"
 }
 
-# Use this instead of test_cmp to compare files that contain expected and
-# actual output from git commands that can be translated.  When running
-# under GIT_TEST_GETTEXT_POISON this pretends that the command produced expected
-# results.
+# Wrapper for test_cmp which used to be used for
+# GIT_TEST_GETTEXT_POISON=false. Only here as a shim for other
+# in-flight changes. Should not be used and will be removed soon.
 test_i18ncmp () {
-	! test_have_prereq C_LOCALE_OUTPUT || test_cmp "$@"
+	test_cmp "$@"
 }
 
-# Use this instead of "grep expected-string actual" to see if the
-# output from a git command that can be translated either contains an
-# expected string, or does not contain an unwanted one.  When running
-# under GIT_TEST_GETTEXT_POISON this pretends that the command produced expected
-# results.
+# Wrapper for grep which used to be used for
+# GIT_TEST_GETTEXT_POISON=false. Only here as a shim for other
+# in-flight changes. Should not be used and will be removed soon.
 test_i18ngrep () {
 	eval "last_arg=\${$#}"
 
@@ -1013,12 +1010,6 @@ test_i18ngrep () {
 		BUG "too few parameters to test_i18ngrep"
 	fi
 
-	if test_have_prereq !C_LOCALE_OUTPUT
-	then
-		# pretend success
-		return 0
-	fi
-
 	if test "x!" = "x$1"
 	then
 		shift
t/test-lib.sh +4 -19
index 9fa7c1d0f6..c1ff5db2c1 100644
@@ -404,15 +404,6 @@ TZ=UTC
 export LANG LC_ALL PAGER TZ
 EDITOR=:
 
-# GIT_TEST_GETTEXT_POISON should not influence git commands executed
-# during initialization of test-lib and the test repo. Back it up,
-# unset and then restore after initialization is finished.
-if test -n "$GIT_TEST_GETTEXT_POISON"
-then
-	GIT_TEST_GETTEXT_POISON_ORIG=$GIT_TEST_GETTEXT_POISON
-	unset GIT_TEST_GETTEXT_POISON
-fi
-
 # A call to "unset" with no arguments causes at least Solaris 10
 # /usr/xpg4/bin/sh and /bin/ksh to bail out.  So keep the unsets
 # deriving from the command substitution clustered with the other
@@ -1529,16 +1520,10 @@ test -n "$USE_LIBPCRE1" && test_set_prereq LIBPCRE1
 test -n "$USE_LIBPCRE2" && test_set_prereq LIBPCRE2
 test -z "$NO_GETTEXT" && test_set_prereq GETTEXT
 
-if test -n "$GIT_TEST_GETTEXT_POISON_ORIG"
-then
-	GIT_TEST_GETTEXT_POISON=$GIT_TEST_GETTEXT_POISON_ORIG
-	export GIT_TEST_GETTEXT_POISON
-	unset GIT_TEST_GETTEXT_POISON_ORIG
-fi
-
-test_lazy_prereq C_LOCALE_OUTPUT '
-	! test_bool_env GIT_TEST_GETTEXT_POISON false
-'
+# Used to be used for GIT_TEST_GETTEXT_POISON=false. Only here as a
+# shim for other in-flight changes. Should not be used and will be
+# removed soon.
+test_set_prereq C_LOCALE_OUTPUT
 
 if test -z "$GIT_TEST_CHECK_CACHE_TREE"
 then
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#127 lore raw

Re: [PATCH 2/6] tests: remove support for GIT_TEST_GETTEXT_POISON

po/README Perl: · line 288

Quoted 10 lines · Ævar Arnfjörð Bjarmason
po/README +2 -0
index 07595d369b..b9b6c9cb24 100644
@@ -284,23 +284,27 @@ Perl:
 Testing marked strings
 ----------------------
 
+Git's tests are run under LANG=C LC_ALL=C. So the tests do not need be
+changed to account for translations as they're added.

After we told readers this ...

po/README Perl: · line 310

Quoted 6 lines
po/README +5 -0
+Testing whether marked strings are used by plumbing
...
+It's still possible to do a one-off test of where a string might be
+used in the tests by simply replacing the "some hardcoded string" with
+something like "POISON ME" locally and running the tests, before
+finally converting it to _("some hardcoded string").

... I do not see how this manual check can still be effective.

People rely on our tests running under C locale, and will add tests to grep for error messages meant for humans in C locale. What does it help to notice that a particular string you are contemplating to translate _is_ being looked for in an existing test? The author of such a test knew it was meant for humans, but trusted our promise to run tests in the C locale and used grep for it, in which case it is a fair game to be marked with _().

I think we should honestly admit that after this change it is no longer possible for our tests to help catching strings that are mistakenly marked for translation when they are meant for machine consumption, and justify why it is still OK to apply this change.

t/test-lib-functions.sh

Quoted 4 lines
t/test-lib-functions.sh +0 -0
index 999982fe4a..c7b1d857c4 100644

The changes to this file all look very attractive ;-) Not having to worry about poison test certainly makes it easier to write tests.

I just do not know if the price we are paying for the easier life is too high.

Thanks.

#128 lore raw

PATCH 1/1 config.c: remove last remnant of GIT_TEST_GETTEXT_POISON

In reply to Ævar Arnfjörð Bjarmason

Remove a use of GIT_TEST_GETTEXT_POISON added in f276e2a4694 (config:
improve error message for boolean config, 2021-02-11).

This was simultaneously in-flight with my d162b25f956 (tests: remove support for GIT_TEST_GETTEXT_POISON, 2021-01-20) which removed the rest of the GIT_TEST_GETTEXT_POISON code.

---

At this point I'm going to call it a win if I'm not submitting a patch in 2022 to remove some more stray GETTEXT_POISON ... :)

 config.c          | 16 +---------------
 t/t1300-config.sh |  2 +-
 2 files changed, 2 insertions(+), 16 deletions(-)
config.c +1 -15
index 6428393a414..870d9534def 100644
@@ -1180,20 +1180,6 @@ static void die_bad_number(const char *name, const char *value)
 	}
 }
 
-NORETURN
-static void die_bad_bool(const char *name, const char *value)
-{
-	if (!strcmp(name, "GIT_TEST_GETTEXT_POISON"))
-		/*
-		 * We explicitly *don't* use _() here since it would
-		 * cause an infinite loop with _() needing to call
-		 * use_gettext_poison().
-		 */
-		die("bad boolean config value '%s' for '%s'", value, name);
-	else
-		die(_("bad boolean config value '%s' for '%s'"), value, name);
-}
-
 int git_config_int(const char *name, const char *value)
 {
 	int ret;
@@ -1268,7 +1254,7 @@ int git_config_bool(const char *name, const char *value)
 {
 	int v = git_parse_maybe_bool(value);
 	if (v < 0)
-		die_bad_bool(name, value);
+		die(_("bad boolean config value '%s' for '%s'"), value, name);
 	return v;
 }
 
t/t1300-config.sh +1 -1
index e0dd5d65ced..2280c2504ac 100755
@@ -679,7 +679,7 @@ test_expect_success 'invalid unit boolean' '
 	git config commit.gpgsign "1true" &&
 	test_cmp_config 1true commit.gpgsign &&
 	test_must_fail git config --bool --get commit.gpgsign 2>actual &&
-	test_i18ngrep "bad boolean config value .1true. for .commit.gpgsign." actual
+	grep "bad boolean config value .1true. for .commit.gpgsign." actual
 '
 
 test_expect_success 'line number is reported correctly' '
Signature · 1 line
2.31.0.354.gc8cbd507b5a
#129 lore raw

Re: [PATCH] config.c: remove last remnant of GIT_TEST_GETTEXT_POISON

Quoted 6 lines · Ævar Arnfjörð Bjarmason

Remove a use of GIT_TEST_GETTEXT_POISON added in f276e2a4694 (config:
improve error message for boolean config, 2021-02-11).

This was simultaneously in-flight with my d162b25f956 (tests: remove support for GIT_TEST_GETTEXT_POISON, 2021-01-20) which removed the rest of the GIT_TEST_GETTEXT_POISON code.

Yay. :)

Quoted 26 lines
config.c +0 -14
index 6428393a414..870d9534def 100644
@@ -1180,20 +1180,6 @@ static void die_bad_number(const char *name, const char *value)
 	}
 }
 
-NORETURN
-static void die_bad_bool(const char *name, const char *value)
-{
-	if (!strcmp(name, "GIT_TEST_GETTEXT_POISON"))
-		/*
-		 * We explicitly *don't* use _() here since it would
-		 * cause an infinite loop with _() needing to call
-		 * use_gettext_poison().
-		 */
-		die("bad boolean config value '%s' for '%s'", value, name);
-	else
-		die(_("bad boolean config value '%s' for '%s'"), value, name);
-}
-
 int git_config_int(const char *name, const char *value)
 {
 	int ret;

config.c int git_config_bool(const char *name, const char *value) · line 1259

Quoted 7 lines
config.c +1 -1
@@ -1268,7 +1254,7 @@ int git_config_bool(const char *name, const char *value)
 {
 	int v = git_parse_maybe_bool(value);
 	if (v < 0)
-		die_bad_bool(name, value);
+		die(_("bad boolean config value '%s' for '%s'"), value, name);
 	return v;
 }

This code change looks good, but...

t/t1300-config.sh test_expect_success 'invalid unit boolean' ' · line 683

Quoted 11 lines
t/t1300-config.sh +1 -1
index e0dd5d65ced..2280c2504ac 100755
@@ -679,7 +679,7 @@ test_expect_success 'invalid unit boolean' '
 	git config commit.gpgsign "1true" &&
 	test_cmp_config 1true commit.gpgsign &&
 	test_must_fail git config --bool --get commit.gpgsign 2>actual &&
-	test_i18ngrep "bad boolean config value .1true. for .commit.gpgsign." actual
+	grep "bad boolean config value .1true. for .commit.gpgsign." actual
 '

why are we losing test_i18ngrep here? The message is still marked for translation. I know we've discussed dropping all of the test_i18n helpers, but that seems unrelated to the rest of the patch.

-Peff

#130 lore raw

Re: [PATCH] config.c: remove last remnant of GIT_TEST_GETTEXT_POISON

Quoted 47 lines · Jeff King
Quoted 6 lines · Ævar Arnfjörð Bjarmason

Remove a use of GIT_TEST_GETTEXT_POISON added in f276e2a4694 (config:
improve error message for boolean config, 2021-02-11).

This was simultaneously in-flight with my d162b25f956 (tests: remove support for GIT_TEST_GETTEXT_POISON, 2021-01-20) which removed the rest of the GIT_TEST_GETTEXT_POISON code.

Yay. :)

Quoted 26 lines
config.c +0 -14
index 6428393a414..870d9534def 100644
@@ -1180,20 +1180,6 @@ static void die_bad_number(const char *name, const char *value)
 	}
 }
 
-NORETURN
-static void die_bad_bool(const char *name, const char *value)
-{
-	if (!strcmp(name, "GIT_TEST_GETTEXT_POISON"))
-		/*
-		 * We explicitly *don't* use _() here since it would
-		 * cause an infinite loop with _() needing to call
-		 * use_gettext_poison().
-		 */
-		die("bad boolean config value '%s' for '%s'", value, name);
-	else
-		die(_("bad boolean config value '%s' for '%s'"), value, name);
-}
-
 int git_config_int(const char *name, const char *value)
 {
 	int ret;

config.c int git_config_bool(const char *name, const char *value) · line 1259

Quoted 7 lines
config.c +1 -1
@@ -1268,7 +1254,7 @@ int git_config_bool(const char *name, const char *value)
 {
 	int v = git_parse_maybe_bool(value);
 	if (v < 0)
-		die_bad_bool(name, value);
+		die(_("bad boolean config value '%s' for '%s'"), value, name);
 	return v;
 }

This code change looks good, but...

Quoted 15 lines

t/t1300-config.sh test_expect_success 'invalid unit boolean' ' · line 683

Quoted 11 lines
t/t1300-config.sh +1 -1
index e0dd5d65ced..2280c2504ac 100755
@@ -679,7 +679,7 @@ test_expect_success 'invalid unit boolean' '
 	git config commit.gpgsign "1true" &&
 	test_cmp_config 1true commit.gpgsign &&
 	test_must_fail git config --bool --get commit.gpgsign 2>actual &&
-	test_i18ngrep "bad boolean config value .1true. for .commit.gpgsign." actual
+	grep "bad boolean config value .1true. for .commit.gpgsign." actual
 '

why are we losing test_i18ngrep here? The message is still marked for translation. I know we've discussed dropping all of the test_i18n helpers, but that seems unrelated to the rest of the patch.

For new tests we're suggesting not to use it, so while I'm holding off on some general s/test_i18ngrep/grep/ refactoring, it seemed natural to adjust the test added by the commit whose code I'm modifying.

It also shows reviewers that there is such a test, so e.g. I didn't invert the name/value parameters or something in refactoring this.

#131 lore raw

Re: [PATCH] config.c: remove last remnant of GIT_TEST_GETTEXT_POISON

Quoted 16 lines · Ævar Arnfjörð Bjarmason
Quoted 7 lines
Quoted 7 lines
t/t1300-config.sh +0 -0
index e0dd5d65ced..2280c2504ac 100755
@@ -679,7 +679,7 @@ test_expect_success 'invalid unit boolean' '
 	git config commit.gpgsign "1true" &&
 	test_cmp_config 1true commit.gpgsign &&
Quoted 8 lines

t/t1300-config.sh test_expect_success 'invalid unit boolean' ' · line 683

Quoted 4 lines
t/t1300-config.sh +1 -1
 	test_must_fail git config --bool --get commit.gpgsign 2>actual &&
-	test_i18ngrep "bad boolean config value .1true. for .commit.gpgsign." actual
+	grep "bad boolean config value .1true. for .commit.gpgsign." actual
 '

why are we losing test_i18ngrep here? The message is still marked for translation. I know we've discussed dropping all of the test_i18n helpers, but that seems unrelated to the rest of the patch.

Quoted 3 lines

For new tests we're suggesting not to use it, so while I'm holding off on some general s/test_i18ngrep/grep/ refactoring, it seemed natural to adjust the test added by the commit whose code I'm modifying.

It would have been understandable if the proposed log message said

Remove a use of GIT_TEST_GETTEXT_POISON added in f276e2a4694 (config:
improve error message for boolean config, 2021-02-11), together with a new test added.

and removed the test.

But the test still has value for the remaining codebase, just like all the other tests that happen to use test_i18n{grep,cmp}. In a sense, the original mixed two separate things into one commit (i.e. use of TEST_GETTEXT_POISON to decide if the message given to die() is localized, and a test to see how "git config --bool --get" behaves when a malformed boolean value is given), and that may have been justifiable back in the world where GETTEXT_POISON was a thing, making these two things closely interrelated. Since we left that world behind, I think we should treat them as two separate things.

In short, I do not think "The C code we are removing was added in the same commit" is a good excuse for this "while at it" change.

Putting it another way, imagine back then there was the t1300 test and there was no die_bad_bool(). The test may have been expecting "bad numeric" or "invalid unit", with grep (with a known bug that the test would not pass under GETTEXT_POISON).

In such an alternative past, the change you are reverting may have been only to config.c to make the die(_()) work correctly with GETTEXT_POISON, and turned grep to test_i18ngrep. And your "we are reverting the whole commit" may have made more sense.

But we are not living in such an alternative world.

Having said all that, I do not particularly care when, in which exact commit, a use of test_i18n* in t/ among 1100+ of them lost its i18n-ness (it just felt a bit out of place in this particular commit, that's all).

Thanks.

#132 lore raw

PATCH v2 1/1 config.c: remove last remnant of GIT_TEST_GETTEXT_POISON

In reply to Ævar Arnfjörð Bjarmason

Remove a use of GIT_TEST_GETTEXT_POISON added in f276e2a4694 (config:
improve error message for boolean config, 2021-02-11).

This was simultaneously in-flight with my d162b25f956 (tests: remove support for GIT_TEST_GETTEXT_POISON, 2021-01-20) which removed the rest of the GIT_TEST_GETTEXT_POISON code.

---

This trivial cleanup now without the objectionable and needless change to the tests.

Range-diff:
1: ea968affa8 ! 1: 2b2ac8471f config.c: remove last remnant of GIT_TEST_GETTEXT_POISON
@@ config.c: int git_config_bool(const char *name, const char *value)
return v;
}

-
- ## t/t1300-config.sh ##
-@@ t/t1300-config.sh: test_expect_success 'invalid unit boolean' '
- git config commit.gpgsign "1true" &&
- test_cmp_config 1true commit.gpgsign &&
- test_must_fail git config --bool --get commit.gpgsign 2>actual &&
-- test_i18ngrep "bad boolean config value .1true. for .commit.gpgsign." actual
-+ grep "bad boolean config value .1true. for .commit.gpgsign." actual
- '
-
- test_expect_success 'line number is reported correctly' '

 config.c | 16 +---------------
 1 file changed, 1 insertion(+), 15 deletions(-)
config.c +1 -15
index 6428393a41..870d9534de 100644
@@ -1180,20 +1180,6 @@ static void die_bad_number(const char *name, const char *value)
 	}
 }
 
-NORETURN
-static void die_bad_bool(const char *name, const char *value)
-{
-	if (!strcmp(name, "GIT_TEST_GETTEXT_POISON"))
-		/*
-		 * We explicitly *don't* use _() here since it would
-		 * cause an infinite loop with _() needing to call
-		 * use_gettext_poison().
-		 */
-		die("bad boolean config value '%s' for '%s'", value, name);
-	else
-		die(_("bad boolean config value '%s' for '%s'"), value, name);
-}
-
 int git_config_int(const char *name, const char *value)
 {
 	int ret;
@@ -1268,7 +1254,7 @@ int git_config_bool(const char *name, const char *value)
 {
 	int v = git_parse_maybe_bool(value);
 	if (v < 0)
-		die_bad_bool(name, value);
+		die(_("bad boolean config value '%s' for '%s'"), value, name);
 	return v;
 }
 
Signature · 1 line
2.31.1.527.g9b8f7de2547
#133 lore raw

Re: [PATCH v2] config.c: remove last remnant of GIT_TEST_GETTEXT_POISON

Quoted 2 lines · Ævar Arnfjörð Bjarmason

Remove a use of GIT_TEST_GETTEXT_POISON added in f276e2a4694 (config:
improve error message for boolean config, 2021-02-11).

Thanks. Will queue.

Quoted 65 lines

This was simultaneously in-flight with my d162b25f956 (tests: remove support for GIT_TEST_GETTEXT_POISON, 2021-01-20) which removed the rest of the GIT_TEST_GETTEXT_POISON code.

---

This trivial cleanup now without the objectionable and needless change to the tests.

Range-diff:
1: ea968affa8 ! 1: 2b2ac8471f config.c: remove last remnant of GIT_TEST_GETTEXT_POISON
@@ config.c: int git_config_bool(const char *name, const char *value)
return v;
}

-
- ## t/t1300-config.sh ##
-@@ t/t1300-config.sh: test_expect_success 'invalid unit boolean' '
- git config commit.gpgsign "1true" &&
- test_cmp_config 1true commit.gpgsign &&
- test_must_fail git config --bool --get commit.gpgsign 2>actual &&
-- test_i18ngrep "bad boolean config value .1true. for .commit.gpgsign." actual
-+ grep "bad boolean config value .1true. for .commit.gpgsign." actual
- '
-
- test_expect_success 'line number is reported correctly' '

 config.c | 16 +---------------
 1 file changed, 1 insertion(+), 15 deletions(-)
config.c +1 -15
index 6428393a41..870d9534de 100644
@@ -1180,20 +1180,6 @@ static void die_bad_number(const char *name, const char *value)
 	}
 }
 
-NORETURN
-static void die_bad_bool(const char *name, const char *value)
-{
-	if (!strcmp(name, "GIT_TEST_GETTEXT_POISON"))
-		/*
-		 * We explicitly *don't* use _() here since it would
-		 * cause an infinite loop with _() needing to call
-		 * use_gettext_poison().
-		 */
-		die("bad boolean config value '%s' for '%s'", value, name);
-	else
-		die(_("bad boolean config value '%s' for '%s'"), value, name);
-}
-
 int git_config_int(const char *name, const char *value)
 {
 	int ret;
@@ -1268,7 +1254,7 @@ int git_config_bool(const char *name, const char *value)
 {
 	int v = git_parse_maybe_bool(value);
 	if (v < 0)
-		die_bad_bool(name, value);
+		die(_("bad boolean config value '%s' for '%s'"), value, name);
 	return v;
 }
#134 lore raw

PATCH 3/6 tests: remove misc use of test_i18n{cmp,grep}

In reply to Ævar Arnfjörð Bjarmason

Remove miscellaneous use of test_i18n{cmp,grep} from contrib/*, lib-*.sh and t/*/*.sh etc. noted in an earlier commit we're removing the GIT_TEST_GETTEXT_POISON=true facility.

This leaves the only users of these functions in t/t[0-9]*.sh tests, which makes it easier to do a follow-up for-loop search-replacement of those occurrences at a later date.

---
 contrib/mw-to-git/t/t9363-mw-to-git-export-import.sh | 2 +-
 t/lib-credential.sh                                  | 2 +-
 t/lib-httpd.sh                                       | 2 +-
 t/lib-log-graph.sh                                   | 4 ++--
 t/lib-submodule-update.sh                            | 2 +-
 t/t5411/test-0026-push-options.sh                    | 3 +--
 t/t5411/test-0027-push-options--porcelain.sh         | 3 +--
 7 files changed, 8 insertions(+), 10 deletions(-)
contrib/mw-to-git/t/t9363-mw-to-git-export-import.sh +1 -1
index 6187ec67fa..3278334b9b 100755
@@ -161,7 +161,7 @@ test_expect_success 'git push properly warns about insufficient permissions' '
 		git add foo.forbidden &&
 		git commit -m "add a file" &&
 		git push 2>actual &&
-		test_i18ngrep "foo.forbidden is not a permitted file" actual
+		grep "foo.forbidden is not a permitted file" actual
 	)
 '
 
t/lib-credential.sh +1 -1
index dea2cbef51..5ea8bc9f1d 100644
@@ -20,7 +20,7 @@ check() {
 		false
 	fi &&
 	test_cmp expect-stdout stdout &&
-	test_i18ncmp expect-stderr stderr
+	test_cmp expect-stderr stderr
 }
 
 read_chunk() {
t/lib-httpd.sh +1 -1
index d2edfa4c50..a3ae0616e6 100644
@@ -219,7 +219,7 @@ test_http_push_nonff () {
 	'
 
 	test_expect_success 'non-fast-forward push shows help message' '
-		test_i18ngrep "Updates were rejected because" output
+		grep "Updates were rejected because" output
 	'
 
 	test_expect_${EXPECT_CAS_RESULT} 'force with lease aka cas' '
t/lib-log-graph.sh +2 -2
index 1184cceef2..bf952ef920 100644
@@ -12,13 +12,13 @@ sanitize_log_output () {
 lib_test_cmp_graph () {
 	git log --graph "$@" >output &&
 	sed 's/ *$//' >output.sanitized <output &&
-	test_i18ncmp expect output.sanitized
+	test_cmp expect output.sanitized
 }
 
 lib_test_cmp_short_graph () {
 	git log --graph --pretty=short "$@" >output &&
 	sanitize_log_output >output.sanitized <output &&
-	test_i18ncmp expect output.sanitized
+	test_cmp expect output.sanitized
 }
 
 lib_test_cmp_colored_graph () {
t/lib-submodule-update.sh +1 -1
index bd3fa3c6da..e04b5c4ba3 100644
@@ -830,7 +830,7 @@ test_submodule_recursing_with_args_common () {
 			cd submodule_update &&
 			git branch -t invalid_sub1 origin/invalid_sub1 &&
 			test_must_fail $command invalid_sub1 2>err &&
-			test_i18ngrep sub1 err &&
+			grep sub1 err &&
 			test_superproject_content origin/add_sub1 &&
 			test_submodule_content sub1 origin/add_sub1
 		)
t/t5411/test-0026-push-options.sh +1 -2
index e88edb16a4..a03f9ac5c4 100644
@@ -18,8 +18,7 @@ test_expect_success "proc-receive: not support push options ($PROTOCOL)" '
 		HEAD:refs/for/main/topic \
 		>out 2>&1 &&
 	make_user_friendly_and_stable_output <out >actual &&
-	test_i18ngrep "fatal: the receiving end does not support push options" \
-		actual &&
+	grep "fatal: the receiving end does not support push options" actual &&
 	git -C "$upstream" show-ref >out &&
 	make_user_friendly_and_stable_output <out >actual &&
 	cat >expect <<-EOF &&
t/t5411/test-0027-push-options--porcelain.sh +1 -2
index 3a6561b5ea..e734247d69 100644
@@ -19,8 +19,7 @@ test_expect_success "proc-receive: not support push options ($PROTOCOL/porcelain
 		HEAD:refs/for/main/topic \
 		>out 2>&1 &&
 	make_user_friendly_and_stable_output <out >actual &&
-	test_i18ngrep "fatal: the receiving end does not support push options" \
-		actual &&
+	grep "fatal: the receiving end does not support push options" actual &&
 	git -C "$upstream" show-ref >out &&
 	make_user_friendly_and_stable_output <out >actual &&
 	cat >expect <<-EOF &&
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#135 lore raw

PATCH 4/6 tests: (almost) remove use of "test_i18ngrep !"

In reply to Ævar Arnfjörð Bjarmason

As noted in previous commits we are removing uses of test_i18ngrep. Let's start with the more rare negated form. This leaves us with "test_i18ngrep" as a mere wrapper for "grep" at the end.

Done with:

    cd t &&
    perl -pi -e 's/test_i18ngrep ! /! grep /g' *.sh

I excluded those things that would conflict with other in-flight changes on the "seen" branch, as well as not removing support from this invocation from "test_i18ngrep" for those & any other in-flight and soon-to-be-submitted changes. We can do a later final cleanup and removal of the unused code in some later change.

---
 t/t0041-usage.sh                    | 12 +++++-----
 t/t1060-object-corruption.sh        |  2 +-
 t/t1091-sparse-checkout-builtin.sh  | 10 ++++-----
 t/t1450-fsck.sh                     |  4 ++--
 t/t1506-rev-parse-diagnosis.sh      |  2 +-
 t/t2019-checkout-ambiguous-ref.sh   |  4 ++--
 t/t2020-checkout-detach.sh          |  4 ++--
 t/t2024-checkout-dwim.sh            |  2 +-
 t/t3418-rebase-continue.sh          |  2 +-
 t/t3507-cherry-pick-conflict.sh     |  2 +-
 t/t3600-rm.sh                       |  2 +-
 t/t4001-diff-rename.sh              |  2 +-
 t/t4014-format-patch.sh             |  2 +-
 t/t4018-diff-funcname.sh            |  8 +++----
 t/t4153-am-resume-override-opts.sh  |  2 +-
 t/t5324-split-commit-graph.sh       |  2 +-
 t/t5505-remote.sh                   |  2 +-
 t/t5510-fetch.sh                    |  2 +-
 t/t5534-push-signed.sh              |  2 +-
 t/t5541-http-push-smart.sh          |  4 ++--
 t/t5580-unc-paths.sh                |  2 +-
 t/t5606-clone-options.sh            |  2 +-
 t/t6001-rev-list-graft.sh           |  2 +-
 t/t6050-replace.sh                  |  2 +-
 t/t6423-merge-rename-directories.sh | 12 +++++-----
 t/t6433-merge-toplevel.sh           |  2 +-
 t/t6437-submodule-merge.sh          |  2 +-
 t/t6500-gc.sh                       |  2 +-
 t/t7400-submodule-basic.sh          | 16 +++++++-------
 t/t7414-submodule-mistakes.sh       |  6 ++---
 t/t7415-submodule-names.sh          |  2 +-
 t/t7416-submodule-dash-url.sh       |  2 +-
 t/t7502-commit-porcelain.sh         | 10 ++++-----
 t/t7508-status.sh                   |  8 +++----
 t/t7518-ident-corner-cases.sh       |  2 +-
 t/t7519-status-fsmonitor.sh         |  4 ++--
 t/t7520-ignored-hook-warning.sh     |  6 ++---
 t/t7601-merge-pull-config.sh        | 34 ++++++++++++++---------------
 t/t7810-grep.sh                     |  2 +-
 t/t7816-grep-binary-pattern.sh      |  2 +-
 t/t9800-git-p4-basic.sh             |  2 +-
 t/t9807-git-p4-submit.sh            |  2 +-
 42 files changed, 98 insertions(+), 98 deletions(-)
t/t0041-usage.sh +6 -6
index 5b927b76fe..7d84671941 100755
@@ -18,7 +18,7 @@ test_expect_success 'tag --contains <inexistent_tag>' '
 	test_must_fail git tag --contains "notag" >actual 2>actual.err &&
 	test_line_count = 0 actual &&
 	test_i18ngrep "error" actual.err &&
-	test_i18ngrep ! "usage" actual.err
+	! grep "usage" actual.err
 '
 
 test_expect_success 'tag --no-contains <existent_tag>' '
@@ -31,7 +31,7 @@ test_expect_success 'tag --no-contains <inexistent_tag>' '
 	test_must_fail git tag --no-contains "notag" >actual 2>actual.err &&
 	test_line_count = 0 actual &&
 	test_i18ngrep "error" actual.err &&
-	test_i18ngrep ! "usage" actual.err
+	! grep "usage" actual.err
 '
 
 test_expect_success 'tag usage error' '
@@ -50,7 +50,7 @@ test_expect_success 'branch --contains <inexistent_commit>' '
 	test_must_fail git branch --no-contains "nocommit" >actual 2>actual.err &&
 	test_line_count = 0 actual &&
 	test_i18ngrep "error" actual.err &&
-	test_i18ngrep ! "usage" actual.err
+	! grep "usage" actual.err
 '
 
 test_expect_success 'branch --no-contains <existent_commit>' '
@@ -63,7 +63,7 @@ test_expect_success 'branch --no-contains <inexistent_commit>' '
 	test_must_fail git branch --no-contains "nocommit" >actual 2>actual.err &&
 	test_line_count = 0 actual &&
 	test_i18ngrep "error" actual.err &&
-	test_i18ngrep ! "usage" actual.err
+	! grep "usage" actual.err
 '
 
 test_expect_success 'branch usage error' '
@@ -82,7 +82,7 @@ test_expect_success 'for-each-ref --contains <inexistent_object>' '
 	test_must_fail git for-each-ref --no-contains "noobject" >actual 2>actual.err &&
 	test_line_count = 0 actual &&
 	test_i18ngrep "error" actual.err &&
-	test_i18ngrep ! "usage" actual.err
+	! grep "usage" actual.err
 '
 
 test_expect_success 'for-each-ref --no-contains <existent_object>' '
@@ -95,7 +95,7 @@ test_expect_success 'for-each-ref --no-contains <inexistent_object>' '
 	test_must_fail git for-each-ref --no-contains "noobject" >actual 2>actual.err &&
 	test_line_count = 0 actual &&
 	test_i18ngrep "error" actual.err &&
-	test_i18ngrep ! "usage" actual.err
+	! grep "usage" actual.err
 '
 
 test_expect_success 'for-each-ref usage error' '
t/t1060-object-corruption.sh +1 -1
index bc89371f53..18dd448c7a 100755
@@ -123,7 +123,7 @@ test_expect_success 'fetch into corrupted repo with index-pack' '
 		cd bit-error-cp &&
 		test_must_fail git -c transfer.unpackLimit=1 \
 			fetch ../no-bit-error 2>stderr &&
-		test_i18ngrep ! -i collision stderr
+		! grep -i collision stderr
 	)
 '
 
t/t1091-sparse-checkout-builtin.sh +5 -5
index 84acfc48b6..e0b3c7e5e4 100755
@@ -180,7 +180,7 @@ test_expect_success 'cone mode: match patterns' '
 	git -C repo config --worktree core.sparseCheckoutCone true &&
 	rm -rf repo/a repo/folder1 repo/folder2 &&
 	git -C repo read-tree -mu HEAD 2>err &&
-	test_i18ngrep ! "disabling cone patterns" err &&
+	! grep "disabling cone patterns" err &&
 	git -C repo reset --hard &&
 	check_files repo a folder1 folder2
 '
@@ -323,8 +323,8 @@ test_expect_success 'revert to old sparse-checkout on empty update' '
 		git add file &&
 		git commit -m "test" &&
 		git sparse-checkout set nothing 2>err &&
-		test_i18ngrep ! "Sparse checkout leaves no entry on working directory" err &&
-		test_i18ngrep ! ".git/index.lock" err &&
+		! grep "Sparse checkout leaves no entry on working directory" err &&
+		! grep ".git/index.lock" err &&
 		git sparse-checkout set file
 	)
 '
@@ -340,7 +340,7 @@ test_expect_success '.gitignore should not warn about cone mode' '
 	git -C repo config --worktree core.sparseCheckoutCone true &&
 	echo "**/bin/*" >repo/.gitignore &&
 	git -C repo reset --hard 2>err &&
-	test_i18ngrep ! "disabling cone patterns" err
+	! grep "disabling cone patterns" err
 '
 
 test_expect_success 'sparse-checkout (init|set|disable) warns with dirty status' '
@@ -417,7 +417,7 @@ test_expect_success 'sparse-checkout reapply' '
 
 	git -C tweak checkout HEAD deep/deeper2/a &&
 	git -C tweak sparse-checkout reapply 2>err &&
-	test_i18ngrep ! "warning.*The following paths are not up to date" err &&
+	! grep "warning.*The following paths are not up to date" err &&
 	test_path_is_missing tweak/deep/deeper2/a &&
 	test_i18ngrep "warning.*The following paths are unmerged" err &&
 	test_path_is_file tweak/folder1/a &&
t/t1450-fsck.sh +2 -2
index b17f5c21fb..884b33569c 100755
@@ -301,7 +301,7 @@ test_expect_success 'unparseable tree object' '
 	test_must_fail git fsck 2>out &&
 	test_i18ngrep "error: empty filename in tree entry" out &&
 	test_i18ngrep "$tree_sha1" out &&
-	test_i18ngrep ! "fatal: empty filename in tree entry" out
+	! grep "fatal: empty filename in tree entry" out
 '
 
 test_expect_success 'tree entry with type mismatch' '
@@ -319,7 +319,7 @@ test_expect_success 'tree entry with type mismatch' '
 	git update-ref refs/heads/type_mismatch $commit &&
 	test_must_fail git fsck >out 2>&1 &&
 	test_i18ngrep "is a blob, not a tree" out &&
-	test_i18ngrep ! "dangling blob" out
+	! grep "dangling blob" out
 '
 
 test_expect_success 'tag pointing to nonexistent' '
t/t1506-rev-parse-diagnosis.sh +1 -1
index e2ae15a2cf..80006b3707 100755
@@ -167,7 +167,7 @@ test_expect_success 'relative path when cwd is outside worktree' '
 
 test_expect_success '<commit>:file correctly diagnosed after a pathname' '
 	test_must_fail git rev-parse file.txt HEAD:file.txt 1>actual 2>error &&
-	test_i18ngrep ! "exists on disk" error &&
+	! grep "exists on disk" error &&
 	test_i18ngrep "no such path in the working tree" error &&
 	cat >expect <<-\EOF &&
 	file.txt
t/t2019-checkout-ambiguous-ref.sh +2 -2
index b99d5192a9..2a46257c76 100755
@@ -31,7 +31,7 @@ test_expect_success 'checkout chooses branch over tag' '
 
 test_expect_success 'checkout reports switch to branch' '
 	test_i18ngrep "Switched to branch" stderr &&
-	test_i18ngrep ! "^HEAD is now at" stderr
+	! grep "^HEAD is now at" stderr
 '
 
 test_expect_success 'checkout vague ref succeeds' '
@@ -53,7 +53,7 @@ test_expect_success VAGUENESS_SUCCESS 'checkout chooses branch over tag' '
 
 test_expect_success VAGUENESS_SUCCESS 'checkout reports switch to branch' '
 	test_i18ngrep "Switched to branch" stderr &&
-	test_i18ngrep ! "^HEAD is now at" stderr
+	! grep "^HEAD is now at" stderr
 '
 
 test_done
t/t2020-checkout-detach.sh +2 -2
index b748db9946..552dd9f505 100755
@@ -14,10 +14,10 @@ check_not_detached () {
 PREV_HEAD_DESC='Previous HEAD position was'
 check_orphan_warning() {
 	test_i18ngrep "you are leaving $2 behind" "$1" &&
-	test_i18ngrep ! "$PREV_HEAD_DESC" "$1"
+	! grep "$PREV_HEAD_DESC" "$1"
 }
 check_no_orphan_warning() {
-	test_i18ngrep ! "you are leaving .* commit.*behind" "$1" &&
+	! grep "you are leaving .* commit.*behind" "$1" &&
 	test_i18ngrep "$PREV_HEAD_DESC" "$1"
 }
 
t/t2024-checkout-dwim.sh +1 -1
index a4f8d3a67e..6e5042c606 100755
@@ -118,7 +118,7 @@ test_expect_success PERL 'checkout -p with multiple remotes does not print advic
 	test_might_fail git branch -D foo &&
 
 	git checkout -p foo 2>stderr &&
-	test_i18ngrep ! "^hint: " stderr &&
+	! grep "^hint: " stderr &&
 	status_uno_is_clean
 '
 
t/t3418-rebase-continue.sh +1 -1
index 7a2da972fd..958653aebf 100755
@@ -178,7 +178,7 @@ test_expect_success '--skip after failed fixup cleans commit message' '
 
 	(test_set_editor "$PWD/copy-editor.sh" && git rebase --skip) &&
 	git show HEAD >out &&
-	test_i18ngrep ! "# This is a combination" out &&
+	! grep "# This is a combination" out &&
 
 	: Final squash failed, but there was still a squash &&
 	test_i18ngrep "# This is a combination of 2 commits" .git/copy.txt &&
t/t3507-cherry-pick-conflict.sh +1 -1
index a21adcf0e4..2cc7338b75 100755
@@ -555,7 +555,7 @@ test_expect_success 'cherry-pick preserves sparse-checkout' '
 	echo /unrelated >.git/info/sparse-checkout &&
 	git read-tree --reset -u HEAD &&
 	test_must_fail git cherry-pick -Xours picked>actual &&
-	test_i18ngrep ! "Changes not staged for commit:" actual
+	! grep "Changes not staged for commit:" actual
 '
 
 test_expect_success 'cherry-pick --continue remembers --keep-redundant-commits' '
t/t3600-rm.sh +1 -1
index efec8d13b6..aa407095e9 100755
@@ -264,7 +264,7 @@ test_expect_success 'Resolving by removal is not a warning-worthy event' '
 		echo "100644 $blob $stage	blob"
 	done | git update-index --index-info &&
 	git rm blob >msg 2>&1 &&
-	test_i18ngrep ! "needs merge" msg &&
+	! grep "needs merge" msg &&
 	test_must_fail git ls-files -s --error-unmatch blob
 '
 
t/t4001-diff-rename.sh +1 -1
index c16486a9d4..9fa37836c0 100755
@@ -145,7 +145,7 @@ test_expect_success 'test diff.renames=true for git status' '
 
 test_expect_success 'test diff.renames=false for git status' '
 	git -c diff.renames=false status >out &&
-	test_i18ngrep ! "renamed: .*path1 -> subdir/path1" out &&
+	! grep "renamed: .*path1 -> subdir/path1" out &&
 	test_i18ngrep "new file: .*subdir/path1" out &&
 	test_i18ngrep "deleted: .*[^/]path1" out
 '
t/t4014-format-patch.sh +1 -1
index c5e5e0da3f..dce16a4741 100755
@@ -2242,7 +2242,7 @@ test_expect_success 'interdiff: cover-letter' '
 	EOF
 	git format-patch --cover-letter --interdiff=boop~2 -1 boop &&
 	test_i18ngrep "^Interdiff:$" 0000-cover-letter.patch &&
-	test_i18ngrep ! "^Interdiff:$" 0001-fleep.patch &&
+	! grep "^Interdiff:$" 0001-fleep.patch &&
 	sed "1,/^@@ /d; /^-- $/q" 0000-cover-letter.patch >actual &&
 	test_cmp expect actual
 '
t/t4018-diff-funcname.sh +4 -4
index 9675bc17db..b75b6358d1 100755
@@ -60,15 +60,15 @@ do
 		echo "*.java diff=$p" >.gitattributes &&
 		test_expect_code 1 git diff --no-index \
 			A.java B.java 2>msg &&
-		test_i18ngrep ! fatal msg &&
-		test_i18ngrep ! error msg
+		! grep fatal msg &&
+		! grep error msg
 	'
 	test_expect_success "builtin $p wordRegex pattern compiles" '
 		echo "*.java diff=$p" >.gitattributes &&
 		test_expect_code 1 git diff --no-index --word-diff \
 			A.java B.java 2>msg &&
-		test_i18ngrep ! fatal msg &&
-		test_i18ngrep ! error msg
+		! grep fatal msg &&
+		! grep error msg
 	'
 done
 
t/t4153-am-resume-override-opts.sh +1 -1
index 8ea22d1bcb..74d4e07632 100755
@@ -53,7 +53,7 @@ test_expect_success '--no-quiet overrides --quiet' '
 	# Applying side1 will be quiet.
 	test_must_fail git am --quiet side[123].eml >out &&
 	test_path_is_dir .git/rebase-apply &&
-	test_i18ngrep ! "^Applying: " out &&
+	! grep "^Applying: " out &&
 	echo side1 >file &&
 	git add file &&
 
t/t5324-split-commit-graph.sh +1 -1
index 4d3842b83b..5dc03be6e6 100755
@@ -342,7 +342,7 @@ test_expect_success 'add octopus merge' '
 	git commit-graph write --reachable --split &&
 	git commit-graph verify --progress 2>err &&
 	test_line_count = 3 err &&
-	test_i18ngrep ! warning err &&
+	! grep warning err &&
 	test_line_count = 3 $graphdir/commit-graph-chain
 '
 
t/t5505-remote.sh +1 -1
index ecadf02d64..8db6c1feaf 100755
@@ -1341,7 +1341,7 @@ test_expect_success 'unqualified <dst> refspec DWIM and advice' '
 			test_must_fail git -c advice.pushUnqualifiedRefName=false \
 				push origin $oid:dst 2>err &&
 			test_i18ngrep "error: The destination you" err &&
-			test_i18ngrep ! "hint: Did you mean" err ||
+			! grep "hint: Did you mean" err ||
 			exit_with=false
 		done &&
 		$exit_with
t/t5510-fetch.sh +1 -1
index 2013051a64..bfb282744e 100755
@@ -993,7 +993,7 @@ test_expect_success '--no-show-forced-updates' '
 	(
 		cd no-forced-update-clone &&
 		git fetch --no-show-forced-updates origin 2>output &&
-		test_i18ngrep ! "(forced update)" output
+		! grep "(forced update)" output
 	)
 '
 
t/t5534-push-signed.sh +1 -1
index af0385fb89..a9a4df21b2 100755
@@ -284,7 +284,7 @@ test_expect_success GPG 'failed atomic push does not execute GPG' '
 			--signed --atomic --porcelain \
 			dst noop ff noff >out 2>err &&
 
-	test_i18ngrep ! "gpg failed to sign" err &&
+	! grep "gpg failed to sign" err &&
 	cat >expect <<-EOF &&
 	To dst
 	=	refs/heads/noop:refs/heads/noop	[up to date]
t/t5541-http-push-smart.sh +2 -2
index 187454f5dd..6969744d93 100755
@@ -321,7 +321,7 @@ test_expect_success TTY 'push --no-progress silences progress but not status' '
 	test_commit no-progress &&
 	test_terminal git push --no-progress >output 2>&1 &&
 	test_i18ngrep "^To http" output &&
-	test_i18ngrep ! "^Writing objects" output
+	! grep "^Writing objects" output
 '
 
 test_expect_success 'push --progress shows progress to non-tty' '
@@ -503,7 +503,7 @@ test_expect_success 'colorize errors/hints' '
 	test_i18ngrep "<RED>.*rejected.*<RESET>" decoded &&
 	test_i18ngrep "<RED>error: failed to push some refs" decoded &&
 	test_i18ngrep "<YELLOW>hint: " decoded &&
-	test_i18ngrep ! "^hint: " decoded
+	! grep "^hint: " decoded
 '
 
 test_done
t/t5580-unc-paths.sh +1 -1
index cf768b3a27..c143989456 100755
@@ -71,7 +71,7 @@ test_expect_success push '
 test_expect_success MINGW 'remote nick cannot contain backslashes' '
 	BACKSLASHED="$(winpwd | tr / \\\\)" &&
 	git ls-remote "$BACKSLASHED" 2>err &&
-	test_i18ngrep ! "unable to access" err
+	! grep "unable to access" err
 '
 
 test_expect_success 'unc alternates' '
t/t5606-clone-options.sh +1 -1
index 7f082fb23b..2eb221adb3 100755
@@ -89,7 +89,7 @@ test_expect_success 'redirected clone does not show progress' '
 
 	git clone "file://$(pwd)/parent" clone-redirected >out 2>err &&
 	! grep % err &&
-	test_i18ngrep ! "Checking connectivity" err
+	! grep "Checking connectivity" err
 
 '
 
t/t6001-rev-list-graft.sh +1 -1
index 7504ba4751..5af149f58a 100755
@@ -116,7 +116,7 @@ test_expect_success 'show advice that grafts are deprecated' '
 	test_i18ngrep "git replace" err &&
 	test_config advice.graftFileDeprecated false &&
 	git show HEAD 2>err &&
-	test_i18ngrep ! "git replace" err
+	! grep "git replace" err
 '
 
 test_done
t/t6050-replace.sh +1 -1
index c80dc10b8f..a6641e8874 100755
@@ -489,7 +489,7 @@ test_expect_success '--convert-graft-file' '
 	git status 2>stderr &&
 	test_i18ngrep "hint:.*grafts is deprecated" stderr &&
 	git replace --convert-graft-file 2>stderr &&
-	test_i18ngrep ! "hint:.*grafts is deprecated" stderr &&
+	! grep "hint:.*grafts is deprecated" stderr &&
 	test_path_is_missing .git/info/grafts &&
 
 	: verify that the history is now "grafted" &&
t/t6423-merge-rename-directories.sh +6 -6
index 4ab133f489..089f220160 100755
@@ -591,7 +591,7 @@ test_expect_success '2b: Directory split into two on one side, with equal number
 		git rev-parse >expect \
 			 O:z/b  O:z/c  B:x/d &&
 		test_cmp expect actual &&
-		test_i18ngrep ! "CONFLICT.*directory rename split" out
+		! grep "CONFLICT.*directory rename split" out
 	)
 '
 
@@ -727,7 +727,7 @@ test_expect_success '3b: Avoid implicit rename if involved as source on current
 
 		test_must_fail git -c merge.directoryRenames=true merge -s recursive B^0 >out &&
 		test_i18ngrep CONFLICT.*rename/rename.*z/d.*x/d.*w/d out &&
-		test_i18ngrep ! CONFLICT.*rename/rename.*y/d out &&
+		! grep CONFLICT.*rename/rename.*y/d out &&
 
 		git ls-files -s >out &&
 		test_line_count = 5 out &&
@@ -3606,7 +3606,7 @@ test_expect_merge_algorithm failure success '10e: Does git complain about untrac
 		echo random >z/c &&
 
 		git -c merge.directoryRenames=true merge -s recursive B^0 >out 2>err &&
-		test_i18ngrep ! "following untracked working tree files would be overwritten by merge" err &&
+		! grep "following untracked working tree files would be overwritten by merge" err &&
 
 		git ls-files -s >out &&
 		test_line_count = 3 out &&
@@ -5356,9 +5356,9 @@ test_expect_success '13e: directory rename detection in recursive case' '
 
 		git -c merge.directoryRenames=conflict merge -s recursive C^0 >out 2>err &&
 
-		test_i18ngrep ! CONFLICT out &&
-		test_i18ngrep ! BUG: err &&
-		test_i18ngrep ! core.dumped err &&
+		! grep CONFLICT out &&
+		! grep BUG: err &&
+		! grep core.dumped err &&
 		test_must_be_empty err &&
 
 		git ls-files >paths &&
t/t6433-merge-toplevel.sh +1 -1
index e29c284b9b..8418050d2d 100755
@@ -148,7 +148,7 @@ test_expect_success 'refuse two-project merge by default, quit before --autostas
 	echo change >>one.t &&
 	git diff >expect &&
 	test_must_fail git merge --autostash five 2>err &&
-	test_i18ngrep ! "stash" err &&
+	! grep "stash" err &&
 	git diff >actual &&
 	test_cmp expect actual
 '
t/t6437-submodule-merge.sh +1 -1
index 3ead2b726f..e42a96cad4 100755
@@ -446,7 +446,7 @@ test_expect_failure !FAIL_PREREQS 'directory/submodule conflict; should not trea
 		# We do not want files within the submodule to prevent the
 		# merge from starting; we should not be writing to such paths
 		# anyway.
-		test_i18ngrep ! "refusing to lose untracked file at" err
+		! grep "refusing to lose untracked file at" err
 	)
 '
 
t/t6500-gc.sh +1 -1
index 4a3b8f48ac..8670da6bf9 100755
@@ -111,7 +111,7 @@ test_expect_success 'auto gc with too many loose objects does not attempt to cre
 	test_commit "$(test_oid obj4)" &&
 
 	git gc --auto 2>err &&
-	test_i18ngrep ! "^warning:" err &&
+	! grep "^warning:" err &&
 	ls .git/objects/pack/ | sort >post_packs &&
 	comm -1 -3 existing_packs post_packs >new &&
 	comm -2 -3 existing_packs post_packs >del &&
t/t7400-submodule-basic.sh +8 -8
index fec7e0299d..113b25892b 100755
@@ -158,7 +158,7 @@ test_expect_success 'redirected submodule add does not show progress' '
 	git -C addtest submodule add "file://$submodurl/parent" submod-redirected \
 		2>err &&
 	! grep % err &&
-	test_i18ngrep ! "Checking connectivity" err
+	! grep "Checking connectivity" err
 '
 
 test_expect_success 'redirected submodule add --progress does show progress' '
@@ -1101,7 +1101,7 @@ test_expect_success 'submodule deinit deinits a submodule when its work tree is
 	git submodule deinit init example2 >actual &&
 	test -z "$(git config --get-regexp "submodule\.example\.")" &&
 	test -z "$(git config --get-regexp "submodule\.example2\.")" &&
-	test_i18ngrep ! "Cleared directory .init" actual &&
+	! grep "Cleared directory .init" actual &&
 	test_i18ngrep "Cleared directory .example2" actual &&
 	rmdir init
 '
@@ -1151,19 +1151,19 @@ test_expect_success 'submodule deinit is silent when used on an uninitialized su
 	test_i18ngrep "Submodule .example. (.*) unregistered for path .init" actual &&
 	test_i18ngrep "Cleared directory .init" actual &&
 	git submodule deinit init >actual &&
-	test_i18ngrep ! "Submodule .example. (.*) unregistered for path .init" actual &&
+	! grep "Submodule .example. (.*) unregistered for path .init" actual &&
 	test_i18ngrep "Cleared directory .init" actual &&
 	git submodule deinit . >actual &&
-	test_i18ngrep ! "Submodule .example. (.*) unregistered for path .init" actual &&
+	! grep "Submodule .example. (.*) unregistered for path .init" actual &&
 	test_i18ngrep "Submodule .example2. (.*) unregistered for path .example2" actual &&
 	test_i18ngrep "Cleared directory .init" actual &&
 	git submodule deinit . >actual &&
-	test_i18ngrep ! "Submodule .example. (.*) unregistered for path .init" actual &&
-	test_i18ngrep ! "Submodule .example2. (.*) unregistered for path .example2" actual &&
+	! grep "Submodule .example. (.*) unregistered for path .init" actual &&
+	! grep "Submodule .example2. (.*) unregistered for path .example2" actual &&
 	test_i18ngrep "Cleared directory .init" actual &&
 	git submodule deinit --all >actual &&
-	test_i18ngrep ! "Submodule .example. (.*) unregistered for path .init" actual &&
-	test_i18ngrep ! "Submodule .example2. (.*) unregistered for path .example2" actual &&
+	! grep "Submodule .example. (.*) unregistered for path .init" actual &&
+	! grep "Submodule .example2. (.*) unregistered for path .example2" actual &&
 	test_i18ngrep "Cleared directory .init" actual &&
 	rmdir init example2
 '
t/t7414-submodule-mistakes.sh +3 -3
index f2e7df59cf..2bae75228e 100755
@@ -17,7 +17,7 @@ test_expect_success 'git-add on embedded repository warns' '
 test_expect_success '--no-warn-embedded-repo suppresses warning' '
 	test_when_finished "git rm --cached -f embed" &&
 	git add --no-warn-embedded-repo embed 2>stderr &&
-	test_i18ngrep ! warning stderr
+	! grep warning stderr
 '
 
 test_expect_success 'no warning when updating entry' '
@@ -25,13 +25,13 @@ test_expect_success 'no warning when updating entry' '
 	git add embed &&
 	git -C embed commit --allow-empty -m two &&
 	git add embed 2>stderr &&
-	test_i18ngrep ! warning stderr
+	! grep warning stderr
 '
 
 test_expect_success 'submodule add does not warn' '
 	test_when_finished "git rm -rf submodule .gitmodules" &&
 	git submodule add ./embed submodule 2>stderr &&
-	test_i18ngrep ! warning stderr
+	! grep warning stderr
 '
 
 test_done
t/t7415-submodule-names.sh +1 -1
index f70368bc2e..5228c88b9f 100755
@@ -187,7 +187,7 @@ test_expect_success 'fsck detects corrupt .gitmodules' '
 
 		git fsck 2>output &&
 		test_i18ngrep gitmodulesParse output &&
-		test_i18ngrep ! "bad config" output
+		! grep "bad config" output
 	)
 '
 
t/t7416-submodule-dash-url.sh +1 -1
index eec96e0ba9..80b36914c4 100755
@@ -57,7 +57,7 @@ test_expect_success 'trailing backslash is handled correctly' '
 	mv .new .gitmodules &&
 	git commit -am "Add testmodule" &&
 	test_must_fail git clone --verbose --recurse-submodules . dolly 2>err &&
-	test_i18ngrep ! "unknown option" err
+	! grep "unknown option" err
 '
 
 test_expect_success 'fsck rejects missing URL scheme' '
t/t7502-commit-porcelain.sh +5 -5
index 14c92e4c25..6639a849c9 100755
@@ -526,7 +526,7 @@ try_commit () {
 	GIT_EDITOR=.git/FAKE_EDITOR git commit -a $* $use_template &&
 	case "$use_template" in
 	'')
-		test_i18ngrep ! "^## Custom template" .git/COMMIT_EDITMSG ;;
+		! grep "^## Custom template" .git/COMMIT_EDITMSG ;;
 	*)
 		test_i18ngrep "^## Custom template" .git/COMMIT_EDITMSG ;;
 	esac
@@ -546,7 +546,7 @@ try_commit_status_combo () {
 
 	test_expect_success 'commit --no-status' '
 		try_commit --no-status &&
-		test_i18ngrep ! "^# Changes to be committed:" .git/COMMIT_EDITMSG
+		! grep "^# Changes to be committed:" .git/COMMIT_EDITMSG
 	'
 
 	test_expect_success 'commit with commit.status = yes' '
@@ -558,7 +558,7 @@ try_commit_status_combo () {
 	test_expect_success 'commit with commit.status = no' '
 		test_config commit.status no &&
 		try_commit "" &&
-		test_i18ngrep ! "^# Changes to be committed:" .git/COMMIT_EDITMSG
+		! grep "^# Changes to be committed:" .git/COMMIT_EDITMSG
 	'
 
 	test_expect_success 'commit --status with commit.status = yes' '
@@ -570,7 +570,7 @@ try_commit_status_combo () {
 	test_expect_success 'commit --no-status with commit.status = yes' '
 		test_config commit.status yes &&
 		try_commit --no-status &&
-		test_i18ngrep ! "^# Changes to be committed:" .git/COMMIT_EDITMSG
+		! grep "^# Changes to be committed:" .git/COMMIT_EDITMSG
 	'
 
 	test_expect_success 'commit --status with commit.status = no' '
@@ -582,7 +582,7 @@ try_commit_status_combo () {
 	test_expect_success 'commit --no-status with commit.status = no' '
 		test_config commit.status no &&
 		try_commit --no-status &&
-		test_i18ngrep ! "^# Changes to be committed:" .git/COMMIT_EDITMSG
+		! grep "^# Changes to be committed:" .git/COMMIT_EDITMSG
 	'
 
 }
t/t7508-status.sh +4 -4
index 963fed6821..091c4dadea 100755
@@ -1547,7 +1547,7 @@ test_expect_success 'git commit will commit a staged but ignored submodule' '
 	export GIT_EDITOR &&
 	git commit -uno &&
 	git status -s --ignore-submodules=dirty >output &&
-	test_i18ngrep ! "^M. sm" output
+	! grep "^M. sm" output
 '
 
 test_expect_success 'git commit --dry-run will show a staged but ignored submodule' '
@@ -1579,7 +1579,7 @@ EOF
 test_expect_success 'git commit -m will commit a staged but ignored submodule' '
 	git commit -uno -m message &&
 	git status -s --ignore-submodules=dirty >output &&
-	test_i18ngrep ! "^M. sm" output &&
+	! grep "^M. sm" output &&
 	git config --remove-section submodule.subname &&
 	git config -f .gitmodules  --remove-section submodule.subname
 '
@@ -1626,7 +1626,7 @@ test_expect_success '"No commits yet" should not be noted in status output' '
 	git checkout --orphan empty-branch-2 &&
 	test_commit test-commit-1 &&
 	git status >output &&
-	test_i18ngrep ! "No commits yet" output
+	! grep "No commits yet" output
 '
 
 test_expect_success '"Initial commit" should be noted in commit template' '
@@ -1643,7 +1643,7 @@ test_expect_success '"Initial commit" should not be noted in commit template' '
 	touch to_be_committed_2 &&
 	git add to_be_committed_2 &&
 	git commit --dry-run >output &&
-	test_i18ngrep ! "Initial commit" output
+	! grep "Initial commit" output
 '
 
 test_expect_success '--no-optional-locks prevents index update' '
t/t7518-ident-corner-cases.sh +1 -1
index 905957bd0a..1310b4381e 100755
@@ -13,7 +13,7 @@ test_expect_success 'empty name and missing email' '
 		sane_unset GIT_AUTHOR_EMAIL &&
 		GIT_AUTHOR_NAME= &&
 		test_must_fail git commit --allow-empty -m foo 2>err &&
-		test_i18ngrep ! "(null)" err
+		! grep "(null)" err
 	)
 '
 
t/t7519-status-fsmonitor.sh +2 -2
index fbfdcca000..f2c1e0b741 100755
@@ -290,10 +290,10 @@ do
 			rm -f marker &&
 			git status >actual &&
 			test_path_is_file marker &&
-			test_i18ngrep ! "Changes not staged for commit:" actual &&
+			! grep "Changes not staged for commit:" actual &&
 			if test $uc_val = true
 			then
-				test_i18ngrep ! "Untracked files:" actual
+				! grep "Untracked files:" actual
 			fi &&
 			if test $uc_val = false
 			then
t/t7520-ignored-hook-warning.sh +3 -3
index 634fb7f23a..06682b20d0 100755
@@ -15,7 +15,7 @@ test_expect_success setup '
 
 test_expect_success 'no warning if hook is not ignored' '
 	git commit --allow-empty -m "more" 2>message &&
-	test_i18ngrep ! -e "hook was ignored" message
+	! grep -e "hook was ignored" message
 '
 
 test_expect_success POSIXPERM 'warning if hook is ignored' '
@@ -28,14 +28,14 @@ test_expect_success POSIXPERM 'no warning if advice.ignoredHook set to false' '
 	test_config advice.ignoredHook false &&
 	chmod -x "$hook" &&
 	git commit --allow-empty -m "even more" 2>message &&
-	test_i18ngrep ! -e "hook was ignored" message
+	! grep -e "hook was ignored" message
 '
 
 test_expect_success 'no warning if unset advice.ignoredHook and hook removed' '
 	rm -f "$hook" &&
 	test_unconfig advice.ignoredHook &&
 	git commit --allow-empty -m "even more" 2>message &&
-	test_i18ngrep ! -e "hook was ignored" message
+	! grep -e "hook was ignored" message
 '
 
 test_done
t/t7601-merge-pull-config.sh +17 -17
index 52e8ccc933..10b392028e 100755
@@ -30,58 +30,58 @@ test_expect_success 'setup' '
 test_expect_success 'pull.rebase not set' '
 	git reset --hard c0 &&
 	git pull . c1 2>err &&
-	test_i18ngrep ! "Pulling without specifying how to reconcile" err
+	! grep "Pulling without specifying how to reconcile" err
 '
 
 test_expect_success 'pull.rebase not set and pull.ff=true' '
 	git reset --hard c0 &&
 	test_config pull.ff true &&
 	git pull . c1 2>err &&
-	test_i18ngrep ! "Pulling without specifying how to reconcile" err
+	! grep "Pulling without specifying how to reconcile" err
 '
 
 test_expect_success 'pull.rebase not set and pull.ff=false' '
 	git reset --hard c0 &&
 	test_config pull.ff false &&
 	git pull . c1 2>err &&
-	test_i18ngrep ! "Pulling without specifying how to reconcile" err
+	! grep "Pulling without specifying how to reconcile" err
 '
 
 test_expect_success 'pull.rebase not set and pull.ff=only' '
 	git reset --hard c0 &&
 	test_config pull.ff only &&
 	git pull . c1 2>err &&
-	test_i18ngrep ! "Pulling without specifying how to reconcile" err
+	! grep "Pulling without specifying how to reconcile" err
 '
 
 test_expect_success 'pull.rebase not set and --rebase given' '
 	git reset --hard c0 &&
 	git pull --rebase . c1 2>err &&
-	test_i18ngrep ! "Pulling without specifying how to reconcile" err
+	! grep "Pulling without specifying how to reconcile" err
 '
 
 test_expect_success 'pull.rebase not set and --no-rebase given' '
 	git reset --hard c0 &&
 	git pull --no-rebase . c1 2>err &&
-	test_i18ngrep ! "Pulling without specifying how to reconcile" err
+	! grep "Pulling without specifying how to reconcile" err
 '
 
 test_expect_success 'pull.rebase not set and --ff given' '
 	git reset --hard c0 &&
 	git pull --ff . c1 2>err &&
-	test_i18ngrep ! "Pulling without specifying how to reconcile" err
+	! grep "Pulling without specifying how to reconcile" err
 '
 
 test_expect_success 'pull.rebase not set and --no-ff given' '
 	git reset --hard c0 &&
 	git pull --no-ff . c1 2>err &&
-	test_i18ngrep ! "Pulling without specifying how to reconcile" err
+	! grep "Pulling without specifying how to reconcile" err
 '
 
 test_expect_success 'pull.rebase not set and --ff-only given' '
 	git reset --hard c0 &&
 	git pull --ff-only . c1 2>err &&
-	test_i18ngrep ! "Pulling without specifying how to reconcile" err
+	! grep "Pulling without specifying how to reconcile" err
 '
 
 test_expect_success 'pull.rebase not set (not-fast-forward)' '
@@ -96,51 +96,51 @@ test_expect_success 'pull.rebase not set and pull.ff=true (not-fast-forward)' '
 	git reset --hard c2 &&
 	test_config pull.ff true &&
 	git pull . c1 2>err &&
-	test_i18ngrep ! "Pulling without specifying how to reconcile" err
+	! grep "Pulling without specifying how to reconcile" err
 '
 
 test_expect_success 'pull.rebase not set and pull.ff=false (not-fast-forward)' '
 	git reset --hard c2 &&
 	test_config pull.ff false &&
 	git pull . c1 2>err &&
-	test_i18ngrep ! "Pulling without specifying how to reconcile" err
+	! grep "Pulling without specifying how to reconcile" err
 '
 
 test_expect_success 'pull.rebase not set and pull.ff=only (not-fast-forward)' '
 	git reset --hard c2 &&
 	test_config pull.ff only &&
 	test_must_fail git pull . c1 2>err &&
-	test_i18ngrep ! "Pulling without specifying how to reconcile" err
+	! grep "Pulling without specifying how to reconcile" err
 '
 
 test_expect_success 'pull.rebase not set and --rebase given (not-fast-forward)' '
 	git reset --hard c2 &&
 	git pull --rebase . c1 2>err &&
-	test_i18ngrep ! "Pulling without specifying how to reconcile" err
+	! grep "Pulling without specifying how to reconcile" err
 '
 
 test_expect_success 'pull.rebase not set and --no-rebase given (not-fast-forward)' '
 	git reset --hard c2 &&
 	git pull --no-rebase . c1 2>err &&
-	test_i18ngrep ! "Pulling without specifying how to reconcile" err
+	! grep "Pulling without specifying how to reconcile" err
 '
 
 test_expect_success 'pull.rebase not set and --ff given (not-fast-forward)' '
 	git reset --hard c2 &&
 	git pull --ff . c1 2>err &&
-	test_i18ngrep ! "Pulling without specifying how to reconcile" err
+	! grep "Pulling without specifying how to reconcile" err
 '
 
 test_expect_success 'pull.rebase not set and --no-ff given (not-fast-forward)' '
 	git reset --hard c2 &&
 	git pull --no-ff . c1 2>err &&
-	test_i18ngrep ! "Pulling without specifying how to reconcile" err
+	! grep "Pulling without specifying how to reconcile" err
 '
 
 test_expect_success 'pull.rebase not set and --ff-only given (not-fast-forward)' '
 	git reset --hard c2 &&
 	test_must_fail git pull --ff-only . c1 2>err &&
-	test_i18ngrep ! "Pulling without specifying how to reconcile" err
+	! grep "Pulling without specifying how to reconcile" err
 '
 
 test_expect_success 'merge c1 with c2' '
t/t7810-grep.sh +1 -1
index 991d5bd9c0..a7e17b1a5e 100755
@@ -1268,7 +1268,7 @@ test_expect_success 'grep --no-index prefers paths to revs' '
 
 test_expect_success 'grep --no-index does not "diagnose" revs' '
 	test_must_fail git grep --no-index o :1:hello.c 2>err &&
-	test_i18ngrep ! -i "did you mean" err
+	! grep -i "did you mean" err
 '
 
 cat >expected <<EOF
t/t7816-grep-binary-pattern.sh +1 -1
index 60bab291e4..25cbc16446 100755
@@ -25,7 +25,7 @@ nul_match_internal () {
 			>stderr &&
 			printf '$pattern' | q_to_nul >f &&
 			test_must_fail env LC_ALL=\"$lc_all\" git grep $extra_flags -f f $flags a 2>stderr &&
-			test_i18ngrep ! 'This is only supported with -P under PCRE v2' stderr
+			! grep 'This is only supported with -P under PCRE v2' stderr
 		"
 	elif test "$matches" = P
 	then
t/t9800-git-p4-basic.sh +1 -1
index c98c1dfc23..3f656bbbaa 100755
@@ -202,7 +202,7 @@ test_expect_success 'exit when p4 fails to produce marshaled output' '
 		export PATH &&
 		test_expect_code 1 git p4 clone --dest="$git" //depot >errs 2>&1
 	) &&
-	test_i18ngrep ! Traceback errs
+	! grep Traceback errs
 '
 
 # Hide a file from p4d, make sure we catch its complaint.  This won't fail in
t/t9807-git-p4-submit.sh +1 -1
index eaaae414a1..22c8106b13 100755
@@ -460,7 +460,7 @@ test_expect_success 'submit --prepare-p4-only' '
 		git p4 submit --prepare-p4-only >out &&
 		test_i18ngrep "prepared for submission" out &&
 		test_i18ngrep "must be deleted" out &&
-		test_i18ngrep ! "everything below this line is just the diff" out
+		! grep "everything below this line is just the diff" out
 	) &&
 	(
 		cd "$cli" &&
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#136 lore raw

PATCH 5/6 tests: (almost) remove C_LOCALE_OUTPUT prerequisites

In reply to Ævar Arnfjörð Bjarmason

Almost entirely Remove the C_LOCALE_OUTPUT prerequisites and its uses as a part of getting rid of the GIT_TEST_GETTEXT_POISON facility.

As with test_i18ngrep we're not removing the stub C_LOCALE_OUTPUT prerequisite yet for the benefit of other in-flight code.

---
 t/t0000-basic.sh                    |  2 +-
 t/t0020-crlf.sh                     |  6 ++----
 t/t0201-gettext-fallbacks.sh        |  2 +-
 t/t1430-bad-ref-name.sh             |  2 +-
 t/t1512-rev-parse-disambiguation.sh | 14 +++++++-------
 t/t3404-rebase-interactive.sh       | 14 +++++++-------
 t/t3415-rebase-autosquash.sh        | 10 +++++-----
 t/t3701-add-interactive.sh          |  2 +-
 t/t4012-diff-binary.sh              |  4 ++--
 t/t4205-log-pretty-formats.sh       |  2 +-
 t/t5300-pack-object.sh              |  4 ++--
 t/t5505-remote.sh                   |  6 +++---
 t/t5510-fetch.sh                    |  4 ++--
 t/t5601-clone.sh                    |  2 +-
 t/t6423-merge-rename-directories.sh |  2 +-
 t/t7300-clean.sh                    |  6 +++---
 t/t7810-grep.sh                     |  2 +-
 t/t9003-help-autocorrect.sh         |  7 ++-----
 18 files changed, 43 insertions(+), 48 deletions(-)
t/t0000-basic.sh +1 -1
index f4ba2e8c85..45a88315ad 100755
@@ -271,7 +271,7 @@ test_expect_success 'pretend we have a mix of all possible results' "
 	EOF
 "
 
-test_expect_success C_LOCALE_OUTPUT 'test --verbose' '
+test_expect_success 'test --verbose' '
 	run_sub_test_lib_test_err \
 		t1234-verbose "test verbose" --verbose <<-\EOF &&
 	test_expect_success "passing test" true
t/t0020-crlf.sh +2 -4
index b63ba62e5d..7081485e21 100755
@@ -84,10 +84,8 @@ test_expect_success 'safecrlf: print warning only once' '
 	git commit -m "nowarn" &&
 	for w in Oh here is CRLFQ in text; do echo $w; done | q_to_cr >doublewarn &&
 	git add doublewarn 2>err &&
-	if test_have_prereq C_LOCALE_OUTPUT
-	then
-		test $(grep "CRLF will be replaced by LF" err | wc -l) = 1
-	fi
+	grep "CRLF will be replaced by LF" err >err.lines &&
+	test_line_count = 1 err.lines
 '
 
 
t/t0201-gettext-fallbacks.sh +1 -1
index 90da1c7ddc..a7b3e4a2c7 100755
@@ -18,7 +18,7 @@ test_expect_success 'sanity: $GIT_INTERNAL_GETTEXT_TEST_FALLBACKS is set' '
     test -n "$GIT_INTERNAL_GETTEXT_TEST_FALLBACKS"
 '
 
-test_expect_success C_LOCALE_OUTPUT 'sanity: $GIT_INTERNAL_GETTEXT_SH_SCHEME" is fallthrough' '
+test_expect_success 'sanity: $GIT_INTERNAL_GETTEXT_SH_SCHEME" is fallthrough' '
     echo fallthrough >expect &&
     echo $GIT_INTERNAL_GETTEXT_SH_SCHEME >actual &&
     test_cmp expect actual
t/t1430-bad-ref-name.sh +1 -1
index c7878a60ed..138fc9b1c3 100755
@@ -122,7 +122,7 @@ test_expect_success 'push cannot create a badly named ref' '
 	! grep -e "broken\.\.\.ref" output
 '
 
-test_expect_failure C_LOCALE_OUTPUT 'push --mirror can delete badly named ref' '
+test_expect_failure 'push --mirror can delete badly named ref' '
 	top=$(pwd) &&
 	git init src &&
 	git init dest &&
t/t1512-rev-parse-disambiguation.sh +7 -7
index 18fa6cf40d..3a4b337ae0 100755
@@ -311,39 +311,39 @@ test_expect_success 'ambiguous short sha1 ref' '
 	grep "refname.*${REF}.*ambiguous" err
 '
 
-test_expect_success C_LOCALE_OUTPUT 'ambiguity errors are not repeated (raw)' '
+test_expect_success 'ambiguity errors are not repeated (raw)' '
 	test_must_fail git rev-parse 00000 2>stderr &&
 	grep "is ambiguous" stderr >errors &&
 	test_line_count = 1 errors
 '
 
-test_expect_success C_LOCALE_OUTPUT 'ambiguity errors are not repeated (treeish)' '
+test_expect_success 'ambiguity errors are not repeated (treeish)' '
 	test_must_fail git rev-parse 00000:foo 2>stderr &&
 	grep "is ambiguous" stderr >errors &&
 	test_line_count = 1 errors
 '
 
-test_expect_success C_LOCALE_OUTPUT 'ambiguity errors are not repeated (peel)' '
+test_expect_success 'ambiguity errors are not repeated (peel)' '
 	test_must_fail git rev-parse 00000^{commit} 2>stderr &&
 	grep "is ambiguous" stderr >errors &&
 	test_line_count = 1 errors
 '
 
-test_expect_success C_LOCALE_OUTPUT 'ambiguity hints' '
+test_expect_success 'ambiguity hints' '
 	test_must_fail git rev-parse 000000000 2>stderr &&
 	grep ^hint: stderr >hints &&
 	# 16 candidates, plus one intro line
 	test_line_count = 17 hints
 '
 
-test_expect_success C_LOCALE_OUTPUT 'ambiguity hints respect type' '
+test_expect_success 'ambiguity hints respect type' '
 	test_must_fail git rev-parse 000000000^{commit} 2>stderr &&
 	grep ^hint: stderr >hints &&
 	# 5 commits, 1 tag (which is a committish), plus intro line
 	test_line_count = 7 hints
 '
 
-test_expect_success C_LOCALE_OUTPUT 'failed type-selector still shows hint' '
+test_expect_success 'failed type-selector still shows hint' '
 	# these two blobs share the same prefix "ee3d", but neither
 	# will pass for a commit
 	echo 851 | git hash-object --stdin -w &&
@@ -367,7 +367,7 @@ test_expect_success 'core.disambiguate does not override context' '
 		git -c core.disambiguate=committish rev-parse $sha1^{tree}
 '
 
-test_expect_success C_LOCALE_OUTPUT 'ambiguous commits are printed by type first, then hash order' '
+test_expect_success 'ambiguous commits are printed by type first, then hash order' '
 	test_must_fail git rev-parse 0000 2>stderr &&
 	grep ^hint: stderr >hints &&
 	grep 0000 hints >objects &&
t/t3404-rebase-interactive.sh +7 -7
index b06fc36159..c53ae6b5c7 100755
@@ -447,7 +447,7 @@ test_expect_success 'verbose flag is heeded, even after --continue' '
 	grep "^ file1 | 2 +-$" output
 '
 
-test_expect_success C_LOCALE_OUTPUT 'multi-squash only fires up editor once' '
+test_expect_success 'multi-squash only fires up editor once' '
 	base=$(git rev-parse HEAD~4) &&
 	(
 		set_fake_editor &&
@@ -460,7 +460,7 @@ test_expect_success C_LOCALE_OUTPUT 'multi-squash only fires up editor once' '
 	test 1 = $(git show | grep ONCE | wc -l)
 '
 
-test_expect_success C_LOCALE_OUTPUT 'multi-fixup does not fire up editor' '
+test_expect_success 'multi-fixup does not fire up editor' '
 	git checkout -b multi-fixup E &&
 	base=$(git rev-parse HEAD~4) &&
 	(
@@ -511,7 +511,7 @@ test_expect_success 'commit message retained after conflict' '
 	git branch -D conflict-squash
 '
 
-test_expect_success C_LOCALE_OUTPUT 'squash and fixup generate correct log messages' '
+test_expect_success 'squash and fixup generate correct log messages' '
 	cat >expect-squash-fixup <<-\EOF &&
 	B
 
@@ -538,7 +538,7 @@ test_expect_success C_LOCALE_OUTPUT 'squash and fixup generate correct log messa
 	git branch -D squash-fixup
 '
 
-test_expect_success C_LOCALE_OUTPUT 'squash ignores comments' '
+test_expect_success 'squash ignores comments' '
 	git checkout -b skip-comments E &&
 	base=$(git rev-parse HEAD~4) &&
 	(
@@ -554,7 +554,7 @@ test_expect_success C_LOCALE_OUTPUT 'squash ignores comments' '
 	git branch -D skip-comments
 '
 
-test_expect_success C_LOCALE_OUTPUT 'squash ignores blank lines' '
+test_expect_success 'squash ignores blank lines' '
 	git checkout -b skip-blank-lines E &&
 	base=$(git rev-parse HEAD~4) &&
 	(
@@ -992,7 +992,7 @@ test_expect_success 'rebase -ix with several instances of --exec' '
 	test_cmp expected actual
 '
 
-test_expect_success C_LOCALE_OUTPUT 'rebase -ix with --autosquash' '
+test_expect_success 'rebase -ix with --autosquash' '
 	git reset --hard execute &&
 	git checkout -b autosquash &&
 	echo second >second.txt &&
@@ -1133,7 +1133,7 @@ test_expect_success 'rebase -i --root reword root when root has untracked file c
 	test "$(git rev-list --count HEAD)" = 2
 '
 
-test_expect_success C_LOCALE_OUTPUT 'rebase --edit-todo does not work on non-interactive rebase' '
+test_expect_success 'rebase --edit-todo does not work on non-interactive rebase' '
 	git checkout reword-original-root-branch &&
 	git reset --hard &&
 	git checkout conflict-branch &&
t/t3415-rebase-autosquash.sh +5 -5
index 7bab6000dc..e678f26c01 100755
@@ -303,23 +303,23 @@ test_auto_fixup_fixup () {
 	fi
 }
 
-test_expect_success C_LOCALE_OUTPUT 'fixup! fixup!' '
+test_expect_success 'fixup! fixup!' '
 	test_auto_fixup_fixup fixup fixup
 '
 
-test_expect_success C_LOCALE_OUTPUT 'fixup! squash!' '
+test_expect_success 'fixup! squash!' '
 	test_auto_fixup_fixup fixup squash
 '
 
-test_expect_success C_LOCALE_OUTPUT 'squash! squash!' '
+test_expect_success 'squash! squash!' '
 	test_auto_fixup_fixup squash squash
 '
 
-test_expect_success C_LOCALE_OUTPUT 'squash! fixup!' '
+test_expect_success 'squash! fixup!' '
 	test_auto_fixup_fixup squash fixup
 '
 
-test_expect_success C_LOCALE_OUTPUT 'autosquash with custom inst format' '
+test_expect_success 'autosquash with custom inst format' '
 	git reset --hard base &&
 	git config --add rebase.instructionFormat "[%an @ %ar] %s"  &&
 	echo 2 >file1 &&
t/t3701-add-interactive.sh +1 -1
index cc3f434a97..6cdde711e0 100755
@@ -367,7 +367,7 @@ test_expect_success 'setup expected' '
 '
 
 # Test splitting the first patch, then adding both
-test_expect_success C_LOCALE_OUTPUT 'add first line works' '
+test_expect_success 'add first line works' '
 	git commit -am "clear local changes" &&
 	git apply patch &&
 	printf "%s\n" s y y | git add -p file 2>error |
t/t4012-diff-binary.sh +2 -2
index 6579c81216..bd59328e4b 100755
@@ -63,7 +63,7 @@ test_expect_success 'apply --numstat understands diff --binary format' '
 
 # apply needs to be able to skip the binary material correctly
 # in order to report the line number of a corrupt patch.
-test_expect_success C_LOCALE_OUTPUT 'apply detecting corrupt patch correctly' '
+test_expect_success 'apply detecting corrupt patch correctly' '
 	git diff >output &&
 	sed -e "s/-CIT/xCIT/" <output >broken &&
 	test_must_fail git apply --stat --summary broken 2>detected &&
@@ -73,7 +73,7 @@ test_expect_success C_LOCALE_OUTPUT 'apply detecting corrupt patch correctly' '
 	test "$detected" = xCIT
 '
 
-test_expect_success C_LOCALE_OUTPUT 'apply detecting corrupt patch correctly' '
+test_expect_success 'apply detecting corrupt patch correctly' '
 	git diff --binary | sed -e "s/-CIT/xCIT/" >broken &&
 	test_must_fail git apply --stat --summary broken 2>detected &&
 	detected=$(cat detected) &&
t/t4205-log-pretty-formats.sh +1 -1
index 749bc1431a..5ec0f47b56 100755
@@ -126,7 +126,7 @@ test_expect_success 'NUL separation with --stat' '
 	test_i18ncmp expected actual
 '
 
-test_expect_failure C_LOCALE_OUTPUT 'NUL termination with --stat' '
+test_expect_failure 'NUL termination with --stat' '
 	stat0_part=$(git diff --stat HEAD^ HEAD) &&
 	stat1_part=$(git diff-tree --no-commit-id --stat --root HEAD^) &&
 	printf "add bar\n$stat0_part\n\0$(commit_msg)\n$stat1_part\n0" >expected &&
t/t5300-pack-object.sh +2 -2
index 392201cabd..d586fdc7a9 100755
@@ -427,7 +427,7 @@ test_expect_success 'index-pack --strict <pack> works in non-repo' '
 	test_path_is_file foo.idx
 '
 
-test_expect_success !PTHREADS,C_LOCALE_OUTPUT 'index-pack --threads=N or pack.threads=N warns when no pthreads' '
+test_expect_success !PTHREADS 'index-pack --threads=N or pack.threads=N warns when no pthreads' '
 	test_must_fail git index-pack --threads=2 2>err &&
 	grep ^warning: err >warnings &&
 	test_line_count = 1 warnings &&
@@ -445,7 +445,7 @@ test_expect_success !PTHREADS,C_LOCALE_OUTPUT 'index-pack --threads=N or pack.th
 	grep -F "no threads support, ignoring pack.threads" err
 '
 
-test_expect_success !PTHREADS,C_LOCALE_OUTPUT 'pack-objects --threads=N or pack.threads=N warns when no pthreads' '
+test_expect_success !PTHREADS 'pack-objects --threads=N or pack.threads=N warns when no pthreads' '
 	git pack-objects --threads=2 --stdout --all </dev/null >/dev/null 2>err &&
 	grep ^warning: err >warnings &&
 	test_line_count = 1 warnings &&
t/t5505-remote.sh +3 -3
index 8db6c1feaf..3d5360d7cd 100755
@@ -56,7 +56,7 @@ test_expect_success 'add remote whose URL agrees with url.<...>.insteadOf' '
 	git remote add myremote [email protected]:team/repo.git
 '
 
-test_expect_success C_LOCALE_OUTPUT 'remote information for the origin' '
+test_expect_success 'remote information for the origin' '
 	(
 		cd test &&
 		tokens_match origin "$(git remote)" &&
@@ -78,7 +78,7 @@ test_expect_success 'add another remote' '
 	)
 '
 
-test_expect_success C_LOCALE_OUTPUT 'check remote-tracking' '
+test_expect_success 'check remote-tracking' '
 	(
 		cd test &&
 		check_remote_track origin main side &&
@@ -104,7 +104,7 @@ test_expect_success 'remove remote' '
 	)
 '
 
-test_expect_success C_LOCALE_OUTPUT 'remove remote' '
+test_expect_success 'remove remote' '
 	(
 		cd test &&
 		tokens_match origin "$(git remote)" &&
t/t5510-fetch.sh +2 -2
index bfb282744e..b7233fc8ce 100755
@@ -944,7 +944,7 @@ test_expect_success 'fetching with auto-gc does not lock up' '
 	)
 '
 
-test_expect_success C_LOCALE_OUTPUT 'fetch aligned output' '
+test_expect_success 'fetch aligned output' '
 	git clone . full-output &&
 	test_commit looooooooooooong-tag &&
 	(
@@ -959,7 +959,7 @@ test_expect_success C_LOCALE_OUTPUT 'fetch aligned output' '
 	test_cmp expect actual
 '
 
-test_expect_success C_LOCALE_OUTPUT 'fetch compact output' '
+test_expect_success 'fetch compact output' '
 	git clone . compact &&
 	test_commit extraaa &&
 	(
t/t5601-clone.sh +1 -1
index 7df3c5373a..d2b02fd25e 100755
@@ -37,7 +37,7 @@ test_expect_success 'clone with excess parameters (2)' '
 
 '
 
-test_expect_success C_LOCALE_OUTPUT 'output from clone' '
+test_expect_success 'output from clone' '
 	rm -fr dst &&
 	git clone -n "file://$(pwd)/src" dst >output 2>&1 &&
 	test $(grep Clon output | wc -l) = 1
t/t6423-merge-rename-directories.sh +1 -1
index 089f220160..2ca70f6274 100755
@@ -2905,7 +2905,7 @@ test_setup_9e () {
 	)
 }
 
-test_expect_success C_LOCALE_OUTPUT '9e: N-to-1 whammo' '
+test_expect_success '9e: N-to-1 whammo' '
 	test_setup_9e &&
 	(
 		cd 9e &&
t/t7300-clean.sh +3 -3
index cb5e34d94c..a74816ca8b 100755
@@ -110,7 +110,7 @@ test_expect_success 'git clean with prefix' '
 
 '
 
-test_expect_success C_LOCALE_OUTPUT 'git clean with relative prefix' '
+test_expect_success 'git clean with relative prefix' '
 
 	mkdir -p build docs &&
 	touch a.out src/part3.c docs/manual.txt obj.o build/lib.so &&
@@ -123,7 +123,7 @@ test_expect_success C_LOCALE_OUTPUT 'git clean with relative prefix' '
 	verbose test "$would_clean" = ../src/part3.c
 '
 
-test_expect_success C_LOCALE_OUTPUT 'git clean with absolute path' '
+test_expect_success 'git clean with absolute path' '
 
 	mkdir -p build docs &&
 	touch a.out src/part3.c docs/manual.txt obj.o build/lib.so &&
@@ -407,7 +407,7 @@ test_expect_success 'clean.requireForce and -f' '
 
 '
 
-test_expect_success C_LOCALE_OUTPUT 'core.excludesfile' '
+test_expect_success 'core.excludesfile' '
 
 	echo excludes >excludes &&
 	echo included >included &&
t/t7810-grep.sh +1 -1
index a7e17b1a5e..0c4a0e3865 100755
@@ -978,7 +978,7 @@ do
 	"
 done
 
-test_expect_success !PTHREADS,C_LOCALE_OUTPUT 'grep --threads=N or pack.threads=N warns when no pthreads' '
+test_expect_success !PTHREADS 'grep --threads=N or pack.threads=N warns when no pthreads' '
 	git grep --threads=2 Hello hello_world 2>err &&
 	grep ^warning: err >warnings &&
 	test_line_count = 1 warnings &&
t/t9003-help-autocorrect.sh +2 -5
index 03cd5c5423..f00deaf381 100755
@@ -56,11 +56,8 @@ test_expect_success 'autocorrect can be declined altogether' '
 	git config help.autocorrect never &&
 
 	test_must_fail git lfg 2>actual &&
-	if test_have_prereq C_LOCALE_OUTPUT
-	then
-		grep "is not a git command" actual &&
-		test_line_count = 1 actual
-	fi
+	grep "is not a git command" actual &&
+	test_line_count = 1 actual
 '
 
 test_done
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#137 lore raw

PATCH 6/6 tests: remove uses of GIT_TEST_GETTEXT_POISON=false

In reply to Ævar Arnfjörð Bjarmason

As noted in previous commits we are removing the use of GIT_TEST_GETTEXT_POISON=false. These tests all relied on the facility being off, it always is off after an earlier change, but we hadn't removed the redundant assignments to "false" in the tests.

I'm preserving the deletion of "error" lines in 38b9197a76a (t5411:
add basic test cases for proc-receive hook, 2020-08-27), it turns out that's useful even without GIT_TEST_GETTEXT_POISON=true in play. Update a comment added in that commit to note that.

---
 t/t1305-config-include.sh   | 4 +---
 t/t5411/common-functions.sh | 5 ++---
 t/t7201-co.sh               | 2 +-
 t/t9902-completion.sh       | 1 -
 4 files changed, 4 insertions(+), 8 deletions(-)
t/t1305-config-include.sh +1 -3
index f1e1b289f9..f08cfcdfae 100755
@@ -352,9 +352,7 @@ test_expect_success 'include cycles are detected' '
 	git init --bare cycle &&
 	git -C cycle config include.path cycle &&
 	git config -f cycle/cycle include.path config &&
-	test_must_fail \
-		env GIT_TEST_GETTEXT_POISON=false \
-		git -C cycle config --get-all test.value 2>stderr &&
+	test_must_fail git -C cycle config --get-all test.value 2>stderr &&
 	grep "exceeded maximum include depth" stderr
 '
 
t/t5411/common-functions.sh +2 -3
index 344d13f61a..13107fcdb6 100644
@@ -36,9 +36,8 @@ create_commits_in () {
 # without having to worry about future changes of the commit ID and spaces
 # of the output.  Single quotes are replaced with double quotes, because
 # it is boring to prepare unquoted single quotes in expect text.  We also
-# remove some locale error messages, which break test if we turn on
-# `GIT_TEST_GETTEXT_POISON=true` in order to test unintentional translations
-# on plumbing commands.
+# remove some locale error messages. The emitted human-readable errors are
+# redundant to the more machine-readable output the tests already assert.
 make_user_friendly_and_stable_output () {
 	sed \
 		-e "s/  *\$//" \
t/t7201-co.sh +1 -1
index b36a93056f..d10076efd7 100755
@@ -245,7 +245,7 @@ test_expect_success 'checkout to detach HEAD' '
 	rev=$(git rev-parse --short renamer^) &&
 	git checkout -f renamer &&
 	git clean -f &&
-	GIT_TEST_GETTEXT_POISON=false git checkout renamer^ 2>messages &&
+	git checkout renamer^ 2>messages &&
 	grep "HEAD is now at $rev" messages &&
 	test_line_count -gt 1 messages &&
 	H=$(git rev-parse --verify HEAD) &&
t/t9902-completion.sh +0 -1
index a1c4f1f6d4..e5adee27d4 100755
@@ -2363,7 +2363,6 @@ test_expect_success 'sourcing the completion script clears cached commands' '
 '
 
 test_expect_success 'sourcing the completion script clears cached merge strategies' '
-	GIT_TEST_GETTEXT_POISON=false &&
 	__git_compute_merge_strategies &&
 	verbose test -n "$__git_merge_strategies" &&
 	. "$GIT_BUILD_DIR/contrib/completion/git-completion.bash" &&
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#138 lore raw

Re: Should you use test_i18ngrep or GIT_TEST_GETTEXT_POISON=false?

In reply to Ævar Arnfjörð Bjarmason
Quoted 1 line · Ævar Arnfjörð Bjarmason

What do you think about just removing it? I.e. make setting it a noop?

I have been seeing occasional CI job failures from new tests that forget to use test_i18ngrep. I actually think marking such a grep as "this is looking for a string that is meant for humans" a good way to document the interface and expected end-user interaction, so I am not sure about just removing it.

So after all, test_i18ngrep may make more sense than setting GIT_TEST_GETTEXT_POISON to false. I dunno.

#139 lore raw

Re: Should you use test_i18ngrep or GIT_TEST_GETTEXT_POISON=false?

Quoted 4 lines · Junio C Hamano
Quoted 1 line · Ævar Arnfjörð Bjarmason

What do you think about just removing it? I.e. make setting it a noop?

Quoted 8 lines

I have been seeing occasional CI job failures from new tests that forget to use test_i18ngrep. I actually think marking such a grep as "this is looking for a string that is meant for humans" a good way to document the interface and expected end-user interaction, so I am not sure about just removing it.

So after all, test_i18ngrep may make more sense than setting GIT_TEST_GETTEXT_POISON to false. I dunno.

I agree that test_i18ngrep is preferable to setting GIT_TEST_GETTEXT_POISON. Since it's tied to the comparison itself, I think it's easier to see which messages we're expecting as human-readable. It's also shorter and more readable IMHO (you'd already be calling grep or test_cmp, so it is only a little longer).

I am on the fence on whether the presence of something like test_i18ngrep in the test suite is really serving as a useful indication of what was meant to be translated and what wasn't. It's such an incomplete coverage of the total set of messages we generate that I wouldn't trust it. I'd be more likely to look at the source to see if a message is actually translated, or just assume we follow some basic rules (in general, things to stderr are translatable; I think the plumbing outputs from unpack_trees() were really the exception).

So I'm open to the idea that the whole poison mechanism has just outlived its usefulness.

-Peff

#140 lore raw

Re: Should you use test_i18ngrep or GIT_TEST_GETTEXT_POISON=false?

In reply to Ævar Arnfjörð Bjarmason
Quoted 58 lines · Ævar Arnfjörð Bjarmason
Quoted 8 lines · SZEDER Gábor
Quoted 6 lines · Ævar Arnfjörð Bjarmason

We need to provide GIT_TEST_GETTEXT_POISON=false here because the "invalid object type" error is emitted by parse_loose_header_extended(), which has that message already marked for translation. Another option would be to use test_i18ngrep, but I prefer always running the test, not skipping it under gettext poison testing.

Quoted 4 lines

This is fairly unconvincing. Why do you prefer it that way? What is so special in these tests of 'git mktags' that could possibly warrant this special treatment WRT gettext poisoning, as opposed to the other ~1500 invocations of test_i18n{grep,cmp}?

You prodded me about this and Johannes also did off-list. So given that this is already in "next" I think it's best to let usage in this mktag series land as-is.

I promise I'll follow-up with something to (at least start to) fix this generally one way or the other on "master" after that. It's a trivial part of this particular series.

I.e. either document & migrate away from test_i18ngrep (mostly, in some cases it's still convenient), or git rid of this and the few existing GIT_TEST_GETTEXT_POISON=false uses.

Anyway, as both the initial author of this poison facility & of its current runtime incarnation in git.git, some context on it:

The need for test_i18ngrep was always a wart. When I submitted the initial i18n patches a big selling point at the time to a (rightly so) skeptical audience on the Git ML was that it could almost entirely be turned off at compile-time, and that we had something in place to make sure we didn't introduce plumbing bugs by overtranslating.

For a while the mass-i18n patches were about as noisy on-list as the sha256 conversion patches were in more recenty memory.

So if you look through the history you'll see that first we skipped whole tests with the C_LOCALE_OUTPUT prereq, and then test_i18ngrep was a more granular way to do that:

git some-command >output &&
test_i18ngrep "c locale string" output

But since 6cdccfce1e0 (i18n: make GETTEXT_POISON a runtime option, 2018-11-08) this hasn't been needed. But yes, we have a bit under 10 years of test_i18ngrep usage in the test suite now.

But that stuff is the wart, and purely a historical artifact of this having once been compile-time only. We don't do this sort of thing with any other GIT_TEST_* option, e.g. we have tests like:

    t/t5512-ls-remote.sh:   GIT_TEST_PROTOCOL_VERSION=0 git ls-remote --symref --heads . >actual &&
    t/t5512-ls-remote.sh-   test_cmp expect actual &&
Quoted 8 lines

Not:

    t/t5512-ls-remote.sh:   git ls-remote --symref --heads . >actual &&
    t/t5512-ls-remote.sh-   test_protocolv0cmp expect actual &&

Because that doesn't make sense. If we know we're testing protocol v0 here why wouldn't we just pass that as a parameter in the test? Same for GIT_TEST_GETTEXT_POISON=false.

I think it's the comparison of GIT_TEST_GETTEXT_POISON and GIT_TEST_PROTOCOL_VERSION that doesn't make sense in the first place.

For the sake of users who might be stuck on Git version not yet understanding protocol v2 or have to connect to a v0-only remote, we must make sure that both v0 and v2 protocols work, so a simple 'make test' must exercise both. Since protocol v2 is the default nowadays, we need a way to tell some tests to use v0 instead, and setting GIT_TEST_PROTOCOL_VERSION=0 ended up our standard way to do that. Besides, the uses of GIT_TEST_PROTOCOL_VERSION is quite contained, as it is set in only 34 times in 13 test scripts.

GETTEXT_POISON is purely a developer aid (of questionable usefulness as you explained above), no end user would ever see it, and yet the test_i18n* functions are used ~1500 times in 250+ test scripts. I sure dont't want to see 1500+ uses of 'GIT_TEST_GETTEXT_POISON=false git cmd...'. Besides, what would be the purpose of 'GIT_TEST_GETTEXT_POSION=true make test' if it would eventually be overriden all over the place?

Quoted 4 lines

I have also wondered if a better approach at this point isn't to just remove the thing entirely. We're just doing incremental i18n-ization at this point, so the odds that it's catching accidental plumbing translations is rather low compared to 2011-era git.git.

I wouldn't mind seeing it go... well, I think I would welcome it, actually :) I can only recall cases where a failure of the GETTEXT_POISON CI job was resolved by another replacement of 'grep' with 'test_i18ngrep', but I don't seem to remember a case where we decided that "Nah, this message should not have been translated in the first place".

I would, however, really miss the error reporting of 'test_i18ngrep', though, i.e. that it shows the contents of the file when it doesn't find a match.

#141 lore raw

PATCH v5 13/23 mktag tests: test verify_object() with replaced objects

In reply to Ævar Arnfjörð Bjarmason

Add tests to demonstrate what "mktag" does in the face of replaced objects.

There was an existing test for replaced objects fed to "mktag" added in cc400f50112 (mktag: call "check_sha1_signature" with the replacement sha1, 2009-01-23), but that one only tests a commit->commit mapping. Not a mapping to a different type as like we're also testing for here. We could remove the "mktag" test in t6050-replace.sh now if the created tag wasn't being used by a subsequent "fsck" test.

---
 t/t3800-mktag.sh | 35 ++++++++++++++++++++++++++++++++++-
 1 file changed, 34 insertions(+), 1 deletion(-)
t/t3800-mktag.sh +34 -1
index ced540035d4..fbaf648491f 100755
@@ -38,7 +38,11 @@ test_expect_mktag_success() {
 # for the tag.
 test_expect_success 'setup' '
 	test_commit A &&
-	head=$(git rev-parse --verify HEAD)
+	test_commit B &&
+	head=$(git rev-parse --verify HEAD) &&
+	head_parent=$(git rev-parse --verify HEAD~) &&
+	tree=$(git rev-parse HEAD^{tree}) &&
+	blob=$(git rev-parse --verify HEAD:B.t)
 '
 
 ############################################################
@@ -180,6 +184,35 @@ tagger . <> 0 +0000
 
 EOF
 
+check_verify_failure 'verify object (hash/type) check -- mismatched type, valid object' \
+	'^error: char7: could not verify object'
+
+############################################################
+#  9.5. verify object (hash/type) check -- replacement
+
+test_expect_success 'setup replacement of commit -> commit and tree -> blob' '
+	git replace $head_parent $head &&
+	git replace -f $tree $blob
+'
+
+cat >tag.sig <<EOF
+object $head_parent
+type commit
+tag mytag
+tagger . <> 0 +0000
+
+EOF
+
+test_expect_mktag_success 'tag to a commit replaced by another commit'
+
+cat >tag.sig <<EOF
+object $tree
+type tree
+tag mytag
+tagger . <> 0 +0000
+
+EOF
+
 check_verify_failure 'verify object (hash/type) check -- mismatched type, valid object' \
 	'^error: char7: could not verify object'
 
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#142 lore raw

PATCH v5 14/23 mktag: use default strbuf_read() hint

In reply to Ævar Arnfjörð Bjarmason

Change the hardcoded hint of 2^12 to 0. The default strbuf hint is perfectly fine here, and the only reason we were hardcoding it is because it survived migration from a pre-strbuf fixed-sized buffer.

See fd17f5b5f77 (Replace all read_fd use with strbuf_read, and get rid of it., 2007-09-10) for that migration.

---
 builtin/mktag.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
builtin/mktag.c +1 -1
index 4982d3a93ef..ff7ac8e0e5d 100644
@@ -161,7 +161,7 @@ int cmd_mktag(int argc, const char **argv, const char *prefix)
 	if (argc != 1)
 		usage("git mktag");
 
-	if (strbuf_read(&buf, 0, 4096) < 0) {
+	if (strbuf_read(&buf, 0, 0) < 0) {
 		die_errno("could not read from stdin");
 	}
 
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#143 lore raw

PATCH v5 15/23 mktag: remove redundant braces in one-line body "if"

In reply to Ævar Arnfjörð Bjarmason

This minor stylistic churn is usually something we'd avoid, but if we don't do this then the file after changes in subsequent commits will only have this minor style inconsistency, so let's change this while we're at it.

---
 builtin/mktag.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
builtin/mktag.c +1 -2
index ff7ac8e0e5d..97ca5f28b1b 100644
@@ -161,9 +161,8 @@ int cmd_mktag(int argc, const char **argv, const char *prefix)
 	if (argc != 1)
 		usage("git mktag");
 
-	if (strbuf_read(&buf, 0, 0) < 0) {
+	if (strbuf_read(&buf, 0, 0) < 0)
 		die_errno("could not read from stdin");
-	}
 
 	/* Verify it for some basic sanity: it needs to start with
 	   "object <sha1>\ntype\ntagger " */
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#144 lore raw

PATCH v5 16/23 mktag: use puts(str) instead of printf("%s\n", str)

In reply to Ævar Arnfjörð Bjarmason

This introduces no functional change, but refactors the print-out of the hash at the end to do the same thing with less code.

---
 builtin/mktag.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
builtin/mktag.c +1 -1
index 97ca5f28b1b..d89a3c201de 100644
@@ -173,6 +173,6 @@ int cmd_mktag(int argc, const char **argv, const char *prefix)
 		die("unable to write tag file");
 
 	strbuf_release(&buf);
-	printf("%s\n", oid_to_hex(&result));
+	puts(oid_to_hex(&result));
 	return 0;
 }
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#145 lore raw

PATCH v5 17/23 mktag: use fsck instead of custom verify_tag()

In reply to Ævar Arnfjörð Bjarmason

Change the validation logic in "mktag" to use fsck's fsck_tag() instead of its own custom parser. Curiously the logic for both dates back to the same commit[1]. Let's unify them so we're not maintaining two sets functions to verify that a tag is OK.

The behavior of fsck_tag() and the old "mktag" code being removed here is different in few aspects.

I think it makes sense to remove some of those checks, namely:

A. fsck only cares that the timezone matches [-+][0-9]{4}. The mktag
code disallowed values larger than 1400.

Yes there's currently no timezone with a greater offset[2], but since we allow any number of non-offical timezones (e.g. +1234) passing this through seems fine. Git also won't break in the future if e.g. French Polynesia decides it needs to outdo the Line Islands when it comes to timezone extravagance.

B. fsck allows missing author names such as "tagger <email>", mktag
wouldn't, but would allow e.g. "tagger [2 spaces] <email>" (but not "tagger [1 space] <email>"). Now we allow all of these.

C. Like B, but "mktag" disallowed spaces in the <email> part, fsck
allows it.

In some ways fsck_tag() is stricter than "mktag" was, namely:

D. fsck disallows zero-padded dates, but mktag didn't care. So
e.g. the timestamp "0000000000 +0000" produces an error now. A test in "t1006-cat-file.sh" relied on this, it's been changed to use "hash-object" (without fsck) instead.

There was one check I deemed worth keeping by porting it over to fsck_tag():

E. "mktag" did not allow any custom headers, and by extension (as an
empty commit is allowed) also forbade an extra stray trailing newline after the headers it knew about.

Add a new check in the "ignore" category to fsck and use it. This somewhat abuses the facility added in efaba7cc77f (fsck:
optionally ignore specific fsck issues completely, 2015-06-22).

This is somewhat of hack, but probably the least invasive change we can make here. The fsck command will shuffle these categories around, e.g. under --strict the "info" becomes a "warn" and "warn" becomes "error". Existing users of fsck's (and others, e.g. index-pack) --strict option rely on this.

So we need to put something into a category that'll be ignored by all existing users of the API. Pretending that fsck.extraHeaderEntry=error ("ignore" by default) was set serves to do this for us.

1. ec4465adb38 (Add "tag" objects that can be used to sign other objects., 2005-04-25)

2. https://en.wikipedia.org/wiki/List_of_UTC_time_offsets

---
 Documentation/git-mktag.txt |   9 +-
 builtin/mktag.c             | 196 +++++++++---------------------------
 fsck.c                      |  32 +++++-
 fsck.h                      |   9 ++
 t/t1006-cat-file.sh         |   2 +-
 t/t3800-mktag.sh            |  63 ++++++------
 6 files changed, 127 insertions(+), 184 deletions(-)
Documentation/git-mktag.txt +8 -1
index 20af1915e9e..fa070e21a0b 100644
@@ -25,7 +25,14 @@ write a tag found in `my-tag`:
     git hash-object -t tag -w --stdin <my-tag
 
 The difference is that mktag will die before writing the tag if the
-tag doesn't pass a sanity check.
+tag doesn't pass a linkgit:git-fsck[1] check.
+
+The "fsck" check done mktag is stricter than what linkgit:git-fsck[1]
+would run by default in that all `fsck.<msg-id>` messages are promoted
+from warnings to errors (so e.g. a missing "tagger" line is an error).
+
+Extra headers in the object are also an error under mktag, but ignored
+by linkgit:git-fsck[1]
 
 Tag Format
 ----------
builtin/mktag.c +50 -146

220 lines not on this page · raw message

fsck.c +29 -3
index f82e2fe9e30..bed5e20e03b 100644
@@ -80,7 +80,9 @@ static struct oidset gitmodules_done = OIDSET_INIT;
 	/* infos (reported as warnings, but ignored by default) */ \
 	FUNC(GITMODULES_PARSE, INFO) \
 	FUNC(BAD_TAG_NAME, INFO) \
-	FUNC(MISSING_TAGGER_ENTRY, INFO)
+	FUNC(MISSING_TAGGER_ENTRY, INFO) \
+	/* ignored (elevated when requested) */ \
+	FUNC(EXTRA_HEADER_ENTRY, IGNORE)
 
 #define MSG_ID(id, msg_type) FSCK_MSG_##id,
 enum fsck_msg_id {
@@ -911,6 +913,16 @@ static int fsck_tag(const struct object_id *oid, const char *buffer,
 		    unsigned long size, struct fsck_options *options)
 {
 	struct object_id tagged_oid;
+	int tagged_type;
+	return fsck_tag_standalone(oid, buffer, size, options, &tagged_oid,
+				   &tagged_type);
+}
+
+int fsck_tag_standalone(const struct object_id *oid, const char *buffer,
+			unsigned long size, struct fsck_options *options,
+			struct object_id *tagged_oid,
+			int *tagged_type)
+{
 	int ret = 0;
 	char *eol;
 	struct strbuf sb = STRBUF_INIT;
@@ -924,7 +936,7 @@ static int fsck_tag(const struct object_id *oid, const char *buffer,
 		ret = report(options, oid, OBJ_TAG, FSCK_MSG_MISSING_OBJECT, "invalid format - expected 'object' line");
 		goto done;
 	}
-	if (parse_oid_hex(buffer, &tagged_oid, &p) || *p != '\n') {
+	if (parse_oid_hex(buffer, tagged_oid, &p) || *p != '\n') {
 		ret = report(options, oid, OBJ_TAG, FSCK_MSG_BAD_OBJECT_SHA1, "invalid 'object' line format - bad sha1");
 		if (ret)
 			goto done;
@@ -940,7 +952,8 @@ static int fsck_tag(const struct object_id *oid, const char *buffer,
 		ret = report(options, oid, OBJ_TAG, FSCK_MSG_MISSING_TYPE, "invalid format - unexpected end after 'type' line");
 		goto done;
 	}
-	if (type_from_string_gently(buffer, eol - buffer, 1) < 0)
+	*tagged_type = type_from_string_gently(buffer, eol - buffer, 1);
+	if (*tagged_type < 0)
 		ret = report(options, oid, OBJ_TAG, FSCK_MSG_BAD_TYPE, "invalid 'type' value");
 	if (ret)
 		goto done;
@@ -975,6 +988,19 @@ static int fsck_tag(const struct object_id *oid, const char *buffer,
 	else
 		ret = fsck_ident(&buffer, oid, OBJ_TAG, options);
 
+	if (!starts_with(buffer, "\n")) {
+		/*
+		 * The verify_headers() check will allow
+		 * e.g. "[...]tagger <tagger>\nsome
+		 * garbage\n\nmessage" to pass, thinking "some
+		 * garbage" could be a custom header. E.g. "mktag"
+		 * doesn't want any unknown headers.
+		 */
+		ret = report(options, oid, OBJ_TAG, FSCK_MSG_EXTRA_HEADER_ENTRY, "invalid format - extra header(s) after 'tagger'");
+		if (ret)
+			goto done;
+	}
+
 done:
 	strbuf_release(&sb);
 	return ret;
fsck.h +9 -0
index 69cf715e798..29ee4c45e87 100644
@@ -62,6 +62,15 @@ int fsck_walk(struct object *obj, void *data, struct fsck_options *options);
 int fsck_object(struct object *obj, void *data, unsigned long size,
 	struct fsck_options *options);
 
+/*
+ * fsck a tag, and pass info about it back to the caller. This is
+ * exposed fsck_object() internals for git-mktag(1).
+ */
+int fsck_tag_standalone(const struct object_id *oid, const char *buffer,
+			unsigned long size, struct fsck_options *options,
+			struct object_id *tagged_oid,
+			int *tag_type);
+
 /*
  * Some fsck checks are context-dependent, and may end up queued; run this
  * after completing all fsck_object() calls in order to resolve any remaining
t/t1006-cat-file.sh +1 -1
index 2f501d2dc94..5d2dc99b74a 100755
@@ -166,7 +166,7 @@ tag_content="$tag_header_without_timestamp 0000000000 +0000
 
 $tag_description"
 
-tag_sha1=$(echo_without_newline "$tag_content" | git mktag)
+tag_sha1=$(echo_without_newline "$tag_content" | git hash-object -t tag --stdin -w)
 tag_size=$(strlen "$tag_content")
 
 run_tests 'tag' $tag_sha1 $tag_size "$tag_content" "$tag_content" 1
t/t3800-mktag.sh +30 -33

235 lines not on this page · raw message

Signature · 1 line
2.29.2.222.g5d2a92d10f8
#146 lore raw

PATCH v5 18/23 fsck: make fsck_config() re-usable

In reply to Ævar Arnfjörð Bjarmason

Move the fsck_config() function from builtin/fsck.c to fsck.[ch]. This allows for re-using it in other tools that expose fsck logic and want to support its configuration variables.

A logical continuation of this change would be to use a common function for all of {fetch,receive}.fsck.* and fsck.*. See 5d477a334a6 (fsck (receive-pack): allow demoting errors to warnings, 2015-06-22) and my own 1362df0d413 (fetch: implement fetch.fsck.*, 2018-07-27) for the relevant code.

However, those routines want to not parse the fsck.skipList into OIDs, but rather pass them along with the --strict option to another process. It would be possible to refactor that whole thing so we support e.g. a "fetch." prefix, then just keep track of the skiplist as a filename instead of parsing it, and learn to spew that all out from our internal structures into something we can append to the --strict option.

But instead I'm planning to re-use this in "mktag", which'll just re-use these "fsck.*" variables as-is.

---
 builtin/fsck.c | 20 +-------------------
 fsck.c         | 25 +++++++++++++++++++++++++
 fsck.h         |  7 +++++++
 3 files changed, 33 insertions(+), 19 deletions(-)
builtin/fsck.c +1 -19
index fbf26cafcfd..821e7798c70 100644
@@ -73,25 +73,7 @@ static const char *printable_type(const struct object_id *oid,
 
 static int fsck_config(const char *var, const char *value, void *cb)
 {
-	if (strcmp(var, "fsck.skiplist") == 0) {
-		const char *path;
-		struct strbuf sb = STRBUF_INIT;
-
-		if (git_config_pathname(&path, var, value))
-			return 1;
-		strbuf_addf(&sb, "skiplist=%s", path);
-		free((char *)path);
-		fsck_set_msg_types(&fsck_obj_options, sb.buf);
-		strbuf_release(&sb);
-		return 0;
-	}
-
-	if (skip_prefix(var, "fsck.", &var)) {
-		fsck_set_msg_type(&fsck_obj_options, var, value);
-		return 0;
-	}
-
-	return git_default_config(var, value, cb);
+	return fsck_config_internal(var, value, cb, &fsck_obj_options);
 }
 
 static int objerror(struct object *obj, const char *err)
fsck.c +25 -0
index bed5e20e03b..9067a290a2e 100644
@@ -1310,3 +1310,28 @@ int fsck_finish(struct fsck_options *options)
 	oidset_clear(&gitmodules_done);
 	return ret;
 }
+
+int fsck_config_internal(const char *var, const char *value, void *cb,
+			 struct fsck_options *options)
+{
+	if (strcmp(var, "fsck.skiplist") == 0) {
+		const char *path;
+		struct strbuf sb = STRBUF_INIT;
+
+		if (git_config_pathname(&path, var, value))
+			return 1;
+		strbuf_addf(&sb, "skiplist=%s", path);
+		free((char *)path);
+		fsck_set_msg_types(options, sb.buf);
+		strbuf_release(&sb);
+		return 0;
+	}
+
+	if (skip_prefix(var, "fsck.", &var)) {
+		fsck_set_msg_type(options, var, value);
+		return 0;
+	}
+
+	return git_default_config(var, value, cb);
+}
+
fsck.h +7 -0
index 29ee4c45e87..423c467feb7 100644
@@ -103,4 +103,11 @@ void fsck_put_object_name(struct fsck_options *options,
 const char *fsck_describe_object(struct fsck_options *options,
 				 const struct object_id *oid);
 
+/*
+ * git_config() callback for use by fsck-y tools that want to support
+ * fsck.<msg> fsck.skipList etc.
+ */
+int fsck_config_internal(const char *var, const char *value, void *cb,
+			 struct fsck_options *options);
+
 #endif
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#147 lore raw

PATCH v5 19/23 mktag: allow turning off fsck.extraHeaderEntry

In reply to Ævar Arnfjörð Bjarmason

In earlier commits mktag learned to use the fsck machinery, at which point we needed to add fsck.extraHeaderEntry so it could be as strict about extra headers as it's been ever since it was implemented.

But it's not nice to need to switch away from "mktag" to "hash-object"
+ manual "fsck" just because you'd like to have an extra header. So let's support turning it off by getting "fsck.*" variables from the config.

Pedantically speaking it's still not possible to make "mktag" behave just like "hash-object -t tag" does, since we're unconditionally going to check the referenced object in verify_object_in_tag(), which is our own check, and not one that exists in fsck.c.

But the spirit of "this works like fsck" is preserved, in that if you created such a tag with "hash-object" and did a full "fsck" on the repository it would also error out about that invalid object, it just wouldn't emit the same message as fsck does.

---
 Documentation/git-mktag.txt |  5 ++++-
 builtin/mktag.c             | 11 ++++++++++-
 t/t3800-mktag.sh            | 14 ++++++++++++++
 3 files changed, 28 insertions(+), 2 deletions(-)
Documentation/git-mktag.txt +4 -1
index fa070e21a0b..79813ff8df0 100644
@@ -32,7 +32,10 @@ would run by default in that all `fsck.<msg-id>` messages are promoted
 from warnings to errors (so e.g. a missing "tagger" line is an error).
 
 Extra headers in the object are also an error under mktag, but ignored
-by linkgit:git-fsck[1]
+by linkgit:git-fsck[1]. This extra check can be turned off by setting
+the appropriate `fsck.<msg-id>` varible:
+
+    git -c fsck.extraHeaderEntry=ignore mktag <my-tag-with-headers
 
 Tag Format
 ----------
builtin/mktag.c +10 -1
index 4dd35bc79e2..373926d7e0c 100644
@@ -3,6 +3,14 @@
 #include "replace-object.h"
 #include "object-store.h"
 #include "fsck.h"
+#include "config.h"
+
+static struct fsck_options fsck_options = FSCK_OPTIONS_STRICT;
+
+static int mktag_config(const char *var, const char *value, void *cb)
+{
+	return fsck_config_internal(var, value, cb, &fsck_options);
+}
 
 static int mktag_fsck_error_func(struct fsck_options *o,
 				 const struct object_id *oid,
@@ -53,7 +61,6 @@ static int verify_object_in_tag(struct object_id *tagged_oid, int *tagged_type)
 int cmd_mktag(int argc, const char **argv, const char *prefix)
 {
 	struct strbuf buf = STRBUF_INIT;
-	struct fsck_options fsck_options = FSCK_OPTIONS_STRICT;
 	struct object_id tagged_oid;
 	int tagged_type;
 	struct object_id result;
@@ -66,6 +73,8 @@ int cmd_mktag(int argc, const char **argv, const char *prefix)
 
 	fsck_options.error_func = mktag_fsck_error_func;
 	fsck_set_msg_type(&fsck_options, "extraheaderentry", "warn");
+	/* config might set fsck.extraHeaderEntry=* again */
+	git_config(mktag_config, NULL);
 	if (fsck_tag_standalone(NULL, buf.buf, buf.len, &fsck_options,
 				&tagged_oid, &tagged_type))
 		die("tag on stdin did not pass our strict fsck check");
t/t3800-mktag.sh +14 -0
index 0e2a9a883ba..dccf4503235 100755
@@ -412,6 +412,20 @@ EOF
 check_verify_failure 'detect invalid header entry' \
 	'^error:.* extraHeaderEntry:'
 
+test_expect_success 'invalid header entry config & fsck' '
+	test_must_fail git mktag <tag.sig &&
+	test_must_fail git -c fsck.extraHeaderEntry=error mktag <tag.sig &&
+	test_must_fail git -c fsck.extraHeaderEntry=warn mktag <tag.sig &&
+	git -c fsck.extraHeaderEntry=ignore mktag <tag.sig &&
+	git fsck &&
+	env GIT_TEST_GETTEXT_POISON=false \
+		git -c fsck.extraHeaderEntry=warn fsck 2>err &&
+	grep "warning .*extraHeaderEntry:" err &&
+	test_must_fail env GIT_TEST_GETTEXT_POISON=false \
+		git -c fsck.extraHeaderEntry=error 2>err fsck &&
+	grep "error .* extraHeaderEntry:" err
+'
+
 cat >tag.sig <<EOF
 object $head
 type commit
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#148 lore raw

PATCH v5 20/23 mktag: allow omitting the header/body \n separator

In reply to Ævar Arnfjörð Bjarmason

Change mktag's acceptance rules to accept an empty body without an empty line after the header again. This fixes an ancient unintended dregression in "mktag".

When "mktag" was introduced in ec4465adb3 (Add "tag" objects that can be used to sign other objects., 2005-04-25) the input checks were much looser. When it was documented it 6cfec03680 (mktag: minimally update the description., 2007-06-10) it was clearly intended for this \n to be optional:

The message, when [it] exists, is separated by a blank line from the header.

But then in e0aaf781f6 (mktag.c: improve verification of tagger field and tests, 2008-03-27) this was made an error, seemingly by accident. It was just a result of the general header checks, and all the tests after that patch have a trailing empty line (but did not before).

Let's allow this again, and tweak the test semantics changed in e0aaf781f6 to remove the redundant empty line. New tests added in previous commits of mine already added an explicit test for allowing the empty line between header and body.

---
 fsck.c           | 2 ++
 t/t3800-mktag.sh | 6 ++----
 2 files changed, 4 insertions(+), 4 deletions(-)
fsck.c +2 -0
index 9067a290a2e..29c1eaca4cf 100644
@@ -987,6 +987,8 @@ int fsck_tag_standalone(const struct object_id *oid, const char *buffer,
 	}
 	else
 		ret = fsck_ident(&buffer, oid, OBJ_TAG, options);
+	if (!*buffer)
+		goto done;
 
 	if (!starts_with(buffer, "\n")) {
 		/*
t/t3800-mktag.sh +2 -4
index dccf4503235..601b064e97a 100755
@@ -446,7 +446,7 @@ tagger T A Gger <[email protected]> 1206478233 -0500
 
 EOF
 
-test_expect_mktag_success 'require a blank line before an empty body (1)'
+test_expect_mktag_success 'allow a blank line before an empty body (1)'
 
 cat >tag.sig <<EOF
 object $head
@@ -455,8 +455,7 @@ tag mytag
 tagger T A Gger <[email protected]> 1206478233 -0500
 EOF
 
-check_verify_failure 'require a blank line before an empty body (2)' \
-	'^error:.* extraHeaderEntry:'
+test_expect_mktag_success 'allow no blank line before an empty body (2)'
 
 ############################################################
 # 24. create valid tag
@@ -466,7 +465,6 @@ object $head
 type commit
 tag mytag
 tagger T A Gger <[email protected]> 1206478233 -0500
-
 EOF
 
 test_expect_mktag_success 'create valid tag object'
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#149 lore raw

PATCH v5 21/23 mktag: convert to parse-options

In reply to Ævar Arnfjörð Bjarmason

Convert the "mktag" command to use parse-options.h instead of its own ad-hoc argc handling. This doesn't matter much in practice since it doesn't support any options, but removes another special-case in our codebase, and makes it easier to add options to it in the future.

It does marginally improve the situation for programs that want to execute git commands in a consistent manner and e.g. always use --end-of-options. E.g. "gitaly" does that, and has a blacklist of built-ins that don't support --end-of-options. This is one less special case for it and other similar programs to support.

---
 builtin/mktag.c  | 14 ++++++++++++--
 t/t3800-mktag.sh | 12 ++++++++++++
 2 files changed, 24 insertions(+), 2 deletions(-)
builtin/mktag.c +12 -2
index 373926d7e0c..18b8492f4d4 100644
@@ -1,10 +1,16 @@
 #include "builtin.h"
+#include "parse-options.h"
 #include "tag.h"
 #include "replace-object.h"
 #include "object-store.h"
 #include "fsck.h"
 #include "config.h"
 
+static char const * const builtin_mktag_usage[] = {
+	N_("git mktag"),
+	NULL
+};
+
 static struct fsck_options fsck_options = FSCK_OPTIONS_STRICT;
 
 static int mktag_config(const char *var, const char *value, void *cb)
@@ -60,13 +66,17 @@ static int verify_object_in_tag(struct object_id *tagged_oid, int *tagged_type)
 
 int cmd_mktag(int argc, const char **argv, const char *prefix)
 {
+	static struct option builtin_mktag_options[] = {
+		OPT_END(),
+	};
 	struct strbuf buf = STRBUF_INIT;
 	struct object_id tagged_oid;
 	int tagged_type;
 	struct object_id result;
 
-	if (argc != 1)
-		usage("git mktag");
+	argc = parse_options(argc, argv, NULL,
+			     builtin_mktag_options,
+			     builtin_mktag_usage, 0);
 
 	if (strbuf_read(&buf, 0, 0) < 0)
 		die_errno("could not read from stdin");
t/t3800-mktag.sh +12 -0
index 601b064e97a..0ddd0f01cc7 100755
@@ -45,6 +45,18 @@ test_expect_success 'setup' '
 	blob=$(git rev-parse --verify HEAD:B.t)
 '
 
+test_expect_success 'basic usage' '
+	cat >tag.sig <<-EOF &&
+	object $head
+	type commit
+	tag mytag
+	tagger T A Gger <[email protected]> 1206478233 -0500
+	EOF
+	git mktag <tag.sig &&
+	git mktag --end-of-options <tag.sig &&
+	test_expect_code 129 git mktag --unknown-option
+'
+
 ############################################################
 #  1. length check
 
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#150 lore raw

PATCH v5 22/23 mktag: mark strings for translation

In reply to Ævar Arnfjörð Bjarmason

Mark the errors mktag might emit for translation. This is a plumbing command, but the errors it emits are intended to be human-readable.

---
 builtin/mktag.c | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)
builtin/mktag.c +8 -8
index 18b8492f4d4..9b04b61c2bb 100644
@@ -31,10 +31,10 @@ static int mktag_fsck_error_func(struct fsck_options *o,
 		 * like missing "tagger" lines are "only" warnings
 		 * under fsck, we've always considered them an error.
 		 */
-		fprintf_ln(stderr, "error: tag input does not pass fsck: %s", message);
+		fprintf_ln(stderr, _("error: tag input does not pass fsck: %s"), message);
 		return 1;
 	default:
-		BUG("%d (FSCK_IGNORE?) should never trigger this callback",
+		BUG(_("%d (FSCK_IGNORE?) should never trigger this callback"),
 		    msg_type);
 	}
 }
@@ -49,10 +49,10 @@ static int verify_object_in_tag(struct object_id *tagged_oid, int *tagged_type)
 
 	buffer = read_object_file(tagged_oid, &type, &size);
 	if (!buffer)
-		die("could not read tagged object '%s'",
+		die(_("could not read tagged object '%s'"),
 		    oid_to_hex(tagged_oid));
 	if (type != *tagged_type)
-		die("object '%s' tagged as '%s', but is a '%s' type",
+		die(_("object '%s' tagged as '%s', but is a '%s' type"),
 		    oid_to_hex(tagged_oid),
 		    type_name(*tagged_type), type_name(type));
 
@@ -79,7 +79,7 @@ int cmd_mktag(int argc, const char **argv, const char *prefix)
 			     builtin_mktag_usage, 0);
 
 	if (strbuf_read(&buf, 0, 0) < 0)
-		die_errno("could not read from stdin");
+		die_errno(_("could not read from stdin"));
 
 	fsck_options.error_func = mktag_fsck_error_func;
 	fsck_set_msg_type(&fsck_options, "extraheaderentry", "warn");
@@ -87,13 +87,13 @@ int cmd_mktag(int argc, const char **argv, const char *prefix)
 	git_config(mktag_config, NULL);
 	if (fsck_tag_standalone(NULL, buf.buf, buf.len, &fsck_options,
 				&tagged_oid, &tagged_type))
-		die("tag on stdin did not pass our strict fsck check");
+		die(_("tag on stdin did not pass our strict fsck check"));
 
 	if (verify_object_in_tag(&tagged_oid, &tagged_type))
-		die("tag on stdin did not refer to a valid object");
+		die(_("tag on stdin did not refer to a valid object"));
 
 	if (write_object_file(buf.buf, buf.len, tag_type, &result) < 0)
-		die("unable to write tag file");
+		die(_("unable to write tag file"));
 
 	strbuf_release(&buf);
 	puts(oid_to_hex(&result));
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#151 lore raw

PATCH v5 23/23 mktag: add a --no-strict option

In reply to Ævar Arnfjörð Bjarmason

Now that mktag has been migrated to use the fsck machinery to check its input, it makes sense to teach it to run in the equivalent of "git fsck"'s default mode, instead of hardcoding "git fsck --strict". Let's do that and support the "--no-strict" option.

Since this is a new option we don't need to cater to parse-option.c's default of automatically supporting --strict. So let's use PARSE_OPT_NONEG, using a new trivial helper macro.

---
 Documentation/git-mktag.txt |  8 ++++++++
 builtin/mktag.c             |  9 +++++++++
 t/t3800-mktag.sh            | 33 +++++++++++++++++++++++----------
 3 files changed, 40 insertions(+), 10 deletions(-)
Documentation/git-mktag.txt +8 -0
index 79813ff8df0..17a2603a600 100644
@@ -11,6 +11,14 @@ SYNOPSIS
 [verse]
 'git mktag'
 
+OPTIONS
+-------
+
+--strict::
+	By default mktag turns on the equivalent of
+	linkgit:git-fsck[1] `--strict` mode. Use `--no-strict` to
+	disable it.
+
 DESCRIPTION
 -----------
 
builtin/mktag.c +9 -0
index 9b04b61c2bb..41a399a69e4 100644
@@ -10,6 +10,7 @@ static char const * const builtin_mktag_usage[] = {
 	N_("git mktag"),
 	NULL
 };
+static int option_strict = 1;
 
 static struct fsck_options fsck_options = FSCK_OPTIONS_STRICT;
 
@@ -25,6 +26,12 @@ static int mktag_fsck_error_func(struct fsck_options *o,
 {
 	switch (msg_type) {
 	case FSCK_WARN:
+		if (!option_strict) {
+			fprintf_ln(stderr, _("warning: tag input does not pass fsck: %s"), message);
+			return 0;
+
+		}
+		/* fallthrough */
 	case FSCK_ERROR:
 		/*
 		 * We treat both warnings and errors as errors, things
@@ -67,6 +74,8 @@ static int verify_object_in_tag(struct object_id *tagged_oid, int *tagged_type)
 int cmd_mktag(int argc, const char **argv, const char *prefix)
 {
 	static struct option builtin_mktag_options[] = {
+		OPT_BOOL(0, "strict", &option_strict,
+			 N_("enable more strict checking")),
 		OPT_END(),
 	};
 	struct strbuf buf = STRBUF_INIT;
t/t3800-mktag.sh +23 -10
index 0ddd0f01cc7..4d76ccbbc19 100755
@@ -12,12 +12,17 @@ test_description='git mktag: tag object verify test'
 # given in the expect.pat file.
 
 check_verify_failure () {
-	expect="$2"
-	test_expect_success "$1" '
+	test_expect_success "$1" "
 		test_must_fail env GIT_TEST_GETTEXT_POISON=false \
 			git mktag <tag.sig 2>message &&
-		grep "$expect" message
-	'
+		grep '$2' message &&
+		if test '$3' != '--no-strict'
+		then
+			test_must_fail env GIT_TEST_GETTEXT_POISON=false \
+				git mktag --no-strict <tag.sig 2>message.no-strict &&
+			grep '$2' message.no-strict
+		fi
+	"
 }
 
 test_expect_mktag_success() {
@@ -65,7 +70,7 @@ too short for a tag
 EOF
 
 check_verify_failure 'Tag object length check' \
-	'^error:.* missingObject:'
+	'^error:.* missingObject:' 'strict'
 
 ############################################################
 #  2. object line label check
@@ -240,7 +245,7 @@ tagger . <> 0 +0000
 EOF
 
 check_verify_failure 'verify tag-name check' \
-	'^error:.* badTagName:'
+	'^error:.* badTagName:' '--no-strict'
 
 ############################################################
 # 11. tagger line label check #1
@@ -254,7 +259,7 @@ This is filler
 EOF
 
 check_verify_failure '"tagger" line label check #1' \
-	'^error:.* missingTaggerEntry:'
+	'^error:.* missingTaggerEntry:' '--no-strict'
 
 ############################################################
 # 12. tagger line label check #2
@@ -269,7 +274,7 @@ This is filler
 EOF
 
 check_verify_failure '"tagger" line label check #2' \
-	'^error:.* missingTaggerEntry:'
+	'^error:.* missingTaggerEntry:' '--no-strict'
 
 ############################################################
 # 13. allow missing tag author name like fsck
@@ -298,7 +303,7 @@ tagger T A Gger <
 EOF
 
 check_verify_failure 'disallow malformed tagger' \
-	'^error:.* badEmail:'
+	'^error:.* badEmail:' '--no-strict'
 
 ############################################################
 # 15. allow empty tag email
@@ -422,13 +427,21 @@ this line should not be here
 EOF
 
 check_verify_failure 'detect invalid header entry' \
-	'^error:.* extraHeaderEntry:'
+	'^error:.* extraHeaderEntry:' '--no-strict'
 
 test_expect_success 'invalid header entry config & fsck' '
 	test_must_fail git mktag <tag.sig &&
+	git mktag --no-strict <tag.sig &&
+
 	test_must_fail git -c fsck.extraHeaderEntry=error mktag <tag.sig &&
+	test_must_fail git -c fsck.extraHeaderEntry=error mktag --no-strict <tag.sig &&
+
 	test_must_fail git -c fsck.extraHeaderEntry=warn mktag <tag.sig &&
+	git -c fsck.extraHeaderEntry=warn mktag --no-strict <tag.sig &&
+
 	git -c fsck.extraHeaderEntry=ignore mktag <tag.sig &&
+	git -c fsck.extraHeaderEntry=ignore mktag --no-strict <tag.sig &&
+
 	git fsck &&
 	env GIT_TEST_GETTEXT_POISON=false \
 		git -c fsck.extraHeaderEntry=warn fsck 2>err &&
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#152 lore raw

Re: [PATCH v5 23/23] mktag: add a --no-strict option

Quoted 8 lines · Ævar Arnfjörð Bjarmason

Now that mktag has been migrated to use the fsck machinery to check its input, it makes sense to teach it to run in the equivalent of "git fsck"'s default mode, instead of hardcoding "git fsck --strict". Let's do that and support the "--no-strict" option.

Since this is a new option we don't need to cater to parse-option.c's default of automatically supporting --strict. So let's use PARSE_OPT_NONEG, using a new trivial helper macro.

I do not understand the last paragraph.

    git mktag --no-strict --strict --no-strict

should follow the usual "the last one wins" rule, so I would say that supporting "--strict", whether it is done automatically as a natural consequence of use of parse-options or not, would be a good thing (also I so not see PARSE_OPT_NONEG in this patch). Perhaps the log message (and the title) needs updating?

mktag: add a --[no-]strict option

Now that mktag has been migrated to use the fsck machinery to check its input, it makes sense to teach it to run in the equivalent of "git fsck"'s default mode. For cases where mktag is used to (re)create a tag object using data from an existing and malformed tag object, the validation may optionally have to be loosened. Teach the command to take the "--[no-]strict" option to do so.

or something?

#153 lore raw

PATCH v6 23/23 mktag: add a --[no-]strict option

Now that mktag has been migrated to use the fsck machinery to check its input, it makes sense to teach it to run in the equivalent of "git fsck"'s default mode.

For cases where mktag is used to (re)create a tag object using data from an existing and malformed tag object, the validation may optionally have to be loosened. Teach the command to take the "--[no-]strict" option to do so.

---

Quoted 7 lines · Junio C Hamano
Quoted 5 lines · Ævar Arnfjörð Bjarmason

Now that mktag has been migrated to use the fsck machinery to check its input, it makes sense to teach it to run in the equivalent of "git fsck"'s default mode, instead of hardcoding "git fsck --strict". Let's do that and support the "--no-strict" option.

Quoted 5 lines
Quoted 3 lines

Since this is a new option we don't need to cater to parse-option.c's default of automatically supporting --strict. So let's use PARSE_OPT_NONEG, using a new trivial helper macro.

I do not understand the last paragraph.

I forgot to update the commit message when changing this from v4->v5. Here's a v6 of just this patch (to save on list traffic) with an updated commit message. Thanks!

 Documentation/git-mktag.txt |  8 ++++++++
 builtin/mktag.c             |  9 +++++++++
 t/t3800-mktag.sh            | 33 +++++++++++++++++++++++----------
 3 files changed, 40 insertions(+), 10 deletions(-)
Documentation/git-mktag.txt +8 -0
index 79813ff8df0..17a2603a600 100644
@@ -11,6 +11,14 @@ SYNOPSIS
 [verse]
 'git mktag'
 
+OPTIONS
+-------
+
+--strict::
+	By default mktag turns on the equivalent of
+	linkgit:git-fsck[1] `--strict` mode. Use `--no-strict` to
+	disable it.
+
 DESCRIPTION
 -----------
 
builtin/mktag.c +9 -0
index 9b04b61c2bb..41a399a69e4 100644
@@ -10,6 +10,7 @@ static char const * const builtin_mktag_usage[] = {
 	N_("git mktag"),
 	NULL
 };
+static int option_strict = 1;
 
 static struct fsck_options fsck_options = FSCK_OPTIONS_STRICT;
 
@@ -25,6 +26,12 @@ static int mktag_fsck_error_func(struct fsck_options *o,
 {
 	switch (msg_type) {
 	case FSCK_WARN:
+		if (!option_strict) {
+			fprintf_ln(stderr, _("warning: tag input does not pass fsck: %s"), message);
+			return 0;
+
+		}
+		/* fallthrough */
 	case FSCK_ERROR:
 		/*
 		 * We treat both warnings and errors as errors, things
@@ -67,6 +74,8 @@ static int verify_object_in_tag(struct object_id *tagged_oid, int *tagged_type)
 int cmd_mktag(int argc, const char **argv, const char *prefix)
 {
 	static struct option builtin_mktag_options[] = {
+		OPT_BOOL(0, "strict", &option_strict,
+			 N_("enable more strict checking")),
 		OPT_END(),
 	};
 	struct strbuf buf = STRBUF_INIT;
t/t3800-mktag.sh +23 -10
index 0ddd0f01cc7..4d76ccbbc19 100755
@@ -12,12 +12,17 @@ test_description='git mktag: tag object verify test'
 # given in the expect.pat file.
 
 check_verify_failure () {
-	expect="$2"
-	test_expect_success "$1" '
+	test_expect_success "$1" "
 		test_must_fail env GIT_TEST_GETTEXT_POISON=false \
 			git mktag <tag.sig 2>message &&
-		grep "$expect" message
-	'
+		grep '$2' message &&
+		if test '$3' != '--no-strict'
+		then
+			test_must_fail env GIT_TEST_GETTEXT_POISON=false \
+				git mktag --no-strict <tag.sig 2>message.no-strict &&
+			grep '$2' message.no-strict
+		fi
+	"
 }
 
 test_expect_mktag_success() {
@@ -65,7 +70,7 @@ too short for a tag
 EOF
 
 check_verify_failure 'Tag object length check' \
-	'^error:.* missingObject:'
+	'^error:.* missingObject:' 'strict'
 
 ############################################################
 #  2. object line label check
@@ -240,7 +245,7 @@ tagger . <> 0 +0000
 EOF
 
 check_verify_failure 'verify tag-name check' \
-	'^error:.* badTagName:'
+	'^error:.* badTagName:' '--no-strict'
 
 ############################################################
 # 11. tagger line label check #1
@@ -254,7 +259,7 @@ This is filler
 EOF
 
 check_verify_failure '"tagger" line label check #1' \
-	'^error:.* missingTaggerEntry:'
+	'^error:.* missingTaggerEntry:' '--no-strict'
 
 ############################################################
 # 12. tagger line label check #2
@@ -269,7 +274,7 @@ This is filler
 EOF
 
 check_verify_failure '"tagger" line label check #2' \
-	'^error:.* missingTaggerEntry:'
+	'^error:.* missingTaggerEntry:' '--no-strict'
 
 ############################################################
 # 13. allow missing tag author name like fsck
@@ -298,7 +303,7 @@ tagger T A Gger <
 EOF
 
 check_verify_failure 'disallow malformed tagger' \
-	'^error:.* badEmail:'
+	'^error:.* badEmail:' '--no-strict'
 
 ############################################################
 # 15. allow empty tag email
@@ -422,13 +427,21 @@ this line should not be here
 EOF
 
 check_verify_failure 'detect invalid header entry' \
-	'^error:.* extraHeaderEntry:'
+	'^error:.* extraHeaderEntry:' '--no-strict'
 
 test_expect_success 'invalid header entry config & fsck' '
 	test_must_fail git mktag <tag.sig &&
+	git mktag --no-strict <tag.sig &&
+
 	test_must_fail git -c fsck.extraHeaderEntry=error mktag <tag.sig &&
+	test_must_fail git -c fsck.extraHeaderEntry=error mktag --no-strict <tag.sig &&
+
 	test_must_fail git -c fsck.extraHeaderEntry=warn mktag <tag.sig &&
+	git -c fsck.extraHeaderEntry=warn mktag --no-strict <tag.sig &&
+
 	git -c fsck.extraHeaderEntry=ignore mktag <tag.sig &&
+	git -c fsck.extraHeaderEntry=ignore mktag --no-strict <tag.sig &&
+
 	git fsck &&
 	env GIT_TEST_GETTEXT_POISON=false \
 		git -c fsck.extraHeaderEntry=warn fsck 2>err &&
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#154 lore raw

Re: [PATCH v6 23/23] mktag: add a --[no-]strict option

Quoted 25 lines · Ævar Arnfjörð Bjarmason

Now that mktag has been migrated to use the fsck machinery to check its input, it makes sense to teach it to run in the equivalent of "git fsck"'s default mode.

For cases where mktag is used to (re)create a tag object using data from an existing and malformed tag object, the validation may optionally have to be loosened. Teach the command to take the "--[no-]strict" option to do so.

---

Quoted 11 lines · Junio C Hamano
Quoted 8 lines · Ævar Arnfjörð Bjarmason

Now that mktag has been migrated to use the fsck machinery to check its input, it makes sense to teach it to run in the equivalent of "git fsck"'s default mode, instead of hardcoding "git fsck --strict". Let's do that and support the "--no-strict" option.

Since this is a new option we don't need to cater to parse-option.c's default of automatically supporting --strict. So let's use PARSE_OPT_NONEG, using a new trivial helper macro.

Quoted 5 lines
Quoted 1 line

I do not understand the last paragraph.

I forgot to update the commit message when changing this from v4->v5. Here's a v6 of just this patch (to save on list traffic) with an updated commit message. Thanks!

Okay. I actually was expecting to hear the reason why the approach taken in v4 to use PARSE_OPT_NONEG is a good one, but I take the change of approach in v5 as the sign that you now agree with me that a plain vanilla bool that happens to default to true is easier to manage.

Will replace. Thanks.

Quoted 153 lines
 Documentation/git-mktag.txt |  8 ++++++++
 builtin/mktag.c             |  9 +++++++++
 t/t3800-mktag.sh            | 33 +++++++++++++++++++++++----------
 3 files changed, 40 insertions(+), 10 deletions(-)
Documentation/git-mktag.txt +8 -0
index 79813ff8df0..17a2603a600 100644
@@ -11,6 +11,14 @@ SYNOPSIS
 [verse]
 'git mktag'
 
+OPTIONS
+-------
+
+--strict::
+	By default mktag turns on the equivalent of
+	linkgit:git-fsck[1] `--strict` mode. Use `--no-strict` to
+	disable it.
+
 DESCRIPTION
 -----------
 
builtin/mktag.c +9 -0
index 9b04b61c2bb..41a399a69e4 100644
@@ -10,6 +10,7 @@ static char const * const builtin_mktag_usage[] = {
 	N_("git mktag"),
 	NULL
 };
+static int option_strict = 1;
 
 static struct fsck_options fsck_options = FSCK_OPTIONS_STRICT;
 
@@ -25,6 +26,12 @@ static int mktag_fsck_error_func(struct fsck_options *o,
 {
 	switch (msg_type) {
 	case FSCK_WARN:
+		if (!option_strict) {
+			fprintf_ln(stderr, _("warning: tag input does not pass fsck: %s"), message);
+			return 0;
+
+		}
+		/* fallthrough */
 	case FSCK_ERROR:
 		/*
 		 * We treat both warnings and errors as errors, things
@@ -67,6 +74,8 @@ static int verify_object_in_tag(struct object_id *tagged_oid, int *tagged_type)
 int cmd_mktag(int argc, const char **argv, const char *prefix)
 {
 	static struct option builtin_mktag_options[] = {
+		OPT_BOOL(0, "strict", &option_strict,
+			 N_("enable more strict checking")),
 		OPT_END(),
 	};
 	struct strbuf buf = STRBUF_INIT;
t/t3800-mktag.sh +23 -10
index 0ddd0f01cc7..4d76ccbbc19 100755
@@ -12,12 +12,17 @@ test_description='git mktag: tag object verify test'
 # given in the expect.pat file.
 
 check_verify_failure () {
-	expect="$2"
-	test_expect_success "$1" '
+	test_expect_success "$1" "
 		test_must_fail env GIT_TEST_GETTEXT_POISON=false \
 			git mktag <tag.sig 2>message &&
-		grep "$expect" message
-	'
+		grep '$2' message &&
+		if test '$3' != '--no-strict'
+		then
+			test_must_fail env GIT_TEST_GETTEXT_POISON=false \
+				git mktag --no-strict <tag.sig 2>message.no-strict &&
+			grep '$2' message.no-strict
+		fi
+	"
 }
 
 test_expect_mktag_success() {
@@ -65,7 +70,7 @@ too short for a tag
 EOF
 
 check_verify_failure 'Tag object length check' \
-	'^error:.* missingObject:'
+	'^error:.* missingObject:' 'strict'
 
 ############################################################
 #  2. object line label check
@@ -240,7 +245,7 @@ tagger . <> 0 +0000
 EOF
 
 check_verify_failure 'verify tag-name check' \
-	'^error:.* badTagName:'
+	'^error:.* badTagName:' '--no-strict'
 
 ############################################################
 # 11. tagger line label check #1
@@ -254,7 +259,7 @@ This is filler
 EOF
 
 check_verify_failure '"tagger" line label check #1' \
-	'^error:.* missingTaggerEntry:'
+	'^error:.* missingTaggerEntry:' '--no-strict'
 
 ############################################################
 # 12. tagger line label check #2
@@ -269,7 +274,7 @@ This is filler
 EOF
 
 check_verify_failure '"tagger" line label check #2' \
-	'^error:.* missingTaggerEntry:'
+	'^error:.* missingTaggerEntry:' '--no-strict'
 
 ############################################################
 # 13. allow missing tag author name like fsck
@@ -298,7 +303,7 @@ tagger T A Gger <
 EOF
 
 check_verify_failure 'disallow malformed tagger' \
-	'^error:.* badEmail:'
+	'^error:.* badEmail:' '--no-strict'
 
 ############################################################
 # 15. allow empty tag email
@@ -422,13 +427,21 @@ this line should not be here
 EOF
 
 check_verify_failure 'detect invalid header entry' \
-	'^error:.* extraHeaderEntry:'
+	'^error:.* extraHeaderEntry:' '--no-strict'
 
 test_expect_success 'invalid header entry config & fsck' '
 	test_must_fail git mktag <tag.sig &&
+	git mktag --no-strict <tag.sig &&
+
 	test_must_fail git -c fsck.extraHeaderEntry=error mktag <tag.sig &&
+	test_must_fail git -c fsck.extraHeaderEntry=error mktag --no-strict <tag.sig &&
+
 	test_must_fail git -c fsck.extraHeaderEntry=warn mktag <tag.sig &&
+	git -c fsck.extraHeaderEntry=warn mktag --no-strict <tag.sig &&
+
 	git -c fsck.extraHeaderEntry=ignore mktag <tag.sig &&
+	git -c fsck.extraHeaderEntry=ignore mktag --no-strict <tag.sig &&
+
 	git fsck &&
 	env GIT_TEST_GETTEXT_POISON=false \
 		git -c fsck.extraHeaderEntry=warn fsck 2>err &&
#155 lore raw

Re: [PATCH v6 23/23] mktag: add a --[no-]strict option

Quoted 33 lines · Junio C Hamano
Quoted 25 lines · Ævar Arnfjörð Bjarmason

Now that mktag has been migrated to use the fsck machinery to check its input, it makes sense to teach it to run in the equivalent of "git fsck"'s default mode.

For cases where mktag is used to (re)create a tag object using data from an existing and malformed tag object, the validation may optionally have to be loosened. Teach the command to take the "--[no-]strict" option to do so.

---

Quoted 11 lines · Junio C Hamano
Quoted 8 lines · Ævar Arnfjörð Bjarmason

Now that mktag has been migrated to use the fsck machinery to check its input, it makes sense to teach it to run in the equivalent of "git fsck"'s default mode, instead of hardcoding "git fsck --strict". Let's do that and support the "--no-strict" option.

Since this is a new option we don't need to cater to parse-option.c's default of automatically supporting --strict. So let's use PARSE_OPT_NONEG, using a new trivial helper macro.

Quoted 5 lines
Quoted 1 line

I do not understand the last paragraph.

I forgot to update the commit message when changing this from v4->v5. Here's a v6 of just this patch (to save on list traffic) with an updated commit message. Thanks!

Quoted 5 lines

Okay. I actually was expecting to hear the reason why the approach taken in v4 to use PARSE_OPT_NONEG is a good one, but I take the change of approach in v5 as the sign that you now agree with me that a plain vanilla bool that happens to default to true is easier to manage.

There was no good reason. I couldn't find whether &no_var with "no-OPT" options & PARSE_OPT_NONEG was the preferred pattern or &var with a default of 1 with "OPT". There's a few examples of the former in the codebase, but I'm happy to go with the latter.

Quoted 153 lines
Quoted 153 lines
 Documentation/git-mktag.txt |  8 ++++++++
 builtin/mktag.c             |  9 +++++++++
 t/t3800-mktag.sh            | 33 +++++++++++++++++++++++----------
 3 files changed, 40 insertions(+), 10 deletions(-)
Documentation/git-mktag.txt +8 -0
index 79813ff8df0..17a2603a600 100644
@@ -11,6 +11,14 @@ SYNOPSIS
 [verse]
 'git mktag'
 
+OPTIONS
+-------
+
+--strict::
+	By default mktag turns on the equivalent of
+	linkgit:git-fsck[1] `--strict` mode. Use `--no-strict` to
+	disable it.
+
 DESCRIPTION
 -----------
 
builtin/mktag.c +9 -0
index 9b04b61c2bb..41a399a69e4 100644
@@ -10,6 +10,7 @@ static char const * const builtin_mktag_usage[] = {
 	N_("git mktag"),
 	NULL
 };
+static int option_strict = 1;
 
 static struct fsck_options fsck_options = FSCK_OPTIONS_STRICT;
 
@@ -25,6 +26,12 @@ static int mktag_fsck_error_func(struct fsck_options *o,
 {
 	switch (msg_type) {
 	case FSCK_WARN:
+		if (!option_strict) {
+			fprintf_ln(stderr, _("warning: tag input does not pass fsck: %s"), message);
+			return 0;
+
+		}
+		/* fallthrough */
 	case FSCK_ERROR:
 		/*
 		 * We treat both warnings and errors as errors, things
@@ -67,6 +74,8 @@ static int verify_object_in_tag(struct object_id *tagged_oid, int *tagged_type)
 int cmd_mktag(int argc, const char **argv, const char *prefix)
 {
 	static struct option builtin_mktag_options[] = {
+		OPT_BOOL(0, "strict", &option_strict,
+			 N_("enable more strict checking")),
 		OPT_END(),
 	};
 	struct strbuf buf = STRBUF_INIT;
t/t3800-mktag.sh +23 -10
index 0ddd0f01cc7..4d76ccbbc19 100755
@@ -12,12 +12,17 @@ test_description='git mktag: tag object verify test'
 # given in the expect.pat file.
 
 check_verify_failure () {
-	expect="$2"
-	test_expect_success "$1" '
+	test_expect_success "$1" "
 		test_must_fail env GIT_TEST_GETTEXT_POISON=false \
 			git mktag <tag.sig 2>message &&
-		grep "$expect" message
-	'
+		grep '$2' message &&
+		if test '$3' != '--no-strict'
+		then
+			test_must_fail env GIT_TEST_GETTEXT_POISON=false \
+				git mktag --no-strict <tag.sig 2>message.no-strict &&
+			grep '$2' message.no-strict
+		fi
+	"
 }
 
 test_expect_mktag_success() {
@@ -65,7 +70,7 @@ too short for a tag
 EOF
 
 check_verify_failure 'Tag object length check' \
-	'^error:.* missingObject:'
+	'^error:.* missingObject:' 'strict'
 
 ############################################################
 #  2. object line label check
@@ -240,7 +245,7 @@ tagger . <> 0 +0000
 EOF
 
 check_verify_failure 'verify tag-name check' \
-	'^error:.* badTagName:'
+	'^error:.* badTagName:' '--no-strict'
 
 ############################################################
 # 11. tagger line label check #1
@@ -254,7 +259,7 @@ This is filler
 EOF
 
 check_verify_failure '"tagger" line label check #1' \
-	'^error:.* missingTaggerEntry:'
+	'^error:.* missingTaggerEntry:' '--no-strict'
 
 ############################################################
 # 12. tagger line label check #2
@@ -269,7 +274,7 @@ This is filler
 EOF
 
 check_verify_failure '"tagger" line label check #2' \
-	'^error:.* missingTaggerEntry:'
+	'^error:.* missingTaggerEntry:' '--no-strict'
 
 ############################################################
 # 13. allow missing tag author name like fsck
@@ -298,7 +303,7 @@ tagger T A Gger <
 EOF
 
 check_verify_failure 'disallow malformed tagger' \
-	'^error:.* badEmail:'
+	'^error:.* badEmail:' '--no-strict'
 
 ############################################################
 # 15. allow empty tag email
@@ -422,13 +427,21 @@ this line should not be here
 EOF
 
 check_verify_failure 'detect invalid header entry' \
-	'^error:.* extraHeaderEntry:'
+	'^error:.* extraHeaderEntry:' '--no-strict'
 
 test_expect_success 'invalid header entry config & fsck' '
 	test_must_fail git mktag <tag.sig &&
+	git mktag --no-strict <tag.sig &&
+
 	test_must_fail git -c fsck.extraHeaderEntry=error mktag <tag.sig &&
+	test_must_fail git -c fsck.extraHeaderEntry=error mktag --no-strict <tag.sig &&
+
 	test_must_fail git -c fsck.extraHeaderEntry=warn mktag <tag.sig &&
+	git -c fsck.extraHeaderEntry=warn mktag --no-strict <tag.sig &&
+
 	git -c fsck.extraHeaderEntry=ignore mktag <tag.sig &&
+	git -c fsck.extraHeaderEntry=ignore mktag --no-strict <tag.sig &&
+
 	git fsck &&
 	env GIT_TEST_GETTEXT_POISON=false \
 		git -c fsck.extraHeaderEntry=warn fsck 2>err &&
#156 lore raw

PATCH v4 2/20 mktag doc: grammar fix, when exists -> when it exists

In reply to Ævar Arnfjörð Bjarmason

Amend the wording of documentation added in 6cfec03680 (mktag:
minimally update the description., 2007-06-10). It makes more sense to say "when it exists" here, as we're referring to "the message".

---
 Documentation/git-mktag.txt | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Documentation/git-mktag.txt +1 -1
index a158428eb9..1b0667e372 100644
@@ -29,7 +29,7 @@ has a very simple fixed format: four lines of
   tagger <tagger>
 
 followed by some 'optional' free-form message (some tags created
-by older Git may not have `tagger` line).  The message, when
+by older Git may not have `tagger` line).  The message, when it
 exists, is separated by a blank line from the header.  The
 message part may contain a signature that Git itself doesn't
 care about, but that can be verified with gpg.
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#157 lore raw

PATCH v4 3/20 mktag doc: update to explain why to use this

In reply to Ævar Arnfjörð Bjarmason

Change the mktag documentation to compare itself to the similar "hash-object -t tag" command. Before this someone reading the documentation wouldn't have much of an idea what the difference was.

Let's allude to our own validation logic, and cross-link the "mktag" and "hash-object" documentation to aid discover-ability. A follow-up change to migrate "mktag" to use "fsck" validation will make the part about validation logic clearer.

---
 Documentation/git-hash-object.txt |  4 ++++
 Documentation/git-mktag.txt       | 21 +++++++++++++++++----
 2 files changed, 21 insertions(+), 4 deletions(-)
Documentation/git-hash-object.txt +4 -0
index df9e2c58bd..c535661ced 100644
@@ -58,6 +58,10 @@ OPTIONS
 	stress-testing Git itself or reproducing characteristics of corrupt or
 	bogus objects encountered in the wild.
 
+SEE ALSO
+--------
+linkgit:git-mktag[1]
+
 GIT
 ---
 Part of the linkgit:git[1] suite
Documentation/git-mktag.txt +17 -4
index 1b0667e372..adc63f6d4e 100644
@@ -3,7 +3,7 @@ git-mktag(1)
 
 NAME
 ----
-git-mktag - Creates a tag object
+git-mktag - Creates a tag object with extra validation
 
 
 SYNOPSIS
@@ -13,10 +13,19 @@ SYNOPSIS
 
 DESCRIPTION
 -----------
-Reads a tag contents on standard input and creates a tag object
-that can also be used to sign other objects.
 
-The output is the new tag's <object> identifier.
+Reads a tag contents on standard input and creates a tag object. The
+output is the new tag's <object> identifier.
+
+This command is mostly equivalent to linkgit:git-hash-object[1]
+invoked with `-t tag -w --stdin`. I.e. both of these will create and
+write a tag found in `my-tag`:
+
+    git mktag <my-tag
+    git hash-object -t tag -w --stdin <my-tag
+
+The difference is that mktag will die before writing the tag if the
+tag doesn't pass a sanity check.
 
 Tag Format
 ----------
@@ -34,6 +43,10 @@ exists, is separated by a blank line from the header.  The
 message part may contain a signature that Git itself doesn't
 care about, but that can be verified with gpg.
 
+SEE ALSO
+--------
+linkgit:git-hash-object[1],
+
 GIT
 ---
 Part of the linkgit:git[1] suite
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#158 lore raw

Re: [PATCH v4 03/20] mktag doc: update to explain why to use this

Quoted 9 lines · Ævar Arnfjörð Bjarmason

Change the mktag documentation to compare itself to the similar "hash-object -t tag" command. Before this someone reading the documentation wouldn't have much of an idea what the difference was.

Let's allude to our own validation logic, and cross-link the "mktag" and "hash-object" documentation to aid discover-ability. A follow-up change to migrate "mktag" to use "fsck" validation will make the part about validation logic clearer.

We do not add links to refer to commands like "git commit-tree", "git mktree" or "git add" just because these other commands can create commit, tree and blob objects like hash-object does, so adding a link to mktag in the hash-object documentation feels a bit fishy, but the extended description in mktag that highlights the added value of the command over hash-object reads very well.

Documentation/git-mktag.txt exists, is separated by a blank line from the header. The · line 48

Quoted 9 lines

Tag Format
----------

Documentation/git-mktag.txt +3 -0
@@ -34,6 +43,10 @@ exists, is separated by a blank line from the header.  The
 message part may contain a signature that Git itself doesn't
 care about, but that can be verified with gpg.
 
+SEE ALSO
+--------
+linkgit:git-hash-object[1],

Comma,

Quoted 4 lines
Documentation/git-hash-object.txt +1 -0
+
 GIT
 ---
 Part of the linkgit:git[1] suite
#159 lore raw

PATCH v4 4/20 mktag tests: don't needlessly use a subshell

In reply to Ævar Arnfjörð Bjarmason

The use of a subshell dates back to e9b20943b77 (t/t3800: do not use a temporary file to hold expected result., 2008-01-04). It's not needed anymore, if it ever was.

---
 t/t3800-mktag.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
t/t3800-mktag.sh +1 -1
index d696aa4e52..0e411e3c45 100755
@@ -14,7 +14,7 @@ test_description='git mktag: tag object verify test'
 check_verify_failure () {
 	expect="$2"
 	test_expect_success "$1" '
-		( test_must_fail git mktag <tag.sig 2>message ) &&
+		test_must_fail git mktag <tag.sig 2>message &&
 		grep "$expect" message
 	'
 }
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#160 lore raw

PATCH v4 5/20 mktag tests: remove needless SHA-1 hardcoding

In reply to Ævar Arnfjörð Bjarmason

Change the tests amended in acb49d1cc8b (t3800: make hash-size independent, 2019-08-18) even more to make them independent of either SHA-1 or SHA-256.

Some of these tests were failing for the wrong reasons. The first one being modified here would fail because the line starts with "xxxxxx" instead of "object", the rest of the line doesn't matter.

Let's just put a valid hash on the rest of the line anyway to narrow the test down for just the s/object/xxxxxx/ case.

The second one being modified here would fail under GIT_TEST_DEFAULT_HASH=sha256 because <some sha-1 length garbage> is an invalid SHA-256, but we should really be testing <some sha-256 length garbage> when under SHA-256.

This doesn't really matter since we should be able to trust other parts of the code to validate things in the 0-9a-f range, but let's keep it for good measure.

There's a later test which tests an invalid SHA which looks like a valid one, to stress the "We refuse to tag something we can't verify[...]" logic in mktag.c.

But here we're testing for a SHA-length string which contains characters outside of the /[0-9a-f]/i set.

---
 t/t3800-mktag.sh | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
t/t3800-mktag.sh +3 -3
index 0e411e3c45..a22a0954d5 100755
@@ -43,7 +43,7 @@ check_verify_failure 'Tag object length check' \
 #  2. object line label check
 
 cat >tag.sig <<EOF
-xxxxxx 139e9b33986b1c2670fff52c5067603117b3e895
+xxxxxx $head
 type tag
 tag mytag
 tagger . <> 0 +0000
@@ -53,10 +53,10 @@ EOF
 check_verify_failure '"object" line label check' '^error: char0: .*"object "$'
 
 ############################################################
-#  3. object line SHA1 check
+#  3. object line hash check
 
 cat >tag.sig <<EOF
-object zz9e9b33986b1c2670fff52c5067603117b3e895
+object $(echo $head | tr 0-9a-f z)
 type tag
 tag mytag
 tagger . <> 0 +0000
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#161 lore raw

PATCH v4 6/20 mktag tests: improve verify_object() test coverage

In reply to Ævar Arnfjörð Bjarmason

The verify_object() function in "mktag.c" is tasked with ensuring that our tag refers to a valid object.

The existing test for this might fail because it was also testing that "type taggg" didn't refer to a valid object type (it should be "type tag"), or because we referred to a valid object but got the type wrong.

Let's split these tests up, so we're testing all combinations of a non-existing object and in invalid/wrong "type" lines.

---
 t/t3800-mktag.sh | 37 +++++++++++++++++++++++++++++++++++--
 1 file changed, 35 insertions(+), 2 deletions(-)
t/t3800-mktag.sh +35 -2
index a22a0954d5..9ae1b0bb0a 100755
@@ -125,19 +125,52 @@ check_verify_failure '"type" line type-name length check' \
 	'^error: char.*: type too long$'
 
 ############################################################
-#  9. verify object (SHA1/type) check
+#  9. verify object (hash/type) check
 
 cat >tag.sig <<EOF
 object $(test_oid deadbeef)
+type tag
+tag mytag
+tagger . <> 0 +0000
+
+EOF
+
+check_verify_failure 'verify object (hash/type) check -- correct type, nonexisting object' \
+	'^error: char7: could not verify object.*$'
+
+cat >tag.sig <<EOF
+object $head
 type tagggg
 tag mytag
 tagger . <> 0 +0000
 
 EOF
 
-check_verify_failure 'verify object (SHA1/type) check' \
+check_verify_failure 'verify object (hash/type) check -- made-up type, nonexisting object' \
+	'^fatal: invalid object type'
+
+cat >tag.sig <<EOF
+object $(test_oid deadbeef)
+type tagggg
+tag mytag
+tagger . <> 0 +0000
+
+EOF
+
+check_verify_failure 'verify object (hash/type) check -- incorrect type, valid object' \
 	'^error: char7: could not verify object.*$'
 
+cat >tag.sig <<EOF
+object $head
+type tree
+tag mytag
+tagger . <> 0 +0000
+
+EOF
+
+check_verify_failure 'verify object (hash/type) check -- incorrect type, valid object' \
+	'^error: char7: could not verify object'
+
 ############################################################
 # 10. verify tag-name check
 
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#162 lore raw

Re: [PATCH v4 06/20] mktag tests: improve verify_object() test coverage

t/t3800-mktag.sh check_verify_failure '"type" line type-name length check' \ · line 140

Quoted 11 lines · Ævar Arnfjörð Bjarmason

cat >tag.sig <<EOF
object $(test_oid deadbeef)

t/t3800-mktag.sh +9 -0
+type tag
+tag mytag
+tagger . <> 0 +0000
+
+EOF
+
+check_verify_failure 'verify object (hash/type) check -- correct type, nonexisting object' \
+	'^error: char7: could not verify object.*$'
+

OK.

t/t3800-mktag.sh check_verify_failure '"type" line type-name length check' \ · line 150

Quoted 11 lines
t/t3800-mktag.sh +4 -1
+cat >tag.sig <<EOF
+object $head
 type tagggg
 tag mytag
 tagger . <> 0 +0000
 
 EOF
 
-check_verify_failure 'verify object (SHA1/type) check' \
+check_verify_failure 'verify object (hash/type) check -- made-up type, nonexisting object' \
+	'^fatal: invalid object type'

tagggg is "made-up type". Where did we grab head from? it is head=$(git rev-parse --verify HEAD) at the very beginning. Is that really a nonexisting object?

t/t3800-mktag.sh check_verify_failure '"type" line type-name length check' \ · line 161

Quoted 10 lines
t/t3800-mktag.sh +9 -0
+cat >tag.sig <<EOF
+object $(test_oid deadbeef)
+type tagggg
+tag mytag
+tagger . <> 0 +0000
+
+EOF
+
+check_verify_failure 'verify object (hash/type) check -- incorrect type, valid object' \
 	'^error: char7: could not verify object.*$'

This is the "made-up type, nonexisting object", and previous one is "made-up type, valid object", no?

t/t3800-mktag.sh check_verify_failure '"type" line type-name length check' \ · line 172

Quoted 10 lines
t/t3800-mktag.sh +10 -0
+cat >tag.sig <<EOF
+object $head
+type tree
+tag mytag
+tagger . <> 0 +0000
+
+EOF
+
+check_verify_failure 'verify object (hash/type) check -- incorrect type, valid object' \
+	'^error: char7: could not verify object'

And this is correct. "type tree" is not a made-up type, and $head is a valid object but it is not of "type tree".

Somehow I feel as if this patch is trying to see if reviewers are paying attention to ;-)

Thanks.

#163 lore raw

PATCH v4 7/20 mktag tests: don't pipe to stderr needlessly

In reply to Ævar Arnfjörð Bjarmason

Remove the piping of stderr to "message" in the valid tag test. This pattern seems to have been copy/pasted from the failure case in 446c6faec6 (New tests and en-passant modifications to mktag., 2006-07-29).

Nothing is piped to "message" here, and in the event of the test failing it only serves to hide the error.

---
 t/t3800-mktag.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
t/t3800-mktag.sh +2 -2
index 9ae1b0bb0a..bbd148618e 100755
@@ -257,7 +257,7 @@ EOF
 
 test_expect_success \
     'allow empty tag email' \
-    'git mktag <tag.sig >.git/refs/tags/mytag 2>message'
+    'git mktag <tag.sig >.git/refs/tags/mytag'
 
 ############################################################
 # 16. disallow spaces in tag email
@@ -385,7 +385,7 @@ EOF
 
 test_expect_success \
     'create valid tag' \
-    'git mktag <tag.sig >.git/refs/tags/mytag 2>message'
+    'git mktag <tag.sig >.git/refs/tags/mytag'
 
 ############################################################
 # 25. check mytag
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#164 lore raw

Re: [PATCH v4 07/20] mktag tests: don't pipe to stderr needlessly

Quoted 7 lines · Ævar Arnfjörð Bjarmason

Remove the piping of stderr to "message" in the valid tag test. This pattern seems to have been copy/pasted from the failure case in 446c6faec6 (New tests and en-passant modifications to mktag., 2006-07-29).

Nothing is piped to "message" here, and in the event of the test failing it only serves to hide the error.

I think the verb "to pipe" specifically means "redirecting the output to a pipe" and nothing else.

mktag tests: don't redirect stderr to a file needlessly

Remove the redirection of stderr to "message" in the valid tag tests. This pattern ... 2006-07-29). Nobody examines the contents of the resulting "message" file, so the net result is that error messages cannot be seen in "sh t3800-mktag.sh -v" output.

or something like that?

Quoted 28 lines
---
 t/t3800-mktag.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
t/t3800-mktag.sh +2 -2
index 9ae1b0bb0a..bbd148618e 100755
@@ -257,7 +257,7 @@ EOF
 
 test_expect_success \
     'allow empty tag email' \
-    'git mktag <tag.sig >.git/refs/tags/mytag 2>message'
+    'git mktag <tag.sig >.git/refs/tags/mytag'
 
 ############################################################
 # 16. disallow spaces in tag email
@@ -385,7 +385,7 @@ EOF
 
 test_expect_success \
     'create valid tag' \
-    'git mktag <tag.sig >.git/refs/tags/mytag 2>message'
+    'git mktag <tag.sig >.git/refs/tags/mytag'
 
 ############################################################
 # 25. check mytag
#165 lore raw

PATCH v4 8/20 mktag tests: don't create "mytag" twice

In reply to Ævar Arnfjörð Bjarmason

Change a test added in e0aaf781f6 (mktag.c: improve verification of tagger field and tests, 2008-03-27) to not create "mytag", which should only be created and verified at the end in an earlier test added in 446c6faec6 (New tests and en-passant modifications to mktag., 2006-07-29).

While we're at it let's prevent a similar logic error from creeping into the test by asserting that "mytag" doesn't exist before we create it. Let's do this by moving the test to use "update-ref", instead of our own homebrew ad-hoc refstore update.

---
 t/t3800-mktag.sh | 17 +++++------------
 1 file changed, 5 insertions(+), 12 deletions(-)
t/t3800-mktag.sh +5 -12
index bbd148618e..b6dcdbebe6 100755
@@ -257,7 +257,7 @@ EOF
 
 test_expect_success \
     'allow empty tag email' \
-    'git mktag <tag.sig >.git/refs/tags/mytag'
+    'git mktag <tag.sig'
 
 ############################################################
 # 16. disallow spaces in tag email
@@ -383,16 +383,9 @@ tagger T A Gger <[email protected]> 1206478233 -0500
 
 EOF
 
-test_expect_success \
-    'create valid tag' \
-    'git mktag <tag.sig >.git/refs/tags/mytag'
-
-############################################################
-# 25. check mytag
-
-test_expect_success \
-    'check mytag' \
-    'git tag -l | grep mytag'
-
+test_expect_success 'create valid tag' '
+	git mktag <tag.sig >hash &&
+	git update-ref refs/tags/mytag $(cat hash) $(test_oid zero)
+'
 
 test_done
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#166 lore raw

Re: [PATCH v4 08/20] mktag tests: don't create "mytag" twice

Quoted 5 lines · Ævar Arnfjörð Bjarmason

Change a test added in e0aaf781f6 (mktag.c: improve verification of tagger field and tests, 2008-03-27) to not create "mytag", which should only be created and verified at the end in an earlier test added in 446c6faec6 (New tests and en-passant modifications to mktag., 2006-07-29).

If mktag fails to create a tag, presumably .git/refs/tags/mytag file would be left empty. Wouldn't "git tag -l" notice that it is not a valid ref and omit it from its output? I suspect if you corrupt the contents of tag.sig temporarily while the first of the original tests were running in such a way that mktag notices a bogosity, the second half of the original would notice.

Having said that, I like the new way much better.

But do we even need to create the tag with update-ref in the first place? What does it check? The fact that "mktag" only creates an object and never creates a new ref to point at the newly created object (hence we expect refs/tags/mytag does not yet exist)?

I am not complaining that it runs update-ref there or creates the mytag tag. I am just saying I do not understand what the test wants to check by doing so.

Quoted 4 lines

While we're at it let's prevent a similar logic error from creeping into the test by asserting that "mytag" doesn't exist before we create it. Let's do this by moving the test to use "update-ref", instead of our own homebrew ad-hoc refstore update.

Great. I really like this part of the change that future-proofs us.

Thanks.

Quoted 39 lines
---
 t/t3800-mktag.sh | 17 +++++------------
 1 file changed, 5 insertions(+), 12 deletions(-)
t/t3800-mktag.sh +5 -12
index bbd148618e..b6dcdbebe6 100755
@@ -257,7 +257,7 @@ EOF
 
 test_expect_success \
     'allow empty tag email' \
-    'git mktag <tag.sig >.git/refs/tags/mytag'
+    'git mktag <tag.sig'
 
 ############################################################
 # 16. disallow spaces in tag email
@@ -383,16 +383,9 @@ tagger T A Gger <[email protected]> 1206478233 -0500
 
 EOF
 
-test_expect_success \
-    'create valid tag' \
-    'git mktag <tag.sig >.git/refs/tags/mytag'
-
-############################################################
-# 25. check mytag
-
-test_expect_success \
-    'check mytag' \
-    'git tag -l | grep mytag'
-
+test_expect_success 'create valid tag' '
+	git mktag <tag.sig >hash &&
+	git update-ref refs/tags/mytag $(cat hash) $(test_oid zero)
+'
 
 test_done
#167 lore raw

PATCH v4 9/20 mktag tests: stress test whitespace handling

In reply to Ævar Arnfjörð Bjarmason

Add tests for various whitespace edge cases around the header/body boundary, such as two newlines separating the header and body, a space at the end of the "tagger" line before the header etc.

Add a test showing that we insist that an empty body must be preceded by an empty line. This test for a long-standing regression which goes against the command's documented behavior. It'll be addressed in a follow-up change.

---
 t/t3800-mktag.sh | 48 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 48 insertions(+)
t/t3800-mktag.sh +48 -0
index b6dcdbebe6..f339321be8 100755
@@ -372,6 +372,54 @@ EOF
 check_verify_failure 'detect invalid header entry' \
 	'^error: char.*: trailing garbage in tag header$'
 
+cat >tag.sig <<EOF
+object $head
+type commit
+tag mytag
+tagger T A Gger <[email protected]> 1206478233 -0500
+
+
+this line comes after an extra newline
+EOF
+
+test_expect_success 'allow extra newlines at start of body' '
+	git mktag <tag.sig
+'
+
+cat >tag.sig <<EOF
+object $head
+type commit
+tag mytag
+tagger T A Gger <[email protected]> 1206478233 -0500
+
+EOF
+
+test_expect_success 'allow extra newlines at end of headers' '
+	git mktag <tag.sig
+'
+
+space=' '
+cat >tag.sig <<EOF
+object $head
+type commit
+tag mytag
+tagger T A Gger <[email protected]> 1206478233 -0500$space
+
+EOF
+
+check_verify_failure 'extra whitespace at end of headers' \
+	'^error: char.*: malformed tag timezone$'
+
+cat >tag.sig <<EOF
+object $head
+type commit
+tag mytag
+tagger T A Gger <[email protected]> 1206478233 -0500
+EOF
+
+check_verify_failure 'disallow no header / body newline separator' \
+	'^error: char.*: trailing garbage in tag header$'
+
 ############################################################
 # 24. create valid tag
 
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#168 lore raw

Re: [PATCH v4 09/20] mktag tests: stress test whitespace handling

t/t3800-mktag.sh EOF · line 387

Quoted 13 lines · Ævar Arnfjörð Bjarmason
t/t3800-mktag.sh +13 -0
+cat >tag.sig <<EOF
+object $head
+type commit
+tag mytag
+tagger T A Gger <[email protected]> 1206478233 -0500
+
+
+this line comes after an extra newline
+EOF
+
+test_expect_success 'allow extra newlines at start of body' '
+	git mktag <tag.sig
+'

OK.

t/t3800-mktag.sh EOF · line 399

Quoted 11 lines
t/t3800-mktag.sh +11 -0
+cat >tag.sig <<EOF
+object $head
+type commit
+tag mytag
+tagger T A Gger <[email protected]> 1206478233 -0500
+
+EOF
+
+test_expect_success 'allow extra newlines at end of headers' '
+	git mktag <tag.sig
+'

I am not sure what the contrast between "at start of body" and "at end of headers" means.

Ahh, OK, we are making sure that we "allow a blank line after the headers before an empty body".

Makes sense (but see below).

t/t3800-mktag.sh EOF · line 411

Quoted 11 lines
t/t3800-mktag.sh +11 -0
+space=' '
+cat >tag.sig <<EOF
+object $head
+type commit
+tag mytag
+tagger T A Gger <[email protected]> 1206478233 -0500$space
+
+EOF
+
+check_verify_failure 'extra whitespace at end of headers' \
+	'^error: char.*: malformed tag timezone$'

OK. Presumably all the other header lines forbid trailing space as a malformed line?

t/t3800-mktag.sh EOF · line 421

Quoted 9 lines
t/t3800-mktag.sh +9 -0
+cat >tag.sig <<EOF
+object $head
+type commit
+tag mytag
+tagger T A Gger <[email protected]> 1206478233 -0500
+EOF
+
+check_verify_failure 'disallow no header / body newline separator' \
+	'^error: char.*: trailing garbage in tag header$'

Now I am confused. This fails because...? We earlier saw that we
"allow" a blank line after the header when no body is present, but this says the test fails if a blank line exists after the header when no body exists. I do not have any objection to the behaviour of the command, but doesn't this mean the earlier test was not labelled correctly? It may make more sense to move this test next to the earlier one, and name them

"require a blank line before even an empty body (1)" "require a blank line before even an empty body (2)"

or something? The earlier one ensures that a payload with such a blank line is OK, and this one ensures that a payload without such a blank line causes a failure.

Thanks.

Quoted 3 lines

+
############################################################
# 24. create valid tag

#169 lore raw

PATCH v4 10/20 mktag tests: test "hash-object" compatibility

In reply to Ævar Arnfjörð Bjarmason

Change all the successful "mktag" tests to test that "hash-object" produces the same hash for the input, and that fsck passes for both.

This tests e.g. that "mktag" doesn't trim its input or otherwise munge it in a way that "hash-object" doesn't.

---
 t/t3800-mktag.sh | 26 +++++++++++++++-----------
 1 file changed, 15 insertions(+), 11 deletions(-)
t/t3800-mktag.sh +15 -11
index f339321be8..c6826762d9 100755
@@ -19,6 +19,16 @@ check_verify_failure () {
 	'
 }
 
+test_expect_mktag_success() {
+	test_expect_success "$1" '
+		git hash-object -t tag -w --stdin <tag.sig >expected &&
+		git fsck --strict &&
+		git mktag <tag.sig >hash &&
+		git fsck --strict &&
+		test_cmp expected hash
+	'
+}
+
 ###########################################################
 # first create a commit, so we have a valid object/type
 # for the tag.
@@ -255,9 +265,7 @@ tagger T A Gger <> 0 +0000
 
 EOF
 
-test_expect_success \
-    'allow empty tag email' \
-    'git mktag <tag.sig'
+test_expect_mktag_success 'allow empty tag email'
 
 ############################################################
 # 16. disallow spaces in tag email
@@ -382,9 +390,7 @@ tagger T A Gger <[email protected]> 1206478233 -0500
 this line comes after an extra newline
 EOF
 
-test_expect_success 'allow extra newlines at start of body' '
-	git mktag <tag.sig
-'
+test_expect_mktag_success 'allow extra newlines at start of body'
 
 cat >tag.sig <<EOF
 object $head
@@ -394,9 +400,7 @@ tagger T A Gger <[email protected]> 1206478233 -0500
 
 EOF
 
-test_expect_success 'allow extra newlines at end of headers' '
-	git mktag <tag.sig
-'
+test_expect_mktag_success 'allow extra newlines at end of headers'
 
 space=' '
 cat >tag.sig <<EOF
@@ -431,8 +435,8 @@ tagger T A Gger <[email protected]> 1206478233 -0500
 
 EOF
 
-test_expect_success 'create valid tag' '
-	git mktag <tag.sig >hash &&
+test_expect_mktag_success 'create valid tag object'
+test_expect_success 'create valid tag name' '
 	git update-ref refs/tags/mytag $(cat hash) $(test_oid zero)
 '
 
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#170 lore raw

Re: [PATCH v4 10/20] mktag tests: test "hash-object" compatibility

Quoted 19 lines · Ævar Arnfjörð Bjarmason

Change all the successful "mktag" tests to test that "hash-object" produces the same hash for the input, and that fsck passes for both.

This tests e.g. that "mktag" doesn't trim its input or otherwise munge it in a way that "hash-object" doesn't.

---
 t/t3800-mktag.sh | 26 +++++++++++++++-----------
 1 file changed, 15 insertions(+), 11 deletions(-)
t/t3800-mktag.sh +0 -0
index f339321be8..c6826762d9 100755
@@ -19,6 +19,16 @@ check_verify_failure () {
 	'
 }

t/t3800-mktag.sh check_verify_failure () { · line 30

Quoted 10 lines
t/t3800-mktag.sh +9 -0
 
+test_expect_mktag_success() {
+	test_expect_success "$1" '
+		git hash-object -t tag -w --stdin <tag.sig >expected &&
+		git fsck --strict &&
+		git mktag <tag.sig >hash &&
+		git fsck --strict &&
+		test_cmp expected hash
+	'
+}

;-) Nice.

I'll stop here for tonight. Thanks for working on this.

#171 lore raw

PATCH v4 11/20 mktag: use default strbuf_read() hint

In reply to Ævar Arnfjörð Bjarmason

Change the hardcoded hint of 2^12 to 0. The default strbuf hint is perfectly fine here, and the only reason we were hardcoding it is because it survived migration from a pre-strbuf fixed-sized buffer.

See fd17f5b5f77 (Replace all read_fd use with strbuf_read, and get rid of it., 2007-09-10) for that migration.

---
 builtin/mktag.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
builtin/mktag.c +1 -1
index 4982d3a93e..ff7ac8e0e5 100644
@@ -161,7 +161,7 @@ int cmd_mktag(int argc, const char **argv, const char *prefix)
 	if (argc != 1)
 		usage("git mktag");
 
-	if (strbuf_read(&buf, 0, 4096) < 0) {
+	if (strbuf_read(&buf, 0, 0) < 0) {
 		die_errno("could not read from stdin");
 	}
 
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#172 lore raw

PATCH v4 12/20 mktag: remove redundant braces in one-line body "if"

In reply to Ævar Arnfjörð Bjarmason

This minor stylistic churn is usually something we'd avoid, but if we don't do this then the file after changes in subsequent commits will only have this minor style inconsistency, so let's change this while we're at it.

---
 builtin/mktag.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
builtin/mktag.c +1 -2
index ff7ac8e0e5..97ca5f28b1 100644
@@ -161,9 +161,8 @@ int cmd_mktag(int argc, const char **argv, const char *prefix)
 	if (argc != 1)
 		usage("git mktag");
 
-	if (strbuf_read(&buf, 0, 0) < 0) {
+	if (strbuf_read(&buf, 0, 0) < 0)
 		die_errno("could not read from stdin");
-	}
 
 	/* Verify it for some basic sanity: it needs to start with
 	   "object <sha1>\ntype\ntagger " */
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#173 lore raw

PATCH v4 13/20 mktag: use puts(str) instead of printf("%s\n", str)

In reply to Ævar Arnfjörð Bjarmason

This introduces no functional change, but refactors the print-out of the hash at the end to do the same thing with less code.

---
 builtin/mktag.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
builtin/mktag.c +1 -1
index 97ca5f28b1..d89a3c201d 100644
@@ -173,6 +173,6 @@ int cmd_mktag(int argc, const char **argv, const char *prefix)
 		die("unable to write tag file");
 
 	strbuf_release(&buf);
-	printf("%s\n", oid_to_hex(&result));
+	puts(oid_to_hex(&result));
 	return 0;
 }
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#174 lore raw

PATCH v4 14/20 mktag: use fsck instead of custom verify_tag()

In reply to Ævar Arnfjörð Bjarmason

Change the validation logic in "mktag" to use fsck's fsck_tag() instead of its own custom parser. Curiously the logic for both dates back to the same commit[1]. Let's unify them so we're not maintaining two sets functions to verify that a tag is OK.

The behavior of fsck_tag() and the old "mktag" code being removed here is different in few aspects.

I think it makes sense to remove some of those checks, namely:

A. fsck only cares that the timezone matches [-+][0-9]{4}. The mktag
code disallowed values larger than 1400.

Yes there's currently no timezone with a greater offset[2], but since we allow any number of non-offical timezones (e.g. +1234) passing this through seems fine. Git also won't break in the future if e.g. French Polynesia decides it needs to outdo the Line Islands when it comes to timezone extravagance.

B. fsck allows missing author names such as "tagger <email>", mktag
wouldn't, but would allow e.g. "tagger [2 spaces] <email>" (but not "tagger [1 space] <email>"). Now we allow all of these.

C. Like B, but "mktag" disallowed spaces in the <email> part, fsck
allows it.

In some ways fsck_tag() is stricter than "mktag" was, namely:

D. fsck disallows zero-padded dates, but mktag didn't care. So
e.g. the timestamp "0000000000 +0000" produces an error now. A test in "t1006-cat-file.sh" relied on this, it's been changed to use "hash-object" (without fsck) instead.

There was one check I deemed worth keeping by porting it over to fsck_tag():

E. "mktag" did not allow any custom headers, and by extension (as an
empty commit is allowed) also forbade an extra stray trailing newline after the headers it knew about.

Add a new check in the "ignore" category to fsck and use it. This somewhat abuses the facility added in efaba7cc77f (fsck:
optionally ignore specific fsck issues completely, 2015-06-22).

This is somewhat of hack, but probably the least invasive change we can make here. The fsck command will shuffle these categories around, e.g. under --strict the "info" becomes a "warn" and "warn" becomes "error". Existing users of fsck's (and others, e.g. index-pack) --strict option rely on this.

So we need to put something into a category that'll be ignored by all existing users of the API. Pretending that fsck.extraHeaderEntry=error ("ignore" by default) was set serves to do this for us.

1. ec4465adb38 (Add "tag" objects that can be used to sign other objects., 2005-04-25)

2. https://en.wikipedia.org/wiki/List_of_UTC_time_offsets

---
 Documentation/git-mktag.txt |   9 +-
 builtin/mktag.c             | 196 +++++++++---------------------------
 fsck.c                      |  32 +++++-
 fsck.h                      |   9 ++
 t/t1006-cat-file.sh         |   2 +-
 t/t3800-mktag.sh            |  63 ++++++------
 6 files changed, 127 insertions(+), 184 deletions(-)
Documentation/git-mktag.txt +8 -1
index adc63f6d4e..03cd63d9f6 100644
@@ -25,7 +25,14 @@ write a tag found in `my-tag`:
     git hash-object -t tag -w --stdin <my-tag
 
 The difference is that mktag will die before writing the tag if the
-tag doesn't pass a sanity check.
+tag doesn't pass a linkgit:git-fsck[1] check.
+
+The "fsck" check done mktag is stricter than what linkgit:git-fsck[1]
+would run by default in that all `fsck.<msg-id>` messages are promoted
+from warnings to errors (so e.g. a missing "tagger" line is an error).
+
+Extra headers in the object are also an error under mktag, but ignored
+by linkgit:git-fsck[1]
 
 Tag Format
 ----------
builtin/mktag.c +50 -146

220 lines not on this page · raw message

fsck.c +29 -3
index f82e2fe9e3..bed5e20e03 100644
@@ -80,7 +80,9 @@ static struct oidset gitmodules_done = OIDSET_INIT;
 	/* infos (reported as warnings, but ignored by default) */ \
 	FUNC(GITMODULES_PARSE, INFO) \
 	FUNC(BAD_TAG_NAME, INFO) \
-	FUNC(MISSING_TAGGER_ENTRY, INFO)
+	FUNC(MISSING_TAGGER_ENTRY, INFO) \
+	/* ignored (elevated when requested) */ \
+	FUNC(EXTRA_HEADER_ENTRY, IGNORE)
 
 #define MSG_ID(id, msg_type) FSCK_MSG_##id,
 enum fsck_msg_id {
@@ -911,6 +913,16 @@ static int fsck_tag(const struct object_id *oid, const char *buffer,
 		    unsigned long size, struct fsck_options *options)
 {
 	struct object_id tagged_oid;
+	int tagged_type;
+	return fsck_tag_standalone(oid, buffer, size, options, &tagged_oid,
+				   &tagged_type);
+}
+
+int fsck_tag_standalone(const struct object_id *oid, const char *buffer,
+			unsigned long size, struct fsck_options *options,
+			struct object_id *tagged_oid,
+			int *tagged_type)
+{
 	int ret = 0;
 	char *eol;
 	struct strbuf sb = STRBUF_INIT;
@@ -924,7 +936,7 @@ static int fsck_tag(const struct object_id *oid, const char *buffer,
 		ret = report(options, oid, OBJ_TAG, FSCK_MSG_MISSING_OBJECT, "invalid format - expected 'object' line");
 		goto done;
 	}
-	if (parse_oid_hex(buffer, &tagged_oid, &p) || *p != '\n') {
+	if (parse_oid_hex(buffer, tagged_oid, &p) || *p != '\n') {
 		ret = report(options, oid, OBJ_TAG, FSCK_MSG_BAD_OBJECT_SHA1, "invalid 'object' line format - bad sha1");
 		if (ret)
 			goto done;
@@ -940,7 +952,8 @@ static int fsck_tag(const struct object_id *oid, const char *buffer,
 		ret = report(options, oid, OBJ_TAG, FSCK_MSG_MISSING_TYPE, "invalid format - unexpected end after 'type' line");
 		goto done;
 	}
-	if (type_from_string_gently(buffer, eol - buffer, 1) < 0)
+	*tagged_type = type_from_string_gently(buffer, eol - buffer, 1);
+	if (*tagged_type < 0)
 		ret = report(options, oid, OBJ_TAG, FSCK_MSG_BAD_TYPE, "invalid 'type' value");
 	if (ret)
 		goto done;
@@ -975,6 +988,19 @@ static int fsck_tag(const struct object_id *oid, const char *buffer,
 	else
 		ret = fsck_ident(&buffer, oid, OBJ_TAG, options);
 
+	if (!starts_with(buffer, "\n")) {
+		/*
+		 * The verify_headers() check will allow
+		 * e.g. "[...]tagger <tagger>\nsome
+		 * garbage\n\nmessage" to pass, thinking "some
+		 * garbage" could be a custom header. E.g. "mktag"
+		 * doesn't want any unknown headers.
+		 */
+		ret = report(options, oid, OBJ_TAG, FSCK_MSG_EXTRA_HEADER_ENTRY, "invalid format - extra header(s) after 'tagger'");
+		if (ret)
+			goto done;
+	}
+
 done:
 	strbuf_release(&sb);
 	return ret;
fsck.h +9 -0
index 69cf715e79..29ee4c45e8 100644
@@ -62,6 +62,15 @@ int fsck_walk(struct object *obj, void *data, struct fsck_options *options);
 int fsck_object(struct object *obj, void *data, unsigned long size,
 	struct fsck_options *options);
 
+/*
+ * fsck a tag, and pass info about it back to the caller. This is
+ * exposed fsck_object() internals for git-mktag(1).
+ */
+int fsck_tag_standalone(const struct object_id *oid, const char *buffer,
+			unsigned long size, struct fsck_options *options,
+			struct object_id *tagged_oid,
+			int *tag_type);
+
 /*
  * Some fsck checks are context-dependent, and may end up queued; run this
  * after completing all fsck_object() calls in order to resolve any remaining
t/t1006-cat-file.sh +1 -1
index 2f501d2dc9..5d2dc99b74 100755
@@ -166,7 +166,7 @@ tag_content="$tag_header_without_timestamp 0000000000 +0000
 
 $tag_description"
 
-tag_sha1=$(echo_without_newline "$tag_content" | git mktag)
+tag_sha1=$(echo_without_newline "$tag_content" | git hash-object -t tag --stdin -w)
 tag_size=$(strlen "$tag_content")
 
 run_tests 'tag' $tag_sha1 $tag_size "$tag_content" "$tag_content" 1
t/t3800-mktag.sh +30 -33

235 lines not on this page · raw message

Signature · 1 line
2.29.2.222.g5d2a92d10f8
#175 lore raw

Re: [PATCH v4 14/20] mktag: use fsck instead of custom verify_tag()

Quoted 11 lines · Ævar Arnfjörð Bjarmason
builtin/mktag.c +10 -0
+static int verify_object_in_tag(struct object_id *tagged_oid, int *tagged_type)
 {
+	int ret;
+	enum object_type type;
+	unsigned long size;
+	void *buffer;
+	const struct object_id *repl;
+
+	buffer = read_object_file(tagged_oid, &type, &size);
+	if (!buffer)
+		die("could not read tagged object '%s'",

builtin/mktag.c line 48

Quoted 10 lines
builtin/mktag.c +10 -0
+		    oid_to_hex(tagged_oid));
+	if (type != *tagged_type)
+		die("object '%s' tagged as '%s', but is a '%s' type",
+		    oid_to_hex(tagged_oid),
+		    type_name(*tagged_type), type_name(type));
+
+	repl = lookup_replace_object(the_repository, tagged_oid);
+	ret = check_object_signature(the_repository, repl,
+				     buffer, size, type_name(*tagged_type));
+	free(buffer);

This is not new, but I wonder if the object whose name is repl can be of different type than the object whose name is tagged_oid, and what happens when such a replacement is defined.

fsck.h int fsck_walk(struct object *obj, void *data, struct fsck_options *options); · line 73

Quoted 12 lines
fsck.h +10 -0
+	return ret;
 }
...
+/*
+ * fsck a tag, and pass info about it back to the caller. This is
+ * exposed fsck_object() internals for git-mktag(1).
+ */
+int fsck_tag_standalone(const struct object_id *oid, const char *buffer,
+			unsigned long size, struct fsck_options *options,
+			struct object_id *tagged_oid,
+			int *tag_type);
+

OK.

t/t3800-mktag.sh too short for a tag · line 47

Quoted 6 lines
t/t3800-mktag.sh +0 -0
index c6826762d9..d20adf0544 100755
@@ -47,7 +47,7 @@ too short for a tag
 EOF

OK. This seems to show rather nicely what checks got loosened (and nothing got tightened it seems).

Thanks. Looking good so far.

Quoted 261 lines
t/t3800-mktag.sh +30 -33
 check_verify_failure 'Tag object length check' \
-	'^error: .*size wrong.*$'
+	'^error:.* missingObject:'
 
 ############################################################
 #  2. object line label check
@@ -60,7 +60,7 @@ tagger . <> 0 +0000
 
 EOF
 
-check_verify_failure '"object" line label check' '^error: char0: .*"object "$'
+check_verify_failure '"object" line label check' '^error:.* missingObject:'
 
 ############################################################
 #  3. object line hash check
@@ -73,7 +73,7 @@ tagger . <> 0 +0000
 
 EOF
 
-check_verify_failure '"object" line SHA1 check' '^error: char7: .*SHA1 hash$'
+check_verify_failure '"object" line check' '^error:.* badObjectSha1:'
 
 ############################################################
 #  4. type line label check
@@ -86,7 +86,7 @@ tagger . <> 0 +0000
 
 EOF
 
-check_verify_failure '"type" line label check' '^error: char.*: .*"\\ntype "$'
+check_verify_failure '"type" line label check' '^error:.* missingTypeEntry:'
 
 ############################################################
 #  5. type line eol check
@@ -94,7 +94,7 @@ check_verify_failure '"type" line label check' '^error: char.*: .*"\\ntype "$'
 echo "object $head" >tag.sig
 printf "type tagsssssssssssssssssssssssssssssss" >>tag.sig
 
-check_verify_failure '"type" line eol check' '^error: char.*: .*"\\n"$'
+check_verify_failure '"type" line eol check' '^error:.* unterminatedHeader:'
 
 ############################################################
 #  6. tag line label check #1
@@ -108,7 +108,7 @@ tagger . <> 0 +0000
 EOF
 
 check_verify_failure '"tag" line label check #1' \
-	'^error: char.*: no "tag " found$'
+	'^error:.* missingTagEntry:'
 
 ############################################################
 #  7. tag line label check #2
@@ -120,7 +120,7 @@ tag
 EOF
 
 check_verify_failure '"tag" line label check #2' \
-	'^error: char.*: no "tag " found$'
+	'^error:.* badType:'
 
 ############################################################
 #  8. type line type-name length check
@@ -132,7 +132,7 @@ tag mytag
 EOF
 
 check_verify_failure '"type" line type-name length check' \
-	'^error: char.*: type too long$'
+	'^error:.* badType:'
 
 ############################################################
 #  9. verify object (hash/type) check
@@ -146,7 +146,7 @@ tagger . <> 0 +0000
 EOF
 
 check_verify_failure 'verify object (hash/type) check -- correct type, nonexisting object' \
-	'^error: char7: could not verify object.*$'
+	'^fatal: could not read tagged object'
 
 cat >tag.sig <<EOF
 object $head
@@ -157,7 +157,7 @@ tagger . <> 0 +0000
 EOF
 
 check_verify_failure 'verify object (hash/type) check -- made-up type, nonexisting object' \
-	'^fatal: invalid object type'
+	'^error:.* badType:'
 
 cat >tag.sig <<EOF
 object $(test_oid deadbeef)
@@ -168,7 +168,7 @@ tagger . <> 0 +0000
 EOF
 
 check_verify_failure 'verify object (hash/type) check -- incorrect type, valid object' \
-	'^error: char7: could not verify object.*$'
+	'^error:.* badType:'
 
 cat >tag.sig <<EOF
 object $head
@@ -179,7 +179,7 @@ tagger . <> 0 +0000
 EOF
 
 check_verify_failure 'verify object (hash/type) check -- incorrect type, valid object' \
-	'^error: char7: could not verify object'
+	'^fatal: object.*tagged as.*tree.*but is.*commit'
 
 ############################################################
 # 10. verify tag-name check
@@ -193,7 +193,7 @@ tagger . <> 0 +0000
 EOF
 
 check_verify_failure 'verify tag-name check' \
-	'^error: char.*: could not verify tag name$'
+	'^error:.* badTagName:'
 
 ############################################################
 # 11. tagger line label check #1
@@ -207,7 +207,7 @@ This is filler
 EOF
 
 check_verify_failure '"tagger" line label check #1' \
-	'^error: char.*: could not find "tagger "$'
+	'^error:.* missingTaggerEntry:'
 
 ############################################################
 # 12. tagger line label check #2
@@ -222,10 +222,10 @@ This is filler
 EOF
 
 check_verify_failure '"tagger" line label check #2' \
-	'^error: char.*: could not find "tagger "$'
+	'^error:.* missingTaggerEntry:'
 
 ############################################################
-# 13. disallow missing tag author name
+# 13. allow missing tag author name like fsck
 
 cat >tag.sig <<EOF
 object $head
@@ -236,8 +236,7 @@ tagger  <> 0 +0000
 This is filler
 EOF
 
-check_verify_failure 'disallow missing tag author name' \
-	'^error: char.*: missing tagger name$'
+test_expect_mktag_success 'allow missing tag author name'
 
 ############################################################
 # 14. disallow missing tag author name
@@ -252,7 +251,7 @@ tagger T A Gger <
 EOF
 
 check_verify_failure 'disallow malformed tagger' \
-	'^error: char.*: malformed tagger field$'
+	'^error:.* badEmail:'
 
 ############################################################
 # 15. allow empty tag email
@@ -268,7 +267,7 @@ EOF
 test_expect_mktag_success 'allow empty tag email'
 
 ############################################################
-# 16. disallow spaces in tag email
+# 16. allow spaces in tag email like fsck
 
 cat >tag.sig <<EOF
 object $head
@@ -278,8 +277,7 @@ tagger T A Gger <tag [email protected]> 0 +0000
 
 EOF
 
-check_verify_failure 'disallow spaces in tag email' \
-	'^error: char.*: malformed tagger field$'
+test_expect_mktag_success 'allow spaces in tag email like fsck'
 
 ############################################################
 # 17. disallow missing tag timestamp
@@ -293,7 +291,7 @@ tagger T A Gger <[email protected]>__
 EOF
 
 check_verify_failure 'disallow missing tag timestamp' \
-	'^error: char.*: missing tag timestamp$'
+	'^error:.* badDate:'
 
 ############################################################
 # 18. detect invalid tag timestamp1
@@ -307,7 +305,7 @@ tagger T A Gger <[email protected]> Tue Mar 25 15:47:44 2008
 EOF
 
 check_verify_failure 'detect invalid tag timestamp1' \
-	'^error: char.*: missing tag timestamp$'
+	'^error:.* badDate:'
 
 ############################################################
 # 19. detect invalid tag timestamp2
@@ -321,7 +319,7 @@ tagger T A Gger <[email protected]> 2008-03-31T12:20:15-0500
 EOF
 
 check_verify_failure 'detect invalid tag timestamp2' \
-	'^error: char.*: malformed tag timestamp$'
+	'^error:.* badDate:'
 
 ############################################################
 # 20. detect invalid tag timezone1
@@ -335,7 +333,7 @@ tagger T A Gger <[email protected]> 1206478233 GMT
 EOF
 
 check_verify_failure 'detect invalid tag timezone1' \
-	'^error: char.*: malformed tag timezone$'
+	'^error:.* badTimezone:'
 
 ############################################################
 # 21. detect invalid tag timezone2
@@ -349,10 +347,10 @@ tagger T A Gger <[email protected]> 1206478233 +  30
 EOF
 
 check_verify_failure 'detect invalid tag timezone2' \
-	'^error: char.*: malformed tag timezone$'
+	'^error:.* badTimezone:'
 
 ############################################################
-# 22. detect invalid tag timezone3
+# 22. allow invalid tag timezone3 (the maximum is -1200/+1400)
 
 cat >tag.sig <<EOF
 object $head
@@ -362,8 +360,7 @@ tagger T A Gger <[email protected]> 1206478233 -1430
 
 EOF
 
-check_verify_failure 'detect invalid tag timezone3' \
-	'^error: char.*: malformed tag timezone$'
+test_expect_mktag_success 'allow invalid tag timezone'
 
 ############################################################
 # 23. detect invalid header entry
@@ -378,7 +375,7 @@ this line should not be here
 EOF
 
 check_verify_failure 'detect invalid header entry' \
-	'^error: char.*: trailing garbage in tag header$'
+	'^error:.* extraHeaderEntry:'
 
 cat >tag.sig <<EOF
 object $head
@@ -412,7 +409,7 @@ tagger T A Gger <[email protected]> 1206478233 -0500$space
 EOF
 
 check_verify_failure 'extra whitespace at end of headers' \
-	'^error: char.*: malformed tag timezone$'
+	'^error:.* badTimezone:'
 
 cat >tag.sig <<EOF
 object $head
@@ -422,7 +419,7 @@ tagger T A Gger <[email protected]> 1206478233 -0500
 EOF
 
 check_verify_failure 'disallow no header / body newline separator' \
-	'^error: char.*: trailing garbage in tag header$'
+	'^error:.* extraHeaderEntry:'
 
 ############################################################
 # 24. create valid tag
#176 lore raw

PATCH v4 15/20 fsck: make fsck_config() re-usable

In reply to Ævar Arnfjörð Bjarmason

Move the fsck_config() function from builtin/fsck.c to fsck.[ch]. This allows for re-using it in other tools that expose fsck logic and want to support its configuration variables.

A logical continuation of this change would be to use a common function for all of {fetch,receive}.fsck.* and fsck.*. See 5d477a334a6 (fsck (receive-pack): allow demoting errors to warnings, 2015-06-22) and my own 1362df0d413 (fetch: implement fetch.fsck.*, 2018-07-27) for the relevant code.

However, those routines want to not parse the fsck.skipList into OIDs, but rather pass them along with the --strict option to another process. It would be possible to refactor that whole thing so we support e.g. a "fetch." prefix, then just keep track of the skiplist as a filename instead of parsing it, and learn to spew that all out from our internal structures into something we can append to the --strict option.

But instead I'm planning to re-use this in "mktag", which'll just re-use these "fsck.*" variables as-is.

---
 builtin/fsck.c | 20 +-------------------
 fsck.c         | 25 +++++++++++++++++++++++++
 fsck.h         |  7 +++++++
 3 files changed, 33 insertions(+), 19 deletions(-)
builtin/fsck.c +1 -19
index fbf26cafcf..821e7798c7 100644
@@ -73,25 +73,7 @@ static const char *printable_type(const struct object_id *oid,
 
 static int fsck_config(const char *var, const char *value, void *cb)
 {
-	if (strcmp(var, "fsck.skiplist") == 0) {
-		const char *path;
-		struct strbuf sb = STRBUF_INIT;
-
-		if (git_config_pathname(&path, var, value))
-			return 1;
-		strbuf_addf(&sb, "skiplist=%s", path);
-		free((char *)path);
-		fsck_set_msg_types(&fsck_obj_options, sb.buf);
-		strbuf_release(&sb);
-		return 0;
-	}
-
-	if (skip_prefix(var, "fsck.", &var)) {
-		fsck_set_msg_type(&fsck_obj_options, var, value);
-		return 0;
-	}
-
-	return git_default_config(var, value, cb);
+	return fsck_config_internal(var, value, cb, &fsck_obj_options);
 }
 
 static int objerror(struct object *obj, const char *err)
fsck.c +25 -0
index bed5e20e03..9067a290a2 100644
@@ -1310,3 +1310,28 @@ int fsck_finish(struct fsck_options *options)
 	oidset_clear(&gitmodules_done);
 	return ret;
 }
+
+int fsck_config_internal(const char *var, const char *value, void *cb,
+			 struct fsck_options *options)
+{
+	if (strcmp(var, "fsck.skiplist") == 0) {
+		const char *path;
+		struct strbuf sb = STRBUF_INIT;
+
+		if (git_config_pathname(&path, var, value))
+			return 1;
+		strbuf_addf(&sb, "skiplist=%s", path);
+		free((char *)path);
+		fsck_set_msg_types(options, sb.buf);
+		strbuf_release(&sb);
+		return 0;
+	}
+
+	if (skip_prefix(var, "fsck.", &var)) {
+		fsck_set_msg_type(options, var, value);
+		return 0;
+	}
+
+	return git_default_config(var, value, cb);
+}
+
fsck.h +7 -0
index 29ee4c45e8..423c467feb 100644
@@ -103,4 +103,11 @@ void fsck_put_object_name(struct fsck_options *options,
 const char *fsck_describe_object(struct fsck_options *options,
 				 const struct object_id *oid);
 
+/*
+ * git_config() callback for use by fsck-y tools that want to support
+ * fsck.<msg> fsck.skipList etc.
+ */
+int fsck_config_internal(const char *var, const char *value, void *cb,
+			 struct fsck_options *options);
+
 #endif
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#177 lore raw

PATCH v4 16/20 mktag: allow turning off fsck.extraHeaderEntry

In reply to Ævar Arnfjörð Bjarmason

In earlier commits mktag learned to use the fsck machinery, at which point we needed to add fsck.extraHeaderEntry so it could be as strict about extra headers as it's been ever since it was implemented.

But it's not nice to need to switch away from "mktag" to "hash-object"
+ manual "fsck" just because you'd like to have an extra header. So let's support turning it off by getting "fsck.*" variables from the config.

Pedantically speaking it's still not possible to make "mktag" behave just like "hash-object -t tag" does, since we're unconditionally going to check the referenced object in verify_object_in_tag(), which is our own check, and not one that exists in fsck.c.

But the spirit of "this works like fsck" is preserved, in that if you created such a tag with "hash-object" and did a full "fsck" on the repository it would also error out about that invalid object, it just wouldn't emit the same message as fsck does.

---
 Documentation/git-mktag.txt |  5 ++++-
 builtin/mktag.c             | 11 ++++++++++-
 t/t3800-mktag.sh            | 14 ++++++++++++++
 3 files changed, 28 insertions(+), 2 deletions(-)
Documentation/git-mktag.txt +4 -1
index 03cd63d9f6..d04657b219 100644
@@ -32,7 +32,10 @@ would run by default in that all `fsck.<msg-id>` messages are promoted
 from warnings to errors (so e.g. a missing "tagger" line is an error).
 
 Extra headers in the object are also an error under mktag, but ignored
-by linkgit:git-fsck[1]
+by linkgit:git-fsck[1]. This extra check can be turned off by setting
+the appropriate `fsck.<msg-id>` varible:
+
+    git -c fsck.extraHeaderEntry=ignore mktag <my-tag-with-headers
 
 Tag Format
 ----------
builtin/mktag.c +10 -1
index 4dd35bc79e..373926d7e0 100644
@@ -3,6 +3,14 @@
 #include "replace-object.h"
 #include "object-store.h"
 #include "fsck.h"
+#include "config.h"
+
+static struct fsck_options fsck_options = FSCK_OPTIONS_STRICT;
+
+static int mktag_config(const char *var, const char *value, void *cb)
+{
+	return fsck_config_internal(var, value, cb, &fsck_options);
+}
 
 static int mktag_fsck_error_func(struct fsck_options *o,
 				 const struct object_id *oid,
@@ -53,7 +61,6 @@ static int verify_object_in_tag(struct object_id *tagged_oid, int *tagged_type)
 int cmd_mktag(int argc, const char **argv, const char *prefix)
 {
 	struct strbuf buf = STRBUF_INIT;
-	struct fsck_options fsck_options = FSCK_OPTIONS_STRICT;
 	struct object_id tagged_oid;
 	int tagged_type;
 	struct object_id result;
@@ -66,6 +73,8 @@ int cmd_mktag(int argc, const char **argv, const char *prefix)
 
 	fsck_options.error_func = mktag_fsck_error_func;
 	fsck_set_msg_type(&fsck_options, "extraheaderentry", "warn");
+	/* config might set fsck.extraHeaderEntry=* again */
+	git_config(mktag_config, NULL);
 	if (fsck_tag_standalone(NULL, buf.buf, buf.len, &fsck_options,
 				&tagged_oid, &tagged_type))
 		die("tag on stdin did not pass our strict fsck check");
t/t3800-mktag.sh +14 -0
index d20adf0544..1019b42378 100755
@@ -377,6 +377,20 @@ EOF
 check_verify_failure 'detect invalid header entry' \
 	'^error:.* extraHeaderEntry:'
 
+test_expect_success 'invalid header entry config & fsck' '
+	test_must_fail git mktag <tag.sig &&
+	test_must_fail git -c fsck.extraHeaderEntry=error mktag <tag.sig &&
+	test_must_fail git -c fsck.extraHeaderEntry=warn mktag <tag.sig &&
+	git -c fsck.extraHeaderEntry=ignore mktag <tag.sig &&
+	git fsck &&
+	env GIT_TEST_GETTEXT_POISON=false \
+		git -c fsck.extraHeaderEntry=warn fsck 2>err &&
+	grep "warning .*extraHeaderEntry:" err &&
+	test_must_fail env GIT_TEST_GETTEXT_POISON=false \
+		git -c fsck.extraHeaderEntry=error 2>err fsck &&
+	grep "error .* extraHeaderEntry:" err
+'
+
 cat >tag.sig <<EOF
 object $head
 type commit
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#178 lore raw

Re: [PATCH v4 16/20] mktag: allow turning off fsck.extraHeaderEntry

Quoted 27 lines · Ævar Arnfjörð Bjarmason

In earlier commits mktag learned to use the fsck machinery, at which point we needed to add fsck.extraHeaderEntry so it could be as strict about extra headers as it's been ever since it was implemented.

But it's not nice to need to switch away from "mktag" to "hash-object"
+ manual "fsck" just because you'd like to have an extra header. So let's support turning it off by getting "fsck.*" variables from the config.

Pedantically speaking it's still not possible to make "mktag" behave just like "hash-object -t tag" does, since we're unconditionally going to check the referenced object in verify_object_in_tag(), which is our own check, and not one that exists in fsck.c.

But the spirit of "this works like fsck" is preserved, in that if you created such a tag with "hash-object" and did a full "fsck" on the repository it would also error out about that invalid object, it just wouldn't emit the same message as fsck does.

---

t/t3800-mktag.sh +0 -0
index d20adf0544..1019b42378 100755

t/t3800-mktag.sh EOF · line 386

Quoted 10 lines
t/t3800-mktag.sh +7 -0
@@ -377,6 +377,20 @@ EOF
 check_verify_failure 'detect invalid header entry' \
 	'^error:.* extraHeaderEntry:'
 
+test_expect_success 'invalid header entry config & fsck' '
+	test_must_fail git mktag <tag.sig &&
+	test_must_fail git -c fsck.extraHeaderEntry=error mktag <tag.sig &&
+	test_must_fail git -c fsck.extraHeaderEntry=warn mktag <tag.sig &&
+	git -c fsck.extraHeaderEntry=ignore mktag <tag.sig &&
+	git fsck &&
+	env GIT_TEST_GETTEXT_POISON=false \

Why this 'env GIT_TEST_GETTEXT_POISON=false' here...

t/t3800-mktag.sh EOF · line 388

Quoted 2 lines
t/t3800-mktag.sh +2 -0
+		git -c fsck.extraHeaderEntry=warn fsck 2>err &&
+	grep "warning .*extraHeaderEntry:" err &&

... instead of test_i18ngrep here?

Quoted 11 lines
t/t3800-mktag.sh +5 -0
+	test_must_fail env GIT_TEST_GETTEXT_POISON=false \
+		git -c fsck.extraHeaderEntry=error 2>err fsck &&
+	grep "error .* extraHeaderEntry:" err
+'
+
 cat >tag.sig <<EOF
 object $head
 type commit
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#179 lore raw

PATCH v4 17/20 mktag: allow omitting the header/body \n separator

In reply to Ævar Arnfjörð Bjarmason

Change mktag's acceptance rules to accept an empty body without an empty line after the header again. This fixes an ancient unintended dregression in "mktag".

When "mktag" was introduced in ec4465adb3 (Add "tag" objects that can be used to sign other objects., 2005-04-25) the input checks were much looser. When it was documented it 6cfec03680 (mktag: minimally update the description., 2007-06-10) it was clearly intended for this \n to be optional:

The message, when [it] exists, is separated by a blank line from the header.

But then in e0aaf781f6 (mktag.c: improve verification of tagger field and tests, 2008-03-27) this was made an error, seemingly by accident. It was just a result of the general header checks, and all the tests after that patch have a trailing empty line (but did not before).

Let's allow this again, and tweak the test semantics changed in e0aaf781f6 to remove the redundant empty line. New tests added in previous commits of mine already added an explicit test for allowing the empty line between header and body.

---
 fsck.c           | 2 ++
 t/t3800-mktag.sh | 6 +++---
 2 files changed, 5 insertions(+), 3 deletions(-)
fsck.c +2 -0
index 9067a290a2..29c1eaca4c 100644
@@ -987,6 +987,8 @@ int fsck_tag_standalone(const struct object_id *oid, const char *buffer,
 	}
 	else
 		ret = fsck_ident(&buffer, oid, OBJ_TAG, options);
+	if (!*buffer)
+		goto done;
 
 	if (!starts_with(buffer, "\n")) {
 		/*
t/t3800-mktag.sh +3 -3
index 1019b42378..9383f7795b 100755
@@ -432,8 +432,9 @@ tag mytag
 tagger T A Gger <[email protected]> 1206478233 -0500
 EOF
 
-check_verify_failure 'disallow no header / body newline separator' \
-	'^error:.* extraHeaderEntry:'
+test_expect_success 'allow no header / body newline separator' '
+	git mktag <tag.sig
+'
 
 ############################################################
 # 24. create valid tag
@@ -443,7 +444,6 @@ object $head
 type commit
 tag mytag
 tagger T A Gger <[email protected]> 1206478233 -0500
-
 EOF
 
 test_expect_mktag_success 'create valid tag object'
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#180 lore raw

PATCH v4 18/20 mktag: convert to parse-options

In reply to Ævar Arnfjörð Bjarmason

Convert the "mktag" command to use parse-options.h instead of its own ad-hoc argc handling. This doesn't matter much in practice since it doesn't support any options, but removes another special-case in our codebase, and makes it easier to add options to it in the future.

It does marginally improve the situation for programs that want to execute git commands in a consistent manner and e.g. always use --end-of-options. E.g. "gitaly" does that, and has a blacklist of built-ins that don't support --end-of-options. This is one less special case for it and other similar programs to support.

---
 builtin/mktag.c  | 14 ++++++++++++--
 t/t3800-mktag.sh | 12 ++++++++++++
 2 files changed, 24 insertions(+), 2 deletions(-)
builtin/mktag.c +12 -2
index 373926d7e0..18b8492f4d 100644
@@ -1,10 +1,16 @@
 #include "builtin.h"
+#include "parse-options.h"
 #include "tag.h"
 #include "replace-object.h"
 #include "object-store.h"
 #include "fsck.h"
 #include "config.h"
 
+static char const * const builtin_mktag_usage[] = {
+	N_("git mktag"),
+	NULL
+};
+
 static struct fsck_options fsck_options = FSCK_OPTIONS_STRICT;
 
 static int mktag_config(const char *var, const char *value, void *cb)
@@ -60,13 +66,17 @@ static int verify_object_in_tag(struct object_id *tagged_oid, int *tagged_type)
 
 int cmd_mktag(int argc, const char **argv, const char *prefix)
 {
+	static struct option builtin_mktag_options[] = {
+		OPT_END(),
+	};
 	struct strbuf buf = STRBUF_INIT;
 	struct object_id tagged_oid;
 	int tagged_type;
 	struct object_id result;
 
-	if (argc != 1)
-		usage("git mktag");
+	argc = parse_options(argc, argv, NULL,
+			     builtin_mktag_options,
+			     builtin_mktag_usage, 0);
 
 	if (strbuf_read(&buf, 0, 0) < 0)
 		die_errno("could not read from stdin");
t/t3800-mktag.sh +12 -0
index 9383f7795b..177ab1682d 100755
@@ -39,6 +39,18 @@ test_expect_success 'setup' '
 	head=$(git rev-parse --verify HEAD)
 '
 
+test_expect_success 'basic usage' '
+	cat >tag.sig <<-EOF &&
+	object $head
+	type commit
+	tag mytag
+	tagger T A Gger <[email protected]> 1206478233 -0500
+	EOF
+	git mktag <tag.sig &&
+	git mktag --end-of-options <tag.sig &&
+	test_expect_code 129 git mktag --unknown-option
+'
+
 ############################################################
 #  1. length check
 
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#181 lore raw

PATCH v4 19/20 mktag: mark strings for translation

In reply to Ævar Arnfjörð Bjarmason

Mark the errors mktag might emit for translation. This is a plumbing command, but the errors it emits are intended to be human-readable.

---
 builtin/mktag.c  | 16 ++++++++--------
 t/t3800-mktag.sh |  3 ++-
 2 files changed, 10 insertions(+), 9 deletions(-)
builtin/mktag.c +8 -8
index 18b8492f4d..9b04b61c2b 100644
@@ -31,10 +31,10 @@ static int mktag_fsck_error_func(struct fsck_options *o,
 		 * like missing "tagger" lines are "only" warnings
 		 * under fsck, we've always considered them an error.
 		 */
-		fprintf_ln(stderr, "error: tag input does not pass fsck: %s", message);
+		fprintf_ln(stderr, _("error: tag input does not pass fsck: %s"), message);
 		return 1;
 	default:
-		BUG("%d (FSCK_IGNORE?) should never trigger this callback",
+		BUG(_("%d (FSCK_IGNORE?) should never trigger this callback"),
 		    msg_type);
 	}
 }
@@ -49,10 +49,10 @@ static int verify_object_in_tag(struct object_id *tagged_oid, int *tagged_type)
 
 	buffer = read_object_file(tagged_oid, &type, &size);
 	if (!buffer)
-		die("could not read tagged object '%s'",
+		die(_("could not read tagged object '%s'"),
 		    oid_to_hex(tagged_oid));
 	if (type != *tagged_type)
-		die("object '%s' tagged as '%s', but is a '%s' type",
+		die(_("object '%s' tagged as '%s', but is a '%s' type"),
 		    oid_to_hex(tagged_oid),
 		    type_name(*tagged_type), type_name(type));
 
@@ -79,7 +79,7 @@ int cmd_mktag(int argc, const char **argv, const char *prefix)
 			     builtin_mktag_usage, 0);
 
 	if (strbuf_read(&buf, 0, 0) < 0)
-		die_errno("could not read from stdin");
+		die_errno(_("could not read from stdin"));
 
 	fsck_options.error_func = mktag_fsck_error_func;
 	fsck_set_msg_type(&fsck_options, "extraheaderentry", "warn");
@@ -87,13 +87,13 @@ int cmd_mktag(int argc, const char **argv, const char *prefix)
 	git_config(mktag_config, NULL);
 	if (fsck_tag_standalone(NULL, buf.buf, buf.len, &fsck_options,
 				&tagged_oid, &tagged_type))
-		die("tag on stdin did not pass our strict fsck check");
+		die(_("tag on stdin did not pass our strict fsck check"));
 
 	if (verify_object_in_tag(&tagged_oid, &tagged_type))
-		die("tag on stdin did not refer to a valid object");
+		die(_("tag on stdin did not refer to a valid object"));
 
 	if (write_object_file(buf.buf, buf.len, tag_type, &result) < 0)
-		die("unable to write tag file");
+		die(_("unable to write tag file"));
 
 	strbuf_release(&buf);
 	puts(oid_to_hex(&result));
t/t3800-mktag.sh +2 -1
index 177ab1682d..2e8b718379 100755
@@ -14,7 +14,8 @@ test_description='git mktag: tag object verify test'
 check_verify_failure () {
 	expect="$2"
 	test_expect_success "$1" '
-		test_must_fail git mktag <tag.sig 2>message &&
+		test_must_fail env GIT_TEST_GETTEXT_POISON=false \
+			git mktag <tag.sig 2>message &&
 		grep "$expect" message
 	'
 }
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#182 lore raw

PATCH v4 20/20 mktag: add a --no-strict option

In reply to Ævar Arnfjörð Bjarmason

Now that mktag has been migrated to use the fsck machinery to check its input, it makes sense to teach it to run in the equivalent of "git fsck"'s default mode, instead of hardcoding "git fsck --strict". Let's do that and support the "--no-strict" option.

Since this is a new option we don't need to cater to parse-option.c's default of automatically supporting --strict. So let's use PARSE_OPT_NONEG, using a new trivial helper macro.

---
 Documentation/git-mktag.txt |  7 +++++++
 builtin/mktag.c             |  9 +++++++++
 parse-options.h             |  1 +
 t/t3800-mktag.sh            | 35 ++++++++++++++++++++++++-----------
 4 files changed, 41 insertions(+), 11 deletions(-)
Documentation/git-mktag.txt +7 -0
index d04657b219..05e49b6787 100644
@@ -11,6 +11,13 @@ SYNOPSIS
 [verse]
 'git mktag'
 
+OPTIONS
+-------
+
+--no-strict::
+	By default mktag turns on the equivalent of
+	linkgit:git-fsck[1] `--strict` mode. This disables it.
+
 DESCRIPTION
 -----------
 
builtin/mktag.c +9 -0
index 9b04b61c2b..a6a4612247 100644
@@ -10,6 +10,7 @@ static char const * const builtin_mktag_usage[] = {
 	N_("git mktag"),
 	NULL
 };
+static int option_no_strict;
 
 static struct fsck_options fsck_options = FSCK_OPTIONS_STRICT;
 
@@ -25,6 +26,12 @@ static int mktag_fsck_error_func(struct fsck_options *o,
 {
 	switch (msg_type) {
 	case FSCK_WARN:
+		if (option_no_strict) {
+			fprintf_ln(stderr, _("warning: tag input does not pass fsck: %s"), message);
+			return 0;
+
+		}
+		/* fallthrough */
 	case FSCK_ERROR:
 		/*
 		 * We treat both warnings and errors as errors, things
@@ -67,6 +74,8 @@ static int verify_object_in_tag(struct object_id *tagged_oid, int *tagged_type)
 int cmd_mktag(int argc, const char **argv, const char *prefix)
 {
 	static struct option builtin_mktag_options[] = {
+		OPT_NO_BOOL(0, "no-strict", &option_no_strict,
+			    N_("don't do strict fsck checks")),
 		OPT_END(),
 	};
 	struct strbuf buf = STRBUF_INIT;
parse-options.h +1 -0
index 7030d8f3da..90d6a817d7 100644
@@ -166,6 +166,7 @@ struct option {
 #define OPT_COUNTUP(s, l, v, h)     OPT_COUNTUP_F(s, l, v, h, 0)
 #define OPT_SET_INT(s, l, v, h, i)  OPT_SET_INT_F(s, l, v, h, i, 0)
 #define OPT_BOOL(s, l, v, h)        OPT_BOOL_F(s, l, v, h, 0)
+#define OPT_NO_BOOL(s, l, v, h)     OPT_BOOL_F(s, l, v, h, PARSE_OPT_NONEG)
 #define OPT_HIDDEN_BOOL(s, l, v, h) { OPTION_SET_INT, (s), (l), (v), NULL, \
 				      (h), PARSE_OPT_NOARG | PARSE_OPT_HIDDEN, NULL, 1}
 #define OPT_CMDMODE(s, l, v, h, i)  { OPTION_SET_INT, (s), (l), (v), NULL, \
t/t3800-mktag.sh +24 -11
index 2e8b718379..b436ae1e44 100755
@@ -12,12 +12,16 @@ test_description='git mktag: tag object verify test'
 # given in the expect.pat file.
 
 check_verify_failure () {
-	expect="$2"
-	test_expect_success "$1" '
+	test_expect_success "$1" "
 		test_must_fail env GIT_TEST_GETTEXT_POISON=false \
 			git mktag <tag.sig 2>message &&
-		grep "$expect" message
-	'
+		grep '$2' message &&
+		if test '$3' != '--no-strict'
+		then
+			test_must_fail git mktag --no-strict <tag.sig 2>message.no-strict &&
+			grep '$2' message.no-strict
+		fi
+	"
 }
 
 test_expect_mktag_success() {
@@ -49,7 +53,8 @@ test_expect_success 'basic usage' '
 	EOF
 	git mktag <tag.sig &&
 	git mktag --end-of-options <tag.sig &&
-	test_expect_code 129 git mktag --unknown-option
+	test_expect_code 129 git mktag --unknown-option &&
+	test_expect_code 129 git mktag --strict
 '
 
 ############################################################
@@ -60,7 +65,7 @@ too short for a tag
 EOF
 
 check_verify_failure 'Tag object length check' \
-	'^error:.* missingObject:'
+	'^error:.* missingObject:' 'strict'
 
 ############################################################
 #  2. object line label check
@@ -206,7 +211,7 @@ tagger . <> 0 +0000
 EOF
 
 check_verify_failure 'verify tag-name check' \
-	'^error:.* badTagName:'
+	'^error:.* badTagName:' '--no-strict'
 
 ############################################################
 # 11. tagger line label check #1
@@ -220,7 +225,7 @@ This is filler
 EOF
 
 check_verify_failure '"tagger" line label check #1' \
-	'^error:.* missingTaggerEntry:'
+	'^error:.* missingTaggerEntry:' '--no-strict'
 
 ############################################################
 # 12. tagger line label check #2
@@ -235,7 +240,7 @@ This is filler
 EOF
 
 check_verify_failure '"tagger" line label check #2' \
-	'^error:.* missingTaggerEntry:'
+	'^error:.* missingTaggerEntry:' '--no-strict'
 
 ############################################################
 # 13. allow missing tag author name like fsck
@@ -264,7 +269,7 @@ tagger T A Gger <
 EOF
 
 check_verify_failure 'disallow malformed tagger' \
-	'^error:.* badEmail:'
+	'^error:.* badEmail:' '--no-strict'
 
 ############################################################
 # 15. allow empty tag email
@@ -388,13 +393,21 @@ this line should not be here
 EOF
 
 check_verify_failure 'detect invalid header entry' \
-	'^error:.* extraHeaderEntry:'
+	'^error:.* extraHeaderEntry:' '--no-strict'
 
 test_expect_success 'invalid header entry config & fsck' '
 	test_must_fail git mktag <tag.sig &&
+	git mktag --no-strict <tag.sig &&
+
 	test_must_fail git -c fsck.extraHeaderEntry=error mktag <tag.sig &&
+	test_must_fail git -c fsck.extraHeaderEntry=error mktag --no-strict <tag.sig &&
+
 	test_must_fail git -c fsck.extraHeaderEntry=warn mktag <tag.sig &&
+	git -c fsck.extraHeaderEntry=warn mktag --no-strict <tag.sig &&
+
 	git -c fsck.extraHeaderEntry=ignore mktag <tag.sig &&
+	git -c fsck.extraHeaderEntry=ignore mktag --no-strict <tag.sig &&
+
 	git fsck &&
 	env GIT_TEST_GETTEXT_POISON=false \
 		git -c fsck.extraHeaderEntry=warn fsck 2>err &&
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#183 lore raw

Re: [PATCH v4 20/20] mktag: add a --no-strict option

Quoted 8 lines · Ævar Arnfjörð Bjarmason

Now that mktag has been migrated to use the fsck machinery to check its input, it makes sense to teach it to run in the equivalent of "git fsck"'s default mode, instead of hardcoding "git fsck --strict". Let's do that and support the "--no-strict" option.

Since this is a new option we don't need to cater to parse-option.c's default of automatically supporting --strict. So let's use PARSE_OPT_NONEG, using a new trivial helper macro.

Hmph, I would have thought that the internal file-scope static done like

    static int option_strict = 1;

with OPT_BOOL() would be more natural, than a double negation.

What problem does this new OPT_NO_BOOL() thing solve?

Quoted 175 lines
---
 Documentation/git-mktag.txt |  7 +++++++
 builtin/mktag.c             |  9 +++++++++
 parse-options.h             |  1 +
 t/t3800-mktag.sh            | 35 ++++++++++++++++++++++++-----------
 4 files changed, 41 insertions(+), 11 deletions(-)
Documentation/git-mktag.txt +7 -0
index d04657b219..05e49b6787 100644
@@ -11,6 +11,13 @@ SYNOPSIS
 [verse]
 'git mktag'
 
+OPTIONS
+-------
+
+--no-strict::
+	By default mktag turns on the equivalent of
+	linkgit:git-fsck[1] `--strict` mode. This disables it.
+
 DESCRIPTION
 -----------
 
builtin/mktag.c +9 -0
index 9b04b61c2b..a6a4612247 100644
@@ -10,6 +10,7 @@ static char const * const builtin_mktag_usage[] = {
 	N_("git mktag"),
 	NULL
 };
+static int option_no_strict;
 
 static struct fsck_options fsck_options = FSCK_OPTIONS_STRICT;
 
@@ -25,6 +26,12 @@ static int mktag_fsck_error_func(struct fsck_options *o,
 {
 	switch (msg_type) {
 	case FSCK_WARN:
+		if (option_no_strict) {
+			fprintf_ln(stderr, _("warning: tag input does not pass fsck: %s"), message);
+			return 0;
+
+		}
+		/* fallthrough */
 	case FSCK_ERROR:
 		/*
 		 * We treat both warnings and errors as errors, things
@@ -67,6 +74,8 @@ static int verify_object_in_tag(struct object_id *tagged_oid, int *tagged_type)
 int cmd_mktag(int argc, const char **argv, const char *prefix)
 {
 	static struct option builtin_mktag_options[] = {
+		OPT_NO_BOOL(0, "no-strict", &option_no_strict,
+			    N_("don't do strict fsck checks")),
 		OPT_END(),
 	};
 	struct strbuf buf = STRBUF_INIT;
parse-options.h +1 -0
index 7030d8f3da..90d6a817d7 100644
@@ -166,6 +166,7 @@ struct option {
 #define OPT_COUNTUP(s, l, v, h)     OPT_COUNTUP_F(s, l, v, h, 0)
 #define OPT_SET_INT(s, l, v, h, i)  OPT_SET_INT_F(s, l, v, h, i, 0)
 #define OPT_BOOL(s, l, v, h)        OPT_BOOL_F(s, l, v, h, 0)
+#define OPT_NO_BOOL(s, l, v, h)     OPT_BOOL_F(s, l, v, h, PARSE_OPT_NONEG)
 #define OPT_HIDDEN_BOOL(s, l, v, h) { OPTION_SET_INT, (s), (l), (v), NULL, \
 				      (h), PARSE_OPT_NOARG | PARSE_OPT_HIDDEN, NULL, 1}
 #define OPT_CMDMODE(s, l, v, h, i)  { OPTION_SET_INT, (s), (l), (v), NULL, \
t/t3800-mktag.sh +24 -11
index 2e8b718379..b436ae1e44 100755
@@ -12,12 +12,16 @@ test_description='git mktag: tag object verify test'
 # given in the expect.pat file.
 
 check_verify_failure () {
-	expect="$2"
-	test_expect_success "$1" '
+	test_expect_success "$1" "
 		test_must_fail env GIT_TEST_GETTEXT_POISON=false \
 			git mktag <tag.sig 2>message &&
-		grep "$expect" message
-	'
+		grep '$2' message &&
+		if test '$3' != '--no-strict'
+		then
+			test_must_fail git mktag --no-strict <tag.sig 2>message.no-strict &&
+			grep '$2' message.no-strict
+		fi
+	"
 }
 
 test_expect_mktag_success() {
@@ -49,7 +53,8 @@ test_expect_success 'basic usage' '
 	EOF
 	git mktag <tag.sig &&
 	git mktag --end-of-options <tag.sig &&
-	test_expect_code 129 git mktag --unknown-option
+	test_expect_code 129 git mktag --unknown-option &&
+	test_expect_code 129 git mktag --strict
 '
 
 ############################################################
@@ -60,7 +65,7 @@ too short for a tag
 EOF
 
 check_verify_failure 'Tag object length check' \
-	'^error:.* missingObject:'
+	'^error:.* missingObject:' 'strict'
 
 ############################################################
 #  2. object line label check
@@ -206,7 +211,7 @@ tagger . <> 0 +0000
 EOF
 
 check_verify_failure 'verify tag-name check' \
-	'^error:.* badTagName:'
+	'^error:.* badTagName:' '--no-strict'
 
 ############################################################
 # 11. tagger line label check #1
@@ -220,7 +225,7 @@ This is filler
 EOF
 
 check_verify_failure '"tagger" line label check #1' \
-	'^error:.* missingTaggerEntry:'
+	'^error:.* missingTaggerEntry:' '--no-strict'
 
 ############################################################
 # 12. tagger line label check #2
@@ -235,7 +240,7 @@ This is filler
 EOF
 
 check_verify_failure '"tagger" line label check #2' \
-	'^error:.* missingTaggerEntry:'
+	'^error:.* missingTaggerEntry:' '--no-strict'
 
 ############################################################
 # 13. allow missing tag author name like fsck
@@ -264,7 +269,7 @@ tagger T A Gger <
 EOF
 
 check_verify_failure 'disallow malformed tagger' \
-	'^error:.* badEmail:'
+	'^error:.* badEmail:' '--no-strict'
 
 ############################################################
 # 15. allow empty tag email
@@ -388,13 +393,21 @@ this line should not be here
 EOF
 
 check_verify_failure 'detect invalid header entry' \
-	'^error:.* extraHeaderEntry:'
+	'^error:.* extraHeaderEntry:' '--no-strict'
 
 test_expect_success 'invalid header entry config & fsck' '
 	test_must_fail git mktag <tag.sig &&
+	git mktag --no-strict <tag.sig &&
+
 	test_must_fail git -c fsck.extraHeaderEntry=error mktag <tag.sig &&
+	test_must_fail git -c fsck.extraHeaderEntry=error mktag --no-strict <tag.sig &&
+
 	test_must_fail git -c fsck.extraHeaderEntry=warn mktag <tag.sig &&
+	git -c fsck.extraHeaderEntry=warn mktag --no-strict <tag.sig &&
+
 	git -c fsck.extraHeaderEntry=ignore mktag <tag.sig &&
+	git -c fsck.extraHeaderEntry=ignore mktag --no-strict <tag.sig &&
+
 	git fsck &&
 	env GIT_TEST_GETTEXT_POISON=false \
 		git -c fsck.extraHeaderEntry=warn fsck 2>err &&
#184 lore raw

Re: [PATCH v4 20/20] mktag: add a --no-strict option

In reply to Ævar Arnfjörð Bjarmason
Quoted 16 lines · Ævar Arnfjörð Bjarmason

Now that mktag has been migrated to use the fsck machinery to check its input, it makes sense to teach it to run in the equivalent of "git fsck"'s default mode, instead of hardcoding "git fsck --strict". Let's do that and support the "--no-strict" option.

Since this is a new option we don't need to cater to parse-option.c's default of automatically supporting --strict. So let's use PARSE_OPT_NONEG, using a new trivial helper macro.

t/t3800-mktag.sh +0 -0
index 2e8b718379..b436ae1e44 100755

t/t3800-mktag.sh test_description='git mktag: tag object verify test' · line 17

Quoted 8 lines
t/t3800-mktag.sh +1 -2
@@ -12,12 +12,16 @@ test_description='git mktag: tag object verify test'
 # given in the expect.pat file.
 
 check_verify_failure () {
-	expect="$2"
-	test_expect_success "$1" '
+	test_expect_success "$1" "
 		test_must_fail env GIT_TEST_GETTEXT_POISON=false \
 			git mktag <tag.sig 2>message &&

So this 'git mktag' is always invoked with GIT_TEST_GETTEXT_POISON=false (instead of, for some reason, using test_i18ngrep on the next line)...

t/t3800-mktag.sh test_description='git mktag: tag object verify test' · line 21

Quoted 6 lines
t/t3800-mktag.sh +4 -2
-		grep "$expect" message
-	'
+		grep '$2' message &&
+		if test '$3' != '--no-strict'
+		then
+			test_must_fail git mktag --no-strict <tag.sig 2>message.no-strict &&

Wouldn't a GIT_TEST_GETTEXT_POISON=false be necessary for this 'git mktag' as well? Or, alternatively (preferably?), a test_i18ngrep below.

Quoted 6 lines
t/t3800-mktag.sh +3 -0
+			grep '$2' message.no-strict
+		fi
+	"
 }
 
 test_expect_mktag_success() {
#185 lore raw

PATCH v3 9/10 fsck: make fsck_config() re-usable

In reply to Ævar Arnfjörð Bjarmason

Move the fsck_config() function from builtin/fsck.c to fsck.[ch]. This allows for re-using it in other tools that expose fsck logic and want to support its configuration variables.

A logical continuation of this change would be to use a common function for all of {fetch,receive}.fsck.* and fsck.*. See 5d477a334a6 (fsck (receive-pack): allow demoting errors to warnings, 2015-06-22) and my own 1362df0d413 (fetch: implement fetch.fsck.*, 2018-07-27) for the relevant code.

However, those routines want to not parse the fsck.skipList into OIDs, but rather pass them along with the --strict option to another process. It would be possible to refactor that whole thing so we support e.g. a "fetch." prefix, then just keep track of the skiplist as a filename instead of parsing it, and learn to spew that all out from our internal structures into something we can append to the --strict option.

But instead I'm planning to re-use this in "mktag", which'll just re-use these "fsck.*" variables as-is.

---
 builtin/fsck.c | 20 +-------------------
 fsck.c         | 25 +++++++++++++++++++++++++
 fsck.h         |  7 +++++++
 3 files changed, 33 insertions(+), 19 deletions(-)
builtin/fsck.c +1 -19
index fbf26cafcfd..821e7798c70 100644
@@ -73,25 +73,7 @@ static const char *printable_type(const struct object_id *oid,
 
 static int fsck_config(const char *var, const char *value, void *cb)
 {
-	if (strcmp(var, "fsck.skiplist") == 0) {
-		const char *path;
-		struct strbuf sb = STRBUF_INIT;
-
-		if (git_config_pathname(&path, var, value))
-			return 1;
-		strbuf_addf(&sb, "skiplist=%s", path);
-		free((char *)path);
-		fsck_set_msg_types(&fsck_obj_options, sb.buf);
-		strbuf_release(&sb);
-		return 0;
-	}
-
-	if (skip_prefix(var, "fsck.", &var)) {
-		fsck_set_msg_type(&fsck_obj_options, var, value);
-		return 0;
-	}
-
-	return git_default_config(var, value, cb);
+	return fsck_config_internal(var, value, cb, &fsck_obj_options);
 }
 
 static int objerror(struct object *obj, const char *err)
fsck.c +25 -0
index bed5e20e03b..9067a290a2e 100644
@@ -1310,3 +1310,28 @@ int fsck_finish(struct fsck_options *options)
 	oidset_clear(&gitmodules_done);
 	return ret;
 }
+
+int fsck_config_internal(const char *var, const char *value, void *cb,
+			 struct fsck_options *options)
+{
+	if (strcmp(var, "fsck.skiplist") == 0) {
+		const char *path;
+		struct strbuf sb = STRBUF_INIT;
+
+		if (git_config_pathname(&path, var, value))
+			return 1;
+		strbuf_addf(&sb, "skiplist=%s", path);
+		free((char *)path);
+		fsck_set_msg_types(options, sb.buf);
+		strbuf_release(&sb);
+		return 0;
+	}
+
+	if (skip_prefix(var, "fsck.", &var)) {
+		fsck_set_msg_type(options, var, value);
+		return 0;
+	}
+
+	return git_default_config(var, value, cb);
+}
+
fsck.h +7 -0
index 29ee4c45e87..423c467feb7 100644
@@ -103,4 +103,11 @@ void fsck_put_object_name(struct fsck_options *options,
 const char *fsck_describe_object(struct fsck_options *options,
 				 const struct object_id *oid);
 
+/*
+ * git_config() callback for use by fsck-y tools that want to support
+ * fsck.<msg> fsck.skipList etc.
+ */
+int fsck_config_internal(const char *var, const char *value, void *cb,
+			 struct fsck_options *options);
+
 #endif
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#186 lore raw

PATCH v3 10/10 mktag: allow turning off fsck.extraHeaderEntry

In reply to Ævar Arnfjörð Bjarmason

In earlier commits mktag learned to use the fsck machinery, at which point we needed to add fsck.extraHeaderEntry so it could be as strict about extra headers as it's been ever since it was implemented.

But it's not nice to need to switch away from "mktag" to "hash-object"
+ manual "fsck" just because you'd like to have an extra header. So let's support turning it off by getting "fsck.*" variables from the config.

Pedantically speaking it's still not possible to make "mktag" behave just like "hash-object -t tag" does, since we're unconditionally going to check the referenced object in verify_object_in_tag(), which is our own check, and not one that exists in fsck.c.

But the spirit of "this works like fsck" is preserved, in that if you created such a tag with "hash-object" and did a full "fsck" on the repository it would also error out about that invalid object, it just wouldn't emit the same message as fsck does.

---
 Documentation/git-mktag.txt |  9 +++++++--
 builtin/mktag.c             | 11 ++++++++++-
 t/t3800-mktag.sh            | 14 ++++++++++++++
 3 files changed, 31 insertions(+), 3 deletions(-)
Documentation/git-mktag.txt +7 -2
index b164ab563bd..e1506dde561 100644
@@ -30,8 +30,13 @@ tag doesn't pass a linkgit:git-fsck[1] check.
 The "fsck" check done mktag is is stricter than what
 linkgit:git-fsck[1] would run by default in that all `fsck.<msg-id>`
 messages are promoted from warnings to errors (so e.g. a missing
-"tagger" line is an error). Extra headers in the object are also an
-error under mktag, but ignored by linkgit:git-fsck[1].
+"tagger" line is an error).
+
+Extra headers in the object are also an error under mktag, but ignored
+by linkgit:git-fsck[1]. This extra check can be turned off by setting
+the appropriate `fsck.<msg-id>` varible:
+
+    git -c fsck.extraHeaderEntry=ignore mktag <my-tag-with-headers
 
 Tag Format
 ----------
builtin/mktag.c +10 -1
index 5765cde0032..c1398ccf133 100644
@@ -3,6 +3,14 @@
 #include "replace-object.h"
 #include "object-store.h"
 #include "fsck.h"
+#include "config.h"
+
+static struct fsck_options fsck_options = FSCK_OPTIONS_STRICT;
+
+static int mktag_config(const char *var, const char *value, void *cb)
+{
+	return fsck_config_internal(var, value, cb, &fsck_options);
+}
 
 static int mktag_fsck_error_func(struct fsck_options *o,
 				 const struct object_id *oid,
@@ -53,7 +61,6 @@ static int verify_object_in_tag(struct object_id *tagged_oid, int *tagged_type)
 int cmd_mktag(int argc, const char **argv, const char *prefix)
 {
 	struct strbuf buf = STRBUF_INIT;
-	struct fsck_options fsck_options = FSCK_OPTIONS_STRICT;
 	struct object_id tagged_oid;
 	int tagged_type;
 	struct object_id result;
@@ -66,6 +73,8 @@ int cmd_mktag(int argc, const char **argv, const char *prefix)
 
 	fsck_options.error_func = mktag_fsck_error_func;
 	fsck_set_msg_type(&fsck_options, "extraheaderentry", "warn");
+	/* config might set fsck.extraHeaderEntry=* again */
+	git_config(mktag_config, NULL);
 	if (fsck_tag_standalone(NULL, buf.buf, buf.len, &fsck_options,
 				&tagged_oid, &tagged_type))
 		die("tag on stdin did not pass our strict fsck check");
t/t3800-mktag.sh +14 -0
index 19b3c1bca9c..59082e06277 100755
@@ -375,6 +375,20 @@ EOF
 check_verify_failure 'detect invalid header entry' \
 	'^error:.* extraHeaderEntry:'
 
+test_expect_success 'invalid header entry config & fsck' '
+	test_must_fail git mktag <tag.sig &&
+	test_must_fail git -c fsck.extraHeaderEntry=error mktag <tag.sig &&
+	test_must_fail git -c fsck.extraHeaderEntry=warn mktag <tag.sig &&
+	git -c fsck.extraHeaderEntry=ignore mktag <tag.sig &&
+	git fsck &&
+	env GIT_TEST_GETTEXT_POISON=false \
+		git -c fsck.extraHeaderEntry=warn fsck 2>err &&
+	grep "warning .*extraHeaderEntry:" err &&
+	test_must_fail env GIT_TEST_GETTEXT_POISON=false \
+		git -c fsck.extraHeaderEntry=error 2>err fsck &&
+	grep "error .* extraHeaderEntry:" err
+'
+
 cat >tag.sig <<EOF
 object $head
 type commit
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#187 lore raw

PATCH v2 1/10 mktag doc: say <hash> not <sha1>

In reply to Ævar Arnfjörð Bjarmason

Change the "mktag" documentation to refer to the input hash as just "hash", not "sha1". This command has supported SHA-256 for a while now.

---
 Documentation/git-mktag.txt | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Documentation/git-mktag.txt +1 -1
index fa6a756123..a158428eb9 100644
@@ -23,7 +23,7 @@ Tag Format
 A tag signature file, to be fed to this command's standard input,
 has a very simple fixed format: four lines of
 
-  object <sha1>
+  object <hash>
   type <typename>
   tag <tagname>
   tagger <tagger>
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#188 lore raw

Re: [PATCH v2 01/10] mktag doc: say <hash> not <sha1>

Quoted 6 lines · Ævar Arnfjörð Bjarmason

Change the "mktag" documentation to refer to the input hash as just "hash", not "sha1". This command has supported SHA-256 for a while now.

---

I think saying <hash> instead of <sha> is a very good idea---it won't tie us to the SHA family of hashes, including SHA-256, forever.

Quoted 16 lines
 Documentation/git-mktag.txt | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Documentation/git-mktag.txt +1 -1
index fa6a756123..a158428eb9 100644
@@ -23,7 +23,7 @@ Tag Format
 A tag signature file, to be fed to this command's standard input,
 has a very simple fixed format: four lines of
 
-  object <sha1>
+  object <hash>
   type <typename>
   tag <tagname>
   tagger <tagger>
#189 lore raw

PATCH v2 2/10 mktag: use default strbuf_read() hint

In reply to Ævar Arnfjörð Bjarmason

Change the hardcoded hint of 2^12 to 0. The default strbuf hint is perfectly fine here, and the only reason we were hardcoding it is because it survived migration from a pre-strbuf fixed-sized buffer.

See fd17f5b5f77 (Replace all read_fd use with strbuf_read, and get rid of it., 2007-09-10) for that migration.

---
 builtin/mktag.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
builtin/mktag.c +1 -1
index 4982d3a93e..ff7ac8e0e5 100644
@@ -161,7 +161,7 @@ int cmd_mktag(int argc, const char **argv, const char *prefix)
 	if (argc != 1)
 		usage("git mktag");
 
-	if (strbuf_read(&buf, 0, 4096) < 0) {
+	if (strbuf_read(&buf, 0, 0) < 0) {
 		die_errno("could not read from stdin");
 	}
 
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#190 lore raw

Re: [PATCH v2 02/10] mktag: use default strbuf_read() hint

builtin/mktag.c int cmd_mktag(int argc, const char **argv, const char *prefix)

Quoted 17 lines · Ævar Arnfjörð Bjarmason

Change the hardcoded hint of 2^12 to 0. The default strbuf hint is perfectly fine here, and the only reason we were hardcoding it is because it survived migration from a pre-strbuf fixed-sized buffer.

See fd17f5b5f77 (Replace all read_fd use with strbuf_read, and get rid of it., 2007-09-10) for that migration.

---
 builtin/mktag.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
builtin/mktag.c +0 -0
index 4982d3a93e..ff7ac8e0e5 100644
@@ -161,7 +161,7 @@ int cmd_mktag(int argc, const char **argv, const char *prefix)

In the lines beyond the pre-context there is a comment that mentions <sha1>, but I think it is sensible not to clean it up with this patch, as it will go away with the main "use fsck machinery" step.

Quoted 7 lines
builtin/mktag.c +1 -1
 	if (argc != 1)
 		usage("git mktag");
 
-	if (strbuf_read(&buf, 0, 4096) < 0) {
+	if (strbuf_read(&buf, 0, 0) < 0) {
 		die_errno("could not read from stdin");
 	}
#191 lore raw

PATCH v2 3/10 mktag: reword write_object_file() error

In reply to Ævar Arnfjörð Bjarmason

Change the error message emitted when write_object_file() fails to make more sense. At this point we're not writing a "tag file" (which as an aside we never do, we just write to stdout). We are writing an annotated tag object, let's say that instead.

---
 builtin/mktag.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
builtin/mktag.c +1 -1
index ff7ac8e0e5..603b55aca0 100644
@@ -171,7 +171,7 @@ int cmd_mktag(int argc, const char **argv, const char *prefix)
 		die("invalid tag signature file");
 
 	if (write_object_file(buf.buf, buf.len, tag_type, &result) < 0)
-		die("unable to write tag file");
+		die("unable to write annotated tag object");
 
 	strbuf_release(&buf);
 	printf("%s\n", oid_to_hex(&result));
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#192 lore raw

Re: [PATCH v2 03/10] mktag: reword write_object_file() error

Quoted 15 lines · Ævar Arnfjörð Bjarmason

Change the error message emitted when write_object_file() fails to make more sense. At this point we're not writing a "tag file" (which as an aside we never do, we just write to stdout). We are writing an annotated tag object, let's say that instead.

---
 builtin/mktag.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
builtin/mktag.c +0 -0
index ff7ac8e0e5..603b55aca0 100644

builtin/mktag.c int cmd_mktag(int argc, const char **argv, const char *prefix) · line 174

Quoted 5 lines
builtin/mktag.c +1 -1
@@ -171,7 +171,7 @@ int cmd_mktag(int argc, const char **argv, const char *prefix)
 		die("invalid tag signature file");
 
 	if (write_object_file(buf.buf, buf.len, tag_type, &result) < 0)
-		die("unable to write tag file");
+		die("unable to write annotated tag object");

"write an annotated tag object"?

It is not just this call to die(), but we'd eventually want to _(l10n/i18n) these messages. Perhaps in a separate step but on all messages fed to die/error/warn in this file.

Quoted 3 lines
builtin/mktag.c +0 -0
 	strbuf_release(&buf);
 	printf("%s\n", oid_to_hex(&result));
#193 lore raw

Re: [PATCH v2 03/10] mktag: reword write_object_file() error

Quoted 19 lines · Junio C Hamano
Quoted 17 lines · Ævar Arnfjörð Bjarmason

Change the error message emitted when write_object_file() fails to make more sense. At this point we're not writing a "tag file" (which as an aside we never do, we just write to stdout). We are writing an annotated tag object, let's say that instead.

---
 builtin/mktag.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
builtin/mktag.c +0 -0
index ff7ac8e0e5..603b55aca0 100644
@@ -171,7 +171,7 @@ int cmd_mktag(int argc, const char **argv, const char *prefix)
 		die("invalid tag signature file");
 
Quoted 5 lines

builtin/mktag.c int cmd_mktag(int argc, const char **argv, const char *prefix) · line 174

Quoted 3 lines
builtin/mktag.c +1 -1
 	if (write_object_file(buf.buf, buf.len, tag_type, &result) < 0)
-		die("unable to write tag file");
+		die("unable to write annotated tag object");

"write an annotated tag object"?

Actually, there is no such thing as an "unannotated tag object". Perhaps _("unable to create a tag object")?

Quoted 8 lines

It is not just this call to die(), but we'd eventually want to _(l10n/i18n) these messages. Perhaps in a separate step but on all messages fed to die/error/warn in this file.

Quoted 3 lines
builtin/mktag.c +0 -0
 	strbuf_release(&buf);
 	printf("%s\n", oid_to_hex(&result));
#194 lore raw

PATCH v2 4/10 mktag: remove redundant braces in one-line body "if"

In reply to Ævar Arnfjörð Bjarmason

This minor stylistic churn is usually something we'd avoid, but if we don't do this then the file after changes in subsequent commits will only have this minor style inconsistency, so let's change this while we're at it.

---
 builtin/mktag.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
builtin/mktag.c +1 -2
index 603b55aca0..dc354828f7 100644
@@ -161,9 +161,8 @@ int cmd_mktag(int argc, const char **argv, const char *prefix)
 	if (argc != 1)
 		usage("git mktag");
 
-	if (strbuf_read(&buf, 0, 0) < 0) {
+	if (strbuf_read(&buf, 0, 0) < 0)
 		die_errno("could not read from stdin");
-	}
 
 	/* Verify it for some basic sanity: it needs to start with
 	   "object <sha1>\ntype\ntagger " */
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#195 lore raw

PATCH v2 5/10 mktag tests: don't needlessly use a subshell

In reply to Ævar Arnfjörð Bjarmason

The use of a subshell dates back to e9b20943b77 (t/t3800: do not use a temporary file to hold expected result., 2008-01-04). It's not needed anymore, if it ever was.

---
 t/t3800-mktag.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
t/t3800-mktag.sh +1 -1
index d696aa4e52..0e411e3c45 100755
@@ -14,7 +14,7 @@ test_description='git mktag: tag object verify test'
 check_verify_failure () {
 	expect="$2"
 	test_expect_success "$1" '
-		( test_must_fail git mktag <tag.sig 2>message ) &&
+		test_must_fail git mktag <tag.sig 2>message &&
 		grep "$expect" message
 	'
 }
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#196 lore raw

PATCH v2 6/10 mktag tests: remove needless SHA-1 hardcoding

In reply to Ævar Arnfjörð Bjarmason

Change the tests amended in acb49d1cc8b (t3800: make hash-size independent, 2019-08-18) even more to make them independent of either SHA-1 or SHA-256.

Some of these tests were failing for the wrong reasons. The first one being modified here would fail because the line starts with "xxxxxx" instead of "object", the rest of the line doesn't matter.

Let's just put a valid hash on the rest of the line anyway to narrow the test down for just the s/object/xxxxxx/ case.

The second one being modified here would fail under GIT_TEST_DEFAULT_HASH=sha256 because <some sha-1 length garbage> is an invalid SHA-256, but we should really be testing <some sha-256 length garbage> when under SHA-256.

This doesn't really matter since we should be able to trust other parts of the code to validate things in the 0-9a-f range, but let's keep it for good measure.

There's a later test which tests an invalid SHA which looks like a valid one, to stress the "We refuse to tag something we can't verify[...]" logic in mktag.c.

But here we're testing for a SHA-length string which contains characters outside of the /[0-9a-f]/i set.

---
 t/t3800-mktag.sh | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
t/t3800-mktag.sh +3 -3
index 0e411e3c45..e9d7799537 100755
@@ -43,7 +43,7 @@ check_verify_failure 'Tag object length check' \
 #  2. object line label check
 
 cat >tag.sig <<EOF
-xxxxxx 139e9b33986b1c2670fff52c5067603117b3e895
+xxxxxx $head
 type tag
 tag mytag
 tagger . <> 0 +0000
@@ -53,10 +53,10 @@ EOF
 check_verify_failure '"object" line label check' '^error: char0: .*"object "$'
 
 ############################################################
-#  3. object line SHA1 check
+#  3. object line SHA check
 
 cat >tag.sig <<EOF
-object zz9e9b33986b1c2670fff52c5067603117b3e895
+object $(echo $head | tr 0-9a-f z)
 type tag
 tag mytag
 tagger . <> 0 +0000
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#197 lore raw

Re: [PATCH v2 06/10] mktag tests: remove needless SHA-1 hardcoding

Quoted 37 lines · Ævar Arnfjörð Bjarmason

Change the tests amended in acb49d1cc8b (t3800: make hash-size independent, 2019-08-18) even more to make them independent of either SHA-1 or SHA-256.

Some of these tests were failing for the wrong reasons. The first one being modified here would fail because the line starts with "xxxxxx" instead of "object", the rest of the line doesn't matter.

Let's just put a valid hash on the rest of the line anyway to narrow the test down for just the s/object/xxxxxx/ case.

The second one being modified here would fail under GIT_TEST_DEFAULT_HASH=sha256 because <some sha-1 length garbage> is an invalid SHA-256, but we should really be testing <some sha-256 length garbage> when under SHA-256.

This doesn't really matter since we should be able to trust other parts of the code to validate things in the 0-9a-f range, but let's keep it for good measure.

There's a later test which tests an invalid SHA which looks like a valid one, to stress the "We refuse to tag something we can't verify[...]" logic in mktag.c.

But here we're testing for a SHA-length string which contains characters outside of the /[0-9a-f]/i set.

---
 t/t3800-mktag.sh | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
t/t3800-mktag.sh +0 -0
index 0e411e3c45..e9d7799537 100755

t/t3800-mktag.sh check_verify_failure 'Tag object length check' \ · line 49

Quoted 8 lines
t/t3800-mktag.sh +1 -1
@@ -43,7 +43,7 @@ check_verify_failure 'Tag object length check' \
 #  2. object line label check
 
 cat >tag.sig <<EOF
-xxxxxx 139e9b33986b1c2670fff52c5067603117b3e895
+xxxxxx $head
 type tag
 tag mytag
 tagger . <> 0 +0000

These steps to prepare the input and the expected output outside the test_expect_* block look too old-fashioned but they are consistently so, so let's let them pass, at least inside this series.

t/t3800-mktag.sh EOF · line 56

Quoted 6 lines
t/t3800-mktag.sh +1 -1
@@ -53,10 +53,10 @@ EOF
 check_verify_failure '"object" line label check' '^error: char0: .*"object "$'
 
 ############################################################
-#  3. object line SHA1 check
+#  3. object line SHA check

Shouldn't we say hash instead of SHA for consistency?

Quoted 7 lines
t/t3800-mktag.sh +1 -1
 cat >tag.sig <<EOF
-object zz9e9b33986b1c2670fff52c5067603117b3e895
+object $(echo $head | tr 0-9a-f z)
 type tag
 tag mytag
 tagger . <> 0 +0000
#198 lore raw

PATCH v2 7/10 mktag tests: improve verify_object() test coverage

In reply to Ævar Arnfjörð Bjarmason

The verify_object() function in "mktag.c" is tasked with ensuring that our tag refers to a valid object.

The existing test for this might fail because it was also testing that "type taggg" didn't refer to a valid object type (it should be "type tag"), or because we referred to a valid object but got the type wrong.

Let's split these tests up, so we're testing all combinations of a non-existing object and in invalid/wrong "type" lines.

---
 t/t3800-mktag.sh | 37 +++++++++++++++++++++++++++++++++++--
 1 file changed, 35 insertions(+), 2 deletions(-)
t/t3800-mktag.sh +35 -2
index e9d7799537..a6a67b6f27 100755
@@ -125,19 +125,52 @@ check_verify_failure '"type" line type-name length check' \
 	'^error: char.*: type too long$'
 
 ############################################################
-#  9. verify object (SHA1/type) check
+#  9. verify object (SHA/type) check
 
 cat >tag.sig <<EOF
 object $(test_oid deadbeef)
+type tag
+tag mytag
+tagger . <> 0 +0000
+
+EOF
+
+check_verify_failure 'verify object (SHA/type) check -- correct type, nonexisting object' \
+	'^error: char7: could not verify object.*$'
+
+cat >tag.sig <<EOF
+object $head
 type tagggg
 tag mytag
 tagger . <> 0 +0000
 
 EOF
 
-check_verify_failure 'verify object (SHA1/type) check' \
+check_verify_failure 'verify object (SHA/type) check -- made-up type, nonexisting object' \
+	'^fatal: invalid object type'
+
+cat >tag.sig <<EOF
+object $(test_oid deadbeef)
+type tagggg
+tag mytag
+tagger . <> 0 +0000
+
+EOF
+
+check_verify_failure 'verify object (SHA/type) check -- incorrect type, valid object' \
 	'^error: char7: could not verify object.*$'
 
+cat >tag.sig <<EOF
+object $head
+type tree
+tag mytag
+tagger . <> 0 +0000
+
+EOF
+
+check_verify_failure 'verify object (SHA/type) check -- incorrect type, valid object' \
+	'^error: char7: could not verify object'
+
 ############################################################
 # 10. verify tag-name check
 
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#199 lore raw

PATCH v2 8/10 fsck: add new "extra" checks for "mktag"

In reply to Ævar Arnfjörð Bjarmason

Add optional "extra" checks to fsck, these are needed to eventually replace the custom not-quite-fsck code in mktag.c.

The mktag checks differ from fsck_tag() in several ways, one of those is that fsck doesn't know how to refuse an object with custom headers, and isn't strict about header and body newline separation.

Teach it how to optionally report these. I thought the best way to do that given the current structure of the code was to add a new "extra" category in addition to error/warn/info. Under --strict the "info" becomes a "warn" and "warn" becomes "error". Existing users of fsck's (and others, e.g. index-pack) --strict option rely on this.

By adding an "extra" category and only reporting it based on a flag in fsck_options callers can opt-in to these "extra" messages, which they'll then need to deal with in their own "error_func".

No tests are being added for this new functionality, they're added in a subsequent commit where we teach "mktag" to use this new validation mode.

I'm not changing fsck_commit() to validate commit objects like this either, we could do that, but unlike in the tag case that code wouldn't be used anywhere. If someone wants to write a "mkcommit" they which behaves like "mktag" they can borrow or refactor this logic for use in fsck_commit().

---
 fsck.c | 32 +++++++++++++++++++++++++++++++-
 fsck.h |  2 ++
 2 files changed, 33 insertions(+), 1 deletion(-)
fsck.c +31 -1
index f82e2fe9e3..3c25df2244 100644
@@ -80,7 +80,10 @@ static struct oidset gitmodules_done = OIDSET_INIT;
 	/* infos (reported as warnings, but ignored by default) */ \
 	FUNC(GITMODULES_PARSE, INFO) \
 	FUNC(BAD_TAG_NAME, INFO) \
-	FUNC(MISSING_TAGGER_ENTRY, INFO)
+	FUNC(MISSING_TAGGER_ENTRY, INFO) \
+	/* extra (only reported when requested) */ \
+	FUNC(EXTRA_HEADER_ENTRY, EXTRA) \
+	FUNC(EXTRA_HEADER_BODY_NEWLINE, EXTRA)
 
 #define MSG_ID(id, msg_type) FSCK_MSG_##id,
 enum fsck_msg_id {
@@ -975,6 +978,33 @@ static int fsck_tag(const struct object_id *oid, const char *buffer,
 	else
 		ret = fsck_ident(&buffer, oid, OBJ_TAG, options);
 
+	if (options->extra && *buffer) {
+		if (!starts_with(buffer, "\n")) {
+			/*
+			 * The verify_headers() check will allow
+			 * e.g. "[...]tagger <tagger>\nsome
+			 * garbage\n\nmessage" to pass, thinking "some
+			 * garbage" could be a custom
+			 * header. E.g. "mktag" doesn't want any
+			 * unknown headers.
+			 */
+			ret = report(options, oid, OBJ_TAG, FSCK_MSG_EXTRA_HEADER_ENTRY, "invalid format - extra header(s) after 'tagger'");
+			if (ret)
+				goto done;
+		}
+		if (starts_with(buffer, "\n\n")) {
+			/*
+			 * Some callers such as "mktag" want to
+			 * disallow "[...]tagger
+			 * <tagger>\n\n\nmessage", only allowing a
+			 * single newline for separation.
+			 */
+			ret = report(options, oid, OBJ_TAG, FSCK_MSG_EXTRA_HEADER_BODY_NEWLINE, "invalid format - headers separated body by more than one newline");
+			if (ret)
+				goto done;
+		}
+	}
+
 done:
 	strbuf_release(&sb);
 	return ret;
fsck.h +2 -0
index 69cf715e79..110efc65fd 100644
@@ -6,6 +6,7 @@
 #define FSCK_ERROR 1
 #define FSCK_WARN 2
 #define FSCK_IGNORE 3
+#define FSCK_EXTRA 4
 
 struct fsck_options;
 struct object;
@@ -40,6 +41,7 @@ struct fsck_options {
 	unsigned strict:1;
 	int *msg_type;
 	struct oidset skiplist;
+	unsigned extra:1;
 	kh_oid_map_t *object_names;
 };
 
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#200 lore raw

Re: [PATCH v2 08/10] fsck: add new "extra" checks for "mktag"

Quoted 6 lines · Ævar Arnfjörð Bjarmason

Add optional "extra" checks to fsck, these are needed to eventually replace the custom not-quite-fsck code in mktag.c.

The mktag checks differ from fsck_tag() in several ways, one of those is that fsck doesn't know how to refuse an object with custom headers, and isn't strict about header and body newline separation.

You say "there must be only one blank line between the header and the body", but viewing from the way we parse header and body, I think that such a rule actually forbids a leading blank line in the body and steps into checking whitespace errors---makes readers wonder if we should be also detecting trailing whitespaces on lines, etc.

Is there actually such a check enforced in the original? Or is this a new rule that appeared out of thin air? We'd have to inspect the lines deleted from builtin/mktag.c in the next step, I gues.

Quoted 3 lines

By adding an "extra" category and only reporting it based on a flag in fsck_options callers can opt-in to these "extra" messages, which they'll then need to deal with in their own "error_func".

Makes sense.

#201 lore raw

PATCH v2 9/10 mktag: use fsck instead of custom verify_tag()

In reply to Ævar Arnfjörð Bjarmason

Change the validation logic in "mktag" to use fsck's fsck_tag() instead of its own custom parser. Curiously the logic for both dates back to the same commit[1]. Let's unify them so we're not maintaining two sets functions to verify that a tag is OK.

Moving to fsck_tag() required teaching it to optionally use some validations that only the old mktag code could perform. That was done in an earlier commit, the "extraHeaderEntry" and "extraHeaderBodyNewline" tests being added here make use of that logic.

There was other "mktag" validation logic that I think makes sense to just remove. Namely:

A. fsck only cares that the timezone matches [-+][0-9]{4}. The mktag
code disallowed values larger than 1400.

Yes there's currently no timezone with a greater offset[2], but since we allow any number of non-offical timezones (e.g. +1234) passing this through seems fine. Git also won't break in the future if e.g. French Polynesia decides it needs to outdo the Line Islands when it comes to timezone extravagance.

B. fsck allows missing author names such as "tagger <email>", mktag
wouldn't, but would allow e.g. "tagger [2 spaces] <email>" (but not "tagger [1 space] <email>"). Now we allow all of these.

C. Like B, but "mktag" disallowed spaces in the <email> part, fsck
allows it.

We didn't only lose obscure validation logic, we also gained some:

D. fsck disallows zero-padded dates, but mktag didn't care. So
e.g. the timestamp "0000000000 +0000" produces an error now. A test in "t1006-cat-file.sh" relied on this, it's been changed to use "hash-object" (without fsck) instead.

1. ec4465adb38 (Add "tag" objects that can be used to sign other objects., 2005-04-25)

2. https://en.wikipedia.org/wiki/List_of_UTC_time_offsets

---
 builtin/mktag.c     | 197 ++++++++++++--------------------------------
 fsck.c              |  15 +++-
 fsck.h              |   9 ++
 t/t1006-cat-file.sh |   2 +-
 t/t3800-mktag.sh    |  78 +++++++++++-------
 5 files changed, 121 insertions(+), 180 deletions(-)
builtin/mktag.c +51 -146
index dc354828f7..54b0d75ff7 100644
@@ -2,160 +2,61 @@
 #include "tag.h"
 #include "replace-object.h"
 #include "object-store.h"
+#include "fsck.h"
 
-/*
- * A signature file has a very simple fixed format: four lines
- * of "object <sha1>" + "type <typename>" + "tag <tagname>" +
- * "tagger <committer>", followed by a blank line, a free-form tag
- * message and a signature block that git itself doesn't care about,
- * but that can be verified with gpg or similar.
- *
- * The first four lines are guaranteed to be at least 83 bytes:
- * "object <sha1>\n" is 48 bytes, "type tag\n" at 9 bytes is the
- * shortest possible type-line, "tag .\n" at 6 bytes is the shortest
- * single-character-tag line, and "tagger . <> 0 +0000\n" at 20 bytes is
- * the shortest possible tagger-line.
- */
-
-/*
- * We refuse to tag something we can't verify. Just because.
- */
-static int verify_object(const struct object_id *oid, const char *expected_type)
+static int mktag_fsck_error_func(struct fsck_options *o,
+				 const struct object_id *oid,
+				 enum object_type object_type,
+				 int msg_type, const char *message)
 {
-	int ret = -1;
-	enum object_type type;
-	unsigned long size;
-	void *buffer = read_object_file(oid, &type, &size);
-	const struct object_id *repl = lookup_replace_object(the_repository, oid);
-
-	if (buffer) {
-		if (type == type_from_string(expected_type)) {
-			ret = check_object_signature(the_repository, repl,
-						     buffer, size,
-						     expected_type);
-		}
-		free(buffer);
+	switch (msg_type) {
+	case FSCK_WARN:
+	case FSCK_ERROR:
+	case FSCK_EXTRA:
+		/*
+		 * We treat both warnings and errors as errors, things
+		 * like missing "tagger" lines are "only" warnings
+		 * under fsck, we've always considered them an error.
+		 */
+		fprintf_ln(stderr, "error: tag input does not pass fsck: %s", message);
+		return 1;
+	default:
+		BUG("%d (FSCK_IGNORE?) should never trigger this callback",
+		    msg_type);
 	}
-	return ret;
 }
 
-static int verify_tag(char *buffer, unsigned long size)
+static int verify_object_in_tag(struct object_id *tagged_oid, int *tagged_type)
 {
-	int typelen;
-	char type[20];
-	struct object_id oid;
-	const char *object, *type_line, *tag_line, *tagger_line, *lb, *rb, *p;
-	size_t len;
-
-	if (size < 84)
-		return error("wanna fool me ? you obviously got the size wrong !");
-
-	buffer[size] = 0;
-
-	/* Verify object line */
-	object = buffer;
-	if (memcmp(object, "object ", 7))
-		return error("char%d: does not start with \"object \"", 0);
-
-	if (parse_oid_hex(object + 7, &oid, &p))
-		return error("char%d: could not get SHA1 hash", 7);
-
-	/* Verify type line */
-	type_line = p + 1;
-	if (memcmp(type_line - 1, "\ntype ", 6))
-		return error("char%d: could not find \"\\ntype \"", 47);
-
-	/* Verify tag-line */
-	tag_line = strchr(type_line, '\n');
-	if (!tag_line)
-		return error("char%"PRIuMAX": could not find next \"\\n\"",
-				(uintmax_t) (type_line - buffer));
-	tag_line++;
-	if (memcmp(tag_line, "tag ", 4) || tag_line[4] == '\n')
-		return error("char%"PRIuMAX": no \"tag \" found",
-				(uintmax_t) (tag_line - buffer));
-
-	/* Get the actual type */
-	typelen = tag_line - type_line - strlen("type \n");
-	if (typelen >= sizeof(type))
-		return error("char%"PRIuMAX": type too long",
-				(uintmax_t) (type_line+5 - buffer));
-
-	memcpy(type, type_line+5, typelen);
-	type[typelen] = 0;
-
-	/* Verify that the object matches */
-	if (verify_object(&oid, type))
-		return error("char%d: could not verify object %s", 7, oid_to_hex(&oid));
-
-	/* Verify the tag-name: we don't allow control characters or spaces in it */
-	tag_line += 4;
-	for (;;) {
-		unsigned char c = *tag_line++;
-		if (c == '\n')
-			break;
-		if (c > ' ')
-			continue;
-		return error("char%"PRIuMAX": could not verify tag name",
-				(uintmax_t) (tag_line - buffer));
-	}
-
-	/* Verify the tagger line */
-	tagger_line = tag_line;
-
-	if (memcmp(tagger_line, "tagger ", 7))
-		return error("char%"PRIuMAX": could not find \"tagger \"",
-			(uintmax_t) (tagger_line - buffer));
-
-	/*
-	 * Check for correct form for name and email
-	 * i.e. " <" followed by "> " on _this_ line
-	 * No angle brackets within the name or email address fields.
-	 * No spaces within the email address field.
-	 */
-	tagger_line += 7;
-	if (!(lb = strstr(tagger_line, " <")) || !(rb = strstr(lb+2, "> ")) ||
-		strpbrk(tagger_line, "<>\n") != lb+1 ||
-		strpbrk(lb+2, "><\n ") != rb)
-		return error("char%"PRIuMAX": malformed tagger field",
-			(uintmax_t) (tagger_line - buffer));
-
-	/* Check for author name, at least one character, space is acceptable */
-	if (lb == tagger_line)
-		return error("char%"PRIuMAX": missing tagger name",
-			(uintmax_t) (tagger_line - buffer));
-
-	/* timestamp, 1 or more digits followed by space */
-	tagger_line = rb + 2;
-	if (!(len = strspn(tagger_line, "0123456789")))
-		return error("char%"PRIuMAX": missing tag timestamp",
-			(uintmax_t) (tagger_line - buffer));
-	tagger_line += len;
-	if (*tagger_line != ' ')
-		return error("char%"PRIuMAX": malformed tag timestamp",
-			(uintmax_t) (tagger_line - buffer));
-	tagger_line++;
-
-	/* timezone, 5 digits [+-]hhmm, max. 1400 */
-	if (!((tagger_line[0] == '+' || tagger_line[0] == '-') &&
-	      strspn(tagger_line+1, "0123456789") == 4 &&
-	      tagger_line[5] == '\n' && atoi(tagger_line+1) <= 1400))
-		return error("char%"PRIuMAX": malformed tag timezone",
-			(uintmax_t) (tagger_line - buffer));
-	tagger_line += 6;
-
-	/* Verify the blank line separating the header from the body */
-	if (*tagger_line != '\n')
-		return error("char%"PRIuMAX": trailing garbage in tag header",
-			(uintmax_t) (tagger_line - buffer));
+	int ret;
+	enum object_type type;
+	unsigned long size;
+	void *buffer;
+	const struct object_id *repl;
+
+	buffer = read_object_file(tagged_oid, &type, &size);
+	if (!buffer)
+		die("could not read tagged object '%s'\n",
+		    oid_to_hex(tagged_oid));
+	if (type != *tagged_type)
+		die("object '%s' tagged as '%s', but is a '%s' type\n",
+		    oid_to_hex(tagged_oid),
+		    type_name(*tagged_type), type_name(type));
+
+	repl = lookup_replace_object(the_repository, tagged_oid);
+	ret = check_object_signature(the_repository, repl,
+				     buffer, size, type_name(*tagged_type));
+	free(buffer);
 
-	/* The actual stuff afterwards we don't care about.. */
-	return 0;
+	return ret;
 }
 
 int cmd_mktag(int argc, const char **argv, const char *prefix)
 {
 	struct strbuf buf = STRBUF_INIT;
+	struct fsck_options fsck_options = FSCK_OPTIONS_STRICT;
+	struct object_id tagged_oid;
+	int tagged_type;
 	struct object_id result;
 
 	if (argc != 1)
@@ -164,10 +65,14 @@ int cmd_mktag(int argc, const char **argv, const char *prefix)
 	if (strbuf_read(&buf, 0, 0) < 0)
 		die_errno("could not read from stdin");
 
-	/* Verify it for some basic sanity: it needs to start with
-	   "object <sha1>\ntype\ntagger " */
-	if (verify_tag(buf.buf, buf.len) < 0)
-		die("invalid tag signature file");
+	fsck_options.extra = 1;
+	fsck_options.error_func = mktag_fsck_error_func;
+	if (fsck_tag_standalone(NULL, buf.buf, buf.len, &fsck_options,
+				&tagged_oid, &tagged_type))
+		die("tag on stdin did not pass our strict fsck check");
+
+	if (verify_object_in_tag(&tagged_oid, &tagged_type))
+		die("tag on stdin did not refer to a valid object");
 
 	if (write_object_file(buf.buf, buf.len, tag_type, &result) < 0)
 		die("unable to write annotated tag object");
fsck.c +13 -2
index 3c25df2244..97da3a13c6 100644
@@ -914,6 +914,16 @@ static int fsck_tag(const struct object_id *oid, const char *buffer,
 		    unsigned long size, struct fsck_options *options)
 {
 	struct object_id tagged_oid;
+	int tagged_type;
+	return fsck_tag_standalone(oid, buffer, size, options, &tagged_oid,
+				   &tagged_type);
+}
+
+int fsck_tag_standalone(const struct object_id *oid, const char *buffer,
+			unsigned long size, struct fsck_options *options,
+			struct object_id *tagged_oid,
+			int *tagged_type)
+{
 	int ret = 0;
 	char *eol;
 	struct strbuf sb = STRBUF_INIT;
@@ -927,7 +937,7 @@ static int fsck_tag(const struct object_id *oid, const char *buffer,
 		ret = report(options, oid, OBJ_TAG, FSCK_MSG_MISSING_OBJECT, "invalid format - expected 'object' line");
 		goto done;
 	}
-	if (parse_oid_hex(buffer, &tagged_oid, &p) || *p != '\n') {
+	if (parse_oid_hex(buffer, tagged_oid, &p) || *p != '\n') {
 		ret = report(options, oid, OBJ_TAG, FSCK_MSG_BAD_OBJECT_SHA1, "invalid 'object' line format - bad sha1");
 		if (ret)
 			goto done;
@@ -943,7 +953,8 @@ static int fsck_tag(const struct object_id *oid, const char *buffer,
 		ret = report(options, oid, OBJ_TAG, FSCK_MSG_MISSING_TYPE, "invalid format - unexpected end after 'type' line");
 		goto done;
 	}
-	if (type_from_string_gently(buffer, eol - buffer, 1) < 0)
+	*tagged_type = type_from_string_gently(buffer, eol - buffer, 1);
+	if (*tagged_type < 0)
 		ret = report(options, oid, OBJ_TAG, FSCK_MSG_BAD_TYPE, "invalid 'type' value");
 	if (ret)
 		goto done;
fsck.h +9 -0
index 110efc65fd..cf43046146 100644
@@ -64,6 +64,15 @@ int fsck_walk(struct object *obj, void *data, struct fsck_options *options);
 int fsck_object(struct object *obj, void *data, unsigned long size,
 	struct fsck_options *options);
 
+/*
+ * fsck a tag, and pass info about it back to the caller. This is
+ * exposed fsck_object() internals for git-mktag(1).
+ */
+int fsck_tag_standalone(const struct object_id *oid, const char *buffer,
+			unsigned long size, struct fsck_options *options,
+			struct object_id *tagged_oid,
+			int *tag_type);
+
 /*
  * Some fsck checks are context-dependent, and may end up queued; run this
  * after completing all fsck_object() calls in order to resolve any remaining
t/t1006-cat-file.sh +1 -1
index 2f501d2dc9..5d2dc99b74 100755
@@ -166,7 +166,7 @@ tag_content="$tag_header_without_timestamp 0000000000 +0000
 
 $tag_description"
 
-tag_sha1=$(echo_without_newline "$tag_content" | git mktag)
+tag_sha1=$(echo_without_newline "$tag_content" | git hash-object -t tag --stdin -w)
 tag_size=$(strlen "$tag_content")
 
 run_tests 'tag' $tag_sha1 $tag_size "$tag_content" "$tag_content" 1
t/t3800-mktag.sh +47 -31

238 lines not on this page · raw message

Signature · 1 line
2.29.2.222.g5d2a92d10f8
#202 lore raw

Re: [PATCH v2 09/10] mktag: use fsck instead of custom verify_tag()

Quoted 5 lines · Ævar Arnfjörð Bjarmason

Change the validation logic in "mktag" to use fsck's fsck_tag() instead of its own custom parser. Curiously the logic for both dates back to the same commit[1]. Let's unify them so we're not maintaining two sets functions to verify that a tag is OK.

Quoted 8 lines

Moving to fsck_tag() required teaching it to optionally use some validations that only the old mktag code could perform. That was done in an earlier commit, the "extraHeaderEntry" and "extraHeaderBodyNewline" tests being added here make use of that logic.

There was other "mktag" validation logic that I think makes sense to just remove. Namely:

I may not agree with the specifics in this list, but I agree that the validation should be consistent between mktag and fsck.

builtin/mktag.c line 20

Quoted 9 lines
builtin/mktag.c +9 -0
+	switch (msg_type) {
+	case FSCK_WARN:
+	case FSCK_ERROR:
+	case FSCK_EXTRA:
+		/*
+		 * We treat both warnings and errors as errors, things
+		 * like missing "tagger" lines are "only" warnings
+		 * under fsck, we've always considered them an error.
+		 */

This is a good comment and design decision.

As far as I am concerned, the only reason we warn on a missing tagger line without erroring out is because very early versions of Git did allow creating such tag objects, and there are annotated tags in the wild that lack the tagger line.

Quoted 16 lines
builtin/mktag.c +5 -7
+		fprintf_ln(stderr, "error: tag input does not pass fsck: %s", message);
+		return 1;
+	default:
+		BUG("%d (FSCK_IGNORE?) should never trigger this callback",
+		    msg_type);
 	}
-	return ret;
 }
 
-static int verify_tag(char *buffer, unsigned long size)
...
-	/* timezone, 5 digits [+-]hhmm, max. 1400 */
-	if (!((tagger_line[0] == '+' || tagger_line[0] == '-') &&
-	      strspn(tagger_line+1, "0123456789") == 4 &&
-	      tagger_line[5] == '\n' && atoi(tagger_line+1) <= 1400))
-		return error("char%"PRIuMAX": malformed tag timezone",

builtin/mktag.c line 153

Quoted 10 lines
builtin/mktag.c +0 -9
-			(uintmax_t) (tagger_line - buffer));
-	tagger_line += 6;
-
-	/* Verify the blank line separating the header from the body */
-	if (*tagger_line != '\n')
-		return error("char%"PRIuMAX": trailing garbage in tag header",
-			(uintmax_t) (tagger_line - buffer));
 
-	/* The actual stuff afterwards we don't care about.. */
-	return 0;

I do not see the "we do not want more than one blank line after the header" in the original, which was one thing I was looking for after seeing what 08/10 did.

builtin/mktag.c int cmd_mktag(int argc, const char **argv, const char *prefix) · line 69

Quoted 2 lines
builtin/mktag.c +2 -0
+	fsck_options.extra = 1;
+	fsck_options.error_func = mktag_fsck_error_func;

I wonder if we really need the new .extra bit member in the fsck_options. Wouldn't it be sufficient to instead teach fsck_error_func() that the extra level is a non-event?

#203 lore raw

Re: [PATCH v2 09/10] mktag: use fsck instead of custom verify_tag()

Quoted 6 lines · Junio C Hamano

I do not see the "we do not want more than one blank line after the header" in the original, which was one thing I was looking for after seeing what 08/10 did.

builtin/mktag.c int cmd_mktag(int argc, const char **argv, const char *prefix) · line 69

Quoted 2 lines
builtin/mktag.c +2 -0
+	fsck_options.extra = 1;
+	fsck_options.error_func = mktag_fsck_error_func;

Hmph, recent "released" versions of Git seems to be fine when

--- >8 ------ >8 ------ >8 ------ >8 ------ >8 --- object 72ffeb997eaf999f6938b2a7e0d9a75dcceaa311 type commit
tag tester
tagger Junio C Hamano <[email protected]> 1606860947 -0800

a message after a blank
--- 8< ------ 8< ------ 8< ------ 8< ------ 8< ---

is fed to their "git mktag". If I am not mistaken, we definitely want to drop the EXTRA_HEADER_BODY_NEWLINE check from 08/10, even though EXTRA_HEADER_ENTRY may be something we want to retain at least for some releases for backward compatibility.

#204 lore raw

PATCH v2 10/10 mktag doc: update to explain why to use this

In reply to Ævar Arnfjörð Bjarmason

Change the mktag documentation to compare itself to the similar "hash-object -t tag" command. Before this someone reading the documentation wouldn't have much of an idea what the difference was.

Let's make it clear that it's to do with slightly different fsck validation logic, and cross-link the "mktag" and "hash-object" documentation to aid discover-ability.

---
 Documentation/git-hash-object.txt |  4 ++++
 Documentation/git-mktag.txt       | 29 +++++++++++++++++++++++++----
 2 files changed, 29 insertions(+), 4 deletions(-)
Documentation/git-hash-object.txt +4 -0
index df9e2c58bd..c535661ced 100644
@@ -58,6 +58,10 @@ OPTIONS
 	stress-testing Git itself or reproducing characteristics of corrupt or
 	bogus objects encountered in the wild.
 
+SEE ALSO
+--------
+linkgit:git-mktag[1]
+
 GIT
 ---
 Part of the linkgit:git[1] suite
Documentation/git-mktag.txt +25 -4
index a158428eb9..3ba15072df 100644
@@ -3,7 +3,7 @@ git-mktag(1)
 
 NAME
 ----
-git-mktag - Creates a tag object
+git-mktag - Creates a tag object with extra validation
 
 
 SYNOPSIS
@@ -13,10 +13,20 @@ SYNOPSIS
 
 DESCRIPTION
 -----------
-Reads a tag contents on standard input and creates a tag object
-that can also be used to sign other objects.
 
-The output is the new tag's <object> identifier.
+Reads a tag contents on standard input and creates a tag object. The
+output is the new tag's <object> identifier.
+
+This command accepts a subset of what linkgit:git-hash-object[1] would
+accept with `-t tag --stdin`. I.e. both of these work:
+
+    git mktag <my-tag
+    git hash-object -t tag --stdin <my-tag
+
+The difference between the two is that mktag does the equivalent of a
+linkgit:git-fsck(1) check on its input, and furthermore disallows some
+thing linkgit:git-hash-object[1] would pass, e.g. extra headers in the
+object before the message.
 
 Tag Format
 ----------
@@ -34,6 +44,17 @@ exists, is separated by a blank line from the header.  The
 message part may contain a signature that Git itself doesn't
 care about, but that can be verified with gpg.
 
+HISTORY
+-------
+
+In versions of Git before v2.30.0 the "mktag" command's validation
+logic was subtly different than that of linkgit:git-fsck[1]. It is now
+a strict superset of linkgit:git-fsck[1]'s validation logic.
+
+SEE ALSO
+--------
+linkgit:git-hash-object[1],
+
 GIT
 ---
 Part of the linkgit:git[1] suite
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#205 lore raw

Re: [PATCH v2 10/10] mktag doc: update to explain why to use this

Documentation/git-mktag.txt SYNOPSIS · line 24

Quoted 8 lines · Ævar Arnfjörð Bjarmason
Documentation/git-mktag.txt +8 -0
+Reads a tag contents on standard input and creates a tag object. The
+output is the new tag's <object> identifier.
+
+This command accepts a subset of what linkgit:git-hash-object[1] would
+accept with `-t tag --stdin`. I.e. both of these work:
+
+    git mktag <my-tag
+    git hash-object -t tag --stdin <my-tag

It's misleading to say "both of these work", I am afraid. If the former works, the latter would. That is what "accepts a subset" means, no?

Documentation/git-mktag.txt SYNOPSIS · line 29

Quoted 4 lines
Documentation/git-mktag.txt +4 -0
+The difference between the two is that mktag does the equivalent of a
+linkgit:git-fsck(1) check on its input, and furthermore disallows some
+thing linkgit:git-hash-object[1] would pass, e.g. extra headers in the
+object before the message.

Good description.

Documentation/git-mktag.txt exists, is separated by a blank line from the header. The · line 50

Quoted 8 lines
Documentation/git-mktag.txt +4 -0
@@ -34,6 +44,17 @@ exists, is separated by a blank line from the header.  The
 message part may contain a signature that Git itself doesn't
 care about, but that can be verified with gpg.
 
+HISTORY
+-------
+
+In versions of Git before v2.30.0 the "mktag" command's validation

Hardcoding v2.30.0 here feels problematic. If the series cooks in 'next' while 2.30 gets released, it has to be kicked out of 'next' to update this line before it gets allowed in 'next'.

Documentation/git-mktag.txt exists, is separated by a blank line from the header. The · line 52

Quoted 2 lines
Documentation/git-mktag.txt +2 -0
+logic was subtly different than that of linkgit:git-fsck[1]. It is now
+a strict superset of linkgit:git-fsck[1]'s validation logic.

It may be a better direction to go to drop this section.

Also, I somehow have a feeling that we'd rather want to loosen the "no other headers allowed" in the longer run to be consistent with what "fsck" does.

I also wonder if we want to teach "commit-tree" to also run fsck check like this new version of mktag does. It is outside the scope of this "mktag" series, of course.

Thanks.

Quoted 7 lines
Documentation/git-mktag.txt +4 -0
+SEE ALSO
+--------
+linkgit:git-hash-object[1],
+
 GIT
 ---
 Part of the linkgit:git[1] suite
#206 lore raw

PATCH 1/12 mktag: use default strbuf_read() hint

In reply to Jeff King

Change the hardcoded hint of 2^12 to 0. The default strbuf hint is perfectly fine here, and the only reason we were hardcoding it is because it survived migration from a pre-strbuf fixed-sized buffer.

See fd17f5b5f77 (Replace all read_fd use with strbuf_read, and get rid of it., 2007-09-10) for that migration.

---
 builtin/mktag.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
builtin/mktag.c +1 -1
index 4982d3a93e..ff7ac8e0e5 100644
@@ -161,7 +161,7 @@ int cmd_mktag(int argc, const char **argv, const char *prefix)
 	if (argc != 1)
 		usage("git mktag");
 
-	if (strbuf_read(&buf, 0, 4096) < 0) {
+	if (strbuf_read(&buf, 0, 0) < 0) {
 		die_errno("could not read from stdin");
 	}
 
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#207 lore raw

PATCH 2/12 mktag: reword write_object_file() error

In reply to Jeff King

Change the error message emitted when write_object_file() fails to make more sense. At this point we're not writing a "tag file" (which as an aside we never do, we just write to stdout). We are writing an annotated tag object, let's say that instead.

---
 builtin/mktag.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
builtin/mktag.c +1 -1
index ff7ac8e0e5..603b55aca0 100644
@@ -171,7 +171,7 @@ int cmd_mktag(int argc, const char **argv, const char *prefix)
 		die("invalid tag signature file");
 
 	if (write_object_file(buf.buf, buf.len, tag_type, &result) < 0)
-		die("unable to write tag file");
+		die("unable to write annotated tag object");
 
 	strbuf_release(&buf);
 	printf("%s\n", oid_to_hex(&result));
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#208 lore raw

PATCH 3/12 mktag: remove redundant braces in one-line body "if"

In reply to Jeff King

This minor stylistic churn is usually something we'd avoid, but if we don't do this then the file after changes in subsequent commits will only have this minor style inconsistency, so let's change this while we're at it.

---
 builtin/mktag.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
builtin/mktag.c +1 -2
index 603b55aca0..dc354828f7 100644
@@ -161,9 +161,8 @@ int cmd_mktag(int argc, const char **argv, const char *prefix)
 	if (argc != 1)
 		usage("git mktag");
 
-	if (strbuf_read(&buf, 0, 0) < 0) {
+	if (strbuf_read(&buf, 0, 0) < 0)
 		die_errno("could not read from stdin");
-	}
 
 	/* Verify it for some basic sanity: it needs to start with
 	   "object <sha1>\ntype\ntagger " */
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#209 lore raw

PATCH 4/12 mktag tests: don't needlessly use a subshell

In reply to Jeff King

The use of a subshell dates back to e9b20943b77 (t/t3800: do not use a temporary file to hold expected result., 2008-01-04). It's not needed anymore, if it ever was.

---
 t/t3800-mktag.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
t/t3800-mktag.sh +1 -1
index d696aa4e52..0e411e3c45 100755
@@ -14,7 +14,7 @@ test_description='git mktag: tag object verify test'
 check_verify_failure () {
 	expect="$2"
 	test_expect_success "$1" '
-		( test_must_fail git mktag <tag.sig 2>message ) &&
+		test_must_fail git mktag <tag.sig 2>message &&
 		grep "$expect" message
 	'
 }
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#210 lore raw

PATCH 5/12 mktag tests: remove needless SHA-1 hardcoding

In reply to Jeff King

Change the tests amended in acb49d1cc8b (t3800: make hash-size independent, 2019-08-18) even more to make them independent of either SHA-1 or SHA-256.

Some of these tests were failing for the wrong reasons. The first one being modified here would fail because the line starts with "xxxxxx" instead of "object", the rest of the line doesn't matter. Let's just put a valid hash on the rest of the line anyway to narrow the test down for just the s/object/xxxxxx/ case.

The second one being modified here would fail under GIT_TEST_DEFAULT_HASH=sha256 because <some sha-1 length garbage> is an invalid SHA-256, but we should really be testing <some sha-256 length garbage> when under SHA-256.

This doesn't really matter since we should be able to trust other parts of the code to validate things in the 0-9a-f range, but let's do it for good measure.

There's a later test which tests an invalid SHA which looks like a valid one, to stress the "We refuse to tag something we can't verify[...]" logic in mktag.c.

But here we're testing for a SHA-length string which contains characters outside of the /[0-9a-f]/i set. Let's just do that with a ROT13 invocation.

We could get really unlucky and switch to a future hash function that just happens to produce all [0-9] output for this particular input, but that's very unlikely.

---
 t/t3800-mktag.sh | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
t/t3800-mktag.sh +4 -3
index 0e411e3c45..b5013af2aa 100755
@@ -43,7 +43,7 @@ check_verify_failure 'Tag object length check' \
 #  2. object line label check
 
 cat >tag.sig <<EOF
-xxxxxx 139e9b33986b1c2670fff52c5067603117b3e895
+xxxxxx $head
 type tag
 tag mytag
 tagger . <> 0 +0000
@@ -53,10 +53,11 @@ EOF
 check_verify_failure '"object" line label check' '^error: char0: .*"object "$'
 
 ############################################################
-#  3. object line SHA1 check
+#  3. object line SHA check
 
+invalid_sha=$(echo $head | tr A-Za-z N-ZA-Mn-za-m)
 cat >tag.sig <<EOF
-object zz9e9b33986b1c2670fff52c5067603117b3e895
+object $invalid_sha
 type tag
 tag mytag
 tagger . <> 0 +0000
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#211 lore raw

Re: [RFC/PATCH 05/12] mktag tests: remove needless SHA-1 hardcoding

Quoted 7 lines · Ævar Arnfjörð Bjarmason

But here we're testing for a SHA-length string which contains characters outside of the /[0-9a-f]/i set. Let's just do that with a ROT13 invocation.

We could get really unlucky and switch to a future hash function that just happens to produce all [0-9] output for this particular input, but that's very unlikely.

Maybe s/[0-9a-f]/z/ would be both simpler and more robust?

-Peff

#212 lore raw

PATCH 6/12 mktag tests: improve verify_object() test coverage

In reply to Jeff King

The verify_object() function in "mktag.c" is tasked with ensuring that our tag refers to a valid object. The existing test for this might fail because it was also testing that "type taggg" didn't refer to a valid object type (it should be "type tag").

Let's split these tests up, so we're testing all combinations of a non-existing object and in invalid "type" line.

---
 t/t3800-mktag.sh | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)
t/t3800-mktag.sh +22 -0
index b5013af2aa..3801d3a285 100755
@@ -128,6 +128,28 @@ check_verify_failure '"type" line type-name length check' \
 ############################################################
 #  9. verify object (SHA1/type) check
 
+cat >tag.sig <<EOF
+object $(test_oid deadbeef)
+type tag
+tag mytag
+tagger . <> 0 +0000
+
+EOF
+
+check_verify_failure 'verify object (SHA1/type) check' \
+	'^error: char7: could not verify object.*$'
+
+cat >tag.sig <<EOF
+object $head
+type tagggg
+tag mytag
+tagger . <> 0 +0000
+
+EOF
+
+check_verify_failure 'verify object (SHA1/type) check' \
+	'^fatal: invalid object type'
+
 cat >tag.sig <<EOF
 object $(test_oid deadbeef)
 type tagggg
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#213 lore raw

PATCH 7/12 fsck: add new "extra" checks for "mktag"

In reply to Jeff King

Add optional "extra" checks to fsck, these are needed to eventually replace the custom not-quite-fsck code in mktag.c.

The mktag checks differ from fsck_tag() in several ways, one of those is that fsck doesn't know how to refuse an object with custom headers, and isn't strict about header and body newline separation.

Teach it how to optionally report these. I thought the best way to do that given the current structure of the code was to add a new "extra" category in addition to error/warn/info. Under --strict the "info" becomes a "warn" and "warn" becomes "error". Existing users of fsck's (and others, e.g. index-pack) --strict option rely on this.

By adding an "extra" category and only reporting it based on a flag in fsck_options callers can opt-in to these "extra" messages, which they'll then need to deal with in their own "error_func".

No tests are being added for this new functionality, they're added in a subsequent commit where we teach "mktag" to use this new validation mode.

I'm not changing fsck_commit() to validate commit objects like this either, we could do that, but unlike in the tag case that code wouldn't be used anywhere. If someone wants to write a "mkcommit" they which behaves like "mktag" they can borrow or refactor this logic for use in fsck_commit().

---
 fsck.c | 32 +++++++++++++++++++++++++++++++-
 fsck.h |  2 ++
 2 files changed, 33 insertions(+), 1 deletion(-)
fsck.c +31 -1
index f82e2fe9e3..3c25df2244 100644
@@ -80,7 +80,10 @@ static struct oidset gitmodules_done = OIDSET_INIT;
 	/* infos (reported as warnings, but ignored by default) */ \
 	FUNC(GITMODULES_PARSE, INFO) \
 	FUNC(BAD_TAG_NAME, INFO) \
-	FUNC(MISSING_TAGGER_ENTRY, INFO)
+	FUNC(MISSING_TAGGER_ENTRY, INFO) \
+	/* extra (only reported when requested) */ \
+	FUNC(EXTRA_HEADER_ENTRY, EXTRA) \
+	FUNC(EXTRA_HEADER_BODY_NEWLINE, EXTRA)
 
 #define MSG_ID(id, msg_type) FSCK_MSG_##id,
 enum fsck_msg_id {
@@ -975,6 +978,33 @@ static int fsck_tag(const struct object_id *oid, const char *buffer,
 	else
 		ret = fsck_ident(&buffer, oid, OBJ_TAG, options);
 
+	if (options->extra && *buffer) {
+		if (!starts_with(buffer, "\n")) {
+			/*
+			 * The verify_headers() check will allow
+			 * e.g. "[...]tagger <tagger>\nsome
+			 * garbage\n\nmessage" to pass, thinking "some
+			 * garbage" could be a custom
+			 * header. E.g. "mktag" doesn't want any
+			 * unknown headers.
+			 */
+			ret = report(options, oid, OBJ_TAG, FSCK_MSG_EXTRA_HEADER_ENTRY, "invalid format - extra header(s) after 'tagger'");
+			if (ret)
+				goto done;
+		}
+		if (starts_with(buffer, "\n\n")) {
+			/*
+			 * Some callers such as "mktag" want to
+			 * disallow "[...]tagger
+			 * <tagger>\n\n\nmessage", only allowing a
+			 * single newline for separation.
+			 */
+			ret = report(options, oid, OBJ_TAG, FSCK_MSG_EXTRA_HEADER_BODY_NEWLINE, "invalid format - headers separated body by more than one newline");
+			if (ret)
+				goto done;
+		}
+	}
+
 done:
 	strbuf_release(&sb);
 	return ret;
fsck.h +2 -0
index 69cf715e79..110efc65fd 100644
@@ -6,6 +6,7 @@
 #define FSCK_ERROR 1
 #define FSCK_WARN 2
 #define FSCK_IGNORE 3
+#define FSCK_EXTRA 4
 
 struct fsck_options;
 struct object;
@@ -40,6 +41,7 @@ struct fsck_options {
 	unsigned strict:1;
 	int *msg_type;
 	struct oidset skiplist;
+	unsigned extra:1;
 	kh_oid_map_t *object_names;
 };
 
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#214 lore raw

Re: [RFC/PATCH 07/12] fsck: add new "extra" checks for "mktag"

Quoted 10 lines · Ævar Arnfjörð Bjarmason

Add optional "extra" checks to fsck, these are needed to eventually replace the custom not-quite-fsck code in mktag.c.

The mktag checks differ from fsck_tag() in several ways, one of those is that fsck doesn't know how to refuse an object with custom headers, and isn't strict about header and body newline separation.

Teach it how to optionally report these. I thought the best way to do that given the current structure of the code was to add a new "extra" category in addition to error/warn/info.

Hmm, this new severity (and the extra options bit) feels a bit backwards. We are already passing the information on what we find to the report() callback. It seems like that is the place that should be deciding what is important and what is not.

Unfortunately the defaults are somewhat backwards here. We'd have to teach the fsck callbacks to ignore these harmless entries, rather than teaching the mktag caller that they need to be respected.

So probably the extra bit in options to say "do these extra tag checks" is the least-bad thing. But then why do we need to put them in their own EXTRA section? The only caller that wants them would treat them as errors.

I'm slightly on the fence on whether mktag really needs to enforce the "unknown header" thing at all. Sure, we don't encourage them, but it's a plumbing tool one could use to experiment with new headers. I guess the downside is that a typo'd header would not be caught.

Quoted 3 lines

Under --strict the "info"
becomes a "warn" and "warn" becomes "error". Existing users of fsck's (and others, e.g. index-pack) --strict option rely on this.

Yeah, this is a weirdness I think we should eventually fix (along with re-prioritizing some of the existing checks). I'm wary of doing anything that further cements that somewhat broken world-view (keep in mind that "index-pack --strict" is not "do fsck more strictly" but "do fsck at all").

Quoted 5 lines

I'm not changing fsck_commit() to validate commit objects like this either, we could do that, but unlike in the tag case that code wouldn't be used anywhere. If someone wants to write a "mkcommit" they which behaves like "mktag" they can borrow or refactor this logic for use in fsck_commit().

Seems reasonable.

-Peff

#215 lore raw

Re: [RFC/PATCH 07/12] fsck: add new "extra" checks for "mktag"

Quoted 18 lines · Jeff King
Quoted 10 lines · Ævar Arnfjörð Bjarmason

Add optional "extra" checks to fsck, these are needed to eventually replace the custom not-quite-fsck code in mktag.c.

The mktag checks differ from fsck_tag() in several ways, one of those is that fsck doesn't know how to refuse an object with custom headers, and isn't strict about header and body newline separation.

Teach it how to optionally report these. I thought the best way to do that given the current structure of the code was to add a new "extra" category in addition to error/warn/info.

Hmm, this new severity (and the extra options bit) feels a bit backwards. We are already passing the information on what we find to the report() callback. It seems like that is the place that should be deciding what is important and what is not.

Quoted 8 lines

Unfortunately the defaults are somewhat backwards here. We'd have to teach the fsck callbacks to ignore these harmless entries, rather than teaching the mktag caller that they need to be respected.

So probably the extra bit in options to say "do these extra tag checks" is the least-bad thing. But then why do we need to put them in their own EXTRA section? The only caller that wants them would treat them as errors.

Right, it'll be hidden behind options->extra, so I could just make them ERROR. I guess I was thinking it would be confusing to stick stuff in the middle of ERROR that wasn't on by default, e.g. I've sometimes skimmed that macro definition and saw "ah, bad parent sha1 is an error", as in transfer.fsckObjects would reject it.

So I'm slightly on the fence about keeping it as it is, what do you think?

Quoted 4 lines

I'm slightly on the fence on whether mktag really needs to enforce the "unknown header" thing at all. Sure, we don't encourage them, but it's a plumbing tool one could use to experiment with new headers. I guess the downside is that a typo'd header would not be caught.

The problem is that since verify_headers() in fsck.c wants to allow it, there's no way for it to distinguish a fat-fingerd "didn't separate the body from the headers" v.s. actually wanting a custom header in some cases.

Quoted 9 lines
Quoted 3 lines

Under --strict the "info"
becomes a "warn" and "warn" becomes "error". Existing users of fsck's (and others, e.g. index-pack) --strict option rely on this.

Yeah, this is a weirdness I think we should eventually fix (along with re-prioritizing some of the existing checks). I'm wary of doing anything that further cements that somewhat broken world-view (keep in mind that "index-pack --strict" is not "do fsck more strictly" but "do fsck at all").

*nod*, will note that.

Quoted 9 lines
Quoted 5 lines

I'm not changing fsck_commit() to validate commit objects like this either, we could do that, but unlike in the tag case that code wouldn't be used anywhere. If someone wants to write a "mkcommit" they which behaves like "mktag" they can borrow or refactor this logic for use in fsck_commit().

Seems reasonable.

-Peff

#216 lore raw

PATCH 8/12 mktag: use fsck instead of custom verify_tag()

In reply to Jeff King

TODO: This subtly breaks one check, see the last patch in this series.

Change the validation logic in "mktag" to use fsck's fsck_tag() instead of its own custom parser. Curiously the logic for both dates back to the same commit[1]. Let's unify them so we're not maintaining two sets functions to verify that a tag is OK.

Moving to fsck_tag() required teaching it to optionally use some validations that only the old mktag code could perform. That was done in an earlier commit, the "extraHeaderEntry" and "extraHeaderBodyNewline" tests being added here make use of that logic.

There was other "mktag" validation logic that I think makes sense to just remove. Namely:

A. fsck only cares that the timezone matches [-+][0-9]{4}. The mktag
code disallowed values larger than 1400.

Yes there's currently no timezone with a greater offset[2], but since we allow any number of non-offical timezones (e.g. +1234) passing this through seems fine. Git also won't break in the future if e.g. French Polynesia decides it needs to outdo the Line Islands when it comes to timezone extravagance.

B. fsck allows missing author names such as "tagger <email>", mktag
wouldn't, but would allow e.g. "tagger <email>" (but not "tagger <email>"). Now we allow all of these.

C. Like B, but "mktag" disallowed spaces in the <email> part, fsck
allows it.

We didn't only lose obscure validation logic, we also gained some:

D. fsck disallows zero-padded dates, but mktag didn't care. So
e.g. the timestamp "0000000000 +0000" produces an error now. A test in "t1006-cat-file.sh" relied on this, it's been changed to use "hash-object" (without fsck) instead.

1. ec4465adb38 (Add "tag" objects that can be used to sign other objects., 2005-04-25)

2. https://en.wikipedia.org/wiki/List_of_UTC_time_offsets

---
 builtin/mktag.c     | 45 +++++++++++++++++++++++++---
 t/t1006-cat-file.sh |  2 +-
 t/t3800-mktag.sh    | 72 +++++++++++++++++++++++++++------------------
 3 files changed, 86 insertions(+), 33 deletions(-)
builtin/mktag.c +41 -4
index dc354828f7..a1ae80702d 100644
@@ -2,6 +2,7 @@
 #include "tag.h"
 #include "replace-object.h"
 #include "object-store.h"
+#include "fsck.h"
 
 /*
  * A signature file has a very simple fixed format: four lines
@@ -47,6 +48,9 @@ static int verify_tag(char *buffer, unsigned long size)
 	const char *object, *type_line, *tag_line, *tagger_line, *lb, *rb, *p;
 	size_t len;
 
+	/* verify_tag() will be removed in the next commit */
+	return 0;
+
 	if (size < 84)
 		return error("wanna fool me ? you obviously got the size wrong !");
 
@@ -153,10 +157,34 @@ static int verify_tag(char *buffer, unsigned long size)
 	return 0;
 }
 
+static int mktag_fsck_error_func(struct fsck_options *o,
+				 const struct object_id *oid,
+				 enum object_type object_type,
+				 int msg_type, const char *message)
+{
+	switch (msg_type) {
+	case FSCK_WARN:
+	case FSCK_ERROR:
+	case FSCK_EXTRA:
+		/*
+		 * We treat both warnings and errors as errors, things
+		 * like missing "tagger" lines are "only" warnings
+		 * under fsck, we've always considered them an error.
+		 */
+		fprintf_ln(stderr, "error: %s", message);
+		return 1;
+	default:
+		BUG("%d (FSCK_IGNORE?) should never trigger this callback",
+		    msg_type);
+	}
+}
+
 int cmd_mktag(int argc, const char **argv, const char *prefix)
 {
+	struct object obj;
 	struct strbuf buf = STRBUF_INIT;
 	struct object_id result;
+	struct fsck_options fsck_options = FSCK_OPTIONS_STRICT;
 
 	if (argc != 1)
 		usage("git mktag");
@@ -164,10 +192,19 @@ int cmd_mktag(int argc, const char **argv, const char *prefix)
 	if (strbuf_read(&buf, 0, 0) < 0)
 		die_errno("could not read from stdin");
 
-	/* Verify it for some basic sanity: it needs to start with
-	   "object <sha1>\ntype\ntagger " */
-	if (verify_tag(buf.buf, buf.len) < 0)
-		die("invalid tag signature file");
+	/* verify_tag() will be removed in the next commit */
+	verify_tag("", 0);
+
+	/*
+	 * Fake up an object for fsck_object()
+	 */
+	obj.parsed = 1;
+	obj.type = OBJ_TAG;
+
+	fsck_options.extra = 1;
+	fsck_options.error_func = mktag_fsck_error_func;
+	if (fsck_object(&obj, buf.buf, buf.len, &fsck_options))
+		die("tag on stdin did not pass our strict fsck check");
 
 	if (write_object_file(buf.buf, buf.len, tag_type, &result) < 0)
 		die("unable to write annotated tag object");
t/t1006-cat-file.sh +1 -1
index 2f501d2dc9..5d2dc99b74 100755
@@ -166,7 +166,7 @@ tag_content="$tag_header_without_timestamp 0000000000 +0000
 
 $tag_description"
 
-tag_sha1=$(echo_without_newline "$tag_content" | git mktag)
+tag_sha1=$(echo_without_newline "$tag_content" | git hash-object -t tag --stdin -w)
 tag_size=$(strlen "$tag_content")
 
 run_tests 'tag' $tag_sha1 $tag_size "$tag_content" "$tag_content" 1
t/t3800-mktag.sh +44 -28

214 lines not on this page · raw message

Signature · 1 line
2.29.2.222.g5d2a92d10f8
#217 lore raw

Re: [RFC/PATCH 08/12] mktag: use fsck instead of custom verify_tag()

Quoted 11 lines · Ævar Arnfjörð Bjarmason

There was other "mktag" validation logic that I think makes sense to just remove. Namely:

A. fsck only cares that the timezone matches [-+][0-9]{4}. The mktag
code disallowed values larger than 1400.

Yes there's currently no timezone with a greater offset[2], but since we allow any number of non-offical timezones (e.g. +1234) passing this through seems fine. Git also won't break in the future if e.g. French Polynesia decides it needs to outdo the Line Islands when it comes to timezone extravagance.

Yeah, I think this is a good choice to loosen.

Quoted 3 lines

B. fsck allows missing author names such as "tagger <email>", mktag
wouldn't, but would allow e.g. "tagger <email>" (but not "tagger <email>"). Now we allow all of these.

Likewise, though I am confused. Should the second "tagger <email>" in that paragraph have something else in it?

Quoted 2 lines

C. Like B, but "mktag" disallowed spaces in the <email> part, fsck
allows it.

Possibly something we'd want to tighten in fsck, but I think keeping them in alignment is a good idea for now.

Quoted 6 lines

We didn't only lose obscure validation logic, we also gained some:

D. fsck disallows zero-padded dates, but mktag didn't care. So
e.g. the timestamp "0000000000 +0000" produces an error now. A test in "t1006-cat-file.sh" relied on this, it's been changed to use "hash-object" (without fsck) instead.

Seems reasonable.

builtin/mktag.c int cmd_mktag(int argc, const char **argv, const char *prefix) · line 202

Quoted 8 lines
builtin/mktag.c +8 -0
+	/* verify_tag() will be removed in the next commit */
+	verify_tag("", 0);
+
+	/*
+	 * Fake up an object for fsck_object()
+	 */
+	obj.parsed = 1;
+	obj.type = OBJ_TAG;

I don't love this "fake object struct on the stack" thing. I can't think of anything that would break outright, but it may be the only place where that struct isn't coming from the usual pool, and representing the common part of a larger object. Two definite gotchas, though:

- if the type is OBJ_TAG, then it may get cast to a "struct tag" by other code, which could look past the end of the struct. I think that fsck_object() doesn't do this, but it could (and it definitely used to)

- you don't initialize the other fields. We'll definitely pass &obj->oid in the fsck code, and even back to our report() callback, even though it's full of garbage. In practice this is OK because our custom report() won't look at them, but it seems awfully fragile.

I recently genericized the type-specific fsck_* functions so that they just need an oid, and not an object struct. I think I didn't do fsck_object() just because it didn't have any callers where it mattered. So I think it would make sense here to either:

- make fsck_tag() specifically available outside of fsck.c, so could call it directly

- convert fsck_object() to take an oid rather than an object struct. It only uses the object itself to check for NULL-ness. Looking at the callers, they all have non-NULL objects already. So I think that check can't be triggered.

t/t3800-mktag.sh too short for a tag · line 40

Quoted 3 lines

check_verify_failure 'Tag object length check' \

t/t3800-mktag.sh +1 -1
-	'^error: .*size wrong.*$'
+	'^error: missingObject:'

We may want to enhance the "error:" here to make it clear we're talking about a format error in the tag input. Maybe:

error: tag input does not pass fsck: missingObject: ...

or something.

-Peff

#218 lore raw

Re: [RFC/PATCH 08/12] mktag: use fsck instead of custom verify_tag()

Quoted 14 lines · Jeff King
Quoted 11 lines · Ævar Arnfjörð Bjarmason

There was other "mktag" validation logic that I think makes sense to just remove. Namely:

A. fsck only cares that the timezone matches [-+][0-9]{4}. The mktag
code disallowed values larger than 1400.

Yes there's currently no timezone with a greater offset[2], but since we allow any number of non-offical timezones (e.g. +1234) passing this through seems fine. Git also won't break in the future if e.g. French Polynesia decides it needs to outdo the Line Islands when it comes to timezone extravagance.

Quoted 8 lines

Yeah, I think this is a good choice to loosen.

Quoted 3 lines

B. fsck allows missing author names such as "tagger <email>", mktag
wouldn't, but would allow e.g. "tagger <email>" (but not "tagger <email>"). Now we allow all of these.

Likewise, though I am confused. Should the second "tagger <email>" in that paragraph have something else in it?

There should be 1-2 and 3 spaces there, don't know why that didn't make it. Will fix.

Quoted 47 lines
Quoted 2 lines

C. Like B, but "mktag" disallowed spaces in the <email> part, fsck
allows it.

Possibly something we'd want to tighten in fsck, but I think keeping them in alignment is a good idea for now.

Quoted 6 lines

We didn't only lose obscure validation logic, we also gained some:

D. fsck disallows zero-padded dates, but mktag didn't care. So
e.g. the timestamp "0000000000 +0000" produces an error now. A test in "t1006-cat-file.sh" relied on this, it's been changed to use "hash-object" (without fsck) instead.

Seems reasonable.

builtin/mktag.c int cmd_mktag(int argc, const char **argv, const char *prefix) · line 202

Quoted 8 lines
builtin/mktag.c +8 -0
+	/* verify_tag() will be removed in the next commit */
+	verify_tag("", 0);
+
+	/*
+	 * Fake up an object for fsck_object()
+	 */
+	obj.parsed = 1;
+	obj.type = OBJ_TAG;

I don't love this "fake object struct on the stack" thing. I can't think of anything that would break outright, but it may be the only place where that struct isn't coming from the usual pool, and representing the common part of a larger object. Two definite gotchas, though:

- if the type is OBJ_TAG, then it may get cast to a "struct tag" by other code, which could look past the end of the struct. I think that fsck_object() doesn't do this, but it could (and it definitely used to)

- you don't initialize the other fields. We'll definitely pass &obj->oid in the fsck code, and even back to our report() callback, even though it's full of garbage. In practice this is OK because our custom report() won't look at them, but it seems awfully fragile.

I recently genericized the type-specific fsck_* functions so that they just need an oid, and not an object struct. I think I didn't do fsck_object() just because it didn't have any callers where it mattered. So I think it would make sense here to either:

- make fsck_tag() specifically available outside of fsck.c, so could call it directly

Quoted 8 lines

- convert fsck_object() to take an oid rather than an object struct. It only uses the object itself to check for NULL-ness. Looking at the callers, they all have non-NULL objects already. So I think that check can't be triggered.

t/t3800-mktag.sh too short for a tag · line 40

Quoted 3 lines

check_verify_failure 'Tag object length check' \

t/t3800-mktag.sh +1 -1
-	'^error: .*size wrong.*$'
+	'^error: missingObject:'

When I started out hacking on this I just made fsck_tag() a non-static function. The faking up obj only got added because I needed to call fsck_object().

Between this and your comment to 12/12 I wonder if the least bad thing isn't just to split up fsck_tag() into fsck_tag() and fsck_tag_standalone(), so I can call the latter with a "populate the object and type" argument, and fsck.c will just NULL those two parameters since it doesn't care.

Gets rid of the obj faking, and gets rid of the custom parser in mktag.c. It'll only need to do the object lookup at that point, and even then since I've added everything else to fsck.c I might as well make that another custom ERROR/EXTRA, i.e. have it learn to do the lookup itself.

Quoted 6 lines

We may want to enhance the "error:" here to make it clear we're talking about a format error in the tag input. Maybe:

error: tag input does not pass fsck: missingObject: ...

or something.

*nod*

#219 lore raw

PATCH 9/12 mktag: remove now-unused verify_tag() code

In reply to Jeff King

The preceding commit removed the use of the verify_tag() function, but didn't remove this code to keep the diff smaller. Let's remove this unused code now.

---
 builtin/mktag.c | 157 ------------------------------------------------
 1 file changed, 157 deletions(-)
builtin/mktag.c +0 -157
index a1ae80702d..e9a0954dcb 100644
@@ -1,162 +1,8 @@
 #include "builtin.h"
 #include "tag.h"
-#include "replace-object.h"
 #include "object-store.h"
 #include "fsck.h"
 
-/*
- * A signature file has a very simple fixed format: four lines
- * of "object <sha1>" + "type <typename>" + "tag <tagname>" +
- * "tagger <committer>", followed by a blank line, a free-form tag
- * message and a signature block that git itself doesn't care about,
- * but that can be verified with gpg or similar.
- *
- * The first four lines are guaranteed to be at least 83 bytes:
- * "object <sha1>\n" is 48 bytes, "type tag\n" at 9 bytes is the
- * shortest possible type-line, "tag .\n" at 6 bytes is the shortest
- * single-character-tag line, and "tagger . <> 0 +0000\n" at 20 bytes is
- * the shortest possible tagger-line.
- */
-
-/*
- * We refuse to tag something we can't verify. Just because.
- */
-static int verify_object(const struct object_id *oid, const char *expected_type)
-{
-	int ret = -1;
-	enum object_type type;
-	unsigned long size;
-	void *buffer = read_object_file(oid, &type, &size);
-	const struct object_id *repl = lookup_replace_object(the_repository, oid);
-
-	if (buffer) {
-		if (type == type_from_string(expected_type)) {
-			ret = check_object_signature(the_repository, repl,
-						     buffer, size,
-						     expected_type);
-		}
-		free(buffer);
-	}
-	return ret;
-}
-
-static int verify_tag(char *buffer, unsigned long size)
-{
-	int typelen;
-	char type[20];
-	struct object_id oid;
-	const char *object, *type_line, *tag_line, *tagger_line, *lb, *rb, *p;
-	size_t len;
-
-	/* verify_tag() will be removed in the next commit */
-	return 0;
-
-	if (size < 84)
-		return error("wanna fool me ? you obviously got the size wrong !");
-
-	buffer[size] = 0;
-
-	/* Verify object line */
-	object = buffer;
-	if (memcmp(object, "object ", 7))
-		return error("char%d: does not start with \"object \"", 0);
-
-	if (parse_oid_hex(object + 7, &oid, &p))
-		return error("char%d: could not get SHA1 hash", 7);
-
-	/* Verify type line */
-	type_line = p + 1;
-	if (memcmp(type_line - 1, "\ntype ", 6))
-		return error("char%d: could not find \"\\ntype \"", 47);
-
-	/* Verify tag-line */
-	tag_line = strchr(type_line, '\n');
-	if (!tag_line)
-		return error("char%"PRIuMAX": could not find next \"\\n\"",
-				(uintmax_t) (type_line - buffer));
-	tag_line++;
-	if (memcmp(tag_line, "tag ", 4) || tag_line[4] == '\n')
-		return error("char%"PRIuMAX": no \"tag \" found",
-				(uintmax_t) (tag_line - buffer));
-
-	/* Get the actual type */
-	typelen = tag_line - type_line - strlen("type \n");
-	if (typelen >= sizeof(type))
-		return error("char%"PRIuMAX": type too long",
-				(uintmax_t) (type_line+5 - buffer));
-
-	memcpy(type, type_line+5, typelen);
-	type[typelen] = 0;
-
-	/* Verify that the object matches */
-	if (verify_object(&oid, type))
-		return error("char%d: could not verify object %s", 7, oid_to_hex(&oid));
-
-	/* Verify the tag-name: we don't allow control characters or spaces in it */
-	tag_line += 4;
-	for (;;) {
-		unsigned char c = *tag_line++;
-		if (c == '\n')
-			break;
-		if (c > ' ')
-			continue;
-		return error("char%"PRIuMAX": could not verify tag name",
-				(uintmax_t) (tag_line - buffer));
-	}
-
-	/* Verify the tagger line */
-	tagger_line = tag_line;
-
-	if (memcmp(tagger_line, "tagger ", 7))
-		return error("char%"PRIuMAX": could not find \"tagger \"",
-			(uintmax_t) (tagger_line - buffer));
-
-	/*
-	 * Check for correct form for name and email
-	 * i.e. " <" followed by "> " on _this_ line
-	 * No angle brackets within the name or email address fields.
-	 * No spaces within the email address field.
-	 */
-	tagger_line += 7;
-	if (!(lb = strstr(tagger_line, " <")) || !(rb = strstr(lb+2, "> ")) ||
-		strpbrk(tagger_line, "<>\n") != lb+1 ||
-		strpbrk(lb+2, "><\n ") != rb)
-		return error("char%"PRIuMAX": malformed tagger field",
-			(uintmax_t) (tagger_line - buffer));
-
-	/* Check for author name, at least one character, space is acceptable */
-	if (lb == tagger_line)
-		return error("char%"PRIuMAX": missing tagger name",
-			(uintmax_t) (tagger_line - buffer));
-
-	/* timestamp, 1 or more digits followed by space */
-	tagger_line = rb + 2;
-	if (!(len = strspn(tagger_line, "0123456789")))
-		return error("char%"PRIuMAX": missing tag timestamp",
-			(uintmax_t) (tagger_line - buffer));
-	tagger_line += len;
-	if (*tagger_line != ' ')
-		return error("char%"PRIuMAX": malformed tag timestamp",
-			(uintmax_t) (tagger_line - buffer));
-	tagger_line++;
-
-	/* timezone, 5 digits [+-]hhmm, max. 1400 */
-	if (!((tagger_line[0] == '+' || tagger_line[0] == '-') &&
-	      strspn(tagger_line+1, "0123456789") == 4 &&
-	      tagger_line[5] == '\n' && atoi(tagger_line+1) <= 1400))
-		return error("char%"PRIuMAX": malformed tag timezone",
-			(uintmax_t) (tagger_line - buffer));
-	tagger_line += 6;
-
-	/* Verify the blank line separating the header from the body */
-	if (*tagger_line != '\n')
-		return error("char%"PRIuMAX": trailing garbage in tag header",
-			(uintmax_t) (tagger_line - buffer));
-
-	/* The actual stuff afterwards we don't care about.. */
-	return 0;
-}
-
 static int mktag_fsck_error_func(struct fsck_options *o,
 				 const struct object_id *oid,
 				 enum object_type object_type,
@@ -192,9 +38,6 @@ int cmd_mktag(int argc, const char **argv, const char *prefix)
 	if (strbuf_read(&buf, 0, 0) < 0)
 		die_errno("could not read from stdin");
 
-	/* verify_tag() will be removed in the next commit */
-	verify_tag("", 0);
-
 	/*
 	 * Fake up an object for fsck_object()
 	 */
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#220 lore raw

Re: [RFC/PATCH 09/12] mktag: remove now-unused verify_tag() code

Quoted 3 lines · Ævar Arnfjörð Bjarmason

The preceding commit removed the use of the verify_tag() function, but didn't remove this code to keep the diff smaller. Let's remove this unused code now.

I do appreciate keeping diffs readable, but IMHO this is making things more confusing, because you had to add a new dummy call to verify_tag() in the previous commit in order to keep the compiler from complaining about the unused static function (I'm actually surprised that it doesn't complain about the immediate "return 0" in that function, which makes the rest of the function unambiguously unreachable).

-Peff

#221 lore raw

PATCH 10/12 mktag doc: update to explain why to use this

In reply to Jeff King

Change the mktag documentation to compare itself to the similar "hash-object -t tag" command. Before this someone reading the documentation wouldn't have much of an idea what the difference was.

Let's make it clear that it's to do with slightly different fsck validation logic, and cross-link the "mktag" and "hash-object" documentation to aid discover-ability.

---
 Documentation/git-hash-object.txt |  4 ++++
 Documentation/git-mktag.txt       | 29 +++++++++++++++++++++++++----
 2 files changed, 29 insertions(+), 4 deletions(-)
Documentation/git-hash-object.txt +4 -0
index df9e2c58bd..c535661ced 100644
@@ -58,6 +58,10 @@ OPTIONS
 	stress-testing Git itself or reproducing characteristics of corrupt or
 	bogus objects encountered in the wild.
 
+SEE ALSO
+--------
+linkgit:git-mktag[1]
+
 GIT
 ---
 Part of the linkgit:git[1] suite
Documentation/git-mktag.txt +25 -4
index fa6a756123..11ce8e4bb4 100644
@@ -3,7 +3,7 @@ git-mktag(1)
 
 NAME
 ----
-git-mktag - Creates a tag object
+git-mktag - Creates a tag object with extra validation
 
 
 SYNOPSIS
@@ -13,10 +13,20 @@ SYNOPSIS
 
 DESCRIPTION
 -----------
-Reads a tag contents on standard input and creates a tag object
-that can also be used to sign other objects.
 
-The output is the new tag's <object> identifier.
+Reads a tag contents on standard input and creates a tag object. The
+output is the new tag's <object> identifier.
+
+This command accepts a subset of what linkgit:git-hash-object[1] would
+accept with `-t tag --stdin`. I.e. both of these work:
+
+    git mktag <my-tag
+    git hash-object -t tag --stdin <my-tag
+
+The difference between the two is that mktag does the equivalent of a
+linkgit:git-fsck(1) check on its input, and furthermore disallows some
+thing linkgit:git-hash-object[1] would pass, e.g. extra headers in the
+object before the message.
 
 Tag Format
 ----------
@@ -34,6 +44,17 @@ exists, is separated by a blank line from the header.  The
 message part may contain a signature that Git itself doesn't
 care about, but that can be verified with gpg.
 
+HISTORY
+-------
+
+In versions of Git before v2.30.0 the "mktag" command's validation
+logic was subtly different than that of linkgit:git-fsck[1]. It is now
+a strict superset of linkgit:git-fsck[1]'s validation logic.
+
+SEE ALSO
+--------
+linkgit:git-hash-object[1],
+
 GIT
 ---
 Part of the linkgit:git[1] suite
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#222 lore raw

PATCH 11/12 mktag docs: say <hash> not <sha1>

In reply to Jeff King

Change the "mktag" documentation to refer to the input hash as just "hash", not "sha1". This command has supported SHA-256 for a while now.

---
 Documentation/git-mktag.txt | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Documentation/git-mktag.txt +1 -1
index 11ce8e4bb4..3ba15072df 100644
@@ -33,7 +33,7 @@ Tag Format
 A tag signature file, to be fed to this command's standard input,
 has a very simple fixed format: four lines of
 
-  object <sha1>
+  object <hash>
   type <typename>
   tag <tagname>
   tagger <tagger>
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#223 lore raw

PATCH 12/12 mktag: bring back some of the verify_object() logic

In reply to Jeff King

When working on this series I saw too late that I'd removed the mktag check for validating the object the tag points to. The fsck_tag() code doesn't do this because it's meant for the context of fsck, where we're validating reachability anyway.

We'd need to either refactor fsck_tag() so that it can pass us back its "tagged_oid" and the "type_from_string_gently()" value it throws away to get rid of the re-parsing of stdin here, or just duplicate the logic as I'm doing here.

---
 builtin/mktag.c  | 48 ++++++++++++++++++++++++++++++++++++++++++++++++
 t/t3800-mktag.sh |  4 ++--
 2 files changed, 50 insertions(+), 2 deletions(-)
builtin/mktag.c +48 -0
index e9a0954dcb..f1f1cf04ff 100644
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "tag.h"
+#include "replace-object.h"
 #include "object-store.h"
 #include "fsck.h"
 
@@ -25,6 +26,50 @@ static int mktag_fsck_error_func(struct fsck_options *o,
 	}
 }
 
+static int verify_object_in_tag(const char *stdin)
+{
+	struct object_id oid;
+	char *eol;
+	const char *p;
+	int expected_type_id;
+	const char *expected_type;
+	int ret = -1;
+	enum object_type type;
+	unsigned long size;
+	void *buffer;
+	const struct object_id *repl;
+
+	if (!skip_prefix(stdin, "object ", &stdin))
+		goto bug;
+	if (parse_oid_hex(stdin, &oid, &p) || *p != '\n')
+		goto bug;
+	stdin = p + 1;
+	if (!skip_prefix(stdin, "type ", &stdin))
+		goto bug;
+	eol = strchr(stdin, '\n');
+	expected_type_id = type_from_string_gently(stdin, eol - stdin, 1);
+	if (expected_type_id < 0)
+		goto bug;
+	expected_type = type_name(expected_type_id);
+
+	buffer = read_object_file(&oid, &type, &size);
+	repl = lookup_replace_object(the_repository, &oid);
+
+	if (buffer) {
+		if (type == type_from_string(expected_type)) {
+			ret = check_object_signature(the_repository, repl,
+						     buffer, size,
+						     expected_type);
+		}
+		free(buffer);
+	}
+	goto ok;
+bug:
+	BUG("fsck_object() should have ensured a sane tag format already!");
+ok:
+	return ret;
+}
+
 int cmd_mktag(int argc, const char **argv, const char *prefix)
 {
 	struct object obj;
@@ -49,6 +94,9 @@ int cmd_mktag(int argc, const char **argv, const char *prefix)
 	if (fsck_object(&obj, buf.buf, buf.len, &fsck_options))
 		die("tag on stdin did not pass our strict fsck check");
 
+	if (verify_object_in_tag(buf.buf))
+		die("tag on stdin did not refer to a valid object");
+
 	if (write_object_file(buf.buf, buf.len, tag_type, &result) < 0)
 		die("unable to write annotated tag object");
 
t/t3800-mktag.sh +2 -2
index bc57ee85c9..74cd2eb141 100755
@@ -137,7 +137,7 @@ tagger . <> 0 +0000
 EOF
 
 check_verify_failure 'verify object (SHA1/type) check' \
-	'^error: char7: could not verify object.*$'
+	'^fatal: tag on stdin did not refer to a valid object'
 
 cat >tag.sig <<EOF
 object $head
@@ -148,7 +148,7 @@ tagger . <> 0 +0000
 EOF
 
 check_verify_failure 'verify object (SHA1/type) check' \
-	'^fatal: invalid object type'
+	'^error: badType:'
 
 cat >tag.sig <<EOF
 object $(test_oid deadbeef)
Signature · 1 line
2.29.2.222.g5d2a92d10f8
#224 lore raw

Re: [RFC/PATCH 12/12] mktag: bring back some of the verify_object() logic

Quoted 9 lines · Ævar Arnfjörð Bjarmason

When working on this series I saw too late that I'd removed the mktag check for validating the object the tag points to. The fsck_tag() code doesn't do this because it's meant for the context of fsck, where we're validating reachability anyway.

We'd need to either refactor fsck_tag() so that it can pass us back its "tagged_oid" and the "type_from_string_gently()" value it throws away to get rid of the re-parsing of stdin here, or just duplicate the logic as I'm doing here.

We have yet another tag parser (because of course there's more): the one in parse_tag_buffer() that we use for reading tags. I think your new function here could just be:

enum object_type real_type;
struct tag t; /* yuck! another fake object */

memset(&t, 0, sizeof(t));
if (parse_tag_buffer(r, &t, buf, len) < 0)

	die("unable to parse");

real_type = oid_object_info(r, &t->tagged->oid, NULL); if (real_type < 0)

	die("tagged object does not exist");
  if (real_type != t->tagged->type)
	die("tagged object's type does not match tag type field");

I almost wonder if we could simply rely on parse_tag_buffer() instead of fsck_tag(), but it is not nearly as picky about finding potential problems (its goal is the opposite: to return something usable if it's at all possible to do so).

The "yuck" above isn't great. We could use pretend_object_file() for that, though it's a bit heavy-weight (it computes the actual oid!). And also it's a weird one-off that we've talked about getting rid of.

-Peff

#225 lore raw

PATCH v2 11/14 t3903: abstract away SHA-1-specific constants

In reply to brian m. carlson

Abstract away the SHA-1-specific constants by sanitizing diff output to remove the index lines, since it's clear from the assertions in question that we are not interested in the specific object IDs.

---
 t/t3903-stash.sh | 32 ++++++++++++++++++++++----------
 1 file changed, 22 insertions(+), 10 deletions(-)
t/t3903-stash.sh +22 -10
index b8e337893f..820b350aeb 100755
@@ -7,6 +7,18 @@ test_description='Test git stash'
 
 . ./test-lib.sh
 
+diff_cmp () {
+	for i in "$1" "$2"
+	do
+		sed -e 's/^index 0000000\.\.[0-9a-f]*/index 0000000..1234567/' \
+		-e 's/^index [0-9a-f]*\.\.[0-9a-f]*/index 1234567..89abcde/' \
+		-e 's/^index [0-9a-f]*,[0-9a-f]*\.\.[0-9a-f]*/index 1234567,7654321..89abcde/' \
+		"$i" >"$i.compare" || return 1
+	done &&
+	test_cmp "$1.compare" "$2.compare" &&
+	rm -f "$1.compare" "$2.compare"
+}
+
 test_expect_success 'stash some dirty working directory' '
 	echo 1 >file &&
 	git add file &&
@@ -36,7 +48,7 @@ EOF
 test_expect_success 'parents of stash' '
 	test $(git rev-parse stash^) = $(git rev-parse HEAD) &&
 	git diff stash^2..stash >output &&
-	test_cmp expect output
+	diff_cmp expect output
 '
 
 test_expect_success 'applying bogus stash does nothing' '
@@ -210,13 +222,13 @@ test_expect_success 'stash branch' '
 	test refs/heads/stashbranch = $(git symbolic-ref HEAD) &&
 	test $(git rev-parse HEAD) = $(git rev-parse master^) &&
 	git diff --cached >output &&
-	test_cmp expect output &&
+	diff_cmp expect output &&
 	git diff >output &&
-	test_cmp expect1 output &&
+	diff_cmp expect1 output &&
 	git add file &&
 	git commit -m alternate\ second &&
 	git diff master..stashbranch >output &&
-	test_cmp output expect2 &&
+	diff_cmp output expect2 &&
 	test 0 = $(git stash list | wc -l)
 '
 
@@ -577,7 +589,7 @@ test_expect_success 'stash show -p - stashes on stack, stash-like argument' '
 	+bar
 	EOF
 	git stash show -p ${STASH_ID} >actual &&
-	test_cmp expected actual
+	diff_cmp expected actual
 '
 
 test_expect_success 'stash show - no stashes on stack, stash-like argument' '
@@ -609,7 +621,7 @@ test_expect_success 'stash show -p - no stashes on stack, stash-like argument' '
 	+foo
 	EOF
 	git stash show -p ${STASH_ID} >actual &&
-	test_cmp expected actual
+	diff_cmp expected actual
 '
 
 test_expect_success 'stash show --patience shows diff' '
@@ -627,7 +639,7 @@ test_expect_success 'stash show --patience shows diff' '
 	+foo
 	EOF
 	git stash show --patience ${STASH_ID} >actual &&
-	test_cmp expected actual
+	diff_cmp expected actual
 '
 
 test_expect_success 'drop: fail early if specified stash is not a stash ref' '
@@ -791,7 +803,7 @@ test_expect_success 'stash where working directory contains "HEAD" file' '
 	git diff-index --cached --quiet HEAD &&
 	test "$(git rev-parse stash^)" = "$(git rev-parse HEAD)" &&
 	git diff stash^..stash >output &&
-	test_cmp expect output
+	diff_cmp expect output
 '
 
 test_expect_success 'store called with invalid commit' '
@@ -847,7 +859,7 @@ test_expect_success 'stash list implies --first-parent -m' '
 	+working
 	EOF
 	git stash list --format=%gd -p >actual &&
-	test_cmp expect actual
+	diff_cmp expect actual
 '
 
 test_expect_success 'stash list --cc shows combined diff' '
@@ -864,7 +876,7 @@ test_expect_success 'stash list --cc shows combined diff' '
 	++working
 	EOF
 	git stash list --format=%gd -p --cc >actual &&
-	test_cmp expect actual
+	diff_cmp expect actual
 '
 
 test_expect_success 'stash is not confused by partial renames' '
#226 lore raw

PATCH v2 12/14 t4000: make hash size independent

In reply to brian m. carlson

Use $ZERO_OID instead of hard-coding a fixed size all-zeros object ID.

---
 t/t4000-diff-format.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
t/t4000-diff-format.sh +1 -1
index 8de36b7d12..e5116a76a1 100755
@@ -78,7 +78,7 @@ test_expect_success 'git diff-files --no-patch --patch shows the patch' '
 
 test_expect_success 'git diff-files --no-patch --patch-with-raw shows the patch and raw data' '
 	git diff-files --no-patch --patch-with-raw >actual &&
-	grep -q "^:100644 100755 .* 0000000000000000000000000000000000000000 M	path0\$" actual &&
+	grep -q "^:100644 100755 .* $ZERO_OID M	path0\$" actual &&
 	tail -n +4 actual >actual-patch &&
 	compare_diff_patch expected actual-patch
 '
#227 lore raw

PATCH v2 13/14 t4002: make hash independent

In reply to brian m. carlson

Factor out the hard-coded object IDs and use test_oid to provide values for both SHA-1 and SHA-256.

---
 t/t4002-diff-basic.sh | 367 +++++++++++++++++++++++++++++-------------
 1 file changed, 258 insertions(+), 109 deletions(-)
t/t4002-diff-basic.sh +258 -109
index 3a6c21e825..cbcdd10464 100755
@@ -7,123 +7,272 @@ test_description='Test diff raw-output.
 
 '
 . ./test-lib.sh
+
 . "$TEST_DIRECTORY"/lib-read-tree-m-3way.sh
 
-cat >.test-plain-OA <<\EOF
-:000000 100644 0000000000000000000000000000000000000000 ccba72ad3888a3520b39efcf780b9ee64167535d A	AA
-:000000 100644 0000000000000000000000000000000000000000 7e426fb079479fd67f6d81f984e4ec649a44bc25 A	AN
-:100644 000000 bcc68ef997017466d5c9094bcf7692295f588c9a 0000000000000000000000000000000000000000 D	DD
-:000000 040000 0000000000000000000000000000000000000000 6d50f65d3bdab91c63444294d38f08aeff328e42 A	DF
-:100644 000000 141c1f1642328e4bc46a7d801a71da392e66791e 0000000000000000000000000000000000000000 D	DM
-:100644 000000 35abde1506ddf806572ff4d407bd06885d0f8ee9 0000000000000000000000000000000000000000 D	DN
-:000000 100644 0000000000000000000000000000000000000000 1d41122ebdd7a640f29d3c9cc4f9d70094374762 A	LL
-:100644 100644 03f24c8c4700babccfd28b654e7e8eac402ad6cd 103d9f89b50b9aad03054b579be5e7aa665f2d57 M	MD
-:100644 100644 b258508afb7ceb449981bd9d63d2d3e971bf8d34 b431b272d829ff3aa4d1a5085f4394ab4d3305b6 M	MM
-:100644 100644 bd084b0c27c7b6cc34f11d6d0509a29be3caf970 a716d58de4a570e0038f5c307bd8db34daea021f M	MN
-:100644 100644 40c959f984c8b89a2b02520d17f00d717f024397 2ac547ae9614a00d1b28275de608131f7a0e259f M	SS
-:100644 100644 4ac13458899ab908ef3b1128fa378daefc88d356 4c86f9a85fbc5e6804ee2e17a797538fbe785bca M	TT
-:040000 040000 7d670fdcdb9929f6c7dac196ff78689cd1c566a1 5e5f22072bb39f6e12cf663a57cb634c76eefb49 M	Z
+test_oid_init
+
+test_oid_cache <<\EOF
+aa_1 sha1:ccba72ad3888a3520b39efcf780b9ee64167535d
+aa_1 sha256:9febfbf18197819b2735c45291f138525d2476d59470f98239647544586ba403
+
+aa_2 sha1:6aa2b5335b16431a0ef71e5c0a28be69183cf6a2
+aa_2 sha256:6eaa3437de83f145a4aaa6ba355303075ade547b128ec6a2cd00a81ff7ce7a56
+
+an_1 sha1:7e426fb079479fd67f6d81f984e4ec649a44bc25
+an_1 sha256:8f92a0bec99e399a38e3bd0e1bf19fbf121e0160efb29b857df79d439f1c4536
+
+dd_1 sha1:bcc68ef997017466d5c9094bcf7692295f588c9a
+dd_1 sha256:07e17428b00639b85485d2b01083d219e2f3e3ba8579e9ca44e9cc8dd554d952
+
+df_1 sha1:6d50f65d3bdab91c63444294d38f08aeff328e42
+df_1 sha256:e367cecc27e9bf5451b1c65828cb21938d36a5f8e39c1b03ad6509cc36bb8e9d
+
+df_2 sha1:71420ab81e254145d26d6fc0cddee64c1acd4787
+df_2 sha256:0f0a86d10347ff6921d03a3c954679f3f1d14fa3d5cd82f57b32c09755f3a47d
+
+dfd1 sha1:68a6d8b91da11045cf4aa3a5ab9f2a781c701249
+dfd1 sha256:f3bd3265b02b6978ce86490d8ad026c573639c974b3de1d9faf30d8d5a77d3d5
+
+dm_1 sha1:141c1f1642328e4bc46a7d801a71da392e66791e
+dm_1 sha256:c89f8656e7b94e21ee5fbaf0e2149bbf783c51edbe2ce110349cac13059ee7ed
+
+dm_2 sha1:3c4d8de5fbad08572bab8e10eef8dbb264cf0231
+dm_2 sha256:83a572e37e0c94086294dae2cecc43d9131afd6f6c906e495c78972230b54988
+
+dn_1 sha1:35abde1506ddf806572ff4d407bd06885d0f8ee9
+dn_1 sha256:775d5852582070e620be63327bfa515fab8f71c7ac3e4f0c3cd6267b4377ba28
+
+ll_2 sha1:1d41122ebdd7a640f29d3c9cc4f9d70094374762
+ll_2 sha256:7917b4948a883cfed0a77d3d5a625dc8577d6ddcc3c6c3bbc56c4d4226a2246d
+
+md_1 sha1:03f24c8c4700babccfd28b654e7e8eac402ad6cd
+md_1 sha256:fc9f30369b978595ad685ba11ca9a17de0af16d79cd4b629975f4f1590033902
+
+md_2 sha1:103d9f89b50b9aad03054b579be5e7aa665f2d57
+md_2 sha256:fc78ec75275628762fe520479a6b2398dec295ce7aabcb1d15e5963c7b4e9317
+
+mm_1 sha1:b258508afb7ceb449981bd9d63d2d3e971bf8d34
+mm_1 sha256:a4b7847d228e900e3000285e240c20fd96f9dd41ce1445305f6eada126d4a04a
+
+mm_2 sha1:b431b272d829ff3aa4d1a5085f4394ab4d3305b6
+mm_2 sha256:3f8b83ea36aacf689bcf1a1290a9a8ed341564d32682ea6f76fea9a979186782
+
+mm_3 sha1:19989d4559aae417fedee240ccf2ba315ea4dc2b
+mm_3 sha256:71b3bfc5747ac033fff9ea0ab39ee453a3af2969890e75d6ef547b87544e2681
+
+mn_1 sha1:bd084b0c27c7b6cc34f11d6d0509a29be3caf970
+mn_1 sha256:47a67450583d7a329eb01a7c4ba644945af72c0ed2c7c95eb5a00d6e46d4d483
+
+mn_2 sha1:a716d58de4a570e0038f5c307bd8db34daea021f
+mn_2 sha256:f95104c1ebe27acb84bac25a7be98c71f6b8d3054b21f357a5be0c524ad97e08
+
+nm_1 sha1:c8f25781e8f1792e3e40b74225e20553041b5226
+nm_1 sha256:09baddc7afaa62e62e152c23c9c3ab94bf15a3894031e227e9be7fe68e1f4e49
+
+nm_2 sha1:cdb9a8c3da571502ac30225e9c17beccb8387983
+nm_2 sha256:58b5227956ac2d2a08d0efa513c0ae37430948b16791ea3869a1308dbf05536d
+
+na_1 sha1:15885881ea69115351c09b38371f0348a3fb8c67
+na_1 sha256:18e4fdd1670cd7968ee23d35bfd29e5418d56fb190c840094c1c57ceee0aad8f
+
+nd_1 sha1:a4e179e4291e5536a5e1c82e091052772d2c5a93
+nd_1 sha256:07dac9b01d00956ea0c65bd993d7de4864aeef2ed3cbb1255d9f1d949fcd6df6
+
+ss_1 sha1:40c959f984c8b89a2b02520d17f00d717f024397
+ss_1 sha256:50fc1b5df74d9910db2f9270993484235f15b69b75b01bcfb53e059289d14af9
+
+ss_2 sha1:2ac547ae9614a00d1b28275de608131f7a0e259f
+ss_2 sha256:a90f02e6044f1497d13db587d22ab12f90150a7d1e084afcf96065fab35ae2bc
+
+tt_1 sha1:4ac13458899ab908ef3b1128fa378daefc88d356
+tt_1 sha256:c53113c7dd5060e86b5b251428bd058f6726f66273c6a24bff1c61a04f498dd3
+
+tt_2 sha1:4c86f9a85fbc5e6804ee2e17a797538fbe785bca
+tt_2 sha256:0775f2a296129a7cf2862b46bc0e88c14d593f2773a3e3fb1c5193db6f5a7e77
+
+tt_3 sha1:c4e4a12231b9fa79a0053cb6077fcb21bb5b135a
+tt_3 sha256:47860f93cdd211f96443e0560f21c57ab6c2f4b0ac27ff03651a352e53fe8484
+
+z__1 sha1:7d670fdcdb9929f6c7dac196ff78689cd1c566a1
+z__1 sha256:44d0f37aff5e51cfcfdd1134c93a6419bcca7b9964f792ffcd5f9b4fcba1ee63
+
+z__2 sha1:5e5f22072bb39f6e12cf663a57cb634c76eefb49
+z__2 sha256:d29de162113190fed104eb5f010820cef4e315f89b9326e8497f7219fb737894
+
+z__3 sha1:1ba523955d5160681af65cb776411f574c1e8155
+z__3 sha256:07422d772b07794ab4369a5648e617719f89c2d2212cbeab05d97214b6471636
+
+zaa1 sha1:8acb8e9750e3f644bf323fcf3d338849db106c77
+zaa1 sha256:e79b029282c8abec2d9f3f7faceaf2a1405e02d1f368e66450ae66cf5b68d1f4
+
+zaa2 sha1:6c0b99286d0bce551ac4a7b3dff8b706edff3715
+zaa2 sha256:c82bd78c3e69ea1796e6b1a7a3ba45bb106c50e819296475b862123d3f5cc5a0
+
+zan1 sha1:087494262084cefee7ed484d20c8dc0580791272
+zan1 sha256:4b159eb3804d05599023dd074f771d06d02870f4ab24a7165add8ac3d703b8d3
+
+zdd1 sha1:879007efae624d2b1307214b24a956f0a8d686a8
+zdd1 sha256:eecfdd4d8092dd0363fb6d4548b54c6afc8982c3ed9b34e393f1d6a921d8eaa3
+
+zdm1 sha1:9b541b2275c06e3a7b13f28badf5294e2ae63df4
+zdm1 sha256:ab136e88e19a843c4bf7713d2090d5a2186ba16a6a80dacc12eeddd256a8e556
+
+zdm2 sha1:d77371d15817fcaa57eeec27f770c505ba974ec1
+zdm2 sha256:1c1a5f57363f46a15d95ce8527b3c2c158d88d16853b4acbf81bd20fd2c89a46
+
+zdn1 sha1:beb5d38c55283d280685ea21a0e50cfcc0ca064a
+zdn1 sha256:0f0eca66183617b0aa5ad74b256540329f841470922ca6760263c996d825eb18
+
+zmd1 sha1:d41fda41b7ec4de46b43cb7ea42a45001ae393d5
+zmd1 sha256:1ed32d481852eddf31a0ce12652a0ad14bf5b7a842667b5dbb0b50f35bf1c80a
+
+zmd2 sha1:a79ac3be9377639e1c7d1edf1ae1b3a5f0ccd8a9
+zmd2 sha256:b238da211b404f8917df2d9c6f7030535e904b2186131007a3c292ec6902f933
+
+zmm1 sha1:4ca22bae2527d3d9e1676498a0fba3b355bd1278
+zmm1 sha256:072b1d85b5f34fabc99dfa46008c5418df68302d3e317430006f49b32d244226
+
+zmm2 sha1:61422ba9c2c873416061a88cd40a59a35b576474
+zmm2 sha256:81dd5d2b3c5cda16fef552256aed4e2ea0802a8450a08f308a92142112ff6dda
+
+zmm3 sha1:697aad7715a1e7306ca76290a3dd4208fbaeddfa
+zmm3 sha256:8b10fab49e9be3414aa5e9a93d0e46f9569053440138a7c19a5eb5536d8e95bf
+
+zmn1 sha1:b16d7b25b869f2beb124efa53467d8a1550ad694
+zmn1 sha256:609e4f75d1295e844c826feeba213acb0b6cfc609adfe8ff705b19e3829ae3e9
+
+zmn2 sha1:a5c544c21cfcb07eb80a4d89a5b7d1570002edfd
+zmn2 sha256:d6d03edf2dc1a3b267a8205de5f41a2ff4b03def8c7ae02052b543fb09d589fc
+
+zna1 sha1:d12979c22fff69c59ca9409e7a8fe3ee25eaee80
+zna1 sha256:b37b80e789e8ea32aa323f004628f02013f632124b0282c7fe00a127d3c64c3c
+
+znd1 sha1:a18393c636b98e9bd7296b8b437ea4992b72440c
+znd1 sha256:af92a22eee8c38410a0c9d2b5135a10aeb052cbc7cf675541ed9a67bfcaf7cf9
+
+znm1 sha1:3fdbe17fd013303a2e981e1ca1c6cd6e72789087
+znm1 sha256:f75aeaa0c11e76918e381c105f0752932c6150e941fec565d24fa31098a13dc1
+
+znm2 sha1:7e09d6a3a14bd630913e8c75693cea32157b606d
+znm2 sha256:938d73cfbaa1c902a84fb5b3afd9736aa0590367fb9bd59c6c4d072ce70fcd6d
+EOF
+
+cat >.test-plain-OA <<EOF
+:000000 100644 $(test_oid zero) $(test_oid aa_1) A	AA
+:000000 100644 $(test_oid zero) $(test_oid an_1) A	AN
+:100644 000000 $(test_oid dd_1) $(test_oid zero) D	DD
+:000000 040000 $(test_oid zero) $(test_oid df_1) A	DF
+:100644 000000 $(test_oid dm_1) $(test_oid zero) D	DM
+:100644 000000 $(test_oid dn_1) $(test_oid zero) D	DN
+:000000 100644 $(test_oid zero) $(test_oid ll_2) A	LL
+:100644 100644 $(test_oid md_1) $(test_oid md_2) M	MD
+:100644 100644 $(test_oid mm_1) $(test_oid mm_2) M	MM
+:100644 100644 $(test_oid mn_1) $(test_oid mn_2) M	MN
+:100644 100644 $(test_oid ss_1) $(test_oid ss_2) M	SS
+:100644 100644 $(test_oid tt_1) $(test_oid tt_2) M	TT
+:040000 040000 $(test_oid z__1) $(test_oid z__2) M	Z
 EOF
 
-cat >.test-recursive-OA <<\EOF
-:000000 100644 0000000000000000000000000000000000000000 ccba72ad3888a3520b39efcf780b9ee64167535d A	AA
-:000000 100644 0000000000000000000000000000000000000000 7e426fb079479fd67f6d81f984e4ec649a44bc25 A	AN
-:100644 000000 bcc68ef997017466d5c9094bcf7692295f588c9a 0000000000000000000000000000000000000000 D	DD
-:000000 100644 0000000000000000000000000000000000000000 68a6d8b91da11045cf4aa3a5ab9f2a781c701249 A	DF/DF
-:100644 000000 141c1f1642328e4bc46a7d801a71da392e66791e 0000000000000000000000000000000000000000 D	DM
-:100644 000000 35abde1506ddf806572ff4d407bd06885d0f8ee9 0000000000000000000000000000000000000000 D	DN
-:000000 100644 0000000000000000000000000000000000000000 1d41122ebdd7a640f29d3c9cc4f9d70094374762 A	LL
-:100644 100644 03f24c8c4700babccfd28b654e7e8eac402ad6cd 103d9f89b50b9aad03054b579be5e7aa665f2d57 M	MD
-:100644 100644 b258508afb7ceb449981bd9d63d2d3e971bf8d34 b431b272d829ff3aa4d1a5085f4394ab4d3305b6 M	MM
-:100644 100644 bd084b0c27c7b6cc34f11d6d0509a29be3caf970 a716d58de4a570e0038f5c307bd8db34daea021f M	MN
-:100644 100644 40c959f984c8b89a2b02520d17f00d717f024397 2ac547ae9614a00d1b28275de608131f7a0e259f M	SS
-:100644 100644 4ac13458899ab908ef3b1128fa378daefc88d356 4c86f9a85fbc5e6804ee2e17a797538fbe785bca M	TT
-:000000 100644 0000000000000000000000000000000000000000 8acb8e9750e3f644bf323fcf3d338849db106c77 A	Z/AA
-:000000 100644 0000000000000000000000000000000000000000 087494262084cefee7ed484d20c8dc0580791272 A	Z/AN
-:100644 000000 879007efae624d2b1307214b24a956f0a8d686a8 0000000000000000000000000000000000000000 D	Z/DD
-:100644 000000 9b541b2275c06e3a7b13f28badf5294e2ae63df4 0000000000000000000000000000000000000000 D	Z/DM
-:100644 000000 beb5d38c55283d280685ea21a0e50cfcc0ca064a 0000000000000000000000000000000000000000 D	Z/DN
-:100644 100644 d41fda41b7ec4de46b43cb7ea42a45001ae393d5 a79ac3be9377639e1c7d1edf1ae1b3a5f0ccd8a9 M	Z/MD
-:100644 100644 4ca22bae2527d3d9e1676498a0fba3b355bd1278 61422ba9c2c873416061a88cd40a59a35b576474 M	Z/MM
-:100644 100644 b16d7b25b869f2beb124efa53467d8a1550ad694 a5c544c21cfcb07eb80a4d89a5b7d1570002edfd M	Z/MN
+cat >.test-recursive-OA <<EOF
+:000000 100644 $(test_oid zero) $(test_oid aa_1) A	AA
+:000000 100644 $(test_oid zero) $(test_oid an_1) A	AN
+:100644 000000 $(test_oid dd_1) $(test_oid zero) D	DD
+:000000 100644 $(test_oid zero) $(test_oid dfd1) A	DF/DF
+:100644 000000 $(test_oid dm_1) $(test_oid zero) D	DM
+:100644 000000 $(test_oid dn_1) $(test_oid zero) D	DN
+:000000 100644 $(test_oid zero) $(test_oid ll_2) A	LL
+:100644 100644 $(test_oid md_1) $(test_oid md_2) M	MD
+:100644 100644 $(test_oid mm_1) $(test_oid mm_2) M	MM
+:100644 100644 $(test_oid mn_1) $(test_oid mn_2) M	MN
+:100644 100644 $(test_oid ss_1) $(test_oid ss_2) M	SS
+:100644 100644 $(test_oid tt_1) $(test_oid tt_2) M	TT
+:000000 100644 $(test_oid zero) $(test_oid zaa1) A	Z/AA
+:000000 100644 $(test_oid zero) $(test_oid zan1) A	Z/AN
+:100644 000000 $(test_oid zdd1) $(test_oid zero) D	Z/DD
+:100644 000000 $(test_oid zdm1) $(test_oid zero) D	Z/DM
+:100644 000000 $(test_oid zdn1) $(test_oid zero) D	Z/DN
+:100644 100644 $(test_oid zmd1) $(test_oid zmd2) M	Z/MD
+:100644 100644 $(test_oid zmm1) $(test_oid zmm2) M	Z/MM
+:100644 100644 $(test_oid zmn1) $(test_oid zmn2) M	Z/MN
 EOF
-cat >.test-plain-OB <<\EOF
-:000000 100644 0000000000000000000000000000000000000000 6aa2b5335b16431a0ef71e5c0a28be69183cf6a2 A	AA
-:100644 000000 bcc68ef997017466d5c9094bcf7692295f588c9a 0000000000000000000000000000000000000000 D	DD
-:000000 100644 0000000000000000000000000000000000000000 71420ab81e254145d26d6fc0cddee64c1acd4787 A	DF
-:100644 100644 141c1f1642328e4bc46a7d801a71da392e66791e 3c4d8de5fbad08572bab8e10eef8dbb264cf0231 M	DM
-:000000 100644 0000000000000000000000000000000000000000 1d41122ebdd7a640f29d3c9cc4f9d70094374762 A	LL
-:100644 000000 03f24c8c4700babccfd28b654e7e8eac402ad6cd 0000000000000000000000000000000000000000 D	MD
-:100644 100644 b258508afb7ceb449981bd9d63d2d3e971bf8d34 19989d4559aae417fedee240ccf2ba315ea4dc2b M	MM
-:000000 100644 0000000000000000000000000000000000000000 15885881ea69115351c09b38371f0348a3fb8c67 A	NA
-:100644 000000 a4e179e4291e5536a5e1c82e091052772d2c5a93 0000000000000000000000000000000000000000 D	ND
-:100644 100644 c8f25781e8f1792e3e40b74225e20553041b5226 cdb9a8c3da571502ac30225e9c17beccb8387983 M	NM
-:100644 100644 40c959f984c8b89a2b02520d17f00d717f024397 2ac547ae9614a00d1b28275de608131f7a0e259f M	SS
-:100644 100644 4ac13458899ab908ef3b1128fa378daefc88d356 c4e4a12231b9fa79a0053cb6077fcb21bb5b135a M	TT
-:040000 040000 7d670fdcdb9929f6c7dac196ff78689cd1c566a1 1ba523955d5160681af65cb776411f574c1e8155 M	Z
+cat >.test-plain-OB <<EOF
+:000000 100644 $(test_oid zero) $(test_oid aa_2) A	AA
+:100644 000000 $(test_oid dd_1) $(test_oid zero) D	DD
+:000000 100644 $(test_oid zero) $(test_oid df_2) A	DF
+:100644 100644 $(test_oid dm_1) $(test_oid dm_2) M	DM
+:000000 100644 $(test_oid zero) $(test_oid ll_2) A	LL
+:100644 000000 $(test_oid md_1) $(test_oid zero) D	MD
+:100644 100644 $(test_oid mm_1) $(test_oid mm_3) M	MM
+:000000 100644 $(test_oid zero) $(test_oid na_1) A	NA
+:100644 000000 $(test_oid nd_1) $(test_oid zero) D	ND
+:100644 100644 $(test_oid nm_1) $(test_oid nm_2) M	NM
+:100644 100644 $(test_oid ss_1) $(test_oid ss_2) M	SS
+:100644 100644 $(test_oid tt_1) $(test_oid tt_3) M	TT
+:040000 040000 $(test_oid z__1) $(test_oid z__3) M	Z
 EOF
-cat >.test-recursive-OB <<\EOF
-:000000 100644 0000000000000000000000000000000000000000 6aa2b5335b16431a0ef71e5c0a28be69183cf6a2 A	AA
-:100644 000000 bcc68ef997017466d5c9094bcf7692295f588c9a 0000000000000000000000000000000000000000 D	DD
-:000000 100644 0000000000000000000000000000000000000000 71420ab81e254145d26d6fc0cddee64c1acd4787 A	DF
-:100644 100644 141c1f1642328e4bc46a7d801a71da392e66791e 3c4d8de5fbad08572bab8e10eef8dbb264cf0231 M	DM
-:000000 100644 0000000000000000000000000000000000000000 1d41122ebdd7a640f29d3c9cc4f9d70094374762 A	LL
-:100644 000000 03f24c8c4700babccfd28b654e7e8eac402ad6cd 0000000000000000000000000000000000000000 D	MD
-:100644 100644 b258508afb7ceb449981bd9d63d2d3e971bf8d34 19989d4559aae417fedee240ccf2ba315ea4dc2b M	MM
-:000000 100644 0000000000000000000000000000000000000000 15885881ea69115351c09b38371f0348a3fb8c67 A	NA
-:100644 000000 a4e179e4291e5536a5e1c82e091052772d2c5a93 0000000000000000000000000000000000000000 D	ND
-:100644 100644 c8f25781e8f1792e3e40b74225e20553041b5226 cdb9a8c3da571502ac30225e9c17beccb8387983 M	NM
-:100644 100644 40c959f984c8b89a2b02520d17f00d717f024397 2ac547ae9614a00d1b28275de608131f7a0e259f M	SS
-:100644 100644 4ac13458899ab908ef3b1128fa378daefc88d356 c4e4a12231b9fa79a0053cb6077fcb21bb5b135a M	TT
-:000000 100644 0000000000000000000000000000000000000000 6c0b99286d0bce551ac4a7b3dff8b706edff3715 A	Z/AA
-:100644 000000 879007efae624d2b1307214b24a956f0a8d686a8 0000000000000000000000000000000000000000 D	Z/DD
-:100644 100644 9b541b2275c06e3a7b13f28badf5294e2ae63df4 d77371d15817fcaa57eeec27f770c505ba974ec1 M	Z/DM
-:100644 000000 d41fda41b7ec4de46b43cb7ea42a45001ae393d5 0000000000000000000000000000000000000000 D	Z/MD
-:100644 100644 4ca22bae2527d3d9e1676498a0fba3b355bd1278 697aad7715a1e7306ca76290a3dd4208fbaeddfa M	Z/MM
-:000000 100644 0000000000000000000000000000000000000000 d12979c22fff69c59ca9409e7a8fe3ee25eaee80 A	Z/NA
-:100644 000000 a18393c636b98e9bd7296b8b437ea4992b72440c 0000000000000000000000000000000000000000 D	Z/ND
-:100644 100644 3fdbe17fd013303a2e981e1ca1c6cd6e72789087 7e09d6a3a14bd630913e8c75693cea32157b606d M	Z/NM
+cat >.test-recursive-OB <<EOF
+:000000 100644 $(test_oid zero) $(test_oid aa_2) A	AA
+:100644 000000 $(test_oid dd_1) $(test_oid zero) D	DD
+:000000 100644 $(test_oid zero) $(test_oid df_2) A	DF
+:100644 100644 $(test_oid dm_1) $(test_oid dm_2) M	DM
+:000000 100644 $(test_oid zero) $(test_oid ll_2) A	LL
+:100644 000000 $(test_oid md_1) $(test_oid zero) D	MD
+:100644 100644 $(test_oid mm_1) $(test_oid mm_3) M	MM
+:000000 100644 $(test_oid zero) $(test_oid na_1) A	NA
+:100644 000000 $(test_oid nd_1) $(test_oid zero) D	ND
+:100644 100644 $(test_oid nm_1) $(test_oid nm_2) M	NM
+:100644 100644 $(test_oid ss_1) $(test_oid ss_2) M	SS
+:100644 100644 $(test_oid tt_1) $(test_oid tt_3) M	TT
+:000000 100644 $(test_oid zero) $(test_oid zaa2) A	Z/AA
+:100644 000000 $(test_oid zdd1) $(test_oid zero) D	Z/DD
+:100644 100644 $(test_oid zdm1) $(test_oid zdm2) M	Z/DM
+:100644 000000 $(test_oid zmd1) $(test_oid zero) D	Z/MD
+:100644 100644 $(test_oid zmm1) $(test_oid zmm3) M	Z/MM
+:000000 100644 $(test_oid zero) $(test_oid zna1) A	Z/NA
+:100644 000000 $(test_oid znd1) $(test_oid zero) D	Z/ND
+:100644 100644 $(test_oid znm1) $(test_oid znm2) M	Z/NM
 EOF
-cat >.test-plain-AB <<\EOF
-:100644 100644 ccba72ad3888a3520b39efcf780b9ee64167535d 6aa2b5335b16431a0ef71e5c0a28be69183cf6a2 M	AA
-:100644 000000 7e426fb079479fd67f6d81f984e4ec649a44bc25 0000000000000000000000000000000000000000 D	AN
-:000000 100644 0000000000000000000000000000000000000000 71420ab81e254145d26d6fc0cddee64c1acd4787 A	DF
-:040000 000000 6d50f65d3bdab91c63444294d38f08aeff328e42 0000000000000000000000000000000000000000 D	DF
-:000000 100644 0000000000000000000000000000000000000000 3c4d8de5fbad08572bab8e10eef8dbb264cf0231 A	DM
-:000000 100644 0000000000000000000000000000000000000000 35abde1506ddf806572ff4d407bd06885d0f8ee9 A	DN
-:100644 000000 103d9f89b50b9aad03054b579be5e7aa665f2d57 0000000000000000000000000000000000000000 D	MD
-:100644 100644 b431b272d829ff3aa4d1a5085f4394ab4d3305b6 19989d4559aae417fedee240ccf2ba315ea4dc2b M	MM
-:100644 100644 a716d58de4a570e0038f5c307bd8db34daea021f bd084b0c27c7b6cc34f11d6d0509a29be3caf970 M	MN
-:000000 100644 0000000000000000000000000000000000000000 15885881ea69115351c09b38371f0348a3fb8c67 A	NA
-:100644 000000 a4e179e4291e5536a5e1c82e091052772d2c5a93 0000000000000000000000000000000000000000 D	ND
-:100644 100644 c8f25781e8f1792e3e40b74225e20553041b5226 cdb9a8c3da571502ac30225e9c17beccb8387983 M	NM
-:100644 100644 4c86f9a85fbc5e6804ee2e17a797538fbe785bca c4e4a12231b9fa79a0053cb6077fcb21bb5b135a M	TT
-:040000 040000 5e5f22072bb39f6e12cf663a57cb634c76eefb49 1ba523955d5160681af65cb776411f574c1e8155 M	Z
+cat >.test-plain-AB <<EOF
+:100644 100644 $(test_oid aa_1) $(test_oid aa_2) M	AA
+:100644 000000 $(test_oid an_1) $(test_oid zero) D	AN
+:000000 100644 $(test_oid zero) $(test_oid df_2) A	DF
+:040000 000000 $(test_oid df_1) $(test_oid zero) D	DF
+:000000 100644 $(test_oid zero) $(test_oid dm_2) A	DM
+:000000 100644 $(test_oid zero) $(test_oid dn_1) A	DN
+:100644 000000 $(test_oid md_2) $(test_oid zero) D	MD
+:100644 100644 $(test_oid mm_2) $(test_oid mm_3) M	MM
+:100644 100644 $(test_oid mn_2) $(test_oid mn_1) M	MN
+:000000 100644 $(test_oid zero) $(test_oid na_1) A	NA
+:100644 000000 $(test_oid nd_1) $(test_oid zero) D	ND
+:100644 100644 $(test_oid nm_1) $(test_oid nm_2) M	NM
+:100644 100644 $(test_oid tt_2) $(test_oid tt_3) M	TT
+:040000 040000 $(test_oid z__2) $(test_oid z__3) M	Z
 EOF
-cat >.test-recursive-AB <<\EOF
-:100644 100644 ccba72ad3888a3520b39efcf780b9ee64167535d 6aa2b5335b16431a0ef71e5c0a28be69183cf6a2 M	AA
-:100644 000000 7e426fb079479fd67f6d81f984e4ec649a44bc25 0000000000000000000000000000000000000000 D	AN
-:000000 100644 0000000000000000000000000000000000000000 71420ab81e254145d26d6fc0cddee64c1acd4787 A	DF
-:100644 000000 68a6d8b91da11045cf4aa3a5ab9f2a781c701249 0000000000000000000000000000000000000000 D	DF/DF
-:000000 100644 0000000000000000000000000000000000000000 3c4d8de5fbad08572bab8e10eef8dbb264cf0231 A	DM
-:000000 100644 0000000000000000000000000000000000000000 35abde1506ddf806572ff4d407bd06885d0f8ee9 A	DN
-:100644 000000 103d9f89b50b9aad03054b579be5e7aa665f2d57 0000000000000000000000000000000000000000 D	MD
-:100644 100644 b431b272d829ff3aa4d1a5085f4394ab4d3305b6 19989d4559aae417fedee240ccf2ba315ea4dc2b M	MM
-:100644 100644 a716d58de4a570e0038f5c307bd8db34daea021f bd084b0c27c7b6cc34f11d6d0509a29be3caf970 M	MN
-:000000 100644 0000000000000000000000000000000000000000 15885881ea69115351c09b38371f0348a3fb8c67 A	NA
-:100644 000000 a4e179e4291e5536a5e1c82e091052772d2c5a93 0000000000000000000000000000000000000000 D	ND
-:100644 100644 c8f25781e8f1792e3e40b74225e20553041b5226 cdb9a8c3da571502ac30225e9c17beccb8387983 M	NM
-:100644 100644 4c86f9a85fbc5e6804ee2e17a797538fbe785bca c4e4a12231b9fa79a0053cb6077fcb21bb5b135a M	TT
-:100644 100644 8acb8e9750e3f644bf323fcf3d338849db106c77 6c0b99286d0bce551ac4a7b3dff8b706edff3715 M	Z/AA
-:100644 000000 087494262084cefee7ed484d20c8dc0580791272 0000000000000000000000000000000000000000 D	Z/AN
-:000000 100644 0000000000000000000000000000000000000000 d77371d15817fcaa57eeec27f770c505ba974ec1 A	Z/DM
-:000000 100644 0000000000000000000000000000000000000000 beb5d38c55283d280685ea21a0e50cfcc0ca064a A	Z/DN
-:100644 000000 a79ac3be9377639e1c7d1edf1ae1b3a5f0ccd8a9 0000000000000000000000000000000000000000 D	Z/MD
-:100644 100644 61422ba9c2c873416061a88cd40a59a35b576474 697aad7715a1e7306ca76290a3dd4208fbaeddfa M	Z/MM
-:100644 100644 a5c544c21cfcb07eb80a4d89a5b7d1570002edfd b16d7b25b869f2beb124efa53467d8a1550ad694 M	Z/MN
-:000000 100644 0000000000000000000000000000000000000000 d12979c22fff69c59ca9409e7a8fe3ee25eaee80 A	Z/NA
-:100644 000000 a18393c636b98e9bd7296b8b437ea4992b72440c 0000000000000000000000000000000000000000 D	Z/ND
-:100644 100644 3fdbe17fd013303a2e981e1ca1c6cd6e72789087 7e09d6a3a14bd630913e8c75693cea32157b606d M	Z/NM
+cat >.test-recursive-AB <<EOF
+:100644 100644 $(test_oid aa_1) $(test_oid aa_2) M	AA
+:100644 000000 $(test_oid an_1) $(test_oid zero) D	AN
+:000000 100644 $(test_oid zero) $(test_oid df_2) A	DF
+:100644 000000 $(test_oid dfd1) $(test_oid zero) D	DF/DF
+:000000 100644 $(test_oid zero) $(test_oid dm_2) A	DM
+:000000 100644 $(test_oid zero) $(test_oid dn_1) A	DN
+:100644 000000 $(test_oid md_2) $(test_oid zero) D	MD
+:100644 100644 $(test_oid mm_2) $(test_oid mm_3) M	MM
+:100644 100644 $(test_oid mn_2) $(test_oid mn_1) M	MN
+:000000 100644 $(test_oid zero) $(test_oid na_1) A	NA
+:100644 000000 $(test_oid nd_1) $(test_oid zero) D	ND
+:100644 100644 $(test_oid nm_1) $(test_oid nm_2) M	NM
+:100644 100644 $(test_oid tt_2) $(test_oid tt_3) M	TT
+:100644 100644 $(test_oid zaa1) $(test_oid zaa2) M	Z/AA
+:100644 000000 $(test_oid zan1) $(test_oid zero) D	Z/AN
+:000000 100644 $(test_oid zero) $(test_oid zdm2) A	Z/DM
+:000000 100644 $(test_oid zero) $(test_oid zdn1) A	Z/DN
+:100644 000000 $(test_oid zmd2) $(test_oid zero) D	Z/MD
+:100644 100644 $(test_oid zmm2) $(test_oid zmm3) M	Z/MM
+:100644 100644 $(test_oid zmn2) $(test_oid zmn1) M	Z/MN
+:000000 100644 $(test_oid zero) $(test_oid zna1) A	Z/NA
+:100644 000000 $(test_oid znd1) $(test_oid zero) D	Z/ND
+:100644 100644 $(test_oid znm1) $(test_oid znm2) M	Z/NM
 EOF
 
 cmp_diff_files_output () {
#228 lore raw

PATCH v2 14/14 t4009: make hash size independent

In reply to brian m. carlson

Instead of hard-coding object IDs, compute them and use those in the comparison. Note that the comparison code ignores the actual object IDs, but does check that they're the right size, so computing them is the easiest way to ensure that they are.

---
 t/t4009-diff-rename-4.sh | 19 +++++++++++--------
 1 file changed, 11 insertions(+), 8 deletions(-)
t/t4009-diff-rename-4.sh +11 -8
index 3641fd84d6..b63bdf031f 100755
@@ -14,6 +14,7 @@ test_expect_success \
     'cat "$TEST_DIRECTORY"/diff-lib/COPYING >COPYING &&
      echo frotz >rezrov &&
     git update-index --add COPYING rezrov &&
+    orig=$(git hash-object COPYING) &&
     tree=$(git write-tree) &&
     echo $tree'
 
@@ -22,6 +23,8 @@ test_expect_success \
     'sed -e 's/HOWEVER/However/' <COPYING >COPYING.1 &&
     sed -e 's/GPL/G.P.L/g' <COPYING >COPYING.2 &&
     rm -f COPYING &&
+    c1=$(git hash-object COPYING.1) &&
+    c2=$(git hash-object COPYING.2) &&
     git update-index --add --remove COPYING COPYING.?'
 
 # tree has COPYING and rezrov.  work tree has COPYING.1 and COPYING.2,
@@ -31,11 +34,11 @@ test_expect_success \
 
 git diff-index -z -C $tree >current
 
-cat >expected <<\EOF
-:100644 100644 6ff87c4664981e4397625791c8ea3bbb5f2279a3 0603b3238a076dc6c8022aedc6648fa523a17178 C1234
+cat >expected <<EOF
+:100644 100644 $orig $c1 C1234
 COPYING
 COPYING.1
-:100644 100644 6ff87c4664981e4397625791c8ea3bbb5f2279a3 06c67961bbaed34a127f76d261f4c0bf73eda471 R1234
+:100644 100644 $orig $c2 R1234
 COPYING
 COPYING.2
 EOF
@@ -57,10 +60,10 @@ test_expect_success \
 # about rezrov.
 
 git diff-index -z -C $tree >current
-cat >expected <<\EOF
-:100644 100644 6ff87c4664981e4397625791c8ea3bbb5f2279a3 06c67961bbaed34a127f76d261f4c0bf73eda471 M
+cat >expected <<EOF
+:100644 100644 $orig $c2 M
 COPYING
-:100644 100644 6ff87c4664981e4397625791c8ea3bbb5f2279a3 0603b3238a076dc6c8022aedc6648fa523a17178 C1234
+:100644 100644 $orig $c1 C1234
 COPYING
 COPYING.1
 EOF
@@ -82,8 +85,8 @@ test_expect_success \
      git update-index --add --remove COPYING COPYING.1'
 
 git diff-index -z -C --find-copies-harder $tree >current
-cat >expected <<\EOF
-:100644 100644 6ff87c4664981e4397625791c8ea3bbb5f2279a3 0603b3238a076dc6c8022aedc6648fa523a17178 C1234
+cat >expected <<EOF
+:100644 100644 $orig $c1 C1234
 COPYING
 COPYING.1
 EOF