Skip to contentSkip to conversation

test-lib: GIT_TEST_SANITIZE_LEAK_LOG enabled by default

PATCH Superseded Rubén Justo 4 participants 10 messages Jul 10, 2024 – Jul 22, 2024

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 test-lib: GIT_TEST_SANITIZE_LEAK_LOG enabled by default

As we describe in t/README, it can happen that:

Some tests run "git" (or "test-tool" etc.) without properly checking the exit code, or git will invoke itself and fail to ferry the abort() exit code to the original caller.

Therefore, GIT_TEST_SANITIZE_LEAK_LOG must be set to true to capture all memory leaks triggered by the tests when SANITIZE=leak.

Set it to true by default, and stop worrying about someone checking for leaks who isn't aware of this option and might be missing some leaks.

---
 ci/lib.sh     | 1 -
 t/README      | 4 ++--
 t/test-lib.sh | 2 +-
 3 files changed, 3 insertions(+), 4 deletions(-)
ci/lib.sh +0 -1
index ff66ad356b..fe52954828 100755
@@ -374,7 +374,6 @@ linux-musl)
 linux-leaks|linux-reftable-leaks)
 	export SANITIZE=leak
 	export GIT_TEST_PASSING_SANITIZE_LEAK=true
-	export GIT_TEST_SANITIZE_LEAK_LOG=true
 	;;
 linux-asan-ubsan)
 	export SANITIZE=address,undefined
t/README +2 -2
index d9e0e07506..1c97bc3331 100644
@@ -382,10 +382,10 @@ mapping between "TEST_PASSES_SANITIZE_LEAK=true" and those tests that
 pass under "SANITIZE=leak". This is especially useful when testing a
 series that fixes various memory leaks with "git rebase -x".
 
-GIT_TEST_SANITIZE_LEAK_LOG=true will log memory leaks to
+GIT_TEST_SANITIZE_LEAK_LOG=<boolean> controls logging of memory leaks to
 "test-results/$TEST_NAME.leak/trace.*" files. The logs include a
 "dedup_token" (see +"ASAN_OPTIONS=help=1 ./git") and other options to
-make logs +machine-readable.
+make logs +machine-readable.  Defaults to "true" when SANITIZE=leak.
 
 With GIT_TEST_SANITIZE_LEAK_LOG=true we'll look at the leak logs
 before exiting and exit on failure if the logs showed that we had a
t/test-lib.sh +1 -1
index 7ed6d3fc47..1dd2ea4e07 100644
@@ -1578,7 +1578,7 @@ then
 		test_done
 	fi
 
-	if test_bool_env GIT_TEST_SANITIZE_LEAK_LOG false
+	if test_bool_env GIT_TEST_SANITIZE_LEAK_LOG true
 	then
 		if ! mkdir -p "$TEST_RESULTS_SAN_DIR"
 		then
Signature · 1 line
2.45.1
#1 lore raw
Quoted 11 lines · Rubén Justo

As we describe in t/README, it can happen that:

Some tests run "git" (or "test-tool" etc.) without properly checking the exit code, or git will invoke itself and fail to ferry the abort() exit code to the original caller.

Therefore, GIT_TEST_SANITIZE_LEAK_LOG must be set to true to capture all memory leaks triggered by the tests when SANITIZE=leak.

Set it to true by default, and stop worrying about someone checking for leaks who isn't aware of this option and might be missing some leaks.

I'm obviously in favor of this direction, but...why stop here? Do we expect somebody to set it to false? If not, then can't we just get rid of it entirely?

-Peff

#2 lore raw
Quoted 11 lines · Jeff King
Quoted 9 lines · Rubén Justo

As we describe in t/README, it can happen that:

Some tests run "git" (or "test-tool" etc.) without properly checking the exit code, or git will invoke itself and fail to ferry the abort() exit code to the original caller.

Therefore, GIT_TEST_SANITIZE_LEAK_LOG must be set to true to capture all memory leaks triggered by the tests when SANITIZE=leak.

Quoted 6 lines
Quoted 2 lines

Set it to true by default, and stop worrying about someone checking for leaks who isn't aware of this option and might be missing some leaks.

I'm obviously in favor of this direction, but...why stop here? Do we expect somebody to set it to false? If not, then can't we just get rid of it entirely?

I'd also be strongly in favor of just removing this variable altogether. I found it quite tedious to remember setting it when working on the memory leak fixes recently, and I'm not aware of any downsides.

Patrick

#3 lore raw

PATCH v2 1/1 test-lib: GIT_TEST_SANITIZE_LEAK_LOG enabled by default

In reply to Rubén Justo

As we currently describe in t/README, it can happen that:

Some tests run "git" (or "test-tool" etc.) without properly checking the exit code, or git will invoke itself and fail to ferry the abort() exit code to the original caller.

Therefore, GIT_TEST_SANITIZE_LEAK_LOG=true is needed to be set to capture all memory leaks triggered by our tests.

It seems unnecessary to force users to remember this option, as forgetting it could lead to missed memory leaks.

We could solve the problem by setting GIT_TEST_SANITIZE_LEAK_LOG to "true" by default, but that might suggest we think "false" makes sense, which isn't the case.

Therefore, the best approach is to remove the option entirely while maintaining the capability to detect memory leaks in blind spots of our tests.

---

@Peff: I thought your preference was to keep the option. I agree to remove it completely. This, v2, address that.

I'm traveling, and while I think the change doesn't break anything, I'd appreciate a double check ;-)

By the way, I used, more or less, the text for the message that Junio suggested.

 ci/lib.sh     |  1 -
 t/README      | 24 ------------------------
 t/test-lib.sh | 41 +++++++++++++++++++----------------------
 3 files changed, 19 insertions(+), 47 deletions(-)
ci/lib.sh +0 -1
index 814578ffc6..51f8f59a29 100755
@@ -370,7 +370,6 @@ linux-musl)
 linux-leaks|linux-reftable-leaks)
 	export SANITIZE=leak
 	export GIT_TEST_PASSING_SANITIZE_LEAK=true
-	export GIT_TEST_SANITIZE_LEAK_LOG=true
 	;;
 linux-asan-ubsan)
 	export SANITIZE=address,undefined
t/README +0 -24
index d9e0e07506..c2a732d59e 100644
@@ -382,33 +382,9 @@ mapping between "TEST_PASSES_SANITIZE_LEAK=true" and those tests that
 pass under "SANITIZE=leak". This is especially useful when testing a
 series that fixes various memory leaks with "git rebase -x".
 
-GIT_TEST_SANITIZE_LEAK_LOG=true will log memory leaks to
-"test-results/$TEST_NAME.leak/trace.*" files. The logs include a
-"dedup_token" (see +"ASAN_OPTIONS=help=1 ./git") and other options to
-make logs +machine-readable.
-
-With GIT_TEST_SANITIZE_LEAK_LOG=true we'll look at the leak logs
-before exiting and exit on failure if the logs showed that we had a
-memory leak, even if the test itself would have otherwise passed. This
-allows us to catch e.g. missing &&-chaining. This is especially useful
-when combined with "GIT_TEST_PASSING_SANITIZE_LEAK", see below.
-
 GIT_TEST_PASSING_SANITIZE_LEAK=check when combined with "--immediate"
 will run to completion faster, and result in the same failing
 tests. The only practical reason to run
-GIT_TEST_PASSING_SANITIZE_LEAK=check without "--immediate" is to
-combine it with "GIT_TEST_SANITIZE_LEAK_LOG=true". If we stop at the
-first failing test case our leak logs won't show subsequent leaks we
-might have run into.
-
-GIT_TEST_PASSING_SANITIZE_LEAK=(true|check) will not catch all memory
-leaks unless combined with GIT_TEST_SANITIZE_LEAK_LOG=true. Some tests
-run "git" (or "test-tool" etc.) without properly checking the exit
-code, or git will invoke itself and fail to ferry the abort() exit
-code to the original caller. When the two modes are combined we'll
-look at the "test-results/$TEST_NAME.leak/trace.*" files at the end of
-the test run to see if had memory leaks which the test itself didn't
-catch.
 
 GIT_TEST_PROTOCOL_VERSION=<n>, when set, makes 'protocol.version'
 default to n.
t/test-lib.sh +0 -0
index 79d3e0e7d9..942828c55d 100644
@@ -1270,8 +1270,8 @@ check_test_results_san_file_ () {
 		say "As TEST_PASSES_SANITIZE_LEAK=true isn't set the above leak is 

'ok' with GIT_TEST_PASSING_SANITIZE_LEAK=check" &&
invert_exit_code=t
else
- say "With GIT_TEST_SANITIZE_LEAK_LOG=true our logs revealed a memory leak, exit non-zero!" &&
- invert_exit_code=t
+ say "Our logs revealed a leak!" &&
+ test "$test_failure" != 0 || invert_exit_code=t
fi
}

@@ -1555,28 +1555,28 @@ then
passes_sanitize_leak=t
fi

- if test "$GIT_TEST_PASSING_SANITIZE_LEAK" = "check"
+ if test -z "$passes_sanitize_leak" &&
+ ! test "$GIT_TEST_PASSING_SANITIZE_LEAK" = "check" &&
+ test_bool_env GIT_TEST_PASSING_SANITIZE_LEAK false
then
- sanitize_leak_check=t
- if test -n "$invert_exit_code"
+ skip_all="skipping $this_test under GIT_TEST_PASSING_SANITIZE_LEAK=true"
+ test_done
+ else
+ if test "$GIT_TEST_PASSING_SANITIZE_LEAK" = "check"
then
- BAIL_OUT "cannot use --invert-exit-code under GIT_TEST_PASSING_SANITIZE_LEAK=check"
- fi
+ sanitize_leak_check=t
+ if test -n "$invert_exit_code"
+ then
+ BAIL_OUT "cannot use --invert-exit-code under GIT_TEST_PASSING_SANITIZE_LEAK=check"
+ fi

- if test -z "$passes_sanitize_leak"
- then
- say "in GIT_TEST_PASSING_SANITIZE_LEAK=check mode, setting --invert-exit-code for TEST_PASSES_SANITIZE_LEAK != true"
- invert_exit_code=t
+ if test -z "$passes_sanitize_leak"
+ then
+ say "in GIT_TEST_PASSING_SANITIZE_LEAK=check mode, setting --invert-exit-code for TEST_PASSES_SANITIZE_LEAK != true"
+ invert_exit_code=t
+ fi
fi
- elif test -z "$passes_sanitize_leak" &&
- test_bool_env GIT_TEST_PASSING_SANITIZE_LEAK false
- then
- skip_all="skipping $this_test under GIT_TEST_PASSING_SANITIZE_LEAK=true"
- test_done
- fi

- if test_bool_env GIT_TEST_SANITIZE_LEAK_LOG false
- then
if ! mkdir -p "$TEST_RESULTS_SAN_DIR"
then
BAIL_OUT "cannot create $TEST_RESULTS_SAN_DIR"
@@ -1599,9 +1599,6 @@ elif test "$GIT_TEST_PASSING_SANITIZE_LEAK" = "check" ||
test_bool_env GIT_TEST_PASSING_SANITIZE_LEAK false
then
BAIL_OUT_ENV_NEEDS_SANITIZE_LEAK "GIT_TEST_PASSING_SANITIZE_LEAK=true"
-elif test_bool_env GIT_TEST_SANITIZE_LEAK_LOG false
-then
- BAIL_OUT_ENV_NEEDS_SANITIZE_LEAK "GIT_TEST_SANITIZE_LEAK_LOG=true"
fi

if test "${GIT_TEST_CHAIN_LINT:-1}" != 0 &&

Signature · 1 line
2.45.1
#4 lore raw
Quoted 23 lines · Rubén Justo

As we currently describe in t/README, it can happen that:

Some tests run "git" (or "test-tool" etc.) without properly checking the exit code, or git will invoke itself and fail to ferry the abort() exit code to the original caller.

Therefore, GIT_TEST_SANITIZE_LEAK_LOG=true is needed to be set to capture all memory leaks triggered by our tests.

It seems unnecessary to force users to remember this option, as forgetting it could lead to missed memory leaks.

We could solve the problem by setting GIT_TEST_SANITIZE_LEAK_LOG to "true" by default, but that might suggest we think "false" makes sense, which isn't the case.

Therefore, the best approach is to remove the option entirely while maintaining the capability to detect memory leaks in blind spots of our tests.

---

Quoted 8 lines

@Peff: I thought your preference was to keep the option. I agree to remove it completely. This, v2, address that.

I'm traveling, and while I think the change doesn't break anything, I'd appreciate a double check ;-)

By the way, I used, more or less, the text for the message that Junio suggested.

Hmph, the above doesn't look like any, but as a standalone patch it is very much readable.

Quoted 4 lines
 ci/lib.sh     |  1 -
 t/README      | 24 ------------------------
 t/test-lib.sh | 41 +++++++++++++++++++----------------------
 3 files changed, 19 insertions(+), 47 deletions(-)

I won't be able to apply this to my tree, with

warning: Patch sent with format=flowed; space at the end of lines might be lost.
error: corrupt patch at line 22

until these get fixed, but is this meant to apply on top of 47c6d4da (test-lib: fix GIT_TEST_SANITIZE_LEAK_LOG, 2024-06-30) which is cooking in 'next'?

Thanks.

Quoted 132 lines
ci/lib.sh +0 -1
index 814578ffc6..51f8f59a29 100755
@@ -370,7 +370,6 @@ linux-musl)
 linux-leaks|linux-reftable-leaks)
 	export SANITIZE=leak
 	export GIT_TEST_PASSING_SANITIZE_LEAK=true
-	export GIT_TEST_SANITIZE_LEAK_LOG=true
 	;;
 linux-asan-ubsan)
 	export SANITIZE=address,undefined
t/README +0 -0
index d9e0e07506..c2a732d59e 100644
@@ -382,33 +382,9 @@ mapping between "TEST_PASSES_SANITIZE_LEAK=true"

and those tests that
pass under "SANITIZE=leak". This is especially useful when testing a series that fixes various memory leaks with "git rebase -x".

t/README +0 -24
-GIT_TEST_SANITIZE_LEAK_LOG=true will log memory leaks to
-"test-results/$TEST_NAME.leak/trace.*" files. The logs include a
-"dedup_token" (see +"ASAN_OPTIONS=help=1 ./git") and other options to
-make logs +machine-readable.
-
-With GIT_TEST_SANITIZE_LEAK_LOG=true we'll look at the leak logs
-before exiting and exit on failure if the logs showed that we had a
-memory leak, even if the test itself would have otherwise passed. This
-allows us to catch e.g. missing &&-chaining. This is especially useful
-when combined with "GIT_TEST_PASSING_SANITIZE_LEAK", see below.
-
 GIT_TEST_PASSING_SANITIZE_LEAK=check when combined with "--immediate"
 will run to completion faster, and result in the same failing
 tests. The only practical reason to run
-GIT_TEST_PASSING_SANITIZE_LEAK=check without "--immediate" is to
-combine it with "GIT_TEST_SANITIZE_LEAK_LOG=true". If we stop at the
-first failing test case our leak logs won't show subsequent leaks we
-might have run into.
-
-GIT_TEST_PASSING_SANITIZE_LEAK=(true|check) will not catch all memory
-leaks unless combined with GIT_TEST_SANITIZE_LEAK_LOG=true. Some tests
-run "git" (or "test-tool" etc.) without properly checking the exit
-code, or git will invoke itself and fail to ferry the abort() exit
-code to the original caller. When the two modes are combined we'll
-look at the "test-results/$TEST_NAME.leak/trace.*" files at the end of
-the test run to see if had memory leaks which the test itself didn't
-catch.
 
 GIT_TEST_PROTOCOL_VERSION=<n>, when set, makes 'protocol.version'
 default to n.
t/test-lib.sh +2 -2
index 79d3e0e7d9..942828c55d 100644
@@ -1270,8 +1270,8 @@ check_test_results_san_file_ () {
 		say "As TEST_PASSES_SANITIZE_LEAK=true isn't set the
 		above leak is 'ok' with
 		GIT_TEST_PASSING_SANITIZE_LEAK=check" &&
 		invert_exit_code=t
 	else
-		say "With GIT_TEST_SANITIZE_LEAK_LOG=true our logs
                 revealed a memory leak, exit non-zero!" &&
-		invert_exit_code=t
+		say "Our logs revealed a leak!" &&
+		test "$test_failure" != 0 || invert_exit_code=t
t/test-lib.sh +15 -10
 	fi
 }
@@ -1555,28 +1555,28 @@ then
 		passes_sanitize_leak=t
 	fi
 
-	if test "$GIT_TEST_PASSING_SANITIZE_LEAK" = "check"
+	if test -z "$passes_sanitize_leak" &&
+	   ! test "$GIT_TEST_PASSING_SANITIZE_LEAK" = "check" &&
+	   test_bool_env GIT_TEST_PASSING_SANITIZE_LEAK false
 	then
-		sanitize_leak_check=t
-		if test -n "$invert_exit_code"
+		skip_all="skipping $this_test under GIT_TEST_PASSING_SANITIZE_LEAK=true"
+		test_done
+	else
+		if test "$GIT_TEST_PASSING_SANITIZE_LEAK" = "check"
 		then
-			BAIL_OUT "cannot use --invert-exit-code under
                         GIT_TEST_PASSING_SANITIZE_LEAK=check"
-		fi
+			sanitize_leak_check=t
+			if test -n "$invert_exit_code"
+			then
+				BAIL_OUT "cannot use
--invert-exit-code under GIT_TEST_PASSING_SANITIZE_LEAK=check"
+			fi
 
-		if test -z "$passes_sanitize_leak"
-		then
-			say "in GIT_TEST_PASSING_SANITIZE_LEAK=check
                         mode, setting --invert-exit-code for
                        TEST_PASSES_SANITIZE_LEAK != true"
-			invert_exit_code=t
+			if test -z "$passes_sanitize_leak"
+			then
+				say "in

GIT_TEST_PASSING_SANITIZE_LEAK=check mode, setting --invert-exit-code for TEST_PASSES_SANITIZE_LEAK != true"

t/test-lib.sh +2 -8
+				invert_exit_code=t
+			fi
 		fi
-	elif test -z "$passes_sanitize_leak" &&
-	     test_bool_env GIT_TEST_PASSING_SANITIZE_LEAK false
-	then
-		skip_all="skipping $this_test under GIT_TEST_PASSING_SANITIZE_LEAK=true"
-		test_done
-	fi
 
-	if test_bool_env GIT_TEST_SANITIZE_LEAK_LOG false
-	then
 		if ! mkdir -p "$TEST_RESULTS_SAN_DIR"
 		then
 			BAIL_OUT "cannot create $TEST_RESULTS_SAN_DIR"
@@ -1599,9 +1599,6 @@ elif test "$GIT_TEST_PASSING_SANITIZE_LEAK" =

"check" ||
test_bool_env GIT_TEST_PASSING_SANITIZE_LEAK false
then

t/test-lib.sh +0 -3
 	BAIL_OUT_ENV_NEEDS_SANITIZE_LEAK "GIT_TEST_PASSING_SANITIZE_LEAK=true"
-elif test_bool_env GIT_TEST_SANITIZE_LEAK_LOG false
-then
-	BAIL_OUT_ENV_NEEDS_SANITIZE_LEAK "GIT_TEST_SANITIZE_LEAK_LOG=true"
 fi
 
 if test "${GIT_TEST_CHAIN_LINT:-1}" != 0 &&
#5 lore raw
Quoted 5 lines · Junio C Hamano
Quoted 2 lines

I used, more or less, the text for the message that Junio suggested.

Hmph, the above doesn't look like any, but as a standalone patch it is very much readable.

Ok. I'll apply the simplification as a separate patch and then remove the reference to GIT_TEST_SANITIZE_LEAK_LOG.

Quoted 6 lines

I won't be able to apply this to my tree, with

warning: Patch sent with format=flowed; space at the end of lines might be lost.
error: corrupt patch at line 22

until these get fixed,

Ouch. My email client is set up incorrectly. I'll fix it. Sorry.

Quoted 3 lines

but is this meant to apply on top of 47c6d4da (test-lib: fix GIT_TEST_SANITIZE_LEAK_LOG, 2024-06-30) which is cooking in 'next'?

It is. I'll use "git format-patch --base=47c6d4da" to reroll.

However, will wait a bit for confirmation that this patch is heading in the expected direction.

Thanks.

#6 lore raw
In reply to Rubén Justo
Quoted 12 lines · Rubén Justo

As we currently describe in t/README, it can happen that:

Some tests run "git" (or "test-tool" etc.) without properly checking the exit code, or git will invoke itself and fail to ferry the abort() exit code to the original caller.

Therefore, GIT_TEST_SANITIZE_LEAK_LOG=true is needed to be set to capture all memory leaks triggered by our tests.

It seems unnecessary to force users to remember this option, as forgetting it could lead to missed memory leaks.

Quoted 7 lines

We could solve the problem by setting GIT_TEST_SANITIZE_LEAK_LOG to "true" by default, but that might suggest we think "false" makes sense, which isn't the case.

Therefore, the best approach is to remove the option entirely while maintaining the capability to detect memory leaks in blind spots of our tests.

Yeah, I think that reasoning makes sense.

ci/lib.sh linux-musl) · line 373

Quoted 10 lines
ci/lib.sh +0 -1
index 814578ffc6..51f8f59a29 100755
@@ -370,7 +370,6 @@ linux-musl)
 linux-leaks|linux-reftable-leaks)
 	export SANITIZE=leak
 	export GIT_TEST_PASSING_SANITIZE_LEAK=true
-	export GIT_TEST_SANITIZE_LEAK_LOG=true
 	;;

OK, we can drop this line snice it's now the default. Good.

Quoted 26 lines
t/README +0 -0
index d9e0e07506..c2a732d59e 100644
@@ -382,33 +382,9 @@ mapping between "TEST_PASSES_SANITIZE_LEAK=true" and

those tests that
pass under "SANITIZE=leak". This is especially useful when testing a series that fixes various memory leaks with "git rebase -x".

t/README +0 -14
-GIT_TEST_SANITIZE_LEAK_LOG=true will log memory leaks to
-"test-results/$TEST_NAME.leak/trace.*" files. The logs include a
-"dedup_token" (see +"ASAN_OPTIONS=help=1 ./git") and other options to
-make logs +machine-readable.
-
-With GIT_TEST_SANITIZE_LEAK_LOG=true we'll look at the leak logs
-before exiting and exit on failure if the logs showed that we had a
-memory leak, even if the test itself would have otherwise passed. This
-allows us to catch e.g. missing &&-chaining. This is especially useful
-when combined with "GIT_TEST_PASSING_SANITIZE_LEAK", see below.
-
 GIT_TEST_PASSING_SANITIZE_LEAK=check when combined with "--immediate"
 will run to completion faster, and result in the same failing
 tests. The only practical reason to run
-GIT_TEST_PASSING_SANITIZE_LEAK=check without "--immediate" is to
-combine it with "GIT_TEST_SANITIZE_LEAK_LOG=true". If we stop at the
-first failing test case our leak logs won't show subsequent leaks we

t/README mapping between "TEST_PASSES_SANITIZE_LEAK=true" and those tests that · line 411

Quoted 10 lines
t/README +0 -10
-might have run into.
-
-GIT_TEST_PASSING_SANITIZE_LEAK=(true|check) will not catch all memory
-leaks unless combined with GIT_TEST_SANITIZE_LEAK_LOG=true. Some tests
-run "git" (or "test-tool" etc.) without properly checking the exit
-code, or git will invoke itself and fail to ferry the abort() exit
-code to the original caller. When the two modes are combined we'll
-look at the "test-results/$TEST_NAME.leak/trace.*" files at the end of
-the test run to see if had memory leaks which the test itself didn't
-catch.

After this patch, the documentation seems to end abruptly with "The only practical reason to run". I think we need to either delete those lines, too, or complete the thought.

I do think they are saying something useful, which is: in "check" mode, you should always use "--immediate" since the point is just to find scripts which aren't labeled correctly. But I think that is true whether you are using the leak log or not. Your log will be incomplete, of course, if you used "--immediate", but the point is to see whether we find even one.

Quoted 13 lines

GIT_TEST_PROTOCOL_VERSION=<n>, when set, makes 'protocol.version' default to n.

t/test-lib.sh +0 -0
index 79d3e0e7d9..942828c55d 100644
@@ -1270,8 +1270,8 @@ check_test_results_san_file_ () {
 		say "As TEST_PASSES_SANITIZE_LEAK=true isn't set the above leak is 'ok'

with GIT_TEST_PASSING_SANITIZE_LEAK=check" &&

t/test-lib.sh +0 -1
 		invert_exit_code=t
 	else
-		say "With GIT_TEST_SANITIZE_LEAK_LOG=true our logs revealed a memory

leak, exit non-zero!" &&

t/test-lib.sh

Quoted 5 lines
t/test-lib.sh +2 -1
-		invert_exit_code=t
+		say "Our logs revealed a leak!" &&
+		test "$test_failure" != 0 || invert_exit_code=t
 	fi
 }

This adds back in the test_failure fix from 47c6d4dad2 (test-lib: fix GIT_TEST_SANITIZE_LEAK_LOG, 2024-06-30), but in a different way. I think we'd want to build on top, and then you just need to update the messages on either side of that final elif/else.

Quoted 7 lines
t/test-lib.sh +2 -1
@@ -1555,28 +1555,28 @@ then
 		passes_sanitize_leak=t
 	fi
 
-	if test "$GIT_TEST_PASSING_SANITIZE_LEAK" = "check"
+	if test -z "$passes_sanitize_leak" &&
+	   ! test "$GIT_TEST_PASSING_SANITIZE_LEAK" = "check" &&

t/test-lib.sh then · line 1566

Quoted 10 lines
t/test-lib.sh +5 -2
+	   test_bool_env GIT_TEST_PASSING_SANITIZE_LEAK false
 	then
-		sanitize_leak_check=t
-		if test -n "$invert_exit_code"
+		skip_all="skipping $this_test under GIT_TEST_PASSING_SANITIZE_LEAK=true"
+		test_done
+	else
+		if test "$GIT_TEST_PASSING_SANITIZE_LEAK" = "check"
 		then
[...]

I'm not sure why we need to touch this block. The "if GIT_TEST_SANITIZE_LEAK_LOG" just below it I assumed would go away. But all of this has to do with "check" versus "true", etc? There might be new refactoring / simplification opportunities opened up by getting rid of the LEAK_LOG variable, but we should do those on top.

I guess what's happening is that you've rearranged it so that:

t/test-lib.sh

Quoted 5 lines
t/test-lib.sh +0 -2
-	if test_bool_env GIT_TEST_SANITIZE_LEAK_LOG false
-	then
 		if ! mkdir -p "$TEST_RESULTS_SAN_DIR"
 		then
 			BAIL_OUT "cannot create $TEST_RESULTS_SAN_DIR"

...when this conditional goes away, the existing body is still in the "else". But even though it would make the diff noisy to reindent, I think we are better off doing so to make it clear what the actual change is.

t/test-lib.sh

Quoted 9 lines
t/test-lib.sh +0 -0
@@ -1599,9 +1599,6 @@ elif test "$GIT_TEST_PASSING_SANITIZE_LEAK" = "check"

||
test_bool_env GIT_TEST_PASSING_SANITIZE_LEAK false
then

t/test-lib.sh +0 -3
 	BAIL_OUT_ENV_NEEDS_SANITIZE_LEAK "GIT_TEST_PASSING_SANITIZE_LEAK=true"
-elif test_bool_env GIT_TEST_SANITIZE_LEAK_LOG false
-then
-	BAIL_OUT_ENV_NEEDS_SANITIZE_LEAK "GIT_TEST_SANITIZE_LEAK_LOG=true"
 fi

OK, this final elif is responsible for complaining when you set LEAK_LOG but don't have an actual leak-checking build. But once it goes away, there's no need to complain. Makes sense.

-Peff

#7 lore raw
Quoted 69 lines · Jeff King
Quoted 12 lines · Rubén Justo

As we currently describe in t/README, it can happen that:

Some tests run "git" (or "test-tool" etc.) without properly checking the exit code, or git will invoke itself and fail to ferry the abort() exit code to the original caller.

Therefore, GIT_TEST_SANITIZE_LEAK_LOG=true is needed to be set to capture all memory leaks triggered by our tests.

It seems unnecessary to force users to remember this option, as forgetting it could lead to missed memory leaks.

Quoted 7 lines

We could solve the problem by setting GIT_TEST_SANITIZE_LEAK_LOG to "true" by default, but that might suggest we think "false" makes sense, which isn't the case.

Therefore, the best approach is to remove the option entirely while maintaining the capability to detect memory leaks in blind spots of our tests.

Yeah, I think that reasoning makes sense.

ci/lib.sh linux-musl) · line 373

Quoted 10 lines
ci/lib.sh +0 -1
index 814578ffc6..51f8f59a29 100755
@@ -370,7 +370,6 @@ linux-musl)
 linux-leaks|linux-reftable-leaks)
 	export SANITIZE=leak
 	export GIT_TEST_PASSING_SANITIZE_LEAK=true
-	export GIT_TEST_SANITIZE_LEAK_LOG=true
 	;;

OK, we can drop this line snice it's now the default. Good.

Quoted 32 lines
t/README +0 -0
index d9e0e07506..c2a732d59e 100644
@@ -382,33 +382,9 @@ mapping between "TEST_PASSES_SANITIZE_LEAK=true" and

those tests that
pass under "SANITIZE=leak". This is especially useful when testing a series that fixes various memory leaks with "git rebase -x".

t/README +0 -20
-GIT_TEST_SANITIZE_LEAK_LOG=true will log memory leaks to
-"test-results/$TEST_NAME.leak/trace.*" files. The logs include a
-"dedup_token" (see +"ASAN_OPTIONS=help=1 ./git") and other options to
-make logs +machine-readable.
-
-With GIT_TEST_SANITIZE_LEAK_LOG=true we'll look at the leak logs
-before exiting and exit on failure if the logs showed that we had a
-memory leak, even if the test itself would have otherwise passed. This
-allows us to catch e.g. missing &&-chaining. This is especially useful
-when combined with "GIT_TEST_PASSING_SANITIZE_LEAK", see below.
-
 GIT_TEST_PASSING_SANITIZE_LEAK=check when combined with "--immediate"
 will run to completion faster, and result in the same failing
 tests. The only practical reason to run
-GIT_TEST_PASSING_SANITIZE_LEAK=check without "--immediate" is to
-combine it with "GIT_TEST_SANITIZE_LEAK_LOG=true". If we stop at the
-first failing test case our leak logs won't show subsequent leaks we
-might have run into.
-
-GIT_TEST_PASSING_SANITIZE_LEAK=(true|check) will not catch all memory
-leaks unless combined with GIT_TEST_SANITIZE_LEAK_LOG=true. Some tests
-run "git" (or "test-tool" etc.) without properly checking the exit
-code, or git will invoke itself and fail to ferry the abort() exit
Quoted 8 lines

t/README mapping between "TEST_PASSES_SANITIZE_LEAK=true" and those tests that · line 411

Quoted 4 lines
t/README +0 -4
-code to the original caller. When the two modes are combined we'll
-look at the "test-results/$TEST_NAME.leak/trace.*" files at the end of
-the test run to see if had memory leaks which the test itself didn't
-catch.

After this patch, the documentation seems to end abruptly with "The only practical reason to run". I think we need to either delete those lines, too, or complete the thought.

I accidentally left that line unfinished. I'll fix it.

Quoted 23 lines

I do think they are saying something useful, which is: in "check" mode, you should always use "--immediate" since the point is just to find scripts which aren't labeled correctly. But I think that is true whether you are using the leak log or not. Your log will be incomplete, of course, if you used "--immediate", but the point is to see whether we find even one.

Quoted 15 lines

GIT_TEST_PROTOCOL_VERSION=<n>, when set, makes 'protocol.version' default to n.

t/test-lib.sh +0 -0
index 79d3e0e7d9..942828c55d 100644
@@ -1270,8 +1270,8 @@ check_test_results_san_file_ () {
 		say "As TEST_PASSES_SANITIZE_LEAK=true isn't set the above leak is 'ok'

with GIT_TEST_PASSING_SANITIZE_LEAK=check" &&

t/test-lib.sh +0 -1
 		invert_exit_code=t
 	else
-		say "With GIT_TEST_SANITIZE_LEAK_LOG=true our logs revealed a memory

leak, exit non-zero!" &&

t/test-lib.sh +1 -1
-		invert_exit_code=t
+		say "Our logs revealed a leak!" &&
Quoted 8 lines

t/test-lib.sh

Quoted 3 lines
t/test-lib.sh +1 -0
+		test "$test_failure" != 0 || invert_exit_code=t
 	fi
 }

This adds back in the test_failure fix from 47c6d4dad2 (test-lib: fix GIT_TEST_SANITIZE_LEAK_LOG, 2024-06-30), but in a different way. I think we'd want to build on top, and then you just need to update the messages on either side of that final elif/else.

OK. I think simplifying those lines introduced unnecessary noise. I'll discard it and just adjust the messages.

Quoted 27 lines
Quoted 8 lines
t/test-lib.sh +2 -1
@@ -1555,28 +1555,28 @@ then
 		passes_sanitize_leak=t
 	fi
 
-	if test "$GIT_TEST_PASSING_SANITIZE_LEAK" = "check"
+	if test -z "$passes_sanitize_leak" &&
+	   ! test "$GIT_TEST_PASSING_SANITIZE_LEAK" = "check" &&

t/test-lib.sh then · line 1566

Quoted 10 lines
t/test-lib.sh +5 -2
+	   test_bool_env GIT_TEST_PASSING_SANITIZE_LEAK false
 	then
-		sanitize_leak_check=t
-		if test -n "$invert_exit_code"
+		skip_all="skipping $this_test under GIT_TEST_PASSING_SANITIZE_LEAK=true"
+		test_done
+	else
+		if test "$GIT_TEST_PASSING_SANITIZE_LEAK" = "check"
 		then
[...]

I'm not sure why we need to touch this block. The "if GIT_TEST_SANITIZE_LEAK_LOG" just below it I assumed would go away. But all of this has to do with "check" versus "true", etc? There might be new refactoring / simplification opportunities opened up by getting rid of the LEAK_LOG variable, but we should do those on top.

I guess what's happening is that you've rearranged it so that:

Quoted 10 lines

t/test-lib.sh

Quoted 5 lines
t/test-lib.sh +0 -2
-	if test_bool_env GIT_TEST_SANITIZE_LEAK_LOG false
-	then
 		if ! mkdir -p "$TEST_RESULTS_SAN_DIR"
 		then
 			BAIL_OUT "cannot create $TEST_RESULTS_SAN_DIR"

...when this conditional goes away, the existing body is still in the "else". But even though it would make the diff noisy to reindent, I think we are better off doing so to make it clear what the actual change is.

OK.

Quoted 15 lines

t/test-lib.sh

Quoted 10 lines
t/test-lib.sh +0 -0
@@ -1599,9 +1599,6 @@ elif test "$GIT_TEST_PASSING_SANITIZE_LEAK" = "check"

||
test_bool_env GIT_TEST_PASSING_SANITIZE_LEAK false
then

t/test-lib.sh +0 -3
 	BAIL_OUT_ENV_NEEDS_SANITIZE_LEAK "GIT_TEST_PASSING_SANITIZE_LEAK=true"
-elif test_bool_env GIT_TEST_SANITIZE_LEAK_LOG false
-then
-	BAIL_OUT_ENV_NEEDS_SANITIZE_LEAK "GIT_TEST_SANITIZE_LEAK_LOG=true"
 fi

OK, this final elif is responsible for complaining when you set LEAK_LOG but don't have an actual leak-checking build. But once it goes away, there's no need to complain. Makes sense.

Thanks for reviewing the patch.

#8 lore raw

PATCH v3 1/1 test-lib: GIT_TEST_SANITIZE_LEAK_LOG enabled by default

In reply to Rubén Justo

As we currently describe in t/README, it can happen that:

Some tests run "git" (or "test-tool" etc.) without properly checking the exit code, or git will invoke itself and fail to ferry the abort() exit code to the original caller.

Therefore, GIT_TEST_SANITIZE_LEAK_LOG=true is needed to be set to capture all memory leaks triggered by our tests.

It seems unnecessary to force users to remember this option, as forgetting it could lead to missed memory leaks.

We could solve the problem by making it "true" by default, but that might suggest we think "false" makes sense, which isn't the case.

Therefore, the best approach is to remove the option entirely while maintaining the capability to detect memory leaks in blind spots of our tests.

---

This iteration mainly reduces the noise introduced in the previous iteration.

 ci/lib.sh     |  1 -
 t/README      | 26 +-------------------------
 t/test-lib.sh | 37 ++++++++++++++++---------------------
 3 files changed, 17 insertions(+), 47 deletions(-)
ci/lib.sh +0 -1
index ff66ad356b..fe52954828 100755
@@ -374,7 +374,6 @@ linux-musl)
 linux-leaks|linux-reftable-leaks)
 	export SANITIZE=leak
 	export GIT_TEST_PASSING_SANITIZE_LEAK=true
-	export GIT_TEST_SANITIZE_LEAK_LOG=true
 	;;
 linux-asan-ubsan)
 	export SANITIZE=address,undefined
t/README +1 -25
index d9e0e07506..ea620de17e 100644
@@ -382,33 +382,9 @@ mapping between "TEST_PASSES_SANITIZE_LEAK=true" and those tests that
 pass under "SANITIZE=leak". This is especially useful when testing a
 series that fixes various memory leaks with "git rebase -x".
 
-GIT_TEST_SANITIZE_LEAK_LOG=true will log memory leaks to
-"test-results/$TEST_NAME.leak/trace.*" files. The logs include a
-"dedup_token" (see +"ASAN_OPTIONS=help=1 ./git") and other options to
-make logs +machine-readable.
-
-With GIT_TEST_SANITIZE_LEAK_LOG=true we'll look at the leak logs
-before exiting and exit on failure if the logs showed that we had a
-memory leak, even if the test itself would have otherwise passed. This
-allows us to catch e.g. missing &&-chaining. This is especially useful
-when combined with "GIT_TEST_PASSING_SANITIZE_LEAK", see below.
-
 GIT_TEST_PASSING_SANITIZE_LEAK=check when combined with "--immediate"
 will run to completion faster, and result in the same failing
-tests. The only practical reason to run
-GIT_TEST_PASSING_SANITIZE_LEAK=check without "--immediate" is to
-combine it with "GIT_TEST_SANITIZE_LEAK_LOG=true". If we stop at the
-first failing test case our leak logs won't show subsequent leaks we
-might have run into.
-
-GIT_TEST_PASSING_SANITIZE_LEAK=(true|check) will not catch all memory
-leaks unless combined with GIT_TEST_SANITIZE_LEAK_LOG=true. Some tests
-run "git" (or "test-tool" etc.) without properly checking the exit
-code, or git will invoke itself and fail to ferry the abort() exit
-code to the original caller. When the two modes are combined we'll
-look at the "test-results/$TEST_NAME.leak/trace.*" files at the end of
-the test run to see if had memory leaks which the test itself didn't
-catch.
+tests.
 
 GIT_TEST_PROTOCOL_VERSION=<n>, when set, makes 'protocol.version'
 default to n.
t/test-lib.sh +16 -21
index 7ed6d3fc47..54247604cb 100644
@@ -1271,10 +1271,10 @@ check_test_results_san_file_ () {
 		invert_exit_code=t
 	elif test "$test_failure" = 0
 	then
-		say "With GIT_TEST_SANITIZE_LEAK_LOG=true our logs revealed a memory leak, exit non-zero!" &&
+		say "Our logs revealed a memory leak, exit non-zero!" &&
 		invert_exit_code=t
 	else
-		say "With GIT_TEST_SANITIZE_LEAK_LOG=true our logs revealed a memory leak..."
+		say "Our logs revealed a memory leak..."
 	fi
 }
 
@@ -1578,33 +1578,28 @@ then
 		test_done
 	fi
 
-	if test_bool_env GIT_TEST_SANITIZE_LEAK_LOG false
+	if ! mkdir -p "$TEST_RESULTS_SAN_DIR"
 	then
-		if ! mkdir -p "$TEST_RESULTS_SAN_DIR"
-		then
-			BAIL_OUT "cannot create $TEST_RESULTS_SAN_DIR"
-		fi &&
-		TEST_RESULTS_SAN_FILE="$TEST_RESULTS_SAN_DIR/$TEST_RESULTS_SAN_FILE_PFX"
+		BAIL_OUT "cannot create $TEST_RESULTS_SAN_DIR"
+	fi &&
+	TEST_RESULTS_SAN_FILE="$TEST_RESULTS_SAN_DIR/$TEST_RESULTS_SAN_FILE_PFX"
 
-		# In case "test-results" is left over from a previous
-		# run: Only report if new leaks show up.
-		TEST_RESULTS_SAN_DIR_NR_LEAKS_STARTUP=$(nr_san_dir_leaks_)
+	# In case "test-results" is left over from a previous
+	# run: Only report if new leaks show up.
+	TEST_RESULTS_SAN_DIR_NR_LEAKS_STARTUP=$(nr_san_dir_leaks_)
 
-		# Don't litter *.leak dirs if there was nothing to report
-		test_atexit "rmdir \"$TEST_RESULTS_SAN_DIR\" 2>/dev/null || :"
+	# Don't litter *.leak dirs if there was nothing to report
+	test_atexit "rmdir \"$TEST_RESULTS_SAN_DIR\" 2>/dev/null || :"
+
+	prepend_var LSAN_OPTIONS : dedup_token_length=9999
+	prepend_var LSAN_OPTIONS : log_exe_name=1
+	prepend_var LSAN_OPTIONS : log_path=\"$TEST_RESULTS_SAN_FILE\"
+	export LSAN_OPTIONS
 
-		prepend_var LSAN_OPTIONS : dedup_token_length=9999
-		prepend_var LSAN_OPTIONS : log_exe_name=1
-		prepend_var LSAN_OPTIONS : log_path=\"$TEST_RESULTS_SAN_FILE\"
-		export LSAN_OPTIONS
-	fi
 elif test "$GIT_TEST_PASSING_SANITIZE_LEAK" = "check" ||
      test_bool_env GIT_TEST_PASSING_SANITIZE_LEAK false
 then
 	BAIL_OUT_ENV_NEEDS_SANITIZE_LEAK "GIT_TEST_PASSING_SANITIZE_LEAK=true"
-elif test_bool_env GIT_TEST_SANITIZE_LEAK_LOG false
-then
-	BAIL_OUT_ENV_NEEDS_SANITIZE_LEAK "GIT_TEST_SANITIZE_LEAK_LOG=true"
 fi
 
 if test "${GIT_TEST_CHAIN_LINT:-1}" != 0 &&

base-commit: 47c6d4dad22a751068a4975f1c4177cc6c0c41d2

Signature · 1 line
2.45.1
#9 lore raw
Quoted 19 lines · Rubén Justo

As we currently describe in t/README, it can happen that:

Some tests run "git" (or "test-tool" etc.) without properly checking the exit code, or git will invoke itself and fail to ferry the abort() exit code to the original caller.

Therefore, GIT_TEST_SANITIZE_LEAK_LOG=true is needed to be set to capture all memory leaks triggered by our tests.

It seems unnecessary to force users to remember this option, as forgetting it could lead to missed memory leaks.

We could solve the problem by making it "true" by default, but that might suggest we think "false" makes sense, which isn't the case.

Therefore, the best approach is to remove the option entirely while maintaining the capability to detect memory leaks in blind spots of our tests.

Quoted 5 lines

---

This iteration mainly reduces the noise introduced in the previous iteration.

Thanks, this one looks good to me.

-Peff