test-lib: GIT_TEST_SANITIZE_LEAK_LOG enabled by default
PATCH 1/1 test-lib: GIT_TEST_SANITIZE_LEAK_LOG enabled by default
As we describe in t/README, it can happen that:
Some tests run "git" (or "test-tool" etc.) without properly checking the exit code, or git will invoke itself and fail to ferry the abort() exit code to the original caller.
Therefore, GIT_TEST_SANITIZE_LEAK_LOG must be set to true to capture all memory leaks triggered by the tests when SANITIZE=leak.
Set it to true by default, and stop worrying about someone checking for leaks who isn't aware of this option and might be missing some leaks.
--- ci/lib.sh | 1 - t/README | 4 ++-- t/test-lib.sh | 2 +- 3 files changed, 3 insertions(+), 4 deletions(-)
ci/lib.sh +0 -1
@@ -374,7 +374,6 @@ linux-musl) linux-leaks|linux-reftable-leaks) export SANITIZE=leak export GIT_TEST_PASSING_SANITIZE_LEAK=true - export GIT_TEST_SANITIZE_LEAK_LOG=true ;; linux-asan-ubsan) export SANITIZE=address,undefined
t/README +2 -2
@@ -382,10 +382,10 @@ mapping between "TEST_PASSES_SANITIZE_LEAK=true" and those tests that pass under "SANITIZE=leak". This is especially useful when testing a series that fixes various memory leaks with "git rebase -x". -GIT_TEST_SANITIZE_LEAK_LOG=true will log memory leaks to +GIT_TEST_SANITIZE_LEAK_LOG=<boolean> controls logging of memory leaks to "test-results/$TEST_NAME.leak/trace.*" files. The logs include a "dedup_token" (see +"ASAN_OPTIONS=help=1 ./git") and other options to -make logs +machine-readable. +make logs +machine-readable. Defaults to "true" when SANITIZE=leak. With GIT_TEST_SANITIZE_LEAK_LOG=true we'll look at the leak logs before exiting and exit on failure if the logs showed that we had a
t/test-lib.sh +1 -1
@@ -1578,7 +1578,7 @@ then test_done fi - if test_bool_env GIT_TEST_SANITIZE_LEAK_LOG false + if test_bool_env GIT_TEST_SANITIZE_LEAK_LOG true then if ! mkdir -p "$TEST_RESULTS_SAN_DIR" then
Signature · 1 line
2.45.1
Quoted 11 lines · Rubén Justo
As we describe in t/README, it can happen that:
Some tests run "git" (or "test-tool" etc.) without properly checking the exit code, or git will invoke itself and fail to ferry the
abort()exit code to the original caller.Therefore,
GIT_TEST_SANITIZE_LEAK_LOGmust be set to true to capture all memory leaks triggered by the tests when SANITIZE=leak.Set it to true by default, and stop worrying about someone checking for leaks who isn't aware of this option and might be missing some leaks.
I'm obviously in favor of this direction, but...why stop here? Do we expect somebody to set it to false? If not, then can't we just get rid of it entirely?
-Peff
Quoted 11 lines · Jeff King
Quoted 9 lines · Rubén Justo
As we describe in t/README, it can happen that:
Some tests run "git" (or "test-tool" etc.) without properly checking the exit code, or git will invoke itself and fail to ferry the
abort()exit code to the original caller.Therefore,
GIT_TEST_SANITIZE_LEAK_LOGmust be set to true to capture all memory leaks triggered by the tests when SANITIZE=leak.
Quoted 6 lines
Quoted 2 lines
Set it to true by default, and stop worrying about someone checking for leaks who isn't aware of this option and might be missing some leaks.
I'm obviously in favor of this direction, but...why stop here? Do we expect somebody to set it to false? If not, then can't we just get rid of it entirely?
I'd also be strongly in favor of just removing this variable altogether. I found it quite tedious to remember setting it when working on the memory leak fixes recently, and I'm not aware of any downsides.
Patrick
PATCH v2 1/1 test-lib: GIT_TEST_SANITIZE_LEAK_LOG enabled by default
In reply to Rubén JustoAs we currently describe in t/README, it can happen that:
Some tests run "git" (or "test-tool" etc.) without properly checking the exit code, or git will invoke itself and fail to ferry the abort() exit code to the original caller.
Therefore, GIT_TEST_SANITIZE_LEAK_LOG=true is needed to be set to capture all memory leaks triggered by our tests.
It seems unnecessary to force users to remember this option, as forgetting it could lead to missed memory leaks.
We could solve the problem by setting GIT_TEST_SANITIZE_LEAK_LOG to "true" by default, but that might suggest we think "false" makes sense, which isn't the case.
Therefore, the best approach is to remove the option entirely while maintaining the capability to detect memory leaks in blind spots of our tests.
---
@Peff: I thought your preference was to keep the option. I agree to remove it completely. This, v2, address that.
I'm traveling, and while I think the change doesn't break anything, I'd appreciate a double check ;-)
By the way, I used, more or less, the text for the message that Junio suggested.
ci/lib.sh | 1 - t/README | 24 ------------------------ t/test-lib.sh | 41 +++++++++++++++++++---------------------- 3 files changed, 19 insertions(+), 47 deletions(-)
ci/lib.sh +0 -1
@@ -370,7 +370,6 @@ linux-musl) linux-leaks|linux-reftable-leaks) export SANITIZE=leak export GIT_TEST_PASSING_SANITIZE_LEAK=true - export GIT_TEST_SANITIZE_LEAK_LOG=true ;; linux-asan-ubsan) export SANITIZE=address,undefined
t/README +0 -24
@@ -382,33 +382,9 @@ mapping between "TEST_PASSES_SANITIZE_LEAK=true" and those tests that pass under "SANITIZE=leak". This is especially useful when testing a series that fixes various memory leaks with "git rebase -x". -GIT_TEST_SANITIZE_LEAK_LOG=true will log memory leaks to -"test-results/$TEST_NAME.leak/trace.*" files. The logs include a -"dedup_token" (see +"ASAN_OPTIONS=help=1 ./git") and other options to -make logs +machine-readable. - -With GIT_TEST_SANITIZE_LEAK_LOG=true we'll look at the leak logs -before exiting and exit on failure if the logs showed that we had a -memory leak, even if the test itself would have otherwise passed. This -allows us to catch e.g. missing &&-chaining. This is especially useful -when combined with "GIT_TEST_PASSING_SANITIZE_LEAK", see below. - GIT_TEST_PASSING_SANITIZE_LEAK=check when combined with "--immediate" will run to completion faster, and result in the same failing tests. The only practical reason to run -GIT_TEST_PASSING_SANITIZE_LEAK=check without "--immediate" is to -combine it with "GIT_TEST_SANITIZE_LEAK_LOG=true". If we stop at the -first failing test case our leak logs won't show subsequent leaks we -might have run into. - -GIT_TEST_PASSING_SANITIZE_LEAK=(true|check) will not catch all memory -leaks unless combined with GIT_TEST_SANITIZE_LEAK_LOG=true. Some tests -run "git" (or "test-tool" etc.) without properly checking the exit -code, or git will invoke itself and fail to ferry the abort() exit -code to the original caller. When the two modes are combined we'll -look at the "test-results/$TEST_NAME.leak/trace.*" files at the end of -the test run to see if had memory leaks which the test itself didn't -catch. GIT_TEST_PROTOCOL_VERSION=<n>, when set, makes 'protocol.version' default to n.
t/test-lib.sh +0 -0
@@ -1270,8 +1270,8 @@ check_test_results_san_file_ () { say "As TEST_PASSES_SANITIZE_LEAK=true isn't set the above leak is
'ok' with GIT_TEST_PASSING_SANITIZE_LEAK=check" &&
invert_exit_code=t
else
- say "With GIT_TEST_SANITIZE_LEAK_LOG=true our logs revealed a memory leak, exit non-zero!" &&
- invert_exit_code=t
+ say "Our logs revealed a leak!" &&
+ test "$test_failure" != 0 || invert_exit_code=t
fi
}
@@ -1555,28 +1555,28 @@ then
passes_sanitize_leak=t
fi
- if test "$GIT_TEST_PASSING_SANITIZE_LEAK" = "check"
+ if test -z "$passes_sanitize_leak" &&
+ ! test "$GIT_TEST_PASSING_SANITIZE_LEAK" = "check" &&
+ test_bool_env GIT_TEST_PASSING_SANITIZE_LEAK false
then
- sanitize_leak_check=t
- if test -n "$invert_exit_code"
+ skip_all="skipping $this_test under GIT_TEST_PASSING_SANITIZE_LEAK=true"
+ test_done
+ else
+ if test "$GIT_TEST_PASSING_SANITIZE_LEAK" = "check"
then
- BAIL_OUT "cannot use --invert-exit-code under GIT_TEST_PASSING_SANITIZE_LEAK=check"
- fi
+ sanitize_leak_check=t
+ if test -n "$invert_exit_code"
+ then
+ BAIL_OUT "cannot use --invert-exit-code under GIT_TEST_PASSING_SANITIZE_LEAK=check"
+ fi
- if test -z "$passes_sanitize_leak"
- then
- say "in GIT_TEST_PASSING_SANITIZE_LEAK=check mode, setting --invert-exit-code for TEST_PASSES_SANITIZE_LEAK != true"
- invert_exit_code=t
+ if test -z "$passes_sanitize_leak"
+ then
+ say "in GIT_TEST_PASSING_SANITIZE_LEAK=check mode, setting --invert-exit-code for TEST_PASSES_SANITIZE_LEAK != true"
+ invert_exit_code=t
+ fi
fi
- elif test -z "$passes_sanitize_leak" &&
- test_bool_env GIT_TEST_PASSING_SANITIZE_LEAK false
- then
- skip_all="skipping $this_test under GIT_TEST_PASSING_SANITIZE_LEAK=true"
- test_done
- fi
- if test_bool_env GIT_TEST_SANITIZE_LEAK_LOG false
- then
if ! mkdir -p "$TEST_RESULTS_SAN_DIR"
then
BAIL_OUT "cannot create $TEST_RESULTS_SAN_DIR"
@@ -1599,9 +1599,6 @@ elif test "$GIT_TEST_PASSING_SANITIZE_LEAK" = "check" ||
test_bool_env GIT_TEST_PASSING_SANITIZE_LEAK false
then
BAIL_OUT_ENV_NEEDS_SANITIZE_LEAK "GIT_TEST_PASSING_SANITIZE_LEAK=true"
-elif test_bool_env GIT_TEST_SANITIZE_LEAK_LOG false
-then
- BAIL_OUT_ENV_NEEDS_SANITIZE_LEAK "GIT_TEST_SANITIZE_LEAK_LOG=true"
fi
if test "${GIT_TEST_CHAIN_LINT:-1}" != 0 &&
Signature · 1 line
2.45.1
Quoted 23 lines · Rubén Justo
As we currently describe in t/README, it can happen that:
Some tests run "git" (or "test-tool" etc.) without properly checking the exit code, or git will invoke itself and fail to ferry the
abort()exit code to the original caller.Therefore, GIT_TEST_SANITIZE_LEAK_LOG=true is needed to be set to capture all memory leaks triggered by our tests.
It seems unnecessary to force users to remember this option, as forgetting it could lead to missed memory leaks.
We could solve the problem by setting
GIT_TEST_SANITIZE_LEAK_LOGto "true" by default, but that might suggest we think "false" makes sense, which isn't the case.Therefore, the best approach is to remove the option entirely while maintaining the capability to detect memory leaks in blind spots of our tests.
Signed-off-by Rubén Justo <[email protected]>---
Quoted 8 lines
@Peff: I thought your preference was to keep the option. I agree to remove it completely. This, v2, address that.
I'm traveling, and while I think the change doesn't break anything, I'd appreciate a double check ;-)
By the way, I used, more or less, the text for the message that Junio suggested.
Hmph, the above doesn't look like any, but as a standalone patch it is very much readable.
Quoted 4 lines
ci/lib.sh | 1 - t/README | 24 ------------------------ t/test-lib.sh | 41 +++++++++++++++++++---------------------- 3 files changed, 19 insertions(+), 47 deletions(-)
I won't be able to apply this to my tree, with
warning: Patch sent with format=flowed; space at the end of lines might be lost.
error: corrupt patch at line 22
until these get fixed, but is this meant to apply on top of 47c6d4da (test-lib: fix GIT_TEST_SANITIZE_LEAK_LOG, 2024-06-30) which is cooking in 'next'?
Thanks.
Quoted 132 lines
ci/lib.sh +0
-1@@ -370,7 +370,6 @@ linux-musl) linux-leaks|linux-reftable-leaks) export SANITIZE=leak export GIT_TEST_PASSING_SANITIZE_LEAK=true - export GIT_TEST_SANITIZE_LEAK_LOG=true ;; linux-asan-ubsan) export SANITIZE=address,undefinedt/README +0
-0@@ -382,33 +382,9 @@ mapping between "TEST_PASSES_SANITIZE_LEAK=true"and those tests that
pass under "SANITIZE=leak". This is especially useful when testing a series that fixes various memory leaks with "git rebase -x".t/README +0
-24-GIT_TEST_SANITIZE_LEAK_LOG=true will log memory leaks to -"test-results/$TEST_NAME.leak/trace.*" files. The logs include a -"dedup_token" (see +"ASAN_OPTIONS=help=1 ./git") and other options to -make logs +machine-readable. - -With GIT_TEST_SANITIZE_LEAK_LOG=true we'll look at the leak logs -before exiting and exit on failure if the logs showed that we had a -memory leak, even if the test itself would have otherwise passed. This -allows us to catch e.g. missing &&-chaining. This is especially useful -when combined with "GIT_TEST_PASSING_SANITIZE_LEAK", see below. - GIT_TEST_PASSING_SANITIZE_LEAK=check when combined with "--immediate" will run to completion faster, and result in the same failing tests. The only practical reason to run -GIT_TEST_PASSING_SANITIZE_LEAK=check without "--immediate" is to -combine it with "GIT_TEST_SANITIZE_LEAK_LOG=true". If we stop at the -first failing test case our leak logs won't show subsequent leaks we -might have run into. - -GIT_TEST_PASSING_SANITIZE_LEAK=(true|check) will not catch all memory -leaks unless combined with GIT_TEST_SANITIZE_LEAK_LOG=true. Some tests -run "git" (or "test-tool" etc.) without properly checking the exit -code, or git will invoke itself and fail to ferry the abort() exit -code to the original caller. When the two modes are combined we'll -look at the "test-results/$TEST_NAME.leak/trace.*" files at the end of -the test run to see if had memory leaks which the test itself didn't -catch. GIT_TEST_PROTOCOL_VERSION=<n>, when set, makes 'protocol.version' default to n.t/test-lib.sh +2
-2@@ -1270,8 +1270,8 @@ check_test_results_san_file_ () { say "As TEST_PASSES_SANITIZE_LEAK=true isn't set the above leak is 'ok' with GIT_TEST_PASSING_SANITIZE_LEAK=check" && invert_exit_code=t else - say "With GIT_TEST_SANITIZE_LEAK_LOG=true our logs revealed a memory leak, exit non-zero!" && - invert_exit_code=t + say "Our logs revealed a leak!" && + test "$test_failure" != 0 || invert_exit_code=tt/test-lib.sh +15
-10fi }@@ -1555,28 +1555,28 @@ then passes_sanitize_leak=t fi - if test "$GIT_TEST_PASSING_SANITIZE_LEAK" = "check" + if test -z "$passes_sanitize_leak" && + ! test "$GIT_TEST_PASSING_SANITIZE_LEAK" = "check" && + test_bool_env GIT_TEST_PASSING_SANITIZE_LEAK false then - sanitize_leak_check=t - if test -n "$invert_exit_code" + skip_all="skipping $this_test under GIT_TEST_PASSING_SANITIZE_LEAK=true" + test_done + else + if test "$GIT_TEST_PASSING_SANITIZE_LEAK" = "check" then - BAIL_OUT "cannot use --invert-exit-code under GIT_TEST_PASSING_SANITIZE_LEAK=check" - fi + sanitize_leak_check=t + if test -n "$invert_exit_code" + then + BAIL_OUT "cannot use --invert-exit-code under GIT_TEST_PASSING_SANITIZE_LEAK=check" + fi - if test -z "$passes_sanitize_leak" - then - say "in GIT_TEST_PASSING_SANITIZE_LEAK=check mode, setting --invert-exit-code for TEST_PASSES_SANITIZE_LEAK != true" - invert_exit_code=t + if test -z "$passes_sanitize_leak" + then + say "inGIT_TEST_PASSING_SANITIZE_LEAK=check mode, setting --invert-exit-code for
TEST_PASSES_SANITIZE_LEAK!= true"t/test-lib.sh +2
-8+ invert_exit_code=t + fi fi - elif test -z "$passes_sanitize_leak" && - test_bool_env GIT_TEST_PASSING_SANITIZE_LEAK false - then - skip_all="skipping $this_test under GIT_TEST_PASSING_SANITIZE_LEAK=true" - test_done - fi - if test_bool_env GIT_TEST_SANITIZE_LEAK_LOG false - then if ! mkdir -p "$TEST_RESULTS_SAN_DIR" then BAIL_OUT "cannot create $TEST_RESULTS_SAN_DIR"@@ -1599,9 +1599,6 @@ elif test "$GIT_TEST_PASSING_SANITIZE_LEAK" ="check" ||
test_bool_envGIT_TEST_PASSING_SANITIZE_LEAKfalse
thent/test-lib.sh +0
-3BAIL_OUT_ENV_NEEDS_SANITIZE_LEAK "GIT_TEST_PASSING_SANITIZE_LEAK=true" -elif test_bool_env GIT_TEST_SANITIZE_LEAK_LOG false -then - BAIL_OUT_ENV_NEEDS_SANITIZE_LEAK "GIT_TEST_SANITIZE_LEAK_LOG=true" fi if test "${GIT_TEST_CHAIN_LINT:-1}" != 0 &&
Quoted 5 lines · Junio C Hamano
Quoted 2 lines
I used, more or less, the text for the message that Junio suggested.
Hmph, the above doesn't look like any, but as a standalone patch it is very much readable.
Ok. I'll apply the simplification as a separate patch and then remove the reference to GIT_TEST_SANITIZE_LEAK_LOG.
Quoted 6 lines
I won't be able to apply this to my tree, with
warning: Patch sent with format=flowed; space at the end of lines might be lost.
error: corrupt patch at line 22until these get fixed,
Ouch. My email client is set up incorrectly. I'll fix it. Sorry.
Quoted 3 lines
but is this meant to apply on top of
47c6d4da(test-lib: fixGIT_TEST_SANITIZE_LEAK_LOG, 2024-06-30) which is cooking in 'next'?
It is. I'll use "git format-patch --base=47c6d4da" to reroll.
However, will wait a bit for confirmation that this patch is heading in the expected direction.
Thanks.
Quoted 12 lines · Rubén Justo
As we currently describe in t/README, it can happen that:
Some tests run "git" (or "test-tool" etc.) without properly checking the exit code, or git will invoke itself and fail to ferry the
abort()exit code to the original caller.Therefore, GIT_TEST_SANITIZE_LEAK_LOG=true is needed to be set to capture all memory leaks triggered by our tests.
It seems unnecessary to force users to remember this option, as forgetting it could lead to missed memory leaks.
Quoted 7 lines
We could solve the problem by setting
GIT_TEST_SANITIZE_LEAK_LOGto "true" by default, but that might suggest we think "false" makes sense, which isn't the case.Therefore, the best approach is to remove the option entirely while maintaining the capability to detect memory leaks in blind spots of our tests.
Yeah, I think that reasoning makes sense.
ci/lib.sh linux-musl) · line 373
Quoted 10 lines
ci/lib.sh +0
-1@@ -370,7 +370,6 @@ linux-musl) linux-leaks|linux-reftable-leaks) export SANITIZE=leak export GIT_TEST_PASSING_SANITIZE_LEAK=true - export GIT_TEST_SANITIZE_LEAK_LOG=true ;;
OK, we can drop this line snice it's now the default. Good.
Quoted 26 lines
t/README +0
-0@@ -382,33 +382,9 @@ mapping between "TEST_PASSES_SANITIZE_LEAK=true" andthose tests that
pass under "SANITIZE=leak". This is especially useful when testing a series that fixes various memory leaks with "git rebase -x".t/README +0
-14-GIT_TEST_SANITIZE_LEAK_LOG=true will log memory leaks to -"test-results/$TEST_NAME.leak/trace.*" files. The logs include a -"dedup_token" (see +"ASAN_OPTIONS=help=1 ./git") and other options to -make logs +machine-readable. - -With GIT_TEST_SANITIZE_LEAK_LOG=true we'll look at the leak logs -before exiting and exit on failure if the logs showed that we had a -memory leak, even if the test itself would have otherwise passed. This -allows us to catch e.g. missing &&-chaining. This is especially useful -when combined with "GIT_TEST_PASSING_SANITIZE_LEAK", see below. - GIT_TEST_PASSING_SANITIZE_LEAK=check when combined with "--immediate" will run to completion faster, and result in the same failing tests. The only practical reason to run -GIT_TEST_PASSING_SANITIZE_LEAK=check without "--immediate" is to -combine it with "GIT_TEST_SANITIZE_LEAK_LOG=true". If we stop at the -first failing test case our leak logs won't show subsequent leaks we
t/README mapping between "TEST_PASSES_SANITIZE_LEAK=true" and those tests that · line 411
Quoted 10 lines
t/README +0
-10-might have run into. - -GIT_TEST_PASSING_SANITIZE_LEAK=(true|check) will not catch all memory -leaks unless combined with GIT_TEST_SANITIZE_LEAK_LOG=true. Some tests -run "git" (or "test-tool" etc.) without properly checking the exit -code, or git will invoke itself and fail to ferry the abort() exit -code to the original caller. When the two modes are combined we'll -look at the "test-results/$TEST_NAME.leak/trace.*" files at the end of -the test run to see if had memory leaks which the test itself didn't -catch.
After this patch, the documentation seems to end abruptly with "The only practical reason to run". I think we need to either delete those lines, too, or complete the thought.
I do think they are saying something useful, which is: in "check" mode, you should always use "--immediate" since the point is just to find scripts which aren't labeled correctly. But I think that is true whether you are using the leak log or not. Your log will be incomplete, of course, if you used "--immediate", but the point is to see whether we find even one.
Quoted 13 lines
GIT_TEST_PROTOCOL_VERSION=<n>, when set, makes 'protocol.version' default to n.
t/test-lib.sh +0
-0@@ -1270,8 +1270,8 @@ check_test_results_san_file_ () { say "As TEST_PASSES_SANITIZE_LEAK=true isn't set the above leak is 'ok'with GIT_TEST_PASSING_SANITIZE_LEAK=check" &&
t/test-lib.sh +0
-1invert_exit_code=t else - say "With GIT_TEST_SANITIZE_LEAK_LOG=true our logs revealed a memoryleak, exit non-zero!" &&
t/test-lib.sh
Quoted 5 lines
t/test-lib.sh +2
-1- invert_exit_code=t + say "Our logs revealed a leak!" && + test "$test_failure" != 0 || invert_exit_code=t fi }
This adds back in the test_failure fix from 47c6d4dad2 (test-lib: fix GIT_TEST_SANITIZE_LEAK_LOG, 2024-06-30), but in a different way. I think we'd want to build on top, and then you just need to update the messages on either side of that final elif/else.
Quoted 7 lines
t/test-lib.sh +2
-1@@ -1555,28 +1555,28 @@ then passes_sanitize_leak=t fi - if test "$GIT_TEST_PASSING_SANITIZE_LEAK" = "check" + if test -z "$passes_sanitize_leak" && + ! test "$GIT_TEST_PASSING_SANITIZE_LEAK" = "check" &&
t/test-lib.sh then · line 1566
Quoted 10 lines
t/test-lib.sh +5
-2+ test_bool_env GIT_TEST_PASSING_SANITIZE_LEAK false then - sanitize_leak_check=t - if test -n "$invert_exit_code" + skip_all="skipping $this_test under GIT_TEST_PASSING_SANITIZE_LEAK=true" + test_done + else + if test "$GIT_TEST_PASSING_SANITIZE_LEAK" = "check" then
I'm not sure why we need to touch this block. The "if GIT_TEST_SANITIZE_LEAK_LOG" just below it I assumed would go away. But all of this has to do with "check" versus "true", etc? There might be new refactoring / simplification opportunities opened up by getting rid of the LEAK_LOG variable, but we should do those on top.
I guess what's happening is that you've rearranged it so that:
t/test-lib.sh
Quoted 5 lines
t/test-lib.sh +0
-2- if test_bool_env GIT_TEST_SANITIZE_LEAK_LOG false - then if ! mkdir -p "$TEST_RESULTS_SAN_DIR" then BAIL_OUT "cannot create $TEST_RESULTS_SAN_DIR"
...when this conditional goes away, the existing body is still in the "else". But even though it would make the diff noisy to reindent, I think we are better off doing so to make it clear what the actual change is.
t/test-lib.sh
Quoted 9 lines
t/test-lib.sh +0
-0@@ -1599,9 +1599,6 @@ elif test "$GIT_TEST_PASSING_SANITIZE_LEAK" = "check"||
test_bool_envGIT_TEST_PASSING_SANITIZE_LEAKfalse
thent/test-lib.sh +0
-3BAIL_OUT_ENV_NEEDS_SANITIZE_LEAK "GIT_TEST_PASSING_SANITIZE_LEAK=true" -elif test_bool_env GIT_TEST_SANITIZE_LEAK_LOG false -then - BAIL_OUT_ENV_NEEDS_SANITIZE_LEAK "GIT_TEST_SANITIZE_LEAK_LOG=true" fi
OK, this final elif is responsible for complaining when you set LEAK_LOG but don't have an actual leak-checking build. But once it goes away, there's no need to complain. Makes sense.
-Peff
Quoted 69 lines · Jeff King
Quoted 12 lines · Rubén Justo
As we currently describe in t/README, it can happen that:
Some tests run "git" (or "test-tool" etc.) without properly checking the exit code, or git will invoke itself and fail to ferry the
abort()exit code to the original caller.Therefore, GIT_TEST_SANITIZE_LEAK_LOG=true is needed to be set to capture all memory leaks triggered by our tests.
It seems unnecessary to force users to remember this option, as forgetting it could lead to missed memory leaks.
Quoted 7 lines
We could solve the problem by setting
GIT_TEST_SANITIZE_LEAK_LOGto "true" by default, but that might suggest we think "false" makes sense, which isn't the case.Therefore, the best approach is to remove the option entirely while maintaining the capability to detect memory leaks in blind spots of our tests.
Yeah, I think that reasoning makes sense.
ci/lib.sh linux-musl) · line 373
Quoted 10 lines
ci/lib.sh +0
-1@@ -370,7 +370,6 @@ linux-musl) linux-leaks|linux-reftable-leaks) export SANITIZE=leak export GIT_TEST_PASSING_SANITIZE_LEAK=true - export GIT_TEST_SANITIZE_LEAK_LOG=true ;;OK, we can drop this line snice it's now the default. Good.
Quoted 32 lines
t/README +0
-0@@ -382,33 +382,9 @@ mapping between "TEST_PASSES_SANITIZE_LEAK=true" andthose tests that
pass under "SANITIZE=leak". This is especially useful when testing a series that fixes various memory leaks with "git rebase -x".t/README +0
-20-GIT_TEST_SANITIZE_LEAK_LOG=true will log memory leaks to -"test-results/$TEST_NAME.leak/trace.*" files. The logs include a -"dedup_token" (see +"ASAN_OPTIONS=help=1 ./git") and other options to -make logs +machine-readable. - -With GIT_TEST_SANITIZE_LEAK_LOG=true we'll look at the leak logs -before exiting and exit on failure if the logs showed that we had a -memory leak, even if the test itself would have otherwise passed. This -allows us to catch e.g. missing &&-chaining. This is especially useful -when combined with "GIT_TEST_PASSING_SANITIZE_LEAK", see below. - GIT_TEST_PASSING_SANITIZE_LEAK=check when combined with "--immediate" will run to completion faster, and result in the same failing tests. The only practical reason to run -GIT_TEST_PASSING_SANITIZE_LEAK=check without "--immediate" is to -combine it with "GIT_TEST_SANITIZE_LEAK_LOG=true". If we stop at the -first failing test case our leak logs won't show subsequent leaks we -might have run into. - -GIT_TEST_PASSING_SANITIZE_LEAK=(true|check) will not catch all memory -leaks unless combined with GIT_TEST_SANITIZE_LEAK_LOG=true. Some tests -run "git" (or "test-tool" etc.) without properly checking the exit -code, or git will invoke itself and fail to ferry the abort() exit
Quoted 8 lines
t/README mapping between "TEST_PASSES_SANITIZE_LEAK=true" and those tests that · line 411
Quoted 4 lines
t/README +0
-4-code to the original caller. When the two modes are combined we'll -look at the "test-results/$TEST_NAME.leak/trace.*" files at the end of -the test run to see if had memory leaks which the test itself didn't -catch.After this patch, the documentation seems to end abruptly with "The only practical reason to run". I think we need to either delete those lines, too, or complete the thought.
I accidentally left that line unfinished. I'll fix it.
Quoted 23 lines
I do think they are saying something useful, which is: in "check" mode, you should always use "--immediate" since the point is just to find scripts which aren't labeled correctly. But I think that is true whether you are using the leak log or not. Your log will be incomplete, of course, if you used "--immediate", but the point is to see whether we find even one.
Quoted 15 lines
GIT_TEST_PROTOCOL_VERSION=<n>, when set, makes 'protocol.version' default to n.
t/test-lib.sh +0
-0@@ -1270,8 +1270,8 @@ check_test_results_san_file_ () { say "As TEST_PASSES_SANITIZE_LEAK=true isn't set the above leak is 'ok'with GIT_TEST_PASSING_SANITIZE_LEAK=check" &&
t/test-lib.sh +0
-1invert_exit_code=t else - say "With GIT_TEST_SANITIZE_LEAK_LOG=true our logs revealed a memoryleak, exit non-zero!" &&
t/test-lib.sh +1
-1- invert_exit_code=t + say "Our logs revealed a leak!" &&
Quoted 8 lines
t/test-lib.sh
Quoted 3 lines
t/test-lib.sh +1
-0+ test "$test_failure" != 0 || invert_exit_code=t fi }This adds back in the
test_failurefix from47c6d4dad2(test-lib: fixGIT_TEST_SANITIZE_LEAK_LOG, 2024-06-30), but in a different way. I think we'd want to build on top, and then you just need to update the messages on either side of that final elif/else.
OK. I think simplifying those lines introduced unnecessary noise. I'll discard it and just adjust the messages.
Quoted 27 lines
Quoted 8 lines
t/test-lib.sh +2
-1@@ -1555,28 +1555,28 @@ then passes_sanitize_leak=t fi - if test "$GIT_TEST_PASSING_SANITIZE_LEAK" = "check" + if test -z "$passes_sanitize_leak" && + ! test "$GIT_TEST_PASSING_SANITIZE_LEAK" = "check" &&t/test-lib.sh then · line 1566
Quoted 10 lines
t/test-lib.sh +5
-2+ test_bool_env GIT_TEST_PASSING_SANITIZE_LEAK false then - sanitize_leak_check=t - if test -n "$invert_exit_code" + skip_all="skipping $this_test under GIT_TEST_PASSING_SANITIZE_LEAK=true" + test_done + else + if test "$GIT_TEST_PASSING_SANITIZE_LEAK" = "check" thenI'm not sure why we need to touch this block. The "if
GIT_TEST_SANITIZE_LEAK_LOG" just below it I assumed would go away. But all of this has to do with "check" versus "true", etc? There might be new refactoring / simplification opportunities opened up by getting rid of theLEAK_LOGvariable, but we should do those on top.I guess what's happening is that you've rearranged it so that:
Quoted 10 lines
t/test-lib.sh
Quoted 5 lines
t/test-lib.sh +0
-2- if test_bool_env GIT_TEST_SANITIZE_LEAK_LOG false - then if ! mkdir -p "$TEST_RESULTS_SAN_DIR" then BAIL_OUT "cannot create $TEST_RESULTS_SAN_DIR"...when this conditional goes away, the existing body is still in the "else". But even though it would make the diff noisy to reindent, I think we are better off doing so to make it clear what the actual change is.
OK.
Quoted 15 lines
t/test-lib.sh
Quoted 10 lines
t/test-lib.sh +0
-0@@ -1599,9 +1599,6 @@ elif test "$GIT_TEST_PASSING_SANITIZE_LEAK" = "check"||
test_bool_envGIT_TEST_PASSING_SANITIZE_LEAKfalse
thent/test-lib.sh +0
-3BAIL_OUT_ENV_NEEDS_SANITIZE_LEAK "GIT_TEST_PASSING_SANITIZE_LEAK=true" -elif test_bool_env GIT_TEST_SANITIZE_LEAK_LOG false -then - BAIL_OUT_ENV_NEEDS_SANITIZE_LEAK "GIT_TEST_SANITIZE_LEAK_LOG=true" fiOK, this final elif is responsible for complaining when you set
LEAK_LOGbut don't have an actual leak-checking build. But once it goes away, there's no need to complain. Makes sense.
Thanks for reviewing the patch.
PATCH v3 1/1 test-lib: GIT_TEST_SANITIZE_LEAK_LOG enabled by default
In reply to Rubén JustoAs we currently describe in t/README, it can happen that:
Some tests run "git" (or "test-tool" etc.) without properly checking the exit code, or git will invoke itself and fail to ferry the abort() exit code to the original caller.
Therefore, GIT_TEST_SANITIZE_LEAK_LOG=true is needed to be set to capture all memory leaks triggered by our tests.
It seems unnecessary to force users to remember this option, as forgetting it could lead to missed memory leaks.
We could solve the problem by making it "true" by default, but that might suggest we think "false" makes sense, which isn't the case.
Therefore, the best approach is to remove the option entirely while maintaining the capability to detect memory leaks in blind spots of our tests.
---
This iteration mainly reduces the noise introduced in the previous iteration.
ci/lib.sh | 1 - t/README | 26 +------------------------- t/test-lib.sh | 37 ++++++++++++++++--------------------- 3 files changed, 17 insertions(+), 47 deletions(-)
ci/lib.sh +0 -1
@@ -374,7 +374,6 @@ linux-musl) linux-leaks|linux-reftable-leaks) export SANITIZE=leak export GIT_TEST_PASSING_SANITIZE_LEAK=true - export GIT_TEST_SANITIZE_LEAK_LOG=true ;; linux-asan-ubsan) export SANITIZE=address,undefined
t/README +1 -25
@@ -382,33 +382,9 @@ mapping between "TEST_PASSES_SANITIZE_LEAK=true" and those tests that pass under "SANITIZE=leak". This is especially useful when testing a series that fixes various memory leaks with "git rebase -x". -GIT_TEST_SANITIZE_LEAK_LOG=true will log memory leaks to -"test-results/$TEST_NAME.leak/trace.*" files. The logs include a -"dedup_token" (see +"ASAN_OPTIONS=help=1 ./git") and other options to -make logs +machine-readable. - -With GIT_TEST_SANITIZE_LEAK_LOG=true we'll look at the leak logs -before exiting and exit on failure if the logs showed that we had a -memory leak, even if the test itself would have otherwise passed. This -allows us to catch e.g. missing &&-chaining. This is especially useful -when combined with "GIT_TEST_PASSING_SANITIZE_LEAK", see below. - GIT_TEST_PASSING_SANITIZE_LEAK=check when combined with "--immediate" will run to completion faster, and result in the same failing -tests. The only practical reason to run -GIT_TEST_PASSING_SANITIZE_LEAK=check without "--immediate" is to -combine it with "GIT_TEST_SANITIZE_LEAK_LOG=true". If we stop at the -first failing test case our leak logs won't show subsequent leaks we -might have run into. - -GIT_TEST_PASSING_SANITIZE_LEAK=(true|check) will not catch all memory -leaks unless combined with GIT_TEST_SANITIZE_LEAK_LOG=true. Some tests -run "git" (or "test-tool" etc.) without properly checking the exit -code, or git will invoke itself and fail to ferry the abort() exit -code to the original caller. When the two modes are combined we'll -look at the "test-results/$TEST_NAME.leak/trace.*" files at the end of -the test run to see if had memory leaks which the test itself didn't -catch. +tests. GIT_TEST_PROTOCOL_VERSION=<n>, when set, makes 'protocol.version' default to n.
t/test-lib.sh +16 -21
@@ -1271,10 +1271,10 @@ check_test_results_san_file_ () { invert_exit_code=t elif test "$test_failure" = 0 then - say "With GIT_TEST_SANITIZE_LEAK_LOG=true our logs revealed a memory leak, exit non-zero!" && + say "Our logs revealed a memory leak, exit non-zero!" && invert_exit_code=t else - say "With GIT_TEST_SANITIZE_LEAK_LOG=true our logs revealed a memory leak..." + say "Our logs revealed a memory leak..." fi }
@@ -1578,33 +1578,28 @@ then test_done fi - if test_bool_env GIT_TEST_SANITIZE_LEAK_LOG false + if ! mkdir -p "$TEST_RESULTS_SAN_DIR" then - if ! mkdir -p "$TEST_RESULTS_SAN_DIR" - then - BAIL_OUT "cannot create $TEST_RESULTS_SAN_DIR" - fi && - TEST_RESULTS_SAN_FILE="$TEST_RESULTS_SAN_DIR/$TEST_RESULTS_SAN_FILE_PFX" + BAIL_OUT "cannot create $TEST_RESULTS_SAN_DIR" + fi && + TEST_RESULTS_SAN_FILE="$TEST_RESULTS_SAN_DIR/$TEST_RESULTS_SAN_FILE_PFX" - # In case "test-results" is left over from a previous - # run: Only report if new leaks show up. - TEST_RESULTS_SAN_DIR_NR_LEAKS_STARTUP=$(nr_san_dir_leaks_) + # In case "test-results" is left over from a previous + # run: Only report if new leaks show up. + TEST_RESULTS_SAN_DIR_NR_LEAKS_STARTUP=$(nr_san_dir_leaks_) - # Don't litter *.leak dirs if there was nothing to report - test_atexit "rmdir \"$TEST_RESULTS_SAN_DIR\" 2>/dev/null || :" + # Don't litter *.leak dirs if there was nothing to report + test_atexit "rmdir \"$TEST_RESULTS_SAN_DIR\" 2>/dev/null || :" + + prepend_var LSAN_OPTIONS : dedup_token_length=9999 + prepend_var LSAN_OPTIONS : log_exe_name=1 + prepend_var LSAN_OPTIONS : log_path=\"$TEST_RESULTS_SAN_FILE\" + export LSAN_OPTIONS - prepend_var LSAN_OPTIONS : dedup_token_length=9999 - prepend_var LSAN_OPTIONS : log_exe_name=1 - prepend_var LSAN_OPTIONS : log_path=\"$TEST_RESULTS_SAN_FILE\" - export LSAN_OPTIONS - fi elif test "$GIT_TEST_PASSING_SANITIZE_LEAK" = "check" || test_bool_env GIT_TEST_PASSING_SANITIZE_LEAK false then BAIL_OUT_ENV_NEEDS_SANITIZE_LEAK "GIT_TEST_PASSING_SANITIZE_LEAK=true" -elif test_bool_env GIT_TEST_SANITIZE_LEAK_LOG false -then - BAIL_OUT_ENV_NEEDS_SANITIZE_LEAK "GIT_TEST_SANITIZE_LEAK_LOG=true" fi if test "${GIT_TEST_CHAIN_LINT:-1}" != 0 &&
base-commit: 47c6d4dad22a751068a4975f1c4177cc6c0c41d2
Signature · 1 line
2.45.1
Quoted 19 lines · Rubén Justo
As we currently describe in t/README, it can happen that:
Some tests run "git" (or "test-tool" etc.) without properly checking the exit code, or git will invoke itself and fail to ferry the
abort()exit code to the original caller.Therefore, GIT_TEST_SANITIZE_LEAK_LOG=true is needed to be set to capture all memory leaks triggered by our tests.
It seems unnecessary to force users to remember this option, as forgetting it could lead to missed memory leaks.
We could solve the problem by making it "true" by default, but that might suggest we think "false" makes sense, which isn't the case.
Therefore, the best approach is to remove the option entirely while maintaining the capability to detect memory leaks in blind spots of our tests.
Quoted 5 lines
Signed-off-by Rubén Justo <[email protected]>---
This iteration mainly reduces the noise introduced in the previous iteration.
Thanks, this one looks good to me.
-Peff