t3419: prevent failure when run with EXPENSIVE
PATCH 1/1 t3419: prevent failure when run with EXPENSIVE
This test runs a function which itself runs several assertions. The last of these assertions cleans up the .git/rebase-apply directory, since when run with EXPENSIVE set, the function is invoked a second time to run the same tests with a larger data set.
However, as of 2ac0d6273f ("rebase: change the default backend from "am" to "merge"", 2020-02-15), the default backend of rebase has changed, and cleaning up the rebase-apply directory has no effect: it no longer exists, since we're using rebase-merge instead.
Since we don't really care which rebase backend is in use, let's just clean up both, which means we'll do the right thing in every case.
---
I noticed this when I was working on another series and running the testsuite with GIT_TEST_LONG=1. We'll probably want to add this before the release if possible.
It may also be desirable to have at least once CI run that runs this way. In my experience, it does not take substantially longer to run the testsuite on a modern Linux system with this option enabled.
t/t3419-rebase-patch-id.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
t/t3419-rebase-patch-id.sh +1 -1
@@ -91,7 +91,7 @@ do_tests () { git commit -q -m squashed && git checkout -q other^{} && test_must_fail git rebase squashed && - rm -rf .git/rebase-apply + rm -rf .git/rebase-merge .git/rebase-apply ' }
Quoted 32 lines · brian m. carlson
This test runs a function which itself runs several assertions. The last of these assertions cleans up the .git/rebase-apply directory, since when run with EXPENSIVE set, the function is invoked a second time to run the same tests with a larger data set.
However, as of
2ac0d6273f("rebase: change the default backend from "am" to "merge"", 2020-02-15), the default backend of rebase has changed, and cleaning up the rebase-apply directory has no effect: it no longer exists, since we're using rebase-merge instead.Since we don't really care which rebase backend is in use, let's just clean up both, which means we'll do the right thing in every case.
Signed-off-by brian m. carlson <[email protected]>---
I noticed this when I was working on another series and running the testsuite with GIT_TEST_LONG=1. We'll probably want to add this before the release if possible.It may also be desirable to have at least once CI run that runs this way. In my experience, it does not take substantially longer to run the testsuite on a modern Linux system with this option enabled.
t/t3419-rebase-patch-id.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)t/t3419-rebase-patch-id.sh +0
-0
t/t3419-rebase-patch-id.sh do_tests () { · line 96
Quoted 7 lines
t/t3419-rebase-patch-id.sh +1
-1@@ -91,7 +91,7 @@ do_tests () { git commit -q -m squashed && git checkout -q other^{} && test_must_fail git rebase squashed && - rm -rf .git/rebase-apply + rm -rf .git/rebase-merge .git/rebase-apply ' }
Good catch, thanks. Perhaps we just want to invoke 'git rebase --quit' and let it clean up instead of manually doing so ourselves, since it may buy us some future-proofing in case we ever want to move the place we store rebase state?
Quoted 4 lines · Elijah Newren
Good catch, thanks. Perhaps we just want to invoke 'git rebase --quit' and let it clean up instead of manually doing so ourselves, since it may buy us some future-proofing in case we ever want to move the place we store rebase state?
Sure, I can do that. I think there are other people (e.g., zsh) relying on these locations for the state of the various rebase versions, but nevertheless it seems like a good idea to use our existing cleanup tools.
I'll send a v2 shortly.
Signature · 2 lines
brian m. carlson: Houston, Texas, US OpenPGP: https://keybase.io/bk2204
Quoted 9 lines · Elijah Newren
t/t3419-rebase-patch-id.sh do_tests () { · line 96
Quoted 4 lines
t/t3419-rebase-patch-id.sh +1
-1- rm -rf .git/rebase-apply + rm -rf .git/rebase-merge .git/rebase-apply ' }Good catch, thanks. Perhaps we just want to invoke 'git rebase --quit' and let it clean up instead of manually doing so ourselves, since it may buy us some future-proofing in case we ever want to move the place we store rebase state?
I started writing "I would agree 100% if this script were not about testing 'rebase', but using 'rebase' itself for a framework to test 'rebase'???" but then if 'rebase --quit' fails to clear these, what we are likely to see is that the next test to fail, so it probably is an OK approach.
Thanks.
PATCH v2 1/1 t3419: prevent failure when run with EXPENSIVE
In reply to brian m. carlsonThis test runs a function which itself runs several assertions. The last of these assertions cleans up the .git/rebase-apply directory, since when run with EXPENSIVE set, the function is invoked a second time to run the same tests with a larger data set.
However, as of 2ac0d6273f ("rebase: change the default backend from "am" to "merge"", 2020-02-15), the default backend of rebase has changed, and cleaning up the rebase-apply directory has no effect: it no longer exists, since we're using rebase-merge instead.
Since we don't really care which rebase backend is in use, let's just use the command "git rebase --quit", which will do the right thing regardless.
--- t/t3419-rebase-patch-id.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
t/t3419-rebase-patch-id.sh +1 -1
@@ -91,7 +91,7 @@ do_tests () { git commit -q -m squashed && git checkout -q other^{} && test_must_fail git rebase squashed && - rm -rf .git/rebase-apply + git rebase --quit ' }
PATCH 1/1 t3419: drop EXPENSIVE tests
In reply to brian m. carlsonQuoted 7 lines · brian m. carlson
I noticed this when I was working on another series and running the testsuite with GIT_TEST_LONG=1. We'll probably want to add this before the release if possible.
It may also be desirable to have at least once CI run that runs this way. In my experience, it does not take substantially longer to run the testsuite on a modern Linux system with this option enabled.
I do agree that anything marked with EXPENSIVE is practically useless if nobody is running it. In many cases these can be made much less expensive by being smarter about setup (e.g., using fast-import or test_commit_bulk to create commits rather than looping and creating many processes).
This one in particular seems to add about 500ms, which isn't too bad. But I actually don't think it's doing anything useful. See below.
-- >8 --
Subject: [PATCH] t3419: drop EXPENSIVE tests
When t3419 was originally written, it was designed to run a smaller test for correctness, and then the same test with a larger number of patches for performance. But it seems unlikely the latter was helping us:
- it was marked with EXPENSIVE, so hardly anybody ran it anyway
- there's no indication that it was more likely to find bugs than the smaller case (the commit message isn't very helpful, but the original cover letter describes it as: "The first patch adds correctness and (optional) performance tests".
- the timing results are shown only via test_debug(). So also not run unless the user says "-d", and then not provided in any machine-readable form.
If we're interested in performance regressions, a script in t/perf would be more appropriate. I didn't add one here, because it's not at all clear to me that what the script is timing is even all that interesting.
Let's simplify the script by dropping the EXPENSIVE run. That in turn lets us drop the do_tests() wrapper, which lets us consistently use single-quotes for our test snippets. And we can drop the useless test_debug() timings, as well as their run() helper. And finally, while we're here, we can replace the count() helper with the standard test_seq().
---
Prepared on top of en/rebase-backend, since it textually conflicts otherwise. Best viewed with "diff -w" because of the re-indenting.
t/t3419-rebase-patch-id.sh | 112 ++++++++++++++----------------------- 1 file changed, 41 insertions(+), 71 deletions(-)
t/t3419-rebase-patch-id.sh +41 -71
@@ -4,15 +4,6 @@ test_description='git rebase - test patch id computation' . ./test-lib.sh -count () { - i=0 - while test $i -lt $1 - do - echo "$i" - i=$(($i+1)) - done -} - scramble () { i=0 while read x
@@ -26,75 +17,54 @@ scramble () { mv -f "$1.new" "$1" } -run () { - echo \$ "$@" - /usr/bin/time "$@" >/dev/null -} - test_expect_success 'setup' ' git commit --allow-empty -m initial && git tag root ' -do_tests () { - nlines=$1 pr=${2-} - - test_expect_success $pr "setup: $nlines lines" " - rm -f .gitattributes && - git checkout -q -f master && - git reset --hard root && - count $nlines >file && - git add file && - git commit -q -m initial && - git branch -f other && - - scramble file && - git add file && - git commit -q -m 'change big file' && - - git checkout -q other && - : >newfile && - git add newfile && - git commit -q -m 'add small file' && - - git cherry-pick master >/dev/null 2>&1 - " - - test_debug " - run git diff master^\! - " - - test_expect_success $pr 'setup attributes' " - echo 'file binary' >.gitattributes - " - - test_debug " - run git format-patch --stdout master && - run git format-patch --stdout --ignore-if-in-upstream master - " +test_expect_success 'setup: 500 lines' ' + rm -f .gitattributes && + git checkout -q -f master && + git reset --hard root && + test_seq 500 >file && + git add file && + git commit -q -m initial && + git branch -f other && + + scramble file && + git add file && + git commit -q -m "change big file" && + + git checkout -q other && + : >newfile && + git add newfile && + git commit -q -m "add small file" && + + git cherry-pick master >/dev/null 2>&1 +' - test_expect_success $pr 'detect upstream patch' ' - git checkout -q master && - scramble file && - git add file && - git commit -q -m "change big file again" && - git checkout -q other^{} && - git rebase master && - test_must_fail test -n "$(git rev-list master...HEAD~)" - ' +test_expect_success 'setup attributes' ' + echo "file binary" >.gitattributes +' - test_expect_success $pr 'do not drop patch' ' - git branch -f squashed master && - git checkout -q -f squashed && - git reset -q --soft HEAD~2 && - git commit -q -m squashed && - git checkout -q other^{} && - test_must_fail git rebase squashed && - git rebase --quit - ' -} +test_expect_success 'detect upstream patch' ' + git checkout -q master && + scramble file && + git add file && + git commit -q -m "change big file again" && + git checkout -q other^{} && + git rebase master && + test_must_fail test -n "$(git rev-list master...HEAD~)" +' -do_tests 500 -do_tests 50000 EXPENSIVE +test_expect_success 'do not drop patch' ' + git branch -f squashed master && + git checkout -q -f squashed && + git reset -q --soft HEAD~2 && + git commit -q -m squashed && + git checkout -q other^{} && + test_must_fail git rebase squashed && + git rebase --quit +' test_done
Signature · 1 line
2.26.0.rc2.641.gac63ba406a
Re: [PATCH] t3419: drop EXPENSIVE tests
Quoted 19 lines · Jeff King
When t3419 was originally written, it was designed to run a smaller test for correctness, and then the same test with a larger number of patches for performance. But it seems unlikely the latter was helping us:
- it was marked with EXPENSIVE, so hardly anybody ran it anyway
- there's no indication that it was more likely to find bugs than the smaller case (the commit message isn't very helpful, but the original cover letter describes it as: "The first patch adds correctness and (optional) performance tests".
- the timing results are shown only via
test_debug(). So also not run unless the user says "-d", and then not provided in any machine-readable form.If we're interested in performance regressions, a script in t/perf would be more appropriate. I didn't add one here, because it's not at all clear to me that what the script is timing is even all that interesting.
Quoted 6 lines
Let's simplify the script by dropping the EXPENSIVE run. That in turn lets us drop the
do_tests()wrapper, which lets us consistently use single-quotes for our test snippets. And we can drop the uselesstest_debug()timings, as well as theirrun()helper. And finally, while we're here, we can replace thecount()helper with the standardtest_seq().
I'm also fine with this solution. As long as this test doesn't fail with EXPENSIVE, I'm happy.
Signature · 2 lines
brian m. carlson: Houston, Texas, US OpenPGP: https://keybase.io/bk2204
Re: [PATCH] t3419: drop EXPENSIVE tests
Quoted 9 lines · brian m. carlson
Quoted 6 lines
Let's simplify the script by dropping the EXPENSIVE run. That in turn lets us drop the
do_tests()wrapper, which lets us consistently use single-quotes for our test snippets. And we can drop the uselesstest_debug()timings, as well as theirrun()helper. And finally, while we're here, we can replace thecount()helper with the standardtest_seq().I'm also fine with this solution. As long as this test doesn't fail with EXPENSIVE, I'm happy.
Just to be clear, this is meant to go on top of your fix, which is already in next, so I think that part is fixed either way. :)
-Peff