Missing file in 2.23 (p5302-pack-index.subtests)?
I was trying to run "make profile" on the master branch (commit 5fa0f5238b: "Git 2.23") and it died in the
$(MAKE) PROFILE=GEN perf
dies with:
cannot open test-results/p5302-pack-index.subtests: No such file or directory at ./aggregate.perl line 153.
I presume that's becuase the file t/perf/test-results/p5302-pack-index.subtests is missing?
- Ted
Quoted 10 lines · Theodore Y. Ts'o
I was trying to run "make profile" on the master branch (commit
5fa0f5238b: "Git 2.23") and it died in the$(MAKE) PROFILE=GEN perfdies with:
cannot open test-results/p5302-pack-index.subtests: No such file or directory at ./aggregate.perl line 153.
I presume that's becuase the file t/perf/test-results/p5302-pack-index.subtests is missing?
That file should be created by running p5302. If you do:
cd t/perf
rm -rf test-results
./run p5302-pack-index.sh
is it generated?
If so, then perhaps something removed it while the perf tests were running. Did you run perhaps run "make clean" simultaneously?
-Peff
Quoted 7 lines · Jeff King
Quoted 5 lines · Theodore Y. Ts'o
I was trying to run "make profile" on the master branch (commit
5fa0f5238b: "Git 2.23") and it died in the$(MAKE) PROFILE=GEN perf
Quoted 7 lines
Quoted 5 lines
dies with:
cannot open test-results/p5302-pack-index.subtests: No such file or directory at ./aggregate.perl line 153.
I presume that's becuase the file t/perf/test-results/p5302-pack-index.subtests is missing?
That file should be created by running p5302. If you do:
By the way, I tried "make perf" as you specified above, and it worked fine for me. And I don't think there are any changes in v2.23 that should touch this area. But certainly it would be interesting to know if it fails consistently for you, and if so, whether older versions work (so we can bisect).
-Peff
Quoted 13 lines · Jeff King
Quoted 10 lines · Theodore Y. Ts'o
I was trying to run "make profile" on the master branch (commit
5fa0f5238b: "Git 2.23") and it died in the$(MAKE) PROFILE=GEN perfdies with:
cannot open test-results/p5302-pack-index.subtests: No such file or directory at ./aggregate.perl line 153.
I presume that's becuase the file t/perf/test-results/p5302-pack-index.subtests is missing?
Quoted 7 lines
That file should be created by running p5302. If you do:
cd t/perf
rm -rf test-results
./runp5302-pack-index.shis it generated?
No, it wasn't. See below....
- Ted
<tytso@lambda> {/usr/projects/git/git-core/t/perf} (master)
1140% git show
commit 5fa0f5238b0cd46cfe7f6fa76c3f526ea98148d9 (HEAD -> master, tag: v2.23.0, origin/master, origin/HEAD)
Author: Junio C Hamano <[email protected]> Date: Fri Aug 16 10:28:23 2019 -0700
Git 2.23
GIT-VERSION-GEN +1 -1
@@ -1,7 +1,7 @@ #!/bin/sh GVF=GIT-VERSION-FILE -DEF_VER=v2.23.0-rc2 +DEF_VER=v2.23.0 LF=' '
<tytso@lambda> {/usr/projects/git/git-core/t/perf} (master)
1141% rm -rf test-results ; ./run p5302-pack-index.sh
=== Running 1 tests in this tree ===
warning: $GIT_PERF_LARGE_REPO is $GIT_BUILD_DIR.
warning: This will work, but may not be a sufficiently large repo
warning: for representative measurements.
not ok 1 - repack
#
# git repack -ad &&
# PACK=$(ls .git/objects/pack/*.pack | head -n1) &&
# test -f "$PACK" &&
# export PACK
#
cannot open test-results/p5302-pack-index.subtests: No such file or directory at ./aggregate.perl line 153.
<tytso@lambda> {/usr/projects/git/git-core/t/perf} (master)
1142% git status
On branch master
Your branch is up to date with 'origin/master'.
Untracked files:
(use "git add <file>..." to include in what will be committed)
../../MAKELOG ../../MAKELOG.1 ../../common-cmds.h ../../git-lost-found ../../git-peek-remote ../../git-rebase--am ../../git-rebase--common ../../git-rebase--helper ../../git-rebase--interactive ../../git-rebase--merge ../../git-relink ../../git-remote-testgit ../../git-repo-config ../../git-tar-tree ../../git_remote_helpers/ ../../perl/MYMETA.json ../../perl/MYMETA.yml ../../perl/PM.stamp ../../perl/blib/ ../../perl/perl.mak ../../perl/pm_to_blib ../../test-chmtime ../../test-config ../../test-credential ../../test-ctype ../../test-date ../../test-delta ../../test-dump-cache-tree ../../test-dump-split-index ../../test-dump-untracked-cache ../../test-fake-ssh ../../test-genrandom ../../test-hashmap ../../test-index-version ../../test-line-buffer ../../test-match-trees ../../test-mergesort ../../test-mktemp ../../test-obj-pool ../../test-parse-options ../../test-path-utils ../../test-prio-queue ../../test-read-cache ../../test-regex ../../test-revision-walking ../../test-run-command ../../test-scrap-cache-tree ../../test-sha1 ../../test-sha1-array ../../test-sigchain ../../test-string-list ../../test-string-pool ../../test-submodule-config ../../test-subprocess ../../test-svn-fe ../../test-treap ../../test-urlmatch-normalization ../../test-wildmatch
nothing added to commit but untracked files present (use "git add" to track)
Quoted 11 lines · Theodore Y. Ts'o
Quoted 9 lines
Quoted 1 line
I presume that's becuase the file t/perf/test-results/p5302-pack-index.subtests is missing?
That file should be created by running p5302. If you do:
cd t/perf
rm -rf test-results
./runp5302-pack-index.shis it generated?
No, it wasn't. See below....
Weird. The output here:
Quoted 5 lines
1141% rm -rf test-results ; ./run
p5302-pack-index.sh=== Running 1 tests in this tree ===
warning: $GIT_PERF_LARGE_REPO is $GIT_BUILD_DIR.
warning: This will work, but may not be a sufficiently large repo
warning: for representative measurements.
Quoted 8 lines
not ok 1 - repack
#
# git repack -ad &&
# PACK=$(ls .git/objects/pack/*.pack | head -n1) &&
# test -f "$PACK" &&
# export PACK
#
cannot open test-results/p5302-pack-index.subtests: No such file or directory at ./aggregate.perl line 153.
Implies that we're trying to _write_ to it, and that the problem is that test-results doesn't exist. That should be set up by this part of perf-lib:
perf_results_dir=$TEST_OUTPUT_DIRECTORY/test-results test -n "$GIT_PERF_SUBSECTION" && perf_results_dir="$perf_results_dir/$GIT_PERF_SUBSECTION" mkdir -p "$perf_results_dir" rm -f "$perf_results_dir"/$(basename "$0" .sh).subtests
I don't know why that would be failing for you (and not producing any kind of message!). You might need to poke at:
./p5302-pack-index.sh -x
to see what is going on.
It's also weird that the repack fails for you. And that (presumably?) this is the only script that fails, since they would all want to do this.
There are some bits of test-lib.sh that re-exec the script, but I think we ironed out the weirdness there (and I use "--verbose-log" myself, which is one such option). But just in case: do you set GIT_TEST_OPTS in your config.mak?
-Peff
Quoted 5 lines · Jeff King
Quoted 1 line
cannot open test-results/p5302-pack-index.subtests: No such file or directory at ./aggregate.perl line 153.
Implies that we're trying to _write_ to it, and that the problem is that test-results doesn't exist. That should be set up by this part of perf-lib:
Hmm.... test-results does exist after the failure:
<tytso@lambda> {/usr/projects/git/git-core/t/perf} (master)
1174% rm -rf test-results ; ./run p5302-pack-index.sh
=== Running 1 tests in this tree ===
warning: $GIT_PERF_LARGE_REPO is $GIT_BUILD_DIR.
warning: This will work, but may not be a sufficiently large repo
warning: for representative measurements.
not ok 1 - repack
#
# git repack -ad &&
# PACK=$(ls .git/objects/pack/*.pack | head -n1) &&
# test -f "$PACK" &&
# export PACK
#
cannot open test-results/p5302-pack-index.subtests: No such file or directory at ./aggregate.perl line 153.
<tytso@lambda> {/usr/projects/git/git-core/t/perf} (master)
1175% ls test-results
total 0
0 run_subsections.names
Maybe we're not in the right cwd for some reason?!?
Quoted 4 lines
There are some bits of
test-lib.shthat re-exec the script, but I think we ironed out the weirdness there (and I use "--verbose-log" myself, which is one such option). But just in case: do you setGIT_TEST_OPTSin your config.mak?
You mean in my top-level directory? I don't have a config.mak file in there:
<tytso@lambda> {/usr/projects/git/git-core} (master)
1151% grep GIT_TEST_OPTS config.mak.*
<tytso@lambda> {/usr/projects/git/git-core} (master)
1181% ls config.mak*
4 config.mak.autogen 4 config.mak.dev 4 config.mak.in 24 config.mak.uname
<tytso@lambda> {/usr/projects/git/git-core} (master)
1182% grep GIT_TEST_OPTS config.mak.*
- Ted
Quoted 8 lines · Theodore Y. Ts'o
Quoted 5 lines · Jeff King
Quoted 1 line
cannot open test-results/p5302-pack-index.subtests: No such file or directory at ./aggregate.perl line 153.
Implies that we're trying to _write_ to it, and that the problem is that test-results doesn't exist. That should be set up by this part of perf-lib:
Hmm.... test-results does exist after the failure:
Ah, never mind. I was thinking it was coming from this line in perf-lib:
echo "$test_count" >>"$perf_results_dir"/$base.subtests
But it it is pretty clear from this line of your output:
Quoted 1 line
cannot open test-results/p5302-pack-index.subtests: No such file or directory at ./aggregate.perl line 153.
...that this is the aggregate script that is run afterwards complaining. So for whatever reason, the actual p5302 script is exiting early, before it writes anything into the subtests file (from the line above).
That subtests write is done in test_wrapper_(), which is only triggered for actual timing tests, not for setup tests (like the repack step that starts this script). So the plausible sequence of events here is:
1. The first "test_expect_success repack" test fails for some reason. Try running "./p5302-pack-index -v -i -x" to see more output.
2. After the initial failure, the script exits totally rather than continue. That's due to this line in perf-lib, which acts as if "-i" is always set:
# Performance tests should never fail. If they do, stop immediately immediate=t
That makes sense, since any timings we do after a setup step fails would invalidate the result. And if it's not the _first_ such step that fails, we'll just have a truncated subtests file, and some timings will be missing. But when the first one fails, there's no file at all.
We could probably leave a more consistent state in that case. E.g., something like this:
t/perf/perf-lib.sh +1 -1
@@ -48,7 +48,7 @@ export MODERN_GIT perf_results_dir=$TEST_OUTPUT_DIRECTORY/test-results test -n "$GIT_PERF_SUBSECTION" && perf_results_dir="$perf_results_dir/$GIT_PERF_SUBSECTION" mkdir -p "$perf_results_dir" -rm -f "$perf_results_dir"/$(basename "$0" .sh).subtests +>"$perf_results_dir"/$(basename "$0" .sh).subtests die_if_build_dir_not_repo () { if ! ( cd "$TEST_DIRECTORY/.." &&
which would just quietly omit the rest of p5302. It is unfortunate that you wouldn't get some note in the output saying "hey, we didn't run some tests!".
A lot of this has to do with the hackish way that the list of tests is generated. If you do something like:
./run HEAD^ HEAD p5302-pack-index.sh
and the tests fail in HEAD^ but not HEAD, you'll get a nice table listing all of the tests, with a nil field for each one that didn't run in HEAD^.
But that's because the second run, on HEAD, actually generated a correct p5302-pack-index.subtests file, that actually mentioned the tests! If the order were reversed, you'd get an empty list.
So I think in the long run, it would be nice to have some way of generating the list that's more robust to failures. But I suspect doing that is going to be hard; a lot of this is rooted in the fact that there's no data structure with the set of tests, but literally an executable shell script that decides what to run.
So short answer: use "-v" to figure out why the repack is failing, and the rest is just Git's perf suite being a bit hacky.
-Peff