Skip to contentSkip to conversation

Missing file in 2.23 (p5302-pack-index.subtests)?

DISCUSSION Theodore Y. Ts'o 2 participants 7 messages Aug 18, 2019 – Aug 27, 2019

lore
Share & tools
lore
#0 lore raw

I was trying to run "make profile" on the master branch (commit 5fa0f5238b: "Git 2.23") and it died in the

	$(MAKE) PROFILE=GEN perf

dies with:

cannot open test-results/p5302-pack-index.subtests: No such file or directory at ./aggregate.perl line 153.

I presume that's becuase the file t/perf/test-results/p5302-pack-index.subtests is missing?

- Ted

#1 lore raw
Quoted 10 lines · Theodore Y. Ts'o

I was trying to run "make profile" on the master branch (commit 5fa0f5238b: "Git 2.23") and it died in the

	$(MAKE) PROFILE=GEN perf

dies with:

cannot open test-results/p5302-pack-index.subtests: No such file or directory at ./aggregate.perl line 153.

I presume that's becuase the file t/perf/test-results/p5302-pack-index.subtests is missing?

That file should be created by running p5302. If you do:

cd t/perf
rm -rf test-results
./run p5302-pack-index.sh

is it generated?

If so, then perhaps something removed it while the perf tests were running. Did you run perhaps run "make clean" simultaneously?

-Peff

#2 lore raw
Quoted 7 lines · Jeff King
Quoted 5 lines · Theodore Y. Ts'o

I was trying to run "make profile" on the master branch (commit 5fa0f5238b: "Git 2.23") and it died in the

	$(MAKE) PROFILE=GEN perf
Quoted 7 lines
Quoted 5 lines

dies with:

cannot open test-results/p5302-pack-index.subtests: No such file or directory at ./aggregate.perl line 153.

I presume that's becuase the file t/perf/test-results/p5302-pack-index.subtests is missing?

That file should be created by running p5302. If you do:

By the way, I tried "make perf" as you specified above, and it worked fine for me. And I don't think there are any changes in v2.23 that should touch this area. But certainly it would be interesting to know if it fails consistently for you, and if so, whether older versions work (so we can bisect).

-Peff

#3 lore raw
In reply to Jeff King
Quoted 13 lines · Jeff King
Quoted 10 lines · Theodore Y. Ts'o

I was trying to run "make profile" on the master branch (commit 5fa0f5238b: "Git 2.23") and it died in the

	$(MAKE) PROFILE=GEN perf

dies with:

cannot open test-results/p5302-pack-index.subtests: No such file or directory at ./aggregate.perl line 153.

I presume that's becuase the file t/perf/test-results/p5302-pack-index.subtests is missing?

Quoted 7 lines

That file should be created by running p5302. If you do:

cd t/perf
rm -rf test-results
./run p5302-pack-index.sh

is it generated?

No, it wasn't. See below....

- Ted

<tytso@lambda> {/usr/projects/git/git-core/t/perf} (master)
1140% git show
commit 5fa0f5238b0cd46cfe7f6fa76c3f526ea98148d9 (HEAD -> master, tag: v2.23.0, origin/master, origin/HEAD)

Author: Junio C Hamano <[email protected]>
Date:   Fri Aug 16 10:28:23 2019 -0700

Git 2.23

GIT-VERSION-GEN +1 -1
index 551e607e73..a1539a7ce6 100755
@@ -1,7 +1,7 @@
 #!/bin/sh
 
 GVF=GIT-VERSION-FILE
-DEF_VER=v2.23.0-rc2
+DEF_VER=v2.23.0
 
 LF='
 '

<tytso@lambda> {/usr/projects/git/git-core/t/perf} (master)
1141% rm -rf test-results ; ./run p5302-pack-index.sh
=== Running 1 tests in this tree ===
warning: $GIT_PERF_LARGE_REPO is $GIT_BUILD_DIR.
warning: This will work, but may not be a sufficiently large repo
warning: for representative measurements.
not ok 1 - repack
#
# git repack -ad &&
# PACK=$(ls .git/objects/pack/*.pack | head -n1) &&
# test -f "$PACK" &&
# export PACK
#
cannot open test-results/p5302-pack-index.subtests: No such file or directory at ./aggregate.perl line 153.
<tytso@lambda> {/usr/projects/git/git-core/t/perf} (master)
1142% git status
On branch master
Your branch is up to date with 'origin/master'.

Untracked files:
(use "git add <file>..." to include in what will be committed)

	../../MAKELOG
	../../MAKELOG.1
	../../common-cmds.h
	../../git-lost-found
	../../git-peek-remote
	../../git-rebase--am
	../../git-rebase--common
	../../git-rebase--helper
	../../git-rebase--interactive
	../../git-rebase--merge
	../../git-relink
	../../git-remote-testgit
	../../git-repo-config
	../../git-tar-tree
	../../git_remote_helpers/
	../../perl/MYMETA.json
	../../perl/MYMETA.yml
	../../perl/PM.stamp
	../../perl/blib/
	../../perl/perl.mak
	../../perl/pm_to_blib
	../../test-chmtime
	../../test-config
	../../test-credential
	../../test-ctype
	../../test-date
	../../test-delta
	../../test-dump-cache-tree
	../../test-dump-split-index
	../../test-dump-untracked-cache
	../../test-fake-ssh
	../../test-genrandom
	../../test-hashmap
	../../test-index-version
	../../test-line-buffer
	../../test-match-trees
	../../test-mergesort
	../../test-mktemp
	../../test-obj-pool
	../../test-parse-options
	../../test-path-utils
	../../test-prio-queue
	../../test-read-cache
	../../test-regex
	../../test-revision-walking
	../../test-run-command
	../../test-scrap-cache-tree
	../../test-sha1
	../../test-sha1-array
	../../test-sigchain
	../../test-string-list
	../../test-string-pool
	../../test-submodule-config
	../../test-subprocess
	../../test-svn-fe
	../../test-treap
	../../test-urlmatch-normalization
	../../test-wildmatch

nothing added to commit but untracked files present (use "git add" to track)

#4 lore raw
Quoted 11 lines · Theodore Y. Ts'o
Quoted 9 lines
Quoted 1 line

I presume that's becuase the file t/perf/test-results/p5302-pack-index.subtests is missing?

That file should be created by running p5302. If you do:

cd t/perf
rm -rf test-results
./run p5302-pack-index.sh

is it generated?

No, it wasn't. See below....

Weird. The output here:

Quoted 5 lines

1141% rm -rf test-results ; ./run p5302-pack-index.sh === Running 1 tests in this tree ===
warning: $GIT_PERF_LARGE_REPO is $GIT_BUILD_DIR.
warning: This will work, but may not be a sufficiently large repo
warning: for representative measurements.

Quoted 8 lines

not ok 1 - repack
#
# git repack -ad &&
# PACK=$(ls .git/objects/pack/*.pack | head -n1) &&
# test -f "$PACK" &&
# export PACK
#
cannot open test-results/p5302-pack-index.subtests: No such file or directory at ./aggregate.perl line 153.

Implies that we're trying to _write_ to it, and that the problem is that test-results doesn't exist. That should be set up by this part of perf-lib:

  perf_results_dir=$TEST_OUTPUT_DIRECTORY/test-results
  test -n "$GIT_PERF_SUBSECTION" && perf_results_dir="$perf_results_dir/$GIT_PERF_SUBSECTION"
  mkdir -p "$perf_results_dir"
  rm -f "$perf_results_dir"/$(basename "$0" .sh).subtests

I don't know why that would be failing for you (and not producing any kind of message!). You might need to poke at:

./p5302-pack-index.sh -x

to see what is going on.

It's also weird that the repack fails for you. And that (presumably?) this is the only script that fails, since they would all want to do this.

There are some bits of test-lib.sh that re-exec the script, but I think we ironed out the weirdness there (and I use "--verbose-log" myself, which is one such option). But just in case: do you set GIT_TEST_OPTS in your config.mak?

-Peff

#5 lore raw
Quoted 5 lines · Jeff King
Quoted 1 line

cannot open test-results/p5302-pack-index.subtests: No such file or directory at ./aggregate.perl line 153.

Implies that we're trying to _write_ to it, and that the problem is that test-results doesn't exist. That should be set up by this part of perf-lib:

Hmm.... test-results does exist after the failure:

<tytso@lambda> {/usr/projects/git/git-core/t/perf} (master)
1174% rm -rf test-results ; ./run p5302-pack-index.sh
=== Running 1 tests in this tree ===
warning: $GIT_PERF_LARGE_REPO is $GIT_BUILD_DIR.
warning: This will work, but may not be a sufficiently large repo
warning: for representative measurements.
not ok 1 - repack
#
# git repack -ad &&
# PACK=$(ls .git/objects/pack/*.pack | head -n1) &&
# test -f "$PACK" &&
# export PACK
#
cannot open test-results/p5302-pack-index.subtests: No such file or directory at ./aggregate.perl line 153.
<tytso@lambda> {/usr/projects/git/git-core/t/perf} (master)
1175% ls test-results
total 0
0 run_subsections.names

Maybe we're not in the right cwd for some reason?!?

Quoted 4 lines

There are some bits of test-lib.sh that re-exec the script, but I think we ironed out the weirdness there (and I use "--verbose-log" myself, which is one such option). But just in case: do you set GIT_TEST_OPTS in your config.mak?

You mean in my top-level directory? I don't have a config.mak file in there:

<tytso@lambda> {/usr/projects/git/git-core} (master)
1151% grep GIT_TEST_OPTS config.mak.*
<tytso@lambda> {/usr/projects/git/git-core} (master)
1181% ls config.mak*
4 config.mak.autogen 4 config.mak.dev 4 config.mak.in 24 config.mak.uname
<tytso@lambda> {/usr/projects/git/git-core} (master)
1182% grep GIT_TEST_OPTS config.mak.*

- Ted

#6 lore raw
Quoted 8 lines · Theodore Y. Ts'o
Quoted 5 lines · Jeff King
Quoted 1 line

cannot open test-results/p5302-pack-index.subtests: No such file or directory at ./aggregate.perl line 153.

Implies that we're trying to _write_ to it, and that the problem is that test-results doesn't exist. That should be set up by this part of perf-lib:

Hmm.... test-results does exist after the failure:

Ah, never mind. I was thinking it was coming from this line in perf-lib:

echo "$test_count" >>"$perf_results_dir"/$base.subtests

But it it is pretty clear from this line of your output:

Quoted 1 line

cannot open test-results/p5302-pack-index.subtests: No such file or directory at ./aggregate.perl line 153.

...that this is the aggregate script that is run afterwards complaining. So for whatever reason, the actual p5302 script is exiting early, before it writes anything into the subtests file (from the line above).

That subtests write is done in test_wrapper_(), which is only triggered for actual timing tests, not for setup tests (like the repack step that starts this script). So the plausible sequence of events here is:

1. The first "test_expect_success repack" test fails for some reason. Try running "./p5302-pack-index -v -i -x" to see more output.

2. After the initial failure, the script exits totally rather than continue. That's due to this line in perf-lib, which acts as if "-i" is always set:

# Performance tests should never fail. If they do, stop immediately immediate=t

That makes sense, since any timings we do after a setup step fails would invalidate the result. And if it's not the _first_ such step that fails, we'll just have a truncated subtests file, and some timings will be missing. But when the first one fails, there's no file at all.

We could probably leave a more consistent state in that case. E.g., something like this:

t/perf/perf-lib.sh +1 -1
index b58a43ea43..cfac2c3cbe 100644
@@ -48,7 +48,7 @@ export MODERN_GIT
 perf_results_dir=$TEST_OUTPUT_DIRECTORY/test-results
 test -n "$GIT_PERF_SUBSECTION" && perf_results_dir="$perf_results_dir/$GIT_PERF_SUBSECTION"
 mkdir -p "$perf_results_dir"
-rm -f "$perf_results_dir"/$(basename "$0" .sh).subtests
+>"$perf_results_dir"/$(basename "$0" .sh).subtests
 
 die_if_build_dir_not_repo () {
 	if ! ( cd "$TEST_DIRECTORY/.." &&

which would just quietly omit the rest of p5302. It is unfortunate that you wouldn't get some note in the output saying "hey, we didn't run some tests!".

A lot of this has to do with the hackish way that the list of tests is generated. If you do something like:

./run HEAD^ HEAD p5302-pack-index.sh

and the tests fail in HEAD^ but not HEAD, you'll get a nice table listing all of the tests, with a nil field for each one that didn't run in HEAD^.

But that's because the second run, on HEAD, actually generated a correct p5302-pack-index.subtests file, that actually mentioned the tests! If the order were reversed, you'd get an empty list.

So I think in the long run, it would be nice to have some way of generating the list that's more robust to failures. But I suspect doing that is going to be hard; a lot of this is rooted in the fact that there's no data structure with the set of tests, but literally an executable shell script that decides what to run.

So short answer: use "-v" to figure out why the repack is failing, and the rest is just Git's perf suite being a bit hacky.

-Peff