Skip to contentSkip to conversation

Windows: Fix intermittent failures of t7701

PATCH Superseded Johannes Sixt 4 participants 7 messages Jun 15, 2016

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 Windows: Fix intermittent failures of t7701

From: Johannes Sixt <[email protected]>

The last test case checks whether unpacked objects receive the time stamp of the pack file. Due to different implementations of stat(2) by MSYS and our version in compat/mingw.c, the test fails in about half of the test runs.

Note the following facts:

- The test uses perl's -M operator to compare the time stamps. Since we depend on MSYS perl, the result of this operator is based on MSYS's implementation of the stat(2) call.

- NTFS on Windows records fractional seconds.

- The MSYS implementation of stat(2) *rounds* fractional seconds to full seconds instead of truncating them. This becomes obvious by comparing the modification times reported by 'ls --full-time $f' and 'stat $f' for various files $f.

- Our implementation of stat(2) in compat/mingw.c *truncates* to full seconds.

The consequence of this is that

- add_packed_git() picks up truncated whole second modification times from the pack file time stamp, which is then used for the loose objects, while the pack file retains its time stamp in fractional seconds;

- but the test case compares the pack file's rounded modification times to the loose objects' truncated modification times.

And half of the time the rounded modification time is not the same as its truncated modification time.

The fix is that we replace perl by 'test-chmtime -v +0', which prints the truncated whole-second mtime without modifying it.

We want to catch failures of test-chmtime; but since it appears in a pipe, we cannot access its exit code. Therefore, we at least make sure that it prints time stamps of all files that are passed on its command line.

---
 t/t7701-repack-unpack-unreachable.sh |   15 +++++++++------
 1 files changed, 9 insertions(+), 6 deletions(-)
t/t7701-repack-unpack-unreachable.sh +9 -6
index 63a8225..e6e9f99 100755
@@ -50,12 +50,15 @@ test_expect_success '-A with -d option leaves unreachable objects unpacked' '
 
 compare_mtimes ()
 {
-	perl -e 'my $reference = shift;
-		 foreach my $file (@ARGV) {
-			exit(1) unless(-f $file && -M $file == -M $reference);
-		 }
-		 exit(0);
-		' -- "$@"
+	test-chmtime -v +0 "$@" |
+	{
+		read ref files &&
+		while read t name; do
+			test $ref = $t || break
+			files="$files $name"
+		done &&
+		test "$files" = "$*"
+	}
 }
 
 test_expect_success '-A without -d option leaves unreachable objects packed' '
Signature · 1 line
1.6.1.1.1203.g5882
#1 lore raw

Hi,

Quoted 4 lines · Johannes Sixt

We want to catch failures of test-chmtime; but since it appears in a pipe, we cannot access its exit code. Therefore, we at least make sure that it prints time stamps of all files that are passed on its command line.

I use this trick in my valgrind series:

	($PROGRAM; echo $? > exit.code) | $OTHER_PROGRAM &&
	test 0 = "$(cat exit.code)"

Ciao,
Dscho

#2 lore raw
Quoted 9 lines · Johannes Schindelin
Quoted 4 lines

We want to catch failures of test-chmtime; but since it appears in a pipe, we cannot access its exit code. Therefore, we at least make sure that it prints time stamps of all files that are passed on its command line.

I use this trick in my valgrind series:

	($PROGRAM; echo $? > exit.code) | $OTHER_PROGRAM &&
	test 0 = "$(cat exit.code)"

Oh, that's far too readable. How about:

  exec 3>&1
  status=$( ( ($PROGRAM ; echo $? >&4) | $OTHER_PROGRAM >&3) 4>&1 )
  exec 3>&-

But seriously, I think if we are talking about tests, then

$PROGRAM >output &&
$OTHER_PROGRAM <output

is very clear to read, and as a bonus makes "output" accessible for viewing when the test breaks. The downside is that it isn't very efficient for a large output, but most of our test output is small (and we don't care that much about efficiency).

Just my shell bikeshedding 2 cents. Feel free to ignore.

-Peff

#3 lore raw

Hi,

Quoted 18 lines · Jeff King
Quoted 9 lines · Johannes Schindelin
Quoted 4 lines

We want to catch failures of test-chmtime; but since it appears in a pipe, we cannot access its exit code. Therefore, we at least make sure that it prints time stamps of all files that are passed on its command line.

I use this trick in my valgrind series:

	($PROGRAM; echo $? > exit.code) | $OTHER_PROGRAM &&
	test 0 = "$(cat exit.code)"

Oh, that's far too readable. How about:

  exec 3>&1
  status=$( ( ($PROGRAM ; echo $? >&4) | $OTHER_PROGRAM >&3) 4>&1 )
  exec 3>&-
Quoted 7 lines

But seriously, I think if we are talking about tests, then

$PROGRAM >output &&
$OTHER_PROGRAM <output

is very clear to read, and as a bonus makes "output" accessible for viewing when the test breaks.

The real problem is that in my case, OTHER_PROGRAM=tee.

But you're right, Hannes was talking about tests, where it might even make sense to have a record of what was passed between the two programs.

Ciao,
Dscho

#4 lore raw
In reply to Johannes Schindelin
Quoted 4 lines · Johannes Schindelin

I use this trick in my valgrind series:

	($PROGRAM; echo $? > exit.code) | $OTHER_PROGRAM &&
	test 0 = "$(cat exit.code)"

Ah, using a file as temporary storage? Why not simply

	$PROGRAM > data &&
	$OTHER_PROGRAM < data

Hm? ;)

-- Hannes

#5 lore raw
Quoted 6 lines · Johannes Sixt
Quoted 4 lines · Johannes Schindelin

I use this trick in my valgrind series:

	($PROGRAM; echo $? > exit.code) | $OTHER_PROGRAM &&
	test 0 = "$(cat exit.code)"
Quoted 6 lines

Ah, using a file as temporary storage? Why not simply

	$PROGRAM > data &&
	$OTHER_PROGRAM < data

Hm? ;)

Because too much cleverness often blinds clever minds.

Care to reroll with the simpler, dumber but clearer version?

#6 lore raw

PATCH v2 1/1 Windows: Fix intermittent failures of t7701

From: Johannes Sixt <[email protected]>

The last test case checks whether unpacked objects receive the time stamp of the pack file. Due to different implementations of stat(2) by MSYS and our version in compat/mingw.c, the test fails in about half of the test runs.

Note the following facts:

- The test uses perl's -M operator to compare the time stamps. Since we depend on MSYS perl, the result of this operator is based on MSYS's implementation of the stat(2) call.

- NTFS on Windows records fractional seconds.

- The MSYS implementation of stat(2) *rounds* fractional seconds to full seconds instead of truncating them. This becomes obvious by comparing the modification times reported by 'ls --full-time $f' and 'stat $f' for various files $f.

- Our implementation of stat(2) in compat/mingw.c *truncates* to full seconds.

The consequence of this is that

- add_packed_git() picks up a truncated whole second modification time from the pack file time stamp, which is then used for the loose objects, while the pack file retains its time stamp in fractional seconds;

- but the test case compared the pack file's rounded modification times to the loose objects' truncated modification times.

And half of the time the rounded modification time is not the same as its truncated modification time.

The fix is that we replace perl by 'test-chmtime -v +0', which prints the truncated whole-second mtime without modifying it.

---

Quoted 1 line · Junio C Hamano

Care to reroll with the simpler, dumber but clearer version?

OK. This time I use an intermediate file and dquotes around $t.

-- Hannes

 t/t7701-repack-unpack-unreachable.sh |   14 +++++++-------
 1 files changed, 7 insertions(+), 7 deletions(-)
t/t7701-repack-unpack-unreachable.sh +7 -7
index 63a8225..5babdf2 100755
@@ -50,12 +50,10 @@ test_expect_success '-A with -d option leaves unreachable objects unpacked' '
 
 compare_mtimes ()
 {
-	perl -e 'my $reference = shift;
-		 foreach my $file (@ARGV) {
-			exit(1) unless(-f $file && -M $file == -M $reference);
-		 }
-		 exit(0);
-		' -- "$@"
+	read tref rest &&
+	while read t rest; do
+		test "$tref" = "$t" || break
+	done
 }
 
 test_expect_success '-A without -d option leaves unreachable objects packed' '
@@ -87,7 +85,9 @@ test_expect_success 'unpacked objects receive timestamp of pack file' '
 	tmppack=".git/objects/pack/tmp_pack" &&
 	ln "$packfile" "$tmppack" &&
 	git repack -A -l -d &&
-	compare_mtimes "$tmppack" "$fsha1path" "$csha1path" "$tsha1path"
+	test-chmtime -v +0 "$tmppack" "$fsha1path" "$csha1path" "$tsha1path" \
+		> mtimes &&
+	compare_mtimes < mtimes
 '
 
 test_done
Signature · 1 line
1.6.1.1.1203.g5882