Skip to contentSkip to conversation

cat-file related doc and test

PATCH In review Eric Wong 4 participants 16 messages Jun 17, 2024 – Jun 24, 2024

lore
Share & tools
lore

Eric Wong

author
#0 lore raw

cover cat-file related doc and test

I was working on reducing syscalls required for cat-file --batch (and readers) in a different patch, but noticed a bug in my yet-to-be-published patch wasn't detected in our test suite, only via 3rd-party Perl code.

Then I noticed Git.pm documentation was wrong..., so fixes are in reverse order.

Eric Wong (2):
Git.pm: use array in command_bidi_pipe example
t9700: ensure cat-file info isn't buffered by default

 perl/Git.pm     |  4 ++--
 t/t9700/test.pl | 14 ++++++++++++++
 2 files changed, 16 insertions(+), 2 deletions(-)
author
#1 lore raw

PATCH 1/2 Git.pm: use array in command_bidi_pipe example

command_bidi_pipe takes the git command and optional arguments as an array, not a string. Make sure the documentation example is usable code.

---
 perl/Git.pm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
perl/Git.pm +2 -2
index 03bf570bf4..aebfe0c6e0 100644
@@ -418,7 +418,7 @@ sub command_bidi_pipe {
 and it is the fourth value returned by C<command_bidi_pipe()>.  The call idiom
 is:
 
-	my ($pid, $in, $out, $ctx) = $r->command_bidi_pipe('cat-file --batch-check');
+	my ($pid, $in, $out, $ctx) = $r->command_bidi_pipe(qw(cat-file --batch-check));
 	print $out "000000000\n";
 	while (<$in>) { ... }
 	$r->command_close_bidi_pipe($pid, $in, $out, $ctx);
@@ -431,7 +431,7 @@ sub command_bidi_pipe {
 calling this function.  This may be useful in a query-response type of
 commands where caller first writes a query and later reads response, eg:
 
-	my ($pid, $in, $out, $ctx) = $r->command_bidi_pipe('cat-file --batch-check');
+	my ($pid, $in, $out, $ctx) = $r->command_bidi_pipe(qw(cat-file --batch-check));
 	print $out "000000000\n";
 	close $out;
 	while (<$in>) { ... }
#2 lore raw

Re: [PATCH 1/2] Git.pm: use array in command_bidi_pipe example

Quoted 3 lines · Eric Wong

command_bidi_pipe takes the git command and optional arguments as an array, not a string. Make sure the documentation example is usable code.

Makes sense.

Quoted 28 lines
---
 perl/Git.pm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
perl/Git.pm +2 -2
index 03bf570bf4..aebfe0c6e0 100644
@@ -418,7 +418,7 @@ sub command_bidi_pipe {
 and it is the fourth value returned by C<command_bidi_pipe()>.  The call idiom
 is:
 
-	my ($pid, $in, $out, $ctx) = $r->command_bidi_pipe('cat-file --batch-check');
+	my ($pid, $in, $out, $ctx) = $r->command_bidi_pipe(qw(cat-file --batch-check));
 	print $out "000000000\n";
 	while (<$in>) { ... }
 	$r->command_close_bidi_pipe($pid, $in, $out, $ctx);
@@ -431,7 +431,7 @@ sub command_bidi_pipe {
 calling this function.  This may be useful in a query-response type of
 commands where caller first writes a query and later reads response, eg:
 
-	my ($pid, $in, $out, $ctx) = $r->command_bidi_pipe('cat-file --batch-check');
+	my ($pid, $in, $out, $ctx) = $r->command_bidi_pipe(qw(cat-file --batch-check));
 	print $out "000000000\n";
 	close $out;
 	while (<$in>) { ... }
author
#3 lore raw

PATCH 2/2 t9700: ensure cat-file info isn't buffered by default

In reply to Eric Wong

While working on buffering changes to git cat-file' in a separate patch, I inadvertently made the output of --batch-check and the info' command of --batch-command buffered by default.

Buffering by default breaks some 3rd-party Perl scripts using cat-file, but this breakage was not detected anywhere in our test suite. The easiest place to test this behavior is with Git.pm, since (AFAIK) other equivalent way to test this behavior from Bourne shell and/or awk would require racy sleeps, non-portable FIFOs or tedious C code.

---
 t/t9700/test.pl | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
t/t9700/test.pl +14 -0
index d8e85482ab..94a2e2c09d 100755
@@ -154,6 +154,20 @@ sub adjust_dirsep {
 		     "abc\"\\ \x07\x08\x09\x0a\x0b\x0c\x0d\x01 ",
 		     'unquote escape sequences');
 
+# ensure --batch-check is unbuffered by default
+my ($pid, $in, $out, $ctx) = $r->command_bidi_pipe(qw(cat-file --batch-check));
+print $out $file1hash, "\n" or die $!;
+my $info = <$in>;
+is $info, "$file1hash blob 15\n", 'command_bidi_pipe w/ --batch-check';
+$r->command_close_bidi_pipe($pid, $in, $out, $ctx);
+
+# ditto with `info' with --batch-command
+($pid, $in, $out, $ctx) = $r->command_bidi_pipe(qw(cat-file --batch-command));
+print $out 'info ', $file1hash, "\n" or die $!;
+$info = <$in>;
+is $info, "$file1hash blob 15\n", 'command_bidi_pipe w/ --batch-command=info';
+$r->command_close_bidi_pipe($pid, $in, $out, $ctx);
+
 printf "1..%d\n", Test::More->builder->current_test;
 
 my $is_passing = eval { Test::More->is_passing };
#4 lore raw

Re: [PATCH 2/2] t9700: ensure cat-file info isn't buffered by default

Quoted 6 lines · Eric Wong

Buffering by default breaks some 3rd-party Perl scripts using cat-file, but this breakage was not detected anywhere in our test suite. The easiest place to test this behavior is with Git.pm, since (AFAIK) other equivalent way to test this behavior from Bourne shell and/or awk would require racy sleeps, non-portable FIFOs or tedious C code.

Yes, using Perl is a good substitute for writing it in C in this case. I however question the choice to use t9700/test.pl here, which is clearly stated that its purpose is to "test perl interface which is Git.pm", and added tests are not testing anything in Git.pm at all.

Using t9700/test.pl only because it happens to use "perl -MTest::More" sounds a bit eh, suboptimal.

It seems that there are Perl snippets in other tests (including t1006 that is specifically about cat-file). How involved would it be to implement these new tests without modifying unrelated test scripts?

Quoted 30 lines
---
 t/t9700/test.pl | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
t/t9700/test.pl +14 -0
index d8e85482ab..94a2e2c09d 100755
@@ -154,6 +154,20 @@ sub adjust_dirsep {
 		     "abc\"\\ \x07\x08\x09\x0a\x0b\x0c\x0d\x01 ",
 		     'unquote escape sequences');
 
+# ensure --batch-check is unbuffered by default
+my ($pid, $in, $out, $ctx) = $r->command_bidi_pipe(qw(cat-file --batch-check));
+print $out $file1hash, "\n" or die $!;
+my $info = <$in>;
+is $info, "$file1hash blob 15\n", 'command_bidi_pipe w/ --batch-check';
+$r->command_close_bidi_pipe($pid, $in, $out, $ctx);
+
+# ditto with `info' with --batch-command
+($pid, $in, $out, $ctx) = $r->command_bidi_pipe(qw(cat-file --batch-command));
+print $out 'info ', $file1hash, "\n" or die $!;
+$info = <$in>;
+is $info, "$file1hash blob 15\n", 'command_bidi_pipe w/ --batch-command=info';
+$r->command_close_bidi_pipe($pid, $in, $out, $ctx);
+
 printf "1..%d\n", Test::More->builder->current_test;
 
 my $is_passing = eval { Test::More->is_passing };
author
#5 lore raw

PATCH v2 2/2 t1006: ensure cat-file info isn't buffered by default

Quoted 9 lines · Junio C Hamano
Quoted 6 lines · Eric Wong

Buffering by default breaks some 3rd-party Perl scripts using cat-file, but this breakage was not detected anywhere in our test suite. The easiest place to test this behavior is with Git.pm, since (AFAIK) other equivalent way to test this behavior from Bourne shell and/or awk would require racy sleeps, non-portable FIFOs or tedious C code.

Quoted 8 lines

Yes, using Perl is a good substitute for writing it in C in this case. I however question the choice to use t9700/test.pl here, which is clearly stated that its purpose is to "test perl interface which is Git.pm", and added tests are not testing anything in Git.pm at all.

Using t9700/test.pl only because it happens to use "perl -MTest::More" sounds a bit eh, suboptimal.

*shrug* I figure Test::More is common enough since it's part of the Perl standard library; but I consider Perl a better scripting language than sh by far and wish our whole test suite were Perl :>

Quoted 7 lines

It seems that there are Perl snippets in other tests (including t1006 that is specifically about cat-file). How involved would it be to implement these new tests without modifying unrelated test scripts?

Quoted 2 lines
 t/t9700/test.pl | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

More code than that. At least IPC::Open2 takes care of the nasty portability bits, but getting the Perl quoting nested properly inside sh was confusing :x

v2: moved test to t1006 to avoid Test::More,
add select timeout in case a buffering bug does get introduced, updated commit message and clarified the bug it's supposed to guard against
(I initially tried stdio buffering, but moved away from it for the patch I'm testing...)

----8<----
Subject: [PATCH] t1006: ensure cat-file info isn't buffered by default

While working on buffering changes to git cat-file' in a separate patch, I inadvertently made the output of --batch-check and the info' command of --batch-command buffered as if opt->buffer_output is turned on by default.

Buffering by default breaks some 3rd-party Perl scripts using cat-file, but this breakage was not detected anywhere in our test suite. Add a small Perl snippet to test this problem since (AFAIK) other equivalent ways to test this behavior from Bourne shell and/or awk would require racy sleeps, non-portable FIFOs or tedious C code.

---
 t/t1006-cat-file.sh | 30 ++++++++++++++++++++++++++++++
 1 file changed, 30 insertions(+)
t/t1006-cat-file.sh +30 -0
index e12b221972..ff9bf213aa 100755
@@ -1294,4 +1294,34 @@ test_expect_success 'batch-command flush without --buffer' '
 	grep "^fatal:.*flush is only for --buffer mode.*" err
 '
 
+script='
+use warnings;
+use strict;
+use IPC::Open2;
+my ($opt, $oid, $expect, @pfx) = @ARGV;
+my @cmd = (qw(git cat-file), $opt);
+my $pid = open2(my $out, my $in, @cmd) or die "open2: @cmd";
+print $in @pfx, $oid, "\n" or die "print $!";
+my $rvec = "";
+vec($rvec, fileno($out), 1) = 1;
+select($rvec, undef, undef, 30) or die "no response to `@pfx $oid` from @cmd";
+my $info = <$out>;
+chop($info) eq "\n" or die "no LF";
+$info eq $expect or die "`$info` != `$expect`";
+close $in or die "close in $!";
+close $out or die "close out $!";
+waitpid $pid, 0;
+$? == 0 or die "\$?=$?";
+'
+
+expect="$hello_oid blob $hello_size"
+
+test_expect_success PERL '--batch-check is unbuffered by default' '
+	perl -e "$script" -- --batch-check $hello_oid "$expect"
+'
+
+test_expect_success PERL '--batch-command info is unbuffered by default' '
+	perl -e "$script" -- --batch-command $hello_oid "$expect" "info "
+'
+
 test_done
#6 lore raw

Re: [PATCH v2 2/2] t1006: ensure cat-file info isn't buffered by default

Quoted 6 lines · Eric Wong
Quoted 6 lines

Yes, using Perl is a good substitute for writing it in C in this case. I however question the choice to use t9700/test.pl here, which is clearly stated that its purpose is to "test perl interface which is Git.pm", and added tests are not testing anything in Git.pm at all.

Quoted 6 lines
Quoted 2 lines

Using t9700/test.pl only because it happens to use "perl -MTest::More" sounds a bit eh, suboptimal.

*shrug* I figure Test::More is common enough since it's part of the Perl standard library; but I consider Perl a better scripting language than sh by far and wish our whole test suite were Perl :>

Oh, I think we (actually the author of t9700) considers it common enough that we have PERL_TEST_MORE prerequisite to allow us to write tests, assuming that it is available, and let us easily skip where it is not available. So I do not think I mind the dependency on Test::More at all. Moving the tests to t1006 and rewriting the tests not to use Test::More are two separate and unrelated things, and if you are more comfortable with Test::More (and more importantly if it is natural to write Perl based tests using Test::More), it is not necessary to switch away from it.

author
#7 lore raw

Re: [PATCH v2 2/2] t1006: ensure cat-file info isn't buffered by default

Quoted 15 lines · Junio C Hamano
Quoted 6 lines · Eric Wong
Quoted 6 lines

Yes, using Perl is a good substitute for writing it in C in this case. I however question the choice to use t9700/test.pl here, which is clearly stated that its purpose is to "test perl interface which is Git.pm", and added tests are not testing anything in Git.pm at all.

Quoted 6 lines
Quoted 2 lines

Using t9700/test.pl only because it happens to use "perl -MTest::More" sounds a bit eh, suboptimal.

*shrug* I figure Test::More is common enough since it's part of the Perl standard library; but I consider Perl a better scripting language than sh by far and wish our whole test suite were Perl :>

Quoted 9 lines

Oh, I think we (actually the author of t9700) considers it common enough that we have PERL_TEST_MORE prerequisite to allow us to write tests, assuming that it is available, and let us easily skip where it is not available. So I do not think I mind the dependency on Test::More at all. Moving the tests to t1006 and rewriting the tests not to use Test::More are two separate and unrelated things, and if you are more comfortable with Test::More (and more importantly if it is natural to write Perl based tests using Test::More), it is not necessary to switch away from it.

OK, fair enough. Given t1006 is mostly sh, I prefer keeping v2 as-is because the Test::More->builder munging of test numbers in t9700/test.pl is nasty too and I wouldn't enjoy duplicating those bits in a hypothetical t1006/test.pl, either.

It would be nice to have first class support for Test::More in our suite so we could just have t/t0006-cat-file.t and t/t9700-perl-git.t implemented in Perl without sh at all, but that's a separate discussion.

#8 lore raw

Re: [PATCH v2 2/2] t1006: ensure cat-file info isn't buffered by default

Quoted 4 lines · Eric Wong

OK, fair enough. Given t1006 is mostly sh, I prefer keeping v2 as-is because the Test::More->builder munging of test numbers in t9700/test.pl is nasty too and I wouldn't enjoy duplicating those bits in a hypothetical t1006/test.pl, either.

That is quite sensible.

#9 lore raw

Re: [PATCH v2 2/2] t1006: ensure cat-file info isn't buffered by default

In reply to Eric Wong
Quoted 15 lines · Eric Wong
(quoted diff) +15 -0
+script='
+use warnings;
+use strict;
+use IPC::Open2;
+my ($opt, $oid, $expect, @pfx) = @ARGV;
+my @cmd = (qw(git cat-file), $opt);
+my $pid = open2(my $out, my $in, @cmd) or die "open2: @cmd";
+print $in @pfx, $oid, "\n" or die "print $!";
+my $rvec = "";
+vec($rvec, fileno($out), 1) = 1;
+select($rvec, undef, undef, 30) or die "no response to `@pfx $oid` from @cmd";
+my $info = <$out>;
+chop($info) eq "\n" or die "no LF";
+$info eq $expect or die "`$info` != `$expect`";
+close $in or die "close in $!";

Review

Quoted 10 lines
(quoted diff) +10 -0
+close $out or die "close out $!";
+waitpid $pid, 0;
+$? == 0 or die "\$?=$?";
+'
+
+expect="$hello_oid blob $hello_size"
+
+test_expect_success PERL '--batch-check is unbuffered by default' '
+	perl -e "$script" -- --batch-check $hello_oid "$expect"
+'

We often use "perl -e" for one-liners, etc, but this is pretty big. Maybe:

cat >foo.pl <<-\EOF
...
EOF
perl foo.pl -- ...

would be more readable? To be clear I don't think there's anything incorrect about your usage, but it would match the style of our suite a bit better.

Likewise, it would be usual in our suite for the helper to do the minimum that needs to be in perl, and use our normal functions for things like comparing output (rather than taking its own "expect" argument).

So maybe:

t/t1006-cat-file.sh +29 -0
index e12b221972..929d7a7579 100755
@@ -1294,4 +1294,33 @@ test_expect_success 'batch-command flush without --buffer' '
 	grep "^fatal:.*flush is only for --buffer mode.*" err
 '
 
+# Copy a single line from stdin to the program specified
+# by @ARGV, and then wait for a response _without_ closing
+# the pipe.
+cat >run-and-wait.pl <<-\EOF
+use IPC::Open2;
+open2(my $out, my $in, @ARGV) or die "open2: @ARGV";
+print $in scalar(<STDIN>) or die "print $!";
+
+my $rvec = "";
+vec($rvec, fileno($out), 1) = 1;
+select($rvec, undef, undef, 30) or die "no response after 30 seconds";
+
+print scalar(<$out>);
+EOF
+
+test_expect_success PERL '--batch-check is unbuffered by default' '
+	echo "$hello_oid" |
+	perl run-and-wait.pl git cat-file --batch-check >out &&
+	echo "$hello_oid blob $hello_size" >expect &&
+	test_cmp expect out
+'
+
+test_expect_success PERL '--batch-command info is unbuffered by default' '
+	echo "info $hello_oid" |
+	perl run-and-wait.pl git cat-file --batch-command >out &&
+	echo "$hello_oid blob $hello_size" >expect &&
+	test_cmp expect out
+'
+
 test_done

I went for brevity above. Notably missing are:

- the use of strict/warnings. I think we've shied away from these in the test suite because we want to run on any version of perl. In my experience most strict/warnings output is actually telling you about obvious garbage, but not always. IIRC perl got more strict about "()" around lists in some contexts a few years back, and code which used to be OK started generating warnings. OTOH, those warnings were probably a sign of problems-to-come, anyway. Without "FATAL", though, I think "use warnings" is not doing much good (nobody is ever going to see its output if the test isn't failing).

- I dropped the close/waitpid. I guess maybe it is valuable to confirm that cat-file did not barf, but IMHO the important thing here is testing that it produced the single line of output we expected.

-Peff

author
#10 lore raw

Re: [PATCH v2 2/2] t1006: ensure cat-file info isn't buffered by default

Quoted 3 lines · Jeff King
Quoted 1 line · Eric Wong

+script='

<snip>

Quoted 9 lines

t/t1006-cat-file.sh test_expect_success 'batch-command flush without --buffer' ' · line 1317

Quoted 5 lines
t/t1006-cat-file.sh +5 -0
+expect="$hello_oid blob $hello_size"
+
+test_expect_success PERL '--batch-check is unbuffered by default' '
+	perl -e "$script" -- --batch-check $hello_oid "$expect"
+'

We often use "perl -e" for one-liners, etc, but this is pretty big. Maybe:

Quoted 8 lines

cat >foo.pl <<-\EOF
...
EOF
perl foo.pl -- ...

would be more readable? To be clear I don't think there's anything incorrect about your usage, but it would match the style of our suite a bit better.

*shrug* It doesn't save the nested quoting/expansion confusion; but it's Junio's call. I don't think a v3 is worth the effort.

Quoted 4 lines

Likewise, it would be usual in our suite for the helper to do the minimum that needs to be in perl, and use our normal functions for things like comparing output (rather than taking its own "expect" argument).

<snip>

t/t1006-cat-file.sh test_expect_success 'batch-command flush without --buffer' ' · line 1316

Quoted 5 lines
t/t1006-cat-file.sh +5 -0
+test_expect_success PERL '--batch-check is unbuffered by default' '
+	echo "$hello_oid" |
+	perl run-and-wait.pl git cat-file --batch-check >out &&
+	echo "$hello_oid blob $hello_size" >expect &&
+	test_cmp expect out

I prefer to avoid process spawning overhead from test_cmp; but that's a small drop in a big bucket.

Quoted 11 lines

I went for brevity above. Notably missing are:

- the use of strict/warnings. I think we've shied away from these in the test suite because we want to run on any version of perl. In my experience most strict/warnings output is actually telling you about obvious garbage, but not always. IIRC perl got more strict about "()" around lists in some contexts a few years back, and code which used to be OK started generating warnings. OTOH, those warnings were probably a sign of problems-to-come, anyway. Without "FATAL", though, I think "use warnings" is not doing much good (nobody is ever going to see its output if the test isn't failing).

It may make problems easier to find if there are failures, so I think the potential benefits outweight any downsides.

Quoted 3 lines

- I dropped the close/waitpid. I guess maybe it is valuable to confirm that cat-file did not barf, but IMHO the important thing here is testing that it produced the single line of output we expected.

I've found some unexpected bugs through excessive error checking in the past, so much preferred to keep them.

#11 lore raw

Re: [PATCH v2 2/2] t1006: ensure cat-file info isn't buffered by default

Quoted 8 lines · Eric Wong
Quoted 8 lines

We often use "perl -e" for one-liners, etc, but this is pretty big. Maybe:

cat >foo.pl <<-\EOF
...
EOF
perl foo.pl -- ...

Quoted 6 lines
Quoted 3 lines

would be more readable? To be clear I don't think there's anything incorrect about your usage, but it would match the style of our suite a bit better.

*shrug* It doesn't save the nested quoting/expansion confusion; but it's Junio's call. I don't think a v3 is worth the effort.

It does allow you to use single quotes in the script, though I think you managed without it.

Quoted 8 lines

t/t1006-cat-file.sh test_expect_success 'batch-command flush without --buffer' ' · line 1316

Quoted 5 lines
t/t1006-cat-file.sh +5 -0
+test_expect_success PERL '--batch-check is unbuffered by default' '
+	echo "$hello_oid" |
+	perl run-and-wait.pl git cat-file --batch-check >out &&
+	echo "$hello_oid blob $hello_size" >expect &&
+	test_cmp expect out

I prefer to avoid process spawning overhead from test_cmp; but that's a small drop in a big bucket.

If we care about that, I'd rather see us make test_cmp zero-process with a shell helper than come up with ad-hoc solutions. I've tried to measure something like that before, but couldn't come up with any conclusive improvements (my findings were mostly that running Git itself accounts for most of the process overhead).

-Peff

#12 lore raw

Re: [PATCH 2/2] t9700: ensure cat-file info isn't buffered by default

In reply to Eric Wong
Quoted 3 lines · Eric Wong

While working on buffering changes to git cat-file' in a separate patch, I inadvertently made the output of --batch-check and the info' command of --batch-command buffered by default.

Here "buffered" means "as if opt->buffer_output is turned on", which in turn means "the output goes through stdio"? Just making sure that my understanding of what the breakage was is in line with what you wanted to convey.

Thanks.

#13 lore raw

Re: [PATCH 2/2] t9700: ensure cat-file info isn't buffered by default

In reply to Eric Wong

Hi Eric

t/t9700/test.pl sub adjust_dirsep { · line 159

Quoted 3 lines · Eric Wong
t/t9700/test.pl +3 -0
+# ensure --batch-check is unbuffered by default
+my ($pid, $in, $out, $ctx) = $r->command_bidi_pipe(qw(cat-file --batch-check));
+print $out $file1hash, "\n" or die $!;

It's been a while since I did any perl scripting and I'm not clear whether $out is buffered or not and if it is whether it is guaranteed to be flushed when we print "\n". It might be worth adding a explicit flush so it is clear that any deadlocks come from cat-file and not our test code.

Quoted 1 line

+my $info = <$in>;

Is there an easy way to add a timeout to this read so that the failure mode isn't "the test hangs without printing anything"? I'm not sure that failure mode is easy to diagnose from our CI output as it is hard to tell which test caused the CI to timeout and it takes ages for the CI to time out.

Best Wishes

Phillip

Quoted 14 lines
t/t9700/test.pl +10 -0
+is $info, "$file1hash blob 15\n", 'command_bidi_pipe w/ --batch-check';
+$r->command_close_bidi_pipe($pid, $in, $out, $ctx);
+
+# ditto with `info' with --batch-command
+($pid, $in, $out, $ctx) = $r->command_bidi_pipe(qw(cat-file --batch-command));
+print $out 'info ', $file1hash, "\n" or die $!;
+$info = <$in>;
+is $info, "$file1hash blob 15\n", 'command_bidi_pipe w/ --batch-command=info';
+$r->command_close_bidi_pipe($pid, $in, $out, $ctx);
+
  printf "1..%d\n", Test::More->builder->current_test;
 
  my $is_passing = eval { Test::More->is_passing };
author
#14 lore raw

Re: [PATCH 2/2] t9700: ensure cat-file info isn't buffered by default

Quoted 11 lines · Phillip Wood

Hi Eric

t/t9700/test.pl sub adjust_dirsep { · line 159

Quoted 3 lines · Eric Wong
t/t9700/test.pl +3 -0
+# ensure --batch-check is unbuffered by default
+my ($pid, $in, $out, $ctx) = $r->command_bidi_pipe(qw(cat-file --batch-check));
+print $out $file1hash, "\n" or die $!;

It's been a while since I did any perl scripting and I'm not clear whether $out is buffered or not and if it is whether it is guaranteed to be flushed when we print "\n". It might be worth adding a explicit flush so it is clear that any deadlocks come from cat-file and not our test code.

Pipes and sockets created by Perl are always unbuffered since 5.8, at least. If they were buffered, Git.pm users (including git-svn) wouldn't have worked at all.

Quoted 6 lines
Quoted 1 line

+my $info = <$in>;

Is there an easy way to add a timeout to this read so that the failure mode isn't "the test hangs without printing anything"? I'm not sure that failure mode is easy to diagnose from our CI output as it is hard to tell which test caused the CI to timeout and it takes ages for the CI to time out.

Yeah, select() has been added in v2.

#15 lore raw

Re: [PATCH 2/2] t9700: ensure cat-file info isn't buffered by default

Quoted 13 lines · Eric Wong
Quoted 11 lines · Phillip Wood

Hi Eric

t/t9700/test.pl sub adjust_dirsep { · line 159

Quoted 3 lines · Eric Wong
t/t9700/test.pl +3 -0
+# ensure --batch-check is unbuffered by default
+my ($pid, $in, $out, $ctx) = $r->command_bidi_pipe(qw(cat-file --batch-check));
+print $out $file1hash, "\n" or die $!;

It's been a while since I did any perl scripting and I'm not clear whether $out is buffered or not and if it is whether it is guaranteed to be flushed when we print "\n". It might be worth adding a explicit flush so it is clear that any deadlocks come from cat-file and not our test code.

Quoted 3 lines

Pipes and sockets created by Perl are always unbuffered since 5.8, at least. If they were buffered, Git.pm users (including git-svn) wouldn't have worked at all.

Thanks for clarifying that

Quoted 8 lines
Quoted 6 lines
Quoted 1 line

+my $info = <$in>;

Is there an easy way to add a timeout to this read so that the failure mode isn't "the test hangs without printing anything"? I'm not sure that failure mode is easy to diagnose from our CI output as it is hard to tell which test caused the CI to timeout and it takes ages for the CI to time out.

Yeah, select() has been added in v2.

That's much nicer.

Thanks

Phillip