t9300-fast-import: don't hang if background fast-import exits too early
PATCH 1/1 t9300-fast-import: don't hang if background fast-import exits too early
The five tests checking 'git fast-import's checkpoint handling in 't9300-fast-import.sh', all with the prefix "V:" in their test description, can hang indefinitely if 'git fast-import' unexpectedly dies early in any of these tests.
These five tests run 'git fast-import' in the background, while feeding instructions to its standard input through a fifo from a background subshell, and reading and verifying its standard output through another fifo in the test script's main shell process. This "reading and verifying" is basically a 'while read ...' shell loop iterating until 'git fast-import' outputs the expected line, ignoring any other output. This doesn't work very well when 'git fast-import' dies before printing that particular line, because the 'read' builtin doesn't get EOF after the death of 'git fast-import', as their input and output are not connected directly but through a fifo. Consequently, that 'read' hangs waiting for the next line from the already dead 'git fast-import', leaving the test script and in turn the whole test suite hanging.
Avoid this hang by checking whether the background 'git fast-import' process exited unexpectedly early, and interrupt the 'while read' loop if it did. We have to jump through some hoops to achive that, though:
- Start the background 'git fast-import' in another background subshell, which then waits until that 'git fast-import' process exits. If it does exit, then report its exit code, and write a message to the fifo used for 'git fast-import's standard output, thus un-block the 'read' builtin in the main shell process.
- Modify that 'while read' loop to break the loop upon seeing that message, and fail the test in the usual way.
- Once the test is finished kill that background subshell as well, and do so before killing the background 'git fast-import'. Otherwise the background 'git fast-import' and subshell would die racily, and if 'git fast-import' were to die sooner, then we might get some undesired and potentially confusing messages in the test's output.
--- t/t9300-fast-import.sh | 23 +++++++++++++++++------ 1 file changed, 17 insertions(+), 6 deletions(-)
t/t9300-fast-import.sh +17 -6
@@ -3148,63 +3148,74 @@ test_expect_success 'U: validate root delete result' ' # The commands in input_file should not produce any output on the file # descriptor set with --cat-blob-fd (or stdout if unspecified). # # To make sure you're observing the side effects of checkpoint *before* # fast-import terminates (and thus writes out its state), check that the # fast-import process is still running using background_import_still_running # *after* evaluating the test conditions. background_import_then_checkpoint () { options=$1 input_file=$2 mkfifo V.input exec 8<>V.input rm V.input mkfifo V.output exec 9<>V.output rm V.output - git fast-import $options <&8 >&9 & - echo $! >V.pid + ( + git fast-import $options <&8 >&9 & + echo $! >V.fi.pid + wait $! + echo >&2 "background fast-import terminated too early with exit code $?" + # Un-block the read loop in the main shell process. + echo >&9 UNEXPECTED + ) & + echo $! >V.sh.pid # We don't mind if fast-import has already died by the time the test # ends. - test_when_finished " + test_when_finished ' exec 8>&-; exec 9>&-; - kill $(cat V.pid) && wait $(cat V.pid) - true" + kill $(cat V.sh.pid) && wait $(cat V.sh.pid) + kill $(cat V.fi.pid) && wait $(cat V.sh.pid) + true' # Start in the background to ensure we adhere strictly to (blocking) # pipes writing sequence. We want to assume that the write below could # block, e.g. if fast-import blocks writing its own output to &9 # because there is no reader on &9 yet. ( cat "$input_file" echo "checkpoint" echo "progress checkpoint" ) >&8 & error=1 ;# assume the worst while read output <&9 do if test "$output" = "progress checkpoint" then error=0 break + elif test "$output" = "UNEXPECTED" + then + break fi # otherwise ignore cruft echo >&2 "cruft: $output" done if test $error -eq 1 then false fi } background_import_still_running () { - if ! kill -0 "$(cat V.pid)" + if ! kill -0 "$(cat V.fi.pid)" then echo >&2 "background fast-import terminated too early" false fi }
Signature · 1 line
2.24.0.643.gac013444ca
Quoted 9 lines · SZEDER Gábor
t/t9300-fast-import.sh +9
-0+ ( + git fast-import $options <&8 >&9 & + echo $! >V.fi.pid + wait $! + echo >&2 "background fast-import terminated too early with exit code $?" + # Un-block the read loop in the main shell process. + echo >&9 UNEXPECTED + ) & + echo $! >V.sh.pid
t/t9300-fast-import.sh test_expect_success 'U: validate root delete result' ' · line 3182
Quoted 10 lines
t/t9300-fast-import.sh +4
-3# We don't mind if fast-import has already died by the time the test # ends. - test_when_finished " + test_when_finished ' exec 8>&-; exec 9>&-; - kill $(cat V.pid) && wait $(cat V.pid) - true" + kill $(cat V.sh.pid) && wait $(cat V.sh.pid) + kill $(cat V.fi.pid) && wait $(cat V.sh.pid) + true'
The original interpolates the PID of the fast-import when "when-finished" program is registered, so it is OK if somebody else removed V.pid file; the new one interpolates when "when-finished" program is run, reading from V.??.pid, so somebody needs to make sure these pid files will stay around. I do not think it is an issue as I suspect we've left it to the global clean-up procedure that removes the trash directory to remove the pid file.
By the way, does the second "kill && wait" wait for the right process?
Quoted 8 lines
background_import_still_running() {t/t9300-fast-import.sh +1
-1- if ! kill -0 "$(cat V.pid)" + if ! kill -0 "$(cat V.fi.pid)" then echo >&2 "background fast-import terminated too early" false fi }
Quoted 22 lines · Junio C Hamano
Quoted 9 lines · SZEDER Gábor
t/t9300-fast-import.sh +9
-0+ ( + git fast-import $options <&8 >&9 & + echo $! >V.fi.pid + wait $! + echo >&2 "background fast-import terminated too early with exit code $?" + # Un-block the read loop in the main shell process. + echo >&9 UNEXPECTED + ) & + echo $! >V.sh.pidt/t9300-fast-import.sh test_expect_success 'U: validate root delete result' ' · line 3182
Quoted 10 lines
t/t9300-fast-import.sh +4
-3# We don't mind if fast-import has already died by the time the test # ends. - test_when_finished " + test_when_finished ' exec 8>&-; exec 9>&-; - kill $(cat V.pid) && wait $(cat V.pid) - true" + kill $(cat V.sh.pid) && wait $(cat V.sh.pid) + kill $(cat V.fi.pid) && wait $(cat V.sh.pid) + true'
Quoted 7 lines
The original interpolates the PID of the fast-import when "when-finished" program is registered, so it is OK if somebody else removed V.pid file; the new one interpolates when "when-finished" program is run, reading from V.??.pid, so somebody needs to make sure these pid files will stay around. I do not think it is an issue as I suspect we've left it to the global clean-up procedure that removes the trash directory to remove the pid file.
In the original the same shell process starts 'git fast-import', writes its pidfile, and registers the test_when_finished commands, so we can be sure that the pid file is already present when the shell runs the $(cat V.pid) command substitutions.
With this patch that's not the case anymore, because the background subshell starts 'git fast-import' and writes the pidfile, but the main shell process registers the test_when_finished commands. IOW these two shell processes are racing, and it's possible that the test_when_finished command is executed before the background subshell can write the pidfile. So double quotes around the block of test_when_finished commands are not good.
Quoted 2 lines
By the way, does the second "kill && wait" wait for the right process?
Ouch, it clearly doesn't. Copy-paste error I suppose. Thanks for spotting it.
Quoted 9 lines
Quoted 9 lines
background_import_still_running() {t/t9300-fast-import.sh +1
-1- if ! kill -0 "$(cat V.pid)" + if ! kill -0 "$(cat V.fi.pid)" then echo >&2 "background fast-import terminated too early" false fi }
Quoted 17 lines · SZEDER Gábor
Quoted 9 lines
t/t9300-fast-import.sh test_expect_success 'U: validate root delete result' ' · line 3182
Quoted 8 lines
t/t9300-fast-import.sh +4
-3- test_when_finished " + test_when_finished ' exec 8>&-; exec 9>&-; - kill $(cat V.pid) && wait $(cat V.pid) - true" + kill $(cat V.sh.pid) && wait $(cat V.sh.pid) + kill $(cat V.fi.pid) && wait $(cat V.sh.pid) + true'Quoted 7 lines
The original interpolates the PID of the fast-import when "when-finished" program is registered, so it is OK if somebody else removed V.pid file; the new one interpolates when "when-finished" program is run, reading from V.??.pid, so somebody needs to make sure these pid files will stay around. I do not think it is an issue as I suspect we've left it to the global clean-up procedure that removes the trash directory to remove the pid file.
Quoted 4 lines
In the original the same shell process starts 'git fast-import', writes its pidfile, and registers the
test_when_finishedcommands, so we can be sure that the pid file is already present when the shell runs the $(cat V.pid) command substitutions.
Yes. It also means that V.pid file was not very useful in the when-finished handler in the original. We could have just used a shell variable.
Quoted 7 lines
With this patch that's not the case anymore, because the background subshell starts 'git fast-import' and writes the pidfile, but the main shell process registers the
test_when_finishedcommands. IOW these two shell processes are racing, and it's possible that thetest_when_finishedcommand is executed before the background subshell can write the pidfile. So double quotes around the block oftest_when_finishedcommands are not good.
Oh, I was not questioning that. I wanted to make sure, and I was doing so aloud, that these files are
(1) created way before, and
(2) left on the filesystem
when-finished handler actually runs, because the original did not need any guarantee for (2), but now the sq version does.
v2 cover t9300-fast-import: don't hang if background fast-import exits too early
In reply to SZEDER GáborThe five tests checking 'git fast-import's checkpoint handling in 't9300-fast-import.sh', all with the prefix "V:" in their test description, can hang indefinitely if 'git fast-import' unexpectedly dies early in any of these tests.
Changes since v1:
- The first patch is new, and refactors those tests to store the PID of the background 'git fast-import' process in a shell variable instead of in a pidfile.
- The second patch is conceptually the same as the only patch v1, but it has been updated to store the PID of the background subshell in a variable as well.
SZEDER Gábor (2):
t9300-fast-import: store the PID in a variable instead of pidfile
t9300-fast-import: don't hang if background fast-import exits too
early
t/t9300-fast-import.sh | 20 ++++++++++++++++---- 1 file changed, 16 insertions(+), 4 deletions(-)
Signature · 1 line
2.24.0.801.g241c134b8d
PATCH v2 1/2 t9300-fast-import: store the PID in a variable instead of pidfile
The five tests running 'git fast-import' in the background in 't9300-fast-import.sh' store the PID of that background process in a pidfile, to be used to check whether that background process survived each test and then to kill it in test_when_finished commands. To achieve this all these five tests run three $(cat <pidfile>) command substitutions each.
Store the PID of the background 'git fast-import' in a variable to avoid those extra processes.
--- t/t9300-fast-import.sh | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
t/t9300-fast-import.sh +3 -3
@@ -3165,12 +3165,12 @@ background_import_then_checkpoint () { rm V.output git fast-import $options <&8 >&9 & - echo $! >V.pid + fi_pid=$! # We don't mind if fast-import has already died by the time the test # ends. test_when_finished " exec 8>&-; exec 9>&-; - kill $(cat V.pid) && wait $(cat V.pid) + kill $fi_pid && wait $fi_pid true" # Start in the background to ensure we adhere strictly to (blocking)
@@ -3202,7 +3202,7 @@ background_import_then_checkpoint () { } background_import_still_running () { - if ! kill -0 "$(cat V.pid)" + if ! kill -0 "$fi_pid" then echo >&2 "background fast-import terminated too early" false
Signature · 1 line
2.24.0.801.g241c134b8d
PATCH v2 2/2 t9300-fast-import: don't hang if background fast-import exits too early
In reply to SZEDER GáborThe five tests checking 'git fast-import's checkpoint handling in 't9300-fast-import.sh', all with the prefix "V:" in their test description, can hang indefinitely if 'git fast-import' unexpectedly dies early in any of these tests.
These five tests run 'git fast-import' in the background, while feeding instructions to its standard input through a fifo (fd 8) from a background subshell, and reading and verifying its standard output through another fifo (fd 9) in the test script's main shell process. This "reading and verifying" is basically a 'while read ...' shell loop iterating until 'git fast-import' outputs the expected line, ignoring any other output. This doesn't work very well when 'git fast-import' dies before printing that particular line, because the 'read' builtin doesn't get EOF after the death of 'git fast-import', as their input and output are not connected directly but through a fifo. Consequently, that 'read' hangs waiting for the next line from the already dead 'git fast-import', leaving the test script and in turn the whole test suite hanging.
Avoid this hang by checking whether the background 'git fast-import' process exited unexpectedly early, and interrupt the 'while read' loop if it did. We have to jump through some hoops to achive that, though:
- Start the background 'git fast-import' in another background subshell, which then:
- prints the PID of that 'git fast-import' process to the fifo, to be read by the main shell process, so it will know which process to kill when the test is finished.
- waits until that 'git fast-import' process exits. If it does exit, then report its exit code, and write a message to the fifo used for 'git fast-import's standard output, thus un-block the 'read' builtin in the main shell process.
- Modify that 'while read' loop to break the loop upon seeing that message, and fail the test in the usual way.
- Once the test is finished kill that background subshell as well, and do so before killing the background 'git fast-import'. Otherwise the background 'git fast-import' and subshell processes would die racily, and if 'git fast-import' were to die sooner, then we might get some undesired and potentially confusing messages in the test's output.
--- t/t9300-fast-import.sh | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-)
t/t9300-fast-import.sh +14 -2
@@ -3164,12 +3164,21 @@ background_import_then_checkpoint () { exec 9<>V.output rm V.output - git fast-import $options <&8 >&9 & - fi_pid=$! + ( + git fast-import $options <&8 >&9 & + echo $! >&9 + wait $! + echo >&2 "background fast-import terminated too early with exit code $?" + # Un-block the read loop in the main shell process. + echo >&9 UNEXPECTED + ) & + sh_pid=$! + read fi_pid <&9 # We don't mind if fast-import has already died by the time the test # ends. test_when_finished " exec 8>&-; exec 9>&-; + kill $sh_pid && wait $sh_pid kill $fi_pid && wait $fi_pid true"
@@ -3190,6 +3199,9 @@ background_import_then_checkpoint () { then error=0 break + elif test "$output" = "UNEXPECTED" + then + break fi # otherwise ignore cruft echo >&2 "cruft: $output"
Signature · 1 line
2.24.0.801.g241c134b8d