test-lib: add the '--stress' option to help reproduce occasional failures in flaky tests

To recap: this patch series tries to make reproducing rare failures in flaky tests easier: it adds the '--stress' option to our test library to run the test script repeatedly in multiple parallel jobs, in the hope that the increased load creates enough variance in the timing of the test's commands that such a failure is eventually triggered.

There is an issue with v3: by parsing all command line options before re-executing the test script for '--tee' or '--verbose-log', we not only parse the options twice (which is harmless), but check the shell's version for untraceable test scripts twice as well. Consequently, the warning about ignoring '-x' are issued twice:

  $ ./t1510-repo-setup.sh -x --tee
  warning: ignoring -x; './t1510-repo-setup.sh' is untraceable without BASH_XTRACEFD
  warning: ignoring -x; './t1510-repo-setup.sh' is untraceable without BASH_XTRACEFD
  ok 1 - #0: nonbare repo, no explicit configuration
  <...>

Furthermore, when TEST_SHELL_PATH is not /bin/sh but is set to Bash v4.1 or later, then the above command issues one warning as it's run with /bin/sh from the shebang line, but then re-execs itself with Bash and does print the '-x' tracing output as well.

This version of the patch series fixes the above issue by checking the shell's version after '--tee' and '--verbose-log' re-execue the test script. To do so it makes more sense to move the 'test-lib: extract Bash version check for '-x' tracing' patch to the beginning of the series. This move, however, made the range-diff quite noisy, hence the interdiff below, as it describes the changes since v3 much better.

It's based on 'sg/test-bash-version-fix'.

SZEDER Gábor (8):
test-lib: translate SIGTERM and SIGHUP to an exit
test-lib: extract Bash version check for '-x' tracing
test-lib: parse options in a for loop to keep $@ intact
test-lib: parse command line options earlier
test-lib: consolidate naming of test-results paths
test-lib: set $TRASH_DIRECTORY earlier
test-lib-functions: introduce the 'test_set_port' helper function
test-lib: add the '--stress' option to run a test repeatedly under
load

 t/README                 |  19 +-
 t/lib-git-daemon.sh      |   2 +-
 t/lib-git-p4.sh          |   9 +-
 t/lib-git-svn.sh         |   2 +-
 t/lib-httpd.sh           |   2 +-
 t/t0410-partial-clone.sh |   1 -
 t/t5512-ls-remote.sh     |   2 +-
 t/test-lib-functions.sh  |  39 +++++
 t/test-lib.sh            | 365 ++++++++++++++++++++++++++-------------
 9 files changed, 308 insertions(+), 133 deletions(-)

Interdiff:

index 3f0b0c07df..a1abb1177a 100644
@@ -180,29 +180,6 @@ then
 	immediate=t
 fi
 
-if test -n "$trace" && test -n "$test_untraceable"
-then
-	# '-x' tracing requested, but this test script can't be reliably
-	# traced, unless it is run with a Bash version supporting
-	# BASH_XTRACEFD (introduced in Bash v4.1).
-	if test -n "$BASH_VERSION" && eval '
-	     test ${BASH_VERSINFO[0]} -gt 4 || {
-	       test ${BASH_VERSINFO[0]} -eq 4 &&
-	       test ${BASH_VERSINFO[1]} -ge 1
-	     }
-	   '
-	then
-		: Executed by a Bash version supporting BASH_XTRACEFD.  Good.
-	else
-		echo >&2 "warning: ignoring -x; '$0' is untraceable without BASH_XTRACEFD"
-		trace=
-	fi
-fi
-if test -n "$trace" && test -z "$verbose_log"
-then
-	verbose=t
-fi
-
 TEST_STRESS_JOB_SFX="${GIT_TEST_STRESS_JOB_NR:+.stress-$GIT_TEST_STRESS_JOB_NR}"
 TEST_NAME="$(basename "$0" .sh)"
 TEST_RESULTS_DIR="$TEST_OUTPUT_DIRECTORY/test-results"

@@ -322,6 +299,34 @@ then
 	exit
 fi
 
+if test -n "$trace" && test -n "$test_untraceable"
+then
+	# '-x' tracing requested, but this test script can't be reliably
+	# traced, unless it is run with a Bash version supporting
+	# BASH_XTRACEFD (introduced in Bash v4.1).
+	#
+	# Perform this version check _after_ the test script was
+	# potentially re-executed with $TEST_SHELL_PATH for '--tee' or
+	# '--verbose-log', so the right shell is checked and the
+	# warning is issued only once.
+	if test -n "$BASH_VERSION" && eval '
+	     test ${BASH_VERSINFO[0]} -gt 4 || {
+	       test ${BASH_VERSINFO[0]} -eq 4 &&
+	       test ${BASH_VERSINFO[1]} -ge 1
+	     }
+	   '
+	then
+		: Executed by a Bash version supporting BASH_XTRACEFD.  Good.
+	else
+		echo >&2 "warning: ignoring -x; '$0' is untraceable without BASH_XTRACEFD"
+		trace=
+	fi
+fi
+if test -n "$trace" && test -z "$verbose_log"
+then
+	verbose=t
+fi
+
 # For repeatability, reset the environment to known value.
 # TERM is sanitized below, after saving color control sequences.
 LANG=C

2.20.1.151.gec613c4b75