Skip to contentSkip to conversation

Torture-test scripting updates for v6.2

PATCH New Paul E. McKenney 1 participant 9 messages Oct 19, 2022

Subsystems (3)
lore
Share & tools
lore

Paul E. McKenney

author maintainer
#0 lore raw

cover Torture-test scripting updates for v6.2

Hello!

This series provides torture-test scripting updates, perhaps most notably elimination of redundant builds by torture.sh.

1. Add --bootargs parameter to kvm-again.sh.

2. Use mktemp instead of guessing at unique names.

3. Make kvm-test-1-run-qemu.sh check for alternative output.

4. Make kvm-recheck.sh export TORTURE_SUITE.

5. Add --datestamp parameter to kvm-again.sh.

6. Avoid redundant builds for rcuscale and refscale in torture.sh.

7. Avoid torture.sh compressing identical files.

8. Make torture.sh create a properly formatted log file.

Thanx, Paul

------------------------------------------------------------------------

 b/tools/testing/selftests/rcutorture/bin/config2csv.sh           |    3 
 b/tools/testing/selftests/rcutorture/bin/config_override.sh      |    3 
 b/tools/testing/selftests/rcutorture/bin/configcheck.sh          |    3 
 b/tools/testing/selftests/rcutorture/bin/configinit.sh           |    3 
 b/tools/testing/selftests/rcutorture/bin/kvm-again.sh            |    9 
 b/tools/testing/selftests/rcutorture/bin/kvm-assign-cpus.sh      |    3 
 b/tools/testing/selftests/rcutorture/bin/kvm-build.sh            |    3 
 b/tools/testing/selftests/rcutorture/bin/kvm-end-run-stats.sh    |    3 
 b/tools/testing/selftests/rcutorture/bin/kvm-recheck.sh          |    2 
 b/tools/testing/selftests/rcutorture/bin/kvm-remote.sh           |   13 
 b/tools/testing/selftests/rcutorture/bin/kvm-test-1-run-batch.sh |    3 
 b/tools/testing/selftests/rcutorture/bin/kvm-test-1-run-qemu.sh  |    3 
 b/tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh       |    3 
 b/tools/testing/selftests/rcutorture/bin/kvm-transform.sh        |   68 ++++
 b/tools/testing/selftests/rcutorture/bin/kvm.sh                  |    3 
 b/tools/testing/selftests/rcutorture/bin/parse-build.sh          |    3 
 b/tools/testing/selftests/rcutorture/bin/torture.sh              |    3 
 tools/testing/selftests/rcutorture/bin/kvm-again.sh              |   40 ++
 tools/testing/selftests/rcutorture/bin/kvm-test-1-run-qemu.sh    |    2 
 tools/testing/selftests/rcutorture/bin/torture.sh                |  140 +++++++++-
 20 files changed, 247 insertions(+), 66 deletions(-)
author maintainer
#1 lore raw

PATCH v3 1/8 rcutorture: Add --bootargs parameter to kvm-again.sh

The kvm-again.sh script can be used to repeat short boot-time tests, but the kernel boot arguments cannot be changed. This means that every change in kernel boot arguments currently necessitates a kernel build, which greatly increases the duration of kernel-boot testing.

This commit therefore adds a --bootargs parameter to kvm-again.sh, which allows a given kernel to be repeatedly booted, but overriding old and adding new kernel boot parameters. This allows an old kernel to be booted with new kernel boot parameters, avoiding the overhead of rebuilding the kernel under test.

---
 .../selftests/rcutorture/bin/kvm-again.sh     |  9 ++-
 .../selftests/rcutorture/bin/kvm-transform.sh | 68 ++++++++++++++++---
 2 files changed, 65 insertions(+), 12 deletions(-)
tools/testing/selftests/rcutorture/bin/kvm-again.sh +8 -1
index 0941f1ddab658..85cfd139f9737 100755
@@ -51,6 +51,7 @@ RCUTORTURE="`pwd`/tools/testing/selftests/rcutorture"; export RCUTORTURE
 PATH=${RCUTORTURE}/bin:$PATH; export PATH
 . functions.sh
 
+bootargs=
 dryrun=
 dur=
 default_link="cp -R"
@@ -61,6 +62,7 @@ starttime="`get_starttime`"
 
 usage () {
 	echo "Usage: $scriptname $oldrun [ arguments ]:"
+	echo "       --bootargs kernel-boot-arguments"
 	echo "       --dryrun"
 	echo "       --duration minutes | <seconds>s | <hours>h | <days>d"
 	echo "       --link hard|soft|copy"
@@ -72,6 +74,11 @@ usage () {
 while test $# -gt 0
 do
 	case "$1" in
+	--bootargs|--bootarg)
+		checkarg --bootargs "(list of kernel boot arguments)" "$#" "$2" '.*' '^--'
+		bootargs="$bootargs $2"
+		shift
+		;;
 	--dryrun)
 		dryrun=1
 		;;
@@ -156,7 +163,7 @@ do
 	qemu_cmd_dir="`dirname "$i"`"
 	kernel_dir="`echo $qemu_cmd_dir | sed -e 's/\.[0-9]\+$//'`"
 	jitter_dir="`dirname "$kernel_dir"`"
-	kvm-transform.sh "$kernel_dir/bzImage" "$qemu_cmd_dir/console.log" "$jitter_dir" $dur < $T/qemu-cmd > $i
+	kvm-transform.sh "$kernel_dir/bzImage" "$qemu_cmd_dir/console.log" "$jitter_dir" $dur "$bootargs" < $T/qemu-cmd > $i
 	if test -n "$arg_remote"
 	then
 		echo "# TORTURE_KCONFIG_GDB_ARG=''" >> $i
tools/testing/selftests/rcutorture/bin/kvm-transform.sh +57 -11

102 lines not on this page · raw message

Signature · 1 line
2.31.1.189.g2e36527f23
author maintainer
#2 lore raw

PATCH v3 2/8 torture: Use mktemp instead of guessing at unique names

In reply to Paul E. McKenney

This commit drags the rcutorture scripting kicking and screaming into the twenty-first century by making use of the BSD-derived mktemp command to create temporary files and directories. In happy contrast to many of its ill-behaved predecessors, mktemp seems to actually work reasonably reliably!

---
 .../testing/selftests/rcutorture/bin/config2csv.sh  |  3 +--
 .../selftests/rcutorture/bin/config_override.sh     |  3 +--
 .../testing/selftests/rcutorture/bin/configcheck.sh |  3 +--
 .../testing/selftests/rcutorture/bin/configinit.sh  |  3 +--
 tools/testing/selftests/rcutorture/bin/kvm-again.sh |  3 +--
 .../selftests/rcutorture/bin/kvm-assign-cpus.sh     |  3 +--
 tools/testing/selftests/rcutorture/bin/kvm-build.sh |  3 +--
 .../selftests/rcutorture/bin/kvm-end-run-stats.sh   |  3 +--
 .../testing/selftests/rcutorture/bin/kvm-remote.sh  | 13 ++++++-------
 .../rcutorture/bin/kvm-test-1-run-batch.sh          |  3 +--
 .../selftests/rcutorture/bin/kvm-test-1-run-qemu.sh |  3 +--
 .../selftests/rcutorture/bin/kvm-test-1-run.sh      |  3 +--
 tools/testing/selftests/rcutorture/bin/kvm.sh       |  3 +--
 .../testing/selftests/rcutorture/bin/parse-build.sh |  3 +--
 tools/testing/selftests/rcutorture/bin/torture.sh   |  3 +--
 15 files changed, 20 insertions(+), 35 deletions(-)
tools/testing/selftests/rcutorture/bin/config2csv.sh +1 -2
index d5a16631b16ee..0cf55f1bf6548 100755
@@ -30,9 +30,8 @@ else
 fi
 scenarios="`echo $scenariosarg | sed -e "s/\<CFLIST\>/$defaultconfigs/g"`"
 
-T=/tmp/config2latex.sh.$$
+T=`mktemp -d /tmp/config2latex.sh.XXXXXX`
 trap 'rm -rf $T' 0
-mkdir $T
 
 cat << '---EOF---' >> $T/p.awk
 END	{
tools/testing/selftests/rcutorture/bin/config_override.sh +1 -2
index 90016c359e839..b3d2e7efa40cd 100755
@@ -29,9 +29,8 @@ else
 	exit 1
 fi
 
-T=${TMPDIR-/tmp}/config_override.sh.$$
+T="`mktemp -d ${TMPDIR-/tmp}/config_override.sh.XXXXXX`"
 trap 'rm -rf $T' 0
-mkdir $T
 
 sed < $override -e 's/^/grep -v "/' -e 's/=.*$/="/' |
 	awk '
tools/testing/selftests/rcutorture/bin/configcheck.sh +1 -2
index 31584cee84d71..83fac1852ab23 100755
@@ -7,9 +7,8 @@
 #
 # Authors: Paul E. McKenney <[email protected]>
 
-T=${TMPDIR-/tmp}/abat-chk-config.sh.$$
+T="`mktemp -d ${TMPDIR-/tmp}/configcheck.sh.XXXXXX`"
 trap 'rm -rf $T' 0
-mkdir $T
 
 cat $1 > $T/.config
 
tools/testing/selftests/rcutorture/bin/configinit.sh +1 -2
index d6e5ce084b1cf..28bdb3ac7ba6f 100755
@@ -15,9 +15,8 @@
 #
 # Authors: Paul E. McKenney <[email protected]>
 
-T=${TMPDIR-/tmp}/configinit.sh.$$
+T="`mktemp -d ${TMPDIR-/tmp}/configinit.sh.XXXXXX`"
 trap 'rm -rf $T' 0
-mkdir $T
 
 # Capture config spec file.
 
tools/testing/selftests/rcutorture/bin/kvm-again.sh +1 -2
index 85cfd139f9737..20941c1051087 100755
@@ -12,9 +12,8 @@
 scriptname=$0
 args="$*"
 
-T=${TMPDIR-/tmp}/kvm-again.sh.$$
+T="`mktemp -d ${TMPDIR-/tmp}/kvm-again.sh.XXXXXX`"
 trap 'rm -rf $T' 0
-mkdir $T
 
 if ! test -d tools/testing/selftests/rcutorture/bin
 then
tools/testing/selftests/rcutorture/bin/kvm-assign-cpus.sh +1 -2
index f99b2c146f835..46b08cd16ba5c 100755
@@ -7,9 +7,8 @@
 #
 # Usage: kvm-assign-cpus.sh /path/to/sysfs
 
-T=/tmp/kvm-assign-cpus.sh.$$
+T="`mktemp -d ${TMPDIR-/tmp}/kvm-assign-cpus.sh.XXXXXX`"
 trap 'rm -rf $T' 0 2
-mkdir $T
 
 sysfsdir=${1-/sys/devices/system/node}
 if ! cd "$sysfsdir" > $T/msg 2>&1
tools/testing/selftests/rcutorture/bin/kvm-build.sh +1 -2
index 5ad973dca8207..e28a82851f7c4 100755
@@ -23,9 +23,8 @@ then
 fi
 resdir=${2}
 
-T=${TMPDIR-/tmp}/test-linux.sh.$$
+T="`mktemp -d ${TMPDIR-/tmp}/kvm-build.sh.XXXXXX`"
 trap 'rm -rf $T' 0
-mkdir $T
 
 cp ${config_template} $T/config
 cat << ___EOF___ >> $T/config
tools/testing/selftests/rcutorture/bin/kvm-end-run-stats.sh +1 -2
index ee886b40a5d2c..2b56baceb05d7 100755
@@ -18,9 +18,8 @@ then
 	exit 1
 fi
 
-T=${TMPDIR-/tmp}/kvm-end-run-stats.sh.$$
+T="`mktemp -d ${TMPDIR-/tmp}/kvm-end-run-stats.sh.XXXXXX`"
 trap 'rm -rf $T' 0
-mkdir $T
 
 RCUTORTURE="`pwd`/tools/testing/selftests/rcutorture"; export RCUTORTURE
 PATH=${RCUTORTURE}/bin:$PATH; export PATH
tools/testing/selftests/rcutorture/bin/kvm-remote.sh +6 -7
index 9f0a5d5ff2ddc..a2328163eba1d 100755
@@ -34,19 +34,18 @@ fi
 shift
 
 # Pathnames:
-# T:	  /tmp/kvm-remote.sh.$$
-# resdir: /tmp/kvm-remote.sh.$$/res
-# rundir: /tmp/kvm-remote.sh.$$/res/$ds ("-remote" suffix)
+# T:	  /tmp/kvm-remote.sh.NNNNNN where "NNNNNN" is set by mktemp
+# resdir: /tmp/kvm-remote.sh.NNNNNN/res
+# rundir: /tmp/kvm-remote.sh.NNNNNN/res/$ds ("-remote" suffix)
 # oldrun: `pwd`/tools/testing/.../res/$otherds
 #
 # Pathname segments:
-# TD:	  kvm-remote.sh.$$
+# TD:	  kvm-remote.sh.NNNNNN
 # ds:	  yyyy.mm.dd-hh.mm.ss-remote
 
-TD=kvm-remote.sh.$$
-T=${TMPDIR-/tmp}/$TD
+T="`mktemp -d ${TMPDIR-/tmp}/kvm-remote.sh.XXXXXX`"
 trap 'rm -rf $T' 0
-mkdir $T
+TD="`basename "$T"`"
 
 resdir="$T/res"
 ds=`date +%Y.%m.%d-%H.%M.%S`-remote
tools/testing/selftests/rcutorture/bin/kvm-test-1-run-batch.sh +1 -2
index 1e29d656501bc..c3808c490d92d 100755
@@ -13,9 +13,8 @@
 #
 # Authors: Paul E. McKenney <[email protected]>
 
-T=${TMPDIR-/tmp}/kvm-test-1-run-batch.sh.$$
+T="`mktemp -d ${TMPDIR-/tmp}/kvm-test-1-run-batch.sh.XXXXXX`"
 trap 'rm -rf $T' 0
-mkdir $T
 
 echo ---- Running batch $*
 # Check arguments
tools/testing/selftests/rcutorture/bin/kvm-test-1-run-qemu.sh +1 -2
index 44280582c594e..9da86e6cd1812 100755
@@ -17,9 +17,8 @@
 #
 # Authors: Paul E. McKenney <[email protected]>
 
-T=${TMPDIR-/tmp}/kvm-test-1-run-qemu.sh.$$
+T="`mktemp -d ${TMPDIR-/tmp}/kvm-test-1-run-qemu.sh.XXXXXX`"
 trap 'rm -rf $T' 0
-mkdir $T
 
 resdir="$1"
 if ! test -d "$resdir"
tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh +1 -2
index f4c8055dbf7ad..d2a3710a5f2ad 100755
@@ -25,9 +25,8 @@
 #
 # Authors: Paul E. McKenney <[email protected]>
 
-T=${TMPDIR-/tmp}/kvm-test-1-run.sh.$$
+T="`mktemp -d ${TMPDIR-/tmp}/kvm-test-1-run.sh.XXXXXX`"
 trap 'rm -rf $T' 0
-mkdir $T
 
 . functions.sh
 . $CONFIGFRAG/ver_functions.sh
tools/testing/selftests/rcutorture/bin/kvm.sh +1 -2
index 6c734818a8757..7710b1e1cddab 100755
@@ -14,9 +14,8 @@
 scriptname=$0
 args="$*"
 
-T=${TMPDIR-/tmp}/kvm.sh.$$
+T="`mktemp -d ${TMPDIR-/tmp}/kvm.sh.XXXXXX`"
 trap 'rm -rf $T' 0
-mkdir $T
 
 cd `dirname $scriptname`/../../../../../
 
tools/testing/selftests/rcutorture/bin/parse-build.sh +1 -2
index 2dbfca3589b17..5a0b7ffcf047a 100755
@@ -15,9 +15,8 @@
 
 F=$1
 title=$2
-T=${TMPDIR-/tmp}/parse-build.sh.$$
+T="`mktemp -d ${TMPDIR-/tmp}/parse-build.sh.XXXXXX`"
 trap 'rm -rf $T' 0
-mkdir $T
 
 . functions.sh
 
tools/testing/selftests/rcutorture/bin/torture.sh +1 -2
index d477618e7261d..4d285db2fbbdd 100755
@@ -206,9 +206,8 @@ ds="`date +%Y.%m.%d-%H.%M.%S`-torture"
 startdate="`date`"
 starttime="`get_starttime`"
 
-T=/tmp/torture.sh.$$
+T="`mktemp -d ${TMPDIR-/tmp}/torture.sh.XXXXXX`"
 trap 'rm -rf $T' 0 2
-mkdir $T
 
 echo " --- " $scriptname $args | tee -a $T/log
 echo " --- Results directory: " $ds | tee -a $T/log
Signature · 1 line
2.31.1.189.g2e36527f23
author maintainer
#3 lore raw

PATCH v3 3/8 rcutorture: Make kvm-test-1-run-qemu.sh check for alternative output

In reply to Paul E. McKenney

The kvm-again.sh script, when running locally, can place the QEMU output into kvm-test-1-run-qemu.sh.out instead of kvm-test-1-run.sh.out. This commit therefore makes kvm-test-1-run-qemu.sh check both locations.

---
 tools/testing/selftests/rcutorture/bin/kvm-test-1-run-qemu.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tools/testing/selftests/rcutorture/bin/kvm-test-1-run-qemu.sh +1 -1
index 9da86e6cd1812..76f24cd5825be 100755
@@ -108,7 +108,7 @@ do
 		if test $kruntime -lt $seconds
 		then
 			echo Completed in $kruntime vs. $seconds >> $resdir/Warnings 2>&1
-			grep "^(qemu) qemu:" $resdir/kvm-test-1-run.sh.out >> $resdir/Warnings 2>&1
+			grep "^(qemu) qemu:" $resdir/kvm-test-1-run*.sh.out >> $resdir/Warnings 2>&1
 			killpid="`sed -n "s/^(qemu) qemu: terminating on signal [0-9]* from pid \([0-9]*\).*$/\1/p" $resdir/Warnings`"
 			if test -n "$killpid"
 			then
Signature · 1 line
2.31.1.189.g2e36527f23
author maintainer
#4 lore raw

PATCH v3 4/8 rcutorture: Make kvm-recheck.sh export TORTURE_SUITE

In reply to Paul E. McKenney

As it should, the kvm-recheck.sh script sets the TORTURE_SUITE bash variable based on the type of rcutorture test being run. However, it does not export it. Which is OK, at least until you try running kvm-again.sh on either a rcuscale or a refscale test, at which point you get false-positive "no success message, N successful version messages" errors. This commit therefore causes the kvm-recheck.sh script to export TORTURE_SUITE, suppressing these false positives.

---
 tools/testing/selftests/rcutorture/bin/kvm-recheck.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tools/testing/selftests/rcutorture/bin/kvm-recheck.sh +1 -1
index 0789c5606d2ab..1df7e695edf75 100755
@@ -30,7 +30,7 @@ do
 			resdir=`echo $i | sed -e 's,/$,,' -e 's,/[^/]*$,,'`
 			head -1 $resdir/log
 		fi
-		TORTURE_SUITE="`cat $i/../torture_suite`"
+		TORTURE_SUITE="`cat $i/../torture_suite`" ; export TORTURE_SUITE
 		configfile=`echo $i | sed -e 's,^.*/,,'`
 		rm -f $i/console.log.*.diags
 		case "${TORTURE_SUITE}" in
Signature · 1 line
2.31.1.189.g2e36527f23
author maintainer
#5 lore raw

PATCH v3 5/8 rcutorture: Add --datestamp parameter to kvm-again.sh

In reply to Paul E. McKenney

This commit adds a --datestamp parameter to kvm-again.sh, which, in contrast to the existing --rundir argument, specifies only the last segments of the pathname. This addition enables torture.sh to use kvm-again.sh in order to avoid redundant kernel builds.

---
 .../selftests/rcutorture/bin/kvm-again.sh     | 37 +++++++++++++++++--
 1 file changed, 34 insertions(+), 3 deletions(-)
tools/testing/selftests/rcutorture/bin/kvm-again.sh +34 -3
index 20941c1051087..8a968fbda02c9 100755
@@ -54,7 +54,10 @@ bootargs=
 dryrun=
 dur=
 default_link="cp -R"
-rundir="`pwd`/tools/testing/selftests/rcutorture/res/`date +%Y.%m.%d-%H.%M.%S-again`"
+resdir="`pwd`/tools/testing/selftests/rcutorture/res"
+rundir="$resdir/`date +%Y.%m.%d-%H.%M.%S-again`"
+got_datestamp=
+got_rundir=
 
 startdate="`date`"
 starttime="`get_starttime`"
@@ -62,11 +65,13 @@ starttime="`get_starttime`"
 usage () {
 	echo "Usage: $scriptname $oldrun [ arguments ]:"
 	echo "       --bootargs kernel-boot-arguments"
+	echo "       --datestamp string"
 	echo "       --dryrun"
 	echo "       --duration minutes | <seconds>s | <hours>h | <days>d"
 	echo "       --link hard|soft|copy"
 	echo "       --remote"
 	echo "       --rundir /new/res/path"
+	echo "Command line: $scriptname $args"
 	exit 1
 }
 
@@ -78,6 +83,23 @@ do
 		bootargs="$bootargs $2"
 		shift
 		;;
+	--datestamp)
+		checkarg --datestamp "(relative pathname)" "$#" "$2" '^[a-zA-Z0-9._/-]*$' '^--'
+		if test -n "$got_rundir" || test -n "$got_datestamp"
+		then
+			echo Only one of --datestamp or --rundir may be specified
+			usage
+		fi
+		got_datestamp=y
+		ds=$2
+		rundir="$resdir/$ds"
+		if test -e "$rundir"
+		then
+			echo "--datestamp $2: Already exists."
+			usage
+		fi
+		shift
+		;;
 	--dryrun)
 		dryrun=1
 		;;
@@ -119,6 +141,12 @@ do
 		;;
 	--rundir)
 		checkarg --rundir "(absolute pathname)" "$#" "$2" '^/' '^error'
+		if test -n "$got_rundir" || test -n "$got_datestamp"
+		then
+			echo Only one of --datestamp or --rundir may be specified
+			usage
+		fi
+		got_rundir=y
 		rundir=$2
 		if test -e "$rundir"
 		then
@@ -128,8 +156,11 @@ do
 		shift
 		;;
 	*)
-		echo Unknown argument $1
-		usage
+		if test -n "$1"
+		then
+			echo Unknown argument $1
+			usage
+		fi
 		;;
 	esac
 	shift
Signature · 1 line
2.31.1.189.g2e36527f23
author maintainer
#6 lore raw

PATCH v3 6/8 rcutorture: Avoid redundant builds for rcuscale and refscale in torture.sh

In reply to Paul E. McKenney

This commit causes torture.sh to use the new --bootargs and --datestamp parameters to kvm-again.sh in order to avoid redundant kernel builds during rcuscale and refscale testing. This trims the better part of an hour off of torture.sh runs that use --do-kasan.

---
 .../selftests/rcutorture/bin/torture.sh       | 87 ++++++++++++++++++-
 1 file changed, 83 insertions(+), 4 deletions(-)
tools/testing/selftests/rcutorture/bin/torture.sh +83 -4
index 4d285db2fbbdd..b376688fe280a 100755
@@ -277,6 +277,8 @@ function torture_one {
 	then
 		cat $T/$curflavor.out | tee -a $T/log
 		echo retcode=$retcode | tee -a $T/log
+	else
+		echo $resdir > $T/last-resdir
 	fi
 	if test "$retcode" == 0
 	then
@@ -302,10 +304,12 @@ function torture_set {
 	shift
 	curflavor=$flavor
 	torture_one "$@"
+	mv $T/last-resdir $T/last-resdir-nodebug || :
 	if test "$do_kasan" = "yes"
 	then
 		curflavor=${flavor}-kasan
 		torture_one "$@" --kasan
+		mv $T/last-resdir $T/last-resdir-kasan || :
 	fi
 	if test "$do_kcsan" = "yes"
 	then
@@ -316,6 +320,7 @@ function torture_set {
 			cur_kcsan_kmake_args="$kcsan_kmake_args"
 		fi
 		torture_one "$@" --kconfig "CONFIG_DEBUG_LOCK_ALLOC=y CONFIG_PROVE_LOCKING=y" $kcsan_kmake_tag $cur_kcsan_kmake_args --kcsan
+		mv $T/last-resdir $T/last-resdir-kcsan || :
 	fi
 }
 
@@ -378,11 +383,48 @@ then
 else
 	primlist=
 fi
+firsttime=1
+do_kasan_save="$do_kasan"
+do_kcsan_save="$do_kcsan"
 for prim in $primlist
 do
-	torture_bootargs="refscale.scale_type="$prim" refscale.nreaders=$HALF_ALLOTED_CPUS refscale.loops=10000 refscale.holdoff=20 torture.disable_onoff_at_boot"
-	torture_set "refscale-$prim" tools/testing/selftests/rcutorture/bin/kvm.sh --torture refscale --allcpus --duration 5 --kconfig "CONFIG_TASKS_TRACE_RCU=y CONFIG_NR_CPUS=$HALF_ALLOTED_CPUS" --bootargs "verbose_batched=$VERBOSE_BATCH_CPUS torture.verbose_sleep_frequency=8 torture.verbose_sleep_duration=$VERBOSE_BATCH_CPUS" --trust-make
+	if test -n "$firsttime"
+	then
+		torture_bootargs="refscale.scale_type="$prim" refscale.nreaders=$HALF_ALLOTED_CPUS refscale.loops=10000 refscale.holdoff=20 torture.disable_onoff_at_boot"
+		torture_set "refscale-$prim" tools/testing/selftests/rcutorture/bin/kvm.sh --torture refscale --allcpus --duration 5 --kconfig "CONFIG_TASKS_TRACE_RCU=y CONFIG_NR_CPUS=$HALF_ALLOTED_CPUS" --bootargs "verbose_batched=$VERBOSE_BATCH_CPUS torture.verbose_sleep_frequency=8 torture.verbose_sleep_duration=$VERBOSE_BATCH_CPUS" --trust-make
+		mv $T/last-resdir-nodebug $T/first-resdir-nodebug || :
+		if test -f "$T/last-resdir-kasan"
+		then
+			mv $T/last-resdir-kasan $T/first-resdir-kasan || :
+		fi
+		if test -f "$T/last-resdir-kcsan"
+		then
+			mv $T/last-resdir-kcsan $T/first-resdir-kcsan || :
+		fi
+		firsttime=
+		do_kasan=
+		do_kcsan=
+	else
+		torture_bootargs=
+		for i in $T/first-resdir-*
+		do
+			case "$i" in
+			*-nodebug)
+				torture_suffix=
+				;;
+			*-kasan)
+				torture_suffix="-kasan"
+				;;
+			*-kcsan)
+				torture_suffix="-kcsan"
+				;;
+			esac
+			torture_set "refscale-$prim$torture_suffix" tools/testing/selftests/rcutorture/bin/kvm-again.sh "`cat "$i"`" --duration 5 --bootargs "refscale.scale_type=$prim"
+		done
+	fi
 done
+do_kasan="$do_kasan_save"
+do_kcsan="$do_kcsan_save"
 
 if test "$do_rcuscale" = yes
 then
@@ -390,11 +432,48 @@ then
 else
 	primlist=
 fi
+firsttime=1
+do_kasan_save="$do_kasan"
+do_kcsan_save="$do_kcsan"
 for prim in $primlist
 do
-	torture_bootargs="rcuscale.scale_type="$prim" rcuscale.nwriters=$HALF_ALLOTED_CPUS rcuscale.holdoff=20 torture.disable_onoff_at_boot"
-	torture_set "rcuscale-$prim" tools/testing/selftests/rcutorture/bin/kvm.sh --torture rcuscale --allcpus --duration 5 --kconfig "CONFIG_TASKS_TRACE_RCU=y CONFIG_NR_CPUS=$HALF_ALLOTED_CPUS" --trust-make
+	if test -n "$firsttime"
+	then
+		torture_bootargs="rcuscale.scale_type="$prim" rcuscale.nwriters=$HALF_ALLOTED_CPUS rcuscale.holdoff=20 torture.disable_onoff_at_boot"
+		torture_set "rcuscale-$prim" tools/testing/selftests/rcutorture/bin/kvm.sh --torture rcuscale --allcpus --duration 5 --kconfig "CONFIG_TASKS_TRACE_RCU=y CONFIG_NR_CPUS=$HALF_ALLOTED_CPUS" --trust-make
+		mv $T/last-resdir-nodebug $T/first-resdir-nodebug || :
+		if test -f "$T/last-resdir-kasan"
+		then
+			mv $T/last-resdir-kasan $T/first-resdir-kasan || :
+		fi
+		if test -f "$T/last-resdir-kcsan"
+		then
+			mv $T/last-resdir-kcsan $T/first-resdir-kcsan || :
+		fi
+		firsttime=
+		do_kasan=
+		do_kcsan=
+	else
+		torture_bootargs=
+		for i in $T/first-resdir-*
+		do
+			case "$i" in
+			*-nodebug)
+				torture_suffix=
+				;;
+			*-kasan)
+				torture_suffix="-kasan"
+				;;
+			*-kcsan)
+				torture_suffix="-kcsan"
+				;;
+			esac
+			torture_set "rcuscale-$prim$torture_suffix" tools/testing/selftests/rcutorture/bin/kvm-again.sh "`cat "$i"`" --duration 5 --bootargs "rcuscale.scale_type=$prim"
+		done
+	fi
 done
+do_kasan="$do_kasan_save"
+do_kcsan="$do_kcsan_save"
 
 if test "$do_kvfree" = "yes"
 then
Signature · 1 line
2.31.1.189.g2e36527f23
author maintainer
#7 lore raw

PATCH v3 7/8 rcutorture: Avoid torture.sh compressing identical files

In reply to Paul E. McKenney

Currently, torture.sh will compress the vmlinux files for KASAN and KCSAN runs. But it will compress all of the files, including those copied verbatim by the kvm-again.sh script. Compression takes around ten minutes, so this is not a good thing. This commit therefore compresses only one of a given set of identical vmlinux files, and then hard-links it to the directories produced by kvm-again.sh.

---
 .../selftests/rcutorture/bin/torture.sh       | 21 ++++++++++++++++++-
 1 file changed, 20 insertions(+), 1 deletion(-)
tools/testing/selftests/rcutorture/bin/torture.sh +20 -1
index b376688fe280a..fc63539377147 100755
@@ -536,7 +536,10 @@ if test -n "$tdir" && test $compress_concurrency -gt 0
 then
 	# KASAN vmlinux files can approach 1GB in size, so compress them.
 	echo Looking for K[AC]SAN files to compress: `date` > "$tdir/log-xz" 2>&1
-	find "$tdir" -type d -name '*-k[ac]san' -print > $T/xz-todo
+	find "$tdir" -type d -name '*-k[ac]san' -print > $T/xz-todo-all
+	find "$tdir" -type f -name 're-run' -print | sed -e 's,/re-run,,' |
+		grep -e '-k[ac]san$' > $T/xz-todo-copy
+	sort $T/xz-todo-all $T/xz-todo-copy | uniq -u > $T/xz-todo
 	ncompresses=0
 	batchno=1
 	if test -s $T/xz-todo
@@ -568,6 +571,22 @@ then
 			echo Waiting for final batch $batchno of $ncompresses compressions `date` | tee -a "$tdir/log-xz" | tee -a $T/log
 		fi
 		wait
+		if test -s $T/xz-todo-copy
+		then
+			echo Linking vmlinux.xz files to re-use scenarios `date` | tee -a "$tdir/log-xz" | tee -a $T/log
+			dirstash="`pwd`"
+			for i in `cat $T/xz-todo-copy`
+			do
+				cd $i
+				find "$i" -name vmlinux -print > $T/xz-todo-copy-vmlinux
+				for v in `cat $T/xz-todo-copy-vmlinux`
+				do
+					rm -f "$v"
+					cp -l "$i/$v".xz "`dirname "$v"`"
+				done
+				cd "$dirstash"
+			done
+		fi
 		echo Size after compressing $n2compress files: `du -sh $tdir | awk '{ print $1 }'` `date` 2>&1 | tee -a "$tdir/log-xz" | tee -a $T/log
 		echo Total duration `get_starttime_duration $starttime`. | tee -a $T/log
 	else
Signature · 1 line
2.31.1.189.g2e36527f23
author maintainer
#8 lore raw

PATCH v3 8/8 torture: Make torture.sh create a properly formatted log file

In reply to Paul E. McKenney

Currently, if the torture.sh allmodconfig step fails, this is counted as an error (as it should be), but there is also an extraneous complaint about a missing log file. This commit therefore adds that log file, which is hoped to reduce confused reactions to the error report.

---
 .../selftests/rcutorture/bin/torture.sh       | 32 +++++++++++++------
 1 file changed, 23 insertions(+), 9 deletions(-)
tools/testing/selftests/rcutorture/bin/torture.sh +23 -9
index fc63539377147..ec3d8a2d513f1 100755
@@ -330,20 +330,34 @@ then
 	echo " --- allmodconfig:" Start `date` | tee -a $T/log
 	amcdir="tools/testing/selftests/rcutorture/res/$ds/allmodconfig"
 	mkdir -p "$amcdir"
-	echo " --- make clean" > "$amcdir/Make.out" 2>&1
+	echo " --- make clean" | tee $amcdir/log > "$amcdir/Make.out" 2>&1
 	make -j$MAKE_ALLOTED_CPUS clean >> "$amcdir/Make.out" 2>&1
-	echo " --- make allmodconfig" >> "$amcdir/Make.out" 2>&1
-	cp .config $amcdir
-	make -j$MAKE_ALLOTED_CPUS allmodconfig >> "$amcdir/Make.out" 2>&1
-	echo " --- make " >> "$amcdir/Make.out" 2>&1
-	make -j$MAKE_ALLOTED_CPUS >> "$amcdir/Make.out" 2>&1
-	retcode="$?"
-	echo $retcode > "$amcdir/Make.exitcode"
-	if test "$retcode" == 0
+	retcode=$?
+	buildphase='"make clean"'
+	if test "$retcode" -eq 0
+	then
+		echo " --- make allmodconfig" | tee -a $amcdir/log >> "$amcdir/Make.out" 2>&1
+		cp .config $amcdir
+		make -j$MAKE_ALLOTED_CPUS allmodconfig >> "$amcdir/Make.out" 2>&1
+		retcode=$?
+		buildphase='"make allmodconfig"'
+	fi
+	if test "$retcode" -eq 0
+	then
+		echo " --- make " | tee -a $amcdir/log >> "$amcdir/Make.out" 2>&1
+		make -j$MAKE_ALLOTED_CPUS >> "$amcdir/Make.out" 2>&1
+		retcode="$?"
+		echo $retcode > "$amcdir/Make.exitcode"
+		buildphase='"make"'
+	fi
+	if test "$retcode" -eq 0
 	then
 		echo "allmodconfig($retcode)" $amcdir >> $T/successes
+		echo Success >> $amcdir/log
 	else
 		echo "allmodconfig($retcode)" $amcdir >> $T/failures
+		echo " --- allmodconfig Test summary:" >> $amcdir/log
+		echo " --- Summary: Exit code $retcode from $buildphase, see Make.out" >> $amcdir/log
 	fi
 fi
 
Signature · 1 line
2.31.1.189.g2e36527f23