ktest: Updates to ktest for 7.1
cover ktest: Updates to ktest for 7.1
git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-ktest.git
for-next
Head SHA1: 229517449879b3ca8ca5588593cbea6a67ba0ad2
Ricardo B. Marlière (9):
ktest: Avoid undef warning when WARNINGS_FILE is unset
ktest: Resolve LOG_FILE in test option context
ktest: Treat undefined self-reference as empty
ktest: Honor empty per-test option overrides
ktest: Run commands through list-form shell open
ktest: Stop dropping console output during power-cycle reboot
ktest: Add PRE_KTEST_DIE for PRE_KTEST failures
ktest: Run POST_KTEST hooks on failure and cancellation
ktest: Add a --dry-run mode
----
tools/testing/ktest/ktest.pl | 158 +++++++++++++++++++++++++++++----------- tools/testing/ktest/sample.conf | 6 ++ 2 files changed, 122 insertions(+), 42 deletions(-)
PATCH 1/9 ktest: Avoid undef warning when WARNINGS_FILE is unset
From: =?UTF-8?q?Ricardo=20B=2E=20Marli=C3=A8re?= <[email protected]>
check_buildlog() probes $warnings_file with -f even when WARNINGS_FILE is not configured. Perl warns about the uninitialized value and adds noise to the test log, which can hide the output we actually care about.
Check that WARNINGS_FILE is defined before testing whether the file exists.
--- tools/testing/ktest/ktest.pl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/testing/ktest/ktest.pl +1 -1
@@ -2508,7 +2508,7 @@ sub check_buildlog { my $save_no_reboot = $no_reboot; $no_reboot = 1; - if (-f $warnings_file) { + if (defined($warnings_file) && -f $warnings_file) { open(IN, $warnings_file) or dodie "Error opening $warnings_file";
Signature · 1 line
2.51.0
PATCH 2/9 ktest: Resolve LOG_FILE in test option context
In reply to Steven RostedtFrom: =?UTF-8?q?Ricardo=20B=2E=20Marli=C3=A8re?= <[email protected]>
LOG_FILE is expanded immediately after the config file is parsed with eval_option(..., -1). That uses the default context, not the same option resolution path used for tests. If LOG_FILE depends on options that are finalized per test, it can be resolved from stale values before the first test starts.
Resolve LOG_FILE through set_test_option("LOG_FILE", 1) instead so it uses the same expansion rules as the rest of the test options.
--- tools/testing/ktest/ktest.pl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/testing/ktest/ktest.pl +1 -1
@@ -4391,7 +4391,7 @@ EOF read_config $ktest_config; if (defined($opt{"LOG_FILE"})) { - $opt{"LOG_FILE"} = eval_option("LOG_FILE", $opt{"LOG_FILE"}, -1); + $opt{"LOG_FILE"} = set_test_option("LOG_FILE", 1); } # Append any configs entered in manually to the config file.
Signature · 1 line
2.51.0
PATCH 3/9 ktest: Treat undefined self-reference as empty
In reply to Steven RostedtFrom: =?UTF-8?q?Ricardo=20B=2E=20Marli=C3=A8re?= <[email protected]>
Config variables are expanded when they are assigned. A first-time append such as:
VAR := ${VAR} fooleaves the literal ${VAR} in the stored value because VAR has not been defined yet. Later expansions then carry the self-reference forward instead of behaving like an empty prefix.
Drop an unescaped self-reference when the variable has no current value, and trim the outer whitespace left behind. Keep escaped \${VAR} references unchanged so literal text still works.
--- tools/testing/ktest/ktest.pl | 8 ++++++++ 1 file changed, 8 insertions(+)
tools/testing/ktest/ktest.pl +8 -0
@@ -910,6 +910,14 @@ sub set_variable { if (defined($command_tmp_vars{$lvalue})) { return; } + + # If a variable is undefined, treat an unescaped self-reference as empty. + if (!defined($variable{$lvalue})) { + $rvalue =~ s/(?<!\\)\$\{\Q$lvalue\E\}//g; + $rvalue =~ s/^\s+//; + $rvalue =~ s/\s+$//; + } + if ($rvalue =~ /^\s*$/) { delete $variable{$lvalue}; } else {
Signature · 1 line
2.51.0
PATCH 4/9 ktest: Honor empty per-test option overrides
In reply to Steven RostedtFrom: =?UTF-8?q?Ricardo=20B=2E=20Marli=C3=A8re?= <[email protected]>
A per-test override can clear an inherited default option by assigning an empty value, but __set_test_option() still used option_defined() to decide whether a per-test key existed. That turned an empty per-test assignment back into "fall back to the default", so tests still could not clear inherited settings.
For example:
DEFAULTS
(...)
LOG_FILE = /tmp/ktest-empty-override.log
CLEAR_LOG = 1
ADD_CONFIG = /tmp/.config
TEST_START
TEST_TYPE = build
BUILD_TYPE = nobuild
ADD_CONFIG =
This would run the test with ADD_CONFIG[1] = /tmp/.config
Fix by checking whether the per-test key exists before falling back. If it does exist but is empty, treat it as unset for that test and stop the fallback chain there.
--- tools/testing/ktest/ktest.pl | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
tools/testing/ktest/ktest.pl +4 -2
@@ -4191,7 +4191,8 @@ sub __set_test_option { my $option = "$name\[$i\]"; - if (option_defined($option)) { + if (exists($opt{$option})) { + return undef if (!option_defined($option)); return $opt{$option}; }
@@ -4199,7 +4200,8 @@ sub __set_test_option { if ($i >= $test && $i < $test + $repeat_tests{$test}) { $option = "$name\[$test\]"; - if (option_defined($option)) { + if (exists($opt{$option})) { + return undef if (!option_defined($option)); return $opt{$option}; } }
Signature · 1 line
2.51.0
PATCH 5/9 ktest: Run commands through list-form shell open
In reply to Steven RostedtFrom: =?UTF-8?q?Ricardo=20B=2E=20Marli=C3=A8re?= <[email protected]>
run_command() currently uses string-form open():
open(CMD, "$command 2>&1 |")
That delegates parsing to the shell but also mixes the stderr redirection into the command string. Switch to list-form open() with an explicit sh -c wrapper so shell syntax errors are captured in the same output stream as command output. Otherwise, important errors can not be retrieved from the ktest LOG_FILE.
--- tools/testing/ktest/ktest.pl | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
tools/testing/ktest/ktest.pl +4 -1
@@ -1921,7 +1921,10 @@ sub run_command { doprint("$command ... "); $start_time = time; - $pid = open(CMD, "$command 2>&1 |") or + $pid = open(CMD, "-|", + "sh", "-c", + 'command=$1; shift; exec 2>&1; eval "$command"', + "sh", $command) or (fail "unable to exec $command" and return 0); if (defined($opt{"LOG_FILE"})) {
Signature · 1 line
2.51.0
PATCH 6/9 ktest: Stop dropping console output during power-cycle reboot
In reply to Steven RostedtFrom: =?UTF-8?q?Ricardo=20B=2E=20Marli=C3=A8re?= <[email protected]>
The POWER_CYCLE fallback added to reboot() flushes monitor output at the wrong time. In the untimed reboot path, flushing immediately after start_monitor() can consume the first output from the new boot before monitor() begins reading it. In the timed path, flushing after POWER_CYCLE can eat the "Linux version" banner or REBOOT_SUCCESS_LINE from the new kernel.
That makes ktest miss the boot it is waiting for and can trigger an unnecessary second power cycle.
Start the monitor before POWER_CYCLE so the reference counting stays balanced, but only flush when reboot() was asked to wait for a timed reboot. Perform that flush before issuing POWER_CYCLE so it drains stale output from the old kernel instead of consuming the next boot.
--- tools/testing/ktest/ktest.pl | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-)
tools/testing/ktest/ktest.pl +6 -5
@@ -1499,12 +1499,13 @@ sub reboot { } if ($powercycle) { - run_command "$power_cycle"; - start_monitor; - # flush out current monitor - # May contain the reboot success line - wait_for_monitor 1; + if (defined($time)) { + # Flush stale console output from the old kernel before power-cycling. + wait_for_monitor 1; + } + + run_command "$power_cycle"; } else { # Make sure everything has been written to disk
Signature · 1 line
2.51.0
PATCH 7/9 ktest: Add PRE_KTEST_DIE for PRE_KTEST failures
In reply to Steven RostedtFrom: =?UTF-8?q?Ricardo=20B=2E=20Marli=C3=A8re?= <[email protected]>
PRE_KTEST runs before the first test, but its return status is currently ignored. A failing setup hook can leave the rest of the run executing in a partially initialized environment.
Add PRE_KTEST_DIE so PRE_KTEST can fail the run in the same way PRE_BUILD_DIE and PRE_TEST_DIE already can. Keep the default behavior unchanged when the new option is not set.
--- tools/testing/ktest/ktest.pl | 8 +++++++- tools/testing/ktest/sample.conf | 6 ++++++ 2 files changed, 13 insertions(+), 1 deletion(-)
tools/testing/ktest/ktest.pl +7 -1
@@ -101,6 +101,7 @@ my $build_type; my $build_options; my $final_post_ktest; my $pre_ktest; +my $pre_ktest_die; my $post_ktest; my $pre_test; my $pre_test_die;
@@ -283,6 +284,7 @@ my %option_map = ( "BUILD_DIR" => \$builddir, "TEST_TYPE" => \$test_type, "PRE_KTEST" => \$pre_ktest, + "PRE_KTEST_DIE" => \$pre_ktest_die, "POST_KTEST" => \$post_ktest, "PRE_TEST" => \$pre_test, "PRE_TEST_DIE" => \$pre_test_die,
@@ -4506,7 +4508,11 @@ for (my $i = 1; $i <= $opt{"NUM_TESTS"}; $i++) { if ($i == 1) { if (defined($pre_ktest)) { doprint "\n"; - run_command $pre_ktest; + my $ret = run_command $pre_ktest; + if (!$ret && defined($pre_ktest_die) && + $pre_ktest_die) { + dodie "failed to pre_ktest\n"; + } } if ($email_when_started) { my $name = get_test_name;
tools/testing/ktest/sample.conf +6 -0
@@ -494,6 +494,12 @@ # # default (undefined) #PRE_KTEST = ${SSH} ~/set_up_test +# +# To specify if the test should fail if PRE_KTEST fails, +# PRE_KTEST_DIE needs to be set to 1. Otherwise the PRE_KTEST +# result is ignored. +# (default 0) +#PRE_KTEST_DIE = 1 # If you want to execute some command after all the tests have # completed, you can set this option. Note, it can be set as a
Signature · 1 line
2.51.0
PATCH 8/9 ktest: Run POST_KTEST hooks on failure and cancellation
In reply to Steven RostedtFrom: =?UTF-8?q?Ricardo=20B=2E=20Marli=C3=A8re?= <[email protected]>
PRE_KTEST can be useful for setting up the environment and POST_KTEST to tear it down, however POST_KTEST only runs on the normal end-of-run path. It is skipped when ktest exits through dodie() or cancel_test(). Final cleanup hooks are skipped.
Factor the final hook execution into run_post_ktest(), call it from the normal exit path and from the early exit paths, and guard it so the hook runs at most once.
--- tools/testing/ktest/ktest.pl | 27 ++++++++++++++++++++++----- 1 file changed, 22 insertions(+), 5 deletions(-)
tools/testing/ktest/ktest.pl +22 -5
@@ -100,6 +100,7 @@ my $test_type; my $build_type; my $build_options; my $final_post_ktest; +my $post_ktest_done = 0; my $pre_ktest; my $pre_ktest_die; my $post_ktest;
@@ -1586,6 +1587,24 @@ sub get_test_name() { return $name; } +sub run_post_ktest { + my $cmd; + + return if ($post_ktest_done); + + if (defined($final_post_ktest)) { + $cmd = $final_post_ktest; + } elsif (defined($post_ktest)) { + $cmd = $post_ktest; + } else { + return; + } + + my $cp_post_ktest = eval_kernel_version($cmd); + run_command $cp_post_ktest; + $post_ktest_done = 1; +} + sub dodie { # avoid recursion return if ($in_die);
@@ -1645,6 +1664,7 @@ sub dodie { if (defined($post_test)) { run_command $post_test; } + run_post_ktest; die @_, "\n"; }
@@ -4314,6 +4334,7 @@ sub cancel_test { send_email("KTEST: Your [$name] test was cancelled", "Your test started at $script_start_time was cancelled: sig int"); } + run_post_ktest; die "\nCaught Sig Int, test interrupted: $!\n" }
@@ -4679,11 +4700,7 @@ for (my $i = 1; $i <= $opt{"NUM_TESTS"}; $i++) { success $i; } -if (defined($final_post_ktest)) { - - my $cp_final_post_ktest = eval_kernel_version $final_post_ktest; - run_command $cp_final_post_ktest; -} +run_post_ktest; if ($opt{"POWEROFF_ON_SUCCESS"}) { halt;
Signature · 1 line
2.51.0
PATCH 9/9 ktest: Add a --dry-run mode
In reply to Steven RostedtFrom: =?UTF-8?q?Ricardo=20B=2E=20Marli=C3=A8re?= <[email protected]>
When working on a ktest configuration, it is often useful to inspect the final option values after includes, defaults, per-test overrides, and variable expansion have been applied, without actually starting a test run.
Add a --dry-run option that reads the configuration, prints the test preamble using resolved option values, and exits before opening LOG_FILE or executing any test logic.
This is useful for debugging ktest configurations and for scripts that need to validate the final resolved settings without triggering side effects.
--- tools/testing/ktest/ktest.pl | 89 +++++++++++++++++++++++++----------- 1 file changed, 63 insertions(+), 26 deletions(-)
tools/testing/ktest/ktest.pl +63 -26
@@ -85,6 +85,7 @@ my %default = ( ); my $test_log_start = 0; +my $dry_run = 0; my $ktest_config = "ktest.conf"; my $version;
@@ -587,7 +588,7 @@ sub end_monitor; sub wait_for_monitor; sub _logit { - if (defined($opt{"LOG_FILE"})) { + if (defined($opt{"LOG_FILE"}) && defined(fileno(LOG))) { print LOG @_; } }
@@ -1365,6 +1366,9 @@ sub read_config { print "$option\n"; } print "Set IGNORE_UNUSED = 1 to have ktest ignore unused variables\n"; + if ($dry_run) { + return; + } if (!read_yn "Do you want to continue?") { exit -1; }
@@ -4249,6 +4253,53 @@ sub set_test_option { return eval_option($name, $option, $i); } +sub print_test_preamble { + my ($resolved) = @_; + + doprint "\n\nSTARTING AUTOMATED TESTS\n\n"; + + for (my $i = 0, my $repeat = 1; $i <= $opt{"NUM_TESTS"}; $i += $repeat) { + + if (!$i) { + doprint "DEFAULT OPTIONS:\n"; + } else { + doprint "\nTEST $i OPTIONS"; + if (defined($repeat_tests{$i})) { + $repeat = $repeat_tests{$i}; + doprint " ITERATE $repeat"; + } + doprint "\n"; + } + + foreach my $option (sort keys %opt) { + my $value; + + if ($option =~ /\[(\d+)\]$/) { + next if ($i != $1); + + if ($resolved) { + my $name = $option; + $name =~ s/\[\d+\]$//; + $value = set_test_option($name, $i); + } else { + $value = $opt{$option}; + } + } else { + next if ($i); + + if ($resolved) { + $value = set_test_option($option, 0); + } else { + $value = $opt{$option}; + } + } + + $value = "" if (!defined($value)); + doprint "$option = $value\n"; + } + } +} + sub find_mailer { my ($mailer) = @_;
@@ -4348,6 +4399,8 @@ ktest.pl version: $VERSION Sets global BUILD_NOCLEAN to 1 -D TEST_TYPE[2]=build Sets TEST_TYPE of test 2 to "build" + --dry-run + Print resolved test options and exit without running tests. It can also override all temp variables. -D USE_TEMP_DIR:=1
@@ -4379,6 +4432,9 @@ while ( $#ARGV >= 0 ) { } else { $command_vars[$#command_vars + 1] = $val; } + } elsif ( $ARGV[0] eq "--dry-run" ) { + $dry_run = 1; + shift; } elsif ( $ARGV[0] eq "-h" ) { die_usage; } else {
@@ -4427,6 +4483,11 @@ EOF } read_config $ktest_config; +if ($dry_run) { + print_test_preamble 1; + exit 0; +} + if (defined($opt{"LOG_FILE"})) { $opt{"LOG_FILE"} = set_test_option("LOG_FILE", 1); }
@@ -4458,31 +4519,7 @@ if (defined($opt{"LOG_FILE"})) { LOG->autoflush(1); } -doprint "\n\nSTARTING AUTOMATED TESTS\n\n"; - -for (my $i = 0, my $repeat = 1; $i <= $opt{"NUM_TESTS"}; $i += $repeat) { - - if (!$i) { - doprint "DEFAULT OPTIONS:\n"; - } else { - doprint "\nTEST $i OPTIONS"; - if (defined($repeat_tests{$i})) { - $repeat = $repeat_tests{$i}; - doprint " ITERATE $repeat"; - } - doprint "\n"; - } - - foreach my $option (sort keys %opt) { - if ($option =~ /\[(\d+)\]$/) { - next if ($i != $1); - } else { - next if ($i); - } - - doprint "$option = $opt{$option}\n"; - } -} +print_test_preamble 0; $SIG{INT} = qw(cancel_test);
Signature · 1 line
2.51.0