Skip to contentSkip to conversation

ktest: Updates for ktest.pl

PATCH In review Steven Rostedt 2 participants 9 messages May 23, 2011 Ktest

lore
Share & tools
lore

Steven Rostedt

author maintainer
#0 lore raw

cover ktest: Updates for ktest.pl

Linus,

Please pull the following patches from:

git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-2.6-ktest.git

branch: for-linus

Steven Rostedt (6):
ktest: Fix post install command
ktest: If test failed due to timeout, print that
ktest: Reboot to good kernel after every bisect run
ktest: Reboot after each patchcheck run
ktest: Create variables for the ktest config files
ktest: Allow options to be used by other options

----

 tools/testing/ktest/ktest.pl    |  156 +++++++++++++++++++++++++++++++++++++--
 tools/testing/ktest/sample.conf |   93 +++++++++++++++++++++++
 2 files changed, 242 insertions(+), 7 deletions(-)
author maintainer
#1 lore raw

PATCH 1/6 ktest: Fix post install command

From: Steven Rostedt <[email protected]>

The command to run post install (for those that want initrds) was broken. Instead of doing a substitution for the $KERNEL_VERSION variable. It was replacing the entire command with nothing.

---
 tools/testing/ktest/ktest.pl |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
tools/testing/ktest/ktest.pl +1 -1
index 8ce792e..7301d03 100755
@@ -907,7 +907,7 @@ sub install {
     return if (!defined($post_install));
 
     my $cp_post_install = $post_install;
-    $cp_post_install = s/\$KERNEL_VERSION/$version/g;
+    $cp_post_install =~ s/\$KERNEL_VERSION/$version/g;
     run_command "$cp_post_install" or
 	dodie "Failed to run post install";
 }
Signature · 1 line
1.7.4.4
author maintainer
#2 lore raw

PATCH 2/6 ktest: If test failed due to timeout, print that

In reply to Steven Rostedt

From: Steven Rostedt <[email protected]>

If the test failed due to timeout for boot, print a message saying so. Otherwise the user will be confused to why their test just failed.

---
 tools/testing/ktest/ktest.pl |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)
tools/testing/ktest/ktest.pl +1 -0
index 7301d03..0603e64 100755
@@ -838,6 +838,7 @@ sub monitor {
 
 	if ($stop_test_after > 0 && !$booted && !$bug) {
 	    if (time - $monitor_start > $stop_test_after) {
+		doprint "STOP_TEST_AFTER ($stop_test_after seconds) timed out\n";
 		$done = 1;
 	    }
 	}
Signature · 1 line
1.7.4.4
author maintainer
#3 lore raw

PATCH 3/6 ktest: Reboot to good kernel after every bisect run

In reply to Steven Rostedt

From: Steven Rostedt <[email protected]>

Reboot after each bisect run regardless if the bisect passed or failed. The test may just be to boot the kernel and that kernel may not have a way to copy the next kerne to it. Reboot to a known good kernel after each bisect run.

---
 tools/testing/ktest/ktest.pl |   10 +++++-----
 1 files changed, 5 insertions(+), 5 deletions(-)
tools/testing/ktest/ktest.pl +5 -5
index 0603e64..2011222 100755
@@ -1248,14 +1248,14 @@ sub run_bisect_test {
 
     if ($failed) {
 	$result = 0;
-
-	# reboot the box to a good kernel
-	if ($type ne "build") {
-	    bisect_reboot;
-	}
     } else {
 	$result = 1;
     }
+
+    # reboot the box to a kernel we can ssh to
+    if ($type ne "build") {
+	bisect_reboot;
+    }
     $in_bisect = 0;
 
     return $result;
Signature · 1 line
1.7.4.4
#4 lore raw

Re: [PATCH 3/6] ktest: Reboot to good kernel after every bisect run

Quoted 7 lines · Steven Rostedt

From: Steven Rostedt <[email protected]>

Reboot after each bisect run regardless if the bisect passed or failed. The test may just be to boot the kernel and that kernel may not have a way to copy the next kerne to it. Reboot to a known good kernel after each bisect run.

How about an option to not reboot for good kernels?

author maintainer
#5 lore raw

Re: [PATCH 3/6] ktest: Reboot to good kernel after every bisect run

Quoted 10 lines · John Kacur
Quoted 7 lines · Steven Rostedt

From: Steven Rostedt <[email protected]>

Reboot after each bisect run regardless if the bisect passed or failed. The test may just be to boot the kernel and that kernel may not have a way to copy the next kerne to it. Reboot to a known good kernel after each bisect run.

How about an option to not reboot for good kernels?

Would be trivial to add. But that would be for 2.6.41.

Thanks,

-- Steve

author maintainer
#6 lore raw

PATCH 4/6 ktest: Reboot after each patchcheck run

In reply to Steven Rostedt

From: Steven Rostedt <[email protected]>

The patches being checked may not leave the kernel in a state that the next run will allow the new kernel to be copied to the machine. Reboot to a known good kernel before continuing to the next kernel to test.

Added option PATCHCHECK_SLEEP_TIME for the max time to sleep between patchcheck reboots.

---
 tools/testing/ktest/ktest.pl    |   13 +++++++++++++
 tools/testing/ktest/sample.conf |    4 ++++
 2 files changed, 17 insertions(+), 0 deletions(-)
tools/testing/ktest/ktest.pl +13 -0
index 2011222..65003a1 100755
@@ -36,6 +36,7 @@ $default{"REBOOT_ON_SUCCESS"}	= 1;
 $default{"POWEROFF_ON_SUCCESS"}	= 0;
 $default{"BUILD_OPTIONS"}	= "";
 $default{"BISECT_SLEEP_TIME"}	= 60;   # sleep time between bisects
+$default{"PATCHCHECK_SLEEP_TIME"} = 60; # sleep time between patch checks
 $default{"CLEAR_LOG"}		= 0;
 $default{"BISECT_MANUAL"}	= 0;
 $default{"BISECT_SKIP"}		= 1;
@@ -96,6 +97,7 @@ my $monitor_pid;
 my $monitor_cnt = 0;
 my $sleep_time;
 my $bisect_sleep_time;
+my $patchcheck_sleep_time;
 my $store_failures;
 my $timeout;
 my $booted_timeout;
@@ -1764,6 +1766,14 @@ sub config_bisect {
     success $i;
 }
 
+sub patchcheck_reboot {
+    doprint "Reboot and sleep $patchcheck_sleep_time seconds\n";
+    reboot;
+    start_monitor;
+    wait_for_monitor $patchcheck_sleep_time;
+    end_monitor;
+}
+
 sub patchcheck {
     my ($i) = @_;
 
@@ -1855,6 +1865,8 @@ sub patchcheck {
 	end_monitor;
 	return 0 if ($failed);
 
+	patchcheck_reboot;
+
     }
     $in_patchcheck = 0;
     success $i;
@@ -2004,6 +2016,7 @@ for (my $i = 1; $i <= $opt{"NUM_TESTS"}; $i++) {
     $poweroff_after_halt = set_test_option("POWEROFF_AFTER_HALT", $i);
     $sleep_time = set_test_option("SLEEP_TIME", $i);
     $bisect_sleep_time = set_test_option("BISECT_SLEEP_TIME", $i);
+    $patchcheck_sleep_time = set_test_option("PATCHCHECK_SLEEP_TIME", $i);
     $bisect_manual = set_test_option("BISECT_MANUAL", $i);
     $bisect_skip = set_test_option("BISECT_SKIP", $i);
     $store_failures = set_test_option("STORE_FAILURES", $i);
tools/testing/ktest/sample.conf +4 -0
index 4c5d6bd..87bf92a 100644
@@ -366,6 +366,10 @@
 # (default 60)
 #BISECT_SLEEP_TIME = 60
 
+# The time in between patch checks to sleep (in seconds)
+# (default 60)
+#PATCHCHECK_SLEEP_TIME = 60
+
 # Reboot the target box on error (default 0)
 #REBOOT_ON_ERROR = 0
 
Signature · 1 line
1.7.4.4
author maintainer
#7 lore raw

PATCH 5/6 ktest: Create variables for the ktest config files

In reply to Steven Rostedt

From: Steven Rostedt <[email protected]>

I found that I constantly reuse information for each test case. It would be nice to just define a variable to reuse.

For example I may have:

TEST_START
[...]
TEST = ssh root@mybox /path/to/my/script

TEST_START
[...]
TEST = ssh root@mybox /path/to/my/script

[etc]

The issue is, I may wont to change that script or one of the other fields. Then I need to update each line individually.

With the addition of config variables (variables only used during parsing the config) we can simplify the config files. These variables can also be defined multiple times and each time the new value will overwrite the old value.

The convention to use a config variable over a ktest option is to use := instead of =.

Now we could do:

USER := root
TARGET := mybox
TEST_SCRIPT := /path/to/my/script
TEST_CASE := ${USER}@${TARGET} ${TEST_SCRIPT}

TEST_START
[...]
TEST = ${TEST_CASE}

TEST_START
[...]
TEST = ${TEST_CASE}

[etc]

Now we just need to update the variables at the top.

---
 tools/testing/ktest/ktest.pl    |   62 +++++++++++++++++++++++++++++++++++++++
 tools/testing/ktest/sample.conf |   59 +++++++++++++++++++++++++++++++++++++
 2 files changed, 121 insertions(+), 0 deletions(-)
tools/testing/ktest/ktest.pl +62 -0
index 65003a1..18ef668 100755
@@ -114,6 +114,7 @@ my $successes = 0;
 
 my %entered_configs;
 my %config_help;
+my %variable;
 
 $config_help{"MACHINE"} = << "EOF"
  The machine hostname that you will test.
@@ -262,6 +263,39 @@ sub get_ktest_configs {
     }
 }
 
+sub process_variables {
+    my ($value) = @_;
+    my $retval = "";
+
+    # We want to check for '\', and it is just easier
+    # to check the previous characet of '$' and not need
+    # to worry if '$' is the first character. By adding
+    # a space to $value, we can just check [^\\]\$ and
+    # it will still work.
+    $value = " $value";
+
+    while ($value =~ /(.*?[^\\])\$\{(.*?)\}(.*)/) {
+	my $begin = $1;
+	my $var = $2;
+	my $end = $3;
+	# append beginning of value to retval
+	$retval = "$retval$begin";
+	if (defined($variable{$var})) {
+	    $retval = "$retval$variable{$var}";
+	} else {
+	    # put back the origin piece.
+	    $retval = "$retval\$\{$var\}";
+	}
+	$value = $end;
+    }
+    $retval = "$retval$value";
+
+    # remove the space added in the beginning
+    $retval =~ s/ //;
+
+    return "$retval"
+}
+
 sub set_value {
     my ($lvalue, $rvalue) = @_;
 
@@ -271,10 +305,22 @@ sub set_value {
     if ($rvalue =~ /^\s*$/) {
 	delete $opt{$lvalue};
     } else {
+	$rvalue = process_variables($rvalue);
 	$opt{$lvalue} = $rvalue;
     }
 }
 
+sub set_variable {
+    my ($lvalue, $rvalue) = @_;
+
+    if ($rvalue =~ /^\s*$/) {
+	delete $variable{$lvalue};
+    } else {
+	$rvalue = process_variables($rvalue);
+	$variable{$lvalue} = $rvalue;
+    }
+}
+
 sub read_config {
     my ($config) = @_;
 
@@ -387,6 +433,22 @@ sub read_config {
 		    $repeats{$val} = $repeat;
 		}
 	    }
+	} elsif (/^\s*([A-Z_\[\]\d]+)\s*:=\s*(.*?)\s*$/) {
+	    next if ($skip);
+
+	    my $lvalue = $1;
+	    my $rvalue = $2;
+
+	    # process config variables.
+	    # Config variables are only active while reading the
+	    # config and can be defined anywhere. They also ignore
+	    # TEST_START and DEFAULTS, but are skipped if they are in
+	    # on of these sections that have SKIP defined.
+	    # The save variable can be
+	    # defined multiple times and the new one simply overrides
+	    # the prevous one.
+	    set_variable($lvalue, $rvalue);
+
 	} else {
 	    die "$name: $.: Garbage found in config\n$_";
 	}
tools/testing/ktest/sample.conf +59 -0

65 lines not on this page · raw message

Signature · 1 line
1.7.4.4
author maintainer
#8 lore raw

PATCH 6/6 ktest: Allow options to be used by other options

In reply to Steven Rostedt

From: Steven Rostedt <[email protected]>

There are cases where one ktest option may be used within another ktest option. Allow them to be reused just like config variables but there are evaluated at time of test not config processing time.

Thus having something like:

MAKE_CMD = make ARCH=${ARCH}

TEST_START
ARCH = powerpc

TEST_START
ARCH = arm

Will have the arch defined for each test iteration.

---
 tools/testing/ktest/ktest.pl    |   68 ++++++++++++++++++++++++++++++++++++++-
 tools/testing/ktest/sample.conf |   30 +++++++++++++++++
 2 files changed, 97 insertions(+), 1 deletions(-)
tools/testing/ktest/ktest.pl +67 -1
index 18ef668..1fd29b2 100755
@@ -2019,7 +2019,7 @@ for (my $i = 0, my $repeat = 1; $i <= $opt{"NUM_TESTS"}; $i += $repeat) {
     }
 }
 
-sub set_test_option {
+sub __set_test_option {
     my ($name, $i) = @_;
 
     my $option = "$name\[$i\]";
@@ -2045,6 +2045,72 @@ sub set_test_option {
     return undef;
 }
 
+sub eval_option {
+    my ($option, $i) = @_;
+
+    # Add space to evaluate the character before $
+    $option = " $option";
+    my $retval = "";
+
+    while ($option =~ /(.*?[^\\])\$\{(.*?)\}(.*)/) {
+	my $start = $1;
+	my $var = $2;
+	my $end = $3;
+
+	# Append beginning of line
+	$retval = "$retval$start";
+
+	# If the iteration option OPT[$i] exists, then use that.
+	# otherwise see if the default OPT (without [$i]) exists.
+
+	my $o = "$var\[$i\]";
+
+	if (defined($opt{$o})) {
+	    $o = $opt{$o};
+	    $retval = "$retval$o";
+	} elsif (defined($opt{$var})) {
+	    $o = $opt{$var};
+	    $retval = "$retval$o";
+	} else {
+	    $retval = "$retval\$\{$var\}";
+	}
+
+	$option = $end;
+    }
+
+    $retval = "$retval$option";
+
+    $retval =~ s/^ //;
+
+    return $retval;
+}
+
+sub set_test_option {
+    my ($name, $i) = @_;
+
+    my $option = __set_test_option($name, $i);
+    return $option if (!defined($option));
+
+    my $prev = "";
+
+    # Since an option can evaluate to another option,
+    # keep iterating until we do not evaluate any more
+    # options.
+    my $r = 0;
+    while ($prev ne $option) {
+	# Check for recursive evaluations.
+	# 100 deep should be more than enough.
+	if ($r++ > 100) {
+	    die "Over 100 evaluations accurred with $name\n" .
+		"Check for recursive variables\n";
+	}
+	$prev = $option;
+	$option = eval_option($option, $i);
+    }
+
+    return $option;
+}
+
 # First we need to do is the builds
 for (my $i = 1; $i <= $opt{"NUM_TESTS"}; $i++) {
 
tools/testing/ktest/sample.conf +30 -0
index 761079e..48cbcc80 100644
@@ -133,6 +133,36 @@
 # the MAKE_CMD option will be evaluated by the shell when
 # the MAKE_CMD option is passed into shell processing.
 
+#### Using options in other options ####
+#
+# Options that are defined in the config file may also be used
+# by other options. All options are evaulated at time of
+# use (except that config variables are evaluated at config
+# processing time).
+#
+# If an ktest option is used within another option, instead of
+# typing it again in that option you can simply use the option
+# just like you can config variables.
+#
+# MACHINE = mybox
+#
+# TEST = ssh root@${MACHINE} /path/to/test
+#
+# The option will be used per test case. Thus:
+#
+# TEST_TYPE = test
+# TEST = ssh root@{MACHINE}
+#
+# TEST_START
+# MACHINE = box1
+#
+# TEST_START
+# MACHINE = box2
+#
+# For both test cases, MACHINE will be evaluated at the time
+# of the test case. The first test will run ssh root@box1
+# and the second will run ssh root@box2.
+
 #### Mandatory Default Options ####
 
 # These options must be in the default section, although most
Signature · 1 line
1.7.4.4