Skip to contentSkip to conversation

ktest: updates for v3.9

GIT PULL New Steven Rostedt 1 participant 8 messages Feb 20, 2013 Ktest

lore
Share & tools
lore
author maintainer
#3 lore raw

PATCH 3/7 ktest: Allow a test option to use its default option

In reply to Steven Rostedt

From: "Steven Rostedt (Red Hat)" <[email protected]>

Options are allowed to use other options, for example:

  LOG_FILE = ${OUTPUT_DIR}/${MACHINE}.log

where the option LOG_FILE used the options OUTPUT_DIR and MACHINE.

But if a test option were to use a default option, it will not get substituted:

  OUTPUT_DIR = ${THIS_DIR}/${MACHINE}

  TEST_START
  OUTPUT_DIR = ${OUTPUT_DIR}/t1

For the above test, OUTPUT_DIR will stay literally "${OUTPUT_DIR}/t1" and not be converted to "${THIS_DIR}/${MACHINE}/t1". When the test runs, it will pass the ${OUTPUT_DIR} to the shell, which would probaly interpret it as "", and the output directory will end up as "/t1".

Change the code where if a test option has its own option name in its defined field, and a default option exists, then substitute the default option in its place.

---
 tools/testing/ktest/ktest.pl |   16 ++++++++++------
 1 file changed, 10 insertions(+), 6 deletions(-)
tools/testing/ktest/ktest.pl +10 -6
index cc9925a..3d19ee4 100755
@@ -1074,7 +1074,7 @@ sub read_config {
 }
 
 sub __eval_option {
-    my ($option, $i) = @_;
+    my ($name, $option, $i) = @_;
 
     # Add space to evaluate the character before $
     $option = " $option";
@@ -1106,7 +1106,11 @@ sub __eval_option {
 	my $o = "$var\[$i\]";
 	my $parento = "$var\[$parent\]";
 
-	if (defined($opt{$o})) {
+	# If a variable contains itself, use the default var
+	if (($var eq $name) && defined($opt{$var})) {
+	    $o = $opt{$var};
+	    $retval = "$retval$o";
+	} elsif (defined($opt{$o})) {
 	    $o = $opt{$o};
 	    $retval = "$retval$o";
 	} elsif ($repeated && defined($opt{$parento})) {
@@ -1130,7 +1134,7 @@ sub __eval_option {
 }
 
 sub eval_option {
-    my ($option, $i) = @_;
+    my ($name, $option, $i) = @_;
 
     my $prev = "";
 
@@ -1146,7 +1150,7 @@ sub eval_option {
 		"Check for recursive variables\n";
 	}
 	$prev = $option;
-	$option = __eval_option($option, $i);
+	$option = __eval_option($name, $option, $i);
     }
 
     return $option;
@@ -3683,7 +3687,7 @@ EOF
 read_config $ktest_config;
 
 if (defined($opt{"LOG_FILE"})) {
-    $opt{"LOG_FILE"} = eval_option($opt{"LOG_FILE"}, -1);
+    $opt{"LOG_FILE"} = eval_option("LOG_FILE", $opt{"LOG_FILE"}, -1);
 }
 
 # Append any configs entered in manually to the config file.
@@ -3760,7 +3764,7 @@ sub set_test_option {
     my $option = __set_test_option($name, $i);
     return $option if (!defined($option));
 
-    return eval_option($option, $i);
+    return eval_option($name, $option, $i);
 }
 
 # First we need to do is the builds
Signature · 1 line
1.7.10.4
author maintainer
#2 lore raw

PATCH 2/7 ktest: Strip off \n when reading which files were modified

In reply to Steven Rostedt

From: "Steven Rostedt (Red Hat)" <[email protected]>

The patchcheck test looks at what files are modified for each patch it checks and makes sure that those files do not produce any warnings.

Unfortunately, when it read the diffstat, the newlines were added on the files and this made compares miss warnings, and commits that should not have passed, ktest let pass.

Fix this by using the perl command "chomp" that strips off whitespace at the end of lines.

---
 tools/testing/ktest/ktest.pl |    4 ++++
 1 file changed, 4 insertions(+)
tools/testing/ktest/ktest.pl +4 -0
index d6690df..cc9925a 100755
@@ -1925,6 +1925,10 @@ sub check_buildlog {
 
     my @files = `git show $patch | diffstat -l`;
 
+    foreach my $file (@files) {
+	chomp $file;
+    }
+
     open(IN, "git show $patch |") or
 	dodie "failed to show $patch";
     while (<IN>) {
Signature · 1 line
1.7.10.4
author maintainer
#1 lore raw

PATCH 1/7 ktest: Do not require CONSOLE for build or install bisects

In reply to Steven Rostedt

From: "Steven Rostedt (Red Hat)" <[email protected]>

If the user is doing a build or install bisect, there's no reason to have them define CONSOLE, as the console does not need to be read. The console only needs to be read for boot tests.

CONSOLE is not required for normal build or install tests, let's not require it for bisect tests with BISECT_TYPE of build or install.

---
 tools/testing/ktest/ktest.pl |   27 ++++++++++++++++++++++-----
 1 file changed, 22 insertions(+), 5 deletions(-)
tools/testing/ktest/ktest.pl +22 -5
index 35fc584..d6690df 100755
@@ -619,6 +619,18 @@ sub set_value {
 	# Note if a test is something other than build, then we
 	# will need other manditory options.
 	if ($prvalue ne "install") {
+	    # for bisect, we need to check BISECT_TYPE
+	    if ($prvalue ne "bisect") {
+		$buildonly = 0;
+	    }
+	} else {
+	    # install still limits some manditory options.
+	    $buildonly = 2;
+	}
+    }
+
+    if ($buildonly && $lvalue =~ /^BISECT_TYPE(\[.*\])?$/ && $prvalue ne "build") {
+	if ($prvalue ne "install") {
 	    $buildonly = 0;
 	} else {
 	    # install still limits some manditory options.
@@ -1279,6 +1291,7 @@ sub start_monitor {
 }
 
 sub end_monitor {
+    return if (!defined $console);
     if (--$monitor_cnt) {
 	return;
     }
@@ -1585,7 +1598,7 @@ sub wait_for_input
 
     $rin = '';
     vec($rin, fileno($fp), 1) = 1;
-    $ready = select($rin, undef, undef, $time);
+    ($ready, $time) = select($rin, undef, undef, $time);
 
     $line = "";
 
@@ -1891,15 +1904,19 @@ sub get_version {
 
 sub start_monitor_and_boot {
     # Make sure the stable kernel has finished booting
-    start_monitor;
-    wait_for_monitor 5;
-    end_monitor;
+
+    # Install bisects, don't need console
+    if (defined $console) {
+	start_monitor;
+	wait_for_monitor 5;
+	end_monitor;
+    }
 
     get_grub_index;
     get_version;
     install;
 
-    start_monitor;
+    start_monitor if (defined $console);
     return monitor;
 }
 
Signature · 1 line
1.7.10.4
author maintainer
#7 lore raw

PATCH 7/7 ktest: Remove indexes from warnings check

In reply to Steven Rostedt

From: "Steven Rostedt (Red Hat)" <[email protected]>

The index of a line where a warning is tested can be returned differently on different versions of gcc (or same version compiled differently). That is, a tab + space can give different results. This causes the warning check to produce a false positive. Removing the index from the check fixes this issue.

---
 tools/testing/ktest/ktest.pl |   34 ++++++++++++++++++++++++++--------
 1 file changed, 26 insertions(+), 8 deletions(-)
tools/testing/ktest/ktest.pl +26 -8
index f3bb5da..4e67d52 100755
@@ -1945,6 +1945,27 @@ sub start_monitor_and_boot {
 my $check_build_re = ".*:.*(warning|error|Error):.*";
 my $utf8_quote = "\\x{e2}\\x{80}(\\x{98}|\\x{99})";
 
+sub process_warning_line {
+    my ($line) = @_;
+
+    chomp $line;
+
+    # for distcc heterogeneous systems, some compilers
+    # do things differently causing warning lines
+    # to be slightly different. This makes an attempt
+    # to fixe those issues.
+
+    # chop off the index into the line
+    # using distcc, some compilers give different indexes
+    # depending on white space
+    $line =~ s/^(\s*\S+:\d+:)\d+/$1/;
+
+    # Some compilers use UTF-8 extended for quotes and some don't.
+    $line =~ s/$utf8_quote/'/g;
+
+    return $line;
+}
+
 # Read buildlog and check against warnings file for any
 # new warnings.
 #
@@ -1965,8 +1986,9 @@ sub check_buildlog {
 
 	while (<IN>) {
 	    if (/$check_build_re/) {
-		chomp;
-		$warnings_list{$_} = 1;
+		my $warning = process_warning_line $_;
+		
+		$warnings_list{$warning} = 1;
 	    }
 	}
 	close(IN);
@@ -1978,13 +2000,9 @@ sub check_buildlog {
     open(IN, $buildlog) or dodie "Can't open $buildlog";
     while (<IN>) {
 	if (/$check_build_re/) {
+	    my $warning = process_warning_line $_;
 
-	    # Some compilers use UTF-8 extended for quotes
-	    # for distcc heterogeneous systems, this causes issues
-	    s/$utf8_quote/'/g;
-
-	    chomp;
-	    if (!defined $warnings_list{$_}) {
+	    if (!defined $warnings_list{$warning}) {
 		fail "New warning found (not in $warnings_file)\n$_\n";
 		$no_reboot = $save_no_reboot;
 		return 0;
Signature · 1 line
1.7.10.4
author maintainer
#5 lore raw

PATCH 5/7 ktest: Search for linux banner for successful reboot

In reply to Steven Rostedt

From: "Steven Rostedt (Red Hat)" <[email protected]>

Sometimes when a test kernel passed fine, but on reboot it crashed, ktest could get stuck and not proceed. This would be frustrating if you let a test run overnight to find out the next morning that it was stuck on the first test.

To fix this, I made reboot check for the REBOOT_SUCCESS_LINE. If the line was not detected, then it would power cycle the box.

What it didn't cover was if the REBOOT_SUCCESS_LINE wasn't defined or if a 'good' kernel did not display the line. Instead have it search for the Linux banner "Linux version". The reboot just needs to get to the start of the next kernel, it does not need to test if the next kernel makes it to a boot prompt.

After we find the next kernel has booted, then we just wait for either the REBOOT_SUCCESS_LINE to appear or the timeout.

---
 tools/testing/ktest/ktest.pl |    7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
tools/testing/ktest/ktest.pl +6 -1
index 3fd768e..839989d 100755
@@ -1212,11 +1212,16 @@ sub reboot {
     }
 
     if (defined($time)) {
-	if (wait_for_monitor($time, $reboot_success_line)) {
+	# Look for the good kernel to boot
+	if (wait_for_monitor($time, "Linux version")) {
 	    # reboot got stuck?
 	    doprint "Reboot did not finish. Forcing power cycle\n";
 	    run_command "$power_cycle";
 	}
+
+	# Still need to wait for the reboot to finish
+	wait_for_monitor($time, $reboot_success_line);
+
 	end_monitor;
     }
 }
Signature · 1 line
1.7.10.4

Steven Rostedt

author maintainer
#0 lore raw

Linus,

Updates include:

o Added ability to have all builds test warnings.

o Fixed failing reboot when the reboot produces a non fatal error.

o Config reading fixes and other cleanups.

Please pull the latest ktest-v3.9 tree, which can be found at:

git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-ktest.git
ktest-v3.9

Head SHA1: b4e059fa0c07e0c054dab9a48c2e3f7d7d4563be

Steven Rostedt (Red Hat) (7):
ktest: Do not require CONSOLE for build or install bisects
ktest: Strip off '\n' when reading which files were modified
ktest: Allow a test option to use its default option
ktest: Add make_warnings_file and process full warnings
ktest: Search for linux banner for successful reboot
ktest: Ignore warnings during reboot
ktest: Remove indexes from warnings check

----

 .../testing/ktest/examples/include/patchcheck.conf |   37 ++++
 tools/testing/ktest/ktest.pl                       |  190 ++++++++++++++++++--
 tools/testing/ktest/sample.conf                    |   44 +++++
 3 files changed, 253 insertions(+), 18 deletions(-)
author maintainer
#4 lore raw

PATCH 4/7 ktest: Add make_warnings_file and process full warnings

From: "Steven Rostedt (Red Hat)" <[email protected]>

Although the patchcheck test checks for warnings in the files that were changed, this check does not catch warnings that were caused by header file changes and the warnings appear in C files not touched by the commit.

Add a new option called WARNINGS_FILE. If this option is set, then the file it points to is read before bulid, and the file should contain a list of known warnings. If a warning appears in the build, this file is checked, and if the warning does not exist in this file, then it fails the build showing the new warning.

If the WARNINGS_FILE points to a file that does not exist, this will cause any warning in the build to fail.

A new test is also added called "make_warnings_file". This test will create do a build and record any warnings it finds into the WARNINGS_FILE. This test is something that can be run before other tests to build a warnings file of "known warnings", ie, warnings that were there before your changes.

---
 .../testing/ktest/examples/include/patchcheck.conf |   37 +++++++
 tools/testing/ktest/ktest.pl                       |  110 ++++++++++++++++++--
 tools/testing/ktest/sample.conf                    |   44 ++++++++
 3 files changed, 185 insertions(+), 6 deletions(-)
tools/testing/ktest/examples/include/patchcheck.conf +37 -0
index 339d3e1..0eb0a5a 100644
@@ -14,6 +14,16 @@
 PATCH_START := HEAD~3
 PATCH_END := HEAD
 
+# Use the oldconfig if build_type wasn't defined
+DEFAULTS IF NOT DEFINED BUILD_TYPE
+DO_BUILD_TYPE := oldconfig
+
+DEFAULTS ELSE
+DO_BUILD_TYPE := ${BUILD_TYPE}
+
+DEFAULTS
+
+
 # Change PATCH_CHECKOUT to be the branch you want to test. The test will
 # do a git checkout of this branch before starting. Obviously both
 # PATCH_START and PATCH_END must be in this branch (and PATCH_START must
@@ -43,6 +53,31 @@ PATCH_TEST_TYPE := boot
 # (space delimited)
 #IGNORE_WARNINGS = 39eaf7ef884dcc44f7ff1bac803ca2a1dcf43544 6edb2a8a385f0cdef51dae37ff23e74d76d8a6ce
 
+# Instead of just checking for warnings to files that are changed
+# it can be advantageous to check for any new warnings. If a
+# header file is changed, it could cause a warning in a file not
+# touched by the commit. To detect these kinds of warnings, you
+# can use the WARNINGS_FILE option.
+#
+# If the variable CREATE_WARNINGS_FILE is set, this config will
+# enable the WARNINGS_FILE during the patchcheck test. Also,
+# before running the patchcheck test, it will create the
+# warnings file.
+#
+DEFAULTS IF DEFINED CREATE_WARNINGS_FILE
+WARNINGS_FILE = ${OUTPUT_DIR}/warnings_file
+
+TEST_START IF DEFINED CREATE_WARNINGS_FILE
+# WARNINGS_FILE is already set by the DEFAULTS above
+TEST_TYPE = make_warnings_file
+# Checkout the commit before the patches to test,
+# and record all the warnings that exist before the patches
+# to test are added
+CHECKOUT = ${PATCHCHECK_START}~1
+# Force a full build
+BUILD_NOCLEAN = 0
+BUILD_TYPE = ${DO_BUILD_TYPE}
+
 # If you are running a multi test, and the test failed on the first
 # test but on, say the 5th patch. If you want to restart on the
 # fifth patch, set PATCH_START1. This will make the first test start
@@ -61,6 +96,7 @@ PATCHCHECK_TYPE = ${PATCH_TEST_TYPE}
 PATCHCHECK_START = ${PATCH_START1}
 PATCHCHECK_END = ${PATCH_END}
 CHECKOUT = ${PATCH_CHECKOUT}
+BUILD_TYPE = ${DO_BUILD_TYPE}
 
 TEST_START IF ${TEST} == patchcheck && ${MULTI}
 TEST_TYPE = patchcheck
@@ -72,3 +108,4 @@ PATCHCHECK_END = ${PATCH_END}
 CHECKOUT = ${PATCH_CHECKOUT}
 # Use multi to test different compilers?
 MAKE_CMD = CC=gcc-4.5.1 make
+BUILD_TYPE = ${DO_BUILD_TYPE}
tools/testing/ktest/ktest.pl +104 -6

165 lines not on this page · raw message

tools/testing/ktest/sample.conf +44 -0

53 lines not on this page · raw message

Signature · 1 line
1.7.10.4
author maintainer
#6 lore raw

PATCH 6/7 ktest: Ignore warnings during reboot

In reply to Steven Rostedt

From: "Steven Rostedt (Red Hat)" <[email protected]>

The reboot just wants to get to the next kernel. But if a warning (Call Trace) appears, the monitor will report an error, and the reboot will think something went wrong and power cycle the box, even though we successfully made it to the next kernel.

Ignore warnings during the reboot until we get to the next kernel. It will still timeout if we never get to the next kernel and then a power cycle will happen. That's what we want it to do.

---
 tools/testing/ktest/ktest.pl |    8 ++++++++
 1 file changed, 8 insertions(+)
tools/testing/ktest/ktest.pl +8 -0
index 839989d..f3bb5da 100755
@@ -1212,6 +1212,12 @@ sub reboot {
     }
 
     if (defined($time)) {
+
+	# We only want to get to the new kernel, don't fail
+	# if we stumble over a call trace.
+	my $save_ignore_errors = $ignore_errors;
+	$ignore_errors = 1;
+
 	# Look for the good kernel to boot
 	if (wait_for_monitor($time, "Linux version")) {
 	    # reboot got stuck?
@@ -1219,6 +1225,8 @@ sub reboot {
 	    run_command "$power_cycle";
 	}
 
+	$ignore_errors = $save_ignore_errors;
+
 	# Still need to wait for the reboot to finish
 	wait_for_monitor($time, $reboot_success_line);
 
Signature · 1 line
1.7.10.4