Skip to contentSkip to conversation

ktest.pl: Various updates and including more content in email on failures

PATCH In review Steven Rostedt 2 participants 17 messages Jul 1, 2020 – Jul 2, 2020 Ktest

lore
Share & tools
lore

Steven Rostedt

author maintainer
#0 lore raw

cover ktest.pl: Various updates and including more content in email on failures

Steven Rostedt (VMware) (8):
ktest.pl: Have config-bisect save each config used in the bisect ktest.pl: Always show log file location if defined even on success ktest.pl: Define PRE_TEST_DIE to kill the test if the PRE_TEST fails ktest.pl: Add a NOT operator
ktest.pl: Just open up the log file once
ktest.pl: Turn off buffering to the log file
ktest.pl: Add the log of last test in email on failure ktest.pl: Add MAIL_MAX_SIZE to limit the amount of log emailed

----

 tools/testing/ktest/ktest.pl    | 100 ++++++++++++++++++++++++++++++++++------
 tools/testing/ktest/sample.conf |  18 ++++++++
 2 files changed, 103 insertions(+), 15 deletions(-)
author maintainer
#1 lore raw

PATCH 1/8 ktest.pl: Have config-bisect save each config used in the bisect

From: "Steven Rostedt (VMware)" <[email protected]>

When performing a automatic config bisect via ktest.pl, it is very useful to have a copy of each of the bisects used. This way, if a bisect were to go wrong, it is possible to retrace the steps and continue at the location before the error was made.

The ktest.pl will make a copy of the good and bad configs, labeled as such, as well as a number attached to it that represents the iteration of the bisect. These files are saved in the ktest temp directory where it currently stores the good and bad config files.

---
 tools/testing/ktest/ktest.pl | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)
tools/testing/ktest/ktest.pl +8 -2
index 7570e36d636d..5f6f88911f5c 100755
@@ -3188,6 +3188,8 @@ sub config_bisect_end {
     doprint "***************************************\n\n";
 }
 
+my $pass = 1;
+
 sub run_config_bisect {
     my ($good, $bad, $last_result) = @_;
     my $reset = "";
@@ -3210,11 +3212,15 @@ sub run_config_bisect {
 
     $ret = run_config_bisect_test $config_bisect_type;
     if ($ret) {
-        doprint "NEW GOOD CONFIG\n";
+        doprint "NEW GOOD CONFIG ($pass)\n";
+	system("cp $output_config $tmpdir/good_config.tmp.$pass");
+	$pass++;
 	# Return 3 for good config
 	return 3;
     } else {
-        doprint "NEW BAD CONFIG\n";
+        doprint "NEW BAD CONFIG ($pass)\n";
+	system("cp $output_config $tmpdir/bad_config.tmp.$pass");
+	$pass++;
 	# Return 4 for bad config
 	return 4;
     }
Signature · 1 line
2.26.2
author maintainer
#2 lore raw

PATCH 2/8 ktest.pl: Always show log file location if defined even on success

In reply to Steven Rostedt

From: "Steven Rostedt (VMware)" <[email protected]>

If a log file is defined and the test were to error, a print statement is made that shows the user where the log file is to examine it further. But this is not done if the test were to succeed.

I find it annoying that it does not show where the log file is on success, as I run several different tests that place their log files in various locations, and even though the test pass, there's things I want to look at in the log file (like warnings). It is much easier to find where the log file is, if it is displayed at the end of a test.

---
 tools/testing/ktest/ktest.pl | 4 ++++
 1 file changed, 4 insertions(+)
tools/testing/ktest/ktest.pl +4 -0
index 5f6f88911f5c..5d5cf3e1e81a 100755
@@ -4441,6 +4441,10 @@ if ($opt{"POWEROFF_ON_SUCCESS"}) {
 }
 
 
+if (defined($opt{"LOG_FILE"})) {
+    print "\n See $opt{LOG_FILE} for the record of results.\n";
+}
+
 doprint "\n    $successes of $opt{NUM_TESTS} tests were successful\n\n";
 
 if ($email_when_finished) {
Signature · 1 line
2.26.2
author maintainer
#3 lore raw

PATCH 3/8 ktest.pl: Define PRE_TEST_DIE to kill the test if the PRE_TEST fails

In reply to Steven Rostedt

From: "Steven Rostedt (VMware)" <[email protected]>

Currently, if a PRE_TEST is defined and ran, but fails, there's nothing currently available to make the test fail too. Add a PRE_TEST_DIE option that when set, if a PRE_TEST is defined and fails, the test will die too.

---
 tools/testing/ktest/ktest.pl    | 8 +++++++-
 tools/testing/ktest/sample.conf | 5 +++++
 2 files changed, 12 insertions(+), 1 deletion(-)
tools/testing/ktest/ktest.pl +7 -1
index 5d5cf3e1e81a..f99cf633ed84 100755
@@ -98,6 +98,7 @@ my $final_post_ktest;
 my $pre_ktest;
 my $post_ktest;
 my $pre_test;
+my $pre_test_die;
 my $post_test;
 my $pre_build;
 my $post_build;
@@ -273,6 +274,7 @@ my %option_map = (
     "PRE_KTEST"			=> \$pre_ktest,
     "POST_KTEST"		=> \$post_ktest,
     "PRE_TEST"			=> \$pre_test,
+    "PRE_TEST_DIE"		=> \$pre_test_die,
     "POST_TEST"			=> \$post_test,
     "BUILD_TYPE"		=> \$build_type,
     "BUILD_OPTIONS"		=> \$build_options,
@@ -4347,7 +4349,11 @@ for (my $i = 1; $i <= $opt{"NUM_TESTS"}; $i++) {
     doprint "RUNNING TEST $i of $opt{NUM_TESTS}$name with option $test_type $run_type$installme\n\n";
 
     if (defined($pre_test)) {
-	run_command $pre_test;
+	my $ret = run_command $pre_test;
+	if (!$ret && defined($pre_test_die) &&
+	    $pre_test_die) {
+	    dodie "failed to pre_test\n";
+	}
     }
 
     unlink $dmesg;
tools/testing/ktest/sample.conf +5 -0
index 27666b8007ed..cb8227fbd01e 100644
@@ -557,6 +557,11 @@
 # default (undefined)
 #PRE_TEST = ${SSH} reboot_to_special_kernel
 
+# To kill the entire test if PRE_TEST is defined but fails set this
+# to 1.
+# (default 0)
+#PRE_TEST_DIE = 1
+
 # If there is a command you want to run after the individual test case
 # completes, then you can set this option.
 #
Signature · 1 line
2.26.2
author maintainer
#4 lore raw

PATCH 4/8 ktest.pl: Add a NOT operator

In reply to Steven Rostedt

From: "Steven Rostedt (VMware)" <[email protected]>

There is a NOT DEFINED operator, but there is not an operator that can negate any other expression.

For example: NOT (${FOO} == boot || ${BAR} == run)

Add the keyword NOT to allow the ktest.pl config files to negate operators.

---
 tools/testing/ktest/ktest.pl | 6 ++++++
 1 file changed, 6 insertions(+)
tools/testing/ktest/ktest.pl +6 -0
index f99cf633ed84..0d04b8a2b5a2 100755
@@ -911,6 +911,12 @@ sub process_expression {
 	}
     }
 
+    if ($val =~ s/^\s*NOT\s+(.*)//) {
+	my $express = $1;
+	my $ret = process_expression($name, $express);
+	return !$ret;
+    }
+
     if ($val =~ /^\s*0\s*$/) {
 	return 0;
     } elsif ($val =~ /^\s*\d+\s*$/) {
Signature · 1 line
2.26.2
author maintainer
#5 lore raw

PATCH 5/8 ktest.pl: Just open up the log file once

In reply to Steven Rostedt

From: "Steven Rostedt (VMware)" <[email protected]>

Currently, every write to the log file is done by opening the file, writing to it, then closing the file. This rather expensive. Just open it at the beginning and close it at the end.

---
 tools/testing/ktest/ktest.pl | 24 ++++++++++++------------
 1 file changed, 12 insertions(+), 12 deletions(-)
tools/testing/ktest/ktest.pl +12 -12
index 0d04b8a2b5a2..f20a81bb3abe 100755
@@ -509,9 +509,7 @@ EOF
 
 sub _logit {
     if (defined($opt{"LOG_FILE"})) {
-	open(OUT, ">> $opt{LOG_FILE}") or die "Can't write to $opt{LOG_FILE}";
-	print OUT @_;
-	close(OUT);
+	print LOG @_;
     }
 }
 
@@ -1780,8 +1778,6 @@ sub run_command {
 	(fail "unable to exec $command" and return 0);
 
     if (defined($opt{"LOG_FILE"})) {
-	open(LOG, ">>$opt{LOG_FILE}") or
-	    dodie "failed to write to log";
 	$dolog = 1;
     }
 
@@ -1829,7 +1825,6 @@ sub run_command {
     }
 
     close(CMD);
-    close(LOG) if ($dolog);
     close(RD)  if ($dord);
 
     $end_time = time;
@@ -4091,8 +4086,11 @@ if ($#new_configs >= 0) {
     }
 }
 
-if ($opt{"CLEAR_LOG"} && defined($opt{"LOG_FILE"})) {
-    unlink $opt{"LOG_FILE"};
+if (defined($opt{"LOG_FILE"})) {
+    if ($opt{"CLEAR_LOG"}) {
+	unlink $opt{"LOG_FILE"};
+    }
+    open(LOG, ">> $opt{LOG_FILE}") or die "Can't write to $opt{LOG_FILE}";
 }
 
 doprint "\n\nSTARTING AUTOMATED TESTS\n\n";
@@ -4453,14 +4451,16 @@ if ($opt{"POWEROFF_ON_SUCCESS"}) {
 }
 
 
-if (defined($opt{"LOG_FILE"})) {
-    print "\n See $opt{LOG_FILE} for the record of results.\n";
-}
-
 doprint "\n    $successes of $opt{NUM_TESTS} tests were successful\n\n";
 
 if ($email_when_finished) {
     send_email("KTEST: Your test has finished!",
             "$successes of $opt{NUM_TESTS} tests started at $script_start_time were successful!");
 }
+
+if (defined($opt{"LOG_FILE"})) {
+    print "\n See $opt{LOG_FILE} for the record of results.\n\n";
+    close LOG;
+}
+
 exit 0;
Signature · 1 line
2.26.2
author maintainer
#6 lore raw

PATCH 6/8 ktest.pl: Turn off buffering to the log file

In reply to Steven Rostedt

From: "Steven Rostedt (VMware)" <[email protected]>

The log file should be up to date to whatever is happening in ktest. Disable buffering to the LOG output file handle.

---
 tools/testing/ktest/ktest.pl | 2 ++
 1 file changed, 2 insertions(+)
tools/testing/ktest/ktest.pl +2 -0
index f20a81bb3abe..e90e2e7cb72c 100755
@@ -11,6 +11,7 @@ use File::Path qw(mkpath);
 use File::Copy qw(cp);
 use FileHandle;
 use FindBin;
+use IO::Handle;
 
 my $VERSION = "0.2";
 
@@ -4091,6 +4092,7 @@ if (defined($opt{"LOG_FILE"})) {
 	unlink $opt{"LOG_FILE"};
     }
     open(LOG, ">> $opt{LOG_FILE}") or die "Can't write to $opt{LOG_FILE}";
+    LOG->autoflush(1);
 }
 
 doprint "\n\nSTARTING AUTOMATED TESTS\n\n";
Signature · 1 line
2.26.2
author maintainer
#7 lore raw

PATCH 7/8 ktest.pl: Add the log of last test in email on failure

In reply to Steven Rostedt

From: "Steven Rostedt (VMware)" <[email protected]>

If a failure happens and an email is sent, show the contents of the log of the last test that failed in the email.

---
 tools/testing/ktest/ktest.pl | 44 ++++++++++++++++++++++++++++++++----
 1 file changed, 40 insertions(+), 4 deletions(-)
tools/testing/ktest/ktest.pl +40 -4
index e90e2e7cb72c..945a7d8c178c 100755
@@ -82,6 +82,8 @@ my %default = (
     "IGNORE_UNUSED"		=> 0,
 );
 
+my $test_log_start = 0;
+
 my $ktest_config = "ktest.conf";
 my $version;
 my $have_version = 0;
@@ -1492,8 +1494,21 @@ sub dodie {
 
     if ($email_on_error) {
 	my $name = get_test_name;
+	my $log_file;
+
+	if (defined($opt{"LOG_FILE"})) {
+	    $log_file = "$tmpdir/log";
+	    open (L, "$opt{LOG_FILE}") or die "Can't open $opt{LOG_FILE} to read)";
+	    open (O, "> $tmpdir/log") or die "Can't open $tmpdir/log\n";
+	    seek(L, $test_log_start, 0);
+	    while (<L>) {
+		print O;
+	    }
+	    close O;
+	    close L;
+	}
         send_email("KTEST: critical failure for test $i [$name]",
-                "Your test started at $script_start_time has failed with:\n@_\n");
+                "Your test started at $script_start_time has failed with:\n@_\n", $log_file);
     }
 
     if ($monitor_cnt) {
@@ -4185,7 +4200,7 @@ sub find_mailer {
 }
 
 sub do_send_mail {
-    my ($subject, $message) = @_;
+    my ($subject, $message, $file) = @_;
 
     if (!defined($mail_path)) {
 	# find the mailer
@@ -4195,22 +4210,37 @@ sub do_send_mail {
 	}
     }
 
+    my $header_file = "$tmpdir/header";
+    open (HEAD, ">$header_file") or die "Can not create $header_file\n";
+    print HEAD "To: $mailto\n";
+    print HEAD "Subject: $subject\n\n";
+    print HEAD "$message\n";
+    close HEAD;
+
     if (!defined($mail_command)) {
 	if ($mailer eq "mail" || $mailer eq "mailx") {
-	    $mail_command = "\$MAIL_PATH/\$MAILER -s \'\$SUBJECT\' \$MAILTO <<< \'\$MESSAGE\'";
+	    $mail_command = "cat \$HEADER_FILE \$BODY_FILE | \$MAIL_PATH/\$MAILER -s \'\$SUBJECT\' \$MAILTO";
 	} elsif ($mailer eq "sendmail" ) {
-	    $mail_command =  "echo \'Subject: \$SUBJECT\n\n\$MESSAGE\' | \$MAIL_PATH/\$MAILER -t \$MAILTO";
+	    $mail_command =  "cat \$HEADER_FILE \$BODY_FILE | \$MAIL_PATH/\$MAILER -t \$MAILTO";
 	} else {
 	    die "\nYour mailer: $mailer is not supported.\n";
 	}
     }
 
+    if (defined($file)) {
+	$mail_command =~ s/\$BODY_FILE/$file/g;
+    } else {
+	$mail_command =~ s/\$BODY_FILE//g;
+    }
+
+    $mail_command =~ s/\$HEADER_FILE/$header_file/g;
     $mail_command =~ s/\$MAILER/$mailer/g;
     $mail_command =~ s/\$MAIL_PATH/$mail_path/g;
     $mail_command =~ s/\$MAILTO/$mailto/g;
     $mail_command =~ s/\$SUBJECT/$subject/g;
     $mail_command =~ s/\$MESSAGE/$message/g;
 
+	    print ">$mail_command<\n";
     run_command $mail_command;
 }
 
@@ -4352,6 +4382,11 @@ for (my $i = 1; $i <= $opt{"NUM_TESTS"}; $i++) {
     }
 
     doprint "\n\n";
+
+    if (defined($opt{"LOG_FILE"})) {
+	$test_log_start = tell(LOG);
+    }
+
     doprint "RUNNING TEST $i of $opt{NUM_TESTS}$name with option $test_type $run_type$installme\n\n";
 
     if (defined($pre_test)) {
@@ -4461,6 +4496,7 @@ if ($email_when_finished) {
 }
 
 if (defined($opt{"LOG_FILE"})) {
+
     print "\n See $opt{LOG_FILE} for the record of results.\n\n";
     close LOG;
 }
Signature · 1 line
2.26.2
author maintainer
#8 lore raw

Re: [for-next][PATCH 7/8] ktest.pl: Add the log of last test in email on failure

tools/testing/ktest/ktest.pl sub do_send_mail { · line 4243

Quoted 10 lines · On Wed, 01 Jul 2020 19:17:24 -0400 Steven Rostedt <[email protected]> wrote:
tools/testing/ktest/ktest.pl +4 -0
+    }
+
+    $mail_command =~ s/\$HEADER_FILE/$header_file/g;
     $mail_command =~ s/\$MAILER/$mailer/g;
     $mail_command =~ s/\$MAIL_PATH/$mail_path/g;
     $mail_command =~ s/\$MAILTO/$mailto/g;
     $mail_command =~ s/\$SUBJECT/$subject/g;
     $mail_command =~ s/\$MESSAGE/$message/g;
 
+	    print ">$mail_command<\n";

Oops, left in this debug print statement.

I nuked it before pushing to linux-next.

-- Steve

Quoted 5 lines
     run_command $mail_command;

}

(quoted diff) +0 -0
@@ -4352,6 +4382,11 @@ for (my $i = 1; $i <= $opt{"NUM_TESTS"}; $i++) {
     }
Reviewed-by on v1 7/8
#9 lore raw

Re: [for-next][PATCH 7/8] ktest.pl: Add the log of last test in email on failure

In reply to Steven Rostedt
Quoted 97 lines · Steven Rostedt

From: "Steven Rostedt (VMware)" <[email protected]>

If a failure happens and an email is sent, show the contents of the log of the last test that failed in the email.

---
 tools/testing/ktest/ktest.pl | 44 ++++++++++++++++++++++++++++++++----
 1 file changed, 40 insertions(+), 4 deletions(-)
tools/testing/ktest/ktest.pl +34 -4
index e90e2e7cb72c..945a7d8c178c 100755
@@ -82,6 +82,8 @@ my %default = (
     "IGNORE_UNUSED"		=> 0,
 );
 
+my $test_log_start = 0;
+
 my $ktest_config = "ktest.conf";
 my $version;
 my $have_version = 0;
@@ -1492,8 +1494,21 @@ sub dodie {
 
     if ($email_on_error) {
 	my $name = get_test_name;
+	my $log_file;
+
+	if (defined($opt{"LOG_FILE"})) {
+	    $log_file = "$tmpdir/log";
+	    open (L, "$opt{LOG_FILE}") or die "Can't open $opt{LOG_FILE} to read)";
+	    open (O, "> $tmpdir/log") or die "Can't open $tmpdir/log\n";
+	    seek(L, $test_log_start, 0);
+	    while (<L>) {
+		print O;
+	    }
+	    close O;
+	    close L;
+	}
         send_email("KTEST: critical failure for test $i [$name]",
-                "Your test started at $script_start_time has failed with:\n@_\n");
+                "Your test started at $script_start_time has failed with:\n@_\n", $log_file);
     }
 
     if ($monitor_cnt) {
@@ -4185,7 +4200,7 @@ sub find_mailer {
 }
 
 sub do_send_mail {
-    my ($subject, $message) = @_;
+    my ($subject, $message, $file) = @_;
 
     if (!defined($mail_path)) {
 	# find the mailer
@@ -4195,22 +4210,37 @@ sub do_send_mail {
 	}
     }
 
+    my $header_file = "$tmpdir/header";
+    open (HEAD, ">$header_file") or die "Can not create $header_file\n";
+    print HEAD "To: $mailto\n";
+    print HEAD "Subject: $subject\n\n";
+    print HEAD "$message\n";
+    close HEAD;
+
     if (!defined($mail_command)) {
 	if ($mailer eq "mail" || $mailer eq "mailx") {
-	    $mail_command = "\$MAIL_PATH/\$MAILER -s \'\$SUBJECT\' \$MAILTO <<< \'\$MESSAGE\'";
+	    $mail_command = "cat \$HEADER_FILE \$BODY_FILE | \$MAIL_PATH/\$MAILER -s \'\$SUBJECT\' \$MAILTO";
 	} elsif ($mailer eq "sendmail" ) {
-	    $mail_command =  "echo \'Subject: \$SUBJECT\n\n\$MESSAGE\' | \$MAIL_PATH/\$MAILER -t \$MAILTO";
+	    $mail_command =  "cat \$HEADER_FILE \$BODY_FILE | \$MAIL_PATH/\$MAILER -t \$MAILTO";
 	} else {
 	    die "\nYour mailer: $mailer is not supported.\n";
 	}
     }
 
+    if (defined($file)) {
+	$mail_command =~ s/\$BODY_FILE/$file/g;
+    } else {
+	$mail_command =~ s/\$BODY_FILE//g;
+    }
+
+    $mail_command =~ s/\$HEADER_FILE/$header_file/g;
     $mail_command =~ s/\$MAILER/$mailer/g;
     $mail_command =~ s/\$MAIL_PATH/$mail_path/g;
     $mail_command =~ s/\$MAILTO/$mailto/g;
     $mail_command =~ s/\$SUBJECT/$subject/g;
     $mail_command =~ s/\$MESSAGE/$message/g;
 
+	    print ">$mail_command<\n";
     run_command $mail_command;
 }
 

tools/testing/ktest/ktest.pl for (my $i = 1; $i <= $opt{"NUM_TESTS"}; $i++) { · line 4388

Quoted 7 lines
tools/testing/ktest/ktest.pl +4 -0
@@ -4352,6 +4382,11 @@ for (my $i = 1; $i <= $opt{"NUM_TESTS"}; $i++) {
     }
 
     doprint "\n\n";
+
+    if (defined($opt{"LOG_FILE"})) {
+	$test_log_start = tell(LOG);
+    }

Nit, the mix of spaces and tabs in this file is always annoying to me, but it's your code...

tools/testing/ktest/ktest.pl if ($email_when_finished) { · line 4502

Quoted 12 lines
tools/testing/ktest/ktest.pl +2 -0
+
     doprint "RUNNING TEST $i of $opt{NUM_TESTS}$name with option $test_type $run_type$installme\n\n";
 
     if (defined($pre_test)) {
@@ -4461,6 +4496,7 @@ if ($email_when_finished) {
 }
 
 if (defined($opt{"LOG_FILE"})) {
+
     print "\n See $opt{LOG_FILE} for the record of results.\n\n";
     close LOG;
 }

Extra blank line?

Anyway, looks sane to me

author maintainer
#10 lore raw

Re: [for-next][PATCH 7/8] ktest.pl: Add the log of last test in email on failure

Quoted 9 lines · On Thu, 2 Jul 2020 09:42:35 +0200 Greg KH <[email protected]> wrote:

tools/testing/ktest/ktest.pl for (my $i = 1; $i <= $opt{"NUM_TESTS"}; $i++) { · line 4388

Quoted 5 lines
     doprint "\n\n";
tools/testing/ktest/ktest.pl +4 -0
+
+    if (defined($opt{"LOG_FILE"})) {
+	$test_log_start = tell(LOG);
+    }  

Nit, the mix of spaces and tabs in this file is always annoying to me, but it's your code...

It's the default way emacs does Perl code. I call it "oyster mode".

I like to keep it that way as it reminds me that this is Perl and not C. ;-)

Quoted 7 lines
Quoted 7 lines
tools/testing/ktest/ktest.pl +1 -0
+
     doprint "RUNNING TEST $i of $opt{NUM_TESTS}$name with option $test_type $run_type$installme\n\n";
 
     if (defined($pre_test)) {
@@ -4461,6 +4496,7 @@ if ($email_when_finished) {
 }
 
Quoted 7 lines

tools/testing/ktest/ktest.pl if ($email_when_finished) { · line 4502

Quoted 5 lines
tools/testing/ktest/ktest.pl +1 -0
 if (defined($opt{"LOG_FILE"})) {
+
     print "\n See $opt{LOG_FILE} for the record of results.\n\n";
     close LOG;
 }  

Extra blank line?

Will nuke. (Left over from having a debug print in there).

Quoted 4 lines

Anyway, looks sane to me

Awesome. Thanks for looking this over!

-- Steve

author maintainer
#11 lore raw

PATCH 8/8 ktest.pl: Add MAIL_MAX_SIZE to limit the amount of log emailed

In reply to Steven Rostedt

From: "Steven Rostedt (VMware)" <[email protected]>

Add the ktest config option MAIL_MAX_SIZE that will limit the size of the log file that is placed into the email on failure.

---
 tools/testing/ktest/ktest.pl    | 12 +++++++++++-
 tools/testing/ktest/sample.conf | 13 +++++++++++++
 2 files changed, 24 insertions(+), 1 deletion(-)
tools/testing/ktest/ktest.pl +11 -1
index 945a7d8c178c..d59a6b6bcfd5 100755
@@ -227,6 +227,7 @@ my $dirname = $FindBin::Bin;
 my $mailto;
 my $mailer;
 my $mail_path;
+my $mail_max_size;
 my $mail_command;
 my $email_on_error;
 my $email_when_finished;
@@ -263,6 +264,7 @@ my %option_map = (
     "MAILTO"			=> \$mailto,
     "MAILER"			=> \$mailer,
     "MAIL_PATH"			=> \$mail_path,
+    "MAIL_MAX_SIZE"		=> \$mail_max_size,
     "MAIL_COMMAND"		=> \$mail_command,
     "EMAIL_ON_ERROR"		=> \$email_on_error,
     "EMAIL_WHEN_FINISHED"	=> \$email_when_finished,
@@ -1497,10 +1499,18 @@ sub dodie {
 	my $log_file;
 
 	if (defined($opt{"LOG_FILE"})) {
+	    my $size = 0;
+	    if (defined($mail_max_size)) {
+		my $log_size = tell LOG;
+		$log_size -= $test_log_start;
+		if ($log_size > $mail_max_size) {
+		    $size = $log_size - $mail_max_size;
+		}
+	    }
 	    $log_file = "$tmpdir/log";
 	    open (L, "$opt{LOG_FILE}") or die "Can't open $opt{LOG_FILE} to read)";
 	    open (O, "> $tmpdir/log") or die "Can't open $tmpdir/log\n";
-	    seek(L, $test_log_start, 0);
+	    seek(L, $test_log_start + $size, 0);
 	    while (<L>) {
 		print O;
 	    }
tools/testing/ktest/sample.conf +13 -0
index cb8227fbd01e..5e7d1d729752 100644
@@ -442,6 +442,19 @@
 # Users can cancel the test by Ctrl^C
 # (default 0)
 #EMAIL_WHEN_CANCELED = 1
+#
+# If a test ends with an error and EMAIL_ON_ERROR is set as well
+# as a LOG_FILE is defined, then the log of the failing test will
+# be included in the email that is sent.
+# It is possible that the log may be very large, in which case,
+# only the last amount of the log should be sent. To limit how
+# much of the log is sent, set MAIL_MAX_SIZE. This will be the
+# size in bytes of the last portion of the log of the failed
+# test file. That is, if this is set to 100000, then only the
+# last 100 thousand bytes of the log file will be included in
+# the email.
+# (default undef)
+#MAIL_MAX_SIZE = 1000000
 
 # Start a test setup. If you leave this off, all options
 # will be default and the test will run once.
Signature · 1 line
2.26.2
Reviewed-by on v1 8/8
#12 lore raw

Re: [for-next][PATCH 8/8] ktest.pl: Add MAIL_MAX_SIZE to limit the amount of log emailed

Quoted 11 lines · Steven Rostedt

From: "Steven Rostedt (VMware)" <[email protected]>

Add the ktest config option MAIL_MAX_SIZE that will limit the size of the log file that is placed into the email on failure.

---
 tools/testing/ktest/ktest.pl    | 12 +++++++++++-
 tools/testing/ktest/sample.conf | 13 +++++++++++++
 2 files changed, 24 insertions(+), 1 deletion(-)

Interesting, but I like full log files for my reports :)

author maintainer
#13 lore raw

Re: [for-next][PATCH 8/8] ktest.pl: Add MAIL_MAX_SIZE to limit the amount of log emailed

Quoted 6 lines · On Thu, 2 Jul 2020 09:41:03 +0200 Greg KH <[email protected]> wrote:
Quoted 5 lines · Steven Rostedt

From: "Steven Rostedt (VMware)" <[email protected]>

Add the ktest config option MAIL_MAX_SIZE that will limit the size of the log file that is placed into the email on failure.

Quoted 8 lines
Quoted 6 lines
---
 tools/testing/ktest/ktest.pl    | 12 +++++++++++-
 tools/testing/ktest/sample.conf | 13 +++++++++++++
 2 files changed, 24 insertions(+), 1 deletion(-)  

Interesting, but I like full log files for my reports :)

I can add an option to do that if you want. My full logs end up being a few hundred megabytes. Perhaps I could add a compress option too.

Quoted 2 lines

Thanks!

-- Steve

#14 lore raw

Re: [for-next][PATCH 8/8] ktest.pl: Add MAIL_MAX_SIZE to limit the amount of log emailed

Quoted 16 lines · Steven Rostedt
Quoted 13 lines · On Thu, 2 Jul 2020 09:41:03 +0200 Greg KH <[email protected]> wrote:
Quoted 11 lines · Steven Rostedt

From: "Steven Rostedt (VMware)" <[email protected]>

Add the ktest config option MAIL_MAX_SIZE that will limit the size of the log file that is placed into the email on failure.

---
 tools/testing/ktest/ktest.pl    | 12 +++++++++++-
 tools/testing/ktest/sample.conf | 13 +++++++++++++
 2 files changed, 24 insertions(+), 1 deletion(-)  
Quoted 4 lines
Quoted 1 line

Interesting, but I like full log files for my reports :)

I can add an option to do that if you want. My full logs end up being a few hundred megabytes. Perhaps I could add a compress option too.

It's fine, the default should be good enough for me for now. If not, I'll just bump the value, or add compression.

thanks,

greg k-h-

author maintainer
#15 lore raw

Re: [for-next][PATCH 8/8] ktest.pl: Add MAIL_MAX_SIZE to limit the amount of log emailed

Quoted 5 lines · On Thu, 2 Jul 2020 14:34:02 +0200 Greg KH <[email protected]> wrote:
Quoted 2 lines

I can add an option to do that if you want. My full logs end up being a few hundred megabytes. Perhaps I could add a compress option too.

It's fine, the default should be good enough for me for now. If not, I'll just bump the value, or add compression.

If we compress, it would need to be an attachment. I'm guessing you are fine with that. Do you already make it an attachment?

BTW, my test just failed and for some reason it created a 13162697 byte log file to include in my email, which failed to send :-p

Strange that it did that, as I had the max set to 950000. Thus, I've changed this code to be:

tools/testing/ktest/ktest.pl +6 -3
index 917810fa4c85..9363a5b27339 100755
@@ -1499,18 +1499,21 @@ sub dodie {
        my $log_file;
 
        if (defined($opt{"LOG_FILE"})) {
-           my $size = 0;
+           my $whence = 0; # beginning of file
+           my $pos = $test_log_start;
+
            if (defined($mail_max_size)) {
                my $log_size = tell LOG;
                $log_size -= $test_log_start;
                if ($log_size > $mail_max_size) {
-                   $size = $log_size - $mail_max_size;
+                   $whence = 2; # end of file
+                   $pos = - $mail_max_size;
                }
            }
            $log_file = "$tmpdir/log";
            open (L, "$opt{LOG_FILE}") or die "Can't open $opt{LOG_FILE} to read)";
            open (O, "> $tmpdir/log") or die "Can't open $tmpdir/log\n";
-           seek(L, $test_log_start + $size, 0);
+           seek(L, $pos, $whence);
            while (<L>) {
                print O;
            }

Let's see if this now limits it :-/

-- Steve

#16 lore raw

Re: [for-next][PATCH 8/8] ktest.pl: Add MAIL_MAX_SIZE to limit the amount of log emailed

Quoted 11 lines · Steven Rostedt
Quoted 5 lines · On Thu, 2 Jul 2020 14:34:02 +0200 Greg KH <[email protected]> wrote:
Quoted 2 lines

I can add an option to do that if you want. My full logs end up being a few hundred megabytes. Perhaps I could add a compress option too.

It's fine, the default should be good enough for me for now. If not, I'll just bump the value, or add compression.

If we compress, it would need to be an attachment. I'm guessing you are fine with that. Do you already make it an attachment?

Yes I do, so attachments are fine.

thanks,

greg k-h