Skip to contentSkip to conversation

Integration of "Fault Injection Framework" into LTP

PATCH In review Subrata Modak 2 participants 10 messages Aug 17, 2009 – Aug 22, 2009

lore
Share & tools
lore
#0 lore raw

v2 cover Integration of "Fault Injection Framework" into LTP

Hi,

Introducing the "Kernel Fault Injection Framework" generation and testing capability in LTP. This is in line with the recent proposal made through LTP Paper at OLS 2009:
"Putting LTP to test - Validating both the Linux kernel and Test-cases"
(http://ltp.sourceforge.net/documentation/technical_papers/Putting_LTP_to_Test.pdf)

This infrasturcture will help LTP directly in the following ways:
1) Allow test developers to test their new test cases against a faulted kernel, and then compare it on stable kernel run, impacting in better test development,
2) Allow test engineers to be able to generate more code coverage by traversing the rarely touched parts of the kernel code,

As we move forward in using this, we would definitely find some other advantages of this framework. This is V2 of patchset after incorporating comments from Mike, Paul & Garret.

Regards--
Subrata

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list

#1 lore raw

PATCH v2 1/5 Provide all necessary information through ltp/README

Provide all necessary information to create/use "Fault Injection Framework" through ltp/README. This is necessary before any test case(s) can be run on this harness. Also describes the general algorithm that would be followed while running LTP tests in "Fault Injection" harness.

Changed the debugfs mount point from /debug/ to /sys/kernel/debug/ as pointed out by Mike.

---

README +107 -1
@@ -272,7 +272,6 @@ The tests also require CUnit Framework t
 http://sourceforge.net/projects/cunit/
 ---------------------------------
 ---------------------------------
-
 Native language support (nls) testsuite requirements
 ----------------------------------------------------
 CONFIG_NLS=m
@@ -287,4 +286,111 @@ CONFIG_CAN_BCM=m
 # CAN Device Drivers
 CONFIG_CAN_VCAN=m
 ---------------------------------
+Enabling Fault Injection Support for your kernel (version 2.6.29).
+Please check with the original kernel for the fault injection
+types it supports. Following supports will be available:
+
+/sys/kernel/debug/fail_io_timeout/interval
+/sys/kernel/debug/fail_io_timeout/probability
+/sys/kernel/debug/fail_io_timeout/reject-end
+/sys/kernel/debug/fail_io_timeout/reject-start
+/sys/kernel/debug/fail_io_timeout/require-end
+/sys/kernel/debug/fail_io_timeout/require-start
+/sys/kernel/debug/fail_io_timeout/space
+/sys/kernel/debug/fail_io_timeout/stacktrace-depth
+/sys/kernel/debug/fail_io_timeout/task-filter
+/sys/kernel/debug/fail_io_timeout/times
+/sys/kernel/debug/fail_io_timeout/verbose
+
+/sys/kernel/debug/fail_make_request/interval
+/sys/kernel/debug/fail_make_request/probability
+/sys/kernel/debug/fail_make_request/reject-end
+/sys/kernel/debug/fail_make_request/reject-start
+/sys/kernel/debug/fail_make_request/require-end
+/sys/kernel/debug/fail_make_request/require-start
+/sys/kernel/debug/fail_make_request/space
+/sys/kernel/debug/fail_make_request/stacktrace-depth
+/sys/kernel/debug/fail_make_request/task-filter
+/sys/kernel/debug/fail_make_request/times
+/sys/kernel/debug/fail_make_request/verbose
+
+/sys/kernel/debug/fail_page_alloc/ignore-gfp-highmem
+/sys/kernel/debug/fail_page_alloc/ignore-gfp-wait
+/sys/kernel/debug/fail_page_alloc/interval
+/sys/kernel/debug/fail_page_alloc/min-order
+/sys/kernel/debug/fail_page_alloc/probability
+/sys/kernel/debug/fail_page_alloc/reject-end
+/sys/kernel/debug/fail_page_alloc/reject-start
+/sys/kernel/debug/fail_page_alloc/require-end
+/sys/kernel/debug/fail_page_alloc/require-start
+/sys/kernel/debug/fail_page_alloc/space
+/sys/kernel/debug/fail_page_alloc/stacktrace-depth
+/sys/kernel/debug/fail_page_alloc/task-filter
+/sys/kernel/debug/fail_page_alloc/times
+/sys/kernel/debug/fail_page_alloc/verbose
+
+/sys/kernel/debug/failslab/ignore-gfp-wait
+/sys/kernel/debug/failslab/interval
+/sys/kernel/debug/failslab/probability
+/sys/kernel/debug/failslab/reject-end
+/sys/kernel/debug/failslab/reject-start
+/sys/kernel/debug/failslab/require-end
+/sys/kernel/debug/failslab/require-start
+/sys/kernel/debug/failslab/space
+/sys/kernel/debug/failslab/stacktrace-depth
+/sys/kernel/debug/failslab/task-filter
+/sys/kernel/debug/failslab/times
+/sys/kernel/debug/failslab/verbose
+
+when the below kernel config options are set:
+
+CONFIG_FAULT_INJECTION=y
+CONFIG_DEBUG_KERNEL=y
+CONFIG_FAILSLAB=y (Fault-injection capability for kmalloc)
+(CONFIG_SLAB=y || CONFIG_SLUB=y) if CONFIG_FAILSLAB=y
+CONFIG_FAIL_PAGE_ALLOC=y (Fault-injection capabilitiy for alloc_pages())
+CONFIG_FAIL_MAKE_REQUEST=y (Fault-injection capability for disk IO)
+CONFIG_BLOCK=y if CONFIG_FAIL_MAKE_REQUEST=y
+CONFIG_FAIL_IO_TIMEOUT=y (Faul-injection capability for faking disk interrupts)
+CONFIG_BLOCK=y if CONFIG_FAIL_IO_TIMEOUT=y
+CONFIG_FAULT_INJECTION_DEBUG_FS=y (Debugfs entries for fault-injection capabilities)
+(CONFIG_SYSFS=y && CONFIG_DEBUG_FS=y) if CONFIG_FAULT_INJECTION_DEBUG_FS=y
+CONFIG_FAULT_INJECTION_STACKTRACE_FILTER=y (stacktrace filter for fault-injection capabilities)
+(CONFIG_FAULT_INJECTION_DEBUG_FS=y && CONFIG_STACKTRACE_SUPPORT=y && !CONFIG_X86_64) if
+	CONFIG_FAULT_INJECTION_STACKTRACE_FILTER=y
+
+For more information on Fault injection, please refer to:
+linux-2.6/Documentation/fault-injection/fault-injection.txt,
+
+You should also have made the following entries in your /etc/fstab file
+once the kernel is booted with the above CONFIG options set:
+
+debugfs		/sys/kernel/debug/		debugfs
+
+# How the Kernel Fault Injection works for LTP ?
+
+1) Build Kernel with all the above possible kernel CONFIG Options,
+2) Create the above entry in /etc/fstab file,
+3) Reboot in the new kernel,
+4) Goto LTPROOT. Build and Install LTP as per ltp/INSTALL file,
+5) Choose your own test(or default) to run with fault injection as follows:
+	./runltp -f <command_file> -F <LOOPS>,<FAULT_PROBABILITY>
+
+The agorithm will work as:
+loop (for each testcase)
+begin
+	execute_testcase(inside_stable_kernel)
+	begin
+		insert_fault_into_kernel()
+		loop X Times
+		begin
+			execute_testcase(inside_fault_kernel)
+		end
+		restore_kernel_to_normal()
+	end
+end
+
+# Eternal TODOs:
+1) Add as many framework as they get added/modifed in the kernel
+---------------------------------
 

---
Regards--
Subrata

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list

#2 lore raw

PATCH v2 2/5 Add Script which would actually do the job of injecting faults

In reply to Subrata Modak

Script which would actually do the job of injecting faults by changing various parametrs available under /debug/fail*. This would be done dynamically during LTP run, and, is capable of taking parameters from 0 to 100 to vary the probability of Fault the user wants to inject in the running kernel.

Incorporated Mike�� & Garrets comments and concerns.

---

tools/insert_kernel_faults.sh +53 -0
@@ -0,0 +1,53 @@
+#!/bin/sh
+################################################################################
+##                                                                            ##
+## Copyright (c) International Business Machines  Corp., 2009                 ##
+##                                                                            ##
+## This program is free software;  you can redistribute it and/or modify      ##
+## it under the terms of the GNU General Public License as published by       ##
+## the Free Software Foundation; either version 2 of the License, or          ##
+## (at your option) any later version.                                        ##
+##                                                                            ##
+## This program is distributed in the hope that it will be useful, but        ##
+## WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY ##
+## or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License   ##
+## for more details.                                                          ##
+##                                                                            ##
+## You should have received a copy of the GNU General Public License          ##
+## along with this program;  if not, write to the Free Software               ##
+## Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA    ##
+##                                                                            ##
+################################################################################
+#									      ##
+# File :        insert_kernel_faults.sh					      ##
+#									      ##
+# Usage:	insert_kernel_faults.sh <fault_percentage>		      ##
+#									      ##
+# Description:  This is a simple script that inserts faults at various	      ##
+#		subsystems of the kernel. Please refer to the ltp/README      ##
+#		for the various kernel CONFIG options needed to exploit	      ##
+#		all those features					      ##
+#									      ##
+# Author:	Subrata Modak <[email protected]>		      ##
+#									      ##
+# History:      Aug 11 2009 - Created - Subrata Modak.			      ##
+#		Aug 17 2009 - Changed the debugfs mount point - Subrata Modak.##
+################################################################################
+
+if [ -z $1 ]
+	then
+	#Check if Useage has been proper
+	echo "Usage: $0 <fault_percentage>"
+	exit 1
+fi
+
+#These are the types of Subsystems where fault will be injected
+#Make sure debugfs has been mounted
+for FAILTYPE in fail_io_timeout fail_make_request fail_page_alloc failslab
+do
+	echo $1 > /sys/kernel/debug/$FAILTYPE/probability
+	echo 100 > /sys/kernel/debug/$FAILTYPE/interval
+	echo -1 > /sys/kernel/debug/$FAILTYPE/times
+	echo 0 > /sys/kernel/debug/$FAILTYPE/space
+done
+

---
Regards--
Subrata

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july

_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list

#3 lore raw

Re: [LTP] [PATCH v2 02/05] Add Script which would actually do the job of injecting faults

Hi Subrata,

Quoted 40 lines · Subrata Modak

Script which would actually do the job of injecting faults by changing various parametrs available under /debug/fail*. This would be done dynamically during LTP run, and, is capable of taking parameters from 0 to 100 to vary the probability of Fault the user wants to inject in the running kernel.

Incorporated Mikeś & Garrets comments and concerns.

---

tools/insert_kernel_faults.sh     2009-08-17 12:17:31.000000000 +0530 renamed from tools/insert_kernel_faults.sh        1970-01-01 05:30:00.000000000 +0530 +27 -0
@@ -0,0 +1,53 @@
+#!/bin/sh
+################################################################################
+##                                                                            ##
+## Copyright (c) International Business Machines  Corp., 2009                 ##
+##                                                                            ##
+## This program is free software;  you can redistribute it and/or modify      ##
+## it under the terms of the GNU General Public License as published by       ##
+## the Free Software Foundation; either version 2 of the License, or          ##
+## (at your option) any later version.                                        ##
+##                                                                            ##
+## This program is distributed in the hope that it will be useful, but        ##
+## WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY ##
+## or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License   ##
+## for more details.                                                          ##
+##                                                                            ##
+## You should have received a copy of the GNU General Public License          ##
+## along with this program;  if not, write to the Free Software               ##
+## Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA    ##
+##                                                                            ##
+################################################################################
+#                                                                            ##
+# File :        insert_kernel_faults.sh                                              ##
+#                                                                            ##
+# Usage:       insert_kernel_faults.sh <fault_percentage>                    ##
+#                                                                            ##
+# Description:  This is a simple script that inserts faults at various       ##
+#              subsystems of the kernel. Please refer to the ltp/README      ##

tools/insert_kernel_faults.sh     2009-08-17 12:17:31.000000000 +0530 line 37

Quoted 10 lines
tools/insert_kernel_faults.sh     2009-08-17 12:17:31.000000000 +0530 renamed from tools/insert_kernel_faults.sh        1970-01-01 05:30:00.000000000 +0530 +10 -0
+#              for the various kernel CONFIG options needed to exploit       ##
+#              all those features                                            ##
+#                                                                            ##
+# Author:      Subrata Modak <[email protected]>                    ##
+#                                                                            ##
+# History:      Aug 11 2009 - Created - Subrata Modak.                       ##
+#              Aug 17 2009 - Changed the debugfs mount point - Subrata Modak.##
+################################################################################
+
+if [ -z $1 ]

One concern: $1 should be quoted.

Thanks!
-Garrett
------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list

#4 lore raw

Re: [LTP] [PATCH v2 02/05] Add Script which would actually do the job of injecting faults

Quoted 48 lines · Garrett Cooper

Hi Subrata,

Quoted 44 lines · Subrata Modak

Script which would actually do the job of injecting faults by changing various parametrs available under /debug/fail*. This would be done dynamically during LTP run, and, is capable of taking parameters from 0 to 100 to vary the probability of Fault the user wants to inject in the running kernel.

Incorporated Mikeś & Garrets comments and concerns.

---

tools/insert_kernel_faults.sh 2009-08-17 12:17:31.000000000 +0530 renamed from tools/insert_kernel_faults.sh 1970-01-01 05:30:00.000000000 +0530 +31 -0
@@ -0,0 +1,53 @@
+#!/bin/sh
+################################################################################
+##                                                                            ##
+## Copyright (c) International Business Machines  Corp., 2009                 ##
+##                                                                            ##
+## This program is free software;  you can redistribute it and/or modify      ##
+## it under the terms of the GNU General Public License as published by       ##
+## the Free Software Foundation; either version 2 of the License, or          ##
+## (at your option) any later version.                                        ##
+##                                                                            ##
+## This program is distributed in the hope that it will be useful, but        ##
+## WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY ##
+## or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License   ##
+## for more details.                                                          ##
+##                                                                            ##
+## You should have received a copy of the GNU General Public License          ##
+## along with this program;  if not, write to the Free Software               ##
+## Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA    ##
+##                                                                            ##
+################################################################################
+#                                                                            ##
+# File :        insert_kernel_faults.sh                                              ##
+#                                                                            ##
+# Usage:       insert_kernel_faults.sh <fault_percentage>                    ##
+#                                                                            ##
+# Description:  This is a simple script that inserts faults at various       ##
+#              subsystems of the kernel. Please refer to the ltp/README      ##
+#              for the various kernel CONFIG options needed to exploit       ##
+#              all those features                                            ##
+#                                                                            ##
+# Author:      Subrata Modak <[email protected]>                    ##
Quoted 8 lines

tools/insert_kernel_faults.sh 2009-08-17 12:17:31.000000000 +0530 line 37

Quoted 6 lines
tools/insert_kernel_faults.sh 2009-08-17 12:17:31.000000000 +0530 renamed from tools/insert_kernel_faults.sh 1970-01-01 05:30:00.000000000 +0530 +6 -0
+#                                                                            ##
+# History:      Aug 11 2009 - Created - Subrata Modak.                       ##
+#              Aug 17 2009 - Changed the debugfs mount point - Subrata Modak.##
+################################################################################
+
+if [ -z $1 ]

One concern: $1 should be quoted.

Thanks Garret. Surely i will take care of that before checking in.

Regards--
Subrata

Quoted 3 lines

Thanks!
-Garrett

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list

#5 lore raw

PATCH v2 3/5 Add Script so the kernel is restored back to its original pristine form

In reply to Subrata Modak

Once the faults has been injected and all the concerned tests have been run completely, the kernel needs to be restored back to its original pristine form so that it is stable again. This Script does just exactly that.

This again has changes regarding the debugfs mount point.

---

tools/restore_kernel_faults_default.sh +86 -0
diff -uprN ltp-full-20090731.orig/tools/restore_kernel_faults_default.sh ltp-full-20090731/tools/restore_kernel_faults_default.sh
@@ -0,0 +1,86 @@
+#!/bin/sh
+################################################################################
+##                                                                            ##
+## Copyright (c) International Business Machines  Corp., 2009                 ##
+##                                                                            ##
+## This program is free software;  you can redistribute it and/or modify      ##
+## it under the terms of the GNU General Public License as published by       ##
+## the Free Software Foundation; either version 2 of the License, or          ##
+## (at your option) any later version.                                        ##
+##                                                                            ##
+## This program is distributed in the hope that it will be useful, but        ##
+## WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY ##
+## or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License   ##
+## for more details.                                                          ##
+##                                                                            ##
+## You should have received a copy of the GNU General Public License          ##
+## along with this program;  if not, write to the Free Software               ##
+## Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA    ##
+##                                                                            ##
+################################################################################
+#                                                                             ##
+# File :        restore_kernel_faults_default.sh			      ##
+#                                                                             ##
+# Usage:        restore_kernel_faults_default.sh	                      ##
+#                                                                             ##
+# Description:  This is a simple script that will restore the /debugfs/fail*  ##
+#		entries to their default values				      ##
+#                                                                             ##
+# Author:       Subrata Modak <[email protected]>                    ##
+#                                                                             ##
+# History:      Aug 11 2009 - Created - Subrata Modak.                        ##
+#		Aug 17 2009 - Changed debugfs mount point - Subrata Modak.    ##
+################################################################################
+
+echo  0 > /sys/kernel/debug/fail_io_timeout/reject-end
+echo  0 > /sys/kernel/debug/fail_io_timeout/reject-start
+echo  4294967295 > /sys/kernel/debug/fail_io_timeout/require-end
+echo  0 > /sys/kernel/debug/fail_io_timeout/require-start
+echo  32 > /sys/kernel/debug/fail_io_timeout/stacktrace-depth
+echo  N > /sys/kernel/debug/fail_io_timeout/task-filter
+echo  2 > /sys/kernel/debug/fail_io_timeout/verbose
+echo  0 > /sys/kernel/debug/fail_io_timeout/space
+echo  1 > /sys/kernel/debug/fail_io_timeout/times
+echo  1 > /sys/kernel/debug/fail_io_timeout/interval
+echo  0 > /sys/kernel/debug/fail_io_timeout/probability
+
+echo  0 > /sys/kernel/debug/fail_make_request/reject-end
+echo  0 > /sys/kernel/debug/fail_make_request/reject-start
+echo  4294967295 > /sys/kernel/debug/fail_make_request/require-end
+echo  0 > /sys/kernel/debug/fail_make_request/require-start
+echo  32 > /sys/kernel/debug/fail_make_request/stacktrace-depth
+echo  N > /sys/kernel/debug/fail_make_request/task-filter
+echo  2 > /sys/kernel/debug/fail_make_request/verbose
+echo  0 > /sys/kernel/debug/fail_make_request/space
+echo  1 > /sys/kernel/debug/fail_make_request/times
+echo  1 > /sys/kernel/debug/fail_make_request/interval
+echo  0 > /sys/kernel/debug/fail_make_request/probability
+
+echo  1 > /sys/kernel/debug/fail_page_alloc/min-order
+echo  Y > /sys/kernel/debug/fail_page_alloc/ignore-gfp-highmem
+echo  Y > /sys/kernel/debug/fail_page_alloc/ignore-gfp-wait
+echo  0 > /sys/kernel/debug/fail_page_alloc/reject-end
+echo  0 > /sys/kernel/debug/fail_page_alloc/reject-start
+echo  4294967295 > /sys/kernel/debug/fail_page_alloc/require-end
+echo  0 > /sys/kernel/debug/fail_page_alloc/require-start
+echo  32 > /sys/kernel/debug/fail_page_alloc/stacktrace-depth
+echo  N > /sys/kernel/debug/fail_page_alloc/task-filter
+echo  2 > /sys/kernel/debug/fail_page_alloc/verbose
+echo  0 > /sys/kernel/debug/fail_page_alloc/space
+echo  1 > /sys/kernel/debug/fail_page_alloc/times
+echo  1 > /sys/kernel/debug/fail_page_alloc/interval
+echo  0 > /sys/kernel/debug/fail_page_alloc/probability
+
+echo  Y > /sys/kernel/debug/failslab/ignore-gfp-wait
+echo  0 > /sys/kernel/debug/failslab/reject-end
+echo  0 > /sys/kernel/debug/failslab/reject-start
+echo  4294967295 > /sys/kernel/debug/failslab/require-end
+echo  0 > /sys/kernel/debug/failslab/require-start
+echo  32 > /sys/kernel/debug/failslab/stacktrace-depth
+echo  N > /sys/kernel/debug/failslab/task-filter
+echo  2 > /sys/kernel/debug/failslab/verbose
+echo  0 > /sys/kernel/debug/failslab/space
+echo  1 > /sys/kernel/debug/failslab/times
+echo  1 > /sys/kernel/debug/failslab/interval
+echo  0 > /sys/kernel/debug/failslab/probability
+

---
Regards--
Subrata

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list

#6 lore raw

PATCH v2 4/5 Add Script which will be at the heart of this infrastructure

In reply to Subrata Modak

At the heart of this infrastructure is this Script, which will actually:
1) Change the temporary command file generated by runltp,
2) Create a new temporary command file which will have the following entries against each one entry in the command file:
i) Same TAG COMMAND_LINE entry,
ii) Entry to call the script to insert faults,
iii) Entry to run as many loops as specified by the user, iv) Entry to call the script to restore kernel to default state,

It is capable of creating new entries in the temporary command file with the following tags and command lines:
TAG_NAME=tag1, COMMANDLINE="test1",
TAG_NAME=tag1_loop1_under_kernel_fault, COMMANDLINE="insert_fault_in_kernel; test1",
TAG_NAME=tag1_loop2_under_kernel_fault, COMMANDLINE="test1",
...
TAG_NAME=tag1_loopn_under_kernel_fault, COMMANDLINE="test1; restore_default_kernel",

Changes from V1 include:
1) Paul's suggestion to tag the results to say when a fault was, or, was not in the process of being generated,

Garrett and Mike wanted this to be in Shell script. So, Garret will change this to a Shell script in future.

---

tools/create_kernel_faults_in_loops_and_probability.pl +100 -0
diff -uprN ltp-full-20090731.orig/tools/create_kernel_faults_in_loops_and_probability.pl ltp-full-20090731/tools/create_kernel_faults_in_loops_and_probability.pl
@@ -0,0 +1,100 @@
+#!/usr/bin/perl
+################################################################################
+##                                                                            ##
+## Copyright (c) International Business Machines  Corp., 2009                 ##
+##                                                                            ##
+## This program is free software;  you can redistribute it and/or modify      ##
+## it under the terms of the GNU General Public License as published by       ##
+## the Free Software Foundation; either version 2 of the License, or          ##
+## (at your option) any later version.                                        ##
+##                                                                            ##
+## This program is distributed in the hope that it will be useful, but        ##
+## WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY ##
+## or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License   ##
+## for more details.                                                          ##
+##                                                                            ##
+## You should have received a copy of the GNU General Public License          ##
+## along with this program;  if not, write to the Free Software               ##
+## Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA    ##
+##                                                                            ##
+################################################################################
+#                                                                             ##
+# File :        create_kernel_faults_in_loops_and_probability.pl	      ##
+#                                                                             ##
+# Usage:        create_kernel_faults_in_loops_and_probability.pl\	      ##
+#		<LTP_COMMAND_FILE> <NO_OF_LOOPS_EACH_TEST_WILL_RUN>\	      ##
+#		<PROBABILITY_OF_FAULT_INJECTION>			      ##
+#                                                                             ##
+# Description:  This is a simple perl script which will take ltp command file ##
+#		as input and then create a final command file while will have ##
+#		the following entries for each test tag:		      ##
+#		1) <tag_name_loop1> <test_binary_name>			      ##
+#		2) <tag_name_loop2> <insert_kernel_faults.sh test_binary_name>##
+#		3) tag entried from loop3 to loop(n-1)			      ##
+#		4) <tag_name_loopn_under_kernel_fault> <restore_kernel_faults_default.sh test_binary_name>##
+#                                                                             ##
+# Author:       Subrata Modak <[email protected]>                    ##
+#                                                                             ##
+# History:      Aug 11 2009 - Created - Subrata Modak.                        ##
+#		Aug 17 2009 - Made some changes as specified by Paul Larson   ##
+#		1) tag the results to say when a fault was or was not in the  ##
+#		process of being generated - Subrata Modak.		      ##
+################################################################################
+
+my $command_file	= shift (@ARGV) || syntax();
+my $loops		= shift (@ARGV) || syntax();
+my $failure_probability	= shift (@ARGV) || syntax();
+
+sub syntax() {
+	print "syntax: create_fault_in_loops_and_probability.pl\
+	<LTP_COMMAND_FILE> <NO_OF_LOOPS_EACH_TEST_WILL_RUN>\
+	<PROBABILITY_OF_FAULT_INJECTION>\n";
+	exit (1);
+}
+#$ENV{TEST_START_TIME})
+
+
+open (FILE, $command_file) || die "Cannot open file: $command_file\n";
+while ($line = <FILE>) {
+	if ($line =~ /^#/) {
+		print "$line";
+		next;
+	}
+	if ($line =~ /^\n$/) {
+		next;
+	}
+	chomp $line;
+	print "$line\n"; #Print one instance for stable execution
+	@tag_and_actual_command = split(/\ /, $line);
+
+	#The remaining loops should be running under fault injection
+	for ($counter=1; $counter<=$loops; $counter++) {
+		my $token_counter = 0;
+		foreach my $token (@tag_and_actual_command) {
+			if ($token_counter == 0 ) {
+				#Time to append the actual command tag with the loop no.
+				print $token . "_loop_" . $counter . "_under_fault_kernel ";
+				$token_counter++;
+				next;
+			}
+			if ($token_counter == 1 && $counter == 1) {
+				#Time to include the fault injection script in the first loop
+				print "\$LTPROOT/tools/insert_kernel_faults.sh " . $failure_probability . "; " . $token;
+				$token_counter++;
+				next;
+			}
+			print " " . $token . " ";
+		}
+		if ($counter == $loops) {
+			#Time to withdraw the faults once the last loop has been executed
+			#Until all faults has been successfully restored to default values...
+			#Keep restoring them
+			print "; " . "\$LTPROOT/tools/restore_kernel_faults_default.sh; RC=\$?; while [ \$RC -ne 0 ]; do \$LTPROOT/tools/restore_kernel_faults_default.sh; RC=\$?; done\n"
+		} else {
+			print "\n"
+		}
+	}
+
+}
+close (FILE);
+

---
Regards--
Subrata

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list

#7 lore raw

PATCH v2 5/5 Add the necessary Interface and Option through "runltp"

In reply to Subrata Modak

Change the runltp script to actually create an interface for the user:
1) Introduce a new option "-F" for ability to run tests under "Fault Injection Framework",
2) "./runltp -h" will display the new option,
3) Verifies whether Kernel has built-in capabilities for "Fault Injection",

This has no much difference from the last version, except that 'runltp' has changed in between to include Paul's -S option. This has just been created over top of it. I will try to move the 'debugfs' checking logic out of 'runltp' some time in future.

---

runltp +42 -1
@@ -62,6 +62,8 @@
 #               - Added a new option to mail back LTP reports
 #               May 19 2008 - Modified - Subrata Modak
 #               - Added capability for default Log file generation
+#		Aug 17 2009 - Modified - Subrata Modak
+#		- Added Fault Injection generation Capability through -F Option
 #
 #################################################################################
 
@@ -108,6 +110,7 @@ usage() 
     [ -i NUM_PROCS ] [ -l LOGFILE ] [ -m NUM_PROCS,CHUNKS,BYTES,HANGUP_FLAG ] 
     -N -n [ -o OUTPUTFILE ] -p -q [ -r LTPROOT ] [ -s PATTERN ] [ -t DURATION ]
     -v [ -w CMDFILEADDR ] [ -x INSTANCES ] [ -b DEVICE ] [-B DEVICE_FS_TYPE]
+	[ -F LOOPS,PERCENTAGE ]
                 
     -a EMAIL_TO     EMAIL all your Reports to this E-mail Address
     -c NUM_PROCS    Run LTP under additional background CPU load
@@ -123,6 +126,7 @@ usage() 
                     [CLEAN_FLAG  = unlink file to which random data written, when value 1]
     -e              Prints the date of the current LTP release
     -f CMDFILES     Execute user defined list of testcases (separate with ',')
+	-F LOOPS,PERCENTAGE Induce PERCENTAGE Fault in the Kernel Subsystems, and, run each test for LOOPS loop
     -g HTMLFILE     Create an additional HTML output format
     -h              Help. Prints all available options.
     -i NUM_PROCS    Run LTP under additional background Load on IO Bus
@@ -181,6 +185,9 @@ main()
     local DURATION=""
     local CMDFILEADDR=""
     local FAILCMDFILE=""
+	local INJECT_KERNEL_FAULT=""
+	local INJECT_KERNEL_FAULT_PERCENTAGE=""
+	local INJECT_FAULT_LOOPS_PER_TEST=""
     local LOGFILE_NAME=""
     local LOGFILE=""
     local OUTPUTFILE_NAME=""
@@ -194,7 +201,7 @@ main()
     local DEFAULT_FILE_NAME_GENERATION_TIME=`date +"%Y_%b_%d-%Hh_%Mm_%Ss"`
     version_date=`head -n 1 $LTPROOT/ChangeLog`
 
-    while getopts a:c:C:d:D:f:ehi:g:l:m:Nno:pqr:s:S:t:T:vw:x:b:B: arg
+    while getopts a:c:C:d:D:f:F:ehi:g:l:m:Nno:pqr:s:S:t:T:vw:x:b:B: arg
     do  case $arg in
         a)  EMAIL_TO=$OPTARG
             ALT_EMAIL_OUT=1;;
@@ -264,6 +271,18 @@ main()
             # Can be more then one file, just separate it with ',', like:
             # -f nfs,commands,/tmp/testfile
             CMDFILES=$OPTARG;;
+	F)	INJECT_KERNEL_FAULT=1
+		#Seperate out the NO_OF_LOOPS & FAULT_PERCENTAGE
+		INJECT_FAULT_LOOPS_PER_TEST=`echo $OPTARG |cut -d',' -f1 | tr -d '\n' | tr -d ' '`
+		INJECT_KERNEL_FAULT_PERCENTAGE=`echo $OPTARG |cut -d',' -f2 | tr -d '\n' | tr -d ' '`
+		if [ ! $INJECT_FAULT_LOOPS_PER_TEST ]; then
+			echo "Loops not properly defined. Resorting to default 5..."
+			export INJECT_FAULT_LOOPS_PER_TEST=5
+		fi
+		if [ ! $INJECT_KERNEL_FAULT_PERCENTAGE ]; then
+			echo "Fault Persentage not properly defined. Resorting to default 10..."
+			export INJECT_KERNEL_FAULT_PERCENTAGE=10
+		fi;;
         g)  HTMLFILE_NAME="$OPTARG"
             case $OPTARG in
             /*)
@@ -754,6 +773,28 @@ main()
     echo "Running tests......."
     test_start_time=$(date)
 
+	# User wants testing with Kernel Fault Injection
+	if [ $INJECT_KERNEL_FAULT -eq 1 ] ; then
+		#See if Debugfs is mounted, and
+		#Fault Injection Framework available through Debugfs
+		if [ -d "/sys/kernel/debug/fail_io_timeout" -o \
+			-d "/sys/kernel/debug/fail_make_request" -o \
+			-d "/sys/kernel/debug/fail_page_alloc" -o \
+			-d "/sys/kernel/debug/failslab" ]; then
+			#If atleast one of the Framework is available
+			#Go ahead to Inject Fault & Create required
+			#Command Files for LTP run
+			echo Running tests with Fault Injection Enabled in the Kernel...
+			${LTPROOT}/tools/create_kernel_faults_in_loops_and_probability.pl\
+				${TMP}/alltests $INJECT_FAULT_LOOPS_PER_TEST $INJECT_KERNEL_FAULT_PERCENTAGE > ${TMP}/alltests.tmp
+			cp ${TMP}/alltests.tmp ${TMP}/alltests
+			rm -rf ${TMP}/alltests.tmp
+		else
+			echo Fault Injection not enabled in the Kernel..
+			echo Running tests normally...
+		fi
+	fi
+
     # Some tests need to run inside the "bin" directory.
     cd "${LTPROOT}/testcases/bin"
     ${LTPROOT}/pan/ltp-pan $QUIET_MODE -e -S $INSTANCES $DURATION -a $$ -n $$ $PRETTY_PRT -f ${TMP}/alltests $LOGFILE $OUTPUTFILE $FAILCMDFILE

---
Regards--
Subrata

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list

#8 lore raw

PATCH v2 1/1 Results of test run for "Fault Injection Framework"

In reply to Subrata Modak

Again comes the actual test output the framework will generate. Below are the test results with and without this Infrastructure on my following machine:

# uname -a
Linux 2.6.29-gcov #1 SMP Fri Jun 19 12:14:27 IST 2009 i686 i686 i386 GNU/Linux

================================================================================

# ./runltp -f math -o ltp_without_fault_injection.out
================================================================================
<<<test_start>>>
tag=abs01 stime=1250492520
cmdline="abs01"
contacts=""
analysis=exit
<<<test_output>>>
abs01       1  TPASS  :  Test passed
abs01       2  TPASS  :  Test passed
abs01       3  TPASS  :  Test passed
<<<execution_status>>>
initiation_status="ok"
duration=0 termination_type=exited termination_id=0 corefile=no
cutime=0 cstime=0
<<<test_end>>>
<<<test_start>>>
tag=atof01 stime=1250492520
cmdline="atof01"
contacts=""
analysis=exit
<<<test_output>>>
atof01      1  TPASS  :  Test passed
atof01      2  TPASS  :  Test passed
atof01      3  TPASS  :  Test passed
atof01      4  TPASS  :  Test passed
<<<execution_status>>>
initiation_status="ok"
duration=0 termination_type=exited termination_id=0 corefile=no
cutime=0 cstime=0
<<<test_end>>>
<<<test_start>>>
tag=fptest01 stime=1250492520
cmdline="fptest01"
contacts=""
analysis=exit
<<<test_output>>>
fptest01    1  TPASS  :  PASS
<<<execution_status>>>
initiation_status="ok"
duration=0 termination_type=exited termination_id=0 corefile=no
cutime=0 cstime=0
<<<test_end>>>
<<<test_start>>>
tag=fptest02 stime=1250492520
cmdline="fptest02"
contacts=""
analysis=exit
<<<test_output>>>
fptest02    1  TPASS  :  PASS
<<<execution_status>>>
initiation_status="ok"
duration=0 termination_type=exited termination_id=0 corefile=no
cutime=1 cstime=0
<<<test_end>>>
<<<test_start>>>
tag=nextafter01 stime=1250492520
cmdline="nextafter01"
contacts=""
analysis=exit
<<<test_output>>>
nextafter01    1  TPASS  :  Test passed
nextafter01    2  TPASS  :  Test passed
nextafter01    3  TPASS  :  Test passed
incrementing stop
<<<execution_status>>>
initiation_status="ok"
duration=0 termination_type=exited termination_id=0 corefile=no
cutime=0 cstime=1
<<<test_end>>>
================================================================================

================================================================================

# ./runltp -f math -F 6,15 -o ltp_with_fault_injection.out
================================================================================
<<<test_start>>>
tag=abs01 stime=1250492531
cmdline="abs01"
contacts=""
analysis=exit
<<<test_output>>>
abs01       1  TPASS  :  Test passed
abs01       2  TPASS  :  Test passed
abs01       3  TPASS  :  Test passed
<<<execution_status>>>
initiation_status="ok"
duration=0 termination_type=exited termination_id=0 corefile=no
cutime=0 cstime=0
<<<test_end>>>
<<<test_start>>>
tag=abs01_loop_1under_fault_kernel stime=1250492531
cmdline="$LTPROOT/tools/insert_kernel_faults.sh 15; abs01"
contacts=""
analysis=exit
<<<test_output>>>
abs01       1  TPASS  :  Test passed
abs01       2  TPASS  :  Test passed
abs01       3  TPASS  :  Test passed
<<<execution_status>>>
initiation_status="ok"
duration=0 termination_type=exited termination_id=0 corefile=no
cutime=0 cstime=1
<<<test_end>>>
<<<test_start>>>
tag=abs01_loop_2under_fault_kernel stime=1250492531
cmdline=" abs01 "
contacts=""
analysis=exit
<<<test_output>>>
abs01       1  TPASS  :  Test passed
abs01       2  TPASS  :  Test passed
abs01       3  TPASS  :  Test passed
<<<execution_status>>>
initiation_status="ok"
duration=0 termination_type=exited termination_id=0 corefile=no
cutime=0 cstime=0
<<<test_end>>>
<<<test_start>>>
tag=abs01_loop_3under_fault_kernel stime=1250492531
cmdline=" abs01 "
contacts=""
analysis=exit
<<<test_output>>>
abs01       1  TPASS  :  Test passed
abs01       2  TPASS  :  Test passed
abs01       3  TPASS  :  Test passed
<<<execution_status>>>
initiation_status="ok"
duration=0 termination_type=exited termination_id=0 corefile=no
cutime=0 cstime=0
<<<test_end>>>
<<<test_start>>>
tag=abs01_loop_4under_fault_kernel stime=1250492531
cmdline=" abs01 "
contacts=""
analysis=exit
<<<test_output>>>
abs01       1  TPASS  :  Test passed
abs01       2  TPASS  :  Test passed
abs01       3  TPASS  :  Test passed
<<<execution_status>>>
initiation_status="ok"
duration=0 termination_type=exited termination_id=0 corefile=no
cutime=0 cstime=0
<<<test_end>>>
<<<test_start>>>
tag=abs01_loop_5under_fault_kernel stime=1250492531
cmdline=" abs01 "
contacts=""
analysis=exit
<<<test_output>>>
abs01       1  TPASS  :  Test passed
abs01       2  TPASS  :  Test passed
abs01       3  TPASS  :  Test passed
<<<execution_status>>>
initiation_status="ok"
duration=0 termination_type=exited termination_id=0 corefile=no
cutime=0 cstime=1
<<<test_end>>>
<<<test_start>>>
tag=abs01_loop_6under_fault_kernel stime=1250492531
cmdline=" abs01 ; $LTPROOT/tools/restore_kernel_faults_default.sh; RC=$?; while [ $RC -ne 0 ]; do $LTPROOT/tools/restore_kernel_faults_default.sh; RC=$?; done"
contacts=""
analysis=exit
<<<test_output>>>
abs01       1  TPASS  :  Test passed
abs01       2  TPASS  :  Test passed
abs01       3  TPASS  :  Test passed
<<<execution_status>>>
initiation_status="ok"
duration=0 termination_type=exited termination_id=0 corefile=no
cutime=0 cstime=1
<<<test_end>>>
<<<test_start>>>
tag=atof01 stime=1250492531
cmdline="atof01"
contacts=""
analysis=exit
<<<test_output>>>
atof01      1  TPASS  :  Test passed
atof01      2  TPASS  :  Test passed
atof01      3  TPASS  :  Test passed
atof01      4  TPASS  :  Test passed
<<<execution_status>>>
initiation_status="ok"
duration=0 termination_type=exited termination_id=0 corefile=no
cutime=0 cstime=0
<<<test_end>>>
<<<test_start>>>
tag=atof01_loop_1under_fault_kernel stime=1250492531
cmdline="$LTPROOT/tools/insert_kernel_faults.sh 15; atof01"
contacts=""
analysis=exit
<<<test_output>>>
atof01      1  TPASS  :  Test passed
atof01      2  TPASS  :  Test passed
atof01      3  TPASS  :  Test passed
atof01      4  TPASS  :  Test passed
<<<execution_status>>>
initiation_status="ok"
duration=0 termination_type=exited termination_id=0 corefile=no
cutime=1 cstime=1
<<<test_end>>>
<<<test_start>>>
tag=atof01_loop_2under_fault_kernel stime=1250492531
cmdline=" atof01 "
contacts=""
analysis=exit
<<<test_output>>>
atof01      1  TPASS  :  Test passed
atof01      2  TPASS  :  Test passed
atof01      3  TPASS  :  Test passed
atof01      4  TPASS  :  Test passed
<<<execution_status>>>
initiation_status="ok"
duration=0 termination_type=exited termination_id=0 corefile=no
cutime=0 cstime=0
<<<test_end>>>
<<<test_start>>>
tag=atof01_loop_3under_fault_kernel stime=1250492531
cmdline=" atof01 "
contacts=""
analysis=exit
<<<test_output>>>
atof01      1  TPASS  :  Test passed
atof01      2  TPASS  :  Test passed
atof01      3  TPASS  :  Test passed
atof01      4  TPASS  :  Test passed
<<<execution_status>>>
initiation_status="ok"
duration=0 termination_type=exited termination_id=0 corefile=no
cutime=1 cstime=0
<<<test_end>>>
<<<test_start>>>
tag=atof01_loop_4under_fault_kernel stime=1250492531
cmdline=" atof01 "
contacts=""
analysis=exit
<<<test_output>>>
atof01      1  TPASS  :  Test passed
atof01      2  TPASS  :  Test passed
atof01      3  TPASS  :  Test passed
atof01      4  TPASS  :  Test passed
<<<execution_status>>>
initiation_status="ok"
duration=0 termination_type=exited termination_id=0 corefile=no
cutime=0 cstime=0
<<<test_end>>>
<<<test_start>>>
tag=atof01_loop_5under_fault_kernel stime=1250492531
cmdline=" atof01 "
contacts=""
analysis=exit
<<<test_output>>>
atof01      1  TPASS  :  Test passed
atof01      2  TPASS  :  Test passed
atof01      3  TPASS  :  Test passed
atof01      4  TPASS  :  Test passed
<<<execution_status>>>
initiation_status="ok"
duration=0 termination_type=exited termination_id=0 corefile=no
cutime=1 cstime=1
<<<test_end>>>
<<<test_start>>>
tag=atof01_loop_6under_fault_kernel stime=1250492531
cmdline=" atof01 ; $LTPROOT/tools/restore_kernel_faults_default.sh; RC=$?; while [ $RC -ne 0 ]; do $LTPROOT/tools/restore_kernel_faults_default.sh; RC=$?; done"
contacts=""
analysis=exit
<<<test_output>>>
atof01      1  TPASS  :  Test passed
atof01      2  TPASS  :  Test passed
atof01      3  TPASS  :  Test passed
atof01      4  TPASS  :  Test passed
<<<execution_status>>>
initiation_status="ok"
duration=0 termination_type=exited termination_id=0 corefile=no
cutime=0 cstime=1
<<<test_end>>>
<<<test_start>>>
tag=fptest01 stime=1250492531
cmdline="fptest01"
contacts=""
analysis=exit
<<<test_output>>>
fptest01    1  TPASS  :  PASS
<<<execution_status>>>
initiation_status="ok"
duration=0 termination_type=exited termination_id=0 corefile=no
cutime=1 cstime=0
<<<test_end>>>
<<<test_start>>>
tag=fptest01_loop_1under_fault_kernel stime=1250492531
cmdline="$LTPROOT/tools/insert_kernel_faults.sh 15; fptest01"
contacts=""
analysis=exit
<<<test_output>>>
fptest01    1  TPASS  :  PASS
<<<execution_status>>>
initiation_status="ok"
duration=0 termination_type=exited termination_id=0 corefile=no
cutime=1 cstime=1
<<<test_end>>>
<<<test_start>>>
tag=fptest01_loop_2under_fault_kernel stime=1250492531
cmdline=" fptest01 "
contacts=""
analysis=exit
<<<test_output>>>
fptest01    1  TPASS  :  PASS
<<<execution_status>>>
initiation_status="ok"
duration=0 termination_type=exited termination_id=0 corefile=no
cutime=0 cstime=0
<<<test_end>>>
<<<test_start>>>
tag=fptest01_loop_3under_fault_kernel stime=1250492531
cmdline=" fptest01 "
contacts=""
analysis=exit
<<<test_output>>>
fptest01    1  TPASS  :  PASS
<<<execution_status>>>
initiation_status="ok"
duration=0 termination_type=exited termination_id=0 corefile=no
cutime=1 cstime=0
<<<test_end>>>
<<<test_start>>>
tag=fptest01_loop_4under_fault_kernel stime=1250492531
cmdline=" fptest01 "
contacts=""
analysis=exit
<<<test_output>>>
fptest01    1  TPASS  :  PASS
<<<execution_status>>>
initiation_status="ok"
duration=0 termination_type=exited termination_id=0 corefile=no
cutime=0 cstime=0
<<<test_end>>>
<<<test_start>>>
tag=fptest01_loop_5under_fault_kernel stime=1250492531
cmdline=" fptest01 "
contacts=""
analysis=exit
<<<test_output>>>
fptest01    1  TPASS  :  PASS
<<<execution_status>>>
initiation_status="ok"
duration=0 termination_type=exited termination_id=0 corefile=no
cutime=1 cstime=1
<<<test_end>>>
<<<test_start>>>
tag=fptest01_loop_6under_fault_kernel stime=1250492531
cmdline=" fptest01 ; $LTPROOT/tools/restore_kernel_faults_default.sh; RC=$?; while [ $RC -ne 0 ]; do $LTPROOT/tools/restore_kernel_faults_default.sh; RC=$?; done"
contacts=""
analysis=exit
<<<test_output>>>
fptest01    1  TPASS  :  PASS
<<<execution_status>>>
initiation_status="ok"
duration=0 termination_type=exited termination_id=0 corefile=no
cutime=1 cstime=1
<<<test_end>>>
<<<test_start>>>
tag=fptest02 stime=1250492531
cmdline="fptest02"
contacts=""
analysis=exit
<<<test_output>>>
fptest02    1  TPASS  :  PASS
<<<execution_status>>>
initiation_status="ok"
duration=0 termination_type=exited termination_id=0 corefile=no
cutime=0 cstime=0
<<<test_end>>>
<<<test_start>>>
tag=fptest02_loop_1under_fault_kernel stime=1250492531
cmdline="$LTPROOT/tools/insert_kernel_faults.sh 15; fptest02"
contacts=""
analysis=exit
<<<test_output>>>
fptest02    1  TPASS  :  PASS
<<<execution_status>>>
initiation_status="ok"
duration=0 termination_type=exited termination_id=0 corefile=no
cutime=1 cstime=1
<<<test_end>>>
<<<test_start>>>
tag=fptest02_loop_2under_fault_kernel stime=1250492531
cmdline=" fptest02 "
contacts=""
analysis=exit
<<<test_output>>>
fptest02    1  TPASS  :  PASS
<<<execution_status>>>
initiation_status="ok"
duration=0 termination_type=exited termination_id=0 corefile=no
cutime=0 cstime=1
<<<test_end>>>
<<<test_start>>>
tag=fptest02_loop_3under_fault_kernel stime=1250492531
cmdline=" fptest02 "
contacts=""
analysis=exit
<<<test_output>>>
fptest02    1  TPASS  :  PASS
<<<execution_status>>>
initiation_status="ok"
duration=0 termination_type=exited termination_id=0 corefile=no
cutime=1 cstime=0
<<<test_end>>>
<<<test_start>>>
tag=fptest02_loop_4under_fault_kernel stime=1250492531
cmdline=" fptest02 "
contacts=""
analysis=exit
<<<test_output>>>
fptest02    1  TPASS  :  PASS
<<<execution_status>>>
initiation_status="ok"
duration=0 termination_type=exited termination_id=0 corefile=no
cutime=0 cstime=0
<<<test_end>>>
<<<test_start>>>
tag=fptest02_loop_5under_fault_kernel stime=1250492531
cmdline=" fptest02 "
contacts=""
analysis=exit
<<<test_output>>>
fptest02    1  TPASS  :  PASS
<<<execution_status>>>
initiation_status="ok"
duration=0 termination_type=exited termination_id=0 corefile=no
cutime=1 cstime=0
<<<test_end>>>
<<<test_start>>>
tag=fptest02_loop_6under_fault_kernel stime=1250492531
cmdline=" fptest02 ; $LTPROOT/tools/restore_kernel_faults_default.sh; RC=$?; while [ $RC -ne 0 ]; do $LTPROOT/tools/restore_kernel_faults_default.sh; RC=$?; done"
contacts=""
analysis=exit
<<<test_output>>>
fptest02    1  TPASS  :  PASS
<<<execution_status>>>
initiation_status="ok"
duration=0 termination_type=exited termination_id=0 corefile=no
cutime=1 cstime=1
<<<test_end>>>
<<<test_start>>>
tag=nextafter01 stime=1250492531
cmdline="nextafter01"
contacts=""
analysis=exit
<<<test_output>>>
nextafter01    1  TPASS  :  Test passed
nextafter01    2  TPASS  :  Test passed
nextafter01    3  TPASS  :  Test passed
<<<execution_status>>>
initiation_status="ok"
duration=0 termination_type=exited termination_id=0 corefile=no
cutime=1 cstime=0
<<<test_end>>>
<<<test_start>>>
tag=nextafter01_loop_1under_fault_kernel stime=1250492531
cmdline="$LTPROOT/tools/insert_kernel_faults.sh 15; nextafter01"
contacts=""
analysis=exit
<<<test_output>>>
nextafter01    1  TPASS  :  Test passed
nextafter01    2  TPASS  :  Test passed
nextafter01    3  TPASS  :  Test passed
<<<execution_status>>>
initiation_status="ok"
duration=0 termination_type=exited termination_id=0 corefile=no
cutime=0 cstime=1
<<<test_end>>>
<<<test_start>>>
tag=nextafter01_loop_2under_fault_kernel stime=1250492531
cmdline=" nextafter01 "
contacts=""
analysis=exit
<<<test_output>>>
nextafter01    1  TPASS  :  Test passed
nextafter01    2  TPASS  :  Test passed
nextafter01    3  TPASS  :  Test passed
<<<execution_status>>>
initiation_status="ok"
duration=0 termination_type=exited termination_id=0 corefile=no
cutime=0 cstime=0
<<<test_end>>>
<<<test_start>>>
tag=nextafter01_loop_3under_fault_kernel stime=1250492531
cmdline=" nextafter01 "
contacts=""
analysis=exit
<<<test_output>>>
nextafter01    1  TPASS  :  Test passed
nextafter01    2  TPASS  :  Test passed
nextafter01    3  TPASS  :  Test passed
<<<execution_status>>>
initiation_status="ok"
duration=0 termination_type=exited termination_id=0 corefile=no
cutime=0 cstime=0
<<<test_end>>>
<<<test_start>>>
tag=nextafter01_loop_4under_fault_kernel stime=1250492531
cmdline=" nextafter01 "
contacts=""
analysis=exit
<<<test_output>>>
nextafter01    1  TPASS  :  Test passed
nextafter01    2  TPASS  :  Test passed
nextafter01    3  TPASS  :  Test passed
<<<execution_status>>>
initiation_status="ok"
duration=0 termination_type=exited termination_id=0 corefile=no
cutime=0 cstime=0
<<<test_end>>>
<<<test_start>>>
tag=nextafter01_loop_5under_fault_kernel stime=1250492531
cmdline=" nextafter01 "
contacts=""
analysis=exit
<<<test_output>>>
nextafter01    1  TPASS  :  Test passed
nextafter01    2  TPASS  :  Test passed
nextafter01    3  TPASS  :  Test passed
<<<execution_status>>>
initiation_status="ok"
duration=0 termination_type=exited termination_id=0 corefile=no
cutime=0 cstime=0
<<<test_end>>>
<<<test_start>>>
tag=nextafter01_loop_6under_fault_kernel stime=1250492531
cmdline=" nextafter01 ; $LTPROOT/tools/restore_kernel_faults_default.sh; RC=$?; while [ $RC -ne 0 ]; do $LTPROOT/tools/restore_kernel_faults_default.sh; RC=$?; done"
contacts=""
analysis=exit
<<<test_output>>>
nextafter01    1  TPASS  :  Test passed
nextafter01    2  TPASS  :  Test passed
nextafter01    3  TPASS  :  Test passed
incrementing stop
<<<execution_status>>>
initiation_status="ok"
duration=0 termination_type=exited termination_id=0 corefile=no
cutime=1 cstime=2
<<<test_end>>>
================================================================================

Regards--
Subrata

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list

#9 lore raw
In reply to Subrata Modak
Quoted 3 lines · Subrata Modak

Hi,

Introducing the "Kernel Fault Injection Framework" generation and testing

This has now become part of LTP. Hope all of you will start using it and help improve it further.

Regards--
Subrata

Quoted 11 lines

capability in LTP. This is in line with the recent proposal made through LTP Paper at OLS 2009:
"Putting LTP to test - Validating both the Linux kernel and Test-cases"
(http://ltp.sourceforge.net/documentation/technical_papers/Putting_LTP_to_Test.pdf)

This infrasturcture will help LTP directly in the following ways:
1) Allow test developers to test their new test cases against a faulted kernel, and then compare it on stable kernel run, impacting in better test development,
2) Allow test engineers to be able to generate more code coverage by traversing the rarely touched parts of the kernel code,

Quoted 7 lines

As we move forward in using this, we would definitely find some other advantages of this framework. This is V2 of patchset after incorporating comments from Mike, Paul & Garret.

Regards--
Subrata

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list