minimal ltp test set
A message that is not in the archive (sent off-list or not yet received): [email protected]
The number of testcases over the past year had increased faster than could be properly debugged by our skeleton team for the s390x platform. Many of the testcases seemed to be designed for features on later kernel versions with were not yet fully integrated by our supported distributions.
Since we mostly used LTP for a quick regression test, I was discussing with Subrata about creating a minimal test set that we expected would run properly on a majority of the distributions with kernel 2.6 and not just the more recent versions of the kernel.
This is my first pass at the test set:
(See attached file: s390x_ltpcore_targets)
It is basically a subset of the default SCENFILES from the ltp-full-20090731 concatenated into one file.
Basic rundown of the contents ...
* Included tests:
syscalls
fs
fsx
dio
mm
ipc
sched
math
nptl
pty
containersfs_bind
filecaps
fcntl-locktests
-----
* Excluded tests - new in 2009 (maybe late 2008) versions (appear to have build, run, or support issues depending on distribution release) iocap_bounds
connectorsadmin_tools
timerspower_management_tests
numa
hugetlb
commands
hyperthreading
-----
* Excluded from original set:
controllers
- notes: this feature was currently available only on SLES-11 and the build and execution of the testcase appeared to need more work for s390x
-----
Ideally I was hoping to come up with a set of tests that were so basic, they should be supported by practically every release and would be expected
to run correctly everywhere. They would only fail if major issues had
been introduced into the system build.
- Mark Ver
office: Building 710 / Room 2-RF-10
phone: (845) 435-7794 [tie 8 295-7794]
------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list
Quoted 52 lines · Mark Ver
The number of testcases over the past year had increased faster than could be properly debugged by our skeleton team for the s390x platform. Many of the testcases seemed to be designed for features on later kernel versions with were not yet fully integrated by our supported distributions.
Since we mostly used LTP for a quick regression test, I was discussing with Subrata about creating a minimal test set that we expected would run properly on a majority of the distributions with kernel 2.6 and not just the more recent versions of the kernel.
This is my first pass at the test set:
(See attached file:s390x_ltpcore_targets)It is basically a subset of the default SCENFILES from the ltp-full-20090731 concatenated into one file.
Basic rundown of the contents ...
* Included tests:
syscalls
fs
fsx
dio
mm
ipc
sched
math
nptl
pty
containersfs_bind
filecaps
fcntl-locktests
-----
* Excluded tests - new in 2009 (maybe late 2008) versions (appear to have build, run, or support issues depending on distribution release) iocap_bounds
connectorsadmin_tools
timerspower_management_tests
numa
hugetlb
commands
hyperthreading
-----
* Excluded from original set:
controllers
- notes: this feature was currently available only on SLES-11 and the build and execution of the testcase appeared to need more work for s390x
-----
Quoted 4 lines
Ideally I was hoping to come up with a set of tests that were so basic, they should be supported by practically every release and would be expected to run correctly everywhere. They would only fail if major issues had been introduced into the system build.
Mark,
An ideal solution to this problem is a two-tiered solution:
1. Feature based conditional compilation.
2. Tool based conditional compilation (some tests require certain
tools, such as perl, python, and various Unix utilities that aren't a part of the minimal set of tools, e.g. useradd, userdel, etc).
The tool based condition compilation is somewhat simple at a 10k
ft level (I have perl, python, ruby, etc -- add / remove these tests / features) -- the feature based conditional compilation issue is a bit more difficult to deal with as it requires a more in-depth understanding of what code exists in the sourcebase, and what functional tests map to what functional blocks under test. This is ideal, but not trivial to perform.
My first goal is to deal with any fallout from the make
infrastructure commit(s), dealing with the make 3.80 incompatibility and other unforeseen usecases like you have with building and executing out of the same source tree. Next, I want to clean up any and all C warnings so the tests are more robust (Mike, Cyril, and I have done a pretty good job but there's a ways to go...). Then I'll undertake completing the ballista cleanup, which will also involve adding the shims for perl and python across the entire sourcebase. We'll need to revisit the feature based enabling a little later.
How does that sound?
Thanks,
-Garrett
------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list
Hi Mark,
Quoted 6 lines · Garrett Cooper
Quoted 5 lines · Mark Ver
The number of testcases over the past year had increased faster than could be properly debugged by our skeleton team for the s390x platform. Many of the testcases seemed to be designed for features on later kernel versions with were not yet fully integrated by our supported distributions.
Quoted 7 lines
Quoted 7 lines
Since we mostly used LTP for a quick regression test, I was discussing with Subrata about creating a minimal test set that we expected would run properly on a majority of the distributions with kernel 2.6 and not just the more recent versions of the kernel.
This is my first pass at the test set:
(See attached file:s390x_ltpcore_targets)
We can definitely make this part of ltp/runtest for your convenience.
Garret,
Can you please check this in. I seemed to have not received Mark´s original mail in this ID.
Regards--
Subrata
Quoted 71 lines
Quoted 40 lines
It is basically a subset of the default SCENFILES from the ltp-full-20090731 concatenated into one file.
Basic rundown of the contents ...
* Included tests:
syscalls
fs
fsx
dio
mm
ipc
sched
math
nptl
pty
containersfs_bind
filecaps
fcntl-locktests
-----
* Excluded tests - new in 2009 (maybe late 2008) versions (appear to have build, run, or support issues depending on distribution release) iocap_bounds
connectorsadmin_tools
timerspower_management_tests
numa
hugetlb
commands
hyperthreading
-----
* Excluded from original set:
controllers
- notes: this feature was currently available only on SLES-11 and the build and execution of the testcase appeared to need more work for s390x
-----Quoted 4 lines
Ideally I was hoping to come up with a set of tests that were so basic, they should be supported by practically every release and would be expected to run correctly everywhere. They would only fail if major issues had been introduced into the system build.
Mark,
An ideal solution to this problem is a two-tiered solution:
1. Feature based conditional compilation.
2. Tool based conditional compilation (some tests require certain
tools, such as perl, python, and various Unix utilities that aren't a part of the minimal set of tools, e.g. useradd, userdel, etc).
The tool based condition compilation is somewhat simple at a 10k
ft level (I have perl, python, ruby, etc -- add / remove these tests / features) -- the feature based conditional compilation issue is a bit more difficult to deal with as it requires a more in-depth understanding of what code exists in the sourcebase, and what functional tests map to what functional blocks under test. This is ideal, but not trivial to perform.
My first goal is to deal with any fallout from the make
infrastructure commit(s), dealing with the make 3.80 incompatibility and other unforeseen usecases like you have with building and executing out of the same source tree. Next, I want to clean up any and all C warnings so the tests are more robust (Mike, Cyril, and I have done a pretty good job but there's a ways to go...). Then I'll undertake completing the ballista cleanup, which will also involve adding the shims for perl and python across the entire sourcebase. We'll need to revisit the feature based enabling a little later.
How does that sound?
Thanks,
-Garrett
Quoted 9 lines
------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list
------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list
Garrett Cooper <[email protected]> wrote on 11/19/2009 11:54:02 PM:
Quoted 17 lines
Mark,
An ideal solution to this problem is a two-tiered solution:
1. Feature based conditional compilation.
2. Tool based conditional compilation (some tests require certain
tools, such as perl, python, and various Unix utilities that aren't a part of the minimal set of tools, e.g. useradd, userdel, etc).
The tool based condition compilation is somewhat simple at a 10k
ft level (I have perl, python, ruby, etc -- add / remove these tests / features) -- the feature based conditional compilation issue is a bit more difficult to deal with as it requires a more in-depth understanding of what code exists in the sourcebase, and what functional tests map to what functional blocks under test. This is ideal, but not trivial to perform.
My first goal is to deal with any fallout from the make
infrastructure commit(s), dealing with the make 3.80 incompatibility and other unforeseen usecases like you have with building and executing out of the same source tree. Next, I want to clean up any
Quoted 8 lines
and all C warnings so the tests are more robust (Mike, Cyril, and I have done a pretty good job but there's a ways to go...). Then I'll undertake completing the ballista cleanup, which will also involve adding the shims for perl and python across the entire sourcebase. We'll need to revisit the feature based enabling a little later.
How does that sound?
Thanks,
-Garrett
Sounds good to me. Definitely getting the conditional compilation straightened out is priority one. Right now we end up having to remove testcases and modify makefiles just to get the test suite to build.
I guess I was thinking of the minimal test set for the cases when new testcases were misbehaving rather than when they couldn't run because they misjudged the environment. But I agree the 2 scenarios probably overlap 95% of the time.
- Mark
------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list