open posix testsuite and names of testcases
Hi,
this testsuite has many testcases, where binary name is in format: [number]-[number].run-test This is not very helpful when trying to find out, which one actually
crashed or which one is responsible for kernel oops/hung tasks.
For example if I get in dmesg:
[ 702.018706] 1-5.run-test[10140]: segfault at 7f84e724bed8 ip 0000000000402208 sp 00007f84e724bed0 error 6 in 1-5.run-test[400000+4000]
it's not clear, which testcase is responsible.
Does anyone have some ideas?
I had these two:
1. rename all testcases - lot of work, they are also referenced in some .sh scripts
2. modify run-tests.sh to copy it under more verbose name and run that one
testcases/open_posix_testsuite/bin/run-tests.sh +4 -1
@@ -46,8 +46,11 @@ run_test() { complog=$(basename $testname).log.$$ - sh -c "$SCRIPT_DIR/t0 $TIMEOUT_VAL ./$1 $(cat ./$(echo "$1" | sed 's,\.[^\.]*,,').args 2>/dev/null)" > $complog 2>&1 + # create a copy with more verbose name + new_testname=$(basename $TEST_PATH)-$1 + cp -f ./$1 $new_testname + sh -c "$SCRIPT_DIR/t0 $TIMEOUT_VAL ./$new_testname $(cat ./$(echo "$1" | sed 's,\.[^\.]*,,').args 2>/dev/null)" > $complog 2>&1 ret_code=$? if [ "$ret_code" = "0" ]; then
Which makes logs a bit more helpful:
[179468.329814] User process fault: interruption code 0x60011 in pthread_create-1-5.run-test[80000000+5000]
Regards,
Jan
------------------------------------------------------------------------------
LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers
http://p.sf.net/sfu/logmein_12329d2d
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list
Hi!
Quoted 6 lines
this testsuite has many testcases, where binary name is in format: [number]-[number].run-test This is not very helpful when trying to find out, which one actually
crashed or which one is responsible for kernel oops/hung tasks.
For example if I get in dmesg:[ 702.018706] 1-5.run-test[10140]: segfault at 7f84e724bed8 ip 0000000000402208 sp 00007f84e724bed0 error 6 in 1-5.run-test[400000+4000]it's not clear, which testcase is responsible.
Here in suse we use CTCS2 to run the testcases and we have a perl script that generates the the test names accordingly to the directory structure which doesn't help when looking into the kernel log but the segfault is logged into the test results anyway.
And mostly because of that the current test execution framework both for LTP and Open Posix Testsuite is a little out of my radar, although I would like to have them fixed too.
Quoted 5 lines
Does anyone have some ideas?
I had these two:
1. rename all testcases - lot of work, they are also referenced in some .sh scripts
2. modifyrun-tests.shto copy it under more verbose name and run that one
Well I don't think that the option one is that hard, at least I'm not aware of much cases of hardcoded names in the Open Posix Testsuite. On the other hand having better test driver that executes the tests and presents the result in more convinient way would be more appealing. Or we can do both of them.
I was thinking of creating a relatively simply test driver that runs the tests, takes care of timeouts (and anything else) and logs information in some reasonable format (I used JSON in similar case). And then about some tools that could compare the results, print list of failed testcases and so on. Does that sound reasonable?
And on more general note we surely need to decide how to do things there. There are some howtos in the Documentation directory none of them are up to date.
Quoted 18 lines
testcases/open_posix_testsuite/bin/run-tests.sh +4
-1@@ -46,8 +46,11 @@ run_test() { complog=$(basename $testname).log.$$ - sh -c "$SCRIPT_DIR/t0 $TIMEOUT_VAL ./$1 $(cat ./$(echo "$1" | sed 's,\.[^\.]*,,').args 2>/dev/null)" > $complog 2>&1 + # create a copy with more verbose name + new_testname=$(basename $TEST_PATH)-$1 + cp -f ./$1 $new_testname + sh -c "$SCRIPT_DIR/t0 $TIMEOUT_VAL ./$new_testname $(cat ./$(echo "$1" | sed 's,\.[^\.]*,,').args 2>/dev/null)" > $complog 2>&1 ret_code=$? if [ "$ret_code" = "0" ]; then
Quoted 2 lines
Which makes logs a bit more helpful:
[179468.329814] User process fault: interruption code 0x60011 in pthread_create-1-5.run-test[80000000+5000]
It's rather crude hack, but I guess that it's better than nothing. And you should probably remove the file after the test has finished.
Signature · 13 lines
Cyril Hrubis [email protected] ------------------------------------------------------------------------------ LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers http://p.sf.net/sfu/logmein_12329d2d _______________________________________________ Ltp-list mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/ltp-list
Quoted 8 lines · Jan Stancek
Hi,
this testsuite has many testcases, where binary name is in format: [number]-[number].run-test This is not very helpful when trying to find out, which one actually
crashed or which one is responsible for kernel oops/hung tasks.
For example if I get in dmesg:[ 702.018706] 1-5.run-test[10140]: segfault at 7f84e724bed8 ip 0000000000402208 sp 00007f84e724bed0 error 6 in 1-5.run-test[400000+4000]it's not clear, which testcase is responsible.
I met the same problem before because running the open_posix on recent RHEL6.4alpha and beta will cause kernel panic...
Thanks,
Wanlong Gao
Quoted 31 lines
Does anyone have some ideas?
I had these two:
1. rename all testcases - lot of work, they are also referenced in some .sh scripts
2. modifyrun-tests.shto copy it under more verbose name and run that onetestcases/open_posix_testsuite/bin/run-tests.sh +4
-1@@ -46,8 +46,11 @@ run_test() { complog=$(basename $testname).log.$$ - sh -c "$SCRIPT_DIR/t0 $TIMEOUT_VAL ./$1 $(cat ./$(echo "$1" | sed 's,\.[^\.]*,,').args 2>/dev/null)" > $complog 2>&1 + # create a copy with more verbose name + new_testname=$(basename $TEST_PATH)-$1 + cp -f ./$1 $new_testname + sh -c "$SCRIPT_DIR/t0 $TIMEOUT_VAL ./$new_testname $(cat ./$(echo "$1" | sed 's,\.[^\.]*,,').args 2>/dev/null)" > $complog 2>&1 ret_code=$? if [ "$ret_code" = "0" ]; thenWhich makes logs a bit more helpful:
[179468.329814] User process fault: interruption code 0x60011 in pthread_create-1-5.run-test[80000000+5000]Regards,
Jan
Quoted 11 lines
------------------------------------------------------------------------------
LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers
http://p.sf.net/sfu/logmein_12329d2d
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list
------------------------------------------------------------------------------
LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers
http://p.sf.net/sfu/logmein_12329d2d
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list
Hi!
Quoted 9 lines
Quoted 6 lines
this testsuite has many testcases, where binary name is in format: [number]-[number].run-test This is not very helpful when trying to find out, which one actually
crashed or which one is responsible for kernel oops/hung tasks.
For example if I get in dmesg:[ 702.018706] 1-5.run-test[10140]: segfault at 7f84e724bed8 ip 0000000000402208 sp 00007f84e724bed0 error 6 in 1-5.run-test[400000+4000]it's not clear, which testcase is responsible.
I met the same problem before because running the
open_posixon recent RHEL6.4alpha and beta will cause kernel panic...
Which could still be fixed by a testdriver that ensures that the logfile was written to the disk before each test is executed. But the renaming is still a good idea because there may be a rare case where the filesystem will be trashed.
Signature · 13 lines
Cyril Hrubis [email protected] ------------------------------------------------------------------------------ LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers http://p.sf.net/sfu/logmein_12329d2d _______________________________________________ Ltp-list mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/ltp-list
Hi!
Quoted 8 lines
Quoted 7 lines
Well I don't think that the option one is that hard, at least I'm not aware of much cases of hardcoded names in the Open Posix Testsuite. On
the other hand having better test driver that executes the tests and presents the result in more convinient way would be more appealing. Or
we can do both of them.
Quoted 4 lines
I think we should do the renaming anyway.
Would the following format work for everyone? "testdirname-[number]-[number].c" For example:pthread_create-1-5.c
It's unique, and it's easy to tell what old name was.
I'm ok with that.
Quoted 11 lines
Quoted 8 lines
I was thinking of creating a relatively simply test driver that runs the
tests, takes care of timeouts (and anything else) and logs information
in some reasonable format (I used JSON in similar case). And then about
some tools that could compare the results, print list of failed testcases and so on. Does that sound reasonable?How would this be related to ltp-pan? Would this be only for open posix testsuite or also for LTP?
Well, that is a good question. The differencies between Open Posix Testsuite and the rest of the LTP aren't too big so it may be possible to create a test driver for both and share most of the code. On the other hand. To just run the tests you just need to export right VARIABLES and modify PATH accordingly. To collect the result you need to translate the test return values.
On the other hand I'm not even sure about all the features that ltp-pan has and what subset of them is used by different people. But I will surely like to have more up to date test driver that produces more usefull data.
There are couple possibilities what we can do.
We may start creating something that could run the Open Posix Testsuite and then gradually extend it to work with the rest of the LTP adding all the features we would need while deprecating the ltp-pan and removing it once we have full replacement.
Or we may look if there is something to fit our meets out there allready and modify it to meet our needs.
Or anything that any of you may come up with.
Signature · 13 lines
Cyril Hrubis [email protected] ------------------------------------------------------------------------------ LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers http://p.sf.net/sfu/logmein_12329d2d _______________________________________________ Ltp-list mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/ltp-list
Hi!
Just recently I've found that one of the posix tests stays running after the testsuite has exited (is reparented to init and runs happily ever after). Which is another usecase for this rename.
But the original patch was rather hack (and it doesn't work for me anyway).
So what about renaming the sources itself? Do we agree on this?
Signature · 12 lines
Cyril Hrubis [email protected] ------------------------------------------------------------------------------ Free Next-Gen Firewall Hardware Offer Buy your Sophos next-gen firewall before the end March 2013 and get the hardware for free! Learn more. http://p.sf.net/sfu/sophos-d2d-feb _______________________________________________ Ltp-list mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/ltp-list
Hi!
Quoted 8 lines
Quoted 8 lines
Just recently I've found that one of the posix tests stays running after
the testsuite has exited (is reparented to init and runs happily ever after). Which is another usecase for this rename.But the original patch was rather hack (and it doesn't work for me anyway).
Quoted 6 lines
Quoted 1 line
So what about renaming the sources itself? Do we agree on this?
I'm not against renaming sources, just thinking if there is less intrusive way to achieve same result. I believe all Makefiles are generated, so other option that comes to mind would be to modify
scripts/generate-makefiles.sh. Any thoughts about this approach?
We can hack the script to generate Makefiles that include the dir name into the binary name which would be less intrusive but on the other hand that would break the rule that foo.c generates foo.run_test...
Signature · 12 lines
Cyril Hrubis [email protected] ------------------------------------------------------------------------------ Free Next-Gen Firewall Hardware Offer Buy your Sophos next-gen firewall before the end March 2013 and get the hardware for free! Learn more. http://p.sf.net/sfu/sophos-d2d-feb _______________________________________________ Ltp-list mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/ltp-list