cgroup_regression_test.sh fix
cover cgroup_regression_test.sh fix
Hi
while working on this LTP tip (LTP Jan19 REL + fixes):
81e28eab1 memcg_stress_test.sh: fix memory usageb25c719de memcg_stress_test.sh: Further cleanup + print info a6d7d16f4 memcg_stress_test.sh: ported to newlib6a651fe16 controllers/cpuset: cpuset.cpus file doesn't exist which triggers TFAIL fa049ec9f controllers/cgroup_regression_test.sh: Fix two issues a6a5caef1 (tag: 20190115, LTP_20190115) LTP 20190115
I realized 'cgroup' testcase is now wrongly reported FAIL even if NO failures happens in the sub testcases:
Running tests.......
<<<test_start>>>
tag=cgroup stim[ 125.419776] LTP: starting cgroup ( cgroup_regression_test.sh)
e=1550611029
cmdline=" cgroup_regression_test.sh"
contacts=""
analysis=exit
<<<test_output>>>
incrementing stopcgroup_regression_test 1 TINFO: timeout per run is 0h 5m 0s cgroup_regression_test 1 TPASS: no kernel bug was found cgroup_regression_test 2 TPASS: notify_on_release is inherited cgroup_regression_test 3 TCONF: CONFIG_SCHED_DEBUG is not enabled cgroup_regression_test 4 TCONF: CONFIG_LOCKDEP is not enabled cgroup_regression_test 5 TPASS: no kernel bug was found cgroup_regression_test 6 TCONF: CONFIG_CGROUP_NS is NOT supported in Kernels >= 3.0 cgroup_regression_test 7 TPASS: no kernel bug was found for test 1
[ 126.879398] option changes via remount are deprecated (pid=8105 comm=mount)
cgroup_regression_test 7 TWARN: skip rest of testing due possible oops triggered by reading /proc/sched_debug cgroup_regression_test 7 TPASS: no kernel bug was found for test 2cgroup_regression_test 8 TPASS: no kernel bug was foundcgroup_regression_test 9 TPASS: no kernel bug was found
mkdir: cannot create directory 'cgroup/0': File existscgroup_regression_test 10 TPASS: no kernel bug was found
Summary:
passed 8 failed 0 skipped 3
warnings 1
<<<execution_status>>>
initiation_status="ok"
duration=61 termination_type=exited termination_id=36 corefile=no cutime=4314 cstime=5792
<<<test_end>>>
INFO: ltp-pan reported some tests FAIL
LTP Version: 20190115
###############################################################
Done executing testcases.
LTP Version: 20190115
###############################################################This seems to be due to the TWARN added in 78bfa7e785e87d5e7e4e6ebd518ca9a67c78a1f3 which leads to an exit code interpreted by ltp-pan as failure.
Is this expected ? An ltp-pan issue ? Is there something wrong in my setup ?
A possibly trivial patch follows, BUT anyway there still an unresolved problem: in the above test run even using TINFO instead of TWARN, I'll have the whole cgroup_regression_test.sh test case reported as TCONF, since some of the contained sub test cases have been in fact skipped: is there any way to address this kind of situation in the LTP framework ? (beside using TINFO everywhere) I understand the scenario is tricky:Â a test case with many sub-testcases that could each PASS/FAIL/SKIP. Any thought/guideline on this ?
Thanks
Cristian
Cristian Marussi (1):
cgroup_regression_test.sh: Fix TWARN usage
testcases/kernel/controllers/cgroup/cgroup_regression_test.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Signature · 1 line
2.17.1
PATCH 1/1 cgroup_regression_test.sh: Fix TWARN usage
Using TWARN to report that a specific sub-testcase is skipped caused the whole cgroup_regression_test.sh to be reported as FAILING. Using TCONF instead.
--- testcases/kernel/controllers/cgroup/cgroup_regression_test.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
testcases/kernel/controllers/cgroup/cgroup_regression_test.sh +1 -1
@@ -408,7 +408,7 @@ test_7_2() grep -q -w "cpu" /proc/cgroups if [ $? -ne 0 -o ! -e /proc/sched_debug ]; then - tst_res TWARN "skip rest of testing due possible oops triggered by reading /proc/sched_debug" + tst_res TCONF "skip rest of testing due possible oops triggered by reading /proc/sched_debug" return fi
Signature · 1 line
2.17.1
PATCH 1/1 cgroup_regression_test.sh: Fix TWARN usage
Hi Cristian,
Quoted 3 lines
Using TWARN to report that a specific sub-testcase is skipped caused the whole
cgroup_regression_test.shto be reported as FAILING. Using TCONF instead.
My commit, sorry :(.
Quoted 10 lines
Signed-off-by Cristian Marussi <[email protected]>--- testcases/kernel/controllers/cgroup/cgroup_regression_test.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)testcases/kernel/controllers/cgroup/cgroup_regression_test.sh +0
-0
testcases/kernel/controllers/cgroup/cgroup_regression_test.sh test_7_2() · line 411
Quoted 5 lines
testcases/kernel/controllers/cgroup/cgroup_regression_test.sh +1
-1@@ -408,7 +408,7 @@ test_7_2() grep -q -w "cpu" /proc/cgroups if [ $? -ne 0 -o ! -e /proc/sched_debug ]; then - tst_res TWARN "skip rest of testing due possible oops triggered by reading /proc/sched_debug" + tst_res TCONF "skip rest of testing due possible oops triggered by reading /proc/sched_debug"
Right, TWARN is not good (IMHO TWARN is not popular in LTP testsuites due marking test failing), but if you don't mind I'd merge it with TBROK (more appropriate in this case).
Other option would be to use TINFO and don't skip test (we're trying to find kernel crashes, don't we), that's probably a bit crazy.
Kind regards,
Petr
PATCH 1/1 cgroup_regression_test.sh: Fix TWARN usage
Quoted 9 lines · Petr Vorel
Hi Cristian,
Quoted 3 lines
Using TWARN to report that a specific sub-testcase is skipped caused the whole
cgroup_regression_test.shto be reported as FAILING. Using TCONF instead.Fixes 78bfa7e78 ("cgroup: Simplify check_kernel_bug usage")My commit, sorry :(.
No worries...:>
Quoted 11 lines
Quoted 10 lines
Signed-off-by Cristian Marussi <[email protected]>--- testcases/kernel/controllers/cgroup/cgroup_regression_test.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)testcases/kernel/controllers/cgroup/cgroup_regression_test.sh +0
-0@@ -408,7 +408,7 @@ test_7_2()
Quoted 6 lines
testcases/kernel/controllers/cgroup/cgroup_regression_test.sh test_7_2() · line 411
Quoted 4 lines
testcases/kernel/controllers/cgroup/cgroup_regression_test.sh +1
-1grep -q -w "cpu" /proc/cgroups if [ $? -ne 0 -o ! -e /proc/sched_debug ]; then - tst_res TWARN "skip rest of testing due possible oops triggered by reading /proc/sched_debug" + tst_res TCONF "skip rest of testing due possible oops triggered by reading /proc/sched_debug"Right, TWARN is not good (IMHO TWARN is not popular in LTP testsuites due marking test failing), but if you don't mind I'd merge it with TBROK (more appropriate in this case).
mmm...I'm NOT so sure that a FAIL should be the outcome here ... we are skipping if cpu is not amongst cgroups or sched_debug is NOT available...BUT at this point we have run (or skipped) 7 tests so far and there are 3 more to go...so why can't we just TCONF ?
(beside the fact that as I said in the cover-letter one TCONF will lead to the whole 10-subtest croup suite to be reported as TCONF...)
Regards
Cristian
Quoted 7 lines
Other option would be to use TINFO and don't skip test (we're trying to find kernel crashes, don't we), that's probably a bit crazy.
Kind regards,
Petr
PATCH 1/1 cgroup_regression_test.sh: Fix TWARN usage
Hi Cristian,
...
Quoted 10 lines
Quoted 9 lines
testcases/kernel/controllers/cgroup/cgroup_regression_test.sh test_7_2() · line 411
Quoted 7 lines
+++
b/testcases/kernel/controllers/cgroup/cgroup_regression_test.shtestcases/kernel/controllers/cgroup/cgroup_regression_test.sh +1
-1@@ -408,7 +408,7 @@ test_7_2() grep -q -w "cpu" /proc/cgroups if [ $? -ne 0 -o ! -e /proc/sched_debug ]; then - tst_res TWARN "skip rest of testing due possible oops triggered by reading /proc/sched_debug" + tst_res TCONF "skip rest of testing due possible oops triggered by reading /proc/sched_debug"Right, TWARN is not good (IMHO TWARN is not popular in LTP testsuites due marking test failing), but if you don't mind I'd merge it with TBROK (more appropriate in this case).
Quoted 4 lines
mmm...I'm NOT so sure that a FAIL should be the outcome here ... we are skipping if cpu is not amongst cgroups or
sched_debugis NOT available...BUT at this point we have run (or skipped) 7 tests so far and there are 3 more to go...so why can't we just TCONF ?
OK I got that :) /proc/sched_debug is enabled by CONFIG_SCHED_DEBUG, os it's kind of configuration issue. Before I thought that failing "grep -q -w "cpu" /proc/cgroups" means failure/bug found by test, but it's not.
So I'm ok with this change.
Quoted 2 lines
(beside the fact that as I said in the cover-letter one TCONF will lead to the whole 10-subtest croup suite to be reported as TCONF...)
IMHO that's not true (see my reply to cover letter).
Kind regards,
Petr
PATCH 1/1 cgroup_regression_test.sh: Fix TWARN usage
In reply to Cristian MarussiHi Cristian,
thanks for your work, merged (with my comments in commit message).
Kind regards,
Petr
cover cgroup_regression_test.sh fix
In reply to Cristian MarussiHi Cristian,
I overlooked this cover letter (read the patch [1] via patchwork and didn't expect cover letter for single patch, I usually put info below additional ---).
Quoted 55 lines
Hi
while working on this LTP tip (LTP Jan19 REL + fixes):
81e28eab1memcg_stress_test.sh: fix memory usageb25c719dememcg_stress_test.sh: Further cleanup + print infoa6d7d16f4memcg_stress_test.sh: ported to newlib6a651fe16controllers/cpuset: cpuset.cpus file doesn't exist which triggers TFAILfa049ec9fcontrollers/cgroup_regression_test.sh: Fix two issuesa6a5caef1(tag: 20190115,LTP_20190115) LTP 20190115I realized 'cgroup' testcase is now wrongly reported FAIL even if NO failures happens in the sub testcases:
Running tests.......
<<<test_start>>>
tag=cgroup stim[ 125.419776] LTP: starting cgroup (cgroup_regression_test.sh)
e=1550611029
cmdline="cgroup_regression_test.sh"
contacts=""
analysis=exit
<<<test_output>>>
incrementing stopcgroup_regression_test1 TINFO: timeout per run is 0h 5m 0scgroup_regression_test1 TPASS: no kernel bug was foundcgroup_regression_test2 TPASS:notify_on_releaseis inheritedcgroup_regression_test3 TCONF:CONFIG_SCHED_DEBUGis not enabledcgroup_regression_test4 TCONF:CONFIG_LOCKDEPis not enabledcgroup_regression_test5 TPASS: no kernel bug was foundcgroup_regression_test6 TCONF:CONFIG_CGROUP_NSis NOT supported in Kernels >= 3.0cgroup_regression_test7 TPASS: no kernel bug was found for test 1[ 126.879398] option changes via remount are deprecated (pid=8105 comm=mount)
cgroup_regression_test7 TWARN: skip rest of testing due possible oops triggered by reading/proc/sched_debugcgroup_regression_test7 TPASS: no kernel bug was found for test 2cgroup_regression_test8 TPASS: no kernel bug was foundcgroup_regression_test9 TPASS: no kernel bug was found
mkdir: cannot create directory 'cgroup/0': File existscgroup_regression_test10 TPASS: no kernel bug was foundSummary:
passed 8 failed 0 skipped 3warnings 1
<<<execution_status>>>
initiation_status="ok"
duration=61 termination_type=exited termination_id=36 corefile=no cutime=4314 cstime=5792
<<<test_end>>>
INFO: ltp-pan reported some tests FAIL
LTP Version: 20190115
Quoted 8 lines
Done executing testcases.
LTP Version: 20190115This seems to be due to the TWARN added in
78bfa7e785e87d5e7e4e6ebd518ca9a67c78a1f3which leads to an exit code interpreted by ltp-pan as failure.
Is this expected ? An ltp-pan issue ? Is there something wrong in my setup ?
This is maybe surprising, but deliberate behavior.
IMHO this leads to to the fact that TWARN is fewer used than the others. @metan: Maybe it'd make sense to change TWARN not producing FAIL.
Quoted 7 lines
A possibly trivial patch follows, BUT anyway there still an unresolved problem: in the above test run even using TINFO instead of TWARN, I'll have the whole
cgroup_regression_test.shtest case reported as TCONF, since some of the contained sub test cases have been in fact skipped: is there any way to address this kind of situation in the LTP framework ? (beside using TINFO everywhere) I understand the scenario is tricky:Â a test case with many sub-testcases that could each PASS/FAIL/SKIP. Any thought/guideline on this ?
FAIL is IMHO in TWARN, TFAIL and TBROK.
TCONF is taken as PASS, see:
cgroup_regression_test 1 TINFO: timeout per run is 0h 5m 0s cgroup_regression_test 1 TPASS: no kernel bug was found cgroup_regression_test 2 TPASS: notify_on_release is inherited cgroup_regression_test 3 TPASS: no kernel bug was found cgroup_regression_test 4 TCONF: CONFIG_LOCKDEP is not enabled cgroup_regression_test 5 TPASS: no kernel bug was found cgroup_regression_test 6 TCONF: CONFIG_CGROUP_NS is NOT supported in Kernels >= 3.0 cgroup_regression_test 7 TPASS: no kernel bug was found for test 1 cgroup_regression_test 7 TPASS: no kernel bug was found for test 2 cgroup_regression_test 8 TPASS: no kernel bug was found cgroup_regression_test 9 TPASS: no kernel bug was found cgroup_regression_test 10 TPASS: no kernel bug was found
Summary:
passed 9 failed 0 skipped 2
warnings 0
incrementing stop
<<<execution_status>>>
initiation_status="ok"
duration=92 termination_type=exited termination_id=32 corefile=no cutime=12606 cstime=5151
<<<test_end>>>
INFO: ltp-pan reported all tests PASS
Kind regards,
Petr
[1] https://patchwork.ozlabs.org/patch/1042942/
Quoted 5 lines
Cristian Marussi (1):
cgroup_regression_test.sh: Fix TWARN usagetestcases/kernel/controllers/cgroup/cgroup_regression_test.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)