Skip to contentSkip to conversation

cgroup_regression_test.sh fix

RFC PATCH New Cristian Marussi 2 participants 7 messages Feb 15, 2019 – Feb 27, 2019

lore
Share & tools
lore
#0 lore raw

cover cgroup_regression_test.sh fix

Hi

while working on this LTP tip (LTP Jan19 REL + fixes):

81e28eab1 memcg_stress_test.sh: fix memory usage
b25c719de memcg_stress_test.sh: Further cleanup + print info a6d7d16f4 memcg_stress_test.sh: ported to newlib
6a651fe16 controllers/cpuset: cpuset.cpus file doesn't exist which triggers TFAIL fa049ec9f controllers/cgroup_regression_test.sh: Fix two issues a6a5caef1 (tag: 20190115, LTP_20190115) LTP 20190115

I realized 'cgroup' testcase is now wrongly reported FAIL even if NO failures happens in the sub testcases:

Running tests.......
<<<test_start>>>
tag=cgroup stim[ 125.419776] LTP: starting cgroup ( cgroup_regression_test.sh)
e=1550611029
cmdline=" cgroup_regression_test.sh"
contacts=""
analysis=exit
<<<test_output>>>
incrementing stop
cgroup_regression_test 1 TINFO: timeout per run is 0h 5m 0s cgroup_regression_test 1 TPASS: no kernel bug was found cgroup_regression_test 2 TPASS: notify_on_release is inherited cgroup_regression_test 3 TCONF: CONFIG_SCHED_DEBUG is not enabled cgroup_regression_test 4 TCONF: CONFIG_LOCKDEP is not enabled cgroup_regression_test 5 TPASS: no kernel bug was found cgroup_regression_test 6 TCONF: CONFIG_CGROUP_NS is NOT supported in Kernels >= 3.0 cgroup_regression_test 7 TPASS: no kernel bug was found for test 1

[  126.879398] option changes via remount are deprecated (pid=8105 comm=mount)

cgroup_regression_test 7 TWARN: skip rest of testing due possible oops triggered by reading /proc/sched_debug cgroup_regression_test 7 TPASS: no kernel bug was found for test 2
cgroup_regression_test 8 TPASS: no kernel bug was found
cgroup_regression_test 9 TPASS: no kernel bug was found
mkdir: cannot create directory 'cgroup/0': File exists
cgroup_regression_test 10 TPASS: no kernel bug was found

Summary:

passed   8
failed   0
skipped  3

warnings 1
<<<execution_status>>>
initiation_status="ok"
duration=61 termination_type=exited termination_id=36 corefile=no cutime=4314 cstime=5792
<<<test_end>>>
INFO: ltp-pan reported some tests FAIL
LTP Version: 20190115

       ###############################################################

            Done executing testcases.
            LTP Version:  20190115
       ###############################################################

This seems to be due to the TWARN added in 78bfa7e785e87d5e7e4e6ebd518ca9a67c78a1f3 which leads to an exit code interpreted by ltp-pan as failure.
Is this expected ? An ltp-pan issue ? Is there something wrong in my setup ?

A possibly trivial patch follows, BUT anyway there still an unresolved problem: in the above test run even using TINFO instead of TWARN, I'll have the whole cgroup_regression_test.sh test case reported as TCONF, since some of the contained sub test cases have been in fact skipped: is there any way to address this kind of situation in the LTP framework ? (beside using TINFO everywhere) I understand the scenario is tricky: a test case with many sub-testcases that could each PASS/FAIL/SKIP. Any thought/guideline on this ?

Thanks

Cristian

Cristian Marussi (1):
cgroup_regression_test.sh: Fix TWARN usage

 testcases/kernel/controllers/cgroup/cgroup_regression_test.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Signature · 1 line
2.17.1
#1 lore raw

PATCH 1/1 cgroup_regression_test.sh: Fix TWARN usage

Using TWARN to report that a specific sub-testcase is skipped caused the whole cgroup_regression_test.sh to be reported as FAILING. Using TCONF instead.

---
 testcases/kernel/controllers/cgroup/cgroup_regression_test.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
testcases/kernel/controllers/cgroup/cgroup_regression_test.sh +1 -1
index 686860923..e197f5d3f 100755
@@ -408,7 +408,7 @@ test_7_2()
 
 	grep -q -w "cpu" /proc/cgroups
 	if [ $? -ne 0 -o ! -e /proc/sched_debug ]; then
-		tst_res TWARN "skip rest of testing due possible oops triggered by reading /proc/sched_debug"
+		tst_res TCONF "skip rest of testing due possible oops triggered by reading /proc/sched_debug"
 		return
 	fi
 
Signature · 1 line
2.17.1
#2 lore raw

PATCH 1/1 cgroup_regression_test.sh: Fix TWARN usage

Hi Cristian,

Quoted 3 lines

Using TWARN to report that a specific sub-testcase is skipped caused the whole cgroup_regression_test.sh to be reported as FAILING. Using TCONF instead.

Fixes 78bfa7e78 ("cgroup: Simplify check_kernel_bug usage")

My commit, sorry :(.

Quoted 10 lines
---
 testcases/kernel/controllers/cgroup/cgroup_regression_test.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
testcases/kernel/controllers/cgroup/cgroup_regression_test.sh +0 -0
index 686860923..e197f5d3f 100755

testcases/kernel/controllers/cgroup/cgroup_regression_test.sh test_7_2() · line 411

Quoted 5 lines
testcases/kernel/controllers/cgroup/cgroup_regression_test.sh +1 -1
@@ -408,7 +408,7 @@ test_7_2()
 
 	grep -q -w "cpu" /proc/cgroups
 	if [ $? -ne 0 -o ! -e /proc/sched_debug ]; then
-		tst_res TWARN "skip rest of testing due possible oops triggered by reading /proc/sched_debug"
+		tst_res TCONF "skip rest of testing due possible oops triggered by reading /proc/sched_debug"

Right, TWARN is not good (IMHO TWARN is not popular in LTP testsuites due marking test failing), but if you don't mind I'd merge it with TBROK (more appropriate in this case).

Other option would be to use TINFO and don't skip test (we're trying to find kernel crashes, don't we), that's probably a bit crazy.

Kind regards,
Petr

#3 lore raw

PATCH 1/1 cgroup_regression_test.sh: Fix TWARN usage

Quoted 9 lines · Petr Vorel

Hi Cristian,

Quoted 3 lines

Using TWARN to report that a specific sub-testcase is skipped caused the whole cgroup_regression_test.sh to be reported as FAILING. Using TCONF instead.

Fixes 78bfa7e78 ("cgroup: Simplify check_kernel_bug usage")

My commit, sorry :(.

No worries...:>

Quoted 11 lines
Quoted 10 lines
---
 testcases/kernel/controllers/cgroup/cgroup_regression_test.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
testcases/kernel/controllers/cgroup/cgroup_regression_test.sh +0 -0
index 686860923..e197f5d3f 100755
@@ -408,7 +408,7 @@ test_7_2()
Quoted 6 lines

testcases/kernel/controllers/cgroup/cgroup_regression_test.sh test_7_2() · line 411

Quoted 4 lines
testcases/kernel/controllers/cgroup/cgroup_regression_test.sh +1 -1
 	grep -q -w "cpu" /proc/cgroups
 	if [ $? -ne 0 -o ! -e /proc/sched_debug ]; then
-		tst_res TWARN "skip rest of testing due possible oops triggered by reading /proc/sched_debug"
+		tst_res TCONF "skip rest of testing due possible oops triggered by reading /proc/sched_debug"

Right, TWARN is not good (IMHO TWARN is not popular in LTP testsuites due marking test failing), but if you don't mind I'd merge it with TBROK (more appropriate in this case).

mmm...I'm NOT so sure that a FAIL should be the outcome here ... we are skipping if cpu is not amongst cgroups or sched_debug is NOT available...BUT at this point we have run (or skipped) 7 tests so far and there are 3 more to go...so why can't we just TCONF ?
(beside the fact that as I said in the cover-letter one TCONF will lead to the whole 10-subtest croup suite to be reported as TCONF...)

Regards

Cristian

Quoted 7 lines

Other option would be to use TINFO and don't skip test (we're trying to find kernel crashes, don't we), that's probably a bit crazy.

Kind regards,
Petr

#4 lore raw

PATCH 1/1 cgroup_regression_test.sh: Fix TWARN usage

Hi Cristian,

...

Quoted 10 lines
Quoted 9 lines

testcases/kernel/controllers/cgroup/cgroup_regression_test.sh test_7_2() · line 411

Quoted 7 lines

+++ b/testcases/kernel/controllers/cgroup/cgroup_regression_test.sh

testcases/kernel/controllers/cgroup/cgroup_regression_test.sh +1 -1
@@ -408,7 +408,7 @@ test_7_2()
 
 	grep -q -w "cpu" /proc/cgroups
 	if [ $? -ne 0 -o ! -e /proc/sched_debug ]; then
-		tst_res TWARN "skip rest of testing due possible oops triggered by reading /proc/sched_debug"
+		tst_res TCONF "skip rest of testing due possible oops triggered by reading /proc/sched_debug"

Right, TWARN is not good (IMHO TWARN is not popular in LTP testsuites due marking test failing), but if you don't mind I'd merge it with TBROK (more appropriate in this case).

Quoted 4 lines

mmm...I'm NOT so sure that a FAIL should be the outcome here ... we are skipping if cpu is not amongst cgroups or sched_debug is NOT available...BUT at this point we have run (or skipped) 7 tests so far and there are 3 more to go...so why can't we just TCONF ?

OK I got that :) /proc/sched_debug is enabled by CONFIG_SCHED_DEBUG, os it's kind of configuration issue. Before I thought that failing "grep -q -w "cpu" /proc/cgroups" means failure/bug found by test, but it's not.

So I'm ok with this change.

Quoted 2 lines

(beside the fact that as I said in the cover-letter one TCONF will lead to the whole 10-subtest croup suite to be reported as TCONF...)

IMHO that's not true (see my reply to cover letter).

Kind regards,
Petr

#6 lore raw

cover cgroup_regression_test.sh fix

In reply to Cristian Marussi

Hi Cristian,

I overlooked this cover letter (read the patch [1] via patchwork and didn't expect cover letter for single patch, I usually put info below additional ---).

Quoted 55 lines

Hi

while working on this LTP tip (LTP Jan19 REL + fixes):

81e28eab1 memcg_stress_test.sh: fix memory usage
b25c719de memcg_stress_test.sh: Further cleanup + print info a6d7d16f4 memcg_stress_test.sh: ported to newlib
6a651fe16 controllers/cpuset: cpuset.cpus file doesn't exist which triggers TFAIL fa049ec9f controllers/cgroup_regression_test.sh: Fix two issues a6a5caef1 (tag: 20190115, LTP_20190115) LTP 20190115

I realized 'cgroup' testcase is now wrongly reported FAIL even if NO failures happens in the sub testcases:

Running tests.......
<<<test_start>>>
tag=cgroup stim[ 125.419776] LTP: starting cgroup ( cgroup_regression_test.sh)
e=1550611029
cmdline=" cgroup_regression_test.sh"
contacts=""
analysis=exit
<<<test_output>>>
incrementing stop
cgroup_regression_test 1 TINFO: timeout per run is 0h 5m 0s cgroup_regression_test 1 TPASS: no kernel bug was found cgroup_regression_test 2 TPASS: notify_on_release is inherited cgroup_regression_test 3 TCONF: CONFIG_SCHED_DEBUG is not enabled cgroup_regression_test 4 TCONF: CONFIG_LOCKDEP is not enabled cgroup_regression_test 5 TPASS: no kernel bug was found cgroup_regression_test 6 TCONF: CONFIG_CGROUP_NS is NOT supported in Kernels >= 3.0 cgroup_regression_test 7 TPASS: no kernel bug was found for test 1

[  126.879398] option changes via remount are deprecated (pid=8105 comm=mount)

cgroup_regression_test 7 TWARN: skip rest of testing due possible oops triggered by reading /proc/sched_debug cgroup_regression_test 7 TPASS: no kernel bug was found for test 2
cgroup_regression_test 8 TPASS: no kernel bug was found
cgroup_regression_test 9 TPASS: no kernel bug was found
mkdir: cannot create directory 'cgroup/0': File exists
cgroup_regression_test 10 TPASS: no kernel bug was found

Summary:

passed   8
failed   0
skipped  3

warnings 1
<<<execution_status>>>
initiation_status="ok"
duration=61 termination_type=exited termination_id=36 corefile=no cutime=4314 cstime=5792
<<<test_end>>>
INFO: ltp-pan reported some tests FAIL
LTP Version: 20190115

Quoted 8 lines

Done executing testcases.
LTP Version: 20190115

This seems to be due to the TWARN added in 78bfa7e785e87d5e7e4e6ebd518ca9a67c78a1f3 which leads to an exit code interpreted by ltp-pan as failure.
Is this expected ? An ltp-pan issue ? Is there something wrong in my setup ?

This is maybe surprising, but deliberate behavior.
IMHO this leads to to the fact that TWARN is fewer used than the others. @metan: Maybe it'd make sense to change TWARN not producing FAIL.

Quoted 7 lines

A possibly trivial patch follows, BUT anyway there still an unresolved problem: in the above test run even using TINFO instead of TWARN, I'll have the whole cgroup_regression_test.sh test case reported as TCONF, since some of the contained sub test cases have been in fact skipped: is there any way to address this kind of situation in the LTP framework ? (beside using TINFO everywhere) I understand the scenario is tricky: a test case with many sub-testcases that could each PASS/FAIL/SKIP. Any thought/guideline on this ?

FAIL is IMHO in TWARN, TFAIL and TBROK.

TCONF is taken as PASS, see:

cgroup_regression_test 1 TINFO: timeout per run is 0h 5m 0s cgroup_regression_test 1 TPASS: no kernel bug was found cgroup_regression_test 2 TPASS: notify_on_release is inherited cgroup_regression_test 3 TPASS: no kernel bug was found cgroup_regression_test 4 TCONF: CONFIG_LOCKDEP is not enabled cgroup_regression_test 5 TPASS: no kernel bug was found cgroup_regression_test 6 TCONF: CONFIG_CGROUP_NS is NOT supported in Kernels >= 3.0 cgroup_regression_test 7 TPASS: no kernel bug was found for test 1 cgroup_regression_test 7 TPASS: no kernel bug was found for test 2 cgroup_regression_test 8 TPASS: no kernel bug was found cgroup_regression_test 9 TPASS: no kernel bug was found cgroup_regression_test 10 TPASS: no kernel bug was found

Summary:

passed   9
failed   0
skipped  2

warnings 0
incrementing stop
<<<execution_status>>>
initiation_status="ok"
duration=92 termination_type=exited termination_id=32 corefile=no cutime=12606 cstime=5151
<<<test_end>>>
INFO: ltp-pan reported all tests PASS

Kind regards,
Petr

[1] https://patchwork.ozlabs.org/patch/1042942/

Quoted 5 lines

Cristian Marussi (1):
cgroup_regression_test.sh: Fix TWARN usage

 testcases/kernel/controllers/cgroup/cgroup_regression_test.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)