Skip to contentSkip to conversation

Failing test on failing shell commands

LTP Krzysztof Kozlowski 2 participants 5 messages Jul 13, 2021

lore
Share & tools
lore
#0 lore raw

Hi all,

I have a failing memcg_use_hierarchy_test (in testcases/kernel/controllers/memcg/functional/) which hits unexpected OOM on one of earlier commands, already in the test:

18 echo 1 > memory.use_hierarchy
19 echo $PAGESIZE > memory.limit_in_bytes
20
21 mkdir subgroup
mkdir: cannot create directory ?subgroup?: Cannot allocate memory /home/ubuntu/ltp-install/testcases/bin/memcg_use_hierarchy_test.sh: 22: cd: can't cd to subgroup

The command did not fail, as there is no "set -e" and test fails later. However there could be a case where such failure does not trigger later issue and test is a false-positive.

What is the practice for LTP shell tests to handle kind of preparation commands? Maybe this should be something like:

--------

testcases/kernel/controllers/memcg/functional/memcg_use_hierarchy_test.sh +2 -2
index 8be342499ece..f8dd8946f80b 100755
@@ -18,8 +18,8 @@ test1()
        echo 1 > memory.use_hierarchy
        echo $PAGESIZE > memory.limit_in_bytes
 
-       mkdir subgroup
-       cd subgroup
+       EXPECT_PASS mkdir subgroup
+       EXPECT_PASS cd subgroup
        test_proc_kill $((PAGESIZE * 3)) "--mmap-lock1" $((PAGESIZE * 2)) 0
 
        cd ..

Best regards,
Krzysztof

#1 lore raw

Hi!

Quoted 11 lines

I have a failing memcg_use_hierarchy_test (in testcases/kernel/controllers/memcg/functional/) which hits unexpected OOM on one of earlier commands, already in the test:

18 echo 1 > memory.use_hierarchy
19 echo $PAGESIZE > memory.limit_in_bytes
20
21 mkdir subgroup
mkdir: cannot create directory ???subgroup???: Cannot allocate memory /home/ubuntu/ltp-install/testcases/bin/memcg_use_hierarchy_test.sh: 22: cd: can't cd to subgroup

Quoted 6 lines

The command did not fail, as there is no "set -e" and test fails later. However there could be a case where such failure does not trigger later issue and test is a false-positive.

What is the practice for LTP shell tests to handle kind of preparation commands? Maybe this should be something like:

We do have ROD (as a shorthand for "run or die") in the shell test library, so this really should be:

ROD mkdir subgroup

Signature · 2 lines
Cyril Hrubis
[email protected]
#2 lore raw
Quoted 16 lines · Cyril Hrubis

Hi!

Quoted 15 lines

I have a failing memcg_use_hierarchy_test (in testcases/kernel/controllers/memcg/functional/) which hits unexpected OOM on one of earlier commands, already in the test:

18 echo 1 > memory.use_hierarchy
19 echo $PAGESIZE > memory.limit_in_bytes
20
21 mkdir subgroup
mkdir: cannot create directory ???subgroup???: Cannot allocate memory /home/ubuntu/ltp-install/testcases/bin/memcg_use_hierarchy_test.sh: 22: cd: can't cd to subgroup

The command did not fail, as there is no "set -e" and test fails later. However there could be a case where such failure does not trigger later issue and test is a false-positive.

Quoted 7 lines
Quoted 2 lines

What is the practice for LTP shell tests to handle kind of preparation commands? Maybe this should be something like:

We do have ROD (as a shorthand for "run or die") in the shell test library, so this really should be:

ROD mkdir subgroup

Thanks. This will however fail entire suite (memcg_use_hierarchy_test) and other tests won't get executed. The failure could be only local, so other tests are still meaningful.

Best regards,
Krzysztof

#3 lore raw

Hi!

Quoted 8 lines
Quoted 4 lines

We do have ROD (as a shorthand for "run or die") in the shell test library, so this really should be:

ROD mkdir subgroup

Thanks. This will however fail entire suite (memcg_use_hierarchy_test) and other tests won't get executed. The failure could be only local, so other tests are still meaningful.

If you want to exit the subtest you really have to handle the return values yourself, report the error, and return from the test function.

The question is which status to report. The TBROK status means that test has failed setup phase and there is no point to continue, so this probably should map to tst_res TWARN "...".

Another solution would be splitting the test into three seprate tests so that we can use ROD.

Signature · 2 lines
Cyril Hrubis
[email protected]
#4 lore raw
In reply to Krzysztof Kozlowski
Quoted 7 lines · Krzysztof Kozlowski

Hi all,

I have a failing memcg_use_hierarchy_test (in testcases/kernel/controllers/memcg/functional/) which hits unexpected OOM on one of earlier commands, already in the test:

Quoted 8 lines

18 echo 1 > memory.use_hierarchy
19 echo $PAGESIZE > memory.limit_in_bytes
20
21 mkdir subgroup
mkdir: cannot create directory ?subgroup?: Cannot allocate memory /home/ubuntu/ltp-install/testcases/bin/memcg_use_hierarchy_test.sh: 22: cd: can't cd to subgroup

The command did not fail, as there is no "set -e" and test fails later.

Ah, I wanted to say:
"The command did not fail entire test, as there is no "set -e" and test fails later."

Quoted 24 lines

However there could be a case where such failure does not trigger later issue and test is a false-positive.

What is the practice for LTP shell tests to handle kind of preparation commands? Maybe this should be something like:

--------

testcases/kernel/controllers/memcg/functional/memcg_use_hierarchy_test.sh +2 -2
index 8be342499ece..f8dd8946f80b 100755
@@ -18,8 +18,8 @@ test1()
        echo 1 > memory.use_hierarchy
        echo $PAGESIZE > memory.limit_in_bytes
 
-       mkdir subgroup
-       cd subgroup
+       EXPECT_PASS mkdir subgroup
+       EXPECT_PASS cd subgroup
        test_proc_kill $((PAGESIZE * 3)) "--mmap-lock1" $((PAGESIZE * 2)) 0
 
        cd ..
Quoted 3 lines

Best regards,
Krzysztof

Best regards,
Krzysztof