consolidate 4 TCs message_queue_test_02_* into message_queue_test_02.sh to avoid fail in case of random execution
PATCH 1/1 consolidate 4 TCs message_queue_test_02_* into message_queue_test_02.sh to avoid fail in case of random execution
--- runtest/ipc | 5 +---- testcases/kernel/ipc/ipc_stress/Makefile | 2 +- testcases/kernel/ipc/ipc_stress/message_queue_test_02.sh | 6 ++++++ 3 files changed, 8 insertions(+), 5 deletions(-) create mode 100644 testcases/kernel/ipc/ipc_stress/message_queue_test_02.sh
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-consolidate-4-TCs-message_queue_test_02_-into-messag.patch
Type: text/x-patch
Size: 1534 bytes
Desc: not available
URL: <http://lists.linux.it/pipermail/ltp/attachments/20160715/edef22ab/attachment.bin>
PATCH 1/1 consolidate 4 TCs message_queue_test_02_* into message_queue_test_02.sh to avoid fail in case of random execution
Hi!
Quoted 6 lines
runtest/ipc +0
-0@@ -32,10 +32,7 @@ sem01 sem01 sem02 sem02
runtest/ipc sem01 sem01 · line 38
Quoted 10 lines
runtest/ipc +1
-4message_queue_test_01 message_queue_test_01 -message_queue_test_02_get message_queue_test_02_get -message_queue_test_02_snd message_queue_test_02_snd -message_queue_test_02_rcv message_queue_test_02_rcv -message_queue_test_02_ctl message_queue_test_02_ctl -r +message_queue_test_02 message_queue_test_02.sh message_queue_test_04 message_queue_test_04 message_queue_test_05 message_queue_test_05 pipe_test_01 pipe_test_01
This is exactly against our guidelines, see:
https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines#211-runtest-files
Signature · 2 lines
Cyril Hrubis [email protected]
PATCH 1/1 consolidate 4 TCs message_queue_test_02_* into message_queue_test_02.sh to avoid fail in case of random execution
Hi Cyril,
Quoted 29 lines
-----Original Message-----
From: ltp [mailto:ltp-bounces+fixed- [email protected]] On Behalf Of Cyril Hrubis
Sent: Friday, July 15, 2016 11:56 AM
To: Hoang Thi Thuy Nga (RBVH/ENG2) <[email protected]>
Cc: Hoang Thi Thuy Nga (RBVH/ENG2) <[email protected]>; [email protected]
Subject: Re: [LTP] [PATCH] consolidate 4 TCs message_queue_test_02_* intomessage_queue_test_02.shto avoid fail in case of random executionHi!
Quoted 5 lines
runtest/ipc +0
-0@@ -32,10 +32,7 @@ sem01 sem01 sem02 sem02runtest/ipc sem01 sem01 · line 38
Quoted 10 lines
runtest/ipc +1
-4message_queue_test_01 message_queue_test_01 -message_queue_test_02_get message_queue_test_02_get -message_queue_test_02_snd message_queue_test_02_snd -message_queue_test_02_rcv message_queue_test_02_rcv -message_queue_test_02_ctl message_queue_test_02_ctl -r +message_queue_test_02 message_queue_test_02.sh message_queue_test_04 message_queue_test_04 message_queue_test_05 message_queue_test_05 pipe_test_01 pipe_test_01This is exactly against our guidelines, see:
Quoted 2 lines
https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines#211-runtest-
files
What exactly do you mean?
To avoid confusions I need to explain background of this patch:
We working on remote execution of LTPs on the target. It means, some script is taking test list and execute each of entry over ssh. In this case we are able to track resets and continue testing after it.
So, now we found that some resets are not actially triggered by some specific test. So we decided to randomize test order.
This allowed us to find some more bugs on the system, but introduced issues with tests which depend on each other. For example message_queue_test_02_* can be executed only in some specific order.
What would be the proper way to solve this issue?
PATCH 1/1 consolidate 4 TCs message_queue_test_02_* into message_queue_test_02.sh to avoid fail in case of random execution
Hi!
Quoted 14 lines
Quoted 14 lines
Review
Quoted 11 lines
sem02 sem02
message_queue_test_01message_queue_test_01(quoted diff) +1
-4-message_queue_test_02_get message_queue_test_02_get -message_queue_test_02_snd message_queue_test_02_snd -message_queue_test_02_rcv message_queue_test_02_rcv -message_queue_test_02_ctl message_queue_test_02_ctl -r +message_queue_test_02 message_queue_test_02.sh message_queue_test_04 message_queue_test_04 message_queue_test_05 message_queue_test_05 pipe_test_01 pipe_test_01This is exactly against our guidelines, see:
Quoted 4 lines
Quoted 2 lines
https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines#211-runtest-
filesWhat exactly do you mean?
What I mean and is written in the guidelines is:
The runtest files should have one entry per a test. Creating a wrapper that runs all your tests and adding it as a single test into runtest file is strongly discouraged.
Quoted 10 lines
To avoid confusions I need to explain background of this patch: We working on remote execution of LTPs on the target. It means, some script is taking test list and execute each of entry over ssh. In this case we are able to track resets and continue testing after it.
So, now we found that some resets are not actially triggered by some specific test. So we decided to randomize test order. This allowed us to find some more bugs on the system, but introduced issues with tests which depend on each other. For example message_queue_test_02_* can be executed only in some specific order.
Ah, so they depend on each other. That is a valid reason for executing them in a defined order.
You should have written better patch description since this is not exactly clear.
Quoted 1 line
What would be the proper way to solve this issue?
Looking at the code we should probably avoid running these testcasese in ipc runtest file, since they were designed to be executed in a loop to stress the target system. They does not seem very useful when they are executed just once as from the runtest file.
Signature · 2 lines
Cyril Hrubis [email protected]
PATCH 1/1 consolidate 4 TCs message_queue_test_02_* into message_queue_test_02.sh to avoid fail in case of random execution
Quoted 37 lines · Cyril Hrubis
Hi!
Quoted 14 lines
Quoted 14 lines
Review
Quoted 11 lines
sem02 sem02
message_queue_test_01message_queue_test_01(quoted diff) +1
-4-message_queue_test_02_get message_queue_test_02_get -message_queue_test_02_snd message_queue_test_02_snd -message_queue_test_02_rcv message_queue_test_02_rcv -message_queue_test_02_ctl message_queue_test_02_ctl -r +message_queue_test_02 message_queue_test_02.sh message_queue_test_04 message_queue_test_04 message_queue_test_05 message_queue_test_05 pipe_test_01 pipe_test_01This is exactly against our guidelines, see:
Quoted 4 lines
Quoted 2 lines
https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines#211-runtest-
filesWhat exactly do you mean?
What I mean and is written in the guidelines is:
The runtest files should have one entry per a test. Creating a wrapper that runs all your tests and adding it as a single test into runtest file is strongly discouraged.
Quoted 10 lines
To avoid confusions I need to explain background of this patch: We working on remote execution of LTPs on the target. It means, some script is taking test list and execute each of entry over ssh. In this case we are able to track resets and continue testing after it.
So, now we found that some resets are not actially triggered by some specific test. So we decided to randomize test order. This allowed us to find some more bugs on the system, but introduced issues with tests which depend on each other. For example message_queue_test_02_* can be executed only in some specific order.
Quoted 5 lines
Ah, so they depend on each other. That is a valid reason for executing them in a defined order.
You should have written better patch description since this is not exactly clear.
Yea, we are learning to work with upstream directly :)
It will take some time until the patches will be good from first try.
Quoted 6 lines
Quoted 1 line
What would be the proper way to solve this issue?
Looking at the code we should probably avoid running these testcasese in ipc runtest file, since they were designed to be executed in a loop to stress the target system. They does not seem very useful when they are executed just once as from the runtest file.
Hmm.. so, they can be removed for now?
Or there is some other place for this kind of tests?
How about cpuhotplug* tests? Some times they fail, but since they are executed only once, i would expect bad reproducibility.