pipe13: Scale child reap window with LTP_TIMEOUT_MUL
PATCH 1/1 pipe13: Scale child reap window with LTP_TIMEOUT_MUL
The post-close wait used a fixed 1000000 us cap on exponential backoff (~524 ms total). Scale that cap with tst_multiply_timeout() so debug kernels and LTP_TIMEOUT_MUL apply, fixing failures under parallel Kirk workers without changing default behavior on non-debug systems.
Before (only with case 100):
pipe13.c:50: TINFO: Creating 100 child processes pipe13.c:81: TINFO: pid 435007 still sleeps
...
pipe13.c:81: TINFO: pid 435300 still sleeps pipe13.c:89: TFAIL: Closed pipe didn't wake up everyone The list varied but sometimes up to 20 would remained asleep.
After using tst_multiply_timeout(), test passes.
The failure before would happen, when using 4 paralell workers on a debug kernel, about 1 to 2 times when iterated 5 times. With the addition of the function, under the same conditions, it has not failed in 100 iterations.
--- testcases/kernel/syscalls/pipe/pipe13.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
testcases/kernel/syscalls/pipe/pipe13.c +2 -1
@@ -44,6 +44,7 @@ static void verify_pipe(unsigned int n)int ret;unsigned int i, cnt =0 , sleep_us =1 , fail =0 ;unsigned int child_num = tcases[n]; +unsigned int sleep_us_cap =tst_multiply_timeout (1000 ) *1000U ;int pid[child_num];SAFE_PIPE (fds);
@@ -60,7 +61,7 @@ static void verify_pipe(unsigned int n)SAFE_CLOSE (fds[0 ]);SAFE_CLOSE (fds[1 ]); -while (cnt < child_num && sleep_us <1000000 ) { +while (cnt < child_num && sleep_us < sleep_us_cap) { ret =waitpid (-1 ,NULL ,WNOHANG );if (ret <0 )tst_brk (TBROK |TERRNO ,"waitpid()" );
Signature · 5 lines
2.54.0 -- Mailing list info: https://lists.linux.it/listinfo/ltp
Hi Stephen,
Quoted 4 lines · Stephen Bertram
pipe13: Scale child reap window with
LTP_TIMEOUT_MULThe post-close wait used a fixed 1000000 us cap on exponential backoff (~524 ms total).
The total accumulated sleep time is 1+2+4+...+524288 = 1048575 us, which is approximately 1048 ms (~1 second), not 524 ms.
524288 us (~524 ms) is the maximum value of a single usleep() call within the loop -- not the total.
Quoted 3 lines
on a debug kernel, about 1 to 2 times when iterated 5 times. With the addition of the function, under the same conditions, it has not failed in 100 iterations.
"paralell" -> "parallel".
Verdict - Needs revision
Pre-existing: pipe13.c carries an old-style "Test Description:" comment block instead of the /*\ ... */ RST doc-comment format now required by LTP conventions. Not introduced by this patch, but worth fixing in a follow-up.
---
Note:
The agent can sometimes produce false positives although often its findings are genuine. If you find issues with the review, please comment this email or ignore the suggestions.
Regards,
LTP AI Reviewer
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
Hi Stephen,
Quoted 31 lines
The post-close wait used a fixed 1000000 us cap on exponential backoff (~524 ms total). Scale that cap with
tst_multiply_timeout()so debug kernels andLTP_TIMEOUT_MULapply, fixing failures under parallel Kirk workers without changing default behavior on non-debug systems.Before (only with case 100):
pipe13.c:50: TINFO: Creating 100 child processes pipe13.c:81: TINFO: pid 435007 still sleeps
...
pipe13.c:81: TINFO: pid 435300 still sleeps pipe13.c:89: TFAIL: Closed pipe didn't wake up everyone The list varied but sometimes up to 20 would remained asleep.After using
tst_multiply_timeout(), test passes.The failure before would happen, when using 4 paralell workers on a debug kernel, about 1 to 2 times when iterated 5 times. With the addition of the function, under the same conditions, it has not failed in 100 iterations.
Signed-off-by Stephen Bertram <[email protected]>--- testcases/kernel/syscalls/pipe/pipe13.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)testcases/kernel/syscalls/pipe/pipe13.c +0
-0
testcases/kernel/syscalls/pipe/pipe13.c static void verify_pipe(unsigned int n) · line 47
Quoted 4 lines
testcases/kernel/syscalls/pipe/pipe13.c +1
-0@@ -44,6 +44,7 @@ static void verify_pipe(unsigned int n)int ret;unsigned int i, cnt =0 , sleep_us =1 , fail =0 ;unsigned int child_num = tcases[n]; +unsigned int sleep_us_cap =tst_multiply_timeout (1000 ) *1000U ;
Even better: we can create a function to reap children and verify if they are completed while counting them.
Then we can use TST_RETRY_FN_EXP_BACKOFF() on it.
The problem is that we are waiting a fixed amount of time (~1s) that is not enough when system is overloaded.
Most likely, when you are calling kirk parallel execution, other tests are overloading the system and this test fails accordingly.
Regards,
--
Andrea Cervesato
SUSE QE Automation Engineer Linux
[email protected]
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
Hi Andrea,
When isolated, I can reproduce the issue, always with the 100 scenario, on a debug kernel. The debug kernel I use has locking and kmemleak active which slows performance greatly.
As for the scaling, LTP_TIMEOUT_MUL will allow users to scale larger as needed, so I don't think the initial time, 1s, is an issue with the implementation. It keeps it as before, without using the multiple, but now allows us to incorporate the multiple for longer wait times if needed. If you think we should start higher I could, but when I run this test without even using the variable multiplier, it works fine because tst_multiply_timeout also adds a 4x multiple for debug kernels.
Thank you for the suggestion and I will make changes to work with TST_RETRY_FN_EXP_BACKOFF().
thanks,
stephen
He/His/Him
Quoted 56 lines · Andrea Cervesato
Hi Stephen,
Quoted 21 lines
The post-close wait used a fixed 1000000 us cap on exponential backoff (~524 ms total). Scale that cap with
tst_multiply_timeout()so debug kernels andLTP_TIMEOUT_MULapply, fixing failures under parallel Kirk workers without changing default behavior on non-debug systems.Before (only with case 100):
pipe13.c:50: TINFO: Creating 100 child processes pipe13.c:81: TINFO: pid 435007 still sleeps
...
pipe13.c:81: TINFO: pid 435300 still sleeps pipe13.c:89: TFAIL: Closed pipe didn't wake up everyone The list varied but sometimes up to 20 would remained asleep.After using
tst_multiply_timeout(), test passes.The failure before would happen, when using 4 paralell workers on a debug kernel, about 1 to 2 times when iterated 5 times. With the addition of the function, under the same conditions, it has not failed in 100 iterations.
Review
Quoted 6 lines
Signed-off-by Stephen Bertram <[email protected]>--- testcases/kernel/syscalls/pipe/pipe13.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)(quoted diff) +0
-0
b/testcases/kernel/syscalls/pipe/pipe13.ctestcases/kernel/syscalls/pipe/pipe13.c static void verify_pipe(unsigned int n) · line 47
Quoted 8 lines
index
5d76e1f00..cf1ccbcbc100644testcases/kernel/syscalls/pipe/pipe13.c +1
-0@@ -44,6 +44,7 @@ static void verify_pipe(unsigned int n)int ret;unsigned int i, cnt =0 , sleep_us =1 , fail =0 ;unsigned int child_num = tcases[n]; +unsigned int sleep_us_cap =tst_multiply_timeout (1000 ) *1000U ;Even better: we can create a function to reap children and verify if they are completed while counting them.
Then we can useTST_RETRY_FN_EXP_BACKOFF()on it.The problem is that we are waiting a fixed amount of time (~1s) that is not enough when system is overloaded.
Most likely, when you are calling kirk parallel execution, other tests are overloading the system and this test fails accordingly.
Regards,
--
Andrea Cervesato
SUSE QE Automation Engineer Linux
[email protected]
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH v2 1/1 pipe13: Scale child reap window with LTP_TIMEOUT_MUL
In reply to Stephen Bertram via ltpThe post-close wait used a fixed 1000000 us cap on exponential backoff. Scale that cap with tst_multiply_timeout() so debug
kernels and LTP_TIMEOUT_MUL apply, fixing failures under parallel Kirk workers without changing default behavior on non-debug systems.
Before (only with case 100):
pipe13.c:50: TINFO: Creating 100 child processes pipe13.c:81: TINFO: pid 435007 still sleeps
...
pipe13.c:81: TINFO: pid 435300 still sleeps pipe13.c:89: TFAIL: Closed pipe didn't wake up everyone The list varied but sometimes up to 20 would remained asleep.
Test is passing after using TST_RETRY_FN_EXP_BACKOFF() so timeout scaling follows the standard LTP path (tst_multiply_timeout, debug kconfig, LTP_TIMEOUT_MUL).
The failure before would happen, when using 4 paralell workers on a debug kernel, about 1 to 2 times when iterated 5 times. Tested on aarch64 debug+PREEMPT_RT: kirk -w 4 pipe13 -i 100, 400 runs, 0 fail.
--- testcases/kernel/syscalls/pipe/pipe13.c | 44 +++++++++++++++---------- 1 file changed, 27 insertions(+), 17 deletions(-)
testcases/kernel/syscalls/pipe/pipe13.c +27 -17
@@ -26,6 +26,28 @@ static unsigned int tcases[] = { };static int fds[2 ]; +static unsigned int reap_child_num; +static unsigned int reap_count; +static int *reap_pids; + +static int reap_children_once (void ) +{ +unsigned int i; +int ret; + +while ((ret =waitpid (-1 ,NULL ,WNOHANG )) >0 ) { + reap_count++; +for (i =0 ; i < reap_child_num; i++) { +if (reap_pids[i] == ret) + reap_pids[i] =0 ; + } + } +if (ret <0 && errno !=ECHILD ) +tst_brk (TBROK |TERRNO ,"waitpid()" ); +return reap_count; +} + +#define ALL_CHILDREN_REAPED (cnt) ((unsigned int)(cnt) >= reap_child_num)static void do_child (unsigned int i) {
@@ -41,8 +63,7 @@ static void do_child(unsigned int i)static void verify_pipe (unsigned int n) { -int ret; -unsigned int i, cnt =0 , sleep_us =1 , fail =0 ; +unsigned int i, fail =0 ;unsigned int child_num = tcases[n];int pid[child_num];
@@ -60,21 +81,10 @@ static void verify_pipe(unsigned int n)SAFE_CLOSE (fds[0 ]);SAFE_CLOSE (fds[1 ]); -while (cnt < child_num && sleep_us <1000000 ) { - ret =waitpid (-1 ,NULL ,WNOHANG ); -if (ret <0 ) -tst_brk (TBROK |TERRNO ,"waitpid()" ); -if (ret >0 ) { - cnt++; -for (i =0 ; i < child_num; i++) { -if (pid[i] == ret) - pid[i] =0 ; - } -continue ; - } -usleep (sleep_us); - sleep_us *=2 ; - } + reap_child_num = child_num; + reap_count =0 ; + reap_pids = pid; +TST_RETRY_FN_EXP_BACKOFF (reap_children_once (),ALL_CHILDREN_REAPED ,1 );for (i =0 ; i < child_num; i++) {if (pid[i]) {
Signature · 5 lines
2.54.0 -- Mailing list info: https://lists.linux.it/listinfo/ltp
Hi Stephen,
Quoted 3 lines · Stephen Bertram
pipe13: Scale child reap window with
LTP_TIMEOUT_MULThe failure before would happen, when using 4 paralell workers
Typo: "paralell" should be "parallel".
Verdict - Needs revision
---
Note:
The agent can sometimes produce false positives although often its findings are genuine. If you find issues with the review, please comment this email or ignore the suggestions.
Regards,
LTP AI Reviewer
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH v3 1/1 pipe13: Scale child reap window with LTP_TIMEOUT_MUL
In reply to Stephen Bertram via ltpThe post-close wait used a fixed 1000000 us cap on exponential backoff. Scale that cap with tst_multiply_timeout() so debug
kernels and LTP_TIMEOUT_MUL apply, fixing failures under parallel Kirk workers without changing default behavior on non-debug systems.
Before (only with case 100):
pipe13.c:50: TINFO: Creating 100 child processes pipe13.c:81: TINFO: pid 435007 still sleeps
...
pipe13.c:81: TINFO: pid 435300 still sleeps pipe13.c:89: TFAIL: Closed pipe didn't wake up everyone The list varied but sometimes up to 20 would remained asleep.
Test is passing after using TST_RETRY_FN_EXP_BACKOFF() so timeout scaling follows the standard LTP path (tst_multiply_timeout, debug kconfig, LTP_TIMEOUT_MUL).
The failure before would happen, when using 4 parallel workers on a debug kernel, about 1 to 2 times when iterated 5 times. Tested on aarch64 debug+PREEMPT_RT: kirk -w 4 pipe13 -i 100, 400 runs, 0 fail.
--- testcases/kernel/syscalls/pipe/pipe13.c | 44 +++++++++++++++---------- 1 file changed, 27 insertions(+), 17 deletions(-)
testcases/kernel/syscalls/pipe/pipe13.c +27 -17
@@ -26,6 +26,28 @@ static unsigned int tcases[] = { };static int fds[2 ]; +static unsigned int reap_child_num; +static unsigned int reap_count; +static int *reap_pids; + +static int reap_children_once (void ) +{ +unsigned int i; +int ret; + +while ((ret =waitpid (-1 ,NULL ,WNOHANG )) >0 ) { + reap_count++; +for (i =0 ; i < reap_child_num; i++) { +if (reap_pids[i] == ret) + reap_pids[i] =0 ; + } + } +if (ret <0 && errno !=ECHILD ) +tst_brk (TBROK |TERRNO ,"waitpid()" ); +return reap_count; +} + +#define ALL_CHILDREN_REAPED (cnt) ((unsigned int)(cnt) >= reap_child_num)static void do_child (unsigned int i) {
@@ -41,8 +63,7 @@ static void do_child(unsigned int i)static void verify_pipe (unsigned int n) { -int ret; -unsigned int i, cnt =0 , sleep_us =1 , fail =0 ; +unsigned int i, fail =0 ;unsigned int child_num = tcases[n];int pid[child_num];
@@ -60,21 +81,10 @@ static void verify_pipe(unsigned int n)SAFE_CLOSE (fds[0 ]);SAFE_CLOSE (fds[1 ]); -while (cnt < child_num && sleep_us <1000000 ) { - ret =waitpid (-1 ,NULL ,WNOHANG ); -if (ret <0 ) -tst_brk (TBROK |TERRNO ,"waitpid()" ); -if (ret >0 ) { - cnt++; -for (i =0 ; i < child_num; i++) { -if (pid[i] == ret) - pid[i] =0 ; - } -continue ; - } -usleep (sleep_us); - sleep_us *=2 ; - } + reap_child_num = child_num; + reap_count =0 ; + reap_pids = pid; +TST_RETRY_FN_EXP_BACKOFF (reap_children_once (),ALL_CHILDREN_REAPED ,1 );for (i =0 ; i < child_num; i++) {if (pid[i]) {
Signature · 5 lines
2.54.0 -- Mailing list info: https://lists.linux.it/listinfo/ltp
Hi Stephen,
Quoted 1 line · Stephen Bertram
pipe13: Scale child reap window with
LTP_TIMEOUT_MUL
Verdict - Reviewed
---
Note:
The agent can sometimes produce false positives although often its findings are genuine. If you find issues with the review, please comment this email or ignore the suggestions.
Regards,
LTP AI Reviewer
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
--
Andrea Cervesato
SUSE QE Automation Engineer Linux
[email protected]
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
Andrea Cervesato via ltp
AppliedMerged, Thanks!
--
Andrea Cervesato
SUSE QE Automation Engineer Linux
[email protected]
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp