Skip to contentSkip to conversation

pipe13: Scale child reap window with LTP_TIMEOUT_MUL

PATCH Superseded Stephen Bertram via ltp 2 participants 10 messages Jul 10 – Jul 30

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 pipe13: Scale child reap window with LTP_TIMEOUT_MUL

The post-close wait used a fixed 1000000 us cap on exponential backoff (~524 ms total). Scale that cap with tst_multiply_timeout() so debug kernels and LTP_TIMEOUT_MUL apply, fixing failures under parallel Kirk workers without changing default behavior on non-debug systems.

Before (only with case 100):

pipe13.c:50: TINFO: Creating 100 child processes pipe13.c:81: TINFO: pid 435007 still sleeps
...
pipe13.c:81: TINFO: pid 435300 still sleeps pipe13.c:89: TFAIL: Closed pipe didn't wake up everyone The list varied but sometimes up to 20 would remained asleep.

After using tst_multiply_timeout(), test passes.

The failure before would happen, when using 4 paralell workers on a debug kernel, about 1 to 2 times when iterated 5 times. With the addition of the function, under the same conditions, it has not failed in 100 iterations.

---
 testcases/kernel/syscalls/pipe/pipe13.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
testcases/kernel/syscalls/pipe/pipe13.c +2 -1
index 5d76e1f00..cf1ccbcbc 100644
@@ -44,6 +44,7 @@ static void verify_pipe(unsigned int n)
 	int ret;
 	unsigned int i, cnt = 0, sleep_us = 1, fail = 0;
 	unsigned int child_num = tcases[n];
+	unsigned int sleep_us_cap = tst_multiply_timeout(1000) * 1000U;
 	int pid[child_num];
 
 	SAFE_PIPE(fds);
@@ -60,7 +61,7 @@ static void verify_pipe(unsigned int n)
 	SAFE_CLOSE(fds[0]);
 	SAFE_CLOSE(fds[1]);
 
-	while (cnt < child_num && sleep_us < 1000000) {
+	while (cnt < child_num && sleep_us < sleep_us_cap) {
 		ret = waitpid(-1, NULL, WNOHANG);
 		if (ret < 0)
 			tst_brk(TBROK | TERRNO, "waitpid()");
Signature · 5 lines
2.54.0


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
#1 lore raw

Hi Stephen,

Quoted 4 lines · Stephen Bertram

pipe13: Scale child reap window with LTP_TIMEOUT_MUL

The post-close wait used a fixed 1000000 us cap on exponential backoff (~524 ms total).

The total accumulated sleep time is 1+2+4+...+524288 = 1048575 us, which is approximately 1048 ms (~1 second), not 524 ms.

524288 us (~524 ms) is the maximum value of a single usleep() call within the loop -- not the total.

Quoted 3 lines

on a debug kernel, about 1 to 2 times when iterated 5 times. With the addition of the function, under the same conditions, it has not failed in 100 iterations.

"paralell" -> "parallel".

Verdict - Needs revision

Pre-existing: pipe13.c carries an old-style "Test Description:" comment block instead of the /*\ ... */ RST doc-comment format now required by LTP conventions. Not introduced by this patch, but worth fixing in a follow-up.

---
Note:

The agent can sometimes produce false positives although often its findings are genuine. If you find issues with the review, please comment this email or ignore the suggestions.

Regards,
LTP AI Reviewer

Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
#2 lore raw
In reply to Stephen Bertram via ltp

Hi Stephen,

Quoted 31 lines

The post-close wait used a fixed 1000000 us cap on exponential backoff (~524 ms total). Scale that cap with tst_multiply_timeout() so debug kernels and LTP_TIMEOUT_MUL apply, fixing failures under parallel Kirk workers without changing default behavior on non-debug systems.

Before (only with case 100):

pipe13.c:50: TINFO: Creating 100 child processes pipe13.c:81: TINFO: pid 435007 still sleeps
...
pipe13.c:81: TINFO: pid 435300 still sleeps pipe13.c:89: TFAIL: Closed pipe didn't wake up everyone The list varied but sometimes up to 20 would remained asleep.

After using tst_multiply_timeout(), test passes.

The failure before would happen, when using 4 paralell workers on a debug kernel, about 1 to 2 times when iterated 5 times. With the addition of the function, under the same conditions, it has not failed in 100 iterations.

---
 testcases/kernel/syscalls/pipe/pipe13.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
testcases/kernel/syscalls/pipe/pipe13.c +0 -0
index 5d76e1f00..cf1ccbcbc 100644

testcases/kernel/syscalls/pipe/pipe13.c static void verify_pipe(unsigned int n) · line 47

Quoted 4 lines
testcases/kernel/syscalls/pipe/pipe13.c +1 -0
@@ -44,6 +44,7 @@ static void verify_pipe(unsigned int n)
 	int ret;
 	unsigned int i, cnt = 0, sleep_us = 1, fail = 0;
 	unsigned int child_num = tcases[n];
+	unsigned int sleep_us_cap = tst_multiply_timeout(1000) * 1000U;

Even better: we can create a function to reap children and verify if they are completed while counting them.
Then we can use TST_RETRY_FN_EXP_BACKOFF() on it.

The problem is that we are waiting a fixed amount of time (~1s) that is not enough when system is overloaded.

Most likely, when you are calling kirk parallel execution, other tests are overloading the system and this test fails accordingly.

Regards,
--
Andrea Cervesato
SUSE QE Automation Engineer Linux
[email protected]

Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
#3 lore raw

Hi Andrea,

When isolated, I can reproduce the issue, always with the 100 scenario, on a debug kernel. The debug kernel I use has locking and kmemleak active which slows performance greatly.

As for the scaling, LTP_TIMEOUT_MUL will allow users to scale larger as needed, so I don't think the initial time, 1s, is an issue with the implementation. It keeps it as before, without using the multiple, but now allows us to incorporate the multiple for longer wait times if needed. If you think we should start higher I could, but when I run this test without even using the variable multiplier, it works fine because tst_multiply_timeout also adds a 4x multiple for debug kernels.

Thank you for the suggestion and I will make changes to work with TST_RETRY_FN_EXP_BACKOFF().

thanks,

stephen
He/His/Him

Quoted 56 lines · Andrea Cervesato

Hi Stephen,

Quoted 21 lines

The post-close wait used a fixed 1000000 us cap on exponential backoff (~524 ms total). Scale that cap with tst_multiply_timeout() so debug kernels and LTP_TIMEOUT_MUL apply, fixing failures under parallel Kirk workers without changing default behavior on non-debug systems.

Before (only with case 100):

pipe13.c:50: TINFO: Creating 100 child processes pipe13.c:81: TINFO: pid 435007 still sleeps
...
pipe13.c:81: TINFO: pid 435300 still sleeps pipe13.c:89: TFAIL: Closed pipe didn't wake up everyone The list varied but sometimes up to 20 would remained asleep.

After using tst_multiply_timeout(), test passes.

The failure before would happen, when using 4 paralell workers on a debug kernel, about 1 to 2 times when iterated 5 times. With the addition of the function, under the same conditions, it has not failed in 100 iterations.

Review

Quoted 6 lines
---
 testcases/kernel/syscalls/pipe/pipe13.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
(quoted diff) +0 -0

b/testcases/kernel/syscalls/pipe/pipe13.c

testcases/kernel/syscalls/pipe/pipe13.c static void verify_pipe(unsigned int n) · line 47

Quoted 8 lines

index 5d76e1f00..cf1ccbcbc 100644

testcases/kernel/syscalls/pipe/pipe13.c +1 -0
@@ -44,6 +44,7 @@ static void verify_pipe(unsigned int n)
      int ret;
      unsigned int i, cnt = 0, sleep_us = 1, fail = 0;
      unsigned int child_num = tcases[n];
+     unsigned int sleep_us_cap = tst_multiply_timeout(1000) * 1000U;

Even better: we can create a function to reap children and verify if they are completed while counting them.
Then we can use TST_RETRY_FN_EXP_BACKOFF() on it.

The problem is that we are waiting a fixed amount of time (~1s) that is not enough when system is overloaded.

Most likely, when you are calling kirk parallel execution, other tests are overloading the system and this test fails accordingly.

Regards,
--
Andrea Cervesato
SUSE QE Automation Engineer Linux
[email protected]

Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
#4 lore raw

PATCH v2 1/1 pipe13: Scale child reap window with LTP_TIMEOUT_MUL

In reply to Stephen Bertram via ltp

The post-close wait used a fixed 1000000 us cap on exponential backoff. Scale that cap with tst_multiply_timeout() so debug
kernels and LTP_TIMEOUT_MUL apply, fixing failures under parallel Kirk workers without changing default behavior on non-debug systems.

Before (only with case 100):

pipe13.c:50: TINFO: Creating 100 child processes pipe13.c:81: TINFO: pid 435007 still sleeps
...
pipe13.c:81: TINFO: pid 435300 still sleeps pipe13.c:89: TFAIL: Closed pipe didn't wake up everyone The list varied but sometimes up to 20 would remained asleep.

Test is passing after using TST_RETRY_FN_EXP_BACKOFF() so timeout scaling follows the standard LTP path (tst_multiply_timeout, debug kconfig, LTP_TIMEOUT_MUL).

The failure before would happen, when using 4 paralell workers on a debug kernel, about 1 to 2 times when iterated 5 times. Tested on aarch64 debug+PREEMPT_RT: kirk -w 4 pipe13 -i 100, 400 runs, 0 fail.

---
 testcases/kernel/syscalls/pipe/pipe13.c | 44 +++++++++++++++----------
 1 file changed, 27 insertions(+), 17 deletions(-)
testcases/kernel/syscalls/pipe/pipe13.c +27 -17
index 5d76e1f00..7f7685a2c 100644
@@ -26,6 +26,28 @@ static unsigned int tcases[] = {
 };
 
 static int fds[2];
+static unsigned int reap_child_num;
+static unsigned int reap_count;
+static int *reap_pids;
+
+static int reap_children_once(void)
+{
+	unsigned int i;
+	int ret;
+
+	while ((ret = waitpid(-1, NULL, WNOHANG)) > 0) {
+		reap_count++;
+		for (i = 0; i < reap_child_num; i++) {
+			if (reap_pids[i] == ret)
+				reap_pids[i] = 0;
+		}
+	}
+	if (ret < 0 && errno != ECHILD)
+		tst_brk(TBROK | TERRNO, "waitpid()");
+	return reap_count;
+}
+
+#define ALL_CHILDREN_REAPED(cnt) ((unsigned int)(cnt) >= reap_child_num)
 
 static void do_child(unsigned int i)
 {
@@ -41,8 +63,7 @@ static void do_child(unsigned int i)
 
 static void verify_pipe(unsigned int n)
 {
-	int ret;
-	unsigned int i, cnt = 0, sleep_us = 1, fail = 0;
+	unsigned int i, fail = 0;
 	unsigned int child_num = tcases[n];
 	int pid[child_num];
 
@@ -60,21 +81,10 @@ static void verify_pipe(unsigned int n)
 	SAFE_CLOSE(fds[0]);
 	SAFE_CLOSE(fds[1]);
 
-	while (cnt < child_num && sleep_us < 1000000) {
-		ret = waitpid(-1, NULL, WNOHANG);
-		if (ret < 0)
-			tst_brk(TBROK | TERRNO, "waitpid()");
-		if (ret > 0) {
-			cnt++;
-			for (i = 0; i < child_num; i++) {
-				if (pid[i] == ret)
-					pid[i] = 0;
-			}
-			continue;
-		}
-		usleep(sleep_us);
-		sleep_us *= 2;
-	}
+	reap_child_num = child_num;
+	reap_count = 0;
+	reap_pids = pid;
+	TST_RETRY_FN_EXP_BACKOFF(reap_children_once(), ALL_CHILDREN_REAPED, 1);
 
 	for (i = 0; i < child_num; i++) {
 		if (pid[i]) {
Signature · 5 lines
2.54.0


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
#5 lore raw

Hi Stephen,

Quoted 3 lines · Stephen Bertram

pipe13: Scale child reap window with LTP_TIMEOUT_MUL

The failure before would happen, when using 4 paralell workers

Typo: "paralell" should be "parallel".

Verdict - Needs revision

---
Note:

The agent can sometimes produce false positives although often its findings are genuine. If you find issues with the review, please comment this email or ignore the suggestions.

Regards,
LTP AI Reviewer

Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
#6 lore raw

PATCH v3 1/1 pipe13: Scale child reap window with LTP_TIMEOUT_MUL

In reply to Stephen Bertram via ltp

The post-close wait used a fixed 1000000 us cap on exponential backoff. Scale that cap with tst_multiply_timeout() so debug
kernels and LTP_TIMEOUT_MUL apply, fixing failures under parallel Kirk workers without changing default behavior on non-debug systems.

Before (only with case 100):

pipe13.c:50: TINFO: Creating 100 child processes pipe13.c:81: TINFO: pid 435007 still sleeps
...
pipe13.c:81: TINFO: pid 435300 still sleeps pipe13.c:89: TFAIL: Closed pipe didn't wake up everyone The list varied but sometimes up to 20 would remained asleep.

Test is passing after using TST_RETRY_FN_EXP_BACKOFF() so timeout scaling follows the standard LTP path (tst_multiply_timeout, debug kconfig, LTP_TIMEOUT_MUL).

The failure before would happen, when using 4 parallel workers on a debug kernel, about 1 to 2 times when iterated 5 times. Tested on aarch64 debug+PREEMPT_RT: kirk -w 4 pipe13 -i 100, 400 runs, 0 fail.

---
 testcases/kernel/syscalls/pipe/pipe13.c | 44 +++++++++++++++----------
 1 file changed, 27 insertions(+), 17 deletions(-)
testcases/kernel/syscalls/pipe/pipe13.c +27 -17
index 5d76e1f00..7f7685a2c 100644
@@ -26,6 +26,28 @@ static unsigned int tcases[] = {
 };
 
 static int fds[2];
+static unsigned int reap_child_num;
+static unsigned int reap_count;
+static int *reap_pids;
+
+static int reap_children_once(void)
+{
+	unsigned int i;
+	int ret;
+
+	while ((ret = waitpid(-1, NULL, WNOHANG)) > 0) {
+		reap_count++;
+		for (i = 0; i < reap_child_num; i++) {
+			if (reap_pids[i] == ret)
+				reap_pids[i] = 0;
+		}
+	}
+	if (ret < 0 && errno != ECHILD)
+		tst_brk(TBROK | TERRNO, "waitpid()");
+	return reap_count;
+}
+
+#define ALL_CHILDREN_REAPED(cnt) ((unsigned int)(cnt) >= reap_child_num)
 
 static void do_child(unsigned int i)
 {
@@ -41,8 +63,7 @@ static void do_child(unsigned int i)
 
 static void verify_pipe(unsigned int n)
 {
-	int ret;
-	unsigned int i, cnt = 0, sleep_us = 1, fail = 0;
+	unsigned int i, fail = 0;
 	unsigned int child_num = tcases[n];
 	int pid[child_num];
 
@@ -60,21 +81,10 @@ static void verify_pipe(unsigned int n)
 	SAFE_CLOSE(fds[0]);
 	SAFE_CLOSE(fds[1]);
 
-	while (cnt < child_num && sleep_us < 1000000) {
-		ret = waitpid(-1, NULL, WNOHANG);
-		if (ret < 0)
-			tst_brk(TBROK | TERRNO, "waitpid()");
-		if (ret > 0) {
-			cnt++;
-			for (i = 0; i < child_num; i++) {
-				if (pid[i] == ret)
-					pid[i] = 0;
-			}
-			continue;
-		}
-		usleep(sleep_us);
-		sleep_us *= 2;
-	}
+	reap_child_num = child_num;
+	reap_count = 0;
+	reap_pids = pid;
+	TST_RETRY_FN_EXP_BACKOFF(reap_children_once(), ALL_CHILDREN_REAPED, 1);
 
 	for (i = 0; i < child_num; i++) {
 		if (pid[i]) {
Signature · 5 lines
2.54.0


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
#7 lore raw

Hi Stephen,

Quoted 1 line · Stephen Bertram

pipe13: Scale child reap window with LTP_TIMEOUT_MUL

Verdict - Reviewed

---
Note:

The agent can sometimes produce false positives although often its findings are genuine. If you find issues with the review, please comment this email or ignore the suggestions.

Regards,
LTP AI Reviewer

Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp