Skip to contentSkip to conversation

netstress: explicitly set a thread stack size

PATCH New [email protected] 3 participants 10 messages Nov 25, 2020 – Dec 1, 2020

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 netstress: explicitly set a thread stack size

Musl libc uses a relatively small thread stack size (128k [1]). This gets used up on 2 local buffers sized max_msg_len (64k by default), which causes a segfault due to a stack overflow in the error reporting path.

Set the stack size to 128kB + 2*max_msg_len instead, which is enough for both buffers with an additional allowance for the remaining stack usage by netstress and called libc or ltp helper functions.

[1]: https://wiki.musl-libc.org/functional-differences-from-glibc.html#Thread_stack_size

---
 testcases/network/netstress/netstress.c | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)
testcases/network/netstress/netstress.c +9 -3
index e79e64220..bcd3cd049 100644
@@ -480,7 +480,7 @@ static void client_init(void)
 	clock_gettime(CLOCK_MONOTONIC_RAW, &tv_client_start);
 	intptr_t i;
 	for (i = 0; i < clients_num; ++i)
-		SAFE_PTHREAD_CREATE(&thread_ids[i], 0, client_fn, (void *)i);
+		SAFE_PTHREAD_CREATE(&thread_ids[i], &attr, client_fn, (void *)i);
 }
 
 static void client_run(void)
@@ -747,8 +747,6 @@ static void server_run(void)
 	struct sockaddr_in6 addr6;
 	socklen_t addr_size = sizeof(addr6);
 
-	pthread_attr_init(&attr);
-
 	/*
 	 * detaching threads allow to reclaim thread's resources
 	 * once a thread finishes its work.
@@ -980,6 +978,14 @@ static void setup(void)
 	break;
 	}
 
+	errno = pthread_attr_init(&attr);
+	if (errno != 0)
+		tst_brk(TBROK | TERRNO, "pthread_attr_init failed");
+
+	errno = pthread_attr_setstacksize(&attr, 128*1024 + 2*max_msg_len);
+	if (errno != 0)
+		tst_brk(TBROK | TERRNO, "pthread_attr_setstacksize failed");
+
 	net.init();
 }
 
Signature · 1 line
2.17.1
#1 lore raw

PATCH 1/1 netstress: explicitly set a thread stack size

Quoted 11 lines · Johannes Nixdorf via ltp

Musl libc uses a relatively small thread stack size (128k [1]). This gets used up on 2 local buffers sized max_msg_len (64k by default), which causes a segfault due to a stack overflow in the error reporting path.

Set the stack size to 128kB + 2*max_msg_len instead, which is enough for both buffers with an additional allowance for the remaining stack usage by netstress and called libc or ltp helper functions.

[1]: https://urldefense.com/v3/__https://wiki.musl-libc.org/functional-differences-from-glibc.html*Thread_stack_size__;Iw!!GqivPVa7Brio!JiKwkv8uMgJNgh9ARntUpseI9zZW6h6u3Pm6AeF_sgqVyDcsHIQ_MyJWVVKbh_rVk-8u$

Hi Johannes,

Review

Quoted 2 lines
(quoted diff) +2 -0
+	errno = pthread_attr_init(&attr);
+	if (errno != 0)

if (errno = pthread_attr_init(&attr))

testcases/network/netstress/netstress.c static void setup(void) · line 985

Quoted 3 lines
testcases/network/netstress/netstress.c +3 -0
+		tst_brk(TBROK | TERRNO, "pthread_attr_init failed");
+
+	errno = pthread_attr_setstacksize(&attr, 128*1024 + 2*max_msg_len);

Since max_msg_len is 65535, the result won't be even 4 bytes aligned, perhaps using just 256 * 1024?

Review

Quoted 2 lines
(quoted diff) +2 -0
+	if (errno != 0)
+		tst_brk(TBROK | TERRNO, "pthread_attr_setstacksize failed");

In case of error, it would be good to print the used stack size.

#2 lore raw

PATCH 1/1 netstress: explicitly set a thread stack size

Hi Alexey,

Quoted 6 lines · Alexey Kodanev

testcases/network/netstress/netstress.c static void setup(void) · line 985

Quoted 3 lines
testcases/network/netstress/netstress.c +3 -0
+		tst_brk(TBROK | TERRNO, "pthread_attr_init failed");
+
+	errno = pthread_attr_setstacksize(&attr, 128*1024 + 2*max_msg_len);

Since max_msg_len is 65535, the result won't be even 4 bytes aligned, perhaps using just 256 * 1024?

The function pthread_attr_setstacksize does not have any alignment requirements specified and only sets the minimum stack size. This means the libc is required to over-allocate and suitably align the stack to match platform requirements. Is this broken on any libraries the LTP project cares about?

Note that this is different from pthread_attr_setstack, as there the memory region is provided by the caller and the libc can't change the alignment later on.

I'm reluctant to use a static value here as max_msg_len may be modified by command line arguments.

Regards,
Johannes

#3 lore raw

PATCH 1/1 netstress: explicitly set a thread stack size

Hi!

Quoted 2 lines

I'm reluctant to use a static value here as max_msg_len may be modified by command line arguments.

Wouldn't it be easier to use malloc() instead of putting large data structures on the stack and working around that by increasing the limits?

Signature · 2 lines
Cyril Hrubis
[email protected]
#4 lore raw

PATCH 1/1 netstress: explicitly set a thread stack size

Quoted 7 lines · Cyril Hrubis

Hi!

Quoted 2 lines

I'm reluctant to use a static value here as max_msg_len may be modified by command line arguments.

Wouldn't it be easier to use malloc() instead of putting large data structures on the stack and working around that by increasing the limits?

In the current server implementation it would hurt performance, so I assume it would also require to make a memory/thread-pool...

#5 lore raw

PATCH 1/1 netstress: explicitly set a thread stack size

Hi!

Quoted 9 lines
Quoted 6 lines
Quoted 2 lines

I'm reluctant to use a static value here as max_msg_len may be modified by command line arguments.

Wouldn't it be easier to use malloc() instead of putting large data structures on the stack and working around that by increasing the limits?

In the current server implementation it would hurt performance, so I assume it would also require to make a memory/thread-pool...

Okay, then increasing limits sound like a reasonable workaround for the time being.

Signature · 2 lines
Cyril Hrubis
[email protected]
#6 lore raw

PATCH 1/1 netstress: explicitly set a thread stack size

In reply to [email protected]
Quoted 10 lines · Johannes Nixdorf via ltp

Hi Alexey,

Quoted 6 lines · Alexey Kodanev

testcases/network/netstress/netstress.c static void setup(void) · line 985

Quoted 3 lines
testcases/network/netstress/netstress.c +3 -0
+		tst_brk(TBROK | TERRNO, "pthread_attr_init failed");
+
+	errno = pthread_attr_setstacksize(&attr, 128*1024 + 2*max_msg_len);

Since max_msg_len is 65535, the result won't be even 4 bytes aligned, perhaps using just 256 * 1024?

Quoted 5 lines

The function pthread_attr_setstacksize does not have any alignment requirements specified and only sets the minimum stack size. This means the libc is required to over-allocate and suitably align the stack to match platform requirements. Is this broken on any libraries the LTP project cares about?

It doesn't mean you should intentionally pass unaligned size.

And on it's man-page [1], in errors section, there is a note that it might return EINVAL for some systems if the stack size not multiple of page size.

[1] https://man7.org/linux/man-pages/man3/pthread_attr_setstacksize.3.html

Quoted 7 lines

Note that this is different from pthread_attr_setstack, as there the memory region is provided by the caller and the libc can't change the alignment later on.

I'm reluctant to use a static value here as max_msg_len may be modified by command line arguments.

max_msg_len is const.

#7 lore raw

PATCH 1/1 netstress: explicitly set a thread stack size

-----"Alexey Kodanev" <[email protected]> wrote: -----

Quoted 7 lines · Johannes Nixdorf via ltp

It doesn't mean you should intentionally pass unaligned size.

And on it's man-page [1], in errors section, there is a note that it might
return EINVAL for some systems if the stack size not multiple of page size.

Ok. I was going by the POSIX man page [1], which doesn't mention that practice.

[1]: https://pubs.opengroup.org/onlinepubs/9699919799/functions/pthread_attr_setstacksize.html

Quoted 1 line

max_msg_len is const.

It seems like I was misreading [2] to mean it sets max_msg_len instead of using it as a limit.

[2]: https://github.com/linux-test-project/ltp/blob/master/testcases/network/netstress/netstress.c#L854

Thanks for the review, I'll change it to go with your proposed static limit.

Regards,
Johannes

#8 lore raw

PATCH v2 1/1 netstress: explicitly set a thread stack size

In reply to [email protected]

Musl libc uses a relatively small thread stack size (128k [1]). This gets used up on 2 local buffers sized max_msg_len (64k by default), which causes a segfault due to a stack overflow in the error reporting path.

Set the stack size to 256k instead, which is enough for both buffers with an additional allowance for the remaining stack usage by netstress and called libc or ltp helper functions.

[1]: https://wiki.musl-libc.org/functional-differences-from-glibc.html#Thread_stack_size

---

v2:
- Use a static limit of 256k.
- Document the requested stack size in the error message.
- Coding style fixup.

 testcases/network/netstress/netstress.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)
testcases/network/netstress/netstress.c +7 -3
index e79e64220..48c0c23dc 100644
@@ -480,7 +480,7 @@ static void client_init(void)
 	clock_gettime(CLOCK_MONOTONIC_RAW, &tv_client_start);
 	intptr_t i;
 	for (i = 0; i < clients_num; ++i)
-		SAFE_PTHREAD_CREATE(&thread_ids[i], 0, client_fn, (void *)i);
+		SAFE_PTHREAD_CREATE(&thread_ids[i], &attr, client_fn, (void *)i);
 }
 
 static void client_run(void)
@@ -747,8 +747,6 @@ static void server_run(void)
 	struct sockaddr_in6 addr6;
 	socklen_t addr_size = sizeof(addr6);
 
-	pthread_attr_init(&attr);
-
 	/*
 	 * detaching threads allow to reclaim thread's resources
 	 * once a thread finishes its work.
@@ -980,6 +978,12 @@ static void setup(void)
 	break;
 	}
 
+	if (errno = pthread_attr_init(&attr))
+		tst_brk(TBROK | TERRNO, "pthread_attr_init failed");
+
+	if (errno = pthread_attr_setstacksize(&attr, 256*1024))
+		tst_brk(TBROK | TERRNO, "pthread_attr_setstacksize(256*1024) failed");
+
 	net.init();
 }
 
Signature · 1 line
2.17.1
#9 lore raw

PATCH v2 1/1 netstress: explicitly set a thread stack size

Quoted 11 lines · Johannes Nixdorf via ltp

Musl libc uses a relatively small thread stack size (128k [1]). This gets used up on 2 local buffers sized max_msg_len (64k by default), which causes a segfault due to a stack overflow in the error reporting path.

Set the stack size to 256k instead, which is enough for both buffers with an additional allowance for the remaining stack usage by netstress and called libc or ltp helper functions.

[1]: https://urldefense.com/v3/__https://wiki.musl-libc.org/functional-differences-from-glibc.html*Thread_stack_size__;Iw!!GqivPVa7Brio!Nat9ZPBR2P2QxOM2IsBWR2WdfOQ1ZNM4IJzjNy5a_bTsHUko9bULz88kJAVK8yQ7Kk8-$

Quoted 9 lines

---

v2:
- Use a static limit of 256k.
- Document the requested stack size in the error message.
- Coding style fixup.

Added extra parenthesis to silence possible compiler warnings and applied, thanks Johannes!