Avoid glibc lazy-unwind machinery in parallel teardown
Subsystems (2)
PATCH 1/1 Avoid glibc lazy-unwind machinery in parallel teardown
From: Maxi Saparov <[email protected]>
Worker threads end with pthread_exit(), and the consumer thread then pthread_cancel()s every worker even after a fully successful pass. Both make glibc lazily dlopen() libgcc_s.so.1 for stack unwinding (misc/unwind-link.c), and glibc aborts the process when that load fails:
libgcc_s.so.1 must be installed for pthread_exit to work
In a minimal kdump initramfs nothing has loaded libgcc_s.so.1 by the time the copy finishes, so the process-wide first load happens exactly when every worker exits and is cancelled concurrently at peak memory pressure in the capture kernel. Observed intermittently with 1.7.7, glibc 2.39 and --num-threads 2 on arm64 crash captures, aborting right after "Copying data" reached 100% with libgcc_s.so.1 present and loadable, leaving an otherwise complete dump marked incomplete.
This bug only affects dynamically linked builds, which is currently what most mainstream distributions ship (Debian / Ubuntu build with LINKTYPE=dynamic). Statically linked builds pull the unwinder (libgcc_eh.a) at build time so there is no concern with dlopen failing on teardown.
Avoid entering the unwind machinery at all on the success path:
- return from the worker function instead of calling pthread_exit(); the return value still reaches pthread_join(), but a plain return never enters glibc's unwinder,
- pthread_cancel() the workers only when bailing out on an error, where they may still be blocked producing pages. On success every worker is already exiting on its own, so joining is sufficient.
--- makedumpfile.c | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-)
makedumpfile.c +13 -7
@@ -8790,7 +8790,9 @@ fail:if (bitmap_memory_parallel.buf !=NULL )free (bitmap_memory_parallel.buf); -pthread_exit (retval); +/* Plain return: pthread_exit() would enter glibc's unwind path, +* which requires loading libgcc_s.so.1. */ +return retval; }int
@@ -8994,12 +8996,16 @@ finish: out:if (threads !=NULL ) { -for (i =0 ; i < info->num_threads; i++) { -if (threads[i] !=NULL ) { - res =pthread_cancel (*threads[i]); -if (res !=0 && res !=ESRCH ) -ERRMSG ("Can't cancel thread %d. %s\n" , - i,strerror (res)); +/* Cancel only on error; on success the workers are already +* exiting and joining is sufficient. */ +if (!ret) { +for (i =0 ; i < info->num_threads; i++) { +if (threads[i] !=NULL ) { + res =pthread_cancel (*threads[i]); +if (res !=0 && res !=ESRCH ) +ERRMSG ("Can't cancel thread %d. %s\n" , + i,strerror (res)); + } } }
Signature · 1 line
2.54.0
Quoted 26 lines · Maxi Saparov
From: Maxi Saparov <[email protected]>
Worker threads end with
pthread_exit(), and the consumer thread then pthread_cancel()s every worker even after a fully successful pass. Both make glibc lazilydlopen()libgcc_s.so.1 for stack unwinding (misc/unwind-link.c), and glibc aborts the process when that load fails:libgcc_s.so.1 must be installed for
pthread_exitto workIn a minimal kdump initramfs nothing has loaded libgcc_s.so.1 by the time the copy finishes, so the process-wide first load happens exactly when every worker exits and is cancelled concurrently at peak memory pressure in the capture kernel. Observed intermittently with 1.7.7, glibc 2.39 and --num-threads 2 on arm64 crash captures, aborting right after "Copying data" reached 100% with libgcc_s.so.1 present and loadable, leaving an otherwise complete dump marked incomplete.
This bug only affects dynamically linked builds, which is currently what most mainstream distributions ship (Debian / Ubuntu build with LINKTYPE=dynamic). Statically linked builds pull the unwinder (libgcc_eh.a) at build time so there is no concern with dlopen failing on teardown.
Avoid entering the unwind machinery at all on the success path:
Quoted 8 lines
- return from the worker function instead of calling
pthread_exit(); the return value still reachespthread_join(), but a plain return never enters glibc's unwinder,
-pthread_cancel()the workers only when bailing out on an error, where they may still be blocked producing pages. On success every worker is already exiting on its own, so joining is sufficient.Signed-off-by Maxi Saparov <[email protected]>
Thank you for the improvement, the patch looks good and tested ok. I tweaked only the comment style.
https://github.com/makedumpfile/makedumpfile/commit/fe9858fb7a59a786fb1c00ddc6f1e2a31031dde9
Thanks,
Kazu
Quoted 42 lines
--- makedumpfile.c | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-)makedumpfile.c +13
-7@@ -8790,7 +8790,9 @@ fail:if (bitmap_memory_parallel.buf !=NULL )free (bitmap_memory_parallel.buf); -pthread_exit (retval); +/* Plain return: pthread_exit() would enter glibc's unwind path, +* which requires loading libgcc_s.so.1. */ +return retval; }int @@ -8994,12 +8996,16 @@ finish: out:if (threads !=NULL ) { -for (i =0 ; i < info->num_threads; i++) { -if (threads[i] !=NULL ) { - res =pthread_cancel (*threads[i]); -if (res !=0 && res !=ESRCH ) -ERRMSG ("Can't cancel thread %d. %s\n" , - i,strerror (res)); +/* Cancel only on error; on success the workers are already +* exiting and joining is sufficient. */ +if (!ret) { +for (i =0 ; i < info->num_threads; i++) { +if (threads[i] !=NULL ) { + res =pthread_cancel (*threads[i]); +if (res !=0 && res !=ESRCH ) +ERRMSG ("Can't cancel thread %d. %s\n" , + i,strerror (res)); + } } }