Process with many NPTL threads terminates slowly on core dump signal
Quoted 7 lines · Michael Kerrisk
Following up (belatedly) from my earlier message, I took Daniel Jacobowitz's suggestion to investigate the result from booting with "profile=2". When running my program (shwon below) on 2.6.14-rc4 to create 100 threads, and sending a core dump signal, the program takes 90 seconds to terminate, and readprofile shows the following:
I think the coredumping code in __group_complete_signal() is bogus and what happens is:
group_send_sig_info(P, SIGQUIT):
adds SIGQUIT to ->shared_pending
__group_complete_signal:
->signal->group_exit_task = P;
for_each_thread(t) {
P->signal->group_stop_count ++;
// sets TIF_SIGPENDING
signal_wake_up(t)
}Now, P receives the signal:
get_signal_to_deliver:
if (->signal->group_stop_count > 0) // YES
handle_group_stop():
if (->signal->group_exit_task == current) { // YES
->signal->group_exit_task = NULL
return 0;
}
signr = dequeue_signal(); // SIGQUIT
do_coredump:
->signal->flags = SIGNAL_GROUP_EXIT;
coredump_wait:
yield();Now all other threads do:
get_signal_to_deliver:
if (->signal->group_stop_count > 0) // YES handle_group_stop(); if (->signal->flags & SIGNAL_GROUP_EXIT) // YES return 0; signr = dequeue_signal(); // no pending signals // recalc_sigpending_tsk() DOES NOT clear TIF_SIGPENDING, // because it sees ->group_stop_count != 0. return 0;
TIF_SIGPENDING is not cleared, so get_signal_to_deliver() will be called again on return to userspace. When all threads will eat their ->time_slice, P will return from yield() and kill all threads.
Could you try this patch (added to mm tree):
http://marc.theaimsgroup.com/?l=linux-kernel&m=112887453531139
? It does not solve the whole problem, but may help.
Please report the result, if possible.
Oleg.
Quoted 14 lines
Von: Oleg Nesterov <[email protected]>
Datum: Wed, 12 Oct 2005 21:33:46 +0400Quoted 7 lines · Michael Kerrisk
Following up (belatedly) from my earlier message, I took Daniel Jacobowitz's suggestion to investigate the result from booting with "profile=2". When running my program (shwon below) on 2.6.14-rc4 to create 100 threads, and sending a core dump signal, the program takes 90 seconds to terminate, and readprofile shows the following:
I think the coredumping code in
__group_complete_signal()is bogus and what happens is:
[...]
Quoted 3 lines
TIF_SIGPENDINGis not cleared, soget_signal_to_deliver()will be called again on return to userspace. When all threads will eat their ->time_slice, P will return fromyield()and kill all threads.
Thanks for investiagting this further.
Quoted 5 lines
Could you try this patch (added to mm tree):
http://marc.theaimsgroup.com/?l=linux-kernel&m=112887453531139? It does not solve the whole problem, but may help.
Please report the result, if possible.
Thanks. I've applied it to 2.6.14-rc4: this patch does fix the specific behaviour that my program demonstrates.
What remains to be solved?
Cheers,
Michael
Signature · 2 lines
10 GB Mailbox, 100 FreeSMS/Monat http://www.gmx.net/de/go/topmail +++ GMX - die erste Adresse für Mail, Message, More +++
Quoted 5 lines · Michael Kerrisk
Thanks. I've applied it to 2.6.14-rc4: this patch does fix the specific behaviour that my program demonstrates.
What remains to be solved?
The problem remains that all threads spin with TIF_SIGPENDING flag after the coredumping thread sets SIGNAL_GROUP_EXIT in do_coredump() and before it kills them in zap_threads(). If one of these threads is SCHED_FIFO we have a deadlock. Also, the coredumping thread could be preempted, placed in ->expired array ...
Oleg.