Strange stop-signal behavior in multithreaded program with defunct main
Bert Wesarg described a scenario that I quickly replicated on 2.6.28-rc2 (and 2.6.25 -- it's not a regression in 2.6.28-rc) using the program below: if we have a multithreaded process with a defunct main thread running on a tty, and that process is sent a stop signal (either ^Z (SIGTSTP) or a stop signal sent from another terminal using kill(1)), then:
a) the terminal is locked up; and
b) the program is unresponsive to any other signal, except SIGKILL or SIGCONT.
An example run:
$ ./pthreads_zombie_main 1 # Creates one thread besides main 0: 0 0: 1 0: 2 ^Z
At this point, no shell prompt appears, and typing ^C (or ^\) has no effect. The process can be killed (and the terminal restored) by sending SIGKILL from another terminal. (If one instead types ^C at the terminal, and then sends SIGCONT from another terminal, then the terminal is restored and the program can be seen (via $?) to have terminated because of SIGINT.)
I'm (wildly) guessing that there is some problem in the terminal driver's understanding of the state and identify of the foreground job, but am not sure how to analyze this further. (I couldn't find a bug report or LKML thread that seemed to describe exactly this problem.) Ideas?
Cheers,
Michael
/* pthreads_zombie_main.c */
#include <pthread.h>
#include <string.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <errno.h>
#include <assert.h>
#define errExitEN(en, msg) { errno = en; perror(msg); \
exit(EXIT_FAILURE); }static void *thread_start(void *arg)
{
int tnum = (int) arg;
int j;
for (j = 0; ; j++) {
sleep(3);
printf("%d: %d\n", tnum, j);
}
}intmain(int argc, char *argv[])
{
int s, tnum;
pthread_t thr;
if (argc != 2) {
fprintf(stderr, "Usage: %s <num-threads>\n", argv[0]);
exit(EXIT_SUCCESS);
}
for (tnum = 0; tnum < atoi(argv[1]); tnum++) {
s = pthread_create(&thr, NULL, &thread_start, (void *) tnum);
if (s != 0)
errExitEN(s, "pthread_create");
}
pthread_exit(NULL);
}Quoted 8 lines · Michael Kerrisk
Bert Wesarg described a scenario that I quickly replicated on 2.6.28-rc2 (and 2.6.25 -- it's not a regression in 2.6.28-rc) using the program below: if we have a multithreaded process with a defunct main thread running on a tty, and that process is sent a stop signal (either ^Z (SIGTSTP) or a stop signal sent from another terminal using kill(1)), then:
Quoted 4 lines
a) the terminal is locked up; and
b) the program is unresponsive to any other signal, except SIGKILL or SIGCONT.
Yes, known problem. Please look at
[RFC,PATCH 3/3] do_wait: fix waiting for stopped group with dead leader http://marc.info/?t=119713920000003
Oleg.
Hi Oleg,
Quoted 11 lines · Oleg Nesterov
Quoted 10 lines · Michael Kerrisk
Bert Wesarg described a scenario that I quickly replicated on 2.6.28-rc2 (and 2.6.25 -- it's not a regression in 2.6.28-rc) using the program below: if we have a multithreaded process with a defunct main thread running on a tty, and that process is sent a stop signal (either ^Z (SIGTSTP) or a stop signal sent from another terminal using kill(1)), then:
a) the terminal is locked up; and
Quoted 7 lines
Quoted 2 lines
b) the program is unresponsive to any other signal, except SIGKILL or SIGCONT.
Yes, known problem. Please look at
[RFC,PATCH 3/3]
do_wait: fix waiting for stopped group with dead leader
http://marc.info/?t=119713920000003
Okay -- thanks for the info. I've added some text to man-pages to cover this bug.
Cheers,
Michael
man3/pthread_exit.3 +13 -1
@@ -21,7 +21,7 @@ .\" Formatted or processed versions of this manual, if unaccompanied by .\" the source, must acknowledge the copyright and authors of this work. .\" -.TH PTHREAD_EXIT 3 2008-10-24 "Linux" "Linux Programmer's Manual" +.TH PTHREAD_EXIT 3 2008-10-30 "Linux" "Linux Programmer's Manual" .SH NAME pthread_exit \- terminate calling thread .SH SYNOPSIS
@@ -87,6 +87,18 @@ The value pointed to by .IR retval should not be located on the calling thread's stack, since the contents of that stack are undefined after the thread terminates. +.SH BUGS +Currently, +.\" Linux 2.6.27 +there are limitations in the kernel implementation logic for +.BR wait (2)ing +on a stopped thread group with a dead thread group leader. +This can manifest in problems such as a locked terminal if a stop signal is +sent to a foreground process whose thread group leader has already called +.BR pthread_exit (3). +.\" FIXME . review a later kernel to see if this gets fixed +.\" http://thread.gmane.org/gmane.linux.kernel/611611 +.\" http://marc.info/?l=linux-kernel&m=122525468300823&w=2 .SH SEE ALSO .BR pthread_create (3), .BR pthread_join (3),
Quoted 16 lines · Michael Kerrisk
Quoted 14 lines · Oleg Nesterov
Quoted 12 lines · Michael Kerrisk
Bert Wesarg described a scenario that I quickly replicated on 2.6.28-rc2 (and 2.6.25 -- it's not a regression in 2.6.28-rc) using the program below: if we have a multithreaded process with a defunct main thread running on a tty, and that process is sent a stop signal (either ^Z (SIGTSTP) or a stop signal sent from another terminal using kill(1)), then:
a) the terminal is locked up; and
b) the program is unresponsive to any other signal, except SIGKILL or SIGCONT.
Quoted 7 lines
Quoted 4 lines
Yes, known problem. Please look at
[RFC,PATCH 3/3]
do_wait: fix waiting for stopped group with dead leader
http://marc.info/?t=119713920000003Okay -- thanks for the info. I've added some text to man-pages to cover this bug.
Well, we should fix this bug, of course.
I'll try to redo my old patch, but fyi I am very busy right now, and most probably I will be completely offline during the next week.
Oleg.