Strange Linux behaviour with blocking syscalls and stop signals+SIGCONT
Gidday,
[Various parties involved in past discussions on this topic CCed.]
First off, it's worth mentioning that the topic that I'll go into below has been visited a few times before, and in particular during one of the more recent visits, Linus's position was:
http://marc.theaimsgroup.com/?l=linux-kernel&m=104502401330898&w=2
List: linux-kernel
Subject: Re: another subtle signals issue
From: Linus Torvalds
Date: 2003-02-12 4:21:02
...
And I have multiple times said that as far as Linux is concerned, ^Z is, has always been, and certainly for the 2.6.x timeframe _will_ be a signal that the kernel considers "caught".
The problem is that the "2.6.x timeframe" means something different now than it did then, and the question is when the following Linux-specific (mis)behaviour will ever be fixed.
==
Linux exhibits a unique behaviour among Unix implementations with respect to signals. When a program that is blocked in the middle of certain system calls is suspended by a signal (SIGSTSTP (^Z), SIGSTOP, SIGGTIIN, SIGTTOU) and then resumed by a SIGCONT signal, the system call fails with the error EINTR. ***This behaviour occurs even when no signal handler is installed for the stop or SIGCONT signals.*** (An example program showing the behaviour for one system call is appended to this message.)
This can cause applications that work on other Unix systems to behave unexpectedly on Linux.
On Linux, the current (2.6.17) system calls and functions that exhibit this behaviour are: futex(FUTEX_WAIT), epoll_wait(), poll(), read() from an inotify file descriptor (but not read()s from any other type of file descriptor, AFAIK), sem_wait(3) (because of futex(2)), semop(), semtimedop(), sigtimedwait(), and sigwaitinfo().
I have never seen this behaviour on any other Unix system, and at various times I've explicitly tested various blocking system calls on at least the following: Solaris 8, FreeBSD 4.8, HP-UX 11, Tru64 5.1B, Irix 6.5, and Darwin 7.2.
My reading of POSIX is that POSIX only permits a system call to fail with EINTR if a signal handler is involved (i.e., a signal is only considered "caught" if a handler is involved, a different definition of Linus's "caught" quoted at the start of this message). Some inquiries on the Austin group list quite a while back:
https://www.opengroup.org/sophocles/show_archive.tpl?source=L&listname=austin-group-l&first=1&pagesize=80&searchstring=signals+and+interruption+of+system+calls&zone=G
Or: http://tinyurl.com/l5at8
Date: Fri, 13 Feb 2004 11:44:50 +0100 (MET)
From: "Michael T Kerrisk"
To: The Austin Group
Subject: Stop signals and interruption of system calls on Linux
got answers that agreed with my interpretation, including:
https://www.opengroup.org/sophocles/show_mail.tpl?CALLER=show_archive.tpl&source=L&listname=austin-group-l&id=6668
Or: http://tinyurl.com/rdd7d
Quoting Paul Eggert:
This topic came up in a POSIX.1 standards meeting on April 22, 1993, and the consensus of that meeting also agreed with you.
Quoted 2 lines
(And of course I'm still curious if any other implementation behaves like Linux.)
Long-ago AIX hosts had that bug as well, but they were fixed after that POSIX.1 discussion of a decade ago. For more details about this, please see:
David A. Willcox (Motorola MCG - Urbana)
Job Control and POSIX
<http://groups.google.com/groups?selm=1r77ojINN85n%40ftp.UU.NET>
1993-04-22
==
Given the current development model, a fix now, in kernel 2.6.x seems in order. Probably all of the system calls listed above should be fixed so that in this circumstance the system call is automatically restarted, just as currently occurs for many other similar blocking system calls (e.g., select(), pselect(), mq_send(), mq_receive(), accep(), connect(), recv(), send(), wait(), flock(), fcntl(F_SETLKW), etc.).
In some past threads on this topic I have seen arguments about ABI compatability advanced, but I don't believe these hold, for the following reasons:
a) We are talking about signals involved in *interactive* job control; therefore any ABI issues are likely to be minimal (and see "c)" below).
b) There is no consistency about which system calls show this behaviour and which do not. For example poll() shows it, but select() does not. Notably, ppoll() also does not have the behaviour, since it does signal processing differently from poll()! Another notable example is read(): on an inotify file descriptor it demonstrates this behaviour, but on other file descriptors it does not.
c) The Linux baehviour has been arbitrary across kernel versions and system calls. In particular, the following system calls showed this behaviour in earlier kernel versions, but then the behaviour was changed without forewarning and (AFAIK) without subsequent complaint:
* nanosleep() in kernel 2.4 and earlier
* msgsnd() and msgrcv() in kernels before 2.6.9.
==
As far as I can see, there seems no compelling argument not to make Linux consistent with other current and historical Unix implementations with respect to the treatment of blocking syscalls and stop signals+SIGCONT. Is there any reason not to fix things now?
Cheers,
Michael
PS Just for completeness, I note that this topic has been visited before:
http://marc.theaimsgroup.com/?l=linux-kernel&m=94464821126712&w=2
List: linux-kernel Subject: SIGCONT misbehaviour in Linux From: Eric PAIRE <eric.paire () ri ! silicomp ! fr> Date: 1999-12-08 10:14:13
and
http://marc.theaimsgroup.com/?l=linux-kernel&m=104501574824496&w=2
List: linux-kernel Subject: another subtle signals issue From: Roland McGrath <roland () redhat ! com> Date: 2003-02-12 2:06:54
==
Below is an example program demonstrating the behaviour for semop(). This is a sample run:
$ ./a.out x <type ^Z> [1]+ Stopped ./r x $ fg ./a.out x semop: Interrupted system call
The last line of output shows that the system call failed with EINTR.
$ cat restart_semop.c /* restart_semop.c */
#define _GNU_SOURCE
#include <string.h>
#include <signal.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <sys/ipc.h>
#include <sys/sem.h>
#include <sys/wait.h>
#include <sys/types.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <errno.h>
#define errMsg(msg) do { perror(msg); } while (0)
#define errExit(msg) do { perror(msg); exit(EXIT_FAILURE); \
} while (0)static void
catcher(int sig)
{
char msg[100];
sprintf(msg, "Caught signal %d %s\n", sig, strsignal(sig));
fprintf(stderr, "%s", msg);
} /* catcher */intmain(int argc, char *argv[])
{
struct sigaction sa;
int semId;
struct sembuf sops[10];
sa.sa_handler = catcher;
sigemptyset(&sa.sa_mask);
/* Make system calls restartable if argc > 1 */
sa.sa_flags = (argc > 1) ? SA_RESTART : 0;
if (sigaction(SIGINT, &sa, NULL) == -1) errMsg("sigaction");
semId = semget(IPC_PRIVATE, 1, IPC_CREAT | S_IRUSR | S_IWUSR);
if (semId == -1) errExit("semget");
if (semctl(semId, 0, SETVAL, 1) == -1) errExit("semctl");
sops[0].sem_num = 0;
sops[0].sem_op = 0;
sops[0].sem_flg = 0;
for (;;)
if (semop(semId, sops, 1) == -1) errMsg("semop");
}Quoted 6 lines · Michael Kerrisk
c) The Linux baehviour has been arbitrary across kernel versions and system calls. In particular, the following system calls showed this behaviour in earlier kernel versions, but then the behaviour was changed without forewarning and (AFAIK) without subsequent complaint:
[snip]
*
msgsnd()andmsgrcv()in kernels before 2.6.9.
That was my change - and I even forgot to mention it in the changelog (hiding in shame):
I replaced -EINTR with -ERESTARTNOHAND.
That hides signals that are handled in the kernel from user space - probably what we want.
Michael: Could you replace the EINTR in inotify.c with ERESTARTNOHAND? That should prevent the kernel from showing the signal to user space.
I'd guess that most instances of EINTR are wrong, except in device drivers: It means we return from the syscall, even if the signal handler wants to restart the system call.
--
Manfred
Hi Manfred,
Quoted 7 lines
Quoted 5 lines · Michael Kerrisk
c) The Linux baehviour has been arbitrary across kernel versions and system calls. In particular, the following system calls showed this behaviour in earlier kernel versions, but then the behaviour was changed without forewarning and (AFAIK) without subsequent complaint:
Quoted 7 lines
Quoted 4 lines
[snip]
*
msgsnd()andmsgrcv()in kernels before 2.6.9.That was my change - and I even forgot to mention it in the changelog (hiding in shame):
I replaced-EINTRwith-ERESTARTNOHAND.
Well, that change was useful for my argument: the change appears to have affected no-one, and so why not make it also for futex(), sigtimedwait(), semop()/semtimedop(), inotify read(), epoll_wait()...
Quoted 2 lines
That hides signals that are handled in the kernel from user space - probably what we want.
Yes.
Quoted 5 lines
Michael: Could you replace the EINTR in
inotify.cwith ERESTARTNOHAND? That should prevent the kernel from showing the signal to user space. I'd guess that most instances of EINTR are wrong, except in device
drivers: It means we return from the syscall, even if the signal handler wants to restart the system call.
I'll try patching a kernel to s/EINTR/ERESTARTNOHAND/ in relevant places, and see how that goes. If it goes well, I'll submit a patch.
Cheers,
Michael
Signature · 8 lines
Michael Kerrisk maintainer of Linux man pages Sections 2, 3, 4, 5, and 7 Want to help with man page maintenance? Grab the latest tarball at ftp://ftp.win.tue.nl/pub/linux-local/manpages/, read the HOWTOHELP file and grep the source files for 'FIXME'.
Quoted 12 lines · Michael Kerrisk
Quoted 6 lines
Michael: Could you replace the EINTR in
inotify.cwith ERESTARTNOHAND? That should prevent the kernel from showing the signal to user space.
I'd guess that most instances of EINTR are wrong, except in device drivers: It means we return from the syscall, even if the signal handler wants to restart the system call.I'll try patching a kernel to s/EINTR/ERESTARTNOHAND/ in relevant places, and see how that goes. If it goes well, I'll submit a patch.
1) I would go further and try ERESTARTSYS: ERESTARTSYS means that the kernel signal handler honors SA_RESTART
2) At least for the futex functions, it won't be as easy as replacing EINTR wiht ERESTARTSYS: the futex functions receive a timeout a the parameter, with the duration of the wait call as a parameter. You must use ERESTART_RESTARTBLOCK.
--
Manfred
Quoted 6 lines · Manfred Spraul
1) I would go further and try ERESTARTSYS: ERESTARTSYS means that the kernel signal handler honors
SA_RESTART
2) At least for the futex functions, it won't be as easy as replacing EINTR wiht ERESTARTSYS: the futex functions receive a timeout a the parameter, with the duration of the wait call as a parameter. You must useERESTART_RESTARTBLOCK.
Whoa, not so fast. At least the futex syscall but be interruptible by signals. It is crucial to return EINTR.
Signature · 1 line
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
Quoted 12 lines · Ulrich Drepper
Quoted 8 lines · Manfred Spraul
1) I would go further and try ERESTARTSYS: ERESTARTSYS means that the kernel signal handler honors
SA_RESTART
2) At least for the futex functions, it won't be as easy as replacing EINTR wiht ERESTARTSYS: the futex functions receive a timeout a the parameter, with the duration of the wait call as a parameter. You must useERESTART_RESTARTBLOCK.
Quoted 5 lines
Whoa, not so fast. At least the futex syscall but be interruptible by signals. It is crucial to return EINTR.
Yes, of course.
ERESTARTSYS means honor SA_RESTART.
EINTR means return from the syscall, even if SA_RESTART is set in the signal handler.
Is it necessary that the futex syscall ignores SA_RESTART?
--
Manfred
Quoted 1 line · Manfred Spraul
Is it necessary that the futex syscall ignores
SA_RESTART?
You might break some incorrectly written code this wait. sem_wait(), one example, directly exposes the EINTR error to program. That behavior would change from what it is today, better or worse depends on the situation.
Signature · 1 line
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
Quoted 2 lines · Manfred Spraul
Is it necessary that the futex syscall ignores
SA_RESTART?
Very possibly. That was definitely the case for "select()" long ago.
Linus
Quoted 11 lines · Linus Torvalds
Quoted 3 lines · Manfred Spraul
Is it necessary that the futex syscall ignores
SA_RESTART?Very possibly. That was definitely the case for "
select()" long ago.
select uses ERESTARTSYS...
--
Manfred
Quoted 2 lines · Manfred Spraul
select uses ERESTARTSYS...
Sorry - I meant ERESTARTNOHAND
--
Manfred
Von: Linus Torvalds <[email protected]>
Quoted 5 lines · Manfred Spraul
Quoted 2 lines
Is it necessary that the futex syscall ignores
SA_RESTART?Very possibly. That was definitely the case for "
select()" long ago.
Why was this necesary back then?
Cheers,
Michael
Signature · 2 lines
Der GMX SmartSurfer hilft bis zu 70% Ihrer Onlinekosten zu sparen! Ideal für Modem und ISDN: http://www.gmx.net/de/go/smartsurfer
Quoted 8 lines · Michael Kerrisk
Von: Linus Torvalds <[email protected]>
Quoted 5 lines · Manfred Spraul
Quoted 2 lines
Is it necessary that the futex syscall ignores
SA_RESTART?Very possibly. That was definitely the case for "
select()" long ago.Why was this necesary back then?
I forget the exact details, but certain X applications would stop responding, because they would restart their select() system call in their main select-loop, rather than go through the loop.
I _think_ it was SIGALARM installed with SA_RESTART (and doing itimers every 10 ms), but I didn't really care enough - we had select() return ERESTARTSYS for exactly one day back in 199x, and when I figured out what broke, it got reverted pronto.
I want to say that the failing app was "oneko" (damn, I should try to find if it still exists - brings back memories of an earlier time), but the point being that it's enough that one app breaks.
Linus
Quoted 3 lines · Linus Torvalds
I want to say that the failing app was "oneko" (damn, I should try to find if it still exists - brings back memories of an earlier time)
Ahh. "yum install oneko" and there it is.
I feel calmer already.
Linus
I would venture to suggest a possible solution. But let's fist see what could be a reasoneable goal to achieve. Compatibility with existing applications is a must, so, solutions must not break it. Easy porting applications written for other Unixes is also a desirable feature. Being able to write correct programs that cope with stopping/ continuing is also a must.
The very last (correctness) cannot be achieved with the existing API:
suppose you are implementing a library, and you want to cope with stopping (i.e. make the process using the library continue properly after having been stopped). You cannot simply restart calls (the 16 ones that are interrupted by stop signals) because EINTR can be returned also when other signals have been catched. You cannot either rely on some flags set in a handler for SIGCONT because handlers might be already in place when your library functions are called (and you cannot change them since they are out of your control). A possible solution to consider is to have the kernel store the number of the signal that interrupted a call so that user code can decide the action that is most appropriate for the signal received. Since this adds to the existing API, it cannot break existing programs. People that want to port code from other Unixes can do that simply writing wrappers for the 16 syscalls that restart them when a stop signal is received. People that want to cope with stop signals in new code can restart them testing the signal that interrupted the calls (it would use a feature that is not in the POSIX standard, but it would work).
--
This message was sent on behalf of [email protected] at openSubscriber.com
http://www.opensubscriber.com/message/[email protected]/4393984.html
Von: Ulrich Drepper <[email protected]>
Quoted 10 lines
Quoted 6 lines · Manfred Spraul
1) I would go further and try ERESTARTSYS: ERESTARTSYS means that the kernel signal handler honors
SA_RESTART
2) At least for the futex functions, it won't be as easy as replacing EINTR wiht ERESTARTSYS: the futex functions receive a timeout a the parameter, with the duration of the wait call as a parameter. You must useERESTART_RESTARTBLOCK.Whoa, not so fast. At least the futex syscall but be interruptible by signals. It is crucial to return EINTR.
When you say "return" do you mean "in kernel", or "return to userspace"? My (possibly naive) understanding is that one could simply s/EINTR/ERESTARTNOHAND/ for FUTEX_WAIT, in order to achieve the change I want: for userland that ERESTARTNOHAND would be returned as EINTR if a signal handler interrupted the FUTEX_WAIT.
Cheers,
Michael
Signature · 2 lines
"Feel free" – 10 GB Mailbox, 100 FreeSMS/Monat ... Jetzt GMX TopMail testen: http://www.gmx.net/de/go/topmail
Quoted 9 lines · Michael Kerrisk
Quoted 2 lines
Whoa, not so fast. At least the futex syscall but be interruptible by signals. It is crucial to return EINTR.
When you say "return" do you mean "in kernel", or "return to userspace"? My (possibly naive) understanding is that one could simply s/EINTR/ERESTARTNOHAND/ for
FUTEX_WAIT, in order to achieve the change I want: for userland that ERESTARTNOHAND would be returned as EINTR if a signal handler interrupted theFUTEX_WAIT.
The futex syscall must _at least_ behave like every other syscall which can block and can be interrupted by a signal. It essential for internal operation (cancellation) but also for programs. For the former any change which still allows EINTR to be reported could be acceptable. We install the signal handler internally and so make sure it's correct.
sem_wait() is another case. Here the EINTR handling is exposed to the programmer. Currently, as I understand it, even SA_RESTART handlers cause EINTR to be returned. Yes, this usually correct but it might disrupt existing code.
This is why I'd caution anybody who thinks about changing something in this area. *I* could live with it, I can fix and recompile all the code I use. But others aren't that lucky.
Signature · 1 line
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
Von: Ulrich Drepper <[email protected]>
Quoted 3 lines
sem_wait()is another case. Here the EINTR handling is exposed to the programmer. Currently, as I understand it, evenSA_RESTARThandlers cause EINTR to be returned.
Yes, this is true for sem_wait().
Quoted 6 lines
Yes, this usually correct but it might
disrupt existing code.This is why I'd caution anybody who thinks about changing something in this area. *I* could live with it, I can fix and recompile all the code I use. But others aren't that lucky.
Yes; this is why I'm only proposing to change EINTR to ERESTARTNOHAND at the moment. The only userspace visible change that I think this will bring about is in the stop+SIGCONT case. Changing EINTR to ERESTARTSYS is likely to have more impact on userland (though it still strikes me as a desirable gola to have all system calls restartable via SA_RESTART).
Cheers,
Michael
Signature · 2 lines
"Feel free" – 10 GB Mailbox, 100 FreeSMS/Monat ... Jetzt GMX TopMail testen: http://www.gmx.net/de/go/topmail
Quoted 3 lines · Michael Kerrisk
Yes; this is why I'm only proposing to change EINTR to ERESTARTNOHAND at the moment. The only userspace visible change that I think this will bring about is in the stop+SIGCONT case.
That's fine.
Quoted 4 lines
Changing EINTR
to ERESTARTSYS is likely to have more impact on userland (though it still strikes me as a desirable gola to have all system calls restartable viaSA_RESTART).
This is certainly a nice goal but it changes the current ABI. Therefore it cannot be anything but an option and I don't assume we want to add so much cruft for just this.
Signature · 1 line
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
Von: Ulrich Drepper <[email protected]>
Quoted 6 lines
Quoted 4 lines
Changing EINTR
to ERESTARTSYS is likely to have more impact on userland (though it still strikes me as a desirable gola to have all system calls restartable viaSA_RESTART).This is certainly a nice goal but it changes the current ABI.
Yes, it does.
Quoted 3 lines
Therefore
it cannot be anything but an option and I don't assume we want to add so much cruft for just this.
There must be some framework for changing the kernel ABI over time. We can't remain forever stuck with an ABI behaviour because of the development model (i.e., no 2.7/2.8). And it probably would not be that much code change to achieve the result?
Cheers,
Michael
Signature · 2 lines
"Feel free" – 10 GB Mailbox, 100 FreeSMS/Monat ... Jetzt GMX TopMail testen: http://www.gmx.net/de/go/topmail
Quoted 4 lines
There must be some framework for changing the kernel ABI over time. We can't remain forever stuck with an ABI behaviour because of the development model (i.e., no 2.7/2.8).
Hi,
this has nothing to do with the development model. The userspace syscall ABI *has* to be stable. If we make a mistake that's a high price but we pay it. This fwiw is one of the reasons we are/should be very careful with adding system calls, and make sure the behavior is indeed right. It's also the reason we're not so happy about new ioctls; they're effectively mini-system calls with the same ABI issues, but just less controlled/reviewed/designed/visible.
Greetings,
Arjan van de Ven
Von: Arjan van de Ven <[email protected]>
Quoted 8 lines
Quoted 4 lines
There must be some framework for changing the kernel ABI over time. We can't remain forever stuck with an ABI behaviour because of the development model (i.e., no 2.7/2.8).
Hi,
this has nothing to do with the development model.
Doh! yes, thanks for pointing that out.
Quoted 7 lines
The userspace syscall
ABI *has* to be stable. If we make a mistake that's a high price but we pay it. This fwiw is one of the reasons we are/should be very careful with adding system calls, and make sure the behavior is indeed right. It's also the reason we're not so happy about new ioctls; they're effectively mini-system calls with the same ABI issues, but just less controlled/reviewed/designed/visible.
Yes.
There have been ABI changes in the past. In the end, I assume it's a question of relative desirability ("how broken is existing behaviour"; or: "was that behaviour ever desirable/portable anyway?") versus relative likelihood of breaking applications.
Cheers,
Michael
Signature · 2 lines
Echte DSL-Flatrate dauerhaft für 0,- Euro*! "Feel free" mit GMX DSL! http://www.gmx.net/de/go/dsl
Quoted 4 lines · Michael Kerrisk
There have been ABI changes in the past. In the end, I assume it's a question of relative desirability ("how broken is existing behaviour"; or: "was that behaviour ever desirable/portable anyway?") versus relative likelihood of breaking applications.
In futex(2) case (except FUTEX_LOCK_PI where we want it to be restartable), getting EINTR rather than the getting the syscall restarted is very desirable though and several NPTL routines rely on it.
Jakub
Quoted 3 lines · Jakub Jelinek
In futex(2) case (except
FUTEX_LOCK_PIwhere we want it to be restartable), getting EINTR rather than the getting the syscall restarted is very desirable though and several NPTL routines rely on it.
Jakub, you're slightly missing the point. Restartable means the only if SA_RESTART is set the syscall restarts. Otherwise it returns. That should always be the behavior if EINTR is an acceptable error.
The difference for the _PI operations is that be never need to see EINTR errors. I.e., the syscall should _always_ restart, regardless of SA_RESTART. That's not done using the error code but through explicit programming. I told Ingo that we don't need EINTR for those operations and I hope that part is still in.
Signature · 1 line
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
Von: Manfred Spraul <[email protected]>
Quoted 13 lines
Quoted 11 lines · Michael Kerrisk
Quoted 10 lines
Michael: Could you replace the EINTR in
inotify.cwith ERESTARTNOHAND? That should prevent the kernel from showing the signal to user space. I'd guess that most instances of EINTR are wrong, except in device
drivers: It means we return from the syscall, even if the signal handler
wants to restart the system call.
Quoted 8 lines
Quoted 6 lines
I'll try patching a kernel to s/EINTR/ERESTARTNOHAND/ in relevant places, and see how that goes. If it goes well, I'll submit a patch.
1) I would go further and try ERESTARTSYS: ERESTARTSYS means that the kernel signal handler honors
SA_RESTART
Yes, this is a separate but related issue: some system calls are not restarted by SA_RESTART. This set of system calls overlaps with, but is not quite the same as, the set of system calls that demonstrate the stop+SIGCONT ==> EINTR strangeness. I thought to tackle the latter problem first (since the fix seems easy), and then perhaps get onto the other one later (it is more likely to lead to visible ABI changes).
Cheers,
Michael
Signature · 2 lines
Der GMX SmartSurfer hilft bis zu 70% Ihrer Onlinekosten zu sparen! Ideal für Modem und ISDN: http://www.gmx.net/de/go/smartsurfer