Skip to contentSkip to conversation

O_NONBLOCK setting "leak" outside of a process??

DISCUSSION Denis Vlasenko 7 participants 14 messages Jan 27, 2007 – Feb 5, 2007

lore
Share & tools
lore
#0 lore raw

Hi,

I am currently on Linux 2.6.18, x86_64.
I came across strange behavior while working on one of busybox applets. I narrowed it down to these two trivial testcases:

#include <unistd.h>
#include <fcntl.h>

int main() {
        fcntl(0, F_SETFL, fcntl(0, F_GETFL, 0) | O_NONBLOCK);
        return 0;
}

#include <unistd.h>
#include <fcntl.h>

int main() {
        fcntl(0, F_SETFL, fcntl(0, F_GETFL, 0) & ~O_NONBLOCK);
        return 0;
}

If I run "nonblock" in Midnight Commander in KDE's Konsole, screen redraw starts to work ~5 times slower. For example, Ctrl-O ("show/hide panels" in MC) takes ~0.5 sec to redraw. This persists after the program exist (which it does immediately as you see).
Running "block" reverts things to normal.

I mean: how can O_NONBLOCK _issued in a process which already exited_ have any effect whatsoever on MC or Konsole? They can't even know that it did it, right?

Either I do not know something subtle about Unix or some sort of bug is at work.

Any advice?
--
vda

#1 lore raw
Quoted 28 lines · Denis Vlasenko

Hi,

I am currently on Linux 2.6.18, x86_64.
I came across strange behavior while working on one of busybox applets. I narrowed it down to these two trivial testcases:

#include <unistd.h>
#include <fcntl.h>

int main() {
        fcntl(0, F_SETFL, fcntl(0, F_GETFL, 0) | O_NONBLOCK);
        return 0;
}

#include <unistd.h>
#include <fcntl.h>

int main() {
        fcntl(0, F_SETFL, fcntl(0, F_GETFL, 0) & ~O_NONBLOCK);
        return 0;
}

If I run "nonblock" in Midnight Commander in KDE's Konsole, screen redraw starts to work ~5 times slower. For example, Ctrl-O ("show/hide panels" in MC) takes ~0.5 sec to redraw. This persists after the program exist (which it does immediately as you see).
Running "block" reverts things to normal.

Quoted 6 lines

I mean: how can O_NONBLOCK _issued in a process which already exited_ have any effect whatsoever on MC or Konsole? They can't even know that it did it, right?

Either I do not know something subtle about Unix or some sort of bug is at work.

Because they all share the same stdin file descriptor, therefore they share the same file descriptor flags?

Phil.

#2 lore raw
Quoted 21 lines · Philippe Troin
Quoted 21 lines
int main() {
        fcntl(0, F_SETFL, fcntl(0, F_GETFL, 0) | O_NONBLOCK);
        return 0;
}
int main() {
        fcntl(0, F_SETFL, fcntl(0, F_GETFL, 0) & ~O_NONBLOCK);
        return 0;
}

If I run "nonblock" in Midnight Commander in KDE's Konsole, screen redraw starts to work ~5 times slower. For example, Ctrl-O ("show/hide panels" in MC) takes ~0.5 sec to redraw. This persists after the program exits (which it does immediately as you see).
Running "block" reverts things to normal.

I mean: how can O_NONBLOCK _issued in a process which already exited_ have any effect whatsoever on MC or Konsole? They can't even know that it did it, right?

Quoted 5 lines
Quoted 2 lines

Either I do not know something subtle about Unix or some sort of bug is at work.

Because they all share the same stdin file descriptor, therefore they share the same file descriptor flags?

What share the same file descriptor? MC and programs started from it?

I thought after exec() fds atre either closed (if CLOEXEC) or becoming independent from parent process
(i.e. it you seek, close, etc your fd, parent would not notice that).

Am I wrong?
--
vda

#3 lore raw
Quoted 1 line · Denis Vlasenko

What share the same file descriptor? MC and programs started from it?

All the processes started from your shell share at least fds 0, 1 and 2.

Quoted 5 lines

I thought after exec() fds atre either closed (if CLOEXEC) or becoming independent from parent process
(i.e. it you seek, close, etc your fd, parent would not notice that).

Am I wrong?

I'm afraid so. Seek position and flags are still shared after an exec.

Phil.

#4 lore raw

Hi Philippe!

Quoted 6 lines · Philippe Troin
Quoted 1 line · Denis Vlasenko

What share the same file descriptor? MC and programs started from it?

All the processes started from your shell share at least fds 0, 1 and 2.

Quoted 9 lines
Quoted 5 lines

I thought after exec() fds atre either closed (if CLOEXEC) or becoming independent from parent process
(i.e. it you seek, close, etc your fd, parent would not notice that).

Am I wrong?

I'm afraid so. Seek position and flags are still shared after an exec.

That's a bug, right? I couldn't find anything to that effect in IEEE Std. 1003.1, 2004 Edition...

Ciao,
Roland

--
TU Muenchen, Physik-Department E18, James-Franck-Str., 85748 Garching Telefon 089/289-12575; Telefax 089/289-12570
--
CERN office: 892-1-D23 phone: +41 22 7676540 mobile: +41 76 487 4482 --
Any society that would give up a little liberty to gain a little security will deserve neither and lose both. - Benjamin Franklin -----BEGIN GEEK CODE BLOCK-----
Version: 3.12
GS/CS/M/MU d-(++) s:+ a-> C+++ UL++++ P+++ L+++ E(+) W+ !N K- w--- M+ !V Y+ PGP++ t+(++) 5 R+ tv-- b+ DI++ e+++>++++ h---- y+++
------END GEEK CODE BLOCK------

#5 lore raw

2007/2/2, Roland Kuhn <[email protected]>:

Quoted 1 line

That's a bug, right?

No, if you want something like: (echo toto; date; echo titi) > file to work in your shell, you'll be happy to have the seek position shared in the processes.

Signature · 1 line
Guillaume
#6 lore raw

Hi Guillaume!

Quoted 8 lines · Guillaume Chazarain

2007/2/2, Roland Kuhn <[email protected]>:

Quoted 1 line

That's a bug, right?

No, if you want something like: (echo toto; date; echo titi) > file to work in your shell, you'll be happy to have the seek position shared in the processes.

As a naive user I'd probably expect that each of the above adds to the output, which perfectly fits the O_APPEND flag (to be set by the shell, of course).

The immediate point was about the flags, though, and having O_NONBLOCK on or off certainly is a _design_ choice when writing a program. If I remove O_NONBLOCK, I have a right to expect that I/O functions do not return EAGAIN!

Ciao,
Roland

--
TU Muenchen, Physik-Department E18, James-Franck-Str., 85748 Garching Telefon 089/289-12575; Telefax 089/289-12570
--
CERN office: 892-1-D23 phone: +41 22 7676540 mobile: +41 76 487 4482 --
Any society that would give up a little liberty to gain a little security will deserve neither and lose both. - Benjamin Franklin -----BEGIN GEEK CODE BLOCK-----
Version: 3.12
GS/CS/M/MU d-(++) s:+ a-> C+++ UL++++ P+++ L+++ E(+) W+ !N K- w--- M+ !V Y+ PGP++ t+(++) 5 R+ tv-- b+ DI++ e+++>++++ h---- y+++
------END GEEK CODE BLOCK------

#7 lore raw
Quoted 11 lines · Roland Kuhn

Hi Guillaume!

Quoted 7 lines · Guillaume Chazarain

2007/2/2, Roland Kuhn <[email protected]>:

Quoted 1 line

That's a bug, right?

No, if you want something like: (echo toto; date; echo titi) > file to work in your shell, you'll be happy to have the seek position shared in the processes.

Absolutely right. This has been part of Unix since the beginning.

Quoted 3 lines

As a naive user I'd probably expect that each of the above adds to the output, which perfectly fits the O_APPEND flag (to be set by the shell, of course).

No, no, O_APPEND has slightly different semantics.

Quoted 4 lines

The immediate point was about the flags, though, and having O_NONBLOCK on or off certainly is a _design_ choice when writing a program. If I remove O_NONBLOCK, I have a right to expect that I/O functions do not return EAGAIN!

Generally you don't want to mess with shared resouces like stdin, stdout and stderr.

Phil.

#8 lore raw

O_NONBLOCK supposedly hits the entire 'ofile' and not just the fd:
http://cr.yp.to/unix/nonblock.html

Signature · 2 lines
http://www.PowerDNS.com      Open source, database driven DNS Software 
http://netherlabs.nl              Open and Closed source services
#9 lore raw
In reply to Roland Kuhn
Quoted 6 lines

That's a bug, right? I couldn't find anything to that effect in IEEE Std. 1003.1, 2004 Edition...

Ciao,
Roland

It's not a bug, there's no rational alternative. What would two indepedent file descriptors for the same end of a TCP connection be? What happens when you call 'dup' on a file descriptor? The behavior is both the only logical behavior and consistent with other cases where a file descriptor is split/duplicated.

DS

#10 lore raw
Quoted 9 lines · David Schwartz
Quoted 6 lines

That's a bug, right? I couldn't find anything to that effect in IEEE Std. 1003.1, 2004 Edition...

Ciao,
Roland

It's not a bug, there's no rational alternative. What would two indepedent file descriptors for the same end of a TCP connection be?

Easy. O_NONBLOCK should only affect whether read/write blocks or returns EAGAIN. It's logical for this setting to be per-process.

Currently changing O_NONBLOCK on stdin/out/err affects other, possibly unrelated processes - they don't expect that *their* reads/writes will start returning EAGAIN!

Worse, it cannot be worked around by dup() because duped fds are still sharing O_NONBLOCK. How can I work around this? --
vda

#11 lore raw
Quoted 3 lines

Easy. O_NONBLOCK should only affect whether read/write blocks or returns EAGAIN. It's logical for this setting to be per-process.

Sadly, that's not what POSIX says. POSIX says that 'dup' and 'fork' create two references to the same file description and that O_NONBLOCK is a per-file-description flag. So such an implementation would not be POSIX-conforming.

Quoted 3 lines

Currently changing O_NONBLOCK on stdin/out/err affects other, possibly unrelated processes - they don't expect that *their* reads/writes will start returning EAGAIN!

Then they're broken. Sorry, that's just the way it is. Code should always correctly handle defined error codes. I agree that it's unexpected and unfortunate, but you have to code defensively.

*Every* blocking fd operation should be followed by a check to see if the operation failed, succeeded, or partially succeeded. If it partially succeeded, it needs to be continued. If it failed, you need to check if the error is fatal or transient. If transient, you need to back off and retry. It has, sadly, always been this way. (Programs can get signals, debuggers can interrupt a system call, the unexpected happens.)

Quoted 2 lines

Worse, it cannot be worked around by dup() because duped fds are still sharing O_NONBLOCK. How can I work around this?

If this causes your code a problem, your code is broken. What does your code currently do if it gets a non-fatal error from a blocking operation? If it does anything other than back off and retry, it's mishandling the condition.

It is also an error to change the modes on an inherited file descriptor unless you know for a fact that this is what the program that gave you the file descriptor expects you to do (or that it has relinquished that descriptor). So it takes two bugs to cause this problem.

I agree that the world might have been a better place had this been thought about from the beginning.

DS

#12 lore raw

David Schwartz wrote:
[]

Quoted 8 lines
Quoted 3 lines

Currently changing O_NONBLOCK on stdin/out/err affects other, possibly unrelated processes - they don't expect that *their* reads/writes will start returning EAGAIN!

Then they're broken. Sorry, that's just the way it is. Code should always correctly handle defined error codes. I agree that it's unexpected and unfortunate, but you have to code defensively.

Quoted 6 lines

*Every* blocking fd operation should be followed by a check to see if the operation failed, succeeded, or partially succeeded. If it partially succeeded, it needs to be continued. If it failed, you need to check if the error is fatal or transient. If transient, you need to back off and retry. It has, sadly, always been this way. (Programs can get signals, debuggers can interrupt a system call, the unexpected happens.)

Well, that's partly nonsense. The only error condition which is always being checked in correctly written software is EINTR - if you've got an interrupt, continue/retry the I/O.

Checking and retrying for EAGAIN is umm.. plain wrong. You'll get a nice busywait eating 100% CPU this way, till the I/O actually happens, and will get another the next try.

Checking I/Os for every possible weird condition is just non-productive.

It's like this:

  if (fcntl(fd, F_SETFL, ~O_NONBLOCK) < 0)  error_out();
  if (fcntl(fd, F_GETFL, 0) & O_NOBLOCK) ??? what to do?
  while(do_something())
    if (fcntl(fd, F_GETFL, 0) & O_NOBLOCK)
      if (fcntl(fd, F_SETFL, ~O_NONBLOCK) < 0)  error_out();

(don't pay attention to ~O_NONBLOCK thing - it's wrong, but it's used like that just to show the "idea" - which is to clear O_NONBLOCK)

Which is a complete nonsense. It's either set or cleared, and once set or cleared it should stay that way, period. Until the app changes it again.

Quoted 4 lines
Quoted 2 lines

Worse, it cannot be worked around by dup() because duped fds are still sharing O_NONBLOCK. How can I work around this?

If this causes your code a problem, your code is broken. What does your code

With dup() - maybe. But definitely NOT with fork().

Quoted 2 lines

currently do if it gets a non-fatal error from a blocking operation? If it does anything other than back off and retry, it's mishandling the condition.

Retrying I/O in case of EAGAIN is *wrong*. See above.
But sure, in case of dup() an app should be prepared to set up all the flags properly.

/mjt

Quoted 2 lines

I agree that the world might have been a better place had this been thought about from the beginning.

#13 lore raw
Quoted 18 lines
Quoted 11 lines

*Every* blocking fd operation should be followed by a check to see if the
operation failed, succeeded, or partially succeeded. If it partially succeeded, it needs to be continued. If it failed, you need to check if the
error is fatal or transient. If transient, you need to back off and retry.
It has, sadly, always been this way. (Programs can get signals, debuggers
can interrupt a system call, the unexpected happens.)

Well, that's partly nonsense. The only error condition which is always being
checked in correctly written software is EINTR - if you've got an interrupt,
continue/retry the I/O.

Quoted 3 lines

Checking and retrying for EAGAIN is umm.. plain wrong. You'll get a nice busywait eating 100% CPU this way, till the I/O actually happens, and will get another the next try.

I said back off and retry.

Quoted 4 lines

Checking I/Os for every possible weird condition is just non-productive.

It's like this:

Quoted 8 lines
  if (fcntl(fd, F_SETFL, ~O_NONBLOCK) < 0)  error_out();
  if (fcntl(fd, F_GETFL, 0) & O_NOBLOCK) ??? what to do?
  while(do_something())
    if (fcntl(fd, F_GETFL, 0) & O_NOBLOCK)
      if (fcntl(fd, F_SETFL, ~O_NONBLOCK) < 0)  error_out();

(don't pay attention to ~O_NONBLOCK thing - it's wrong, but it's used like that just to show the "idea" - which is to clear O_NONBLOCK)

I agree that checking for a condition that there's no sane way to handle is non-productive. But here we're talking about testing for a condition that has been proven to happen and for which a sane way to handle it is obvious -- back off and retry.

Quoted 3 lines

Which is a complete nonsense. It's either set or cleared, and once set or cleared it should stay that way, period. Until the app changes it again.

Until anything with access to it changes it.

Quoted 7 lines
Quoted 5 lines
Quoted 2 lines

Worse, it cannot be worked around by dup() because duped fds are still sharing O_NONBLOCK. How can I work around this?

If this causes your code a problem, your code is broken. What does your code

With dup() - maybe. But definitely NOT with fork().

With 'fork', you either give the other process the file descriptor or you share it. Any shared resource requires cooperation for sane results.

Quoted 6 lines
Quoted 4 lines

currently do if it gets a non-fatal error from a blocking operation? If it
does anything other than back off and retry, it's mishandling the condition.

Retrying I/O in case of EAGAIN is *wrong*. See above.

You missed the "back off" part.

DS