select() bug
I've uncovered a bug in select() when checking if it's okay to write on a pipe. It will report "false negatives" if there is any unread data already on the pipe, even a single byte. As soon as the pipe gets flushed, select() does the right thing.
Normally, I wouldn't think this such a big deal, except that Apache uses the writability of s pipe to determine if any of its children that are log file handlers are dead. If select() reports it can't write immediately, Apache terminates and restarts the child process, creating unnecessary load on the system.
The bug exists in the 2.2.x kernels up to and including 2.2.17 that I've tried and I don't know it extends beyond pipes. I've attached sample code to demonstrate the problem, which works correctly on BSD and Solaris.
Is this a know bug?
Signature · 4 lines
Paul Marquis [email protected] If it's tourist season, why can't we shoot them?
Quoted 3 lines
that are log file handlers are dead. If
select()reports it can't write immediately, Apache terminates and restarts the child process, creating unnecessary load on the system.
Is there anything saying that select has to report ready the instant a byte would fit. Certainly its better for performance to reduce the context switch rate by encouraging blocking
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/
Quoted 8 lines · Alan Cox
Quoted 3 lines
that are log file handlers are dead. If
select()reports it can't write immediately, Apache terminates and restarts the child process, creating unnecessary load on the system.Is there anything saying that select has to report ready the instant a byte would fit. Certainly its better for performance to reduce the context switch rate by encouraging blocking
It looks as though, when select() is reporting that the pipe is NOT writable, the code writes anyway -- then complains that it could be written.
This is a code bug. The pipe can certainly become writable during the time interval between when it was checked by select() and the attempt to write the byte.
Cheers,
Dick Johnson
Penguin : Linux version 2.2.17 on an i686 machine (801.18 BogoMips).
"Memory is like gasoline. You use it up when you are running. Of course you get it all back when you reboot..."; Actual explanation obtained from the Micro$oft help desk.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/
In the code sample, there is a loop that is run four times. During each iteration, a call to select() and write() is done, while every other iteration does a read(). Between the 1st and 2nd calls to write(), as well as the 3rd and 4th, select() fails, but all write()'s and read()'s succeed. There is no code bug.
Quoted 32 lines · Richard B. Johnson
Quoted 8 lines · Alan Cox
Quoted 3 lines
that are log file handlers are dead. If
select()reports it can't write immediately, Apache terminates and restarts the child process, creating unnecessary load on the system.Is there anything saying that select has to report ready the instant a byte would fit. Certainly its better for performance to reduce the context switch rate by encouraging blocking
It looks as though, when
select()is reporting that the pipe is NOT writable, the code writes anyway -- then complains that it could be written.This is a code bug. The pipe can certainly become writable during the time interval between when it was checked by
select()and the attempt to write the byte.Cheers,
Dick JohnsonPenguin : Linux version 2.2.17 on an i686 machine (801.18 BogoMips).
"Memory is like gasoline. You use it up when you are running. Of course you get it all back when you reboot..."; Actual explanation obtained from the Micro$oft help desk.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/
Signature · 8 lines
Paul Marquis [email protected] If it's tourist season, why can't we shoot them? - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected] Please read the FAQ at http://www.tux.org/lkml/
Quoted 5 lines · Paul Marquis
In the code sample, there is a loop that is run four times. During each iteration, a call to
select()andwrite()is done, while every other iteration does aread(). Between the 1st and 2nd calls towrite(), as well as the 3rd and 4th,select()fails, but all write()'s and read()'s succeed. There is no code bug.
Not as I see it. This is clearly a code bug. Look:
iteration 1
select says pipe is writable
write succeeded - 1 bytes written iteration 2
select returned 0 -- pipe not writable
for (i = 0; i < 4; i++)
{
printf("iteration %d\n", i + 1);
status = 0;
/* do select */
FD_ZERO(&write_fds);
FD_SET(fd[1], &write_fds);
tv.tv_sec = 0;
tv.tv_usec = 0;
switch (select(fd[1] + 1, NULL, &write_fds, NULL, &tv))
{
case -1:
/* should probably check for EINTR and/or EWOULDBLOCK */
printf("select error - %s\n", strerror(errno));
break;
case 0:
/* no I/O */
puts("select returned 0 -- pipe not writable");
break;
The BREAK breaks out of the switch-statement ---
|
|-----------------------------------------------------------
|
| default:
| /* make sure our fd is writable */
| if (FD_ISSET(fd[1], &write_fds))
| {
| puts("select says pipe is writable");
| status = 1;
| }
| else
| puts("select says pipe is not writable");
| break;
| }
|
| /* do write */
|-------> n = write(fd[1], buf + (i % 2), 1);Then you write it anyway!
write succeeded - 1 bytes written
select bug
Cheers,
Dick Johnson
Penguin : Linux version 2.2.17 on an i686 machine (801.18 BogoMips).
"Memory is like gasoline. You use it up when you are running. Of course you get it all back when you reboot..."; Actual explanation obtained from the Micro$oft help desk.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/
It shows that even though select() says a file descriptor is not writable, a write() can still succeed. This code is not used anywhere in the real world, or at least my real world :-P. It just demonstrates "the bug".
Quoted 53 lines · Richard B. Johnson
Not as I see it. This is clearly a code bug. Look:
iteration 1
select says pipe is writable
write succeeded - 1 bytes written iteration 2
select returned 0 -- pipe not writablefor (i = 0; i < 4; i++) { printf("iteration %d\n", i + 1); status = 0; /* do select */ FD_ZERO(&write_fds); FD_SET(fd[1], &write_fds); tv.tv_sec = 0; tv.tv_usec = 0; switch (select(fd[1] + 1, NULL, &write_fds, NULL, &tv)) { case -1: /* should probably check for EINTR and/or EWOULDBLOCK */ printf("select error - %s\n", strerror(errno)); break; case 0: /* no I/O */ puts("select returned 0 -- pipe not writable"); break; The BREAK breaks out of the switch-statement --- | |----------------------------------------------------------- | | default: | /* make sure our fd is writable */ | if (FD_ISSET(fd[1], &write_fds)) | { | puts("select says pipe is writable"); | status = 1; | } | else | puts("select says pipe is not writable"); | break; | } | | /* do write */ |-------> n = write(fd[1], buf + (i % 2), 1);Then you write it anyway!
write succeeded - 1 bytes written
select bug
Quoted 8 lines
Cheers,
Dick JohnsonPenguin : Linux version 2.2.17 on an i686 machine (801.18 BogoMips).
"Memory is like gasoline. You use it up when you are running. Of course you get it all back when you reboot..."; Actual explanation obtained from the Micro$oft help desk.
--
Paul Marquis
[email protected]
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/
Quoted 5 lines · Paul Marquis
It shows that even though
select()says a file descriptor is not writable, awrite()can still succeed. This code is not used anywhere in the real world, or at least my real world :-P. It just demonstrates "the bug".
It demonstrates nothing except bad code. Anybody should know that such a descriptor can become writable at any instant, simply because the kernel may be finding room for more unread data.
So, the code sees that, at this instant, it's not writable. Then it writes to it anyway. Sometimes the fd becomes writable between the time that it was checked, and the time it was written. So the write succeeded. So what. It shows nothing but bad code. It demonstrates no bug whatsover.
There may be a bug. However, this code doesn't demonstrate anything but bad coding practice.
Cheers,
Dick Johnson
Penguin : Linux version 2.2.17 on an i686 machine (801.18 BogoMips).
"Memory is like gasoline. You use it up when you are running. Of course you get it all back when you reboot..."; Actual explanation obtained from the Micro$oft help desk.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/
I guess in theory, you're right, though if a write() could succeed, shouldn't select() say that it would?
And this assumes you're calling select() with a timeout. In Apache, the caretaker process wakes up periodically and polls the pipe with a timeout of zero. If it gets back the pipe is not writable, it kills the process. With this false negative situation, this is a bad thing.
Quoted 8 lines · Alan Cox
Quoted 4 lines
that are log file handlers are dead. If
select()reports it can't write immediately, Apache terminates and restarts the child process, creating unnecessary load on the system.Is there anything saying that select has to report ready the instant a byte would fit. Certainly its better for performance to reduce the context switch rate by encouraging blocking
Signature · 8 lines
Paul Marquis [email protected] If it's tourist season, why can't we shoot them? - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected] Please read the FAQ at http://www.tux.org/lkml/
Quoted 2 lines
I guess in theory, you're right, though if a
write()could succeed, shouldn'tselect()say that it would?
Thats certainly not normal for a lot of devices. Most fast devices wake up when buffers are half empty for example.
Alan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/
I'm not exactly sure what you mean by this statement. Would you mind explaining further?
Quoted 1 line · Alan Cox
Most fast devices wake up when buffers are half empty for example.
Signature · 8 lines
Paul Marquis [email protected] If it's tourist season, why can't we shoot them? - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected] Please read the FAQ at http://www.tux.org/lkml/
Quoted 2 lines
I'm not exactly sure what you mean by this statement. Would you mind explaining further?
Well take a socket with 64K of buffering. You don't want to wake processes waiting in select or in write every time you can scribble another 1460 bytes to the buffer. Instead you wait until there is 32K of room then wake the user. That means that there is one wakeup/trip through userspace every 32K rather than potentially every time a byte is read the other end
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/
Okay, I see your point, thanks. A couple of comments/questions:
- Does this make sense with devices with small kernel buffers? From my experimentation, pipes on Linux have a 4K buffer and tend to be read and written very quickly.
- If I'm correct that pipes have a 4K kernel buffer, then writing 1 byte shouldn't cause this situation, as the buffer is well more than half empty. Is this still a bug?
Semantic issues aside, since Apache does the test I mentionned earlier to determine child status and since it could be misled, should this feature be turned off?
Thanks for your input.
Quoted 8 lines · Alan Cox
Quoted 2 lines
I'm not exactly sure what you mean by this statement. Would you mind explaining further?
Well take a socket with 64K of buffering. You don't want to wake processes waiting in select or in write every time you can scribble another 1460 bytes to the buffer. Instead you wait until there is 32K of room then wake the user. That means that there is one wakeup/trip through userspace every 32K rather than potentially every time a byte is read the other end
Signature · 8 lines
Paul Marquis [email protected] If it's tourist season, why can't we shoot them? - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected] Please read the FAQ at http://www.tux.org/lkml/
Quoted 3 lines
- Does this make sense with devices with small kernel buffers? From my experimentation, pipes on Linux have a 4K buffer and tend to be read and written very quickly.
It makes sense for all things I suspect
Quoted 3 lines
- If I'm correct that pipes have a 4K kernel buffer, then writing 1 byte shouldn't cause this situation, as the buffer is well more than half empty. Is this still a bug?
The pipe code uses totally full/empty. Im not sure why that was chosen
Quoted 3 lines
Semantic issues aside, since Apache does the test I mentionned earlier to determine child status and since it could be misled, should this feature be turned off?
Or made smarter yes
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/
Quoted 5 lines
Quoted 3 lines
Semantic issues aside, since Apache does the test I mentionned earlier to determine child status and since it could be misled, should this feature be turned off?
Or made smarter yes
i'm scratching my head wondering what i was thinking when i wrote that code. the specific thing the parent wants to handle there is the case of a stuck logger... you'd think i would have at least put some debouncing logic in there.
(the parent is able to replace a logger without disturbing the children because it keeps a copy of both halves of the logging pipes.)
an alternative would be to look for many children stuck in the logging state (they make a note in the scoreboard before going into it).
paul -- if you want to just eliminate that feature (it'll still be able to replace the logger if the logger exits) go into src/http_log.c, piped_log_maintenance and comment out the OC_REASON_UNWRITEABLE.
-dean
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/
Quoted 5 lines · Alan Cox
Quoted 3 lines
- If I'm correct that pipes have a 4K kernel buffer, then writing 1 byte shouldn't cause this situation, as the buffer is well more than half empty. Is this still a bug?
The pipe code uses totally full/empty. Im not sure why that was chosen
Just a quick guess: maybe because of the POSIX atomicity guarantees (if select returned, write might have to block which is not what is expected), and maybe this limitation was used not only on write but on read (Although it's not necessary on the read side, AFAIK).
Signature · 11 lines
-----==- |
----==-- _ |
---==---(_)__ __ ____ __ Marc Lehmann +--
--==---/ / _ \/ // /\ \/ / [email protected] |e|
-=====/_/_//_/\_,_/ /_/\_\ XX11-RIPE --+
The choice of a GNU generation |
|
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/Followup to: <[email protected]> By author: Paul Marquis <[email protected]> In newsgroup: linux.dev.kernel
Quoted 11 lines
Okay, I see your point, thanks. A couple of comments/questions:
- Does this make sense with devices with small kernel buffers? From my experimentation, pipes on Linux have a 4K buffer and tend to be read and written very quickly.
- If I'm correct that pipes have a 4K kernel buffer, then writing 1 byte shouldn't cause this situation, as the buffer is well more than half empty. Is this still a bug?
Quoted 6 lines
Semantic issues aside, since Apache does the test I mentionned earlier to determine child status and since it could be misled, should this feature be turned off?
Thanks for your input.
Has anyone considered the possibility of expanding the buffer of high-traffic pipes?
-hpa
Signature · 7 lines
<[email protected]> at work, <[email protected]> in private! "Unix gives you enough rope to shoot yourself in the foot." http://www.zytor.com/~hpa/puzzle.txt - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected] Please read the FAQ at http://www.tux.org/lkml/
Quoted 2 lines
Has anyone considered the possibility of expanding the buffer of high-traffic pipes?
Do that too much and the data falls out of L1 cache before we context switch. Its a rather complex juggling game. DaveM's kiovec pipe patches avoid some of this by cheating and going user->user
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/
From: "H. Peter Anvin" <[email protected]>
Date: 2 Nov 2000 15:53:29 -0800
Has anyone considered the possibility of expanding the buffer of high-traffic pipes?
The kiobuf pipe patches are a more effective performance improvement for this type of usage. It has the benefit of not requiring a temporary kernel buffer of any size :-)
Later,
David S. Miller
[email protected]
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/
Quoted 11 lines · David S. Miller
From: "H. Peter Anvin" <[email protected]>
Date: 2 Nov 2000 15:53:29 -0800Has anyone considered the possibility of expanding the buffer of high-traffic pipes?
The kiobuf pipe patches are a more effective performance improvement for this type of usage. It has the benefit of not requiring a temporary kernel buffer of any size :-)
That's (very) nice, but it does assume there is currently a reader listening.
-hpa
Signature · 7 lines
<[email protected]> at work, <[email protected]> in private! "Unix gives you enough rope to shoot yourself in the foot." http://www.zytor.com/~hpa/puzzle.txt - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected] Please read the FAQ at http://www.tux.org/lkml/
Date: Thu, 02 Nov 2000 16:04:05 -0800
From: "H. Peter Anvin" <[email protected]>
That's (very) nice, but it does assume there is currently a reader listening.
No, it has no such assumption.
Later,
David S. Miller
[email protected]
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/
Quoted 9 lines · David S. Miller
Date: Thu, 02 Nov 2000 16:04:05 -0800
From: "H. Peter Anvin" <[email protected]>That's (very) nice, but it does assume there is currently a reader listening.
No, it has no such assumption.
Oh. What do you do if there isn't... link up the contents of the write() in a kiovec and hold the writer?
-hpa
Signature · 7 lines
<[email protected]> at work, <[email protected]> in private! "Unix gives you enough rope to shoot yourself in the foot." http://www.zytor.com/~hpa/puzzle.txt - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected] Please read the FAQ at http://www.tux.org/lkml/
Date: Thu, 02 Nov 2000 16:13:13 -0800
From: "H. Peter Anvin" <[email protected]>
Oh. What do you do if there isn't... link up the contents of the write() in a kiovec and hold the writer?
Right, the writer blocks.
I've already posted the patches here within the past week, I'll send them to you under seperate cover so you can see for yourself how it works.
Later,
David S. Miller
[email protected]
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/
Quoted 4 lines · David S. Miller
Date: Thu, 02 Nov 2000 16:13:13 -0800
From: "H. Peter Anvin" <[email protected]>
Quoted 9 lines
Oh. What do you do if there isn't... link up the contents of the
write()in a kiovec and hold the writer?Right, the writer blocks.
I've already posted the patches here within the past week, I'll send them to you under seperate cover so you can see for yourself how it works.
Sure, but my point was that it would be nice for high-traffic pipes to allow a larger volume of data with the two processes still running concurrently.
-hpa
Signature · 7 lines
<[email protected]> at work, <[email protected]> in private! "Unix gives you enough rope to shoot yourself in the foot." http://www.zytor.com/~hpa/puzzle.txt - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected] Please read the FAQ at http://www.tux.org/lkml/