PROBLEM: select() says closed socket readable
Subsystems (3)
For linux 2.4.2, select() indicates socket ready for read on a socket that's never been connected. This is inconsistent with other versions of Unix including Linux 2.2.
The following program demonstrates:
#include <stdio.h>
#include <unistd.h>
#include <sys/time.h>
#include <sys/types.h>
#include <sys/socket.h>
main(int argc, char *argv[])
{
fd_set rfds;
int retval;
int sock;
struct timeval tv;
sock = socket(AF_INET, SOCK_STREAM, 0);
if (sock == -1) abort();
FD_ZERO(&rfds);
FD_SET(sock, &rfds);
tv.tv_sec = 0;
tv.tv_usec = 0;
retval = select(sock + 1, &rfds, NULL, NULL, &tv);
printf("retval from select(): %d\n", retval);
exit(0);
} /* end main program */Output from "ver_linux" follows:
Linux costarica 2.4.2-2 #1 Sun Apr 8 20:41:30 EDT 2001 i686 unknown
Gnu C 2.96 Gnu make 3.79.1 binutils 2.10.91.0.2 util-linux 2.10r modutils 2.4.2 e2fsprogs 1.19 reiserfsprogs 3.x.0f pcmcia-cs 3.1.22 PPP 2.4.0 Linux C Library 2.2.2 Dynamic linker (ldd) 2.2.2 Procps 2.0.7 Net-tools 1.57 Console-tools 0.3.3 Sh-utils 2.0 Modules Loaded nfs lockd sunrpc nls_iso8859-1 ide-cd cdrom autofs 3c59x ipchains usb-uhci usbcore
Hello!
Quoted 2 lines
For linux 2.4.2,
select()indicates socket ready for read on a socket that's never been connected.
Right. It does not block on read, hence it is readable.
Quoted 1 line
This is inconsistent
This is perfectly consistent. Reaction to bugs in applications is undefined.
Alexey
In article <[email protected]>,
Quoted 7 lines · [email protected] writes:
Hello!
Quoted 2 lines
For linux 2.4.2,
select()indicates socket ready for read on a socket that's never been connected.Right. It does not block on read, hence it is readable.
mm, that's not really the rule, since e.g. you want to be able to select on read and write on the write side of a pipe, and would expect to only get readable on error/close:
But there's weird stuff going on here (examples need perl >= 5.6.0):
perl -wle 'use POSIX; pipe(my $rd, my $wr)||die $!; my $fd = fileno($wr); my $mask=""; vec($mask, $fd, 1)=1; select(my $r=$mask, my $w=$mask, "", undef); print "readable" if vec($r, $fd, 1); print "writable" if vec($w, $fd, 1); my $rc = POSIX::read($fd, my $buf, 4096) || die "read returns $!"; print STDERR $rc'
prints:
writable
read returns Bad file descriptor at -e line 1.
Which is in fact weird, since I still have the $rd descriptor around, so the read should block (it does on solaris). This seems a bug.
perl -wle 'use POSIX; pipe(my $rd, my $wr)||die $!; close $rd; my $fd = fileno($wr); my $mask=""; vec($mask, $fd, 1)=1; select(my $r=$mask, my $w=$mask, "", undef); print "readable" if vec($r, $fd, 1); print "writable" if vec($w, $fd, 1); my $rc = POSIX::read($fd, my $buf, 4096) || die "read returns $!"; print STDERR $rc'
prints:
readable
writable
read returns Bad file descriptor at -e line 1.
Which is the behaviour you want (the read does not block in either case), closing the other side causes readability. This allows you to watch a connection for errors without being bothered by EOF (so it's good, but contrary to your statement).
However, the reaction on read is still weird. on solaris I get in fact EOF, not Bad file descriptor
In fact, somewhat related, i last week ran into this:
perl -wle 'use Socket; use POSIX; socketpair(my $rd, my $wr, AF_UNIX, SOCK_STREAM, PF_UNSPEC)||die $!; shutdown($rd, 1); shutdown($wr, 0); my $fd = fileno($wr); my $mask=""; vec($mask, $fd, 1)=1; select(my $r=$mask, my $w=$mask, "", undef); print "readable" if vec($r, $fd, 1); print "writable" if vec($w, $fd, 1); my $rc = POSIX::read($fd, my $buf, 4096) || die "read returns $!"; print STDERR $rc'
which prints:
readable
writable
0 but true
(so the read returns EOF). Which is very unfortunate, because it means a pipe is not equivalent to a socketpair where you make one side only readable and one only writable (the way it's supposedly implemented on some OSes) Solaris and freebsd behaves the same as linux by the way. On the other hand, you *do* want to know if the other side does a shutdown, so in that sense the behaviour is good.
Damn, i wish this socket stuff would finally get completely standardized.
Quoted 5 lines
From: [email protected]
Quoted 2 lines
For linux 2.4.2,
select()indicates socket ready for read on a socket that's never been connected.Right. It does not block on read, hence it is readable.
No, a socket that's never been connected isn't readable, hence select() shouldn't be returning a value of 1 on it.
Quoted 4 lines
Quoted 1 line
This is inconsistent
This is perfectly consistent. Reaction to bugs in applications is undefined.
No this behavior isn't consistent with previous versions of Linux nor is it consistent with other implementations of Unix.
From: Jay Rogers <[email protected]>
Date: Mon, 20 Aug 2001 10:34:09 -0400
Quoted 1 line
Right. It does not block on read, hence it is readable.
No, a socket that's never been connected isn't readable, hence select() shouldn't be returning a value of 1 on it.
You may read without blocking, select() returns 1.
Please, fix your app.
Later,
David S. Miller
[email protected]
Quoted 3 lines
No, a socket that's never been connected isn't readable, hence
select()shouldn't be returning a value of 1 on it.
It is readable. A read that produces an error is still a read.
DS
Quoted 7 lines · David S. Miller
No, a socket that's never been connected isn't readable, hence
select()shouldn't be returning a value of 1 on it.You may read without blocking,
select()returns 1.Please, fix your app.
Hello,
While we're at it - are there any plans to fix the other select issue which was already discussed in December 2000? The original thread can be found at:
http://uwsg.iu.edu/hypermail/linux/net/0012.2/0008.html
I realize that the behaviour doesn't violate the standards, but as Alexey said - it's still somewhat wrong.
-Udo.
Quoted 9 lines · David S. Miller
From: Jay Rogers <[email protected]>
Date: Mon, 20 Aug 2001 10:34:09 -0400Quoted 1 line
Right. It does not block on read, hence it is readable.
No, a socket that's never been connected isn't readable, hence
select()shouldn't be returning a value of 1 on it.You may read without blocking,
select()returns 1.
By this logic a socket that is set non-blocking should always be treated as readable. I think we can all agree that argument is flawed :-).
The prevailing view from other systems appears to be that reading from an unconnected (or unconnectING) socket is meaningless so the socket is not readable.
Presumably there is a damn good reason, or a standards reference, why that is the wrong behaviour and should be changed?
Mike
Quoted 7 lines
Quoted 5 lines · David S. Miller
From: Jay Rogers <[email protected]>
Date: Mon, 20 Aug 2001 10:34:09 -0400Quoted 1 line
Right. It does not block on read, hence it is readable.
Quoted 8 lines
Quoted 4 lines
No, a socket that's never been connected isn't readable, hence
select()shouldn't be returning a value of 1 on it.You may read without blocking,
select()returns 1.By this logic a socket that is set non-blocking should always be treated as readable. I think we can all agree that argument is flawed :-).
No, because 'select' is defined to work the same on both blocking and
non-blocking sockets. Roughly, select should hit on read if a non-blocking read wouldn't return 'would block'.
Quoted 3 lines
The prevailing view from other systems appears to be that reading from an unconnected (or unconnectING) socket is meaningless so the socket is not readable.
So is reading from a closed or errored socket. There is nothing to wait
for, so why should the system call wait?
Quoted 2 lines
Presumably there is a damn good reason, or a standards reference, why that is the wrong behaviour and should be changed?
If you think about the cases where someone might actually do this, odds are
you want the application's error handling code to launch. That won't happen if you never break out of select. However, if you do break out of select, do a read, and get an error, the problem will then be handled, rather than ignored.
DS
Quoted 3 lines
No, because 'select' is defined to work the same on both blocking and
non-blocking sockets. Roughly, select should hit on read if a non-blocking read wouldn't return 'would block'.
Select is not reliable for a blocking socket in all cases. There is always a risk select may return "data to read" and the read will find there is now none. It isnt going to bite anyone on Linux with our current protocols but it may bite portable code
Quoted 7 lines
Quoted 6 lines
No, because 'select' is defined to work the same on both blocking and
non-blocking sockets. Roughly, select should hit on read if a non-blocking
read wouldn't return 'would block'.
Quoted 6 lines
Select is not reliable for a blocking socket in all cases. There is always
a risk select may return "data to read" and the read will find there is now
none. It isnt going to bite anyone on Linux with our current protocols but it may bite portable code
I should have continued my sentence with "if it was issued at the instant
'select' made that decision." Using 'select' on blocking sockets is usually an error. Nevertheless, select itself is defined to work the same on both blocking and non-blocking sockets. In general, there is no way the operating system can make guarantees about the future state of a socket.
DS
Quoted 3 lines · David Schwartz
No, because 'select' is defined to work the same on both blocking and
non-blocking sockets. Roughly, select should hit on read if a non-blocking read wouldn't return 'would block'.
Which may be the root of the problem. Linux seems to consider that select indicates "would not block", whereas Solaris et. al. use "ready to read". (The difference seems to exist in man pages too) Clearly an unconnected socket "would not block" but is not "ready to read".
Quoted 5 lines
If you think about the cases where someone might actually do this, odds are
you want the application's error handling code to launch. That won't happen if you never break out of select. However, if you do break out of select, do a read, and get an error, the problem will then be handled, rather than ignored.
No. If there is a correct behaviour defined in a standard we should do that. Otherwise we should do what other systems do _unless_ there is a clear benefit to doing something else. In this case doing something else appears to create porting problems and confusion over what select(2) means without any clear benefit.
So, we're back to the beginning: justify with reference or reason :-).
Mike
Quoted 8 lines
No. If there is a correct behaviour defined in a standard we should do that. Otherwise we should do what other systems do _unless_ there is a clear benefit to doing something else. In this case doing something else appears to create porting problems and confusion over what select(2) means without any clear benefit.
Mike
There is a clear and significant benefit. Bugs that result in a program
calling 'select' on an unconnected socket will be easily and quickly detected. During debugging, they can then be fixed. During release execution, they can be worked around.
There are a large number of possible mistakes that can result in this
behavior. A program that heavily uses sockets could sometimes forget to remove a socket from its active poll/select set. A program might accidentally close the wrong socket (and that socket might get reused by a subsequent call to 'socket'). It's nice to have a way to catch these. During debug, socket errors are routinely logged or displayed, so this would get caught.
Of course, this isn't so much of a benefit that it's worth violating a
standard like POSIX. But it could be considered enough of a benefit that it's worth not being compatable outside the bounds of such a standard.
DS
David Schwartz wrote:
[assumptions deleted]
Quoted 3 lines
Of course, this isn't so much of a benefit that it's worth violating a
standard like POSIX. But it could be considered enough of a benefit that it's worth not being compatable outside the bounds of such a standard.
Let's think about the _facts_ for a second. In the case where select on an unconnected socket succeeds (because a read would not block) a process that does this would not sleep in select. If the select does not succeed (because the socket is not ready to read) then the process sleeps.
In the first case the process either eats CPU time or gets an error when it reads the socket. But does it handle the error? Does it even do the read - why should it read the unconnected socket at all?
In the second case the process either hangs or behaves as expected. It may even recover gracefully if the socket is subsequently connected.
Case 1 leads to spin-or-die and case 2 leads to block-and-work.
Case 1 assumes that selecting for read on an unconnected socket is an error. Case 2 doesn't - an unconnected socket is simply never "ready to read".
Other systems use the "ready to read" interpretation. The Single UNIX Spec states "ready to read". The Linux man page uses "ready to read" but adds a parenthetical "read will not block" to allow EOF. Linux itself implements "will not block".
I can find nothing that even suggests that selecting on an unconnected socket is an error.
Linux is wrong. Patch follows.
Mike
PATCH 1/1 select() says closed socket readable
In reply to David SchwartzWhoa! It isn't that select/poll is saying that a read wouldn't block. It's saying that the socket has been hung up (POLLHUP not POLLIN)! This is even worse - POLLHUP is always tested for even of the user didn't ask for it. So if an unconnected socket appears _anywhere_ in a select or poll we are heading for spin city, regardless of whether we asked for read, write, errors or even nothing (in the case of poll)!
Older Linux kernels (2.2.18 at least) blocked on unconnected TCP sockets but returned POLLHUP on unix sockets. Current kernels return POLLHUP in both cases. It looks like TCP was fixed to match unix instead of vice-versa.
A quick poll (<grin>) of available machines here show that both Sun and FreeBSD block for both TCP and unix unconnected sockets.
The attached patch _should_ be applied :-).
Mike
net/ipv4/tcp.c +1 -4
@@ -415,11 +415,8 @@* then we could set it on SND_SHUTDOWN. BTW examples given * in Stevens' books assume exactly this behaviour, it explains * why PULLHUP is incompatible with POLLOUT. --ANK -* -* NOTE. Check for TCP_CLOSE is added. The goal is to prevent -* blocking on fresh not-connected or disconnected socket. --ANK */ -if (sk->shutdown ==SHUTDOWN_MASK || sk->state ==TCP_CLOSE ) +if (sk->shutdown ==SHUTDOWN_MASK ) mask |=POLLHUP ;if (sk->shutdown &RCV_SHUTDOWN ) mask |=POLLIN |POLLRDNORM ;
net/unix/af_unix.c +2 -2
@@ -1711,8 +1711,8 @@if (!skb_queue_empty (&sk->receive_queue) || (sk->shutdown&RCV_SHUTDOWN )) mask |=POLLIN |POLLRDNORM ; -/* Connection-based need to check for termination and startup */ -if (sk->type ==SOCK_STREAM && sk->state==TCP_CLOSE ) +/* Connection-based need to check for termination */ +if (sk->type ==SOCK_STREAM ) mask |=POLLHUP ;/*