dup2() vs dup3() inconsistency when
Al,
your 2008-07-26 commit 6c5d0512a091480c9f981162227fdb1c9d70e555 for fs/fcntl.c:sys_dup3 contains
+ if (unlikely(oldfd == newfd))
+ return -EINVAL;
This makes dup2() and dup3() differ with respect to the oldfd==newfd case (dup2() becomes a no-op, just returning oldfd, dup3() gives the EINVAL error.
Your commit log doesn't explain the rationale for this change. What is it? (I could guess that it is to error on a case that may be a user programming error, but I want to check this with you, so that I add the right text to the man page.)
Cheers,
Michael
Signature · 5 lines
Michael Kerrisk Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/ git://git.kernel.org/pub/scm/docs/man-pages/man-pages.git man-pages online: http://www.kernel.org/doc/man-pages/online_pages.html Found a bug? http://www.kernel.org/doc/man-pages/reporting_bugs.html
Quoted 8 lines · Michael Kerrisk
Al,
your 2008-07-26 commit
6c5d0512a091480c9f981162227fdb1c9d70e555for fs/fcntl.c:sys_dup3 contains(quoted diff) +2
-0+ if (unlikely(oldfd == newfd)) + return -EINVAL;
Quoted 9 lines
This makes
dup2()anddup3()differ with respect to the oldfd==newfd case (dup2()becomes a no-op, just returning oldfd,dup3()gives the EINVAL error.Your commit log doesn't explain the rationale for this change. What is it? (I could guess that it is to error on a case that may be a user programming error, but I want to check this with you, so that I add the right text to the man page.)
The dup2() behavior comes from the logical consequence of dup2()'s "close on reuse"; one would think it would be logical for dup3() to behave the same way.
-hpa
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Quoted 3 lines · H. Peter Anvin
The
dup2()behavior comes from the logical consequence of dup2()'s "close on reuse"; one would think it would be logical fordup3()to behave the same way.
No. We deliberately decided on this change. Otherwise, what is the result of dup3(fd, fd, O_CLOEXEC)? There is no reason to use dup2(fd,fd), so why the hell somebody wants to defend this is beyond me.
- --
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖ -----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
iEYEARECAAYFAkjuaisACgkQ2ijCOnn/RHRBBgCeMtzyHtpv7jt5a2XxIq9LEoDN ZVYAnixMwtW6d6SL55MvrKwV/B5Yv1Cm
=MCqO
-----END PGP SIGNATURE-----
Quoted 12 lines · Ulrich Drepper
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1Quoted 3 lines · H. Peter Anvin
The
dup2()behavior comes from the logical consequence of dup2()'s "close on reuse"; one would think it would be logical fordup3()to behave the same way.No. We deliberately decided on this change. Otherwise, what is the result of
dup3(fd, fd, O_CLOEXEC)? There is no reason to usedup2(fd,fd), so why the hell somebody wants to defend this is beyond me.
The result of dup3(fd, fd, O_CLOEXEC) is to set the O_CLOEXEC flag on fd.
The behaviour of dup2() is functionally the following:
1. Duplicate the file descriptor from file_table[oldfd].
2. If file_table[newfd] is in use, close it.
3. Install the duplicate file descriptor at file_table[newfd].
Step (2) could be considered a bit dubious, but the behaviour of dup2(fd, fd) is a direct consequence of the chosen semantics.
-hpa
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Quoted 1 line · H. Peter Anvin
The result of
dup3(fd, fd, O_CLOEXEC)is to set theO_CLOEXECflag on fd.
That's bad and disregarded by Al and myself because it is one and the same descriptor and therefore it changes the source descriptor.
Quoted 2 lines
Step (2) could be considered a bit dubious, but the behaviour of
dup2(fd, fd)is a direct consequence of the chosen semantics.
The behavior of dup2(fd,fd) is just a result of an accident in the original implementation. It makes no sense and the mistake doesn't have to be repeated.
- --
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖ -----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
iEYEARECAAYFAkjubyIACgkQ2ijCOnn/RHRIMwCfdFeW08lSPRh12C+qKzF99AWf idEAn0x0jqcVEIzmcgIVuZLlHKleNmC0
=Dx/d
-----END PGP SIGNATURE-----
Quoted 8 lines · Ulrich Drepper
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1Quoted 1 line · H. Peter Anvin
The result of
dup3(fd, fd, O_CLOEXEC)is to set theO_CLOEXECflag on fd.That's bad and disregarded by Al and myself because it is one and the same descriptor and therefore it changes the source descriptor.
It's not the source descriptor, per se, it is the "new" descriptor, which happens to have a side effect of closing the "old" descriptor.
Quoted 6 lines
Quoted 2 lines
Step (2) could be considered a bit dubious, but the behaviour of
dup2(fd, fd)is a direct consequence of the chosen semantics.The behavior of
dup2(fd,fd)is just a result of an accident in the original implementation. It makes no sense and the mistake doesn't have to be repeated.
Inconsistency is bad, too, and one could *definitely* argue that the fundamental problem is the one of closing a pre-existing descriptor rather than forcing the user to do that explicitly if that behaviour was desired.
-hpa
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Quoted 4 lines · H. Peter Anvin
Inconsistency is bad, too, and one could *definitely* argue that the fundamental problem is the one of closing a pre-existing descriptor rather than forcing the user to do that explicitly if that behaviour was desired.
This is a new interface. It's not breaking any code and fixing problems with older interfaces is exactly what you do in such situations.
- --
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖ -----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
iEYEARECAAYFAkjucdAACgkQ2ijCOnn/RHQJ2gCgm3vuOgcPJPl3RgdehydbQjkc 2BgAoIM8KBKh94ge2IVBVmqknWs4hwEk
=2KCr
-----END PGP SIGNATURE-----
Quoted 18 lines · H. Peter Anvin
Quoted 10 lines · Ulrich Drepper
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1Quoted 2 lines · H. Peter Anvin
The result of
dup3(fd, fd, O_CLOEXEC)is to set theO_CLOEXECflag on fd.That's bad and disregarded by Al and myself because it is one and the same descriptor and therefore it changes the source descriptor.
It's not the source descriptor, per se, it is the "new" descriptor, which happens to have a side effect of closing the "old" descriptor.
Quoted 3 lines
Quoted 2 lines
Step (2) could be considered a bit dubious, but the behaviour of
dup2(fd, fd)is a direct consequence of the chosen semantics.
Quoted 7 lines
Quoted 3 lines
The behavior of
dup2(fd,fd)is just a result of an accident in the original implementation. It makes no sense and the mistake doesn't have to be repeated.Inconsistency is bad, too, and one could *definitely* argue that the fundamental problem is the one of closing a pre-existing descriptor rather than forcing the user to do that explicitly if that behaviour was desired.
Well, as long as we are fixing the dup3() interface in the way that Al and Ulrich have suggested, what about another fix:
give an error if newfd is already open, thus forcing the user to do an explicit close
?
This silent close in dup2() is an implementation blemish. Why not eliminate it?
Cheers,
Michael
Signature · 5 lines
Michael Kerrisk Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/ git://git.kernel.org/pub/scm/docs/man-pages/man-pages.git man-pages online: http://www.kernel.org/doc/man-pages/online_pages.html Found a bug? http://www.kernel.org/doc/man-pages/reporting_bugs.html
On Fri, 2008-10-10 at 07:04 +0200, Michael Kerrisk wrote:
[....]
Quoted 9 lines
Well, as long as we are fixing the
dup3()interface in the way that Al and Ulrich have suggested, what about another fix:give an error if newfd is already open, thus forcing the user to do an explicit close
?
This silent close in
dup2()is an implementation blemish. Why not eliminate it?
Apart from the usual "do not break almost all existing apps" killer
reason: The alternative is that people will simply add a "close(newfd)" everytime before "dup2(oldfd,newfd)" since close() is harmless on a non-open fd.
Bernd
Signature · 3 lines
Firmix Software GmbH http://www.firmix.at/
mobil: +43 664 4416156 fax: +43 1 7890849-55
Embedded Linux Development and ServicesQuoted 8 lines · Bernd Petrovitsch
On Fri, 2008-10-10 at 07:04 +0200, Michael Kerrisk wrote:
[....]Quoted 6 lines
Well, as long as we are fixing the
dup3()interface in the way that Al and Ulrich have suggested, what about another fix:give an error if newfd is already open, thus forcing the user to do an explicit close
Quoted 8 lines
Quoted 3 lines
?
This silent close in
dup2()is an implementation blemish. Why not eliminate it?Apart from the usual "do not break almost all existing apps" killer
reason: The alternative is that people will simply add a "close(newfd)" everytime before "dup2(oldfd,newfd)" sinceclose()is harmless on a non-open fd.
Bernd,
I think you've missed the point. The idea is not to change to dup2(), but to eliminate the blemish in its design in the new dup3() (since we have alrady eliminated one other blemish).
Cheers,
Michael
Signature · 5 lines
Michael Kerrisk Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/ git://git.kernel.org/pub/scm/docs/man-pages/man-pages.git man-pages online: http://www.kernel.org/doc/man-pages/online_pages.html Found a bug? http://www.kernel.org/doc/man-pages/reporting_bugs.html
Quoted 13 lines · Michael Kerrisk
Quoted 12 lines · Bernd Petrovitsch
On Fri, 2008-10-10 at 07:04 +0200, Michael Kerrisk wrote:
[....]Quoted 9 lines
Well, as long as we are fixing the
dup3()interface in the way that Al and Ulrich have suggested, what about another fix:give an error if newfd is already open, thus forcing the user to do an explicit close
?
This silent close in
dup2()is an implementation blemish. Why not eliminate it?
Quoted 8 lines
Quoted 4 lines
Apart from the usual "do not break almost all existing apps" killer
reason: The alternative is that people will simply add a "close(newfd)" everytime before "dup2(oldfd,newfd)" sinceclose()is harmless on a non-open fd.Bernd,
I think you've missed the point. The idea is not to change to
dup2(),
That may well be. So the "eliminate it" apparently doesn't mean "eliminate it in dup2()".
Quoted 2 lines
but to eliminate the blemish in its design in the new
dup3()(since we have alrady eliminated one other blemish).
FWIW I consider the automatic close() in dup2() a feature (if only that it avoids an additional system call). So I would include it in dup3() too.
But a sane, clear and consistent definition and handling of flags are more important the auto-close() feature.
Bernd
Signature · 3 lines
Firmix Software GmbH http://www.firmix.at/
mobil: +43 664 4416156 fax: +43 1 7890849-55
Embedded Linux Development and ServicesQuoted 19 lines · Bernd Petrovitsch
Quoted 18 lines · Michael Kerrisk
Quoted 16 lines · Bernd Petrovitsch
On Fri, 2008-10-10 at 07:04 +0200, Michael Kerrisk wrote:
[....]Quoted 9 lines
Well, as long as we are fixing the
dup3()interface in the way that Al and Ulrich have suggested, what about another fix:give an error if newfd is already open, thus forcing the user to do an explicit close
?
This silent close in
dup2()is an implementation blemish. Why not eliminate it?Apart from the usual "do not break almost all existing apps" killer
reason: The alternative is that people will simply add a "close(newfd)" everytime before "dup2(oldfd,newfd)" sinceclose()is harmless on a non-open fd.
Quoted 6 lines
Quoted 3 lines
Bernd,
I think you've missed the point. The idea is not to change to
dup2(),That may well be. So the "eliminate it" apparently doesn't mean "eliminate it in
dup2()".
Right.
Quoted 5 lines
Quoted 2 lines
but to eliminate the blemish in its design in the new
dup3()(since we have alrady eliminated one other blemish).FWIW I consider the automatic
close()indup2()a feature (if only that it avoids an additional system call).
Exploiting this "feature" can hide real errors that would be detected if one does an explicit close(). (See the dup2 man page.)
Cheers,
Michael
Quoted 1 line · Michael Kerrisk
On Fri, Oct 10, 2008 at 3:02 PM, Bernd Petrovitsch <[email protected]> wrote:
[....]
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
but to eliminate the blemish in its design in the new
dup3()(since we have alrady eliminated one other blemish).FWIW I consider the automatic
close()indup2()a feature (if only that it avoids an additional system call).Exploiting this "feature" can hide real errors that would be detected if one does an explicit
close(). (See the dup2 man page.)
If someone is interested in the return value of the automatic close(), he/she can call close() anyways and check the the result.
Bernd
Signature · 3 lines
Firmix Software GmbH http://www.firmix.at/
mobil: +43 664 4416156 fax: +43 1 7890849-55
Embedded Linux Development and ServicesQuoted 10 lines · Ulrich Drepper
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1Quoted 3 lines · H. Peter Anvin
The
dup2()behavior comes from the logical consequence of dup2()'s "close on reuse"; one would think it would be logical fordup3()to behave the same way.No. We deliberately decided on this change. Otherwise, what is the result of
dup3(fd, fd, O_CLOEXEC)?
Good point. However, I don't find any list mail that discussed this implementation choice or its rationale. That would have been useful.
Quoted 2 lines
There is no reason to use
dup2(fd,fd), so why the hell somebody wants to defend this is beyond me.
There is no defense, except consistency with historical behavior. My reasons for raising this thread were:
a) to point out the inconsistency with dup2().
b) to check that the change was intentional (though it was already fairly clear that it was)
c) to determine the reason(s) for the change (which were not made clear)
As you point out, we have various possibilities with dup3(fd, fd, flags)
1) Do nothing to fd.
2) change the O_CLOEXEC flag for fd.
3) Give an error.
The problem is that depending on one's perspective one argue that either 1 or 2 is consistent with dup2(), so 3 seems a reasonable thing to do.
Cheers,
Michael
Signature · 5 lines
Michael Kerrisk Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/ git://git.kernel.org/pub/scm/docs/man-pages/man-pages.git man-pages online: http://www.kernel.org/doc/man-pages/online_pages.html Found a bug? http://www.kernel.org/doc/man-pages/reporting_bugs.html
Quoted 8 lines · Ulrich Drepper
Quoted 3 lines · H. Peter Anvin
The
dup2()behavior comes from the logical consequence of dup2()'s "close on reuse"; one would think it would be logical fordup3()to behave the same way.No. We deliberately decided on this change. Otherwise, what is the result of
dup3(fd, fd, O_CLOEXEC)? There is no reason to usedup2(fd,fd), so why the hell somebody wants to defend this is beyond me.
The reason is: application programmers expect it to behave that way. The interface is mostly targeted for typical application programmers, and consistency decreases bugs. In this respect, it would be a good idea for dup3() to have the same semantics. Doing that might not make practical sense, but it is secondary to obviousness.
Signature · 3 lines
Heikki Orsila [email protected] http://www.iki.fi/shd