Skip to contentSkip to conversation

dup2() vs dup3() inconsistency when

DISCUSSION Michael Kerrisk 5 participants 15 messages Oct 9, 2008 – Oct 10, 2008

lore
Share & tools
lore
#0 lore raw

Al,

your 2008-07-26 commit 6c5d0512a091480c9f981162227fdb1c9d70e555 for fs/fcntl.c:sys_dup3 contains

+ if (unlikely(oldfd == newfd))
+ return -EINVAL;

This makes dup2() and dup3() differ with respect to the oldfd==newfd case (dup2() becomes a no-op, just returning oldfd, dup3() gives the EINVAL error.

Your commit log doesn't explain the rationale for this change. What is it? (I could guess that it is to error on a case that may be a user programming error, but I want to check this with you, so that I add the right text to the man page.)

Cheers,

Michael

Signature · 5 lines
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
git://git.kernel.org/pub/scm/docs/man-pages/man-pages.git
man-pages online: http://www.kernel.org/doc/man-pages/online_pages.html
Found a bug? http://www.kernel.org/doc/man-pages/reporting_bugs.html
#1 lore raw
Quoted 8 lines · Michael Kerrisk

Al,

your 2008-07-26 commit 6c5d0512a091480c9f981162227fdb1c9d70e555 for fs/fcntl.c:sys_dup3 contains

(quoted diff) +2 -0
+       if (unlikely(oldfd == newfd))
+               return -EINVAL;
Quoted 9 lines

This makes dup2() and dup3() differ with respect to the oldfd==newfd case (dup2() becomes a no-op, just returning oldfd, dup3() gives the EINVAL error.

Your commit log doesn't explain the rationale for this change. What is it? (I could guess that it is to error on a case that may be a user programming error, but I want to check this with you, so that I add the right text to the man page.)

The dup2() behavior comes from the logical consequence of dup2()'s "close on reuse"; one would think it would be logical for dup3() to behave the same way.

-hpa

#2 lore raw

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Quoted 3 lines · H. Peter Anvin

The dup2() behavior comes from the logical consequence of dup2()'s "close on reuse"; one would think it would be logical for dup3() to behave the same way.

No. We deliberately decided on this change. Otherwise, what is the result of dup3(fd, fd, O_CLOEXEC)? There is no reason to use dup2(fd,fd), so why the hell somebody wants to defend this is beyond me.

- --
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖ -----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEARECAAYFAkjuaisACgkQ2ijCOnn/RHRBBgCeMtzyHtpv7jt5a2XxIq9LEoDN ZVYAnixMwtW6d6SL55MvrKwV/B5Yv1Cm
=MCqO
-----END PGP SIGNATURE-----

#3 lore raw
Quoted 12 lines · Ulrich Drepper

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Quoted 3 lines · H. Peter Anvin

The dup2() behavior comes from the logical consequence of dup2()'s "close on reuse"; one would think it would be logical for dup3() to behave the same way.

No. We deliberately decided on this change. Otherwise, what is the result of dup3(fd, fd, O_CLOEXEC)? There is no reason to use dup2(fd,fd), so why the hell somebody wants to defend this is beyond me.

The result of dup3(fd, fd, O_CLOEXEC) is to set the O_CLOEXEC flag on fd.

The behaviour of dup2() is functionally the following:

1. Duplicate the file descriptor from file_table[oldfd].
2. If file_table[newfd] is in use, close it.
3. Install the duplicate file descriptor at file_table[newfd].

Step (2) could be considered a bit dubious, but the behaviour of dup2(fd, fd) is a direct consequence of the chosen semantics.

-hpa

#4 lore raw

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Quoted 1 line · H. Peter Anvin

The result of dup3(fd, fd, O_CLOEXEC) is to set the O_CLOEXEC flag on fd.

That's bad and disregarded by Al and myself because it is one and the same descriptor and therefore it changes the source descriptor.

Quoted 2 lines

Step (2) could be considered a bit dubious, but the behaviour of dup2(fd, fd) is a direct consequence of the chosen semantics.

The behavior of dup2(fd,fd) is just a result of an accident in the original implementation. It makes no sense and the mistake doesn't have to be repeated.

- --
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖ -----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEARECAAYFAkjubyIACgkQ2ijCOnn/RHRIMwCfdFeW08lSPRh12C+qKzF99AWf idEAn0x0jqcVEIzmcgIVuZLlHKleNmC0
=Dx/d
-----END PGP SIGNATURE-----

#5 lore raw
Quoted 8 lines · Ulrich Drepper

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Quoted 1 line · H. Peter Anvin

The result of dup3(fd, fd, O_CLOEXEC) is to set the O_CLOEXEC flag on fd.

That's bad and disregarded by Al and myself because it is one and the same descriptor and therefore it changes the source descriptor.

It's not the source descriptor, per se, it is the "new" descriptor, which happens to have a side effect of closing the "old" descriptor.

Quoted 6 lines
Quoted 2 lines

Step (2) could be considered a bit dubious, but the behaviour of dup2(fd, fd) is a direct consequence of the chosen semantics.

The behavior of dup2(fd,fd) is just a result of an accident in the original implementation. It makes no sense and the mistake doesn't have to be repeated.

Inconsistency is bad, too, and one could *definitely* argue that the fundamental problem is the one of closing a pre-existing descriptor rather than forcing the user to do that explicitly if that behaviour was desired.

-hpa

#6 lore raw

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Quoted 4 lines · H. Peter Anvin

Inconsistency is bad, too, and one could *definitely* argue that the fundamental problem is the one of closing a pre-existing descriptor rather than forcing the user to do that explicitly if that behaviour was desired.

This is a new interface. It's not breaking any code and fixing problems with older interfaces is exactly what you do in such situations.

- --
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖ -----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEARECAAYFAkjucdAACgkQ2ijCOnn/RHQJ2gCgm3vuOgcPJPl3RgdehydbQjkc 2BgAoIM8KBKh94ge2IVBVmqknWs4hwEk
=2KCr
-----END PGP SIGNATURE-----

#7 lore raw
In reply to H. Peter Anvin
Quoted 18 lines · H. Peter Anvin
Quoted 10 lines · Ulrich Drepper

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Quoted 2 lines · H. Peter Anvin

The result of dup3(fd, fd, O_CLOEXEC) is to set the O_CLOEXEC flag on fd.

That's bad and disregarded by Al and myself because it is one and the same descriptor and therefore it changes the source descriptor.

It's not the source descriptor, per se, it is the "new" descriptor, which happens to have a side effect of closing the "old" descriptor.

Quoted 3 lines
Quoted 2 lines

Step (2) could be considered a bit dubious, but the behaviour of dup2(fd, fd) is a direct consequence of the chosen semantics.

Quoted 7 lines
Quoted 3 lines

The behavior of dup2(fd,fd) is just a result of an accident in the original implementation. It makes no sense and the mistake doesn't have to be repeated.

Inconsistency is bad, too, and one could *definitely* argue that the fundamental problem is the one of closing a pre-existing descriptor rather than forcing the user to do that explicitly if that behaviour was desired.

Well, as long as we are fixing the dup3() interface in the way that Al and Ulrich have suggested, what about another fix:

give an error if newfd is already open, thus forcing the user to do an explicit close

?

This silent close in dup2() is an implementation blemish. Why not eliminate it?

Cheers,

Michael

Signature · 5 lines
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
git://git.kernel.org/pub/scm/docs/man-pages/man-pages.git
man-pages online: http://www.kernel.org/doc/man-pages/online_pages.html
Found a bug? http://www.kernel.org/doc/man-pages/reporting_bugs.html
#8 lore raw

On Fri, 2008-10-10 at 07:04 +0200, Michael Kerrisk wrote:
[....]

Quoted 9 lines

Well, as long as we are fixing the dup3() interface in the way that Al and Ulrich have suggested, what about another fix:

give an error if newfd is already open, thus forcing the user to do an explicit close

?

This silent close in dup2() is an implementation blemish. Why not eliminate it?

Apart from the usual "do not break almost all existing apps" killer
reason: The alternative is that people will simply add a "close(newfd)" everytime before "dup2(oldfd,newfd)" since close() is harmless on a non-open fd.

Bernd

Signature · 3 lines
Firmix Software GmbH                   http://www.firmix.at/
mobil: +43 664 4416156                 fax: +43 1 7890849-55
          Embedded Linux Development and Services
#9 lore raw
Quoted 8 lines · Bernd Petrovitsch

On Fri, 2008-10-10 at 07:04 +0200, Michael Kerrisk wrote:
[....]

Quoted 6 lines

Well, as long as we are fixing the dup3() interface in the way that Al and Ulrich have suggested, what about another fix:

give an error if newfd is already open, thus forcing the user to do an explicit close

Quoted 8 lines
Quoted 3 lines

?

This silent close in dup2() is an implementation blemish. Why not eliminate it?

Apart from the usual "do not break almost all existing apps" killer
reason: The alternative is that people will simply add a "close(newfd)" everytime before "dup2(oldfd,newfd)" since close() is harmless on a non-open fd.

Bernd,

I think you've missed the point. The idea is not to change to dup2(), but to eliminate the blemish in its design in the new dup3() (since we have alrady eliminated one other blemish).

Cheers,

Michael

Signature · 5 lines
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
git://git.kernel.org/pub/scm/docs/man-pages/man-pages.git
man-pages online: http://www.kernel.org/doc/man-pages/online_pages.html
Found a bug? http://www.kernel.org/doc/man-pages/reporting_bugs.html
#10 lore raw
Quoted 13 lines · Michael Kerrisk
Quoted 12 lines · Bernd Petrovitsch

On Fri, 2008-10-10 at 07:04 +0200, Michael Kerrisk wrote:
[....]

Quoted 9 lines

Well, as long as we are fixing the dup3() interface in the way that Al and Ulrich have suggested, what about another fix:

give an error if newfd is already open, thus forcing the user to do an explicit close

?

This silent close in dup2() is an implementation blemish. Why not eliminate it?

Quoted 8 lines
Quoted 4 lines

Apart from the usual "do not break almost all existing apps" killer
reason: The alternative is that people will simply add a "close(newfd)" everytime before "dup2(oldfd,newfd)" since close() is harmless on a non-open fd.

Bernd,

I think you've missed the point. The idea is not to change to dup2(),

That may well be. So the "eliminate it" apparently doesn't mean "eliminate it in dup2()".

Quoted 2 lines

but to eliminate the blemish in its design in the new dup3() (since we have alrady eliminated one other blemish).

FWIW I consider the automatic close() in dup2() a feature (if only that it avoids an additional system call). So I would include it in dup3() too.
But a sane, clear and consistent definition and handling of flags are more important the auto-close() feature.

Bernd

Signature · 3 lines
Firmix Software GmbH                   http://www.firmix.at/
mobil: +43 664 4416156                 fax: +43 1 7890849-55
          Embedded Linux Development and Services
#11 lore raw
Quoted 19 lines · Bernd Petrovitsch
Quoted 18 lines · Michael Kerrisk
Quoted 16 lines · Bernd Petrovitsch

On Fri, 2008-10-10 at 07:04 +0200, Michael Kerrisk wrote:
[....]

Quoted 9 lines

Well, as long as we are fixing the dup3() interface in the way that Al and Ulrich have suggested, what about another fix:

give an error if newfd is already open, thus forcing the user to do an explicit close

?

This silent close in dup2() is an implementation blemish. Why not eliminate it?

Apart from the usual "do not break almost all existing apps" killer
reason: The alternative is that people will simply add a "close(newfd)" everytime before "dup2(oldfd,newfd)" since close() is harmless on a non-open fd.

Quoted 6 lines
Quoted 3 lines

Bernd,

I think you've missed the point. The idea is not to change to dup2(),

That may well be. So the "eliminate it" apparently doesn't mean "eliminate it in dup2()".

Right.

Quoted 5 lines
Quoted 2 lines

but to eliminate the blemish in its design in the new dup3() (since we have alrady eliminated one other blemish).

FWIW I consider the automatic close() in dup2() a feature (if only that it avoids an additional system call).

Exploiting this "feature" can hide real errors that would be detected if one does an explicit close(). (See the dup2 man page.)

Cheers,

Michael

#12 lore raw
Quoted 1 line · Michael Kerrisk

On Fri, Oct 10, 2008 at 3:02 PM, Bernd Petrovitsch <[email protected]> wrote:

[....]

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

but to eliminate the blemish in its design in the new dup3() (since we have alrady eliminated one other blemish).

FWIW I consider the automatic close() in dup2() a feature (if only that it avoids an additional system call).

Exploiting this "feature" can hide real errors that would be detected if one does an explicit close(). (See the dup2 man page.)

If someone is interested in the return value of the automatic close(), he/she can call close() anyways and check the the result.

Bernd

Signature · 3 lines
Firmix Software GmbH                   http://www.firmix.at/
mobil: +43 664 4416156                 fax: +43 1 7890849-55
          Embedded Linux Development and Services
#13 lore raw
In reply to Ulrich Drepper
Quoted 10 lines · Ulrich Drepper

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Quoted 3 lines · H. Peter Anvin

The dup2() behavior comes from the logical consequence of dup2()'s "close on reuse"; one would think it would be logical for dup3() to behave the same way.

No. We deliberately decided on this change. Otherwise, what is the result of dup3(fd, fd, O_CLOEXEC)?

Good point. However, I don't find any list mail that discussed this implementation choice or its rationale. That would have been useful.

Quoted 2 lines

There is no reason to use
dup2(fd,fd), so why the hell somebody wants to defend this is beyond me.

There is no defense, except consistency with historical behavior. My reasons for raising this thread were:

a) to point out the inconsistency with dup2().
b) to check that the change was intentional (though it was already fairly clear that it was)
c) to determine the reason(s) for the change (which were not made clear)

As you point out, we have various possibilities with dup3(fd, fd, flags)

1) Do nothing to fd.
2) change the O_CLOEXEC flag for fd.
3) Give an error.

The problem is that depending on one's perspective one argue that either 1 or 2 is consistent with dup2(), so 3 seems a reasonable thing to do.

Cheers,

Michael

Signature · 5 lines
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
git://git.kernel.org/pub/scm/docs/man-pages/man-pages.git
man-pages online: http://www.kernel.org/doc/man-pages/online_pages.html
Found a bug? http://www.kernel.org/doc/man-pages/reporting_bugs.html
#14 lore raw
In reply to Ulrich Drepper
Quoted 8 lines · Ulrich Drepper
Quoted 3 lines · H. Peter Anvin

The dup2() behavior comes from the logical consequence of dup2()'s "close on reuse"; one would think it would be logical for dup3() to behave the same way.

No. We deliberately decided on this change. Otherwise, what is the result of dup3(fd, fd, O_CLOEXEC)? There is no reason to use dup2(fd,fd), so why the hell somebody wants to defend this is beyond me.

The reason is: application programmers expect it to behave that way. The interface is mostly targeted for typical application programmers, and consistency decreases bugs. In this respect, it would be a good idea for dup3() to have the same semantics. Doing that might not make practical sense, but it is secondary to obviousness.

Signature · 3 lines
Heikki Orsila
[email protected]
http://www.iki.fi/shd