Allow O_SYNC to be set by fcntl(F_SETFL)
Subsystems (2)
PATCH 1/1 Allow O_SYNC to be set by fcntl(F_SETFL)
This has probably been a problem since day 1 (I ran into this running the 2.4 kernel years ago; finally got around to fixing it). The problem is that fcntl(fd, F_SETFL, flags|O_SYNC) appears to work, but silently ignores the O_SYNC flag.
Opening the file with O_SYNC works okay, but setting it later on via fcntl doesn't work.
--- fs/fcntl.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
fs/fcntl.c +1 -1
@@ -143,7 +143,7 @@ SYSCALL_DEFINE1(dup, unsigned int, fildes)return ret; } -#define SETFL_MASK (O_APPEND | O_NONBLOCK | O_NDELAY | O_DIRECT | O_NOATIME) +#define SETFL_MASK (O_APPEND | O_NONBLOCK | O_NDELAY | O_DIRECT | O_NOATIME | O_SYNC)static int setfl (int fd,struct file * filp,unsigned long arg) {
--
1.7.2.1
(did I ever reply to this? I meant to ;))
Quoted 15 lines · On Fri, 25 Feb 2011 16:52:36 -0500 Steve Rago <[email protected]> wrote:
This has probably been a problem since day 1 (I ran into this running the 2.4 kernel years ago; finally got around to fixing it). The problem is that
fcntl(fd, F_SETFL, flags|O_SYNC)appears to work, but silently ignores theO_SYNCflag.
Opening the file withO_SYNCworks okay, but setting it later on via fcntl doesn't work.Signed-off-by Steve Rago <[email protected]>--- fs/fcntl.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)fs/fcntl.c +0
-0
fs/fcntl.c SYSCALL_DEFINE1(dup, unsigned int, fildes) · line 146
Quoted 5 lines
fs/fcntl.c +1
-1@@ -143,7 +143,7 @@ SYSCALL_DEFINE1(dup, unsigned int, fildes)return ret; } -#define SETFL_MASK (O_APPEND | O_NONBLOCK | O_NDELAY | O_DIRECT | O_NOATIME) +#define SETFL_MASK (O_APPEND | O_NONBLOCK | O_NDELAY | O_DIRECT | O_NOATIME | O_SYNC)
Does any standard say that we should do this?
http://pubs.opengroup.org/onlinepubs/007908799/xsh/fcntl.html does, I guess.
I worry a bit that this change will surprise people. For example, this person:
http://koders.com/c/fidA34D8D5EE9AA5D0AB0F3C604678E2E935E5B0246.aspx?s=dupa
is going to wonder why his app suddenly got a lot slower!
Sadly, the kernel silently ignores invalid set bits in `arg', so we have no reliable way of signaling to the user that our behaviour here changed.
I wonder if we should sync the file when someone sets O_SYNC this way. If we don't then there is a period during which we have an fd which has O_SYNC set, but it has pending unwritten data. An O_SYNC fd should never be in such a state!
Ho hum. yes, I guess we should apply the patch. But it would have been better to not have screwed this up in the first place!
I actually prototypes this patch independetly a while ago, and in addition to the data writeout when removing O_SYNC there are the following caveats:
- O_SYNC is not actually one flag, but two: O_DSYNC and __O_SYNC. setfl() needs to make sure __O_SYNC cannot be in f_flags without O_DSYNC also beeing present.
- we need to audit all filesystems that they don't do stupid things when the O_SYNC flags appear or disappear during a write, that is make sure it is checked in just one place. The generic write code is fine in that respect, but I didn't go through all filesystems to verify it yet.
Quoted 23 lines · Andrew Morton
(did I ever reply to this? I meant to ;))
Quoted 18 lines · On Fri, 25 Feb 2011 16:52:36 -0500 Steve Rago<[email protected]> wrote:
This has probably been a problem since day 1 (I ran into this running the 2.4 kernel years ago; finally got around to fixing it). The problem is that
fcntl(fd, F_SETFL, flags|O_SYNC)appears to work, but silently ignores theO_SYNCflag.
Opening the file withO_SYNCworks okay, but setting it later on via fcntl doesn't work.Signed-off-by Steve Rago<[email protected]>--- fs/fcntl.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)fs/fcntl.c +0
-0@@ -143,7 +143,7 @@ SYSCALL_DEFINE1(dup, unsigned int, fildes)return ret; }
Quoted 6 lines
fs/fcntl.c SYSCALL_DEFINE1(dup, unsigned int, fildes) · line 146
Quoted 2 lines
fs/fcntl.c +1
-1-#define SETFL_MASK (O_APPEND | O_NONBLOCK | O_NDELAY | O_DIRECT | O_NOATIME) +#define SETFL_MASK (O_APPEND | O_NONBLOCK | O_NDELAY | O_DIRECT | O_NOATIME | O_SYNC)Does any standard say that we should do this?
http://pubs.opengroup.org/onlinepubs/007908799/xsh/fcntl.html does, I guess.
It's required by the Single UNIX Specification (POSIX.1). All other major platforms allow it to be set via fcntl. See bugzilla.kernel.org bug ID #5994.
Quoted 6 lines
I worry a bit that this change will surprise people. For example, this person:
http://koders.com/c/fidA34D8D5EE9AA5D0AB0F3C604678E2E935E5B0246.aspx?s=dupa
is going to wonder why his app suddenly got a lot slower!
Quoted 8 lines
Sadly, the kernel silently ignores invalid set bits in `arg', so we have no reliable way of signaling to the user that our behaviour here changed.
I wonder if we should sync the file when someone sets
O_SYNCthis way. If we don't then there is a period during which we have an fd which hasO_SYNCset, but it has pending unwritten data. AnO_SYNCfd should never be in such a state!
Why not? If I write something in non-synchronous mode, then change the file descriptor to synchronous mode, I should not make any assumptions about what was written prior to this point. If I care that much, I'll call fsync. All that matters is that the operating system honors the contract as specified by the system call API.
Quoted 5 lines
Ho hum. yes, I guess we should apply the patch. But it would have been better to not have screwed this up in the first place!
Agreed. Thanks for not letting this fall through the cracks.
Steve
Quoted 6 lines · On Fri, 08 Apr 2011 13:39:16 -0400 Steve Rago <[email protected]> wrote:
Quoted 4 lines
I wonder if we should sync the file when someone sets
O_SYNCthis way. If we don't then there is a period during which we have an fd which hasO_SYNCset, but it has pending unwritten data. AnO_SYNCfd should never be in such a state!Why not?
Because it's inconsistent. An O_SYNC fd never has outstanding writeout. Except for in this one new and special time window between a setfl() and the next write().
It's not a big deal, but it's somewhat ugly and merits thinking about.
Quoted 2 lines
If I write something in non-synchronous mode, then change the file descriptor to synchronous mode, I should
not make any assumptions about what was written prior to this point. If I care that much, I'll call fsync.
Well. You can call fsync() after every write() too.
Quoted 2 lines
All that
matters is that the operating system honors the contract as specified by the system call API.
There's a lot more to it than that. Things like quality-of-implementation and principle-of-least-surprise. We used to have a particular relationship between an O_SYNC fd and the state of the inode which it represents. With this patch, that relationship no longer holds.
As I say: not a big deal IMO, but it should be aired and thought about.
Quoted 3 lines · Andrew Morton
Because it's inconsistent. An
O_SYNCfd never has outstanding writeout. Except for in this one new and special time window between asetfl()and the nextwrite().
It might actually have outstanding writes for as long as it eventually takes the writeback code to push them out. O_SYNC only does a range writeout for the area that was written.