aio_fsync() a directory ?
This change is completely contrary to real world behaviour. No modern filesystem I know of implements this behaviour as the default, and performance with the coresponding mount options (usually -o dirsync on Linux) is terrible as it forces a write out of the log (or corresponding action on non-log based filesystems) and in the common case of volatile write caches a cache flush.
Retrospectively claiming this as standards behaviour in a "clarification" is utterly wrong.
Quoted 119 lines · Xavier Roche
Just to close this thread, Geoff Clare has summarized the changes regarding
aio_fsync()and relationship wrt. directory entry data.<http://austingroupbugs.net/view.php?id=672>
"In the April 18 teleconference it was agreed that the standard should mandate that directory operations are always synchronized on conforming file systems, and should include warnings about non-conforming configurations. The proposed changes are as follows.
Changes to XBD...
At page 94 line 2581-2588 section 3.376 change:
For read, when the operation has been completed or diagnosed if unsuccessful. The read is complete only when an image of the data has been successfully transferred to the requesting process. If there were any pending write requests affecting the data to be read at the time that the synchronized read operation was requested, these write requests are successfully transferred prior to reading the data.
For write, when the operation has been completed or diagnosed if unsuccessful. The write is complete only when the data specified in the write request is successfully transferred and all file system information required to retrieve the data is successfully transferred.
to:
For read operations, when the operation has been completed or diagnosed if unsuccessful. The operation is complete only when an image of the data has been successfully transferred to the requesting process. If there were any pending write requests or (if the file is a directory) directory modifications affecting the data to be read at the time that the synchronized read operation was requested, these requests are successfully transferred prior to reading the data.
For write operations and directory modification operations, when the operation has been completed or diagnosed if unsuccessful. The operation is complete only when the written data or (if the file is a directory) modified directory entries have been successfully transferred to storage and all file system information required to retrieve them is successfully transferred.
At page 107 line 2859 add a new XBD 4.2 section (and renumber the current 4.2 and all later 4.x sections):
4.2 Directory Operations
All file system operations that read a directory or that modify the contents of a directory (for example creating, unlinking, or renaming a file) shall be completed as defined for synchronized I/O data integrity completion (see section 3.376).
<small>Note: Although conforming file systems are required to
perform all directory modifications as synchronized I/O operations, some file systems may support non-conforming configurations (for example via mount options) where directory modifications are not synchronized. Applications that rely on directory modifications being synchronized should only be used with such file systems in their conforming configuration(s).</small>Changes to XSH...
At page 574 line 19833 section
aio_fsync()change the APPLICATION USAGE section from:None.
to:
Refer to
fdatasync()andfsync().At page 815 line 27215 section
fdatasync()append to the first paragraph:If the file is a directory, an implicit
fdatasync()is already performed on every I/O operation (see XBD 4.2) and consequently iffdatasync()is called explicitly it shall take no action and shall return the value 0.At page 815 line 27232 section
fdatasync()change the APPLICATION USAGE section from:None.
to:
Although conforming file systems are required to complete all directory modifications as defined for synchronized I/O data integrity completion, some file systems may support non-conforming configurations (for example via mount options) where directory modifications are not synchronized. When the file system is configured in this way, calls to
fdatasync()on directories may cause I/O operations to be synchronized, rather than being a no-op.At page 954 line 31987 section
fsync()add a new paragraph to the APPLICATION USAGE section:Since conforming file systems are required to complete all directory modifications as defined for synchronized I/O data integrity completion (see XBD 4.2), calling
fsync()on a directory only synchronizes the file attributes such as timestamps. However, some file systems may support non-conforming configurations (for example via mount options) where modifications to directory contents are not synchronized. When the file system is configured in this way, calls tofsync()on directories may cause directory contents to be synchronized in addition to file attributes.Changes to XRAT...
At page 3444 line 115531 add a new XRAT A.4.2 section (and renumber the current A.4.2 and all later A.4.x sections):
A.4.2 Directory Operations
Quoted 10 lines
Earlier versions of this standard did not make clear that all directory modifications are performed as synchronized I/O operations, although that is the historical behavior and was always intended. Applications have no need to call
fdatasync()orfsync()on a directory unless they want to synchronize the file attributes (usingfsync()), provided the directory is on a conforming file system. However, since applications may wish to usefdatasync()orfsync()to synchronize directory modifications on non-conforming file systems, implementations are required to supportfdatasync()on directories as a no-op on conforming file systems."
---end quoted text---
Quoted 8 lines · Christoph Hellwig
This change is completely contrary to real world behaviour. No modern filesystem I know of implements this behaviour as the default, and performance with the coresponding mount options (usually -o dirsync on Linux) is terrible as it forces a write out of the log (or corresponding action on non-log based filesystems) and in the common case of volatile write caches a cache flush.
Retrospectively claiming this as standards behaviour in a "clarification" is utterly wrong.
As I understand the wording, the original intent was to have all file operations synchronized (ie. operation committed permanently) at the "beginning" of the specification. The updated one suggests that an implementation may be non-conformant, and allow fsync() on a directory entry.
I'm also a bit puzzled by the default synchronization guarantee (which would lead an I/O per open/rename/etc. operation ?) ; even if many developers just do not care about filename synchronization. [ they are probably assuming that the fsync() on the file itself will also flush the related directory entrie(s) ]
Al Viro
maintainerQuoted 7 lines · Xavier Roche
Quoted 6 lines · Christoph Hellwig
This change is completely contrary to real world behaviour. No modern filesystem I know of implements this behaviour as the default, and performance with the coresponding mount options (usually -o dirsync on Linux) is terrible as it forces a write out of the log (or corresponding action on non-log based filesystems) and in the common case of volatile write caches a cache flush.
Quoted 8 lines
Quoted 2 lines
Retrospectively claiming this as standards behaviour in a "clarification" is utterly wrong.
As I understand the wording, the original intent was to have all file operations synchronized (ie. operation committed permanently) at the "beginning" of the specification. The updated one suggests that an implementation may be non-conformant, and allow
fsync()on a directory entry.
Original intent of _what_? With all due respect, original intent of Austin Group is none of our concern - "we had always intended to make that promise on your behalf, now we'd simply got around to saying so clearly" inspires many things, but "oh, sure, it's binding for us now" is *not* one of those. Suggestions to do anatomically impossible things, OTOH...
Take this up with the POSIX committee. Complaining on implementors' mailing lists won't have any effect on the future text of standards.
Quoted 2 lines · Roland McGrath
Take this up with the POSIX committee. Complaining on implementors' mailing lists won't have any effect on the future text of standards.
Absolutely. I'd like to mention that the austin-group-l mailing-list is opened to anyone wishing to contribute (with proper technical arguments, of course), and AFAICS opened to remarks (this bug was opened following one of my remarks - so you may consider this as my fault :p)
Quoted 9 lines · Xavier Roche
Quoted 2 lines · Roland McGrath
Take this up with the POSIX committee. Complaining on implementors' mailing lists won't have any effect on the future text of standards.
Absolutely. I'd like to mention that the austin-group-l mailing-list is opened to anyone wishing to contribute (with proper technical arguments, of course), and AFAICS opened to remarks (this bug was opened following one of my remarks - so you may consider this as my fault :p)
I'm guilty of a similar mistake, with issue #663, and would appreciate any supporting arguments against the ridiculous change in the standard they want to make and pretend is just a clarification of historic practice. The proposed change also renders glibc non-conforming so I think it's on-topic.
Rich
I have followed up with the Austin Group, here:
http://austingroupbugs.net/view.php?id=672#c1545
and would appreciate further review and followup as needed. To add further comments to that page, one must be an Austin Group member; if you're not one already, you can easily become one by using the "Signup for a new account" link on that page.