Correct behavior on O_DIRECT sparse file writes
Hello everyone,
The test below creates a sparse file and then fills a hole with O_DIRECT. As far as I can tell from reading generic_osync_inode, the filesystem metadata is only forced to disk if i_size changes during the file write. I've tested ext3, xfs and reiserfs and they all skip the commit when filling holes.
I would argue that filling holes via O_DIRECT is supposed to commit the metadata required to find those file blocks later. At least on ext3, O_SYNC does force a commit on fill holes (haven't tested others).
So, is the current behavior a bug or a feature?
dd if=/dev/zero of=foo bs=1M seek=1 count=1 oflag=direct
hexdump foo | head -n 2
0000000 62b1 ea2d 73e8 c64f f5ef 1af5 dd09 8ccd
0000010 75ec 9581 e0ea ae9b e28f b76d a700 4d5b
dd if=/dev/urandom of=foo bs=4k count=1 conv=notrunc oflag=direct reboot -nf
(after reboot)
hexdump foo
0000000 0000 0000 0000 0000 0000 0000 0000 0000
*
0200000
-chris
Quoted 8 lines · On Fri, 12 Oct 2007 16:39:27 -0400 Chris Mason <[email protected]> wrote:
Hello everyone,
The test below creates a sparse file and then fills a hole with
O_DIRECT. As far as I can tell from readinggeneric_osync_inode, the filesystem metadata is only forced to disk ifi_sizechanges during the file write. I've tested ext3, xfs and reiserfs and they all skip the commit when filling holes.
Quoted 5 lines
I would argue that filling holes via
O_DIRECTis supposed to commit the metadata required to find those file blocks later. At least on ext3,O_SYNCdoes force a commit on fill holes (haven't tested others).So, is the current behavior a bug or a feature?
I don't think it's a bug. Sure, O_DIRECT is synchronous, but that's because it is, err, direct. Not because it provides extra data-integrity guarantees. If you want those guarantees, use O_SYNC as well.
Quoted 18 lines
dd if=/dev/zero of=foo bs=1M seek=1 count=1 oflag=direct
hexdump foo | head -n 2
0000000 62b1 ea2d 73e8 c64f f5ef 1af5 dd09 8ccd
0000010 75ec 9581 e0ea ae9b e28f b76d a700 4d5bdd if=/dev/urandom of=foo bs=4k count=1 conv=notrunc oflag=direct reboot -nf
(after reboot)
hexdump foo
0000000 0000 0000 0000 0000 0000 0000 0000 0000
*
0200000-chris
* Andrew Morton:
Quoted 3 lines
I don't think it's a bug. Sure,
O_DIRECTis synchronous, but that's because it is, err, direct. Not because it provides extra data-integrity guarantees. If you want those guarantees, useO_SYNCas well.
This needs to be prominently documented. Right now, it's far from clear that you need both O_DIRECT and O_SYNC.
Quoted 8 lines · Florian Weimer
* Andrew Morton:
Quoted 3 lines
I don't think it's a bug. Sure,
O_DIRECTis synchronous, but that's because it is, err, direct. Not because it provides extra data-integrity guarantees. If you want those guarantees, useO_SYNCas well.This needs to be prominently documented. Right now, it's far from clear that you need both
O_DIRECTandO_SYNC.
It's certainly not a requirement for NFS. O_DIRECT on NFS forces data to the server, which always updates a file's metadata on each write, including indirect blocks.
begin:vcard
fn:Chuck Lever
n:Lever;Chuck
org:Oracle Corporation;Corporate Architecture: Linux Projects Group adr:;;1015 Granger Avenue;Ann Arbor;MI;48104;USA title:Principal Member of Staff
tel;work:+1 248 614 5091
x-mozilla-html:FALSE
version:2.1
end:vcard
Quoted 6 lines
Quoted 6 lines
The test below creates a sparse file and then fills a hole with
O_DIRECT. As far as I can tell from readinggeneric_osync_inode, the filesystem metadata is only forced to disk ifi_sizechanges during the file write. I've tested ext3, xfs and reiserfs and they all skip the commit when filling holes.
Quoted 8 lines
Quoted 4 lines
I would argue that filling holes via
O_DIRECTis supposed to commit the metadata required to find those file blocks later. At least on ext3,O_SYNCdoes force a commit on fill holes (haven't tested others).I don't think it's a bug. Sure,
O_DIRECTis synchronous, but that's because it is, err, direct. Not because it provides extra data-integrity guarantees. If you want those guarantees, useO_SYNCas well.
That makes sense, but how do you explain the committing of the size change without O_SYNC? That seems wrong to me.
This does need to be documented carefully, because a person could easily believe, even subconsciously, that O_DIRECT makes the entire file write direct, and sloppy documentation might actually use words to that effect.
--
Bryan Henderson IBM Almaden Research Center
San Jose CA Filesystems