Skip to contentSkip to conversation

Correct behavior on O_DIRECT sparse file writes

DISCUSSION Chris Mason 5 participants 5 messages Oct 12, 2007 – Oct 15, 2007 VFS

lore
Share & tools
lore
#0 lore raw

Hello everyone,

The test below creates a sparse file and then fills a hole with O_DIRECT. As far as I can tell from reading generic_osync_inode, the filesystem metadata is only forced to disk if i_size changes during the file write. I've tested ext3, xfs and reiserfs and they all skip the commit when filling holes.

I would argue that filling holes via O_DIRECT is supposed to commit the metadata required to find those file blocks later. At least on ext3, O_SYNC does force a commit on fill holes (haven't tested others).

So, is the current behavior a bug or a feature?

dd if=/dev/zero of=foo bs=1M seek=1 count=1 oflag=direct

hexdump foo | head -n 2
0000000 62b1 ea2d 73e8 c64f f5ef 1af5 dd09 8ccd
0000010 75ec 9581 e0ea ae9b e28f b76d a700 4d5b

dd if=/dev/urandom of=foo bs=4k count=1 conv=notrunc oflag=direct reboot -nf

(after reboot)

hexdump foo
0000000 0000 0000 0000 0000 0000 0000 0000 0000
*
0200000

-chris

#1 lore raw
Quoted 8 lines · On Fri, 12 Oct 2007 16:39:27 -0400 Chris Mason <[email protected]> wrote:

Hello everyone,

The test below creates a sparse file and then fills a hole with O_DIRECT. As far as I can tell from reading generic_osync_inode, the filesystem metadata is only forced to disk if i_size changes during the file write. I've tested ext3, xfs and reiserfs and they all skip the commit when filling holes.

Quoted 5 lines

I would argue that filling holes via O_DIRECT is supposed to commit the metadata required to find those file blocks later. At least on ext3, O_SYNC does force a commit on fill holes (haven't tested others).

So, is the current behavior a bug or a feature?

I don't think it's a bug. Sure, O_DIRECT is synchronous, but that's because it is, err, direct. Not because it provides extra data-integrity guarantees. If you want those guarantees, use O_SYNC as well.

Quoted 18 lines

dd if=/dev/zero of=foo bs=1M seek=1 count=1 oflag=direct

hexdump foo | head -n 2
0000000 62b1 ea2d 73e8 c64f f5ef 1af5 dd09 8ccd
0000010 75ec 9581 e0ea ae9b e28f b76d a700 4d5b

dd if=/dev/urandom of=foo bs=4k count=1 conv=notrunc oflag=direct reboot -nf

(after reboot)

hexdump foo
0000000 0000 0000 0000 0000 0000 0000 0000 0000
*
0200000

-chris

#2 lore raw

* Andrew Morton:

Quoted 3 lines

I don't think it's a bug. Sure, O_DIRECT is synchronous, but that's because it is, err, direct. Not because it provides extra data-integrity guarantees. If you want those guarantees, use O_SYNC as well.

This needs to be prominently documented. Right now, it's far from clear that you need both O_DIRECT and O_SYNC.

#3 lore raw
Quoted 8 lines · Florian Weimer

* Andrew Morton:

Quoted 3 lines

I don't think it's a bug. Sure, O_DIRECT is synchronous, but that's because it is, err, direct. Not because it provides extra data-integrity guarantees. If you want those guarantees, use O_SYNC as well.

This needs to be prominently documented. Right now, it's far from clear that you need both O_DIRECT and O_SYNC.

It's certainly not a requirement for NFS. O_DIRECT on NFS forces data to the server, which always updates a file's metadata on each write, including indirect blocks.

begin:vcard
fn:Chuck Lever
n:Lever;Chuck
org:Oracle Corporation;Corporate Architecture: Linux Projects Group adr:;;1015 Granger Avenue;Ann Arbor;MI;48104;USA title:Principal Member of Staff
tel;work:+1 248 614 5091
x-mozilla-html:FALSE
version:2.1
end:vcard

#4 lore raw
In reply to Andrew Morton
Quoted 6 lines
Quoted 6 lines

The test below creates a sparse file and then fills a hole with O_DIRECT. As far as I can tell from reading generic_osync_inode, the filesystem metadata is only forced to disk if i_size changes during the file write. I've tested ext3, xfs and reiserfs and they all skip the commit when filling holes.

Quoted 8 lines
Quoted 4 lines

I would argue that filling holes via O_DIRECT is supposed to commit the metadata required to find those file blocks later. At least on ext3, O_SYNC does force a commit on fill holes (haven't tested others).

I don't think it's a bug. Sure, O_DIRECT is synchronous, but that's because it is, err, direct. Not because it provides extra data-integrity guarantees. If you want those guarantees, use O_SYNC as well.

That makes sense, but how do you explain the committing of the size change without O_SYNC? That seems wrong to me.

This does need to be documented carefully, because a person could easily believe, even subconsciously, that O_DIRECT makes the entire file write direct, and sloppy documentation might actually use words to that effect.

--
Bryan Henderson IBM Almaden Research Center
San Jose CA Filesystems