Question about fallocate
Hi all,
I'm doing some test using a small BTRFS volume on CentOS 6.5 x86_64 (I know that CentOS 6 use an old kernel and btrfs version and I plan to replicate the same test on Fedora 20).
From my understanding, disabling CoW and fallocate a file should give a non-fragmented file. The followind commands show that:
[root@blackhole test]# fallocate test.img -l 1G [root@blackhole test]# sync
[root@blackhole test]# filefrag -v test.img
Filesystem type is: 9123683e
File size of test.img is 1073741824 (262144 blocks, blocksize 4096)
ext logical physical expected length flags
0 0 269312 262144 eof
test.img: 1 extent found
As you can see, I have a single, continuous block stream.
However, write some 4k blocks into the file leads to fragmentation:
[root@blackhole test]# for id in seq 1 32; do dd if=/dev/zero of=test.img bs=4k count=1 seek=$id conv=notrunc,nocreat oflag=direct,sync; done
...
[root@blackhole test]# filefrag -v test.img
Filesystem type is: 9123683e
File size of test.img is 1073741824 (262144 blocks, blocksize 4096)
ext logical physical expected length flags
0 0 269312 1 1 1 269313 1 2 2 531456 269314 31 3 33 269345 531487 262111 eof
test.img: 3 extents found
If I don't use fallocate to reserve space, using a simple dd to write 0s to the target file, the fragmentation do not occour:
[root@blackhole test]# dd if=/dev/zero of=test.img bs=2M count=512
[root@blackhole test]# sync
[root@blackhole test]# for id in seq 1 32; do dd if=/dev/zero of=test.img bs=4k count=1 seek=$id conv=notrunc,nocreat oflag=direct,sync; done
...
[root@blackhole test]# filefrag -v test.img
Filesystem type is: 9123683e
File size of test.img is 1073741824 (262144 blocks, blocksize 4096)
ext logical physical expected length flags
0 0 269312 262144 eof
test.img: 1 extent found
So, my question is: why writing to a fallocated file produce fragmentation, even with CoW disabled?
Regards.
Signature · 5 lines
Danti Gionatan Supporto Tecnico Assyoma S.r.l. - www.assyoma.it email: [email protected] - [email protected] GPG public key ID: FF5F32A8
Gionatan Danti posted on Tue, 01 Jul 2014 12:16:34 +0200 as excerpted:
Quoted 2 lines
So, my question is: why writing to a fallocated file produce fragmentation, even with CoW disabled?
Good question.
But how did you disable COW? The nodatacow mount option? Setting the NOCOW attribute on the file or parent-dir (chattr +C)? Something else?
Because there are caveats to both the mount option and the file attribute methods.
For the nodatacow mount option, the caveat is that to the best of my knowledge, that option is one of several btrfs-specific options that's still whole-filesystem-based. So if you are mounting multiple subvolumes from the same filesystem, nodatacow will toggle together for all of them -- you can't use it for just one.
The possible explanation there is thus that if multiple subvolumes are mounted and not all of them have the same nodatacow option applied, you might not have nodatacow set on that subvolume after all.
For the nocow file attribute case, there's two possibilities. First, it's critical how the file attribute was set. If it was set on a file with existing data, nocow behavior isn't guaranteed. The nocow attribute must be set on the file while it is zero-sized, before it has any data in it. The easiest way to do that is to set the attribute on the directory, such that newly created files (and subdirs) in it inherit the attribute. Nocow doesn't affect the directory itself and thus only determines inheritance.
Second, there's the snapshotting exception. Because a btrfs snapshot locks the existing file data in place with the snapshot, the first modification to a fileblock after a snapshot will force a COW for that block, even on an otherwise nocow file. The nocow attribute remains in effect, however, and further writes to the same block will modify it in- place... until the next snapshot of course. This becomes a real issue for people running automated snapshotting scripts, particularly if they've set it to something extreme like once a minute, since in that case the existing file content is locked in place once a minute, meaning the nocow attribute is effectively worthless for them. The recommendation in that case is to put the nocow files in a dedicated subvolume, since snapshots stop at subvolume boundaries, and to use conventional backup instead of snapshotting for that subvolume.
So the possible explanations for the file attribute case is either that the file attribute wasn't applied correctly (the file already had content when the attribute was set), or that a snapshot intervened between file creation and modification, locking the existing file data in-place and thus forcing a cow for the first post-snapshot modification per-block.
Other than that, I don't know, but it'd be interesting to see if the behavior replicates on a current kernel.
Signature · 3 lines
Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman