copy: adjust fiemap handling of sparse files

PATCH Pádraig Brady 2 participants 2 messages Feb 13, 2011 – Feb 14, 2011 5 messages not in the archive Btrfs file system

lore

Share & tools

lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

Pádraig Brady

author

2011-02-13 23:53 UTC #5 lore raw

Unfortunately, after checking BTRFS I see that fiemap behaves differently to EXT4. IMHO the EXT4 operation seems correct, and gives full info about the structure of a file, which cp for example can use to efficiently and accurately reproduce the structure at the destination.

On EXT4 (on kernel-2.6.35.11-83.fc14.i686) there are no extents returned for holes in a file.

However on btrfs it does return an extent for holes? So with btrfs there is no way to know an extent is allocated but unwritten (zero), so one must read and write all the data, rather than just fallocating the space in the destination.

One can also see this with the following on btrfs:

$ fallocate -l 100000000 unwritten
$ truncate -s 100000000 sparse
$ dd count=3D1000 bs=3D100000 if=3D/dev/zero of=3Dzero

$ filefrag -vs *
=46ilesystem type is: 9123683e
=46ile size of sparse is 100000000 (24415 blocks, blocksize 4096)
 ext logical physical expected length flags
   0       0        0           24415 unwritten,eof
sparse: 1 extent found
=46ile size of unwritten is 100000000 (24415 blocks, blocksize 4096)
 ext logical physical expected length flags
   0       0    68160           12207
   1   12207    92560    80366  12208 eof
unwritten: 2 extents found
=46ile size of zeros is 100000000 (24415 blocks, blocksize 4096)
 ext logical physical expected length flags
   0       0    19360           20678
   1   20678    43760    40037   3737 eof
zeros: 2 extents found

So is this expected?
Has this already been changed to match ext4?

=46or my own reference, I can probably skip performance tests on (older) btrfs by checking filefrag output. Also in cp, if we see an "unwritten extent" we should probably create a hole rather than an empty allocation by default. It's better to decrease file allocation than increase it.

cheers,
P=E1draig.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Jim Meyering

2011-02-14 06:05 UTC #6 lore raw

Quoted 38 lines · P=E1draig Brady wrote:

Unfortunately, after checking BTRFS I see that fiemap behaves differently to EXT4. IMHO the EXT4 operation seems correct, and gives full info about the structure of a file, which cp for example can use to efficiently and accurately reproduce the structure at the destination.
On EXT4 (on kernel-2.6.35.11-83.fc14.i686) there are no extents returned for holes in a file.
However on btrfs it does return an extent for holes? So with btrfs there is no way to know an extent is allocated but unwritten (zero), so one must read and write all the data, rather than just fallocating the space in the destination.
One can also see this with the following on btrfs:
$ fallocate -l 100000000 unwritten
$ truncate -s 100000000 sparse
$ dd count=3D1000 bs=3D100000 if=3D/dev/zero of=3Dzero
$ filefrag -vs *
Filesystem type is: 9123683e
File size of sparse is 100000000 (24415 blocks, blocksize 4096)
 ext logical physical expected length flags
   0       0        0           24415 unwritten,eof
sparse: 1 extent found
File size of unwritten is 100000000 (24415 blocks, blocksize 4096)
 ext logical physical expected length flags
   0       0    68160           12207
   1   12207    92560    80366  12208 eof
unwritten: 2 extents found
File size of zeros is 100000000 (24415 blocks, blocksize 4096)
 ext logical physical expected length flags
   0       0    19360           20678
   1   20678    43760    40037   3737 eof
zeros: 2 extents found

Quoted 1 line

So is this expected?

I heard it's a bug in F14's btrfs.
To test btrfs properly, I had to use rawhide/F15.

Quoted 1 line

Has this already been changed to match ext4?

Yes, with a new-enough kernel.

Quoted 6 lines

For my own reference, I can probably skip performance tests on (older) btrfs by checking filefrag output. Also in cp, if we see an "unwritten extent" we should probably create a hole rather than an empty allocation by default. It's better to decrease file allocation than increase it.

Makes sense.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html