btrfs: defrag: don't try to merge regular extents with preallocated extents
PATCH 1/1 btrfs: defrag: don't try to merge regular extents with preallocated extents
[BUG]
With older kernels (before v5.16), btrfs will defrag preallocated extents. While with newer kernels (v5.16 and newer) btrfs will not defrag preallocated extents, but it will defrag the extent just before the preallocated extent, even it's just a single sector.
This can be exposed by the following small script:
mkfs.btrfs -f $dev > /dev/null
mount $dev $mnt
xfs_io -f -c "pwrite 0 4k" -c sync -c "falloc 4k 16K" $mnt/file xfs_io -c "fiemap -v" $mnt/file
btrfs fi defrag $mnt/file
sync
xfs_io -c "fiemap -v" $mnt/file
The output looks like this on older kernels:
/mnt/btrfs/file:
EXT: FILE-OFFSET BLOCK-RANGE TOTAL FLAGS 0: [0..7]: 26624..26631 8 0x0 1: [8..39]: 26632..26663 32 0x801
/mnt/btrfs/file:
EXT: FILE-OFFSET BLOCK-RANGE TOTAL FLAGS
0: [0..39]: 26664..26703 40 0x1
Which defrags the single sector along with the preallocated extent, and replace them with an regular extent into a new location (caused by data COW).
This wastes most of the data IO just for the preallocated range.
On the other hand, v5.16 is slightly better:
/mnt/btrfs/file:
EXT: FILE-OFFSET BLOCK-RANGE TOTAL FLAGS 0: [0..7]: 26624..26631 8 0x0 1: [8..39]: 26632..26663 32 0x801 /mnt/btrfs/file: EXT: FILE-OFFSET BLOCK-RANGE TOTAL FLAGS 0: [0..7]: 26664..26671 8 0x0 1: [8..39]: 26632..26663 32 0x801
The preallocated range is not defragged, but the sector before it still gets defragged, which has no need for it.
[CAUSE]
One of the function reused by the old and new behavior is defrag_check_next_extent(), it will determine if we should defrag current extent by checking the next one.
It only checks if the next extent is a hole or inlined, but it doesn't check if it's preallocated.
On the other hand, out of the function, both old and new kernel will reject preallocated extents.
Such inconsistent behavior causes above behavior.
[FIX]
- Also check if next extent is preallocated
If so, don't defrag current extent
- Add comments on each case we don't defrag
This will reduce the IO caused by defrag ioctl and autodefrag.
--- fs/btrfs/ioctl.c | 29 +++++++++++++++++++++++------ 1 file changed, 23 insertions(+), 6 deletions(-)
fs/btrfs/ioctl.c +23 -6
@@ -1049,23 +1049,40 @@ static struct extent_map *defrag_lookup_extent(struct inode *inode, u64 start,return em; } +/* +* Return if current extent @em is a good candidate for defrag. +* +* This is done by checking against the next extent after @em. +*/ static bool defrag_check_next_extent (struct inode *inode,struct extent_map *em,bool locked) {struct extent_map *next; -bool ret = true; +bool ret = false;/* this is the last extent */ if (em->start + em->len >=i_size_read (inode)) -return false; +return ret; next =defrag_lookup_extent (inode, em->start + em->len, locked); +/* No next extent or a hole, no way to merge */ if (!next || next->block_start >=EXTENT_MAP_LAST_BYTE ) - ret = false; -else if ((em->block_start + em->block_len == next->block_start) && - (em->block_len >SZ_128K && next->block_len >SZ_128K )) - ret = false; + goto out; +/* Next extent is preallocated, no sense to defrag current extent */ +if (test_bit (EXTENT_FLAG_PREALLOC , &next->flags)) + goto out; + +/* +* Next extent are not only mergable but also adjacent in their +* logical address, normally an excellent candicate, but if they +* are already large enough, then no need to defrag current extent. +*/ +if ((em->block_start + em->block_len == next->block_start) && + (em->block_len >SZ_128K && next->block_len >SZ_128K )) + goto out; + ret = true; +out:free_extent_map (next);return ret; }
Signature · 1 line
2.34.1
Quoted 102 lines · Qu Wenruo
[BUG]
With older kernels (before v5.16), btrfs will defrag preallocated extents. While with newer kernels (v5.16 and newer) btrfs will not defrag preallocated extents, but it will defrag the extent just before the preallocated extent, even it's just a single sector.This can be exposed by the following small script:
mkfs.btrfs -f $dev >
/dev/nullmount $dev $mnt
xfs_io-f -c "pwrite 0 4k" -c sync -c "falloc 4k 16K" $mnt/filexfs_io-c "fiemap -v" $mnt/file
btrfs fi defrag $mnt/file
sync
xfs_io-c "fiemap -v" $mnt/fileThe output looks like this on older kernels:
/mnt/btrfs/file:EXT: FILE-OFFSET BLOCK-RANGE TOTAL FLAGS 0: [0..7]: 26624..26631 8 0x0 1: [8..39]: 26632..26663 32 0x801
/mnt/btrfs/file:
EXT: FILE-OFFSET BLOCK-RANGE TOTAL FLAGS
0: [0..39]: 26664..26703 400x1Which defrags the single sector along with the preallocated extent, and replace them with an regular extent into a new location (caused by data COW).
This wastes most of the data IO just for the preallocated range.On the other hand, v5.16 is slightly better:
/mnt/btrfs/file:EXT: FILE-OFFSET BLOCK-RANGE TOTAL FLAGS 0: [0..7]: 26624..26631 8 0x0 1: [8..39]: 26632..26663 32 0x801 /mnt/btrfs/file: EXT: FILE-OFFSET BLOCK-RANGE TOTAL FLAGS 0: [0..7]: 26664..26671 8 0x0 1: [8..39]: 26632..26663 32 0x801The preallocated range is not defragged, but the sector before it still gets defragged, which has no need for it.
[CAUSE]
One of the function reused by the old and new behavior isdefrag_check_next_extent(), it will determine if we should defrag current extent by checking the next one.It only checks if the next extent is a hole or inlined, but it doesn't check if it's preallocated.
On the other hand, out of the function, both old and new kernel will reject preallocated extents.
Such inconsistent behavior causes above behavior.
[FIX]
- Also check if next extent is preallocated
If so, don't defrag current extent- Add comments on each case we don't defrag
This will reduce the IO caused by defrag ioctl and autodefrag.
Signed-off-by Qu Wenruo <[email protected]>--- fs/btrfs/ioctl.c | 29 +++++++++++++++++++++++------ 1 file changed, 23 insertions(+), 6 deletions(-)fs/btrfs/ioctl.c +8
-4@@ -1049,23 +1049,40 @@ static struct extent_map *defrag_lookup_extent(struct inode *inode, u64 start,return em; } +/* +* Return if current extent @em is a good candidate for defrag. +* +* This is done by checking against the next extent after @em. +*/ static bool defrag_check_next_extent (struct inode *inode,struct extent_map *em,bool locked) {struct extent_map *next; -bool ret = true; +bool ret = false;/* this is the last extent */ if (em->start + em->len >=i_size_read (inode)) -return false; +return ret; next =defrag_lookup_extent (inode, em->start + em->len, locked); +/* No next extent or a hole, no way to merge */ if (!next || next->block_start >=EXTENT_MAP_LAST_BYTE ) - ret = false; - else if ((em->block_start + em->block_len == next->block_start) &&
fs/btrfs/ioctl.c static struct extent_map *defrag_lookup_extent(struct inode *inode, u64 start, · line 1077
Quoted 10 lines
fs/btrfs/ioctl.c +7
-2- (em->block_len > SZ_128K && next->block_len > SZ_128K)) - ret = false; + goto out; +/* Next extent is preallocated, no sense to defrag current extent */ +if (test_bit (EXTENT_FLAG_PREALLOC , &next->flags)) + goto out; + +/* +* Next extent are not only mergable but also adjacent in their
are not -> is not
mergable -> mergeable
their -> its
fs/btrfs/ioctl.c static struct extent_map *defrag_lookup_extent(struct inode *inode, u64 start, · line 1078
Quoted 1 line
fs/btrfs/ioctl.c +1
-0+* logical address, normally an excellent candicate, but if they
candicate -> candidate
fs/btrfs/ioctl.c static struct extent_map *defrag_lookup_extent(struct inode *inode, u64 start, · line 1080
Quoted 2 lines
fs/btrfs/ioctl.c +2
-0+* are already large enough, then no need to defrag current extent. +*/
It still sounds a bit odd to me, maybe:
Next extent is mergeable and its logical address is contiguous with this extent, so normally an excellent candidate, but if this extent or the next one is already large enough, then we don't need to defrag. We use SZ_128K because in case of enabled compression, extents can never be larger than that.
Adding this comment is unrelated to this fix about prealloc extents, but I'm fine with it.
Other than that it looks fine.
Thanks.
Quoted 11 lines
fs/btrfs/ioctl.c +5
-0+if ((em->block_start + em->block_len == next->block_start) && + (em->block_len >SZ_128K && next->block_len >SZ_128K )) + goto out; + ret = true; +out:free_extent_map (next);return ret; }Signature · 1 line
2.34.1
Quoted 127 lines · Filipe Manana
Quoted 102 lines · Qu Wenruo
[BUG]
With older kernels (before v5.16), btrfs will defrag preallocated extents. While with newer kernels (v5.16 and newer) btrfs will not defrag preallocated extents, but it will defrag the extent just before the preallocated extent, even it's just a single sector.This can be exposed by the following small script:
mkfs.btrfs -f $dev >
/dev/nullmount $dev $mnt
xfs_io-f -c "pwrite 0 4k" -c sync -c "falloc 4k 16K" $mnt/filexfs_io-c "fiemap -v" $mnt/file
btrfs fi defrag $mnt/file
sync
xfs_io-c "fiemap -v" $mnt/fileThe output looks like this on older kernels:
/mnt/btrfs/file:EXT: FILE-OFFSET BLOCK-RANGE TOTAL FLAGS 0: [0..7]: 26624..26631 8 0x0 1: [8..39]: 26632..26663 32 0x801 /mnt/btrfs/file: EXT: FILE-OFFSET BLOCK-RANGE TOTAL FLAGS 0: [0..39]: 26664..26703 40 0x1Which defrags the single sector along with the preallocated extent, and replace them with an regular extent into a new location (caused by data COW).
This wastes most of the data IO just for the preallocated range.On the other hand, v5.16 is slightly better:
/mnt/btrfs/file:EXT: FILE-OFFSET BLOCK-RANGE TOTAL FLAGS 0: [0..7]: 26624..26631 8 0x0 1: [8..39]: 26632..26663 32 0x801 /mnt/btrfs/file: EXT: FILE-OFFSET BLOCK-RANGE TOTAL FLAGS 0: [0..7]: 26664..26671 8 0x0 1: [8..39]: 26632..26663 32 0x801The preallocated range is not defragged, but the sector before it still gets defragged, which has no need for it.
[CAUSE]
One of the function reused by the old and new behavior isdefrag_check_next_extent(), it will determine if we should defrag current extent by checking the next one.It only checks if the next extent is a hole or inlined, but it doesn't check if it's preallocated.
On the other hand, out of the function, both old and new kernel will reject preallocated extents.
Such inconsistent behavior causes above behavior.
[FIX]
- Also check if next extent is preallocated
If so, don't defrag current extent- Add comments on each case we don't defrag
This will reduce the IO caused by defrag ioctl and autodefrag.
Signed-off-by Qu Wenruo <[email protected]>--- fs/btrfs/ioctl.c | 29 +++++++++++++++++++++++------ 1 file changed, 23 insertions(+), 6 deletions(-)fs/btrfs/ioctl.c +8
-4@@ -1049,23 +1049,40 @@ static struct extent_map *defrag_lookup_extent(struct inode *inode, u64 start,return em; } +/* +* Return if current extent @em is a good candidate for defrag. +* +* This is done by checking against the next extent after @em. +*/ static bool defrag_check_next_extent (struct inode *inode,struct extent_map *em,bool locked) {struct extent_map *next; -bool ret = true; +bool ret = false;/* this is the last extent */ if (em->start + em->len >=i_size_read (inode)) -return false; +return ret; next =defrag_lookup_extent (inode, em->start + em->len, locked); +/* No next extent or a hole, no way to merge */ if (!next || next->block_start >=EXTENT_MAP_LAST_BYTE ) - ret = false; - else if ((em->block_start + em->block_len == next->block_start) &&fs/btrfs/ioctl.c static struct extent_map *defrag_lookup_extent(struct inode *inode, u64 start, · line 1077
Quoted 10 lines
fs/btrfs/ioctl.c +7
-2- (em->block_len > SZ_128K && next->block_len > SZ_128K)) - ret = false; + goto out; +/* Next extent is preallocated, no sense to defrag current extent */ +if (test_bit (EXTENT_FLAG_PREALLOC , &next->flags)) + goto out; + +/* +* Next extent are not only mergable but also adjacent in their are not -> is not
mergable -> mergeable
their -> itsfs/btrfs/ioctl.c static struct extent_map *defrag_lookup_extent(struct inode *inode, u64 start, · line 1078
Quoted 1 line
fs/btrfs/ioctl.c +1
-0+* logical address, normally an excellent candicate, but if they candicate -> candidate
fs/btrfs/ioctl.c static struct extent_map *defrag_lookup_extent(struct inode *inode, u64 start, · line 1080
Quoted 2 lines
fs/btrfs/ioctl.c +2
-0+* are already large enough, then no need to defrag current extent. +*/ It still sounds a bit odd to me, maybe:
Quoted 5 lines
Next extent is mergeable and its logical address is contiguous with this extent, so normally an excellent candidate, but if this extent or the next one is already large enough, then we don't need to defrag. We use
SZ_128Kbecause in case of enabled compression, extents can never be larger than that.
In fact, I'm a little more concerned of the original condition now.
One thing is, the threshold here, it's hard coded 128K, while the extent_threshold for defrag can be specified by the ioctl caller.
Another thing is, the original condition is using block_start, I'm not sure if we really need to check that.
As long as the next extent is not a hole/preallocated one, we're completely happy to defrag.
In fact, if the disk bytenr/num_bytes of @em is not adjacent to the next extent, it's even better, we can merge it into one extent without an extra seek.
So I tend to change the condition more, like this:
- Skip holes/preallocated
That's already here in the patch
- Skip large extents, using @threshold passed into this function No longer hard coded values, let the defrag caller to have more control, and have more consistent behavior.
- No more check on em::block_start
There are some pros and cons of defragging already physically adjacent file extents:
Pros:
* Reduces the number of extents
Which may be what the defrag users want, to defrag extents caused by small but sequential direct IO.
With reduced number of extents, there is a slight chance to reduce mount time by a little.
Cons:
* Extra IO and no saving in seeking time
So why the existing code checks on the em::block_start is already questionable, as it's not a clear win.
I know this sounds weird especially after I have broken so many defrag code, but I still want to remove the checks, replacing with more reasonable checks, even it means it will change the behavior again.
Thanks,
Qu
Quoted 22 lines
Adding this comment is unrelated to this fix about prealloc extents, but I'm fine with it.
Other than that it looks fine.
Reviewed-by Filipe Manana <[email protected]>Thanks.
Quoted 11 lines
fs/btrfs/ioctl.c +5
-0+if ((em->block_start + em->block_len == next->block_start) && + (em->block_len >SZ_128K && next->block_len >SZ_128K )) + goto out; + ret = true; +out:free_extent_map (next);return ret; }Signature · 1 line
2.34.1