Skip to contentSkip to conversation

btrfs: defrag: don't try to merge regular extents with preallocated extents

PATCH Superseded Qu Wenruo 4 participants 16 messages Jan 26, 2022 – Jan 28, 2022 Btrfs file system

v3 of 4 Superseded Latest is v4 →

lore
Share & tools
lore

Qu Wenruo

author
#0 lore raw

PATCH v3 1/3 btrfs: defrag: don't try to merge regular extents with preallocated extents

[BUG]
With older kernels (before v5.16), btrfs will defrag preallocated extents. While with newer kernels (v5.16 and newer) btrfs will not defrag preallocated extents, but it will defrag the extent just before the preallocated extent, even it's just a single sector.

This can be exposed by the following small script:

mkfs.btrfs -f $dev > /dev/null

mount $dev $mnt
xfs_io -f -c "pwrite 0 4k" -c sync -c "falloc 4k 16K" $mnt/file xfs_io -c "fiemap -v" $mnt/file
btrfs fi defrag $mnt/file
sync
xfs_io -c "fiemap -v" $mnt/file

The output looks like this on older kernels:

/mnt/btrfs/file:

 EXT: FILE-OFFSET      BLOCK-RANGE      TOTAL FLAGS
   0: [0..7]:          26624..26631         8   0x0
   1: [8..39]:         26632..26663        32 0x801

/mnt/btrfs/file:
EXT: FILE-OFFSET BLOCK-RANGE TOTAL FLAGS
0: [0..39]: 26664..26703 40 0x1

Which defrags the single sector along with the preallocated extent, and replace them with an regular extent into a new location (caused by data COW).
This wastes most of the data IO just for the preallocated range.

On the other hand, v5.16 is slightly better:

/mnt/btrfs/file:

 EXT: FILE-OFFSET      BLOCK-RANGE      TOTAL FLAGS
   0: [0..7]:          26624..26631         8   0x0
   1: [8..39]:         26632..26663        32 0x801
/mnt/btrfs/file:
 EXT: FILE-OFFSET      BLOCK-RANGE      TOTAL FLAGS
   0: [0..7]:          26664..26671         8   0x0
   1: [8..39]:         26632..26663        32 0x801

The preallocated range is not defragged, but the sector before it still gets defragged, which has no need for it.

[CAUSE]
One of the function reused by the old and new behavior is defrag_check_next_extent(), it will determine if we should defrag current extent by checking the next one.

It only checks if the next extent is a hole or inlined, but it doesn't check if it's preallocated.

On the other hand, out of the function, both old and new kernel will reject preallocated extents.

Such inconsistent behavior causes above behavior.

[FIX]
- Also check if next extent is preallocated
If so, don't defrag current extent.

- Add comments for each branch why we reject the extent

This will reduce the IO caused by defrag ioctl and autodefrag.

---
Changelog:
v2:
- Use @extent_thresh from caller to replace the harded coded threshold Now caller has full control over the extent threshold value.

- Remove the old ambiguous check based on physical address The original check is too specific, only reject extents which are physically adjacent, AND too large.
Since we have correct size check now, and the physically adjacent check is not always a win.
So remove the old check completely.

v3:
- Split the @extent_thresh and physicall adjacent check into other patches

- Simplify the comment

---
 fs/btrfs/ioctl.c | 20 +++++++++++++-------
 1 file changed, 13 insertions(+), 7 deletions(-)
fs/btrfs/ioctl.c +13 -7
index 91ba2efe9792..0d8bfc716e6b 100644
@@ -1053,19 +1053,25 @@ static bool defrag_check_next_extent(struct inode *inode, struct extent_map *em,
 				     bool locked)
 {
 	struct extent_map *next;
-	bool ret = true;
+	bool ret = false;
 
 	/* this is the last extent */
 	if (em->start + em->len >= i_size_read(inode))
-		return false;
+		return ret;
 
 	next = defrag_lookup_extent(inode, em->start + em->len, locked);
+	/* No more em or hole */
 	if (!next || next->block_start >= EXTENT_MAP_LAST_BYTE)
-		ret = false;
-	else if ((em->block_start + em->block_len == next->block_start) &&
-		 (em->block_len > SZ_128K && next->block_len > SZ_128K))
-		ret = false;
-
+		goto out;
+	/* Preallocated */
+	if (test_bit(EXTENT_FLAG_PREALLOC, &em->flags))
+		goto out;
+	/* Physically adjacent and large enough */
+	if ((em->block_start + em->block_len == next->block_start) &&
+	    (em->block_len > SZ_128K && next->block_len > SZ_128K))
+		goto out;
+	ret = true;
+out:
 	free_extent_map(next);
 	return ret;
 }
Signature · 1 line
2.34.1
author
#1 lore raw

PATCH 2/3 btrfs: defrag: use extent_thresh to replace the hardcoded size limit

In defrag_lookup_extent() we use hardcoded extent size threshold, SZ_128K, other than @extent_thresh in btrfs_defrag_file().

This can lead to some inconsistent behavior, especially the default extent size threshold is 256K.

Fix this by passing @extent_thresh into defrag_check_next_extent() and use that value.

Also, since the extent_thresh check should be applied to all extents, not only physically adjacent extents, move the threshold check into a dedicate if ().

---
 fs/btrfs/ioctl.c | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)
fs/btrfs/ioctl.c +7 -5
index 0d8bfc716e6b..2911df12fc48 100644
@@ -1050,7 +1050,7 @@ static struct extent_map *defrag_lookup_extent(struct inode *inode, u64 start,
 }
 
 static bool defrag_check_next_extent(struct inode *inode, struct extent_map *em,
-				     bool locked)
+				     u32 extent_thresh, bool locked)
 {
 	struct extent_map *next;
 	bool ret = false;
@@ -1066,9 +1066,11 @@ static bool defrag_check_next_extent(struct inode *inode, struct extent_map *em,
 	/* Preallocated */
 	if (test_bit(EXTENT_FLAG_PREALLOC, &em->flags))
 		goto out;
-	/* Physically adjacent and large enough */
-	if ((em->block_start + em->block_len == next->block_start) &&
-	    (em->block_len > SZ_128K && next->block_len > SZ_128K))
+	/* Extent is already large enough */
+	if (next->len >= extent_thresh)
+		goto out;
+	/* Physically adjacent */
+	if ((em->block_start + em->block_len == next->block_start))
 		goto out;
 	ret = true;
 out:
@@ -1231,7 +1233,7 @@ static int defrag_collect_targets(struct btrfs_inode *inode,
 			goto next;
 
 		next_mergeable = defrag_check_next_extent(&inode->vfs_inode, em,
-							  locked);
+							  extent_thresh, locked);
 		if (!next_mergeable) {
 			struct defrag_target_range *last;
 
Signature · 1 line
2.34.1
#2 lore raw

Re: [PATCH 2/3] btrfs: defrag: use extent_thresh to replace the hardcoded size limit

Quoted 32 lines · Qu Wenruo

In defrag_lookup_extent() we use hardcoded extent size threshold, SZ_128K, other than @extent_thresh in btrfs_defrag_file().

This can lead to some inconsistent behavior, especially the default extent size threshold is 256K.

Fix this by passing @extent_thresh into defrag_check_next_extent() and use that value.

Also, since the extent_thresh check should be applied to all extents, not only physically adjacent extents, move the threshold check into a dedicate if ().

---
 fs/btrfs/ioctl.c | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)
fs/btrfs/ioctl.c +1 -1
index 0d8bfc716e6b..2911df12fc48 100644
@@ -1050,7 +1050,7 @@ static struct extent_map *defrag_lookup_extent(struct inode *inode, u64 start,
 }
 
 static bool defrag_check_next_extent(struct inode *inode, struct extent_map *em,
-				     bool locked)
+				     u32 extent_thresh, bool locked)
 {
 	struct extent_map *next;
 	bool ret = false;

fs/btrfs/ioctl.c static bool defrag_check_next_extent(struct inode *inode, struct extent_map *em, · line 1071

Quoted 9 lines
fs/btrfs/ioctl.c +3 -3
@@ -1066,9 +1066,11 @@ static bool defrag_check_next_extent(struct inode *inode, struct extent_map *em,
 	/* Preallocated */
 	if (test_bit(EXTENT_FLAG_PREALLOC, &em->flags))
 		goto out;
-	/* Physically adjacent and large enough */
-	if ((em->block_start + em->block_len == next->block_start) &&
-	    (em->block_len > SZ_128K && next->block_len > SZ_128K))
+	/* Extent is already large enough */
+	if (next->len >= extent_thresh)
+		goto out;

So this will trigger unnecessary rewrites of compressed extents. The SZ_128K is there to deal with compressed extents, it has nothing to do with the threshold passed to the ioctl.

After applying this patchset, if you run a trivial test like this:

#!/bin/bash

DEV=/dev/sdj
MNT=/mnt/sdj

mkfs.btrfs -f $DEV
mount -o compress $DEV $MNT

xfs_io -f -c "pwrite -S 0xab 0 128K" $MNT/foobar sync
# Write to some other file so that the next extent for foobar

   # is not contiguous with the first extent.
   xfs_io -f -c "pwrite 0 128K" $MNT/baz
   sync
   xfs_io -f -c "pwrite -S 0xcd 128K 128K" $MNT/foobar
   sync

echo -e "\n\nTree after creating file:\n\n"
btrfs inspect-internal dump-tree -t 5 $DEV

btrfs filesystem defragment $MNT/foobar
sync

echo -e "\n\nTree after defrag:\n\n"
btrfs inspect-internal dump-tree -t 5 $DEV

umount $MNT

It will result in rewriting the two 128K compressed extents:

(...)
Tree after write and sync:

btrfs-progs v5.12.1
fs tree key (FS_TREE ROOT_ITEM 0)
(...)

	item 7 key (257 INODE_REF 256) itemoff 15797 itemsize 16
		index 2 namelen 6 name: foobar
	item 8 key (257 EXTENT_DATA 0) itemoff 15744 itemsize 53
		generation 6 type 1 (regular)
		extent data disk byte 13631488 nr 4096
		extent data offset 0 nr 131072 ram 131072
		extent compression 1 (zlib)
	item 9 key (257 EXTENT_DATA 131072) itemoff 15691 itemsize 53
		generation 8 type 1 (regular)
		extent data disk byte 14163968 nr 4096
		extent data offset 0 nr 131072 ram 131072
		extent compression 1 (zlib)

(...)

Tree after defrag:

btrfs-progs v5.12.1
fs tree key (FS_TREE ROOT_ITEM 0)
(...)

	item 7 key (257 INODE_REF 256) itemoff 15797 itemsize 16
		index 2 namelen 6 name: foobar
	item 8 key (257 EXTENT_DATA 0) itemoff 15744 itemsize 53
		generation 9 type 1 (regular)
		extent data disk byte 14430208 nr 4096
		extent data offset 0 nr 131072 ram 131072
		extent compression 1 (zlib)
	item 9 key (257 EXTENT_DATA 131072) itemoff 15691 itemsize 53
		generation 9 type 1 (regular)
		extent data disk byte 13635584 nr 4096
		extent data offset 0 nr 131072 ram 131072
		extent compression 1 (zlib)

In other words, a waste of IO and CPU time.

So it needs to check if we are dealing with compressed extents, and if so, skip either of them has a size of SZ_128K (and changelog updated).

Thanks.

Quoted 17 lines
fs/btrfs/ioctl.c +3 -1
+	/* Physically adjacent */
+	if ((em->block_start + em->block_len == next->block_start))
 		goto out;
 	ret = true;
 out:
@@ -1231,7 +1233,7 @@ static int defrag_collect_targets(struct btrfs_inode *inode,
 			goto next;
 
 		next_mergeable = defrag_check_next_extent(&inode->vfs_inode, em,
-							  locked);
+							  extent_thresh, locked);
 		if (!next_mergeable) {
 			struct defrag_target_range *last;
 
Signature · 1 line
2.34.1
author
#3 lore raw

Re: [PATCH 2/3] btrfs: defrag: use extent_thresh to replace the hardcoded size limit

Quoted 38 lines · Filipe Manana
Quoted 37 lines · Qu Wenruo

In defrag_lookup_extent() we use hardcoded extent size threshold, SZ_128K, other than @extent_thresh in btrfs_defrag_file().

This can lead to some inconsistent behavior, especially the default extent size threshold is 256K.

Fix this by passing @extent_thresh into defrag_check_next_extent() and use that value.

Also, since the extent_thresh check should be applied to all extents, not only physically adjacent extents, move the threshold check into a dedicate if ().

---
  fs/btrfs/ioctl.c | 12 +++++++-----
  1 file changed, 7 insertions(+), 5 deletions(-)
fs/btrfs/ioctl.c +1 -3
index 0d8bfc716e6b..2911df12fc48 100644
@@ -1050,7 +1050,7 @@ static struct extent_map *defrag_lookup_extent(struct inode *inode, u64 start,
  }
 
  static bool defrag_check_next_extent(struct inode *inode, struct extent_map *em,
-				     bool locked)
+				     u32 extent_thresh, bool locked)
  {
  	struct extent_map *next;
  	bool ret = false;
@@ -1066,9 +1066,11 @@ static bool defrag_check_next_extent(struct inode *inode, struct extent_map *em,
  	/* Preallocated */
  	if (test_bit(EXTENT_FLAG_PREALLOC, &em->flags))
  		goto out;
-	/* Physically adjacent and large enough */
-	if ((em->block_start + em->block_len == next->block_start) &&
Quoted 8 lines

fs/btrfs/ioctl.c static bool defrag_check_next_extent(struct inode *inode, struct extent_map *em, · line 1071

Quoted 4 lines
fs/btrfs/ioctl.c +3 -1
-	    (em->block_len > SZ_128K && next->block_len > SZ_128K))
+	/* Extent is already large enough */
+	if (next->len >= extent_thresh)
+		goto out;

So this will trigger unnecessary rewrites of compressed extents. The SZ_128K is there to deal with compressed extents, it has nothing to do with the threshold passed to the ioctl.

Then there is still something wrong.

The original check will only reject it when both conditions are met.

So based on your script, I can still find a way to defrag the extents, with or without this modification:

	mkfs.btrfs -f $DEV
	mount -o compress $DEV $MNT
	
	xfs_io -f -c "pwrite -S 0xab 0 128K" $MNT/file1
	sync
	xfs_io -f -c "pwrite -S 0xab 0 128K" $MNT/file2
	sync
	xfs_io -f -c "pwrite -S 0xab 128K 128K" $MNT/file1
	sync

	echo "=== file1 before defrag ==="
	xfs_io -f -c "fiemap -v" $MNT/file1
	echo "=== file1 after defrag ==="
	btrfs fi defrag $MNT/file1
	sync
	xfs_io -f -c "fiemap -v" $MNT/file1

The output looks like this:

=== before ===
/mnt/btrfs/file1:

 EXT: FILE-OFFSET      BLOCK-RANGE      TOTAL FLAGS
   0: [0..255]:        26624..26879       256   0x8
   1: [256..511]:      26640..26895       256   0x9
=== after ===
/mnt/btrfs/file1:
 EXT: FILE-OFFSET      BLOCK-RANGE      TOTAL FLAGS
   0: [0..255]:        26648..26903       256   0x8
   1: [256..511]:      26656..26911       256   0x9

No matter if the patch is applied, the result is the same.

So thank you very much for finding another case we're not handling well...

BTW, if the check is want to reject adjacent non-compressed extent, the original one is still incorrect, we can have extents smaller than 128K and is still uncompressed.

So what we really want is to reject physically adjacent, non-compressed extents?

Thanks,
Qu

Quoted 96 lines

After applying this patchset, if you run a trivial test like this:

#!/bin/bash

DEV=/dev/sdj
MNT=/mnt/sdj

mkfs.btrfs -f $DEV
mount -o compress $DEV $MNT

xfs_io -f -c "pwrite -S 0xab 0 128K" $MNT/foobar sync
# Write to some other file so that the next extent for foobar # is not contiguous with the first extent.
xfs_io -f -c "pwrite 0 128K" $MNT/baz
sync
xfs_io -f -c "pwrite -S 0xcd 128K 128K" $MNT/foobar sync

echo -e "\n\nTree after creating file:\n\n"
btrfs inspect-internal dump-tree -t 5 $DEV

btrfs filesystem defragment $MNT/foobar
sync

echo -e "\n\nTree after defrag:\n\n"
btrfs inspect-internal dump-tree -t 5 $DEV

umount $MNT

It will result in rewriting the two 128K compressed extents:

(...)
Tree after write and sync:

btrfs-progs v5.12.1
fs tree key (FS_TREE ROOT_ITEM 0)
(...)

	item 7 key (257 INODE_REF 256) itemoff 15797 itemsize 16
		index 2 namelen 6 name: foobar
	item 8 key (257 EXTENT_DATA 0) itemoff 15744 itemsize 53
		generation 6 type 1 (regular)
		extent data disk byte 13631488 nr 4096
		extent data offset 0 nr 131072 ram 131072
		extent compression 1 (zlib)
	item 9 key (257 EXTENT_DATA 131072) itemoff 15691 itemsize 53
		generation 8 type 1 (regular)
		extent data disk byte 14163968 nr 4096
		extent data offset 0 nr 131072 ram 131072
		extent compression 1 (zlib)

(...)

Tree after defrag:

btrfs-progs v5.12.1
fs tree key (FS_TREE ROOT_ITEM 0)
(...)

	item 7 key (257 INODE_REF 256) itemoff 15797 itemsize 16
		index 2 namelen 6 name: foobar
	item 8 key (257 EXTENT_DATA 0) itemoff 15744 itemsize 53
		generation 9 type 1 (regular)
		extent data disk byte 14430208 nr 4096
		extent data offset 0 nr 131072 ram 131072
		extent compression 1 (zlib)
	item 9 key (257 EXTENT_DATA 131072) itemoff 15691 itemsize 53
		generation 9 type 1 (regular)
		extent data disk byte 13635584 nr 4096
		extent data offset 0 nr 131072 ram 131072
		extent compression 1 (zlib)

In other words, a waste of IO and CPU time.

So it needs to check if we are dealing with compressed extents, and if so, skip either of them has a size of SZ_128K (and changelog updated).

Thanks.

Quoted 17 lines
fs/btrfs/ioctl.c +3 -1
+	/* Physically adjacent */
+	if ((em->block_start + em->block_len == next->block_start))
  		goto out;
  	ret = true;
  out:
@@ -1231,7 +1233,7 @@ static int defrag_collect_targets(struct btrfs_inode *inode,
  			goto next;
 
  		next_mergeable = defrag_check_next_extent(&inode->vfs_inode, em,
-							  locked);
+							  extent_thresh, locked);
  		if (!next_mergeable) {
  			struct defrag_target_range *last;
 
Signature · 1 line
2.34.1
#4 lore raw

Re: [PATCH 2/3] btrfs: defrag: use extent_thresh to replace the hardcoded size limit

Quoted 51 lines · Qu Wenruo
Quoted 38 lines · Filipe Manana
Quoted 37 lines · Qu Wenruo

In defrag_lookup_extent() we use hardcoded extent size threshold, SZ_128K, other than @extent_thresh in btrfs_defrag_file().

This can lead to some inconsistent behavior, especially the default extent size threshold is 256K.

Fix this by passing @extent_thresh into defrag_check_next_extent() and use that value.

Also, since the extent_thresh check should be applied to all extents, not only physically adjacent extents, move the threshold check into a dedicate if ().

---
  fs/btrfs/ioctl.c | 12 +++++++-----
  1 file changed, 7 insertions(+), 5 deletions(-)
fs/btrfs/ioctl.c +1 -3
index 0d8bfc716e6b..2911df12fc48 100644
@@ -1050,7 +1050,7 @@ static struct extent_map *defrag_lookup_extent(struct inode *inode, u64 start,
  }
 
  static bool defrag_check_next_extent(struct inode *inode, struct extent_map *em,
-                                 bool locked)
+                                 u32 extent_thresh, bool locked)
  {
     struct extent_map *next;
     bool ret = false;
@@ -1066,9 +1066,11 @@ static bool defrag_check_next_extent(struct inode *inode, struct extent_map *em,
     /* Preallocated */
     if (test_bit(EXTENT_FLAG_PREALLOC, &em->flags))
             goto out;
-    /* Physically adjacent and large enough */
-    if ((em->block_start + em->block_len == next->block_start) &&
Quoted 8 lines

fs/btrfs/ioctl.c static bool defrag_check_next_extent(struct inode *inode, struct extent_map *em, · line 1071

Quoted 4 lines
fs/btrfs/ioctl.c +3 -1
-        (em->block_len > SZ_128K && next->block_len > SZ_128K))
+    /* Extent is already large enough */
+    if (next->len >= extent_thresh)
+            goto out;

So this will trigger unnecessary rewrites of compressed extents. The SZ_128K is there to deal with compressed extents, it has nothing to do with the threshold passed to the ioctl.

Quoted 6 lines

Then there is still something wrong.

The original check will only reject it when both conditions are met.

So based on your script, I can still find a way to defrag the extents, with or without this modification:

Right, without the intermediary write to file "baz", this patchset brings a regression in regards to
compressed extents - when they are adjacent, which is typically the case when doing large writes,
as they'll create multiple extents covering consecutive 128K ranges.

With the write to file "baz", as I pasted it, it happens before and after the patchset.

Quoted 31 lines
        mkfs.btrfs -f $DEV
        mount -o compress $DEV $MNT

        xfs_io -f -c "pwrite -S 0xab 0 128K" $MNT/file1
        sync
        xfs_io -f -c "pwrite -S 0xab 0 128K" $MNT/file2
        sync
        xfs_io -f -c "pwrite -S 0xab 128K 128K" $MNT/file1
        sync

        echo "=== file1 before defrag ==="
        xfs_io -f -c "fiemap -v" $MNT/file1
        echo "=== file1 after defrag ==="
        btrfs fi defrag $MNT/file1
        sync
        xfs_io -f -c "fiemap -v" $MNT/file1

The output looks like this:

=== before ===
/mnt/btrfs/file1:

  EXT: FILE-OFFSET      BLOCK-RANGE      TOTAL FLAGS
    0: [0..255]:        26624..26879       256   0x8
    1: [256..511]:      26640..26895       256   0x9
=== after ===
/mnt/btrfs/file1:
  EXT: FILE-OFFSET      BLOCK-RANGE      TOTAL FLAGS
    0: [0..255]:        26648..26903       256   0x8
    1: [256..511]:      26656..26911       256   0x9
Quoted 1 line

No matter if the patch is applied, the result is the same.

Yes, explained above.

Quoted 10 lines

So thank you very much for finding another case we're not handling well...

BTW, if the check is want to reject adjacent non-compressed extent, the original one is still incorrect, we can have extents smaller than 128K and is still uncompressed.

So what we really want is to reject physically adjacent, non-compressed extents?

We want to avoid doing work that does nothing.
If 2 consecutive extents are compressed and at least one is already 128K, then it's a waste of time, IO and CPU.

And that's a fairly common scenario. Do a one megabyte write for example, then after writeback we end up with several 128K extents with compression.
In that case defrag should do nothing for the whole range.

Quoted 100 lines

Thanks,
Qu

Quoted 96 lines

After applying this patchset, if you run a trivial test like this:

#!/bin/bash

DEV=/dev/sdj
MNT=/mnt/sdj

mkfs.btrfs -f $DEV
mount -o compress $DEV $MNT

xfs_io -f -c "pwrite -S 0xab 0 128K" $MNT/foobar sync
# Write to some other file so that the next extent for foobar # is not contiguous with the first extent.
xfs_io -f -c "pwrite 0 128K" $MNT/baz
sync
xfs_io -f -c "pwrite -S 0xcd 128K 128K" $MNT/foobar sync

echo -e "\n\nTree after creating file:\n\n"
btrfs inspect-internal dump-tree -t 5 $DEV

btrfs filesystem defragment $MNT/foobar
sync

echo -e "\n\nTree after defrag:\n\n"
btrfs inspect-internal dump-tree -t 5 $DEV

umount $MNT

It will result in rewriting the two 128K compressed extents:

(...)
Tree after write and sync:

btrfs-progs v5.12.1
fs tree key (FS_TREE ROOT_ITEM 0)
(...)

      item 7 key (257 INODE_REF 256) itemoff 15797 itemsize 16
              index 2 namelen 6 name: foobar
      item 8 key (257 EXTENT_DATA 0) itemoff 15744 itemsize 53
              generation 6 type 1 (regular)
              extent data disk byte 13631488 nr 4096
              extent data offset 0 nr 131072 ram 131072
              extent compression 1 (zlib)
      item 9 key (257 EXTENT_DATA 131072) itemoff 15691 itemsize 53
              generation 8 type 1 (regular)
              extent data disk byte 14163968 nr 4096
              extent data offset 0 nr 131072 ram 131072
              extent compression 1 (zlib)

(...)

Tree after defrag:

btrfs-progs v5.12.1
fs tree key (FS_TREE ROOT_ITEM 0)
(...)

      item 7 key (257 INODE_REF 256) itemoff 15797 itemsize 16
              index 2 namelen 6 name: foobar
      item 8 key (257 EXTENT_DATA 0) itemoff 15744 itemsize 53
              generation 9 type 1 (regular)
              extent data disk byte 14430208 nr 4096
              extent data offset 0 nr 131072 ram 131072
              extent compression 1 (zlib)
      item 9 key (257 EXTENT_DATA 131072) itemoff 15691 itemsize 53
              generation 9 type 1 (regular)
              extent data disk byte 13635584 nr 4096
              extent data offset 0 nr 131072 ram 131072
              extent compression 1 (zlib)

In other words, a waste of IO and CPU time.

So it needs to check if we are dealing with compressed extents, and if so, skip either of them has a size of SZ_128K (and changelog updated).

Thanks.

Quoted 17 lines
fs/btrfs/ioctl.c +3 -2
+    /* Physically adjacent */
+    if ((em->block_start + em->block_len == next->block_start))
             goto out;
     ret = true;
  out:
@@ -1231,7 +1233,7 @@ static int defrag_collect_targets(struct btrfs_inode *inode,
                     goto next;
 
             next_mergeable = defrag_check_next_extent(&inode->vfs_inode, em,
-                                                      locked);
+                                                      extent_thresh, locked);
             if (!next_mergeable) {
                     struct defrag_target_range *last;
 
--

2.34.1

#5 lore raw

Re: [PATCH 2/3] btrfs: defrag: use extent_thresh to replace the hardcoded size limit

Quoted 111 lines · Filipe Manana
Quoted 51 lines · Qu Wenruo
Quoted 38 lines · Filipe Manana
Quoted 37 lines · Qu Wenruo

In defrag_lookup_extent() we use hardcoded extent size threshold, SZ_128K, other than @extent_thresh in btrfs_defrag_file().

This can lead to some inconsistent behavior, especially the default extent size threshold is 256K.

Fix this by passing @extent_thresh into defrag_check_next_extent() and use that value.

Also, since the extent_thresh check should be applied to all extents, not only physically adjacent extents, move the threshold check into a dedicate if ().

---
   fs/btrfs/ioctl.c | 12 +++++++-----
   1 file changed, 7 insertions(+), 5 deletions(-)
fs/btrfs/ioctl.c +1 -3
index 0d8bfc716e6b..2911df12fc48 100644
@@ -1050,7 +1050,7 @@ static struct extent_map *defrag_lookup_extent(struct inode *inode, u64 start,
   }
 
   static bool defrag_check_next_extent(struct inode *inode, struct extent_map *em,
-                                 bool locked)
+                                 u32 extent_thresh, bool locked)
   {
      struct extent_map *next;
      bool ret = false;
@@ -1066,9 +1066,11 @@ static bool defrag_check_next_extent(struct inode *inode, struct extent_map *em,
      /* Preallocated */
      if (test_bit(EXTENT_FLAG_PREALLOC, &em->flags))
              goto out;
-    /* Physically adjacent and large enough */
-    if ((em->block_start + em->block_len == next->block_start) &&
Quoted 8 lines

fs/btrfs/ioctl.c static bool defrag_check_next_extent(struct inode *inode, struct extent_map *em, · line 1071

Quoted 4 lines
fs/btrfs/ioctl.c +3 -1
-        (em->block_len > SZ_128K && next->block_len > SZ_128K))
+    /* Extent is already large enough */
+    if (next->len >= extent_thresh)
+            goto out;

So this will trigger unnecessary rewrites of compressed extents. The SZ_128K is there to deal with compressed extents, it has nothing to do with the threshold passed to the ioctl.

Quoted 6 lines

Then there is still something wrong.

The original check will only reject it when both conditions are met.

So based on your script, I can still find a way to defrag the extents, with or without this modification:

Right, without the intermediary write to file "baz", this patchset brings a regression in regards to
compressed extents - when they are adjacent, which is typically the case when doing large writes,
as they'll create multiple extents covering consecutive 128K ranges.

With the write to file "baz", as I pasted it, it happens before and after the patchset.

Quoted 31 lines
         mkfs.btrfs -f $DEV
         mount -o compress $DEV $MNT

         xfs_io -f -c "pwrite -S 0xab 0 128K" $MNT/file1
         sync
         xfs_io -f -c "pwrite -S 0xab 0 128K" $MNT/file2
         sync
         xfs_io -f -c "pwrite -S 0xab 128K 128K" $MNT/file1
         sync

         echo "=== file1 before defrag ==="
         xfs_io -f -c "fiemap -v" $MNT/file1
         echo "=== file1 after defrag ==="
         btrfs fi defrag $MNT/file1
         sync
         xfs_io -f -c "fiemap -v" $MNT/file1

The output looks like this:

=== before ===
/mnt/btrfs/file1:

   EXT: FILE-OFFSET      BLOCK-RANGE      TOTAL FLAGS
     0: [0..255]:        26624..26879       256   0x8
     1: [256..511]:      26640..26895       256   0x9
=== after ===
/mnt/btrfs/file1:
   EXT: FILE-OFFSET      BLOCK-RANGE      TOTAL FLAGS
     0: [0..255]:        26648..26903       256   0x8
     1: [256..511]:      26656..26911       256   0x9
Quoted 1 line

No matter if the patch is applied, the result is the same.

Yes, explained above.

Quoted 8 lines

So thank you very much for finding another case we're not handling well...

BTW, if the check is want to reject adjacent non-compressed extent, the original one is still incorrect, we can have extents smaller than 128K and is still uncompressed.

Quoted 6 lines
Quoted 2 lines

So what we really want is to reject physically adjacent, non-compressed extents?

We want to avoid doing work that does nothing.
If 2 consecutive extents are compressed and at least one is already 128K, then it's a waste of time, IO and CPU.

So can we define the behavior like this?

If the extent is already at its max capacity (compressed 128K,
non-compressed 128M), we don't defrag it.

This also means, we need to do the same check in defrag_collect_targets() to avoid defragging such extent.

Thanks,
Qu

Quoted 107 lines

And that's a fairly common scenario. Do a one megabyte write for example, then after writeback we end up with several 128K extents with compression.
In that case defrag should do nothing for the whole range.

Quoted 100 lines

Thanks,
Qu

Quoted 96 lines

After applying this patchset, if you run a trivial test like this:

#!/bin/bash

DEV=/dev/sdj
MNT=/mnt/sdj

mkfs.btrfs -f $DEV
mount -o compress $DEV $MNT

xfs_io -f -c "pwrite -S 0xab 0 128K" $MNT/foobar sync
# Write to some other file so that the next extent for foobar # is not contiguous with the first extent.
xfs_io -f -c "pwrite 0 128K" $MNT/baz
sync
xfs_io -f -c "pwrite -S 0xcd 128K 128K" $MNT/foobar sync

echo -e "\n\nTree after creating file:\n\n"
btrfs inspect-internal dump-tree -t 5 $DEV

btrfs filesystem defragment $MNT/foobar
sync

echo -e "\n\nTree after defrag:\n\n"
btrfs inspect-internal dump-tree -t 5 $DEV

umount $MNT

It will result in rewriting the two 128K compressed extents:

(...)
Tree after write and sync:

btrfs-progs v5.12.1
fs tree key (FS_TREE ROOT_ITEM 0)
(...)

       item 7 key (257 INODE_REF 256) itemoff 15797 itemsize 16
               index 2 namelen 6 name: foobar
       item 8 key (257 EXTENT_DATA 0) itemoff 15744 itemsize 53
               generation 6 type 1 (regular)
               extent data disk byte 13631488 nr 4096
               extent data offset 0 nr 131072 ram 131072
               extent compression 1 (zlib)
       item 9 key (257 EXTENT_DATA 131072) itemoff 15691 itemsize 53
               generation 8 type 1 (regular)
               extent data disk byte 14163968 nr 4096
               extent data offset 0 nr 131072 ram 131072
               extent compression 1 (zlib)

(...)

Tree after defrag:

btrfs-progs v5.12.1
fs tree key (FS_TREE ROOT_ITEM 0)
(...)

       item 7 key (257 INODE_REF 256) itemoff 15797 itemsize 16
               index 2 namelen 6 name: foobar
       item 8 key (257 EXTENT_DATA 0) itemoff 15744 itemsize 53
               generation 9 type 1 (regular)
               extent data disk byte 14430208 nr 4096
               extent data offset 0 nr 131072 ram 131072
               extent compression 1 (zlib)
       item 9 key (257 EXTENT_DATA 131072) itemoff 15691 itemsize 53
               generation 9 type 1 (regular)
               extent data disk byte 13635584 nr 4096
               extent data offset 0 nr 131072 ram 131072
               extent compression 1 (zlib)

In other words, a waste of IO and CPU time.

So it needs to check if we are dealing with compressed extents, and if so, skip either of them has a size of SZ_128K (and changelog updated).

Thanks.

Quoted 17 lines
fs/btrfs/ioctl.c +3 -2
+    /* Physically adjacent */
+    if ((em->block_start + em->block_len == next->block_start))
              goto out;
      ret = true;
   out:
@@ -1231,7 +1233,7 @@ static int defrag_collect_targets(struct btrfs_inode *inode,
                      goto next;
 
              next_mergeable = defrag_check_next_extent(&inode->vfs_inode, em,
-                                                      locked);
+                                                      extent_thresh, locked);
              if (!next_mergeable) {
                      struct defrag_target_range *last;
 
--

2.34.1

#6 lore raw

Re: [PATCH 2/3] btrfs: defrag: use extent_thresh to replace the hardcoded size limit

Quoted 118 lines · Qu Wenruo
Quoted 114 lines · Filipe Manana
Quoted 51 lines · Qu Wenruo
Quoted 38 lines · Filipe Manana
Quoted 37 lines · Qu Wenruo

In defrag_lookup_extent() we use hardcoded extent size threshold, SZ_128K, other than @extent_thresh in btrfs_defrag_file().

This can lead to some inconsistent behavior, especially the default extent size threshold is 256K.

Fix this by passing @extent_thresh into defrag_check_next_extent() and use that value.

Also, since the extent_thresh check should be applied to all extents, not only physically adjacent extents, move the threshold check into a dedicate if ().

---
   fs/btrfs/ioctl.c | 12 +++++++-----
   1 file changed, 7 insertions(+), 5 deletions(-)
fs/btrfs/ioctl.c +1 -3
index 0d8bfc716e6b..2911df12fc48 100644
@@ -1050,7 +1050,7 @@ static struct extent_map *defrag_lookup_extent(struct inode *inode, u64 start,
   }
 
   static bool defrag_check_next_extent(struct inode *inode, struct extent_map *em,
-                                 bool locked)
+                                 u32 extent_thresh, bool locked)
   {
      struct extent_map *next;
      bool ret = false;
@@ -1066,9 +1066,11 @@ static bool defrag_check_next_extent(struct inode *inode, struct extent_map *em,
      /* Preallocated */
      if (test_bit(EXTENT_FLAG_PREALLOC, &em->flags))
              goto out;
-    /* Physically adjacent and large enough */
-    if ((em->block_start + em->block_len == next->block_start) &&
Quoted 8 lines

fs/btrfs/ioctl.c static bool defrag_check_next_extent(struct inode *inode, struct extent_map *em, · line 1071

Quoted 4 lines
fs/btrfs/ioctl.c +3 -1
-        (em->block_len > SZ_128K && next->block_len > SZ_128K))
+    /* Extent is already large enough */
+    if (next->len >= extent_thresh)
+            goto out;

So this will trigger unnecessary rewrites of compressed extents. The SZ_128K is there to deal with compressed extents, it has nothing to do with the threshold passed to the ioctl.

Quoted 6 lines

Then there is still something wrong.

The original check will only reject it when both conditions are met.

So based on your script, I can still find a way to defrag the extents, with or without this modification:

Right, without the intermediary write to file "baz", this patchset brings a regression in regards to
compressed extents - when they are adjacent, which is typically the case when doing large writes,
as they'll create multiple extents covering consecutive 128K ranges.

With the write to file "baz", as I pasted it, it happens before and after the patchset.

Quoted 31 lines
         mkfs.btrfs -f $DEV
         mount -o compress $DEV $MNT

         xfs_io -f -c "pwrite -S 0xab 0 128K" $MNT/file1
         sync
         xfs_io -f -c "pwrite -S 0xab 0 128K" $MNT/file2
         sync
         xfs_io -f -c "pwrite -S 0xab 128K 128K" $MNT/file1
         sync

         echo "=== file1 before defrag ==="
         xfs_io -f -c "fiemap -v" $MNT/file1
         echo "=== file1 after defrag ==="
         btrfs fi defrag $MNT/file1
         sync
         xfs_io -f -c "fiemap -v" $MNT/file1

The output looks like this:

=== before ===
/mnt/btrfs/file1:

   EXT: FILE-OFFSET      BLOCK-RANGE      TOTAL FLAGS
     0: [0..255]:        26624..26879       256   0x8
     1: [256..511]:      26640..26895       256   0x9
=== after ===
/mnt/btrfs/file1:
   EXT: FILE-OFFSET      BLOCK-RANGE      TOTAL FLAGS
     0: [0..255]:        26648..26903       256   0x8
     1: [256..511]:      26656..26911       256   0x9
Quoted 1 line

No matter if the patch is applied, the result is the same.

Yes, explained above.

Quoted 10 lines

So thank you very much for finding another case we're not handling well...

BTW, if the check is want to reject adjacent non-compressed extent, the original one is still incorrect, we can have extents smaller than 128K and is still uncompressed.

So what we really want is to reject physically adjacent, non-compressed extents?

Quoted 8 lines
Quoted 3 lines

We want to avoid doing work that does nothing.
If 2 consecutive extents are compressed and at least one is already 128K, then it's a waste of time, IO and CPU.

So can we define the behavior like this?

If the extent is already at its max capacity (compressed 128K,
non-compressed 128M), we don't defrag it.

My previous suggestion was: if one of the extents is compressed and its size is 128K, don't include it for defrag.

There's probably other cases to think about: 1 compressed extent representing 100K of data, followed by another compressed extent representing 64K of data for example.
In that case using both for defrag will still result in 2 extents, 1 for 128K of data and another for 36K of data - still not worth it to defrag them, we end up with 2 extents, just different sizes.

At the very least we should not regress on what we did not defrag before:

2 extents with physically contiguous ranges, representing 128K of data each, both compressed.

Which is a very common case.

Thanks.

Quoted 115 lines

This also means, we need to do the same check in defrag_collect_targets() to avoid defragging such extent.

Thanks,
Qu

Quoted 107 lines

And that's a fairly common scenario. Do a one megabyte write for example, then after writeback we end up with several 128K extents with compression.
In that case defrag should do nothing for the whole range.

Quoted 100 lines

Thanks,
Qu

Quoted 96 lines

After applying this patchset, if you run a trivial test like this:

#!/bin/bash

DEV=/dev/sdj
MNT=/mnt/sdj

mkfs.btrfs -f $DEV
mount -o compress $DEV $MNT

xfs_io -f -c "pwrite -S 0xab 0 128K" $MNT/foobar sync
# Write to some other file so that the next extent for foobar # is not contiguous with the first extent.
xfs_io -f -c "pwrite 0 128K" $MNT/baz
sync
xfs_io -f -c "pwrite -S 0xcd 128K 128K" $MNT/foobar sync

echo -e "\n\nTree after creating file:\n\n"
btrfs inspect-internal dump-tree -t 5 $DEV

btrfs filesystem defragment $MNT/foobar
sync

echo -e "\n\nTree after defrag:\n\n"
btrfs inspect-internal dump-tree -t 5 $DEV

umount $MNT

It will result in rewriting the two 128K compressed extents:

(...)
Tree after write and sync:

btrfs-progs v5.12.1
fs tree key (FS_TREE ROOT_ITEM 0)
(...)

       item 7 key (257 INODE_REF 256) itemoff 15797 itemsize 16
               index 2 namelen 6 name: foobar
       item 8 key (257 EXTENT_DATA 0) itemoff 15744 itemsize 53
               generation 6 type 1 (regular)
               extent data disk byte 13631488 nr 4096
               extent data offset 0 nr 131072 ram 131072
               extent compression 1 (zlib)
       item 9 key (257 EXTENT_DATA 131072) itemoff 15691 itemsize 53
               generation 8 type 1 (regular)
               extent data disk byte 14163968 nr 4096
               extent data offset 0 nr 131072 ram 131072
               extent compression 1 (zlib)

(...)

Tree after defrag:

btrfs-progs v5.12.1
fs tree key (FS_TREE ROOT_ITEM 0)
(...)

       item 7 key (257 INODE_REF 256) itemoff 15797 itemsize 16
               index 2 namelen 6 name: foobar
       item 8 key (257 EXTENT_DATA 0) itemoff 15744 itemsize 53
               generation 9 type 1 (regular)
               extent data disk byte 14430208 nr 4096
               extent data offset 0 nr 131072 ram 131072
               extent compression 1 (zlib)
       item 9 key (257 EXTENT_DATA 131072) itemoff 15691 itemsize 53
               generation 9 type 1 (regular)
               extent data disk byte 13635584 nr 4096
               extent data offset 0 nr 131072 ram 131072
               extent compression 1 (zlib)

In other words, a waste of IO and CPU time.

So it needs to check if we are dealing with compressed extents, and if so, skip either of them has a size of SZ_128K (and changelog updated).

Thanks.

Quoted 17 lines
fs/btrfs/ioctl.c +3 -2
+    /* Physically adjacent */
+    if ((em->block_start + em->block_len == next->block_start))
              goto out;
      ret = true;
   out:
@@ -1231,7 +1233,7 @@ static int defrag_collect_targets(struct btrfs_inode *inode,
                      goto next;
 
              next_mergeable = defrag_check_next_extent(&inode->vfs_inode, em,
-                                                      locked);
+                                                      extent_thresh, locked);
              if (!next_mergeable) {
                      struct defrag_target_range *last;
 
--

2.34.1

#7 lore raw

Re: [PATCH 2/3] btrfs: defrag: use extent_thresh to replace the hardcoded size limit

Quoted 123 lines · Filipe Manana
Quoted 122 lines · Qu Wenruo
Quoted 117 lines · Filipe Manana
Quoted 51 lines · Qu Wenruo
Quoted 38 lines · Filipe Manana
Quoted 37 lines · Qu Wenruo

In defrag_lookup_extent() we use hardcoded extent size threshold, SZ_128K, other than @extent_thresh in btrfs_defrag_file().

This can lead to some inconsistent behavior, especially the default extent size threshold is 256K.

Fix this by passing @extent_thresh into defrag_check_next_extent() and use that value.

Also, since the extent_thresh check should be applied to all extents, not only physically adjacent extents, move the threshold check into a dedicate if ().

---
    fs/btrfs/ioctl.c | 12 +++++++-----
    1 file changed, 7 insertions(+), 5 deletions(-)
fs/btrfs/ioctl.c +1 -3
index 0d8bfc716e6b..2911df12fc48 100644
@@ -1050,7 +1050,7 @@ static struct extent_map *defrag_lookup_extent(struct inode *inode, u64 start,
    }
 
    static bool defrag_check_next_extent(struct inode *inode, struct extent_map *em,
-                                 bool locked)
+                                 u32 extent_thresh, bool locked)
    {
       struct extent_map *next;
       bool ret = false;
@@ -1066,9 +1066,11 @@ static bool defrag_check_next_extent(struct inode *inode, struct extent_map *em,
       /* Preallocated */
       if (test_bit(EXTENT_FLAG_PREALLOC, &em->flags))
               goto out;
-    /* Physically adjacent and large enough */
-    if ((em->block_start + em->block_len == next->block_start) &&
Quoted 8 lines

fs/btrfs/ioctl.c static bool defrag_check_next_extent(struct inode *inode, struct extent_map *em, · line 1071

Quoted 4 lines
fs/btrfs/ioctl.c +3 -1
-        (em->block_len > SZ_128K && next->block_len > SZ_128K))
+    /* Extent is already large enough */
+    if (next->len >= extent_thresh)
+            goto out;

So this will trigger unnecessary rewrites of compressed extents. The SZ_128K is there to deal with compressed extents, it has nothing to do with the threshold passed to the ioctl.

Quoted 6 lines

Then there is still something wrong.

The original check will only reject it when both conditions are met.

So based on your script, I can still find a way to defrag the extents, with or without this modification:

Right, without the intermediary write to file "baz", this patchset brings a regression in regards to
compressed extents - when they are adjacent, which is typically the case when doing large writes,
as they'll create multiple extents covering consecutive 128K ranges.

With the write to file "baz", as I pasted it, it happens before and after the patchset.

Quoted 31 lines
          mkfs.btrfs -f $DEV
          mount -o compress $DEV $MNT

          xfs_io -f -c "pwrite -S 0xab 0 128K" $MNT/file1
          sync
          xfs_io -f -c "pwrite -S 0xab 0 128K" $MNT/file2
          sync
          xfs_io -f -c "pwrite -S 0xab 128K 128K" $MNT/file1
          sync

          echo "=== file1 before defrag ==="
          xfs_io -f -c "fiemap -v" $MNT/file1
          echo "=== file1 after defrag ==="
          btrfs fi defrag $MNT/file1
          sync
          xfs_io -f -c "fiemap -v" $MNT/file1

The output looks like this:

=== before ===
/mnt/btrfs/file1:

    EXT: FILE-OFFSET      BLOCK-RANGE      TOTAL FLAGS
      0: [0..255]:        26624..26879       256   0x8
      1: [256..511]:      26640..26895       256   0x9
=== after ===
/mnt/btrfs/file1:
    EXT: FILE-OFFSET      BLOCK-RANGE      TOTAL FLAGS
      0: [0..255]:        26648..26903       256   0x8
      1: [256..511]:      26656..26911       256   0x9
Quoted 1 line

No matter if the patch is applied, the result is the same.

Yes, explained above.

Quoted 10 lines

So thank you very much for finding another case we're not handling well...

BTW, if the check is want to reject adjacent non-compressed extent, the original one is still incorrect, we can have extents smaller than 128K and is still uncompressed.

So what we really want is to reject physically adjacent, non-compressed extents?

We want to avoid doing work that does nothing.
If 2 consecutive extents are compressed and at least one is already 128K, then it's a waste of time, IO and CPU.

Quoted 7 lines
Quoted 4 lines

So can we define the behavior like this?

If the extent is already at its max capacity (compressed 128K,
non-compressed 128M), we don't defrag it.

My previous suggestion was: if one of the extents is compressed and its size is 128K, don't include it for defrag.

Yep, your previous one can handling it well, I'd just want to add the similar check for uncompressed one (which may be too rare to hit though)

Quoted 7 lines

There's probably other cases to think about: 1 compressed extent representing 100K of data, followed by another compressed extent representing 64K of data for example.
In that case using both for defrag will still result in 2 extents, 1 for 128K of data and another for 36K of data - still not worth it to defrag them, we end up with 2 extents, just different sizes.

Yes, that's also a factor to consider.

And with the target list we can determine how many compressed extents it would result.
The missing piece is the number of the original extents.

I think this would be a target for later optimization.

Quoted 5 lines

At the very least we should not regress on what we did not defrag before:

2 extents with physically contiguous ranges, representing 128K of data each, both compressed.

Although the original code is not really working as it's doing block_len

Quoted 1 line

SZ_128K, while our maximum compressed extent size is exactly 128K.

I'll fix the problem first with better check and comments.

Thanks,
Qu

Quoted 120 lines

Which is a very common case.

Thanks.

Quoted 115 lines

This also means, we need to do the same check in defrag_collect_targets() to avoid defragging such extent.

Thanks,
Qu

Quoted 107 lines

And that's a fairly common scenario. Do a one megabyte write for example, then after writeback we end up with several 128K extents with compression.
In that case defrag should do nothing for the whole range.

Quoted 100 lines

Thanks,
Qu

Quoted 96 lines

After applying this patchset, if you run a trivial test like this:

#!/bin/bash

DEV=/dev/sdj
MNT=/mnt/sdj

mkfs.btrfs -f $DEV
mount -o compress $DEV $MNT

xfs_io -f -c "pwrite -S 0xab 0 128K" $MNT/foobar sync
# Write to some other file so that the next extent for foobar # is not contiguous with the first extent.
xfs_io -f -c "pwrite 0 128K" $MNT/baz
sync
xfs_io -f -c "pwrite -S 0xcd 128K 128K" $MNT/foobar sync

echo -e "\n\nTree after creating file:\n\n"
btrfs inspect-internal dump-tree -t 5 $DEV

btrfs filesystem defragment $MNT/foobar
sync

echo -e "\n\nTree after defrag:\n\n"
btrfs inspect-internal dump-tree -t 5 $DEV

umount $MNT

It will result in rewriting the two 128K compressed extents:

(...)
Tree after write and sync:

btrfs-progs v5.12.1
fs tree key (FS_TREE ROOT_ITEM 0)
(...)

        item 7 key (257 INODE_REF 256) itemoff 15797 itemsize 16
                index 2 namelen 6 name: foobar
        item 8 key (257 EXTENT_DATA 0) itemoff 15744 itemsize 53
                generation 6 type 1 (regular)
                extent data disk byte 13631488 nr 4096
                extent data offset 0 nr 131072 ram 131072
                extent compression 1 (zlib)
        item 9 key (257 EXTENT_DATA 131072) itemoff 15691 itemsize 53
                generation 8 type 1 (regular)
                extent data disk byte 14163968 nr 4096
                extent data offset 0 nr 131072 ram 131072
                extent compression 1 (zlib)

(...)

Tree after defrag:

btrfs-progs v5.12.1
fs tree key (FS_TREE ROOT_ITEM 0)
(...)

        item 7 key (257 INODE_REF 256) itemoff 15797 itemsize 16
                index 2 namelen 6 name: foobar
        item 8 key (257 EXTENT_DATA 0) itemoff 15744 itemsize 53
                generation 9 type 1 (regular)
                extent data disk byte 14430208 nr 4096
                extent data offset 0 nr 131072 ram 131072
                extent compression 1 (zlib)
        item 9 key (257 EXTENT_DATA 131072) itemoff 15691 itemsize 53
                generation 9 type 1 (regular)
                extent data disk byte 13635584 nr 4096
                extent data offset 0 nr 131072 ram 131072
                extent compression 1 (zlib)

In other words, a waste of IO and CPU time.

So it needs to check if we are dealing with compressed extents, and if so, skip either of them has a size of SZ_128K (and changelog updated).

Thanks.

Quoted 17 lines
fs/btrfs/ioctl.c +3 -2
+    /* Physically adjacent */
+    if ((em->block_start + em->block_len == next->block_start))
               goto out;
       ret = true;
    out:
@@ -1231,7 +1233,7 @@ static int defrag_collect_targets(struct btrfs_inode *inode,
                       goto next;
 
               next_mergeable = defrag_check_next_extent(&inode->vfs_inode, em,
-                                                      locked);
+                                                      extent_thresh, locked);
               if (!next_mergeable) {
                       struct defrag_target_range *last;
 
--

2.34.1

#8 lore raw

Re: [PATCH 2/3] btrfs: defrag: use extent_thresh to replace the hardcoded size limit

Quoted 145 lines · Qu Wenruo
Quoted 123 lines · Filipe Manana
Quoted 122 lines · Qu Wenruo
Quoted 117 lines · Filipe Manana
Quoted 51 lines · Qu Wenruo
Quoted 38 lines · Filipe Manana
Quoted 37 lines · Qu Wenruo

In defrag_lookup_extent() we use hardcoded extent size threshold, SZ_128K, other than @extent_thresh in btrfs_defrag_file().

This can lead to some inconsistent behavior, especially the default extent size threshold is 256K.

Fix this by passing @extent_thresh into defrag_check_next_extent() and use that value.

Also, since the extent_thresh check should be applied to all extents, not only physically adjacent extents, move the threshold check into a dedicate if ().

---
    fs/btrfs/ioctl.c | 12 +++++++-----
    1 file changed, 7 insertions(+), 5 deletions(-)
fs/btrfs/ioctl.c +1 -3
index 0d8bfc716e6b..2911df12fc48 100644
@@ -1050,7 +1050,7 @@ static struct extent_map *defrag_lookup_extent(struct inode *inode, u64 start,
    }
 
    static bool defrag_check_next_extent(struct inode *inode, struct extent_map *em,
-                                 bool locked)
+                                 u32 extent_thresh, bool locked)
    {
       struct extent_map *next;
       bool ret = false;
@@ -1066,9 +1066,11 @@ static bool defrag_check_next_extent(struct inode *inode, struct extent_map *em,
       /* Preallocated */
       if (test_bit(EXTENT_FLAG_PREALLOC, &em->flags))
               goto out;
-    /* Physically adjacent and large enough */
-    if ((em->block_start + em->block_len == next->block_start) &&
Quoted 8 lines

fs/btrfs/ioctl.c static bool defrag_check_next_extent(struct inode *inode, struct extent_map *em, · line 1071

Quoted 4 lines
fs/btrfs/ioctl.c +3 -1
-        (em->block_len > SZ_128K && next->block_len > SZ_128K))
+    /* Extent is already large enough */
+    if (next->len >= extent_thresh)
+            goto out;

So this will trigger unnecessary rewrites of compressed extents. The SZ_128K is there to deal with compressed extents, it has nothing to do with the threshold passed to the ioctl.

Quoted 6 lines

Then there is still something wrong.

The original check will only reject it when both conditions are met.

So based on your script, I can still find a way to defrag the extents, with or without this modification:

Right, without the intermediary write to file "baz", this patchset brings a regression in regards to
compressed extents - when they are adjacent, which is typically the case when doing large writes,
as they'll create multiple extents covering consecutive 128K ranges.

With the write to file "baz", as I pasted it, it happens before and after the patchset.

Quoted 31 lines
          mkfs.btrfs -f $DEV
          mount -o compress $DEV $MNT

          xfs_io -f -c "pwrite -S 0xab 0 128K" $MNT/file1
          sync
          xfs_io -f -c "pwrite -S 0xab 0 128K" $MNT/file2
          sync
          xfs_io -f -c "pwrite -S 0xab 128K 128K" $MNT/file1
          sync

          echo "=== file1 before defrag ==="
          xfs_io -f -c "fiemap -v" $MNT/file1
          echo "=== file1 after defrag ==="
          btrfs fi defrag $MNT/file1
          sync
          xfs_io -f -c "fiemap -v" $MNT/file1

The output looks like this:

=== before ===
/mnt/btrfs/file1:

    EXT: FILE-OFFSET      BLOCK-RANGE      TOTAL FLAGS
      0: [0..255]:        26624..26879       256   0x8
      1: [256..511]:      26640..26895       256   0x9
=== after ===
/mnt/btrfs/file1:
    EXT: FILE-OFFSET      BLOCK-RANGE      TOTAL FLAGS
      0: [0..255]:        26648..26903       256   0x8
      1: [256..511]:      26656..26911       256   0x9
Quoted 1 line

No matter if the patch is applied, the result is the same.

Yes, explained above.

Quoted 10 lines

So thank you very much for finding another case we're not handling well...

BTW, if the check is want to reject adjacent non-compressed extent, the original one is still incorrect, we can have extents smaller than 128K and is still uncompressed.

So what we really want is to reject physically adjacent, non-compressed extents?

We want to avoid doing work that does nothing.
If 2 consecutive extents are compressed and at least one is already 128K, then it's a waste of time, IO and CPU.

Quoted 7 lines
Quoted 4 lines

So can we define the behavior like this?

If the extent is already at its max capacity (compressed 128K,
non-compressed 128M), we don't defrag it.

My previous suggestion was: if one of the extents is compressed and its size is 128K, don't include it for defrag.

Yep, your previous one can handling it well, I'd just want to add the similar check for uncompressed one (which may be too rare to hit though)

Quoted 7 lines

There's probably other cases to think about: 1 compressed extent representing 100K of data, followed by another compressed extent representing 64K of data for example.
In that case using both for defrag will still result in 2 extents, 1 for 128K of data and another for 36K of data - still not worth it to defrag them, we end up with 2 extents, just different sizes.

Quoted 7 lines

Yes, that's also a factor to consider.

And with the target list we can determine how many compressed extents it would result.
The missing piece is the number of the original extents.

I think this would be a target for later optimization.

Sure. Don't forget about possibly less common cases like having a mix of compressed extents, smaller than 128K followed with non-compressed extents, with sizes like in that above example, where after defrag we would still end up with 2 extents, just of different sizes.

It's certainly possible after cloning from a compressed file to one without compression and vice-versa.

Quoted 11 lines
Quoted 6 lines

At the very least we should not regress on what we did not defrag before:

2 extents with physically contiguous ranges, representing 128K of data each, both compressed.

Although the original code is not really working as it's doing block_len

Quoted 1 line

SZ_128K, while our maximum compressed extent size is exactly 128K.

I'll fix the problem first with better check and comments.

Sounds fine.

Thanks.

Quoted 124 lines

Thanks,
Qu

Quoted 120 lines

Which is a very common case.

Thanks.

Quoted 115 lines

This also means, we need to do the same check in defrag_collect_targets() to avoid defragging such extent.

Thanks,
Qu

Quoted 107 lines

And that's a fairly common scenario. Do a one megabyte write for example, then after writeback we end up with several 128K extents with compression.
In that case defrag should do nothing for the whole range.

Quoted 100 lines

Thanks,
Qu

Quoted 96 lines

After applying this patchset, if you run a trivial test like this:

#!/bin/bash

DEV=/dev/sdj
MNT=/mnt/sdj

mkfs.btrfs -f $DEV
mount -o compress $DEV $MNT

xfs_io -f -c "pwrite -S 0xab 0 128K" $MNT/foobar sync
# Write to some other file so that the next extent for foobar # is not contiguous with the first extent.
xfs_io -f -c "pwrite 0 128K" $MNT/baz
sync
xfs_io -f -c "pwrite -S 0xcd 128K 128K" $MNT/foobar sync

echo -e "\n\nTree after creating file:\n\n"
btrfs inspect-internal dump-tree -t 5 $DEV

btrfs filesystem defragment $MNT/foobar
sync

echo -e "\n\nTree after defrag:\n\n"
btrfs inspect-internal dump-tree -t 5 $DEV

umount $MNT

It will result in rewriting the two 128K compressed extents:

(...)
Tree after write and sync:

btrfs-progs v5.12.1
fs tree key (FS_TREE ROOT_ITEM 0)
(...)

        item 7 key (257 INODE_REF 256) itemoff 15797 itemsize 16
                index 2 namelen 6 name: foobar
        item 8 key (257 EXTENT_DATA 0) itemoff 15744 itemsize 53
                generation 6 type 1 (regular)
                extent data disk byte 13631488 nr 4096
                extent data offset 0 nr 131072 ram 131072
                extent compression 1 (zlib)
        item 9 key (257 EXTENT_DATA 131072) itemoff 15691 itemsize 53
                generation 8 type 1 (regular)
                extent data disk byte 14163968 nr 4096
                extent data offset 0 nr 131072 ram 131072
                extent compression 1 (zlib)

(...)

Tree after defrag:

btrfs-progs v5.12.1
fs tree key (FS_TREE ROOT_ITEM 0)
(...)

        item 7 key (257 INODE_REF 256) itemoff 15797 itemsize 16
                index 2 namelen 6 name: foobar
        item 8 key (257 EXTENT_DATA 0) itemoff 15744 itemsize 53
                generation 9 type 1 (regular)
                extent data disk byte 14430208 nr 4096
                extent data offset 0 nr 131072 ram 131072
                extent compression 1 (zlib)
        item 9 key (257 EXTENT_DATA 131072) itemoff 15691 itemsize 53
                generation 9 type 1 (regular)
                extent data disk byte 13635584 nr 4096
                extent data offset 0 nr 131072 ram 131072
                extent compression 1 (zlib)

In other words, a waste of IO and CPU time.

So it needs to check if we are dealing with compressed extents, and if so, skip either of them has a size of SZ_128K (and changelog updated).

Thanks.

Quoted 17 lines
fs/btrfs/ioctl.c +3 -2
+    /* Physically adjacent */
+    if ((em->block_start + em->block_len == next->block_start))
               goto out;
       ret = true;
    out:
@@ -1231,7 +1233,7 @@ static int defrag_collect_targets(struct btrfs_inode *inode,
                       goto next;
 
               next_mergeable = defrag_check_next_extent(&inode->vfs_inode, em,
-                                                      locked);
+                                                      extent_thresh, locked);
               if (!next_mergeable) {
                       struct defrag_target_range *last;
 
--

2.34.1

#9 lore raw

Re: [PATCH 2/3] btrfs: defrag: use extent_thresh to replace the hardcoded size limit

---- From: Filipe Manana <[email protected]> -- Sent: 2022-01-27 - 11:58 ----

Quoted 154 lines · Qu Wenruo
Quoted 145 lines · Filipe Manana
Quoted 123 lines · Qu Wenruo
Quoted 122 lines · Filipe Manana
Quoted 117 lines · Qu Wenruo
Quoted 51 lines · Filipe Manana
Quoted 38 lines · Qu Wenruo
Quoted 37 lines

In defrag_lookup_extent() we use hardcoded extent size threshold, SZ_128K, other than @extent_thresh in btrfs_defrag_file().

This can lead to some inconsistent behavior, especially the default extent size threshold is 256K.

Fix this by passing @extent_thresh into defrag_check_next_extent() and use that value.

Also, since the extent_thresh check should be applied to all extents, not only physically adjacent extents, move the threshold check into a dedicate if ().

---
    fs/btrfs/ioctl.c | 12 +++++++-----
    1 file changed, 7 insertions(+), 5 deletions(-)
fs/btrfs/ioctl.c +1 -3
index 0d8bfc716e6b..2911df12fc48 100644
@@ -1050,7 +1050,7 @@ static struct extent_map *defrag_lookup_extent(struct inode *inode, u64 start,
    }
 
    static bool defrag_check_next_extent(struct inode *inode, struct extent_map *em,
-                                 bool locked)
+                                 u32 extent_thresh, bool locked)
    {
       struct extent_map *next;
       bool ret = false;
@@ -1066,9 +1066,11 @@ static bool defrag_check_next_extent(struct inode *inode, struct extent_map *em,
       /* Preallocated */
       if (test_bit(EXTENT_FLAG_PREALLOC, &em->flags))
               goto out;
-    /* Physically adjacent and large enough */
-    if ((em->block_start + em->block_len == next->block_start) &&
Quoted 8 lines

fs/btrfs/ioctl.c static bool defrag_check_next_extent(struct inode *inode, struct extent_map *em, · line 1071

Quoted 4 lines
fs/btrfs/ioctl.c +3 -1
-        (em->block_len > SZ_128K && next->block_len > SZ_128K))
+    /* Extent is already large enough */
+    if (next->len >= extent_thresh)
+            goto out;

So this will trigger unnecessary rewrites of compressed extents. The SZ_128K is there to deal with compressed extents, it has nothing to do with the threshold passed to the ioctl.

Quoted 6 lines

Then there is still something wrong.

The original check will only reject it when both conditions are met.

So based on your script, I can still find a way to defrag the extents, with or without this modification:

Right, without the intermediary write to file "baz", this patchset brings a regression in regards to
compressed extents - when they are adjacent, which is typically the case when doing large writes,
as they'll create multiple extents covering consecutive 128K ranges.

With the write to file "baz", as I pasted it, it happens before and after the patchset.

Quoted 31 lines
          mkfs.btrfs -f $DEV
          mount -o compress $DEV $MNT

          xfs_io -f -c "pwrite -S 0xab 0 128K" $MNT/file1
          sync
          xfs_io -f -c "pwrite -S 0xab 0 128K" $MNT/file2
          sync
          xfs_io -f -c "pwrite -S 0xab 128K 128K" $MNT/file1
          sync

          echo "=== file1 before defrag ==="
          xfs_io -f -c "fiemap -v" $MNT/file1
          echo "=== file1 after defrag ==="
          btrfs fi defrag $MNT/file1
          sync
          xfs_io -f -c "fiemap -v" $MNT/file1

The output looks like this:

=== before ===
/mnt/btrfs/file1:

    EXT: FILE-OFFSET      BLOCK-RANGE      TOTAL FLAGS
      0: [0..255]:        26624..26879       256   0x8
      1: [256..511]:      26640..26895       256   0x9
=== after ===
/mnt/btrfs/file1:
    EXT: FILE-OFFSET      BLOCK-RANGE      TOTAL FLAGS
      0: [0..255]:        26648..26903       256   0x8
      1: [256..511]:      26656..26911       256   0x9
Quoted 1 line

No matter if the patch is applied, the result is the same.

Yes, explained above.

Quoted 10 lines

So thank you very much for finding another case we're not handling well...

BTW, if the check is want to reject adjacent non-compressed extent, the original one is still incorrect, we can have extents smaller than 128K and is still uncompressed.

So what we really want is to reject physically adjacent, non-compressed extents?

We want to avoid doing work that does nothing.
If 2 consecutive extents are compressed and at least one is already 128K, then it's a waste of time, IO and CPU.

Quoted 7 lines
Quoted 4 lines

So can we define the behavior like this?

If the extent is already at its max capacity (compressed 128K,
non-compressed 128M), we don't defrag it.

My previous suggestion was: if one of the extents is compressed and its size is 128K, don't include it for defrag.

Yep, your previous one can handling it well, I'd just want to add the similar check for uncompressed one (which may be too rare to hit though)

Quoted 7 lines

There's probably other cases to think about: 1 compressed extent representing 100K of data, followed by another compressed extent representing 64K of data for example.
In that case using both for defrag will still result in 2 extents, 1 for 128K of data and another for 36K of data - still not worth it to defrag them, we end up with 2 extents, just different sizes.

Quoted 7 lines

Yes, that's also a factor to consider.

And with the target list we can determine how many compressed extents it would result.
The missing piece is the number of the original extents.

I think this would be a target for later optimization.

Quoted 8 lines

Sure. Don't forget about possibly less common cases like having a mix of compressed extents, smaller than 128K followed with non-compressed extents, with sizes like in that above example, where after defrag we would still end up with 2 extents, just of different sizes.

It's certainly possible after cloning from a compressed file to one without compression and vice-versa.

It may also be possible that two compressed extents with a combined size >128K would compress better when combined into a single extent, than each of them individually, and so end up fitting inside one final 128K extent.

How advanced logic is reasonable to aim for?

Quoted 140 lines
Quoted 11 lines
Quoted 6 lines

At the very least we should not regress on what we did not defrag before:

2 extents with physically contiguous ranges, representing 128K of data each, both compressed.

Although the original code is not really working as it's doing block_len

Quoted 1 line

SZ_128K, while our maximum compressed extent size is exactly 128K.

I'll fix the problem first with better check and comments.

Sounds fine.

Thanks.

Quoted 124 lines

Thanks,
Qu

Quoted 120 lines

Which is a very common case.

Thanks.

Quoted 115 lines

This also means, we need to do the same check in defrag_collect_targets() to avoid defragging such extent.

Thanks,
Qu

Quoted 107 lines

And that's a fairly common scenario. Do a one megabyte write for example, then after writeback we end up with several 128K extents with compression.
In that case defrag should do nothing for the whole range.

Quoted 100 lines

Thanks,
Qu

Quoted 96 lines

After applying this patchset, if you run a trivial test like this:

#!/bin/bash

DEV=/dev/sdj
MNT=/mnt/sdj

mkfs.btrfs -f $DEV
mount -o compress $DEV $MNT

xfs_io -f -c "pwrite -S 0xab 0 128K" $MNT/foobar sync
# Write to some other file so that the next extent for foobar # is not contiguous with the first extent.
xfs_io -f -c "pwrite 0 128K" $MNT/baz
sync
xfs_io -f -c "pwrite -S 0xcd 128K 128K" $MNT/foobar sync

echo -e "\n\nTree after creating file:\n\n"
btrfs inspect-internal dump-tree -t 5 $DEV

btrfs filesystem defragment $MNT/foobar
sync

echo -e "\n\nTree after defrag:\n\n"
btrfs inspect-internal dump-tree -t 5 $DEV

umount $MNT

It will result in rewriting the two 128K compressed extents:

(...)
Tree after write and sync:

btrfs-progs v5.12.1
fs tree key (FS_TREE ROOT_ITEM 0)
(...)

        item 7 key (257 INODE_REF 256) itemoff 15797 itemsize 16
                index 2 namelen 6 name: foobar
        item 8 key (257 EXTENT_DATA 0) itemoff 15744 itemsize 53
                generation 6 type 1 (regular)
                extent data disk byte 13631488 nr 4096
                extent data offset 0 nr 131072 ram 131072
                extent compression 1 (zlib)
        item 9 key (257 EXTENT_DATA 131072) itemoff 15691 itemsize 53
                generation 8 type 1 (regular)
                extent data disk byte 14163968 nr 4096
                extent data offset 0 nr 131072 ram 131072
                extent compression 1 (zlib)

(...)

Tree after defrag:

btrfs-progs v5.12.1
fs tree key (FS_TREE ROOT_ITEM 0)
(...)

        item 7 key (257 INODE_REF 256) itemoff 15797 itemsize 16
                index 2 namelen 6 name: foobar
        item 8 key (257 EXTENT_DATA 0) itemoff 15744 itemsize 53
                generation 9 type 1 (regular)
                extent data disk byte 14430208 nr 4096
                extent data offset 0 nr 131072 ram 131072
                extent compression 1 (zlib)
        item 9 key (257 EXTENT_DATA 131072) itemoff 15691 itemsize 53
                generation 9 type 1 (regular)
                extent data disk byte 13635584 nr 4096
                extent data offset 0 nr 131072 ram 131072
                extent compression 1 (zlib)

In other words, a waste of IO and CPU time.

So it needs to check if we are dealing with compressed extents, and if so, skip either of them has a size of SZ_128K (and changelog updated).

Thanks.

Quoted 17 lines
fs/btrfs/ioctl.c +3 -2
+    /* Physically adjacent */
+    if ((em->block_start + em->block_len == next->block_start))
               goto out;
       ret = true;
    out:
@@ -1231,7 +1233,7 @@ static int defrag_collect_targets(struct btrfs_inode *inode,
                       goto next;
 
               next_mergeable = defrag_check_next_extent(&inode->vfs_inode, em,
-                                                      locked);
+                                                      extent_thresh, locked);
               if (!next_mergeable) {
                       struct defrag_target_range *last;
 
--

2.34.1

author
#10 lore raw

PATCH 3/3 btrfs: defrag: remove the physical adjacent extents rejection in defrag_check_next_extent()

In reply to Qu Wenruo

There is a rejection for physically adjacent extents in defrag_check_next_extent() from the very beginning.

The check will reject physically adjacent extents which are also large enough.

The extent size threshold check is now a generic check, and the benefit of rejecting physically adjacent extents is unsure.

Sure physically adjacent extents means no extra seek time, thus defragging them may not bring much help.

On the other hand, btrfs also benefits from reduced number of extents (which can reduce the size of extent tree, thus reduce the mount time).

So such rejection is not a full win.

Remove such check, and policy on which extents should be defragged is mostly on @extent_thresh and @newer_than parameters.

---
 fs/btrfs/ioctl.c | 3 ---
 1 file changed, 3 deletions(-)
fs/btrfs/ioctl.c +0 -3
index 2911df12fc48..0929d08bb378 100644
@@ -1069,9 +1069,6 @@ static bool defrag_check_next_extent(struct inode *inode, struct extent_map *em,
 	/* Extent is already large enough */
 	if (next->len >= extent_thresh)
 		goto out;
-	/* Physically adjacent */
-	if ((em->block_start + em->block_len == next->block_start))
-		goto out;
 	ret = true;
 out:
 	free_extent_map(next);
Signature · 1 line
2.34.1
Reviewed-by on v1 3/3
#11 lore raw

Re: [PATCH 3/3] btrfs: defrag: remove the physical adjacent extents rejection in defrag_check_next_extent()

Quoted 6 lines · Qu Wenruo

There is a rejection for physically adjacent extents in defrag_check_next_extent() from the very beginning.

The check will reject physically adjacent extents which are also large enough.

Quoted 8 lines

The extent size threshold check is now a generic check, and the benefit of rejecting physically adjacent extents is unsure.

Sure physically adjacent extents means no extra seek time, thus defragging them may not bring much help.

On the other hand, btrfs also benefits from reduced number of extents (which can reduce the size of extent tree, thus reduce the mount time).

And also reduce the number of file extent items in the subvolume's tree.

I think it's fine.

Thanks.

Quoted 28 lines

So such rejection is not a full win.

Remove such check, and policy on which extents should be defragged is mostly on @extent_thresh and @newer_than parameters.

---
 fs/btrfs/ioctl.c | 3 ---
 1 file changed, 3 deletions(-)
fs/btrfs/ioctl.c +0 -3
index 2911df12fc48..0929d08bb378 100644
@@ -1069,9 +1069,6 @@ static bool defrag_check_next_extent(struct inode *inode, struct extent_map *em,
 	/* Extent is already large enough */
 	if (next->len >= extent_thresh)
 		goto out;
-	/* Physically adjacent */
-	if ((em->block_start + em->block_len == next->block_start))
-		goto out;
 	ret = true;
 out:
 	free_extent_map(next);
Signature · 1 line
2.34.1
Reviewed-by on v3 1/3
#12 lore raw
In reply to Qu Wenruo
Quoted 5 lines · Qu Wenruo

[BUG]
With older kernels (before v5.16), btrfs will defrag preallocated extents. While with newer kernels (v5.16 and newer) btrfs will not defrag preallocated extents, but it will defrag the extent just before the preallocated extent, even it's just a single sector.

In that case, isn't a Fixes: tag missing?

Quoted 102 lines

This can be exposed by the following small script:

mkfs.btrfs -f $dev > /dev/null

mount $dev $mnt
xfs_io -f -c "pwrite 0 4k" -c sync -c "falloc 4k 16K" $mnt/file xfs_io -c "fiemap -v" $mnt/file
btrfs fi defrag $mnt/file
sync
xfs_io -c "fiemap -v" $mnt/file

The output looks like this on older kernels:

/mnt/btrfs/file:

 EXT: FILE-OFFSET      BLOCK-RANGE      TOTAL FLAGS
   0: [0..7]:          26624..26631         8   0x0
   1: [8..39]:         26632..26663        32 0x801

/mnt/btrfs/file:
EXT: FILE-OFFSET BLOCK-RANGE TOTAL FLAGS
0: [0..39]: 26664..26703 40 0x1

Which defrags the single sector along with the preallocated extent, and replace them with an regular extent into a new location (caused by data COW).
This wastes most of the data IO just for the preallocated range.

On the other hand, v5.16 is slightly better:

/mnt/btrfs/file:

 EXT: FILE-OFFSET      BLOCK-RANGE      TOTAL FLAGS
   0: [0..7]:          26624..26631         8   0x0
   1: [8..39]:         26632..26663        32 0x801
/mnt/btrfs/file:
 EXT: FILE-OFFSET      BLOCK-RANGE      TOTAL FLAGS
   0: [0..7]:          26664..26671         8   0x0
   1: [8..39]:         26632..26663        32 0x801

The preallocated range is not defragged, but the sector before it still gets defragged, which has no need for it.

[CAUSE]
One of the function reused by the old and new behavior is defrag_check_next_extent(), it will determine if we should defrag current extent by checking the next one.

It only checks if the next extent is a hole or inlined, but it doesn't check if it's preallocated.

On the other hand, out of the function, both old and new kernel will reject preallocated extents.

Such inconsistent behavior causes above behavior.

[FIX]
- Also check if next extent is preallocated
If so, don't defrag current extent.

- Add comments for each branch why we reject the extent

This will reduce the IO caused by defrag ioctl and autodefrag.

---
Changelog:
v2:
- Use @extent_thresh from caller to replace the harded coded threshold Now caller has full control over the extent threshold value.

- Remove the old ambiguous check based on physical address The original check is too specific, only reject extents which are physically adjacent, AND too large.
Since we have correct size check now, and the physically adjacent check is not always a win.
So remove the old check completely.

v3:
- Split the @extent_thresh and physicall adjacent check into other patches

- Simplify the comment

---
 fs/btrfs/ioctl.c | 20 +++++++++++++-------
 1 file changed, 13 insertions(+), 7 deletions(-)
fs/btrfs/ioctl.c +2 -2
index 91ba2efe9792..0d8bfc716e6b 100644
@@ -1053,19 +1053,25 @@ static bool defrag_check_next_extent(struct inode *inode, struct extent_map *em,
 				     bool locked)
 {
 	struct extent_map *next;
-	bool ret = true;
+	bool ret = false;
 
 	/* this is the last extent */
 	if (em->start + em->len >= i_size_read(inode))
-		return false;
+		return ret;
 
 	next = defrag_lookup_extent(inode, em->start + em->len, locked);

fs/btrfs/ioctl.c static bool defrag_check_next_extent(struct inode *inode, struct extent_map *em, · line 1067

Quoted 10 lines
fs/btrfs/ioctl.c +4 -5
+	/* No more em or hole */
 	if (!next || next->block_start >= EXTENT_MAP_LAST_BYTE)
-		ret = false;
-	else if ((em->block_start + em->block_len == next->block_start) &&
-		 (em->block_len > SZ_128K && next->block_len > SZ_128K))
-		ret = false;
-
+		goto out;
+	/* Preallocated */
+	if (test_bit(EXTENT_FLAG_PREALLOC, &em->flags))

The comment is superfluous, the name of the flag is pretty informative that we are checking for a preallocated extent. You don't need to send a new version just to remove the comment however.

For the Fixes: tag, you can just comment and David will pick it.

Thanks.

Quoted 13 lines
fs/btrfs/ioctl.c +7 -0
+		goto out;
+	/* Physically adjacent and large enough */
+	if ((em->block_start + em->block_len == next->block_start) &&
+	    (em->block_len > SZ_128K && next->block_len > SZ_128K))
+		goto out;
+	ret = true;
+out:
 	free_extent_map(next);
 	return ret;
 }
Signature · 1 line
2.34.1
#13 lore raw
Quoted 8 lines · Filipe Manana
Quoted 5 lines · Qu Wenruo

[BUG]
With older kernels (before v5.16), btrfs will defrag preallocated extents. While with newer kernels (v5.16 and newer) btrfs will not defrag preallocated extents, but it will defrag the extent just before the preallocated extent, even it's just a single sector.

In that case, isn't a Fixes: tag missing?

The function defrag_check_next_extent() is reused from older kernel code, dating back to 2012.
(And even that 2012 commit is not really the root cause).
Thus it's missing preallocated check from the very beginning unfortunately.

Does it still make sense to include such an old commit?

It would make more sense to CC this to v5.x stable branch, though.

Any advice on this situation?

Thanks,
Qu

Quoted 137 lines
Quoted 103 lines

This can be exposed by the following small script:

mkfs.btrfs -f $dev > /dev/null

mount $dev $mnt
xfs_io -f -c "pwrite 0 4k" -c sync -c "falloc 4k 16K" $mnt/file xfs_io -c "fiemap -v" $mnt/file
btrfs fi defrag $mnt/file
sync
xfs_io -c "fiemap -v" $mnt/file

The output looks like this on older kernels:

/mnt/btrfs/file:

  EXT: FILE-OFFSET      BLOCK-RANGE      TOTAL FLAGS
    0: [0..7]:          26624..26631         8   0x0
    1: [8..39]:         26632..26663        32 0x801
/mnt/btrfs/file:
  EXT: FILE-OFFSET      BLOCK-RANGE      TOTAL FLAGS
    0: [0..39]:         26664..26703        40   0x1

Which defrags the single sector along with the preallocated extent, and replace them with an regular extent into a new location (caused by data COW).
This wastes most of the data IO just for the preallocated range.

On the other hand, v5.16 is slightly better:

/mnt/btrfs/file:

  EXT: FILE-OFFSET      BLOCK-RANGE      TOTAL FLAGS
    0: [0..7]:          26624..26631         8   0x0
    1: [8..39]:         26632..26663        32 0x801
/mnt/btrfs/file:
  EXT: FILE-OFFSET      BLOCK-RANGE      TOTAL FLAGS
    0: [0..7]:          26664..26671         8   0x0
    1: [8..39]:         26632..26663        32 0x801

The preallocated range is not defragged, but the sector before it still gets defragged, which has no need for it.

[CAUSE]
One of the function reused by the old and new behavior is defrag_check_next_extent(), it will determine if we should defrag current extent by checking the next one.

It only checks if the next extent is a hole or inlined, but it doesn't check if it's preallocated.

On the other hand, out of the function, both old and new kernel will reject preallocated extents.

Such inconsistent behavior causes above behavior.

[FIX]
- Also check if next extent is preallocated
If so, don't defrag current extent.

- Add comments for each branch why we reject the extent

This will reduce the IO caused by defrag ioctl and autodefrag.

---
Changelog:
v2:
- Use @extent_thresh from caller to replace the harded coded threshold Now caller has full control over the extent threshold value.

- Remove the old ambiguous check based on physical address
The original check is too specific, only reject extents which are physically adjacent, AND too large.
Since we have correct size check now, and the physically adjacent check is not always a win.
So remove the old check completely.

v3:
- Split the @extent_thresh and physicall adjacent check into other patches

- Simplify the comment

---
  fs/btrfs/ioctl.c | 20 +++++++++++++-------
  1 file changed, 13 insertions(+), 7 deletions(-)
fs/btrfs/ioctl.c +2 -2
index 91ba2efe9792..0d8bfc716e6b 100644
@@ -1053,19 +1053,25 @@ static bool defrag_check_next_extent(struct inode *inode, struct extent_map *em,
  				     bool locked)
  {
  	struct extent_map *next;
-	bool ret = true;
+	bool ret = false;
 
  	/* this is the last extent */
  	if (em->start + em->len >= i_size_read(inode))
-		return false;
+		return ret;
 
  	next = defrag_lookup_extent(inode, em->start + em->len, locked);

fs/btrfs/ioctl.c static bool defrag_check_next_extent(struct inode *inode, struct extent_map *em, · line 1067

Quoted 10 lines
fs/btrfs/ioctl.c +4 -5
+	/* No more em or hole */
  	if (!next || next->block_start >= EXTENT_MAP_LAST_BYTE)
-		ret = false;
-	else if ((em->block_start + em->block_len == next->block_start) &&
-		 (em->block_len > SZ_128K && next->block_len > SZ_128K))
-		ret = false;
-
+		goto out;
+	/* Preallocated */
+	if (test_bit(EXTENT_FLAG_PREALLOC, &em->flags))

The comment is superfluous, the name of the flag is pretty informative that we are checking for a preallocated extent. You don't need to send a new version just to remove the comment however.

For the Fixes: tag, you can just comment and David will pick it.

Thanks.

Quoted 13 lines
fs/btrfs/ioctl.c +7 -1
+		goto out;
+	/* Physically adjacent and large enough */
+	if ((em->block_start + em->block_len == next->block_start) &&
+	    (em->block_len > SZ_128K && next->block_len > SZ_128K))
+		goto out;
+	ret = true;
+out:
  	free_extent_map(next);
  	return ret;
  }
--

2.34.1

#14 lore raw
Quoted 18 lines · Qu Wenruo
Quoted 8 lines · Filipe Manana
Quoted 5 lines · Qu Wenruo

[BUG]
With older kernels (before v5.16), btrfs will defrag preallocated extents. While with newer kernels (v5.16 and newer) btrfs will not defrag preallocated extents, but it will defrag the extent just before the preallocated extent, even it's just a single sector.

In that case, isn't a Fixes: tag missing?

The function defrag_check_next_extent() is reused from older kernel code, dating back to 2012.
(And even that 2012 commit is not really the root cause).
Thus it's missing preallocated check from the very beginning unfortunately.

Quoted 5 lines

Does it still make sense to include such an old commit?

It would make more sense to CC this to v5.x stable branch, though.

Any advice on this situation?

Hum, ok, then I think it's fine without the tag.
Thanks.

Quoted 140 lines

Thanks,
Qu

Quoted 137 lines
Quoted 103 lines

This can be exposed by the following small script:

mkfs.btrfs -f $dev > /dev/null

mount $dev $mnt
xfs_io -f -c "pwrite 0 4k" -c sync -c "falloc 4k 16K" $mnt/file xfs_io -c "fiemap -v" $mnt/file
btrfs fi defrag $mnt/file
sync
xfs_io -c "fiemap -v" $mnt/file

The output looks like this on older kernels:

/mnt/btrfs/file:

  EXT: FILE-OFFSET      BLOCK-RANGE      TOTAL FLAGS
    0: [0..7]:          26624..26631         8   0x0
    1: [8..39]:         26632..26663        32 0x801
/mnt/btrfs/file:
  EXT: FILE-OFFSET      BLOCK-RANGE      TOTAL FLAGS
    0: [0..39]:         26664..26703        40   0x1

Which defrags the single sector along with the preallocated extent, and replace them with an regular extent into a new location (caused by data COW).
This wastes most of the data IO just for the preallocated range.

On the other hand, v5.16 is slightly better:

/mnt/btrfs/file:

  EXT: FILE-OFFSET      BLOCK-RANGE      TOTAL FLAGS
    0: [0..7]:          26624..26631         8   0x0
    1: [8..39]:         26632..26663        32 0x801
/mnt/btrfs/file:
  EXT: FILE-OFFSET      BLOCK-RANGE      TOTAL FLAGS
    0: [0..7]:          26664..26671         8   0x0
    1: [8..39]:         26632..26663        32 0x801

The preallocated range is not defragged, but the sector before it still gets defragged, which has no need for it.

[CAUSE]
One of the function reused by the old and new behavior is defrag_check_next_extent(), it will determine if we should defrag current extent by checking the next one.

It only checks if the next extent is a hole or inlined, but it doesn't check if it's preallocated.

On the other hand, out of the function, both old and new kernel will reject preallocated extents.

Such inconsistent behavior causes above behavior.

[FIX]
- Also check if next extent is preallocated
If so, don't defrag current extent.

- Add comments for each branch why we reject the extent

This will reduce the IO caused by defrag ioctl and autodefrag.

---
Changelog:
v2:
- Use @extent_thresh from caller to replace the harded coded threshold Now caller has full control over the extent threshold value.

- Remove the old ambiguous check based on physical address
The original check is too specific, only reject extents which are physically adjacent, AND too large.
Since we have correct size check now, and the physically adjacent check is not always a win.
So remove the old check completely.

v3:
- Split the @extent_thresh and physicall adjacent check into other patches

- Simplify the comment

---
  fs/btrfs/ioctl.c | 20 +++++++++++++-------
  1 file changed, 13 insertions(+), 7 deletions(-)
fs/btrfs/ioctl.c +2 -2
index 91ba2efe9792..0d8bfc716e6b 100644
@@ -1053,19 +1053,25 @@ static bool defrag_check_next_extent(struct inode *inode, struct extent_map *em,
                                  bool locked)
  {
     struct extent_map *next;
-    bool ret = true;
+    bool ret = false;
 
     /* this is the last extent */
     if (em->start + em->len >= i_size_read(inode))
-            return false;
+            return ret;
 
     next = defrag_lookup_extent(inode, em->start + em->len, locked);

fs/btrfs/ioctl.c static bool defrag_check_next_extent(struct inode *inode, struct extent_map *em, · line 1067

Quoted 10 lines
fs/btrfs/ioctl.c +4 -5
+    /* No more em or hole */
     if (!next || next->block_start >= EXTENT_MAP_LAST_BYTE)
-            ret = false;
-    else if ((em->block_start + em->block_len == next->block_start) &&
-             (em->block_len > SZ_128K && next->block_len > SZ_128K))
-            ret = false;
-
+            goto out;
+    /* Preallocated */
+    if (test_bit(EXTENT_FLAG_PREALLOC, &em->flags))

The comment is superfluous, the name of the flag is pretty informative that we are checking for a preallocated extent. You don't need to send a new version just to remove the comment however.

For the Fixes: tag, you can just comment and David will pick it.

Thanks.

Quoted 13 lines
fs/btrfs/ioctl.c +7 -1
+            goto out;
+    /* Physically adjacent and large enough */
+    if ((em->block_start + em->block_len == next->block_start) &&
+        (em->block_len > SZ_128K && next->block_len > SZ_128K))
+            goto out;
+    ret = true;
+out:
     free_extent_map(next);
     return ret;
  }
--

2.34.1

author
#15 lore raw
In reply to Qu Wenruo
Quoted 107 lines · Qu Wenruo

[BUG]
With older kernels (before v5.16), btrfs will defrag preallocated extents. While with newer kernels (v5.16 and newer) btrfs will not defrag preallocated extents, but it will defrag the extent just before the preallocated extent, even it's just a single sector.

This can be exposed by the following small script:

mkfs.btrfs -f $dev > /dev/null

mount $dev $mnt
xfs_io -f -c "pwrite 0 4k" -c sync -c "falloc 4k 16K" $mnt/file xfs_io -c "fiemap -v" $mnt/file
btrfs fi defrag $mnt/file
sync
xfs_io -c "fiemap -v" $mnt/file

The output looks like this on older kernels:

/mnt/btrfs/file:

  EXT: FILE-OFFSET      BLOCK-RANGE      TOTAL FLAGS
    0: [0..7]:          26624..26631         8   0x0
    1: [8..39]:         26632..26663        32 0x801
/mnt/btrfs/file:
  EXT: FILE-OFFSET      BLOCK-RANGE      TOTAL FLAGS
    0: [0..39]:         26664..26703        40   0x1

Which defrags the single sector along with the preallocated extent, and replace them with an regular extent into a new location (caused by data COW).
This wastes most of the data IO just for the preallocated range.

On the other hand, v5.16 is slightly better:

/mnt/btrfs/file:

  EXT: FILE-OFFSET      BLOCK-RANGE      TOTAL FLAGS
    0: [0..7]:          26624..26631         8   0x0
    1: [8..39]:         26632..26663        32 0x801
/mnt/btrfs/file:
  EXT: FILE-OFFSET      BLOCK-RANGE      TOTAL FLAGS
    0: [0..7]:          26664..26671         8   0x0
    1: [8..39]:         26632..26663        32 0x801

The preallocated range is not defragged, but the sector before it still gets defragged, which has no need for it.

[CAUSE]
One of the function reused by the old and new behavior is defrag_check_next_extent(), it will determine if we should defrag current extent by checking the next one.

It only checks if the next extent is a hole or inlined, but it doesn't check if it's preallocated.

On the other hand, out of the function, both old and new kernel will reject preallocated extents.

Such inconsistent behavior causes above behavior.

[FIX]
- Also check if next extent is preallocated
If so, don't defrag current extent.

- Add comments for each branch why we reject the extent

This will reduce the IO caused by defrag ioctl and autodefrag.

---
Changelog:
v2:
- Use @extent_thresh from caller to replace the harded coded threshold Now caller has full control over the extent threshold value.

- Remove the old ambiguous check based on physical address
The original check is too specific, only reject extents which are physically adjacent, AND too large.
Since we have correct size check now, and the physically adjacent check is not always a win.
So remove the old check completely.

v3:
- Split the @extent_thresh and physicall adjacent check into other patches

- Simplify the comment

---
  fs/btrfs/ioctl.c | 20 +++++++++++++-------
  1 file changed, 13 insertions(+), 7 deletions(-)
fs/btrfs/ioctl.c +2 -2
index 91ba2efe9792..0d8bfc716e6b 100644
@@ -1053,19 +1053,25 @@ static bool defrag_check_next_extent(struct inode *inode, struct extent_map *em,
  				     bool locked)
  {
  	struct extent_map *next;
-	bool ret = true;
+	bool ret = false;
 
  	/* this is the last extent */
  	if (em->start + em->len >= i_size_read(inode))
-		return false;
+		return ret;
 
  	next = defrag_lookup_extent(inode, em->start + em->len, locked);

fs/btrfs/ioctl.c static bool defrag_check_next_extent(struct inode *inode, struct extent_map *em, · line 1067

Quoted 10 lines
fs/btrfs/ioctl.c +4 -5
+	/* No more em or hole */
  	if (!next || next->block_start >= EXTENT_MAP_LAST_BYTE)
-		ret = false;
-	else if ((em->block_start + em->block_len == next->block_start) &&
-		 (em->block_len > SZ_128K && next->block_len > SZ_128K))
-		ret = false;
-
+		goto out;
+	/* Preallocated */
+	if (test_bit(EXTENT_FLAG_PREALLOC, &em->flags))

Nope, during the extra tests for unrelated work, I exposed this bug, we should check @next, not @em.

Needs to update it again...

Thanks,
Qu

Quoted 10 lines
fs/btrfs/ioctl.c +7 -0
+		goto out;
+	/* Physically adjacent and large enough */
+	if ((em->block_start + em->block_len == next->block_start) &&
+	    (em->block_len > SZ_128K && next->block_len > SZ_128K))
+		goto out;
+	ret = true;
+out:
  	free_extent_map(next);
  	return ret;
  }