Skip to contentSkip to conversation

generic/301: skip extent count check on btrfs

PATCH In review Leo Martins 5 participants 5 messages Mar 24 – Mar 31 Btrfs file system

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 generic/301: skip extent count check on btrfs

The 2/3 fragmentation threshold was designed for XFS's cowextsize mechanism and is not applicable to btrfs. After commit 0dc118b3c327 ("btrfs: be less aggressive with metadata overcommit when we can do full flushing"), btrfs commits transactions more frequently during writeback, reducing the extent allocator's opportunity to coalesce adjacent CoW extents. This causes the extent count to sometimes cross the threshold, making the test flaky on btrfs.

Skip only the extent count assertion for btrfs while keeping the CoW write + remount + md5sum data integrity checks, which remain useful.

---
 tests/generic/301 | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
tests/generic/301 +6 -1
index 0303f25d..985d943a 100755
@@ -62,7 +62,12 @@ new_extents=$(_count_extents $testdir/file2)
 echo "old extents: $old_extents" >> $seqres.full
 echo "new extents: $new_extents" >> $seqres.full
 echo "maximum extents: $internal_blks" >> $seqres.full
-test $new_extents -lt $((internal_blks * 2 / 3)) || echo "file2 badly fragmented"
+# btrfs CoW extent allocation depends on transaction commit frequency and
+# metadata reservation behavior, so the 2/3 fragmentation threshold (designed
+# for XFS cowextsize) is not applicable.
+if [ "$FSTYP" != "btrfs" ]; then
+	test $new_extents -lt $((internal_blks * 2 / 3)) || echo "file2 badly fragmented"
+fi
 
 # success, all done
 status=0
Signature · 1 line
2.52.0
Reviewed-by on v1 1/1
#1 lore raw
Quoted 8 lines · Leo Martins

The 2/3 fragmentation threshold was designed for XFS's cowextsize mechanism and is not applicable to btrfs. After commit 0dc118b3c327 ("btrfs: be less aggressive with metadata overcommit when we can do full flushing"), btrfs commits transactions more frequently during writeback, reducing the extent allocator's opportunity to coalesce adjacent CoW extents. This causes the extent count to sometimes cross the threshold, making the test flaky on btrfs.

Quoted 4 lines

Skip only the extent count assertion for btrfs while keeping the CoW write + remount + md5sum data integrity checks, which remain useful.

Seems reasonable to me,

--D

Quoted 26 lines
---
 tests/generic/301 | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
tests/generic/301 +6 -1
index 0303f25d..985d943a 100755
@@ -62,7 +62,12 @@ new_extents=$(_count_extents $testdir/file2)
 echo "old extents: $old_extents" >> $seqres.full
 echo "new extents: $new_extents" >> $seqres.full
 echo "maximum extents: $internal_blks" >> $seqres.full
-test $new_extents -lt $((internal_blks * 2 / 3)) || echo "file2 badly fragmented"
+# btrfs CoW extent allocation depends on transaction commit frequency and
+# metadata reservation behavior, so the 2/3 fragmentation threshold (designed
+# for XFS cowextsize) is not applicable.
+if [ "$FSTYP" != "btrfs" ]; then
+	test $new_extents -lt $((internal_blks * 2 / 3)) || echo "file2 badly fragmented"
+fi
 
 # success, all done
 status=0
Signature · 1 line
2.52.0
Reviewed-by on v1 1/1
#2 lore raw
In reply to Leo Martins
Quoted 8 lines · Leo Martins

The 2/3 fragmentation threshold was designed for XFS's cowextsize mechanism and is not applicable to btrfs. After commit 0dc118b3c327 ("btrfs: be less aggressive with metadata overcommit when we can do full flushing"), btrfs commits transactions more frequently during writeback, reducing the extent allocator's opportunity to coalesce adjacent CoW extents. This causes the extent count to sometimes cross the threshold, making the test flaky on btrfs.

Quoted 4 lines

Skip only the extent count assertion for btrfs while keeping the CoW write + remount + md5sum data integrity checks, which remain useful.

For the btrfs part:

Thanks,
Qu

Quoted 22 lines
---
  tests/generic/301 | 7 ++++++-
  1 file changed, 6 insertions(+), 1 deletion(-)
tests/generic/301 +6 -1
index 0303f25d..985d943a 100755
@@ -62,7 +62,12 @@ new_extents=$(_count_extents $testdir/file2)
  echo "old extents: $old_extents" >> $seqres.full
  echo "new extents: $new_extents" >> $seqres.full
  echo "maximum extents: $internal_blks" >> $seqres.full
-test $new_extents -lt $((internal_blks * 2 / 3)) || echo "file2 badly fragmented"
+# btrfs CoW extent allocation depends on transaction commit frequency and
+# metadata reservation behavior, so the 2/3 fragmentation threshold (designed
+# for XFS cowextsize) is not applicable.
+if [ "$FSTYP" != "btrfs" ]; then
+	test $new_extents -lt $((internal_blks * 2 / 3)) || echo "file2 badly fragmented"
+fi
 
  # success, all done
  status=0
#3 lore raw
In reply to Leo Martins

tests/generic/301 new_extents=$(_count_extents $testdir/file2) · line 67

Quoted 3 lines · Leo Martins
tests/generic/301 +3 -0
+# btrfs CoW extent allocation depends on transaction commit frequency and
+# metadata reservation behavior, so the 2/3 fragmentation threshold (designed
+# for XFS cowextsize) is not applicable.

It really is a quality of implementation issue and not specifically designed for XFS. But I gues it is hard to force quality of implementation on specific file systems.

tests/generic/301 new_extents=$(_count_extents $testdir/file2) · line 69

Quoted 2 lines
tests/generic/301 +2 -0
+if [ "$FSTYP" != "btrfs" ]; then
+	test $new_extents -lt $((internal_blks * 2 / 3)) || echo "file2 badly fragmented"

Please break the echo into a separate line to keep this readable, either by just breaking using \ or by turning this into a proper if statement.

#4 lore raw
Quoted 9 lines · Christoph Hellwig

tests/generic/301 new_extents=$(_count_extents $testdir/file2) · line 67

Quoted 3 lines · Leo Martins
tests/generic/301 +3 -0
+# btrfs CoW extent allocation depends on transaction commit frequency and
+# metadata reservation behavior, so the 2/3 fragmentation threshold (designed
+# for XFS cowextsize) is not applicable.

It really is a quality of implementation issue and not specifically designed for XFS. But I gues it is hard to force quality of implementation on specific file systems.

Quoted 6 lines

tests/generic/301 new_extents=$(_count_extents $testdir/file2) · line 69

Quoted 2 lines
tests/generic/301 +2 -0
+if [ "$FSTYP" != "btrfs" ]; then
+	test $new_extents -lt $((internal_blks * 2 / 3)) || echo "file2 badly fragmented"

Please break the echo into a separate line to keep this readable, either by just breaking using \ or by turning this into a proper if statement.

Sure, I'll help to change it to:

	test $new_extents -lt $((internal_blks * 2 / 3)) || \
		echo "file2 badly fragmented"

when I merge it.

Thanks,
Zorro

Quoted 2 lines