Skip to contentSkip to conversation

Btrfs fstests fixups

PATCH In review David Sterba 2 participants 11 messages Mar 5, 2024 – Mar 7, 2024 Btrfs file system

lore
Share & tools
lore

David Sterba

author maintainer
#0 lore raw

cover Btrfs fstests fixups

Hi,

I'm sending first batch of fixups on behalf of Josef. I think most of them have been sent to fstests@ at some point. They have been in our CI branch and consider them tested and working for our needs. All of them are btrfs-specific and will not affect other filesystems. (The change in common/rc only moves a helper.) The last patch is a new test split from another one.

Josef Bacik (8):
btrfs/011: increase the runtime for replace cancel btrfs/012: adjust how we populate the fs to convert btrfs/131: don't run with subpage blocksizes
btrfs/213: make the test more reliable
btrfs/271: adjust failure condition
btrfs/287,btrfs/293: filter all btrfs subvolume delete calls btrfs/291: remove image file after teardown
btrfs/400: test normal qgroup operations in a compress friendly way

 check               |   6 ---
 common/rc           |   5 +++
 tests/btrfs/011     |   9 +++-
 tests/btrfs/012     |  14 +++---
 tests/btrfs/022     |  86 ++---------------------------------
 tests/btrfs/131     |   4 ++
 tests/btrfs/213     |  20 ++++-----
 tests/btrfs/271     |  11 ++---
 tests/btrfs/287     |   4 +-
 tests/btrfs/287.out |   2 +-
 tests/btrfs/291     |   2 +-
 tests/btrfs/293     |   6 +--
 tests/btrfs/293.out |   4 +-
 tests/btrfs/400     | 107 ++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/400.out |   2 +
 15 files changed, 161 insertions(+), 121 deletions(-)
 create mode 100755 tests/btrfs/400
 create mode 100644 tests/btrfs/400.out
Signature · 1 line
2.42.1
author maintainer
#1 lore raw

PATCH 1/8 btrfs/011: increase the runtime for replace cancel

From: Josef Bacik <[email protected]>

This test exercises the btrfs replace cancel path, but in order to do this we have to have enough work to do in order to successfully cancel the balance, otherwise the test fails because the operation has completed before we're able to cancel. This test has a very low pass rate because we do not generate a large enough file system for replace to have enough work, passing around 5% of the time. Increase the time spent to 10x the time we wait for the replace to start its work before we cancel, this allows us to consistently pass this test.

---
 tests/btrfs/011 | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)
tests/btrfs/011 +7 -2
index ff52ada94a17d0..d8b5a978275032 100755
@@ -60,6 +60,7 @@ wait_time=1
 fill_scratch()
 {
 	local fssize=$1
+	local with_cancel=$2
 	local filler_pid
 
 	# Fill inline extents.
@@ -88,7 +89,11 @@ fill_scratch()
 	$XFS_IO_PROG -f -d -c "pwrite -b 64k 0 1E" "$SCRATCH_MNT/t_filler" &>\
 		$tmp.filler_result &
 	filler_pid=$!
-	sleep $((2 * $wait_time))
+	if [ "${with_cancel}" = "cancel" ]; then
+		sleep $((10 * $wait_time))
+	else
+		sleep $((2 * $wait_time))
+	fi
 	kill -KILL $filler_pid &> /dev/null
 	wait $filler_pid &> /dev/null
 
@@ -125,7 +130,7 @@ workout()
 	_scratch_mount
 	_require_fs_space $SCRATCH_MNT $((2 * 512 * 1024)) #2.5G
 
-	fill_scratch $fssize
+	fill_scratch $fssize $with_cancel
 	_run_btrfs_util_prog filesystem show -m $SCRATCH_MNT
 
 	echo -e "Replace from $source_dev to $SPARE_DEV\\n" >> $seqres.full
Signature · 1 line
2.42.1
author maintainer
#2 lore raw

PATCH 2/8 btrfs/012: adjust how we populate the fs to convert

In reply to David Sterba

From: Josef Bacik <[email protected]>

/lib/modules/$(uname -r)/ can have symlinks to the source tree where the kernel was built from, which can have all sorts of stuff, which will make the runtime for this test exceedingly long. We're just trying to copy some data into our tree to test with, we don't need the entire devel tree of whatever we're doing. Additionally VM's that aren't built with modules will fail this test.

Update the test to use /etc, which will always exist. Additionally use timeout just in case there's large files or some other shenanigans so the test doesn't run forever copying large amounts of files.

---
 tests/btrfs/012 | 14 ++++++++------
 1 file changed, 8 insertions(+), 6 deletions(-)
tests/btrfs/012 +8 -6
index d9faf81ce1ad8e..7bc0c3ce59d28f 100755
@@ -33,6 +33,8 @@ _require_non_zoned_device "${SCRATCH_DEV}"
 _require_loop
 _require_extra_fs ext4
 
+SOURCE_DIR=/etc
+BASENAME=$(basename $SOURCE_DIR)
 BLOCK_SIZE=`_get_block_size $TEST_DIR`
 
 # Create & populate an ext4 filesystem
@@ -41,9 +43,9 @@ $MKFS_EXT4_PROG -F -b $BLOCK_SIZE $SCRATCH_DEV > $seqres.full 2>&1 || \
 # Manual mount so we don't use -t btrfs or selinux context
 mount -t ext4 $SCRATCH_DEV $SCRATCH_MNT
 
-_require_fs_space $SCRATCH_MNT $(du -s /lib/modules/`uname -r` | ${AWK_PROG} '{print $1}')
+_require_fs_space $SCRATCH_MNT $(du -s $SOURCE_DIR | ${AWK_PROG} '{print $1}')
 
-cp -aR /lib/modules/`uname -r`/ $SCRATCH_MNT
+timeout 10 cp -aRP $SOURCE_DIR $SCRATCH_MNT
 _scratch_unmount
 
 # Convert it to btrfs, mount it, verify the data
@@ -51,7 +53,7 @@ $BTRFS_CONVERT_PROG $SCRATCH_DEV >> $seqres.full 2>&1 || \
 	_fail "btrfs-convert failed"
 _try_scratch_mount || _fail "Could not mount new btrfs fs"
 # (Ignore the symlinks which may be broken/nonexistent)
-diff -r /lib/modules/`uname -r`/ $SCRATCH_MNT/`uname -r`/ 2>&1 | grep -vw "source\|build"
+diff --no-dereference -r $SOURCE_DIR $SCRATCH_MNT/$BASENAME/ 2>&1
 
 # Old ext4 image file should exist & be consistent
 $E2FSCK_PROG -fn $SCRATCH_MNT/ext2_saved/image >> $seqres.full 2>&1 || \
@@ -62,12 +64,12 @@ mkdir -p $SCRATCH_MNT/mnt
 mount -o loop $SCRATCH_MNT/ext2_saved/image $SCRATCH_MNT/mnt || \
 	_fail "could not loop mount saved ext4 image"
 # Ignore the symlinks which may be broken/nonexistent
-diff -r /lib/modules/`uname -r`/ $SCRATCH_MNT/mnt/`uname -r`/ 2>&1 | grep -vw "source\|build"
+diff --no-dereference -r $SOURCE_DIR $SCRATCH_MNT/mnt/$BASENAME/ 2>&1
 umount $SCRATCH_MNT/mnt
 
 # Now put some fresh data on the btrfs fs
 mkdir -p $SCRATCH_MNT/new 
-cp -aR /lib/modules/`uname -r`/ $SCRATCH_MNT/new
+timeout 10 cp -aRP $SOURCE_DIR $SCRATCH_MNT/new
 
 _scratch_unmount
 
@@ -82,7 +84,7 @@ $E2FSCK_PROG -fn $SCRATCH_DEV >> $seqres.full 2>&1 || \
 # Mount the un-converted ext4 device & check the contents
 mount -t ext4 $SCRATCH_DEV $SCRATCH_MNT
 # (Ignore the symlinks which may be broken/nonexistent)
-diff -r /lib/modules/`uname -r`/ $SCRATCH_MNT/`uname -r`/ 2>&1 | grep -vw "source\|build"
+diff --no-dereference -r $SOURCE_DIR $SCRATCH_MNT/$BASENAME/ 2>&1
 
 _scratch_unmount
 
Signature · 1 line
2.42.1
#3 lore raw

Re: [PATCH 2/8] btrfs/012: adjust how we populate the fs to convert

Quoted 32 lines · David Sterba

From: Josef Bacik <[email protected]>

/lib/modules/$(uname -r)/ can have symlinks to the source tree where the kernel was built from, which can have all sorts of stuff, which will make the runtime for this test exceedingly long. We're just trying to copy some data into our tree to test with, we don't need the entire devel tree of whatever we're doing. Additionally VM's that aren't built with modules will fail this test.

Update the test to use /etc, which will always exist. Additionally use timeout just in case there's large files or some other shenanigans so the test doesn't run forever copying large amounts of files.

---
  tests/btrfs/012 | 14 ++++++++------
  1 file changed, 8 insertions(+), 6 deletions(-)
tests/btrfs/012 +2 -0
index d9faf81ce1ad8e..7bc0c3ce59d28f 100755
@@ -33,6 +33,8 @@ _require_non_zoned_device "${SCRATCH_DEV}"
  _require_loop
  _require_extra_fs ext4
 
+SOURCE_DIR=/etc
+BASENAME=$(basename $SOURCE_DIR)
  BLOCK_SIZE=`_get_block_size $TEST_DIR`
 
  # Create & populate an ext4 filesystem

tests/btrfs/012 $MKFS_EXT4_PROG -F -b $BLOCK_SIZE $SCRATCH_DEV > $seqres.full 2>&1 || \ · line 48

Quoted 8 lines
tests/btrfs/012 +2 -2
@@ -41,9 +43,9 @@ $MKFS_EXT4_PROG -F -b $BLOCK_SIZE $SCRATCH_DEV > $seqres.full 2>&1 || \
  # Manual mount so we don't use -t btrfs or selinux context
  mount -t ext4 $SCRATCH_DEV $SCRATCH_MNT
 
-_require_fs_space $SCRATCH_MNT $(du -s /lib/modules/`uname -r` | ${AWK_PROG} '{print $1}')
+_require_fs_space $SCRATCH_MNT $(du -s $SOURCE_DIR | ${AWK_PROG} '{print $1}')
 
-cp -aR /lib/modules/`uname -r`/ $SCRATCH_MNT
+timeout 10 cp -aRP $SOURCE_DIR $SCRATCH_MNT

TIMEOUT_PROG

Quoted 15 lines
tests/btrfs/012 +1 -1
  _scratch_unmount
 
  # Convert it to btrfs, mount it, verify the data
@@ -51,7 +53,7 @@ $BTRFS_CONVERT_PROG $SCRATCH_DEV >> $seqres.full 2>&1 || \
  	_fail "btrfs-convert failed"
  _try_scratch_mount || _fail "Could not mount new btrfs fs"
  # (Ignore the symlinks which may be broken/nonexistent)
-diff -r /lib/modules/`uname -r`/ $SCRATCH_MNT/`uname -r`/ 2>&1 | grep -vw "source\|build"
+diff --no-dereference -r $SOURCE_DIR $SCRATCH_MNT/$BASENAME/ 2>&1
 
  # Old ext4 image file should exist & be consistent
  $E2FSCK_PROG -fn $SCRATCH_MNT/ext2_saved/image >> $seqres.full 2>&1 || \
@@ -62,12 +64,12 @@ mkdir -p $SCRATCH_MNT/mnt
  mount -o loop $SCRATCH_MNT/ext2_saved/image $SCRATCH_MNT/mnt || \
  	_fail "could not loop mount saved ext4 image"

tests/btrfs/012 mkdir -p $SCRATCH_MNT/mnt · line 73

Quoted 10 lines
tests/btrfs/012 +2 -2
  # Ignore the symlinks which may be broken/nonexistent
-diff -r /lib/modules/`uname -r`/ $SCRATCH_MNT/mnt/`uname -r`/ 2>&1 | grep -vw "source\|build"
+diff --no-dereference -r $SOURCE_DIR $SCRATCH_MNT/mnt/$BASENAME/ 2>&1
  umount $SCRATCH_MNT/mnt
 
  # Now put some fresh data on the btrfs fs
  mkdir -p $SCRATCH_MNT/new
-cp -aR /lib/modules/`uname -r`/ $SCRATCH_MNT/new
+timeout 10 cp -aRP $SOURCE_DIR $SCRATCH_MNT/new
 

TIMEOUT_PROG

looks good.

Added for the pr.

Quoted 11 lines
tests/btrfs/012 +1 -1
  _scratch_unmount
 
@@ -82,7 +84,7 @@ $E2FSCK_PROG -fn $SCRATCH_DEV >> $seqres.full 2>&1 || \
  # Mount the un-converted ext4 device & check the contents
  mount -t ext4 $SCRATCH_DEV $SCRATCH_MNT
  # (Ignore the symlinks which may be broken/nonexistent)
-diff -r /lib/modules/`uname -r`/ $SCRATCH_MNT/`uname -r`/ 2>&1 | grep -vw "source\|build"
+diff --no-dereference -r $SOURCE_DIR $SCRATCH_MNT/$BASENAME/ 2>&1
 
  _scratch_unmount
 
author maintainer
#4 lore raw

PATCH 3/8 btrfs/131: don't run with subpage blocksizes

In reply to David Sterba

From: Josef Bacik <[email protected]>

This test requires a feature that is incompatible with subpage blocksizes. Check to see if that's what we're testing and simply skip this test.

---
 tests/btrfs/131 | 4 ++++
 1 file changed, 4 insertions(+)
tests/btrfs/131 +4 -0
index 1e072b285ecfea..529ee3e80f87eb 100755
@@ -21,6 +21,10 @@ _require_btrfs_fs_feature free_space_tree
 # Zoned btrfs does not support space_cache(v1)
 _require_non_zoned_device "${SCRATCH_DEV}"
 
+_scratch_mkfs >/dev/null 2>&1
+[ "$(_get_page_size)" -gt "$(_scratch_btrfs_sectorsize)" ] && \
+	_notrun "cannot run with subpage sectorsize"
+
 mkfs_v1()
 {
 	_scratch_mkfs >/dev/null 2>&1
Signature · 1 line
2.42.1
author maintainer
#5 lore raw

PATCH 4/8 btrfs/213: make the test more reliable

In reply to David Sterba

From: Josef Bacik <[email protected]>

This test will write for 8 seconds and then try to balance, but for some setups 8 seconds may be enough to fill the disk. Instead figure out what half the size of the disk is and write at most that many bytes, or for 8 seconds, whichever comes first. Then use the amount of time it took to do the write to determine how long we should allow the balance to continue before we attempt to cancel it.

Additionally the macro is '_notrun' not '_not_run'. With this change this test now does the correct thing on my ARM CI VM.

---
 check           |  6 ------
 common/rc       |  5 +++++
 tests/btrfs/213 | 20 ++++++++++----------
 3 files changed, 15 insertions(+), 16 deletions(-)
check +0 -6
index 71b9fbd075223f..c6dba89b5b506e 100755
@@ -204,12 +204,6 @@ trim_test_list()
 	rm -f $tmp.grep
 }
 
-
-_wallclock()
-{
-    date "+%s"
-}
-
 _timestamp()
 {
     local now=`date "+%T"`
common/rc +5 -0
index 6cbceb7ae7c6bb..9b6dfcaaeddadb 100644
@@ -6,6 +6,11 @@
 
 BC="$(type -P bc)" || BC=
 
+_wallclock()
+{
+    date "+%s"
+}
+
 _require_math()
 {
 	if [ -z "$BC" ]; then
tests/btrfs/213 +10 -10
index 6def4f6ef79acf..816041a0cc2ea0 100755
@@ -31,23 +31,23 @@ _fixed_by_kernel_commit 1dae7e0e58b4 \
 _scratch_mkfs >> $seqres.full
 _scratch_mount
 
-runtime=8
+max_space=$(_get_total_space $SCRATCH_MNT)
+max_space=$(( max_space / 2 ))
 
-# Create enough IO so that we need around $runtime seconds to relocate it.
-#
-# Here we don't want any wrapper, as we want full control of the process.
-$XFS_IO_PROG -f -c "pwrite -D -b 1M 0 1024T" "$SCRATCH_MNT/file" &> /dev/null &
-write_pid=$!
-sleep $runtime
-kill $write_pid
-wait $write_pid
+# Create enough IO so that we need around 8 seconds to relocate it.
+start_ts=$(_wallclock)
+$TIMEOUT_PROG 8s $XFS_IO_PROG -f -c "pwrite -D -b 1M 0 $max_space" \
+	"$SCRATCH_MNT/file" > /dev/null 2>&1
+stop_ts=$(_wallclock)
+
+runtime=$(( stop_ts - start_ts ))
 
 # Unmount and mount again the fs to clear any cached data and metadata, so that
 # it's less likely balance has already finished when we try to cancel it below.
 _scratch_cycle_mount
 
 # Now balance should take at least $runtime seconds, we can cancel it at
-# $runtime/2 to ensure a success cancel.
+# $runtime/4 to ensure a success cancel.
 _run_btrfs_balance_start -d --bg "$SCRATCH_MNT"
 sleep $(($runtime / 4))
 # It's possible that balance has already completed. It's unlikely but often
Signature · 1 line
2.42.1
author maintainer
#6 lore raw

PATCH 5/8 btrfs/271: adjust failure condition

In reply to David Sterba

From: Josef Bacik <[email protected]>

btrfs/271 was failing with the subpage blocksize VM's. This is because there's an assumption made that the device error counters are per-sector, but they're per-io. With a 16kib pagesize and a 4k sectorsize/nodesize the threshold was expecting 16 failed IO's, but instead we were getting 5.

This other gotcha here is that with the tree log we will write the log tree first, and then update the log root tree with the location of the log tree root node. With pagesize == nodesize this is fine, we will only write the log tree root node. However with subpage blocksize both of these nodes could be on the same page, and thus they are both written out during that initial write. When we update the pointer for the log root tree we will COW the log root tree root node and submit another IO, resulting in 3 metadata IO's instead of 2.

Fix the failure case to be < 4 blocks, which is the minimum number of IO's we should be seeing.

---
 tests/btrfs/271 | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)
tests/btrfs/271 +6 -5
index 273799f17924c0..a342af3aed44cf 100755
@@ -25,10 +25,6 @@ _scratch_mount
 
 dev2=`echo $SCRATCH_DEV_POOL | $AWK_PROG '{print $2}'`
 
-pagesize=$(_get_page_size)
-blocksize=$(_get_block_size $SCRATCH_MNT)
-sectors_per_page=$(($pagesize / $blocksize))
-
 _allow_fail_make_request
 
 echo "Step 1: writing with one failing mirror:"
@@ -36,9 +32,14 @@ _bdev_fail_make_request $SCRATCH_DEV 1
 $XFS_IO_PROG -f -c "pwrite -W -S 0xaa 0 8K" $SCRATCH_MNT/foobar | _filter_xfs_io
 _bdev_fail_make_request $SCRATCH_DEV 0
 
+# btrfs counts errors per IO, assuming the data is merged that'll be 1 IO, then
+# the log tree block and then the log root tree block and then the super block.
+# We should see at least 4 failed IO's, but with subpage blocksize we could see
+# more if the log blocks end up on the same page, or if the data IO gets split
+# at all.
 errs=$($BTRFS_UTIL_PROG device stats $SCRATCH_DEV | \
 	$AWK_PROG '/write_io_errs/ { print $2 }')
-if [ $errs -ne $((4 * $sectors_per_page)) ]; then
+if [ $errs -lt 4 ]; then
         _fail "Errors: $errs expected: 4"
 fi
 
Signature · 1 line
2.42.1
author maintainer
#7 lore raw

PATCH 6/8 btrfs/287,btrfs/293: filter all btrfs subvolume delete calls

In reply to David Sterba

From: Josef Bacik <[email protected]>

Some of our btrfs subvolume delete calls get put into the golden output, and many of them simply _filter_scratch. This works fine, but we recently changed btrfs subvolume delete output, and it would have been nice to simply filter this in one place. We have a _filter_btrfs_subvol_delete helper, but it's only used in one place. Fix all of these uses to call _filter_btrfs_subvol_delete, this will allow for follow up fixes against _filter_btrfs_subvol_delete itself to deal with changed output.

---
 tests/btrfs/287     | 4 ++--
 tests/btrfs/287.out | 2 +-
 tests/btrfs/293     | 6 +++---
 tests/btrfs/293.out | 4 ++--
 4 files changed, 8 insertions(+), 8 deletions(-)
tests/btrfs/287 +2 -2
index 04871d46036aa2..64e6ef35250c8e 100755
@@ -9,7 +9,7 @@
 . ./common/preamble
 _begin_fstest auto quick snapshot clone punch logical_resolve
 
-. ./common/filter
+. ./common/filter.btrfs
 . ./common/reflink
 
 _supported_fs btrfs
@@ -148,7 +148,7 @@ echo "resolve second extent with ignore offset option:"
 query_logical_ino -o $second_extent_bytenr | filter_snapshot_ids
 
 # Now delete the first snapshot and repeat the last 2 queries.
-$BTRFS_UTIL_PROG subvolume delete -C $SCRATCH_MNT/snap1 | _filter_scratch
+$BTRFS_UTIL_PROG subvolume delete -C $SCRATCH_MNT/snap1 | _filter_btrfs_subvol_delete
 
 # Query the second extent with an offset of 0, should return file offsets 12M
 # and 20M for the default subvolume (root 5) and file offsets 4M, 12M and 20M
tests/btrfs/287.out +1 -1
index 0d69473364fa31..30eac8fa444c8f 100644
@@ -79,7 +79,7 @@ inode 257 offset 4194304 snap1
 inode 257 offset 20971520 root 5
 inode 257 offset 12582912 root 5
 inode 257 offset 5242880 root 5
-Delete subvolume (commit): 'SCRATCH_MNT/snap1'
+Delete subvolume 'SCRATCH_MNT/snap1'
 resolve second extent:
 inode 257 offset 20971520 snap2
 inode 257 offset 12582912 snap2
tests/btrfs/293 +3 -3
index cded956468ee9a..06f96dc414b05b 100755
@@ -18,7 +18,7 @@ _cleanup()
 	test -n "$swap_file" && swapoff $swap_file &> /dev/null
 }
 
-. ./common/filter
+. ./common/filter.btrfs
 
 _supported_fs btrfs
 _fixed_by_kernel_commit deccae40e4b3 \
@@ -40,7 +40,7 @@ echo "Activating swap file... (should fail due to snapshots)"
 _swapon_file $swap_file 2>&1 | _filter_scratch
 
 echo "Deleting first snapshot..."
-$BTRFS_UTIL_PROG subvolume delete -c $SCRATCH_MNT/snap1 | _filter_scratch
+$BTRFS_UTIL_PROG subvolume delete -c $SCRATCH_MNT/snap1 | _filter_btrfs_subvol_delete
 
 # We deleted the snapshot and committed the transaction used to delete it (-c),
 # but all its extents are actually only deleted in the background, by the cleaner
@@ -55,7 +55,7 @@ echo "Activating swap file... (should fail due to snapshot)"
 _swapon_file $swap_file 2>&1 | _filter_scratch
 
 echo "Deleting second snapshot..."
-$BTRFS_UTIL_PROG subvolume delete -c $SCRATCH_MNT/snap2 | _filter_scratch
+$BTRFS_UTIL_PROG subvolume delete -c $SCRATCH_MNT/snap2 | _filter_btrfs_subvol_delete
 
 echo "Remounting and waiting for cleaner thread to remove the second snapshot..."
 _scratch_remount commit=1
tests/btrfs/293.out +2 -2
index 2ac1663281947a..fd04ac9139b849 100644
@@ -6,12 +6,12 @@ Create a snapshot of 'SCRATCH_MNT' in 'SCRATCH_MNT/snap2'
 Activating swap file... (should fail due to snapshots)
 swapon: SCRATCH_MNT/swapfile: swapon failed: Invalid argument
 Deleting first snapshot...
-Delete subvolume (commit): 'SCRATCH_MNT/snap1'
+Delete subvolume 'SCRATCH_MNT/snap1'
 Remounting and waiting for cleaner thread to remove the first snapshot...
 Activating swap file... (should fail due to snapshot)
 swapon: SCRATCH_MNT/swapfile: swapon failed: Invalid argument
 Deleting second snapshot...
-Delete subvolume (commit): 'SCRATCH_MNT/snap2'
+Delete subvolume 'SCRATCH_MNT/snap2'
 Remounting and waiting for cleaner thread to remove the second snapshot...
 Activating swap file...
 Disabling swap file...
Signature · 1 line
2.42.1
author maintainer
#8 lore raw

PATCH 7/8 btrfs/291: remove image file after teardown

In reply to David Sterba

From: Josef Bacik <[email protected]>

LVM doesn't like it when you remove the file out from underneath the backing device.

---
 tests/btrfs/291 | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tests/btrfs/291 +1 -1
index c5947133239940..bfffb845fea740 100755
@@ -20,9 +20,9 @@ _cleanup()
 {
 	cd /
 	_log_writes_cleanup &> /dev/null
-	rm -f $img
 	$LVM_PROG vgremove -f -y $vgname >>$seqres.full 2>&1
 	losetup -d $loop_dev >>$seqres.full 2>&1
+	rm -f $img
 	_restore_fsverity_signatures
 }
 
Signature · 1 line
2.42.1
author maintainer
#9 lore raw

PATCH 8/8 btrfs/400: test normal qgroup operations in a compress friendly way

In reply to David Sterba

From: Josef Bacik <[email protected]>

btrfs/022 currently fails if you are testing with -o compress because it does a limit exceed test which will pass with compression on.

However the other functionality this test tests is completely acceptable with compression enabled. Handle this by breaking the test into two tests, one that simply tests the qgroup exceed limits test that requires no compression, and the rest of the tests that do not have the no compression restriction.

---
 tests/btrfs/022     |  86 ++---------------------------------
 tests/btrfs/400     | 107 ++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/400.out |   2 +
 3 files changed, 112 insertions(+), 83 deletions(-)
 create mode 100755 tests/btrfs/400
 create mode 100644 tests/btrfs/400.out
tests/btrfs/022 +3 -83
index b1ef2fdf787640..32ad80bf9c64e8 100755
@@ -4,7 +4,7 @@
 #
 # FS QA Test No. 022
 #
-# Test the basic functionality of qgroups
+# Test the basic qgroup exceed case
 #
 . ./common/preamble
 _begin_fstest auto qgroup limit
@@ -17,59 +17,8 @@ _require_scratch
 _require_qgroup_rescan
 _require_btrfs_qgroup_report
 
-# Test to make sure we can actually turn it on and it makes sense
-_basic_test()
-{
-	echo "=== basic test ===" >> $seqres.full
-	_run_btrfs_util_prog subvolume create $SCRATCH_MNT/a
-	_run_btrfs_util_prog quota enable $SCRATCH_MNT/a
-	_run_btrfs_util_prog quota rescan -w $SCRATCH_MNT
-	subvolid=$(_btrfs_get_subvolid $SCRATCH_MNT a)
-	$BTRFS_UTIL_PROG qgroup show $units $SCRATCH_MNT | grep $subvolid >> \
-		$seqres.full 2>&1
-	[ $? -eq 0 ] || _fail "couldn't find our subvols quota group"
-	run_check $FSSTRESS_PROG -d $SCRATCH_MNT/a -w -p 1 -n 2000 \
-		$FSSTRESS_AVOID
-	_run_btrfs_util_prog subvolume snapshot $SCRATCH_MNT/a \
-		$SCRATCH_MNT/b
-
-	# the shared values of both the original subvol and snapshot should
-	# match
-	a_shared=$($BTRFS_UTIL_PROG qgroup show $units $SCRATCH_MNT | grep "0/$subvolid")
-	a_shared=$(echo $a_shared | $AWK_PROG '{ print $2 }')
-	echo "subvol a id=$subvolid" >> $seqres.full
-	subvolid=$(_btrfs_get_subvolid $SCRATCH_MNT b)
-	echo "subvol b id=$subvolid" >> $seqres.full
-	b_shared=$($BTRFS_UTIL_PROG qgroup show $units $SCRATCH_MNT | grep "0/$subvolid")
-	b_shared=$(echo $b_shared | $AWK_PROG '{ print $2 }')
-	$BTRFS_UTIL_PROG qgroup show $units $SCRATCH_MNT >> $seqres.full
-	[ $b_shared -eq $a_shared ] || _fail "shared values don't match"
-}
-
-#enable quotas, do some work, check our values and then rescan and make sure we
-#come up with the same answer
-_rescan_test()
-{
-	echo "=== rescan test ===" >> $seqres.full
-	# first with a blank subvol
-	_run_btrfs_util_prog subvolume create $SCRATCH_MNT/a
-	_run_btrfs_util_prog quota enable $SCRATCH_MNT/a
-	subvolid=$(_btrfs_get_subvolid $SCRATCH_MNT a)
-	run_check $FSSTRESS_PROG -d $SCRATCH_MNT/a -w -p 1 -n 2000 \
-		$FSSTRESS_AVOID
-	sync
-	output=$($BTRFS_UTIL_PROG qgroup show $units $SCRATCH_MNT | grep "0/$subvolid")
-	echo "qgroup values before rescan: $output" >> $seqres.full
-	refer=$(echo $output | $AWK_PROG '{ print $2 }')
-	excl=$(echo $output | $AWK_PROG '{ print $3 }')
-	_run_btrfs_util_prog quota rescan -w $SCRATCH_MNT
-	output=$($BTRFS_UTIL_PROG qgroup show $units $SCRATCH_MNT | grep "0/$subvolid")
-	echo "qgroup values after rescan: $output" >> $seqres.full
-	[ $refer -eq $(echo $output | $AWK_PROG '{ print $2 }') ] || \
-		_fail "reference values don't match after rescan"
-	[ $excl -eq $(echo $output | $AWK_PROG '{ print $3 }') ] || \
-		_fail "exclusive values don't match after rescan"
-}
+# This test requires specific data usage, skip if we have compression enabled
+_require_no_compress
 
 #basic exceed limit testing
 _limit_test_exceed()
@@ -82,43 +31,14 @@ _limit_test_exceed()
 	_ddt of=$SCRATCH_MNT/a/file bs=10M count=1 >> $seqres.full 2>&1
 	[ $? -ne 0 ] || _fail "quota should have limited us"
 }
-
-#basic noexceed limit testing
-_limit_test_noexceed()
-{
-	echo "=== limit not exceed test ===" >> $seqres.full
-	_run_btrfs_util_prog subvolume create $SCRATCH_MNT/a
-	_run_btrfs_util_prog quota enable $SCRATCH_MNT
-	subvolid=$(_btrfs_get_subvolid $SCRATCH_MNT a)
-	_run_btrfs_util_prog qgroup limit 5M 0/$subvolid $SCRATCH_MNT
-	_ddt of=$SCRATCH_MNT/a/file bs=4M count=1 >> $seqres.full 2>&1
-	[ $? -eq 0 ] || _fail "should have been allowed to write"
-}
-
 units=`_btrfs_qgroup_units`
 
-_scratch_mkfs > /dev/null 2>&1
-_scratch_mount
-_basic_test
-_scratch_unmount
-_check_scratch_fs
-
-_scratch_mkfs > /dev/null 2>&1
-_scratch_mount
-_rescan_test
-_scratch_unmount
-_check_scratch_fs
-
 _scratch_mkfs > /dev/null 2>&1
 _scratch_mount
 _limit_test_exceed
 _scratch_unmount
 _check_scratch_fs
 
-_scratch_mkfs > /dev/null 2>&1
-_scratch_mount
-_limit_test_noexceed
-
 # success, all done
 echo "Silence is golden"
 status=0
tests/btrfs/400 new file +107 -0

107 lines not on this page · raw message

tests/btrfs/400.out new file +2 -0
new file mode 100644
index 00000000000000..c940c6206bcd81
@@ -0,0 +1,2 @@
+QA output created by 400
+Silence is golden
Signature · 1 line
2.42.1
Reviewed-by on the v1 series
#10 lore raw
In reply to David Sterba

Looks good.

for the whole series.

Staged for the PR

Thanks, Anand

Quoted 38 lines · David Sterba

Hi,

I'm sending first batch of fixups on behalf of Josef. I think most of them have been sent to fstests@ at some point. They have been in our CI branch and consider them tested and working for our needs. All of them are btrfs-specific and will not affect other filesystems. (The change in common/rc only moves a helper.) The last patch is a new test split from another one.

Josef Bacik (8):
btrfs/011: increase the runtime for replace cancel btrfs/012: adjust how we populate the fs to convert btrfs/131: don't run with subpage blocksizes
btrfs/213: make the test more reliable
btrfs/271: adjust failure condition
btrfs/287,btrfs/293: filter all btrfs subvolume delete calls btrfs/291: remove image file after teardown
btrfs/400: test normal qgroup operations in a compress friendly way

  check               |   6 ---
  common/rc           |   5 +++
  tests/btrfs/011     |   9 +++-
  tests/btrfs/012     |  14 +++---
  tests/btrfs/022     |  86 ++---------------------------------
  tests/btrfs/131     |   4 ++
  tests/btrfs/213     |  20 ++++-----
  tests/btrfs/271     |  11 ++---
  tests/btrfs/287     |   4 +-
  tests/btrfs/287.out |   2 +-
  tests/btrfs/291     |   2 +-
  tests/btrfs/293     |   6 +--
  tests/btrfs/293.out |   4 +-
  tests/btrfs/400     | 107 ++++++++++++++++++++++++++++++++++++++++++++
  tests/btrfs/400.out |   2 +
  15 files changed, 161 insertions(+), 121 deletions(-)

create mode 100755 tests/btrfs/400
create mode 100644 tests/btrfs/400.out