Skip to contentSkip to conversation

new tests for zoned xfs

PATCH In review Christoph Hellwig 4 participants 34 messages May 1, 2025 – May 7, 2025

lore
Share & tools
lore
#0 lore raw

Hi all,

this series adds the various new tests for zoned xfs, including testing data placement.

#1 lore raw

PATCH 1/15 xfs: add a zoned growfs test

Zoned file systems require zone aligned RT volume sizes. Because of that xfs/596 won't work as-is. Copy it and test two zone capacity aligned values.

---
 tests/xfs/4202     | 81 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/4202.out | 11 +++++++
 2 files changed, 92 insertions(+)
 create mode 100755 tests/xfs/4202
 create mode 100644 tests/xfs/4202.out
tests/xfs/4202 new file +81 -0
new file mode 100755
index 000000000000..7830d48a8d2f
@@ -0,0 +1,81 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2000-2002 Silicon Graphics, Inc.  All Rights Reserved.
+# Copyright (c) 2024 Christoph Hellwig
+#
+# FS QA Test No. 4202
+#
+# growfs QA tests - repeatedly fill/grow the rt volume of the filesystem check
+# the filesystem contents after each operation.  This is the zoned equivalent of
+# xfs/596
+#
+. ./common/preamble
+_begin_fstest growfs ioctl auto zone
+
+_cleanup()
+{
+	cd /
+	_scratch_unmount
+	rm -f $tmp.*
+}
+
+. ./common/filter
+. ./common/zoned
+
+
+_require_scratch
+_require_realtime
+_require_no_large_scratch_dev
+
+zone_capacity=$(_zone_capacity 0 $SCRATCH_RTDEV)
+
+_fill()
+{
+    if [ $# -ne 1 ]; then echo "Usage: _fill \"path\"" 1>&2 ; exit 1; fi
+    _do "Fill filesystem" \
+	"$here/src/fill2fs --verbose --dir=$1 --seed=0 --filesize=65536 --stddev=32768 --list=- >>$tmp.manifest"
+}
+
+_do_die_on_error=message_only
+rtsize=$zone_capacity
+echo -n "Make rt filesystem on SCRATCH_DEV and mount... "
+_scratch_mkfs_xfs -r size=${rtsize} | \
+	_filter_mkfs 2> "$tmp.mkfs" >> $seqres.full
+
+# for $dbsize
+. $tmp.mkfs
+
+_scratch_mount
+_require_xfs_scratch_zoned
+
+# We're growing the realtime device, so force new file creation there
+_xfs_force_bdev realtime $SCRATCH_MNT
+
+echo "done"
+
+#
+# Zone RT devices can only grow by entire zones.
+# Do that twice.  The high starting code looks weird, but is neededed
+# due to the automatically added OP
+#
+for size in $(( 6 * $zone_capacity )) $(( 7 * $zone_capacity )); do
+    grow_size=$(( $size / $dbsize ))
+    _fill $SCRATCH_MNT/fill_$size
+    _do "Grow filesystem" "xfs_growfs -R $grow_size $SCRATCH_MNT"
+    echo -n "Flush filesystem... "
+    _do "_scratch_unmount"
+    _do "_try_scratch_mount"
+    echo "done"
+    echo -n "Check files... "
+    if ! _do "$here/src/fill2fs_check $tmp.manifest"; then
+      echo "fail (see $seqres.full)"
+      _do "cat $tmp.manifest"
+      _do "ls -altrR $SCRATCH_MNT"
+      status=1 ; exit
+    fi
+    echo "done"
+done
+
+# success, all done
+echo "Growfs tests passed."
+status=0 ; exit
tests/xfs/4202.out new file +11 -0
new file mode 100644
index 000000000000..66cab6aba8e2
@@ -0,0 +1,11 @@
+QA output created by 4202
+Make rt filesystem on SCRATCH_DEV and mount... done
+Fill filesystem... done
+Grow filesystem... done
+Flush filesystem... done
+Check files... done
+Fill filesystem... done
+Grow filesystem... done
+Flush filesystem... done
+Check files... done
+Growfs tests passed.
Signature · 1 line
2.47.2
#2 lore raw

PATCH 2/15 xfs: add a test for zoned block accounting after remount

In reply to Christoph Hellwig

Test for a problem with an earlier version of the zoned XFS mount code where freeded blocks in an open zone weren't properly accounted for.

---
 tests/xfs/4201     | 47 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/4201.out |  6 ++++++
 2 files changed, 53 insertions(+)
 create mode 100755 tests/xfs/4201
 create mode 100644 tests/xfs/4201.out
tests/xfs/4201 new file +47 -0
new file mode 100755
index 000000000000..5367291f3e87
@@ -0,0 +1,47 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2024 Christoph Hellwig.
+#
+# FS QA Test No. 4201
+#
+# Regression test for mount time accounting of an open zone with freed blocks.
+#
+
+. ./common/preamble
+_begin_fstest auto quick zone
+
+_require_scratch
+_require_odirect
+
+tmp=`mktemp -d`
+
+_cleanup()
+{
+	rm -rf $tmp
+}
+
+#
+# Create a 256MB file system.  This is picked as the lowest common zone size
+# to ensure both files are placed into the same zone.
+#
+_scratch_mkfs_sized $((256 * 1024 * 1024)) >> $seqres.full 2>&1
+_scratch_mount
+
+dd if=/dev/zero of=$SCRATCH_MNT/test1 oflag=direct conv=fsync bs=1M count=32
+dd if=/dev/zero of=$SCRATCH_MNT/test2 oflag=direct conv=fsync bs=1M count=32
+rm $SCRATCH_MNT/test1
+
+# let delayed inode deactivate do its work
+sleep 1
+df -h $SCRATCH_MNT > $tmp/df.old
+
+_scratch_cycle_mount
+
+echo "Check that df output matches after remount"
+df -h $SCRATCH_MNT > $tmp/df.new
+diff -u $tmp/df.old $tmp/df.new
+
+_scratch_unmount
+
+status=0
+exit
tests/xfs/4201.out new file +6 -0
new file mode 100644
index 000000000000..4cff86d90b0f
@@ -0,0 +1,6 @@
+QA output created by 4201
+32+0 records in
+32+0 records out
+32+0 records in
+32+0 records out
+Check that df output matches after remount
Signature · 1 line
2.47.2
#3 lore raw

Re: [PATCH 02/15] xfs: add a test for zoned block accounting after remount

Quoted 23 lines · Christoph Hellwig

Test for a problem with an earlier version of the zoned XFS mount code where freeded blocks in an open zone weren't properly accounted for.

---
 tests/xfs/4201     | 47 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/4201.out |  6 ++++++
 2 files changed, 53 insertions(+)
 create mode 100755 tests/xfs/4201
 create mode 100644 tests/xfs/4201.out
tests/xfs/4201 new file +6 -0
new file mode 100755
index 000000000000..5367291f3e87
@@ -0,0 +1,47 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2024 Christoph Hellwig.
+#
+# FS QA Test No. 4201
+#

tests/xfs/4201 line 16

Quoted 10 lines
tests/xfs/4201 new file +10 -0
+# Regression test for mount time accounting of an open zone with freed blocks.
+#
+
+. ./common/preamble
+_begin_fstest auto quick zone
+
+_require_scratch
+_require_odirect
+
+tmp=`mktemp -d`

Why not use "$tmp" directly, it's defined in _begin_fstest?

tests/xfs/4201 line 20

Quoted 4 lines
tests/xfs/4201 new file +4 -0
+
+_cleanup()
+{
+	rm -rf $tmp

Then we can save this specific _cleanup.

Quoted 6 lines
tests/xfs/4201 new file +6 -0
+}
+
+#
+# Create a 256MB file system.  This is picked as the lowest common zone size
+# to ensure both files are placed into the same zone.
+#

tests/xfs/4201 line 36

Quoted 10 lines
tests/xfs/4201 new file +10 -0
+_scratch_mkfs_sized $((256 * 1024 * 1024)) >> $seqres.full 2>&1
+_scratch_mount
+
+dd if=/dev/zero of=$SCRATCH_MNT/test1 oflag=direct conv=fsync bs=1M count=32
+dd if=/dev/zero of=$SCRATCH_MNT/test2 oflag=direct conv=fsync bs=1M count=32
+rm $SCRATCH_MNT/test1
+
+# let delayed inode deactivate do its work
+sleep 1
+df -h $SCRATCH_MNT > $tmp/df.old

then use $tmp.df.old and ..

tests/xfs/4201 line 42

Quoted 6 lines
tests/xfs/4201 new file +6 -0
+
+_scratch_cycle_mount
+
+echo "Check that df output matches after remount"
+df -h $SCRATCH_MNT > $tmp/df.new
+diff -u $tmp/df.old $tmp/df.new

$tmp.df.new

Thanks,
Zorro

Quoted 20 lines
tests/xfs/4201 new file +5 -0
+
+_scratch_unmount
+
+status=0
+exit
tests/xfs/4201.out new file +6 -0
new file mode 100644
index 000000000000..4cff86d90b0f
@@ -0,0 +1,6 @@
+QA output created by 4201
+32+0 records in
+32+0 records out
+32+0 records in
+32+0 records out
+Check that df output matches after remount
Signature · 1 line
2.47.2
#4 lore raw

PATCH 3/15 xfs: add test to check for block layer reordering

In reply to Christoph Hellwig

Zoned writes using zone append can be easily fragmented when the block layer or the driver reorders I/O. Check that a simple sequential direct write creates a single extent. This was broken in the kernel until recently when using the ->commit_rqs interface on devices with a relatively small max_hw_sectors / max_zone_append_sectors.

---
 tests/xfs/4203     | 37 +++++++++++++++++++++++++++++++++++++
 tests/xfs/4203.out |  5 +++++
 2 files changed, 42 insertions(+)
 create mode 100755 tests/xfs/4203
 create mode 100644 tests/xfs/4203.out
tests/xfs/4203 new file +37 -0
new file mode 100755
index 000000000000..9f0280c26fc8
@@ -0,0 +1,37 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2024 Christoph Hellwig
+#
+# FS QA Test No. 4203
+#
+# Ensure that direct I/O writes are not pointlessly reordered on zoned
+# devices.
+#
+# This is a regression test for the block layer or drivers reordering
+# writes and thus creating more extents than required.
+#
+. ./common/preamble
+_begin_fstest quick auto rw zone
+
+. ./common/filter
+. ./common/zoned
+
+_require_scratch
+_require_realtime
+
+_scratch_mkfs >/dev/null 2>&1
+
+_scratch_mount
+_require_xfs_scratch_zoned
+_xfs_force_bdev realtime $SCRATCH_MNT
+
+dd if=/dev/zero of=$SCRATCH_MNT/test bs=1M count=16 oflag=direct
+
+echo "Check extent counts"
+extents=$(_count_extents $SCRATCH_MNT/test)
+
+# There should not be more than a single extent when there are
+# no other concurrent writers
+echo "number of extents: $extents"
+
+status=0
tests/xfs/4203.out new file +5 -0
new file mode 100644
index 000000000000..f5aaece908fa
@@ -0,0 +1,5 @@
+QA output created by 4203
+16+0 records in
+16+0 records out
+Check extent counts
+number of extents: 1
Signature · 1 line
2.47.2
#5 lore raw

PATCH 4/15 xfs: add a test to check that data growfs fails with internal rt device

In reply to Christoph Hellwig

The internal RT device directly follows the data device on the same block device. This implies the data device can't be grown, and growfs should handle this gracefully.

---
 tests/xfs/4204     | 33 +++++++++++++++++++++++++++++++++
 tests/xfs/4204.out |  3 +++
 2 files changed, 36 insertions(+)
 create mode 100755 tests/xfs/4204
 create mode 100644 tests/xfs/4204.out
tests/xfs/4204 new file +33 -0
new file mode 100755
index 000000000000..0b73cee23ba5
@@ -0,0 +1,33 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2024 Christoph Hellwig.
+#
+# FS QA Test No. 4204
+#
+# Check that trying to grow a data device followed by the internal RT device
+# fails gracefully with EINVAL.
+#
+. ./common/preamble
+_begin_fstest quick auto growfs ioctl zone
+
+_cleanup()
+{
+	cd /
+	_scratch_unmount
+}
+
+# Import common functions.
+. ./common/filter
+
+_require_scratch
+_require_zoned_device $SCRATCH_DEV
+
+echo "Creating file system"
+_scratch_mkfs_xfs >>$seqres.full 2>&1
+_scratch_mount
+
+echo "Trying to grow file system (should fail)"
+$XFS_GROWFS_PROG -d $SCRATCH_MNT >>$seqres.full 2>&1
+
+status=0
+exit
tests/xfs/4204.out new file +3 -0
new file mode 100644
index 000000000000..b3593cf60d16
@@ -0,0 +1,3 @@
+QA output created by 4204
+Creating file system
+Trying to grow file system (should fail)
Signature · 1 line
2.47.2
#6 lore raw

Re: [PATCH 04/15] xfs: add a test to check that data growfs fails with internal rt device

Quoted 24 lines · Christoph Hellwig

The internal RT device directly follows the data device on the same block device. This implies the data device can't be grown, and growfs should handle this gracefully.

---
 tests/xfs/4204     | 33 +++++++++++++++++++++++++++++++++
 tests/xfs/4204.out |  3 +++
 2 files changed, 36 insertions(+)
 create mode 100755 tests/xfs/4204
 create mode 100644 tests/xfs/4204.out
tests/xfs/4204 new file +6 -0
new file mode 100755
index 000000000000..0b73cee23ba5
@@ -0,0 +1,33 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2024 Christoph Hellwig.
+#
+# FS QA Test No. 4204
+#

tests/xfs/4204 line 16

Quoted 10 lines
tests/xfs/4204 new file +10 -0
+# Check that trying to grow a data device followed by the internal RT device
+# fails gracefully with EINVAL.
+#
+. ./common/preamble
+_begin_fstest quick auto growfs ioctl zone
+
+_cleanup()
+{
+	cd /
+	_scratch_unmount

Won't $SCRATCH_DEV be unmounted after each test case done. Is there any reason we must do this in a specific _cleanup?

Quoted 29 lines
tests/xfs/4204 new file +17 -0
+}
+
+# Import common functions.
+. ./common/filter
+
+_require_scratch
+_require_zoned_device $SCRATCH_DEV
+
+echo "Creating file system"
+_scratch_mkfs_xfs >>$seqres.full 2>&1
+_scratch_mount
+
+echo "Trying to grow file system (should fail)"
+$XFS_GROWFS_PROG -d $SCRATCH_MNT >>$seqres.full 2>&1
+
+status=0
+exit
tests/xfs/4204.out new file +3 -0
new file mode 100644
index 000000000000..b3593cf60d16
@@ -0,0 +1,3 @@
+QA output created by 4204
+Creating file system
+Trying to grow file system (should fail)
Signature · 1 line
2.47.2
#7 lore raw

PATCH 5/15 add a new rw_hint helper

In reply to Christoph Hellwig

Add a tool to set the life time hint via fcntl.

---
 .gitignore    |  1 +
 src/Makefile  |  3 ++-
 src/rw_hint.c | 48 ++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 51 insertions(+), 1 deletion(-)
 create mode 100644 src/rw_hint.c
.gitignore +1 -0
index 4fd817243dca..f22cff8fb6c4 100644
@@ -203,6 +203,7 @@ tags
 /src/aio-dio-regress/aio-last-ref-held-by-io
 /src/aio-dio-regress/aiocp
 /src/aio-dio-regress/aiodio_sparse2
+/src/rw_hint
 /src/vfs/vfstest
 /src/vfs/mount-idmapped
 /src/log-writes/replay-log
src/Makefile +2 -1
index 6ac72b366257..2cc1fb40d9f1 100644
@@ -35,7 +35,8 @@ LINUX_TARGETS = xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader \
 	attr_replace_test swapon mkswap t_attr_corruption t_open_tmpfiles \
 	fscrypt-crypt-util bulkstat_null_ocount splice-test chprojid_fail \
 	detached_mounts_propagation ext4_resize t_readdir_3 splice2pipe \
-	uuid_ioctl t_snapshot_deleted_subvolume fiemap-fault min_dio_alignment
+	uuid_ioctl t_snapshot_deleted_subvolume fiemap-fault min_dio_alignment \
+	rw_hint
 
 EXTRA_EXECS = dmerror fill2attr fill2fs fill2fs_check scaleread.sh \
 	      btrfs_crc32c_forged_name.py popdir.pl popattr.py \
src/rw_hint.c new file +48 -0

48 lines not on this page · raw message

Signature · 1 line
2.47.2
#8 lore raw

PATCH 6/15 xfs: add a test for write lifetime hints

In reply to Christoph Hellwig

Test that the zone allocator actually places by temperature bucket.

---
 tests/xfs/4205     | 105 +++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/4205.out |   4 ++
 2 files changed, 109 insertions(+)
 create mode 100755 tests/xfs/4205
 create mode 100644 tests/xfs/4205.out
tests/xfs/4205 new file +105 -0
new file mode 100755
index 000000000000..be508806ec0a
@@ -0,0 +1,105 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2024 Christoph Hellwig.
+#
+# FS QA Test No. 4205
+#
+# Test data placement by write hints.
+#
+. ./common/preamble
+_begin_fstest auto rw zone
+
+_cleanup()
+{
+	cd /
+	_scratch_unmount >/dev/null 2>&1
+}
+
+# Import common functions.
+. ./common/filter
+
+_require_scratch
+
+_filter_rgno()
+{
+	# the rg number is in column 4 of xfs_bmap output
+	perl -ne '
+		$rg = (split /\s+/)[4] ;
+		if ($rg =~ /\d+/) {print "$rg "} ;
+	'
+}
+
+_test_placement()
+{
+	xfs_io_opts=$1
+
+	_scratch_mkfs_xfs >>$seqres.full 2>&1
+	_scratch_mount
+	_require_xfs_scratch_zoned 3
+
+	# Create a bunch of files for the three major temperature buckets
+	for i in `seq 1 100`; do
+		for hint in "short" "medium" "long"; do
+			file=$SCRATCH_MNT/$hint.$i
+
+			touch $file
+			$here/src/rw_hint $file $hint
+			$XFS_IO_PROG ${xfs_io_opts} \
+				-c 'pwrite 0 1m' \
+				$file >>$seqres.full
+		done
+	done
+
+	sync
+
+	# Check that all short lifetime files are placed together
+	short_rg=`xfs_bmap -v $SCRATCH_MNT/short.1 | _filter_rgno`
+	for i in `seq 2 100`; do
+		file=$SCRATCH_MNT/short.$i
+		rg=`xfs_bmap -v $file | _filter_rgno`
+		if [ "${rg}" != "${short_rg}" ]; then
+			echo "short RG mismatch for file $i: $short_rg/$rg"
+		fi
+	done
+
+	# Check that all medium lifetime files are placed together,
+	# but not in the short RG
+	medium_rg=`xfs_bmap -v $SCRATCH_MNT/medium.1 | _filter_rgno`
+	if [ "${medium}" == "${short_rg}" ]; then
+		echo "medium rg == short_rg"
+	fi
+	for i in `seq 2 100`; do
+		file=$SCRATCH_MNT/medium.$i
+		rg=`xfs_bmap -v $file | _filter_rgno`
+		if [ "${rg}" != "${medium_rg}" ]; then
+			echo "medium RG mismatch for file $i: $medium_rg/$rg"
+		fi
+	done
+
+	# Check that none of the long lifetime files are colocated with
+	# short and medium ones
+	for i in `seq 1 100`; do
+		file=$SCRATCH_MNT/long.$i
+		rg=`xfs_bmap -v $file | _filter_rgno`
+		if [ "${rg}" == "${short_rg}" ]; then
+			echo "long file $i placed into short RG "
+		fi
+		if [ "${rg}" == "${medium_rg}" ]; then
+			echo "long file $i placed into medium RG"
+		fi
+	done
+
+	_scratch_unmount
+}
+
+echo "Testing buffered I/O:"
+_test_placement ""
+
+echo "Testing synchronous buffered I/O:"
+_test_placement "-s"
+
+echo "Testing direct I/O:"
+_test_placement "-d"
+
+status=0
+exit
tests/xfs/4205.out new file +4 -0
new file mode 100644
index 000000000000..3331e361a36d
@@ -0,0 +1,4 @@
+QA output created by 4205
+Testing buffered I/O:
+Testing synchronous buffered I/O:
+Testing direct I/O:
Signature · 1 line
2.47.2
#9 lore raw

Re: [PATCH 06/15] xfs: add a test for write lifetime hints

Quoted 21 lines · Christoph Hellwig

Test that the zone allocator actually places by temperature bucket.

---
 tests/xfs/4205     | 105 +++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/4205.out |   4 ++
 2 files changed, 109 insertions(+)
 create mode 100755 tests/xfs/4205
 create mode 100644 tests/xfs/4205.out
tests/xfs/4205 new file +5 -0
new file mode 100755
index 000000000000..be508806ec0a
@@ -0,0 +1,105 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2024 Christoph Hellwig.
+#
+# FS QA Test No. 4205

tests/xfs/4205 line 15

Quoted 10 lines
tests/xfs/4205 new file +10 -0
+#
+# Test data placement by write hints.
+#
+. ./common/preamble
+_begin_fstest auto rw zone
+
+_cleanup()
+{
+	cd /
+	_scratch_unmount >/dev/null 2>&1

Is this unmount necessary here? If not, this _cleanup can be removed.

Quoted 7 lines
tests/xfs/4205 new file +7 -0
+}
+
+# Import common functions.
+. ./common/filter
+
+_require_scratch
+

tests/xfs/4205 line 32

Quoted 10 lines
tests/xfs/4205 new file +10 -0
+_filter_rgno()
+{
+	# the rg number is in column 4 of xfs_bmap output
+	perl -ne '
+		$rg = (split /\s+/)[4] ;
+		if ($rg =~ /\d+/) {print "$rg "} ;
+	'
+}
+
+_test_placement()

I think we don't remind using "_" prefix for local function of each test case :)

Quoted 86 lines
tests/xfs/4205 new file +73 -0
+{
+	xfs_io_opts=$1
+
+	_scratch_mkfs_xfs >>$seqres.full 2>&1
+	_scratch_mount
+	_require_xfs_scratch_zoned 3
+
+	# Create a bunch of files for the three major temperature buckets
+	for i in `seq 1 100`; do
+		for hint in "short" "medium" "long"; do
+			file=$SCRATCH_MNT/$hint.$i
+
+			touch $file
+			$here/src/rw_hint $file $hint
+			$XFS_IO_PROG ${xfs_io_opts} \
+				-c 'pwrite 0 1m' \
+				$file >>$seqres.full
+		done
+	done
+
+	sync
+
+	# Check that all short lifetime files are placed together
+	short_rg=`xfs_bmap -v $SCRATCH_MNT/short.1 | _filter_rgno`
+	for i in `seq 2 100`; do
+		file=$SCRATCH_MNT/short.$i
+		rg=`xfs_bmap -v $file | _filter_rgno`
+		if [ "${rg}" != "${short_rg}" ]; then
+			echo "short RG mismatch for file $i: $short_rg/$rg"
+		fi
+	done
+
+	# Check that all medium lifetime files are placed together,
+	# but not in the short RG
+	medium_rg=`xfs_bmap -v $SCRATCH_MNT/medium.1 | _filter_rgno`
+	if [ "${medium}" == "${short_rg}" ]; then
+		echo "medium rg == short_rg"
+	fi
+	for i in `seq 2 100`; do
+		file=$SCRATCH_MNT/medium.$i
+		rg=`xfs_bmap -v $file | _filter_rgno`
+		if [ "${rg}" != "${medium_rg}" ]; then
+			echo "medium RG mismatch for file $i: $medium_rg/$rg"
+		fi
+	done
+
+	# Check that none of the long lifetime files are colocated with
+	# short and medium ones
+	for i in `seq 1 100`; do
+		file=$SCRATCH_MNT/long.$i
+		rg=`xfs_bmap -v $file | _filter_rgno`
+		if [ "${rg}" == "${short_rg}" ]; then
+			echo "long file $i placed into short RG "
+		fi
+		if [ "${rg}" == "${medium_rg}" ]; then
+			echo "long file $i placed into medium RG"
+		fi
+	done
+
+	_scratch_unmount
+}
+
+echo "Testing buffered I/O:"
+_test_placement ""
+
+echo "Testing synchronous buffered I/O:"
+_test_placement "-s"
+
+echo "Testing direct I/O:"
+_test_placement "-d"
+
+status=0
+exit
tests/xfs/4205.out new file +4 -0
new file mode 100644
index 000000000000..3331e361a36d
@@ -0,0 +1,4 @@
+QA output created by 4205
+Testing buffered I/O:
+Testing synchronous buffered I/O:
+Testing direct I/O:
Signature · 1 line
2.47.2
#10 lore raw

PATCH 7/15 xfs: add a test for writeback after close

In reply to Christoph Hellwig

Test that files written back after closing are packed tightly instead of using up open zone resources.

---
 tests/xfs/4206     | 57 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/4206.out |  1 +
 2 files changed, 58 insertions(+)
 create mode 100755 tests/xfs/4206
 create mode 100644 tests/xfs/4206.out
tests/xfs/4206 new file +57 -0
new file mode 100755
index 000000000000..63e6aebeaeec
@@ -0,0 +1,57 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2024 Christoph Hellwig.
+#
+# FS QA Test No. 4206
+#
+# Test that data is packed tighly for writeback after the files were
+# closed.
+#
+. ./common/preamble
+_begin_fstest auto quick rw zone
+
+_cleanup()
+{
+	cd /
+	_scratch_unmount >/dev/null 2>&1
+}
+
+# Import common functions.
+. ./common/filter
+
+_require_scratch
+
+_filter_rgno()
+{
+	# the rg number is in column 4 of xfs_bmap output
+	perl -ne '
+		$rg = (split /\s+/)[4] ;
+		if ($rg =~ /\d+/) {print "$rg "} ;
+	'
+}
+
+_scratch_mkfs_xfs >>$seqres.full 2>&1
+_scratch_mount
+_require_xfs_scratch_zoned
+
+# Create a bunch of small files
+for i in `seq 1 100`; do
+	file=$SCRATCH_MNT/$i
+
+	$XFS_IO_PROG -f -c 'pwrite 0 8k' $file >>$seqres.full
+done
+
+sync
+
+# Check that all small files are placed together
+short_rg=`xfs_bmap -v $SCRATCH_MNT/1 | _filter_rgno`
+for i in `seq 2 100`; do
+	file=$SCRATCH_MNT/$i
+	rg=`xfs_bmap -v $file | _filter_rgno`
+	if [ "${rg}" != "${short_rg}" ]; then
+		echo "RG mismatch for file $i: $short_rg/$rg"
+	fi
+done
+
+status=0
+exit
tests/xfs/4206.out new file +1 -0
new file mode 100644
index 000000000000..4835b5053ae5
@@ -0,0 +1 @@
+QA output created by 4206
Signature · 1 line
2.47.2
#11 lore raw

Re: [PATCH 07/15] xfs: add a test for writeback after close

Quoted 23 lines · Christoph Hellwig

Test that files written back after closing are packed tightly instead of using up open zone resources.

---
 tests/xfs/4206     | 57 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/4206.out |  1 +
 2 files changed, 58 insertions(+)
 create mode 100755 tests/xfs/4206
 create mode 100644 tests/xfs/4206.out
tests/xfs/4206 new file +6 -0
new file mode 100755
index 000000000000..63e6aebeaeec
@@ -0,0 +1,57 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2024 Christoph Hellwig.
+#
+# FS QA Test No. 4206
+#

tests/xfs/4206 line 16

Quoted 10 lines
tests/xfs/4206 new file +10 -0
+# Test that data is packed tighly for writeback after the files were
+# closed.
+#
+. ./common/preamble
+_begin_fstest auto quick rw zone
+
+_cleanup()
+{
+	cd /
+	_scratch_unmount >/dev/null 2>&1

Same, is this unmount necessary?

tests/xfs/4206 line 24

Quoted 8 lines
tests/xfs/4206 new file +8 -0
+}
+
+# Import common functions.
+. ./common/filter
+
+_require_scratch
+
+_filter_rgno()

If this filter is useful for 1+ case, how about moving it to common/filter? Or remove the "_" prefix of a local function.

Quoted 44 lines
tests/xfs/4206 new file +33 -0
+{
+	# the rg number is in column 4 of xfs_bmap output
+	perl -ne '
+		$rg = (split /\s+/)[4] ;
+		if ($rg =~ /\d+/) {print "$rg "} ;
+	'
+}
+
+_scratch_mkfs_xfs >>$seqres.full 2>&1
+_scratch_mount
+_require_xfs_scratch_zoned
+
+# Create a bunch of small files
+for i in `seq 1 100`; do
+	file=$SCRATCH_MNT/$i
+
+	$XFS_IO_PROG -f -c 'pwrite 0 8k' $file >>$seqres.full
+done
+
+sync
+
+# Check that all small files are placed together
+short_rg=`xfs_bmap -v $SCRATCH_MNT/1 | _filter_rgno`
+for i in `seq 2 100`; do
+	file=$SCRATCH_MNT/$i
+	rg=`xfs_bmap -v $file | _filter_rgno`
+	if [ "${rg}" != "${short_rg}" ]; then
+		echo "RG mismatch for file $i: $short_rg/$rg"
+	fi
+done
+
+status=0
+exit
tests/xfs/4206.out new file +1 -0
new file mode 100644
index 000000000000..4835b5053ae5
@@ -0,0 +1 @@
+QA output created by 4206
Signature · 1 line
2.47.2
#12 lore raw

PATCH 8/15 xfs: test zone stream separation for two direct writers

In reply to Christoph Hellwig

Check that two parallel direct sequential writers are separated into different zones.

---
 tests/xfs/4207     | 79 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/4207.out |  3 ++
 2 files changed, 82 insertions(+)
 create mode 100755 tests/xfs/4207
 create mode 100644 tests/xfs/4207.out
tests/xfs/4207 new file +79 -0
new file mode 100755
index 000000000000..edc22da73bfb
@@ -0,0 +1,79 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2024 Christoph Hellwig.
+#
+# FS QA Test No. 4207
+#
+# Test that multiple direct I/O write streams are directed to separate zones.
+#
+. ./common/preamble
+_begin_fstest quick auto rw zone
+
+_cleanup()
+{
+	cd /
+	_scratch_unmount >/dev/null 2>&1
+}
+
+# Import common functions.
+. ./common/filter
+
+_require_scratch
+
+_filter_rgno()
+{
+	# the rg number is in column 4 of xfs_bmap output
+	perl -ne '
+		$rg = (split /\s+/)[4] ;
+		if ($rg =~ /\d+/) {print "$rg "} ;
+	'
+}
+
+_scratch_mkfs_xfs >>$seqres.full 2>&1
+_scratch_mount
+_require_xfs_scratch_zoned 3
+
+fio_config=$tmp.fio
+fio_out=$tmp.fio.out
+fio_err=$tmp.fio.err
+
+cat >$fio_config <<EOF
+[global]
+bs=64k
+iodepth=16
+iodepth_batch=8
+size=1m
+directory=$SCRATCH_MNT
+ioengine=libaio
+rw=write
+direct=1
+
+[file1]
+filename=file1
+size=128m
+
+[file2]
+filename=file2
+size=128m
+EOF
+
+_require_fio $fio_config
+
+$FIO_PROG $fio_config --output=$fio_out
+cat $fio_out >> $seqres.full
+
+# Check the files only have a single extent each and are in separate zones
+extents1=$(_count_extents $SCRATCH_MNT/file1)
+extents2=$(_count_extents $SCRATCH_MNT/file2)
+
+echo "number of file 1 extents: $extents1"
+echo "number of file 2 extents: $extents2"
+
+rg1=`xfs_bmap -v $SCRATCH_MNT/file1 | _filter_rgno`
+rg2=`xfs_bmap -v $SCRATCH_MNT/file2 | _filter_rgno`
+if [ "${rg1}" == "${rg2}" ]; then
+	echo "same RG used for both files"
+fi
+
+status=0
+exit
tests/xfs/4207.out new file +3 -0
new file mode 100644
index 000000000000..5d33658de474
@@ -0,0 +1,3 @@
+QA output created by 4207
+number of file 1 extents: 1
+number of file 2 extents: 1
Signature · 1 line
2.47.2
#13 lore raw

Re: [PATCH 08/15] xfs: test zone stream separation for two direct writers

Quoted 22 lines · Christoph Hellwig

Check that two parallel direct sequential writers are separated into different zones.

---
 tests/xfs/4207     | 79 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/4207.out |  3 ++
 2 files changed, 82 insertions(+)
 create mode 100755 tests/xfs/4207
 create mode 100644 tests/xfs/4207.out
tests/xfs/4207 new file +5 -0
new file mode 100755
index 000000000000..edc22da73bfb
@@ -0,0 +1,79 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2024 Christoph Hellwig.
+#
+# FS QA Test No. 4207

tests/xfs/4207 line 15

Quoted 10 lines
tests/xfs/4207 new file +10 -0
+#
+# Test that multiple direct I/O write streams are directed to separate zones.
+#
+. ./common/preamble
+_begin_fstest quick auto rw zone
+
+_cleanup()
+{
+	cd /
+	_scratch_unmount >/dev/null 2>&1

Same... remove this _cleanup if the "unmount" isn't necessary.

tests/xfs/4207 line 19

Quoted 4 lines
tests/xfs/4207 new file +4 -0
+}
+
+# Import common functions.
+. ./common/filter

You don't use any filter helpers in common/filter ...

tests/xfs/4207 line 23

Quoted 4 lines
tests/xfs/4207 new file +4 -0
+
+_require_scratch
+
+_filter_rgno()

I've seen this function in many cases of this patchset, how about move it to common/filter or common/xfs ?

Quoted 14 lines
tests/xfs/4207 new file +14 -0
+{
+	# the rg number is in column 4 of xfs_bmap output
+	perl -ne '
+		$rg = (split /\s+/)[4] ;
+		if ($rg =~ /\d+/) {print "$rg "} ;
+	'
+}
+
+_scratch_mkfs_xfs >>$seqres.full 2>&1
+_scratch_mount
+_require_xfs_scratch_zoned 3
+
+fio_config=$tmp.fio
+fio_out=$tmp.fio.out

tests/xfs/4207 line 47

Quoted 10 lines
tests/xfs/4207 new file +10 -0
+fio_err=$tmp.fio.err
+
+cat >$fio_config <<EOF
+[global]
+bs=64k
+iodepth=16
+iodepth_batch=8
+size=1m
+directory=$SCRATCH_MNT
+ioengine=libaio

_require_aio?

tests/xfs/4207 line 49

Quoted 2 lines
tests/xfs/4207 new file +2 -0
+rw=write
+direct=1

_require_odirect ? or _require_aiodio?

Quoted 43 lines
tests/xfs/4207 new file +30 -0
+
+[file1]
+filename=file1
+size=128m
+
+[file2]
+filename=file2
+size=128m
+EOF
+
+_require_fio $fio_config
+
+$FIO_PROG $fio_config --output=$fio_out
+cat $fio_out >> $seqres.full
+
+# Check the files only have a single extent each and are in separate zones
+extents1=$(_count_extents $SCRATCH_MNT/file1)
+extents2=$(_count_extents $SCRATCH_MNT/file2)
+
+echo "number of file 1 extents: $extents1"
+echo "number of file 2 extents: $extents2"
+
+rg1=`xfs_bmap -v $SCRATCH_MNT/file1 | _filter_rgno`
+rg2=`xfs_bmap -v $SCRATCH_MNT/file2 | _filter_rgno`
+if [ "${rg1}" == "${rg2}" ]; then
+	echo "same RG used for both files"
+fi
+
+status=0
+exit
tests/xfs/4207.out new file +3 -0
new file mode 100644
index 000000000000..5d33658de474
@@ -0,0 +1,3 @@
+QA output created by 4207
+number of file 1 extents: 1
+number of file 2 extents: 1
Signature · 1 line
2.47.2
#14 lore raw

PATCH 9/15 xfs: test zone stream separation for two buffered writers

In reply to Christoph Hellwig

Check that two parallel buffered sequential writers are separated into different zones when writeback happens before closing the files.

---
 tests/xfs/4208     | 79 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/4208.out |  3 ++
 2 files changed, 82 insertions(+)
 create mode 100755 tests/xfs/4208
 create mode 100644 tests/xfs/4208.out
tests/xfs/4208 new file +79 -0
new file mode 100755
index 000000000000..b85105704b65
@@ -0,0 +1,79 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2024 Christoph Hellwig.
+#
+# FS QA Test No. 4208
+#
+# Test that multiple buffered I/O write streams are directed to separate zones
+# when written back with the file still open.
+#
+. ./common/preamble
+_begin_fstest quick auto rw zone
+
+_cleanup()
+{
+	cd /
+	_scratch_unmount >/dev/null 2>&1
+}
+
+# Import common functions.
+. ./common/filter
+
+_require_scratch
+
+_filter_rgno()
+{
+	# the rg number is in column 4 of xfs_bmap output
+	perl -ne '
+		$rg = (split /\s+/)[4] ;
+		if ($rg =~ /\d+/) {print "$rg "} ;
+	'
+}
+
+_scratch_mkfs_xfs >>$seqres.full 2>&1
+_scratch_mount
+_require_xfs_scratch_zoned 3
+
+fio_config=$tmp.fio
+fio_out=$tmp.fio.out
+fio_err=$tmp.fio.err
+
+cat >$fio_config <<EOF
+[global]
+bs=64k
+iodepth=16
+iodepth_batch=8
+size=1m
+directory=$SCRATCH_MNT
+rw=write
+fsync_on_close=1
+
+[file1]
+filename=file1
+size=128m
+
+[file2]
+filename=file2
+size=128m
+EOF
+
+_require_fio $fio_config
+
+$FIO_PROG $fio_config --output=$fio_out
+cat $fio_out >> $seqres.full
+
+# Check the files only have a single extent each and are in separate zones
+extents1=$(_count_extents $SCRATCH_MNT/file1)
+extents2=$(_count_extents $SCRATCH_MNT/file2)
+
+echo "number of file 1 extents: $extents1"
+echo "number of file 2 extents: $extents2"
+
+rg1=`xfs_bmap -v $SCRATCH_MNT/file1 | _filter_rgno`
+rg2=`xfs_bmap -v $SCRATCH_MNT/file2 | _filter_rgno`
+if [ "${rg1}" == "${rg2}" ]; then
+	echo "same RG used for both files"
+fi
+
+status=0
+exit
tests/xfs/4208.out new file +3 -0
new file mode 100644
index 000000000000..1aaea308fe6a
@@ -0,0 +1,3 @@
+QA output created by 4208
+number of file 1 extents: 1
+number of file 2 extents: 1
Signature · 1 line
2.47.2
#15 lore raw

Re: [PATCH 09/15] xfs: test zone stream separation for two buffered writers

Quoted 23 lines · Christoph Hellwig

Check that two parallel buffered sequential writers are separated into different zones when writeback happens before closing the files.

---
 tests/xfs/4208     | 79 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/4208.out |  3 ++
 2 files changed, 82 insertions(+)
 create mode 100755 tests/xfs/4208
 create mode 100644 tests/xfs/4208.out
tests/xfs/4208 new file +6 -0
new file mode 100755
index 000000000000..b85105704b65
@@ -0,0 +1,79 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2024 Christoph Hellwig.
+#
+# FS QA Test No. 4208
+#

tests/xfs/4208 line 16

Quoted 10 lines
tests/xfs/4208 new file +10 -0
+# Test that multiple buffered I/O write streams are directed to separate zones
+# when written back with the file still open.
+#
+. ./common/preamble
+_begin_fstest quick auto rw zone
+
+_cleanup()
+{
+	cd /
+	_scratch_unmount >/dev/null 2>&1

Same review points with patch 8/15.

tests/xfs/4208 line 24

Quoted 8 lines
tests/xfs/4208 new file +8 -0
+}
+
+# Import common functions.
+. ./common/filter
+
+_require_scratch
+
+_filter_rgno()

Same review points with patch 8/15.

Quoted 67 lines
tests/xfs/4208 new file +55 -0
+{
+	# the rg number is in column 4 of xfs_bmap output
+	perl -ne '
+		$rg = (split /\s+/)[4] ;
+		if ($rg =~ /\d+/) {print "$rg "} ;
+	'
+}
+
+_scratch_mkfs_xfs >>$seqres.full 2>&1
+_scratch_mount
+_require_xfs_scratch_zoned 3
+
+fio_config=$tmp.fio
+fio_out=$tmp.fio.out
+fio_err=$tmp.fio.err
+
+cat >$fio_config <<EOF
+[global]
+bs=64k
+iodepth=16
+iodepth_batch=8
+size=1m
+directory=$SCRATCH_MNT
+rw=write
+fsync_on_close=1
+
+[file1]
+filename=file1
+size=128m
+
+[file2]
+filename=file2
+size=128m
+EOF
+
+_require_fio $fio_config
+
+$FIO_PROG $fio_config --output=$fio_out
+cat $fio_out >> $seqres.full
+
+# Check the files only have a single extent each and are in separate zones
+extents1=$(_count_extents $SCRATCH_MNT/file1)
+extents2=$(_count_extents $SCRATCH_MNT/file2)
+
+echo "number of file 1 extents: $extents1"
+echo "number of file 2 extents: $extents2"
+
+rg1=`xfs_bmap -v $SCRATCH_MNT/file1 | _filter_rgno`
+rg2=`xfs_bmap -v $SCRATCH_MNT/file2 | _filter_rgno`
+if [ "${rg1}" == "${rg2}" ]; then
+	echo "same RG used for both files"
+fi
+
+status=0
+exit
tests/xfs/4208.out new file +3 -0
new file mode 100644
index 000000000000..1aaea308fe6a
@@ -0,0 +1,3 @@
+QA output created by 4208
+number of file 1 extents: 1
+number of file 2 extents: 1
Signature · 1 line
2.47.2
#16 lore raw

PATCH 10/15 xfs: test zoned ENOSPC behavior with multiple writers

In reply to Christoph Hellwig

Test that multiple parallel writers can't accidentally dip into the reserved space pool.

---
 tests/xfs/4209     | 90 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/4209.out |  2 ++
 2 files changed, 92 insertions(+)
 create mode 100755 tests/xfs/4209
 create mode 100644 tests/xfs/4209.out
tests/xfs/4209 new file +90 -0
new file mode 100755
index 000000000000..18c84a968c40
@@ -0,0 +1,90 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2024 Christoph Hellwig.
+#
+# FS QA Test No. 4209
+#
+# Test that multiple parallel writers can't accidentally dip into the reserved
+# space pool.
+#
+. ./common/preamble
+_begin_fstest quick auto rw zone enospc
+
+_cleanup()
+{
+	cd /
+	_scratch_unmount >/dev/null 2>&1
+}
+
+# Import common functions.
+. ./common/filter
+. ./common/zoned
+
+_require_scratch
+
+_scratch_mkfs_sized $((256 * 1024 * 1024))  >>$seqres.full 2>&1
+
+# limit to two max open zones so that all writes get thrown into the blender
+export MOUNT_OPTIONS="$MOUNT_OPTIONS -o max_open_zones=2"
+_try_scratch_mount || _notrun "mount option not supported"
+_require_xfs_scratch_zoned
+
+fio_config=$tmp.fio
+
+cat >$fio_config <<EOF
+[global]
+bs=64k
+iodepth=16
+iodepth_batch=8
+directory=$SCRATCH_MNT
+ioengine=libaio
+rw=write
+direct=1
+size=60m
+
+[file1]
+filename=file1
+
+[file2]
+filename=file2
+
+[file3]
+filename=file3
+
+[file4]
+filename=file4
+
+[file5]
+filename=file5
+
+[file6]
+filename=file6
+
+[file7]
+filename=file7
+
+[file8]
+filename=file8
+EOF
+
+_require_fio $fio_config
+
+# try to overfill the file system
+$FIO_PROG $fio_config 2>&1 | \
+	grep -q "No space left on dev" || \
+	_fail "Overfill did not cause ENOSPC"
+
+sync
+
+#
+# Compare the df and du values to ensure we did not overshoot
+#
+# Use within_tolerance to paper over the fact that the du output includes
+# the root inode, which does not sit in the RT device, while df does not
+#
+df_val=`df --output=size $SCRATCH_MNT | tail -n 1`
+du_val=`du -s $SCRATCH_MNT | awk '{print $1}'`
+_within_tolerance "file space usage" $df_val $du_val 64 -v
+
+status=0
+exit
tests/xfs/4209.out new file +2 -0
new file mode 100644
index 000000000000..cb72138a1bf6
@@ -0,0 +1,2 @@
+QA output created by 4209
+file space usage is in range
Signature · 1 line
2.47.2
#17 lore raw

Re: [PATCH 10/15] xfs: test zoned ENOSPC behavior with multiple writers

Quoted 23 lines · Christoph Hellwig

Test that multiple parallel writers can't accidentally dip into the reserved space pool.

---
 tests/xfs/4209     | 90 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/4209.out |  2 ++
 2 files changed, 92 insertions(+)
 create mode 100755 tests/xfs/4209
 create mode 100644 tests/xfs/4209.out
tests/xfs/4209 new file +6 -0
new file mode 100755
index 000000000000..18c84a968c40
@@ -0,0 +1,90 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2024 Christoph Hellwig.
+#
+# FS QA Test No. 4209
+#

tests/xfs/4209 line 16

Quoted 10 lines
tests/xfs/4209 new file +10 -0
+# Test that multiple parallel writers can't accidentally dip into the reserved
+# space pool.
+#
+. ./common/preamble
+_begin_fstest quick auto rw zone enospc
+
+_cleanup()
+{
+	cd /
+	_scratch_unmount >/dev/null 2>&1

Same review points with patch 8/15

tests/xfs/4209 line 20

Quoted 4 lines
tests/xfs/4209 new file +4 -0
+}
+
+# Import common functions.
+. ./common/filter

Looks like this case doesn't use any filter helper.

Quoted 12 lines
tests/xfs/4209 new file +12 -0
+. ./common/zoned
+
+_require_scratch
+
+_scratch_mkfs_sized $((256 * 1024 * 1024))  >>$seqres.full 2>&1
+
+# limit to two max open zones so that all writes get thrown into the blender
+export MOUNT_OPTIONS="$MOUNT_OPTIONS -o max_open_zones=2"
+_try_scratch_mount || _notrun "mount option not supported"
+_require_xfs_scratch_zoned
+
+fio_config=$tmp.fio

tests/xfs/4209 line 42

Quoted 10 lines
tests/xfs/4209 new file +10 -0
+
+cat >$fio_config <<EOF
+[global]
+bs=64k
+iodepth=16
+iodepth_batch=8
+directory=$SCRATCH_MNT
+ioengine=libaio
+rw=write
+direct=1

Same review points with patch 8/15

Quoted 59 lines
tests/xfs/4209 new file +48 -0
+size=60m
+
+[file1]
+filename=file1
+
+[file2]
+filename=file2
+
+[file3]
+filename=file3
+
+[file4]
+filename=file4
+
+[file5]
+filename=file5
+
+[file6]
+filename=file6
+
+[file7]
+filename=file7
+
+[file8]
+filename=file8
+EOF
+
+_require_fio $fio_config
+
+# try to overfill the file system
+$FIO_PROG $fio_config 2>&1 | \
+	grep -q "No space left on dev" || \
+	_fail "Overfill did not cause ENOSPC"
+
+sync
+
+#
+# Compare the df and du values to ensure we did not overshoot
+#
+# Use within_tolerance to paper over the fact that the du output includes
+# the root inode, which does not sit in the RT device, while df does not
+#
+df_val=`df --output=size $SCRATCH_MNT | tail -n 1`
+du_val=`du -s $SCRATCH_MNT | awk '{print $1}'`
+_within_tolerance "file space usage" $df_val $du_val 64 -v
+
+status=0
+exit
tests/xfs/4209.out new file +2 -0
new file mode 100644
index 000000000000..cb72138a1bf6
@@ -0,0 +1,2 @@
+QA output created by 4209
+file space usage is in range
Signature · 1 line
2.47.2
#18 lore raw

PATCH 11/15 xfs: test zoned GC file defragmentation for sequential writers

In reply to Christoph Hellwig

Test that zoned GC defragments sequential writers forced into the same zone.

---
 tests/xfs/4210     | 124 +++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/4210.out |   5 ++
 2 files changed, 129 insertions(+)
 create mode 100755 tests/xfs/4210
 create mode 100644 tests/xfs/4210.out
tests/xfs/4210 new file +124 -0
new file mode 100755
index 000000000000..2984339fd86e
@@ -0,0 +1,124 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2024 Christoph Hellwig.
+#
+# FS QA Test No. 4210
+#
+# Test that GC defragments sequentially written files.
+#
+. ./common/preamble
+_begin_fstest auto rw zone
+
+_cleanup()
+{
+	cd /
+	_scratch_unmount >/dev/null 2>&1
+}
+
+# Import common functions.
+. ./common/filter
+. ./common/zoned
+
+_require_scratch
+
+_scratch_mkfs_sized $((256 * 1024 * 1024))  >>$seqres.full 2>&1
+
+# limit to two max open zones so that all writes get thrown into the blender
+export MOUNT_OPTIONS="$MOUNT_OPTIONS -o max_open_zones=2"
+_try_scratch_mount || _notrun "mount option not supported"
+_require_xfs_scratch_zoned
+
+fio_config=$tmp.fio
+fio_out=$tmp.fio.out
+fio_err=$tmp.fio.err
+
+cat >$fio_config <<EOF
+[global]
+bs=64k
+iodepth=16
+iodepth_batch=8
+directory=$SCRATCH_MNT
+ioengine=libaio
+rw=write
+direct=1
+size=30m
+
+[file1]
+filename=file1
+
+[file2]
+filename=file2
+
+[file3]
+filename=file3
+
+[file4]
+filename=file4
+
+[file5]
+filename=file5
+
+[file6]
+filename=file6
+
+[file7]
+filename=file7
+
+[file8]
+filename=file8
+EOF
+
+_require_fio $fio_config
+
+# create fragmented files
+$FIO_PROG $fio_config --output=$fio_out
+cat $fio_out >> $seqres.full
+
+# fill up all remaining user capacity
+dd if=/dev/zero of=$SCRATCH_MNT/fill bs=4k >> $seqres.full 2>&1
+
+sync
+
+# all files should be badly fragmented now
+extents2=$(_count_extents $SCRATCH_MNT/file2)
+echo "number of file 2 extents: $extents2" >>$seqres.full
+test $extents2 -gt 200 || _fail "fio did not fragment file"
+
+extents4=$(_count_extents $SCRATCH_MNT/file4)
+echo "number of file 4 extents: $extents4" >>$seqres.full
+test $extents4 -gt 200 || _fail "fio did not fragment file"
+
+extents6=$(_count_extents $SCRATCH_MNT/file6)
+echo "number of file 6 extents: $extents6" >>$seqres.full
+test $extents6 -gt 200 || _fail "fio did not fragment file"
+
+extents8=$(_count_extents $SCRATCH_MNT/file8)
+echo "number of file 8 extents: $extents8" >>$seqres.full
+test $extents8 -gt 200 || _fail "fio did not fragment file"
+
+# remove half of the files to create work for GC
+rm $SCRATCH_MNT/file1
+rm $SCRATCH_MNT/file3
+rm $SCRATCH_MNT/file5
+rm $SCRATCH_MNT/file7
+
+# fill up all remaining user capacity a few times to force GC
+for i in `seq 1 10`; do
+	dd if=/dev/zero of=$SCRATCH_MNT/fill bs=4k >> $seqres.full 2>&1
+	$XFS_IO_PROG -c "fsync" $SCRATCH_MNT/fill >> $seqres.full 2>&1
+done
+
+#
+# All files should have a no more than a handful of extents now
+#
+extents2=$(_count_extents $SCRATCH_MNT/file2)
+_within_tolerance "file 2 extents" $extents2 3 2 -v
+extents4=$(_count_extents $SCRATCH_MNT/file4)
+_within_tolerance "file 4 extents" $extents4 3 2 -v
+extents6=$(_count_extents $SCRATCH_MNT/file6)
+_within_tolerance "file 6 extents" $extents6 3 2 -v
+extents8=$(_count_extents $SCRATCH_MNT/file8)
+_within_tolerance "file 8 extents" $extents8 3 2 -v
+
+status=0
+exit
tests/xfs/4210.out new file +5 -0
new file mode 100644
index 000000000000..488dd9db790b
@@ -0,0 +1,5 @@
+QA output created by 4210
+file 2 extents is in range
+file 4 extents is in range
+file 6 extents is in range
+file 8 extents is in range
Signature · 1 line
2.47.2
#19 lore raw

Re: [PATCH 11/15] xfs: test zoned GC file defragmentation for sequential writers

Quoted 5 lines · Christoph Hellwig

Test that zoned GC defragments sequential writers forced into the same zone.

---

Same review points with patch 10/15

Quoted 150 lines
 tests/xfs/4210     | 124 +++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/4210.out |   5 ++
 2 files changed, 129 insertions(+)
 create mode 100755 tests/xfs/4210
 create mode 100644 tests/xfs/4210.out
tests/xfs/4210 new file +124 -0
new file mode 100755
index 000000000000..2984339fd86e
@@ -0,0 +1,124 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2024 Christoph Hellwig.
+#
+# FS QA Test No. 4210
+#
+# Test that GC defragments sequentially written files.
+#
+. ./common/preamble
+_begin_fstest auto rw zone
+
+_cleanup()
+{
+	cd /
+	_scratch_unmount >/dev/null 2>&1
+}
+
+# Import common functions.
+. ./common/filter
+. ./common/zoned
+
+_require_scratch
+
+_scratch_mkfs_sized $((256 * 1024 * 1024))  >>$seqres.full 2>&1
+
+# limit to two max open zones so that all writes get thrown into the blender
+export MOUNT_OPTIONS="$MOUNT_OPTIONS -o max_open_zones=2"
+_try_scratch_mount || _notrun "mount option not supported"
+_require_xfs_scratch_zoned
+
+fio_config=$tmp.fio
+fio_out=$tmp.fio.out
+fio_err=$tmp.fio.err
+
+cat >$fio_config <<EOF
+[global]
+bs=64k
+iodepth=16
+iodepth_batch=8
+directory=$SCRATCH_MNT
+ioengine=libaio
+rw=write
+direct=1
+size=30m
+
+[file1]
+filename=file1
+
+[file2]
+filename=file2
+
+[file3]
+filename=file3
+
+[file4]
+filename=file4
+
+[file5]
+filename=file5
+
+[file6]
+filename=file6
+
+[file7]
+filename=file7
+
+[file8]
+filename=file8
+EOF
+
+_require_fio $fio_config
+
+# create fragmented files
+$FIO_PROG $fio_config --output=$fio_out
+cat $fio_out >> $seqres.full
+
+# fill up all remaining user capacity
+dd if=/dev/zero of=$SCRATCH_MNT/fill bs=4k >> $seqres.full 2>&1
+
+sync
+
+# all files should be badly fragmented now
+extents2=$(_count_extents $SCRATCH_MNT/file2)
+echo "number of file 2 extents: $extents2" >>$seqres.full
+test $extents2 -gt 200 || _fail "fio did not fragment file"
+
+extents4=$(_count_extents $SCRATCH_MNT/file4)
+echo "number of file 4 extents: $extents4" >>$seqres.full
+test $extents4 -gt 200 || _fail "fio did not fragment file"
+
+extents6=$(_count_extents $SCRATCH_MNT/file6)
+echo "number of file 6 extents: $extents6" >>$seqres.full
+test $extents6 -gt 200 || _fail "fio did not fragment file"
+
+extents8=$(_count_extents $SCRATCH_MNT/file8)
+echo "number of file 8 extents: $extents8" >>$seqres.full
+test $extents8 -gt 200 || _fail "fio did not fragment file"
+
+# remove half of the files to create work for GC
+rm $SCRATCH_MNT/file1
+rm $SCRATCH_MNT/file3
+rm $SCRATCH_MNT/file5
+rm $SCRATCH_MNT/file7
+
+# fill up all remaining user capacity a few times to force GC
+for i in `seq 1 10`; do
+	dd if=/dev/zero of=$SCRATCH_MNT/fill bs=4k >> $seqres.full 2>&1
+	$XFS_IO_PROG -c "fsync" $SCRATCH_MNT/fill >> $seqres.full 2>&1
+done
+
+#
+# All files should have a no more than a handful of extents now
+#
+extents2=$(_count_extents $SCRATCH_MNT/file2)
+_within_tolerance "file 2 extents" $extents2 3 2 -v
+extents4=$(_count_extents $SCRATCH_MNT/file4)
+_within_tolerance "file 4 extents" $extents4 3 2 -v
+extents6=$(_count_extents $SCRATCH_MNT/file6)
+_within_tolerance "file 6 extents" $extents6 3 2 -v
+extents8=$(_count_extents $SCRATCH_MNT/file8)
+_within_tolerance "file 8 extents" $extents8 3 2 -v
+
+status=0
+exit
tests/xfs/4210.out new file +5 -0
new file mode 100644
index 000000000000..488dd9db790b
@@ -0,0 +1,5 @@
+QA output created by 4210
+file 2 extents is in range
+file 4 extents is in range
+file 6 extents is in range
+file 8 extents is in range
Signature · 1 line
2.47.2
#20 lore raw

PATCH 12/15 xfs: test zoned GC file defragmentation for random writers

In reply to Christoph Hellwig

Test that zoned GC defragments sequential writers forced into the same zone.

---
 tests/xfs/4211     | 129 +++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/4211.out |   5 ++
 2 files changed, 134 insertions(+)
 create mode 100755 tests/xfs/4211
 create mode 100644 tests/xfs/4211.out
tests/xfs/4211 new file +129 -0
new file mode 100755
index 000000000000..9679da72a4eb
@@ -0,0 +1,129 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2024 Christoph Hellwig.
+#
+# FS QA Test No. 4211
+#
+# Test that GC defragments randomly written files.
+#
+. ./common/preamble
+_begin_fstest auto rw zone
+
+_cleanup()
+{
+	cd /
+	_scratch_unmount >/dev/null 2>&1
+}
+
+# Import common functions.
+. ./common/filter
+. ./common/zoned
+
+_require_scratch
+
+_scratch_mkfs_sized $((256 * 1024 * 1024))  >>$seqres.full 2>&1
+
+# limit to two max open zones so that all writes get thrown into the blender
+export MOUNT_OPTIONS="$MOUNT_OPTIONS -o max_open_zones=2"
+_try_scratch_mount || _notrun "mount option not supported"
+_require_xfs_scratch_zoned
+
+fio_config=$tmp.fio
+fio_out=$tmp.fio.out
+fio_err=$tmp.fio.err
+
+cat >$fio_config <<EOF
+[global]
+bs=64k
+iodepth=16
+iodepth_batch=8
+directory=$SCRATCH_MNT
+ioengine=libaio
+rw=randwrite
+direct=1
+size=30m
+
+[file1]
+filename=file1
+
+[file2]
+filename=file2
+
+[file3]
+filename=file3
+
+[file4]
+filename=file4
+
+[file5]
+filename=file5
+
+[file6]
+filename=file6
+
+[file7]
+filename=file7
+
+[file8]
+filename=file8
+EOF
+
+_require_fio $fio_config
+
+# create fragmented files
+$FIO_PROG $fio_config --output=$fio_out
+cat $fio_out >> $seqres.full
+
+# fill up all remaining user capacity
+dd if=/dev/zero of=$SCRATCH_MNT/fill bs=4k >> $seqres.full 2>&1
+
+sync
+
+# all files should be badly fragmented now
+extents2=$(_count_extents $SCRATCH_MNT/file2)
+echo "number of file 2 extents: $extents2" >>$seqres.full
+test $extents2 -gt 200 || _fail "fio did not fragment file"
+
+extents4=$(_count_extents $SCRATCH_MNT/file4)
+echo "number of file 4 extents: $extents4" >>$seqres.full
+test $extents4 -gt 200 || _fail "fio did not fragment file"
+
+extents6=$(_count_extents $SCRATCH_MNT/file6)
+echo "number of file 6 extents: $extents6" >>$seqres.full
+test $extents6 -gt 200 || _fail "fio did not fragment file"
+
+extents8=$(_count_extents $SCRATCH_MNT/file8)
+echo "number of file 8 extents: $extents8" >>$seqres.full
+test $extents8 -gt 200 || _fail "fio did not fragment file"
+
+# remove half of the files to create work for GC
+rm $SCRATCH_MNT/file1
+rm $SCRATCH_MNT/file3
+rm $SCRATCH_MNT/file5
+rm $SCRATCH_MNT/file7
+
+#
+# Fill up all remaining user capacity a few times to force GC.
+#
+# This needs to be a very large number of larger zones sizes that have a lot
+# of OP for the small file system size
+#
+for i in `seq 1 200`; do
+	dd if=/dev/zero of=$SCRATCH_MNT/fill bs=4k >> $seqres.full 2>&1
+	$XFS_IO_PROG -c "fsync" $SCRATCH_MNT/fill >> $seqres.full 2>&1
+done
+
+#
+# All files should have a no more than a handful of extents now
+#
+extents2=$(_count_extents $SCRATCH_MNT/file2)
+_within_tolerance "file 2 extents" $extents2 3 2 -v
+extents4=$(_count_extents $SCRATCH_MNT/file4)
+_within_tolerance "file 4 extents" $extents4 3 2 -v
+extents6=$(_count_extents $SCRATCH_MNT/file6)
+_within_tolerance "file 6 extents" $extents6 3 2 -v
+extents8=$(_count_extents $SCRATCH_MNT/file8)
+_within_tolerance "file 8 extents" $extents8 3 2 -v
+
+status=0
+exit
tests/xfs/4211.out new file +5 -0
new file mode 100644
index 000000000000..348e59950a47
@@ -0,0 +1,5 @@
+QA output created by 4211
+file 2 extents is in range
+file 4 extents is in range
+file 6 extents is in range
+file 8 extents is in range
Signature · 1 line
2.47.2
#21 lore raw

Re: [PATCH 12/15] xfs: test zoned GC file defragmentation for random writers

Quoted 5 lines · Christoph Hellwig

Test that zoned GC defragments sequential writers forced into the same zone.

---

Same review points with patch 10/15

Quoted 156 lines
 tests/xfs/4211     | 129 +++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/4211.out |   5 ++
 2 files changed, 134 insertions(+)
 create mode 100755 tests/xfs/4211
 create mode 100644 tests/xfs/4211.out
tests/xfs/4211 new file +129 -0
new file mode 100755
index 000000000000..9679da72a4eb
@@ -0,0 +1,129 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2024 Christoph Hellwig.
+#
+# FS QA Test No. 4211
+#
+# Test that GC defragments randomly written files.
+#
+. ./common/preamble
+_begin_fstest auto rw zone
+
+_cleanup()
+{
+	cd /
+	_scratch_unmount >/dev/null 2>&1
+}
+
+# Import common functions.
+. ./common/filter
+. ./common/zoned
+
+_require_scratch
+
+_scratch_mkfs_sized $((256 * 1024 * 1024))  >>$seqres.full 2>&1
+
+# limit to two max open zones so that all writes get thrown into the blender
+export MOUNT_OPTIONS="$MOUNT_OPTIONS -o max_open_zones=2"
+_try_scratch_mount || _notrun "mount option not supported"
+_require_xfs_scratch_zoned
+
+fio_config=$tmp.fio
+fio_out=$tmp.fio.out
+fio_err=$tmp.fio.err
+
+cat >$fio_config <<EOF
+[global]
+bs=64k
+iodepth=16
+iodepth_batch=8
+directory=$SCRATCH_MNT
+ioengine=libaio
+rw=randwrite
+direct=1
+size=30m
+
+[file1]
+filename=file1
+
+[file2]
+filename=file2
+
+[file3]
+filename=file3
+
+[file4]
+filename=file4
+
+[file5]
+filename=file5
+
+[file6]
+filename=file6
+
+[file7]
+filename=file7
+
+[file8]
+filename=file8
+EOF
+
+_require_fio $fio_config
+
+# create fragmented files
+$FIO_PROG $fio_config --output=$fio_out
+cat $fio_out >> $seqres.full
+
+# fill up all remaining user capacity
+dd if=/dev/zero of=$SCRATCH_MNT/fill bs=4k >> $seqres.full 2>&1
+
+sync
+
+# all files should be badly fragmented now
+extents2=$(_count_extents $SCRATCH_MNT/file2)
+echo "number of file 2 extents: $extents2" >>$seqres.full
+test $extents2 -gt 200 || _fail "fio did not fragment file"
+
+extents4=$(_count_extents $SCRATCH_MNT/file4)
+echo "number of file 4 extents: $extents4" >>$seqres.full
+test $extents4 -gt 200 || _fail "fio did not fragment file"
+
+extents6=$(_count_extents $SCRATCH_MNT/file6)
+echo "number of file 6 extents: $extents6" >>$seqres.full
+test $extents6 -gt 200 || _fail "fio did not fragment file"
+
+extents8=$(_count_extents $SCRATCH_MNT/file8)
+echo "number of file 8 extents: $extents8" >>$seqres.full
+test $extents8 -gt 200 || _fail "fio did not fragment file"
+
+# remove half of the files to create work for GC
+rm $SCRATCH_MNT/file1
+rm $SCRATCH_MNT/file3
+rm $SCRATCH_MNT/file5
+rm $SCRATCH_MNT/file7
+
+#
+# Fill up all remaining user capacity a few times to force GC.
+#
+# This needs to be a very large number of larger zones sizes that have a lot
+# of OP for the small file system size
+#
+for i in `seq 1 200`; do
+	dd if=/dev/zero of=$SCRATCH_MNT/fill bs=4k >> $seqres.full 2>&1
+	$XFS_IO_PROG -c "fsync" $SCRATCH_MNT/fill >> $seqres.full 2>&1
+done
+
+#
+# All files should have a no more than a handful of extents now
+#
+extents2=$(_count_extents $SCRATCH_MNT/file2)
+_within_tolerance "file 2 extents" $extents2 3 2 -v
+extents4=$(_count_extents $SCRATCH_MNT/file4)
+_within_tolerance "file 4 extents" $extents4 3 2 -v
+extents6=$(_count_extents $SCRATCH_MNT/file6)
+_within_tolerance "file 6 extents" $extents6 3 2 -v
+extents8=$(_count_extents $SCRATCH_MNT/file8)
+_within_tolerance "file 8 extents" $extents8 3 2 -v
+
+status=0
+exit
tests/xfs/4211.out new file +5 -0
new file mode 100644
index 000000000000..348e59950a47
@@ -0,0 +1,5 @@
+QA output created by 4211
+file 2 extents is in range
+file 4 extents is in range
+file 6 extents is in range
+file 8 extents is in range
Signature · 1 line
2.47.2
#22 lore raw

PATCH 13/15 xfs: test that xfs_repair does not mess up the zone used counter

In reply to Christoph Hellwig

Check that xfs_repair actually rebuilds the used counter after blowing away the rmap inode and recreating it.

---
 tests/xfs/4212     | 32 ++++++++++++++++++++++++++++++++
 tests/xfs/4212.out |  5 +++++
 2 files changed, 37 insertions(+)
 create mode 100755 tests/xfs/4212
 create mode 100644 tests/xfs/4212.out
tests/xfs/4212 new file +32 -0
new file mode 100755
index 000000000000..f392a978c7a6
@@ -0,0 +1,32 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2024 Christoph Hellwig.
+#
+# FS QA Test No. 4212
+#
+# Regression test for xfs_repair messing up the per-zone used counter.
+#
+
+. ./common/preamble
+_begin_fstest auto quick zone repair
+
+_require_scratch
+_require_odirect
+
+_scratch_mkfs >> $seqres.full 2>&1
+_scratch_mount
+
+dd if=/dev/zero of=$SCRATCH_MNT/test1 oflag=direct bs=1M count=64
+
+_scratch_unmount
+
+echo "Repairing"
+_scratch_xfs_repair 2>> $seqres.full
+
+echo "Removing file after repair"
+_scratch_mount
+rm -f $SCRATCH_MNT/test1
+_scratch_unmount
+
+status=0
+exit
tests/xfs/4212.out new file +5 -0
new file mode 100644
index 000000000000..70a45a381f2d
@@ -0,0 +1,5 @@
+QA output created by 4212
+64+0 records in
+64+0 records out
+Repairing
+Removing file after repair
Signature · 1 line
2.47.2
#23 lore raw

PATCH 14/15 xfs: test that truncate does not spuriously return ENOSPC

In reply to Christoph Hellwig

For zoned file systems, truncate to an offset not aligned to the block size need to allocate a new block for zeroing the remainder.

Test that this allocation can dip into the reserved pool even when other threads are waiting for space freed by GC.

---
 tests/xfs/4213     | 45 +++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/4213.out |  1 +
 2 files changed, 46 insertions(+)
 create mode 100755 tests/xfs/4213
 create mode 100644 tests/xfs/4213.out
tests/xfs/4213 new file +45 -0
new file mode 100755
index 000000000000..1509307d39d0
@@ -0,0 +1,45 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2024 Christoph Hellwig.
+#
+# FS QA Test No. 4213
+#
+# Ensure that a truncate that needs to zero the EOFblock doesn't get ENOSPC
+# when another thread is waiting for space to become available through GC.
+#
+. ./common/preamble
+_begin_fstest auto rw zone
+
+_cleanup()
+{
+	cd /
+	_scratch_unmount >/dev/null 2>&1
+}
+
+# Import common functions.
+. ./common/filter
+. ./common/zoned
+
+_require_scratch
+
+_scratch_mkfs_sized $((256 * 1024 * 1024))  >>$seqres.full 2>&1
+_scratch_mount
+_require_xfs_scratch_zoned
+
+for i in `seq 1 20`; do
+	# fill up all user capacity
+	PUNCH_FILE=$SCRATCH_MNT/punch.$i
+	TEST_FILE=$SCRATCH_MNT/file.$i
+
+	dd if=/dev/zero of=$PUNCH_FILE bs=1M count=128 conv=fdatasync \
+		>> $seqres.full 2>&1
+
+	dd if=/dev/zero of=$TEST_FILE bs=4k >> $seqres.full 2>&1 &
+	# truncate to a value not rounded to the block size
+	$XFS_IO_PROG -c "truncate 3275" $PUNCH_FILE
+	sync $SCRATCH_MNT
+	rm -f $TEST_FILE
+done
+
+status=0
+exit
tests/xfs/4213.out new file +1 -0
new file mode 100644
index 000000000000..acf8716f9e13
@@ -0,0 +1 @@
+QA output created by 4213
Signature · 1 line
2.47.2
#24 lore raw

Re: [PATCH 14/15] xfs: test that truncate does not spuriously return ENOSPC

Quoted 26 lines · Christoph Hellwig

For zoned file systems, truncate to an offset not aligned to the block size need to allocate a new block for zeroing the remainder.

Test that this allocation can dip into the reserved pool even when other threads are waiting for space freed by GC.

---
 tests/xfs/4213     | 45 +++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/4213.out |  1 +
 2 files changed, 46 insertions(+)
 create mode 100755 tests/xfs/4213
 create mode 100644 tests/xfs/4213.out
tests/xfs/4213 new file +6 -0
new file mode 100755
index 000000000000..1509307d39d0
@@ -0,0 +1,45 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2024 Christoph Hellwig.
+#
+# FS QA Test No. 4213
+#

tests/xfs/4213 line 16

Quoted 10 lines
tests/xfs/4213 new file +10 -0
+# Ensure that a truncate that needs to zero the EOFblock doesn't get ENOSPC
+# when another thread is waiting for space to become available through GC.
+#
+. ./common/preamble
+_begin_fstest auto rw zone
+
+_cleanup()
+{
+	cd /
+	_scratch_unmount >/dev/null 2>&1

Remove this _cleanup if "unmount" isn't necessary at here.

Quoted 40 lines
tests/xfs/4213 new file +29 -0
+}
+
+# Import common functions.
+. ./common/filter
+. ./common/zoned
+
+_require_scratch
+
+_scratch_mkfs_sized $((256 * 1024 * 1024))  >>$seqres.full 2>&1
+_scratch_mount
+_require_xfs_scratch_zoned
+
+for i in `seq 1 20`; do
+	# fill up all user capacity
+	PUNCH_FILE=$SCRATCH_MNT/punch.$i
+	TEST_FILE=$SCRATCH_MNT/file.$i
+
+	dd if=/dev/zero of=$PUNCH_FILE bs=1M count=128 conv=fdatasync \
+		>> $seqres.full 2>&1
+
+	dd if=/dev/zero of=$TEST_FILE bs=4k >> $seqres.full 2>&1 &
+	# truncate to a value not rounded to the block size
+	$XFS_IO_PROG -c "truncate 3275" $PUNCH_FILE
+	sync $SCRATCH_MNT
+	rm -f $TEST_FILE
+done
+
+status=0
+exit
tests/xfs/4213.out new file +1 -0
new file mode 100644
index 000000000000..acf8716f9e13
@@ -0,0 +1 @@
+QA output created by 4213
Signature · 1 line
2.47.2
#25 lore raw

PATCH 15/15 xfs: test that we can handle spurious zone wp advancements

In reply to Christoph Hellwig

From: Hans Holmberg <[email protected]>

Test that we can gracefully handle spurious zone write pointer advancements while unmounted.

Any space covered by the wp unexpectedly moving forward should just be treated as unused space, so check that we can still mount the file system and that the zone will be reset when all used blocks have been freed.

---
 tests/xfs/4214     | 61 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/4214.out |  2 ++
 2 files changed, 63 insertions(+)
 create mode 100755 tests/xfs/4214
 create mode 100644 tests/xfs/4214.out
tests/xfs/4214 new file +61 -0
new file mode 100755
index 000000000000..3e73a54614d5
@@ -0,0 +1,61 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2025 Western Digital Corporation.  All Rights Reserved.
+#
+# FS QA Test 4214
+#
+# Test that we can gracefully handle spurious zone write pointer
+# advancements while unmounted.
+#
+
+. ./common/preamble
+_begin_fstest auto quick zone
+
+# Import common functions.
+. ./common/filter
+. ./common/zoned
+
+_require_scratch
+_require_zoned_device $SCRATCH_RTDEV
+_require_command "$BLKZONE_PROG" blkzone
+
+_scratch_mkfs >> $seqres.full 2>&1 || _fail "mkfs failed"
+_scratch_mount
+blksz=$(_get_file_block_size $SCRATCH_MNT)
+
+test_file=$SCRATCH_MNT/test.dat
+dd if=/dev/zero of=$test_file bs=1M count=16 >> $seqres.full 2>&1 \
+	oflag=direct || _fail "file creation failed"
+
+_scratch_unmount
+
+#
+# Figure out which zone was opened to store the test file and where
+# the write pointer is in that zone
+#
+open_zone=$($BLKZONE_PROG report $SCRATCH_RTDEV | \
+	$AWK_PROG '/oi/ { print $2 }' | sed 's/,//')
+open_zone_wp=$($BLKZONE_PROG report $SCRATCH_RTDEV | \
+       	grep "start: $open_zone" | $AWK_PROG '{ print $8 }')
+wp=$(( $open_zone + $open_zone_wp ))
+
+# Advance the write pointer manually by one block
+dd if=/dev/zero of=$SCRATCH_RTDEV bs=$blksz count=1 seek=$(($wp * 512 / $blksz))\
+	oflag=direct >> $seqres.full 2>&1 || _fail "wp advancement failed"
+
+_scratch_mount
+_scratch_unmount
+
+# Finish the open zone
+$BLKZONE_PROG finish -c 1 -o $open_zone $SCRATCH_RTDEV
+
+_scratch_mount
+rm $test_file
+_scratch_unmount
+
+# The previously open zone, now finished and unused, should have been reset
+nr_open=$($BLKZONE_PROG report $SCRATCH_RTDEV | grep -wc "oi")
+echo "Number of open zones: $nr_open"
+
+status=0
+exit
tests/xfs/4214.out new file +2 -0
new file mode 100644
index 000000000000..a746546bc8f6
@@ -0,0 +1,2 @@
+QA output created by 4214
+Number of open zones: 0
Signature · 1 line
2.47.2
#26 lore raw

Re: [PATCH 15/15] xfs: test that we can handle spurious zone wp advancements

Quoted 33 lines · Christoph Hellwig

From: Hans Holmberg <[email protected]>

Test that we can gracefully handle spurious zone write pointer advancements while unmounted.

Any space covered by the wp unexpectedly moving forward should just be treated as unused space, so check that we can still mount the file system and that the zone will be reset when all used blocks have been freed.

---
 tests/xfs/4214     | 61 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/4214.out |  2 ++
 2 files changed, 63 insertions(+)
 create mode 100755 tests/xfs/4214
 create mode 100644 tests/xfs/4214.out
tests/xfs/4214 new file +8 -0
new file mode 100755
index 000000000000..3e73a54614d5
@@ -0,0 +1,61 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2025 Western Digital Corporation.  All Rights Reserved.
+#
+# FS QA Test 4214
+#
+# Test that we can gracefully handle spurious zone write pointer
+# advancements while unmounted.

tests/xfs/4214 line 18

Quoted 10 lines
tests/xfs/4214 new file +10 -0
+#
+
+. ./common/preamble
+_begin_fstest auto quick zone
+
+# Import common functions.
+. ./common/filter
+. ./common/zoned
+
+_require_scratch

Needs _require_realtime so that the next command doesn't fail on undefined SCRATCH_RTDEV

Quoted 11 lines
tests/xfs/4214 new file +11 -0
+_require_zoned_device $SCRATCH_RTDEV
+_require_command "$BLKZONE_PROG" blkzone
+
+_scratch_mkfs >> $seqres.full 2>&1 || _fail "mkfs failed"
+_scratch_mount
+blksz=$(_get_file_block_size $SCRATCH_MNT)
+
+test_file=$SCRATCH_MNT/test.dat
+dd if=/dev/zero of=$test_file bs=1M count=16 >> $seqres.full 2>&1 \
+	oflag=direct || _fail "file creation failed"
+

tests/xfs/4214 line 39

Quoted 10 lines
tests/xfs/4214 new file +10 -0
+_scratch_unmount
+
+#
+# Figure out which zone was opened to store the test file and where
+# the write pointer is in that zone
+#
+open_zone=$($BLKZONE_PROG report $SCRATCH_RTDEV | \
+	$AWK_PROG '/oi/ { print $2 }' | sed 's/,//')
+open_zone_wp=$($BLKZONE_PROG report $SCRATCH_RTDEV | \
+       	grep "start: $open_zone" | $AWK_PROG '{ print $8 }')

^ spaces here before a tab

--D

Quoted 34 lines
tests/xfs/4214 new file +22 -0
+wp=$(( $open_zone + $open_zone_wp ))
+
+# Advance the write pointer manually by one block
+dd if=/dev/zero of=$SCRATCH_RTDEV bs=$blksz count=1 seek=$(($wp * 512 / $blksz))\
+	oflag=direct >> $seqres.full 2>&1 || _fail "wp advancement failed"
+
+_scratch_mount
+_scratch_unmount
+
+# Finish the open zone
+$BLKZONE_PROG finish -c 1 -o $open_zone $SCRATCH_RTDEV
+
+_scratch_mount
+rm $test_file
+_scratch_unmount
+
+# The previously open zone, now finished and unused, should have been reset
+nr_open=$($BLKZONE_PROG report $SCRATCH_RTDEV | grep -wc "oi")
+echo "Number of open zones: $nr_open"
+
+status=0
+exit
tests/xfs/4214.out new file +2 -0
new file mode 100644
index 000000000000..a746546bc8f6
@@ -0,0 +1,2 @@
+QA output created by 4214
+Number of open zones: 0
Signature · 1 line
2.47.2
#27 lore raw

Re: [PATCH 15/15] xfs: test that we can handle spurious zone wp advancements

Quoted 21 lines · Darrick J. Wong

Review

Quoted 4 lines

./common/filter
+. ./common/zoned

(quoted diff) +2 -0
+
+_require_scratch

Needs _require_realtime so that the next command doesn't fail on undefined SCRATCH_RTDEV

Quoted 14 lines
tests/xfs/4214 new file +14 -0
+_require_zoned_device $SCRATCH_RTDEV
+_require_command "$BLKZONE_PROG" blkzone
+
+_scratch_mkfs >> $seqres.full 2>&1 || _fail "mkfs failed"
+_scratch_mount
+blksz=$(_get_file_block_size $SCRATCH_MNT)
+
+test_file=$SCRATCH_MNT/test.dat
+dd if=/dev/zero of=$test_file bs=1M count=16 >> $seqres.full 2>&1 \
+	oflag=direct || _fail "file creation failed"
+
+_scratch_unmount
+
+#
Quoted 8 lines

tests/xfs/4214 line 39

Quoted 7 lines
tests/xfs/4214 new file +7 -0
+# Figure out which zone was opened to store the test file and where
+# the write pointer is in that zone
+#
+open_zone=$($BLKZONE_PROG report $SCRATCH_RTDEV | \
+	$AWK_PROG '/oi/ { print $2 }' | sed 's/,//')
+open_zone_wp=$($BLKZONE_PROG report $SCRATCH_RTDEV | \
+       	grep "start: $open_zone" | $AWK_PROG '{ print $8 }')

^ spaces here before a tab

Fixed, thanks!

#28 lore raw

PATCH v2 15/15 xfs: test that we can handle spurious zone wp advancements

In reply to Christoph Hellwig

From: Hans Holmberg <[email protected]>

Test that we can gracefully handle spurious zone write pointer advancements while unmounted.

Any space covered by the wp unexpectedly moving forward should just be treated as unused space, so check that we can still mount the file system and that the zone will be reset when all used blocks have been freed.

---

Canges since v1:
Added _require_realtime and fixed a white space error based on Darrick's review comments.

 tests/xfs/4214     | 62 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/4214.out |  2 ++
 2 files changed, 64 insertions(+)
 create mode 100755 tests/xfs/4214
 create mode 100644 tests/xfs/4214.out
tests/xfs/4214 new file +62 -0
new file mode 100755
index 000000000000..0637bbc7250e
@@ -0,0 +1,62 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2025 Western Digital Corporation.  All Rights Reserved.
+#
+# FS QA Test 4214
+#
+# Test that we can gracefully handle spurious zone write pointer
+# advancements while unmounted.
+#
+
+. ./common/preamble
+_begin_fstest auto quick zone
+
+# Import common functions.
+. ./common/filter
+. ./common/zoned
+
+_require_scratch
+_require_realtime
+_require_zoned_device $SCRATCH_RTDEV
+_require_command "$BLKZONE_PROG" blkzone
+
+_scratch_mkfs >> $seqres.full 2>&1 || _fail "mkfs failed"
+_scratch_mount
+blksz=$(_get_file_block_size $SCRATCH_MNT)
+
+test_file=$SCRATCH_MNT/test.dat
+dd if=/dev/zero of=$test_file bs=1M count=16 >> $seqres.full 2>&1 \
+	oflag=direct || _fail "file creation failed"
+
+_scratch_unmount
+
+#
+# Figure out which zone was opened to store the test file and where
+# the write pointer is in that zone
+#
+open_zone=$($BLKZONE_PROG report $SCRATCH_RTDEV | \
+	$AWK_PROG '/oi/ { print $2 }' | sed 's/,//')
+open_zone_wp=$($BLKZONE_PROG report $SCRATCH_RTDEV | \
+	grep "start: $open_zone" | $AWK_PROG '{ print $8 }')
+wp=$(( $open_zone + $open_zone_wp ))
+
+# Advance the write pointer manually by one block
+dd if=/dev/zero of=$SCRATCH_RTDEV bs=$blksz count=1 seek=$(($wp * 512 / $blksz))\
+	oflag=direct >> $seqres.full 2>&1 || _fail "wp advancement failed"
+
+_scratch_mount
+_scratch_unmount
+
+# Finish the open zone
+$BLKZONE_PROG finish -c 1 -o $open_zone $SCRATCH_RTDEV
+
+_scratch_mount
+rm $test_file
+_scratch_unmount
+
+# The previously open zone, now finished and unused, should have been reset
+nr_open=$($BLKZONE_PROG report $SCRATCH_RTDEV | grep -wc "oi")
+echo "Number of open zones: $nr_open"
+
+status=0
+exit
tests/xfs/4214.out new file +2 -0
new file mode 100644
index 000000000000..a746546bc8f6
@@ -0,0 +1,2 @@
+QA output created by 4214
+Number of open zones: 0
Signature · 1 line
2.34.1
Reviewed-by on v2 15/15
#29 lore raw

Re: [PATCH v2 15/15] xfs: test that we can handle spurious zone wp advancements

Quoted 5 lines · Hans Holmberg

From: Hans Holmberg <[email protected]>

Test that we can gracefully handle spurious zone write pointer advancements while unmounted.

Quoted 7 lines

Any space covered by the wp unexpectedly moving forward should just be treated as unused space, so check that we can still mount the file system and that the zone will be reset when all used blocks have been freed.

Looks good now,

--D

Quoted 91 lines

---

Canges since v1:
Added _require_realtime and fixed a white space error based on Darrick's review comments.

 tests/xfs/4214     | 62 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/4214.out |  2 ++
 2 files changed, 64 insertions(+)
 create mode 100755 tests/xfs/4214
 create mode 100644 tests/xfs/4214.out
tests/xfs/4214 new file +62 -0
new file mode 100755
index 000000000000..0637bbc7250e
@@ -0,0 +1,62 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2025 Western Digital Corporation.  All Rights Reserved.
+#
+# FS QA Test 4214
+#
+# Test that we can gracefully handle spurious zone write pointer
+# advancements while unmounted.
+#
+
+. ./common/preamble
+_begin_fstest auto quick zone
+
+# Import common functions.
+. ./common/filter
+. ./common/zoned
+
+_require_scratch
+_require_realtime
+_require_zoned_device $SCRATCH_RTDEV
+_require_command "$BLKZONE_PROG" blkzone
+
+_scratch_mkfs >> $seqres.full 2>&1 || _fail "mkfs failed"
+_scratch_mount
+blksz=$(_get_file_block_size $SCRATCH_MNT)
+
+test_file=$SCRATCH_MNT/test.dat
+dd if=/dev/zero of=$test_file bs=1M count=16 >> $seqres.full 2>&1 \
+	oflag=direct || _fail "file creation failed"
+
+_scratch_unmount
+
+#
+# Figure out which zone was opened to store the test file and where
+# the write pointer is in that zone
+#
+open_zone=$($BLKZONE_PROG report $SCRATCH_RTDEV | \
+	$AWK_PROG '/oi/ { print $2 }' | sed 's/,//')
+open_zone_wp=$($BLKZONE_PROG report $SCRATCH_RTDEV | \
+	grep "start: $open_zone" | $AWK_PROG '{ print $8 }')
+wp=$(( $open_zone + $open_zone_wp ))
+
+# Advance the write pointer manually by one block
+dd if=/dev/zero of=$SCRATCH_RTDEV bs=$blksz count=1 seek=$(($wp * 512 / $blksz))\
+	oflag=direct >> $seqres.full 2>&1 || _fail "wp advancement failed"
+
+_scratch_mount
+_scratch_unmount
+
+# Finish the open zone
+$BLKZONE_PROG finish -c 1 -o $open_zone $SCRATCH_RTDEV
+
+_scratch_mount
+rm $test_file
+_scratch_unmount
+
+# The previously open zone, now finished and unused, should have been reset
+nr_open=$($BLKZONE_PROG report $SCRATCH_RTDEV | grep -wc "oi")
+echo "Number of open zones: $nr_open"
+
+status=0
+exit
tests/xfs/4214.out new file +2 -0
new file mode 100644
index 000000000000..a746546bc8f6
@@ -0,0 +1,2 @@
+QA output created by 4214
+Number of open zones: 0
Signature · 1 line
2.34.1
#30 lore raw

Re: [PATCH v2 15/15] xfs: test that we can handle spurious zone wp advancements

In reply to Hans Holmberg
Quoted 5 lines · Hans Holmberg

From: Hans Holmberg <[email protected]>

Test that we can gracefully handle spurious zone write pointer advancements while unmounted.

Quoted 8 lines

Any space covered by the wp unexpectedly moving forward should just be treated as unused space, so check that we can still mount the file system and that the zone will be reset when all used blocks have been freed.

---

This's good to me, just below picky review points.

Quoted 22 lines

Canges since v1:
Added _require_realtime and fixed a white space error based on Darrick's review comments.

 tests/xfs/4214     | 62 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/4214.out |  2 ++
 2 files changed, 64 insertions(+)
 create mode 100755 tests/xfs/4214
 create mode 100644 tests/xfs/4214.out
tests/xfs/4214 new file +5 -0
new file mode 100755
index 000000000000..0637bbc7250e
@@ -0,0 +1,62 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2025 Western Digital Corporation.  All Rights Reserved.
+#
+# FS QA Test 4214

tests/xfs/4214 line 15

Quoted 10 lines
tests/xfs/4214 new file +10 -0
+#
+# Test that we can gracefully handle spurious zone write pointer
+# advancements while unmounted.
+#
+
+. ./common/preamble
+_begin_fstest auto quick zone
+
+# Import common functions.
+. ./common/filter

Looks like this case doesn't use any filter helper.

Quoted 4 lines
tests/xfs/4214 new file +4 -0
+. ./common/zoned
+
+_require_scratch
+_require_realtime

tests/xfs/4214 line 29

Quoted 10 lines
tests/xfs/4214 new file +10 -0
+_require_zoned_device $SCRATCH_RTDEV
+_require_command "$BLKZONE_PROG" blkzone
+
+_scratch_mkfs >> $seqres.full 2>&1 || _fail "mkfs failed"
+_scratch_mount
+blksz=$(_get_file_block_size $SCRATCH_MNT)
+
+test_file=$SCRATCH_MNT/test.dat
+dd if=/dev/zero of=$test_file bs=1M count=16 >> $seqres.full 2>&1 \
+	oflag=direct || _fail "file creation failed"

Why put the "oflag=direct" behind the ">> $seqres.full 2>&1" ? I'm saying it's wrong, just a bit weird :)

Thanks,
Zorro

Quoted 44 lines
tests/xfs/4214 new file +33 -0
+
+_scratch_unmount
+
+#
+# Figure out which zone was opened to store the test file and where
+# the write pointer is in that zone
+#
+open_zone=$($BLKZONE_PROG report $SCRATCH_RTDEV | \
+	$AWK_PROG '/oi/ { print $2 }' | sed 's/,//')
+open_zone_wp=$($BLKZONE_PROG report $SCRATCH_RTDEV | \
+	grep "start: $open_zone" | $AWK_PROG '{ print $8 }')
+wp=$(( $open_zone + $open_zone_wp ))
+
+# Advance the write pointer manually by one block
+dd if=/dev/zero of=$SCRATCH_RTDEV bs=$blksz count=1 seek=$(($wp * 512 / $blksz))\
+	oflag=direct >> $seqres.full 2>&1 || _fail "wp advancement failed"
+
+_scratch_mount
+_scratch_unmount
+
+# Finish the open zone
+$BLKZONE_PROG finish -c 1 -o $open_zone $SCRATCH_RTDEV
+
+_scratch_mount
+rm $test_file
+_scratch_unmount
+
+# The previously open zone, now finished and unused, should have been reset
+nr_open=$($BLKZONE_PROG report $SCRATCH_RTDEV | grep -wc "oi")
+echo "Number of open zones: $nr_open"
+
+status=0
+exit
tests/xfs/4214.out new file +2 -0
new file mode 100644
index 000000000000..a746546bc8f6
@@ -0,0 +1,2 @@
+QA output created by 4214
+Number of open zones: 0
Signature · 1 line
2.34.1
author
#31 lore raw

Re: [PATCH v2 15/15] xfs: test that we can handle spurious zone wp advancements

In reply to Hans Holmberg

While testing the updates for the comments from Zorro, it turns out that this doesn't handle internal RT devices correctly. Can you fix it up to handle that as well?

#32 lore raw

Re: [PATCH v2 15/15] xfs: test that we can handle spurious zone wp advancements

Quoted 5 lines · hch

While testing the updates for the comments from Zorro, it turns out that this doesn't handle internal RT devices correctly. Can you fix it up to handle that as well?

Argh. Yep, I'll fix it up and send it out separately.

Reviewed-by
#33 lore raw
In reply to Christoph Hellwig
Quoted 4 lines · Christoph Hellwig

Hi all,

this series adds the various new tests for zoned xfs, including testing data placement.

The first 14 patches I think I've already been running for a few weeks against software/hardware zoned support and haven't seen many problems. I will have a few fixups to apply after this, though.

So for patches 1-14,

(Will go look at #15 separately)

--D