Skip to contentSkip to conversation

formalize and fix disabling the RT subvolume

PATCH In review Christoph Hellwig 2 participants 6 messages Jun 30 – Jul 2

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 formalize and fix disabling the RT subvolume

Various test unset SCRATCH_RTDEV to avoid dealing with metadata allocations for the RT subvolume in the main device or differences in RT subvolume behavior. Besides being rather ad-hoc, this unset also fails to disable the internal RT zoned subvolume.

Add a _disable_scratch_realtime helper that not only unsets SCRATCH_RTDEV, but also disables the zoned allocator and with that the internal RT subvolume.

---
 common/filestreams | 2 +-
 common/rc          | 9 +++++++++
 tests/generic/250  | 2 +-
 tests/generic/252  | 2 +-
 tests/generic/441  | 2 +-
 tests/generic/484  | 2 +-
 tests/generic/487  | 2 +-
 tests/xfs/076      | 2 +-
 tests/xfs/189      | 2 +-
 tests/xfs/205      | 2 +-
 tests/xfs/306      | 2 +-
 tests/xfs/444      | 2 +-
 tests/xfs/445      | 2 +-
 tests/xfs/655      | 2 +-
 tests/xfs/656      | 2 +-
 15 files changed, 23 insertions(+), 14 deletions(-)
common/filestreams +1 -1
index 838b5921520a..459de8c6b6a6 100644
@@ -84,7 +84,7 @@ _test_streams() {
 	# Disable the scratch rt device to avoid test failures relating to the
 	# rt bitmap consuming free space in our small data device and throwing
 	# off the filestreams allocator.
-	unset SCRATCH_RTDEV
+	_disable_scratch_realtime
 
 	# Skip these tests on zoned file systems as filestreams don't work
 	# with the zoned allocator, and the operation below would force it into
common/rc +9 -0
index 4947255479ca..d8d4d5292f8c 100644
@@ -2435,6 +2435,15 @@ _require_no_realtime()
 		_notrun "Test not compatible with realtime subvolumes, skipped this test"
 }
 
+# Disable usage of internal or external scratch devices
+_disable_scratch_realtime()
+{
+	unset SCRATCH_RTDEV
+	if [ "$FSTYP" = "xfs" ]; then
+		MKFS_OPTIONS="${MKFS_OPTIONS//-r zoned=1}"
+	fi
+}
+
 # this test requires that a specified command (executable) exists
 # $1 - command, $2 - name for error message
 #
tests/generic/250 +1 -1
index 605d7c8ade3d..32326114798c 100755
@@ -29,7 +29,7 @@ _require_odirect
 
 # Disable the scratch rt device to avoid test failures relating to the rt
 # bitmap consuming all the free space in our small data device.
-unset SCRATCH_RTDEV
+_disable_scratch_realtime
 
 fssize=$((196 * 1048576))
 echo "Format and mount"
tests/generic/252 +1 -1
index 2ad0524c2037..2e462b61fce4 100755
@@ -30,7 +30,7 @@ AIO_TEST="$here/src/aio-dio-regress/aiocp"
 
 # Disable the scratch rt device to avoid test failures relating to the rt
 # bitmap consuming all the free space in our small data device.
-unset SCRATCH_RTDEV
+_disable_scratch_realtime
 
 fssize=$((196 * 1048576))
 echo "Format and mount"
tests/generic/441 +1 -1
index cc370bea5052..c3a096926eaf 100755
@@ -47,7 +47,7 @@ _require_test_program dmerror
 
 # Disable the scratch rt device to avoid test failures relating to the rt
 # bitmap consuming all the free space in our small data device.
-unset SCRATCH_RTDEV
+_disable_scratch_realtime
 
 echo "Format and mount"
 _scratch_mkfs > $seqres.full 2>&1
tests/generic/484 +1 -1
index 0455efcb6000..e2fea8c6b18d 100755
@@ -35,7 +35,7 @@ _require_xfs_io_command "syncfs"
 
 # Disable the scratch rt device to avoid test failures relating to the rt
 # bitmap consuming all the free space in our small data device.
-unset SCRATCH_RTDEV
+_disable_scratch_realtime
 
 echo "Format and mount"
 _scratch_mkfs > $seqres.full 2>&1
tests/generic/487 +1 -1
index 364f7eb4c032..cc6c8f0e63d9 100755
@@ -40,7 +40,7 @@ _require_dm_target error
 
 # Disable the scratch rt device to avoid test failures relating to the rt
 # bitmap consuming all the free space in our small data device.
-unset SCRATCH_RTDEV
+_disable_scratch_realtime
 
 echo "Format and mount"
 _scratch_mkfs > $seqres.full 2>&1
tests/xfs/076 +1 -1
index 2ea6459b0dbe..103ad7509e0f 100755
@@ -66,7 +66,7 @@ _require_xfs_sparse_inodes
 
 # Disable the scratch rt device to avoid test failures relating to the rt
 # bitmap consuming all the free space in our small data device.
-unset SCRATCH_RTDEV
+_disable_scratch_realtime
 
 _scratch_mkfs "-d size=96m -m crc=1 -i sparse" | tee -a $seqres.full | \
 	_filter_mkfs > /dev/null 2> $tmp.mkfs
tests/xfs/189 +1 -1
index 1770023760fd..aa9712afac35 100755
@@ -250,7 +250,7 @@ _require_no_realtime
 _require_scratch
 _require_noattr2
 
-unset SCRATCH_RTDEV
+_disable_scratch_realtime
 unset SCRATCH_LOGDEV
 
 _scratch_mkfs_xfs | _filter_mkfs 2>/dev/null
tests/xfs/205 +1 -1
index 73d51d8d602f..7d8c0e5e33dc 100755
@@ -18,7 +18,7 @@ _require_scratch_nocheck
 
 # Disable the scratch rt device to avoid test failures relating to the rt
 # bitmap consuming all the free space in our small data device.
-unset SCRATCH_RTDEV
+_disable_scratch_realtime
 
 fsblksz=1024
 _scratch_mkfs_xfs -d size=$((32768*fsblksz)) -b size=$fsblksz >> $seqres.full 2>&1
tests/xfs/306 +1 -1
index d48b753632d5..60c2b486f671 100755
@@ -26,7 +26,7 @@ _require_test_program "punch-alternating"
 
 # Disable the scratch rt device to avoid test failures relating to the rt
 # bitmap consuming all the free space in our small data device.
-unset SCRATCH_RTDEV
+_disable_scratch_realtime
 
 # Create a small fs with a large directory block size. We want to fill up the fs
 # quickly and then create multi-fsb dirblocks over fragmented free space.
tests/xfs/444 +1 -1
index 13c9de528ec8..949c5711e2c8 100755
@@ -31,7 +31,7 @@ _require_scratch_xfs_crc
 # Disable the scratch rt device to avoid test failures relating to the rt
 # bitmap consuming free space in our small data device and throwing off the
 # filestreams allocator.
-unset SCRATCH_RTDEV
+_disable_scratch_realtime
 
 mount_loop() {
 	if ! _try_scratch_mount >> $seqres.full 2>&1; then
tests/xfs/445 +1 -1
index 0ea85ad018d6..2e9b6ca1be34 100755
@@ -42,7 +42,7 @@ _check_filestreams_support || _notrun "filestreams not available"
 # Disable the scratch rt device to avoid test failures relating to the rt
 # bitmap consuming free space in our small data device and throwing off the
 # filestreams allocator.
-unset SCRATCH_RTDEV
+_disable_scratch_realtime
 
 # use small AGs for frequent stream switching
 _scratch_mkfs_xfs -d agsize=20m,size=2g >> $seqres.full 2>&1
tests/xfs/655 +1 -1
index a4162d70b321..d6a3d4a95cc0 100755
@@ -26,7 +26,7 @@ _require_xfs_io_command healthmon
 
 # Disable the scratch rt device to avoid test failures relating to the rt
 # bitmap consuming all the free space in our small data device.
-unset SCRATCH_RTDEV
+_disable_scratch_realtime
 
 echo "Format and mount"
 _scratch_mkfs -d agcount=1 | _filter_mkfs 2> $tmp.mkfs >> $seqres.full
tests/xfs/656 +1 -1
index 3830d7193fcb..1eb82991521c 100755
@@ -42,7 +42,7 @@ filter_healer_errors() {
 
 # Disable the scratch rt device to avoid test failures relating to the rt
 # bitmap consuming all the free space in our small data device.
-unset SCRATCH_RTDEV
+_disable_scratch_realtime
 
 echo "Format and mount"
 _scratch_mkfs > $seqres.full 2>&1
Signature · 1 line
2.53.0
#1 lore raw
Quoted 8 lines · Christoph Hellwig

Various test unset SCRATCH_RTDEV to avoid dealing with metadata allocations for the RT subvolume in the main device or differences in RT subvolume behavior. Besides being rather ad-hoc, this unset also fails to disable the internal RT zoned subvolume.

Add a _disable_scratch_realtime helper that not only unsets SCRATCH_RTDEV, but also disables the zoned allocator and with that the internal RT subvolume.

What if we want to run these tests on the internal rt volume? I guess this means I should go look at what exactly happens on that QA config once I get the 7.2-rc1 rebase stabilized.

--D

Quoted 224 lines
---
 common/filestreams | 2 +-
 common/rc          | 9 +++++++++
 tests/generic/250  | 2 +-
 tests/generic/252  | 2 +-
 tests/generic/441  | 2 +-
 tests/generic/484  | 2 +-
 tests/generic/487  | 2 +-
 tests/xfs/076      | 2 +-
 tests/xfs/189      | 2 +-
 tests/xfs/205      | 2 +-
 tests/xfs/306      | 2 +-
 tests/xfs/444      | 2 +-
 tests/xfs/445      | 2 +-
 tests/xfs/655      | 2 +-
 tests/xfs/656      | 2 +-
 15 files changed, 23 insertions(+), 14 deletions(-)
common/filestreams +1 -1
index 838b5921520a..459de8c6b6a6 100644
@@ -84,7 +84,7 @@ _test_streams() {
 	# Disable the scratch rt device to avoid test failures relating to the
 	# rt bitmap consuming free space in our small data device and throwing
 	# off the filestreams allocator.
-	unset SCRATCH_RTDEV
+	_disable_scratch_realtime
 
 	# Skip these tests on zoned file systems as filestreams don't work
 	# with the zoned allocator, and the operation below would force it into
common/rc +9 -0
index 4947255479ca..d8d4d5292f8c 100644
@@ -2435,6 +2435,15 @@ _require_no_realtime()
 		_notrun "Test not compatible with realtime subvolumes, skipped this test"
 }
 
+# Disable usage of internal or external scratch devices
+_disable_scratch_realtime()
+{
+	unset SCRATCH_RTDEV
+	if [ "$FSTYP" = "xfs" ]; then
+		MKFS_OPTIONS="${MKFS_OPTIONS//-r zoned=1}"
+	fi
+}
+
 # this test requires that a specified command (executable) exists
 # $1 - command, $2 - name for error message
 #
tests/generic/250 +1 -1
index 605d7c8ade3d..32326114798c 100755
@@ -29,7 +29,7 @@ _require_odirect
 
 # Disable the scratch rt device to avoid test failures relating to the rt
 # bitmap consuming all the free space in our small data device.
-unset SCRATCH_RTDEV
+_disable_scratch_realtime
 
 fssize=$((196 * 1048576))
 echo "Format and mount"
tests/generic/252 +1 -1
index 2ad0524c2037..2e462b61fce4 100755
@@ -30,7 +30,7 @@ AIO_TEST="$here/src/aio-dio-regress/aiocp"
 
 # Disable the scratch rt device to avoid test failures relating to the rt
 # bitmap consuming all the free space in our small data device.
-unset SCRATCH_RTDEV
+_disable_scratch_realtime
 
 fssize=$((196 * 1048576))
 echo "Format and mount"
tests/generic/441 +1 -1
index cc370bea5052..c3a096926eaf 100755
@@ -47,7 +47,7 @@ _require_test_program dmerror
 
 # Disable the scratch rt device to avoid test failures relating to the rt
 # bitmap consuming all the free space in our small data device.
-unset SCRATCH_RTDEV
+_disable_scratch_realtime
 
 echo "Format and mount"
 _scratch_mkfs > $seqres.full 2>&1
tests/generic/484 +1 -1
index 0455efcb6000..e2fea8c6b18d 100755
@@ -35,7 +35,7 @@ _require_xfs_io_command "syncfs"
 
 # Disable the scratch rt device to avoid test failures relating to the rt
 # bitmap consuming all the free space in our small data device.
-unset SCRATCH_RTDEV
+_disable_scratch_realtime
 
 echo "Format and mount"
 _scratch_mkfs > $seqres.full 2>&1
tests/generic/487 +1 -1
index 364f7eb4c032..cc6c8f0e63d9 100755
@@ -40,7 +40,7 @@ _require_dm_target error
 
 # Disable the scratch rt device to avoid test failures relating to the rt
 # bitmap consuming all the free space in our small data device.
-unset SCRATCH_RTDEV
+_disable_scratch_realtime
 
 echo "Format and mount"
 _scratch_mkfs > $seqres.full 2>&1
tests/xfs/076 +1 -1
index 2ea6459b0dbe..103ad7509e0f 100755
@@ -66,7 +66,7 @@ _require_xfs_sparse_inodes
 
 # Disable the scratch rt device to avoid test failures relating to the rt
 # bitmap consuming all the free space in our small data device.
-unset SCRATCH_RTDEV
+_disable_scratch_realtime
 
 _scratch_mkfs "-d size=96m -m crc=1 -i sparse" | tee -a $seqres.full | \
 	_filter_mkfs > /dev/null 2> $tmp.mkfs
tests/xfs/189 +1 -1
index 1770023760fd..aa9712afac35 100755
@@ -250,7 +250,7 @@ _require_no_realtime
 _require_scratch
 _require_noattr2
 
-unset SCRATCH_RTDEV
+_disable_scratch_realtime
 unset SCRATCH_LOGDEV
 
 _scratch_mkfs_xfs | _filter_mkfs 2>/dev/null
tests/xfs/205 +1 -1
index 73d51d8d602f..7d8c0e5e33dc 100755
@@ -18,7 +18,7 @@ _require_scratch_nocheck
 
 # Disable the scratch rt device to avoid test failures relating to the rt
 # bitmap consuming all the free space in our small data device.
-unset SCRATCH_RTDEV
+_disable_scratch_realtime
 
 fsblksz=1024
 _scratch_mkfs_xfs -d size=$((32768*fsblksz)) -b size=$fsblksz >> $seqres.full 2>&1
tests/xfs/306 +1 -1
index d48b753632d5..60c2b486f671 100755
@@ -26,7 +26,7 @@ _require_test_program "punch-alternating"
 
 # Disable the scratch rt device to avoid test failures relating to the rt
 # bitmap consuming all the free space in our small data device.
-unset SCRATCH_RTDEV
+_disable_scratch_realtime
 
 # Create a small fs with a large directory block size. We want to fill up the fs
 # quickly and then create multi-fsb dirblocks over fragmented free space.
tests/xfs/444 +1 -1
index 13c9de528ec8..949c5711e2c8 100755
@@ -31,7 +31,7 @@ _require_scratch_xfs_crc
 # Disable the scratch rt device to avoid test failures relating to the rt
 # bitmap consuming free space in our small data device and throwing off the
 # filestreams allocator.
-unset SCRATCH_RTDEV
+_disable_scratch_realtime
 
 mount_loop() {
 	if ! _try_scratch_mount >> $seqres.full 2>&1; then
tests/xfs/445 +1 -1
index 0ea85ad018d6..2e9b6ca1be34 100755
@@ -42,7 +42,7 @@ _check_filestreams_support || _notrun "filestreams not available"
 # Disable the scratch rt device to avoid test failures relating to the rt
 # bitmap consuming free space in our small data device and throwing off the
 # filestreams allocator.
-unset SCRATCH_RTDEV
+_disable_scratch_realtime
 
 # use small AGs for frequent stream switching
 _scratch_mkfs_xfs -d agsize=20m,size=2g >> $seqres.full 2>&1
tests/xfs/655 +1 -1
index a4162d70b321..d6a3d4a95cc0 100755
@@ -26,7 +26,7 @@ _require_xfs_io_command healthmon
 
 # Disable the scratch rt device to avoid test failures relating to the rt
 # bitmap consuming all the free space in our small data device.
-unset SCRATCH_RTDEV
+_disable_scratch_realtime
 
 echo "Format and mount"
 _scratch_mkfs -d agcount=1 | _filter_mkfs 2> $tmp.mkfs >> $seqres.full
tests/xfs/656 +1 -1
index 3830d7193fcb..1eb82991521c 100755
@@ -42,7 +42,7 @@ filter_healer_errors() {
 
 # Disable the scratch rt device to avoid test failures relating to the rt
 # bitmap consuming all the free space in our small data device.
-unset SCRATCH_RTDEV
+_disable_scratch_realtime
 
 echo "Format and mount"
 _scratch_mkfs > $seqres.full 2>&1
Signature · 1 line
2.53.0
#2 lore raw
Quoted 11 lines · Darrick J. Wong
Quoted 8 lines · Christoph Hellwig

Various test unset SCRATCH_RTDEV to avoid dealing with metadata allocations for the RT subvolume in the main device or differences in RT subvolume behavior. Besides being rather ad-hoc, this unset also fails to disable the internal RT zoned subvolume.

Add a _disable_scratch_realtime helper that not only unsets SCRATCH_RTDEV, but also disables the zoned allocator and with that the internal RT subvolume.

What if we want to run these tests on the internal rt volume?

Then we need to stop disabling the RT volume for them..

Reviewed-by
#3 lore raw
Quoted 7 lines · Christoph Hellwig
Quoted 6 lines · Darrick J. Wong
Quoted 5 lines · Christoph Hellwig

Various test unset SCRATCH_RTDEV to avoid dealing with metadata allocations for the RT subvolume in the main device or differences in RT subvolume behavior. Besides being rather ad-hoc, this unset also fails to disable the internal RT zoned subvolume.

Quoted 7 lines
Quoted 5 lines
Quoted 3 lines

Add a _disable_scratch_realtime helper that not only unsets SCRATCH_RTDEV, but also disables the zoned allocator and with that the internal RT subvolume.

What if we want to run these tests on the internal rt volume?

Then we need to stop disabling the RT volume for them..

Ahaha ok with this patch applied, the regressions I see in generic/441 and xfs/656 go away. generic/441 chokes because the scratch fs goes down due to writeback error, so I assume it's ok to start excluding this test.

xfs/656 is a different story:

run/fstests/logs/xfs/656.out.bad renamed from run/fstests/bin/tests/xfs/656.out +1 -2
@@ -1,10 +1,9 @@
 QA output created by 656
 Format and mount
 pwrite: Input/output error
-pread: Input/output error
+stat: Input/output error
 pread: Input/output error
 fsync: Input/output error
 VICTIM pos NUM len NUM: directio_write: Input/output error
-VICTIM pos NUM len NUM: directio_read: Input/output error
 VICTIM pos NUM len NUM: buffered_read: Input/output error
 VICTIM pos NUM len NUM: buffered_write: Input/output error

I'm guessing that the write failure takes down the filesystem, which is why the subsequent xfs_io pread can't even open the file.

Also the comment in xfs/655 and 656 doesn't make sense:

# Disable the scratch rt device to avoid test failures relating to the

 # rt bitmap consuming all the free space in our small data device.
 unset SCRATCH_RTDEV

We're creating a regular sized data device, so this shouldn't be an issue.

Eh, whatever, I'll clean up those two healer tests. :)

--D

#4 lore raw
Quoted 4 lines · Darrick J. Wong

Ahaha ok with this patch applied, the regressions I see in generic/441 and xfs/656 go away. generic/441 chokes because the scratch fs goes down due to writeback error, so I assume it's ok to start excluding this test.

There also is a dedicated patch disabling them for zoned on the list":

"common: skip data write EIO survival tests on fatal configs"

from Yao Sang.

Quoted 2 lines

I'm guessing that the write failure takes down the filesystem, which is why the subsequent xfs_io pread can't even open the file.

Yes.

Quoted 9 lines

Also the comment in xfs/655 and 656 doesn't make sense:

# Disable the scratch rt device to avoid test failures relating to the

 # rt bitmap consuming all the free space in our small data device.
 unset SCRATCH_RTDEV

We're creating a regular sized data device, so this shouldn't be an issue.

I think this has been copy and pasted with the unsetting of SCRATCH_RTDEV, as several tests have it..

#5 lore raw
Quoted 24 lines · Christoph Hellwig
Quoted 4 lines · Darrick J. Wong

Ahaha ok with this patch applied, the regressions I see in generic/441 and xfs/656 go away. generic/441 chokes because the scratch fs goes down due to writeback error, so I assume it's ok to start excluding this test.

There also is a dedicated patch disabling them for zoned on the list":

"common: skip data write EIO survival tests on fatal configs"

from Yao Sang.

Quoted 2 lines

I'm guessing that the write failure takes down the filesystem, which is why the subsequent xfs_io pread can't even open the file.

Yes.

Quoted 7 lines

Also the comment in xfs/655 and 656 doesn't make sense:

# Disable the scratch rt device to avoid test failures relating to the

 # rt bitmap consuming all the free space in our small data device.
 unset SCRATCH_RTDEV
Quoted 5 lines
Quoted 2 lines

We're creating a regular sized data device, so this shouldn't be an issue.

I think this has been copy and pasted with the unsetting of SCRATCH_RTDEV, as several tests have it..

Yes. I apologize for my contributions to the copy-pasta :(

--D