Fix test failures caused by storage devcies with 4k sectors
cover Fix test failures caused by storage devcies with 4k sectors
When testing with an SSD with a 4k logical sector size, I ran into a number of test failures that were caused by assumption that the storage device can support 1k block sizes.
Fix this by skipping those tests, or in the case of generic/563, make sure the loop device has the same block size as the backing scratch device. (Arguably losetup should do that, but at least today, it doesn't.)
This test series was tested using:
gce-xfstests --local-ssd-nvme -c ext4,xfs,btrfs -g auto
and comparing it against the results of running the same set of tests without the --local-ssd-nvme option, which introduces the use of a 4k sector storage device.
With these patches applied, there is one remaining failure with xfs/157 but I'm not sure how to deal with it, since Google searches regarding the failure message just simply say "Don't try to use a regular file as a logdev", which isn't particularly helpful here. Maybe the right thing is to just hard code a _notrun if "blockdev --getss $SCRATCH_DEV" is not 512? It's not clear to me, so I've left it.
I also noted two new failures with btrfs, generic/175 and generic/251. The cause of why these tests are failing with a 4k sector device is not obvious to me. But certainly things are much better with this patch, and perhaps the the btrfs and xfs developers can address these last new test failures if they care about this particular test scenario.
Theodore Ts'o (2):
common: check if the scratch device can support 1024 block sizes generic/563: create the loop dev with the same block size as the
scratch dev
common/rc | 22 ++++++++++++++++++++-- tests/ext4/055 | 1 + tests/generic/563 | 2 +- tests/xfs/205 | 1 + tests/xfs/432 | 1 + tests/xfs/516 | 1 + 6 files changed, 25 insertions(+), 3 deletions(-)
Signature · 1 line
2.31.0
PATCH 1/2 common: check if the scratch device can support 1024 block sizes
If the scratch device has as logical blocksize larger than 512 --- for example, some SSD or HDD's may have a 4k logical blocksize, and so will not support a file system with a 1k block size.
Add a new function, _require_scratch_support_blocksize so we can skip tests that use _scratch_mkfs_blocksized with a size less than the scratch device's logical block size.
--- common/rc | 12 ++++++++++++ tests/ext4/055 | 1 + tests/xfs/205 | 1 + tests/xfs/432 | 1 + tests/xfs/516 | 1 + 5 files changed, 16 insertions(+)
common/rc +12 -0
@@ -1124,6 +1124,9 @@ _scratch_mkfs_blocksized() if [ $blocksize -lt $(_get_page_size) ]; then _exclude_scratch_mount_option dax fi + if [ $blocksize -lt $(blockdev --getss $SCRATCH_DEV) ]; then + _require_scratch_support_blocksize "$blocksize" + fi case $FSTYP in btrfs)
@@ -4452,6 +4455,15 @@ _get_device_size() echo $(($(blockdev --getsz $1) >> 1)) } +_require_scratch_support_blocksize() +{ + local blocksize=$1 + + if [ $blocksize -lt $(blockdev --getss $SCRATCH_DEV) ]; then + _notrun "$SCRATCH_DEV does not support a block size of $blocksize." + fi +} + # Make sure we actually have dmesg checking set up. _require_check_dmesg() {
tests/ext4/055 +1 -0
@@ -27,6 +27,7 @@ echo "Silence is golden" # The 1K blocksize is designed for debugfs. _exclude_scratch_mount_option dax +_require_scratch_support_blocksize 1024 _scratch_mkfs "-F -O quota -b 1024" > $seqres.full 2>&1 # Start from 0, fill block 1 with 6,replace the original 2.
tests/xfs/205 +1 -0
@@ -23,6 +23,7 @@ _require_scratch_nocheck unset SCRATCH_RTDEV fsblksz=1024 +_require_scratch_support_blocksize $fsblksz _scratch_mkfs_xfs -d size=$((32768*fsblksz)) -b size=$fsblksz >> $seqres.full 2>&1 _scratch_mount
tests/xfs/432 +1 -0
@@ -50,6 +50,7 @@ echo "Format and mount" # block. 8187 hashes/dablk / 248 dirents/dirblock = ~33 dirblocks per # dablock. 33 dirblocks * 64k mean that we can expand a directory by # 2112k before we have to allocate another da btree block. +_require_scratch_support_blocksize 1024 _scratch_mkfs -b size=1k -n size=64k > "$seqres.full" 2>&1 _scratch_mount >> "$seqres.full" 2>&1
tests/xfs/516 +1 -0
@@ -23,6 +23,7 @@ _cleanup() # real QA test starts here _supported_fs xfs _require_scratch_nocheck +_require_scratch_support_blocksize 1024 # Assume that if we can run scrub on the test dev we can run it on the scratch # fs too.
Signature · 1 line
2.31.0
Re: [PATCH 1/2] common: check if the scratch device can support 1024 block sizes
Quoted 22 lines · Theodore Ts'o
If the scratch device has as logical blocksize larger than 512 --- for example, some SSD or HDD's may have a 4k logical blocksize, and so will not support a file system with a 1k block size.
Add a new function,
_require_scratch_support_blocksizeso we can skip tests that use_scratch_mkfs_blocksizedwith a size less than the scratch device's logical block size.Signed-off-by Theodore Ts'o <[email protected]>--- common/rc | 12 ++++++++++++ tests/ext4/055 | 1 + tests/xfs/205 | 1 + tests/xfs/432 | 1 + tests/xfs/516 | 1 + 5 files changed, 16 insertions(+)common/rc +0
-0
common/rc _scratch_mkfs_blocksized() · line 1129
Quoted 6 lines
common/rc +3
-0@@ -1124,6 +1124,9 @@ _scratch_mkfs_blocksized() if [ $blocksize -lt $(_get_page_size) ]; then _exclude_scratch_mount_option dax fi + if [ $blocksize -lt $(blockdev --getss $SCRATCH_DEV) ]; then + _require_scratch_support_blocksize "$blocksize" + fi
This duplicates the logic in _require_scratch_support_blocksize, so I think you can drop it.
common/rc _get_device_size() · line 4463
Quoted 13 lines
common/rc +6
-0case $FSTYP in btrfs)@@ -4452,6 +4455,15 @@ _get_device_size() echo $(($(blockdev --getsz $1) >> 1)) } +_require_scratch_support_blocksize() +{ + local blocksize=$1 + + if [ $blocksize -lt $(blockdev --getss $SCRATCH_DEV) ]; then + _notrun "$SCRATCH_DEV does not support a block size of $blocksize."
"block" is a bit vague in this context -- you mean the LBA size, not the internal physical block size, right?
May I suggest "...does not support an LBA size of $blocksize." ?
--D
Quoted 57 lines
common/rc +3
-0+ fi +} + # Make sure we actually have dmesg checking set up. _require_check_dmesg() {tests/ext4/055 +1
-0@@ -27,6 +27,7 @@ echo "Silence is golden" # The 1K blocksize is designed for debugfs. _exclude_scratch_mount_option dax +_require_scratch_support_blocksize 1024 _scratch_mkfs "-F -O quota -b 1024" > $seqres.full 2>&1 # Start from 0, fill block 1 with 6,replace the original 2.tests/xfs/205 +1
-0@@ -23,6 +23,7 @@ _require_scratch_nocheck unset SCRATCH_RTDEV fsblksz=1024 +_require_scratch_support_blocksize $fsblksz _scratch_mkfs_xfs -d size=$((32768*fsblksz)) -b size=$fsblksz >> $seqres.full 2>&1 _scratch_mounttests/xfs/432 +1
-0@@ -50,6 +50,7 @@ echo "Format and mount" # block. 8187 hashes/dablk / 248 dirents/dirblock = ~33 dirblocks per # dablock. 33 dirblocks * 64k mean that we can expand a directory by # 2112k before we have to allocate another da btree block. +_require_scratch_support_blocksize 1024 _scratch_mkfs -b size=1k -n size=64k > "$seqres.full" 2>&1 _scratch_mount >> "$seqres.full" 2>&1tests/xfs/516 +1
-0@@ -23,6 +23,7 @@ _cleanup() # real QA test starts here _supported_fs xfs _require_scratch_nocheck +_require_scratch_support_blocksize 1024 # Assume that if we can run scrub on the test dev we can run it on the scratch # fs too.Signature · 1 line
2.31.0
Re: [PATCH 1/2] common: check if the scratch device can support 1024 block sizes
Quoted 6 lines · Darrick J. Wong
common/rc _scratch_mkfs_blocksized() · line 1129
Quoted 3 lines
common/rc +3
-0+ if [ $blocksize -lt $(blockdev --getss $SCRATCH_DEV) ]; then + _require_scratch_support_blocksize "$blocksize" + fiThis duplicates the logic in
_require_scratch_support_blocksize, so I think you can drop it.
Yes, oops. I'll drop the if statement.
Quoted 11 lines
common/rc _get_device_size() · line 4463
Quoted 6 lines
common/rc +6
-0+_require_scratch_support_blocksize() +{ + local blocksize=$1 + + if [ $blocksize -lt $(blockdev --getss $SCRATCH_DEV) ]; then + _notrun "$SCRATCH_DEV does not support a block size of $blocksize.""block" is a bit vague in this context -- you mean the LBA size, not the internal physical block size, right?
May I suggest "...does not support an LBA size of $blocksize." ?
How about "does not support a file system block size of $blocksize"? LBA size refers to the granularity of "logical block address" for a particular block device, which would be confusing and not quite right here.
- Ted
PATCH 2/2 generic/563: create the loop dev with the same block size as the scratch dev
In reply to Theodore Ts'oThe generic/563 test creates the loop device using $SCRATCH_DEV directly. We need to create the loop device with the same logical block size. Otherwise, the loop device will always be created with the default logical block size of 512, and if its underlying backing store has a different logical block size, then mkfs may create a file system in the loop device that will fail to mount.
--- common/rc | 10 ++++++++-- tests/generic/563 | 2 +- 2 files changed, 9 insertions(+), 3 deletions(-)
common/rc +8 -2
@@ -4246,9 +4246,15 @@ _require_userns() _create_loop_device() { - local file=$1 dev - dev=`losetup -f --show $file` || _fail "Cannot assign $file to a loop device" + local file=$1 + local blocksize=$2 + local dev + + if [ -n "$blocksize" ]; then + blocksize="-b $blocksize" + fi + dev=`losetup -f $blocksize --show $file` || _fail "Cannot assign $file to a loop device" # Try to enable asynchronous directio mode on the loopback device so # that writeback started by a filesystem mounted on the loop device # won't be throttled by buffered writes to the lower filesystem. This
tests/generic/563 +1 -1
@@ -89,7 +89,7 @@ reset() # cgroup I/O accounting doesn't work on partitions. Use a loop device to rule # that out. -LOOP_DEV=$(_create_loop_device $SCRATCH_DEV) +LOOP_DEV=$(_create_loop_device $SCRATCH_DEV $(blockdev --getss $SCRATCH_DEV)) smajor=$((0x`stat -L -c %t $LOOP_DEV`)) sminor=$((0x`stat -L -c %T $LOOP_DEV`))
Signature · 1 line
2.31.0
Re: [PATCH 2/2] generic/563: create the loop dev with the same block size as the scratch dev
Quoted 8 lines · Theodore Ts'o
The generic/563 test creates the loop device using $SCRATCH_DEV directly. We need to create the loop device with the same logical block size. Otherwise, the loop device will always be created with the default logical block size of 512, and if its underlying backing store has a different logical block size, then mkfs may create a file system in the loop device that will fail to mount.
Signed-off-by Theodore Ts'o <[email protected]>
Looks fine to me,
--D
Quoted 43 lines
--- common/rc | 10 ++++++++-- tests/generic/563 | 2 +- 2 files changed, 9 insertions(+), 3 deletions(-)common/rc +8
-2@@ -4246,9 +4246,15 @@ _require_userns() _create_loop_device() { - local file=$1 dev - dev=`losetup -f --show $file` || _fail "Cannot assign $file to a loop device" + local file=$1 + local blocksize=$2 + local dev + + if [ -n "$blocksize" ]; then + blocksize="-b $blocksize" + fi + dev=`losetup -f $blocksize --show $file` || _fail "Cannot assign $file to a loop device" # Try to enable asynchronous directio mode on the loopback device so # that writeback started by a filesystem mounted on the loop device # won't be throttled by buffered writes to the lower filesystem. Thistests/generic/563 +1
-1@@ -89,7 +89,7 @@ reset() # cgroup I/O accounting doesn't work on partitions. Use a loop device to rule # that out. -LOOP_DEV=$(_create_loop_device $SCRATCH_DEV) +LOOP_DEV=$(_create_loop_device $SCRATCH_DEV $(blockdev --getss $SCRATCH_DEV)) smajor=$((0x`stat -L -c %t $LOOP_DEV`)) sminor=$((0x`stat -L -c %T $LOOP_DEV`))Signature · 1 line
2.31.0