generic/563: use a loop device to avoid partition incompatibility
PATCH 1/1 generic/563: use a loop device to avoid partition incompatibility
cgroup writeback accounting does not track partition level statistics. Instead, I/O is accounted against the parent device. As a result, the test fails if the scratch device happens to be a device partition. Since parent level stats are potentially polluted by factors external to the test, wrap the scratch device in a loopback device to guarantee the test always runs on a top-level block device.
--- tests/generic/563 | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-)
tests/generic/563 +14 -7
@@ -2,7 +2,7 @@ # SPDX-License-Identifier: GPL-2.0 # Copyright (c) 2019 Red Hat, Inc. All Rights Reserved. # -# FS QA Test No. 011 +# FS QA Test No. 563 # # This test verifies that cgroup aware writeback properly accounts I/Os in # various scenarios. We perform reads/writes from different combinations of
@@ -26,6 +26,8 @@ _cleanup() echo $$ > $cgdir/cgroup.procs rmdir $cgdir/$seq-cg* > /dev/null 2>&1 + umount $SCRATCH_MNT > /dev/null 2>&1 + _destroy_loop_device $LOOP_DEV > /dev/null 2>&1 } # get standard environment, filters and checks
@@ -42,14 +44,12 @@ rm -f $seqres.full _supported_fs generic _require_scratch _require_cgroup2 io +_require_loop # cgroup v2 writeback is only support on block devices so far _require_block_device $SCRATCH_DEV -smajor=$((0x`stat -L -c %t $SCRATCH_DEV`)) -sminor=$((0x`stat -L -c %T $SCRATCH_DEV`)) cgdir=$CGROUP2_PATH - iosize=$((1024 * 1024 * 8)) # Check cgroup read/write charges against expected values. Allow for some
@@ -89,12 +89,19 @@ reset() rmdir $cgdir/$seq-cg* > /dev/null 2>&1 $XFS_IO_PROG -fc "pwrite 0 $iosize" $SCRATCH_MNT/file \ >> $seqres.full 2>&1 - _scratch_cycle_mount || _fail "mount failed" + umount $SCRATCH_MNT || _fail "umount failed" + _mount $LOOP_DEV $SCRATCH_MNT || _fail "mount failed" stat $SCRATCH_MNT/file > /dev/null } -_scratch_mkfs >> $seqres.full 2>&1 -_scratch_mount +# cgroup I/O accounting doesn't work on partitions. Use a loop device to rule +# that out. +LOOP_DEV=$(_create_loop_device $SCRATCH_DEV) +smajor=$((0x`stat -L -c %t $LOOP_DEV`)) +sminor=$((0x`stat -L -c %T $LOOP_DEV`)) + +_mkfs_dev $LOOP_DEV >> $seqres.full 2>&1 +_mount $LOOP_DEV $SCRATCH_MNT || _fail "mount failed" echo "+io" > $cgdir/cgroup.subtree_control || _fail "subtree control"
Signature · 1 line
2.26.2
Quoted 57 lines · Brian Foster
cgroup writeback accounting does not track partition level statistics. Instead, I/O is accounted against the parent device. As a result, the test fails if the scratch device happens to be a device partition. Since parent level stats are potentially polluted by factors external to the test, wrap the scratch device in a loopback device to guarantee the test always runs on a top-level block device.
Reported-by Boyang Xue <[email protected]>Signed-off-by Brian Foster <[email protected]>--- tests/generic/563 | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-)tests/generic/563 +4
-5@@ -2,7 +2,7 @@ # SPDX-License-Identifier: GPL-2.0 # Copyright (c) 2019 Red Hat, Inc. All Rights Reserved. # -# FS QA Test No. 011 +# FS QA Test No. 563 # # This test verifies that cgroup aware writeback properly accounts I/Os in # various scenarios. We perform reads/writes from different combinations of@@ -26,6 +26,8 @@ _cleanup() echo $$ > $cgdir/cgroup.procs rmdir $cgdir/$seq-cg* > /dev/null 2>&1 + umount $SCRATCH_MNT > /dev/null 2>&1 + _destroy_loop_device $LOOP_DEV > /dev/null 2>&1 } # get standard environment, filters and checks@@ -42,14 +44,12 @@ rm -f $seqres.full _supported_fs generic _require_scratch _require_cgroup2 io +_require_loop # cgroup v2 writeback is only support on block devices so far _require_block_device $SCRATCH_DEV -smajor=$((0x`stat -L -c %t $SCRATCH_DEV`)) -sminor=$((0x`stat -L -c %T $SCRATCH_DEV`)) cgdir=$CGROUP2_PATH - iosize=$((1024 * 1024 * 8)) # Check cgroup read/write charges against expected values. Allow for some@@ -89,12 +89,19 @@ reset() rmdir $cgdir/$seq-cg* > /dev/null 2>&1 $XFS_IO_PROG -fc "pwrite 0 $iosize" $SCRATCH_MNT/file \ >> $seqres.full 2>&1 - _scratch_cycle_mount || _fail "mount failed"
tests/generic/563 reset() · line 99
Quoted 10 lines
tests/generic/563 +5
-2+ umount $SCRATCH_MNT || _fail "umount failed" + _mount $LOOP_DEV $SCRATCH_MNT || _fail "mount failed" stat $SCRATCH_MNT/file > /dev/null } -_scratch_mkfs >> $seqres.full 2>&1 -_scratch_mount +# cgroup I/O accounting doesn't work on partitions. Use a loop device to rule +# that out. +LOOP_DEV=$(_create_loop_device $SCRATCH_DEV)
I recommend using a file to create loop device. If you'd like to use SCRATCH_DEV to create loop device directly, you'd better to change the "_require_scratch" to "_require_scratch_nocheck". Or I think it might be failed, e.g. if SCRATCH_DEV is a 4k sector size device.
Others look good to me.
Thanks,
Zorro
Quoted 11 lines
tests/generic/563 +5
-0+smajor=$((0x`stat -L -c %t $LOOP_DEV`)) +sminor=$((0x`stat -L -c %T $LOOP_DEV`)) + +_mkfs_dev $LOOP_DEV >> $seqres.full 2>&1 +_mount $LOOP_DEV $SCRATCH_MNT || _fail "mount failed" echo "+io" > $cgdir/cgroup.subtree_control || _fail "subtree control"Signature · 1 line
2.26.2
Quoted 63 lines · Zorro Lang
Quoted 62 lines · Brian Foster
cgroup writeback accounting does not track partition level statistics. Instead, I/O is accounted against the parent device. As a result, the test fails if the scratch device happens to be a device partition. Since parent level stats are potentially polluted by factors external to the test, wrap the scratch device in a loopback device to guarantee the test always runs on a top-level block device.
Reported-by Boyang Xue <[email protected]>Signed-off-by Brian Foster <[email protected]>--- tests/generic/563 | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-)tests/generic/563 +6
-5@@ -2,7 +2,7 @@ # SPDX-License-Identifier: GPL-2.0 # Copyright (c) 2019 Red Hat, Inc. All Rights Reserved. # -# FS QA Test No. 011 +# FS QA Test No. 563 # # This test verifies that cgroup aware writeback properly accounts I/Os in # various scenarios. We perform reads/writes from different combinations of@@ -26,6 +26,8 @@ _cleanup() echo $$ > $cgdir/cgroup.procs rmdir $cgdir/$seq-cg* > /dev/null 2>&1 + umount $SCRATCH_MNT > /dev/null 2>&1 + _destroy_loop_device $LOOP_DEV > /dev/null 2>&1 } # get standard environment, filters and checks@@ -42,14 +44,12 @@ rm -f $seqres.full _supported_fs generic _require_scratch _require_cgroup2 io +_require_loop # cgroup v2 writeback is only support on block devices so far _require_block_device $SCRATCH_DEV -smajor=$((0x`stat -L -c %t $SCRATCH_DEV`)) -sminor=$((0x`stat -L -c %T $SCRATCH_DEV`)) cgdir=$CGROUP2_PATH - iosize=$((1024 * 1024 * 8)) # Check cgroup read/write charges against expected values. Allow for some@@ -89,12 +89,19 @@ reset() rmdir $cgdir/$seq-cg* > /dev/null 2>&1 $XFS_IO_PROG -fc "pwrite 0 $iosize" $SCRATCH_MNT/file \ >> $seqres.full 2>&1 - _scratch_cycle_mount || _fail "mount failed" + umount $SCRATCH_MNT || _fail "umount failed" + _mount $LOOP_DEV $SCRATCH_MNT || _fail "mount failed" stat $SCRATCH_MNT/file > /dev/null }
Quoted 11 lines
tests/generic/563 reset() · line 99
Quoted 5 lines
tests/generic/563 +3
-2-_scratch_mkfs >> $seqres.full 2>&1 -_scratch_mount +# cgroup I/O accounting doesn't work on partitions. Use a loop device to rule +# that out. +LOOP_DEV=$(_create_loop_device $SCRATCH_DEV)I recommend using a file to create loop device. If you'd like to use
SCRATCH_DEVto create loop device directly, you'd better to change the "_require_scratch" to "_require_scratch_nocheck". Or I think it might be failed, e.g. ifSCRATCH_DEVis a 4k sector size device.
What's the error that occurs with a 4k device, out of curiosity? I suppose if it's just a repair thing then using _nocheck probably makes sense (or technically might make sense regardless since we're not formatting the scratch device directly). I don't mind creating a file and using loop on that, but would like to make sure I understand if/why it's necessary.
Quoted 2 lines
Others look good to me.
Thanks for the feedback.
Brian
Quoted 15 lines
Thanks,
ZorroQuoted 11 lines
tests/generic/563 +5
-0+smajor=$((0x`stat -L -c %t $LOOP_DEV`)) +sminor=$((0x`stat -L -c %T $LOOP_DEV`)) + +_mkfs_dev $LOOP_DEV >> $seqres.full 2>&1 +_mount $LOOP_DEV $SCRATCH_MNT || _fail "mount failed" echo "+io" > $cgdir/cgroup.subtree_control || _fail "subtree control"Signature · 1 line
2.26.2
Quoted 76 lines · Brian Foster
Quoted 63 lines · Zorro Lang
Quoted 62 lines · Brian Foster
cgroup writeback accounting does not track partition level statistics. Instead, I/O is accounted against the parent device. As a result, the test fails if the scratch device happens to be a device partition. Since parent level stats are potentially polluted by factors external to the test, wrap the scratch device in a loopback device to guarantee the test always runs on a top-level block device.
Reported-by Boyang Xue <[email protected]>Signed-off-by Brian Foster <[email protected]>--- tests/generic/563 | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-)tests/generic/563 +6
-5@@ -2,7 +2,7 @@ # SPDX-License-Identifier: GPL-2.0 # Copyright (c) 2019 Red Hat, Inc. All Rights Reserved. # -# FS QA Test No. 011 +# FS QA Test No. 563 # # This test verifies that cgroup aware writeback properly accounts I/Os in # various scenarios. We perform reads/writes from different combinations of@@ -26,6 +26,8 @@ _cleanup() echo $$ > $cgdir/cgroup.procs rmdir $cgdir/$seq-cg* > /dev/null 2>&1 + umount $SCRATCH_MNT > /dev/null 2>&1 + _destroy_loop_device $LOOP_DEV > /dev/null 2>&1 } # get standard environment, filters and checks@@ -42,14 +44,12 @@ rm -f $seqres.full _supported_fs generic _require_scratch _require_cgroup2 io +_require_loop # cgroup v2 writeback is only support on block devices so far _require_block_device $SCRATCH_DEV -smajor=$((0x`stat -L -c %t $SCRATCH_DEV`)) -sminor=$((0x`stat -L -c %T $SCRATCH_DEV`)) cgdir=$CGROUP2_PATH - iosize=$((1024 * 1024 * 8)) # Check cgroup read/write charges against expected values. Allow for some@@ -89,12 +89,19 @@ reset() rmdir $cgdir/$seq-cg* > /dev/null 2>&1 $XFS_IO_PROG -fc "pwrite 0 $iosize" $SCRATCH_MNT/file \ >> $seqres.full 2>&1 - _scratch_cycle_mount || _fail "mount failed" + umount $SCRATCH_MNT || _fail "umount failed" + _mount $LOOP_DEV $SCRATCH_MNT || _fail "mount failed" stat $SCRATCH_MNT/file > /dev/null }Quoted 11 lines
tests/generic/563 reset() · line 99
Quoted 5 lines
tests/generic/563 +3
-2-_scratch_mkfs >> $seqres.full 2>&1 -_scratch_mount +# cgroup I/O accounting doesn't work on partitions. Use a loop device to rule +# that out. +LOOP_DEV=$(_create_loop_device $SCRATCH_DEV)I recommend using a file to create loop device. If you'd like to use
SCRATCH_DEVto create loop device directly, you'd better to change the "_require_scratch" to "_require_scratch_nocheck". Or I think it might be failed, e.g. ifSCRATCH_DEVis a 4k sector size device.
Quoted 6 lines
What's the error that occurs with a 4k device, out of curiosity? I suppose if it's just a repair thing then using
_nocheckprobably makes sense (or technically might make sense regardless since we're not formatting the scratch device directly). I don't mind creating a file and using loop on that, but would like to make sure I understand if/why it's necessary.
The XFS on underlying device will cause fsck fail, likes this:
# modprobe scsi_debug sector_size=4096 physblk_exp=0 dev_size_mb=1024
# losetup -f --show /dev/sdc
/dev/loop0
# # mkfs.xfs -f /dev/loop0
meta-data=/dev/loop0 isize=512 agcount=4, agsize=65536 blks
= sectsz=512 attr=2, projid32bit=1
= crc=1 finobt=1, sparse=1, rmapbt=0
= reflink=1
data = bsize=4096 blocks=262144, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0, ftype=1
log =internal log bsize=4096 blocks=2560, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
Discarding blocks...Done.
# xfs_repair -n /dev/loop0
[passed]
# losetup -d /dev/loop0
# xfs_repair -n /dev/sdc
Phase 1 - find and verify superblock...
xfs_repair: read failed: Invalid argument
xfs_repair: data size check failed
xfs_repair: cannot repair this filesystem. Sorry.The xfstests always do fsck on SCRATCH_DEV except you use _require_scratch_nocheck at the beginning of a sub-case, to skip the fsck.
Thanks,
Zorro
Quoted 24 lines
Quoted 3 lines
Others look good to me.
Thanks for the feedback.
Brian
Quoted 15 lines
Thanks,
ZorroQuoted 11 lines
tests/generic/563 +5
-0+smajor=$((0x`stat -L -c %t $LOOP_DEV`)) +sminor=$((0x`stat -L -c %T $LOOP_DEV`)) + +_mkfs_dev $LOOP_DEV >> $seqres.full 2>&1 +_mount $LOOP_DEV $SCRATCH_MNT || _fail "mount failed" echo "+io" > $cgdir/cgroup.subtree_control || _fail "subtree control"Signature · 1 line
2.26.2
Quoted 110 lines · Zorro Lang
Quoted 76 lines · Brian Foster
Quoted 63 lines · Zorro Lang
Quoted 62 lines · Brian Foster
cgroup writeback accounting does not track partition level statistics. Instead, I/O is accounted against the parent device. As a result, the test fails if the scratch device happens to be a device partition. Since parent level stats are potentially polluted by factors external to the test, wrap the scratch device in a loopback device to guarantee the test always runs on a top-level block device.
Reported-by Boyang Xue <[email protected]>Signed-off-by Brian Foster <[email protected]>--- tests/generic/563 | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-)tests/generic/563 +6
-5@@ -2,7 +2,7 @@ # SPDX-License-Identifier: GPL-2.0 # Copyright (c) 2019 Red Hat, Inc. All Rights Reserved. # -# FS QA Test No. 011 +# FS QA Test No. 563 # # This test verifies that cgroup aware writeback properly accounts I/Os in # various scenarios. We perform reads/writes from different combinations of@@ -26,6 +26,8 @@ _cleanup() echo $$ > $cgdir/cgroup.procs rmdir $cgdir/$seq-cg* > /dev/null 2>&1 + umount $SCRATCH_MNT > /dev/null 2>&1 + _destroy_loop_device $LOOP_DEV > /dev/null 2>&1 } # get standard environment, filters and checks@@ -42,14 +44,12 @@ rm -f $seqres.full _supported_fs generic _require_scratch _require_cgroup2 io +_require_loop # cgroup v2 writeback is only support on block devices so far _require_block_device $SCRATCH_DEV -smajor=$((0x`stat -L -c %t $SCRATCH_DEV`)) -sminor=$((0x`stat -L -c %T $SCRATCH_DEV`)) cgdir=$CGROUP2_PATH - iosize=$((1024 * 1024 * 8)) # Check cgroup read/write charges against expected values. Allow for some@@ -89,12 +89,19 @@ reset() rmdir $cgdir/$seq-cg* > /dev/null 2>&1 $XFS_IO_PROG -fc "pwrite 0 $iosize" $SCRATCH_MNT/file \ >> $seqres.full 2>&1 - _scratch_cycle_mount || _fail "mount failed" + umount $SCRATCH_MNT || _fail "umount failed" + _mount $LOOP_DEV $SCRATCH_MNT || _fail "mount failed" stat $SCRATCH_MNT/file > /dev/null }Quoted 11 lines
tests/generic/563 reset() · line 99
Quoted 5 lines
tests/generic/563 +3
-2-_scratch_mkfs >> $seqres.full 2>&1 -_scratch_mount +# cgroup I/O accounting doesn't work on partitions. Use a loop device to rule +# that out. +LOOP_DEV=$(_create_loop_device $SCRATCH_DEV)I recommend using a file to create loop device. If you'd like to use
SCRATCH_DEVto create loop device directly, you'd better to change the "_require_scratch" to "_require_scratch_nocheck". Or I think it might be failed, e.g. ifSCRATCH_DEVis a 4k sector size device.Quoted 6 lines
What's the error that occurs with a 4k device, out of curiosity? I suppose if it's just a repair thing then using
_nocheckprobably makes sense (or technically might make sense regardless since we're not formatting the scratch device directly). I don't mind creating a file and using loop on that, but would like to make sure I understand if/why it's necessary.The XFS on underlying device will cause fsck fail, likes this:
# modprobe scsi_debug sector_size=4096 physblk_exp=0 dev_size_mb=1024 # losetup -f --show /dev/sdc /dev/loop0 # # mkfs.xfs -f /dev/loop0 meta-data=/dev/loop0 isize=512 agcount=4, agsize=65536 blks = sectsz=512 attr=2, projid32bit=1 = crc=1 finobt=1, sparse=1, rmapbt=0 = reflink=1 data = bsize=4096 blocks=262144, imaxpct=25 = sunit=0 swidth=0 blks naming =version 2 bsize=4096 ascii-ci=0, ftype=1 log =internal log bsize=4096 blocks=2560, version=2 = sectsz=512 sunit=0 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0 Discarding blocks...Done. # xfs_repair -n /dev/loop0 [passed] # losetup -d /dev/loop0 # xfs_repair -n /dev/sdc Phase 1 - find and verify superblock... xfs_repair: read failed: Invalid argument xfs_repair: data size check failed xfs_repair: cannot repair this filesystem. Sorry.
Quoted 3 lines
The xfstests always do fsck on
SCRATCH_DEVexcept you use_require_scratch_nocheckat the beginning of a sub-case, to skip the fsck.
Ah, Ok. If repair is the only issue then I'll update the test to use _nocheck. Thanks for catching this..
Brian
Quoted 28 lines
Thanks,
ZorroQuoted 24 lines
Quoted 3 lines
Others look good to me.
Thanks for the feedback.
Brian
Quoted 15 lines
Thanks,
ZorroQuoted 11 lines
tests/generic/563 +5
-0+smajor=$((0x`stat -L -c %t $LOOP_DEV`)) +sminor=$((0x`stat -L -c %T $LOOP_DEV`)) + +_mkfs_dev $LOOP_DEV >> $seqres.full 2>&1 +_mount $LOOP_DEV $SCRATCH_MNT || _fail "mount failed" echo "+io" > $cgdir/cgroup.subtree_control || _fail "subtree control"Signature · 1 line
2.26.2