fstests: add test case to make sure btrfs can handle one corrupted device
PATCH 1/1 fstests: add test case to make sure btrfs can handle one corrupted device
The new test case will verify that btrfs can handle one corrupted device without affecting the consistency of the filesystem.
Unlike a missing device, one corrupted device can return garbage to the fs, thus btrfs has to utilize its data/metadata checksum to verify which data is correct.
The test case will:
- Create a small fs
Mostly to speedup the test
- Fill the fs with a regular file
- Use fsstress to create some contents
- Save the fssum for later verification
- Corrupt one device with garbage but keep the primary superblock untouched
- Run fssum verification
- Run scrub to fix the fs
- Run scrub again to make sure the fs is fine
--- tests/btrfs/261 | 94 +++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/261.out | 2 + 2 files changed, 96 insertions(+) create mode 100755 tests/btrfs/261 create mode 100644 tests/btrfs/261.out
tests/btrfs/261 new file +94 -0
@@ -0,0 +1,94 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) 2022 SUSE Linux Products GmbH. All Rights Reserved. +# +# FS QA Test 261 +# +# Make sure btrfs raid profiles can handling one corrupted device +# without affecting the consistency of the fs. +# +. ./common/preamble +_begin_fstest raid + +. ./common/filter +. ./common/populate + +_supported_fs btrfs +_require_scratch_dev_pool 4 +_require_fssum + +prepare_fs() +{ + local profile=$1 + + # We don't want too large fs which can take too long to populate + # And the extra redirection of stderr is to avoid the RAID56 warning + # message to polluate the golden output + _scratch_pool_mkfs -m $profile -d $profile -b 1G >> $seqres.full 2>&1 + if [ $? -ne 0 ]; then + echo "mkfs $mkfs_opts failed" + return + fi + + # Disable compression, as compressed read repair is known to have problems + _scratch_mount -o compress=no + + # Fill some part of the fs first + $XFS_IO_PROG -f -c "pwrite -S 0xfe 0 400M" $SCRATCH_MNT/garbage > /dev/null 2>&1 + + # Then use fsstress to generate some extra contents. + # Disable setattr related operations, as it may set NODATACOW which will + # not allow us to use btrfs checksum to verify the content. + $FSSTRESS_PROG -f setattr=0 -d $SCRATCH_MNT -w -n 3000 > /dev/null 2>&1 + sync + + # Save the fssum of this fs + $FSSUM_PROG -A -f -w $tmp.saved_fssum $SCRATCH_MNT + $BTRFS_UTIL_PROG fi show $SCRATCH_MNT >> $seqres.full + _scratch_unmount +} + +workload() +{ + local target=$(echo $SCRATCH_DEV_POOL | $AWK_PROG '{print $1}') + local profile=$1 + local num_devs=$2 + + _scratch_dev_pool_get $num_devs + echo "=== Testing profile $profile ===" >> $seqres.full + rm -f -- $tmp.saved_fssum + prepare_fs $profile + + # Corrupt the target device, only keep the superblock. + $XFS_IO_PROG -c "pwrite 1M 1023M" $target > /dev/null 2>&1 + + _scratch_mount + + # All content should be fine + $FSSUM_PROG -r $tmp.saved_fssum $SCRATCH_MNT > /dev/null + + # Scrub to fix the fs, this is known to report various correctable + # errors + $BTRFS_UTIL_PROG scrub start -B $SCRATCH_MNT >> $seqres.full 2>&1 + + # Make sure above scrub fixed the fs + $BTRFS_UTIL_PROG scrub start -Br $SCRATCH_MNT >> $seqres.full + if [ $? -ne 0 ]; then + echo "scrub failed to fix the fs for profile $profile" + fi + _scratch_unmount + _scratch_dev_pool_put +} + +workload raid1 2 +workload raid1c3 3 +workload raid1c4 4 +workload raid10 4 +workload raid5 3 +workload raid6 4 + +echo "Silence is golden" + +# success, all done +status=0 +exit
tests/btrfs/261.out new file +2 -0
@@ -0,0 +1,2 @@ +QA output created by 261 +Silence is golden
Signature · 1 line
2.36.1
Quoted 22 lines · Qu Wenruo
The new test case will verify that btrfs can handle one corrupted device without affecting the consistency of the filesystem.
Unlike a missing device, one corrupted device can return garbage to the fs, thus btrfs has to utilize its data/metadata checksum to verify which data is correct.
The test case will:
- Create a small fs
Mostly to speedup the test- Fill the fs with a regular file
- Use fsstress to create some contents
- Save the fssum for later verification
- Corrupt one device with garbage but keep the primary superblock untouched
Quoted 7 lines
- Run fssum verification
- Run scrub to fix the fs
- Run scrub again to make sure the fs is fine
Signed-off-by Qu Wenruo <[email protected]>
Works for me, and looks like a nice test to complement btrfs/027.
Quoted 91 lines
--- tests/btrfs/261 | 94 +++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/261.out | 2 + 2 files changed, 96 insertions(+) create mode 100755 tests/btrfs/261 create mode 100644 tests/btrfs/261.outtests/btrfs/261 new file +78
-0@@ -0,0 +1,94 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) 2022 SUSE Linux Products GmbH. All Rights Reserved. +# +# FS QA Test 261 +# +# Make sure btrfs raid profiles can handling one corrupted device +# without affecting the consistency of the fs. +# +. ./common/preamble +_begin_fstest raid + +. ./common/filter +. ./common/populate + +_supported_fs btrfs +_require_scratch_dev_pool 4 +_require_fssum + +prepare_fs() +{ + local profile=$1 + + # We don't want too large fs which can take too long to populate + # And the extra redirection of stderr is to avoid the RAID56 warning + # message to polluate the golden output + _scratch_pool_mkfs -m $profile -d $profile -b 1G >> $seqres.full 2>&1 + if [ $? -ne 0 ]; then + echo "mkfs $mkfs_opts failed" + return + fi + + # Disable compression, as compressed read repair is known to have problems + _scratch_mount -o compress=no + + # Fill some part of the fs first + $XFS_IO_PROG -f -c "pwrite -S 0xfe 0 400M" $SCRATCH_MNT/garbage > /dev/null 2>&1 + + # Then use fsstress to generate some extra contents. + # Disable setattr related operations, as it may set NODATACOW which will + # not allow us to use btrfs checksum to verify the content. + $FSSTRESS_PROG -f setattr=0 -d $SCRATCH_MNT -w -n 3000 > /dev/null 2>&1 + sync + + # Save the fssum of this fs + $FSSUM_PROG -A -f -w $tmp.saved_fssum $SCRATCH_MNT + $BTRFS_UTIL_PROG fi show $SCRATCH_MNT >> $seqres.full + _scratch_unmount +} + +workload() +{ + local target=$(echo $SCRATCH_DEV_POOL | $AWK_PROG '{print $1}') + local profile=$1 + local num_devs=$2 + + _scratch_dev_pool_get $num_devs + echo "=== Testing profile $profile ===" >> $seqres.full + rm -f -- $tmp.saved_fssum + prepare_fs $profile + + # Corrupt the target device, only keep the superblock. + $XFS_IO_PROG -c "pwrite 1M 1023M" $target > /dev/null 2>&1 + + _scratch_mount + + # All content should be fine + $FSSUM_PROG -r $tmp.saved_fssum $SCRATCH_MNT > /dev/null + + # Scrub to fix the fs, this is known to report various correctable + # errors + $BTRFS_UTIL_PROG scrub start -B $SCRATCH_MNT >> $seqres.full 2>&1 + + # Make sure above scrub fixed the fs + $BTRFS_UTIL_PROG scrub start -Br $SCRATCH_MNT >> $seqres.full + if [ $? -ne 0 ]; then + echo "scrub failed to fix the fs for profile $profile" + fi
tests/btrfs/261 line 88
Quoted 10 lines
tests/btrfs/261 new file +10
-0+ _scratch_unmount + _scratch_dev_pool_put +} + +workload raid1 2 +workload raid1c3 3 +workload raid1c4 4 +workload raid10 4 +workload raid5 3 +workload raid6 4
Speaking of 027,
Can you implement this with _btrfs_profile_configs?
Further, you could imagine writing a more generic test that does:
for raid in raids:
create-fs raid
screw-up disk(s)
check-condition
and make 027 and this new one (and others?) special cases of that. I think this might fall under YAGNI.. Food for thought :)
Quoted 17 lines
tests/btrfs/261 new file +6
-0+ +echo "Silence is golden" + +# success, all done +status=0 +exittests/btrfs/261.out new file +2
-0@@ -0,0 +1,2 @@ +QA output created by 261 +Silence is goldenSignature · 1 line
2.36.1
Quoted 138 lines · Boris Burkov
Quoted 22 lines · Qu Wenruo
The new test case will verify that btrfs can handle one corrupted device without affecting the consistency of the filesystem.
Unlike a missing device, one corrupted device can return garbage to the fs, thus btrfs has to utilize its data/metadata checksum to verify which data is correct.
The test case will:
- Create a small fs
Mostly to speedup the test- Fill the fs with a regular file
- Use fsstress to create some contents
- Save the fssum for later verification
- Corrupt one device with garbage but keep the primary superblock untouched
Quoted 7 lines
- Run fssum verification
- Run scrub to fix the fs
- Run scrub again to make sure the fs is fine
Signed-off-by Qu Wenruo <[email protected]>Works for me, and looks like a nice test to complement btrfs/027.
Reviewed-by Boris Burkov <[email protected]>Quoted 91 lines
--- tests/btrfs/261 | 94 +++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/261.out | 2 + 2 files changed, 96 insertions(+)create mode 100755 tests/btrfs/261
create mode 100644 tests/btrfs/261.outtests/btrfs/261 new file +78
-0@@ -0,0 +1,94 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) 2022 SUSE Linux Products GmbH. All Rights Reserved. +# +# FS QA Test 261 +# +# Make sure btrfs raid profiles can handling one corrupted device +# without affecting the consistency of the fs. +# +. ./common/preamble +_begin_fstest raid + +. ./common/filter +. ./common/populate + +_supported_fs btrfs +_require_scratch_dev_pool 4 +_require_fssum + +prepare_fs() +{ + local profile=$1 + + # We don't want too large fs which can take too long to populate + # And the extra redirection of stderr is to avoid the RAID56 warning + # message to polluate the golden output + _scratch_pool_mkfs -m $profile -d $profile -b 1G >> $seqres.full 2>&1 + if [ $? -ne 0 ]; then + echo "mkfs $mkfs_opts failed" + return + fi + + # Disable compression, as compressed read repair is known to have problems + _scratch_mount -o compress=no + + # Fill some part of the fs first + $XFS_IO_PROG -f -c "pwrite -S 0xfe 0 400M" $SCRATCH_MNT/garbage > /dev/null 2>&1 + + # Then use fsstress to generate some extra contents. + # Disable setattr related operations, as it may set NODATACOW which will + # not allow us to use btrfs checksum to verify the content. + $FSSTRESS_PROG -f setattr=0 -d $SCRATCH_MNT -w -n 3000 > /dev/null 2>&1 + sync + + # Save the fssum of this fs + $FSSUM_PROG -A -f -w $tmp.saved_fssum $SCRATCH_MNT + $BTRFS_UTIL_PROG fi show $SCRATCH_MNT >> $seqres.full + _scratch_unmount +} + +workload() +{ + local target=$(echo $SCRATCH_DEV_POOL | $AWK_PROG '{print $1}') + local profile=$1 + local num_devs=$2 + + _scratch_dev_pool_get $num_devs + echo "=== Testing profile $profile ===" >> $seqres.full + rm -f -- $tmp.saved_fssum + prepare_fs $profile + + # Corrupt the target device, only keep the superblock. + $XFS_IO_PROG -c "pwrite 1M 1023M" $target > /dev/null 2>&1 + + _scratch_mount + + # All content should be fine + $FSSUM_PROG -r $tmp.saved_fssum $SCRATCH_MNT > /dev/null + + # Scrub to fix the fs, this is known to report various correctable + # errors + $BTRFS_UTIL_PROG scrub start -B $SCRATCH_MNT >> $seqres.full 2>&1 + + # Make sure above scrub fixed the fs + $BTRFS_UTIL_PROG scrub start -Br $SCRATCH_MNT >> $seqres.full + if [ $? -ne 0 ]; then + echo "scrub failed to fix the fs for profile $profile" + fitests/btrfs/261 line 88
Quoted 10 lines
tests/btrfs/261 new file +10
-0+ _scratch_unmount + _scratch_dev_pool_put +} + +workload raid1 2 +workload raid1c3 3 +workload raid1c4 4 +workload raid10 4 +workload raid5 3 +workload raid6 4Speaking of 027,
Quoted 1 line
Can you implement this with
_btrfs_profile_configs?
In fact _btrfs_profile_configs() is what I went initially, but the following factors prevent it from being used:
- No way to specify the number of devices
That's not a big deal though.
- No way to skip unsupported profiles
Like DUP/SINGLE can not meet the duplication requirement.
And since _btrfs_profiles_configs() directly provides the mkfs options, it's not easy to just pick what we need.
Personally speaking, it would be much easier if we make _btrfs_profiles_configs() to just return a profile, not full mkfs options.
Quoted 6 lines
Further, you could imagine writing a more generic test that does:
for raid in raids:
create-fs raid
screw-up disk(s)
check-condition
I have seen some helper in common/populate, but unfortunately it only supports XFS and EXT4.
It may be a good idea to enhance that file though.
Thanks,
Qu
Quoted 21 lines
and make 027 and this new one (and others?) special cases of that. I think this might fall under YAGNI.. Food for thought :)
Quoted 17 lines
tests/btrfs/261 new file +6
-0+ +echo "Silence is golden" + +# success, all done +status=0 +exittests/btrfs/261.out new file +2
-1@@ -0,0 +1,2 @@ +QA output created by 261 +Silence is golden --2.36.1
Quoted 143 lines · Qu Wenruo
Quoted 138 lines · Boris Burkov
Quoted 22 lines · Qu Wenruo
The new test case will verify that btrfs can handle one corrupted device without affecting the consistency of the filesystem.
Unlike a missing device, one corrupted device can return garbage to the fs, thus btrfs has to utilize its data/metadata checksum to verify which data is correct.
The test case will:
- Create a small fs
Mostly to speedup the test- Fill the fs with a regular file
- Use fsstress to create some contents
- Save the fssum for later verification
- Corrupt one device with garbage but keep the primary superblock untouched
Quoted 7 lines
- Run fssum verification
- Run scrub to fix the fs
- Run scrub again to make sure the fs is fine
Signed-off-by Qu Wenruo <[email protected]>Works for me, and looks like a nice test to complement btrfs/027.
Reviewed-by Boris Burkov <[email protected]>Quoted 91 lines
--- tests/btrfs/261 | 94 +++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/261.out | 2 + 2 files changed, 96 insertions(+)create mode 100755 tests/btrfs/261
create mode 100644 tests/btrfs/261.outtests/btrfs/261 new file +78
-0@@ -0,0 +1,94 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) 2022 SUSE Linux Products GmbH. All Rights Reserved. +# +# FS QA Test 261 +# +# Make sure btrfs raid profiles can handling one corrupted device +# without affecting the consistency of the fs. +# +. ./common/preamble +_begin_fstest raid + +. ./common/filter +. ./common/populate + +_supported_fs btrfs +_require_scratch_dev_pool 4 +_require_fssum + +prepare_fs() +{ + local profile=$1 + + # We don't want too large fs which can take too long to populate + # And the extra redirection of stderr is to avoid the RAID56 warning + # message to polluate the golden output + _scratch_pool_mkfs -m $profile -d $profile -b 1G >> $seqres.full 2>&1 + if [ $? -ne 0 ]; then + echo "mkfs $mkfs_opts failed" + return + fi + + # Disable compression, as compressed read repair is known to have problems + _scratch_mount -o compress=no + + # Fill some part of the fs first + $XFS_IO_PROG -f -c "pwrite -S 0xfe 0 400M" $SCRATCH_MNT/garbage > /dev/null 2>&1 + + # Then use fsstress to generate some extra contents. + # Disable setattr related operations, as it may set NODATACOW which will + # not allow us to use btrfs checksum to verify the content. + $FSSTRESS_PROG -f setattr=0 -d $SCRATCH_MNT -w -n 3000 > /dev/null 2>&1 + sync + + # Save the fssum of this fs + $FSSUM_PROG -A -f -w $tmp.saved_fssum $SCRATCH_MNT + $BTRFS_UTIL_PROG fi show $SCRATCH_MNT >> $seqres.full + _scratch_unmount +} + +workload() +{ + local target=$(echo $SCRATCH_DEV_POOL | $AWK_PROG '{print $1}') + local profile=$1 + local num_devs=$2 + + _scratch_dev_pool_get $num_devs + echo "=== Testing profile $profile ===" >> $seqres.full + rm -f -- $tmp.saved_fssum + prepare_fs $profile + + # Corrupt the target device, only keep the superblock. + $XFS_IO_PROG -c "pwrite 1M 1023M" $target > /dev/null 2>&1 + + _scratch_mount + + # All content should be fine + $FSSUM_PROG -r $tmp.saved_fssum $SCRATCH_MNT > /dev/null + + # Scrub to fix the fs, this is known to report various correctable + # errors + $BTRFS_UTIL_PROG scrub start -B $SCRATCH_MNT >> $seqres.full 2>&1 + + # Make sure above scrub fixed the fs + $BTRFS_UTIL_PROG scrub start -Br $SCRATCH_MNT >> $seqres.full + if [ $? -ne 0 ]; then + echo "scrub failed to fix the fs for profile $profile" + fitests/btrfs/261 line 88
Quoted 10 lines
tests/btrfs/261 new file +10
-0+ _scratch_unmount + _scratch_dev_pool_put +} + +workload raid1 2 +workload raid1c3 3 +workload raid1c4 4 +workload raid10 4 +workload raid5 3 +workload raid6 4Speaking of 027,
Quoted 1 line
Can you implement this with
_btrfs_profile_configs?
Quoted 8 lines
In fact
_btrfs_profile_configs()is what I went initially, but the following factors prevent it from being used:- No way to specify the number of devices
That's not a big deal though.- No way to skip unsupported profiles
Like DUP/SINGLE can not meet the duplication requirement.
It takes an argument and if you pass "replace-missing" I think it does what you want. You could also add another string it accepts for different semantics.
Quoted 7 lines
And since
_btrfs_profiles_configs()directly provides the mkfs options, it's not easy to just pick what we need.Personally speaking, it would be much easier if we make
_btrfs_profiles_configs()to just return a profile, not full mkfs options.
Giving the caller more flexibility with their call to mkfs does seem like a good idea to me.
Quoted 8 lines
Quoted 7 lines
Further, you could imagine writing a more generic test that does:
for raid in raids:
create-fs raid
screw-up disk(s)
check-condition
Quoted 4 lines
I have seen some helper in
common/populate, but unfortunately it only supports XFS and EXT4.It may be a good idea to enhance that file though.
Cool, never noticed that before. Would be interesting to make it work with btrfs.
Quoted 25 lines
Thanks,
QuQuoted 21 lines
and make 027 and this new one (and others?) special cases of that. I think this might fall under YAGNI.. Food for thought :)
Quoted 17 lines
tests/btrfs/261 new file +6
-0+ +echo "Silence is golden" + +# success, all done +status=0 +exittests/btrfs/261.out new file +2
-1@@ -0,0 +1,2 @@ +QA output created by 261 +Silence is golden --2.36.1
Quoted 42 lines · Qu Wenruo
The new test case will verify that btrfs can handle one corrupted device without affecting the consistency of the filesystem.
Unlike a missing device, one corrupted device can return garbage to the fs, thus btrfs has to utilize its data/metadata checksum to verify which data is correct.
The test case will:
- Create a small fs
Mostly to speedup the test- Fill the fs with a regular file
- Use fsstress to create some contents
- Save the fssum for later verification
- Corrupt one device with garbage but keep the primary superblock untouched
- Run fssum verification
- Run scrub to fix the fs
- Run scrub again to make sure the fs is fine
Signed-off-by Qu Wenruo <[email protected]>--- tests/btrfs/261 | 94 +++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/261.out | 2 + 2 files changed, 96 insertions(+) create mode 100755 tests/btrfs/261 create mode 100644 tests/btrfs/261.outtests/btrfs/261 new file +1
-0@@ -0,0 +1,94 @@ +#! /bin/bash
tests/btrfs/261 line 11
Quoted 10 lines
tests/btrfs/261 new file +10
-0+# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) 2022 SUSE Linux Products GmbH. All Rights Reserved. +# +# FS QA Test 261 +# +# Make sure btrfs raid profiles can handling one corrupted device +# without affecting the consistency of the fs. +# +. ./common/preamble +_begin_fstest raid
Do you think about add it into auto group, or more groups?
Quoted 10 lines
tests/btrfs/261 new file +10
-0+ +. ./common/filter +. ./common/populate + +_supported_fs btrfs +_require_scratch_dev_pool 4 +_require_fssum + +prepare_fs() +{
tests/btrfs/261 line 31
Quoted 10 lines
tests/btrfs/261 new file +10
-0+ local profile=$1 + + # We don't want too large fs which can take too long to populate + # And the extra redirection of stderr is to avoid the RAID56 warning + # message to polluate the golden output + _scratch_pool_mkfs -m $profile -d $profile -b 1G >> $seqres.full 2>&1 + if [ $? -ne 0 ]; then + echo "mkfs $mkfs_opts failed" + return + fi
If mkfs fails, below workload() will keep running, I think it's not what you want, right?
Quoted 12 lines
tests/btrfs/261 new file +12
-0+ + # Disable compression, as compressed read repair is known to have problems + _scratch_mount -o compress=no + + # Fill some part of the fs first + $XFS_IO_PROG -f -c "pwrite -S 0xfe 0 400M" $SCRATCH_MNT/garbage > /dev/null 2>&1 + + # Then use fsstress to generate some extra contents. + # Disable setattr related operations, as it may set NODATACOW which will + # not allow us to use btrfs checksum to verify the content. + $FSSTRESS_PROG -f setattr=0 -d $SCRATCH_MNT -w -n 3000 > /dev/null 2>&1 + sync
tests/btrfs/261 line 53
Quoted 10 lines
tests/btrfs/261 new file +10
-0+ + # Save the fssum of this fs + $FSSUM_PROG -A -f -w $tmp.saved_fssum $SCRATCH_MNT + $BTRFS_UTIL_PROG fi show $SCRATCH_MNT >> $seqres.full + _scratch_unmount +} + +workload() +{ + local target=$(echo $SCRATCH_DEV_POOL | $AWK_PROG '{print $1}')
In common/config, we always set SCRATCH_DEV to the first of $SCRATCH_DEV_POOL, as below:
# a btrfs tester will set only SCRATCH_DEV_POOL, we will put first of its dev
# to SCRATCH_DEV and rest to SCRATCH_DEV_POOL to maintain the backward compatibility
if [ ! -z "$SCRATCH_DEV_POOL" ]; then
if [ ! -z "$SCRATCH_DEV" ]; then
echo "common/config: Error: \$SCRATCH_DEV ($SCRATCH_DEV) should be unset when \$SCRATCH_DEV_POOL ($SCRATCH_DEV_POOL) is set"
exit 1
fi
SCRATCH_DEV=`echo $SCRATCH_DEV_POOL | awk '{print $1}'`is that help?
tests/btrfs/261 line 63
Quoted 10 lines
tests/btrfs/261 new file +10
-0+ local profile=$1 + local num_devs=$2 + + _scratch_dev_pool_get $num_devs + echo "=== Testing profile $profile ===" >> $seqres.full + rm -f -- $tmp.saved_fssum + prepare_fs $profile + + # Corrupt the target device, only keep the superblock. + $XFS_IO_PROG -c "pwrite 1M 1023M" $target > /dev/null 2>&1
Do you need "_require_scratch_nocheck", if you corrupt the fs purposely?
But I think it won't check the SCRATCH_DEV currently, due to you even don't use _require_scratch, and the _require_scratch_dev_pool might not trigger a fsck at the end of the testing.
Hmm... I prefer having a "_require_scratch_nocheck", and you can use $SCRATCH_DEV to replace your "$target".
Quoted 42 lines
tests/btrfs/261 new file +31
-0+ + _scratch_mount + + # All content should be fine + $FSSUM_PROG -r $tmp.saved_fssum $SCRATCH_MNT > /dev/null + + # Scrub to fix the fs, this is known to report various correctable + # errors + $BTRFS_UTIL_PROG scrub start -B $SCRATCH_MNT >> $seqres.full 2>&1 + + # Make sure above scrub fixed the fs + $BTRFS_UTIL_PROG scrub start -Br $SCRATCH_MNT >> $seqres.full + if [ $? -ne 0 ]; then + echo "scrub failed to fix the fs for profile $profile" + fi + _scratch_unmount + _scratch_dev_pool_put +} + +workload raid1 2 +workload raid1c3 3 +workload raid1c4 4 +workload raid10 4 +workload raid5 3 +workload raid6 4 + +echo "Silence is golden" + +# success, all done +status=0 +exittests/btrfs/261.out new file +2
-0@@ -0,0 +1,2 @@ +QA output created by 261 +Silence is goldenSignature · 1 line
2.36.1
Quoted 47 lines · Zorro Lang
Quoted 46 lines · Qu Wenruo
The new test case will verify that btrfs can handle one corrupted device without affecting the consistency of the filesystem.
Unlike a missing device, one corrupted device can return garbage to the fs, thus btrfs has to utilize its data/metadata checksum to verify which data is correct.
The test case will:
- Create a small fs
Mostly to speedup the test- Fill the fs with a regular file
- Use fsstress to create some contents
- Save the fssum for later verification
- Corrupt one device with garbage but keep the primary superblock untouched
- Run fssum verification
- Run scrub to fix the fs
- Run scrub again to make sure the fs is fine
Signed-off-by Qu Wenruo <[email protected]>--- tests/btrfs/261 | 94 +++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/261.out | 2 + 2 files changed, 96 insertions(+)create mode 100755 tests/btrfs/261
create mode 100644 tests/btrfs/261.outtests/btrfs/261 new file +5
-0@@ -0,0 +1,94 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) 2022 SUSE Linux Products GmbH. All Rights Reserved. +# +# FS QA Test 261
Quoted 8 lines
tests/btrfs/261 line 11
Quoted 6 lines
tests/btrfs/261 new file +6
-0+# +# Make sure btrfs raid profiles can handling one corrupted device +# without affecting the consistency of the fs. +# +. ./common/preamble +_begin_fstest raidDo you think about add it into auto group, or more groups?
Oh all my bad, was planning to add to auto later but forgot.
Quoted 16 lines
Quoted 16 lines
tests/btrfs/261 new file +15
-0+ +. ./common/filter +. ./common/populate + +_supported_fs btrfs +_require_scratch_dev_pool 4 +_require_fssum + +prepare_fs() +{ + local profile=$1 + + # We don't want too large fs which can take too long to populate + # And the extra redirection of stderr is to avoid the RAID56 warning + # message to polluate the golden output
Quoted 8 lines
tests/btrfs/261 line 31
Quoted 5 lines
tests/btrfs/261 new file +5
-0+ _scratch_pool_mkfs -m $profile -d $profile -b 1G >> $seqres.full 2>&1 + if [ $? -ne 0 ]; then + echo "mkfs $mkfs_opts failed" + return + fiIf mkfs fails, below
workload()will keep running, I think it's not what you want, right?
Right, it should call _fail() instead.
Quoted 36 lines
Quoted 13 lines
tests/btrfs/261 new file +12
-0+ + # Disable compression, as compressed read repair is known to have problems + _scratch_mount -o compress=no + + # Fill some part of the fs first + $XFS_IO_PROG -f -c "pwrite -S 0xfe 0 400M" $SCRATCH_MNT/garbage > /dev/null 2>&1 + + # Then use fsstress to generate some extra contents. + # Disable setattr related operations, as it may set NODATACOW which will + # not allow us to use btrfs checksum to verify the content. + $FSSTRESS_PROG -f setattr=0 -d $SCRATCH_MNT -w -n 3000 > /dev/null 2>&1 + synctests/btrfs/261 line 53
Quoted 10 lines
tests/btrfs/261 new file +10
-0+ + # Save the fssum of this fs + $FSSUM_PROG -A -f -w $tmp.saved_fssum $SCRATCH_MNT + $BTRFS_UTIL_PROG fi show $SCRATCH_MNT >> $seqres.full + _scratch_unmount +} + +workload() +{ + local target=$(echo $SCRATCH_DEV_POOL | $AWK_PROG '{print $1}')In common/config, we always set
SCRATCH_DEVto the first of $SCRATCH_DEV_POOL, as below:# a btrfs tester will set only SCRATCH_DEV_POOL, we will put first of its dev # to SCRATCH_DEV and rest to SCRATCH_DEV_POOL to maintain the backward compatibility if [ ! -z "$SCRATCH_DEV_POOL" ]; then if [ ! -z "$SCRATCH_DEV" ]; then echo "common/config: Error: \$SCRATCH_DEV ($SCRATCH_DEV) should be unset when \$SCRATCH_DEV_POOL ($SCRATCH_DEV_POOL) is set" exit 1 fi SCRATCH_DEV=`echo $SCRATCH_DEV_POOL | awk '{print $1}'`
Quoted 1 line
is that help?
Thanks, then I can use SCRATCH_DEV directly.
Quoted 13 lines
tests/btrfs/261 line 63
Quoted 11 lines
tests/btrfs/261 new file +10
-0+ local profile=$1 + local num_devs=$2 + + _scratch_dev_pool_get $num_devs + echo "=== Testing profile $profile ===" >> $seqres.full + rm -f -- $tmp.saved_fssum + prepare_fs $profile + + # Corrupt the target device, only keep the superblock. + $XFS_IO_PROG -c "pwrite 1M 1023M" $target > /dev/null 2>&1Do you need "
_require_scratch_nocheck", if you corrupt the fs purposely?
I don't think we need that.
The point here is, we will immediately mount the fs and do a RW scrub, which should make all devices good.
Thus if fsck after the test case found something wrong, it means the RW scrub doesn't properly repair the fs.
Thanks for the review,
Qu
Quoted 51 lines
But I think it won't check the
SCRATCH_DEVcurrently, due to you even don't use_require_scratch, and the_require_scratch_dev_poolmight not trigger a fsck at the end of the testing.Hmm... I prefer having a "
_require_scratch_nocheck", and you can use $SCRATCH_DEV to replace your "$target".Quoted 42 lines
tests/btrfs/261 new file +31
-0+ + _scratch_mount + + # All content should be fine + $FSSUM_PROG -r $tmp.saved_fssum $SCRATCH_MNT > /dev/null + + # Scrub to fix the fs, this is known to report various correctable + # errors + $BTRFS_UTIL_PROG scrub start -B $SCRATCH_MNT >> $seqres.full 2>&1 + + # Make sure above scrub fixed the fs + $BTRFS_UTIL_PROG scrub start -Br $SCRATCH_MNT >> $seqres.full + if [ $? -ne 0 ]; then + echo "scrub failed to fix the fs for profile $profile" + fi + _scratch_unmount + _scratch_dev_pool_put +} + +workload raid1 2 +workload raid1c3 3 +workload raid1c4 4 +workload raid10 4 +workload raid5 3 +workload raid6 4 + +echo "Silence is golden" + +# success, all done +status=0 +exittests/btrfs/261.out new file +2
-1@@ -0,0 +1,2 @@ +QA output created by 261 +Silence is golden --2.36.1