two more btrfs read repair tests
Hi all,
this series adds two more tests for the btrfs read-repair code that exercise the raid1c3 profile.
PATCH 1/2 btrfs: test repair with sectors corrupted in multiple mirrors
Test that repair handles the case where it needs to read from more than a single mirror on the raid1c3 profile.
--- tests/btrfs/265 | 127 ++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/265.out | 75 ++++++++++++++++++++++++++ 2 files changed, 202 insertions(+) create mode 100755 tests/btrfs/265 create mode 100644 tests/btrfs/265.out
tests/btrfs/265 new file +127 -0
@@ -0,0 +1,127 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2017 Liu Bo. All Rights Reserved. +# Copyright (c) 2022 Christoph Hellwig. +# +# FS QA Test 265 +# +# Test that btrfs raid repair on a raid1c3 profile can repair corruption on two +# mirrors for the same logical offset. +# +. ./common/preamble +_begin_fstest auto quick read_repair + +# Import common functions. +. ./common/filter + +# real QA test starts here + +_supported_fs btrfs +_require_scratch_dev_pool 3 + +BTRFS_MAP_LOGICAL_PROG=$(type -P btrfs-map-logical) + +_require_command "$BTRFS_MAP_LOGICAL_PROG" btrfs-map-logical +_require_command "$FILEFRAG_PROG" filefrag +_require_odirect +# Overwriting data is forbidden on a zoned block device +_require_non_zoned_device "${SCRATCH_DEV}" + +get_physical() +{ + local logical=$1 + local stripe=$2 + + $BTRFS_MAP_LOGICAL_PROG -l $logical $SCRATCH_DEV >> $seqres.full 2>&1 + $BTRFS_MAP_LOGICAL_PROG -l $logical $SCRATCH_DEV | \ + $AWK_PROG "(\$1 ~ /mirror/ && \$2 ~ /$stripe/) { print \$6 }" +} + +get_device_path() +{ + local logical=$1 + local stripe=$2 + + $BTRFS_MAP_LOGICAL_PROG -l $logical $SCRATCH_DEV | \ + $AWK_PROG "(\$1 ~ /mirror/ && \$2 ~ /$stripe/) { print \$8 }" +} + +_scratch_dev_pool_get 3 +# step 1, create a raid1 btrfs which contains one 128k file. +echo "step 1......mkfs.btrfs" + +mkfs_opts="-d raid1c3 -b 1G" +_scratch_pool_mkfs $mkfs_opts >>$seqres.full 2>&1 + +# make sure data is written to the start position of the data chunk +_scratch_mount $(_btrfs_no_v1_cache_opt) + +$XFS_IO_PROG -f -d -c "pwrite -S 0xaa -b 128K 0 128K" \ + "$SCRATCH_MNT/foobar" | \ + _filter_xfs_io_offset + +# ensure btrfs-map-logical sees the tree updates +sync + +# step 2, corrupt the first 64k of one copy (on SCRATCH_DEV which is the first +# one in $SCRATCH_DEV_POOL +echo "step 2......corrupt file extent" + +${FILEFRAG_PROG} -v $SCRATCH_MNT/foobar >> $seqres.full +logical_in_btrfs=`${FILEFRAG_PROG} -v $SCRATCH_MNT/foobar | _filter_filefrag | cut -d '#' -f 1` + +physical1=$(get_physical ${logical_in_btrfs} 1) +devpath1=$(get_device_path ${logical_in_btrfs} 1) + +physical2=$(get_physical ${logical_in_btrfs} 2) +devpath2=$(get_device_path ${logical_in_btrfs} 2) + +_scratch_unmount + +echo " corrupt stripe #1, devpath $devpath1 physical $physical1" \ + >> $seqres.full +$XFS_IO_PROG -d -c "pwrite -S 0xbf -b 64K $physical1 64K" $devpath1 \ + > /dev/null + +echo " corrupt stripe #2, devpath $devpath2 physical $physical2" \ + >> $seqres.full +$XFS_IO_PROG -d -c "pwrite -S 0xbf -b 64K $physical2 64K" $devpath2 \ + > /dev/null + +_scratch_mount + +# step 3, 128k dio read (this read can repair bad copy) +echo "step 3......repair the bad copy" + +# since raid1c3 consists of three copies, and the bad copy was put on stripe #1 +# while the good copy lies the other stripes, the bad copy only gets accessed +# when the reader's pid % 3 is 1 +while true; do + $XFS_IO_PROG -d -c "pread -b 128K 0 128K" "$SCRATCH_MNT/foobar" > /dev/null & + pid=$! + wait + if [ $((pid % 3)) == 1 ]; then + break + fi +done +while true; do + $XFS_IO_PROG -d -c "pread -b 128K 0 128K" "$SCRATCH_MNT/foobar" > /dev/null & + pid=$! + wait + if [ $((pid % 3)) == 2 ]; then + break + fi +done + +_scratch_unmount + +echo "step 4......check if the repair works" +$XFS_IO_PROG -d -c "pread -v -b 512 $physical1 512" $devpath1 |\ + _filter_xfs_io_offset +$XFS_IO_PROG -d -c "pread -v -b 512 $physical2 512" $devpath2 |\ + _filter_xfs_io_offset + +_scratch_dev_pool_put +# success, all done +status=0 +exit
tests/btrfs/265.out new file +75 -0
75 lines not on this page · raw message
Signature · 1 line
2.30.2
Re: [PATCH 1/2] btrfs: test repair with sectors corrupted in multiple mirrors
Quoted 29 lines · Christoph Hellwig
Test that repair handles the case where it needs to read from more than a single mirror on the raid1c3 profile.
Signed-off-by Christoph Hellwig <[email protected]>--- tests/btrfs/265 | 127 ++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/265.out | 75 ++++++++++++++++++++++++++ 2 files changed, 202 insertions(+) create mode 100755 tests/btrfs/265 create mode 100644 tests/btrfs/265.outtests/btrfs/265 new file +12
-0@@ -0,0 +1,127 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2017 Liu Bo. All Rights Reserved. +# Copyright (c) 2022 Christoph Hellwig. +# +# FS QA Test 265 +# +# Test that btrfs raid repair on a raid1c3 profile can repair corruption on two +# mirrors for the same logical offset. +# +. ./common/preamble +_begin_fstest auto quick read_repair
tests/btrfs/265 line 22
Quoted 10 lines
tests/btrfs/265 new file +10
-0+ +# Import common functions. +. ./common/filter + +# real QA test starts here + +_supported_fs btrfs +_require_scratch_dev_pool 3 + +BTRFS_MAP_LOGICAL_PROG=$(type -P btrfs-map-logical)
Hi Christoph,
Thanks for the new test cases. I'm not an btrfs expert, so I'll only give this patchset some review points from fstests side. Btrfs forks please feel free to review this patchset :)
If btrfs-map-logical is a general command from btffs-progs, I think we can move above line into common/config.
Quoted 45 lines
tests/btrfs/265 new file +45
-0+ +_require_command "$BTRFS_MAP_LOGICAL_PROG" btrfs-map-logical +_require_command "$FILEFRAG_PROG" filefrag +_require_odirect +# Overwriting data is forbidden on a zoned block device +_require_non_zoned_device "${SCRATCH_DEV}" + +get_physical() +{ + local logical=$1 + local stripe=$2 + + $BTRFS_MAP_LOGICAL_PROG -l $logical $SCRATCH_DEV >> $seqres.full 2>&1 + $BTRFS_MAP_LOGICAL_PROG -l $logical $SCRATCH_DEV | \ + $AWK_PROG "(\$1 ~ /mirror/ && \$2 ~ /$stripe/) { print \$6 }" +} + +get_device_path() +{ + local logical=$1 + local stripe=$2 + + $BTRFS_MAP_LOGICAL_PROG -l $logical $SCRATCH_DEV | \ + $AWK_PROG "(\$1 ~ /mirror/ && \$2 ~ /$stripe/) { print \$8 }" +} + +_scratch_dev_pool_get 3 +# step 1, create a raid1 btrfs which contains one 128k file. +echo "step 1......mkfs.btrfs" + +mkfs_opts="-d raid1c3 -b 1G" +_scratch_pool_mkfs $mkfs_opts >>$seqres.full 2>&1 + +# make sure data is written to the start position of the data chunk +_scratch_mount $(_btrfs_no_v1_cache_opt) + +$XFS_IO_PROG -f -d -c "pwrite -S 0xaa -b 128K 0 128K" \ + "$SCRATCH_MNT/foobar" | \ + _filter_xfs_io_offset + +# ensure btrfs-map-logical sees the tree updates +sync + +# step 2, corrupt the first 64k of one copy (on SCRATCH_DEV which is the first +# one in $SCRATCH_DEV_POOL
tests/btrfs/265 line 77
Quoted 10 lines
tests/btrfs/265 new file +10
-0+echo "step 2......corrupt file extent" + +${FILEFRAG_PROG} -v $SCRATCH_MNT/foobar >> $seqres.full +logical_in_btrfs=`${FILEFRAG_PROG} -v $SCRATCH_MNT/foobar | _filter_filefrag | cut -d '#' -f 1` + +physical1=$(get_physical ${logical_in_btrfs} 1) +devpath1=$(get_device_path ${logical_in_btrfs} 1) + +physical2=$(get_physical ${logical_in_btrfs} 2) +devpath2=$(get_device_path ${logical_in_btrfs} 2)
I saw you do the same steps from "${FILEFRAG_PROG} -v $SCRATCH_MNT/foobar" to this line in 2 patches, so I'm wondering are the FILEFRAG_PROG lines and get_physical() and get_device_path() worth moving to common/btrfs (might need to change name), to make them get "device_path" or "physical" from a filename (e.g. $SCRATCH_MNT/foobar) argument?
Thanks,
Zorro
Quoted 134 lines
tests/btrfs/265 new file +50
-0+ +_scratch_unmount + +echo " corrupt stripe #1, devpath $devpath1 physical $physical1" \ + >> $seqres.full +$XFS_IO_PROG -d -c "pwrite -S 0xbf -b 64K $physical1 64K" $devpath1 \ + > /dev/null + +echo " corrupt stripe #2, devpath $devpath2 physical $physical2" \ + >> $seqres.full +$XFS_IO_PROG -d -c "pwrite -S 0xbf -b 64K $physical2 64K" $devpath2 \ + > /dev/null + +_scratch_mount + +# step 3, 128k dio read (this read can repair bad copy) +echo "step 3......repair the bad copy" + +# since raid1c3 consists of three copies, and the bad copy was put on stripe #1 +# while the good copy lies the other stripes, the bad copy only gets accessed +# when the reader's pid % 3 is 1 +while true; do + $XFS_IO_PROG -d -c "pread -b 128K 0 128K" "$SCRATCH_MNT/foobar" > /dev/null & + pid=$! + wait + if [ $((pid % 3)) == 1 ]; then + break + fi +done +while true; do + $XFS_IO_PROG -d -c "pread -b 128K 0 128K" "$SCRATCH_MNT/foobar" > /dev/null & + pid=$! + wait + if [ $((pid % 3)) == 2 ]; then + break + fi +done + +_scratch_unmount + +echo "step 4......check if the repair works" +$XFS_IO_PROG -d -c "pread -v -b 512 $physical1 512" $devpath1 |\ + _filter_xfs_io_offset +$XFS_IO_PROG -d -c "pread -v -b 512 $physical2 512" $devpath2 |\ + _filter_xfs_io_offset + +_scratch_dev_pool_put +# success, all done +status=0 +exittests/btrfs/265.out new file +75
-0@@ -0,0 +1,75 @@ +QA output created by 265 +step 1......mkfs.btrfs +wrote 131072/131072 bytes +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +step 2......corrupt file extent +step 3......repair the bad copy +step 4......check if the repair works +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +read 512/512 bytes +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +read 512/512 bytes +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)Signature · 1 line
2.30.2
Re: [PATCH 1/2] btrfs: test repair with sectors corrupted in multiple mirrors
In reply to Christoph HellwigQuoted 4 lines · Christoph Hellwig
Test that repair handles the case where it needs to read from more than a single mirror on the raid1c3 profile.
Signed-off-by Christoph Hellwig <[email protected]>
Looks good.
A few nits below.
Quoted 53 lines
--- tests/btrfs/265 | 127 ++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/265.out | 75 ++++++++++++++++++++++++++ 2 files changed, 202 insertions(+)create mode 100755 tests/btrfs/265
create mode 100644 tests/btrfs/265.outtests/btrfs/265 new file +38
-0@@ -0,0 +1,127 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2017 Liu Bo. All Rights Reserved. +# Copyright (c) 2022 Christoph Hellwig. +# +# FS QA Test 265 +# +# Test that btrfs raid repair on a raid1c3 profile can repair corruption on two +# mirrors for the same logical offset. +# +. ./common/preamble +_begin_fstest auto quick read_repair + +# Import common functions. +. ./common/filter + +# real QA test starts here + +_supported_fs btrfs +_require_scratch_dev_pool 3 + +BTRFS_MAP_LOGICAL_PROG=$(type -P btrfs-map-logical) + +_require_command "$BTRFS_MAP_LOGICAL_PROG" btrfs-map-logical +_require_command "$FILEFRAG_PROG" filefrag +_require_odirect +# Overwriting data is forbidden on a zoned block device +_require_non_zoned_device "${SCRATCH_DEV}" + +get_physical() +{ + local logical=$1 + local stripe=$2 + + $BTRFS_MAP_LOGICAL_PROG -l $logical $SCRATCH_DEV >> $seqres.full 2>&1 + $BTRFS_MAP_LOGICAL_PROG -l $logical $SCRATCH_DEV | \ + $AWK_PROG "(\$1 ~ /mirror/ && \$2 ~ /$stripe/) { print \$6 }" +}
tests/btrfs/265 line 48
Quoted 10 lines
tests/btrfs/265 new file +10
-0+ +get_device_path() +{ + local logical=$1 + local stripe=$2 + + $BTRFS_MAP_LOGICAL_PROG -l $logical $SCRATCH_DEV | \ + $AWK_PROG "(\$1 ~ /mirror/ && \$2 ~ /$stripe/) { print \$8 }" +} +
4-5 more test cases use these functions. Why not bring them to common/btrfs.
Quoted 58 lines
tests/btrfs/265 new file +56
-0+_scratch_dev_pool_get 3 +# step 1, create a raid1 btrfs which contains one 128k file. +echo "step 1......mkfs.btrfs" + +mkfs_opts="-d raid1c3 -b 1G" +_scratch_pool_mkfs $mkfs_opts >>$seqres.full 2>&1 + +# make sure data is written to the start position of the data chunk +_scratch_mount $(_btrfs_no_v1_cache_opt) + +$XFS_IO_PROG -f -d -c "pwrite -S 0xaa -b 128K 0 128K" \ + "$SCRATCH_MNT/foobar" | \ + _filter_xfs_io_offset + +# ensure btrfs-map-logical sees the tree updates +sync + +# step 2, corrupt the first 64k of one copy (on SCRATCH_DEV which is the first +# one in $SCRATCH_DEV_POOL +echo "step 2......corrupt file extent" + +${FILEFRAG_PROG} -v $SCRATCH_MNT/foobar >> $seqres.full +logical_in_btrfs=`${FILEFRAG_PROG} -v $SCRATCH_MNT/foobar | _filter_filefrag | cut -d '#' -f 1` + +physical1=$(get_physical ${logical_in_btrfs} 1) +devpath1=$(get_device_path ${logical_in_btrfs} 1) + +physical2=$(get_physical ${logical_in_btrfs} 2) +devpath2=$(get_device_path ${logical_in_btrfs} 2) + +_scratch_unmount + +echo " corrupt stripe #1, devpath $devpath1 physical $physical1" \ + >> $seqres.full +$XFS_IO_PROG -d -c "pwrite -S 0xbf -b 64K $physical1 64K" $devpath1 \ + > /dev/null + +echo " corrupt stripe #2, devpath $devpath2 physical $physical2" \ + >> $seqres.full +$XFS_IO_PROG -d -c "pwrite -S 0xbf -b 64K $physical2 64K" $devpath2 \ + > /dev/null + +_scratch_mount + +# step 3, 128k dio read (this read can repair bad copy) +echo "step 3......repair the bad copy" + +# since raid1c3 consists of three copies, and the bad copy was put on stripe #1 +# while the good copy lies the other stripes, the bad copy only gets accessed +# when the reader's pid % 3 is 1 +while true; do + $XFS_IO_PROG -d -c "pread -b 128K 0 128K" "$SCRATCH_MNT/foobar" > /dev/null & + pid=$! + wait + if [ $((pid % 3)) == 1 ]; then + break
tests/btrfs/265 line 114
Quoted 10 lines
tests/btrfs/265 new file +10
-0+ fi +done +while true; do + $XFS_IO_PROG -d -c "pread -b 128K 0 128K" "$SCRATCH_MNT/foobar" > /dev/null & + pid=$! + wait + if [ $((pid % 3)) == 2 ]; then + break + fi +done
More test cases use the same logic. They can wrap into a helper.
Thanks, Anand
Quoted 94 lines
tests/btrfs/265 new file +13
-0+ +_scratch_unmount + +echo "step 4......check if the repair works" +$XFS_IO_PROG -d -c "pread -v -b 512 $physical1 512" $devpath1 |\ + _filter_xfs_io_offset +$XFS_IO_PROG -d -c "pread -v -b 512 $physical2 512" $devpath2 |\ + _filter_xfs_io_offset + +_scratch_dev_pool_put +# success, all done +status=0 +exittests/btrfs/265.out new file +75
-0@@ -0,0 +1,75 @@ +QA output created by 265 +step 1......mkfs.btrfs +wrote 131072/131072 bytes +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +step 2......corrupt file extent +step 3......repair the bad copy +step 4......check if the repair works +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +read 512/512 bytes +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +read 512/512 bytes +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
PATCH 2/2 btrfs: test repair with corrupted sectors interleaved over multiple mirrors
In reply to Christoph HellwigTest that repair handles the case where it needs to read from more than a single mirror on the raid1c3 profile and needs to take turns over the mirrors to recover data for the whole read.
--- tests/btrfs/266 | 145 ++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/266.out | 109 +++++++++++++++++++++++++++++++++ 2 files changed, 254 insertions(+) create mode 100755 tests/btrfs/266 create mode 100644 tests/btrfs/266.out
tests/btrfs/266 new file +145 -0
@@ -0,0 +1,145 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2017 Liu Bo. All Rights Reserved. +# Copyright (c) 2022 Christoph Hellwig. +# +# FS QA Test 266 +# +# Test that btrfs raid repair on a raid1c3 profile can repair interleaving +# errors on all mirrors. +# + +. ./common/preamble +_begin_fstest auto quick read_repair + +# Import common functions. +. ./common/filter + +# real QA test starts here + +_supported_fs btrfs +_require_scratch_dev_pool 3 + +BTRFS_MAP_LOGICAL_PROG=$(type -P btrfs-map-logical) + +_require_command "$BTRFS_MAP_LOGICAL_PROG" btrfs-map-logical +_require_command "$FILEFRAG_PROG" filefrag +_require_odirect +# Overwriting data is forbidden on a zoned block device +_require_non_zoned_device "${SCRATCH_DEV}" + +get_physical() +{ + local logical=$1 + local stripe=$2 + + $BTRFS_MAP_LOGICAL_PROG -l $logical $SCRATCH_DEV >> $seqres.full 2>&1 + $BTRFS_MAP_LOGICAL_PROG -l $logical $SCRATCH_DEV | \ + $AWK_PROG "(\$1 ~ /mirror/ && \$2 ~ /$stripe/) { print \$6 }" +} + +get_device_path() +{ + local logical=$1 + local stripe=$2 + + $BTRFS_MAP_LOGICAL_PROG -l $logical $SCRATCH_DEV | \ + $AWK_PROG "(\$1 ~ /mirror/ && \$2 ~ /$stripe/) { print \$8 }" +} + +_scratch_dev_pool_get 3 +# step 1, create a raid1 btrfs which contains one 128k file. +echo "step 1......mkfs.btrfs" + +mkfs_opts="-d raid1c3 -b 1G" +_scratch_pool_mkfs $mkfs_opts >>$seqres.full 2>&1 + +# make sure data is written to the start position of the data chunk +_scratch_mount $(_btrfs_no_v1_cache_opt) + +$XFS_IO_PROG -f -d -c "pwrite -S 0xaa -b 128K 0 128K" \ + "$SCRATCH_MNT/foobar" | \ + _filter_xfs_io_offset + +# ensure btrfs-map-logical sees the tree updates +sync + +# step 2, corrupt 4k in each copy +echo "step 2......corrupt file extent" + +${FILEFRAG_PROG} -v $SCRATCH_MNT/foobar >> $seqres.full +logical=`${FILEFRAG_PROG} -v $SCRATCH_MNT/foobar | _filter_filefrag | cut -d '#' -f 1` + +physical1=$(get_physical ${logical} 1) +devpath1=$(get_device_path ${logical} 1) + +physical2=$(get_physical ${logical} 2) +devpath2=$(get_device_path ${logical} 2) + +physical3=$(get_physical ${logical} 3) +devpath3=$(get_device_path ${logical} 3) + +_scratch_unmount + +$XFS_IO_PROG -d -c "pwrite -S 0xbd -b 4K $physical3 4K" $devpath3 \ + > /dev/null + +$XFS_IO_PROG -d -c "pwrite -S 0xba -b 4K $physical1 8K" $devpath1 \ + > /dev/null + +$XFS_IO_PROG -d -c "pwrite -S 0xbb -b 4K $((physical2 + 4096)) 8K" $devpath2 \ + > /dev/null + +$XFS_IO_PROG -d -c "pwrite -S 0xbc -b 4K $((physical3 + (2 * 4096))) 8K" \ + $devpath3 > /dev/null + +_scratch_mount + +# step 3, 128k dio read (this read can repair bad copy) +echo "step 3......repair the bad copy" + +# since raid1c3 consists of three copies, and the bad copy was put on stripe #1 +# while the good copy lies the other stripes, the bad copy only gets accessed +# when the reader's pid % 3 is 1 +while true; do + echo 3 > /proc/sys/vm/drop_caches + $XFS_IO_PROG -c "pread -b 128K 0 128K" "$SCRATCH_MNT/foobar" > /dev/null & + pid=$! + wait + if [ $((pid % 3)) == 1 ]; then + break + fi +done +while true; do + echo 3 > /proc/sys/vm/drop_caches + $XFS_IO_PROG -c "pread -b 128K 0 128K" "$SCRATCH_MNT/foobar" > /dev/null & + pid=$! + wait + if [ $((pid % 3)) == 2 ]; then + break + fi +done +while true; do + echo 3 > /proc/sys/vm/drop_caches + $XFS_IO_PROG -c "pread -b 128K 0 128K" "$SCRATCH_MNT/foobar" > /dev/null & + pid=$! + wait + if [ $((pid % 3)) == 0 ]; then + break + fi +done + +_scratch_unmount + +echo "step 4......check if the repair works" +$XFS_IO_PROG -d -c "pread -v -b 512 $physical1 512" $devpath1 |\ + _filter_xfs_io_offset +$XFS_IO_PROG -d -c "pread -v -b 512 $physical2 512" $devpath2 |\ + _filter_xfs_io_offset +$XFS_IO_PROG -d -c "pread -v -b 512 $physical3 512" $devpath3 |\ + _filter_xfs_io_offset + +_scratch_dev_pool_put +# success, all done +status=0 +exit
tests/btrfs/266.out new file +109 -0
109 lines not on this page · raw message
Signature · 1 line
2.30.2
Re: [PATCH 2/2] btrfs: test repair with corrupted sectors interleaved over multiple mirrors
Quoted 57 lines · Christoph Hellwig
Test that repair handles the case where it needs to read from more than a single mirror on the raid1c3 profile and needs to take turns over the mirrors to recover data for the whole read.
Signed-off-by Christoph Hellwig <[email protected]>--- tests/btrfs/266 | 145 ++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/266.out | 109 +++++++++++++++++++++++++++++++++ 2 files changed, 254 insertions(+)create mode 100755 tests/btrfs/266
create mode 100644 tests/btrfs/266.outtests/btrfs/266 new file +39
-0@@ -0,0 +1,145 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2017 Liu Bo. All Rights Reserved. +# Copyright (c) 2022 Christoph Hellwig. +# +# FS QA Test 266 +# +# Test that btrfs raid repair on a raid1c3 profile can repair interleaving +# errors on all mirrors. +# + +. ./common/preamble +_begin_fstest auto quick read_repair + +# Import common functions. +. ./common/filter + +# real QA test starts here + +_supported_fs btrfs +_require_scratch_dev_pool 3 + +BTRFS_MAP_LOGICAL_PROG=$(type -P btrfs-map-logical) + +_require_command "$BTRFS_MAP_LOGICAL_PROG" btrfs-map-logical +_require_command "$FILEFRAG_PROG" filefrag +_require_odirect +# Overwriting data is forbidden on a zoned block device +_require_non_zoned_device "${SCRATCH_DEV}" + +get_physical() +{ + local logical=$1 + local stripe=$2 + + $BTRFS_MAP_LOGICAL_PROG -l $logical $SCRATCH_DEV >> $seqres.full 2>&1 + $BTRFS_MAP_LOGICAL_PROG -l $logical $SCRATCH_DEV | \ + $AWK_PROG "(\$1 ~ /mirror/ && \$2 ~ /$stripe/) { print \$6 }" +}
tests/btrfs/266 line 49
Quoted 10 lines
tests/btrfs/266 new file +10
-0+ +get_device_path() +{ + local logical=$1 + local stripe=$2 + + $BTRFS_MAP_LOGICAL_PROG -l $logical $SCRATCH_DEV | \ + $AWK_PROG "(\$1 ~ /mirror/ && \$2 ~ /$stripe/) { print \$8 }" +} +
Can wrap into a helper in common/btrfs.
Quoted 39 lines
tests/btrfs/266 new file +36
-0+_scratch_dev_pool_get 3 +# step 1, create a raid1 btrfs which contains one 128k file. +echo "step 1......mkfs.btrfs" + +mkfs_opts="-d raid1c3 -b 1G" +_scratch_pool_mkfs $mkfs_opts >>$seqres.full 2>&1 + +# make sure data is written to the start position of the data chunk +_scratch_mount $(_btrfs_no_v1_cache_opt) + +$XFS_IO_PROG -f -d -c "pwrite -S 0xaa -b 128K 0 128K" \ + "$SCRATCH_MNT/foobar" | \ + _filter_xfs_io_offset + +# ensure btrfs-map-logical sees the tree updates +sync + +# step 2, corrupt 4k in each copy +echo "step 2......corrupt file extent" + +${FILEFRAG_PROG} -v $SCRATCH_MNT/foobar >> $seqres.full +logical=`${FILEFRAG_PROG} -v $SCRATCH_MNT/foobar | _filter_filefrag | cut -d '#' -f 1` + +physical1=$(get_physical ${logical} 1) +devpath1=$(get_device_path ${logical} 1) + +physical2=$(get_physical ${logical} 2) +devpath2=$(get_device_path ${logical} 2) + +physical3=$(get_physical ${logical} 3) +devpath3=$(get_device_path ${logical} 3) + +_scratch_unmount + +$XFS_IO_PROG -d -c "pwrite -S 0xbd -b 4K $physical3 4K" $devpath3 \ + > /dev/null
tests/btrfs/266 line 95
Quoted 10 lines
tests/btrfs/266 new file +10
-0+ +$XFS_IO_PROG -d -c "pwrite -S 0xba -b 4K $physical1 8K" $devpath1 \ + > /dev/null + +$XFS_IO_PROG -d -c "pwrite -S 0xbb -b 4K $((physical2 + 4096)) 8K" $devpath2 \ + > /dev/null + +$XFS_IO_PROG -d -c "pwrite -S 0xbc -b 4K $((physical3 + (2 * 4096))) 8K" \ + $devpath3 > /dev/null +
Could you please make this test case compatible with the 64K sectorsize configs too?
Quoted 26 lines
tests/btrfs/266 new file +26
-0+_scratch_mount + +# step 3, 128k dio read (this read can repair bad copy) +echo "step 3......repair the bad copy" + +# since raid1c3 consists of three copies, and the bad copy was put on stripe #1 +# while the good copy lies the other stripes, the bad copy only gets accessed +# when the reader's pid % 3 is 1 +while true; do + echo 3 > /proc/sys/vm/drop_caches + $XFS_IO_PROG -c "pread -b 128K 0 128K" "$SCRATCH_MNT/foobar" > /dev/null & + pid=$! + wait + if [ $((pid % 3)) == 1 ]; then + break + fi +done +while true; do + echo 3 > /proc/sys/vm/drop_caches + $XFS_IO_PROG -c "pread -b 128K 0 128K" "$SCRATCH_MNT/foobar" > /dev/null & + pid=$! + wait + if [ $((pid % 3)) == 2 ]; then + break + fi +done
tests/btrfs/266 line 131
Quoted 10 lines
tests/btrfs/266 new file +10
-0+while true; do + echo 3 > /proc/sys/vm/drop_caches + $XFS_IO_PROG -c "pread -b 128K 0 128K" "$SCRATCH_MNT/foobar" > /dev/null & + pid=$! + wait + if [ $((pid % 3)) == 0 ]; then + break + fi +done +
Same here. They can wrap into a helper.
Thanks, Anand
Quoted 129 lines
tests/btrfs/266 new file +14
-0+_scratch_unmount + +echo "step 4......check if the repair works" +$XFS_IO_PROG -d -c "pread -v -b 512 $physical1 512" $devpath1 |\ + _filter_xfs_io_offset +$XFS_IO_PROG -d -c "pread -v -b 512 $physical2 512" $devpath2 |\ + _filter_xfs_io_offset +$XFS_IO_PROG -d -c "pread -v -b 512 $physical3 512" $devpath3 |\ + _filter_xfs_io_offset + +_scratch_dev_pool_put +# success, all done +status=0 +exittests/btrfs/266.out new file +109
-0@@ -0,0 +1,109 @@ +QA output created by 266 +step 1......mkfs.btrfs +wrote 131072/131072 bytes +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +step 2......corrupt file extent +step 3......repair the bad copy +step 4......check if the repair works +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +read 512/512 bytes +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +read 512/512 bytes +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +read 512/512 bytes +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
Re: [PATCH 2/2] btrfs: test repair with corrupted sectors interleaved over multiple mirrors
Quoted 167 lines · Anand Jain
Quoted 59 lines · Christoph Hellwig
Test that repair handles the case where it needs to read from more than a single mirror on the raid1c3 profile and needs to take turns over the mirrors to recover data for the whole read.
Signed-off-by Christoph Hellwig <[email protected]>---
tests/btrfs/266 | 145 ++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/266.out | 109 +++++++++++++++++++++++++++++++++2 files changed, 254 insertions(+)create mode 100755 tests/btrfs/266
create mode 100644 tests/btrfs/266.outtests/btrfs/266 new file +8
-0@@ -0,0 +1,145 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2017 Liu Bo. All Rights Reserved. +# Copyright (c) 2022 Christoph Hellwig. +# +# FS QA Test 266 +# +# Test that btrfs raid repair on a raid1c3 profile can repairinterleaving
tests/btrfs/266 new file +28
-0+# errors on all mirrors. +# + +. ./common/preamble +_begin_fstest auto quick read_repair + +# Import common functions. +. ./common/filter + +# real QA test starts here + +_supported_fs btrfs +_require_scratch_dev_pool 3 + +BTRFS_MAP_LOGICAL_PROG=$(type -P btrfs-map-logical) + +_require_command "$BTRFS_MAP_LOGICAL_PROG" btrfs-map-logical +_require_command "$FILEFRAG_PROG" filefrag +_require_odirect +# Overwriting data is forbidden on a zoned block device +_require_non_zoned_device "${SCRATCH_DEV}" + +get_physical() +{ + local logical=$1 + local stripe=$2 + + $BTRFS_MAP_LOGICAL_PROG -l $logical $SCRATCH_DEV >> $seqres.full2>&1
+ $BTRFS_MAP_LOGICAL_PROG -l $logical $SCRATCH_DEV | \
+ $AWK_PROG "(\$1 ~ /mirror/ && \$2 ~ /$stripe/) { print \$6 }"tests/btrfs/266 new file +1
-0+}tests/btrfs/266 line 49
Quoted 10 lines
tests/btrfs/266 new file +10
-0+ +get_device_path() +{ + local logical=$1 + local stripe=$2 + + $BTRFS_MAP_LOGICAL_PROG -l $logical $SCRATCH_DEV | \ + $AWK_PROG "(\$1 ~ /mirror/ && \$2 ~ /$stripe/) { print \$8 }" +} +Can wrap into a helper in common/btrfs.
Quoted 45 lines
tests/btrfs/266 new file +22
-0+_scratch_dev_pool_get 3 +# step 1, create a raid1 btrfs which contains one 128k file. +echo "step 1......mkfs.btrfs" + +mkfs_opts="-d raid1c3 -b 1G" +_scratch_pool_mkfs $mkfs_opts >>$seqres.full 2>&1 + +# make sure data is written to the start position of the data chunk +_scratch_mount $(_btrfs_no_v1_cache_opt) + +$XFS_IO_PROG -f -d -c "pwrite -S 0xaa -b 128K 0 128K" \ + "$SCRATCH_MNT/foobar" | \ + _filter_xfs_io_offset + +# ensure btrfs-map-logical sees the tree updates +sync + +# step 2, corrupt 4k in each copy +echo "step 2......corrupt file extent" + +${FILEFRAG_PROG} -v $SCRATCH_MNT/foobar >> $seqres.full +logical=`${FILEFRAG_PROG} -v $SCRATCH_MNT/foobar | _filter_filefrag |cut -d '#' -f 1`
tests/btrfs/266 new file +19
-0+ +physical1=$(get_physical ${logical} 1) +devpath1=$(get_device_path ${logical} 1) + +physical2=$(get_physical ${logical} 2) +devpath2=$(get_device_path ${logical} 2) + +physical3=$(get_physical ${logical} 3) +devpath3=$(get_device_path ${logical} 3) + +_scratch_unmount + +$XFS_IO_PROG -d -c "pwrite -S 0xbd -b 4K $physical3 4K" $devpath3 \ + > /dev/null + +$XFS_IO_PROG -d -c "pwrite -S 0xba -b 4K $physical1 8K" $devpath1 \ + > /dev/null + +$XFS_IO_PROG -d -c "pwrite -S 0xbb -b 4K $((physical2 + 4096)) 8K"Quoted 7 lines
$devpath2 \
+ >/dev/null
+
+$XFS_IO_PROG -d -c "pwrite -S0xbc-b 4K $((physical3 + (2 * 4096))) 8K" \
+ $devpath3 >/dev/null
+Could you please make this test case compatible with the 64K sectorsize configs too?
Quoted 35 lines
tests/btrfs/266 new file +6
-0+_scratch_mount + +# step 3, 128k dio read (this read can repair bad copy) +echo "step 3......repair the bad copy" + +# since raid1c3 consists of three copies, and the bad copy was put onstripe #1
+# while the good copy lies the other stripes, the bad copy only gets accessedtests/btrfs/266 new file +4
-0+# when the reader's pid % 3 is 1 +while true; do + echo 3 > /proc/sys/vm/drop_caches + $XFS_IO_PROG -c "pread -b 128K 0 128K" "$SCRATCH_MNT/foobar" >
/dev/null&
+ pid=$!
+ wait
+ if [ $((pid % 3)) == 1 ]; then
+ break
+ fitests/btrfs/266 new file +4
-0+done +while true; do + echo 3 > /proc/sys/vm/drop_caches + $XFS_IO_PROG -c "pread -b 128K 0 128K" "$SCRATCH_MNT/foobar" >
/dev/null&
+ pid=$!
+ wait
+ if [ $((pid % 3)) == 2 ]; then
+ break
+ fitests/btrfs/266 new file +4
-0+done +while true; do + echo 3 > /proc/sys/vm/drop_caches + $XFS_IO_PROG -c "pread -b 128K 0 128K" "$SCRATCH_MNT/foobar" >
/dev/null&
+ pid=$!
Quoted 8 lines
tests/btrfs/266
Quoted 6 lines
+ wait
+ if [ $((pid % 3)) == 0 ]; then
+ break
+ fitests/btrfs/266 new file +2
-0+done +Same here. They can wrap into a helper.
In fact, we should recheck the way we handle the read on specific mirror behavior.
For a lot of runs, btrfs/14[01] can take super long time (over 100s) to really trigger the data read on expected mirror.
For a lot of runs, the read just got the same pid again and again.
Can't we at least add some noise at background to make the pid more random?
Thanks,
Qu
Quoted 229 lines
Thanks, Anand
Quoted 225 lines
tests/btrfs/266 new file +14
-0+_scratch_unmount + +echo "step 4......check if the repair works" +$XFS_IO_PROG -d -c "pread -v -b 512 $physical1 512" $devpath1 |\ + _filter_xfs_io_offset +$XFS_IO_PROG -d -c "pread -v -b 512 $physical2 512" $devpath2 |\ + _filter_xfs_io_offset +$XFS_IO_PROG -d -c "pread -v -b 512 $physical3 512" $devpath3 |\ + _filter_xfs_io_offset + +_scratch_dev_pool_put +# success, all done +status=0 +exittests/btrfs/266.out new file +109
-0@@ -0,0 +1,109 @@ +QA output created by 266 +step 1......mkfs.btrfs +wrote 131072/131072 bytes +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +step 2......corrupt file extent +step 3......repair the bad copy +step 4......check if the repair works +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +read 512/512 bytes +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +read 512/512 bytes +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +read 512/512 bytes +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
Re: [PATCH 2/2] btrfs: test repair with corrupted sectors interleaved over multiple mirrors
Quoted 182 lines · Qu Wenruo
Quoted 167 lines · Anand Jain
Quoted 59 lines · Christoph Hellwig
Test that repair handles the case where it needs to read from more than a single mirror on the raid1c3 profile and needs to take turns over the mirrors to recover data for the whole read.
Signed-off-by Christoph Hellwig <[email protected]>---
tests/btrfs/266 | 145 ++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/266.out | 109 +++++++++++++++++++++++++++++++++2 files changed, 254 insertions(+)create mode 100755 tests/btrfs/266
create mode 100644 tests/btrfs/266.outtests/btrfs/266 new file +8
-0@@ -0,0 +1,145 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2017 Liu Bo. All Rights Reserved. +# Copyright (c) 2022 Christoph Hellwig. +# +# FS QA Test 266 +# +# Test that btrfs raid repair on a raid1c3 profile can repairinterleaving
tests/btrfs/266 new file +28
-0+# errors on all mirrors. +# + +. ./common/preamble +_begin_fstest auto quick read_repair + +# Import common functions. +. ./common/filter + +# real QA test starts here + +_supported_fs btrfs +_require_scratch_dev_pool 3 + +BTRFS_MAP_LOGICAL_PROG=$(type -P btrfs-map-logical) + +_require_command "$BTRFS_MAP_LOGICAL_PROG" btrfs-map-logical +_require_command "$FILEFRAG_PROG" filefrag +_require_odirect +# Overwriting data is forbidden on a zoned block device +_require_non_zoned_device "${SCRATCH_DEV}" + +get_physical() +{ + local logical=$1 + local stripe=$2 + + $BTRFS_MAP_LOGICAL_PROG -l $logical $SCRATCH_DEV >> $seqres.full2>&1
+ $BTRFS_MAP_LOGICAL_PROG -l $logical $SCRATCH_DEV | \
+ $AWK_PROG "(\$1 ~ /mirror/ && \$2 ~ /$stripe/) { print \$6 }"tests/btrfs/266 new file +1
-0+}tests/btrfs/266 line 49
Quoted 10 lines
tests/btrfs/266 new file +10
-0+ +get_device_path() +{ + local logical=$1 + local stripe=$2 + + $BTRFS_MAP_LOGICAL_PROG -l $logical $SCRATCH_DEV | \ + $AWK_PROG "(\$1 ~ /mirror/ && \$2 ~ /$stripe/) { print \$8 }" +} +Can wrap into a helper in common/btrfs.
Quoted 45 lines
tests/btrfs/266 new file +22
-0+_scratch_dev_pool_get 3 +# step 1, create a raid1 btrfs which contains one 128k file. +echo "step 1......mkfs.btrfs" + +mkfs_opts="-d raid1c3 -b 1G" +_scratch_pool_mkfs $mkfs_opts >>$seqres.full 2>&1 + +# make sure data is written to the start position of the data chunk +_scratch_mount $(_btrfs_no_v1_cache_opt) + +$XFS_IO_PROG -f -d -c "pwrite -S 0xaa -b 128K 0 128K" \ + "$SCRATCH_MNT/foobar" | \ + _filter_xfs_io_offset + +# ensure btrfs-map-logical sees the tree updates +sync + +# step 2, corrupt 4k in each copy +echo "step 2......corrupt file extent" + +${FILEFRAG_PROG} -v $SCRATCH_MNT/foobar >> $seqres.full +logical=`${FILEFRAG_PROG} -v $SCRATCH_MNT/foobar | _filter_filefrag |cut -d '#' -f 1`
tests/btrfs/266 new file +19
-0+ +physical1=$(get_physical ${logical} 1) +devpath1=$(get_device_path ${logical} 1) + +physical2=$(get_physical ${logical} 2) +devpath2=$(get_device_path ${logical} 2) + +physical3=$(get_physical ${logical} 3) +devpath3=$(get_device_path ${logical} 3) + +_scratch_unmount + +$XFS_IO_PROG -d -c "pwrite -S 0xbd -b 4K $physical3 4K" $devpath3 \ + > /dev/null + +$XFS_IO_PROG -d -c "pwrite -S 0xba -b 4K $physical1 8K" $devpath1 \ + > /dev/null + +$XFS_IO_PROG -d -c "pwrite -S 0xbb -b 4K $((physical2 + 4096)) 8K"Quoted 7 lines
$devpath2 \
+ >/dev/null
+
+$XFS_IO_PROG -d -c "pwrite -S0xbc-b 4K $((physical3 + (2 * 4096))) 8K" \
+ $devpath3 >/dev/null
+Could you please make this test case compatible with the 64K sectorsize configs too?
Quoted 35 lines
tests/btrfs/266 new file +6
-0+_scratch_mount + +# step 3, 128k dio read (this read can repair bad copy) +echo "step 3......repair the bad copy" + +# since raid1c3 consists of three copies, and the bad copy was put onstripe #1
+# while the good copy lies the other stripes, the bad copy only gets accessedtests/btrfs/266 new file +4
-0+# when the reader's pid % 3 is 1 +while true; do + echo 3 > /proc/sys/vm/drop_caches + $XFS_IO_PROG -c "pread -b 128K 0 128K" "$SCRATCH_MNT/foobar" >
/dev/null&
+ pid=$!
+ wait
+ if [ $((pid % 3)) == 1 ]; then
+ break
+ fitests/btrfs/266 new file +4
-0+done +while true; do + echo 3 > /proc/sys/vm/drop_caches + $XFS_IO_PROG -c "pread -b 128K 0 128K" "$SCRATCH_MNT/foobar" >
/dev/null&
+ pid=$!
+ wait
+ if [ $((pid % 3)) == 2 ]; then
+ break
+ fitests/btrfs/266 new file +4
-0+done +while true; do + echo 3 > /proc/sys/vm/drop_caches + $XFS_IO_PROG -c "pread -b 128K 0 128K" "$SCRATCH_MNT/foobar" >
/dev/null&
+ pid=$!Quoted 8 lines
tests/btrfs/266
Quoted 6 lines
+ wait
+ if [ $((pid % 3)) == 0 ]; then
+ break
+ fitests/btrfs/266 new file +2
-0+done +Same here. They can wrap into a helper.
In fact, we should recheck the way we handle the read on specific mirror behavior.
Quoted 7 lines
For a lot of runs, btrfs/14[01] can take super long time (over 100s) to really trigger the data read on expected mirror.
For a lot of runs, the read just got the same pid again and again.
Can't we at least add some noise at background to make the pid more random?
Agreed this method is unreliable. But there is no other choice. Unless we integrated [1] patch in the ML.
[1] btrfs: introduce new read_policy device
[1] is more reliable. You can set which mirrored device to read.
Thanks, Anand
Quoted 231 lines
Thanks,
QuQuoted 229 lines
Thanks, Anand
Quoted 225 lines
tests/btrfs/266 new file +14
-0+_scratch_unmount + +echo "step 4......check if the repair works" +$XFS_IO_PROG -d -c "pread -v -b 512 $physical1 512" $devpath1 |\ + _filter_xfs_io_offset +$XFS_IO_PROG -d -c "pread -v -b 512 $physical2 512" $devpath2 |\ + _filter_xfs_io_offset +$XFS_IO_PROG -d -c "pread -v -b 512 $physical3 512" $devpath3 |\ + _filter_xfs_io_offset + +_scratch_dev_pool_put +# success, all done +status=0 +exittests/btrfs/266.out new file +109
-0@@ -0,0 +1,109 @@ +QA output created by 266 +step 1......mkfs.btrfs +wrote 131072/131072 bytes +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +step 2......corrupt file extent +step 3......repair the bad copy +step 4......check if the repair works +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +read 512/512 bytes +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +read 512/512 bytes +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +read 512/512 bytes +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
Re: [PATCH 2/2] btrfs: test repair with corrupted sectors interleaved over multiple mirrors
Quoted 6 lines · Anand Jain
Agreed this method is unreliable. But there is no other choice. Unless we integrated [1] patch in the ML.
[1] btrfs: introduce new
read_policydevice[1] is more reliable. You can set which mirrored device to read.
I actually thought about something like this as the reliable way to hit a mirror. Besides testing I could also thing of some other use cases for it where you want to avoid a mirror by default for some reason (e.g. because it is network attached store while the other one is local).