fstests: btrfs/022 fixes
cover fstests: btrfs/022 fixes
Since now fsstress can create new subvolumes, it causes new problem for btrfs/022.
First two patches will fix the problem by:
- Enhance _btrfs_get_subvolid()
So it can catch correct subvolid
- Use proper qgroupid for grep
This solves the random failure even after _btrfs_get_subvolid enhancement.
The last patch will add extra debug output for btrfs/022 to make our lives easier.
Qu Wenruo (3):
fstests: btrfs: Use word mathcing for _btrfs_get_subvolid()
fstests: btrfs/022: Match qgroup id more correctly
fstests: btrfs/022: Add debug output
common/btrfs | 2 +- tests/btrfs/022 | 19 +++++++++++++------ 2 files changed, 14 insertions(+), 7 deletions(-)
Signature · 1 line
2.23.0
PATCH 1/3 fstests: btrfs: Use word mathcing for _btrfs_get_subvolid()
Current _btrfs_get_subvolid() can't handle the following case at all:
# btrfs subvol list $SCRATCH_MNT ID 256 gen 9 top level 5 path subv1 ID 257 gen 7 top level 256 path subv1/subv2 ID 258 gen 8 top level 256 path subv1/subv3 ID 259 gen 9 top level 256 path subv1/subv4
If we call "_btrfs_get_subvolid $SCRATCH_MNT subv1" we will get a list of all subvolumes, not the subvolid of subv1.
To address this problem, we go egrep to match $name which starts with a space, and at the end of a line.
So that all other subvolumes won't hit.
--- common/btrfs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
common/btrfs +1 -1
@@ -7,7 +7,7 @@ _btrfs_get_subvolid() mnt=$1 name=$2 - $BTRFS_UTIL_PROG sub list $mnt | grep $name | awk '{ print $2 }' + $BTRFS_UTIL_PROG sub list $mnt | egrep "\s$name$" | awk '{ print $2 }' } # _require_btrfs_command <command> [<subcommand>|<option>]
Signature · 1 line
2.23.0
Re: [PATCH 1/3] fstests: btrfs: Use word mathcing for _btrfs_get_subvolid()
Quoted 10 lines · Qu Wenruo
Current
_btrfs_get_subvolid()can't handle the following case at all:# btrfs subvol list $SCRATCH_MNT ID 256 gen 9 top level 5 path subv1 ID 257 gen 7 top level 256 path subv1/subv2 ID 258 gen 8 top level 256 path subv1/subv3 ID 259 gen 9 top level 256 path subv1/subv4If we call "
_btrfs_get_subvolid$SCRATCH_MNT subv1" we will get a list of all subvolumes, not the subvolid of subv1.
Quoted 6 lines
To address this problem, we go egrep to match $name which starts with a space, and at the end of a line.
So that all other subvolumes won't hit.Suggested-by Josef Bacik <[email protected]>Signed-off-by Qu Wenruo <[email protected]>
Thanks,
Josef
Re: [PATCH 1/3] fstests: btrfs: Use word mathcing for _btrfs_get_subvolid()
In reply to Qu WenruoQuoted 10 lines · Qu Wenruo
Current
_btrfs_get_subvolid()can't handle the following case at all:# btrfs subvol list $SCRATCH_MNT ID 256 gen 9 top level 5 path subv1 ID 257 gen 7 top level 256 path subv1/subv2 ID 258 gen 8 top level 256 path subv1/subv3 ID 259 gen 9 top level 256 path subv1/subv4If we call "
_btrfs_get_subvolid$SCRATCH_MNT subv1" we will get a list of all subvolumes, not the subvolid of subv1.
Quoted 6 lines
To address this problem, we go egrep to match $name which starts with a space, and at the end of a line.
So that all other subvolumes won't hit.Suggested-by Josef Bacik <[email protected]>Signed-off-by Qu Wenruo <[email protected]>
Quoted 9 lines
--- common/btrfs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)common/btrfs +0
-0
common/btrfs _btrfs_get_subvolid() · line 10
Quoted 5 lines
common/btrfs +1
-1@@ -7,7 +7,7 @@ _btrfs_get_subvolid() mnt=$1 name=$2 - $BTRFS_UTIL_PROG sub list $mnt | grep $name | awk '{ print $2 }' + $BTRFS_UTIL_PROG sub list $mnt | egrep "\s$name$" | awk '{ print $2 }'
nit: But you don't even need egrep for this, you could have simply used "grep $name$"
Quoted 4 lines
common/btrfs +0
-0} # _require_btrfs_command <command> [<subcommand>|<option>]
Re: [PATCH 1/3] fstests: btrfs: Use word mathcing for _btrfs_get_subvolid()
Quoted 34 lines · Nikolay Borisov
Quoted 10 lines · Qu Wenruo
Current
_btrfs_get_subvolid()can't handle the following case at all:# btrfs subvol list $SCRATCH_MNT ID 256 gen 9 top level 5 path subv1 ID 257 gen 7 top level 256 path subv1/subv2 ID 258 gen 8 top level 256 path subv1/subv3 ID 259 gen 9 top level 256 path subv1/subv4If we call "
_btrfs_get_subvolid$SCRATCH_MNT subv1" we will get a list of all subvolumes, not the subvolid of subv1.Quoted 6 lines
To address this problem, we go egrep to match $name which starts with a space, and at the end of a line.
So that all other subvolumes won't hit.Suggested-by Josef Bacik <[email protected]>Signed-off-by Qu Wenruo <[email protected]>Reviewed-by Nikolay Borisov <[email protected]>Quoted 12 lines
--- common/btrfs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)common/btrfs +0
-0@@ -7,7 +7,7 @@ _btrfs_get_subvolid() mnt=$1 name=$2
Quoted 5 lines
common/btrfs _btrfs_get_subvolid() · line 10
Quoted 2 lines
common/btrfs +1
-1- $BTRFS_UTIL_PROG sub list $mnt | grep $name | awk '{ print $2 }' + $BTRFS_UTIL_PROG sub list $mnt | egrep "\s$name$" | awk '{ print $2 }'nit: But you don't even need egrep for this, you could have simply used "grep $name$"
That \s is needed. Or the following case can't be handled:
ID 256 gen 9 top level 5 path subv1
ID 257 gen 7 top level 256 path subv1/subv1
Thanks,
Qu
Quoted 5 lines
Quoted 5 lines
common/btrfs +0
-0} # _require_btrfs_command <command> [<subcommand>|<option>]
Re: [PATCH 1/3] fstests: btrfs: Use word mathcing for _btrfs_get_subvolid()
Quoted 43 lines · Qu Wenruo
Quoted 34 lines · Nikolay Borisov
Quoted 10 lines · Qu Wenruo
Current
_btrfs_get_subvolid()can't handle the following case at all:# btrfs subvol list $SCRATCH_MNT ID 256 gen 9 top level 5 path subv1 ID 257 gen 7 top level 256 path subv1/subv2 ID 258 gen 8 top level 256 path subv1/subv3 ID 259 gen 9 top level 256 path subv1/subv4If we call "
_btrfs_get_subvolid$SCRATCH_MNT subv1" we will get a list of all subvolumes, not the subvolid of subv1.Quoted 6 lines
To address this problem, we go egrep to match $name which starts with a space, and at the end of a line.
So that all other subvolumes won't hit.Suggested-by Josef Bacik <[email protected]>Signed-off-by Qu Wenruo <[email protected]>Reviewed-by Nikolay Borisov <[email protected]>Quoted 12 lines
--- common/btrfs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)common/btrfs +0
-0@@ -7,7 +7,7 @@ _btrfs_get_subvolid() mnt=$1 name=$2Quoted 5 lines
common/btrfs _btrfs_get_subvolid() · line 10
Quoted 2 lines
common/btrfs +1
-1- $BTRFS_UTIL_PROG sub list $mnt | grep $name | awk '{ print $2 }' + $BTRFS_UTIL_PROG sub list $mnt | egrep "\s$name$" | awk '{ print $2 }'nit: But you don't even need egrep for this, you could have simply used "grep $name$"
Quoted 4 lines
That \s is needed. Or the following case can't be handled:
ID 256 gen 9 top level 5 path subv1
ID 257 gen 7 top level 256 path subv1/subv1
Good point.
Quoted 9 lines
Thanks,
QuQuoted 5 lines
Quoted 5 lines
common/btrfs +0
-0} # _require_btrfs_command <command> [<subcommand>|<option>]
PATCH 2/3 fstests: btrfs/022: Match qgroup id more correctly
In reply to Qu Wenruo[BUG]
Btrfs/022 sometimes fails with snapshot's reference mismatch with its source.
[CAUSE]
Since commit fd0830929573 ("fsstress: add the ability to create snapshots") adds the ability for fsstress to create/delete snapshot and subvolumes, fsstress will create new subvolumes under test dir.
For example, we could have the following subvolumes created by fsstress:
subvol a id=256
subvol b id=306
qgroupid rfer excl -------- ---- ----
0/5 16384 16384
0/256 13914112 16384
...
0/263 3080192 2306048 << 2 *306* 048
...
0/306 13914112 16384 << 0/ *306
So when we're greping for subvolid 306, it matches qgroup 0/263 first, which has difference size, and caused false alert.
[FIX]
Instead of greping "$subvolid" blindly, now grep "0/$subvolid" to catch qgroupid correctly, without hitting rfer/excl values.
--- tests/btrfs/022 | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
tests/btrfs/022 +4 -4
@@ -49,10 +49,10 @@ _basic_test() # the shared values of both the original subvol and snapshot should # match - a_shared=$($BTRFS_UTIL_PROG qgroup show $units $SCRATCH_MNT | grep $subvolid) + a_shared=$($BTRFS_UTIL_PROG qgroup show $units $SCRATCH_MNT | grep "0/$subvolid") a_shared=$(echo $a_shared | awk '{ print $2 }') subvolid=$(_btrfs_get_subvolid $SCRATCH_MNT b) - b_shared=$($BTRFS_UTIL_PROG qgroup show $units $SCRATCH_MNT | grep $subvolid) + b_shared=$($BTRFS_UTIL_PROG qgroup show $units $SCRATCH_MNT | grep "0/$subvolid") b_shared=$(echo $b_shared | awk '{ print $2 }') [ $b_shared -eq $a_shared ] || _fail "shared values don't match" }
@@ -68,12 +68,12 @@ _rescan_test() run_check $FSSTRESS_PROG -d $SCRATCH_MNT/a -w -p 1 -n 2000 \ $FSSTRESS_AVOID sync - output=$($BTRFS_UTIL_PROG qgroup show $units $SCRATCH_MNT | grep $subvolid) + output=$($BTRFS_UTIL_PROG qgroup show $units $SCRATCH_MNT | grep "0/$subvolid") echo $output >> $seqres.full refer=$(echo $output | awk '{ print $2 }') excl=$(echo $output | awk '{ print $3 }') _run_btrfs_util_prog quota rescan -w $SCRATCH_MNT - output=$($BTRFS_UTIL_PROG qgroup show $units $SCRATCH_MNT | grep $subvolid) + output=$($BTRFS_UTIL_PROG qgroup show $units $SCRATCH_MNT | grep "0/$subvolid") echo $output >> $seqres.full [ $refer -eq $(echo $output | awk '{ print $2 }') ] || \ _fail "reference values don't match after rescan"
Signature · 1 line
2.23.0
Re: [PATCH 2/3] fstests: btrfs/022: Match qgroup id more correctly
Quoted 24 lines · Qu Wenruo
[BUG]
Btrfs/022 sometimes fails with snapshot's reference mismatch with its source.[CAUSE]
Since commitfd0830929573("fsstress: add the ability to create snapshots") adds the ability for fsstress to create/delete snapshot and subvolumes, fsstress will create new subvolumes under test dir.For example, we could have the following subvolumes created by fsstress:
subvol a id=256
subvol b id=306qgroupid rfer excl -------- ---- ----0/5 16384 16384
0/256 13914112 16384
...
0/263 3080192 2306048 << 2 *306* 048
...
0/306 13914112 16384 << 0/ *306So when we're greping for subvolid 306, it matches qgroup 0/263 first, which has difference size, and caused false alert.
Quoted 6 lines
[FIX]
Instead of greping "$subvolid" blindly, now grep "0/$subvolid" to catch qgroupid correctly, without hitting rfer/excl values.Suggested-by Josef Bacik <[email protected]>Signed-off-by Qu Wenruo <[email protected]>
Thanks,
Josef
Re: [PATCH 2/3] fstests: btrfs/022: Match qgroup id more correctly
In reply to Qu WenruoQuoted 21 lines · Qu Wenruo
[BUG]
Btrfs/022 sometimes fails with snapshot's reference mismatch with its source.[CAUSE]
Since commitfd0830929573("fsstress: add the ability to create snapshots") adds the ability for fsstress to create/delete snapshot and subvolumes, fsstress will create new subvolumes under test dir.For example, we could have the following subvolumes created by fsstress:
subvol a id=256
subvol b id=306qgroupid rfer excl -------- ---- ----0/5 16384 16384
0/256 13914112 16384
...
0/263 3080192 2306048 << 2 *306* 048
...
0/306 13914112 16384 << 0/ *306
Quoted 6 lines
So when we're greping for subvolid 306, it matches qgroup 0/263 first, which has difference size, and caused false alert.
[FIX]
Instead of greping "$subvolid" blindly, now grep "0/$subvolid" to catch qgroupid correctly, without hitting rfer/excl values.
That 0/ can it ever be a number different than 0, if so a more correct regular expression should be:
grep "[[:digit:]]/306" ?
Quoted 40 lines
Suggested-by Josef Bacik <[email protected]>Signed-off-by Qu Wenruo <[email protected]>--- tests/btrfs/022 | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)tests/btrfs/022 +4
-4@@ -49,10 +49,10 @@ _basic_test() # the shared values of both the original subvol and snapshot should # match - a_shared=$($BTRFS_UTIL_PROG qgroup show $units $SCRATCH_MNT | grep $subvolid) + a_shared=$($BTRFS_UTIL_PROG qgroup show $units $SCRATCH_MNT | grep "0/$subvolid") a_shared=$(echo $a_shared | awk '{ print $2 }') subvolid=$(_btrfs_get_subvolid $SCRATCH_MNT b) - b_shared=$($BTRFS_UTIL_PROG qgroup show $units $SCRATCH_MNT | grep $subvolid) + b_shared=$($BTRFS_UTIL_PROG qgroup show $units $SCRATCH_MNT | grep "0/$subvolid") b_shared=$(echo $b_shared | awk '{ print $2 }') [ $b_shared -eq $a_shared ] || _fail "shared values don't match" }@@ -68,12 +68,12 @@ _rescan_test() run_check $FSSTRESS_PROG -d $SCRATCH_MNT/a -w -p 1 -n 2000 \ $FSSTRESS_AVOID sync - output=$($BTRFS_UTIL_PROG qgroup show $units $SCRATCH_MNT | grep $subvolid) + output=$($BTRFS_UTIL_PROG qgroup show $units $SCRATCH_MNT | grep "0/$subvolid") echo $output >> $seqres.full refer=$(echo $output | awk '{ print $2 }') excl=$(echo $output | awk '{ print $3 }') _run_btrfs_util_prog quota rescan -w $SCRATCH_MNT - output=$($BTRFS_UTIL_PROG qgroup show $units $SCRATCH_MNT | grep $subvolid) + output=$($BTRFS_UTIL_PROG qgroup show $units $SCRATCH_MNT | grep "0/$subvolid") echo $output >> $seqres.full [ $refer -eq $(echo $output | awk '{ print $2 }') ] || \ _fail "reference values don't match after rescan"
Re: [PATCH 2/3] fstests: btrfs/022: Match qgroup id more correctly
Quoted 27 lines · Nikolay Borisov
Quoted 24 lines · Qu Wenruo
[BUG]
Btrfs/022 sometimes fails with snapshot's reference mismatch with its source.[CAUSE]
Since commitfd0830929573("fsstress: add the ability to create snapshots") adds the ability for fsstress to create/delete snapshot and subvolumes, fsstress will create new subvolumes under test dir.For example, we could have the following subvolumes created by fsstress:
subvol a id=256
subvol b id=306qgroupid rfer excl -------- ---- ----0/5 16384 16384
0/256 13914112 16384
...
0/263 3080192 2306048 << 2 *306* 048
...
0/306 13914112 16384 << 0/ *306So when we're greping for subvolid 306, it matches qgroup 0/263 first, which has difference size, and caused false alert.
Quoted 7 lines
Quoted 3 lines
[FIX]
Instead of greping "$subvolid" blindly, now grep "0/$subvolid" to catch qgroupid correctly, without hitting rfer/excl values.That 0/ can it ever be a number different than 0, if so a more correct regular expression should be:
grep "[[:digit:]]/306" ?
In this particular case, we only care level 0 qgroup, thus it's enough.
If we're extracting it into a generic wrapper, then your [[:digit:]]/$subvolid will be needed.
Thanks,
Qu
Quoted 42 lines
Quoted 42 lines
Suggested-by Josef Bacik <[email protected]>Signed-off-by Qu Wenruo <[email protected]>--- tests/btrfs/022 | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)tests/btrfs/022 +4
-4@@ -49,10 +49,10 @@ _basic_test() # the shared values of both the original subvol and snapshot should # match - a_shared=$($BTRFS_UTIL_PROG qgroup show $units $SCRATCH_MNT | grep $subvolid) + a_shared=$($BTRFS_UTIL_PROG qgroup show $units $SCRATCH_MNT | grep "0/$subvolid") a_shared=$(echo $a_shared | awk '{ print $2 }') subvolid=$(_btrfs_get_subvolid $SCRATCH_MNT b) - b_shared=$($BTRFS_UTIL_PROG qgroup show $units $SCRATCH_MNT | grep $subvolid) + b_shared=$($BTRFS_UTIL_PROG qgroup show $units $SCRATCH_MNT | grep "0/$subvolid") b_shared=$(echo $b_shared | awk '{ print $2 }') [ $b_shared -eq $a_shared ] || _fail "shared values don't match" }@@ -68,12 +68,12 @@ _rescan_test() run_check $FSSTRESS_PROG -d $SCRATCH_MNT/a -w -p 1 -n 2000 \ $FSSTRESS_AVOID sync - output=$($BTRFS_UTIL_PROG qgroup show $units $SCRATCH_MNT | grep $subvolid) + output=$($BTRFS_UTIL_PROG qgroup show $units $SCRATCH_MNT | grep "0/$subvolid") echo $output >> $seqres.full refer=$(echo $output | awk '{ print $2 }') excl=$(echo $output | awk '{ print $3 }') _run_btrfs_util_prog quota rescan -w $SCRATCH_MNT - output=$($BTRFS_UTIL_PROG qgroup show $units $SCRATCH_MNT | grep $subvolid) + output=$($BTRFS_UTIL_PROG qgroup show $units $SCRATCH_MNT | grep "0/$subvolid") echo $output >> $seqres.full [ $refer -eq $(echo $output | awk '{ print $2 }') ] || \ _fail "reference values don't match after rescan"
PATCH 3/3 fstests: btrfs/022: Add debug output
In reply to Qu WenruoWhen btrfs/022 fails, its $seqres.full doesn't contain much useful info to debug.
This patch will add extra debug, including subvolid and full "btrfs qgroup show" output.
--- tests/btrfs/022 | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-)
tests/btrfs/022 +9 -2
@@ -35,6 +35,7 @@ rm -f $seqres.full # Test to make sure we can actually turn it on and it makes sense _basic_test() { + echo "=== basic test ===" >> $seqres.full _run_btrfs_util_prog subvolume create $SCRATCH_MNT/a _run_btrfs_util_prog quota enable $SCRATCH_MNT/a _run_btrfs_util_prog quota rescan -w $SCRATCH_MNT
@@ -51,9 +52,12 @@ _basic_test() # match a_shared=$($BTRFS_UTIL_PROG qgroup show $units $SCRATCH_MNT | grep "0/$subvolid") a_shared=$(echo $a_shared | awk '{ print $2 }') + echo "subvol a id=$subvolid" >> $seqres.full subvolid=$(_btrfs_get_subvolid $SCRATCH_MNT b) + echo "subvol b id=$subvolid" >> $seqres.full b_shared=$($BTRFS_UTIL_PROG qgroup show $units $SCRATCH_MNT | grep "0/$subvolid") b_shared=$(echo $b_shared | awk '{ print $2 }') + $BTRFS_UTIL_PROG qgroup show $units $SCRATCH_MNT >> $seqres.full [ $b_shared -eq $a_shared ] || _fail "shared values don't match" }
@@ -61,6 +65,7 @@ _basic_test() #come up with the same answer _rescan_test() { + echo "=== rescan test ===" >> $seqres.full # first with a blank subvol _run_btrfs_util_prog subvolume create $SCRATCH_MNT/a _run_btrfs_util_prog quota enable $SCRATCH_MNT/a
@@ -69,12 +74,12 @@ _rescan_test() $FSSTRESS_AVOID sync output=$($BTRFS_UTIL_PROG qgroup show $units $SCRATCH_MNT | grep "0/$subvolid") - echo $output >> $seqres.full + echo "qgroup values before rescan: $output" >> $seqres.full refer=$(echo $output | awk '{ print $2 }') excl=$(echo $output | awk '{ print $3 }') _run_btrfs_util_prog quota rescan -w $SCRATCH_MNT output=$($BTRFS_UTIL_PROG qgroup show $units $SCRATCH_MNT | grep "0/$subvolid") - echo $output >> $seqres.full + echo "qgroup values after rescan: $output" >> $seqres.full [ $refer -eq $(echo $output | awk '{ print $2 }') ] || \ _fail "reference values don't match after rescan" [ $excl -eq $(echo $output | awk '{ print $3 }') ] || \
@@ -84,6 +89,7 @@ _rescan_test() #basic exceed limit testing _limit_test_exceed() { + echo "=== limit exceed test ===" >> $seqres.full _run_btrfs_util_prog subvolume create $SCRATCH_MNT/a _run_btrfs_util_prog quota enable $SCRATCH_MNT subvolid=$(_btrfs_get_subvolid $SCRATCH_MNT a)
@@ -95,6 +101,7 @@ _limit_test_exceed() #basic noexceed limit testing _limit_test_noexceed() { + echo "=== limit not exceed test ===" >> $seqres.full _run_btrfs_util_prog subvolume create $SCRATCH_MNT/a _run_btrfs_util_prog quota enable $SCRATCH_MNT subvolid=$(_btrfs_get_subvolid $SCRATCH_MNT a)
Signature · 1 line
2.23.0
Re: [PATCH 3/3] fstests: btrfs/022: Add debug output
Quoted 7 lines · Qu Wenruo
When btrfs/022 fails, its $seqres.full doesn't contain much useful info to debug.
This patch will add extra debug, including subvolid and full "btrfs qgroup show" output.
Signed-off-by Qu Wenruo <[email protected]>
Thanks,
Josef
Re: [PATCH 3/3] fstests: btrfs/022: Add debug output
In reply to Qu WenruoQuoted 7 lines · Qu Wenruo
When btrfs/022 fails, its $seqres.full doesn't contain much useful info to debug.
This patch will add extra debug, including subvolid and full "btrfs qgroup show" output.
Signed-off-by Qu Wenruo <[email protected]>