btrfs: fix some test failures with btrfs-progs 5.19 and 6.0
cover btrfs: fix some test failures with btrfs-progs 5.19 and 6.0
From: Filipe Manana <[email protected]>
The output of some commands changed with btrfs-progs 5.19 and 6.0, as well as the --leafsize/-l command line option was removed from mkfs. These make some tests fail. This patchset changes the tests so that they work with the new btrfs-progs versions as well as older versions.
Filipe Manana (3):
btrfs/003: fix failure on new btrfs-progs versions btrfs/053: fix test failure when running with btrfs-progs v6.0+
btrfs: fix failure of tests that use defrag on btrfs-progs v6.0+
tests/btrfs/003 | 5 +++-- tests/btrfs/021 | 4 +++- tests/btrfs/053 | 4 ++-- tests/btrfs/256 | 4 +++- 4 files changed, 11 insertions(+), 6 deletions(-)
Signature · 1 line
2.35.1
PATCH 1/3 btrfs/003: fix failure on new btrfs-progs versions
From: Filipe Manana <[email protected]>
Starting with btrfs-progs version 5.19, the output of 'filesystem show' command changed when we have a missing device. The old output was like the following:
Label: none uuid: 139ef309-021f-4b98-a3a8-ce230a83b1e2
Total devices 2 FS bytes used 128.00KiB
devid 1 size 5.00GiB used 1.26GiB path /dev/loop0
*** Some devices missing
While the new output (btrfs-progs 5.19+) is like the following:
Label: none uuid: 4a85a40b-9b79-4bde-8e52-c65a550a176b
Total devices 2 FS bytes used 128.00KiB
devid 1 size 5.00GiB used 1.26GiB path /dev/loop0
devid 2 size 0 used 0 path /dev/loop1 MISSING
More specifically it happened in the following btrfs-progs commit:
957a79c9b016 ("btrfs-progs: fi show: print missing device for a mounted file system")
This is making btrfs/003 fail with btrfs-progs 5.19+. Update the grep filter in btrfs/003 so that it works with both output formats.
--- tests/btrfs/003 | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
tests/btrfs/003 +3 -2
@@ -141,8 +141,9 @@ _test_replace() _devmgt_remove ${removed_dev_htl} $ds dev_removed=1 - $BTRFS_UTIL_PROG filesystem show $SCRATCH_DEV | grep "Some devices missing" >> $seqres.full || _fail \ - "btrfs did not report device missing" + $BTRFS_UTIL_PROG filesystem show $SCRATCH_DEV | \ + grep -ie '\bmissing\b' >> $seqres.full || \ + _fail "btrfs did not report device missing" # add a new disk to btrfs ds=${devs[@]:$(($n)):1}
Signature · 1 line
2.35.1
PATCH 2/3 btrfs/053: fix test failure when running with btrfs-progs v6.0+
In reply to [email protected]From: Filipe Manana <[email protected]>
In btrfs-progs v6.0 the --leafsize (-l) command line option was removed, so btrfs/053 always fails with v6.0+.
The change was introduced by the following btrfs-progs commit:
f7a768d62498 ("btrfs-progs: mkfs: remove support for option --leafsize")
Change the test to use --nodesize (-n) instead, since it exists in both old and new btrfs-progs versions.
--- tests/btrfs/053 | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
tests/btrfs/053 +2 -2
@@ -44,7 +44,7 @@ send_files_dir=$TEST_DIR/btrfs-test-$seq rm -fr $send_files_dir mkdir $send_files_dir -_scratch_mkfs "-l $leaf_size" >/dev/null 2>&1 +_scratch_mkfs "--nodesize $leaf_size" >/dev/null 2>&1 _scratch_mount echo "hello world" > $SCRATCH_MNT/foobar
@@ -72,7 +72,7 @@ _run_btrfs_util_prog send -p $SCRATCH_MNT/mysnap1 -f $send_files_dir/2.snap \ _scratch_unmount _check_scratch_fs -_scratch_mkfs "-l $leaf_size" >/dev/null 2>&1 +_scratch_mkfs "--nodesize $leaf_size" >/dev/null 2>&1 _scratch_mount _run_btrfs_util_prog receive -f $send_files_dir/1.snap $SCRATCH_MNT
Signature · 1 line
2.35.1
PATCH 3/3 btrfs: fix failure of tests that use defrag on btrfs-progs v6.0+
In reply to [email protected]From: Filipe Manana <[email protected]>
Starting with btrfs-progs v6.0, the defrag command now prints to stdout the full path of the files it processes. This makes test cases btrfs/021 and btrfs/256 fail because they don't expect any output from the defrag command.
The change happened with the following commit in btrfs-progs:
dd724f21803d ("btrfs-progs: add logic to handle LOG_DEFAULT messages")
So update the tests to ignore the stdout of the defrag command.
--- tests/btrfs/021 | 4 +++- tests/btrfs/256 | 4 +++- 2 files changed, 6 insertions(+), 2 deletions(-)
tests/btrfs/021 +3 -1
@@ -22,8 +22,10 @@ run_test() sleep 0.5 + # In new versions of btrfs-progs (6.0+), the defrag command outputs to + # stdout the path of the files it operates on. So ignore that. find $SCRATCH_MNT -type f -print0 | xargs -0 \ - $BTRFS_UTIL_PROG filesystem defrag -f + $BTRFS_UTIL_PROG filesystem defrag -f > /dev/null sync wait
tests/btrfs/256 +3 -1
@@ -50,7 +50,9 @@ $FSSUM_PROG -A -f -w "$checksums_file" "$SCRATCH_MNT" # Now defrag each file. for sz in ${file_sizes[@]}; do echo "Defragging file with $sz bytes..." >> $seqres.full - $BTRFS_UTIL_PROG filesystem defragment "$SCRATCH_MNT/f_$sz" + # In new versions of btrfs-progs (6.0+), the defrag command outputs to + # stdout the path of the files it operates on. So ignore that. + $BTRFS_UTIL_PROG filesystem defragment "$SCRATCH_MNT/f_$sz" > /dev/null done # Verify the checksums after the defrag operations.
Signature · 1 line
2.35.1
Re: [PATCH 1/3] btrfs/003: fix failure on new btrfs-progs versions
In reply to [email protected]Quoted 20 lines · On Wed, Nov 09, 2022 at 11:43:34AM +0000, [email protected] wrote:
From: Filipe Manana <[email protected]>
Starting with btrfs-progs version 5.19, the output of 'filesystem show' command changed when we have a missing device. The old output was like the following:
Label: none uuid: 139ef309-021f-4b98-a3a8-ce230a83b1e2
Total devices 2 FS bytes used 128.00KiB
devid 1 size 5.00GiB used 1.26GiB path/dev/loop0
*** Some devices missingWhile the new output (btrfs-progs 5.19+) is like the following:
Label: none uuid: 4a85a40b-9b79-4bde-8e52-c65a550a176b
Total devices 2 FS bytes used 128.00KiB
devid 1 size 5.00GiB used 1.26GiB path/dev/loop0
devid 2 size 0 used 0 path/dev/loop1MISSINGMore specifically it happened in the following btrfs-progs commit:
Quoted 7 lines
957a79c9b016("btrfs-progs: fi show: print missing device for a mounted file system")This is making btrfs/003 fail with btrfs-progs 5.19+. Update the grep filter in btrfs/003 so that it works with both output formats.
Signed-off-by Filipe Manana <[email protected]>---
OK, make sense,
Quoted 22 lines
tests/btrfs/003 | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)tests/btrfs/003 +3
-2@@ -141,8 +141,9 @@ _test_replace() _devmgt_remove ${removed_dev_htl} $ds dev_removed=1 - $BTRFS_UTIL_PROG filesystem show $SCRATCH_DEV | grep "Some devices missing" >> $seqres.full || _fail \ - "btrfs did not report device missing" + $BTRFS_UTIL_PROG filesystem show $SCRATCH_DEV | \ + grep -ie '\bmissing\b' >> $seqres.full || \ + _fail "btrfs did not report device missing" # add a new disk to btrfs ds=${devs[@]:$(($n)):1}Signature · 1 line
2.35.1
Re: [PATCH 2/3] btrfs/053: fix test failure when running with btrfs-progs v6.0+
In reply to [email protected]Quoted 22 lines · On Wed, Nov 09, 2022 at 11:43:35AM +0000, [email protected] wrote:
From: Filipe Manana <[email protected]>
In btrfs-progs v6.0 the --leafsize (-l) command line option was removed, so btrfs/053 always fails with v6.0+.
The change was introduced by the following btrfs-progs commit:
f7a768d62498("btrfs-progs: mkfs: remove support for option --leafsize")Change the test to use --nodesize (-n) instead, since it exists in both old and new btrfs-progs versions.
Signed-off-by Filipe Manana <[email protected]>--- tests/btrfs/053 | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)tests/btrfs/053 +0
-0
tests/btrfs/053 send_files_dir=$TEST_DIR/btrfs-test-$seq · line 47
Quoted 5 lines
tests/btrfs/053 +1
-1@@ -44,7 +44,7 @@ send_files_dir=$TEST_DIR/btrfs-test-$seq rm -fr $send_files_dir mkdir $send_files_dir -_scratch_mkfs "-l $leaf_size" >/dev/null 2>&1 +_scratch_mkfs "--nodesize $leaf_size" >/dev/null 2>&1
We you said this case starts to fail on btrfs-progs v6.0, I'm wondering how it fail (I don't doubt it fails), at least not fails at here right?
Actually I recommend mkfs output to .full file at here, especially when you use specified mkfs options to _scratch_mkfs helper. That really might fail, and the case might keep running (with old fs on SCRATCH_DEV), and we hard to notice that if no message output and no return value checking.
The _scratch_mkfs doesn't _fail if it fails, it just return nonzero and output error message. So generally I recommend writting likes this (or other proper way which can detect mkfs failure):
_scratch_mkfs "--nodesize $leaf_size" >>$seqres.full 2>&1 || _fail "mkfs failed"
especially if there's specified mkfs options.
Thanks,
Zorro
Quoted 15 lines
tests/btrfs/053 +1
-1_scratch_mount echo "hello world" > $SCRATCH_MNT/foobar@@ -72,7 +72,7 @@ _run_btrfs_util_prog send -p $SCRATCH_MNT/mysnap1 -f $send_files_dir/2.snap \ _scratch_unmount _check_scratch_fs -_scratch_mkfs "-l $leaf_size" >/dev/null 2>&1 +_scratch_mkfs "--nodesize $leaf_size" >/dev/null 2>&1 _scratch_mount _run_btrfs_util_prog receive -f $send_files_dir/1.snap $SCRATCH_MNTSignature · 1 line
2.35.1
Re: [PATCH 3/3] btrfs: fix failure of tests that use defrag on btrfs-progs v6.0+
In reply to [email protected]Quoted 7 lines · On Wed, Nov 09, 2022 at 11:43:36AM +0000, [email protected] wrote:
From: Filipe Manana <[email protected]>
Starting with btrfs-progs v6.0, the defrag command now prints to stdout the full path of the files it processes. This makes test cases btrfs/021 and btrfs/256 fail because they don't expect any output from the defrag command.
Quoted 8 lines
The change happened with the following commit in btrfs-progs:
dd724f21803d("btrfs-progs: add logic to handleLOG_DEFAULTmessages")So update the tests to ignore the stdout of the defrag command.
Signed-off-by Filipe Manana <[email protected]>---
Good to me,
Quoted 38 lines
tests/btrfs/021 | 4 +++- tests/btrfs/256 | 4 +++- 2 files changed, 6 insertions(+), 2 deletions(-)tests/btrfs/021 +3
-1@@ -22,8 +22,10 @@ run_test() sleep 0.5 + # In new versions of btrfs-progs (6.0+), the defrag command outputs to + # stdout the path of the files it operates on. So ignore that. find $SCRATCH_MNT -type f -print0 | xargs -0 \ - $BTRFS_UTIL_PROG filesystem defrag -f + $BTRFS_UTIL_PROG filesystem defrag -f > /dev/null sync waittests/btrfs/256 +3
-1@@ -50,7 +50,9 @@ $FSSUM_PROG -A -f -w "$checksums_file" "$SCRATCH_MNT" # Now defrag each file. for sz in ${file_sizes[@]}; do echo "Defragging file with $sz bytes..." >> $seqres.full - $BTRFS_UTIL_PROG filesystem defragment "$SCRATCH_MNT/f_$sz" + # In new versions of btrfs-progs (6.0+), the defrag command outputs to + # stdout the path of the files it operates on. So ignore that. + $BTRFS_UTIL_PROG filesystem defragment "$SCRATCH_MNT/f_$sz" > /dev/null done # Verify the checksums after the defrag operations.Signature · 1 line
2.35.1
Re: [PATCH 2/3] btrfs/053: fix test failure when running with btrfs-progs v6.0+
In reply to Zorro LangQuoted 27 lines · Zorro Lang
Quoted 25 lines · On Wed, Nov 09, 2022 at 11:43:35AM +0000, [email protected] wrote:
From: Filipe Manana <[email protected]>
In btrfs-progs v6.0 the --leafsize (-l) command line option was removed, so btrfs/053 always fails with v6.0+.
The change was introduced by the following btrfs-progs commit:
f7a768d62498("btrfs-progs: mkfs: remove support for option --leafsize")Change the test to use --nodesize (-n) instead, since it exists in both old and new btrfs-progs versions.
Signed-off-by Filipe Manana <[email protected]>--- tests/btrfs/053 | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)tests/btrfs/053 +0
-0@@ -44,7 +44,7 @@ send_files_dir=$TEST_DIR/btrfs-test-$seq rm -fr $send_files_dir mkdir $send_files_dir
Quoted 5 lines
tests/btrfs/053 send_files_dir=$TEST_DIR/btrfs-test-$seq · line 47
Quoted 2 lines
tests/btrfs/053 +1
-1-_scratch_mkfs "-l $leaf_size" >/dev/null 2>&1 +_scratch_mkfs "--nodesize $leaf_size" >/dev/null 2>&1We you said this case starts to fail on btrfs-progs v6.0, I'm wondering how it fail (I don't doubt it fails), at least not fails at here right?
It fails during mount in case no previous mkfs on the scratch device happened. First time after booting a vm and without running any other test case.
Quoted 11 lines
Actually I recommend mkfs output to .full file at here, especially when you use specified mkfs options to
_scratch_mkfshelper. That really might fail, and the case might keep running (with old fs onSCRATCH_DEV), and we hard to notice that if no message output and no return value checking.The
_scratch_mkfsdoesn't_failif it fails, it just return nonzero and output error message. So generally I recommend writting likes this (or other proper way which can detect mkfs failure):_scratch_mkfs "--nodesize $leaf_size" >>$seqres.full 2>&1 || _fail "mkfs failed"
Yes, I remember some years ago a thread discussing silent mkfs failures being ignored and getting unexpected results or tests passing when they shouldn't.
I'll add that, thanks.
Quoted 22 lines
especially if there's specified mkfs options.
Thanks,
ZorroQuoted 15 lines
tests/btrfs/053 +1
-2_scratch_mount echo "hello world" > $SCRATCH_MNT/foobar@@ -72,7 +72,7 @@ _run_btrfs_util_prog send -p $SCRATCH_MNT/mysnap1 -f $send_files_dir/2.snap \ _scratch_unmount _check_scratch_fs -_scratch_mkfs "-l $leaf_size" >/dev/null 2>&1 +_scratch_mkfs "--nodesize $leaf_size" >/dev/null 2>&1 _scratch_mount _run_btrfs_util_prog receive -f $send_files_dir/1.snap $SCRATCH_MNT --2.35.1
v2 cover btrfs: fix some test failures with btrfs-progs 5.19 and 6.0
In reply to [email protected]From: Filipe Manana <[email protected]>
The output of some commands changed with btrfs-progs 5.19 and 6.0, as well as the --leafsize/-l command line option was removed from mkfs. These make some tests fail. This patchset changes the tests so that they work with the new btrfs-progs versions as well as older versions.
v2: Updated patch 02/03, redirect mkfs output to log file and explicitly
fail if mkfs failed.
Filipe Manana (3):
btrfs/003: fix failure on new btrfs-progs versions btrfs/053: fix test failure when running with btrfs-progs v6.0+
btrfs: fix failure of tests that use defrag on btrfs-progs v6.0+
tests/btrfs/003 | 5 +++-- tests/btrfs/021 | 4 +++- tests/btrfs/053 | 4 ++-- tests/btrfs/256 | 4 +++- 4 files changed, 11 insertions(+), 6 deletions(-)
Signature · 1 line
2.35.1
PATCH v2 1/3 btrfs/003: fix failure on new btrfs-progs versions
From: Filipe Manana <[email protected]>
Starting with btrfs-progs version 5.19, the output of 'filesystem show' command changed when we have a missing device. The old output was like the following:
Label: none uuid: 139ef309-021f-4b98-a3a8-ce230a83b1e2
Total devices 2 FS bytes used 128.00KiB
devid 1 size 5.00GiB used 1.26GiB path /dev/loop0
*** Some devices missing
While the new output (btrfs-progs 5.19+) is like the following:
Label: none uuid: 4a85a40b-9b79-4bde-8e52-c65a550a176b
Total devices 2 FS bytes used 128.00KiB
devid 1 size 5.00GiB used 1.26GiB path /dev/loop0
devid 2 size 0 used 0 path /dev/loop1 MISSING
More specifically it happened in the following btrfs-progs commit:
957a79c9b016 ("btrfs-progs: fi show: print missing device for a mounted file system")
This is making btrfs/003 fail with btrfs-progs 5.19+. Update the grep filter in btrfs/003 so that it works with both output formats.
--- tests/btrfs/003 | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
tests/btrfs/003 +3 -2
@@ -141,8 +141,9 @@ _test_replace() _devmgt_remove ${removed_dev_htl} $ds dev_removed=1 - $BTRFS_UTIL_PROG filesystem show $SCRATCH_DEV | grep "Some devices missing" >> $seqres.full || _fail \ - "btrfs did not report device missing" + $BTRFS_UTIL_PROG filesystem show $SCRATCH_DEV | \ + grep -ie '\bmissing\b' >> $seqres.full || \ + _fail "btrfs did not report device missing" # add a new disk to btrfs ds=${devs[@]:$(($n)):1}
Signature · 1 line
2.35.1
PATCH v2 3/3 btrfs: fix failure of tests that use defrag on btrfs-progs v6.0+
In reply to [email protected]From: Filipe Manana <[email protected]>
Starting with btrfs-progs v6.0, the defrag command now prints to stdout the full path of the files it processes. This makes test cases btrfs/021 and btrfs/256 fail because they don't expect any output from the defrag command.
The change happened with the following commit in btrfs-progs:
dd724f21803d ("btrfs-progs: add logic to handle LOG_DEFAULT messages")
So update the tests to ignore the stdout of the defrag command.
--- tests/btrfs/021 | 4 +++- tests/btrfs/256 | 4 +++- 2 files changed, 6 insertions(+), 2 deletions(-)
tests/btrfs/021 +3 -1
@@ -22,8 +22,10 @@ run_test() sleep 0.5 + # In new versions of btrfs-progs (6.0+), the defrag command outputs to + # stdout the path of the files it operates on. So ignore that. find $SCRATCH_MNT -type f -print0 | xargs -0 \ - $BTRFS_UTIL_PROG filesystem defrag -f + $BTRFS_UTIL_PROG filesystem defrag -f > /dev/null sync wait
tests/btrfs/256 +3 -1
@@ -50,7 +50,9 @@ $FSSUM_PROG -A -f -w "$checksums_file" "$SCRATCH_MNT" # Now defrag each file. for sz in ${file_sizes[@]}; do echo "Defragging file with $sz bytes..." >> $seqres.full - $BTRFS_UTIL_PROG filesystem defragment "$SCRATCH_MNT/f_$sz" + # In new versions of btrfs-progs (6.0+), the defrag command outputs to + # stdout the path of the files it operates on. So ignore that. + $BTRFS_UTIL_PROG filesystem defragment "$SCRATCH_MNT/f_$sz" > /dev/null done # Verify the checksums after the defrag operations.
Signature · 1 line
2.35.1
Re: [PATCH v2 3/3] btrfs: fix failure of tests that use defrag on btrfs-progs v6.0+
Quoted 7 lines · On 2022/11/10 00:44, [email protected] wrote:
From: Filipe Manana <[email protected]>
Starting with btrfs-progs v6.0, the defrag command now prints to stdout the full path of the files it processes. This makes test cases btrfs/021 and btrfs/256 fail because they don't expect any output from the defrag command.
Quoted 7 lines
The change happened with the following commit in btrfs-progs:
dd724f21803d("btrfs-progs: add logic to handleLOG_DEFAULTmessages")So update the tests to ignore the stdout of the defrag command.
Signed-off-by Filipe Manana <[email protected]>
Thanks,
Qu
Quoted 36 lines
--- tests/btrfs/021 | 4 +++- tests/btrfs/256 | 4 +++- 2 files changed, 6 insertions(+), 2 deletions(-)tests/btrfs/021 +3
-1@@ -22,8 +22,10 @@ run_test() sleep 0.5 + # In new versions of btrfs-progs (6.0+), the defrag command outputs to + # stdout the path of the files it operates on. So ignore that. find $SCRATCH_MNT -type f -print0 | xargs -0 \ - $BTRFS_UTIL_PROG filesystem defrag -f + $BTRFS_UTIL_PROG filesystem defrag -f > /dev/null sync waittests/btrfs/256 +3
-1@@ -50,7 +50,9 @@ $FSSUM_PROG -A -f -w "$checksums_file" "$SCRATCH_MNT" # Now defrag each file. for sz in ${file_sizes[@]}; do echo "Defragging file with $sz bytes..." >> $seqres.full - $BTRFS_UTIL_PROG filesystem defragment "$SCRATCH_MNT/f_$sz" + # In new versions of btrfs-progs (6.0+), the defrag command outputs to + # stdout the path of the files it operates on. So ignore that. + $BTRFS_UTIL_PROG filesystem defragment "$SCRATCH_MNT/f_$sz" > /dev/null done # Verify the checksums after the defrag operations.
Re: [PATCH v2 1/3] btrfs/003: fix failure on new btrfs-progs versions
In reply to [email protected]Quoted 18 lines · On 2022/11/10 00:44, [email protected] wrote:
From: Filipe Manana <[email protected]>
Starting with btrfs-progs version 5.19, the output of 'filesystem show' command changed when we have a missing device. The old output was like the following:
Label: none uuid: 139ef309-021f-4b98-a3a8-ce230a83b1e2
Total devices 2 FS bytes used 128.00KiB
devid 1 size 5.00GiB used 1.26GiB path/dev/loop0
*** Some devices missingWhile the new output (btrfs-progs 5.19+) is like the following:
Label: none uuid: 4a85a40b-9b79-4bde-8e52-c65a550a176b
Total devices 2 FS bytes used 128.00KiB
devid 1 size 5.00GiB used 1.26GiB path/dev/loop0
devid 2 size 0 used 0 path/dev/loop1MISSING
Quoted 8 lines
More specifically it happened in the following btrfs-progs commit:
957a79c9b016("btrfs-progs: fi show: print missing device for a mounted file system")This is making btrfs/003 fail with btrfs-progs 5.19+. Update the grep filter in btrfs/003 so that it works with both output formats.
Signed-off-by Filipe Manana <[email protected]>
After more digging, it shows that the test case itself will skip non-deletable disks, thus it doesn't cover my LVM based test environment at all.
Thanks,
Qu
Quoted 20 lines
--- tests/btrfs/003 | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)tests/btrfs/003 +3
-2@@ -141,8 +141,9 @@ _test_replace() _devmgt_remove ${removed_dev_htl} $ds dev_removed=1 - $BTRFS_UTIL_PROG filesystem show $SCRATCH_DEV | grep "Some devices missing" >> $seqres.full || _fail \ - "btrfs did not report device missing" + $BTRFS_UTIL_PROG filesystem show $SCRATCH_DEV | \ + grep -ie '\bmissing\b' >> $seqres.full || \ + _fail "btrfs did not report device missing" # add a new disk to btrfs ds=${devs[@]:$(($n)):1}