fstests: btrfs/09[58]: Use hash to replace unreliable od output
PATCH 1/1 fstests: btrfs/09[58]: Use hash to replace unreliable od output
[BUG]
With latest master, btrfs/09[58] fails like:
btrfs/095 2s ... - output mismatch (see xfstests-dev/results//btrfs/095.out.bad)
--- tests/btrfs/095.out 2019-12-12 13:23:24.266697540 +0800
+++ xfstests-dev/results//btrfs/095.out.bad 2019-12-12 13:23:29.340030879 +0800
@@ -4,32 +4,32 @@
File contents before power failure:
0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
-207 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa
+771 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa
*
-226 bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb
...
(Run 'diff -u xfstests-dev/tests/btrfs/095.out xfstests-dev/results//btrfs/095.out.bad' to see the entire diff)
btrfs/098 2s ... - output mismatch (see xfstests-dev/results//btrfs/098.out.bad)
--- tests/btrfs/098.out 2019-12-12 13:23:24.266697540 +0800
+++ xfstests-dev/results//btrfs/098.out.bad 2019-12-12 13:23:31.306697545 +0800
@@ -3,20 +3,20 @@
File contents before power failure:
0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
-144 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa
+537 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa
*
-151 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
...
(Run 'diff -u xfstests-dev/tests/btrfs/098.out xfstests-dev/results//btrfs/098.out.bad' to see the entire diff) Ran: btrfs/095 btrfs/098
Failures: btrfs/095 btrfs/098
Failed 2 of 2 tests
[CAUSE]
It looks like commit 37520a314bd4 ("fstests: Don't use gawk's strtonum") is making _filter_od doing stupid filtering.
[FIX]
Personally speaking, I don't really care about whatever _filter_od is doing at all.
And since these two test cases only care about the content, let's use hash to replace unreliable _filter_od output.
While move the filtered (and meaningless) od output to $seqres.full.
--- tests/btrfs/095 | 13 +++++++++---- tests/btrfs/095.out | 36 ++++-------------------------------- tests/btrfs/098 | 14 ++++++++++---- tests/btrfs/098.out | 24 ++++-------------------- 4 files changed, 27 insertions(+), 60 deletions(-)
tests/btrfs/095 +9 -4
@@ -122,8 +122,10 @@ $CLONER_PROG -s $((768 * $BLOCK_SIZE)) -d $((150 * $BLOCK_SIZE)) -l $((25 * $BLO # fsync and before the next transaction commit. $XFS_IO_PROG -c "fsync" $SCRATCH_MNT/foo -echo "File contents before power failure:" -od -t x1 $SCRATCH_MNT/foo | _filter_od +echo "File hash before power failure:" +_md5_checksum $SCRATCH_MNT/foo +echo "File contens before power failure:" >> $seqres.full +od -t x1 $SCRATCH_MNT/foo | _filter_od >> $seqres.full # During log replay, the btrfs delayed references implementation used to run the # deletion of back references before the addition of new back references, which
@@ -135,8 +137,11 @@ od -t x1 $SCRATCH_MNT/foo | _filter_od # failure of the insertion that ended up turning the fs into read-only mode. _flakey_drop_and_remount -echo "File contents after log replay:" -od -t x1 $SCRATCH_MNT/foo | _filter_od +echo "File hash after log replay:" +_md5_checksum $SCRATCH_MNT/foo + +echo "File contents after log replay:" >> $seqres.full +od -t x1 $SCRATCH_MNT/foo | _filter_od >> $seqres.full _unmount_flakey
tests/btrfs/095.out +4 -32
@@ -1,35 +1,7 @@ QA output created by 095 Blocks modified: [135 - 164] Blocks modified: [768 - 792] -File contents before power failure: -0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 -* -207 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa -* -226 bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb -* -257 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 -* -1137 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa -* -1175 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 -* -1400 bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb -* -1431 -File contents after log replay: -0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 -* -207 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa -* -226 bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb -* -257 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 -* -1137 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa -* -1175 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 -* -1400 bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb -* -1431 +File hash before power failure: +beaf47c36659ac29bb9363fb8ffa10a1 +File hash after log replay: +beaf47c36659ac29bb9363fb8ffa10a1
tests/btrfs/098 +10 -4
@@ -69,8 +69,11 @@ $CLONER_PROG -s $(((200 * $BLOCK_SIZE) + (5 * $BLOCK_SIZE))) \ # first fsync we did before. $XFS_IO_PROG -c "fsync" $SCRATCH_MNT/foo -echo "File contents before power failure:" -od -t x1 $SCRATCH_MNT/foo | _filter_od + +echo "File hash before power failure:" +_md5_checksum $SCRATCH_MNT/foo +echo "File contents before power failure:" >> $seqres.full +od -t x1 $SCRATCH_MNT/foo | _filter_od >> $seqres.full # The fsync log replay first processes the file extent item corresponding to the # file offset mapped by 100th block (the one which refers to the [5, 10[ block
@@ -95,10 +98,13 @@ od -t x1 $SCRATCH_MNT/foo | _filter_od # _flakey_drop_and_remount -echo "File contents after log replay:" # Must match the file contents we had after cloning the extent and before # the power failure happened. -od -t x1 $SCRATCH_MNT/foo | _filter_od +echo "File hash after log replay:" +_md5_checksum $SCRATCH_MNT/foo + +echo "File contents after log replay:" >> $seqres.full +od -t x1 $SCRATCH_MNT/foo | _filter_od >> $seqres.full _unmount_flakey
tests/btrfs/098.out +4 -20
@@ -1,22 +1,6 @@ QA output created by 098 Blocks modified: [200 - 224] -File contents before power failure: -0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 -* -144 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa -* -151 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 -* -310 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa -* -341 -File contents after log replay: -0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 -* -144 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa -* -151 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 -* -310 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa -* -341 +File hash before power failure: +39b386375971248740ed8651d5a2ed9f +File hash after log replay: +39b386375971248740ed8651d5a2ed9f
Signature · 1 line
2.23.0
Quoted 29 lines · Qu Wenruo
[BUG]
With latest master, btrfs/09[58] fails like:btrfs/095 2s ... - output mismatch (see xfstests-dev/results//btrfs/095.out.bad)
--- tests/btrfs/095.out 2019-12-12 13:23:24.266697540 +0800 +++ xfstests-dev/results//btrfs/095.out.bad 2019-12-12 13:23:29.340030879 +0800 @@ -4,32 +4,32 @@ File contents before power failure: 0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 * -207 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +771 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa * -226 bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb ... (Run 'diff -u xfstests-dev/tests/btrfs/095.out xfstests-dev/results//btrfs/095.out.bad' to see the entire diff) btrfs/098 2s ... - output mismatch (see xfstests-dev/results//btrfs/098.out.bad) --- tests/btrfs/098.out 2019-12-12 13:23:24.266697540 +0800 +++ xfstests-dev/results//btrfs/098.out.bad 2019-12-12 13:23:31.306697545 +0800 @@ -3,20 +3,20 @@ File contents before power failure: 0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 * -144 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +537 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa * -151 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ... (Run 'diff -u xfstests-dev/tests/btrfs/098.out xfstests-dev/results//btrfs/098.out.bad' to see the entire diff)
Quoted 7 lines
Ran: btrfs/095 btrfs/098
Failures: btrfs/095 btrfs/098
Failed 2 of 2 tests[CAUSE]
It looks like commit37520a314bd4("fstests: Don't use gawk's strtonum") is making_filter_oddoing stupid filtering.
I sent a fix to the list. That commit is parsing od's offsets as decimal which actually is octal.
https://lore.kernel.org/fstests/[email protected]/T/#u
Quoted 6 lines
[FIX]
Personally speaking, I don't really care about whatever_filter_odis doing at all.
And since these two test cases only care about the content, let's use hash to replace unreliable_filter_odoutput.
While move the filtered (and meaningless) od output to $seqres.full.
I think using hash break the tests on non-4k blocks.
Actually, they both once used hash, but changed to use od with commit 55144172825f ("btrfs/095: work on non-4k block sized filesystems") and commit 2099e00681dd ("btrfs/098: work on non-4k block sized filesystems").
Quoted 153 lines
Reported-by Nikolay Borisov <[email protected]>Signed-off-by Qu Wenruo <[email protected]>---
tests/btrfs/095 | 13 +++++++++---- tests/btrfs/095.out | 36 ++++-------------------------------- tests/btrfs/098 | 14 ++++++++++---- tests/btrfs/098.out | 24 ++++-------------------- 4 files changed, 27 insertions(+), 60 deletions(-)tests/btrfs/095 +0
-0@@ -122,8 +122,10 @@ $CLONER_PROG -s $((768 * $BLOCK_SIZE)) -d $((150 * $BLOCK_SIZE)) -l $((25 * $BLO # fsync and before the next transaction commit.$XFS_IO_PROG -c "fsync" $SCRATCH_MNT/foo
tests/btrfs/095 +4
-2-echo "File contents before power failure:" -od -t x1 $SCRATCH_MNT/foo | _filter_od +echo "File hash before power failure:" +_md5_checksum $SCRATCH_MNT/foo +echo "File contens before power failure:" >> $seqres.full +od -t x1 $SCRATCH_MNT/foo | _filter_od >> $seqres.full# During log replay, the btrfs delayed references implementation used to run the
# deletion of back references before the addition of new back references, whichtests/btrfs/095 +0
-0@@ -135,8 +137,11 @@ od -t x1 $SCRATCH_MNT/foo | _filter_od # failure of the insertion that ended up turning the fs into read-only mode.
_flakey_drop_and_remounttests/btrfs/095 +5
-2-echo "File contents after log replay:" -od -t x1 $SCRATCH_MNT/foo | _filter_od +echo "File hash after log replay:" +_md5_checksum $SCRATCH_MNT/foo + +echo "File contents after log replay:" >> $seqres.full +od -t x1 $SCRATCH_MNT/foo | _filter_od >> $seqres.full
_unmount_flakeytests/btrfs/095.out +0
-0@@ -1,35 +1,7 @@QA output created by 095
Blocks modified: [135 - 164]
Blocks modified: [768 - 792]tests/btrfs/095.out +4
-32-File contents before power failure: -0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 -* -207 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa -* -226 bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb -* -257 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 -* -1137 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa -* -1175 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 -* -1400 bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb -* -1431 -File contents after log replay: -0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 -* -207 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa -* -226 bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb -* -257 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 -* -1137 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa -* -1175 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 -* -1400 bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb -* -1431 +File hash before power failure: +beaf47c36659ac29bb9363fb8ffa10a1 +File hash after log replay: +beaf47c36659ac29bb9363fb8ffa10a1tests/btrfs/098 +0
-0@@ -69,8 +69,11 @@ $CLONER_PROG -s $(((200 * $BLOCK_SIZE) + (5 * $BLOCK_SIZE))) \ # first fsync we did before.$XFS_IO_PROG -c "fsync" $SCRATCH_MNT/foo
tests/btrfs/098 +5
-2-echo "File contents before power failure:" -od -t x1 $SCRATCH_MNT/foo | _filter_od + +echo "File hash before power failure:" +_md5_checksum $SCRATCH_MNT/foo +echo "File contents before power failure:" >> $seqres.full +od -t x1 $SCRATCH_MNT/foo | _filter_od >> $seqres.full# The fsync log replay first processes the file extent item corresponding to the
# file offset mapped by 100th block (the one which refers to the [5, 10[ blocktests/btrfs/098 +0
-0@@ -95,10 +98,13 @@ od -t x1 $SCRATCH_MNT/foo | _filter_od #
_flakey_drop_and_remount-echo "File contents after log replay:"
# Must match the file contents we had after cloning the extent and before# the power failure happened.tests/btrfs/098 +5
-1-od -t x1 $SCRATCH_MNT/foo | _filter_od +echo "File hash after log replay:" +_md5_checksum $SCRATCH_MNT/foo + +echo "File contents after log replay:" >> $seqres.full +od -t x1 $SCRATCH_MNT/foo | _filter_od >> $seqres.full
_unmount_flakeytests/btrfs/098.out +0
-0@@ -1,22 +1,6 @@QA output created by 098
Blocks modified: [200 - 224]tests/btrfs/098.out +4
-20-File contents before power failure: -0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 -* -144 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa -* -151 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 -* -310 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa -* -341 -File contents after log replay: -0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 -* -144 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa -* -151 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 -* -310 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa -* -341 +File hash before power failure: +39b386375971248740ed8651d5a2ed9f +File hash after log replay: +39b386375971248740ed8651d5a2ed9fSignature · 1 line
2.23.0
Quoted 40 lines · Naohiro Aota
Quoted 39 lines · Qu Wenruo
[BUG]
With latest master, btrfs/09[58] fails like:btrfs/095 2s ... - output mismatch (see xfstests-dev/results//btrfs/095.out.bad)
--- tests/btrfs/095.out 2019-12-12 13:23:24.266697540 +0800 +++ xfstests-dev/results//btrfs/095.out.bad 2019-12-12 13:23:29.340030879 +0800
@@ -4,32 +4,32 @@
File contents before power failure:
0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
-207 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +771 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa *
-226 bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb
...
(Run 'diff -u xfstests-dev/tests/btrfs/095.out xfstests-dev/results//btrfs/095.out.bad' to see the entire diff) btrfs/098 2s ... - output mismatch (see xfstests-dev/results//btrfs/098.out.bad)
--- tests/btrfs/098.out 2019-12-12 13:23:24.266697540 +0800 +++ xfstests-dev/results//btrfs/098.out.bad 2019-12-12 13:23:31.306697545 +0800
@@ -3,20 +3,20 @@
File contents before power failure:
0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
-144 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +537 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa *
-151 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
...
(Run 'diff -u xfstests-dev/tests/btrfs/098.out xfstests-dev/results//btrfs/098.out.bad' to see the entire diff)
Ran: btrfs/095 btrfs/098
Failures: btrfs/095 btrfs/098
Failed 2 of 2 tests
Quoted 8 lines
Quoted 3 lines
[CAUSE]
It looks like commit37520a314bd4("fstests: Don't use gawk's strtonum") is making_filter_oddoing stupid filtering.I sent a fix to the list. That commit is parsing od's offsets as decimal which actually is octal.
Oh, that's much better.
Although that _filter_od still seems can't handle hex.
But anyway, that's a better root solution.
Quoted 9 lines
Quoted 8 lines
[FIX]
Personally speaking, I don't really care about whatever_filter_odis doing at all.
And since these two test cases only care about the content, let's use hash to replace unreliable_filter_odoutput.
While move the filtered (and meaningless) od output to $seqres.full.
Quoted 5 lines
I think using hash break the tests on non-4k blocks.
Actually, they both once used hash, but changed to use od with commit
55144172825f("btrfs/095: work on non-4k block sized filesystems") and commit2099e00681dd("btrfs/098: work on non-4k block sized filesystems").
Forgot that.
Thanks for the info,
Qu
Quoted 161 lines
Quoted 161 lines
Reported-by Nikolay Borisov <[email protected]>Signed-off-by Qu Wenruo <[email protected]>---
tests/btrfs/095 | 13 +++++++++---- tests/btrfs/095.out | 36 ++++-------------------------------- tests/btrfs/098 | 14 ++++++++++---- tests/btrfs/098.out | 24 ++++--------------------4 files changed, 27 insertions(+), 60 deletions(-)tests/btrfs/095 +0
-0@@ -122,8 +122,10 @@ $CLONER_PROG -s $((768 * $BLOCK_SIZE)) -d $((150 * $BLOCK_SIZE)) -l $((25 * $BLO # fsync and before the next transaction commit.$XFS_IO_PROG -c "fsync" $SCRATCH_MNT/foo
tests/btrfs/095 +4
-2-echo "File contents before power failure:" -od -t x1 $SCRATCH_MNT/foo | _filter_od +echo "File hash before power failure:" +_md5_checksum $SCRATCH_MNT/foo +echo "File contens before power failure:" >> $seqres.full +od -t x1 $SCRATCH_MNT/foo | _filter_od >> $seqres.full# During log replay, the btrfs delayed references implementation used to run the
# deletion of back references before the addition of new back references, whichtests/btrfs/095 +0
-0@@ -135,8 +137,11 @@ od -t x1 $SCRATCH_MNT/foo | _filter_od # failure of the insertion that ended up turning the fs into read-onlymode.
_flakey_drop_and_remounttests/btrfs/095 +5
-2-echo "File contents after log replay:" -od -t x1 $SCRATCH_MNT/foo | _filter_od +echo "File hash after log replay:" +_md5_checksum $SCRATCH_MNT/foo + +echo "File contents after log replay:" >> $seqres.full +od -t x1 $SCRATCH_MNT/foo | _filter_od >> $seqres.full
_unmount_flakeytests/btrfs/095.out +0
-0@@ -1,35 +1,7 @@QA output created by 095
Blocks modified: [135 - 164]
Blocks modified: [768 - 792]tests/btrfs/095.out +4
-32-File contents before power failure: -0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 -* -207 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa -* -226 bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb -* -257 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 -* -1137 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa -* -1175 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 -* -1400 bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb -* -1431 -File contents after log replay: -0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 -* -207 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa -* -226 bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb -* -257 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 -* -1137 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa -* -1175 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 -* -1400 bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb -* -1431 +File hash before power failure: +beaf47c36659ac29bb9363fb8ffa10a1 +File hash after log replay: +beaf47c36659ac29bb9363fb8ffa10a1tests/btrfs/098 +0
-0@@ -69,8 +69,11 @@ $CLONER_PROG -s $(((200 * $BLOCK_SIZE) + (5 * $BLOCK_SIZE))) \ # first fsync we did before.$XFS_IO_PROG -c "fsync" $SCRATCH_MNT/foo
tests/btrfs/098 +5
-2-echo "File contents before power failure:" -od -t x1 $SCRATCH_MNT/foo | _filter_od + +echo "File hash before power failure:" +_md5_checksum $SCRATCH_MNT/foo +echo "File contents before power failure:" >> $seqres.full +od -t x1 $SCRATCH_MNT/foo | _filter_od >> $seqres.full# The fsync log replay first processes the file extent item corresponding to the
# file offset mapped by 100th block (the one which refers to the [5, 10[ blocktests/btrfs/098 +0
-0@@ -95,10 +98,13 @@ od -t x1 $SCRATCH_MNT/foo | _filter_od #
_flakey_drop_and_remount-echo "File contents after log replay:"
# Must match the file contents we had after cloning the extent and before# the power failure happened.tests/btrfs/098 +5
-1-od -t x1 $SCRATCH_MNT/foo | _filter_od +echo "File hash after log replay:" +_md5_checksum $SCRATCH_MNT/foo + +echo "File contents after log replay:" >> $seqres.full +od -t x1 $SCRATCH_MNT/foo | _filter_od >> $seqres.full
_unmount_flakeytests/btrfs/098.out +0
-0@@ -1,22 +1,6 @@QA output created by 098
Blocks modified: [200 - 224]tests/btrfs/098.out +4
-20-File contents before power failure: -0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 -* -144 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa -* -151 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 -* -310 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa -* -341 -File contents after log replay: -0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 -* -144 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa -* -151 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 -* -310 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa -* -341 +File hash before power failure: +39b386375971248740ed8651d5a2ed9f +File hash after log replay: +39b386375971248740ed8651d5a2ed9f--
2.23.0
Quoted 47 lines · Qu Wenruo
Quoted 44 lines · Naohiro Aota
Quoted 42 lines · Qu Wenruo
[BUG]
With latest master, btrfs/09[58] fails like:btrfs/095 2s ... - output mismatch (see xfstests-dev/results//btrfs/095.out.bad)
--- tests/btrfs/095.out 2019-12-12 13:23:24.266697540 +0800 +++ xfstests-dev/results//btrfs/095.out.bad 2019-12-12 13:23:29.340030879 +0800
@@ -4,32 +4,32 @@
File contents before power failure:
0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
-207 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +771 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa *
-226 bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb
...
(Run 'diff -u xfstests-dev/tests/btrfs/095.out xfstests-dev/results//btrfs/095.out.bad' to see the entire diff) btrfs/098 2s ... - output mismatch (see xfstests-dev/results//btrfs/098.out.bad)
--- tests/btrfs/098.out 2019-12-12 13:23:24.266697540 +0800 +++ xfstests-dev/results//btrfs/098.out.bad 2019-12-12 13:23:31.306697545 +0800
@@ -3,20 +3,20 @@
File contents before power failure:
0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
-144 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +537 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa *
-151 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
...
(Run 'diff -u xfstests-dev/tests/btrfs/098.out xfstests-dev/results//btrfs/098.out.bad' to see the entire diff)
Ran: btrfs/095 btrfs/098
Failures: btrfs/095 btrfs/098
Failed 2 of 2 tests[CAUSE]
It looks like commit37520a314bd4("fstests: Don't use gawk's strtonum") is making_filter_oddoing stupid filtering.
Quoted 8 lines
Quoted 4 lines
I sent a fix to the list. That commit is parsing od's offsets as decimal which actually is octal.
Oh, that's much better.
Although that
_filter_odstill seems can't handle hex.
I agree with you that ocatal makes no sense these days. I'm considering to extend _filter_od to take an argument to specify "oct|dec|hex" so that callers can use "od -A x" and see outputs in hex (or decimal).
Thanks,
Quoted 38 lines · Qu Wenruo
[BUG]
With latest master, btrfs/09[58] fails like:btrfs/095 2s ... - output mismatch (see xfstests-dev/results//btrfs/095.out.bad)
--- tests/btrfs/095.out 2019-12-12 13:23:24.266697540 +0800 +++ xfstests-dev/results//btrfs/095.out.bad 2019-12-12 13:23:29.340030879 +0800 @@ -4,32 +4,32 @@ File contents before power failure: 0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 * -207 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +771 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa * -226 bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb ... (Run 'diff -u xfstests-dev/tests/btrfs/095.out xfstests-dev/results//btrfs/095.out.bad' to see the entire diff) btrfs/098 2s ... - output mismatch (see xfstests-dev/results//btrfs/098.out.bad) --- tests/btrfs/098.out 2019-12-12 13:23:24.266697540 +0800 +++ xfstests-dev/results//btrfs/098.out.bad 2019-12-12 13:23:31.306697545 +0800 @@ -3,20 +3,20 @@ File contents before power failure: 0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 * -144 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa +537 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa * -151 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ... (Run 'diff -u xfstests-dev/tests/btrfs/098.out xfstests-dev/results//btrfs/098.out.bad' to see the entire diff)Ran: btrfs/095 btrfs/098
Failures: btrfs/095 btrfs/098
Failed 2 of 2 tests[CAUSE]
It looks like commit37520a314bd4("fstests: Don't use gawk's strtonum") is making_filter_oddoing stupid filtering.
Quoted 5 lines
[FIX]
Personally speaking, I don't really care about whatever_filter_odis doing at all.
And since these two test cases only care about the content, let's use hash to replace unreliable_filter_odoutput.
Nop, this won't work.
Have you read carefully the tests and the changelogs?
The file contents are different depending on the page size of the system where it runs. If you look at the tests, they do things like:
$CLONER_PROG -s $(((200 * $BLOCK_SIZE) + (5 * $BLOCK_SIZE))) ....
So hardcoding a checksum in the golden output is not an option, it won't be the same on x86 as on powerpc (with a page size of 64Kb).
Quoted 1 line
While move the filtered (and meaningless) od output to $seqres.full.
Well, it does't make sense to use the filter for the output to $seqres.full.
Given that the author of the commit that introduced the regression is asking Eryu to revert it, as it's causing problems on xfs tests as well, this won't be necessary.
Thanks.
Quoted 153 lines
Reported-by Nikolay Borisov <[email protected]>Signed-off-by Qu Wenruo <[email protected]>--- tests/btrfs/095 | 13 +++++++++---- tests/btrfs/095.out | 36 ++++-------------------------------- tests/btrfs/098 | 14 ++++++++++---- tests/btrfs/098.out | 24 ++++-------------------- 4 files changed, 27 insertions(+), 60 deletions(-)tests/btrfs/095 +9
-4@@ -122,8 +122,10 @@ $CLONER_PROG -s $((768 * $BLOCK_SIZE)) -d $((150 * $BLOCK_SIZE)) -l $((25 * $BLO # fsync and before the next transaction commit. $XFS_IO_PROG -c "fsync" $SCRATCH_MNT/foo -echo "File contents before power failure:" -od -t x1 $SCRATCH_MNT/foo | _filter_od +echo "File hash before power failure:" +_md5_checksum $SCRATCH_MNT/foo +echo "File contens before power failure:" >> $seqres.full +od -t x1 $SCRATCH_MNT/foo | _filter_od >> $seqres.full # During log replay, the btrfs delayed references implementation used to run the # deletion of back references before the addition of new back references, which@@ -135,8 +137,11 @@ od -t x1 $SCRATCH_MNT/foo | _filter_od # failure of the insertion that ended up turning the fs into read-only mode. _flakey_drop_and_remount -echo "File contents after log replay:" -od -t x1 $SCRATCH_MNT/foo | _filter_od +echo "File hash after log replay:" +_md5_checksum $SCRATCH_MNT/foo + +echo "File contents after log replay:" >> $seqres.full +od -t x1 $SCRATCH_MNT/foo | _filter_od >> $seqres.full _unmount_flakeytests/btrfs/095.out +4
-32@@ -1,35 +1,7 @@ QA output created by 095 Blocks modified: [135 - 164] Blocks modified: [768 - 792] -File contents before power failure: -0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 -* -207 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa -* -226 bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb -* -257 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 -* -1137 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa -* -1175 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 -* -1400 bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb -* -1431 -File contents after log replay: -0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 -* -207 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa -* -226 bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb -* -257 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 -* -1137 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa -* -1175 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 -* -1400 bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb -* -1431 +File hash before power failure: +beaf47c36659ac29bb9363fb8ffa10a1 +File hash after log replay: +beaf47c36659ac29bb9363fb8ffa10a1tests/btrfs/098 +10
-4@@ -69,8 +69,11 @@ $CLONER_PROG -s $(((200 * $BLOCK_SIZE) + (5 * $BLOCK_SIZE))) \ # first fsync we did before. $XFS_IO_PROG -c "fsync" $SCRATCH_MNT/foo -echo "File contents before power failure:" -od -t x1 $SCRATCH_MNT/foo | _filter_od + +echo "File hash before power failure:" +_md5_checksum $SCRATCH_MNT/foo +echo "File contents before power failure:" >> $seqres.full +od -t x1 $SCRATCH_MNT/foo | _filter_od >> $seqres.full # The fsync log replay first processes the file extent item corresponding to the # file offset mapped by 100th block (the one which refers to the [5, 10[ block@@ -95,10 +98,13 @@ od -t x1 $SCRATCH_MNT/foo | _filter_od # _flakey_drop_and_remount -echo "File contents after log replay:" # Must match the file contents we had after cloning the extent and before # the power failure happened. -od -t x1 $SCRATCH_MNT/foo | _filter_od +echo "File hash after log replay:" +_md5_checksum $SCRATCH_MNT/foo + +echo "File contents after log replay:" >> $seqres.full +od -t x1 $SCRATCH_MNT/foo | _filter_od >> $seqres.full _unmount_flakeytests/btrfs/098.out +4
-21@@ -1,22 +1,6 @@ QA output created by 098 Blocks modified: [200 - 224] -File contents before power failure: -0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 -* -144 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa -* -151 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 -* -310 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa -* -341 -File contents after log replay: -0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 -* -144 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa -* -151 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 -* -310 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa -* -341 +File hash before power failure: +39b386375971248740ed8651d5a2ed9f +File hash after log replay: +39b386375971248740ed8651d5a2ed9f --2.23.0
Signature · 3 lines
Filipe David Manana, “Whether you think you can, or you think you can't — you're right.”