fstests: btrfs: add a new test case to verify compressed read
PATCH v3 1/1 fstests: btrfs: add a new test case to verify compressed read
The new test case is a regression test related to the block size < page size handling of compressed read.
The test case will only be triggered with 64K page size and 4K btrfs block size.
I'm using aarch64 with 64K page size to trigger the regression.
The test case will create the following file layout:
base:
[0, 64K): Single compressed data extent at bytenr X.
new:
[0, 32K): Reflinked from base [32K, 64K)
[32K, 60K): Reflinked from base [0, 28K)
[60K, 64K): New 4K write
The range [0, 32K) and [32K, 64K) are pointing to the same compressed data.
The last 4K write is a special workaround. It is a block aligned write, thus it will create the folio but without reading out the remaing part.
This is to avoid readahead path, which has the proper fix. We want single folio read without readahead.
Then output the file "new" just after the last 4K write, then cycle mount and output the content again.
For patched kernel, or with 4K page sized system, the test will pass, the resulted content will not change during mount cycles.
For unpatched kernel and with 64K page size, the test will fail, the content after the write will be incorrect (the range [32K, 60K) will be zero), but after a mount cycle the content is correct again.
---
Changelog:
v3:
- Fix the golden output which is generated by an unpatched kernel
v2:
- Remove the unnecessary sync inherited from the kernel fix
- Use _hexdump instead of open-coded od dumps
- Use the correct 'clone' group instead of 'reflink'
- Minor grammar fixes
All thanks to Filipe.
--- tests/btrfs/337 | 55 +++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/337.out | 23 +++++++++++++++++++ 2 files changed, 78 insertions(+) create mode 100755 tests/btrfs/337 create mode 100644 tests/btrfs/337.out
tests/btrfs/337 new file +55 -0
@@ -0,0 +1,55 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025 SUSE Linux Products GmbH. All Rights Reserved. +# +# FS QA Test 337 +# +# Test compressed read with shared extents, especially for bs < ps cases. +# +. ./common/preamble +_begin_fstest auto quick compress clone + +_fixed_by_kernel_commit xxxxxxxxxxxx \ + "btrfs: fix corruption reading compressed range when block size is smaller than page size" + +. ./common/filter +. ./common/reflink + +_require_btrfs_support_sectorsize 4096 +_require_scratch_reflink + +# The layout used in the test case is all 4K based, and can only be reproduced +# with page size larger than 4K. +_scratch_mkfs -s 4k >> $seqres.full +_scratch_mount "-o compress" + +# Create the reflink source, which must be a compressed extent. +$XFS_IO_PROG -f -c "pwrite -S 0x0f 0 32K" \ + -c "pwrite -S 0xf0 32K 32K" \ + $SCRATCH_MNT/base >> $seqres.full +echo "Reflink source:" +_hexdump $SCRATCH_MNT/base + +# Create the reflink dest, which reverses the order of the two 32K ranges. +# +# And do a further aligned write into the last block. +# This write is to make sure the folio exists in filemap, so that we won't go +# through the readahead path (which has the proper handling) for the folio. +$XFS_IO_PROG -f -c "reflink $SCRATCH_MNT/base 32K 0 32K" \ + -c "reflink $SCRATCH_MNT/base 0 32K 32K" \ + -c "pwrite 60K 4K" $SCRATCH_MNT/new >> $seqres.full + +# This will result an incorrect output for unpatched kernel. +# The range [32K, 60K) will be zero due to incorrectly merged compressed read. +echo "Before mount cycle:" +_hexdump $SCRATCH_MNT/new + +_scratch_cycle_mount + +# This will go through readahead path, which has the proper handling, thus give +# the correct content. +echo "After mount cycle:" +_hexdump $SCRATCH_MNT/new + +status=0 +_exit 0
tests/btrfs/337.out new file +23 -0
@@ -0,0 +1,23 @@ +QA output created by 337 +Reflink source: +000000 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f >................< +* +008000 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 >................< +* +010000 +Before mount cycle: +000000 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 >................< +* +008000 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f >................< +* +00f000 cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd >................< +* +010000 +After mount cycle: +000000 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 >................< +* +008000 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f >................< +* +00f000 cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd >................< +* +010000
Signature · 1 line
2.49.0
Quoted 31 lines · Qu Wenruo
The new test case is a regression test related to the block size < page size handling of compressed read.
The test case will only be triggered with 64K page size and 4K btrfs block size.
I'm using aarch64 with 64K page size to trigger the regression.The test case will create the following file layout:
base:
[0, 64K): Single compressed data extent at bytenr X.new:
[0, 32K): Reflinked from base [32K, 64K)
[32K, 60K): Reflinked from base [0, 28K)
[60K, 64K): New 4K writeThe range [0, 32K) and [32K, 64K) are pointing to the same compressed data.
The last 4K write is a special workaround. It is a block aligned write, thus it will create the folio but without reading out the remaing part.
This is to avoid readahead path, which has the proper fix. We want single folio read without readahead.
Then output the file "new" just after the last 4K write, then cycle mount and output the content again.
Quoted 8 lines
For patched kernel, or with 4K page sized system, the test will pass, the resulted content will not change during mount cycles.
For unpatched kernel and with 64K page size, the test will fail, the content after the write will be incorrect (the range [32K, 60K) will be zero), but after a mount cycle the content is correct again.
Signed-off-by Qu Wenruo <[email protected]>
Looks good, thanks.
Quoted 112 lines
---
Changelog:
v3:
- Fix the golden output which is generated by an unpatched kernelv2:
- Remove the unnecessary sync inherited from the kernel fix
- Use_hexdumpinstead of open-coded od dumps
- Use the correct 'clone' group instead of 'reflink'
- Minor grammar fixes
All thanks to Filipe.--- tests/btrfs/337 | 55 +++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/337.out | 23 +++++++++++++++++++ 2 files changed, 78 insertions(+) create mode 100755 tests/btrfs/337 create mode 100644 tests/btrfs/337.outtests/btrfs/337 new file +55
-0@@ -0,0 +1,55 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025 SUSE Linux Products GmbH. All Rights Reserved. +# +# FS QA Test 337 +# +# Test compressed read with shared extents, especially for bs < ps cases. +# +. ./common/preamble +_begin_fstest auto quick compress clone + +_fixed_by_kernel_commit xxxxxxxxxxxx \ + "btrfs: fix corruption reading compressed range when block size is smaller than page size" + +. ./common/filter +. ./common/reflink + +_require_btrfs_support_sectorsize 4096 +_require_scratch_reflink + +# The layout used in the test case is all 4K based, and can only be reproduced +# with page size larger than 4K. +_scratch_mkfs -s 4k >> $seqres.full +_scratch_mount "-o compress" + +# Create the reflink source, which must be a compressed extent. +$XFS_IO_PROG -f -c "pwrite -S 0x0f 0 32K" \ + -c "pwrite -S 0xf0 32K 32K" \ + $SCRATCH_MNT/base >> $seqres.full +echo "Reflink source:" +_hexdump $SCRATCH_MNT/base + +# Create the reflink dest, which reverses the order of the two 32K ranges. +# +# And do a further aligned write into the last block. +# This write is to make sure the folio exists in filemap, so that we won't go +# through the readahead path (which has the proper handling) for the folio. +$XFS_IO_PROG -f -c "reflink $SCRATCH_MNT/base 32K 0 32K" \ + -c "reflink $SCRATCH_MNT/base 0 32K 32K" \ + -c "pwrite 60K 4K" $SCRATCH_MNT/new >> $seqres.full + +# This will result an incorrect output for unpatched kernel. +# The range [32K, 60K) will be zero due to incorrectly merged compressed read. +echo "Before mount cycle:" +_hexdump $SCRATCH_MNT/new + +_scratch_cycle_mount + +# This will go through readahead path, which has the proper handling, thus give +# the correct content. +echo "After mount cycle:" +_hexdump $SCRATCH_MNT/new + +status=0 +_exit 0tests/btrfs/337.out new file +23
-1@@ -0,0 +1,23 @@ +QA output created by 337 +Reflink source: +000000 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f >................< +* +008000 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 >................< +* +010000 +Before mount cycle: +000000 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 >................< +* +008000 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f >................< +* +00f000 cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd >................< +* +010000 +After mount cycle: +000000 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 >................< +* +008000 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f >................< +* +00f000 cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd >................< +* +010000 --2.49.0
Quoted 43 lines · Qu Wenruo
The new test case is a regression test related to the block size < page size handling of compressed read.
The test case will only be triggered with 64K page size and 4K btrfs block size.
I'm using aarch64 with 64K page size to trigger the regression.The test case will create the following file layout:
base:
[0, 64K): Single compressed data extent at bytenr X.new:
[0, 32K): Reflinked from base [32K, 64K)
[32K, 60K): Reflinked from base [0, 28K)
[60K, 64K): New 4K writeThe range [0, 32K) and [32K, 64K) are pointing to the same compressed data.
The last 4K write is a special workaround. It is a block aligned write, thus it will create the folio but without reading out the remaing part.
This is to avoid readahead path, which has the proper fix. We want single folio read without readahead.
Then output the file "new" just after the last 4K write, then cycle mount and output the content again.
For patched kernel, or with 4K page sized system, the test will pass, the resulted content will not change during mount cycles.
For unpatched kernel and with 64K page size, the test will fail, the content after the write will be incorrect (the range [32K, 60K) will be zero), but after a mount cycle the content is correct again.
Signed-off-by Qu Wenruo <[email protected]>---
Changelog:
v3:
- Fix the golden output which is generated by an unpatched kernel
Quoted 7 lines
v2:
- Remove the unnecessary sync inherited from the kernel fix
- Use_hexdumpinstead of open-coded od dumps
- Use the correct 'clone' group instead of 'reflink'
- Minor grammar fixes
All thanks to Filipe.
---
Hi Qu,
This patch is good to me, just a few picky points below:
Quoted 17 lines
tests/btrfs/337 | 55 +++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/337.out | 23 +++++++++++++++++++ 2 files changed, 78 insertions(+) create mode 100755 tests/btrfs/337 create mode 100644 tests/btrfs/337.outtests/btrfs/337 new file +5
-0@@ -0,0 +1,55 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025 SUSE Linux Products GmbH. All Rights Reserved. +# +# FS QA Test 337
tests/btrfs/337 line 15
Quoted 10 lines
tests/btrfs/337 new file +10
-0+# +# Test compressed read with shared extents, especially for bs < ps cases. +# +. ./common/preamble +_begin_fstest auto quick compress clone + +_fixed_by_kernel_commit xxxxxxxxxxxx \ + "btrfs: fix corruption reading compressed range when block size is smaller than page size" + +. ./common/filter
Looks like you didn't use any helpers of common/filter in this case :)
tests/btrfs/337 line 23
Quoted 8 lines
tests/btrfs/337 new file +8
-0+. ./common/reflink + +_require_btrfs_support_sectorsize 4096 +_require_scratch_reflink + +# The layout used in the test case is all 4K based, and can only be reproduced +# with page size larger than 4K. +_scratch_mkfs -s 4k >> $seqres.full
|| _fail "...."
Fail the test if the mkfs fails.
Quoted 21 lines
tests/btrfs/337 new file +21
-0+_scratch_mount "-o compress" + +# Create the reflink source, which must be a compressed extent. +$XFS_IO_PROG -f -c "pwrite -S 0x0f 0 32K" \ + -c "pwrite -S 0xf0 32K 32K" \ + $SCRATCH_MNT/base >> $seqres.full +echo "Reflink source:" +_hexdump $SCRATCH_MNT/base + +# Create the reflink dest, which reverses the order of the two 32K ranges. +# +# And do a further aligned write into the last block. +# This write is to make sure the folio exists in filemap, so that we won't go +# through the readahead path (which has the proper handling) for the folio. +$XFS_IO_PROG -f -c "reflink $SCRATCH_MNT/base 32K 0 32K" \ + -c "reflink $SCRATCH_MNT/base 0 32K 32K" \ + -c "pwrite 60K 4K" $SCRATCH_MNT/new >> $seqres.full + +# This will result an incorrect output for unpatched kernel. +# The range [32K, 60K) will be zero due to incorrectly merged compressed read. +echo "Before mount cycle:"
tests/btrfs/337 line 54
Quoted 10 lines
tests/btrfs/337 new file +10
-0+_hexdump $SCRATCH_MNT/new + +_scratch_cycle_mount + +# This will go through readahead path, which has the proper handling, thus give +# the correct content. +echo "After mount cycle:" +_hexdump $SCRATCH_MNT/new + +status=0
^^^
"_exit 0" does "status=0", so you can save this line.
Others looks good to me, with above changes:
Quoted 34 lines
tests/btrfs/337 new file +1
-0+_exit 0tests/btrfs/337.out new file +23
-0@@ -0,0 +1,23 @@ +QA output created by 337 +Reflink source: +000000 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f >................< +* +008000 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 >................< +* +010000 +Before mount cycle: +000000 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 >................< +* +008000 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f >................< +* +00f000 cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd >................< +* +010000 +After mount cycle: +000000 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 >................< +* +008000 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f >................< +* +00f000 cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd >................< +* +010000Signature · 1 line
2.49.0
Quoted 133 lines · Zorro Lang
Quoted 43 lines · Qu Wenruo
The new test case is a regression test related to the block size < page size handling of compressed read.
The test case will only be triggered with 64K page size and 4K btrfs block size.
I'm using aarch64 with 64K page size to trigger the regression.The test case will create the following file layout:
base:
[0, 64K): Single compressed data extent at bytenr X.new:
[0, 32K): Reflinked from base [32K, 64K)
[32K, 60K): Reflinked from base [0, 28K)
[60K, 64K): New 4K writeThe range [0, 32K) and [32K, 64K) are pointing to the same compressed data.
The last 4K write is a special workaround. It is a block aligned write, thus it will create the folio but without reading out the remaing part.
This is to avoid readahead path, which has the proper fix. We want single folio read without readahead.
Then output the file "new" just after the last 4K write, then cycle mount and output the content again.
For patched kernel, or with 4K page sized system, the test will pass, the resulted content will not change during mount cycles.
For unpatched kernel and with 64K page size, the test will fail, the content after the write will be incorrect (the range [32K, 60K) will be zero), but after a mount cycle the content is correct again.
Signed-off-by Qu Wenruo <[email protected]>---
Changelog:
v3:
- Fix the golden output which is generated by an unpatched kernelQuoted 7 lines
v2:
- Remove the unnecessary sync inherited from the kernel fix
- Use_hexdumpinstead of open-coded od dumps
- Use the correct 'clone' group instead of 'reflink'
- Minor grammar fixes
All thanks to Filipe.
---Hi Qu,
This patch is good to me, just a few picky points below:
Quoted 17 lines
tests/btrfs/337 | 55 +++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/337.out | 23 +++++++++++++++++++ 2 files changed, 78 insertions(+) create mode 100755 tests/btrfs/337 create mode 100644 tests/btrfs/337.outtests/btrfs/337 new file +5
-0@@ -0,0 +1,55 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025 SUSE Linux Products GmbH. All Rights Reserved. +# +# FS QA Test 337tests/btrfs/337 line 15
Quoted 10 lines
tests/btrfs/337 new file +10
-0+# +# Test compressed read with shared extents, especially for bs < ps cases. +# +. ./common/preamble +_begin_fstest auto quick compress clone + +_fixed_by_kernel_commit xxxxxxxxxxxx \ + "btrfs: fix corruption reading compressed range when block size is smaller than page size" + +. ./common/filterLooks like you didn't use any helpers of common/filter in this case :)
tests/btrfs/337 line 23
Quoted 8 lines
tests/btrfs/337 new file +8
-0+. ./common/reflink + +_require_btrfs_support_sectorsize 4096 +_require_scratch_reflink + +# The layout used in the test case is all 4K based, and can only be reproduced +# with page size larger than 4K. +_scratch_mkfs -s 4k >> $seqres.full||
_fail"...."Fail the test if the mkfs fails.
Quoted 21 lines
tests/btrfs/337 new file +21
-0+_scratch_mount "-o compress" + +# Create the reflink source, which must be a compressed extent. +$XFS_IO_PROG -f -c "pwrite -S 0x0f 0 32K" \ + -c "pwrite -S 0xf0 32K 32K" \ + $SCRATCH_MNT/base >> $seqres.full +echo "Reflink source:" +_hexdump $SCRATCH_MNT/base + +# Create the reflink dest, which reverses the order of the two 32K ranges. +# +# And do a further aligned write into the last block. +# This write is to make sure the folio exists in filemap, so that we won't go +# through the readahead path (which has the proper handling) for the folio. +$XFS_IO_PROG -f -c "reflink $SCRATCH_MNT/base 32K 0 32K" \ + -c "reflink $SCRATCH_MNT/base 0 32K 32K" \ + -c "pwrite 60K 4K" $SCRATCH_MNT/new >> $seqres.full + +# This will result an incorrect output for unpatched kernel. +# The range [32K, 60K) will be zero due to incorrectly merged compressed read. +echo "Before mount cycle:"tests/btrfs/337 line 54
Quoted 10 lines
tests/btrfs/337 new file +10
-0+_hexdump $SCRATCH_MNT/new + +_scratch_cycle_mount + +# This will go through readahead path, which has the proper handling, thus give +# the correct content. +echo "After mount cycle:" +_hexdump $SCRATCH_MNT/new + +status=0^^^
"_exit0" does "status=0", so you can save this line.
Quoted 3 lines
Others looks good to me, with above changes:
Reviewed-by Zorro Lang <[email protected]>
These're simple enough changes, I've helped to change them, you don't need to send one more version (except you hope to change more, please tell me :)
Thanks,
Zorro
Quoted 35 lines
Quoted 35 lines
tests/btrfs/337 new file +1
-0+_exit 0tests/btrfs/337.out new file +23
-0@@ -0,0 +1,23 @@ +QA output created by 337 +Reflink source: +000000 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f >................< +* +008000 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 >................< +* +010000 +Before mount cycle: +000000 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 >................< +* +008000 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f >................< +* +00f000 cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd >................< +* +010000 +After mount cycle: +000000 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 f0 >................< +* +008000 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f 0f >................< +* +00f000 cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd >................< +* +010000Signature · 1 line
2.49.0