Skip to contentSkip to conversation

fstests: btrfs: verify the read behavior of compressed inline extent

PATCH Superseded Qu Wenruo 5 participants 8 messages Jan 23, 2024 – Jan 27, 2024 Btrfs file system

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Qu Wenruo

author
#0 lore raw

PATCH 1/1 fstests: btrfs: verify the read behavior of compressed inline extent

[BUG]
There is a report about reading a zstd compressed inline file extent would lead to either a VM_BUG_ON() crash, or lead to incorrect file content.

[CAUSE]
The root cause is a incorrect memcpy_to_page() call, which uses incorrect page offset, and can lead to either the VM_BUG_ON() as we may write beyond the page boundary, or writes into the incorrect offset of the page.

[TEST CASE]
The test case would:

- Mount with the specified compress algorithm
- Create a 4K file
- Verify the 4K file is all inlined and compressed
- Verify the content of the initial write
- Cycle mount to drop all the page cache
- Verify the content of the file again
- Unmount and fsck the fs

This workload would be applied to all supported compression algorithms. And it can catch the problem correctly by triggering VM_BUG_ON(), as our workload would result decompressed extent size to be 4K, and would trigger the VM_BUG_ON() 100%.
And with the revert or the new fix, the test case can pass safely.

---
 tests/btrfs/310     | 81 +++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/310.out |  2 ++
 2 files changed, 83 insertions(+)
 create mode 100755 tests/btrfs/310
 create mode 100644 tests/btrfs/310.out
tests/btrfs/310 new file +81 -0
new file mode 100755
index 00000000..b514a8bc
@@ -0,0 +1,81 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2024 SUSE Linux Products GmbH. All Rights Reserved.
+#
+# FS QA Test 310
+#
+# Make sure reading on an compressed inline extent is behaving correctly
+#
+. ./common/preamble
+_begin_fstest auto quick compress
+
+# Import common functions.
+# . ./common/filter
+
+# real QA test starts here
+
+# Modify as appropriate.
+_supported_fs btrfs
+_require_scratch
+
+# This test require inlined compressed extents creation, and all the writes
+# are designed for 4K sector size.
+_require_btrfs_inline_extents_creation
+_require_btrfs_support_sectorsize 4096
+
+_fixed_by_kernel_commit e01a83e12604 \
+	"Revert \"btrfs: zstd: fix and simplify the inline extent decompression\""
+
+# The correct md5 for the correct 4K file filled with "0xcd"
+md5sum_correct="5fed275e7617a806f94c173746a2a723"
+
+workload()
+{
+	local algo="$1"
+
+	echo "=== Testing compression algorithm ${algo} ===" >> $seqres.full
+	_scratch_mkfs >> $seqres.full
+	_scratch_mount -o compress=${algo}
+
+	_pwrite_byte 0xcd 0 4k "$SCRATCH_MNT/inline_file" > /dev/null
+	result=$(_md5_checksum "$SCRATCH_MNT/inline_file")
+	echo "after initial write, md5sum=${result}" >> $seqres.full
+	if [ "$result" != "$md5sum_correct" ]; then
+		_fail "initial write results incorrect content for \"$algo\""
+	fi
+	sync
+
+	$XFS_IO_PROG -c "fiemap -v" $SCRATCH_MNT/inline_file | tail -n 1 > $tmp.fiemap
+	cat $tmp.fiemap >> $seqres.full
+	# Make sure we got an inlined compressed file extent.
+	# 0x200 means inlined, 0x100 means not block aligned, 0x8 means encoded
+	# (compressed in this case), and 0x1 means the last extent.
+	if ! grep -q "0x309" $tmp.fiemap; then
+		rm -f -- $tmp.fiemap
+		_notrun "No compressed inline extent created, maybe subpage?"
+	fi
+	rm -f -- $tmp.fiemap
+
+	# Unmount to clear the page cache.
+	_scratch_cycle_mount
+
+	# For v6.8-rc1 without the revert or the newer fix, this can
+	# crash or lead to incorrect contents for zstd.
+	result=$(_md5_checksum "$SCRATCH_MNT/inline_file")
+	echo "after cycle mount, md5sum=${result}" >> $seqres.full
+	if [ "$result" != "$md5sum_correct" ]; then
+		_fail "read for compressed inline extent failed for \"$algo\""
+	fi
+	_scratch_unmount
+	_check_scratch_fs
+}
+
+algo_list=($(_btrfs_compression_algos))
+for algo in ${algo_list[@]}; do
+	workload $algo
+done
+
+echo "Silence is golden"
+
+status=0
+exit
tests/btrfs/310.out new file +2 -0
new file mode 100644
index 00000000..7b9eaf78
@@ -0,0 +1,2 @@
+QA output created by 310
+Silence is golden
Signature · 1 line
2.42.0
Reviewed-by on v1 1/1
#1 lore raw
Quoted 56 lines · Qu Wenruo

[BUG]
There is a report about reading a zstd compressed inline file extent would lead to either a VM_BUG_ON() crash, or lead to incorrect file content.

[CAUSE]
The root cause is a incorrect memcpy_to_page() call, which uses incorrect page offset, and can lead to either the VM_BUG_ON() as we may write beyond the page boundary, or writes into the incorrect offset of the page.

[TEST CASE]
The test case would:

- Mount with the specified compress algorithm
- Create a 4K file
- Verify the 4K file is all inlined and compressed
- Verify the content of the initial write
- Cycle mount to drop all the page cache
- Verify the content of the file again
- Unmount and fsck the fs

This workload would be applied to all supported compression algorithms. And it can catch the problem correctly by triggering VM_BUG_ON(), as our workload would result decompressed extent size to be 4K, and would trigger the VM_BUG_ON() 100%.
And with the revert or the new fix, the test case can pass safely.

---
 tests/btrfs/310     | 81 +++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/310.out |  2 ++
 2 files changed, 83 insertions(+)
 create mode 100755 tests/btrfs/310
 create mode 100644 tests/btrfs/310.out
tests/btrfs/310 new file +14 -0
new file mode 100755
index 00000000..b514a8bc
@@ -0,0 +1,81 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2024 SUSE Linux Products GmbH. All Rights Reserved.
+#
+# FS QA Test 310
+#
+# Make sure reading on an compressed inline extent is behaving correctly
+#
+. ./common/preamble
+_begin_fstest auto quick compress
+
+# Import common functions.
+# . ./common/filter
+

tests/btrfs/310 line 24

Quoted 10 lines
tests/btrfs/310 new file +10 -0
+# real QA test starts here
+
+# Modify as appropriate.
+_supported_fs btrfs
+_require_scratch
+
+# This test require inlined compressed extents creation, and all the writes
+# are designed for 4K sector size.
+_require_btrfs_inline_extents_creation
+_require_btrfs_support_sectorsize 4096

Hopefully coverage can be expanded in future to cover some other sector sizes. This looks fine for now:

#2 lore raw
In reply to Qu Wenruo
Quoted 133 lines · Qu Wenruo

[BUG]
There is a report about reading a zstd compressed inline file extent would lead to either a VM_BUG_ON() crash, or lead to incorrect file content.

[CAUSE]
The root cause is a incorrect memcpy_to_page() call, which uses incorrect page offset, and can lead to either the VM_BUG_ON() as we may write beyond the page boundary, or writes into the incorrect offset of the page.

[TEST CASE]
The test case would:

- Mount with the specified compress algorithm
- Create a 4K file
- Verify the 4K file is all inlined and compressed
- Verify the content of the initial write
- Cycle mount to drop all the page cache
- Verify the content of the file again
- Unmount and fsck the fs

This workload would be applied to all supported compression algorithms. And it can catch the problem correctly by triggering VM_BUG_ON(), as our workload would result decompressed extent size to be 4K, and would trigger the VM_BUG_ON() 100%.
And with the revert or the new fix, the test case can pass safely.

---
 tests/btrfs/310     | 81 +++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/310.out |  2 ++
 2 files changed, 83 insertions(+)
 create mode 100755 tests/btrfs/310
 create mode 100644 tests/btrfs/310.out
tests/btrfs/310 new file +81 -0
new file mode 100755
index 00000000..b514a8bc
@@ -0,0 +1,81 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2024 SUSE Linux Products GmbH. All Rights Reserved.
+#
+# FS QA Test 310
+#
+# Make sure reading on an compressed inline extent is behaving correctly
+#
+. ./common/preamble
+_begin_fstest auto quick compress
+
+# Import common functions.
+# . ./common/filter
+
+# real QA test starts here
+
+# Modify as appropriate.
+_supported_fs btrfs
+_require_scratch
+
+# This test require inlined compressed extents creation, and all the writes
+# are designed for 4K sector size.
+_require_btrfs_inline_extents_creation
+_require_btrfs_support_sectorsize 4096
+
+_fixed_by_kernel_commit e01a83e12604 \
+       "Revert \"btrfs: zstd: fix and simplify the inline extent decompression\""
+
+# The correct md5 for the correct 4K file filled with "0xcd"
+md5sum_correct="5fed275e7617a806f94c173746a2a723"
+
+workload()
+{
+       local algo="$1"
+
+       echo "=== Testing compression algorithm ${algo} ===" >> $seqres.full
+       _scratch_mkfs >> $seqres.full
+       _scratch_mount -o compress=${algo}
+
+       _pwrite_byte 0xcd 0 4k "$SCRATCH_MNT/inline_file" > /dev/null
+       result=$(_md5_checksum "$SCRATCH_MNT/inline_file")
+       echo "after initial write, md5sum=${result}" >> $seqres.full
+       if [ "$result" != "$md5sum_correct" ]; then
+               _fail "initial write results incorrect content for \"$algo\""
+       fi
+       sync
+
+       $XFS_IO_PROG -c "fiemap -v" $SCRATCH_MNT/inline_file | tail -n 1 > $tmp.fiemap
+       cat $tmp.fiemap >> $seqres.full
+       # Make sure we got an inlined compressed file extent.
+       # 0x200 means inlined, 0x100 means not block aligned, 0x8 means encoded
+       # (compressed in this case), and 0x1 means the last extent.
+       if ! grep -q "0x309" $tmp.fiemap; then
+               rm -f -- $tmp.fiemap
+               _notrun "No compressed inline extent created, maybe subpage?"
+       fi
+       rm -f -- $tmp.fiemap
+
+       # Unmount to clear the page cache.
+       _scratch_cycle_mount
+
+       # For v6.8-rc1 without the revert or the newer fix, this can
+       # crash or lead to incorrect contents for zstd.
+       result=$(_md5_checksum "$SCRATCH_MNT/inline_file")
+       echo "after cycle mount, md5sum=${result}" >> $seqres.full
+       if [ "$result" != "$md5sum_correct" ]; then
+               _fail "read for compressed inline extent failed for \"$algo\""
+       fi
+       _scratch_unmount
+       _check_scratch_fs
+}
+
+algo_list=($(_btrfs_compression_algos))
+for algo in ${algo_list[@]}; do
+       workload $algo
+done
+
+echo "Silence is golden"
+
+status=0
+exit
tests/btrfs/310.out new file +2 -1
new file mode 100644
index 00000000..7b9eaf78
@@ -0,0 +1,2 @@
+QA output created by 310
+Silence is golden
--
Quoted 3 lines

2.42.0

This looks reasonable to me, but how is $_btrfs_compression_algos defined? Does it include all the algorithm options supported in Btrfs?

Signature · 1 line
真実はいつも一つ!/ Always, there's only one truth!
#3 lore raw
Quoted 132 lines · Neal Gompa
Quoted 131 lines · Qu Wenruo

[BUG]
There is a report about reading a zstd compressed inline file extent would lead to either a VM_BUG_ON() crash, or lead to incorrect file content.

[CAUSE]
The root cause is a incorrect memcpy_to_page() call, which uses incorrect page offset, and can lead to either the VM_BUG_ON() as we may write beyond the page boundary, or writes into the incorrect offset of the page.

[TEST CASE]
The test case would:

- Mount with the specified compress algorithm
- Create a 4K file
- Verify the 4K file is all inlined and compressed
- Verify the content of the initial write
- Cycle mount to drop all the page cache
- Verify the content of the file again
- Unmount and fsck the fs

This workload would be applied to all supported compression algorithms. And it can catch the problem correctly by triggering VM_BUG_ON(), as our workload would result decompressed extent size to be 4K, and would trigger the VM_BUG_ON() 100%.
And with the revert or the new fix, the test case can pass safely.

---
  tests/btrfs/310     | 81 +++++++++++++++++++++++++++++++++++++++++++++
  tests/btrfs/310.out |  2 ++
  2 files changed, 83 insertions(+)

create mode 100755 tests/btrfs/310
create mode 100644 tests/btrfs/310.out

tests/btrfs/310 new file +81 -0
new file mode 100755
index 00000000..b514a8bc
@@ -0,0 +1,81 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2024 SUSE Linux Products GmbH. All Rights Reserved.
+#
+# FS QA Test 310
+#
+# Make sure reading on an compressed inline extent is behaving correctly
+#
+. ./common/preamble
+_begin_fstest auto quick compress
+
+# Import common functions.
+# . ./common/filter
+
+# real QA test starts here
+
+# Modify as appropriate.
+_supported_fs btrfs
+_require_scratch
+
+# This test require inlined compressed extents creation, and all the writes
+# are designed for 4K sector size.
+_require_btrfs_inline_extents_creation
+_require_btrfs_support_sectorsize 4096
+
+_fixed_by_kernel_commit e01a83e12604 \
+       "Revert \"btrfs: zstd: fix and simplify the inline extent decompression\""
+
+# The correct md5 for the correct 4K file filled with "0xcd"
+md5sum_correct="5fed275e7617a806f94c173746a2a723"
+
+workload()
+{
+       local algo="$1"
+
+       echo "=== Testing compression algorithm ${algo} ===" >> $seqres.full
+       _scratch_mkfs >> $seqres.full
+       _scratch_mount -o compress=${algo}
+
+       _pwrite_byte 0xcd 0 4k "$SCRATCH_MNT/inline_file" > /dev/null
+       result=$(_md5_checksum "$SCRATCH_MNT/inline_file")
+       echo "after initial write, md5sum=${result}" >> $seqres.full
+       if [ "$result" != "$md5sum_correct" ]; then
+               _fail "initial write results incorrect content for \"$algo\""
+       fi
+       sync
+
+       $XFS_IO_PROG -c "fiemap -v" $SCRATCH_MNT/inline_file | tail -n 1 > $tmp.fiemap
+       cat $tmp.fiemap >> $seqres.full
+       # Make sure we got an inlined compressed file extent.
+       # 0x200 means inlined, 0x100 means not block aligned, 0x8 means encoded
+       # (compressed in this case), and 0x1 means the last extent.
+       if ! grep -q "0x309" $tmp.fiemap; then
+               rm -f -- $tmp.fiemap
+               _notrun "No compressed inline extent created, maybe subpage?"
+       fi
+       rm -f -- $tmp.fiemap
+
+       # Unmount to clear the page cache.
+       _scratch_cycle_mount
+
+       # For v6.8-rc1 without the revert or the newer fix, this can
+       # crash or lead to incorrect contents for zstd.
+       result=$(_md5_checksum "$SCRATCH_MNT/inline_file")
+       echo "after cycle mount, md5sum=${result}" >> $seqres.full
+       if [ "$result" != "$md5sum_correct" ]; then
+               _fail "read for compressed inline extent failed for \"$algo\""
+       fi
+       _scratch_unmount
+       _check_scratch_fs
+}
+
+algo_list=($(_btrfs_compression_algos))
+for algo in ${algo_list[@]}; do
+       workload $algo
+done
+
+echo "Silence is golden"
+
+status=0
+exit
tests/btrfs/310.out new file +1 -0
new file mode 100644
index 00000000..7b9eaf78
@@ -0,0 +1,2 @@
+QA output created by 310
Quoted 8 lines
Quoted 5 lines
tests/btrfs/310.out new file +1 -1
+Silence is golden
--

2.42.0

This looks reasonable to me, but how is $_btrfs_compression_algos defined? Does it include all the algorithm options supported in Btrfs?

It fetches all the supported compression algo through the sysfs interfaces:

/sys/fs/btrfs/features/compress_*

Along with the default supported zlib compression.

Thanks,
Qu

Quoted 2 lines
Reviewed-by
#4 lore raw
Quoted 145 lines · Qu Wenruo
Quoted 132 lines · Neal Gompa
Quoted 131 lines · Qu Wenruo

[BUG]
There is a report about reading a zstd compressed inline file extent would lead to either a VM_BUG_ON() crash, or lead to incorrect file content.

[CAUSE]
The root cause is a incorrect memcpy_to_page() call, which uses incorrect page offset, and can lead to either the VM_BUG_ON() as we may write beyond the page boundary, or writes into the incorrect offset of the page.

[TEST CASE]
The test case would:

- Mount with the specified compress algorithm
- Create a 4K file
- Verify the 4K file is all inlined and compressed
- Verify the content of the initial write
- Cycle mount to drop all the page cache
- Verify the content of the file again
- Unmount and fsck the fs

This workload would be applied to all supported compression algorithms. And it can catch the problem correctly by triggering VM_BUG_ON(), as our workload would result decompressed extent size to be 4K, and would trigger the VM_BUG_ON() 100%.
And with the revert or the new fix, the test case can pass safely.

---
  tests/btrfs/310     | 81 +++++++++++++++++++++++++++++++++++++++++++++
  tests/btrfs/310.out |  2 ++
  2 files changed, 83 insertions(+)

create mode 100755 tests/btrfs/310
create mode 100644 tests/btrfs/310.out

tests/btrfs/310 new file +81 -0
new file mode 100755
index 00000000..b514a8bc
@@ -0,0 +1,81 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2024 SUSE Linux Products GmbH. All Rights Reserved.
+#
+# FS QA Test 310
+#
+# Make sure reading on an compressed inline extent is behaving correctly
+#
+. ./common/preamble
+_begin_fstest auto quick compress
+
+# Import common functions.
+# . ./common/filter
+
+# real QA test starts here
+
+# Modify as appropriate.
+_supported_fs btrfs
+_require_scratch
+
+# This test require inlined compressed extents creation, and all the writes
+# are designed for 4K sector size.
+_require_btrfs_inline_extents_creation
+_require_btrfs_support_sectorsize 4096
+
+_fixed_by_kernel_commit e01a83e12604 \
+       "Revert \"btrfs: zstd: fix and simplify the inline extent decompression\""
+
+# The correct md5 for the correct 4K file filled with "0xcd"
+md5sum_correct="5fed275e7617a806f94c173746a2a723"
+
+workload()
+{
+       local algo="$1"
+
+       echo "=== Testing compression algorithm ${algo} ===" >> $seqres.full
+       _scratch_mkfs >> $seqres.full
+       _scratch_mount -o compress=${algo}
+
+       _pwrite_byte 0xcd 0 4k "$SCRATCH_MNT/inline_file" > /dev/null
+       result=$(_md5_checksum "$SCRATCH_MNT/inline_file")
+       echo "after initial write, md5sum=${result}" >> $seqres.full
+       if [ "$result" != "$md5sum_correct" ]; then
+               _fail "initial write results incorrect content for \"$algo\""
+       fi
+       sync
+
+       $XFS_IO_PROG -c "fiemap -v" $SCRATCH_MNT/inline_file | tail -n 1 > $tmp.fiemap
+       cat $tmp.fiemap >> $seqres.full
+       # Make sure we got an inlined compressed file extent.
+       # 0x200 means inlined, 0x100 means not block aligned, 0x8 means encoded
+       # (compressed in this case), and 0x1 means the last extent.
+       if ! grep -q "0x309" $tmp.fiemap; then
+               rm -f -- $tmp.fiemap
+               _notrun "No compressed inline extent created, maybe subpage?"
+       fi
+       rm -f -- $tmp.fiemap
+
+       # Unmount to clear the page cache.
+       _scratch_cycle_mount
+
+       # For v6.8-rc1 without the revert or the newer fix, this can
+       # crash or lead to incorrect contents for zstd.
+       result=$(_md5_checksum "$SCRATCH_MNT/inline_file")
+       echo "after cycle mount, md5sum=${result}" >> $seqres.full
+       if [ "$result" != "$md5sum_correct" ]; then
+               _fail "read for compressed inline extent failed for \"$algo\""
+       fi
+       _scratch_unmount
+       _check_scratch_fs
+}
+
+algo_list=($(_btrfs_compression_algos))
+for algo in ${algo_list[@]}; do
+       workload $algo
+done
+
+echo "Silence is golden"
+
+status=0
+exit
tests/btrfs/310.out new file +1 -0
new file mode 100644
index 00000000..7b9eaf78
@@ -0,0 +1,2 @@
+QA output created by 310
Quoted 8 lines
Quoted 5 lines
tests/btrfs/310.out new file +1 -1
+Silence is golden
--

2.42.0

This looks reasonable to me, but how is $_btrfs_compression_algos defined? Does it include all the algorithm options supported in Btrfs?

Quoted 6 lines

It fetches all the supported compression algo through the sysfs interfaces:

/sys/fs/btrfs/features/compress_*

Along with the default supported zlib compression.

Cool then.

Signature · 1 line
真実はいつも一つ!/ Always, there's only one truth!
#5 lore raw
In reply to Qu Wenruo
Quoted 80 lines · Qu Wenruo

[BUG]
There is a report about reading a zstd compressed inline file extent would lead to either a VM_BUG_ON() crash, or lead to incorrect file content.

[CAUSE]
The root cause is a incorrect memcpy_to_page() call, which uses incorrect page offset, and can lead to either the VM_BUG_ON() as we may write beyond the page boundary, or writes into the incorrect offset of the page.

[TEST CASE]
The test case would:

- Mount with the specified compress algorithm
- Create a 4K file
- Verify the 4K file is all inlined and compressed
- Verify the content of the initial write
- Cycle mount to drop all the page cache
- Verify the content of the file again
- Unmount and fsck the fs

This workload would be applied to all supported compression algorithms. And it can catch the problem correctly by triggering VM_BUG_ON(), as our workload would result decompressed extent size to be 4K, and would trigger the VM_BUG_ON() 100%.
And with the revert or the new fix, the test case can pass safely.

---
  tests/btrfs/310     | 81 +++++++++++++++++++++++++++++++++++++++++++++
  tests/btrfs/310.out |  2 ++
  2 files changed, 83 insertions(+)

create mode 100755 tests/btrfs/310
create mode 100644 tests/btrfs/310.out

tests/btrfs/310 new file +38 -0
new file mode 100755
index 00000000..b514a8bc
@@ -0,0 +1,81 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2024 SUSE Linux Products GmbH. All Rights Reserved.
+#
+# FS QA Test 310
+#
+# Make sure reading on an compressed inline extent is behaving correctly
+#
+. ./common/preamble
+_begin_fstest auto quick compress
+
+# Import common functions.
+# . ./common/filter
+
+# real QA test starts here
+
+# Modify as appropriate.
+_supported_fs btrfs
+_require_scratch
+
+# This test require inlined compressed extents creation, and all the writes
+# are designed for 4K sector size.
+_require_btrfs_inline_extents_creation
+_require_btrfs_support_sectorsize 4096
+
+_fixed_by_kernel_commit e01a83e12604 \
+	"Revert \"btrfs: zstd: fix and simplify the inline extent decompression\""
+
+# The correct md5 for the correct 4K file filled with "0xcd"
+md5sum_correct="5fed275e7617a806f94c173746a2a723"
+
+workload()
+{
+	local algo="$1"
+
+	echo "=== Testing compression algorithm ${algo} ===" >> $seqres.full
+	_scratch_mkfs >> $seqres.full
+	_scratch_mount -o compress=${algo}

tests/btrfs/310 line 45

Quoted 10 lines
tests/btrfs/310 new file +7 -0
+
+	_pwrite_byte 0xcd 0 4k "$SCRATCH_MNT/inline_file" > /dev/null
 
 
 
+	result=$(_md5_checksum "$SCRATCH_MNT/inline_file")
+	echo "after initial write, md5sum=${result}" >> $seqres.full
+	if [ "$result" != "$md5sum_correct" ]; then
+		_fail "initial write results incorrect content for \"$algo\""
+	fi

General rule of thumb is where possible use stdout and compare it with the golden output.

So something like the below shall suffice.

echo "after initial write with alog=$algo"
_md5_checksum "$SCRATCH_MNT/inline_file"

Also, helps quick debug, when fails we have the diff.

tests/btrfs/310 line 46

Quoted 1 line
tests/btrfs/310 new file +1 -0
+	sync

Generally, we need comments to explain why sync is necessary.

tests/btrfs/310 line 55

Quoted 9 lines
tests/btrfs/310 new file +9 -0
+
+	$XFS_IO_PROG -c "fiemap -v" $SCRATCH_MNT/inline_file | tail -n 1 > $tmp.fiemap
+	cat $tmp.fiemap >> $seqres.full
+	# Make sure we got an inlined compressed file extent.
+	# 0x200 means inlined, 0x100 means not block aligned, 0x8 means encoded
+	# (compressed in this case), and 0x1 means the last extent.
+	if ! grep -q "0x309" $tmp.fiemap; then
+		rm -f -- $tmp.fiemap
+		_notrun "No compressed inline extent created, maybe subpage?"

workload() is called for each compress algo. If we fail for one of the algo then notrun is not a good option here.

IMO, stdout (with filters?) and comparing it with golden output is better.

Quoted 5 lines
tests/btrfs/310 new file +3 -0
+	fi
 
 
+	rm -f -- $tmp.fiemap
+

tests/btrfs/310 line 68

Quoted 10 lines
tests/btrfs/310 new file +10 -0
+	# Unmount to clear the page cache.
+	_scratch_cycle_mount
+
+	# For v6.8-rc1 without the revert or the newer fix, this can
+	# crash or lead to incorrect contents for zstd.
+	result=$(_md5_checksum "$SCRATCH_MNT/inline_file")
+	echo "after cycle mount, md5sum=${result}" >> $seqres.full
+	if [ "$result" != "$md5sum_correct" ]; then
+		_fail "read for compressed inline extent failed for \"$algo\""
+	fi

Here too, same as above, golden output to compare can be done. And remove _fail.

Thanks, Anand

Quoted 21 lines
tests/btrfs/310 new file +13 -0
+	_scratch_unmount
+	_check_scratch_fs
+}
+
+algo_list=($(_btrfs_compression_algos))
+for algo in ${algo_list[@]}; do
+	workload $algo
+done
+
+echo "Silence is golden"
+
+status=0
+exit
tests/btrfs/310.out new file +2 -0
new file mode 100644
index 00000000..7b9eaf78
@@ -0,0 +1,2 @@
+QA output created by 310
+Silence is golden
#6 lore raw
Quoted 97 lines · Anand Jain
Quoted 82 lines · Qu Wenruo

[BUG]
There is a report about reading a zstd compressed inline file extent would lead to either a VM_BUG_ON() crash, or lead to incorrect file content.

[CAUSE]
The root cause is a incorrect memcpy_to_page() call, which uses incorrect page offset, and can lead to either the VM_BUG_ON() as we may write beyond the page boundary, or writes into the incorrect offset of the page.

[TEST CASE]
The test case would:

- Mount with the specified compress algorithm
- Create a 4K file
- Verify the 4K file is all inlined and compressed
- Verify the content of the initial write
- Cycle mount to drop all the page cache
- Verify the content of the file again
- Unmount and fsck the fs

This workload would be applied to all supported compression algorithms. And it can catch the problem correctly by triggering VM_BUG_ON(), as our workload would result decompressed extent size to be 4K, and would trigger the VM_BUG_ON() 100%.
And with the revert or the new fix, the test case can pass safely.

---
  tests/btrfs/310     | 81 +++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/310.out |  2 ++

  2 files changed, 83 insertions(+)

  create mode 100755 tests/btrfs/310
  create mode 100644 tests/btrfs/310.out

tests/btrfs/310 new file +21 -0
new file mode 100755
index 00000000..b514a8bc
@@ -0,0 +1,81 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2024 SUSE Linux Products GmbH. All Rights Reserved.
+#
+# FS QA Test 310
+#
+# Make sure reading on an compressed inline extent is behaving correctly
+#
+. ./common/preamble
+_begin_fstest auto quick compress
+
+# Import common functions.
+# . ./common/filter
+
+# real QA test starts here
+
+# Modify as appropriate.
+_supported_fs btrfs
+_require_scratch
+
+# This test require inlined compressed extents creation, and all the

writes

tests/btrfs/310 new file +6 -0
+# are designed for 4K sector size.
+_require_btrfs_inline_extents_creation
+_require_btrfs_support_sectorsize 4096
+
+_fixed_by_kernel_commit e01a83e12604 \
+    "Revert \"btrfs: zstd: fix and simplify the inline extent

decompression\""

tests/btrfs/310 new file +11 -0
+
+# The correct md5 for the correct 4K file filled with "0xcd"
+md5sum_correct="5fed275e7617a806f94c173746a2a723"
+
+workload()
+{
+    local algo="$1"
+
+    echo "=== Testing compression algorithm ${algo} ===" >> $seqres.full
+    _scratch_mkfs >> $seqres.full
+    _scratch_mount -o compress=${algo}

tests/btrfs/310 line 45

Quoted 10 lines
tests/btrfs/310 new file +7 -0
+
+    _pwrite_byte 0xcd 0 4k "$SCRATCH_MNT/inline_file" > /dev/null
 
 
 
+    result=$(_md5_checksum "$SCRATCH_MNT/inline_file")
+    echo "after initial write, md5sum=${result}" >> $seqres.full
+    if [ "$result" != "$md5sum_correct" ]; then
+        _fail "initial write results incorrect content for \"$algo\""
+    fi

General rule of thumb is where possible use stdout and compare it with the golden output.

Quoted 6 lines

So something like the below shall suffice.

echo "after initial write with alog=$algo"
_md5_checksum "$SCRATCH_MNT/inline_file"

Also, helps quick debug, when  fails we have the diff.

Nope, for this particular case, golden output is not suitable.

As the workload is dependent on the support compression algorithm, we can not reply on golden output to cover all algorithms.
Or it would always fail for older kernels without zstd, or for newer kernel with newer algorithm.

That's why I personally don't believe golden output is always the best way to go.

Thanks,
Qu

Quoted 68 lines
Quoted 3 lines

+    sync

 Generally, we need comments to explain why sync is necessary.

Quoted 11 lines

+
+    $XFS_IO_PROG -c "fiemap -v" $SCRATCH_MNT/inline_file | tail -n 1

Quoted 1 line

$tmp.fiemap

+    cat $tmp.fiemap >> $seqres.full
+    # Make sure we got an inlined compressed file extent. +    # 0x200 means inlined, 0x100 means not block aligned, 0x8 means encoded
+    # (compressed in this case), and 0x1 means the last extent. +    if ! grep -q "0x309" $tmp.fiemap; then
+        rm -f -- $tmp.fiemap
+        _notrun "No compressed inline extent created, maybe subpage?"

 workload() is called for each compress algo. If we fail for one of the algo then notrun is not a good option here.

 IMO, stdout (with filters?) and comparing it with golden output is better.

Quoted 5 lines

+    fi

+    rm -f -- $tmp.fiemap

tests/btrfs/310 new file +1 -0
+

tests/btrfs/310 line 68

Quoted 10 lines
tests/btrfs/310 new file +10 -0
+    # Unmount to clear the page cache.
+    _scratch_cycle_mount
+
+    # For v6.8-rc1 without the revert or the newer fix, this can
+    # crash or lead to incorrect contents for zstd.
+    result=$(_md5_checksum "$SCRATCH_MNT/inline_file")
+    echo "after cycle mount, md5sum=${result}" >> $seqres.full
+    if [ "$result" != "$md5sum_correct" ]; then
+        _fail "read for compressed inline extent failed for \"$algo\""
+    fi

 Here too, same as above, golden output to compare can be done. And remove _fail.

Thanks, Anand

Quoted 21 lines

+    _scratch_unmount
+    _check_scratch_fs

tests/btrfs/310 new file +11 -0
+}
+
+algo_list=($(_btrfs_compression_algos))
+for algo in ${algo_list[@]}; do
+    workload $algo
+done
+
+echo "Silence is golden"
+
+status=0
+exit
tests/btrfs/310.out new file +2 -0
new file mode 100644
index 00000000..7b9eaf78
@@ -0,0 +1,2 @@
+QA output created by 310
+Silence is golden
#7 lore raw
Quoted 107 lines · Qu Wenruo
Quoted 104 lines · Anand Jain
Quoted 89 lines · Qu Wenruo

[BUG]
There is a report about reading a zstd compressed inline file extent would lead to either a VM_BUG_ON() crash, or lead to incorrect file content.

[CAUSE]
The root cause is a incorrect memcpy_to_page() call, which uses incorrect page offset, and can lead to either the VM_BUG_ON() as we may write beyond the page boundary, or writes into the incorrect offset of the page.

[TEST CASE]
The test case would:

- Mount with the specified compress algorithm
- Create a 4K file
- Verify the 4K file is all inlined and compressed
- Verify the content of the initial write
- Cycle mount to drop all the page cache
- Verify the content of the file again
- Unmount and fsck the fs

This workload would be applied to all supported compression algorithms. And it can catch the problem correctly by triggering VM_BUG_ON(), as our workload would result decompressed extent size to be 4K, and would trigger the VM_BUG_ON() 100%.
And with the revert or the new fix, the test case can pass safely.

---
  tests/btrfs/310     | 81 +++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/310.out |  2 ++

  2 files changed, 83 insertions(+)

  create mode 100755 tests/btrfs/310
  create mode 100644 tests/btrfs/310.out

tests/btrfs/310 new file +7 -0
new file mode 100755
index 00000000..b514a8bc
@@ -0,0 +1,81 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2024 SUSE Linux Products GmbH. All Rights Reserved.
+#
+# FS QA Test 310
+#
+# Make sure reading on an compressed inline extent is behaving 

correctly

tests/btrfs/310 new file +14 -0
+#
+. ./common/preamble
+_begin_fstest auto quick compress
+
+# Import common functions.
+# . ./common/filter
+
+# real QA test starts here
+
+# Modify as appropriate.
+_supported_fs btrfs
+_require_scratch
+
+# This test require inlined compressed extents creation, and all the

writes

tests/btrfs/310 new file +6 -0
+# are designed for 4K sector size.
+_require_btrfs_inline_extents_creation
+_require_btrfs_support_sectorsize 4096
+
+_fixed_by_kernel_commit e01a83e12604 \
+    "Revert \"btrfs: zstd: fix and simplify the inline extent

decompression\""

tests/btrfs/310 new file +9 -0
+
+# The correct md5 for the correct 4K file filled with "0xcd"
+md5sum_correct="5fed275e7617a806f94c173746a2a723"
+
+workload()
+{
+    local algo="$1"
+
+    echo "=== Testing compression algorithm ${algo} ===" >> 

$seqres.full
+    _scratch_mkfs >> $seqres.full
+    _scratch_mount -o compress=${algo}
+
+    _pwrite_byte 0xcd 0 4k "$SCRATCH_MNT/inline_file" > /dev/null

Quoted 5 lines

+    result=$(_md5_checksum "$SCRATCH_MNT/inline_file")
+    echo "after initial write, md5sum=${result}" >> $seqres.full +    if [ "$result" != "$md5sum_correct" ]; then
+        _fail "initial write results incorrect content for \"$algo\"" +    fi

General rule of thumb is where possible use stdout and compare it with the golden output.

So something like the below shall suffice.

echo "after initial write with alog=$algo"
_md5_checksum "$SCRATCH_MNT/inline_file"

Quoted 7 lines
Quoted 1 line

Also, helps quick debug, when  fails we have the diff.

Nope, for this particular case, golden output is not suitable.

As the workload is dependent on the support compression algorithm, we can not reply on golden output to cover all algorithms.

That's a fair reason not to use golden output.

Looks like you missed more comments below.

Quoted 3 lines
Quoted 3 lines
Quoted 1 line

+    sync

  Generally, we need comments to explain why sync is necessary.

Needs a comment for calling sync.

Quoted 12 lines
Quoted 12 lines
Quoted 11 lines

+
+    $XFS_IO_PROG -c "fiemap -v" $SCRATCH_MNT/inline_file | tail -n 1

Quoted 1 line

$tmp.fiemap

+    cat $tmp.fiemap >> $seqres.full
+    # Make sure we got an inlined compressed file extent. +    # 0x200 means inlined, 0x100 means not block aligned, 0x8 means encoded
+    # (compressed in this case), and 0x1 means the last extent. +    if ! grep -q "0x309" $tmp.fiemap; then
+        rm -f -- $tmp.fiemap
+        _notrun "No compressed inline extent created, maybe subpage?"

Quoted 2 lines
Quoted 2 lines

  workload() is called for each compress algo. If we fail for one of the algo then notrun is not a good option here.

There is no good alternative, lets keep the notrun for now.

Thanks, Anand

Quoted 47 lines
Quoted 47 lines

  IMO, stdout (with filters?) and comparing it with golden output is better.

Quoted 5 lines

+    fi

+    rm -f -- $tmp.fiemap

tests/btrfs/310 new file +1 -0
+

tests/btrfs/310 line 68

Quoted 10 lines
tests/btrfs/310 new file +10 -0
+    # Unmount to clear the page cache.
+    _scratch_cycle_mount
+
+    # For v6.8-rc1 without the revert or the newer fix, this can
+    # crash or lead to incorrect contents for zstd.
+    result=$(_md5_checksum "$SCRATCH_MNT/inline_file")
+    echo "after cycle mount, md5sum=${result}" >> $seqres.full
+    if [ "$result" != "$md5sum_correct" ]; then
+        _fail "read for compressed inline extent failed for \"$algo\""
+    fi

  Here too, same as above, golden output to compare can be done. And remove _fail.

Thanks, Anand

Quoted 21 lines

+    _scratch_unmount
+    _check_scratch_fs

tests/btrfs/310 new file +11 -0
+}
+
+algo_list=($(_btrfs_compression_algos))
+for algo in ${algo_list[@]}; do
+    workload $algo
+done
+
+echo "Silence is golden"
+
+status=0
+exit
tests/btrfs/310.out new file +2 -0
new file mode 100644
index 00000000..7b9eaf78
@@ -0,0 +1,2 @@
+QA output created by 310
+Silence is golden