Skip to contentSkip to conversation

fstests: btrfs/246: add test case to make sure btrfs can create compressed inline extent

PATCH In review Qu Wenruo 3 participants 4 messages Aug 26, 2021 – Aug 29, 2021 Btrfs file system

lore
Share & tools
lore

Qu Wenruo

author
#0 lore raw

PATCH v2 1/1 fstests: btrfs/246: add test case to make sure btrfs can create compressed inline extent

Btrfs has the ability to inline small file extents into its metadata, and such inlined extents can be further compressed if needed.

The new test case is for a regression caused by commit f2165627319f ("btrfs: compression: don't try to compress if we don't have enough pages").

That commit prevents btrfs from creating compressed inline extents, even "-o compress,max_inline=2048" is specified, only uncompressed inline extents can be created.

The test case will make sure that the content of the small file is consistent between cycle mount, then use "btrfs inspect dump-tree" to verify the created extent is both inlined and compressed.

---
Changelog:
v2:
- Also output the sha256sum to make sure the content is consistent

---
 tests/btrfs/246     | 53 +++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/246.out |  5 +++++
 2 files changed, 58 insertions(+)
 create mode 100755 tests/btrfs/246
 create mode 100644 tests/btrfs/246.out
tests/btrfs/246 new file +53 -0
new file mode 100755
index 00000000..e0d8016f
@@ -0,0 +1,53 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2021 SUSE Linux Products GmbH.  All Rights Reserved.
+#
+# FS QA Test 246
+#
+# Make sure btrfs can create compressed inline extents
+#
+. ./common/preamble
+_begin_fstest auto quick compress
+
+# Override the default cleanup function.
+_cleanup()
+{
+	cd /
+	rm -r -f $tmp.*
+}
+
+# Import common functions.
+. ./common/filter
+# For __populate_find_inode()
+. ./common/populate
+
+# real QA test starts here
+
+# Modify as appropriate.
+_supported_fs btrfs
+_require_scratch
+
+_scratch_mkfs > /dev/null
+_scratch_mount -o compress,max_inline=2048
+
+# This should create compressed inline extent
+$XFS_IO_PROG -f -c "pwrite 0 2048" $SCRATCH_MNT/foobar > /dev/null
+ino=$(__populate_find_inode $SCRATCH_MNT/foobar)
+echo "sha256sum before mount cycle"
+sha256sum $SCRATCH_MNT/foobar | _filter_scratch
+_scratch_cycle_mount
+echo "sha256sum after mount cycle"
+sha256sum $SCRATCH_MNT/foobar | _filter_scratch
+_scratch_unmount
+
+$BTRFS_UTIL_PROG inspect dump-tree -t 5 $SCRATCH_DEV | \
+	grep "($ino EXTENT_DATA 0" -A2 > $tmp.dump-tree
+echo "dump tree result for ino $ino:" >> $seqres.full
+cat $tmp.dump-tree >> $seqres.full
+
+grep -q "inline extent" $tmp.dump-tree || echo "no inline extent found"
+grep -q "compression 1" $tmp.dump-tree || echo "no compressed extent found"
+
+# success, all done
+status=0
+exit
tests/btrfs/246.out new file +5 -0
new file mode 100644
index 00000000..3908cc50
@@ -0,0 +1,5 @@
+QA output created by 246
+sha256sum before mount cycle
+0ca3bfdeda1ef5036bfa5dad078a9f15724e79cf296bd4388cf786bfaf4195d0  SCRATCH_MNT/foobar
+sha256sum after mount cycle
+0ca3bfdeda1ef5036bfa5dad078a9f15724e79cf296bd4388cf786bfaf4195d0  SCRATCH_MNT/foobar
Signature · 1 line
2.31.1
#1 lore raw
Quoted 11 lines · Qu Wenruo

Btrfs has the ability to inline small file extents into its metadata, and such inlined extents can be further compressed if needed.

The new test case is for a regression caused by commit f2165627319f ("btrfs: compression: don't try to compress if we don't have enough pages").

That commit prevents btrfs from creating compressed inline extents, even "-o compress,max_inline=2048" is specified, only uncompressed inline extents can be created.

Quoted 5 lines

The test case will make sure that the content of the small file is consistent between cycle mount, then use "btrfs inspect dump-tree" to verify the created extent is both inlined and compressed.

Is there a proposed fix available that could be referenced in the commit log?

Quoted 29 lines

---
Changelog:
v2:
- Also output the sha256sum to make sure the content is consistent

---
 tests/btrfs/246     | 53 +++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/246.out |  5 +++++
 2 files changed, 58 insertions(+)
 create mode 100755 tests/btrfs/246
 create mode 100644 tests/btrfs/246.out
tests/btrfs/246 new file +12 -0
new file mode 100755
index 00000000..e0d8016f
@@ -0,0 +1,53 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2021 SUSE Linux Products GmbH.  All Rights Reserved.
+#
+# FS QA Test 246
+#
+# Make sure btrfs can create compressed inline extents
+#
+. ./common/preamble
+_begin_fstest auto quick compress
+
+# Override the default cleanup function.

tests/btrfs/246 line 22

Quoted 10 lines
tests/btrfs/246 new file +10 -0
+_cleanup()
+{
+	cd /
+	rm -r -f $tmp.*
+}
+
+# Import common functions.
+. ./common/filter
+# For __populate_find_inode()
+. ./common/populate

This function starts with double underscore, I take it as a 'private' function in common/populate. But all it does is returning the inode number of the given file, I think we could just open-code it in this test as

ino=$(stat -c %i $SCRATCH_MNT/foobar)

Otherwise test looks fine to me.

Thanks,
Eryu

Quoted 44 lines
tests/btrfs/246 new file +31 -0
+
+# real QA test starts here
+
+# Modify as appropriate.
+_supported_fs btrfs
+_require_scratch
+
+_scratch_mkfs > /dev/null
+_scratch_mount -o compress,max_inline=2048
+
+# This should create compressed inline extent
+$XFS_IO_PROG -f -c "pwrite 0 2048" $SCRATCH_MNT/foobar > /dev/null
+ino=$(__populate_find_inode $SCRATCH_MNT/foobar)
+echo "sha256sum before mount cycle"
+sha256sum $SCRATCH_MNT/foobar | _filter_scratch
+_scratch_cycle_mount
+echo "sha256sum after mount cycle"
+sha256sum $SCRATCH_MNT/foobar | _filter_scratch
+_scratch_unmount
+
+$BTRFS_UTIL_PROG inspect dump-tree -t 5 $SCRATCH_DEV | \
+	grep "($ino EXTENT_DATA 0" -A2 > $tmp.dump-tree
+echo "dump tree result for ino $ino:" >> $seqres.full
+cat $tmp.dump-tree >> $seqres.full
+
+grep -q "inline extent" $tmp.dump-tree || echo "no inline extent found"
+grep -q "compression 1" $tmp.dump-tree || echo "no compressed extent found"
+
+# success, all done
+status=0
+exit
tests/btrfs/246.out new file +5 -0
new file mode 100644
index 00000000..3908cc50
@@ -0,0 +1,5 @@
+QA output created by 246
+sha256sum before mount cycle
+0ca3bfdeda1ef5036bfa5dad078a9f15724e79cf296bd4388cf786bfaf4195d0  SCRATCH_MNT/foobar
+sha256sum after mount cycle
+0ca3bfdeda1ef5036bfa5dad078a9f15724e79cf296bd4388cf786bfaf4195d0  SCRATCH_MNT/foobar
Signature · 1 line
2.31.1
author
#2 lore raw
Quoted 12 lines · Eryu Guan
Quoted 11 lines · Qu Wenruo

Btrfs has the ability to inline small file extents into its metadata, and such inlined extents can be further compressed if needed.

The new test case is for a regression caused by commit f2165627319f ("btrfs: compression: don't try to compress if we don't have enough pages").

That commit prevents btrfs from creating compressed inline extents, even "-o compress,max_inline=2048" is specified, only uncompressed inline extents can be created.

Quoted 8 lines
Quoted 5 lines

The test case will make sure that the content of the small file is consistent between cycle mount, then use "btrfs inspect dump-tree" to verify the created extent is both inlined and compressed.

Is there a proposed fix available that could be referenced in the commit log?

The upstream commit is 4e9655763b82 ("Revert "btrfs: compression: don't try to compress if we don't have enough pages""), which is merged after I submitted the patch.

Quoted 46 lines
Quoted 30 lines

---
Changelog:
v2:
- Also output the sha256sum to make sure the content is consistent

---
  tests/btrfs/246     | 53 +++++++++++++++++++++++++++++++++++++++++++++
  tests/btrfs/246.out |  5 +++++
  2 files changed, 58 insertions(+)

create mode 100755 tests/btrfs/246
create mode 100644 tests/btrfs/246.out

tests/btrfs/246 new file +12 -0
new file mode 100755
index 00000000..e0d8016f
@@ -0,0 +1,53 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2021 SUSE Linux Products GmbH.  All Rights Reserved.
+#
+# FS QA Test 246
+#
+# Make sure btrfs can create compressed inline extents
+#
+. ./common/preamble
+_begin_fstest auto quick compress
+
+# Override the default cleanup function.

tests/btrfs/246 line 22

Quoted 10 lines
tests/btrfs/246 new file +10 -0
+_cleanup()
+{
+	cd /
+	rm -r -f $tmp.*
+}
+
+# Import common functions.
+. ./common/filter
+# For __populate_find_inode()
+. ./common/populate

This function starts with double underscore, I take it as a 'private' function in common/populate. But all it does is returning the inode number of the given file, I think we could just open-code it in this test as

Quoted 3 lines

ino=$(stat -c %i $SCRATCH_MNT/foobar)

Otherwise test looks fine to me.

Mind me to send an update to include the fix in commit message and use the local ino helper?

Thanks,
Qu

Quoted 49 lines

Thanks,
Eryu

Quoted 44 lines
tests/btrfs/246 new file +31 -0
+
+# real QA test starts here
+
+# Modify as appropriate.
+_supported_fs btrfs
+_require_scratch
+
+_scratch_mkfs > /dev/null
+_scratch_mount -o compress,max_inline=2048
+
+# This should create compressed inline extent
+$XFS_IO_PROG -f -c "pwrite 0 2048" $SCRATCH_MNT/foobar > /dev/null
+ino=$(__populate_find_inode $SCRATCH_MNT/foobar)
+echo "sha256sum before mount cycle"
+sha256sum $SCRATCH_MNT/foobar | _filter_scratch
+_scratch_cycle_mount
+echo "sha256sum after mount cycle"
+sha256sum $SCRATCH_MNT/foobar | _filter_scratch
+_scratch_unmount
+
+$BTRFS_UTIL_PROG inspect dump-tree -t 5 $SCRATCH_DEV | \
+	grep "($ino EXTENT_DATA 0" -A2 > $tmp.dump-tree
+echo "dump tree result for ino $ino:" >> $seqres.full
+cat $tmp.dump-tree >> $seqres.full
+
+grep -q "inline extent" $tmp.dump-tree || echo "no inline extent found"
+grep -q "compression 1" $tmp.dump-tree || echo "no compressed extent found"
+
+# success, all done
+status=0
+exit
tests/btrfs/246.out new file +5 -0
new file mode 100644
index 00000000..3908cc50
@@ -0,0 +1,5 @@
+QA output created by 246
+sha256sum before mount cycle
+0ca3bfdeda1ef5036bfa5dad078a9f15724e79cf296bd4388cf786bfaf4195d0  SCRATCH_MNT/foobar
+sha256sum after mount cycle
+0ca3bfdeda1ef5036bfa5dad078a9f15724e79cf296bd4388cf786bfaf4195d0  SCRATCH_MNT/foobar
Signature · 1 line
2.31.1
#3 lore raw
Quoted 19 lines · Qu Wenruo
Quoted 16 lines · Eryu Guan
Quoted 15 lines · Qu Wenruo

Btrfs has the ability to inline small file extents into its metadata, and such inlined extents can be further compressed if needed.

The new test case is for a regression caused by commit f2165627319f ("btrfs: compression: don't try to compress if we don't have enough pages").

That commit prevents btrfs from creating compressed inline extents, even "-o compress,max_inline=2048" is specified, only uncompressed inline extents can be created.

The test case will make sure that the content of the small file is consistent between cycle mount, then use "btrfs inspect dump-tree" to verify the created extent is both inlined and compressed.

Quoted 8 lines
Quoted 4 lines
Quoted 1 line

Is there a proposed fix available that could be referenced in the commit log?

The upstream commit is 4e9655763b82 ("Revert "btrfs: compression: don't try to compress if we don't have enough pages""), which is merged after I submitted the patch.

Ok. It also could be helpful to just include the proposed patch title if it's not merged yet.

Quoted 47 lines
Quoted 47 lines
Quoted 31 lines

---
Changelog:
v2:
- Also output the sha256sum to make sure the content is consistent

---
  tests/btrfs/246     | 53 +++++++++++++++++++++++++++++++++++++++++++++
  tests/btrfs/246.out |  5 +++++
  2 files changed, 58 insertions(+)

create mode 100755 tests/btrfs/246
create mode 100644 tests/btrfs/246.out

tests/btrfs/246 new file +12 -0
new file mode 100755
index 00000000..e0d8016f
@@ -0,0 +1,53 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2021 SUSE Linux Products GmbH.  All Rights Reserved.
+#
+# FS QA Test 246
+#
+# Make sure btrfs can create compressed inline extents
+#
+. ./common/preamble
+_begin_fstest auto quick compress
+
+# Override the default cleanup function.

tests/btrfs/246 line 22

Quoted 10 lines
tests/btrfs/246 new file +10 -0
+_cleanup()
+{
+	cd /
+	rm -r -f $tmp.*
+}
+
+# Import common functions.
+. ./common/filter
+# For __populate_find_inode()
+. ./common/populate

This function starts with double underscore, I take it as a 'private' function in common/populate. But all it does is returning the inode number of the given file, I think we could just open-code it in this test as

Quoted 6 lines
Quoted 3 lines

ino=$(stat -c %i $SCRATCH_MNT/foobar)

Otherwise test looks fine to me.

Mind me to send an update to include the fix in commit message and use the local ino helper?

You're responding quickly, I haven't finalized this week's update yet, so I can add the fix commit info and remove __populate_find_inode on commit :)

Thanks,
Eryu