Skip to contentSkip to conversation

fstests: btrfs: Validate that balance and qgroups work correctly when balance needs to be resumed on mount

PATCH In review Qu Wenruo 4 participants 4 messages May 23, 2019 – May 24, 2019 Btrfs file system

lore
Share & tools
lore

Qu Wenruo

author
#0 lore raw

PATCH v2 1/1 fstests: btrfs: Validate that balance and qgroups work correctly when balance needs to be resumed on mount

There are two regressions related to balance resume:
- Kernel NULL pointer dereference at mount time Introduced in v5.0
- Kernel BUG_ON() just after mount Introduced in v5.1

The kernel fixes are:
"btrfs: qgroup: Check if @bg is NULL to avoid NULL pointer
dereference"
"btrfs: reloc: Also queue orphan reloc tree for cleanup to
avoid BUG_ON()"

---
changelog:
v2:
- Subject change to describe the test case in a more generic way
- Update commit message and comment to avoid ambitious/confusing words
- Add to 'balance' and 'qgroup' groups

---
 tests/btrfs/188     | 92 +++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/188.out |  2 +
 tests/btrfs/group   |  1 +
 3 files changed, 95 insertions(+)
 create mode 100755 tests/btrfs/188
 create mode 100644 tests/btrfs/188.out
tests/btrfs/188 new file +92 -0
new file mode 100755
index 00000000..e12db87c
@@ -0,0 +1,92 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2019 SUSE Linux Products GmbH.  All Rights Reserved.
+#
+# FS QA Test 188
+#
+# A general test to validate that balance and qgroups work correctly when
+# balance needs to be resumed on mount.
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/dmlogwrites
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+
+# Modify as appropriate.
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+# and we need extra device as log device
+_require_log_writes
+
+nr_files=512				# enough metadata to bump tree height
+file_size=2048				# small enough to be inlined
+
+_log_writes_init $SCRATCH_DEV
+_log_writes_mkfs >> $seqres.full 2>&1
+
+_log_writes_mount
+$BTRFS_UTIL_PROG quota enable $SCRATCH_MNT >> $seqres.full
+$BTRFS_UTIL_PROG quota rescan -w $SCRATCH_MNT >> $seqres.full
+
+# Create enough metadata for later balance
+for ((i = 0; i < $nr_files; i++)); do
+	_pwrite_byte 0xcd 0 $file_size $SCRATCH_MNT/file_$i > /dev/null
+done
+
+# Flush delalloc so that balance has work to do.
+sync
+
+# Balance metadata so we will have at least one transaction committed with
+# valid reloc tree, and hopefully another commit with orphan reloc tree.
+$BTRFS_UTIL_PROG balance start -f -m $SCRATCH_MNT >> $seqres.full
+_log_writes_unmount
+_log_writes_remove
+
+cur=$(_log_writes_find_next_fua 0)
+echo "cur=$cur" >> $seqres.full
+while [ ! -z "$cur" ]; do
+	_log_writes_replay_log_range $cur $SCRATCH_DEV >> $seqref.full
+
+	# Test that no crashes happen or any other kind of failure.
+	_scratch_mount
+	_scratch_unmount
+
+	# Don't trigger fsck here, as relocation get paused,
+	# at that transistent state, qgroup number may differ
+	# and cause false alert.
+
+	prev=$cur
+	cur=$(_log_writes_find_next_fua $(($cur + 1)))
+	[ -z "$cur" ] && break
+done
+
+# Now the fs has finished its balance and qgroup should be consistent.
+# Fstest will automatically check the fs and btrfs check will report
+# any qgroup inconsistent if something went wrong.
+
+echo "Silence is golden"
+
+# success, all done
+status=0
+exit
tests/btrfs/188.out new file +2 -0
new file mode 100644
index 00000000..6f23fda0
@@ -0,0 +1,2 @@
+QA output created by 188
+Silence is golden
tests/btrfs/group +1 -0
index 44ee0dd9..cfad878f 100644
@@ -190,3 +190,4 @@
 185 volume
 186 auto quick send volume
 187 auto send dedupe clone balance
+188 auto quick replay balance qgroup
Signature · 1 line
2.21.0
Reviewed-by on v2 1/1
#1 lore raw
Quoted 7 lines · Qu Wenruo

There are two regressions related to balance resume:
- Kernel NULL pointer dereference at mount time Introduced in v5.0
- Kernel BUG_ON() just after mount Introduced in v5.1

Quoted 7 lines

The kernel fixes are:
"btrfs: qgroup: Check if @bg is NULL to avoid NULL pointer
dereference"
"btrfs: reloc: Also queue orphan reloc tree for cleanup to
avoid BUG_ON()"

Looks good, thanks.

Quoted 132 lines

---
changelog:
v2:
- Subject change to describe the test case in a more generic way
- Update commit message and comment to avoid ambitious/confusing words
- Add to 'balance' and 'qgroup' groups

---
 tests/btrfs/188     | 92 +++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/188.out |  2 +
 tests/btrfs/group   |  1 +
 3 files changed, 95 insertions(+)
 create mode 100755 tests/btrfs/188
 create mode 100644 tests/btrfs/188.out
tests/btrfs/188 new file +92 -0
new file mode 100755
index 00000000..e12db87c
@@ -0,0 +1,92 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2019 SUSE Linux Products GmbH.  All Rights Reserved.
+#
+# FS QA Test 188
+#
+# A general test to validate that balance and qgroups work correctly when
+# balance needs to be resumed on mount.
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1       # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+       cd /
+       rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/dmlogwrites
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+
+# Modify as appropriate.
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+# and we need extra device as log device
+_require_log_writes
+
+nr_files=512                           # enough metadata to bump tree height
+file_size=2048                         # small enough to be inlined
+
+_log_writes_init $SCRATCH_DEV
+_log_writes_mkfs >> $seqres.full 2>&1
+
+_log_writes_mount
+$BTRFS_UTIL_PROG quota enable $SCRATCH_MNT >> $seqres.full
+$BTRFS_UTIL_PROG quota rescan -w $SCRATCH_MNT >> $seqres.full
+
+# Create enough metadata for later balance
+for ((i = 0; i < $nr_files; i++)); do
+       _pwrite_byte 0xcd 0 $file_size $SCRATCH_MNT/file_$i > /dev/null
+done
+
+# Flush delalloc so that balance has work to do.
+sync
+
+# Balance metadata so we will have at least one transaction committed with
+# valid reloc tree, and hopefully another commit with orphan reloc tree.
+$BTRFS_UTIL_PROG balance start -f -m $SCRATCH_MNT >> $seqres.full
+_log_writes_unmount
+_log_writes_remove
+
+cur=$(_log_writes_find_next_fua 0)
+echo "cur=$cur" >> $seqres.full
+while [ ! -z "$cur" ]; do
+       _log_writes_replay_log_range $cur $SCRATCH_DEV >> $seqref.full
+
+       # Test that no crashes happen or any other kind of failure.
+       _scratch_mount
+       _scratch_unmount
+
+       # Don't trigger fsck here, as relocation get paused,
+       # at that transistent state, qgroup number may differ
+       # and cause false alert.
+
+       prev=$cur
+       cur=$(_log_writes_find_next_fua $(($cur + 1)))
+       [ -z "$cur" ] && break
+done
+
+# Now the fs has finished its balance and qgroup should be consistent.
+# Fstest will automatically check the fs and btrfs check will report
+# any qgroup inconsistent if something went wrong.
+
+echo "Silence is golden"
+
+# success, all done
+status=0
+exit
tests/btrfs/188.out new file +2 -0
new file mode 100644
index 00000000..6f23fda0
@@ -0,0 +1,2 @@
+QA output created by 188
+Silence is golden
tests/btrfs/group +1 -1
index 44ee0dd9..cfad878f 100644
@@ -190,3 +190,4 @@
 185 volume
 186 auto quick send volume
 187 auto send dedupe clone balance
+188 auto quick replay balance qgroup
--

2.21.0

Signature · 3 lines
Filipe David Manana,

“Whether you think you can, or you think you can't — you're right.”
#2 lore raw
In reply to Qu Wenruo
Quoted 6 lines · Qu Wenruo

There are two regressions related to balance resume:
- Kernel NULL pointer dereference at mount time Introduced in v5.0
- Kernel BUG_ON() just after mount Introduced in v5.1

Quoted 7 lines

The kernel fixes are:
"btrfs: qgroup: Check if @bg is NULL to avoid NULL pointer
dereference"
"btrfs: reloc: Also queue orphan reloc tree for cleanup to
avoid BUG_ON()"

The summary seems a bit long, does this look like a proper summary to describe the test?

"btrfs: resume balance on mount with qgroup"

If so, I'll use this summary on commit.

Thanks,
Eryu

#3 lore raw
Quoted 13 lines · Eryu Guan
Quoted 12 lines · Qu Wenruo

There are two regressions related to balance resume:
- Kernel NULL pointer dereference at mount time Introduced in v5.0
- Kernel BUG_ON() just after mount Introduced in v5.1

The kernel fixes are:
"btrfs: qgroup: Check if @bg is NULL to avoid NULL pointer
dereference"
"btrfs: reloc: Also queue orphan reloc tree for cleanup to
avoid BUG_ON()"

Quoted 6 lines
Quoted 1 line

The summary seems a bit long, does this look like a proper summary to describe the test?

"btrfs: resume balance on mount with qgroup"

Wow, that's short.

I'm OK with that.

Thanks,
Qu

Quoted 6 lines

If so, I'll use this summary on commit.

Thanks,
Eryu