Skip to contentSkip to conversation

xfs: Check if rt summary/bitmap buffers are logged with correct xfs_buf type

PATCH Superseded Chandan Babu R 3 participants 4 messages Sep 14, 2020 – Sep 15, 2020

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 xfs: Check if rt summary/bitmap buffers are logged with correct xfs_buf type

This commit adds a test to check if growing a real-time device can end up logging an xfs_buf with the "type" subfield of bip->bli_formats->blf_flags set to XFS_BLFT_UNKNOWN_BUF. When this occurs the following call trace is printed on the console,

XFS: Assertion failed: (bip->bli_flags & XFS_BLI_STALE) || (xfs_blft_from_flags(&bip->__bli_format) > XFS_BLFT_UNKNOWN_BUF && xfs_blft_from_flags(&bip->__bli_format) < XFS_BLFT_MAX_BUF), file: fs/xfs/xfs_buf_item.c, line: 331

Call Trace:
 xfs_buf_item_format+0x632/0x680
 ? kmem_alloc_large+0x29/0x90
 ? kmem_alloc+0x70/0x120
 ? xfs_log_commit_cil+0x132/0x940
 xfs_log_commit_cil+0x26f/0x940
 ? xfs_buf_item_init+0x1ad/0x240
 ? xfs_growfs_rt_alloc+0x1fc/0x280
 __xfs_trans_commit+0xac/0x370
 xfs_growfs_rt_alloc+0x1fc/0x280
 xfs_growfs_rt+0x1a0/0x5e0
 xfs_file_ioctl+0x3fd/0xc70
 ? selinux_file_ioctl+0x174/0x220
 ksys_ioctl+0x87/0xc0
 __x64_sys_ioctl+0x16/0x20
 do_syscall_64+0x3e/0x70
 entry_SYSCALL_64_after_hwframe+0x44/0xa9

The kernel patch "xfs: Set xfs_buf type flag when growing summary/bitmap files" is required to fix this issue.

---
 tests/xfs/260     | 52 +++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/260.out |  2 ++
 tests/xfs/group   |  1 +
 3 files changed, 55 insertions(+)
 create mode 100755 tests/xfs/260
 create mode 100644 tests/xfs/260.out
tests/xfs/260 new file +52 -0
new file mode 100755
index 00000000..5fc1a5fc
@@ -0,0 +1,52 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2020 Chandan Babu R.  All Rights Reserved.
+#
+# FS QA Test 260
+#
+# Test to check if growing a real-time device can end up logging an
+# xfs_buf with the "type" subfield of bip->bli_formats->blf_flags set
+# to XFS_BLFT_UNKNOWN_BUF.
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+
+# Modify as appropriate.
+_supported_fs generic
+_supported_os Linux
+_require_realtime
+
+MKFS_OPTIONS="-f -m reflink=0,rmapbt=0 -r rtdev=${SCRATCH_RTDEV},size=10M" \
+ 	    _mkfs_dev $SCRATCH_DEV >> $seqres.full
+_scratch_mount -o rtdev=$SCRATCH_RTDEV
+
+$XFS_GROWFS_PROG $SCRATCH_MNT >> $seqres.full
+
+_scratch_unmount
+
+echo "Silence is golden"
+
+# success, all done
+status=0
+exit
tests/xfs/260.out new file +2 -0
new file mode 100644
index 00000000..18ca517c
@@ -0,0 +1,2 @@
+QA output created by 260
+Silence is golden
tests/xfs/group +1 -0
index ed0d389e..6f30a2e7 100644
@@ -257,6 +257,7 @@
 257 auto quick clone
 258 auto quick clone
 259 auto quick
+260 auto
 261 auto quick quota
 262 dangerous_fuzzers dangerous_scrub dangerous_online_repair
 263 auto quick quota
Signature · 1 line
2.28.0
#1 lore raw
Quoted 67 lines · Chandan Babu R

This commit adds a test to check if growing a real-time device can end up logging an xfs_buf with the "type" subfield of bip->bli_formats->blf_flags set to XFS_BLFT_UNKNOWN_BUF. When this occurs the following call trace is printed on the console,

XFS: Assertion failed: (bip->bli_flags & XFS_BLI_STALE) || (xfs_blft_from_flags(&bip->__bli_format) > XFS_BLFT_UNKNOWN_BUF && xfs_blft_from_flags(&bip->__bli_format) < XFS_BLFT_MAX_BUF), file: fs/xfs/xfs_buf_item.c, line: 331

Call Trace:
 xfs_buf_item_format+0x632/0x680
 ? kmem_alloc_large+0x29/0x90
 ? kmem_alloc+0x70/0x120
 ? xfs_log_commit_cil+0x132/0x940
 xfs_log_commit_cil+0x26f/0x940
 ? xfs_buf_item_init+0x1ad/0x240
 ? xfs_growfs_rt_alloc+0x1fc/0x280
 __xfs_trans_commit+0xac/0x370
 xfs_growfs_rt_alloc+0x1fc/0x280
 xfs_growfs_rt+0x1a0/0x5e0
 xfs_file_ioctl+0x3fd/0xc70
 ? selinux_file_ioctl+0x174/0x220
 ksys_ioctl+0x87/0xc0
 __x64_sys_ioctl+0x16/0x20
 do_syscall_64+0x3e/0x70
 entry_SYSCALL_64_after_hwframe+0x44/0xa9

The kernel patch "xfs: Set xfs_buf type flag when growing summary/bitmap files" is required to fix this issue.

---
 tests/xfs/260     | 52 +++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/260.out |  2 ++
 tests/xfs/group   |  1 +
 3 files changed, 55 insertions(+)
 create mode 100755 tests/xfs/260
 create mode 100644 tests/xfs/260.out
tests/xfs/260 new file +25 -0
new file mode 100755
index 00000000..5fc1a5fc
@@ -0,0 +1,52 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2020 Chandan Babu R.  All Rights Reserved.
+#
+# FS QA Test 260
+#
+# Test to check if growing a real-time device can end up logging an
+# xfs_buf with the "type" subfield of bip->bli_formats->blf_flags set
+# to XFS_BLFT_UNKNOWN_BUF.
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+

tests/xfs/260 line 35

Quoted 10 lines
tests/xfs/260 new file +10 -0
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+
+# Modify as appropriate.
     ^^^^

I think this comment line is useless

tests/xfs/260 line 41

Quoted 6 lines
tests/xfs/260 new file +6 -0
+_supported_fs generic
+_supported_os Linux
+_require_realtime
+
+MKFS_OPTIONS="-f -m reflink=0,rmapbt=0 -r rtdev=${SCRATCH_RTDEV},size=10M" \
+ 	    _mkfs_dev $SCRATCH_DEV >> $seqres.full

Hmm... if you need a sized rtdev, the _scratch_mkfs really can't help that for now. You have to use _mkfs_dev(as you did) or make the helper to support rtdev size:)

I don't know why you need "reflink=0,rmapbt=0", but not old xfsprogs doesn't supports this two options, so you might need _scratch_mkfs_xfs_supported() to check that. If they're not supported, they won't be enabled either. And better to add comment to explain why make sure reflink and rmapbt are disabled.

tests/xfs/260 line 42

Quoted 1 line
tests/xfs/260 new file +1 -0
+_scratch_mount -o rtdev=$SCRATCH_RTDEV

As I known, xfstests deal with SCRATCH_RTDEV things in common/rc _scratch_options() properly, _require_realtime with _scratch_mount are enough, don't need the "-o rtdev=$SCRATCH_RTDEV".

Quoted 23 lines
tests/xfs/260 new file +10 -0
+
+$XFS_GROWFS_PROG $SCRATCH_MNT >> $seqres.full
+
+_scratch_unmount
+
+echo "Silence is golden"
+
+# success, all done
+status=0
+exit
tests/xfs/260.out new file +2 -0
new file mode 100644
index 00000000..18ca517c
@@ -0,0 +1,2 @@
+QA output created by 260
+Silence is golden
tests/xfs/group +0 -0
index ed0d389e..6f30a2e7 100644

tests/xfs/group line 260

Quoted 4 lines
tests/xfs/group +1 -0
@@ -257,6 +257,7 @@
 257 auto quick clone
 258 auto quick clone
 259 auto quick
+260 auto

Better to add 'growfs' group, and if the case is quick enough, 'quick' is acceptable:)

Quoted 6 lines
tests/xfs/group +0 -0
 261 auto quick quota
 262 dangerous_fuzzers dangerous_scrub dangerous_online_repair
 263 auto quick quota
Signature · 1 line
2.28.0
#2 lore raw
Quoted 37 lines · Zorro Lang
Quoted 36 lines · Chandan Babu R

This commit adds a test to check if growing a real-time device can end up logging an xfs_buf with the "type" subfield of bip->bli_formats->blf_flags set to XFS_BLFT_UNKNOWN_BUF. When this occurs the following call trace is printed on the console,

XFS: Assertion failed: (bip->bli_flags & XFS_BLI_STALE) || (xfs_blft_from_flags(&bip->__bli_format) > XFS_BLFT_UNKNOWN_BUF && xfs_blft_from_flags(&bip->__bli_format) < XFS_BLFT_MAX_BUF), file: fs/xfs/xfs_buf_item.c, line: 331

Call Trace:
 xfs_buf_item_format+0x632/0x680
 ? kmem_alloc_large+0x29/0x90
 ? kmem_alloc+0x70/0x120
 ? xfs_log_commit_cil+0x132/0x940
 xfs_log_commit_cil+0x26f/0x940
 ? xfs_buf_item_init+0x1ad/0x240
 ? xfs_growfs_rt_alloc+0x1fc/0x280
 __xfs_trans_commit+0xac/0x370
 xfs_growfs_rt_alloc+0x1fc/0x280
 xfs_growfs_rt+0x1a0/0x5e0
 xfs_file_ioctl+0x3fd/0xc70
 ? selinux_file_ioctl+0x174/0x220
 ksys_ioctl+0x87/0xc0
 __x64_sys_ioctl+0x16/0x20
 do_syscall_64+0x3e/0x70
 entry_SYSCALL_64_after_hwframe+0x44/0xa9

The kernel patch "xfs: Set xfs_buf type flag when growing summary/bitmap files" is required to fix this issue.

---
 tests/xfs/260     | 52 +++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/260.out |  2 ++
 tests/xfs/group   |  1 +
 3 files changed, 55 insertions(+)
 create mode 100755 tests/xfs/260
 create mode 100644 tests/xfs/260.out
Quoted 15 lines
Quoted 6 lines
tests/xfs/260 new file +0 -0
new file mode 100755
index 00000000..5fc1a5fc

tests/xfs/260 line 9

Quoted 9 lines
tests/xfs/260 new file +9 -0
@@ -0,0 +1,52 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2020 Chandan Babu R.  All Rights Reserved.
+#
+# FS QA Test 260
+#
+# Test to check if growing a real-time device can end up logging an
+# xfs_buf with the "type" subfield of bip->bli_formats->blf_flags set
+# to XFS_BLFT_UNKNOWN_BUF.

Please state explicitly that this is a regression test for "xfs: Set xfs_buf type flag when growing summary/bitmap files".

Quoted 29 lines
Quoted 16 lines
tests/xfs/260 new file +16 -0
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+

tests/xfs/260 line 35

Quoted 10 lines
tests/xfs/260 new file +10 -0
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+
+# Modify as appropriate.
     ^^^^

I think this comment line is useless

Quoted 10 lines

tests/xfs/260 line 41

Quoted 6 lines
tests/xfs/260 new file +6 -0
+_supported_fs generic
+_supported_os Linux
+_require_realtime
+
+MKFS_OPTIONS="-f -m reflink=0,rmapbt=0 -r rtdev=${SCRATCH_RTDEV},size=10M" \
+ 	    _mkfs_dev $SCRATCH_DEV >> $seqres.full

Hmm... if you need a sized rtdev, the _scratch_mkfs really can't help that for now. You have to use _mkfs_dev(as you did) or make the helper to support rtdev size:)

Does "_scratch_mkfs -r size=10M" not work here?

Quoted 4 lines

I don't know why you need "reflink=0,rmapbt=0", but not old xfsprogs doesn't supports this two options, so you might need _scratch_mkfs_xfs_supported() to check that. If they're not supported, they won't be enabled either. And better to add comment to explain why make sure reflink and rmapbt are disabled.

That's a bug in mkfs.xfs, which should be fixed by "mkfs: fix reflink/rmap logic w.r.t. realtime devices and crc=0 support".

Quoted 27 lines

tests/xfs/260 line 42

Quoted 1 line
tests/xfs/260 new file +1 -0
+_scratch_mount -o rtdev=$SCRATCH_RTDEV

As I known, xfstests deal with SCRATCH_RTDEV things in common/rc _scratch_options() properly, _require_realtime with _scratch_mount are enough, don't need the "-o rtdev=$SCRATCH_RTDEV".

Quoted 21 lines
tests/xfs/260 new file +10 -0
+
+$XFS_GROWFS_PROG $SCRATCH_MNT >> $seqres.full
+
+_scratch_unmount
+
+echo "Silence is golden"
+
+# success, all done
+status=0
+exit
tests/xfs/260.out new file +2 -0
new file mode 100644
index 00000000..18ca517c
@@ -0,0 +1,2 @@
+QA output created by 260
+Silence is golden
tests/xfs/group +0 -1
index ed0d389e..6f30a2e7 100644
--- a/tests/xfs/group
Quoted 8 lines

tests/xfs/group line 260

Quoted 6 lines

+++ b/tests/xfs/group

tests/xfs/group +1 -0
@@ -257,6 +257,7 @@
 257 auto quick clone
 258 auto quick clone
 259 auto quick
+260 auto

Better to add 'growfs' group, and if the case is quick enough, 'quick' is acceptable:)

And maybe the 'realtime' group.

--D

Quoted 7 lines
Quoted 6 lines
tests/xfs/group +0 -0
 261 auto quick quota
 262 dangerous_fuzzers dangerous_scrub dangerous_online_repair
 263 auto quick quota
Signature · 1 line
2.28.0
#3 lore raw
Quoted 48 lines · IST Darrick J. Wong
Quoted 47 lines · Zorro Lang
Quoted 46 lines · Chandan Babu R

This commit adds a test to check if growing a real-time device can end up logging an xfs_buf with the "type" subfield of bip->bli_formats->blf_flags set to XFS_BLFT_UNKNOWN_BUF. When this occurs the following call trace is printed on the console,

XFS: Assertion failed: (bip->bli_flags & XFS_BLI_STALE) || (xfs_blft_from_flags(&bip->__bli_format) > XFS_BLFT_UNKNOWN_BUF && xfs_blft_from_flags(&bip->__bli_format) < XFS_BLFT_MAX_BUF), file: fs/xfs/xfs_buf_item.c, line: 331

Call Trace:
 xfs_buf_item_format+0x632/0x680
 ? kmem_alloc_large+0x29/0x90
 ? kmem_alloc+0x70/0x120
 ? xfs_log_commit_cil+0x132/0x940
 xfs_log_commit_cil+0x26f/0x940
 ? xfs_buf_item_init+0x1ad/0x240
 ? xfs_growfs_rt_alloc+0x1fc/0x280
 __xfs_trans_commit+0xac/0x370
 xfs_growfs_rt_alloc+0x1fc/0x280
 xfs_growfs_rt+0x1a0/0x5e0
 xfs_file_ioctl+0x3fd/0xc70
 ? selinux_file_ioctl+0x174/0x220
 ksys_ioctl+0x87/0xc0
 __x64_sys_ioctl+0x16/0x20
 do_syscall_64+0x3e/0x70
 entry_SYSCALL_64_after_hwframe+0x44/0xa9

The kernel patch "xfs: Set xfs_buf type flag when growing summary/bitmap files" is required to fix this issue.

---
 tests/xfs/260     | 52 +++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/260.out |  2 ++
 tests/xfs/group   |  1 +
 3 files changed, 55 insertions(+)
 create mode 100755 tests/xfs/260
 create mode 100644 tests/xfs/260.out
tests/xfs/260 new file +4 -0
new file mode 100755
index 00000000..5fc1a5fc
@@ -0,0 +1,52 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2020 Chandan Babu R.  All Rights Reserved.
+#
Quoted 9 lines
Quoted 5 lines

tests/xfs/260 line 9

Quoted 5 lines
tests/xfs/260 new file +5 -0
+# FS QA Test 260
+#
+# Test to check if growing a real-time device can end up logging an
+# xfs_buf with the "type" subfield of bip->bli_formats->blf_flags set
+# to XFS_BLFT_UNKNOWN_BUF.

Please state explicitly that this is a regression test for "xfs: Set xfs_buf type flag when growing summary/bitmap files".

Sure. I will add a description to the comment header.

Quoted 20 lines
Quoted 20 lines
Quoted 20 lines
tests/xfs/260 new file +20 -0
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
Quoted 8 lines
Quoted 8 lines

tests/xfs/260 line 35

Quoted 6 lines
tests/xfs/260 new file +6 -0
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+
+# Modify as appropriate.
     ^^^^

I think this comment line is useless

Sorry, I missed out on removing that.

Quoted 12 lines
Quoted 11 lines

tests/xfs/260 line 41

Quoted 7 lines
tests/xfs/260 new file +6 -0
+_supported_fs generic
+_supported_os Linux
+_require_realtime
+
+MKFS_OPTIONS="-f -m reflink=0,rmapbt=0 -r rtdev=${SCRATCH_RTDEV},size=10M" \
+ 	    _mkfs_dev $SCRATCH_DEV >> $seqres.full

Hmm... if you need a sized rtdev, the _scratch_mkfs really can't help that for now. You have to use _mkfs_dev(as you did) or make the helper to support rtdev size:)

Quoted 1 line

Does "_scratch_mkfs -r size=10M" not work here?

It works! Thanks for the suggestion.

Quoted 8 lines
Quoted 5 lines

I don't know why you need "reflink=0,rmapbt=0", but not old xfsprogs doesn't supports this two options, so you might need _scratch_mkfs_xfs_supported() to check that. If they're not supported, they won't be enabled either. And better to add comment to explain why make sure reflink and rmapbt are disabled.

That's a bug in mkfs.xfs, which should be fixed by "mkfs: fix reflink/rmap logic w.r.t. realtime devices and crc=0 support".

I have replaced this part with just
_scratch_mkfs -r size=10M >> $seqres.full.

IMHO, the tester has to specify reflink=0 explicitly when testing realtime configurations when using xfsprogs which support reflink and don't have "mkfs: fix reflink/rmap logic w.r.t. realtime devices and crc=0 support" patch applied . Otherwise, the call to _scratch_mkfs() inside "check" script will fail.

Quoted 6 lines
Quoted 6 lines

tests/xfs/260 line 42

Quoted 2 lines
tests/xfs/260 new file +1 -0
+_scratch_mount -o rtdev=$SCRATCH_RTDEV

As I known, xfstests deal with SCRATCH_RTDEV things in common/rc _scratch_options() properly, _require_realtime with _scratch_mount are enough, don't need the "-o rtdev=$SCRATCH_RTDEV".

You are right. I will remove it before posting the next version.

Quoted 31 lines
Quoted 22 lines
Quoted 22 lines
tests/xfs/260 new file +10 -0
+
+$XFS_GROWFS_PROG $SCRATCH_MNT >> $seqres.full
+
+_scratch_unmount
+
+echo "Silence is golden"
+
+# success, all done
+status=0
+exit
tests/xfs/260.out new file +2 -0
new file mode 100644
index 00000000..18ca517c
@@ -0,0 +1,2 @@
+QA output created by 260
+Silence is golden
tests/xfs/group +0 -1
index ed0d389e..6f30a2e7 100644
--- a/tests/xfs/group
Quoted 8 lines

tests/xfs/group line 260

Quoted 6 lines

+++ b/tests/xfs/group

tests/xfs/group +1 -0
@@ -257,6 +257,7 @@
 257 auto quick clone
 258 auto quick clone
 259 auto quick
+260 auto

Better to add 'growfs' group, and if the case is quick enough, 'quick' is acceptable:)

Quoted 2 lines

And maybe the 'realtime' group.

Sure, will do.

Darrick and Zorro, Thanks for your review comments.

Quoted 6 lines
Quoted 5 lines
Quoted 4 lines
tests/xfs/group +0 -0
 261 auto quick quota
 262 dangerous_fuzzers dangerous_scrub dangerous_online_repair
 263 auto quick quota
Signature · 1 line
chandan