Skip to contentSkip to conversation

generic: test mapped write after shutdown and failed writeback

PATCH In review Brian Foster 3 participants 4 messages Feb 10, 2021 – Mar 2, 2021

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 generic: test mapped write after shutdown and failed writeback

XFS has a regression where it failed to check shutdown status in the write fault path. This produced an iomap warning if the page happened to recently fail a writeback attempt because writeback failure can clear Uptodate status on the page. Add a test for this scenario to help ensure mapped write failures are handled as expected in the event of filesystem shutdown.

---

Note that this test currently fails on XFS. The fix is posted for review on linux-xfs:

https://lore.kernel.org/linux-xfs/[email protected]/

Brian

 tests/generic/999     | 45 +++++++++++++++++++++++++++++++++++++++++++
 tests/generic/999.out |  4 ++++
 tests/generic/group   |  1 +
 3 files changed, 50 insertions(+)
 create mode 100755 tests/generic/999
 create mode 100644 tests/generic/999.out
tests/generic/999 new file +45 -0
new file mode 100755
index 00000000..5e5408e7
@@ -0,0 +1,45 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-only
+# Copyright 2021 Red Hat, Inc.
+#
+# FS QA Test No. 999
+#
+# Test a write fault scenario on a shutdown fs.
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	rm -f $tmp.*
+}
+
+. ./common/rc
+. ./common/filter
+
+rm -f $seqres.full
+
+_supported_fs generic
+_require_scratch_nocheck
+_require_scratch_shutdown
+
+_scratch_mkfs &>> $seqres.full
+_scratch_mount
+
+# XFS had a regression where it failed to check shutdown status in the fault
+# path. This produced an iomap warning because writeback failure clears Uptodate
+# status on the page.
+file=$SCRATCH_MNT/file
+$XFS_IO_PROG -fc "pwrite 0 4k" -c fsync $file | _filter_xfs_io
+$XFS_IO_PROG -x -c "mmap 0 4k" -c "mwrite 0 4k" -c shutdown -c fsync \
+	-c "mwrite 0 4k" $file | _filter_xfs_io
+
+# success, all done
+status=0
+exit
tests/generic/999.out new file +4 -0
new file mode 100644
index 00000000..f55569ff
@@ -0,0 +1,4 @@
+QA output created by 999
+wrote 4096/4096 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+fsync: Input/output error
tests/generic/group +1 -0
index b10fdea4..edd54ce5 100644
@@ -625,3 +625,4 @@
 620 auto mount quick
 621 auto quick encrypt
 622 auto shutdown metadata atime
+999 auto quick shutdown
Signature · 1 line
2.26.2
Reviewed-by on v1 1/1
#1 lore raw
Quoted 8 lines · Brian Foster

XFS has a regression where it failed to check shutdown status in the write fault path. This produced an iomap warning if the page happened to recently fail a writeback attempt because writeback failure can clear Uptodate status on the page. Add a test for this scenario to help ensure mapped write failures are handled as expected in the event of filesystem shutdown.

This looks reasonable to me, I have run the same xfs_io tests when looking at this behavior and they do provoke it.

This could maybe run on $TEST? But I don't really care much either way.

I spot-checked this on ext4 and it passes.

Quoted 87 lines

---

Note that this test currently fails on XFS. The fix is posted for review on linux-xfs:

https://lore.kernel.org/linux-xfs/[email protected]/

Brian

 tests/generic/999     | 45 +++++++++++++++++++++++++++++++++++++++++++
 tests/generic/999.out |  4 ++++
 tests/generic/group   |  1 +
 3 files changed, 50 insertions(+)
 create mode 100755 tests/generic/999
 create mode 100644 tests/generic/999.out
tests/generic/999 new file +45 -0
new file mode 100755
index 00000000..5e5408e7
@@ -0,0 +1,45 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-only
+# Copyright 2021 Red Hat, Inc.
+#
+# FS QA Test No. 999
+#
+# Test a write fault scenario on a shutdown fs.
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	rm -f $tmp.*
+}
+
+. ./common/rc
+. ./common/filter
+
+rm -f $seqres.full
+
+_supported_fs generic
+_require_scratch_nocheck
+_require_scratch_shutdown
+
+_scratch_mkfs &>> $seqres.full
+_scratch_mount
+
+# XFS had a regression where it failed to check shutdown status in the fault
+# path. This produced an iomap warning because writeback failure clears Uptodate
+# status on the page.
+file=$SCRATCH_MNT/file
+$XFS_IO_PROG -fc "pwrite 0 4k" -c fsync $file | _filter_xfs_io
+$XFS_IO_PROG -x -c "mmap 0 4k" -c "mwrite 0 4k" -c shutdown -c fsync \
+	-c "mwrite 0 4k" $file | _filter_xfs_io
+
+# success, all done
+status=0
+exit
tests/generic/999.out new file +4 -0
new file mode 100644
index 00000000..f55569ff
@@ -0,0 +1,4 @@
+QA output created by 999
+wrote 4096/4096 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+fsync: Input/output error
tests/generic/group +1 -0
index b10fdea4..edd54ce5 100644
@@ -625,3 +625,4 @@
 620 auto mount quick
 621 auto quick encrypt
 622 auto shutdown metadata atime
+999 auto quick shutdown
#2 lore raw
Quoted 8 lines · Eric Sandeen
Quoted 7 lines · Brian Foster

XFS has a regression where it failed to check shutdown status in the write fault path. This produced an iomap warning if the page happened to recently fail a writeback attempt because writeback failure can clear Uptodate status on the page. Add a test for this scenario to help ensure mapped write failures are handled as expected in the event of filesystem shutdown.

Quoted 8 lines
Quoted 1 line

This looks reasonable to me, I have run the same xfs_io tests when looking at this behavior and they do provoke it.

This could maybe run on $TEST? But I don't really care much either way.

I suppose that would be possible, but I'd prefer not to shut down the test device intentionally. I don't _think_ we currently do that anywhere. AFAICT, there's only a handful of open-coded shutdowns and most others use _scratch_shutdown...

Quoted 4 lines

I spot-checked this on ext4 and it passes.

Thanks.

Brian

Quoted 89 lines
Quoted 88 lines

---

Note that this test currently fails on XFS. The fix is posted for review on linux-xfs:

https://lore.kernel.org/linux-xfs/[email protected]/

Brian

 tests/generic/999     | 45 +++++++++++++++++++++++++++++++++++++++++++
 tests/generic/999.out |  4 ++++
 tests/generic/group   |  1 +
 3 files changed, 50 insertions(+)
 create mode 100755 tests/generic/999
 create mode 100644 tests/generic/999.out
tests/generic/999 new file +45 -0
new file mode 100755
index 00000000..5e5408e7
@@ -0,0 +1,45 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-only
+# Copyright 2021 Red Hat, Inc.
+#
+# FS QA Test No. 999
+#
+# Test a write fault scenario on a shutdown fs.
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	rm -f $tmp.*
+}
+
+. ./common/rc
+. ./common/filter
+
+rm -f $seqres.full
+
+_supported_fs generic
+_require_scratch_nocheck
+_require_scratch_shutdown
+
+_scratch_mkfs &>> $seqres.full
+_scratch_mount
+
+# XFS had a regression where it failed to check shutdown status in the fault
+# path. This produced an iomap warning because writeback failure clears Uptodate
+# status on the page.
+file=$SCRATCH_MNT/file
+$XFS_IO_PROG -fc "pwrite 0 4k" -c fsync $file | _filter_xfs_io
+$XFS_IO_PROG -x -c "mmap 0 4k" -c "mwrite 0 4k" -c shutdown -c fsync \
+	-c "mwrite 0 4k" $file | _filter_xfs_io
+
+# success, all done
+status=0
+exit
tests/generic/999.out new file +4 -0
new file mode 100644
index 00000000..f55569ff
@@ -0,0 +1,4 @@
+QA output created by 999
+wrote 4096/4096 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+fsync: Input/output error
tests/generic/group +1 -0
index b10fdea4..edd54ce5 100644
@@ -625,3 +625,4 @@
 620 auto mount quick
 621 auto quick encrypt
 622 auto shutdown metadata atime
+999 auto quick shutdown
#3 lore raw
In reply to Brian Foster
Quoted 61 lines · Brian Foster

XFS has a regression where it failed to check shutdown status in the write fault path. This produced an iomap warning if the page happened to recently fail a writeback attempt because writeback failure can clear Uptodate status on the page. Add a test for this scenario to help ensure mapped write failures are handled as expected in the event of filesystem shutdown.

---

Note that this test currently fails on XFS. The fix is posted for review on linux-xfs:

https://lore.kernel.org/linux-xfs/[email protected]/

Brian

 tests/generic/999     | 45 +++++++++++++++++++++++++++++++++++++++++++
 tests/generic/999.out |  4 ++++
 tests/generic/group   |  1 +
 3 files changed, 50 insertions(+)
 create mode 100755 tests/generic/999
 create mode 100644 tests/generic/999.out
tests/generic/999 new file +31 -0
new file mode 100755
index 00000000..5e5408e7
@@ -0,0 +1,45 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-only
+# Copyright 2021 Red Hat, Inc.
+#
+# FS QA Test No. 999
+#
+# Test a write fault scenario on a shutdown fs.
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	rm -f $tmp.*
+}
+
+. ./common/rc
+. ./common/filter
+
+rm -f $seqres.full
+
+_supported_fs generic
+_require_scratch_nocheck
+_require_scratch_shutdown
+

tests/generic/999 line 41

Quoted 10 lines
tests/generic/999 new file +10 -0
+_scratch_mkfs &>> $seqres.full
+_scratch_mount
+
+# XFS had a regression where it failed to check shutdown status in the fault
+# path. This produced an iomap warning because writeback failure clears Uptodate
+# status on the page.
+file=$SCRATCH_MNT/file
+$XFS_IO_PROG -fc "pwrite 0 4k" -c fsync $file | _filter_xfs_io
+$XFS_IO_PROG -x -c "mmap 0 4k" -c "mwrite 0 4k" -c shutdown -c fsync \
+	-c "mwrite 0 4k" $file | _filter_xfs_io

On my fstests (which sets unlimited core dumps) this test generates a false negative because ./check trips on the core dump that the dying xfs_io creates. Assuming that we're /supposed/ to segfault here, I'll send a patch to ulimit -c0.

--D

Quoted 26 lines
tests/generic/999 new file +4 -0
+
+# success, all done
+status=0
+exit
tests/generic/999.out new file +4 -0
new file mode 100644
index 00000000..f55569ff
@@ -0,0 +1,4 @@
+QA output created by 999
+wrote 4096/4096 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+fsync: Input/output error
tests/generic/group +1 -0
index b10fdea4..edd54ce5 100644
@@ -625,3 +625,4 @@
 620 auto mount quick
 621 auto quick encrypt
 622 auto shutdown metadata atime
+999 auto quick shutdown
Signature · 1 line
2.26.2