Skip to contentSkip to conversation

add device removal test v2

PATCH In review Christoph Hellwig 3 participants 6 messages Jul 11, 2023 – Jul 24, 2023

lore
Share & tools
lore
#0 lore raw

Hi all,

this series adds two tests for removing a device underneath the file system. This covers the new shutdown handling series I'm posting.

Changes since v1:
- renumbered the tests
- add a check that the tests are only run on block based file systems

#1 lore raw

PATCH 1/2 generic: add a test for device removal with dirty data

Test the removal of the underlying device when the file system still has dirty data.

---
 tests/generic/730     | 57 +++++++++++++++++++++++++++++++++++++++++++
 tests/generic/730.out |  2 ++
 2 files changed, 59 insertions(+)
 create mode 100755 tests/generic/730
 create mode 100644 tests/generic/730.out
tests/generic/730 new file +57 -0
new file mode 100755
index 00000000..6fba31fc
@@ -0,0 +1,57 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2015 Red Hat Inc. All Rights Reserved.
+# Copyright (c) 2023 Christoph Hellwig
+#
+# Test proper file system shut down when the block device is removed underneath
+# and there is dirty data.
+#
+. ./common/preamble
+_begin_fstest auto quick
+
+_cleanup()
+{
+	cd /
+	$UMOUNT_PROG $SCRATCH_MNT >>$seqres.full 2>&1
+	_put_scsi_debug_dev
+	rm -f $tmp.*
+}
+
+. ./common/filter
+. ./common/scsi_debug
+
+_supported_fs generic
+
+# We don't actually use the test device, but we need a block based fs
+_require_test
+_require_block_device $TEST_DEV
+_require_scsi_debug
+
+physical=`blockdev --getpbsz $SCRATCH_DEV`
+logical=`blockdev --getss $SCRATCH_DEV`
+
+SCSI_DEBUG_DEV=`_get_scsi_debug_dev ${physical:-512} ${logical:-512} 0 300`
+test -b "$SCSI_DEBUG_DEV" || _notrun "Failed to initialize scsi debug device"
+echo "SCSI debug device $SCSI_DEBUG_DEV" >>$seqres.full
+
+_mkfs_dev $SCSI_DEBUG_DEV
+
+_mount $SCSI_DEBUG_DEV $SCRATCH_MNT
+
+# create a test file
+$XFS_IO_PROG -f -c "pwrite 0 1M" $SCRATCH_MNT/testfile >>$seqres.full
+
+# open a file descriptor for reading the file
+exec 3< $SCRATCH_MNT/testfile
+
+# delete the scsi debug device while it still has dirty data
+echo 1 > /sys/block/`_short_dev $SCSI_DEBUG_DEV`/device/delete
+
+# try to read from the file, which should give us -EIO
+cat <&3 > /dev/null
+
+# close the file descriptor to not block unmount
+exec 3<&-
+
+status=0
+exit
tests/generic/730.out new file +2 -0
new file mode 100644
index 00000000..79e96db8
@@ -0,0 +1,2 @@
+QA output created by 730
+cat: -: Input/output error
Signature · 1 line
2.39.2
#2 lore raw

Re: [PATCH 1/2] generic: add a test for device removal with dirty data

Quoted 5 lines · Christoph Hellwig

Test the removal of the underlying device when the file system still has dirty data.

---

Thanks this new version! Sorry for the late review. Most of the cases look good to me, just a bit picky reveiw points as below. (Same to patch 2/2).

Quoted 33 lines
 tests/generic/730     | 57 +++++++++++++++++++++++++++++++++++++++++++
 tests/generic/730.out |  2 ++
 2 files changed, 59 insertions(+)
 create mode 100755 tests/generic/730
 create mode 100644 tests/generic/730.out
tests/generic/730 new file +21 -0
new file mode 100755
index 00000000..6fba31fc
@@ -0,0 +1,57 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2015 Red Hat Inc. All Rights Reserved.
+# Copyright (c) 2023 Christoph Hellwig
+#
+# Test proper file system shut down when the block device is removed underneath
+# and there is dirty data.
+#
+. ./common/preamble
+_begin_fstest auto quick
+
+_cleanup()
+{
+	cd /
+	$UMOUNT_PROG $SCRATCH_MNT >>$seqres.full 2>&1
+	_put_scsi_debug_dev
+	rm -f $tmp.*
+}
+
+. ./common/filter
+. ./common/scsi_debug

tests/generic/730 line 31

Quoted 10 lines
tests/generic/730 new file +10 -0
+
+_supported_fs generic
+
+# We don't actually use the test device, but we need a block based fs
+_require_test
+_require_block_device $TEST_DEV
+_require_scsi_debug
+
+physical=`blockdev --getpbsz $SCRATCH_DEV`
+logical=`blockdev --getss $SCRATCH_DEV`

There's not _require_scratch, so I doubt we should use SCRATCH_DEV directly. How about use TEST_DEV (due to this case calls _require_test) ?

tests/generic/730 line 37

Quoted 6 lines
tests/generic/730 new file +6 -0
+
+SCSI_DEBUG_DEV=`_get_scsi_debug_dev ${physical:-512} ${logical:-512} 0 300`
+test -b "$SCSI_DEBUG_DEV" || _notrun "Failed to initialize scsi debug device"
+echo "SCSI debug device $SCSI_DEBUG_DEV" >>$seqres.full
+
+_mkfs_dev $SCSI_DEBUG_DEV

How about _fail if it fails?

tests/generic/730 line 39

Quoted 2 lines
tests/generic/730 new file +2 -0
+
+_mount $SCSI_DEBUG_DEV $SCRATCH_MNT

I don't know if mount can return non-zero without any output. If you hope to check the return status, you can use run_check() in common/rc.

tests/generic/730 line 48

Quoted 9 lines
tests/generic/730 new file +9 -0
+
+# create a test file
+$XFS_IO_PROG -f -c "pwrite 0 1M" $SCRATCH_MNT/testfile >>$seqres.full
+
+# open a file descriptor for reading the file
+exec 3< $SCRATCH_MNT/testfile
+
+# delete the scsi debug device while it still has dirty data
+echo 1 > /sys/block/`_short_dev $SCSI_DEBUG_DEV`/device/delete

Is this file "/sys/block/_short_dev $SCSI_DEBUG_DEV/device/delete" 100% supported? Or we'd better to _notrun, not report pass.

(BTW, the command and $(command) are both work, but the $() is more recommended in fstests (not mandatory:). If both formats are good, choose $(command).
http://mywiki.wooledge.org/BashFAQ/082)

Thanks,
Zorro

Quoted 20 lines
tests/generic/730 new file +9 -0
+
+# try to read from the file, which should give us -EIO
+cat <&3 > /dev/null
+
+# close the file descriptor to not block unmount
+exec 3<&-
+
+status=0
+exit
tests/generic/730.out new file +2 -0
new file mode 100644
index 00000000..79e96db8
@@ -0,0 +1,2 @@
+QA output created by 730
+cat: -: Input/output error
Signature · 1 line
2.39.2
#3 lore raw

PATCH 1/1 generic: add a test for device removal with dirty data

In reply to Christoph Hellwig

From: Christoph Hellwig <[email protected]>

Test the removal of the underlying device when the file system still has dirty data.

---

Hi Christoph,

I'd like to help this patchset to catch this or next fstests release. So I changed this patch a bit, if you feel good, I'll merge this patch as below. (Same to the 2nd patch).

Change logs:
1) Replace SCRATCH_DEV with TEST_DEV, replace SCRATCH_MNT with SCSI_DEBUG_MNT which is a directory in TEST_DIR, due to we don't call _require_scratch in this case.
2) Call _fail if _mkfs_dev or _mount fails

Thanks,
Zorro

 tests/generic/730     | 59 +++++++++++++++++++++++++++++++++++++++++++
 tests/generic/730.out |  2 ++
 2 files changed, 61 insertions(+)
 create mode 100755 tests/generic/730
 create mode 100644 tests/generic/730.out
tests/generic/730 new file +59 -0
new file mode 100755
index 00000000..7886b38d
@@ -0,0 +1,59 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2015 Red Hat Inc. All Rights Reserved.
+# Copyright (c) 2023 Christoph Hellwig
+#
+# Test proper file system shut down when the block device is removed underneath
+# and there is dirty data.
+#
+. ./common/preamble
+_begin_fstest auto quick
+
+_cleanup()
+{
+	cd /
+	$UMOUNT_PROG $SCSI_DEBUG_MNT >>$seqres.full 2>&1
+	_put_scsi_debug_dev
+	rm -r -f $tmp.*
+}
+
+. ./common/filter
+. ./common/scsi_debug
+
+_supported_fs generic
+
+# We don't actually use the test device, but we need a block based fs
+_require_test
+_require_block_device $TEST_DEV
+_require_scsi_debug
+
+physical=`blockdev --getpbsz $TEST_DEV`
+logical=`blockdev --getss $TEST_DEV`
+
+SCSI_DEBUG_DEV=`_get_scsi_debug_dev ${physical:-512} ${logical:-512} 0 300`
+test -b "$SCSI_DEBUG_DEV" || _notrun "Failed to initialize scsi debug device"
+echo "SCSI debug device $SCSI_DEBUG_DEV" >>$seqres.full
+
+_mkfs_dev $SCSI_DEBUG_DEV || _fail "make $FSTYP on $SCSI_DEBUG_DEV"
+SCSI_DEBUG_MNT="$TEST_DIR/scsi_debug_$seq"
+rm -rf $SCSI_DEBUG_MNT
+mkdir $SCSI_DEBUG_MNT
+run_check _mount $SCSI_DEBUG_DEV $SCSI_DEBUG_MNT
+
+# create a test file
+$XFS_IO_PROG -f -c "pwrite 0 1M" $SCSI_DEBUG_MNT/testfile >>$seqres.full
+
+# open a file descriptor for reading the file
+exec 3< $SCSI_DEBUG_MNT/testfile
+
+# delete the scsi debug device while it still has dirty data
+echo 1 > /sys/block/$(_short_dev $SCSI_DEBUG_DEV)/device/delete
+
+# try to read from the file, which should give us -EIO
+cat <&3 > /dev/null
+
+# close the file descriptor to not block unmount
+exec 3<&-
+
+status=0
+exit
tests/generic/730.out new file +2 -0
new file mode 100644
index 00000000..79e96db8
@@ -0,0 +1,2 @@
+QA output created by 730
+cat: -: Input/output error
Signature · 1 line
2.40.1
#4 lore raw

Re: [PATCH] generic: add a test for device removal with dirty data

Quoted 3 lines · Zorro Lang

I'd like to help this patchset to catch this or next fstests release. So I changed this patch a bit, if you feel good, I'll merge this patch as below. (Same to the 2nd patch).

I've actually got a few other changes locally nad plant to send it out.

Quoted 4 lines

Change logs:
1) Replace SCRATCH_DEV with TEST_DEV, replace SCRATCH_MNT with SCSI_DEBUG_MNT which is a directory in TEST_DIR, due to we don't call _require_scratch in this case.

The SCSI_DEBUG_MNT change is fine. The SCRATCH_DEV stuff I think we don't need at all, after all the sector size doesn't really matter for this test, so I've just dropped it.

Quoted 1 line

2) Call _fail if _mkfs_dev or _mount fails

Shouldn't the mkfs side also use run_check ?

Btw, your cleanups should also go into generic/108 from where I took the scsi_debug boilerplate.

I'll resend later today, also incorporating your changes.

#5 lore raw

PATCH 2/2 generic: add a test for device removal without dirty data

In reply to Christoph Hellwig

Test the removal of the underlying device when the file system still does not have dirty data.

---
 tests/generic/731     | 58 +++++++++++++++++++++++++++++++++++++++++++
 tests/generic/731.out |  2 ++
 2 files changed, 60 insertions(+)
 create mode 100755 tests/generic/731
 create mode 100644 tests/generic/731.out
tests/generic/731 new file +58 -0
new file mode 100755
index 00000000..4a6a2355
@@ -0,0 +1,58 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2015 Red Hat Inc. All Rights Reserved.
+# Copyright (c) 2023 Christoph Hellwig
+#
+# Test proper file system shut down when the block device is removed underneath
+# and it has no dirty data.
+#
+#
+. ./common/preamble
+_begin_fstest auto quick
+
+_cleanup()
+{
+	cd /
+	$UMOUNT_PROG $SCRATCH_MNT >>$seqres.full 2>&1
+	_put_scsi_debug_dev
+	rm -f $tmp.*
+}
+
+. ./common/filter
+. ./common/scsi_debug
+
+# We don't actually use the test device, but we need a block based fs
+_require_test
+_require_block_device $TEST_DEV
+_supported_fs generic
+_require_scsi_debug
+
+physical=`blockdev --getpbsz $SCRATCH_DEV`
+logical=`blockdev --getss $SCRATCH_DEV`
+
+SCSI_DEBUG_DEV=`_get_scsi_debug_dev ${physical:-512} ${logical:-512} 0 300`
+test -b "$SCSI_DEBUG_DEV" || _notrun "Failed to initialize scsi debug device"
+echo "SCSI debug device $SCSI_DEBUG_DEV" >>$seqres.full
+
+_mkfs_dev $SCSI_DEBUG_DEV
+
+_mount $SCSI_DEBUG_DEV $SCRATCH_MNT
+
+# create a test file
+$XFS_IO_PROG -f -c "pwrite 0 1M" -c "fsync" $SCRATCH_MNT/testfile >>$seqres.full
+
+# open a file descriptor for reading the file
+exec 3< $SCRATCH_MNT/testfile
+
+# drop all caches and delete the scsi debug device
+echo 3 > /proc/sys/vm/drop_caches
+echo 1 > /sys/block/`_short_dev $SCSI_DEBUG_DEV`/device/delete
+
+# try to read from the file, which should give us -EIO
+cat <&3 > /dev/null
+
+# close the file descriptor to not block unmount
+exec 3<&-
+
+status=0
+exit
tests/generic/731.out new file +2 -0
new file mode 100644
index 00000000..102c0522
@@ -0,0 +1,2 @@
+QA output created by 731
+cat: -: Input/output error
Signature · 1 line
2.39.2