Skip to contentSkip to conversation

fstests: btrfs/219 add a test for some disk caching usecases

PATCH In review Josef Bacik 5 participants 6 messages Sep 2, 2020 – Sep 13, 2020 Btrfs file system

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 fstests: btrfs/219 add a test for some disk caching usecases

This is a test to check the behavior of the disk caching code inside btrfs. It's a regression test for the patch

btrfs: allow single disk devices to mount with older generations

Thanks,

---
 tests/btrfs/219     | 101 ++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/219.out |   2 +
 tests/btrfs/group   |   1 +
 3 files changed, 104 insertions(+)
 create mode 100755 tests/btrfs/219
 create mode 100644 tests/btrfs/219.out
tests/btrfs/219 new file +101 -0
new file mode 100755
index 00000000..14e2792f
@@ -0,0 +1,101 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2020 Facebook.  All Rights Reserved.
+#
+# FS QA Test 219
+#
+# Test a variety of stale device usecases.  We cache the device and generation
+# to make sure we do not allow stale devices, which can end up with some wonky
+# behavior for loop back devices.  This was changed with
+#
+#   btrfs: allow single disk devices to mount with older generations
+#
+# But I've added a few other test cases so it's clear what we expect to happen
+# currently.
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+	$UMOUNT_PROG $loop_mnt
+	rm -rf $loop_mnt
+	rm -rf $loop_mnt1
+	rm -f $fs_img2 $fs_img1
+	[ ! -z "$loop_dev" ] && _destroy_loop_device $loop_dev
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+
+_supported_fs btrfs
+_supported_os Linux
+_require_test
+_require_loop
+_require_btrfs_forget_or_module_loadable
+
+loop_mnt=$TEST_DIR/$seq.mnt
+loop_mnt1=$TEST_DIR/$seq.mnt1
+fs_img1=$TEST_DIR/$seq.img1
+fs_img2=$TEST_DIR/$seq.img2
+
+mkdir $loop_mnt
+mkdir $loop_mnt1
+
+$XFS_IO_PROG -f -c "truncate 256m" $fs_img1 >>$seqres.full 2>&1
+
+_mkfs_dev $fs_img1 >>$seqres.full 2>&1
+cp $fs_img1 $fs_img2
+
+# Normal single device case, should pass just fine
+_mount -o loop $fs_img1 $loop_mnt > /dev/null  2>&1 || \
+	_fail "Couldn't do initial mount"
+$UMOUNT_PROG $loop_mnt
+
+_btrfs_forget_or_module_reload
+
+# Now mount the new version again to get the higher generation cached, umount
+# and try to mount the old version.  Mount the new version again just for good
+# measure.
+loop_dev=`_create_loop_device $fs_img1`
+
+_mount $loop_dev $loop_mnt > /dev/null 2>&1 || \
+	_fail "Failed to mount the second time"
+$UMOUNT_PROG $loop_mnt
+
+_mount -o loop $fs_img2 $loop_mnt > /dev/null 2>&1 || \
+	_fail "We couldn't mount the old generation"
+$UMOUNT_PROG $loop_mnt
+
+_mount $loop_dev $loop_mnt > /dev/null 2>&1 || \
+	_fail "Failed to mount the second time"
+$UMOUNT_PROG $loop_mnt
+
+# Now we definitely can't mount them at the same time, because we're still tied
+# to the limitation of one fs_devices per fsid.
+_btrfs_forget_or_module_reload
+
+_mount $loop_dev $loop_mnt > /dev/null 2>&1 || \
+	_fail "Failed to mount the third time"
+_mount -o loop $fs_img2 $loop_mnt1 > /dev/null 2>&1 && \
+	_fail "We were allowed to mount when we should have failed"
+
+# success, all done
+echo "Silence is golden"
+status=0
+exit
tests/btrfs/219.out new file +2 -0
new file mode 100644
index 00000000..162074d3
@@ -0,0 +1,2 @@
+QA output created by 219
+Silence is golden
tests/btrfs/group +1 -0
index 3295856d..660cdc25 100644
@@ -221,3 +221,4 @@
 216 auto quick seed
 217 auto quick trim dangerous
 218 auto quick volume
+219 auto quick volume
Signature · 1 line
2.24.1
#1 lore raw

PATCH v4 1/1 fstests: add generic/609 to test O_DIRECT|O_DSYNC

We had a problem recently where btrfs would deadlock with O_DIRECT|O_DSYNC because of an unexpected dependency on ->fsync in iomap. This was only caught by chance with aiostress, because weirdly we don't actually test this particular configuration anywhere in xfstests. Fix this by adding a basic test that just does O_DIRECT|O_DSYNC writes. With this test the box deadlocks right away with Btrfs, which would have been helpful in finding this issue before the patches were merged.

---
v3->v4:
- Trying to see how many times I can fuck this thing up.
- Simplified the xfs_io command per Darrick's suggestion.
- Added it to the rw group.

 tests/generic/609     | 43 +++++++++++++++++++++++++++++++++++++++++++
 tests/generic/609.out |  3 +++
 tests/generic/group   |  1 +
 3 files changed, 47 insertions(+)
 create mode 100755 tests/generic/609
 create mode 100644 tests/generic/609.out
tests/generic/609 new file +43 -0
new file mode 100755
index 00000000..6c74ae63
@@ -0,0 +1,43 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2020 Josef Bacik.  All Rights Reserved.
+#
+# FS QA Test 609
+#
+# iomap can call generic_write_sync() if we're O_DSYNC, so write a basic test to
+# exercise O_DSYNC so any unsuspecting file systems will get lockdep warnings if
+# their locking isn't compatible.
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+	rm -rf $TEST_DIR/file
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# Modify as appropriate.
+_supported_fs generic
+_supported_os Linux
+_require_test
+_require_xfs_io_command "pwrite"
+
+$XFS_IO_PROG -f -d -s -c "pwrite 0 64k" $TEST_DIR/file | _filter_xfs_io
+
+status=0
+exit
tests/generic/609.out new file +3 -0
new file mode 100644
index 00000000..111c7fe9
@@ -0,0 +1,3 @@
+QA output created by 609
+wrote 65536/65536 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
tests/generic/group +1 -0
index aa969bcb..ae2567a0 100644
@@ -611,3 +611,4 @@
 606 auto attr quick dax
 607 auto attr quick dax
 608 auto attr quick dax
+609 auto quick rw
Signature · 1 line
2.28.0
Reviewed-by on v4 1/1
#2 lore raw

Re: [PATCH][v4] fstests: add generic/609 to test O_DIRECT|O_DSYNC

Quoted 11 lines · Josef Bacik

We had a problem recently where btrfs would deadlock with O_DIRECT|O_DSYNC because of an unexpected dependency on ->fsync in iomap. This was only caught by chance with aiostress, because weirdly we don't actually test this particular configuration anywhere in xfstests. Fix this by adding a basic test that just does O_DIRECT|O_DSYNC writes. With this test the box deadlocks right away with Btrfs, which would have been helpful in finding this issue before the patches were merged.

Looks fine to me, thanks! Sorry about Yet Another Iomap Dustup. :(

--D

Quoted 83 lines

---
v3->v4:
- Trying to see how many times I can fuck this thing up.
- Simplified the xfs_io command per Darrick's suggestion.
- Added it to the rw group.

 tests/generic/609     | 43 +++++++++++++++++++++++++++++++++++++++++++
 tests/generic/609.out |  3 +++
 tests/generic/group   |  1 +
 3 files changed, 47 insertions(+)
 create mode 100755 tests/generic/609
 create mode 100644 tests/generic/609.out
tests/generic/609 new file +43 -0
new file mode 100755
index 00000000..6c74ae63
@@ -0,0 +1,43 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2020 Josef Bacik.  All Rights Reserved.
+#
+# FS QA Test 609
+#
+# iomap can call generic_write_sync() if we're O_DSYNC, so write a basic test to
+# exercise O_DSYNC so any unsuspecting file systems will get lockdep warnings if
+# their locking isn't compatible.
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+	rm -rf $TEST_DIR/file
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# Modify as appropriate.
+_supported_fs generic
+_supported_os Linux
+_require_test
+_require_xfs_io_command "pwrite"
+
+$XFS_IO_PROG -f -d -s -c "pwrite 0 64k" $TEST_DIR/file | _filter_xfs_io
+
+status=0
+exit
tests/generic/609.out new file +3 -0
new file mode 100644
index 00000000..111c7fe9
@@ -0,0 +1,3 @@
+QA output created by 609
+wrote 65536/65536 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
tests/generic/group +1 -0
index aa969bcb..ae2567a0 100644
@@ -611,3 +611,4 @@
 606 auto attr quick dax
 607 auto attr quick dax
 608 auto attr quick dax
+609 auto quick rw
Signature · 1 line
2.28.0
Reviewed-by on v4 1/1
#3 lore raw

Re: [PATCH][v4] fstests: add generic/609 to test O_DIRECT|O_DSYNC

In reply to Josef Bacik
Quoted 59 lines · Josef Bacik

We had a problem recently where btrfs would deadlock with O_DIRECT|O_DSYNC because of an unexpected dependency on ->fsync in iomap. This was only caught by chance with aiostress, because weirdly we don't actually test this particular configuration anywhere in xfstests. Fix this by adding a basic test that just does O_DIRECT|O_DSYNC writes. With this test the box deadlocks right away with Btrfs, which would have been helpful in finding this issue before the patches were merged.

---
v3->v4:
- Trying to see how many times I can fuck this thing up.
- Simplified the xfs_io command per Darrick's suggestion.
- Added it to the rw group.

 tests/generic/609     | 43 +++++++++++++++++++++++++++++++++++++++++++
 tests/generic/609.out |  3 +++
 tests/generic/group   |  1 +
 3 files changed, 47 insertions(+)
 create mode 100755 tests/generic/609
 create mode 100644 tests/generic/609.out
tests/generic/609 new file +28 -0
new file mode 100755
index 00000000..6c74ae63
@@ -0,0 +1,43 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2020 Josef Bacik.  All Rights Reserved.
+#
+# FS QA Test 609
+#
+# iomap can call generic_write_sync() if we're O_DSYNC, so write a basic test to
+# exercise O_DSYNC so any unsuspecting file systems will get lockdep warnings if
+# their locking isn't compatible.
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1       # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+       cd /
+       rm -f $tmp.*
+       rm -rf $TEST_DIR/file
+}
+
+# get standard environment, filters and checks
+. ./common/rc

tests/generic/609 line 38

Quoted 10 lines
tests/generic/609 new file +10 -0
+. ./common/filter
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# Modify as appropriate.
+_supported_fs generic
+_supported_os Linux
+_require_test
+_require_xfs_io_command "pwrite"

missing a:

_require_odirect

Other than that, it looks good. Perhaps Eryu can add that when picking this, so you avoid sending a v5.

Thanks.

Quoted 26 lines
tests/generic/609 new file +5 -0
+
+$XFS_IO_PROG -f -d -s -c "pwrite 0 64k" $TEST_DIR/file | _filter_xfs_io
+
+status=0
+exit
tests/generic/609.out new file +3 -0
new file mode 100644
index 00000000..111c7fe9
@@ -0,0 +1,3 @@
+QA output created by 609
+wrote 65536/65536 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
tests/generic/group +1 -1
index aa969bcb..ae2567a0 100644
@@ -611,3 +611,4 @@
 606 auto attr quick dax
 607 auto attr quick dax
 608 auto attr quick dax
+609 auto quick rw
--

2.28.0

Signature · 3 lines
Filipe David Manana,

“Whether you think you can, or you think you can't — you're right.”
#4 lore raw

Re: [PATCH][v4] fstests: add generic/609 to test O_DIRECT|O_DSYNC

Quoted 73 lines · Filipe Manana
Quoted 59 lines · Josef Bacik

We had a problem recently where btrfs would deadlock with O_DIRECT|O_DSYNC because of an unexpected dependency on ->fsync in iomap. This was only caught by chance with aiostress, because weirdly we don't actually test this particular configuration anywhere in xfstests. Fix this by adding a basic test that just does O_DIRECT|O_DSYNC writes. With this test the box deadlocks right away with Btrfs, which would have been helpful in finding this issue before the patches were merged.

---
v3->v4:
- Trying to see how many times I can fuck this thing up.
- Simplified the xfs_io command per Darrick's suggestion.
- Added it to the rw group.

 tests/generic/609     | 43 +++++++++++++++++++++++++++++++++++++++++++
 tests/generic/609.out |  3 +++
 tests/generic/group   |  1 +
 3 files changed, 47 insertions(+)
 create mode 100755 tests/generic/609
 create mode 100644 tests/generic/609.out
tests/generic/609 new file +28 -0
new file mode 100755
index 00000000..6c74ae63
@@ -0,0 +1,43 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2020 Josef Bacik.  All Rights Reserved.
+#
+# FS QA Test 609
+#
+# iomap can call generic_write_sync() if we're O_DSYNC, so write a basic test to
+# exercise O_DSYNC so any unsuspecting file systems will get lockdep warnings if
+# their locking isn't compatible.
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1       # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+       cd /
+       rm -f $tmp.*
+       rm -rf $TEST_DIR/file
+}
+
+# get standard environment, filters and checks
+. ./common/rc

tests/generic/609 line 38

Quoted 10 lines
tests/generic/609 new file +10 -0
+. ./common/filter
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# Modify as appropriate.
+_supported_fs generic
+_supported_os Linux
+_require_test
+_require_xfs_io_command "pwrite"

missing a:

Quoted 4 lines

_require_odirect

Other than that, it looks good. Perhaps Eryu can add that when picking this, so you avoid sending a v5.

Yeah, I've added that on commit.

Quoted 2 lines

Thanks for the review!

Eryu