Skip to contentSkip to conversation

btrfs: test creating a snapshot after RWF_NOWAIT write works as expected

PATCH In review [email protected] 2 participants 4 messages Jun 15, 2020 – Jul 19, 2020 Btrfs file system

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 btrfs: test creating a snapshot after RWF_NOWAIT write works as expected

From: Filipe Manana <[email protected]>

Test that creating a snapshot after writing to a file using a RWF_NOWAIT works, does not hang the snapshot creation task, and we are able to read the data after.

Currently btrfs hangs when creating the snapshot due to a missing unlock of a snapshot lock, but it is fixed by a patch with the following subject:

"btrfs: fix hang on snapshot creation after RWF_NOWAIT write"

---
 tests/btrfs/214     | 66 +++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/214.out | 14 ++++++++++
 tests/btrfs/group   |  1 +
 3 files changed, 81 insertions(+)
 create mode 100755 tests/btrfs/214
 create mode 100644 tests/btrfs/214.out
tests/btrfs/214 new file +66 -0
new file mode 100755
index 00000000..c835e844
@@ -0,0 +1,66 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2020 SUSE Linux Products GmbH. All Rights Reserved.
+#
+# FSQA Test No. 214
+#
+# Test that creating a snapshot after writing to a file using a RWF_NOWAIT
+# works, does not hang the snapshot creation task, and we are able to read
+# the data after.
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/attr
+
+# real QA test starts here
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+_require_odirect
+_require_xfs_io_command pwrite -N
+_require_chattr C
+
+rm -f $seqres.full
+
+_scratch_mkfs >>$seqres.full 2>&1
+_scratch_mount
+
+# RWF_NOWAIT writes require NOCOW
+touch $SCRATCH_MNT/f
+$CHATTR_PROG +C $SCRATCH_MNT/f
+
+$XFS_IO_PROG -d -c "pwrite -S 0xab 0 64K" $SCRATCH_MNT/f | _filter_xfs_io
+
+# Now do a WEF_WRITE into a range containing a NOCOWable extent.
+$XFS_IO_PROG -d -c "pwrite -N -V 1 -S 0xfe 0 64K" $SCRATCH_MNT/f \
+	| _filter_xfs_io
+
+$BTRFS_UTIL_PROG subvolume snapshot -r $SCRATCH_MNT $SCRATCH_MNT/snap \
+	| _filter_scratch
+
+# Unmount, mount again and verify the file in the subvolume and snapshot has
+# the correct data.
+_scratch_cycle_mount
+
+echo "File data in the subvolume:"
+od -A d -t x1 $SCRATCH_MNT/f
+
+echo "File data in the snapshot:"
+od -A d -t x1 $SCRATCH_MNT/snap/f
+
+status=0
+exit
tests/btrfs/214.out new file +14 -0
new file mode 100644
index 00000000..6cc66972
@@ -0,0 +1,14 @@
+QA output created by 214
+wrote 65536/65536 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 65536/65536 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+Create a readonly snapshot of 'SCRATCH_MNT' in 'SCRATCH_MNT/snap'
+File data in the subvolume:
+0000000 fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe
+*
+0065536
+File data in the snapshot:
+0000000 fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe
+*
+0065536
tests/btrfs/group +1 -0
index 9e48ecc1..a3706e7d 100644
@@ -216,3 +216,4 @@
 211 auto quick log prealloc
 212 auto balance dangerous
 213 auto balance dangerous
+214 auto quick snapshot
Signature · 1 line
2.26.2
#1 lore raw
Quoted 6 lines · On Mon, Jun 15, 2020 at 06:50:28PM +0100, [email protected] wrote:

From: Filipe Manana <[email protected]>

Test that creating a snapshot after writing to a file using a RWF_NOWAIT works, does not hang the snapshot creation task, and we are able to read the data after.

Quoted 6 lines

Currently btrfs hangs when creating the snapshot due to a missing unlock of a snapshot lock, but it is fixed by a patch with the following subject:

"btrfs: fix hang on snapshot creation after RWF_NOWAIT write"

So sorry for the late review.. But I hit the following failure with v5.8-rc5 kernel, which contains the mentioned fix

QA output created by 215
wrote 65536/65536 bytes at offset 0
XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-wrote 65536/65536 bytes at offset 0
-XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +pwrite: Input/output error
Create a readonly snapshot of 'SCRATCH_MNT' in 'SCRATCH_MNT/snap' File data in the subvolume:
-0000000 fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe +0000000 ab ab ab ab ab ab ab ab ab ab ab ab ab ab ab ab
*
0065536
File data in the snapshot:
-0000000 fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe +0000000 ab ab ab ab ab ab ab ab ab ab ab ab ab ab ab ab
*
0065536

And I did hit hang when testing without the fix. Is this something that should be fixed in the test?

Thanks,
Eryu

Quoted 111 lines
---
 tests/btrfs/214     | 66 +++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/214.out | 14 ++++++++++
 tests/btrfs/group   |  1 +
 3 files changed, 81 insertions(+)
 create mode 100755 tests/btrfs/214
 create mode 100644 tests/btrfs/214.out
tests/btrfs/214 new file +66 -0
new file mode 100755
index 00000000..c835e844
@@ -0,0 +1,66 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2020 SUSE Linux Products GmbH. All Rights Reserved.
+#
+# FSQA Test No. 214
+#
+# Test that creating a snapshot after writing to a file using a RWF_NOWAIT
+# works, does not hang the snapshot creation task, and we are able to read
+# the data after.
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/attr
+
+# real QA test starts here
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+_require_odirect
+_require_xfs_io_command pwrite -N
+_require_chattr C
+
+rm -f $seqres.full
+
+_scratch_mkfs >>$seqres.full 2>&1
+_scratch_mount
+
+# RWF_NOWAIT writes require NOCOW
+touch $SCRATCH_MNT/f
+$CHATTR_PROG +C $SCRATCH_MNT/f
+
+$XFS_IO_PROG -d -c "pwrite -S 0xab 0 64K" $SCRATCH_MNT/f | _filter_xfs_io
+
+# Now do a WEF_WRITE into a range containing a NOCOWable extent.
+$XFS_IO_PROG -d -c "pwrite -N -V 1 -S 0xfe 0 64K" $SCRATCH_MNT/f \
+	| _filter_xfs_io
+
+$BTRFS_UTIL_PROG subvolume snapshot -r $SCRATCH_MNT $SCRATCH_MNT/snap \
+	| _filter_scratch
+
+# Unmount, mount again and verify the file in the subvolume and snapshot has
+# the correct data.
+_scratch_cycle_mount
+
+echo "File data in the subvolume:"
+od -A d -t x1 $SCRATCH_MNT/f
+
+echo "File data in the snapshot:"
+od -A d -t x1 $SCRATCH_MNT/snap/f
+
+status=0
+exit
tests/btrfs/214.out new file +14 -0
new file mode 100644
index 00000000..6cc66972
@@ -0,0 +1,14 @@
+QA output created by 214
+wrote 65536/65536 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 65536/65536 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+Create a readonly snapshot of 'SCRATCH_MNT' in 'SCRATCH_MNT/snap'
+File data in the subvolume:
+0000000 fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe
+*
+0065536
+File data in the snapshot:
+0000000 fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe
+*
+0065536
tests/btrfs/group +1 -0
index 9e48ecc1..a3706e7d 100644
@@ -216,3 +216,4 @@
 211 auto quick log prealloc
 212 auto balance dangerous
 213 auto balance dangerous
+214 auto quick snapshot
Signature · 1 line
2.26.2
#2 lore raw
Quoted 36 lines · Eryu Guan
Quoted 6 lines · On Mon, Jun 15, 2020 at 06:50:28PM +0100, [email protected] wrote:

From: Filipe Manana <[email protected]>

Test that creating a snapshot after writing to a file using a RWF_NOWAIT works, does not hang the snapshot creation task, and we are able to read the data after.

Quoted 6 lines

Currently btrfs hangs when creating the snapshot due to a missing unlock of a snapshot lock, but it is fixed by a patch with the following subject:

"btrfs: fix hang on snapshot creation after RWF_NOWAIT write"

So sorry for the late review.. But I hit the following failure with v5.8-rc5 kernel, which contains the mentioned fix

QA output created by 215
wrote 65536/65536 bytes at offset 0
XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)

(quoted diff) +3 -4
-wrote 65536/65536 bytes at offset 0
-XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+pwrite: Input/output error
 Create a readonly snapshot of 'SCRATCH_MNT' in 'SCRATCH_MNT/snap'
 File data in the subvolume:
-0000000 fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe
+0000000 ab ab ab ab ab ab ab ab ab ab ab ab ab ab ab ab
 *
 0065536
 File data in the snapshot:
-0000000 fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe
+0000000 ab ab ab ab ab ab ab ab ab ab ab ab ab ab ab ab
 *
 0065536
Quoted 2 lines

And I did hit hang when testing without the fix. Is this something that should be fixed in the test?

Odd.

Works for me on 5.8-rc5:

$ ./check btrfs/215
FSTYP         -- btrfs
PLATFORM      -- Linux/x86_64 debian9 5.8.0-rc5-btrfs-next-63 #1 SMP
PREEMPT Sun Jul 19 17:53:27 WEST 2020
MKFS_OPTIONS  -- /dev/sdb
MOUNT_OPTIONS -- /dev/sdb /home/fdmanana/btrfs-tests/scratch_1

btrfs/215 1s
Ran: btrfs/215
Passed all 1 tests

Is there anything in dmesg?

Quoted 115 lines

Thanks,
Eryu

Quoted 111 lines
---
 tests/btrfs/214     | 66 +++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/214.out | 14 ++++++++++
 tests/btrfs/group   |  1 +
 3 files changed, 81 insertions(+)
 create mode 100755 tests/btrfs/214
 create mode 100644 tests/btrfs/214.out
tests/btrfs/214 new file +66 -0
new file mode 100755
index 00000000..c835e844
@@ -0,0 +1,66 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2020 SUSE Linux Products GmbH. All Rights Reserved.
+#
+# FSQA Test No. 214
+#
+# Test that creating a snapshot after writing to a file using a RWF_NOWAIT
+# works, does not hang the snapshot creation task, and we are able to read
+# the data after.
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+tmp=/tmp/$$
+status=1     # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+     cd /
+     rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/attr
+
+# real QA test starts here
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+_require_odirect
+_require_xfs_io_command pwrite -N
+_require_chattr C
+
+rm -f $seqres.full
+
+_scratch_mkfs >>$seqres.full 2>&1
+_scratch_mount
+
+# RWF_NOWAIT writes require NOCOW
+touch $SCRATCH_MNT/f
+$CHATTR_PROG +C $SCRATCH_MNT/f
+
+$XFS_IO_PROG -d -c "pwrite -S 0xab 0 64K" $SCRATCH_MNT/f | _filter_xfs_io
+
+# Now do a WEF_WRITE into a range containing a NOCOWable extent.
+$XFS_IO_PROG -d -c "pwrite -N -V 1 -S 0xfe 0 64K" $SCRATCH_MNT/f \
+     | _filter_xfs_io
+
+$BTRFS_UTIL_PROG subvolume snapshot -r $SCRATCH_MNT $SCRATCH_MNT/snap \
+     | _filter_scratch
+
+# Unmount, mount again and verify the file in the subvolume and snapshot has
+# the correct data.
+_scratch_cycle_mount
+
+echo "File data in the subvolume:"
+od -A d -t x1 $SCRATCH_MNT/f
+
+echo "File data in the snapshot:"
+od -A d -t x1 $SCRATCH_MNT/snap/f
+
+status=0
+exit
tests/btrfs/214.out new file +14 -0
new file mode 100644
index 00000000..6cc66972
@@ -0,0 +1,14 @@
+QA output created by 214
+wrote 65536/65536 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 65536/65536 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+Create a readonly snapshot of 'SCRATCH_MNT' in 'SCRATCH_MNT/snap'
+File data in the subvolume:
+0000000 fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe
+*
+0065536
+File data in the snapshot:
+0000000 fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe
+*
+0065536
tests/btrfs/group +1 -1
index 9e48ecc1..a3706e7d 100644
@@ -216,3 +216,4 @@
 211 auto quick log prealloc
 212 auto balance dangerous
 213 auto balance dangerous
+214 auto quick snapshot
--

2.26.2

#3 lore raw
Quoted 51 lines · Filipe Manana
Quoted 36 lines · Eryu Guan
Quoted 6 lines · On Mon, Jun 15, 2020 at 06:50:28PM +0100, [email protected] wrote:

From: Filipe Manana <[email protected]>

Test that creating a snapshot after writing to a file using a RWF_NOWAIT works, does not hang the snapshot creation task, and we are able to read the data after.

Quoted 6 lines

Currently btrfs hangs when creating the snapshot due to a missing unlock of a snapshot lock, but it is fixed by a patch with the following subject:

"btrfs: fix hang on snapshot creation after RWF_NOWAIT write"

So sorry for the late review.. But I hit the following failure with v5.8-rc5 kernel, which contains the mentioned fix

QA output created by 215
wrote 65536/65536 bytes at offset 0
XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)

(quoted diff) +3 -4
-wrote 65536/65536 bytes at offset 0
-XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+pwrite: Input/output error
 Create a readonly snapshot of 'SCRATCH_MNT' in 'SCRATCH_MNT/snap'
 File data in the subvolume:
-0000000 fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe
+0000000 ab ab ab ab ab ab ab ab ab ab ab ab ab ab ab ab
 *
 0065536
 File data in the snapshot:
-0000000 fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe
+0000000 ab ab ab ab ab ab ab ab ab ab ab ab ab ab ab ab
 *
 0065536
Quoted 2 lines

And I did hit hang when testing without the fix. Is this something that should be fixed in the test?

Odd.

Works for me on 5.8-rc5:

$ ./check btrfs/215
FSTYP         -- btrfs
PLATFORM      -- Linux/x86_64 debian9 5.8.0-rc5-btrfs-next-63 #1 SMP
PREEMPT Sun Jul 19 17:53:27 WEST 2020
MKFS_OPTIONS  -- /dev/sdb
MOUNT_OPTIONS -- /dev/sdb /home/fdmanana/btrfs-tests/scratch_1
Quoted 5 lines

btrfs/215 1s
Ran: btrfs/215
Passed all 1 tests

Is there anything in dmesg?

Hmm, there is a dio failure message.

[ 8077.648741] run fstests btrfs/216 at 2020-07-20 02:42:37
[ 8078.169516] BTRFS: device fsid 0de433a0-628f-4f7c-bb81-a9632d690d0f devid 1 transid 5 /dev/mapper/testvg-lv2 scanned by mkfs.btrfs (76661)
[ 8078.199269] BTRFS info (device dm-5): disk space caching is enabled
[ 8078.200897] BTRFS info (device dm-5): has skinny extents
[ 8078.202227] BTRFS info (device dm-5): flagging fs with big metadata feature
[ 8078.208853] BTRFS info (device dm-5): checking UUID tree
[ 8078.231519] BTRFS warning (device dm-5): direct IO failed ino 257 rw 1,2131969 sector 0x6800 len 4096 err no 10
[ 8078.357124] BTRFS info (device dm-5): disk space caching is enabled
[ 8078.357906] BTRFS info (device dm-5): has skinny extents

My kernel version and btrfs-progs version

[root@fedoravm xfstests]# uname -a
Linux fedoravm 5.8.0-rc5 #52 SMP Mon Jul 20 00:25:44 CST 2020 x86_64 x86_64 x86_64 GNU/Linux [root@fedoravm xfstests]# btrfs --version
btrfs-progs v5.4

Thanks,
Eryu

Quoted 118 lines
Quoted 118 lines

Thanks,
Eryu

Quoted 111 lines
---
 tests/btrfs/214     | 66 +++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/214.out | 14 ++++++++++
 tests/btrfs/group   |  1 +
 3 files changed, 81 insertions(+)
 create mode 100755 tests/btrfs/214
 create mode 100644 tests/btrfs/214.out
tests/btrfs/214 new file +66 -0
new file mode 100755
index 00000000..c835e844
@@ -0,0 +1,66 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2020 SUSE Linux Products GmbH. All Rights Reserved.
+#
+# FSQA Test No. 214
+#
+# Test that creating a snapshot after writing to a file using a RWF_NOWAIT
+# works, does not hang the snapshot creation task, and we are able to read
+# the data after.
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+tmp=/tmp/$$
+status=1     # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+     cd /
+     rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/attr
+
+# real QA test starts here
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+_require_odirect
+_require_xfs_io_command pwrite -N
+_require_chattr C
+
+rm -f $seqres.full
+
+_scratch_mkfs >>$seqres.full 2>&1
+_scratch_mount
+
+# RWF_NOWAIT writes require NOCOW
+touch $SCRATCH_MNT/f
+$CHATTR_PROG +C $SCRATCH_MNT/f
+
+$XFS_IO_PROG -d -c "pwrite -S 0xab 0 64K" $SCRATCH_MNT/f | _filter_xfs_io
+
+# Now do a WEF_WRITE into a range containing a NOCOWable extent.
+$XFS_IO_PROG -d -c "pwrite -N -V 1 -S 0xfe 0 64K" $SCRATCH_MNT/f \
+     | _filter_xfs_io
+
+$BTRFS_UTIL_PROG subvolume snapshot -r $SCRATCH_MNT $SCRATCH_MNT/snap \
+     | _filter_scratch
+
+# Unmount, mount again and verify the file in the subvolume and snapshot has
+# the correct data.
+_scratch_cycle_mount
+
+echo "File data in the subvolume:"
+od -A d -t x1 $SCRATCH_MNT/f
+
+echo "File data in the snapshot:"
+od -A d -t x1 $SCRATCH_MNT/snap/f
+
+status=0
+exit
tests/btrfs/214.out new file +14 -0
new file mode 100644
index 00000000..6cc66972
@@ -0,0 +1,14 @@
+QA output created by 214
+wrote 65536/65536 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 65536/65536 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+Create a readonly snapshot of 'SCRATCH_MNT' in 'SCRATCH_MNT/snap'
+File data in the subvolume:
+0000000 fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe
+*
+0065536
+File data in the snapshot:
+0000000 fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe
+*
+0065536
tests/btrfs/group +1 -1
index 9e48ecc1..a3706e7d 100644
@@ -216,3 +216,4 @@
 211 auto quick log prealloc
 212 auto balance dangerous
 213 auto balance dangerous
+214 auto quick snapshot
--

2.26.2