btrfs: test creating a snapshot after RWF_NOWAIT write works as expected
PATCH 1/1 btrfs: test creating a snapshot after RWF_NOWAIT write works as expected
From: Filipe Manana <[email protected]>
Test that creating a snapshot after writing to a file using a RWF_NOWAIT works, does not hang the snapshot creation task, and we are able to read the data after.
Currently btrfs hangs when creating the snapshot due to a missing unlock of a snapshot lock, but it is fixed by a patch with the following subject:
"btrfs: fix hang on snapshot creation after RWF_NOWAIT write"
--- tests/btrfs/214 | 66 +++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/214.out | 14 ++++++++++ tests/btrfs/group | 1 + 3 files changed, 81 insertions(+) create mode 100755 tests/btrfs/214 create mode 100644 tests/btrfs/214.out
tests/btrfs/214 new file +66 -0
@@ -0,0 +1,66 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) 2020 SUSE Linux Products GmbH. All Rights Reserved. +# +# FSQA Test No. 214 +# +# Test that creating a snapshot after writing to a file using a RWF_NOWAIT +# works, does not hang the snapshot creation task, and we are able to read +# the data after. +# +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter +. ./common/attr + +# real QA test starts here +_supported_fs btrfs +_supported_os Linux +_require_scratch +_require_odirect +_require_xfs_io_command pwrite -N +_require_chattr C + +rm -f $seqres.full + +_scratch_mkfs >>$seqres.full 2>&1 +_scratch_mount + +# RWF_NOWAIT writes require NOCOW +touch $SCRATCH_MNT/f +$CHATTR_PROG +C $SCRATCH_MNT/f + +$XFS_IO_PROG -d -c "pwrite -S 0xab 0 64K" $SCRATCH_MNT/f | _filter_xfs_io + +# Now do a WEF_WRITE into a range containing a NOCOWable extent. +$XFS_IO_PROG -d -c "pwrite -N -V 1 -S 0xfe 0 64K" $SCRATCH_MNT/f \ + | _filter_xfs_io + +$BTRFS_UTIL_PROG subvolume snapshot -r $SCRATCH_MNT $SCRATCH_MNT/snap \ + | _filter_scratch + +# Unmount, mount again and verify the file in the subvolume and snapshot has +# the correct data. +_scratch_cycle_mount + +echo "File data in the subvolume:" +od -A d -t x1 $SCRATCH_MNT/f + +echo "File data in the snapshot:" +od -A d -t x1 $SCRATCH_MNT/snap/f + +status=0 +exit
tests/btrfs/214.out new file +14 -0
@@ -0,0 +1,14 @@ +QA output created by 214 +wrote 65536/65536 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 65536/65536 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +Create a readonly snapshot of 'SCRATCH_MNT' in 'SCRATCH_MNT/snap' +File data in the subvolume: +0000000 fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe +* +0065536 +File data in the snapshot: +0000000 fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe +* +0065536
tests/btrfs/group +1 -0
@@ -216,3 +216,4 @@ 211 auto quick log prealloc 212 auto balance dangerous 213 auto balance dangerous +214 auto quick snapshot
Signature · 1 line
2.26.2
Quoted 6 lines · On Mon, Jun 15, 2020 at 06:50:28PM +0100, [email protected] wrote:
From: Filipe Manana <[email protected]>
Test that creating a snapshot after writing to a file using a
RWF_NOWAITworks, does not hang the snapshot creation task, and we are able to read the data after.
Quoted 6 lines
Currently btrfs hangs when creating the snapshot due to a missing unlock of a snapshot lock, but it is fixed by a patch with the following subject:
"btrfs: fix hang on snapshot creation after
RWF_NOWAITwrite"Signed-off-by Filipe Manana <[email protected]>
So sorry for the late review.. But I hit the following failure with v5.8-rc5 kernel, which contains the mentioned fix
QA output created by 215
wrote 65536/65536 bytes at offset 0
XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-wrote 65536/65536 bytes at offset 0
-XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +pwrite: Input/output error
Create a readonly snapshot of 'SCRATCH_MNT' in 'SCRATCH_MNT/snap' File data in the subvolume:
-0000000 fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe +0000000 ab ab ab ab ab ab ab ab ab ab ab ab ab ab ab ab
*
0065536
File data in the snapshot:
-0000000 fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe +0000000 ab ab ab ab ab ab ab ab ab ab ab ab ab ab ab ab
*
0065536
And I did hit hang when testing without the fix. Is this something that should be fixed in the test?
Thanks,
Eryu
Quoted 111 lines
--- tests/btrfs/214 | 66 +++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/214.out | 14 ++++++++++ tests/btrfs/group | 1 + 3 files changed, 81 insertions(+) create mode 100755 tests/btrfs/214 create mode 100644 tests/btrfs/214.outtests/btrfs/214 new file +66
-0@@ -0,0 +1,66 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) 2020 SUSE Linux Products GmbH. All Rights Reserved. +# +# FSQA Test No. 214 +# +# Test that creating a snapshot after writing to a file using a RWF_NOWAIT +# works, does not hang the snapshot creation task, and we are able to read +# the data after. +# +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter +. ./common/attr + +# real QA test starts here +_supported_fs btrfs +_supported_os Linux +_require_scratch +_require_odirect +_require_xfs_io_command pwrite -N +_require_chattr C + +rm -f $seqres.full + +_scratch_mkfs >>$seqres.full 2>&1 +_scratch_mount + +# RWF_NOWAIT writes require NOCOW +touch $SCRATCH_MNT/f +$CHATTR_PROG +C $SCRATCH_MNT/f + +$XFS_IO_PROG -d -c "pwrite -S 0xab 0 64K" $SCRATCH_MNT/f | _filter_xfs_io + +# Now do a WEF_WRITE into a range containing a NOCOWable extent. +$XFS_IO_PROG -d -c "pwrite -N -V 1 -S 0xfe 0 64K" $SCRATCH_MNT/f \ + | _filter_xfs_io + +$BTRFS_UTIL_PROG subvolume snapshot -r $SCRATCH_MNT $SCRATCH_MNT/snap \ + | _filter_scratch + +# Unmount, mount again and verify the file in the subvolume and snapshot has +# the correct data. +_scratch_cycle_mount + +echo "File data in the subvolume:" +od -A d -t x1 $SCRATCH_MNT/f + +echo "File data in the snapshot:" +od -A d -t x1 $SCRATCH_MNT/snap/f + +status=0 +exittests/btrfs/214.out new file +14
-0@@ -0,0 +1,14 @@ +QA output created by 214 +wrote 65536/65536 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 65536/65536 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +Create a readonly snapshot of 'SCRATCH_MNT' in 'SCRATCH_MNT/snap' +File data in the subvolume: +0000000 fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe +* +0065536 +File data in the snapshot: +0000000 fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe +* +0065536tests/btrfs/group +1
-0@@ -216,3 +216,4 @@ 211 auto quick log prealloc 212 auto balance dangerous 213 auto balance dangerous +214 auto quick snapshotSignature · 1 line
2.26.2
Quoted 36 lines · Eryu Guan
Quoted 6 lines · On Mon, Jun 15, 2020 at 06:50:28PM +0100, [email protected] wrote:
From: Filipe Manana <[email protected]>
Test that creating a snapshot after writing to a file using a
RWF_NOWAITworks, does not hang the snapshot creation task, and we are able to read the data after.Quoted 6 lines
Currently btrfs hangs when creating the snapshot due to a missing unlock of a snapshot lock, but it is fixed by a patch with the following subject:
"btrfs: fix hang on snapshot creation after
RWF_NOWAITwrite"Signed-off-by Filipe Manana <[email protected]>So sorry for the late review.. But I hit the following failure with v5.8-rc5 kernel, which contains the mentioned fix
QA output created by 215
wrote 65536/65536 bytes at offset 0
XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)(quoted diff) +3
-4-wrote 65536/65536 bytes at offset 0 -XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +pwrite: Input/output error Create a readonly snapshot of 'SCRATCH_MNT' in 'SCRATCH_MNT/snap' File data in the subvolume: -0000000 fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe +0000000 ab ab ab ab ab ab ab ab ab ab ab ab ab ab ab ab * 0065536 File data in the snapshot: -0000000 fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe +0000000 ab ab ab ab ab ab ab ab ab ab ab ab ab ab ab ab * 0065536
Quoted 2 lines
And I did hit hang when testing without the fix. Is this something that should be fixed in the test?
Odd.
Works for me on 5.8-rc5:
$ ./check btrfs/215 FSTYP -- btrfs PLATFORM -- Linux/x86_64 debian9 5.8.0-rc5-btrfs-next-63 #1 SMP PREEMPT Sun Jul 19 17:53:27 WEST 2020 MKFS_OPTIONS -- /dev/sdb MOUNT_OPTIONS -- /dev/sdb /home/fdmanana/btrfs-tests/scratch_1
btrfs/215 1s
Ran: btrfs/215
Passed all 1 tests
Is there anything in dmesg?
Quoted 115 lines
Thanks,
EryuQuoted 111 lines
--- tests/btrfs/214 | 66 +++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/214.out | 14 ++++++++++ tests/btrfs/group | 1 + 3 files changed, 81 insertions(+) create mode 100755 tests/btrfs/214 create mode 100644 tests/btrfs/214.outtests/btrfs/214 new file +66
-0@@ -0,0 +1,66 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) 2020 SUSE Linux Products GmbH. All Rights Reserved. +# +# FSQA Test No. 214 +# +# Test that creating a snapshot after writing to a file using a RWF_NOWAIT +# works, does not hang the snapshot creation task, and we are able to read +# the data after. +# +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter +. ./common/attr + +# real QA test starts here +_supported_fs btrfs +_supported_os Linux +_require_scratch +_require_odirect +_require_xfs_io_command pwrite -N +_require_chattr C + +rm -f $seqres.full + +_scratch_mkfs >>$seqres.full 2>&1 +_scratch_mount + +# RWF_NOWAIT writes require NOCOW +touch $SCRATCH_MNT/f +$CHATTR_PROG +C $SCRATCH_MNT/f + +$XFS_IO_PROG -d -c "pwrite -S 0xab 0 64K" $SCRATCH_MNT/f | _filter_xfs_io + +# Now do a WEF_WRITE into a range containing a NOCOWable extent. +$XFS_IO_PROG -d -c "pwrite -N -V 1 -S 0xfe 0 64K" $SCRATCH_MNT/f \ + | _filter_xfs_io + +$BTRFS_UTIL_PROG subvolume snapshot -r $SCRATCH_MNT $SCRATCH_MNT/snap \ + | _filter_scratch + +# Unmount, mount again and verify the file in the subvolume and snapshot has +# the correct data. +_scratch_cycle_mount + +echo "File data in the subvolume:" +od -A d -t x1 $SCRATCH_MNT/f + +echo "File data in the snapshot:" +od -A d -t x1 $SCRATCH_MNT/snap/f + +status=0 +exittests/btrfs/214.out new file +14
-0@@ -0,0 +1,14 @@ +QA output created by 214 +wrote 65536/65536 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 65536/65536 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +Create a readonly snapshot of 'SCRATCH_MNT' in 'SCRATCH_MNT/snap' +File data in the subvolume: +0000000 fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe +* +0065536 +File data in the snapshot: +0000000 fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe +* +0065536tests/btrfs/group +1
-1@@ -216,3 +216,4 @@ 211 auto quick log prealloc 212 auto balance dangerous 213 auto balance dangerous +214 auto quick snapshot --2.26.2
Quoted 51 lines · Filipe Manana
Quoted 36 lines · Eryu Guan
Quoted 6 lines · On Mon, Jun 15, 2020 at 06:50:28PM +0100, [email protected] wrote:
From: Filipe Manana <[email protected]>
Test that creating a snapshot after writing to a file using a
RWF_NOWAITworks, does not hang the snapshot creation task, and we are able to read the data after.Quoted 6 lines
Currently btrfs hangs when creating the snapshot due to a missing unlock of a snapshot lock, but it is fixed by a patch with the following subject:
"btrfs: fix hang on snapshot creation after
RWF_NOWAITwrite"Signed-off-by Filipe Manana <[email protected]>So sorry for the late review.. But I hit the following failure with v5.8-rc5 kernel, which contains the mentioned fix
QA output created by 215
wrote 65536/65536 bytes at offset 0
XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)(quoted diff) +3
-4-wrote 65536/65536 bytes at offset 0 -XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +pwrite: Input/output error Create a readonly snapshot of 'SCRATCH_MNT' in 'SCRATCH_MNT/snap' File data in the subvolume: -0000000 fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe +0000000 ab ab ab ab ab ab ab ab ab ab ab ab ab ab ab ab * 0065536 File data in the snapshot: -0000000 fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe +0000000 ab ab ab ab ab ab ab ab ab ab ab ab ab ab ab ab * 0065536Quoted 2 lines
And I did hit hang when testing without the fix. Is this something that should be fixed in the test?
Odd.
Works for me on 5.8-rc5:
$ ./check btrfs/215 FSTYP -- btrfs PLATFORM -- Linux/x86_64 debian9 5.8.0-rc5-btrfs-next-63 #1 SMP PREEMPT Sun Jul 19 17:53:27 WEST 2020 MKFS_OPTIONS -- /dev/sdb MOUNT_OPTIONS -- /dev/sdb /home/fdmanana/btrfs-tests/scratch_1
Quoted 5 lines
btrfs/215 1s
Ran: btrfs/215
Passed all 1 testsIs there anything in dmesg?
Hmm, there is a dio failure message.
[ 8077.648741] run fstests btrfs/216 at 2020-07-20 02:42:37 [ 8078.169516] BTRFS: device fsid 0de433a0-628f-4f7c-bb81-a9632d690d0f devid 1 transid 5 /dev/mapper/testvg-lv2 scanned by mkfs.btrfs (76661) [ 8078.199269] BTRFS info (device dm-5): disk space caching is enabled [ 8078.200897] BTRFS info (device dm-5): has skinny extents [ 8078.202227] BTRFS info (device dm-5): flagging fs with big metadata feature [ 8078.208853] BTRFS info (device dm-5): checking UUID tree [ 8078.231519] BTRFS warning (device dm-5): direct IO failed ino 257 rw 1,2131969 sector 0x6800 len 4096 err no 10 [ 8078.357124] BTRFS info (device dm-5): disk space caching is enabled [ 8078.357906] BTRFS info (device dm-5): has skinny extents
My kernel version and btrfs-progs version
[root@fedoravm xfstests]# uname -a
Linux fedoravm 5.8.0-rc5 #52 SMP Mon Jul 20 00:25:44 CST 2020 x86_64 x86_64 x86_64 GNU/Linux [root@fedoravm xfstests]# btrfs --version
btrfs-progs v5.4
Thanks,
Eryu
Quoted 118 lines
Quoted 118 lines
Thanks,
EryuQuoted 111 lines
--- tests/btrfs/214 | 66 +++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/214.out | 14 ++++++++++ tests/btrfs/group | 1 + 3 files changed, 81 insertions(+) create mode 100755 tests/btrfs/214 create mode 100644 tests/btrfs/214.outtests/btrfs/214 new file +66
-0@@ -0,0 +1,66 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) 2020 SUSE Linux Products GmbH. All Rights Reserved. +# +# FSQA Test No. 214 +# +# Test that creating a snapshot after writing to a file using a RWF_NOWAIT +# works, does not hang the snapshot creation task, and we are able to read +# the data after. +# +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter +. ./common/attr + +# real QA test starts here +_supported_fs btrfs +_supported_os Linux +_require_scratch +_require_odirect +_require_xfs_io_command pwrite -N +_require_chattr C + +rm -f $seqres.full + +_scratch_mkfs >>$seqres.full 2>&1 +_scratch_mount + +# RWF_NOWAIT writes require NOCOW +touch $SCRATCH_MNT/f +$CHATTR_PROG +C $SCRATCH_MNT/f + +$XFS_IO_PROG -d -c "pwrite -S 0xab 0 64K" $SCRATCH_MNT/f | _filter_xfs_io + +# Now do a WEF_WRITE into a range containing a NOCOWable extent. +$XFS_IO_PROG -d -c "pwrite -N -V 1 -S 0xfe 0 64K" $SCRATCH_MNT/f \ + | _filter_xfs_io + +$BTRFS_UTIL_PROG subvolume snapshot -r $SCRATCH_MNT $SCRATCH_MNT/snap \ + | _filter_scratch + +# Unmount, mount again and verify the file in the subvolume and snapshot has +# the correct data. +_scratch_cycle_mount + +echo "File data in the subvolume:" +od -A d -t x1 $SCRATCH_MNT/f + +echo "File data in the snapshot:" +od -A d -t x1 $SCRATCH_MNT/snap/f + +status=0 +exittests/btrfs/214.out new file +14
-0@@ -0,0 +1,14 @@ +QA output created by 214 +wrote 65536/65536 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 65536/65536 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +Create a readonly snapshot of 'SCRATCH_MNT' in 'SCRATCH_MNT/snap' +File data in the subvolume: +0000000 fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe +* +0065536 +File data in the snapshot: +0000000 fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe +* +0065536tests/btrfs/group +1
-1@@ -216,3 +216,4 @@ 211 auto quick log prealloc 212 auto balance dangerous 213 auto balance dangerous +214 auto quick snapshot --2.26.2