fstests: add a another gap extent testcase for btrfs
PATCH 1/1 fstests: add a another gap extent testcase for btrfs
This is a testcase for a corner that I missed when trying to fix gap extents for btrfs. We would end up with gaps if we hole punched past isize and then extended past the gap in a specific way. This is a simple reproducer to show the problem, and has been properly fixed by my patches now.
--- tests/btrfs/204 | 85 +++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/204.out | 5 +++ tests/btrfs/group | 1 + 3 files changed, 91 insertions(+) create mode 100755 tests/btrfs/204 create mode 100644 tests/btrfs/204.out
tests/btrfs/204 new file +85 -0
@@ -0,0 +1,85 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2020 Facebook. All Rights Reserved. +# +# FS QA Test 204 +# +# Validate that without no-holes we do not get a i_size that is after a gap in +# the file extents on disk when punching a hole past i_size. This is fixed by +# the following patches +# +# btrfs: use the file extent tree infrastructure +# btrfs: replace all uses of btrfs_ordered_update_i_size +# +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter +. ./common/dmlogwrites + +# remove previous $seqres.full before test +rm -f $seqres.full + +# real QA test starts here + +# Modify as appropriate. +_supported_fs generic +_supported_os Linux +_require_test +_require_scratch +_require_log_writes + +_log_writes_init $SCRATCH_DEV +_log_writes_mkfs "-O ^no-holes" >> $seqres.full 2>&1 + +# There's not a straightforward way to commit the transaction without also +# flushing dirty pages, so shorten the commit interval to 1 so we're sure to get +# a commit with our broken file +_log_writes_mount -o commit=1 + +# This creates a gap extent because fpunch doesn't insert hole extents past +# i_size +xfs_io -f -c "falloc -k 4k 8k" $SCRATCH_MNT/file +xfs_io -f -c "fpunch 4k 4k" $SCRATCH_MNT/file + +# The pwrite extends the i_size to cover the gap extent, and then the truncate +# sets the disk_i_size to 12k because it assumes everything was a-ok. +xfs_io -f -c "pwrite 0 4k" $SCRATCH_MNT/file | _filter_xfs_io +xfs_io -f -c "pwrite 0 8k" $SCRATCH_MNT/file | _filter_xfs_io +xfs_io -f -c "truncate 12k" $SCRATCH_MNT/file + +# Wait for a transaction commit +sleep 2 + +_log_writes_unmount +_log_writes_remove + +cur=$(_log_writes_find_next_fua 0) +echo "cur=$cur" >> $seqres.full +while [ ! -z "$cur" ]; do + _log_writes_replay_log_range $cur $SCRATCH_DEV >> $seqres.full + + # We only care about the fs consistency, so just run fsck, we don't have + # to mount the fs to validate it + _check_scratch_fs + + cur=$(_log_writes_find_next_fua $(($cur + 1))) +done + +# success, all done +status=0 +exit
tests/btrfs/204.out new file +5 -0
@@ -0,0 +1,5 @@ +QA output created by 204 +wrote 4096/4096 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 8192/8192 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
tests/btrfs/group +1 -0
@@ -206,3 +206,4 @@ 201 auto quick punch log 202 auto quick subvol snapshot 203 auto quick send clone +204 auto quick log replay
Signature · 1 line
2.24.1
Quoted 56 lines · Josef Bacik
This is a testcase for a corner that I missed when trying to fix gap extents for btrfs. We would end up with gaps if we hole punched past isize and then extended past the gap in a specific way. This is a simple reproducer to show the problem, and has been properly fixed by my patches now.
Signed-off-by Josef Bacik <[email protected]>--- tests/btrfs/204 | 85 +++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/204.out | 5 +++ tests/btrfs/group | 1 + 3 files changed, 91 insertions(+) create mode 100755 tests/btrfs/204 create mode 100644 tests/btrfs/204.outtests/btrfs/204 new file +34
-0@@ -0,0 +1,85 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2020 Facebook. All Rights Reserved. +# +# FS QA Test 204 +# +# Validate that without no-holes we do not get a i_size that is after a gap in +# the file extents on disk when punching a hole past i_size. This is fixed by +# the following patches +# +# btrfs: use the file extent tree infrastructure +# btrfs: replace all uses of btrfs_ordered_update_i_size +# +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter +. ./common/dmlogwrites + +# remove previous $seqres.full before test
tests/btrfs/204 line 44
Quoted 10 lines
tests/btrfs/204 new file +10
-0+rm -f $seqres.full + +# real QA test starts here + +# Modify as appropriate. +_supported_fs generic +_supported_os Linux +_require_test +_require_scratch +_require_log_writes
_require_xfs_io_command "falloc" "-k"_require_xfs_io_command "fpunch"
Quoted 57 lines
tests/btrfs/204 new file +41
-0+ +_log_writes_init $SCRATCH_DEV +_log_writes_mkfs "-O ^no-holes" >> $seqres.full 2>&1 + +# There's not a straightforward way to commit the transaction without also +# flushing dirty pages, so shorten the commit interval to 1 so we're sure to get +# a commit with our broken file +_log_writes_mount -o commit=1 + +# This creates a gap extent because fpunch doesn't insert hole extents past +# i_size +xfs_io -f -c "falloc -k 4k 8k" $SCRATCH_MNT/file +xfs_io -f -c "fpunch 4k 4k" $SCRATCH_MNT/file + +# The pwrite extends the i_size to cover the gap extent, and then the truncate +# sets the disk_i_size to 12k because it assumes everything was a-ok. +xfs_io -f -c "pwrite 0 4k" $SCRATCH_MNT/file | _filter_xfs_io +xfs_io -f -c "pwrite 0 8k" $SCRATCH_MNT/file | _filter_xfs_io +xfs_io -f -c "truncate 12k" $SCRATCH_MNT/file + +# Wait for a transaction commit +sleep 2 + +_log_writes_unmount +_log_writes_remove + +cur=$(_log_writes_find_next_fua 0) +echo "cur=$cur" >> $seqres.full +while [ ! -z "$cur" ]; do + _log_writes_replay_log_range $cur $SCRATCH_DEV >> $seqres.full + + # We only care about the fs consistency, so just run fsck, we don't have + # to mount the fs to validate it + _check_scratch_fs + + cur=$(_log_writes_find_next_fua $(($cur + 1))) +done + +# success, all done +status=0 +exittests/btrfs/204.out new file +5
-0@@ -0,0 +1,5 @@ +QA output created by 204 +wrote 4096/4096 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 8192/8192 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)tests/btrfs/group +0
-0
tests/btrfs/group line 209
Quoted 4 lines
tests/btrfs/group +1
-0@@ -206,3 +206,4 @@ 201 auto quick punch log 202 auto quick subvol snapshot 203 auto quick send clone +204 auto quick log replay
"prealloc" and "punch" groups as well.
Since this just tests another variant of the same problem, maybe it could be added to btrfs/172, since that test is very recent and you authored it as well.
Anyway, I don't have a strong preference.
The test itself looks good to me, and with the _require_xfs_io_command thing added and the groups (maybe Eryu can add these at commit time):
Quoted 3 lines
--
2.24.1
Signature · 3 lines
Filipe David Manana, “Whether you think you can, or you think you can't — you're right.”
Quoted 140 lines · Filipe Manana
Quoted 56 lines · Josef Bacik
This is a testcase for a corner that I missed when trying to fix gap extents for btrfs. We would end up with gaps if we hole punched past isize and then extended past the gap in a specific way. This is a simple reproducer to show the problem, and has been properly fixed by my patches now.
Signed-off-by Josef Bacik <[email protected]>--- tests/btrfs/204 | 85 +++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/204.out | 5 +++ tests/btrfs/group | 1 + 3 files changed, 91 insertions(+)create mode 100755 tests/btrfs/204
create mode 100644 tests/btrfs/204.outtests/btrfs/204 new file +34
-0@@ -0,0 +1,85 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2020 Facebook. All Rights Reserved. +# +# FS QA Test 204 +# +# Validate that without no-holes we do not get a i_size that is after a gap in +# the file extents on disk when punching a hole past i_size. This is fixed by +# the following patches +# +# btrfs: use the file extent tree infrastructure +# btrfs: replace all uses of btrfs_ordered_update_i_size +# +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter +. ./common/dmlogwrites + +# remove previous $seqres.full before testtests/btrfs/204 line 44
Quoted 10 lines
tests/btrfs/204 new file +10
-0+rm -f $seqres.full + +# real QA test starts here + +# Modify as appropriate. +_supported_fs generic +_supported_os Linux +_require_test +_require_scratch +_require_log_writes
_require_xfs_io_command"falloc" "-k"_require_xfs_io_command"fpunch"Quoted 57 lines
tests/btrfs/204 new file +41
-0+ +_log_writes_init $SCRATCH_DEV +_log_writes_mkfs "-O ^no-holes" >> $seqres.full 2>&1 + +# There's not a straightforward way to commit the transaction without also +# flushing dirty pages, so shorten the commit interval to 1 so we're sure to get +# a commit with our broken file +_log_writes_mount -o commit=1 + +# This creates a gap extent because fpunch doesn't insert hole extents past +# i_size +xfs_io -f -c "falloc -k 4k 8k" $SCRATCH_MNT/file +xfs_io -f -c "fpunch 4k 4k" $SCRATCH_MNT/file + +# The pwrite extends the i_size to cover the gap extent, and then the truncate +# sets the disk_i_size to 12k because it assumes everything was a-ok. +xfs_io -f -c "pwrite 0 4k" $SCRATCH_MNT/file | _filter_xfs_io +xfs_io -f -c "pwrite 0 8k" $SCRATCH_MNT/file | _filter_xfs_io +xfs_io -f -c "truncate 12k" $SCRATCH_MNT/file + +# Wait for a transaction commit +sleep 2 + +_log_writes_unmount +_log_writes_remove + +cur=$(_log_writes_find_next_fua 0) +echo "cur=$cur" >> $seqres.full +while [ ! -z "$cur" ]; do + _log_writes_replay_log_range $cur $SCRATCH_DEV >> $seqres.full + + # We only care about the fs consistency, so just run fsck, we don't have + # to mount the fs to validate it + _check_scratch_fs + + cur=$(_log_writes_find_next_fua $(($cur + 1))) +done + +# success, all done +status=0 +exittests/btrfs/204.out new file +5
-0@@ -0,0 +1,5 @@ +QA output created by 204 +wrote 4096/4096 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 8192/8192 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)tests/btrfs/group +0
-0tests/btrfs/group line 209
Quoted 4 lines
tests/btrfs/group +1
-0@@ -206,3 +206,4 @@ 201 auto quick punch log 202 auto quick subvol snapshot 203 auto quick send clone +204 auto quick log replay"prealloc" and "punch" groups as well.
Since this just tests another variant of the same problem, maybe it could be added to btrfs/172, since that test is very recent and you authored it as well.
Anyway, I don't have a strong preference.
Quoted 5 lines
The test itself looks good to me, and with the
_require_xfs_io_commandthing added and the groups (maybe Eryu can add these at commit time):Reviewed-by Filipe Manana <[email protected]>
I like to keep these things discrete, if a test is testing two different things and it fails I have to go comment out one part and re-run to figure out which actually failed. Thanks,
Josef
Quoted 55 lines · Filipe Manana
Quoted 54 lines · Josef Bacik
This is a testcase for a corner that I missed when trying to fix gap extents for btrfs. We would end up with gaps if we hole punched past isize and then extended past the gap in a specific way. This is a simple reproducer to show the problem, and has been properly fixed by my patches now.
Signed-off-by Josef Bacik <[email protected]>--- tests/btrfs/204 | 85 +++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/204.out | 5 +++ tests/btrfs/group | 1 + 3 files changed, 91 insertions(+) create mode 100755 tests/btrfs/204 create mode 100644 tests/btrfs/204.outtests/btrfs/204 new file +32
-0@@ -0,0 +1,85 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2020 Facebook. All Rights Reserved. +# +# FS QA Test 204 +# +# Validate that without no-holes we do not get a i_size that is after a gap in +# the file extents on disk when punching a hole past i_size. This is fixed by +# the following patches +# +# btrfs: use the file extent tree infrastructure +# btrfs: replace all uses of btrfs_ordered_update_i_size +# +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter +. ./common/dmlogwrites
Quoted 8 lines
tests/btrfs/204 line 40
Quoted 8 lines
tests/btrfs/204 new file +8
-0+ +# remove previous $seqres.full before test +rm -f $seqres.full + +# real QA test starts here + +# Modify as appropriate. +_supported_fs generic
I also changed 'generic' to 'btrfs'
Quoted 7 lines
tests/btrfs/204 line 44
Quoted 4 lines
tests/btrfs/204 new file +4
-0+_supported_os Linux +_require_test +_require_scratch +_require_log_writes
_require_xfs_io_command"falloc" "-k"_require_xfs_io_command"fpunch"
Added.
Quoted 12 lines
Quoted 12 lines
tests/btrfs/204 new file +11
-0+ +_log_writes_init $SCRATCH_DEV +_log_writes_mkfs "-O ^no-holes" >> $seqres.full 2>&1 + +# There's not a straightforward way to commit the transaction without also +# flushing dirty pages, so shorten the commit interval to 1 so we're sure to get +# a commit with our broken file +_log_writes_mount -o commit=1 + +# This creates a gap extent because fpunch doesn't insert hole extents past +# i_size
Quoted 8 lines
tests/btrfs/204 line 63
Quoted 8 lines
tests/btrfs/204 new file +8
-0+xfs_io -f -c "falloc -k 4k 8k" $SCRATCH_MNT/file +xfs_io -f -c "fpunch 4k 4k" $SCRATCH_MNT/file + +# The pwrite extends the i_size to cover the gap extent, and then the truncate +# sets the disk_i_size to 12k because it assumes everything was a-ok. +xfs_io -f -c "pwrite 0 4k" $SCRATCH_MNT/file | _filter_xfs_io +xfs_io -f -c "pwrite 0 8k" $SCRATCH_MNT/file | _filter_xfs_io +xfs_io -f -c "truncate 12k" $SCRATCH_MNT/file
Changed all 'xfs_io' to '$XFS_IO_PROG'.
Quoted 36 lines
Quoted 36 lines
tests/btrfs/204 new file +22
-0+ +# Wait for a transaction commit +sleep 2 + +_log_writes_unmount +_log_writes_remove + +cur=$(_log_writes_find_next_fua 0) +echo "cur=$cur" >> $seqres.full +while [ ! -z "$cur" ]; do + _log_writes_replay_log_range $cur $SCRATCH_DEV >> $seqres.full + + # We only care about the fs consistency, so just run fsck, we don't have + # to mount the fs to validate it + _check_scratch_fs + + cur=$(_log_writes_find_next_fua $(($cur + 1))) +done + +# success, all done +status=0 +exittests/btrfs/204.out new file +5
-0@@ -0,0 +1,5 @@ +QA output created by 204 +wrote 4096/4096 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 8192/8192 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)tests/btrfs/group +0
-1--- a/tests/btrfs/group
Quoted 8 lines
tests/btrfs/group line 209
Quoted 6 lines
+++ b/tests/btrfs/group
tests/btrfs/group +1
-0@@ -206,3 +206,4 @@ 201 auto quick punch log 202 auto quick subvol snapshot 203 auto quick send clone +204 auto quick log replay"prealloc" and "punch" groups as well.
Added.
Quoted 10 lines
Since this just tests another variant of the same problem, maybe it could be added to btrfs/172, since that test is very recent and you authored it as well.
Anyway, I don't have a strong preference.The test itself looks good to me, and with the
_require_xfs_io_commandthing added and the groups (maybe Eryu can add these at commit time):Reviewed-by Filipe Manana <[email protected]>
Thanks for the review!
Eryu
Quoted 11 lines
Quoted 5 lines
--
2.24.1Signature · 3 lines
Filipe David Manana, “Whether you think you can, or you think you can't — you're right.”