__btrfs_drop_extents() BUG_ON reproducer
cover __btrfs_drop_extents() BUG_ON reproducer
This patch-set provides a reproducer for hitting the 3.14.0-rc1 BUG_ON() at:
692 int __btrfs_drop_extents(struct btrfs_trans_handle *trans,
...
839 /*
840 * | ---- range to drop ----- | 841 * | -------- extent -------- | 842 */
843 if (start <= key.offset && end < extent_end) {
844 BUG_ON(extent_type == BTRFS_FILE_EXTENT_INLINE);
845
846 memcpy(&new_key, &key, sizeof(new_key));
The first patch adds a small cloner binary which is used by btrfs/035 to dispatch BTRFS_IOC_CLONE_RANGE requests.
This workload resembles that of Samba's vfs_btrfs module, when a Windows client restores a file from a shadow-copy (snapshot) using server-side copy requests.
Feedback appreciated.
Cheers, David
src/Makefile | 2 +- src/cloner.c | 168 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/035 | 76 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/035.out | 3 +++ tests/btrfs/group | 1 + 5 files changed, 249 insertions(+), 1 deletion(-)
PATCH 1/2 btrfs: add small program for clone testing
The cloner program is capable of cloning files using the BTRFS_IOC_CLONE and BTRFS_IOC_CLONE_RANGE ioctls.
--- src/Makefile | 2 +- src/cloner.c | 168 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 169 insertions(+), 1 deletion(-) create mode 100644 src/cloner.c
src/Makefile +1 -1
@@ -18,7 +18,7 @@ LINUX_TARGETS = xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader \ locktest unwritten_mmap bulkstat_unlink_test t_stripealign \ bulkstat_unlink_test_modified t_dir_offset t_futimens t_immutable \ stale_handle pwrite_mmap_blocked t_dir_offset2 seek_sanity_test \ - seek_copy_test t_readdir_1 t_readdir_2 fsync-tester nsexec + seek_copy_test t_readdir_1 t_readdir_2 fsync-tester nsexec cloner SUBDIRS =
src/cloner.c new file +168 -0
168 lines not on this page · raw message
Signature · 1 line
1.8.4.5
Re: [PATCH 1/2] btrfs: add small program for clone testing
Quoted 4 lines · David Disseldorp
The cloner program is capable of cloning files using the
BTRFS_IOC_CLONEandBTRFS_IOC_CLONE_RANGEioctls.Signed-off-by David Disseldorp <[email protected]>
Hi Dave - long time since I've seen your head pop up around here ;)
A few comments below.
src/cloner.c line 44
Quoted 11 lines
src/cloner.c new file +11
-0+struct btrfs_ioctl_clone_range_args { +int64_t src_fd; +uint64_t src_offset; +uint64_t src_length; +uint64_t dest_offset; +}; + +#define BTRFS_IOCTL_MAGIC 0x94 +#define BTRFS_IOC_CLONE _IOW(BTRFS_IOCTL_MAGIC, 9, int) +#define BTRFS_IOC_CLONE_RANGE _IOW(BTRFS_IOCTL_MAGIC, 13, \ + struct btrfs_ioctl_clone_range_args)
Is there some published header file that these belong to? i.e. somewhere in the include/linux/uapi/ kernel directory? Normally the way to handle this sort of thing is by autoconf - if the header file exists, then we include it, otherwise we use the manual definitions. This just means that if the public api ever changes, we'll pick it up automatically in future...
Quoted 8 lines
src/cloner.c new file +8
-0+int +main (int argc,char **argv) +{ +bool full_file = true; +uint64_t src_off =0 ; +uint64_t dst_off =0 ; +uint64_t len =0 ; +char *src_file;
src/cloner.c line 105
Quoted 10 lines
src/cloner.c new file +10
-0+int src_fd; +char *dst_file; +int dst_fd; +int ret; +int opt; + +while ((opt =getopt (argc, argv,"s:d:l:" )) !=-1 ) { +switch (opt) { +case 's' : + src_off =atoi (optarg);
atoi() only returns 32 bit numbers. You probably should use strtoull() as the offset parameters are 64 bit.
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
Re: [PATCH 1/2] btrfs: add small program for clone testing
Hi Dave,
Quoted 7 lines · Dave Chinner
Quoted 4 lines · David Disseldorp
The cloner program is capable of cloning files using the
BTRFS_IOC_CLONEandBTRFS_IOC_CLONE_RANGEioctls.Signed-off-by David Disseldorp <[email protected]>Hi Dave - long time since I've seen your head pop up around here ;)
Indeed, it's been a while. Thanks for the review :)
Quoted 13 lines
A few comments below.
Quoted 10 lines
src/cloner.c new file +10
-0+struct btrfs_ioctl_clone_range_args { +int64_t src_fd; +uint64_t src_offset; +uint64_t src_length; +uint64_t dest_offset; +}; + +#define BTRFS_IOCTL_MAGIC 0x94 +#define BTRFS_IOC_CLONE _IOW(BTRFS_IOCTL_MAGIC, 9, int) +#define BTRFS_IOC_CLONE_RANGE _IOW(BTRFS_IOCTL_MAGIC, 13, \
Quoted 8 lines
src/cloner.c line 44
Quoted 1 line
src/cloner.c new file +1
-0+ struct btrfs_ioctl_clone_range_args)Is there some published header file that these belong to? i.e. somewhere in the
include/linux/uapi/kernel directory? Normally the way to handle this sort of thing is by autoconf - if the header file exists, then we include it, otherwise we use the manual definitions. This just means that if the public api ever changes, we'll pick it up automatically in future...
I'd wanted to avoid the addition of another xfsqa prereq, but I guess it'll work with the fall-back.
I'll add the autoconf logic to the next round, along with changes addressing your other remarks.
Cheers, David
PATCH 2/2 btrfs/035: add new clone overwrite regression test
In reply to David DisseldorpThis test uses the newly added cloner binary to dispatch full file and range specific clone (reflink) requests.
--- tests/btrfs/035 | 76 +++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/035.out | 3 +++ tests/btrfs/group | 1 + 3 files changed, 80 insertions(+) create mode 100755 tests/btrfs/035 create mode 100644 tests/btrfs/035.out
tests/btrfs/035 new file +76 -0
@@ -0,0 +1,76 @@ +#!/bin/bash +# FS QA Test No. btrfs/035 +# +# Regression test for overwriting clones +# +#----------------------------------------------------------------------- +# Copyright (C) 2014 SUSE Linux Products GmbH. All Rights Reserved. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +# +#----------------------------------------------------------------------- +# + +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! + +_cleanup() +{ + rm -f $tmp.* +} + +trap "_cleanup ; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter + +# real QA test starts here +_supported_fs btrfs +_supported_os Linux +_require_scratch + +_scratch_mkfs > /dev/null 2>&1 +_scratch_mount + +CLONER_PROG=$here/src/cloner + +src_str="aaaaaaaaaa" + +echo -n "$src_str" > $SCRATCH_MNT/src || _fail "failed to create src" + +$CLONER_PROG $SCRATCH_MNT/src $SCRATCH_MNT/src.clone1 + +src_str="bbbbbbbbbbcccccccccc" + +echo -n "$src_str" > $SCRATCH_MNT/src || _fail "failed to create src" + +$CLONER_PROG $SCRATCH_MNT/src $SCRATCH_MNT/src.clone2 + +snap_src_sz=`ls -lah $SCRATCH_MNT/src.clone1 | awk '{print $5}'` +echo "attempting ioctl (src.clone1 src)" +$CLONER_PROG -s 0 -d 0 -l ${snap_src_sz} \ + $SCRATCH_MNT/src.clone1 $SCRATCH_MNT/src || _fail "ioctl failed" + +snap_src_sz=`ls -lah $SCRATCH_MNT/src.clone2 | awk '{print $5}'` +echo "attempting ioctl (src.clone2 src)" +$CLONER_PROG -s 0 -d 0 -l ${snap_src_sz} \ + $SCRATCH_MNT/src.clone2 $SCRATCH_MNT/src || _fail "ioctl failed" + +status=0 ; exit
tests/btrfs/035.out new file +3 -0
@@ -0,0 +1,3 @@ +QA output created by 035 +attempting ioctl (src.clone1 src) +attempting ioctl (src.clone2 src)
tests/btrfs/group +1 -0
@@ -37,3 +37,4 @@ 032 auto quick 033 auto quick 034 auto quick +035 auto quick
Signature · 1 line
1.8.4.5
Re: [PATCH 2/2] btrfs/035: add new clone overwrite regression test
Quoted 2 lines · David Disseldorp
This test uses the newly added cloner binary to dispatch full file and range specific clone (reflink) requests.
A couple of small nits:
Quoted 1 line
+CLONER_PROG=$here/src/cloner
Need to test that the binary was build and is present.
tests/btrfs/035 line 56
Quoted 4 lines
tests/btrfs/035 new file +4
-0+ +src_str="aaaaaaaaaa" + +echo -n "$src_str" > $SCRATCH_MNT/src || _fail "failed to create src"
No need for the "|| _fail ..." in any part of this test. Failures will be caught in the output and hence cause golden output mismatches.
Letting the test run even after a failure exercises the filesystem in interesting ways, so it's worthwhile ignoring failures in the test and letting the harness pick up the failures through error messages.
tests/btrfs/035 line 69
Quoted 12 lines
+$CLONER_PROG $SCRATCH_MNT/src $SCRATCH_MNT/src.clone1
tests/btrfs/035 new file +11
-0+ +src_str="bbbbbbbbbbcccccccccc" + +echo -n "$src_str" > $SCRATCH_MNT/src || _fail "failed to create src" + +$CLONER_PROG $SCRATCH_MNT/src $SCRATCH_MNT/src.clone2 + +snap_src_sz=`ls -lah $SCRATCH_MNT/src.clone1 | awk '{print $5}'` +echo "attempting ioctl (src.clone1 src)" +$CLONER_PROG -s 0 -d 0 -l ${snap_src_sz} \ + $SCRATCH_MNT/src.clone1 $SCRATCH_MNT/src || _fail "ioctl failed"
And to do that here, you probably need to add perror() output to the cloner program when it detects an error. i.e. let it give you the exact error that was detected, rather than lumping them all into a catchall here...
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]