Skip to contentSkip to conversation

__btrfs_drop_extents() BUG_ON reproducer

PATCH Superseded David Disseldorp 2 participants 6 messages Feb 5, 2014 – Feb 6, 2014 Btrfs file system

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

cover __btrfs_drop_extents() BUG_ON reproducer

This patch-set provides a reproducer for hitting the 3.14.0-rc1 BUG_ON() at:
692 int __btrfs_drop_extents(struct btrfs_trans_handle *trans,
...
839 /*

 840                  *  | ---- range to drop ----- |
 841                  *      | -------- extent -------- |
 842                  */

843 if (start <= key.offset && end < extent_end) {
844 BUG_ON(extent_type == BTRFS_FILE_EXTENT_INLINE);
845
846 memcpy(&new_key, &key, sizeof(new_key));

The first patch adds a small cloner binary which is used by btrfs/035 to dispatch BTRFS_IOC_CLONE_RANGE requests.

This workload resembles that of Samba's vfs_btrfs module, when a Windows client restores a file from a shadow-copy (snapshot) using server-side copy requests.

Feedback appreciated.

Cheers, David

 src/Makefile        |   2 +-
 src/cloner.c        | 168 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/035     |  76 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/035.out |   3 +++
 tests/btrfs/group   |   1 +
 5 files changed, 249 insertions(+), 1 deletion(-)
#1 lore raw

PATCH 1/2 btrfs: add small program for clone testing

The cloner program is capable of cloning files using the BTRFS_IOC_CLONE and BTRFS_IOC_CLONE_RANGE ioctls.

---
 src/Makefile |   2 +-
 src/cloner.c | 168 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 169 insertions(+), 1 deletion(-)
 create mode 100644 src/cloner.c
src/Makefile +1 -1
index 84c8297..6509f2d 100644
@@ -18,7 +18,7 @@ LINUX_TARGETS = xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader \
 	locktest unwritten_mmap bulkstat_unlink_test t_stripealign \
 	bulkstat_unlink_test_modified t_dir_offset t_futimens t_immutable \
 	stale_handle pwrite_mmap_blocked t_dir_offset2 seek_sanity_test \
-	seek_copy_test t_readdir_1 t_readdir_2 fsync-tester nsexec
+	seek_copy_test t_readdir_1 t_readdir_2 fsync-tester nsexec cloner
 
 SUBDIRS =
 
src/cloner.c new file +168 -0

168 lines not on this page · raw message

Signature · 1 line
1.8.4.5
#2 lore raw

Re: [PATCH 1/2] btrfs: add small program for clone testing

Quoted 4 lines · David Disseldorp

The cloner program is capable of cloning files using the BTRFS_IOC_CLONE and BTRFS_IOC_CLONE_RANGE ioctls.

Hi Dave - long time since I've seen your head pop up around here ;)

A few comments below.

src/cloner.c line 44

Quoted 11 lines
src/cloner.c new file +11 -0
+struct btrfs_ioctl_clone_range_args {
+	int64_t src_fd;
+	uint64_t src_offset;
+	uint64_t src_length;
+	uint64_t dest_offset;
+};
+
+#define BTRFS_IOCTL_MAGIC 0x94
+#define BTRFS_IOC_CLONE       _IOW(BTRFS_IOCTL_MAGIC, 9, int)
+#define BTRFS_IOC_CLONE_RANGE _IOW(BTRFS_IOCTL_MAGIC, 13, \
+				   struct btrfs_ioctl_clone_range_args)

Is there some published header file that these belong to? i.e. somewhere in the include/linux/uapi/ kernel directory? Normally the way to handle this sort of thing is by autoconf - if the header file exists, then we include it, otherwise we use the manual definitions. This just means that if the public api ever changes, we'll pick it up automatically in future...

Quoted 8 lines
src/cloner.c new file +8 -0
+int
+main(int argc, char **argv)
+{
+	bool full_file = true;
+	uint64_t src_off = 0;
+	uint64_t dst_off = 0;
+	uint64_t len = 0;
+	char *src_file;

src/cloner.c line 105

Quoted 10 lines
src/cloner.c new file +10 -0
+	int src_fd;
+	char *dst_file;
+	int dst_fd;
+	int ret;
+	int opt;
+
+	while ((opt = getopt(argc, argv, "s:d:l:")) != -1) {
+		switch (opt) {
+		case 's':
+			src_off = atoi(optarg);

atoi() only returns 32 bit numbers. You probably should use strtoull() as the offset parameters are 64 bit.

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#3 lore raw

Re: [PATCH 1/2] btrfs: add small program for clone testing

Hi Dave,

Quoted 7 lines · Dave Chinner
Quoted 4 lines · David Disseldorp

The cloner program is capable of cloning files using the BTRFS_IOC_CLONE and BTRFS_IOC_CLONE_RANGE ioctls.

Hi Dave - long time since I've seen your head pop up around here ;)

Indeed, it's been a while. Thanks for the review :)

Quoted 13 lines

A few comments below.

Quoted 10 lines
src/cloner.c new file +10 -0
+struct btrfs_ioctl_clone_range_args {
+	int64_t src_fd;
+	uint64_t src_offset;
+	uint64_t src_length;
+	uint64_t dest_offset;
+};
+
+#define BTRFS_IOCTL_MAGIC 0x94
+#define BTRFS_IOC_CLONE       _IOW(BTRFS_IOCTL_MAGIC, 9, int)
+#define BTRFS_IOC_CLONE_RANGE _IOW(BTRFS_IOCTL_MAGIC, 13, \
Quoted 8 lines

src/cloner.c line 44

Quoted 1 line
src/cloner.c new file +1 -0
+				   struct btrfs_ioctl_clone_range_args)

Is there some published header file that these belong to? i.e. somewhere in the include/linux/uapi/ kernel directory? Normally the way to handle this sort of thing is by autoconf - if the header file exists, then we include it, otherwise we use the manual definitions. This just means that if the public api ever changes, we'll pick it up automatically in future...

I'd wanted to avoid the addition of another xfsqa prereq, but I guess it'll work with the fall-back.
I'll add the autoconf logic to the next round, along with changes addressing your other remarks.

Cheers, David

#4 lore raw

PATCH 2/2 btrfs/035: add new clone overwrite regression test

In reply to David Disseldorp

This test uses the newly added cloner binary to dispatch full file and range specific clone (reflink) requests.

---
 tests/btrfs/035     | 76 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/035.out |  3 +++
 tests/btrfs/group   |  1 +
 3 files changed, 80 insertions(+)
 create mode 100755 tests/btrfs/035
 create mode 100644 tests/btrfs/035.out
tests/btrfs/035 new file +76 -0
new file mode 100755
index 0000000..03c2cd3
@@ -0,0 +1,76 @@
+#!/bin/bash
+# FS QA Test No. btrfs/035
+#
+# Regression test for overwriting clones
+#
+#-----------------------------------------------------------------------
+# Copyright (C) 2014 SUSE Linux Products GmbH. All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#
+#-----------------------------------------------------------------------
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+
+_cleanup()
+{
+    rm -f $tmp.*
+}
+
+trap "_cleanup ; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# real QA test starts here
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+
+_scratch_mkfs > /dev/null 2>&1
+_scratch_mount
+
+CLONER_PROG=$here/src/cloner
+
+src_str="aaaaaaaaaa"
+
+echo -n "$src_str" > $SCRATCH_MNT/src || _fail "failed to create src"
+
+$CLONER_PROG $SCRATCH_MNT/src  $SCRATCH_MNT/src.clone1
+
+src_str="bbbbbbbbbbcccccccccc"
+
+echo -n "$src_str" > $SCRATCH_MNT/src || _fail "failed to create src"
+
+$CLONER_PROG $SCRATCH_MNT/src $SCRATCH_MNT/src.clone2
+
+snap_src_sz=`ls -lah $SCRATCH_MNT/src.clone1 | awk '{print $5}'`
+echo "attempting ioctl (src.clone1 src)"
+$CLONER_PROG -s 0 -d 0 -l ${snap_src_sz} \
+	$SCRATCH_MNT/src.clone1 $SCRATCH_MNT/src || _fail "ioctl failed"
+
+snap_src_sz=`ls -lah $SCRATCH_MNT/src.clone2 | awk '{print $5}'`
+echo "attempting ioctl (src.clone2 src)"
+$CLONER_PROG -s 0 -d 0 -l ${snap_src_sz} \
+	$SCRATCH_MNT/src.clone2 $SCRATCH_MNT/src || _fail "ioctl failed"
+
+status=0 ; exit
tests/btrfs/035.out new file +3 -0
new file mode 100644
index 0000000..f86cadf
@@ -0,0 +1,3 @@
+QA output created by 035
+attempting ioctl (src.clone1 src)
+attempting ioctl (src.clone2 src)
tests/btrfs/group +1 -0
index f9f062f..bee57cb 100644
@@ -37,3 +37,4 @@
 032 auto quick
 033 auto quick
 034 auto quick
+035 auto quick
Signature · 1 line
1.8.4.5
#5 lore raw

Re: [PATCH 2/2] btrfs/035: add new clone overwrite regression test

Quoted 2 lines · David Disseldorp

This test uses the newly added cloner binary to dispatch full file and range specific clone (reflink) requests.

A couple of small nits:

Quoted 1 line

+CLONER_PROG=$here/src/cloner

Need to test that the binary was build and is present.

tests/btrfs/035 line 56

Quoted 4 lines
tests/btrfs/035 new file +4 -0
+
+src_str="aaaaaaaaaa"
+
+echo -n "$src_str" > $SCRATCH_MNT/src || _fail "failed to create src"

No need for the "|| _fail ..." in any part of this test. Failures will be caught in the output and hence cause golden output mismatches.

Letting the test run even after a failure exercises the filesystem in interesting ways, so it's worthwhile ignoring failures in the test and letting the harness pick up the failures through error messages.

tests/btrfs/035 line 69

Quoted 12 lines

+$CLONER_PROG $SCRATCH_MNT/src $SCRATCH_MNT/src.clone1

tests/btrfs/035 new file +11 -0
+
+src_str="bbbbbbbbbbcccccccccc"
+
+echo -n "$src_str" > $SCRATCH_MNT/src || _fail "failed to create src"
+
+$CLONER_PROG $SCRATCH_MNT/src $SCRATCH_MNT/src.clone2
+
+snap_src_sz=`ls -lah $SCRATCH_MNT/src.clone1 | awk '{print $5}'`
+echo "attempting ioctl (src.clone1 src)"
+$CLONER_PROG -s 0 -d 0 -l ${snap_src_sz} \
+	$SCRATCH_MNT/src.clone1 $SCRATCH_MNT/src || _fail "ioctl failed"

And to do that here, you probably need to add perror() output to the cloner program when it detects an error. i.e. let it give you the exact error that was detected, rather than lumping them all into a catchall here...

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]