Skip to contentSkip to conversation

Initial CephFS tests

PATCH New Luis Henriques 1 participant 4 messages Oct 19, 2020

lore
Share & tools
lore
#0 lore raw

v2 cover Initial CephFS tests

This is my second attempt to have an initial set of ceph-specific tests merged into fstests. In this patchset I'm pushing a different set of tests, focusing on the copy_file_range testing, although I *do* plan to get back to the quota tests soon.

This syscall has a few peculiarities in ceph as it is able to use remote object copies without the need to download/upload data from the OSDs. However, in order to take advantage of this remote copy, the copy ranges and sizes need to include at least one object. Thus, all the currently existing generic tests won't actually take advantage of this feature.

Let me know any comments/concerns about this patchset. Also note that currently, in order to have cephfs copy_file_range to take advantage of remote object copies, the additional 'copyfrom' mount parameter is required; if not used, the copy will fallback to the default VFS implementation. (Hopefully this additional param will be dropped in the future.)

Changes since v1:

- New _ceph_create_file_layout() function (in common/ceph) that creates and sets the file layout
- Added commit IDs relevant to tracker issue#37378 (test 002)
- Fixed tests file mode to 755

(Also clarified cover-letter text regarding the 'copyfrom' usage.)

Luis Henriques (3):
ceph: add copy_file_range (remote copy operation) testing
ceph: test combination of copy_file_range with truncate
ceph: test copy_file_range with infile = outfile

 common/ceph        |  23 +++++
 common/rc          |   1 +
 tests/ceph/001     | 233 +++++++++++++++++++++++++++++++++++++++++++++
 tests/ceph/001.out | 129 +++++++++++++++++++++++++
 tests/ceph/002     |  79 +++++++++++++++
 tests/ceph/002.out |   8 ++
 tests/ceph/003     | 116 ++++++++++++++++++++++
 tests/ceph/003.out |  11 +++
 tests/ceph/group   |   3 +
 9 files changed, 603 insertions(+)
 create mode 100644 common/ceph
 create mode 100755 tests/ceph/001
 create mode 100644 tests/ceph/001.out
 create mode 100755 tests/ceph/002
 create mode 100644 tests/ceph/002.out
 create mode 100755 tests/ceph/003
 create mode 100644 tests/ceph/003.out
 create mode 100644 tests/ceph/group
#1 lore raw

PATCH v2 1/3 ceph: add copy_file_range (remote copy operation) testing

From: Luis Henriques <[email protected]>

Test remote copy operation (CEPH_OSD_OP_COPY_FROM) with different combinations of both object sizes and copy sizes.

Test remote copy operation (CEPH_OSD_OP_COPY_FROM) with several combinations of both object sizes and copy sizes. It also uses several combinations of copy ranges. For example, copying the 1st object in the src file into:

1) the beginning (1st object) of dst file,
2) the end (last object) of dst file and
3) the middle of the dst file.

---
 common/ceph        |  23 +++++
 common/rc          |   1 +
 tests/ceph/001     | 233 +++++++++++++++++++++++++++++++++++++++++++++
 tests/ceph/001.out | 129 +++++++++++++++++++++++++
 tests/ceph/group   |   1 +
 5 files changed, 387 insertions(+)
 create mode 100644 common/ceph
 create mode 100755 tests/ceph/001
 create mode 100644 tests/ceph/001.out
 create mode 100644 tests/ceph/group
common/ceph new file +23 -0
new file mode 100644
index 000000000000..f80209f98b23
@@ -0,0 +1,23 @@
+#
+# CephFS specific common functions.
+#
+
+# _ceph_create_file_layout <filename> <stripe unit> <stripe count> <object size>
+# This function creates a new empty file and sets the file layout according to
+# parameters.  It will exit if the file already exists.
+_ceph_create_file_layout()
+{
+	local fname=$1
+	local stripe_unit=$2
+	local stripe_count=$3
+	local obj_size=$4
+
+	if [ -e $fname ]; then
+		echo "File $fname already exists."
+		exit 1
+	fi
+	touch $fname
+	$SETFATTR_PROG -n ceph.file.layout \
+		-v "stripe_unit=$objsz stripe_count=1 object_size=$objsz" \
+		$fname
+}
common/rc +1 -0
index 27a27ea36f75..65ebfe201139 100644
@@ -138,6 +138,7 @@ case "$FSTYP" in
     9p)
 	 ;;
     ceph)
+	 . ./common/ceph
 	 ;;
     glusterfs)
 	 ;;
tests/ceph/001 new file +233 -0

233 lines not on this page · raw message

tests/ceph/001.out new file +129 -0

129 lines not on this page · raw message

tests/ceph/group new file +1 -0
new file mode 100644
index 000000000000..11f0b9ad03d3
@@ -0,0 +1 @@
+001 auto quick copy
#2 lore raw

PATCH v2 2/3 ceph: test combination of copy_file_range with truncate

In reply to Luis Henriques

From: Luis Henriques <[email protected]>

This tests a bug found while testing copy_file_range. This bug was an issue with how the OSDs handled the truncate_seq value, which was being copied from the original object into the destination object. This test ensures the kernel client correctly handles fixed/non-fixed OSDs.

---
 tests/ceph/002     | 79 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/ceph/002.out |  8 +++++
 tests/ceph/group   |  1 +
 3 files changed, 88 insertions(+)
 create mode 100755 tests/ceph/002
 create mode 100644 tests/ceph/002.out
tests/ceph/002 new file +79 -0
new file mode 100755
index 000000000000..f0fd28a14991
@@ -0,0 +1,79 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2020 SUSE Linux Products GmbH. All Rights Reserved.
+#
+# FS QA Test No. ceph/002
+#
+# Test bug found while testing copy_file_range.
+#
+# This bug was an issue with how the OSDs handled the truncate_seq, copying it
+# from the original object into the destination object.  This test ensures the
+# kernel client correctly handles fixed/non-fixed OSDs.
+#
+# The bug was tracked here:
+#
+#   https://tracker.ceph.com/issues/37378
+#
+# The most relevant commits are:
+#
+#   ceph OSD:     dcd6a99ef9f5 ("osd: add new 'copy-from2' operation")
+#   linux kernel: 78beb0ff2fec ("ceph: use copy-from2 op in copy_file_range")
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1    # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -rf $tmp.*
+}
+
+# get standard environment
+. common/rc
+. common/filter
+. common/attr
+
+# real QA test starts here
+_supported_fs ceph
+
+_require_xfs_io_command "copy_range"
+_require_attrs
+_require_test
+
+workdir=$TEST_DIR/test-$seq
+rm -rf $workdir
+mkdir $workdir
+rm -f $seqres.full
+
+# Use 4M object size
+objsz=4194304
+file="$workdir/file-$objsz"
+dest="$workdir/dest-$objsz"
+
+# object_size has to be a multiple of stripe_unit
+_ceph_create_file_layout $file $objsz 1 $objsz
+_ceph_create_file_layout $dest $objsz 1 $objsz
+
+# Create a 3 objects size files
+$XFS_IO_PROG -c "pwrite -S 0x61 0 $objsz" $file >> $seqres.full 2>&1
+$XFS_IO_PROG -c "pwrite -S 0x62 $objsz $objsz" $file >> $seqres.full 2>&1
+$XFS_IO_PROG -c "pwrite -S 0x63 $(($objsz * 2)) $objsz" $file >> $seqres.full 2>&1
+
+$XFS_IO_PROG -c "pwrite -S 0x64 0 $(($objsz * 3))" $dest >> $seqres.full 2>&1
+# Truncate the destination file (messing up with the truncate_seq)
+$XFS_IO_PROG -c "truncate 0" $dest >> $seqres.full 2>&1
+
+# copy the whole file over
+$XFS_IO_PROG -c "copy_range -s 0 -d 0 -l $(($objsz * 3)) $file" "$dest"
+
+hexdump $dest
+
+#success, all done
+status=0
+exit
tests/ceph/002.out new file +8 -0
new file mode 100644
index 000000000000..6f067250afff
@@ -0,0 +1,8 @@
+QA output created by 002
+0000000 6161 6161 6161 6161 6161 6161 6161 6161
+*
+0400000 6262 6262 6262 6262 6262 6262 6262 6262
+*
+0800000 6363 6363 6363 6363 6363 6363 6363 6363
+*
+0c00000
tests/ceph/group +1 -0
index 11f0b9ad03d3..c28fe473c1a4 100644
@@ -1 +1,2 @@
 001 auto quick copy
+002 auto quick copy
#3 lore raw

PATCH v2 3/3 ceph: test copy_file_range with infile = outfile

In reply to Luis Henriques

From: Luis Henriques <[email protected]>

This runs a set of simple tests where the infile file is the same as the outfile.

---
 tests/ceph/003     | 116 +++++++++++++++++++++++++++++++++++++++++++++
 tests/ceph/003.out |  11 +++++
 tests/ceph/group   |   1 +
 3 files changed, 128 insertions(+)
 create mode 100755 tests/ceph/003
 create mode 100644 tests/ceph/003.out
tests/ceph/003 new file +116 -0
new file mode 100755
index 000000000000..6a5a4ea429f8
@@ -0,0 +1,116 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2020 SUSE Linux Products GmbH. All Rights Reserved.
+#
+# FS QA Test No. ceph/005
+#
+# Test copy_file_range with infile = outfile
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1    # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -rf $tmp.*
+}
+
+# get standard environment
+. common/rc
+. common/filter
+. common/attr
+. common/reflink
+
+# real QA test starts here
+_supported_fs ceph
+
+_require_xfs_io_command "copy_range"
+_require_attrs
+_require_test
+
+workdir=$TEST_DIR/test-$seq
+rm -rf $workdir
+mkdir $workdir
+rm -f $seqres.full
+
+check_range()
+{
+	local file=$1
+	local off0=$2
+	local off1=$3
+	local val=$4
+	_read_range $file $off0 $off1 | grep -v -q $val
+	[ $? -eq 0 ] && echo "file $file is not '$val' in [ $off0 $off1 ]"
+}
+
+objsz=4194304
+halfobj=$(($objsz / 2))
+file="$workdir/file-$objsz"
+copy="$workdir/copy-$objsz"
+dest="$workdir/dest-$objsz"
+backup="$file.backup"
+
+# object_size has to be a multiple of stripe_unit
+_ceph_create_file_layout $file $objsz 1 $objsz
+_ceph_create_file_layout $backup $objsz 1 $objsz
+
+$XFS_IO_PROG -c "pwrite -S 0x61 0 $objsz" $file >> $seqres.full 2>&1
+$XFS_IO_PROG -c "pwrite -S 0x62 $objsz $objsz" $file >> $seqres.full 2>&1
+$XFS_IO_PROG -c "pwrite -S 0x63 $(($objsz * 2)) $objsz" $file >> $seqres.full 2>&1
+
+cp $file $backup
+
+echo "  Copy single object to the end:"
+echo "    aaaa|bbbb|cccc => aaaa|bbbb|aaaa"
+$XFS_IO_PROG -c "copy_range -s 0 -d $(($objsz * 2)) -l $objsz $file" "$file"
+check_range $file 0 $objsz 61
+check_range $file $objsz $objsz 62
+check_range $file $(($objsz * 2)) $objsz 61
+
+echo "  Copy single object to the beginning:"
+echo "    aaaa|bbbb|aaaa => bbbb|bbbb|aaaa"
+$XFS_IO_PROG -c "copy_range -s $objsz -d 0 -l $objsz $file" "$file"
+check_range $file 0 $(($objsz * 2)) 62
+check_range $file $(($objsz * 2)) $objsz 61
+
+echo "  Copy single object to the middle:"
+echo "    bbbb|bbbb|aaaa => bbbb|aaaa|aaaa"
+$XFS_IO_PROG -c "copy_range -s $(($objsz * 2)) -d $objsz -l $objsz $file" "$file"
+check_range $file 0 $objsz 62
+check_range $file $objsz $(($objsz * 2)) 61
+
+cp $backup $file
+echo "  Cross object boundary (no full object copy)"
+echo "    aaaa|bbbb|cccc => aaaa|bbaa|aacc"
+$XFS_IO_PROG -c "copy_range -s 0 -d $(($objsz + $halfobj)) -l $objsz $file" "$file"
+check_range $file 0 $objsz 61
+check_range $file $objsz $halfobj 62
+check_range $file $(($objsz + $halfobj)) $objsz 61
+check_range $file $(($objsz * 2 + $halfobj)) $halfobj 63
+
+cp $backup $file
+echo "    aaaa|bbbb|cccc => aaaa|bbaa|bbcc"
+$XFS_IO_PROG -c "copy_range -s $halfobj -d $(($objsz + $halfobj)) -l $objsz $file" "$file"
+check_range $file 0 $objsz 61
+check_range $file $objsz $halfobj 62
+check_range $file $(($objsz + $halfobj)) $halfobj 61
+check_range $file $(($objsz * 2)) $halfobj 62
+check_range $file $(($objsz * 2 + $halfobj)) $halfobj 63
+
+cp $backup $file
+echo "    aaaa|bbbb|cccc => aaaa|bbbb|aabb"
+$XFS_IO_PROG -c "copy_range -s $halfobj -d $(($objsz * 2)) -l $objsz $file" "$file"
+check_range $file 0 $objsz 61
+check_range $file $objsz $objsz 62
+check_range $file $(($objsz * 2)) $halfobj 61
+check_range $file $(($objsz * 2 + $halfobj)) $halfobj 62
+
+#success, all done
+status=0
+exit
tests/ceph/003.out new file +11 -0
new file mode 100644
index 000000000000..76c83b265253
@@ -0,0 +1,11 @@
+QA output created by 003
+  Copy single object to the end:
+    aaaa|bbbb|cccc => aaaa|bbbb|aaaa
+  Copy single object to the beginning:
+    aaaa|bbbb|aaaa => bbbb|bbbb|aaaa
+  Copy single object to the middle:
+    bbbb|bbbb|aaaa => bbbb|aaaa|aaaa
+  Cross object boundary (no full object copy)
+    aaaa|bbbb|cccc => aaaa|bbaa|aacc
+    aaaa|bbbb|cccc => aaaa|bbaa|bbcc
+    aaaa|bbbb|cccc => aaaa|bbbb|aabb
tests/ceph/group +1 -0
index c28fe473c1a4..adbf61547766 100644
@@ -1,2 +1,3 @@
 001 auto quick copy
 002 auto quick copy
+003 auto quick copy