Initial CephFS tests (take 2)
cover Initial CephFS tests (take 2)
This is my second attempt to have an initial set of ceph-specific tests merged into fstests. In this patchset I'm pushing a different set of tests, focusing on the copy_file_range testing, although I *do* plan to get back to the quota tests soon.
This syscall has a few peculiarities in ceph as it is able to use remote object copies without the need to download/upload data from the OSDs. However, in order to take advantage of this remote copy, the copy ranges and sizes need to include at least one object. Thus, all the currently existing generic tests won't actually take advantage of this feature.
Let me know any comments/concerns about this patchset. Also note that currently, in order to enable copy_file_range in cephfs, the additional 'copyfrom' mount parameter is required. (Hopefully this additional param may be dropped in the future.)
Luis Henriques (3):
ceph: add copy_file_range (remote copy operation) testing
ceph: test combination of copy_file_range with truncate
ceph: test copy_file_range with infile = outfile
tests/ceph/001 | 233 +++++++++++++++++++++++++++++++++++++++++++++ tests/ceph/001.out | 129 +++++++++++++++++++++++++ tests/ceph/002 | 74 ++++++++++++++ tests/ceph/002.out | 8 ++ tests/ceph/003 | 118 +++++++++++++++++++++++ tests/ceph/003.out | 11 +++ tests/ceph/group | 3 + 7 files changed, 576 insertions(+) create mode 100644 tests/ceph/001 create mode 100644 tests/ceph/001.out create mode 100644 tests/ceph/002 create mode 100644 tests/ceph/002.out create mode 100644 tests/ceph/003 create mode 100644 tests/ceph/003.out create mode 100644 tests/ceph/group
PATCH 1/3 ceph: add copy_file_range (remote copy operation) testing
From: Luis Henriques <[email protected]>
Test remote copy operation (CEPH_OSD_OP_COPY_FROM) with several combinations of both object sizes and copy sizes. It also uses several combinations of copy ranges. For example, copying the 1st object in the src file into:
1) the beginning (1st object) of dst file,
2) the end (last object) of dst file and
3) the middle of the dst file.
--- tests/ceph/001 | 233 +++++++++++++++++++++++++++++++++++++++++++++ tests/ceph/001.out | 129 +++++++++++++++++++++++++ tests/ceph/group | 1 + 3 files changed, 363 insertions(+) create mode 100644 tests/ceph/001 create mode 100644 tests/ceph/001.out create mode 100644 tests/ceph/group
tests/ceph/001 new file +233 -0
@@ -0,0 +1,233 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) 2020 SUSE Linux Products GmbH. All Rights Reserved. +# +# FS QA Test No. ceph/001 +# +# Test remote copy operation (CEPH_OSD_OP_COPY_FROM) with several combinations +# of both object sizes and copy sizes. It also uses several combinations of +# copy ranges. For example, copying the 1st object in the src file into +# 1) the beginning (1st object) of dst file, 2) the end (last object) of dst +# file and 3) the middle of the dst file. +# +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -rf $tmp.* +} + +# get standard environment +. common/rc +. common/filter +. common/attr +. common/reflink + +# real QA test starts here +_supported_fs ceph + +_require_xfs_io_command "copy_range" +_require_attrs +_require_test + +workdir=$TEST_DIR/test-$seq +rm -rf $workdir +mkdir $workdir +rm -f $seqres.full + +check_range() +{ + local file=$1 + local off0=$2 + local off1=$3 + local val=$4 + _read_range $file $off0 $off1 | grep -v -q $val + [ $? -eq 0 ] && echo "file $file is not '$val' in [ $off0 $off1 ]" +} + +run_copy_range_tests() +{ + objsz=$1 + halfobj=$(($objsz / 2)) + file="$workdir/file-$objsz" + copy="$workdir/copy-$objsz" + dest="$workdir/dest-$objsz" + + # setting the file layout needs to be done before writing any data + touch $file $copy $dest + $SETFATTR_PROG -n ceph.file.layout \ + -v "stripe_unit=$objsz stripe_count=1 object_size=$objsz" \ + $file $copy $dest + + # file containing 3 objects with 'aaaa|bbbb|cccc' + $XFS_IO_PROG -c "pwrite -S 0x61 0 $objsz" $file >> $seqres.full 2>&1 + $XFS_IO_PROG -c "pwrite -S 0x62 $objsz $objsz" $file >> $seqres.full 2>&1 + $XFS_IO_PROG -c "pwrite -S 0x63 $(($objsz * 2)) $objsz" $file >> $seqres.full 2>&1 + + echo " Copy whole file (3 objects):" + echo " aaaa|bbbb|cccc => aaaa|bbbb|cccc" + $XFS_IO_PROG -c "copy_range -s 0 -d 0 -l $(($objsz * 3)) $file" "$copy" + cmp $file $copy + + echo " Copy single object to beginning:" + # dest file with 3 objects with 'dddd|dddd|dddd' + $XFS_IO_PROG -c "pwrite -S 0x64 0 $(($objsz * 3))" $dest >> $seqres.full 2>&1 + + echo " dddd|dddd|dddd => aaaa|dddd|dddd" + $XFS_IO_PROG -c "copy_range -s 0 -d 0 -l $objsz $file" "$dest" + check_range $dest 0 $objsz 61 + check_range $dest $objsz $(($objsz * 2)) 64 + + echo " aaaa|dddd|dddd => bbbb|dddd|dddd" + $XFS_IO_PROG -c "copy_range -s $objsz -d 0 -l $objsz $file" "$dest" + check_range $dest 0 $objsz 62 + check_range $dest $objsz $(($objsz * 2)) 64 + + echo " bbbb|dddd|dddd => cccc|dddd|dddd" + $XFS_IO_PROG -c "copy_range -s $(($objsz * 2)) -d 0 -l $objsz $file" "$dest" + check_range $dest 0 $objsz 63 + check_range $dest $objsz $(($objsz * 2)) 64 + + echo " Copy single object to middle:" + + echo " cccc|dddd|dddd => cccc|aaaa|dddd" + $XFS_IO_PROG -c "copy_range -s 0 -d $objsz -l $objsz $file" "$dest" + check_range $dest 0 $objsz 63 + check_range $dest $objsz $objsz 61 + check_range $dest $(($objsz * 2)) $objsz 64 + + echo " cccc|aaaa|dddd => cccc|bbbb|dddd" + $XFS_IO_PROG -c "copy_range -s $objsz -d $objsz -l $objsz $file" "$dest" + check_range $dest 0 $objsz 63 + check_range $dest $objsz $objsz 62 + check_range $dest $(($objsz * 2)) $objsz 64 + + echo " cccc|bbbb|dddd => cccc|cccc|dddd" + $XFS_IO_PROG -c "copy_range -s $((objsz * 2)) -d $objsz -l $objsz $file" "$dest" + check_range $dest 0 $objsz 63 + check_range $dest $objsz $objsz 63 + check_range $dest $(($objsz * 2)) $objsz 64 + + echo " Copy single object to end:" + + echo " cccc|cccc|dddd => cccc|cccc|aaaa" + $XFS_IO_PROG -c "copy_range -s 0 -d $(($objsz * 2)) -l $objsz $file" "$dest" + check_range $dest 0 $(($objsz * 2)) 63 + check_range $dest $(($objsz * 2)) $objsz 61 + + echo " cccc|cccc|aaaa => cccc|cccc|bbbb" + $XFS_IO_PROG -c "copy_range -s $objsz -d $(($objsz * 2)) -l $objsz $file" "$dest" + check_range $dest 0 $(($objsz * 2)) 63 + check_range $dest $(($objsz * 2)) $objsz 62 + + echo " cccc|cccc|aaaa => cccc|cccc|cccc" + $XFS_IO_PROG -c "copy_range -s $(($objsz * 2)) -d $(($objsz * 2)) -l $objsz $file" "$dest" + check_range $dest 0 $(($objsz * 3)) 63 + + echo " Copy 2 objects to beginning:" + + echo " cccc|cccc|cccc => aaaa|bbbb|cccc" + $XFS_IO_PROG -c "copy_range -s 0 -d 0 -l $(($objsz * 2)) $file" "$dest" + cmp $file $dest + + echo " aaaa|bbbb|cccc => bbbb|cccc|cccc" + $XFS_IO_PROG -c "copy_range -s $objsz -d 0 -l $(($objsz * 2)) $file" "$dest" + check_range $dest 0 $objsz 62 + check_range $dest $objsz $(($objsz * 2)) 63 + + echo " Copy 2 objects to end:" + + echo " bbbb|cccc|cccc => bbbb|aaaa|bbbb" + $XFS_IO_PROG -c "copy_range -s 0 -d $objsz -l $(($objsz * 2)) $file" "$dest" + check_range $dest 0 $objsz 62 + check_range $dest $objsz $objsz 61 + check_range $dest $(($objsz * 2)) $objsz 62 + + echo " bbbb|aaaa|bbbb => bbbb|bbbb|cccc" + $XFS_IO_PROG -c "copy_range -s $objsz -d $objsz -l $(($objsz * 2)) $file" "$dest" + check_range $dest 0 $(($objsz * 2)) 62 + check_range $dest $(($objsz * 2)) $objsz 63 + + echo " Append 1 object:" + + echo " bbbb|bbbb|cccc => bbbb|bbbb|cccc|aaaa" + $XFS_IO_PROG -c "copy_range -s 0 -d $(($objsz * 3)) -l $objsz $file" "$dest" + check_range $dest 0 $(($objsz * 2)) 62 + check_range $dest $(($objsz * 2)) $objsz 63 + check_range $dest $(($objsz * 3)) $objsz 61 + + echo " Cross object boundary (no full object copy)" + echo " dddd|dddd|dddd|dddd => ddaa|aadd|dddd|dddd" + $XFS_IO_PROG -c "pwrite -S 0x64 0 $(($objsz * 4))" $dest >> $seqres.full 2>&1 + $XFS_IO_PROG -c "copy_range -s 0 -d $halfobj -l $objsz $file" "$dest" + check_range $dest 0 $halfobj 64 + check_range $dest $halfobj $objsz 61 + check_range $dest $(($objsz + $halfobj)) $(($objsz * 2 + $halfobj)) 64 + + echo " dddd|dddd|dddd|dddd => ddaa|bbdd|dddd|dddd" + $XFS_IO_PROG -c "pwrite -S 0x64 0 $(($objsz * 4))" $dest >> $seqres.full 2>&1 + $XFS_IO_PROG -c "copy_range -s $halfobj -d $halfobj -l $objsz $file" "$dest" + check_range $dest 0 $halfobj 64 + check_range $dest $halfobj $halfobj 61 + check_range $dest $objsz $halfobj 62 + check_range $dest $(($objsz + $halfobj)) $(($objsz * 2 + $halfobj)) 64 + + echo " Cross object boundaries (with full object copy)" + echo " dddd|dddd|dddd|dddd => ddaa|bbbb|dddd|dddd" + $XFS_IO_PROG -c "pwrite -S 0x64 0 $(($objsz * 4))" $dest >> $seqres.full 2>&1 + $XFS_IO_PROG -c "copy_range -s $halfobj -d $halfobj -l $(($objsz + $halfobj)) $file" "$dest" + check_range $dest 0 $halfobj 64 + check_range $dest $halfobj $halfobj 61 + check_range $dest $objsz $objsz 62 + check_range $dest $(($objsz * 2)) $(($objsz * 2)) 64 + + echo " dddd|dddd|dddd|dddd => ddaa|bbbb|ccdd|dddd" + $XFS_IO_PROG -c "pwrite -S 0x64 0 $(($objsz * 4))" $dest >> $seqres.full 2>&1 + $XFS_IO_PROG -c "copy_range -s $halfobj -d $halfobj -l $(($objsz * 2)) $file" "$dest" + check_range $dest 0 $halfobj 64 + check_range $dest $halfobj $halfobj 61 + check_range $dest $objsz $objsz 62 + check_range $dest $(($objsz * 2)) $halfobj 63 + check_range $dest $(($objsz * 2 + $halfobj)) $(($objsz + $halfobj)) 64 + + echo " dddd|dddd|dddd|dddd => dddd|aaaa|bbdd|dddd" + $XFS_IO_PROG -c "pwrite -S 0x64 0 $(($objsz * 4))" $dest >> $seqres.full 2>&1 + $XFS_IO_PROG -c "copy_range -s 0 -d $objsz -l $(($objsz + $halfobj)) $file" "$dest" + check_range $dest 0 $objsz 64 + check_range $dest $objsz $objsz 61 + check_range $dest $(($objsz * 2)) $halfobj 62 + check_range $dest $(($objsz * 2 + $halfobj)) $(($objsz + $halfobj)) 64 + + echo " Cross object boundaries (with 2 full object copies)" + echo " dddd|dddd|dddd|dddd => ddaa|aabb|bbcc|ccdd" + $XFS_IO_PROG -c "pwrite -S 0x64 0 $(($objsz * 4))" $dest >> $seqres.full 2>&1 + $XFS_IO_PROG -c "copy_range -s 0 -d $halfobj -l $(($objsz * 3)) $file" "$dest" + check_range $dest 0 $halfobj 64 + check_range $dest $halfobj $objsz 61 + check_range $dest $(($objsz + $halfobj)) $objsz 62 + check_range $dest $(($objsz * 2 + $halfobj)) $objsz 63 + check_range $dest $(($objsz * 3 + $halfobj)) $halfobj 64 + +} + +echo "Object size: 65536" # CEPH_MIN_STRIPE_UNIT +run_copy_range_tests 65536 +echo "Object size: 1M" +run_copy_range_tests 1048576 +echo "Object size: 4M" +run_copy_range_tests 4194304 +# the max object size is 1TB, but by default OSDs only accept a max of 128M objects +echo "Object size: 128M" +run_copy_range_tests 134217728 + +#success, all done +status=0 +exit
tests/ceph/001.out new file +129 -0
129 lines not on this page · raw message
tests/ceph/group new file +1 -0
@@ -0,0 +1 @@ +001 auto quick copy
Re: [PATCH 1/3] ceph: add copy_file_range (remote copy operation) testing
Quoted 85 lines · Luis Henriques
From: Luis Henriques <[email protected]>
Test remote copy operation (
CEPH_OSD_OP_COPY_FROM) with several combinations of both object sizes and copy sizes. It also uses several combinations of copy ranges. For example, copying the 1st object in the src file into:1) the beginning (1st object) of dst file,
2) the end (last object) of dst file and
3) the middle of the dst file.Signed-off-by Luis Henriques <[email protected]>--- tests/ceph/001 | 233 +++++++++++++++++++++++++++++++++++++++++++++ tests/ceph/001.out | 129 +++++++++++++++++++++++++ tests/ceph/group | 1 + 3 files changed, 363 insertions(+) create mode 100644 tests/ceph/001 create mode 100644 tests/ceph/001.out create mode 100644 tests/ceph/grouptests/ceph/001 new file +58
-0@@ -0,0 +1,233 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) 2020 SUSE Linux Products GmbH. All Rights Reserved. +# +# FS QA Test No. ceph/001 +# +# Test remote copy operation (CEPH_OSD_OP_COPY_FROM) with several combinations +# of both object sizes and copy sizes. It also uses several combinations of +# copy ranges. For example, copying the 1st object in the src file into +# 1) the beginning (1st object) of dst file, 2) the end (last object) of dst +# file and 3) the middle of the dst file. +# +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -rf $tmp.* +} + +# get standard environment +. common/rc +. common/filter +. common/attr +. common/reflink + +# real QA test starts here +_supported_fs ceph + +_require_xfs_io_command "copy_range" +_require_attrs +_require_test + +workdir=$TEST_DIR/test-$seq +rm -rf $workdir +mkdir $workdir +rm -f $seqres.full + +check_range() +{ + local file=$1 + local off0=$2 + local off1=$3 + local val=$4 + _read_range $file $off0 $off1 | grep -v -q $val + [ $? -eq 0 ] && echo "file $file is not '$val' in [ $off0 $off1 ]" +} + +run_copy_range_tests() +{ + objsz=$1
tests/ceph/001 line 68
Quoted 10 lines
tests/ceph/001 new file +10
-0+ halfobj=$(($objsz / 2)) + file="$workdir/file-$objsz" + copy="$workdir/copy-$objsz" + dest="$workdir/dest-$objsz" + + # setting the file layout needs to be done before writing any data + touch $file $copy $dest + $SETFATTR_PROG -n ceph.file.layout \ + -v "stripe_unit=$objsz stripe_count=1 object_size=$objsz" \ + $file $copy $dest
I noticed this pattern repeated a few times in this patchset, I think it'd be good to factor it into a new helper function? And put it in common/ceph file.
Quoted 153 lines
tests/ceph/001 new file +153
-0+ + # file containing 3 objects with 'aaaa|bbbb|cccc' + $XFS_IO_PROG -c "pwrite -S 0x61 0 $objsz" $file >> $seqres.full 2>&1 + $XFS_IO_PROG -c "pwrite -S 0x62 $objsz $objsz" $file >> $seqres.full 2>&1 + $XFS_IO_PROG -c "pwrite -S 0x63 $(($objsz * 2)) $objsz" $file >> $seqres.full 2>&1 + + echo " Copy whole file (3 objects):" + echo " aaaa|bbbb|cccc => aaaa|bbbb|cccc" + $XFS_IO_PROG -c "copy_range -s 0 -d 0 -l $(($objsz * 3)) $file" "$copy" + cmp $file $copy + + echo " Copy single object to beginning:" + # dest file with 3 objects with 'dddd|dddd|dddd' + $XFS_IO_PROG -c "pwrite -S 0x64 0 $(($objsz * 3))" $dest >> $seqres.full 2>&1 + + echo " dddd|dddd|dddd => aaaa|dddd|dddd" + $XFS_IO_PROG -c "copy_range -s 0 -d 0 -l $objsz $file" "$dest" + check_range $dest 0 $objsz 61 + check_range $dest $objsz $(($objsz * 2)) 64 + + echo " aaaa|dddd|dddd => bbbb|dddd|dddd" + $XFS_IO_PROG -c "copy_range -s $objsz -d 0 -l $objsz $file" "$dest" + check_range $dest 0 $objsz 62 + check_range $dest $objsz $(($objsz * 2)) 64 + + echo " bbbb|dddd|dddd => cccc|dddd|dddd" + $XFS_IO_PROG -c "copy_range -s $(($objsz * 2)) -d 0 -l $objsz $file" "$dest" + check_range $dest 0 $objsz 63 + check_range $dest $objsz $(($objsz * 2)) 64 + + echo " Copy single object to middle:" + + echo " cccc|dddd|dddd => cccc|aaaa|dddd" + $XFS_IO_PROG -c "copy_range -s 0 -d $objsz -l $objsz $file" "$dest" + check_range $dest 0 $objsz 63 + check_range $dest $objsz $objsz 61 + check_range $dest $(($objsz * 2)) $objsz 64 + + echo " cccc|aaaa|dddd => cccc|bbbb|dddd" + $XFS_IO_PROG -c "copy_range -s $objsz -d $objsz -l $objsz $file" "$dest" + check_range $dest 0 $objsz 63 + check_range $dest $objsz $objsz 62 + check_range $dest $(($objsz * 2)) $objsz 64 + + echo " cccc|bbbb|dddd => cccc|cccc|dddd" + $XFS_IO_PROG -c "copy_range -s $((objsz * 2)) -d $objsz -l $objsz $file" "$dest" + check_range $dest 0 $objsz 63 + check_range $dest $objsz $objsz 63 + check_range $dest $(($objsz * 2)) $objsz 64 + + echo " Copy single object to end:" + + echo " cccc|cccc|dddd => cccc|cccc|aaaa" + $XFS_IO_PROG -c "copy_range -s 0 -d $(($objsz * 2)) -l $objsz $file" "$dest" + check_range $dest 0 $(($objsz * 2)) 63 + check_range $dest $(($objsz * 2)) $objsz 61 + + echo " cccc|cccc|aaaa => cccc|cccc|bbbb" + $XFS_IO_PROG -c "copy_range -s $objsz -d $(($objsz * 2)) -l $objsz $file" "$dest" + check_range $dest 0 $(($objsz * 2)) 63 + check_range $dest $(($objsz * 2)) $objsz 62 + + echo " cccc|cccc|aaaa => cccc|cccc|cccc" + $XFS_IO_PROG -c "copy_range -s $(($objsz * 2)) -d $(($objsz * 2)) -l $objsz $file" "$dest" + check_range $dest 0 $(($objsz * 3)) 63 + + echo " Copy 2 objects to beginning:" + + echo " cccc|cccc|cccc => aaaa|bbbb|cccc" + $XFS_IO_PROG -c "copy_range -s 0 -d 0 -l $(($objsz * 2)) $file" "$dest" + cmp $file $dest + + echo " aaaa|bbbb|cccc => bbbb|cccc|cccc" + $XFS_IO_PROG -c "copy_range -s $objsz -d 0 -l $(($objsz * 2)) $file" "$dest" + check_range $dest 0 $objsz 62 + check_range $dest $objsz $(($objsz * 2)) 63 + + echo " Copy 2 objects to end:" + + echo " bbbb|cccc|cccc => bbbb|aaaa|bbbb" + $XFS_IO_PROG -c "copy_range -s 0 -d $objsz -l $(($objsz * 2)) $file" "$dest" + check_range $dest 0 $objsz 62 + check_range $dest $objsz $objsz 61 + check_range $dest $(($objsz * 2)) $objsz 62 + + echo " bbbb|aaaa|bbbb => bbbb|bbbb|cccc" + $XFS_IO_PROG -c "copy_range -s $objsz -d $objsz -l $(($objsz * 2)) $file" "$dest" + check_range $dest 0 $(($objsz * 2)) 62 + check_range $dest $(($objsz * 2)) $objsz 63 + + echo " Append 1 object:" + + echo " bbbb|bbbb|cccc => bbbb|bbbb|cccc|aaaa" + $XFS_IO_PROG -c "copy_range -s 0 -d $(($objsz * 3)) -l $objsz $file" "$dest" + check_range $dest 0 $(($objsz * 2)) 62 + check_range $dest $(($objsz * 2)) $objsz 63 + check_range $dest $(($objsz * 3)) $objsz 61 + + echo " Cross object boundary (no full object copy)" + echo " dddd|dddd|dddd|dddd => ddaa|aadd|dddd|dddd" + $XFS_IO_PROG -c "pwrite -S 0x64 0 $(($objsz * 4))" $dest >> $seqres.full 2>&1 + $XFS_IO_PROG -c "copy_range -s 0 -d $halfobj -l $objsz $file" "$dest" + check_range $dest 0 $halfobj 64 + check_range $dest $halfobj $objsz 61 + check_range $dest $(($objsz + $halfobj)) $(($objsz * 2 + $halfobj)) 64 + + echo " dddd|dddd|dddd|dddd => ddaa|bbdd|dddd|dddd" + $XFS_IO_PROG -c "pwrite -S 0x64 0 $(($objsz * 4))" $dest >> $seqres.full 2>&1 + $XFS_IO_PROG -c "copy_range -s $halfobj -d $halfobj -l $objsz $file" "$dest" + check_range $dest 0 $halfobj 64 + check_range $dest $halfobj $halfobj 61 + check_range $dest $objsz $halfobj 62 + check_range $dest $(($objsz + $halfobj)) $(($objsz * 2 + $halfobj)) 64 + + echo " Cross object boundaries (with full object copy)" + echo " dddd|dddd|dddd|dddd => ddaa|bbbb|dddd|dddd" + $XFS_IO_PROG -c "pwrite -S 0x64 0 $(($objsz * 4))" $dest >> $seqres.full 2>&1 + $XFS_IO_PROG -c "copy_range -s $halfobj -d $halfobj -l $(($objsz + $halfobj)) $file" "$dest" + check_range $dest 0 $halfobj 64 + check_range $dest $halfobj $halfobj 61 + check_range $dest $objsz $objsz 62 + check_range $dest $(($objsz * 2)) $(($objsz * 2)) 64 + + echo " dddd|dddd|dddd|dddd => ddaa|bbbb|ccdd|dddd" + $XFS_IO_PROG -c "pwrite -S 0x64 0 $(($objsz * 4))" $dest >> $seqres.full 2>&1 + $XFS_IO_PROG -c "copy_range -s $halfobj -d $halfobj -l $(($objsz * 2)) $file" "$dest" + check_range $dest 0 $halfobj 64 + check_range $dest $halfobj $halfobj 61 + check_range $dest $objsz $objsz 62 + check_range $dest $(($objsz * 2)) $halfobj 63 + check_range $dest $(($objsz * 2 + $halfobj)) $(($objsz + $halfobj)) 64 + + echo " dddd|dddd|dddd|dddd => dddd|aaaa|bbdd|dddd" + $XFS_IO_PROG -c "pwrite -S 0x64 0 $(($objsz * 4))" $dest >> $seqres.full 2>&1 + $XFS_IO_PROG -c "copy_range -s 0 -d $objsz -l $(($objsz + $halfobj)) $file" "$dest" + check_range $dest 0 $objsz 64 + check_range $dest $objsz $objsz 61 + check_range $dest $(($objsz * 2)) $halfobj 62 + check_range $dest $(($objsz * 2 + $halfobj)) $(($objsz + $halfobj)) 64 + + echo " Cross object boundaries (with 2 full object copies)" + echo " dddd|dddd|dddd|dddd => ddaa|aabb|bbcc|ccdd" + $XFS_IO_PROG -c "pwrite -S 0x64 0 $(($objsz * 4))" $dest >> $seqres.full 2>&1 + $XFS_IO_PROG -c "copy_range -s 0 -d $halfobj -l $(($objsz * 3)) $file" "$dest" + check_range $dest 0 $halfobj 64 + check_range $dest $halfobj $objsz 61 + check_range $dest $(($objsz + $halfobj)) $objsz 62 + check_range $dest $(($objsz * 2 + $halfobj)) $objsz 63 + check_range $dest $(($objsz * 3 + $halfobj)) $halfobj 64 + +} + +echo "Object size: 65536" # CEPH_MIN_STRIPE_UNIT
tests/ceph/001 line 231
Quoted 10 lines
tests/ceph/001 new file +10
-0+run_copy_range_tests 65536 +echo "Object size: 1M" +run_copy_range_tests 1048576 +echo "Object size: 4M" +run_copy_range_tests 4194304 +# the max object size is 1TB, but by default OSDs only accept a max of 128M objects +echo "Object size: 128M" +run_copy_range_tests 134217728 + +#success, all done
Minor nit, add a space after '#' for comments.
Thanks,
Eryu
Quoted 144 lines
tests/ceph/001 new file +2
-0+status=0 +exittests/ceph/001.out new file +129
-0@@ -0,0 +1,129 @@ +QA output created by 001 +Object size: 65536 + Copy whole file (3 objects): + aaaa|bbbb|cccc => aaaa|bbbb|cccc + Copy single object to beginning: + dddd|dddd|dddd => aaaa|dddd|dddd + aaaa|dddd|dddd => bbbb|dddd|dddd + bbbb|dddd|dddd => cccc|dddd|dddd + Copy single object to middle: + cccc|dddd|dddd => cccc|aaaa|dddd + cccc|aaaa|dddd => cccc|bbbb|dddd + cccc|bbbb|dddd => cccc|cccc|dddd + Copy single object to end: + cccc|cccc|dddd => cccc|cccc|aaaa + cccc|cccc|aaaa => cccc|cccc|bbbb + cccc|cccc|aaaa => cccc|cccc|cccc + Copy 2 objects to beginning: + cccc|cccc|cccc => aaaa|bbbb|cccc + aaaa|bbbb|cccc => bbbb|cccc|cccc + Copy 2 objects to end: + bbbb|cccc|cccc => bbbb|aaaa|bbbb + bbbb|aaaa|bbbb => bbbb|bbbb|cccc + Append 1 object: + bbbb|bbbb|cccc => bbbb|bbbb|cccc|aaaa + Cross object boundary (no full object copy) + dddd|dddd|dddd|dddd => ddaa|aadd|dddd|dddd + dddd|dddd|dddd|dddd => ddaa|bbdd|dddd|dddd + Cross object boundaries (with full object copy) + dddd|dddd|dddd|dddd => ddaa|bbbb|dddd|dddd + dddd|dddd|dddd|dddd => ddaa|bbbb|ccdd|dddd + dddd|dddd|dddd|dddd => dddd|aaaa|bbdd|dddd + Cross object boundaries (with 2 full object copies) + dddd|dddd|dddd|dddd => ddaa|aabb|bbcc|ccdd +Object size: 1M + Copy whole file (3 objects): + aaaa|bbbb|cccc => aaaa|bbbb|cccc + Copy single object to beginning: + dddd|dddd|dddd => aaaa|dddd|dddd + aaaa|dddd|dddd => bbbb|dddd|dddd + bbbb|dddd|dddd => cccc|dddd|dddd + Copy single object to middle: + cccc|dddd|dddd => cccc|aaaa|dddd + cccc|aaaa|dddd => cccc|bbbb|dddd + cccc|bbbb|dddd => cccc|cccc|dddd + Copy single object to end: + cccc|cccc|dddd => cccc|cccc|aaaa + cccc|cccc|aaaa => cccc|cccc|bbbb + cccc|cccc|aaaa => cccc|cccc|cccc + Copy 2 objects to beginning: + cccc|cccc|cccc => aaaa|bbbb|cccc + aaaa|bbbb|cccc => bbbb|cccc|cccc + Copy 2 objects to end: + bbbb|cccc|cccc => bbbb|aaaa|bbbb + bbbb|aaaa|bbbb => bbbb|bbbb|cccc + Append 1 object: + bbbb|bbbb|cccc => bbbb|bbbb|cccc|aaaa + Cross object boundary (no full object copy) + dddd|dddd|dddd|dddd => ddaa|aadd|dddd|dddd + dddd|dddd|dddd|dddd => ddaa|bbdd|dddd|dddd + Cross object boundaries (with full object copy) + dddd|dddd|dddd|dddd => ddaa|bbbb|dddd|dddd + dddd|dddd|dddd|dddd => ddaa|bbbb|ccdd|dddd + dddd|dddd|dddd|dddd => dddd|aaaa|bbdd|dddd + Cross object boundaries (with 2 full object copies) + dddd|dddd|dddd|dddd => ddaa|aabb|bbcc|ccdd +Object size: 4M + Copy whole file (3 objects): + aaaa|bbbb|cccc => aaaa|bbbb|cccc + Copy single object to beginning: + dddd|dddd|dddd => aaaa|dddd|dddd + aaaa|dddd|dddd => bbbb|dddd|dddd + bbbb|dddd|dddd => cccc|dddd|dddd + Copy single object to middle: + cccc|dddd|dddd => cccc|aaaa|dddd + cccc|aaaa|dddd => cccc|bbbb|dddd + cccc|bbbb|dddd => cccc|cccc|dddd + Copy single object to end: + cccc|cccc|dddd => cccc|cccc|aaaa + cccc|cccc|aaaa => cccc|cccc|bbbb + cccc|cccc|aaaa => cccc|cccc|cccc + Copy 2 objects to beginning: + cccc|cccc|cccc => aaaa|bbbb|cccc + aaaa|bbbb|cccc => bbbb|cccc|cccc + Copy 2 objects to end: + bbbb|cccc|cccc => bbbb|aaaa|bbbb + bbbb|aaaa|bbbb => bbbb|bbbb|cccc + Append 1 object: + bbbb|bbbb|cccc => bbbb|bbbb|cccc|aaaa + Cross object boundary (no full object copy) + dddd|dddd|dddd|dddd => ddaa|aadd|dddd|dddd + dddd|dddd|dddd|dddd => ddaa|bbdd|dddd|dddd + Cross object boundaries (with full object copy) + dddd|dddd|dddd|dddd => ddaa|bbbb|dddd|dddd + dddd|dddd|dddd|dddd => ddaa|bbbb|ccdd|dddd + dddd|dddd|dddd|dddd => dddd|aaaa|bbdd|dddd + Cross object boundaries (with 2 full object copies) + dddd|dddd|dddd|dddd => ddaa|aabb|bbcc|ccdd +Object size: 128M + Copy whole file (3 objects): + aaaa|bbbb|cccc => aaaa|bbbb|cccc + Copy single object to beginning: + dddd|dddd|dddd => aaaa|dddd|dddd + aaaa|dddd|dddd => bbbb|dddd|dddd + bbbb|dddd|dddd => cccc|dddd|dddd + Copy single object to middle: + cccc|dddd|dddd => cccc|aaaa|dddd + cccc|aaaa|dddd => cccc|bbbb|dddd + cccc|bbbb|dddd => cccc|cccc|dddd + Copy single object to end: + cccc|cccc|dddd => cccc|cccc|aaaa + cccc|cccc|aaaa => cccc|cccc|bbbb + cccc|cccc|aaaa => cccc|cccc|cccc + Copy 2 objects to beginning: + cccc|cccc|cccc => aaaa|bbbb|cccc + aaaa|bbbb|cccc => bbbb|cccc|cccc + Copy 2 objects to end: + bbbb|cccc|cccc => bbbb|aaaa|bbbb + bbbb|aaaa|bbbb => bbbb|bbbb|cccc + Append 1 object: + bbbb|bbbb|cccc => bbbb|bbbb|cccc|aaaa + Cross object boundary (no full object copy) + dddd|dddd|dddd|dddd => ddaa|aadd|dddd|dddd + dddd|dddd|dddd|dddd => ddaa|bbdd|dddd|dddd + Cross object boundaries (with full object copy) + dddd|dddd|dddd|dddd => ddaa|bbbb|dddd|dddd + dddd|dddd|dddd|dddd => ddaa|bbbb|ccdd|dddd + dddd|dddd|dddd|dddd => dddd|aaaa|bbdd|dddd + Cross object boundaries (with 2 full object copies) + dddd|dddd|dddd|dddd => ddaa|aabb|bbcc|ccddtests/ceph/group new file +1
-0@@ -0,0 +1 @@ +001 auto quick copy
PATCH 2/3 ceph: test combination of copy_file_range with truncate
In reply to Luis HenriquesFrom: Luis Henriques <[email protected]>
This test was motivated by an OSD bug found while testing copy_file_range. This bug was an issue with the way the OSDs handled the truncate_seq value, which was being copied from the original object into the destination object. This test ensures the kernel client correctly handles fixed/non-fixed OSDs.
--- tests/ceph/002 | 74 ++++++++++++++++++++++++++++++++++++++++++++++ tests/ceph/002.out | 8 +++++ tests/ceph/group | 1 + 3 files changed, 83 insertions(+) create mode 100644 tests/ceph/002 create mode 100644 tests/ceph/002.out
tests/ceph/002 new file +74 -0
@@ -0,0 +1,74 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) 2020 SUSE Linux Products GmbH. All Rights Reserved. +# +# FS QA Test No. ceph/002 +# +# Test bug found while testing copy_file_range. +# +# This bug was an issue with how the OSDs handled the truncate_seq, copying it +# from the original object into the destination object. This test ensures the +# kernel client correctly handles fixed/non-fixed OSDs. +# +# https://tracker.ceph.com/issues/37378 +# +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -rf $tmp.* +} + +# get standard environment +. common/rc +. common/filter +. common/attr + +# real QA test starts here +_supported_fs ceph + +_require_xfs_io_command "copy_range" +_require_attrs +_require_test + +workdir=$TEST_DIR/test-$seq +rm -rf $workdir +mkdir $workdir +rm -f $seqres.full + +# Use 4M object size +objsz=4194304 +file="$workdir/file-$objsz" +dest="$workdir/dest-$objsz" +touch $file $dest + +# object_size has to be a multiple of stripe_unit +$SETFATTR_PROG -n ceph.file.layout \ + -v "stripe_unit=$objsz stripe_count=1 object_size=$objsz" \ + $file $dest + +# Create a 3 objects size files +$XFS_IO_PROG -c "pwrite -S 0x61 0 $objsz" $file >> $seqres.full 2>&1 +$XFS_IO_PROG -c "pwrite -S 0x62 $objsz $objsz" $file >> $seqres.full 2>&1 +$XFS_IO_PROG -c "pwrite -S 0x63 $(($objsz * 2)) $objsz" $file >> $seqres.full 2>&1 + +$XFS_IO_PROG -c "pwrite -S 0x64 0 $(($objsz * 3))" $dest >> $seqres.full 2>&1 +# Truncate the destination file (messing up with the truncate_seq) +$XFS_IO_PROG -c "truncate 0" $dest >> $seqres.full 2>&1 + +# copy the whole file over +$XFS_IO_PROG -c "copy_range -s 0 -d 0 -l $(($objsz * 3)) $file" "$dest" + +hexdump $dest + +#success, all done +status=0 +exit
tests/ceph/002.out new file +8 -0
@@ -0,0 +1,8 @@ +QA output created by 002 +0000000 6161 6161 6161 6161 6161 6161 6161 6161 +* +0400000 6262 6262 6262 6262 6262 6262 6262 6262 +* +0800000 6363 6363 6363 6363 6363 6363 6363 6363 +* +0c00000
tests/ceph/group +1 -0
@@ -1 +1,2 @@ 001 auto quick copy +002 auto quick copy
Re: [PATCH 2/3] ceph: test combination of copy_file_range with truncate
Quoted 27 lines · Luis Henriques
From: Luis Henriques <[email protected]>
This test was motivated by an OSD bug found while testing
copy_file_range. This bug was an issue with the way the OSDs handled thetruncate_seqvalue, which was being copied from the original object into the destination object. This test ensures the kernel client correctly handles fixed/non-fixed OSDs.Signed-off-by Luis Henriques <[email protected]>--- tests/ceph/002 | 74 ++++++++++++++++++++++++++++++++++++++++++++++ tests/ceph/002.out | 8 +++++ tests/ceph/group | 1 + 3 files changed, 83 insertions(+) create mode 100644 tests/ceph/002 create mode 100644 tests/ceph/002.outtests/ceph/002 new file +3
-0@@ -0,0 +1,74 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) 2020 SUSE Linux Products GmbH. All Rights Reserved.
tests/ceph/002 line 13
Quoted 10 lines
tests/ceph/002 new file +10
-0+# +# FS QA Test No. ceph/002 +# +# Test bug found while testing copy_file_range. +# +# This bug was an issue with how the OSDs handled the truncate_seq, copying it +# from the original object into the destination object. This test ensures the +# kernel client correctly handles fixed/non-fixed OSDs. +# +# https://tracker.ceph.com/issues/37378
I noticed that the bug in above issue link has been fixed, it'd be good to reference the fix commit as well, so people will know where the fix is when test fails.
Thanks,
Eryu
Quoted 82 lines
tests/ceph/002 new file +61
-0+# +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -rf $tmp.* +} + +# get standard environment +. common/rc +. common/filter +. common/attr + +# real QA test starts here +_supported_fs ceph + +_require_xfs_io_command "copy_range" +_require_attrs +_require_test + +workdir=$TEST_DIR/test-$seq +rm -rf $workdir +mkdir $workdir +rm -f $seqres.full + +# Use 4M object size +objsz=4194304 +file="$workdir/file-$objsz" +dest="$workdir/dest-$objsz" +touch $file $dest + +# object_size has to be a multiple of stripe_unit +$SETFATTR_PROG -n ceph.file.layout \ + -v "stripe_unit=$objsz stripe_count=1 object_size=$objsz" \ + $file $dest + +# Create a 3 objects size files +$XFS_IO_PROG -c "pwrite -S 0x61 0 $objsz" $file >> $seqres.full 2>&1 +$XFS_IO_PROG -c "pwrite -S 0x62 $objsz $objsz" $file >> $seqres.full 2>&1 +$XFS_IO_PROG -c "pwrite -S 0x63 $(($objsz * 2)) $objsz" $file >> $seqres.full 2>&1 + +$XFS_IO_PROG -c "pwrite -S 0x64 0 $(($objsz * 3))" $dest >> $seqres.full 2>&1 +# Truncate the destination file (messing up with the truncate_seq) +$XFS_IO_PROG -c "truncate 0" $dest >> $seqres.full 2>&1 + +# copy the whole file over +$XFS_IO_PROG -c "copy_range -s 0 -d 0 -l $(($objsz * 3)) $file" "$dest" + +hexdump $dest + +#success, all done +status=0 +exittests/ceph/002.out new file +8
-0@@ -0,0 +1,8 @@ +QA output created by 002 +0000000 6161 6161 6161 6161 6161 6161 6161 6161 +* +0400000 6262 6262 6262 6262 6262 6262 6262 6262 +* +0800000 6363 6363 6363 6363 6363 6363 6363 6363 +* +0c00000tests/ceph/group +1
-0@@ -1 +1,2 @@ 001 auto quick copy +002 auto quick copy
PATCH 3/3 ceph: test copy_file_range with infile = outfile
In reply to Luis HenriquesFrom: Luis Henriques <[email protected]>
This test runs a set of simple checks where the infile file is the same as the outfile.
--- tests/ceph/003 | 118 +++++++++++++++++++++++++++++++++++++++++++++ tests/ceph/003.out | 11 +++++ tests/ceph/group | 1 + 3 files changed, 130 insertions(+) create mode 100644 tests/ceph/003 create mode 100644 tests/ceph/003.out
tests/ceph/003 new file +118 -0
@@ -0,0 +1,118 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) 2020 SUSE Linux Products GmbH. All Rights Reserved. +# +# FS QA Test No. ceph/005 +# +# Test copy_file_range with infile = outfile +# +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -rf $tmp.* +} + +# get standard environment +. common/rc +. common/filter +. common/attr +. common/reflink + +# real QA test starts here +_supported_fs ceph + +_require_xfs_io_command "copy_range" +_require_attrs +_require_test + +workdir=$TEST_DIR/test-$seq +rm -rf $workdir +mkdir $workdir +rm -f $seqres.full + +check_range() +{ + local file=$1 + local off0=$2 + local off1=$3 + local val=$4 + _read_range $file $off0 $off1 | grep -v -q $val + [ $? -eq 0 ] && echo "file $file is not '$val' in [ $off0 $off1 ]" +} + +objsz=4194304 +halfobj=$(($objsz / 2)) +file="$workdir/file-$objsz" +copy="$workdir/copy-$objsz" +dest="$workdir/dest-$objsz" +backup="$file.backup" +touch $file $backup + +# object_size has to be a multiple of stripe_unit +$SETFATTR_PROG -n ceph.file.layout \ + -v "stripe_unit=$objsz stripe_count=1 object_size=$objsz" \ + $file $backup + +$XFS_IO_PROG -c "pwrite -S 0x61 0 $objsz" $file >> $seqres.full 2>&1 +$XFS_IO_PROG -c "pwrite -S 0x62 $objsz $objsz" $file >> $seqres.full 2>&1 +$XFS_IO_PROG -c "pwrite -S 0x63 $(($objsz * 2)) $objsz" $file >> $seqres.full 2>&1 + +cp $file $backup + +echo " Copy single object to the end:" +echo " aaaa|bbbb|cccc => aaaa|bbbb|aaaa" +$XFS_IO_PROG -c "copy_range -s 0 -d $(($objsz * 2)) -l $objsz $file" "$file" +check_range $file 0 $objsz 61 +check_range $file $objsz $objsz 62 +check_range $file $(($objsz * 2)) $objsz 61 + +echo " Copy single object to the beginning:" +echo " aaaa|bbbb|aaaa => bbbb|bbbb|aaaa" +$XFS_IO_PROG -c "copy_range -s $objsz -d 0 -l $objsz $file" "$file" +check_range $file 0 $(($objsz * 2)) 62 +check_range $file $(($objsz * 2)) $objsz 61 + +echo " Copy single object to the middle:" +echo " bbbb|bbbb|aaaa => bbbb|aaaa|aaaa" +$XFS_IO_PROG -c "copy_range -s $(($objsz * 2)) -d $objsz -l $objsz $file" "$file" +check_range $file 0 $objsz 62 +check_range $file $objsz $(($objsz * 2)) 61 + +cp $backup $file +echo " Cross object boundary (no full object copy)" +echo " aaaa|bbbb|cccc => aaaa|bbaa|aacc" +$XFS_IO_PROG -c "copy_range -s 0 -d $(($objsz + $halfobj)) -l $objsz $file" "$file" +check_range $file 0 $objsz 61 +check_range $file $objsz $halfobj 62 +check_range $file $(($objsz + $halfobj)) $objsz 61 +check_range $file $(($objsz * 2 + $halfobj)) $halfobj 63 + +cp $backup $file +echo " aaaa|bbbb|cccc => aaaa|bbaa|bbcc" +$XFS_IO_PROG -c "copy_range -s $halfobj -d $(($objsz + $halfobj)) -l $objsz $file" "$file" +check_range $file 0 $objsz 61 +check_range $file $objsz $halfobj 62 +check_range $file $(($objsz + $halfobj)) $halfobj 61 +check_range $file $(($objsz * 2)) $halfobj 62 +check_range $file $(($objsz * 2 + $halfobj)) $halfobj 63 + +cp $backup $file +echo " aaaa|bbbb|cccc => aaaa|bbbb|aabb" +$XFS_IO_PROG -c "copy_range -s $halfobj -d $(($objsz * 2)) -l $objsz $file" "$file" +check_range $file 0 $objsz 61 +check_range $file $objsz $objsz 62 +check_range $file $(($objsz * 2)) $halfobj 61 +check_range $file $(($objsz * 2 + $halfobj)) $halfobj 62 + +#success, all done +status=0 +exit
tests/ceph/003.out new file +11 -0
@@ -0,0 +1,11 @@ +QA output created by 003 + Copy single object to the end: + aaaa|bbbb|cccc => aaaa|bbbb|aaaa + Copy single object to the beginning: + aaaa|bbbb|aaaa => bbbb|bbbb|aaaa + Copy single object to the middle: + bbbb|bbbb|aaaa => bbbb|aaaa|aaaa + Cross object boundary (no full object copy) + aaaa|bbbb|cccc => aaaa|bbaa|aacc + aaaa|bbbb|cccc => aaaa|bbaa|bbcc + aaaa|bbbb|cccc => aaaa|bbbb|aabb
tests/ceph/group +1 -0
@@ -1,2 +1,3 @@ 001 auto quick copy 002 auto quick copy +003 auto quick copy
Quoted 4 lines · Luis Henriques
This is my second attempt to have an initial set of ceph-specific tests merged into fstests. In this patchset I'm pushing a different set of tests, focusing on the
copy_file_rangetesting, although I *do* plan to get back to the quota tests soon.
I have no knowledge about cephfs, I don't have a cephfs test env either, so I can only comment from fstests' perspect of view. It'd be great if other ceph folks could help review this patchset as well.
From fstests perspect of view, this patchset looks fine to me, just some minor comments go to individual patch.
Quoted 10 lines
This syscall has a few peculiarities in ceph as it is able to use remote object copies without the need to download/upload data from the OSDs. However, in order to take advantage of this remote copy, the copy ranges and sizes need to include at least one object. Thus, all the currently existing generic tests won't actually take advantage of this feature.
Let me know any comments/concerns about this patchset. Also note that currently, in order to enable
copy_file_rangein cephfs, the additional 'copyfrom' mount parameter is required. (Hopefully this additional param
I assume '_require_xfs_io_command "copy_range"' will _notrun the test if there's no 'copyfrom' mount option provided, and test won't fail.
Quoted 7 lines
may be dropped in the future.)
Luis Henriques (3):
ceph: addcopy_file_range(remote copy operation) testing
ceph: test combination ofcopy_file_rangewith truncate
ceph: testcopy_file_rangewith infile = outfile
Quoted 9 lines
tests/ceph/001 | 233 +++++++++++++++++++++++++++++++++++++++++++++ tests/ceph/001.out | 129 +++++++++++++++++++++++++ tests/ceph/002 | 74 ++++++++++++++ tests/ceph/002.out | 8 ++ tests/ceph/003 | 118 +++++++++++++++++++++++ tests/ceph/003.out | 11 +++ tests/ceph/group | 3 + 7 files changed, 576 insertions(+) create mode 100644 tests/ceph/001
New test should have file mode 755, i.e. with x bit set. The 'new' script should have done this for you.
Thanks,
Eryu
Quoted 6 lines
create mode 100644 tests/ceph/001.out create mode 100644 tests/ceph/002 create mode 100644 tests/ceph/002.out create mode 100644 tests/ceph/003 create mode 100644 tests/ceph/003.out create mode 100644 tests/ceph/group
Quoted 6 lines · Eryu Guan
Quoted 4 lines · Luis Henriques
This is my second attempt to have an initial set of ceph-specific tests merged into fstests. In this patchset I'm pushing a different set of tests, focusing on the
copy_file_rangetesting, although I *do* plan to get back to the quota tests soon.
Quoted 6 lines
I have no knowledge about cephfs, I don't have a cephfs test env either, so I can only comment from fstests' perspect of view. It'd be great if other ceph folks could help review this patchset as well.
From fstests perspect of view, this patchset looks fine to me, just some minor comments go to individual patch.
Thank you Eryu, for taking some time to review these tests. Much appreciated. I've gone through all your comments and they all make sense to me. I'll send out v2 shortly, addressing your concerns.
Quoted 7 lines
Quoted 7 lines
This syscall has a few peculiarities in ceph as it is able to use remote object copies without the need to download/upload data from the OSDs. However, in order to take advantage of this remote copy, the copy ranges and sizes need to include at least one object. Thus, all the currently existing generic tests won't actually take advantage of this feature.
Quoted 6 lines
Quoted 3 lines
Let me know any comments/concerns about this patchset. Also note that currently, in order to enable
copy_file_rangein cephfs, the additional 'copyfrom' mount parameter is required. (Hopefully this additional paramI assume '
_require_xfs_io_command"copy_range"' will_notrunthe test if there's no 'copyfrom' mount option provided, and test won't fail.
So, my cover-letter text was a bit confusing and not very clear about this mount option. These tests will run just fine (and won't fail) even without the 'copyfrom' mount option. The difference is that the copy won't be done remotely on the OSDs but rather using a local read+write loop (i.e. with the generic VFS copy_file_range implementation). 'copyfrom' is required only to enable the usage of the OSDs 'COPY_FROM' operation. I'm update the cover-letter text in v2 to clarify this.
Cheers,
Signature · 1 line
Luis
Quoted 29 lines
Quoted 7 lines
may be dropped in the future.)
Luis Henriques (3):
ceph: addcopy_file_range(remote copy operation) testing
ceph: test combination ofcopy_file_rangewith truncate
ceph: testcopy_file_rangewith infile = outfileQuoted 9 lines
tests/ceph/001 | 233 +++++++++++++++++++++++++++++++++++++++++++++ tests/ceph/001.out | 129 +++++++++++++++++++++++++ tests/ceph/002 | 74 ++++++++++++++ tests/ceph/002.out | 8 ++ tests/ceph/003 | 118 +++++++++++++++++++++++ tests/ceph/003.out | 11 +++ tests/ceph/group | 3 + 7 files changed, 576 insertions(+) create mode 100644 tests/ceph/001New test should have file mode 755, i.e. with x bit set. The 'new' script should have done this for you.
Thanks,
EryuQuoted 6 lines
create mode 100644 tests/ceph/001.out create mode 100644 tests/ceph/002 create mode 100644 tests/ceph/002.out create mode 100644 tests/ceph/003 create mode 100644 tests/ceph/003.out create mode 100644 tests/ceph/group