fstests: Add lustre support
PATCH 1/1 fstests: Add lustre support
Add basic lustre support so that we can continuously improve lustre by fstests.
--- README | 2 ++ README.config-sections | 7 +++++++ check | 3 ++- common/config | 5 +++++ common/rc | 29 ++++++++++++++++++++++++++++- 5 files changed, 44 insertions(+), 2 deletions(-)
README +2 -0
@@ -87,6 +87,8 @@ L4: Active support from the fs list, has lots of own cases. +------------+-------+---------------------------------------------------------+ | 9p | L1 | N/A | +------------+-------+---------------------------------------------------------+ +| lustre | L1 | N/A | ++------------+-------+---------------------------------------------------------+ _______________________ BUILDING THE FSQA SUITE
README.config-sections +7 -0
@@ -135,3 +135,10 @@ TEST_DEV=%example.com:xfstest.test TEST_DIR=/mnt/xfstest.test SCRATCH_DEV=%example.com:xfstest.scratch SCRATCH_MNT=/mnt/xfstest.scratch + +[lustre] +FSTYP=lustre +TEST_DIR=/mnt/test +TEST_DEV=192.168.1.1@tcp:/test +SCRATCH_MNT=/mnt/scratch +SCRATCH_DEV=192.168.1.1@tcp:/scratch
check +2 -1
@@ -70,6 +70,7 @@ check options -pvfs2 test PVFS2 -tmpfs test TMPFS -ubifs test ubifs + -lustre test lustre -l line mode diff -udiff show unified diff (default) -n show me, do not run tests
@@ -277,7 +278,7 @@ while [ $# -gt 0 ]; do case "$1" in -\? | -h | --help) usage ;; - -nfs|-afs|-glusterfs|-cifs|-9p|-fuse|-virtiofs|-pvfs2|-tmpfs|-ubifs) + -nfs|-afs|-glusterfs|-cifs|-9p|-fuse|-virtiofs|-pvfs2|-tmpfs|-ubifs|-lustre) FSTYP="${1:1}" ;; -overlay)
common/config +5 -0
@@ -394,6 +394,9 @@ _common_mount_opts() ubifs) echo $UBIFS_MOUNT_OPTIONS ;; + lustre) + echo $LUSTRE_MOUNT_OPTIONS + ;; *) ;; esac
@@ -539,6 +542,8 @@ _source_specific_fs() [ "$UBIUPDATEVOL_PROG" = "" ] && _fatal "ubiupdatevol not found" . ./common/ubifs ;; + lustre) + ;; esac }
common/rc +28 -1
@@ -867,6 +867,9 @@ _test_mkfs() pvfs2) # do nothing for pvfs2 ;; + lustre) + # do nothing for lustre + ;; udf) $MKFS_UDF_PROG $MKFS_OPTIONS $* $TEST_DEV > /dev/null ;;
@@ -981,7 +984,7 @@ _scratch_mkfs() local mkfs_status case $FSTYP in - nfs*|afs|cifs|ceph|overlay|glusterfs|pvfs2|9p|fuse|virtiofs) + nfs*|afs|cifs|ceph|overlay|glusterfs|pvfs2|9p|fuse|virtiofs|lustre) # unable to re-create this fstyp, just remove all files in # $SCRATCH_MNT to avoid EEXIST caused by the leftover files # created in previous runs
@@ -2064,6 +2067,15 @@ _require_scratch_nocheck() _notrun "this test requires a valid \$SCRATCH_MNT" fi ;; + lustre) + echo $SCRATCH_DEV | grep -q "@.*:/" > /dev/null 2>&1 + if [ -z "$SCRATCH_DEV" -o "$?" != "0" ]; then + _notrun "this test requires a valid \$SCRATCH_DEV" + fi + if [ ! -d "$SCRATCH_MNT" ]; then + _notrun "this test requires a valid \$SCRATCH_MNT" + fi + ;; *) if [ -z "$SCRATCH_DEV" -o "`_is_block_dev "$SCRATCH_DEV"`" = "" ] then
@@ -2288,6 +2300,15 @@ _require_test() _notrun "this test requires a valid \$TEST_DIR" fi ;; + lustre) + echo $TEST_DEV | grep -q "@.*:/" > /dev/null 2>&1 + if [ -z "$TEST_DEV" -o "$?" != "0" ]; then + _notrun "this test requires a valid \$TEST_DEV" + fi + if [ ! -d "$TEST_DIR" ]; then + _notrun "this test requires a valid \$TEST_DIR" + fi + ;; *) if [ -z "$TEST_DEV" ] || [ "`_is_block_dev "$TEST_DEV"`" = "" ] then
@@ -3658,6 +3679,9 @@ _check_test_fs() ;; pvfs2) ;; + lustre) + # no way to check consistency for lustre + ;; udf) # do nothing for now ;;
@@ -3720,6 +3744,9 @@ _check_scratch_fs() ;; pvfs2) ;; + lustre) + # no way to check consistency for lustre + ;; btrfs) _check_btrfs_filesystem $device ;;
Signature · 1 line
2.43.7
Quoted 2 lines · Xiao Yang
Add basic lustre support so that we can continuously improve lustre by fstests.
xfstests is only supposed to test upstream file systems and not perpetual out of tree things like Lustre. Community is a giving and taking.
Quoted 6 lines
Quoted 2 lines
Add basic lustre support so that we can continuously improve lustre by fstests.
xfstests is only supposed to test upstream file systems and not perpetual out of tree things like Lustre. Community is a giving and taking.
For better or worse, xfstests does support some out-of-tree filesystems like gluster and bcachefs (both listed in the support matrix in the README, which also suggests that " Firstly, there's not hard restriction about which filesystem can use fstests. Any filesystem can give fstests a try"). Lustre support wouldn't seem that out of place, even if it takes a while for Lustre to make its way in-tree.
Tim Day
Quoted 9 lines · Timothy
For better or worse, xfstests does support some out-of-tree filesystems like gluster and bcachefs (both listed in the support matrix in the README, which also suggests that " Firstly, there's not hard restriction about which filesystem can use fstests. Any filesystem can give fstests a try"). Lustre support wouldn't seem that out of place, even if it takes a while for Lustre to make its way in-tree.
Tim Day
Agreed. According to the README, I think xfstests can also support lustre. We can improve lustre through xfstests and give various feedback to xfstests.
Best Regards,
Xiao Yang
Quoted 9 lines · Timothy
Quoted 6 lines
Quoted 2 lines
Add basic lustre support so that we can continuously improve lustre by fstests.
xfstests is only supposed to test upstream file systems and not perpetual out of tree things like Lustre. Community is a giving and taking.
For better or worse, xfstests does support some out-of-tree filesystems like gluster
gluster is a userspace file system using fuse.
Quoted 1 line
and bcachefs (both listed in the support
bcachefs was upstream, but yes this should be cleaned up.
Quoted 2 lines
Add basic lustre support so that we can continuously improve lustre by fstests.
fstests accepts a wildcard filesystem type. I _think_ the wildcard assumes the filesystem is a disk filesystem. But could we improve that enough that it would work for Lustre? The changes required for Lustre aren't huge, so that seems plausible.
Tim Day
Quoted 7 lines · Timothy
fstests accepts a wildcard filesystem type.
I_think_the wildcard assumes the filesystem is a disk filesystem. But could we improve that enough that it would work for Lustre? The changes required for Lustre aren't huge, so that seems plausible.Tim Day
Hello, Tim
Thanks for your reply.
I think this patch can make xfstests support lustre.
Would you like to improve the patch or add other features? If so, could you provide some more details?
Best Regards,
Xiao Yang
Quoted 5 lines · Xiao Yang
Add basic lustre support so that we can continuously improve lustre by fstests.
Signed-off-by Xiao Yang <[email protected]>---
Hi Yang,
As you can see, the fstests community doesn't like to spend time on maintaining fstests for out-of-tree fs, especially downstream-only filesystems. I'm not a dictator. If we accept to support someone filesystem and its patches, that means we'd like to spend time to review patches for it, think about it when update fstests, fix issues on it, and so on. So I'd like to respect the willing of fstests community.
For bcachefs, it was in-tree, and we know its contributors are still trying to keep it up to date. So I'd like to keep its supporting.
For glusterfs, it's an upstream open-source filesystem, it's base on FUSE of linux kernel. It can't be a kernel module, but it's a pure upstream fs at least. Actually I brought in it for testing FUSE at beginning.
So do you do this for *upstreaming* the lustre filesystem (any active source code link?) or just for using xfstests downstream? If it's the latter, I'd suggest you to fork xfstests, and maintain this patch downstream.
I noticed that there was a discussion about Lustre filesystem upstreaming:
https://lore.kernel.org/linux-fsdevel/[email protected]/T/#med943c38d27ddac1b68e9db37d6b0f2803ee6133
But I haven't found it's online. So if the "lustre upstreaming" project get an offical version be acked by mainline linux, fstests willing to support its later developing/testing. At your early developing stage, I think you can fork and maintain a downstream xfstests temporarily.
Thanks,
Zorro
Quoted 157 lines
README | 2 ++ README.config-sections | 7 +++++++ check | 3 ++- common/config | 5 +++++ common/rc | 29 ++++++++++++++++++++++++++++- 5 files changed, 44 insertions(+), 2 deletions(-)README +2
-0@@ -87,6 +87,8 @@ L4: Active support from the fs list, has lots of own cases. +------------+-------+---------------------------------------------------------+ | 9p | L1 | N/A | +------------+-------+---------------------------------------------------------+ +| lustre | L1 | N/A | ++------------+-------+---------------------------------------------------------+ _______________________ BUILDING THE FSQA SUITEREADME.config-sections +7
-0@@ -135,3 +135,10 @@ TEST_DEV=%example.com:xfstest.test TEST_DIR=/mnt/xfstest.test SCRATCH_DEV=%example.com:xfstest.scratch SCRATCH_MNT=/mnt/xfstest.scratch + +[lustre] +FSTYP=lustre +TEST_DIR=/mnt/test +TEST_DEV=192.168.1.1@tcp:/test +SCRATCH_MNT=/mnt/scratch +SCRATCH_DEV=192.168.1.1@tcp:/scratchcheck +2
-1@@ -70,6 +70,7 @@ check options -pvfs2 test PVFS2 -tmpfs test TMPFS -ubifs test ubifs + -lustre test lustre -l line mode diff -udiff show unified diff (default) -n show me, do not run tests@@ -277,7 +278,7 @@ while [ $# -gt 0 ]; do case "$1" in -\? | -h | --help) usage ;; - -nfs|-afs|-glusterfs|-cifs|-9p|-fuse|-virtiofs|-pvfs2|-tmpfs|-ubifs) + -nfs|-afs|-glusterfs|-cifs|-9p|-fuse|-virtiofs|-pvfs2|-tmpfs|-ubifs|-lustre) FSTYP="${1:1}" ;; -overlay)common/config +5
-0@@ -394,6 +394,9 @@ _common_mount_opts() ubifs) echo $UBIFS_MOUNT_OPTIONS ;; + lustre) + echo $LUSTRE_MOUNT_OPTIONS + ;; *) ;; esac@@ -539,6 +542,8 @@ _source_specific_fs() [ "$UBIUPDATEVOL_PROG" = "" ] && _fatal "ubiupdatevol not found" . ./common/ubifs ;; + lustre) + ;; esac }common/rc +28
-1@@ -867,6 +867,9 @@ _test_mkfs() pvfs2) # do nothing for pvfs2 ;; + lustre) + # do nothing for lustre + ;; udf) $MKFS_UDF_PROG $MKFS_OPTIONS $* $TEST_DEV > /dev/null ;;@@ -981,7 +984,7 @@ _scratch_mkfs() local mkfs_status case $FSTYP in - nfs*|afs|cifs|ceph|overlay|glusterfs|pvfs2|9p|fuse|virtiofs) + nfs*|afs|cifs|ceph|overlay|glusterfs|pvfs2|9p|fuse|virtiofs|lustre) # unable to re-create this fstyp, just remove all files in # $SCRATCH_MNT to avoid EEXIST caused by the leftover files # created in previous runs@@ -2064,6 +2067,15 @@ _require_scratch_nocheck() _notrun "this test requires a valid \$SCRATCH_MNT" fi ;; + lustre) + echo $SCRATCH_DEV | grep -q "@.*:/" > /dev/null 2>&1 + if [ -z "$SCRATCH_DEV" -o "$?" != "0" ]; then + _notrun "this test requires a valid \$SCRATCH_DEV" + fi + if [ ! -d "$SCRATCH_MNT" ]; then + _notrun "this test requires a valid \$SCRATCH_MNT" + fi + ;; *) if [ -z "$SCRATCH_DEV" -o "`_is_block_dev "$SCRATCH_DEV"`" = "" ] then@@ -2288,6 +2300,15 @@ _require_test() _notrun "this test requires a valid \$TEST_DIR" fi ;; + lustre) + echo $TEST_DEV | grep -q "@.*:/" > /dev/null 2>&1 + if [ -z "$TEST_DEV" -o "$?" != "0" ]; then + _notrun "this test requires a valid \$TEST_DEV" + fi + if [ ! -d "$TEST_DIR" ]; then + _notrun "this test requires a valid \$TEST_DIR" + fi + ;; *) if [ -z "$TEST_DEV" ] || [ "`_is_block_dev "$TEST_DEV"`" = "" ] then@@ -3658,6 +3679,9 @@ _check_test_fs() ;; pvfs2) ;; + lustre) + # no way to check consistency for lustre + ;; udf) # do nothing for now ;;@@ -3720,6 +3744,9 @@ _check_scratch_fs() ;; pvfs2) ;; + lustre) + # no way to check consistency for lustre + ;; btrfs) _check_btrfs_filesystem $device ;;Signature · 1 line
2.43.7
Quoted 23 lines · Zorro Lang
Hi Yang,
As you can see, the fstests community doesn't like to spend time on maintaining fstests for out-of-tree fs, especially downstream-only filesystems. I'm not a dictator. If we accept to support someone filesystem and its patches, that means we'd like to spend time to review patches for it, think about it when update fstests, fix issues on it, and so on. So I'd like to respect the willing of fstests community.
For bcachefs, it was in-tree, and we know its contributors are still trying to keep it up to date. So I'd like to keep its supporting.
For glusterfs, it's an upstream open-source filesystem, it's base on FUSE of linux kernel. It can't be a kernel module, but it's a pure upstream fs at least. Actually I brought in it for testing FUSE at beginning.
So do you do this for*upstreaming* the lustre filesystem (any active source code link?) or just for using xfstests downstream? If it's the latter, I'd suggest you to fork xfstests, and maintain this patch downstream.
I noticed that there was a discussion about Lustre filesystem upstreaming:
https://lore.kernel.org/linux-fsdevel/[email protected]/T/#med943c38d27ddac1b68e9db37d6b0f2803ee6133
Quoted 7 lines
But I haven't found it's online. So if the "lustre upstreaming" project get an offical version be acked by mainline linux, fstests willing to support its later developing/testing. At your early developing stage, I think you can fork and maintain a downstream xfstests temporarily.
Thanks,
Zorro
Hi zorro
Thanks for your explanation.
OK, I will fork xfstests and maintain it downstream for now.
Best Regards,
Xiao Yang