NFS testing
Subsystems (2)
Chuck was asking about the regular NFS testing I do. Cc'ing the list in case this is useful to anyone else.
We'd really like to set up regular testing on some kind of common infrastructure that more people can get to. I'm not sure how to get there yet.
For now all I have is this pile of bash scripts that build kernels and boot test VMs and run tests between them, all on a machine in my basement:
http://git.linux-nfs.org/?p=bfields/testd.git;a=tree
The "install.sh" file has notes on my setup.
The thing I run most regularly is this:
http://git.linux-nfs.org/?p=bfields/testd.git;a=blob;f=bin/do-all
which runs:
- connectathon, on v3, v4, v4.1, and v4.2, each with both auth_sys and krb5. Except v4.2, which I run with auth_sys, krb5i, and krb5p.
- at the same time as connectathon (so with the same permutations as the above), I also run the tests in
http://git.linux-nfs.org/?p=bfields/lock-tests.git
which check basic lock and lease functionality plus some odd corner cases I ran into some years ago.
- pynfs 4.0 tests:
http://git.linux-nfs.org/?p=bfields/testd.git;a=blob;f=bin/do-pynfs
I do two auth_sys runs, one as root, one not. (Tests that create special files need root, tests that expect permission failures need non-root). A couple krb5 runs are temporarily disabled.
My goal is that a normal result should be "everything passed", so that it stands out when there's a regression. So I skip a few known failures:
http://git.linux-nfs.org/?p=bfields/testd.git;a=blob;f=data/pynfs-skip
- pynfs 4.1 tests: similar if a little less complicated:
http://git.linux-nfs.org/?p=bfields/testd.git;a=blob;f=bin/do-pynfs41
4.1 tests to skip:
http://git.linux-nfs.org/?p=bfields/testd.git;a=blob;f=data/pynfs41-skip
- xfstests:
http://git.linux-nfs.org/?p=bfields/testd.git;a=blob;f=bin/do-xfstests
This is the most time-consuming, so I just do a single run over NFSv4.2 with auth_sys.
As with pynfs I aim for the normal result to be "everything passed", so I skip these known failures:
http://git.linux-nfs.org/?p=bfields/testd.git;a=blob;f=data/xfstests-failed
I also normally skip anything that takes more than a couple minutes, unfortunately:
http://git.linux-nfs.org/?p=bfields/testd.git;a=blob;f=data/xfstests-slow
I also have some other test sets in there that I don't use as often. And I "bisect" script that will run other tests, e.g.
qtst <host> bisect <good> <bad> xfstests generic/465
and it'll crank away for a few hours and (hopefully) mail me a git-bisect result telling me where generic/465 started failing.
--b.
I'm embarrassed that I don't do anything close to this for my weekly nfs-ganesha merges...
I'd love to see some tools to make it easier to do more testing.
I wish there was a solution that didn't require two hosts for testing NFSv3 locks against nfs-ganesha.
I need to get back to the pynfs v3 tests...
Another test I have run is pjdfstests, which there seem to be multiple versions of, I have repos of two versions here that are Ganesha aware. These tests include a lot of POSIX compliance so will catch permission issues and other POSIX oddities. I also haven't run them in ages...
https://github.com/ffilz/ntfs-3g-pjd-fstest
https://github.com/ffilz/pjdfstest
The nfs-ganesha tree also has a tool to help write lock tests (and can be used for a few other things too - I recently added more flags to open so it can test most open system call invocations of interest) in:
https://github.com/nfs-ganesha/nfs-ganesha/tree/next/src/tools/multilock
Hmm, I should incorporate the features of Bruce's lock tests into that...
The multilock tool can drive multiple clients to use system calls (so any POSIX client can be used including a local process on the server) or Gluster's libgfapi or Cephs libcephfs.
Frank
Quoted 8 lines
-----Original Message-----
From: J. Bruce Fields [mailto:[email protected]]
Sent: Friday, January 7, 2022 1:35 PM
To: Chuck Lever <[email protected]>
Cc: [email protected]
Subject: NFS testingChuck was asking about the regular NFS testing I do. Cc'ing the list in
case this is
Quoted 3 lines
useful to anyone else.
We'd really like to set up regular testing on some kind of common
infrastructure
Quoted 3 lines
that more people can get to. I'm not sure how to get there yet.
For now all I have is this pile of bash scripts that build kernels and
boot test VMs
Quoted 19 lines
and run tests between them, all on a machine in my basement:
http://git.linux-nfs.org/?p=bfields/testd.git;a=tree
The "
install.sh" file has notes on my setup.The thing I run most regularly is this:
http://git.linux-nfs.org/?p=bfields/testd.git;a=blob;f=bin/do-allwhich runs:
- connectathon, on v3, v4, v4.1, and v4.2, each with both
auth_sysand krb5. Except v4.2, which I run withauth_sys, krb5i, and krb5p.
- at the same time as connectathon (so with the same permutations as the above), I also run the tests in
Quoted 6 lines
http://git.linux-nfs.org/?p=bfields/lock-tests.git
which check basic lock and lease functionality plus some odd corner cases I ran into some years ago.
- pynfs 4.0 tests:
http://git.linux-nfs.org/?p=bfields/testd.git;a=blob;f=bin/do-
Quoted 6 lines
pynfs
I do twoauth_sysruns, one as root, one not. (Tests that create special files need root, tests that expect permission failures need non-root). A couple krb5 runs are temporarily disabled.
Quoted 7 lines
My goal is that a normal result should be "everything passed", so that it stands out when there's a regression. So I skip a few known failures:
http://git.linux-
nfs.org/?p=bfields/testd.git;a=blob;f=data/pynfs-skip
- pynfs 4.1 tests: similar if a little less complicated:
http://git.linux-nfs.org/?p=bfields/testd.git;a=blob;f=bin/do-
Quoted 6 lines
pynfs41
4.1 tests to skip: http://git.linux-nfs.org/?p=bfields/testd.git;a=blob;f=data/pynfs41-skip
- xfstests:
http://git.linux-nfs.org/?p=bfields/testd.git;a=blob;f=bin/do-
Quoted 11 lines
xfstests
This is the most time-consuming, so I just do a single run over NFSv4.2 withauth_sys.As with pynfs I aim for the normal result to be "everything passed", so I skip these known failures:
http://git.linux-
nfs.org/?p=bfields/testd.git;a=blob;f=data/xfstests-failedI also normally skip anything that takes more than a couple minutes, unfortunately: http://git.linux-
Quoted 8 lines
nfs.org/?p=bfields/testd.git;a=blob;f=data/xfstests-slow
I also have some other test sets in there that I don't use as often. And I "bisect" script that will run other tests, e.g.
qtst <host> bisect <good> <bad> xfstests generic/465
and it'll crank away for a few hours and (hopefully) mail me a git-bisect
result
Quoted 3 lines
telling me where generic/465 started failing.
--b.
Quoted 10 lines · J. Bruce Fields
Chuck was asking about the regular NFS testing I do. Cc'ing the list in case this is useful to anyone else.
We'd really like to set up regular testing on some kind of common infrastructure that more people can get to. I'm not sure how to get there yet.
[ useful stuff snipped ]
A little context: Since Bruce has announced he is stepping away from his NFSD maintainer role, I'd like to help give him a breather with his testing responsibilities too. We've been talking about shifting some of this testing to a public CI infrastructure like kernelci.org.
And in the short term, obviously I want to ensure that NFSD pull requests are adequately exercised...
It would be nice to have a facility that automatically grabbed new commits pushed to our for-next branches and ran tests on a variety of platforms, much like GitHub Actions does. The kernelci.org infrastructure is intriguing to me, though their usual case is booting up a single guest to run a series of tests. In our community, there usually needs to be a client host and a separate server host.
--
Chuck Lever