Revert "NFSD: Force all NFSv4.2 COPY requests to be synchronous"
Subsystems (2)
Chuck Lever
maintainerRFC PATCH 1/1 Revert "NFSD: Force all NFSv4.2 COPY requests to be synchronous"
From: Chuck Lever <[email protected]>
In the past several kernel releases, we've made NFSv4.2 async copy reliable:
- The Linux NFS client and server now both implement and use the NFSv4.2 OFFLOAD_STATUS operation
- The Linux NFS server keeps copy stateids around longer
- The Linux NFS client and server now both implement referring call lists
And resilient against DoS:
- The Linux NFS server limits the number of concurrent async copy operations
--- fs/nfsd/nfs4proc.c | 7 ------- 1 file changed, 7 deletions(-)
fs/nfsd/nfs4proc.c +0 -7
@@ -1917,13 +1917,6 @@ nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,struct nfsd42_write_res *result;__be32 status; -/* -* Currently, async COPY is not reliable. Force all COPY -* requests to be synchronous to avoid client application -* hangs waiting for COPY completion. -*/ -nfsd4_copy_set_sync (copy, true); - result = ©->cp_res;nfsd_copy_write_verifier ((__be32 *)&result->wr_verifier.data, nn);
Signature · 1 line
2.49.0
Quoted 27 lines · Chuck Lever
From: Chuck Lever <[email protected]>
In the past several kernel releases, we've made NFSv4.2 async copy reliable:
- The Linux NFS client and server now both implement and use the NFSv4.2OFFLOAD_STATUSoperation
- The Linux NFS server keeps copy stateids around longer
- The Linux NFS client and server now both implement referring call listsAnd resilient against DoS:
- The Linux NFS server limits the number of concurrent async copy operationsSigned-off-by Chuck Lever <[email protected]>--- fs/nfsd/nfs4proc.c | 7 ------- 1 file changed, 7 deletions(-)fs/nfsd/nfs4proc.c +0
-0@@ -1917,13 +1917,6 @@ nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,struct nfsd42_write_res *result;__be32 status;
fs/nfsd/nfs4proc.c nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, · line 1922
Quoted 10 lines
fs/nfsd/nfs4proc.c +0
-7-/* -* Currently, async COPY is not reliable. Force all COPY -* requests to be synchronous to avoid client application -* hangs waiting for COPY completion. -*/ -nfsd4_copy_set_sync (copy, true); - result = ©->cp_res;nfsd_copy_write_verifier ((__be32 *)&result->wr_verifier.data, nn);
Quoted 11 lines · Chuck Lever
From: Chuck Lever <[email protected]>
In the past several kernel releases, we've made NFSv4.2 async copy reliable:
- The Linux NFS client and server now both implement and use the NFSv4.2OFFLOAD_STATUSoperation
- The Linux NFS server keeps copy stateids around longer
- The Linux NFS client and server now both implement referring call lists
Quoted 3 lines
And resilient against DoS:
- The Linux NFS server limits the number of concurrent async copy operations
And how does an amin change that limit? There is zero documentation for admins, and zero training or reference material for COPY, CLONE, ALLOCATE, ...
Aurélien
Signature · 2 lines
Aurélien Couderc <[email protected]> Big Data/Data mining expert, chess enthusiast
?
Quoted 25 lines · Aurélien Couderc
Quoted 11 lines · Chuck Lever
From: Chuck Lever <[email protected]>
In the past several kernel releases, we've made NFSv4.2 async copy reliable:
- The Linux NFS client and server now both implement and use the NFSv4.2OFFLOAD_STATUSoperation
- The Linux NFS server keeps copy stateids around longer
- The Linux NFS client and server now both implement referring call listsQuoted 3 lines
And resilient against DoS:
- The Linux NFS server limits the number of concurrent async copy operationsAnd how does an amin change that limit? There is zero documentation for admins, and zero training or reference material for COPY, CLONE, ALLOCATE, ...
Aurélien
--
Aurélien Couderc <[email protected]>
Big Data/Data mining expert, chess enthusiast
Signature · 2 lines
Aurélien Couderc <[email protected]> Big Data/Data mining expert, chess enthusiast
Chuck Lever
maintainerQuoted 32 lines · Aurélien Couderc
?
Quoted 25 lines · Aurélien Couderc
Quoted 11 lines · Chuck Lever
From: Chuck Lever <[email protected]>
In the past several kernel releases, we've made NFSv4.2 async copy reliable:
- The Linux NFS client and server now both implement and use the NFSv4.2OFFLOAD_STATUSoperation
- The Linux NFS server keeps copy stateids around longer
- The Linux NFS client and server now both implement referring call listsQuoted 3 lines
And resilient against DoS:
- The Linux NFS server limits the number of concurrent async copy operationsAnd how does an amin change that limit? There is zero documentation for admins, and zero training or reference material for COPY, CLONE, ALLOCATE, ...
Aurélien
--
Aurélien Couderc <[email protected]>
Big Data/Data mining expert, chess enthusiast
The tone of your original email suggested to me that it was a whine rather than a genuine request for more information.
Also the request for "training material" for individual NFSv4.2 operations does not make sense. We do not have training material for the NFSv3 READDIRPLUS procedure, for example.
Therefore I ignored the email.
If you want reference material, the obvious place to look is the Internet standard that specifies these new operations (RFC 7862). It is publicly accessible on the web. If you need something more basic, I highly recommend Callaghan's "NFS Illustrated" (ISBN: 9780321618924).
NFSv4.2 COPY, CLONE, and ALLOCATE are exposed to applications via the POSIX file system API available in Linux, as documented in man pages.
There is no specific administrative setting that controls the number of concurrent asynchronous COPY operations. The limit increases with the number of NFSD threads. If someone demonstrates a specific use case where manually adjusting that limit is necessary, a setting can be implemented.
The specific guidance you might be looking for is typically provided by the documentation staff at Linux distributions. You could file an issue with your favorite Linux vendor to let them know what you are looking for.
Signature · 1 line
Chuck Lever
Quoted 35 lines · Chuck Lever
Quoted 32 lines · Aurélien Couderc
?
Quoted 25 lines · Aurélien Couderc
Quoted 11 lines · Chuck Lever
From: Chuck Lever <[email protected]>
In the past several kernel releases, we've made NFSv4.2 async copy reliable:
- The Linux NFS client and server now both implement and use the NFSv4.2OFFLOAD_STATUSoperation
- The Linux NFS server keeps copy stateids around longer
- The Linux NFS client and server now both implement referring call listsQuoted 3 lines
And resilient against DoS:
- The Linux NFS server limits the number of concurrent async copy operationsAnd how does an amin change that limit? There is zero documentation for admins, and zero training or reference material for COPY, CLONE, ALLOCATE, ...
Aurélien
--
Aurélien Couderc <[email protected]>
Big Data/Data mining expert, chess enthusiast
Quoted 2 lines
The tone of your original email suggested to me that it was a whine rather than a genuine request for more information.
Well, put it that way: I am not a native english speaker.
Imagine a world in the multiverse where the French retained their colonies instead of the British empire. French would be the dominant language of that blue marble. Programming languages would use french words. Someone from the tiny leftover remnants of the British empire would post in the linnaeus-sdfr@ (système de fichiers réseau, AKA NFS) mailing list, and that post would be called "a whine", because his French is not perfect...
Quoted 6 lines
Also the request for "training material" for individual NFSv4.2 operations does not make sense. We do not have training material for the NFSv3 READDIRPLUS procedure, for example.
Therefore I ignored the email.
OK, but as an analog: SMB is infamous for "too many" features, all which can cause trouble. Over time SAMBA added controls to turn features on/off or use different ways of emulation. So far NFSv4.2 has no controls to turn specific features on/off, or even get statistics, or put limits on certain features.
That IS a problem, which SAMBA and even Windows Server SMB have solved. Otherwise you're at the mercy of whatever combination of NFS client and NFS server you have, and that is NOT good.
Aurélien
Signature · 2 lines
Aurélien Couderc <[email protected]> Big Data/Data mining expert, chess enthusiast
Quoted 55 lines · Aurélien Couderc
Quoted 35 lines · Chuck Lever
Quoted 32 lines · Aurélien Couderc
?
Quoted 25 lines · Aurélien Couderc
Quoted 11 lines · Chuck Lever
From: Chuck Lever <[email protected]>
In the past several kernel releases, we've made NFSv4.2 async copy reliable:
- The Linux NFS client and server now both implement and use the NFSv4.2OFFLOAD_STATUSoperation
- The Linux NFS server keeps copy stateids around longer
- The Linux NFS client and server now both implement referring call listsQuoted 3 lines
And resilient against DoS:
- The Linux NFS server limits the number of concurrent async copy operationsAnd how does an amin change that limit? There is zero documentation for admins, and zero training or reference material for COPY, CLONE, ALLOCATE, ...
Aurélien
--
Aurélien Couderc <[email protected]>
Big Data/Data mining expert, chess enthusiastQuoted 2 lines
The tone of your original email suggested to me that it was a whine rather than a genuine request for more information.
Well, put it that way: I am not a native english speaker.
Imagine a world in the multiverse where the French retained their colonies instead of the British empire. French would be the dominant language of that blue marble. Programming languages would use french words. Someone from the tiny leftover remnants of the British empire would post in the linnaeus-sdfr@ (système de fichiers réseau, AKA NFS) mailing list, and that post would be called "a whine", because his French is not perfect...
Quoted 5 lines
Also the request for "training material" for individual NFSv4.2 operations does not make sense. We do not have training material for the NFSv3 READDIRPLUS procedure, for example.
Quoted 5 lines
Quoted 1 line
Therefore I ignored the email.
OK, but as an analog: SMB is infamous for "too many" features, all which can cause trouble. Over time SAMBA added controls to turn features on/off or use different ways of emulation. So far NFSv4.2
Apologies, that should have been "the LINUX NFSv4.2 server implementation"
Quoted 4 lines
has
no controls to turn specific features on/off, or even get statistics, or put limits on certain features.
Quoted 8 lines
That IS a problem, which SAMBA and even Windows Server SMB have solved. Otherwise you're at the mercy of whatever combination of NFS client and NFS server you have, and that is NOT good.
Aurélien
--
Aurélien Couderc <[email protected]>
Big Data/Data mining expert, chess enthusiast
Aurélien
Signature · 2 lines
Aurélien Couderc <[email protected]> Big Data/Data mining expert, chess enthusiast
Quoted 10 lines · Aurélien Couderc
Quoted 5 lines · Chuck Lever
Also the request for "training material" for individual NFSv4.2 operations does not make sense. We do not have training material for the NFSv3 READDIRPLUS procedure, for example.
Therefore I ignored the email.
OK, but as an analog: SMB is infamous for "too many" features, all which can cause trouble. Over time SAMBA added controls to turn features on/off or use different ways of emulation.
Samba has to navigate between two rather incompatible worlds: POSIX as it is implemented on Linux and FreeBSD, and the Windows world, via a non-POSIX network file protocol (SMB). There needs to be some flexibility of configuration there.
Quoted 3 lines
So far NFSv4.2 has
no controls to turn specific features on/off, or even get statistics, or put limits on certain features.
There has to be a demonstrated need for each such control. We're not going to add controls that don't have any real use because controls actually have a long-term cost. One or two might not be expensive to maintain, but when you add them with abandon, it adds up:
- Administrative complexity increases
- Our test matrix increases exponentially
- The documentation workload increases
- Kernel API rules make it difficult to fix mistakes or remove deprecated controls
- Replacing a constant with a control setting has a small run-time cost
- Developing around these controls can sometimes be difficult
There are very good reasons why Gnome removed most of their configuration settings a few years back. Eventually it becomes impossible to manage and maintain the software.
This is not to say we won't add a control if it should become necessary.
So do you have a need to disable an NFSv4.2 feature? If so, which ones, and why?
Do you have a need to limit some feature? If so, why?
What operations and events do you want to count? Why? Why can't you use eBPF, kprobes, Dtrace, systemtap ?
Since this is open source, can you contribute what you need rather than asking us to implement it?
Quoted 3 lines
That IS a problem, which SAMBA and even Windows Server SMB have solved. Otherwise you're at the mercy of whatever combination of NFS client and NFS server you have, and that is NOT good.
That is still a very generic complaint. If you have a specific issue or question, please post it. Simply saying "we need more controls" is just not actionable.
Signature · 1 line
Chuck Lever