nfsd: don't hand out write delegations on O_WRONLY opens
Subsystems (2)
Jeff Layton
maintainerPATCH v2 1/1 nfsd: don't hand out write delegations on O_WRONLY opens
I noticed that xfstests generic/001 was failing against linux-next nfsd.
The client would request a OPEN4_SHARE_ACCESS_WRITE open, and the server would hand out a write delegation. The client would then try to use that write delegation as the source stateid in a COPY or CLONE operation, and the server would respond with NFS4ERR_STALE.
The problem is that the struct file associated with the delegation does not necessarily have read permissions. It's handing out a write delegation on what is effectively an O_WRONLY open. RFC 8881 states:
"An OPEN_DELEGATE_WRITE delegation allows the client to handle, on its
own, all opens."
Given that the client didn't request any read permissions, and that nfsd didn't check for any, it seems wrong to give out a write delegation.
Only hand out a write delegation if we have a O_RDWR descriptor available. If it fails to find an appropriate write descriptor, go ahead and try for a read delegation if NFS4_SHARE_ACCESS_READ was requested.
This fixes xfstest generic/001.
---
Changes in v2:
- Rework the logic when finding struct file for the delegation. The earlier patch might still have attached a O_WRONLY file to the deleg in some cases, and could still have handed out a write delegation on an O_WRONLY OPEN request in some cases.
--- fs/nfsd/nfs4state.c | 29 ++++++++++++++++++----------- 1 file changed, 18 insertions(+), 11 deletions(-)
fs/nfsd/nfs4state.c +18 -11
@@ -5449,7 +5449,7 @@ nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp,struct nfs4_file *fp = stp->st_stid.sc_file;struct nfs4_clnt_odstate *odstate = stp->st_clnt_odstate;struct nfs4_delegation *dp; -struct nfsd_file *nf; +struct nfsd_file *nf =NULL ;struct file_lock *fl;u32 dl_type;
@@ -5461,21 +5461,28 @@ nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp,if (fp->fi_had_conflict)return ERR_PTR (-EAGAIN ); -if (open->op_share_access &NFS4_SHARE_ACCESS_WRITE ) { - nf =find_writeable_file (fp); +/* +* Try for a write delegation first. We need an O_RDWR file +* since a write delegation allows the client to perform any open +* from its cache. +*/ +if ((open->op_share_access &NFS4_SHARE_ACCESS_BOTH ) ==NFS4_SHARE_ACCESS_BOTH ) { + nf =nfsd_file_get (fp->fi_fds[O_RDWR ]); dl_type =NFS4_OPEN_DELEGATE_WRITE ; - }else { + } + +/* +* If the file is being opened O_RDONLY or we couldn't get a O_RDWR +* file for some reason, then try for a read deleg instead. +*/ +if (!nf && (open->op_share_access &NFS4_SHARE_ACCESS_READ )) { nf =find_readable_file (fp); dl_type =NFS4_OPEN_DELEGATE_READ ; } -if (!nf) { -/* -* We probably could attempt another open and get a read -* delegation, but for now, don't bother until the -* client actually sends us one. -*/ + +if (!nf)return ERR_PTR (-EAGAIN ); - } +spin_lock (&state_lock);spin_lock (&fp->fi_lock);if (nfs4_delegation_exists (clp, fp))
---
base-commit: a734662572708cf062e974f659ae50c24fc1ad17
change-id: 20230731-wdeleg-bbdb6b25a3c6
Best regards,
Signature · 1 line
Jeff Layton <[email protected]>
Quoted 52 lines · Jeff Layton
I noticed that xfstests generic/001 was failing against linux-next nfsd.
The client would request a
OPEN4_SHARE_ACCESS_WRITEopen, and the server would hand out a write delegation. The client would then try to use that write delegation as the source stateid in a COPY or CLONE operation, and the server would respond withNFS4ERR_STALE.The problem is that the struct file associated with the delegation does not necessarily have read permissions. It's handing out a write delegation on what is effectively an
O_WRONLYopen. RFC 8881 states:"An
OPEN_DELEGATE_WRITEdelegation allows the client to handle, on its
own, all opens."Given that the client didn't request any read permissions, and that nfsd didn't check for any, it seems wrong to give out a write delegation.
Only hand out a write delegation if we have a
O_RDWRdescriptor available. If it fails to find an appropriate write descriptor, go ahead and try for a read delegation ifNFS4_SHARE_ACCESS_READwas requested.This fixes xfstest generic/001.
Signed-off-by Jeff Layton <[email protected]>---
Changes in v2:
- Rework the logic when finding struct file for the delegation. The earlier patch might still have attached aO_WRONLYfile to the deleg in some cases, and could still have handed out a write delegation on anO_WRONLYOPEN request in some cases.--- fs/nfsd/nfs4state.c | 29 ++++++++++++++++++----------- 1 file changed, 18 insertions(+), 11 deletions(-)fs/nfsd/nfs4state.c +1
-1@@ -5449,7 +5449,7 @@ nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp,struct nfs4_file *fp = stp->st_stid.sc_file;struct nfs4_clnt_odstate *odstate = stp->st_clnt_odstate;struct nfs4_delegation *dp; -struct nfsd_file *nf; +struct nfsd_file *nf =NULL ;struct file_lock *fl;u32 dl_type;@@ -5461,21 +5461,28 @@ nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp,if (fp->fi_had_conflict)return ERR_PTR (-EAGAIN );
fs/nfsd/nfs4state.c nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp, · line 5470
Quoted 10 lines
fs/nfsd/nfs4state.c +7
-2-if (open->op_share_access &NFS4_SHARE_ACCESS_WRITE ) { - nf =find_writeable_file (fp); +/* +* Try for a write delegation first. We need an O_RDWR file +* since a write delegation allows the client to perform any open +* from its cache. +*/ +if ((open->op_share_access &NFS4_SHARE_ACCESS_BOTH ) ==NFS4_SHARE_ACCESS_BOTH ) { + nf =nfsd_file_get (fp->fi_fds[O_RDWR ]);
This doesn't take fp->fi_lock before accessing ->fi_fds[], while the find_readable_file() call below does. This inconsistency suggests a bug?
Maybe the provided API is awkward. Should we have find_suitable_file() and find_suitable_file_locked() that gets passed an nfs4_file and an O_MODE? It tries the given mode, then O_RDWR
NeilBrown
Quoted 36 lines
fs/nfsd/nfs4state.c +10
-9dl_type =NFS4_OPEN_DELEGATE_WRITE ; - }else { + } + +/* +* If the file is being opened O_RDONLY or we couldn't get a O_RDWR +* file for some reason, then try for a read deleg instead. +*/ +if (!nf && (open->op_share_access &NFS4_SHARE_ACCESS_READ )) { nf =find_readable_file (fp); dl_type =NFS4_OPEN_DELEGATE_READ ; } -if (!nf) { -/* -* We probably could attempt another open and get a read -* delegation, but for now, don't bother until the -* client actually sends us one. -*/ + +if (!nf)return ERR_PTR (-EAGAIN ); - } +spin_lock (&state_lock);spin_lock (&fp->fi_lock);if (nfs4_delegation_exists (clp, fp)) ---base-commit:
a734662572708cf062e974f659ae50c24fc1ad17
change-id: 20230731-wdeleg-bbdb6b25a3c6Best regards,
Signature · 1 line
Jeff Layton <[email protected]>
Quoted 54 lines · NeilBrown
Quoted 53 lines · Jeff Layton
I noticed that xfstests generic/001 was failing against linux-next nfsd.
The client would request a
OPEN4_SHARE_ACCESS_WRITEopen, and the server would hand out a write delegation. The client would then try to use that write delegation as the source stateid in a COPY or CLONE operation, and the server would respond withNFS4ERR_STALE.The problem is that the struct file associated with the delegation does not necessarily have read permissions. It's handing out a write delegation on what is effectively an
O_WRONLYopen. RFC 8881 states:"An
OPEN_DELEGATE_WRITEdelegation allows the client to handle, on its
own, all opens."Given that the client didn't request any read permissions, and that nfsd didn't check for any, it seems wrong to give out a write delegation.
Only hand out a write delegation if we have a
O_RDWRdescriptor available. If it fails to find an appropriate write descriptor, go ahead and try for a read delegation ifNFS4_SHARE_ACCESS_READwas requested.This fixes xfstest generic/001.
Signed-off-by Jeff Layton <[email protected]>---
Changes in v2:
- Rework the logic when finding struct file for the delegation. The earlier patch might still have attached aO_WRONLYfile to the deleg in some cases, and could still have handed out a write delegation on anO_WRONLYOPEN request in some cases.--- fs/nfsd/nfs4state.c | 29 ++++++++++++++++++----------- 1 file changed, 18 insertions(+), 11 deletions(-)fs/nfsd/nfs4state.c +1
-1@@ -5449,7 +5449,7 @@ nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp,struct nfs4_file *fp = stp->st_stid.sc_file;struct nfs4_clnt_odstate *odstate = stp->st_clnt_odstate;struct nfs4_delegation *dp; -struct nfsd_file *nf; +struct nfsd_file *nf =NULL ;struct file_lock *fl;u32 dl_type;@@ -5461,21 +5461,28 @@ nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp,if (fp->fi_had_conflict)return ERR_PTR (-EAGAIN );
Quoted 12 lines
fs/nfsd/nfs4state.c nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp, · line 5470
Quoted 9 lines
fs/nfsd/nfs4state.c +7
-2-if (open->op_share_access &NFS4_SHARE_ACCESS_WRITE ) { - nf =find_writeable_file (fp); +/* +* Try for a write delegation first. We need an O_RDWR file +* since a write delegation allows the client to perform any open +* from its cache. +*/ +if ((open->op_share_access &NFS4_SHARE_ACCESS_BOTH ) ==NFS4_SHARE_ACCESS_BOTH ) { + nf =nfsd_file_get (fp->fi_fds[O_RDWR ]);This doesn't take
fp->fi_lockbefore accessing ->fi_fds[], while thefind_readable_file()call below does.
Note that the code it replaces (find_writeable_file) takes the fi_lock, so that seems like an important omission.
I noticed this earlier, but I was anxious to test whether this fix is on the right path. So far, NFSv4.2 behavior seems much improved. And, I like the new comments.
Quoted 47 lines
This inconsistency suggests a bug?
Maybe the provided API is awkward. Should we have
find_suitable_file()andfind_suitable_file_locked()that gets passed annfs4_fileand anO_MODE? It tries the given mode, thenO_RDWRNeilBrown
Quoted 36 lines
fs/nfsd/nfs4state.c +10
-9dl_type =NFS4_OPEN_DELEGATE_WRITE ; - }else { + } + +/* +* If the file is being opened O_RDONLY or we couldn't get a O_RDWR +* file for some reason, then try for a read deleg instead. +*/ +if (!nf && (open->op_share_access &NFS4_SHARE_ACCESS_READ )) { nf =find_readable_file (fp); dl_type =NFS4_OPEN_DELEGATE_READ ; } -if (!nf) { -/* -* We probably could attempt another open and get a read -* delegation, but for now, don't bother until the -* client actually sends us one. -*/ + +if (!nf)return ERR_PTR (-EAGAIN ); - } +spin_lock (&state_lock);spin_lock (&fp->fi_lock);if (nfs4_delegation_exists (clp, fp)) ---base-commit:
a734662572708cf062e974f659ae50c24fc1ad17
change-id: 20230731-wdeleg-bbdb6b25a3c6Best regards,
Signature · 1 line
Jeff Layton <[email protected]>
Signature · 1 line
Chuck Lever
Jeff Layton
maintainerQuoted 68 lines · Chuck Lever
Quoted 54 lines · NeilBrown
Quoted 53 lines · Jeff Layton
I noticed that xfstests generic/001 was failing against linux-next nfsd.
The client would request a
OPEN4_SHARE_ACCESS_WRITEopen, and the server would hand out a write delegation. The client would then try to use that write delegation as the source stateid in a COPY or CLONE operation, and the server would respond withNFS4ERR_STALE.The problem is that the struct file associated with the delegation does not necessarily have read permissions. It's handing out a write delegation on what is effectively an
O_WRONLYopen. RFC 8881 states:"An
OPEN_DELEGATE_WRITEdelegation allows the client to handle, on its
own, all opens."Given that the client didn't request any read permissions, and that nfsd didn't check for any, it seems wrong to give out a write delegation.
Only hand out a write delegation if we have a
O_RDWRdescriptor available. If it fails to find an appropriate write descriptor, go ahead and try for a read delegation ifNFS4_SHARE_ACCESS_READwas requested.This fixes xfstest generic/001.
Signed-off-by Jeff Layton <[email protected]>---
Changes in v2:
- Rework the logic when finding struct file for the delegation. The earlier patch might still have attached aO_WRONLYfile to the deleg in some cases, and could still have handed out a write delegation on anO_WRONLYOPEN request in some cases.--- fs/nfsd/nfs4state.c | 29 ++++++++++++++++++----------- 1 file changed, 18 insertions(+), 11 deletions(-)fs/nfsd/nfs4state.c +1
-1@@ -5449,7 +5449,7 @@ nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp,struct nfs4_file *fp = stp->st_stid.sc_file;struct nfs4_clnt_odstate *odstate = stp->st_clnt_odstate;struct nfs4_delegation *dp; -struct nfsd_file *nf; +struct nfsd_file *nf =NULL ;struct file_lock *fl;u32 dl_type;@@ -5461,21 +5461,28 @@ nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp,if (fp->fi_had_conflict)return ERR_PTR (-EAGAIN );Quoted 12 lines
fs/nfsd/nfs4state.c nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp, · line 5470
Quoted 9 lines
fs/nfsd/nfs4state.c +7
-2-if (open->op_share_access &NFS4_SHARE_ACCESS_WRITE ) { - nf =find_writeable_file (fp); +/* +* Try for a write delegation first. We need an O_RDWR file +* since a write delegation allows the client to perform any open +* from its cache. +*/ +if ((open->op_share_access &NFS4_SHARE_ACCESS_BOTH ) ==NFS4_SHARE_ACCESS_BOTH ) { + nf =nfsd_file_get (fp->fi_fds[O_RDWR ]);This doesn't take
fp->fi_lockbefore accessing ->fi_fds[], while thefind_readable_file()call below does.
Quoted 3 lines
Note that the code it replaces (
find_writeable_file) takes thefi_lock, so that seems like an important omission.
Yes, you and Neil are correct. We need the lock there. I'll respin the patch, re-test and resend soon (once I sort out an issue with my test setup).
Quoted 53 lines
I noticed this earlier, but I was anxious to test whether this fix is on the right path. So far, NFSv4.2 behavior seems much improved. And, I like the new comments.
Quoted 47 lines
This inconsistency suggests a bug?
Maybe the provided API is awkward. Should we have
find_suitable_file()andfind_suitable_file_locked()that gets passed annfs4_fileand anO_MODE? It tries the given mode, thenO_RDWRNeilBrown
Quoted 36 lines
fs/nfsd/nfs4state.c +10
-9dl_type =NFS4_OPEN_DELEGATE_WRITE ; - }else { + } + +/* +* If the file is being opened O_RDONLY or we couldn't get a O_RDWR +* file for some reason, then try for a read deleg instead. +*/ +if (!nf && (open->op_share_access &NFS4_SHARE_ACCESS_READ )) { nf =find_readable_file (fp); dl_type =NFS4_OPEN_DELEGATE_READ ; } -if (!nf) { -/* -* We probably could attempt another open and get a read -* delegation, but for now, don't bother until the -* client actually sends us one. -*/ + +if (!nf)return ERR_PTR (-EAGAIN ); - } +spin_lock (&state_lock);spin_lock (&fp->fi_lock);if (nfs4_delegation_exists (clp, fp)) ---base-commit:
a734662572708cf062e974f659ae50c24fc1ad17
change-id: 20230731-wdeleg-bbdb6b25a3c6Best regards,
Signature · 1 line
Jeff Layton <[email protected]>
Signature · 1 line
Jeff Layton <[email protected]>
[email protected]
reviewerQuoted 5 lines · Jeff Layton
I noticed that xfstests generic/001 was failing against linux-next nfsd.
The client would request a
OPEN4_SHARE_ACCESS_WRITEopen, and the server would hand out a write delegation. The client would then try to use that write delegation as the source stateid in a COPY
not sure why the client opens the source file of a COPY operation with OPEN4_SHARE_ACCESS_WRITE?
Quoted 2 lines
or CLONE operation, and
the server would respond withNFS4ERR_STALE.
If the server does not allow client to use write delegation for the READ, should the correct error return be NFS4ERR_OPENMODE?
Quoted 48 lines
The problem is that the struct file associated with the delegation does not necessarily have read permissions. It's handing out a write delegation on what is effectively an
O_WRONLYopen. RFC 8881 states:"An
OPEN_DELEGATE_WRITEdelegation allows the client to handle, on its
own, all opens."Given that the client didn't request any read permissions, and that nfsd didn't check for any, it seems wrong to give out a write delegation.
Only hand out a write delegation if we have a
O_RDWRdescriptor available. If it fails to find an appropriate write descriptor, go ahead and try for a read delegation ifNFS4_SHARE_ACCESS_READwas requested.This fixes xfstest generic/001.
Signed-off-by Jeff Layton <[email protected]>---
Changes in v2:
- Rework the logic when finding struct file for the delegation. The earlier patch might still have attached aO_WRONLYfile to the deleg in some cases, and could still have handed out a write delegation on anO_WRONLYOPEN request in some cases.--- fs/nfsd/nfs4state.c | 29 ++++++++++++++++++----------- 1 file changed, 18 insertions(+), 11 deletions(-)fs/nfsd/nfs4state.c +1
-2@@ -5449,7 +5449,7 @@ nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp,struct nfs4_file *fp = stp->st_stid.sc_file;struct nfs4_clnt_odstate *odstate = stp->st_clnt_odstate;struct nfs4_delegation *dp; -struct nfsd_file *nf; +struct nfsd_file *nf =NULL ;struct file_lock *fl;u32 dl_type;@@ -5461,21 +5461,28 @@ nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp,if (fp->fi_had_conflict)return ERR_PTR (-EAGAIN ); -if (open->op_share_access &NFS4_SHARE_ACCESS_WRITE ) {
fs/nfsd/nfs4state.c nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp, · line 5467
Quoted 10 lines
fs/nfsd/nfs4state.c +7
-2- nf =find_writeable_file (fp); +/* +* Try for a write delegation first. We need an O_RDWR file +* since a write delegation allows the client to perform any open +* from its cache. +*/ +if ((open->op_share_access &NFS4_SHARE_ACCESS_BOTH ) ==NFS4_SHARE_ACCESS_BOTH ) { + nf =nfsd_file_get (fp->fi_fds[O_RDWR ]); dl_type =NFS4_OPEN_DELEGATE_WRITE ; - }else {
Does this mean OPEN4_SHARE_ACCESS_WRITE do not get a write delegation? It does not seem right.
-Dai
Quoted 30 lines
fs/nfsd/nfs4state.c +10
-8+ } + +/* +* If the file is being opened O_RDONLY or we couldn't get a O_RDWR +* file for some reason, then try for a read deleg instead. +*/ +if (!nf && (open->op_share_access &NFS4_SHARE_ACCESS_READ )) { nf =find_readable_file (fp); dl_type =NFS4_OPEN_DELEGATE_READ ; } -if (!nf) { -/* -* We probably could attempt another open and get a read -* delegation, but for now, don't bother until the -* client actually sends us one. -*/ + +if (!nf)return ERR_PTR (-EAGAIN ); - } +spin_lock (&state_lock);spin_lock (&fp->fi_lock);if (nfs4_delegation_exists (clp, fp)) ---base-commit:
a734662572708cf062e974f659ae50c24fc1ad17
change-id: 20230731-wdeleg-bbdb6b25a3c6Best regards,
Jeff Layton
maintainerQuoted 10 lines · On Wed, 2023-08-02 at 09:29 -0700, [email protected] wrote:
Quoted 5 lines · Jeff Layton
I noticed that xfstests generic/001 was failing against linux-next nfsd.
The client would request a
OPEN4_SHARE_ACCESS_WRITEopen, and the server would hand out a write delegation. The client would then try to use that write delegation as the source stateid in a COPYnot sure why the client opens the source file of a COPY operation with
OPEN4_SHARE_ACCESS_WRITE?
It doesn't. The original open is to write the data for the file being copied. It then opens the file again for READ, but since it has a write delegation, it doesn't need to talk to the server at all -- it can just use that stateid for later operations.
Quoted 6 lines
Quoted 2 lines
or CLONE operation, and
the server would respond withNFS4ERR_STALE.If the server does not allow client to use write delegation for the READ, should the correct error return be
NFS4ERR_OPENMODE?
The server must allow the client to use a write delegation for read operations. It's required by the spec, AFAIU.
The error in this case was just bogus. The vfs copy routine would return -EBADF since the file didn't have FMODE_READ, and the nfs server would translate that into NFS4ERR_STALE.
Probably there is a better v4 error code that we could translate EBADF to, but with this patch it shouldn't be a problem any longer.
Quoted 62 lines
Quoted 48 lines
The problem is that the struct file associated with the delegation does not necessarily have read permissions. It's handing out a write delegation on what is effectively an
O_WRONLYopen. RFC 8881 states:"An
OPEN_DELEGATE_WRITEdelegation allows the client to handle, on its
own, all opens."Given that the client didn't request any read permissions, and that nfsd didn't check for any, it seems wrong to give out a write delegation.
Only hand out a write delegation if we have a
O_RDWRdescriptor available. If it fails to find an appropriate write descriptor, go ahead and try for a read delegation ifNFS4_SHARE_ACCESS_READwas requested.This fixes xfstest generic/001.
Signed-off-by Jeff Layton <[email protected]>---
Changes in v2:
- Rework the logic when finding struct file for the delegation. The earlier patch might still have attached aO_WRONLYfile to the deleg in some cases, and could still have handed out a write delegation on anO_WRONLYOPEN request in some cases.--- fs/nfsd/nfs4state.c | 29 ++++++++++++++++++----------- 1 file changed, 18 insertions(+), 11 deletions(-)fs/nfsd/nfs4state.c +1
-2@@ -5449,7 +5449,7 @@ nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp,struct nfs4_file *fp = stp->st_stid.sc_file;struct nfs4_clnt_odstate *odstate = stp->st_clnt_odstate;struct nfs4_delegation *dp; -struct nfsd_file *nf; +struct nfsd_file *nf =NULL ;struct file_lock *fl;u32 dl_type;@@ -5461,21 +5461,28 @@ nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp,if (fp->fi_had_conflict)return ERR_PTR (-EAGAIN ); -if (open->op_share_access &NFS4_SHARE_ACCESS_WRITE ) {fs/nfsd/nfs4state.c nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp, · line 5467
Quoted 10 lines
fs/nfsd/nfs4state.c +7
-2- nf =find_writeable_file (fp); +/* +* Try for a write delegation first. We need an O_RDWR file +* since a write delegation allows the client to perform any open +* from its cache. +*/ +if ((open->op_share_access &NFS4_SHARE_ACCESS_BOTH ) ==NFS4_SHARE_ACCESS_BOTH ) { + nf =nfsd_file_get (fp->fi_fds[O_RDWR ]); dl_type =NFS4_OPEN_DELEGATE_WRITE ; - }else {Does this mean
OPEN4_SHARE_ACCESS_WRITEdo not get a write delegation? It does not seem right.
Quoted 2 lines
-Dai
Why? Per RFC 8881:
"An OPEN_DELEGATE_WRITE delegation allows the client to handle, on its own, all opens."
All opens. That includes read opens.
An OPEN4_SHARE_ACCESS_WRITE open will succeed on a file to which the user has no read permissions. Therefore, we can't grant a write delegation since can't guarantee that the user is allowed to do that.
Quoted 30 lines
Quoted 30 lines
fs/nfsd/nfs4state.c +10
-8+ } + +/* +* If the file is being opened O_RDONLY or we couldn't get a O_RDWR +* file for some reason, then try for a read deleg instead. +*/ +if (!nf && (open->op_share_access &NFS4_SHARE_ACCESS_READ )) { nf =find_readable_file (fp); dl_type =NFS4_OPEN_DELEGATE_READ ; } -if (!nf) { -/* -* We probably could attempt another open and get a read -* delegation, but for now, don't bother until the -* client actually sends us one. -*/ + +if (!nf)return ERR_PTR (-EAGAIN ); - } +spin_lock (&state_lock);spin_lock (&fp->fi_lock);if (nfs4_delegation_exists (clp, fp)) ---base-commit:
a734662572708cf062e974f659ae50c24fc1ad17
change-id: 20230731-wdeleg-bbdb6b25a3c6Best regards,
Signature · 1 line
Jeff Layton <[email protected]>
Quoted 85 lines · Jeff Layton
Quoted 10 lines · On Wed, 2023-08-02 at 09:29 -0700, [email protected] wrote:
Quoted 5 lines · Jeff Layton
I noticed that xfstests generic/001 was failing against linux-next nfsd.
The client would request a
OPEN4_SHARE_ACCESS_WRITEopen, and the server would hand out a write delegation. The client would then try to use that write delegation as the source stateid in a COPYnot sure why the client opens the source file of a COPY operation with
OPEN4_SHARE_ACCESS_WRITE?It doesn't. The original open is to write the data for the file being copied. It then opens the file again for READ, but since it has a write delegation, it doesn't need to talk to the server at all -- it can just use that stateid for later operations.
Quoted 6 lines
Quoted 2 lines
or CLONE operation, and
the server would respond withNFS4ERR_STALE.If the server does not allow client to use write delegation for the READ, should the correct error return be
NFS4ERR_OPENMODE?The server must allow the client to use a write delegation for read operations. It's required by the spec, AFAIU.
The error in this case was just bogus. The vfs copy routine would return
-EBADFsince the file didn't haveFMODE_READ, and the nfs server would translate that intoNFS4ERR_STALE.Probably there is a better v4 error code that we could translate EBADF to, but with this patch it shouldn't be a problem any longer.
Quoted 47 lines
Quoted 47 lines
The problem is that the struct file associated with the delegation does not necessarily have read permissions. It's handing out a write delegation on what is effectively an
O_WRONLYopen. RFC 8881 states:"An
OPEN_DELEGATE_WRITEdelegation allows the client to handle, on its
own, all opens."Given that the client didn't request any read permissions, and that nfsd didn't check for any, it seems wrong to give out a write delegation.
Only hand out a write delegation if we have a
O_RDWRdescriptor available. If it fails to find an appropriate write descriptor, go ahead and try for a read delegation ifNFS4_SHARE_ACCESS_READwas requested.This fixes xfstest generic/001.
Signed-off-by Jeff Layton <[email protected]>---
Changes in v2:
- Rework the logic when finding struct file for the delegation. The earlier patch might still have attached aO_WRONLYfile to the deleg in some cases, and could still have handed out a write delegation on anO_WRONLYOPEN request in some cases.--- fs/nfsd/nfs4state.c | 29 ++++++++++++++++++----------- 1 file changed, 18 insertions(+), 11 deletions(-)fs/nfsd/nfs4state.c +1
-1@@ -5449,7 +5449,7 @@ nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp,struct nfs4_file *fp = stp->st_stid.sc_file;struct nfs4_clnt_odstate *odstate = stp->st_clnt_odstate;struct nfs4_delegation *dp; -struct nfsd_file *nf; +struct nfsd_file *nf =NULL ;struct file_lock *fl;u32 dl_type;@@ -5461,21 +5461,28 @@ nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp,if (fp->fi_had_conflict)return ERR_PTR (-EAGAIN );
Quoted 6 lines
Quoted 6 lines
fs/nfsd/nfs4state.c nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp, · line 5467
Quoted 6 lines
fs/nfsd/nfs4state.c +4
-2-if (open->op_share_access &NFS4_SHARE_ACCESS_WRITE ) { - nf =find_writeable_file (fp); +/* +* Try for a write delegation first. We need an O_RDWR file +* since a write delegation allows the client to perform any open +* from its cache.
Since you're sending a v3... can you cite the section of RFC 8881 that specifies this "all open" case in this commment? I'm sure we're all going to forget this requirement.
Quoted 49 lines
Quoted 11 lines
fs/nfsd/nfs4state.c nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp, · line 5467
Quoted 5 lines
fs/nfsd/nfs4state.c +3
-1+*/ +if ((open->op_share_access &NFS4_SHARE_ACCESS_BOTH ) ==NFS4_SHARE_ACCESS_BOTH ) { + nf =nfsd_file_get (fp->fi_fds[O_RDWR ]); dl_type =NFS4_OPEN_DELEGATE_WRITE ; - }else {Does this mean
OPEN4_SHARE_ACCESS_WRITEdo not get a write delegation? It does not seem right.-Dai
Why? Per RFC 8881:
"An
OPEN_DELEGATE_WRITEdelegation allows the client to handle, on its own, all opens."All opens. That includes read opens.
An
OPEN4_SHARE_ACCESS_WRITEopen will succeed on a file to which the user has no read permissions. Therefore, we can't grant a write delegation since can't guarantee that the user is allowed to do that.Quoted 25 lines
Quoted 25 lines
fs/nfsd/nfs4state.c +10
-7+ } + +/* +* If the file is being opened O_RDONLY or we couldn't get a O_RDWR +* file for some reason, then try for a read deleg instead. +*/ +if (!nf && (open->op_share_access &NFS4_SHARE_ACCESS_READ )) { nf =find_readable_file (fp); dl_type =NFS4_OPEN_DELEGATE_READ ; } -if (!nf) { -/* -* We probably could attempt another open and get a read -* delegation, but for now, don't bother until the -* client actually sends us one. -*/ + +if (!nf)return ERR_PTR (-EAGAIN ); - } +spin_lock (&state_lock);spin_lock (&fp->fi_lock);if (nfs4_delegation_exists (clp, fp))
Quoted 8 lines
Quoted 5 lines
Quoted 5 lines
---
base-commit:a734662572708cf062e974f659ae50c24fc1ad17
change-id: 20230731-wdeleg-bbdb6b25a3c6Best regards,
Signature · 1 line
Jeff Layton <[email protected]>
--
Chuck Lever
[email protected]
reviewerQuoted 94 lines · Jeff Layton
Quoted 9 lines · On Wed, 2023-08-02 at 09:29 -0700, [email protected] wrote:
Quoted 5 lines · Jeff Layton
I noticed that xfstests generic/001 was failing against linux-next nfsd.
The client would request a
OPEN4_SHARE_ACCESS_WRITEopen, and the server would hand out a write delegation. The client would then try to use that write delegation as the source stateid in a COPYnot sure why the client opens the source file of a COPY operation with
OPEN4_SHARE_ACCESS_WRITE?It doesn't. The original open is to write the data for the file being copied. It then opens the file again for READ, but since it has a write delegation, it doesn't need to talk to the server at all -- it can just use that stateid for later operations.
Quoted 5 lines
Quoted 2 lines
or CLONE operation, and
the server would respond withNFS4ERR_STALE.If the server does not allow client to use write delegation for the READ, should the correct error return be
NFS4ERR_OPENMODE?The server must allow the client to use a write delegation for read operations. It's required by the spec, AFAIU.
The error in this case was just bogus. The vfs copy routine would return
-EBADFsince the file didn't haveFMODE_READ, and the nfs server would translate that intoNFS4ERR_STALE.Probably there is a better v4 error code that we could translate EBADF to, but with this patch it shouldn't be a problem any longer.
Quoted 60 lines
Quoted 47 lines
The problem is that the struct file associated with the delegation does not necessarily have read permissions. It's handing out a write delegation on what is effectively an
O_WRONLYopen. RFC 8881 states:"An
OPEN_DELEGATE_WRITEdelegation allows the client to handle, on its
own, all opens."Given that the client didn't request any read permissions, and that nfsd didn't check for any, it seems wrong to give out a write delegation.
Only hand out a write delegation if we have a
O_RDWRdescriptor available. If it fails to find an appropriate write descriptor, go ahead and try for a read delegation ifNFS4_SHARE_ACCESS_READwas requested.This fixes xfstest generic/001.
Signed-off-by Jeff Layton <[email protected]>---
Changes in v2:
- Rework the logic when finding struct file for the delegation. The earlier patch might still have attached aO_WRONLYfile to the deleg in some cases, and could still have handed out a write delegation on anO_WRONLYOPEN request in some cases.--- fs/nfsd/nfs4state.c | 29 ++++++++++++++++++----------- 1 file changed, 18 insertions(+), 11 deletions(-)fs/nfsd/nfs4state.c +1
-2@@ -5449,7 +5449,7 @@ nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp,struct nfs4_file *fp = stp->st_stid.sc_file;struct nfs4_clnt_odstate *odstate = stp->st_clnt_odstate;struct nfs4_delegation *dp; -struct nfsd_file *nf; +struct nfsd_file *nf =NULL ;struct file_lock *fl;u32 dl_type;@@ -5461,21 +5461,28 @@ nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp,if (fp->fi_had_conflict)return ERR_PTR (-EAGAIN ); -if (open->op_share_access &NFS4_SHARE_ACCESS_WRITE ) {fs/nfsd/nfs4state.c nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp, · line 5467
Quoted 10 lines
fs/nfsd/nfs4state.c +7
-2- nf =find_writeable_file (fp); +/* +* Try for a write delegation first. We need an O_RDWR file +* since a write delegation allows the client to perform any open +* from its cache. +*/ +if ((open->op_share_access &NFS4_SHARE_ACCESS_BOTH ) ==NFS4_SHARE_ACCESS_BOTH ) { + nf =nfsd_file_get (fp->fi_fds[O_RDWR ]); dl_type =NFS4_OPEN_DELEGATE_WRITE ; - }else {Does this mean
OPEN4_SHARE_ACCESS_WRITEdo not get a write delegation? It does not seem right.Quoted 2 lines
-Dai
Why? Per RFC 8881:
Quoted 8 lines
"An
OPEN_DELEGATE_WRITEdelegation allows the client to handle, on its own, all opens."All opens. That includes read opens.
An
OPEN4_SHARE_ACCESS_WRITEopen will succeed on a file to which the user has no read permissions. Therefore, we can't grant a write delegation since can't guarantee that the user is allowed to do that.
If the server grants the write delegation on an OPEN with OPEN4_SHARE_ACCESS_WRITE on the file with WR-only access mode then why can't the server checks and denies the subsequent READ?
Per RFC 8881, section 9.1.2:
For delegation stateids, the access mode is based on the type of delegation.
When a READ, WRITE, or SETATTR (that specifies the size attribute) operation is done, the operation is subject to checking against the access mode to verify that the operation is appropriate given the stateid with which the operation is associated.
In the case of WRITE-type operations (i.e., WRITEs and SETATTRs that set size), the server MUST verify that the access mode allows writing and MUST return an NFS4ERR_OPENMODE error if it does not. In the case of READ, the server may perform the corresponding check on the access mode, or it may choose to allow READ on OPENs for OPEN4_SHARE_ACCESS_WRITE, to accommodate clients whose WRITE implementation may unavoidably do reads (e.g., due to buffer cache constraints). However, even if READs are allowed in these circumstances, the server MUST still check for locks that conflict with the READ (e.g., another OPEN specified OPEN4_SHARE_DENY_READ or OPEN4_SHARE_DENY_BOTH). Note that a server that does enforce the access mode check on READs need not explicitly check for conflicting share reservations since the existence of OPEN for OPEN4_SHARE_ACCESS_READ guarantees that no conflicting share reservation can exist.
FWIW, The Solaris server grants write delegation on OPEN with OPEN4_SHARE_ACCESS_WRITE on file with access mode either RW or WR-only. Maybe this is a bug? or the spec is not clear?
It'd would be interesting to know how ONTAP server behaves in this scenario.
-Dai
Quoted 32 lines
Quoted 32 lines
Quoted 32 lines
fs/nfsd/nfs4state.c +10
-8+ } + +/* +* If the file is being opened O_RDONLY or we couldn't get a O_RDWR +* file for some reason, then try for a read deleg instead. +*/ +if (!nf && (open->op_share_access &NFS4_SHARE_ACCESS_READ )) { nf =find_readable_file (fp); dl_type =NFS4_OPEN_DELEGATE_READ ; } -if (!nf) { -/* -* We probably could attempt another open and get a read -* delegation, but for now, don't bother until the -* client actually sends us one. -*/ + +if (!nf)return ERR_PTR (-EAGAIN ); - } +spin_lock (&state_lock);spin_lock (&fp->fi_lock);if (nfs4_delegation_exists (clp, fp)) ---base-commit:
a734662572708cf062e974f659ae50c24fc1ad17
change-id: 20230731-wdeleg-bbdb6b25a3c6Best regards,
Jeff Layton
maintainerQuoted 133 lines · On Wed, 2023-08-02 at 13:15 -0700, [email protected] wrote:
Quoted 94 lines · Jeff Layton
Quoted 9 lines · On Wed, 2023-08-02 at 09:29 -0700, [email protected] wrote:
Quoted 5 lines · Jeff Layton
I noticed that xfstests generic/001 was failing against linux-next nfsd.
The client would request a
OPEN4_SHARE_ACCESS_WRITEopen, and the server would hand out a write delegation. The client would then try to use that write delegation as the source stateid in a COPYnot sure why the client opens the source file of a COPY operation with
OPEN4_SHARE_ACCESS_WRITE?It doesn't. The original open is to write the data for the file being copied. It then opens the file again for READ, but since it has a write delegation, it doesn't need to talk to the server at all -- it can just use that stateid for later operations.
Quoted 5 lines
Quoted 2 lines
or CLONE operation, and
the server would respond withNFS4ERR_STALE.If the server does not allow client to use write delegation for the READ, should the correct error return be
NFS4ERR_OPENMODE?The server must allow the client to use a write delegation for read operations. It's required by the spec, AFAIU.
The error in this case was just bogus. The vfs copy routine would return
-EBADFsince the file didn't haveFMODE_READ, and the nfs server would translate that intoNFS4ERR_STALE.Probably there is a better v4 error code that we could translate EBADF to, but with this patch it shouldn't be a problem any longer.
Quoted 60 lines
Quoted 47 lines
The problem is that the struct file associated with the delegation does not necessarily have read permissions. It's handing out a write delegation on what is effectively an
O_WRONLYopen. RFC 8881 states:"An
OPEN_DELEGATE_WRITEdelegation allows the client to handle, on its
own, all opens."Given that the client didn't request any read permissions, and that nfsd didn't check for any, it seems wrong to give out a write delegation.
Only hand out a write delegation if we have a
O_RDWRdescriptor available. If it fails to find an appropriate write descriptor, go ahead and try for a read delegation ifNFS4_SHARE_ACCESS_READwas requested.This fixes xfstest generic/001.
Signed-off-by Jeff Layton <[email protected]>---
Changes in v2:
- Rework the logic when finding struct file for the delegation. The earlier patch might still have attached aO_WRONLYfile to the deleg in some cases, and could still have handed out a write delegation on anO_WRONLYOPEN request in some cases.--- fs/nfsd/nfs4state.c | 29 ++++++++++++++++++----------- 1 file changed, 18 insertions(+), 11 deletions(-)fs/nfsd/nfs4state.c +1
-2@@ -5449,7 +5449,7 @@ nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp,struct nfs4_file *fp = stp->st_stid.sc_file;struct nfs4_clnt_odstate *odstate = stp->st_clnt_odstate;struct nfs4_delegation *dp; -struct nfsd_file *nf; +struct nfsd_file *nf =NULL ;struct file_lock *fl;u32 dl_type;@@ -5461,21 +5461,28 @@ nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp,if (fp->fi_had_conflict)return ERR_PTR (-EAGAIN ); -if (open->op_share_access &NFS4_SHARE_ACCESS_WRITE ) {fs/nfsd/nfs4state.c nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp, · line 5467
Quoted 10 lines
fs/nfsd/nfs4state.c +7
-2- nf =find_writeable_file (fp); +/* +* Try for a write delegation first. We need an O_RDWR file +* since a write delegation allows the client to perform any open +* from its cache. +*/ +if ((open->op_share_access &NFS4_SHARE_ACCESS_BOTH ) ==NFS4_SHARE_ACCESS_BOTH ) { + nf =nfsd_file_get (fp->fi_fds[O_RDWR ]); dl_type =NFS4_OPEN_DELEGATE_WRITE ; - }else {Does this mean
OPEN4_SHARE_ACCESS_WRITEdo not get a write delegation? It does not seem right.Quoted 2 lines
-Dai
Why? Per RFC 8881:
Quoted 8 lines
"An
OPEN_DELEGATE_WRITEdelegation allows the client to handle, on its own, all opens."All opens. That includes read opens.
An
OPEN4_SHARE_ACCESS_WRITEopen will succeed on a file to which the user has no read permissions. Therefore, we can't grant a write delegation since can't guarantee that the user is allowed to do that.If the server grants the write delegation on an OPEN with
OPEN4_SHARE_ACCESS_WRITEon the file with WR-only access mode then why can't the server checks and denies the subsequent READ?Per RFC 8881, section 9.1.2:
For delegation stateids, the access mode is based on the type of delegation.
When a READ, WRITE, or SETATTR (that specifies the size attribute) operation is done, the operation is subject to checking against the access mode to verify that the operation is appropriate given the stateid with which the operation is associated.
In the case of WRITE-type operations (i.e., WRITEs and SETATTRs that set size), the server MUST verify that the access mode allows writing and MUST return an
NFS4ERR_OPENMODEerror if it does not. In the case of READ, the server may perform the corresponding check on the access mode, or it may choose to allow READ on OPENs forOPEN4_SHARE_ACCESS_WRITE, to accommodate clients whose WRITE implementation may unavoidably do reads (e.g., due to buffer cache constraints). However, even if READs are allowed in these circumstances, the server MUST still check for locks that conflict with the READ (e.g., another OPEN specifiedOPEN4_SHARE_DENY_READorOPEN4_SHARE_DENY_BOTH). Note that a server that does enforce the access mode check on READs need not explicitly check for conflicting share reservations since the existence of OPEN forOPEN4_SHARE_ACCESS_READguarantees that no conflicting share reservation can exist.
Quoted 4 lines
FWIW, The Solaris server grants write delegation on OPEN with
OPEN4_SHARE_ACCESS_WRITEon file with access mode either RW or WR-only. Maybe this is a bug? or the spec is not clear?
I don't think that's necessarily a bug.
It's not that the spec demands that we only hand out delegations on BOTH opens. This is more of a quirk of the Linux implementation. Linux' write delegations require an open O_RDWR file descriptor because we may be called upon to do a read on its behalf.
Technically, we could probably just have it check for OPEN4_SHARE_ACCESS_WRITE, but in the case where READ isn't also set, then you're unlikely to get a delegation. Either the O_RDWR descriptor will be NULL, or there are other, conflicting opens already present.
Solaris may have a completely different design that doesn't require this. I haven't looked at its code to know.
Quoted 3 lines
It'd would be interesting to know how ONTAP server behaves in this scenario.
Indeed. Most likely it behaves more like Solaris does, but it'd nice to know.
Quoted 33 lines
Quoted 33 lines
Quoted 33 lines
Quoted 33 lines
fs/nfsd/nfs4state.c +10
-8+ } + +/* +* If the file is being opened O_RDONLY or we couldn't get a O_RDWR +* file for some reason, then try for a read deleg instead. +*/ +if (!nf && (open->op_share_access &NFS4_SHARE_ACCESS_READ )) { nf =find_readable_file (fp); dl_type =NFS4_OPEN_DELEGATE_READ ; } -if (!nf) { -/* -* We probably could attempt another open and get a read -* delegation, but for now, don't bother until the -* client actually sends us one. -*/ + +if (!nf)return ERR_PTR (-EAGAIN ); - } +spin_lock (&state_lock);spin_lock (&fp->fi_lock);if (nfs4_delegation_exists (clp, fp)) ---base-commit:
a734662572708cf062e974f659ae50c24fc1ad17
change-id: 20230731-wdeleg-bbdb6b25a3c6Best regards,
Signature · 1 line
Jeff Layton <[email protected]>
Quoted 150 lines · Jeff Layton
Quoted 133 lines · On Wed, 2023-08-02 at 13:15 -0700, [email protected] wrote:
Quoted 94 lines · Jeff Layton
Quoted 9 lines · On Wed, 2023-08-02 at 09:29 -0700, [email protected] wrote:
Quoted 5 lines · Jeff Layton
I noticed that xfstests generic/001 was failing against linux-next nfsd.
The client would request a
OPEN4_SHARE_ACCESS_WRITEopen, and the server would hand out a write delegation. The client would then try to use that write delegation as the source stateid in a COPYnot sure why the client opens the source file of a COPY operation with
OPEN4_SHARE_ACCESS_WRITE?It doesn't. The original open is to write the data for the file being copied. It then opens the file again for READ, but since it has a write delegation, it doesn't need to talk to the server at all -- it can just use that stateid for later operations.
Quoted 5 lines
Quoted 2 lines
or CLONE operation, and
the server would respond withNFS4ERR_STALE.If the server does not allow client to use write delegation for the READ, should the correct error return be
NFS4ERR_OPENMODE?The server must allow the client to use a write delegation for read operations. It's required by the spec, AFAIU.
The error in this case was just bogus. The vfs copy routine would return
-EBADFsince the file didn't haveFMODE_READ, and the nfs server would translate that intoNFS4ERR_STALE.Probably there is a better v4 error code that we could translate EBADF to, but with this patch it shouldn't be a problem any longer.
Quoted 60 lines
Quoted 47 lines
The problem is that the struct file associated with the delegation does not necessarily have read permissions. It's handing out a write delegation on what is effectively an
O_WRONLYopen. RFC 8881 states:"An
OPEN_DELEGATE_WRITEdelegation allows the client to handle, on its
own, all opens."Given that the client didn't request any read permissions, and that nfsd didn't check for any, it seems wrong to give out a write delegation.
Only hand out a write delegation if we have a
O_RDWRdescriptor available. If it fails to find an appropriate write descriptor, go ahead and try for a read delegation ifNFS4_SHARE_ACCESS_READwas requested.This fixes xfstest generic/001.
Signed-off-by Jeff Layton <[email protected]>---
Changes in v2:
- Rework the logic when finding struct file for the delegation. The earlier patch might still have attached aO_WRONLYfile to the deleg in some cases, and could still have handed out a write delegation on anO_WRONLYOPEN request in some cases.--- fs/nfsd/nfs4state.c | 29 ++++++++++++++++++----------- 1 file changed, 18 insertions(+), 11 deletions(-)fs/nfsd/nfs4state.c +1
-2@@ -5449,7 +5449,7 @@ nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp,struct nfs4_file *fp = stp->st_stid.sc_file;struct nfs4_clnt_odstate *odstate = stp->st_clnt_odstate;struct nfs4_delegation *dp; -struct nfsd_file *nf; +struct nfsd_file *nf =NULL ;struct file_lock *fl;u32 dl_type;@@ -5461,21 +5461,28 @@ nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp,if (fp->fi_had_conflict)return ERR_PTR (-EAGAIN ); -if (open->op_share_access &NFS4_SHARE_ACCESS_WRITE ) {fs/nfsd/nfs4state.c nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp, · line 5467
Quoted 10 lines
fs/nfsd/nfs4state.c +7
-2- nf =find_writeable_file (fp); +/* +* Try for a write delegation first. We need an O_RDWR file +* since a write delegation allows the client to perform any open +* from its cache. +*/ +if ((open->op_share_access &NFS4_SHARE_ACCESS_BOTH ) ==NFS4_SHARE_ACCESS_BOTH ) { + nf =nfsd_file_get (fp->fi_fds[O_RDWR ]); dl_type =NFS4_OPEN_DELEGATE_WRITE ; - }else {Does this mean
OPEN4_SHARE_ACCESS_WRITEdo not get a write delegation? It does not seem right.Quoted 2 lines
-Dai
Why? Per RFC 8881:
Quoted 8 lines
"An
OPEN_DELEGATE_WRITEdelegation allows the client to handle, on its own, all opens."All opens. That includes read opens.
An
OPEN4_SHARE_ACCESS_WRITEopen will succeed on a file to which the user has no read permissions. Therefore, we can't grant a write delegation since can't guarantee that the user is allowed to do that.If the server grants the write delegation on an OPEN with
OPEN4_SHARE_ACCESS_WRITEon the file with WR-only access mode then why can't the server checks and denies the subsequent READ?Per RFC 8881, section 9.1.2:
For delegation stateids, the access mode is based on the type of delegation.
When a READ, WRITE, or SETATTR (that specifies the size attribute) operation is done, the operation is subject to checking against the access mode to verify that the operation is appropriate given the stateid with which the operation is associated.
In the case of WRITE-type operations (i.e., WRITEs and SETATTRs that set size), the server MUST verify that the access mode allows writing and MUST return an
NFS4ERR_OPENMODEerror if it does not. In the case of READ, the server may perform the corresponding check on the access mode, or it may choose to allow READ on OPENs forOPEN4_SHARE_ACCESS_WRITE, to accommodate clients whose WRITE implementation may unavoidably do reads (e.g., due to buffer cache constraints). However, even if READs are allowed in these circumstances, the server MUST still check for locks that conflict with the READ (e.g., another OPEN specifiedOPEN4_SHARE_DENY_READorOPEN4_SHARE_DENY_BOTH). Note that a server that does enforce the access mode check on READs need not explicitly check for conflicting share reservations since the existence of OPEN forOPEN4_SHARE_ACCESS_READguarantees that no conflicting share reservation can exist.Quoted 4 lines
FWIW, The Solaris server grants write delegation on OPEN with
OPEN4_SHARE_ACCESS_WRITEon file with access mode either RW or WR-only. Maybe this is a bug? or the spec is not clear?I don't think that's necessarily a bug.
It's not that the spec demands that we only hand out delegations on BOTH opens. This is more of a quirk of the Linux implementation. Linux' write delegations require an open
O_RDWRfile descriptor because we may be called upon to do a read on its behalf.
Quoted 7 lines
Technically, we could probably just have it check for
OPEN4_SHARE_ACCESS_WRITE, but in the case where READ isn't also set, then you're unlikely to get a delegation. Either theO_RDWRdescriptor will be NULL, or there are other, conflicting opens already present.Solaris may have a completely different design that doesn't require this. I haven't looked at its code to know.
I'm comfortable for now with not handing out write delegations for SHARE_ACCESS_WRITE opens. I prefer that to permission checking on every READ operation.
If we find that it's a significant performance issue, we can revisit.
Quoted 35 lines
Quoted 3 lines
It'd would be interesting to know how ONTAP server behaves in this scenario.
Indeed. Most likely it behaves more like Solaris does, but it'd nice to know.
Quoted 28 lines
Quoted 28 lines
Quoted 28 lines
Quoted 28 lines
fs/nfsd/nfs4state.c +10
-7+ } + +/* +* If the file is being opened O_RDONLY or we couldn't get a O_RDWR +* file for some reason, then try for a read deleg instead. +*/ +if (!nf && (open->op_share_access &NFS4_SHARE_ACCESS_READ )) { nf =find_readable_file (fp); dl_type =NFS4_OPEN_DELEGATE_READ ; } -if (!nf) { -/* -* We probably could attempt another open and get a read -* delegation, but for now, don't bother until the -* client actually sends us one. -*/ + +if (!nf)return ERR_PTR (-EAGAIN ); - } +spin_lock (&state_lock);spin_lock (&fp->fi_lock);if (nfs4_delegation_exists (clp, fp))
Quoted 8 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
---
base-commit:a734662572708cf062e974f659ae50c24fc1ad17
change-id: 20230731-wdeleg-bbdb6b25a3c6Best regards,
Signature · 1 line
Jeff Layton <[email protected]>
--
Chuck Lever
Jeff Layton
maintainerQuoted 154 lines · Chuck Lever III
Quoted 154 lines · Jeff Layton
Quoted 133 lines · On Wed, 2023-08-02 at 13:15 -0700, [email protected] wrote:
Quoted 94 lines · Jeff Layton
Quoted 9 lines · On Wed, 2023-08-02 at 09:29 -0700, [email protected] wrote:
Quoted 5 lines · Jeff Layton
I noticed that xfstests generic/001 was failing against linux-next nfsd.
The client would request a
OPEN4_SHARE_ACCESS_WRITEopen, and the server would hand out a write delegation. The client would then try to use that write delegation as the source stateid in a COPYnot sure why the client opens the source file of a COPY operation with
OPEN4_SHARE_ACCESS_WRITE?It doesn't. The original open is to write the data for the file being copied. It then opens the file again for READ, but since it has a write delegation, it doesn't need to talk to the server at all -- it can just use that stateid for later operations.
Quoted 5 lines
Quoted 2 lines
or CLONE operation, and
the server would respond withNFS4ERR_STALE.If the server does not allow client to use write delegation for the READ, should the correct error return be
NFS4ERR_OPENMODE?The server must allow the client to use a write delegation for read operations. It's required by the spec, AFAIU.
The error in this case was just bogus. The vfs copy routine would return
-EBADFsince the file didn't haveFMODE_READ, and the nfs server would translate that intoNFS4ERR_STALE.Probably there is a better v4 error code that we could translate EBADF to, but with this patch it shouldn't be a problem any longer.
Quoted 60 lines
Quoted 47 lines
The problem is that the struct file associated with the delegation does not necessarily have read permissions. It's handing out a write delegation on what is effectively an
O_WRONLYopen. RFC 8881 states:"An
OPEN_DELEGATE_WRITEdelegation allows the client to handle, on its
own, all opens."Given that the client didn't request any read permissions, and that nfsd didn't check for any, it seems wrong to give out a write delegation.
Only hand out a write delegation if we have a
O_RDWRdescriptor available. If it fails to find an appropriate write descriptor, go ahead and try for a read delegation ifNFS4_SHARE_ACCESS_READwas requested.This fixes xfstest generic/001.
Signed-off-by Jeff Layton <[email protected]>---
Changes in v2:
- Rework the logic when finding struct file for the delegation. The earlier patch might still have attached aO_WRONLYfile to the deleg in some cases, and could still have handed out a write delegation on anO_WRONLYOPEN request in some cases.--- fs/nfsd/nfs4state.c | 29 ++++++++++++++++++----------- 1 file changed, 18 insertions(+), 11 deletions(-)fs/nfsd/nfs4state.c +1
-2@@ -5449,7 +5449,7 @@ nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp,struct nfs4_file *fp = stp->st_stid.sc_file;struct nfs4_clnt_odstate *odstate = stp->st_clnt_odstate;struct nfs4_delegation *dp; -struct nfsd_file *nf; +struct nfsd_file *nf =NULL ;struct file_lock *fl;u32 dl_type;@@ -5461,21 +5461,28 @@ nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp,if (fp->fi_had_conflict)return ERR_PTR (-EAGAIN ); -if (open->op_share_access &NFS4_SHARE_ACCESS_WRITE ) {fs/nfsd/nfs4state.c nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp, · line 5467
Quoted 10 lines
fs/nfsd/nfs4state.c +7
-2- nf =find_writeable_file (fp); +/* +* Try for a write delegation first. We need an O_RDWR file +* since a write delegation allows the client to perform any open +* from its cache. +*/ +if ((open->op_share_access &NFS4_SHARE_ACCESS_BOTH ) ==NFS4_SHARE_ACCESS_BOTH ) { + nf =nfsd_file_get (fp->fi_fds[O_RDWR ]); dl_type =NFS4_OPEN_DELEGATE_WRITE ; - }else {Does this mean
OPEN4_SHARE_ACCESS_WRITEdo not get a write delegation? It does not seem right.Quoted 2 lines
-Dai
Why? Per RFC 8881:
Quoted 8 lines
"An
OPEN_DELEGATE_WRITEdelegation allows the client to handle, on its own, all opens."All opens. That includes read opens.
An
OPEN4_SHARE_ACCESS_WRITEopen will succeed on a file to which the user has no read permissions. Therefore, we can't grant a write delegation since can't guarantee that the user is allowed to do that.If the server grants the write delegation on an OPEN with
OPEN4_SHARE_ACCESS_WRITEon the file with WR-only access mode then why can't the server checks and denies the subsequent READ?Per RFC 8881, section 9.1.2:
For delegation stateids, the access mode is based on the type of delegation.
When a READ, WRITE, or SETATTR (that specifies the size attribute) operation is done, the operation is subject to checking against the access mode to verify that the operation is appropriate given the stateid with which the operation is associated.
In the case of WRITE-type operations (i.e., WRITEs and SETATTRs that set size), the server MUST verify that the access mode allows writing and MUST return an
NFS4ERR_OPENMODEerror if it does not. In the case of READ, the server may perform the corresponding check on the access mode, or it may choose to allow READ on OPENs forOPEN4_SHARE_ACCESS_WRITE, to accommodate clients whose WRITE implementation may unavoidably do reads (e.g., due to buffer cache constraints). However, even if READs are allowed in these circumstances, the server MUST still check for locks that conflict with the READ (e.g., another OPEN specifiedOPEN4_SHARE_DENY_READorOPEN4_SHARE_DENY_BOTH). Note that a server that does enforce the access mode check on READs need not explicitly check for conflicting share reservations since the existence of OPEN forOPEN4_SHARE_ACCESS_READguarantees that no conflicting share reservation can exist.Quoted 4 lines
FWIW, The Solaris server grants write delegation on OPEN with
OPEN4_SHARE_ACCESS_WRITEon file with access mode either RW or WR-only. Maybe this is a bug? or the spec is not clear?I don't think that's necessarily a bug.
It's not that the spec demands that we only hand out delegations on BOTH opens. This is more of a quirk of the Linux implementation. Linux' write delegations require an open
O_RDWRfile descriptor because we may be called upon to do a read on its behalf.Technically, we could probably just have it check for
OPEN4_SHARE_ACCESS_WRITE, but in the case where READ isn't also set, then you're unlikely to get a delegation. Either theO_RDWRdescriptor will be NULL, or there are other, conflicting opens already present.
Quoted 10 lines
Quoted 2 lines
Solaris may have a completely different design that doesn't require this. I haven't looked at its code to know.
I'm comfortable for now with not handing out write delegations for
SHARE_ACCESS_WRITEopens. I prefer that to permission checking on every READ operation.If we find that it's a significant performance issue, we can revisit.
Yeah. The thing to remember here is that delegations are optional. We can always say no.
One thing we could consider to allow this is trying to open O_RDWR first, and then only fall back to doing an O_WRONLY open if that fails. I'm not sure how that would work out in practice though.
One thing that'd be interesting to know with Solaris (and maybe OnTap) is whether they will give you a write delegation for an O_WRONLY open when you don't have any read permissions on the file.
If they do, then is the client expected to do permissions enforcement for the cached open and reject local openers for read? I guess I ought to be looking at the Linux client code for this...
Quoted 48 lines
Quoted 35 lines
Quoted 3 lines
It'd would be interesting to know how ONTAP server behaves in this scenario.
Indeed. Most likely it behaves more like Solaris does, but it'd nice to know.
Quoted 28 lines
Quoted 28 lines
Quoted 28 lines
Quoted 28 lines
fs/nfsd/nfs4state.c +10
-7+ } + +/* +* If the file is being opened O_RDONLY or we couldn't get a O_RDWR +* file for some reason, then try for a read deleg instead. +*/ +if (!nf && (open->op_share_access &NFS4_SHARE_ACCESS_READ )) { nf =find_readable_file (fp); dl_type =NFS4_OPEN_DELEGATE_READ ; } -if (!nf) { -/* -* We probably could attempt another open and get a read -* delegation, but for now, don't bother until the -* client actually sends us one. -*/ + +if (!nf)return ERR_PTR (-EAGAIN ); - } +spin_lock (&state_lock);spin_lock (&fp->fi_lock);if (nfs4_delegation_exists (clp, fp))Quoted 8 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
---
base-commit:a734662572708cf062e974f659ae50c24fc1ad17
change-id: 20230731-wdeleg-bbdb6b25a3c6Best regards,
Signature · 1 line
Jeff Layton <[email protected]>--
Chuck Lever
Signature · 1 line
Jeff Layton <[email protected]>
[email protected]
reviewerQuoted 164 lines · Jeff Layton
Quoted 151 lines · Chuck Lever III
Quoted 151 lines · Jeff Layton
Quoted 132 lines · On Wed, 2023-08-02 at 13:15 -0700, [email protected] wrote:
Quoted 94 lines · Jeff Layton
Quoted 9 lines · On Wed, 2023-08-02 at 09:29 -0700, [email protected] wrote:
Quoted 5 lines · Jeff Layton
I noticed that xfstests generic/001 was failing against linux-next nfsd.
The client would request a
OPEN4_SHARE_ACCESS_WRITEopen, and the server would hand out a write delegation. The client would then try to use that write delegation as the source stateid in a COPYnot sure why the client opens the source file of a COPY operation with
OPEN4_SHARE_ACCESS_WRITE?It doesn't. The original open is to write the data for the file being copied. It then opens the file again for READ, but since it has a write delegation, it doesn't need to talk to the server at all -- it can just use that stateid for later operations.
Quoted 5 lines
Quoted 2 lines
or CLONE operation, and
the server would respond withNFS4ERR_STALE.If the server does not allow client to use write delegation for the READ, should the correct error return be
NFS4ERR_OPENMODE?The server must allow the client to use a write delegation for read operations. It's required by the spec, AFAIU.
The error in this case was just bogus. The vfs copy routine would return
-EBADFsince the file didn't haveFMODE_READ, and the nfs server would translate that intoNFS4ERR_STALE.Probably there is a better v4 error code that we could translate EBADF to, but with this patch it shouldn't be a problem any longer.
Quoted 60 lines
Quoted 47 lines
The problem is that the struct file associated with the delegation does not necessarily have read permissions. It's handing out a write delegation on what is effectively an
O_WRONLYopen. RFC 8881 states:"An
OPEN_DELEGATE_WRITEdelegation allows the client to handle, on its
own, all opens."Given that the client didn't request any read permissions, and that nfsd didn't check for any, it seems wrong to give out a write delegation.
Only hand out a write delegation if we have a
O_RDWRdescriptor available. If it fails to find an appropriate write descriptor, go ahead and try for a read delegation ifNFS4_SHARE_ACCESS_READwas requested.This fixes xfstest generic/001.
Signed-off-by Jeff Layton <[email protected]>---
Changes in v2:
- Rework the logic when finding struct file for the delegation. The earlier patch might still have attached aO_WRONLYfile to the deleg in some cases, and could still have handed out a write delegation on anO_WRONLYOPEN request in some cases.--- fs/nfsd/nfs4state.c | 29 ++++++++++++++++++----------- 1 file changed, 18 insertions(+), 11 deletions(-)fs/nfsd/nfs4state.c +1
-2@@ -5449,7 +5449,7 @@ nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp,struct nfs4_file *fp = stp->st_stid.sc_file;struct nfs4_clnt_odstate *odstate = stp->st_clnt_odstate;struct nfs4_delegation *dp; -struct nfsd_file *nf; +struct nfsd_file *nf =NULL ;struct file_lock *fl;u32 dl_type;@@ -5461,21 +5461,28 @@ nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp,if (fp->fi_had_conflict)return ERR_PTR (-EAGAIN ); -if (open->op_share_access &NFS4_SHARE_ACCESS_WRITE ) {fs/nfsd/nfs4state.c nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp, · line 5467
Quoted 10 lines
fs/nfsd/nfs4state.c +7
-2- nf =find_writeable_file (fp); +/* +* Try for a write delegation first. We need an O_RDWR file +* since a write delegation allows the client to perform any open +* from its cache. +*/ +if ((open->op_share_access &NFS4_SHARE_ACCESS_BOTH ) ==NFS4_SHARE_ACCESS_BOTH ) { + nf =nfsd_file_get (fp->fi_fds[O_RDWR ]); dl_type =NFS4_OPEN_DELEGATE_WRITE ; - }else {Does this mean
OPEN4_SHARE_ACCESS_WRITEdo not get a write delegation? It does not seem right.Quoted 2 lines
-Dai
Why? Per RFC 8881:
Quoted 8 lines
"An
OPEN_DELEGATE_WRITEdelegation allows the client to handle, on its own, all opens."All opens. That includes read opens.
An
OPEN4_SHARE_ACCESS_WRITEopen will succeed on a file to which the user has no read permissions. Therefore, we can't grant a write delegation since can't guarantee that the user is allowed to do that.If the server grants the write delegation on an OPEN with
OPEN4_SHARE_ACCESS_WRITEon the file with WR-only access mode then why can't the server checks and denies the subsequent READ?Per RFC 8881, section 9.1.2:
For delegation stateids, the access mode is based on the type of delegation.
When a READ, WRITE, or SETATTR (that specifies the size attribute) operation is done, the operation is subject to checking against the access mode to verify that the operation is appropriate given the stateid with which the operation is associated.
In the case of WRITE-type operations (i.e., WRITEs and SETATTRs that set size), the server MUST verify that the access mode allows writing and MUST return an
NFS4ERR_OPENMODEerror if it does not. In the case of READ, the server may perform the corresponding check on the access mode, or it may choose to allow READ on OPENs forOPEN4_SHARE_ACCESS_WRITE, to accommodate clients whose WRITE implementation may unavoidably do reads (e.g., due to buffer cache constraints). However, even if READs are allowed in these circumstances, the server MUST still check for locks that conflict with the READ (e.g., another OPEN specifiedOPEN4_SHARE_DENY_READorOPEN4_SHARE_DENY_BOTH). Note that a server that does enforce the access mode check on READs need not explicitly check for conflicting share reservations since the existence of OPEN forOPEN4_SHARE_ACCESS_READguarantees that no conflicting share reservation can exist.Quoted 4 lines
FWIW, The Solaris server grants write delegation on OPEN with
OPEN4_SHARE_ACCESS_WRITEon file with access mode either RW or WR-only. Maybe this is a bug? or the spec is not clear?I don't think that's necessarily a bug.
It's not that the spec demands that we only hand out delegations on BOTH opens. This is more of a quirk of the Linux implementation. Linux' write delegations require an open
O_RDWRfile descriptor because we may be called upon to do a read on its behalf.Technically, we could probably just have it check for
OPEN4_SHARE_ACCESS_WRITE, but in the case where READ isn't also set, then you're unlikely to get a delegation. Either theO_RDWRdescriptor will be NULL, or there are other, conflicting opens already present.Quoted 9 lines
Quoted 2 lines
Solaris may have a completely different design that doesn't require this. I haven't looked at its code to know.
I'm comfortable for now with not handing out write delegations for
SHARE_ACCESS_WRITEopens. I prefer that to permission checking on every READ operation.If we find that it's a significant performance issue, we can revisit.
Yeah. The thing to remember here is that delegations are optional. We can always say no.
Quoted 7 lines
One thing we could consider to allow this is trying to open
O_RDWRfirst, and then only fall back to doing anO_WRONLYopen if that fails. I'm not sure how that would work out in practice though.One thing that'd be interesting to know with Solaris (and maybe OnTap) is whether they will give you a write delegation for an
O_WRONLYopen when you don't have any read permissions on the file.
Yes, the Solaris server does hand out the write delegation for a O_WRONLY open on a file with write only permission for the owner.
-Dai
Quoted 48 lines
If they do, then is the client expected to do permissions enforcement for the cached open and reject local openers for read? I guess I ought to be looking at the Linux client code for this...
Quoted 43 lines
Quoted 32 lines
Quoted 3 lines
It'd would be interesting to know how ONTAP server behaves in this scenario.
Indeed. Most likely it behaves more like Solaris does, but it'd nice to know.
Quoted 26 lines
Quoted 26 lines
Quoted 26 lines
Quoted 26 lines
fs/nfsd/nfs4state.c +10
-7+ } + +/* +* If the file is being opened O_RDONLY or we couldn't get a O_RDWR +* file for some reason, then try for a read deleg instead. +*/ +if (!nf && (open->op_share_access &NFS4_SHARE_ACCESS_READ )) { nf =find_readable_file (fp); dl_type =NFS4_OPEN_DELEGATE_READ ; } -if (!nf) { -/* -* We probably could attempt another open and get a read -* delegation, but for now, don't bother until the -* client actually sends us one. -*/ + +if (!nf)return ERR_PTR (-EAGAIN ); - } +spin_lock (&state_lock);spin_lock (&fp->fi_lock);if (nfs4_delegation_exists (clp, fp))Quoted 7 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
---
base-commit:a734662572708cf062e974f659ae50c24fc1ad17
change-id: 20230731-wdeleg-bbdb6b25a3c6Best regards,
Signature · 1 line
Jeff Layton <[email protected]>--
Chuck Lever
[email protected]
reviewerQuoted 152 lines · Chuck Lever III
Quoted 152 lines · Jeff Layton
Quoted 132 lines · On Wed, 2023-08-02 at 13:15 -0700, [email protected] wrote:
Quoted 94 lines · Jeff Layton
Quoted 9 lines · On Wed, 2023-08-02 at 09:29 -0700, [email protected] wrote:
Quoted 5 lines · Jeff Layton
I noticed that xfstests generic/001 was failing against linux-next nfsd.
The client would request a
OPEN4_SHARE_ACCESS_WRITEopen, and the server would hand out a write delegation. The client would then try to use that write delegation as the source stateid in a COPYnot sure why the client opens the source file of a COPY operation with
OPEN4_SHARE_ACCESS_WRITE?It doesn't. The original open is to write the data for the file being copied. It then opens the file again for READ, but since it has a write delegation, it doesn't need to talk to the server at all -- it can just use that stateid for later operations.
Quoted 5 lines
Quoted 2 lines
or CLONE operation, and
the server would respond withNFS4ERR_STALE.If the server does not allow client to use write delegation for the READ, should the correct error return be
NFS4ERR_OPENMODE?The server must allow the client to use a write delegation for read operations. It's required by the spec, AFAIU.
The error in this case was just bogus. The vfs copy routine would return
-EBADFsince the file didn't haveFMODE_READ, and the nfs server would translate that intoNFS4ERR_STALE.Probably there is a better v4 error code that we could translate EBADF to, but with this patch it shouldn't be a problem any longer.
Quoted 60 lines
Quoted 47 lines
The problem is that the struct file associated with the delegation does not necessarily have read permissions. It's handing out a write delegation on what is effectively an
O_WRONLYopen. RFC 8881 states:"An
OPEN_DELEGATE_WRITEdelegation allows the client to handle, on its
own, all opens."Given that the client didn't request any read permissions, and that nfsd didn't check for any, it seems wrong to give out a write delegation.
Only hand out a write delegation if we have a
O_RDWRdescriptor available. If it fails to find an appropriate write descriptor, go ahead and try for a read delegation ifNFS4_SHARE_ACCESS_READwas requested.This fixes xfstest generic/001.
Signed-off-by Jeff Layton <[email protected]>---
Changes in v2:
- Rework the logic when finding struct file for the delegation. The earlier patch might still have attached aO_WRONLYfile to the deleg in some cases, and could still have handed out a write delegation on anO_WRONLYOPEN request in some cases.--- fs/nfsd/nfs4state.c | 29 ++++++++++++++++++----------- 1 file changed, 18 insertions(+), 11 deletions(-)fs/nfsd/nfs4state.c +1
-2@@ -5449,7 +5449,7 @@ nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp,struct nfs4_file *fp = stp->st_stid.sc_file;struct nfs4_clnt_odstate *odstate = stp->st_clnt_odstate;struct nfs4_delegation *dp; -struct nfsd_file *nf; +struct nfsd_file *nf =NULL ;struct file_lock *fl;u32 dl_type;@@ -5461,21 +5461,28 @@ nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp,if (fp->fi_had_conflict)return ERR_PTR (-EAGAIN ); -if (open->op_share_access &NFS4_SHARE_ACCESS_WRITE ) {fs/nfsd/nfs4state.c nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp, · line 5467
Quoted 10 lines
fs/nfsd/nfs4state.c +7
-2- nf =find_writeable_file (fp); +/* +* Try for a write delegation first. We need an O_RDWR file +* since a write delegation allows the client to perform any open +* from its cache. +*/ +if ((open->op_share_access &NFS4_SHARE_ACCESS_BOTH ) ==NFS4_SHARE_ACCESS_BOTH ) { + nf =nfsd_file_get (fp->fi_fds[O_RDWR ]); dl_type =NFS4_OPEN_DELEGATE_WRITE ; - }else {Does this mean
OPEN4_SHARE_ACCESS_WRITEdo not get a write delegation? It does not seem right.Quoted 2 lines
-Dai
Why? Per RFC 8881:
Quoted 8 lines
"An
OPEN_DELEGATE_WRITEdelegation allows the client to handle, on its own, all opens."All opens. That includes read opens.
An
OPEN4_SHARE_ACCESS_WRITEopen will succeed on a file to which the user has no read permissions. Therefore, we can't grant a write delegation since can't guarantee that the user is allowed to do that.If the server grants the write delegation on an OPEN with
OPEN4_SHARE_ACCESS_WRITEon the file with WR-only access mode then why can't the server checks and denies the subsequent READ?Per RFC 8881, section 9.1.2:
For delegation stateids, the access mode is based on the type of delegation.
When a READ, WRITE, or SETATTR (that specifies the size attribute) operation is done, the operation is subject to checking against the access mode to verify that the operation is appropriate given the stateid with which the operation is associated.
In the case of WRITE-type operations (i.e., WRITEs and SETATTRs that set size), the server MUST verify that the access mode allows writing and MUST return an
NFS4ERR_OPENMODEerror if it does not. In the case of READ, the server may perform the corresponding check on the access mode, or it may choose to allow READ on OPENs forOPEN4_SHARE_ACCESS_WRITE, to accommodate clients whose WRITE implementation may unavoidably do reads (e.g., due to buffer cache constraints). However, even if READs are allowed in these circumstances, the server MUST still check for locks that conflict with the READ (e.g., another OPEN specifiedOPEN4_SHARE_DENY_READorOPEN4_SHARE_DENY_BOTH). Note that a server that does enforce the access mode check on READs need not explicitly check for conflicting share reservations since the existence of OPEN forOPEN4_SHARE_ACCESS_READguarantees that no conflicting share reservation can exist.Quoted 4 lines
FWIW, The Solaris server grants write delegation on OPEN with
OPEN4_SHARE_ACCESS_WRITEon file with access mode either RW or WR-only. Maybe this is a bug? or the spec is not clear?I don't think that's necessarily a bug.
It's not that the spec demands that we only hand out delegations on BOTH opens. This is more of a quirk of the Linux implementation. Linux' write delegations require an open
O_RDWRfile descriptor because we may be called upon to do a read on its behalf.Technically, we could probably just have it check for
OPEN4_SHARE_ACCESS_WRITE, but in the case where READ isn't also set, then you're unlikely to get a delegation. Either theO_RDWRdescriptor will be NULL, or there are other, conflicting opens already present.
Quoted 5 lines
Quoted 2 lines
Solaris may have a completely different design that doesn't require this. I haven't looked at its code to know.
I'm comfortable for now with not handing out write delegations for
SHARE_ACCESS_WRITEopens. I prefer that to permission checking on every READ operation.
I'm fine with just handling out write delegation for SHARE_ACCESS_BOTH only.
Just a concern about not checking for access at the time of READ operation. If the file was opened with SHARE_ACCESS_WRITE (no write delegation granted) and the file access mode was changed to read-only, is it a correct behavior for the server to allow the READ to go through?
-Dai
Quoted 47 lines
If we find that it's a significant performance issue, we can revisit.
Quoted 32 lines
Quoted 3 lines
It'd would be interesting to know how ONTAP server behaves in this scenario.
Indeed. Most likely it behaves more like Solaris does, but it'd nice to know.
Quoted 26 lines
Quoted 26 lines
Quoted 26 lines
Quoted 26 lines
fs/nfsd/nfs4state.c +10
-7+ } + +/* +* If the file is being opened O_RDONLY or we couldn't get a O_RDWR +* file for some reason, then try for a read deleg instead. +*/ +if (!nf && (open->op_share_access &NFS4_SHARE_ACCESS_READ )) { nf =find_readable_file (fp); dl_type =NFS4_OPEN_DELEGATE_READ ; } -if (!nf) { -/* -* We probably could attempt another open and get a read -* delegation, but for now, don't bother until the -* client actually sends us one. -*/ + +if (!nf)return ERR_PTR (-EAGAIN ); - } +spin_lock (&state_lock);spin_lock (&fp->fi_lock);if (nfs4_delegation_exists (clp, fp))Quoted 7 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
---
base-commit:a734662572708cf062e974f659ae50c24fc1ad17
change-id: 20230731-wdeleg-bbdb6b25a3c6Best regards,
Signature · 1 line
Jeff Layton <[email protected]>--
Chuck Lever
[email protected]
reviewerQuoted 160 lines · On 8/2/23 2:22 PM, [email protected] wrote:
Quoted 158 lines · Chuck Lever III
Quoted 158 lines · Jeff Layton
Quoted 136 lines · On Wed, 2023-08-02 at 13:15 -0700, [email protected] wrote:
Quoted 98 lines · Jeff Layton
Quoted 9 lines · On Wed, 2023-08-02 at 09:29 -0700, [email protected] wrote:
Quoted 5 lines · Jeff Layton
I noticed that xfstests generic/001 was failing against linux-next nfsd.
The client would request a
OPEN4_SHARE_ACCESS_WRITEopen, and the server would hand out a write delegation. The client would then try to use that write delegation as the source stateid in a COPYnot sure why the client opens the source file of a COPY operation with
OPEN4_SHARE_ACCESS_WRITE?It doesn't. The original open is to write the data for the file being copied. It then opens the file again for READ, but since it has a write delegation, it doesn't need to talk to the server at all -- it can just use that stateid for later operations.
Quoted 5 lines
Quoted 2 lines
or CLONE operation, and
the server would respond withNFS4ERR_STALE.If the server does not allow client to use write delegation for the READ, should the correct error return be
NFS4ERR_OPENMODE?The server must allow the client to use a write delegation for read operations. It's required by the spec, AFAIU.
The error in this case was just bogus. The vfs copy routine would return
-EBADFsince the file didn't haveFMODE_READ, and the nfs server would
translate that intoNFS4ERR_STALE.Probably there is a better v4 error code that we could translate EBADF to, but with this patch it shouldn't be a problem any longer.
Quoted 63 lines
Quoted 50 lines
The problem is that the struct file associated with the delegation does not necessarily have read permissions. It's handing out a write delegation on what is effectively an
O_WRONLYopen. RFC 8881 states:"An
OPEN_DELEGATE_WRITEdelegation allows the client to handle, on its own, all opens."Given that the client didn't request any read permissions, and that nfsd didn't check for any, it seems wrong to give out a write delegation.
Only hand out a write delegation if we have a
O_RDWRdescriptor available. If it fails to find an appropriate write descriptor, go ahead and try for a read delegation ifNFS4_SHARE_ACCESS_READwas requested.This fixes xfstest generic/001.
Signed-off-by Jeff Layton <[email protected]>---
Changes in v2:
- Rework the logic when finding struct file for the delegation. The earlier patch might still have attached aO_WRONLYfile to the deleg in some cases, and could still have handed out a write delegation on anO_WRONLYOPEN request in some cases.
---
fs/nfsd/nfs4state.c| 29 ++++++++++++++++++-----------1 file changed, 18 insertions(+), 11 deletions(-)fs/nfsd/nfs4state.c +1
-3@@ -5449,7 +5449,7 @@ nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp, struct nfs4_file *fp = stp->st_stid.sc_file; struct nfs4_clnt_odstate *odstate = stp->st_clnt_odstate; struct nfs4_delegation *dp; -struct nfsd_file *nf; +struct nfsd_file *nf =NULL ; struct file_lock *fl; u32 dl_type;@@ -5461,21 +5461,28 @@ nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp, if (fp->fi_had_conflict) return ERR_PTR(-EAGAIN); -if (open->op_share_access &NFS4_SHARE_ACCESS_WRITE ) { - nf =find_writeable_file (fp);fs/nfsd/nfs4state.c nfs4_set_delegation(struct nfsd4_open · line 5469
Quoted 10 lines
fs/nfsd/nfs4state.c +7
-1+/* +* Try for a write delegation first. We need an O_RDWR file +* since a write delegation allows the client to perform any open +* from its cache. +*/ +if ((open->op_share_access &NFS4_SHARE_ACCESS_BOTH ) ==NFS4_SHARE_ACCESS_BOTH ) { + nf =nfsd_file_get (fp->fi_fds[O_RDWR ]); dl_type = NFS4_OPEN_DELEGATE_WRITE; - } else {Does this mean
OPEN4_SHARE_ACCESS_WRITEdo not get a write delegation? It does not seem right.Quoted 2 lines
-Dai
Why? Per RFC 8881:
Quoted 8 lines
"An
OPEN_DELEGATE_WRITEdelegation allows the client to handle, on its own, all opens."All opens. That includes read opens.
An
OPEN4_SHARE_ACCESS_WRITEopen will succeed on a file to which the user has no read permissions. Therefore, we can't grant a write delegation since can't guarantee that the user is allowed to do that.If the server grants the write delegation on an OPEN with
OPEN4_SHARE_ACCESS_WRITEon the file with WR-only access mode then why can't the server checks and denies the subsequent READ?Per RFC 8881, section 9.1.2:
For delegation stateids, the access mode is based on the type of delegation.
When a READ, WRITE, or SETATTR (that specifies the size attribute) operation is done, the operation is subject to checking against the access mode to verify that the operation is appropriate given the stateid with which the operation is associated.
In the case of WRITE-type operations (i.e., WRITEs and SETATTRs that set size), the server MUST verify that the access mode allows writing and MUST return an
NFS4ERR_OPENMODEerror if it does not. In the case of READ, the server may perform the corresponding check on the access mode, or it may choose to allow READ on OPENs forOPEN4_SHARE_ACCESS_WRITE, to accommodate clients whose WRITE implementation may unavoidably do reads (e.g., due to buffer cache constraints). However, even if READs are allowed in these circumstances, the server MUST still check for locks that conflict with the READ (e.g., another OPEN specifiedOPEN4_SHARE_DENY_READorOPEN4_SHARE_DENY_BOTH). Note that a server that does enforce the access mode check on READs need not explicitly check for conflicting share reservations since the existence of OPEN forOPEN4_SHARE_ACCESS_READguarantees that no conflicting share reservation can exist.Quoted 4 lines
FWIW, The Solaris server grants write delegation on OPEN with
OPEN4_SHARE_ACCESS_WRITEon file with access mode either RW or WR-only. Maybe this is a bug? or the spec is not clear?I don't think that's necessarily a bug.
It's not that the spec demands that we only hand out delegations on BOTH opens. This is more of a quirk of the Linux implementation. Linux' write delegations require an open
O_RDWRfile descriptor because we may be called upon to do a read on its behalf.Technically, we could probably just have it check for
OPEN4_SHARE_ACCESS_WRITE, but in the case where READ isn't also set, then you're unlikely to get a delegation. Either theO_RDWRdescriptor will be NULL, or there are other, conflicting opens already present.Solaris may have a completely different design that doesn't require this. I haven't looked at its code to know.
Quoted 8 lines
Quoted 3 lines
I'm comfortable for now with not handing out write delegations for
SHARE_ACCESS_WRITEopens. I prefer that to permission checking on every READ operation.I'm fine with just handling out write delegation for
SHARE_ACCESS_BOTHonly.Just a concern about not checking for access at the time of READ operation.
or not checking file permission at the time WRITE.
Quoted 3 lines
If the file was opened with
SHARE_ACCESS_WRITE(no write delegation granted) and the file access mode was changed to read-only, is it a correct behavior for the server to allow the READ to go through?
I meant for the WRITE to go through.
Quoted 51 lines
-Dai
Quoted 48 lines
If we find that it's a significant performance issue, we can revisit.
Quoted 40 lines
Quoted 3 lines
It'd would be interesting to know how ONTAP server behaves in this scenario.
Indeed. Most likely it behaves more like Solaris does, but it'd nice to know.
Quoted 32 lines
Quoted 32 lines
Quoted 32 lines
Quoted 32 lines
fs/nfsd/nfs4state.c +4
-0+ } + +/* +* If the file is being opened O_RDONLY or we couldn't get a
O_RDWR
+ * file for some reason, then try for a read deleg instead.
+ */
+ if (!nf && (open->op_share_access&NFS4_SHARE_ACCESS_READ)) { nf =find_readable_file(fp);
dl_type=NFS4_OPEN_DELEGATE_READ;
}
- if (!nf) {
- /*
- * We probably could attempt another open and get a read
- * delegation, but for now, don't bother until the
- * client actually sends us one.
- */
+
+ if (!nf)
returnERR_PTR(-EAGAIN);
- }
+
spin_lock(&state_lock);
spin_lock(&fp->fi_lock);
if (nfs4_delegation_exists(clp, fp))---
base-commit:a734662572708cf062e974f659ae50c24fc1ad17
change-id: 20230731-wdeleg-bbdb6b25a3c6Best regards,
Signature · 1 line
Jeff Layton <[email protected]>Signature · 1 line
Chuck Lever
Jeff Layton
maintainerQuoted 201 lines · On Wed, 2023-08-02 at 14:32 -0700, [email protected] wrote:
Quoted 195 lines · On 8/2/23 2:22 PM, [email protected] wrote:
Quoted 187 lines · Chuck Lever III
Quoted 187 lines · Jeff Layton
Quoted 166 lines · On Wed, 2023-08-02 at 13:15 -0700, [email protected] wrote:
Quoted 118 lines · Jeff Layton
Quoted 4 lines · On Wed, 2023-08-02 at 09:29 -0700, [email protected] wrote:
Quoted 3 lines · Jeff Layton
I noticed that xfstests generic/001 was failing against linux-next nfsd.
Quoted 9 lines
Quoted 5 lines
The client would request a
OPEN4_SHARE_ACCESS_WRITEopen, and the server
would hand out a write delegation. The client would then try to use that
write delegation as the source stateid in a COPYnot sure why the client opens the source file of a COPY operation with
OPEN4_SHARE_ACCESS_WRITE?It doesn't. The original open is to write the data for the file being copied. It then opens the file again for READ, but since it has a write
delegation, it doesn't need to talk to the server at all -- it can just
use that stateid for later operations.Quoted 5 lines
Quoted 2 lines
or CLONE operation, and
the server would respond withNFS4ERR_STALE.If the server does not allow client to use write delegation for the READ, should the correct error return be
NFS4ERR_OPENMODE?The server must allow the client to use a write delegation for read operations. It's required by the spec, AFAIU.
The error in this case was just bogus. The vfs copy routine would return
-EBADFsince the file didn't haveFMODE_READ, and the nfs server would
translate that intoNFS4ERR_STALE.Probably there is a better v4 error code that we could translate EBADF
to, but with this patch it shouldn't be a problem any longer.Quoted 71 lines
Quoted 57 lines
The problem is that the struct file associated with the delegation does
not necessarily have read permissions. It's handing out a write delegation on what is effectively anO_WRONLYopen. RFC 8881 states:"An
OPEN_DELEGATE_WRITEdelegation allows the client to handle, on its
own, all opens."Given that the client didn't request any read permissions, and that nfsd
didn't check for any, it seems wrong to give out a write delegation.Only hand out a write delegation if we have a
O_RDWRdescriptor available. If it fails to find an appropriate write descriptor, go ahead and try for a read delegation ifNFS4_SHARE_ACCESS_READwas requested.This fixes xfstest generic/001.
Signed-off-by Jeff Layton <[email protected]>---
Changes in v2:
- Rework the logic when finding struct file for the delegation. The earlier patch might still have attached aO_WRONLYfile to the deleg
in some cases, and could still have handed out a write delegation on
anO_WRONLYOPEN request in some cases.
---
fs/nfsd/nfs4state.c| 29 ++++++++++++++++++-----------1 file changed, 18 insertions(+), 11 deletions(-)fs/nfsd/nfs4state.c +1
-3@@ -5449,7 +5449,7 @@ nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp, struct nfs4_file *fp = stp->st_stid.sc_file; struct nfs4_clnt_odstate *odstate = stp->st_clnt_odstate; struct nfs4_delegation *dp; -struct nfsd_file *nf; +struct nfsd_file *nf =NULL ; struct file_lock *fl; u32 dl_type;@@ -5461,21 +5461,28 @@ nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp, if (fp->fi_had_conflict) return ERR_PTR(-EAGAIN); -if (open->op_share_access &NFS4_SHARE_ACCESS_WRITE ) { - nf =find_writeable_file (fp);fs/nfsd/nfs4state.c nfs4_set_delegation(struct nfsd4_open · line 5469
Quoted 10 lines
fs/nfsd/nfs4state.c +7
-1+/* +* Try for a write delegation first. We need an O_RDWR file +* since a write delegation allows the client to perform any open +* from its cache. +*/ +if ((open->op_share_access &NFS4_SHARE_ACCESS_BOTH ) ==NFS4_SHARE_ACCESS_BOTH ) { + nf =nfsd_file_get (fp->fi_fds[O_RDWR ]); dl_type = NFS4_OPEN_DELEGATE_WRITE; - } else {Does this mean
OPEN4_SHARE_ACCESS_WRITEdo not get a write delegation?
It does not seem right.Quoted 2 lines
-Dai
Why? Per RFC 8881:
"An
OPEN_DELEGATE_WRITEdelegation allows the client to handle, on its
own, all opens."Quoted 5 lines
All opens. That includes read opens.
An
OPEN4_SHARE_ACCESS_WRITEopen will succeed on a file to which the user has no read permissions. Therefore, we can't grant a write delegation since can't guarantee that the user is allowed to do that.If the server grants the write delegation on an OPEN with
OPEN4_SHARE_ACCESS_WRITEon the file with WR-only access mode then why can't the server checks and denies the subsequent READ?Per RFC 8881, section 9.1.2:
For delegation stateids, the access mode is based on the type of delegation.
When a READ, WRITE, or SETATTR (that specifies the size attribute)
operation is done, the operation is subject to checking against the
access mode to verify that the operation is appropriate given the stateid with which the operation is associated.In the case of WRITE-type operations (i.e., WRITEs and SETATTRs that
set size), the server MUST verify that the access mode allows writing
and MUST return anNFS4ERR_OPENMODEerror if it does not. In the case
of READ, the server may perform the corresponding check on the access
mode, or it may choose to allow READ on OPENs forOPEN4_SHARE_ACCESS_WRITE,
to accommodate clients whose WRITE implementation may unavoidably do
reads (e.g., due to buffer cache constraints). However, even if READs
are allowed in these circumstances, the server MUST still check for
locks that conflict with the READ (e.g., another OPEN specifiedOPEN4_SHARE_DENY_READorOPEN4_SHARE_DENY_BOTH). Note that a server
that does enforce the access mode check on READs need not explicitly
check for conflicting share reservations since the existence of OPEN
forOPEN4_SHARE_ACCESS_READguarantees that no conflicting share reservation can exist.Quoted 4 lines
FWIW, The Solaris server grants write delegation on OPEN with
OPEN4_SHARE_ACCESS_WRITEon file with access mode either RW or WR-only. Maybe this is a bug? or the spec is not clear?I don't think that's necessarily a bug.
It's not that the spec demands that we only hand out delegations on BOTH
opens. This is more of a quirk of the Linux implementation. Linux' write delegations require an openO_RDWRfile descriptor because we may be called upon to do a read on its behalf.Technically, we could probably just have it check for
OPEN4_SHARE_ACCESS_WRITE, but in the case where READ isn't also set, then you're unlikely to get a delegation. Either theO_RDWRdescriptor will be NULL, or there are other, conflicting opens already present.Quoted 5 lines
Quoted 2 lines
Solaris may have a completely different design that doesn't require this. I haven't looked at its code to know.
I'm comfortable for now with not handing out write delegations for
SHARE_ACCESS_WRITEopens. I prefer that to permission checking on every READ operation.Quoted 5 lines
I'm fine with just handling out write delegation for
SHARE_ACCESS_BOTHonly.Just a concern about not checking for access at the time of READ operation.
Quoted 7 lines
or not checking file permission at the time WRITE.
Quoted 5 lines
If the file was opened with
SHARE_ACCESS_WRITE(no write delegation granted)
and the file access mode was changed to read-only, is it a correct behavior
for the server to allow the READ to go through?I meant for the WRITE to go through.
Yes:
POSIX permissions enforcement is done at open time, not when doing actual reads and writes. If you open a file on (e.g.) xfs and start streaming writes to it, then you don't expect that you will lose the ability to write to that fd if the permissions change.
In the old v2/3 days of stateless NFS, we had to check permissions on every READ or WRITE operation, but we generally did an open on every RPC too, so it just worked out that we checked permissions on each operation.
With v4 we can better approximate POSIX semantics by just associating a stateid with an open file to allow the client to keep writing in this case.
Quoted 51 lines
Quoted 51 lines
-Dai
Quoted 48 lines
If we find that it's a significant performance issue, we can revisit.
Quoted 40 lines
Quoted 3 lines
It'd would be interesting to know how ONTAP server behaves in this scenario.
Indeed. Most likely it behaves more like Solaris does, but it'd nice to know.
Quoted 32 lines
Quoted 32 lines
Quoted 32 lines
Quoted 32 lines
fs/nfsd/nfs4state.c +4
-0+ } + +/* +* If the file is being opened O_RDONLY or we couldn't get a
O_RDWR
+ * file for some reason, then try for a read deleg instead.
+ */
+ if (!nf && (open->op_share_access&NFS4_SHARE_ACCESS_READ)) { nf =find_readable_file(fp);
dl_type=NFS4_OPEN_DELEGATE_READ;
}
- if (!nf) {
- /*
- * We probably could attempt another open and get a read
- * delegation, but for now, don't bother until the
- * client actually sends us one.
- */
+
+ if (!nf)
returnERR_PTR(-EAGAIN);
- }
+
spin_lock(&state_lock);
spin_lock(&fp->fi_lock);
if (nfs4_delegation_exists(clp, fp))---
base-commit:a734662572708cf062e974f659ae50c24fc1ad17
change-id: 20230731-wdeleg-bbdb6b25a3c6Best regards,
Signature · 1 line
Jeff Layton <[email protected]>Signature · 1 line
Chuck Lever
Signature · 1 line
Jeff Layton <[email protected]>
A message that is not in the archive (sent off-list or not yet received): [email protected]
Jeff Layton
maintainerQuoted 330 lines · On Wed, 2023-08-02 at 16:38 -0700, [email protected] wrote:
On 8/2/23 2:52 PM, Jeff Layton wrote:
Quoted 318 lines
On Wed, 2023-08-02 at 14:32 -0700, [email protected] wrote:
Quoted 294 lines
On 8/2/23 2:22 PM, [email protected] wrote:
Quoted 266 lines
On 8/2/23 1:57 PM, Chuck Lever III wrote:
Quoted 254 lines
Quoted 243 lines
On Aug 2, 2023, at 4:48 PM, Jeff Layton <[email protected]> wrote:
On Wed, 2023-08-02 at 13:15 -0700, [email protected] wrote:
Quoted 220 lines
On 8/2/23 11:15 AM, Jeff Layton wrote:
Quoted 164 lines
On Wed, 2023-08-02 at 09:29 -0700, [email protected] wrote:
Quoted 17 lines
On 8/1/23 6:33 AM, Jeff Layton wrote:
Quoted 5 lines
I noticed that xfstests generic/001 was failing against linux-next nfsd.
Quoted 7 lines
The client would request a
OPEN4_SHARE_ACCESS_WRITEopen, and the server
would hand out a write delegation. The client would then try to use that
write delegation as the source stateid in a COPYQuoted 8 lines
not sure why the client opens the source file of a COPY operation with
OPEN4_SHARE_ACCESS_WRITE?It doesn't. The original open is to write the data for the file being copied. It then opens the file again for READ, but since it has a write
delegation, it doesn't need to talk to the server at all -- it can just
use that stateid for later operations.Quoted 8 lines
Quoted 6 lines
or CLONE operation, and
the server would respond withNFS4ERR_STALE.Quoted 7 lines
If the server does not allow client to use write delegation for the READ, should the correct error return be
NFS4ERR_OPENMODE?The server must allow the client to use a write delegation for read operations. It's required by the spec, AFAIU.
The error in this case was just bogus. The vfs copy routine would return
-EBADFsince the file didn't haveFMODE_READ, and the nfs server would
translate that intoNFS4ERR_STALE.Probably there is a better v4 error code that we could translate EBADF
to, but with this patch it shouldn't be a problem any longer.Quoted 75 lines
Quoted 69 lines
The problem is that the struct file associated with the delegation does
not necessarily have read permissions. It's handing out a write delegation on what is effectively anO_WRONLYopen. RFC 8881 states:"An
OPEN_DELEGATE_WRITEdelegation allows the client to handle, on its
own, all opens."Given that the client didn't request any read permissions, and that nfsd
didn't check for any, it seems wrong to give out a write delegation.Only hand out a write delegation if we have a
O_RDWRdescriptor available. If it fails to find an appropriate write descriptor, go ahead and try for a read delegation ifNFS4_SHARE_ACCESS_READwas requested.This fixes xfstest generic/001.
Signed-off-by Jeff Layton <[email protected]>---
Changes in v2:
- Rework the logic when finding struct file for the delegation. The earlier patch might still have attached aO_WRONLYfile to the deleg
in some cases, and could still have handed out a write delegation on
anO_WRONLYOPEN request in some cases.
---
fs/nfsd/nfs4state.c| 29 ++++++++++++++++++-----------1 file changed, 18 insertions(+), 11 deletions(-)fs/nfsd/nfs4state.c +8
-4@@ -5449,7 +5449,7 @@ nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp, struct nfs4_file *fp = stp->st_stid.sc_file; struct nfs4_clnt_odstate *odstate = stp->st_clnt_odstate; struct nfs4_delegation *dp; -struct nfsd_file *nf; +struct nfsd_file *nf =NULL ; struct file_lock *fl; u32 dl_type;@@ -5461,21 +5461,28 @@ nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp, if (fp->fi_had_conflict) return ERR_PTR(-EAGAIN); -if (open->op_share_access &NFS4_SHARE_ACCESS_WRITE ) { - nf =find_writeable_file (fp); +/* +* Try for a write delegation first. We need an O_RDWR file +* since a write delegation allows the client to perform any open +* from its cache. +*/ +if ((open->op_share_access &NFS4_SHARE_ACCESS_BOTH ) ==NFS4_SHARE_ACCESS_BOTH ) { + nf =nfsd_file_get (fp->fi_fds[O_RDWR ]); dl_type = NFS4_OPEN_DELEGATE_WRITE; - }else {Quoted 2 lines
Quoted 8 lines
Does this mean
OPEN4_SHARE_ACCESS_WRITEdo not get a write delegation?
It does not seem right.-Dai
Why? Per RFC 8881:
"An
OPEN_DELEGATE_WRITEdelegation allows the client to handle, on its
own, all opens."Quoted 7 lines
All opens. That includes read opens.
An
OPEN4_SHARE_ACCESS_WRITEopen will succeed on a file to which the user has no read permissions. Therefore, we can't grant a write delegation since can't guarantee that the user is allowed to do that.If the server grants the write delegation on an OPEN with
OPEN4_SHARE_ACCESS_WRITEon the file with WR-only access mode then why can't the server checks and denies the subsequent READ?Per RFC 8881, section 9.1.2:
For delegation stateids, the access mode is based on the type of delegation.
When a READ, WRITE, or SETATTR (that specifies the size attribute)
operation is done, the operation is subject to checking against the
access mode to verify that the operation is appropriate given the stateid with which the operation is associated.In the case of WRITE-type operations (i.e., WRITEs and SETATTRs that
set size), the server MUST verify that the access mode allows writing
and MUST return anNFS4ERR_OPENMODEerror if it does not. In the case
of READ, the server may perform the corresponding check on the access
mode, or it may choose to allow READ on OPENs forOPEN4_SHARE_ACCESS_WRITE,
to accommodate clients whose WRITE implementation may unavoidably do
reads (e.g., due to buffer cache constraints). However, even if READs
are allowed in these circumstances, the server MUST still check for
locks that conflict with the READ (e.g., another OPEN specifiedOPEN4_SHARE_DENY_READorOPEN4_SHARE_DENY_BOTH). Note that a server
that does enforce the access mode check on READs need not explicitly
check for conflicting share reservations since the existence of OPEN
forOPEN4_SHARE_ACCESS_READguarantees that no conflicting share reservation can exist.Quoted 6 lines
FWIW, The Solaris server grants write delegation on OPEN with
OPEN4_SHARE_ACCESS_WRITEon file with access mode either RW or WR-only. Maybe this is a bug? or the spec is not clear?I don't think that's necessarily a bug.
It's not that the spec demands that we only hand out delegations on BOTH
opens. This is more of a quirk of the Linux implementation. Linux' write delegations require an openO_RDWRfile descriptor because we may be called upon to do a read on its behalf.Quoted 9 lines
Technically, we could probably just have it check for
OPEN4_SHARE_ACCESS_WRITE, but in the case where READ isn't also set, then you're unlikely to get a delegation. Either theO_RDWRdescriptor will be NULL, or there are other, conflicting opens already present.Solaris may have a completely different design that doesn't require this. I haven't looked at its code to know.
Quoted 7 lines
I'm comfortable for now with not handing out write delegations for
SHARE_ACCESS_WRITEopens. I prefer that to permission checking on every READ operation.Quoted 9 lines
I'm fine with just handling out write delegation for
SHARE_ACCESS_BOTHonly.Just a concern about not checking for access at the time of READ operation.
or not checking file permission at the time WRITE.
Quoted 9 lines
If the file was opened with
SHARE_ACCESS_WRITE(no write delegation granted)
and the file access mode was changed to read-only, is it a correct behavior
for the server to allow the READ to go through?Quoted 5 lines
I meant for the WRITE to go through.
Yes:
POSIX permissions enforcement is done at open time, not when doing actual reads and writes. If you open a file on (e.g.) xfs and start streaming writes to it, then you don't expect that you will lose the ability to write to that fd if the permissions change.
In the old v2/3 days of stateless NFS, we had to check permissions on every READ or WRITE operation, but we generally did an open on every RPC too, so it just worked out that we checked permissions on each operation.
Quoted 8 lines
Quoted 5 lines
With v4 we can better approximate POSIX semantics by just associating a stateid with an open file to allow the client to keep writing in this case.
Thanks Jeff,
Don't thank me yet. I went back and looked at the code, and it looks like we still do check permissions on every READ/WRITE (see nfs4_check_file).
I'm unclear on whether that's required, but it's probably safest to always check permissions like we are. That does mean that if the mode of the file changes after we open it we could end up being unable to read or write to it (much like with v2/3), but at this point most people are used to that sort of behavior on NFS, so I don't worry about it too much.
Signature · 1 line
Jeff Layton <[email protected]>
[email protected]
reviewerQuoted 343 lines · Jeff Layton
Quoted 330 lines · On Wed, 2023-08-02 at 16:38 -0700, [email protected] wrote:
Quoted 318 lines · Jeff Layton
Quoted 296 lines · On Wed, 2023-08-02 at 14:32 -0700, [email protected] wrote:
Quoted 268 lines · On 8/2/23 2:22 PM, [email protected] wrote:
Quoted 256 lines · Chuck Lever III
Quoted 245 lines · Jeff Layton
Quoted 220 lines · On Wed, 2023-08-02 at 13:15 -0700, [email protected] wrote:
Quoted 164 lines · Jeff Layton
Quoted 19 lines · On Wed, 2023-08-02 at 09:29 -0700, [email protected] wrote:
Quoted 12 lines · Jeff Layton
I noticed that xfstests generic/001 was failing against linux-next nfsd.
The client would request a
OPEN4_SHARE_ACCESS_WRITEopen, and the server
would hand out a write delegation. The client would then try to use that
write delegation as the source stateid in a COPYQuoted 6 lines
not sure why the client opens the source file of a COPY operation with
OPEN4_SHARE_ACCESS_WRITE?It doesn't. The original open is to write the data for the file being copied. It then opens the file again for READ, but since it has a write
delegation, it doesn't need to talk to the server at all -- it can just
use that stateid for later operations.Quoted 4 lines
Quoted 4 lines
Quoted 11 lines
Quoted 4 lines
or CLONE operation, and
the server would respond withNFS4ERR_STALE.If the server does not allow client to use write delegation for the READ, should the correct error return be
NFS4ERR_OPENMODE?The server must allow the client to use a write delegation for read operations. It's required by the spec, AFAIU.
The error in this case was just bogus. The vfs copy routine would return
-EBADFsince the file didn't haveFMODE_READ, and the nfs server would
translate that intoNFS4ERR_STALE.Probably there is a better v4 error code that we could translate EBADF
to, but with this patch it shouldn't be a problem any longer.Quoted 79 lines
Quoted 63 lines
The problem is that the struct file associated with the delegation does
not necessarily have read permissions. It's handing out a write delegation on what is effectively anO_WRONLYopen. RFC 8881 states:"An
OPEN_DELEGATE_WRITEdelegation allows the client to
handle, on its
own, all opens."Given that the client didn't request any read permissions, and that nfsd
didn't check for any, it seems wrong to give out a write delegation.Only hand out a write delegation if we have a
O_RDWRdescriptor available. If it fails to find an appropriate write descriptor, go ahead and try for a read delegation ifNFS4_SHARE_ACCESS_READwas requested.This fixes xfstest generic/001.
Signed-off-by Jeff Layton <[email protected]>---
Changes in v2:
- Rework the logic when finding struct file for the delegation. The earlier patch might still have attached aO_WRONLYfile to
the deleg
in some cases, and could still have handed out a write
delegation on
anO_WRONLYOPEN request in some cases.--- fs/nfsd/nfs4state.c | 29 ++++++++++++++++++----------- 1 file changed, 18 insertions(+), 11 deletions(-)fs/nfsd/nfs4state.c +3
-3@@ -5449,7 +5449,7 @@ nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp, struct nfs4_file *fp = stp->st_stid.sc_file; struct nfs4_clnt_odstate *odstate = stp->st_clnt_odstate; struct nfs4_delegation *dp; -struct nfsd_file *nf; +struct nfsd_file *nf =NULL ; struct file_lock *fl; u32 dl_type;@@ -5461,21 +5461,28 @@ nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp, if (fp->fi_had_conflict) return ERR_PTR(-EAGAIN); -if (open->op_share_access &NFS4_SHARE_ACCESS_WRITE ) { - nf =find_writeable_file (fp); +/* +* Try for a write delegation first. We need an O_RDWR file fs/nfsd/nfs4state.c nfs4_set_delegation(struct nfsd4_open · line 5475
Quoted 10 lines
fs/nfsd/nfs4state.c +5
-1+* since a write delegation allows the client to perform any open +* from its cache. +*/ +if ((open->op_share_access &NFS4_SHARE_ACCESS_BOTH ) ==NFS4_SHARE_ACCESS_BOTH ) { + nf =nfsd_file_get (fp->fi_fds[O_RDWR ]); dl_type = NFS4_OPEN_DELEGATE_WRITE; - } else {Does this mean
OPEN4_SHARE_ACCESS_WRITEdo not get a write delegation?
It does not seem right.Quoted 4 lines
-Dai
Why? Per RFC 8881:
"An
OPEN_DELEGATE_WRITEdelegation allows the client to handle, on its
own, all opens."Quoted 7 lines
All opens. That includes read opens.
An
OPEN4_SHARE_ACCESS_WRITEopen will succeed on a file to which the user has no read permissions. Therefore, we can't grant a write delegation since can't guarantee that the user is allowed to do that.If the server grants the write delegation on an OPEN with
OPEN4_SHARE_ACCESS_WRITEon the file with WR-only access mode then why can't the server checks and denies the subsequent READ?Per RFC 8881, section 9.1.2:
For delegation stateids, the access mode is based on the type of delegation.
When a READ, WRITE, or SETATTR (that specifies the size
attribute)
operation is done, the operation is subject to checking
against the
access mode to verify that the operation is appropriate given the stateid with which the operation is associated.In the case of WRITE-type operations (i.e., WRITEs and
SETATTRs that
set size), the server MUST verify that the access mode allows
writing
and MUST return anNFS4ERR_OPENMODEerror if it does not. In
the case
of READ, the server may perform the corresponding check on the
access
mode, or it may choose to allow READ on OPENs forOPEN4_SHARE_ACCESS_WRITE,
to accommodate clients whose WRITE implementation may
unavoidably do
reads (e.g., due to buffer cache constraints). However, even
if READs
are allowed in these circumstances, the server MUST still
check for
locks that conflict with the READ (e.g., another OPEN specifiedOPEN4_SHARE_DENY_READorOPEN4_SHARE_DENY_BOTH). Note that a
server
that does enforce the access mode check on READs need not
explicitly
check for conflicting share reservations since the existence
of OPEN
forOPEN4_SHARE_ACCESS_READguarantees that no conflicting share reservation can exist.Quoted 6 lines
FWIW, The Solaris server grants write delegation on OPEN with
OPEN4_SHARE_ACCESS_WRITEon file with access mode either RW or WR-only. Maybe this is a bug? or the spec is not clear?I don't think that's necessarily a bug.
It's not that the spec demands that we only hand out delegations on BOTH
opens. This is more of a quirk of the Linux implementation. Linux' write delegations require an openO_RDWRfile descriptor because we may be called upon to do a read on its behalf.Quoted 9 lines
Technically, we could probably just have it check for
OPEN4_SHARE_ACCESS_WRITE, but in the case where READ isn't also set, then you're unlikely to get a delegation. Either theO_RDWRdescriptor will be NULL, or there are other, conflicting opens already present.Solaris may have a completely different design that doesn't require this. I haven't looked at its code to know.
Quoted 5 lines
I'm comfortable for now with not handing out write delegations for
SHARE_ACCESS_WRITEopens. I prefer that to permission checking on every READ operation.Quoted 7 lines
I'm fine with just handling out write delegation for
SHARE_ACCESS_BOTHonly.Just a concern about not checking for access at the time of READ operation.
or not checking file permission at the time WRITE.
Quoted 9 lines
If the file was opened with
SHARE_ACCESS_WRITE(no write delegation granted)
and the file access mode was changed to read-only, is it a correct behavior
for the server to allow the READ to go through?Quoted 3 lines
I meant for the WRITE to go through.
Yes:
POSIX permissions enforcement is done at open time, not when doing actual reads and writes. If you open a file on (e.g.) xfs and start streaming writes to it, then you don't expect that you will lose the ability to write to that fd if the permissions change.
In the old v2/3 days of stateless NFS, we had to check permissions on every READ or WRITE operation, but we generally did an open on every RPC too, so it just worked out that we checked permissions on each operation.
Quoted 8 lines
Quoted 5 lines
With v4 we can better approximate POSIX semantics by just associating a stateid with an open file to allow the client to keep writing in this case.
Thanks Jeff,
Don't thank me yet. I went back and looked at the code, and it looks like we still do check permissions on every READ/WRITE (see
nfs4_check_file).
Quoted 6 lines
I'm unclear on whether that's required, but it's probably safest to always check permissions like we are. That does mean that if the mode of the file changes after we open it we could end up being unable to read or write to it (much like with v2/3), but at this point most people are used to that sort of behavior on NFS, so I don't worry about it too much.
It might not conform to Posix permissions enforcement but I like what the server is doing right now, correctness of permissions enforcement and consistent behavior of v2/3/4.
-Dai