Skip to contentSkip to conversation

nfsd: close potential race between open and setting delegation

RFC PATCH In review Jeff Layton 3 participants 14 messages Jul 14, 2022 – Jul 18, 2022

Subsystems (2)
lore
Share & tools
lore

Jeff Layton

author maintainer
#0 lore raw

cover nfsd: close potential race between open and setting delegation

Here's a first stab at a patchset to close a potential race when setting a delegation on a file. Between the point where we open the file and where we set the delegation, another task or client could unlink or rename the dentry. If that occurs, we shouldn't hand out a delegation in the open response, but we don't prevent that today.

The basic idea here is to re-do the lookup after setting the delegation. If the resulting dentry is not the one we have in the open, then we can reject handing out a delegation.

Only lightly tested, so this is an RFC for now.

Jeff Layton (3):
nfsd: drop fh argument from alloc_init_deleg
nfsd: rework arguments to nfs4_set_delegation
nfsd: vet the opened dentry after setting a delegation

 fs/nfsd/nfs4state.c | 65 ++++++++++++++++++++++++++++++++++++++-------
 1 file changed, 55 insertions(+), 10 deletions(-)
Signature · 1 line
2.36.1
author maintainer
#1 lore raw

RFC PATCH 1/3 nfsd: drop fh argument from alloc_init_deleg

Currently, we pass the fh of the opened file down through several functions so that alloc_init_deleg can pass it to delegation_blocked. The filehandle of the open file is available in the nfs4_file struct however, so there's no need to pass it in a separate argument.

Drop the argument from alloc_init_deleg, and follow suit in several of the callers.

---
 fs/nfsd/nfs4state.c | 14 ++++++--------
 1 file changed, 6 insertions(+), 8 deletions(-)
fs/nfsd/nfs4state.c +6 -8
index 994bd11bafe0..4f81c0bbd27b 100644
@@ -1131,7 +1131,6 @@ static void block_delegations(struct knfsd_fh *fh)
 
 static struct nfs4_delegation *
 alloc_init_deleg(struct nfs4_client *clp, struct nfs4_file *fp,
-		 struct svc_fh *current_fh,
 		 struct nfs4_clnt_odstate *odstate)
 {
 	struct nfs4_delegation *dp;
@@ -1141,7 +1140,7 @@ alloc_init_deleg(struct nfs4_client *clp, struct nfs4_file *fp,
 	n = atomic_long_inc_return(&num_delegations);
 	if (n < 0 || n > max_delegations)
 		goto out_dec;
-	if (delegation_blocked(&current_fh->fh_handle))
+	if (delegation_blocked(&fp->fi_fhandle))
 		goto out_dec;
 	dp = delegstateid(nfs4_alloc_stid(clp, deleg_slab, nfs4_free_deleg));
 	if (dp == NULL)
@@ -5261,7 +5260,7 @@ static int nfsd4_check_conflicting_opens(struct nfs4_client *clp,
 }
 
 static struct nfs4_delegation *
-nfs4_set_delegation(struct nfs4_client *clp, struct svc_fh *fh,
+nfs4_set_delegation(struct nfs4_client *clp,
 		    struct nfs4_file *fp, struct nfs4_clnt_odstate *odstate)
 {
 	int status = 0;
@@ -5306,7 +5305,7 @@ nfs4_set_delegation(struct nfs4_client *clp, struct svc_fh *fh,
 		return ERR_PTR(status);
 
 	status = -ENOMEM;
-	dp = alloc_init_deleg(clp, fp, fh, odstate);
+	dp = alloc_init_deleg(clp, fp, odstate);
 	if (!dp)
 		goto out_delegees;
 
@@ -5374,8 +5373,7 @@ static void nfsd4_open_deleg_none_ext(struct nfsd4_open *open, int status)
  * proper support for them.
  */
 static void
-nfs4_open_delegation(struct svc_fh *fh, struct nfsd4_open *open,
-			struct nfs4_ol_stateid *stp)
+nfs4_open_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp)
 {
 	struct nfs4_delegation *dp;
 	struct nfs4_openowner *oo = openowner(stp->st_stateowner);
@@ -5407,7 +5405,7 @@ nfs4_open_delegation(struct svc_fh *fh, struct nfsd4_open *open,
 		default:
 			goto out_no_deleg;
 	}
-	dp = nfs4_set_delegation(clp, fh, stp->st_stid.sc_file, stp->st_clnt_odstate);
+	dp = nfs4_set_delegation(clp, stp->st_stid.sc_file, stp->st_clnt_odstate);
 	if (IS_ERR(dp))
 		goto out_no_deleg;
 
@@ -5539,7 +5537,7 @@ nfsd4_process_open2(struct svc_rqst *rqstp, struct svc_fh *current_fh, struct nf
 	* Attempt to hand out a delegation. No error return, because the
 	* OPEN succeeds even if we fail.
 	*/
-	nfs4_open_delegation(current_fh, open, stp);
+	nfs4_open_delegation(open, stp);
 nodeleg:
 	status = nfs_ok;
 	trace_nfsd_open(&stp->st_stid.sc_stateid);
Signature · 1 line
2.36.1
author maintainer
#2 lore raw

RFC PATCH 2/3 nfsd: rework arguments to nfs4_set_delegation

In reply to Jeff Layton

We'll need the nfs4_open to vet the filename. Change nfs4_set_delegation to take the same arguments are nfs4_open_delegation.

---
 fs/nfsd/nfs4state.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
fs/nfsd/nfs4state.c +5 -3
index 4f81c0bbd27b..347794028c98 100644
@@ -5260,10 +5260,12 @@ static int nfsd4_check_conflicting_opens(struct nfs4_client *clp,
 }
 
 static struct nfs4_delegation *
-nfs4_set_delegation(struct nfs4_client *clp,
-		    struct nfs4_file *fp, struct nfs4_clnt_odstate *odstate)
+nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp)
 {
 	int status = 0;
+	struct nfs4_client *clp = stp->st_stid.sc_client;
+	struct nfs4_file *fp = stp->st_stid.sc_file;
+	struct nfs4_clnt_odstate *odstate = stp->st_clnt_odstate;
 	struct nfs4_delegation *dp;
 	struct nfsd_file *nf;
 	struct file_lock *fl;
@@ -5405,7 +5407,7 @@ nfs4_open_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp)
 		default:
 			goto out_no_deleg;
 	}
-	dp = nfs4_set_delegation(clp, stp->st_stid.sc_file, stp->st_clnt_odstate);
+	dp = nfs4_set_delegation(open, stp);
 	if (IS_ERR(dp))
 		goto out_no_deleg;
 
Signature · 1 line
2.36.1
#3 lore raw

Re: [RFC PATCH 2/3] nfsd: rework arguments to nfs4_set_delegation

Quoted 6 lines · Jeff Layton

We'll need the nfs4_open to vet the filename. Change nfs4_set_delegation to take the same arguments are nfs4_open_delegation.

^are^as

Nit: Considering that in the next patch you change the synopsis of nfs4_open_delegation again but not nfs4_set_delegation, this description causes a little whiplash.

Quoted 36 lines

---
fs/nfsd/nfs4state.c | 8 +++++---

1 file changed, 5 insertions(+), 3 deletions(-)
fs/nfsd/nfs4state.c +0 -0
index 4f81c0bbd27b..347794028c98 100644
@@ -5260,10 +5260,12 @@ static int nfsd4_check_conflicting_opens(struct nfs4_client *clp,
 }
 

static struct nfs4_delegation *

fs/nfsd/nfs4state.c +1 -2
-nfs4_set_delegation(struct nfs4_client *clp,
-		    struct nfs4_file *fp, struct nfs4_clnt_odstate *odstate)
+nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp)
{
	int status = 0;
fs/nfsd/nfs4state.c +3 -0
+	struct nfs4_client *clp = stp->st_stid.sc_client;
+	struct nfs4_file *fp = stp->st_stid.sc_file;
+	struct nfs4_clnt_odstate *odstate = stp->st_clnt_odstate;
	struct nfs4_delegation *dp;
	struct nfsd_file *nf;
	struct file_lock *fl;
fs/nfsd/nfs4state.c +1 -1
@@ -5405,7 +5407,7 @@ nfs4_open_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp)
 		default:
 			goto out_no_deleg;
 	}
-	dp = nfs4_set_delegation(clp, stp->st_stid.sc_file, stp->st_clnt_odstate);
+	dp = nfs4_set_delegation(open, stp);
 	if (IS_ERR(dp))
 		goto out_no_deleg;
 
Signature · 1 line
2.36.1

--
Chuck Lever

author maintainer
#4 lore raw

Re: [RFC PATCH 2/3] nfsd: rework arguments to nfs4_set_delegation

Quoted 13 lines · Chuck Lever III
Quoted 5 lines · Jeff Layton

We'll need the nfs4_open to vet the filename. Change nfs4_set_delegation to take the same arguments are nfs4_open_delegation.

^are^as

Nit: Considering that in the next patch you change the synopsis of nfs4_open_delegation again but not nfs4_set_delegation, this description causes a little whiplash.

Yeah, I should have squashed a couple of those together. I _did_ say it was an RFC. I can resend a cleaned-up version later if you want to take this in.

Quoted 42 lines
Quoted 36 lines

---
fs/nfsd/nfs4state.c | 8 +++++---

1 file changed, 5 insertions(+), 3 deletions(-)
fs/nfsd/nfs4state.c +0 -0
index 4f81c0bbd27b..347794028c98 100644
@@ -5260,10 +5260,12 @@ static int nfsd4_check_conflicting_opens(struct nfs4_client *clp,
 }
 

static struct nfs4_delegation *

fs/nfsd/nfs4state.c +1 -2
-nfs4_set_delegation(struct nfs4_client *clp,
-		    struct nfs4_file *fp, struct nfs4_clnt_odstate *odstate)
+nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp)
{
	int status = 0;
fs/nfsd/nfs4state.c +3 -0
+	struct nfs4_client *clp = stp->st_stid.sc_client;
+	struct nfs4_file *fp = stp->st_stid.sc_file;
+	struct nfs4_clnt_odstate *odstate = stp->st_clnt_odstate;
	struct nfs4_delegation *dp;
	struct nfsd_file *nf;
	struct file_lock *fl;
fs/nfsd/nfs4state.c +1 -1
@@ -5405,7 +5407,7 @@ nfs4_open_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp)
 		default:
 			goto out_no_deleg;
 	}
-	dp = nfs4_set_delegation(clp, stp->st_stid.sc_file, stp->st_clnt_odstate);
+	dp = nfs4_set_delegation(open, stp);
 	if (IS_ERR(dp))
 		goto out_no_deleg;
 
Signature · 1 line
2.36.1

--
Chuck Lever

Signature · 1 line
Jeff Layton <[email protected]>
#5 lore raw

Re: [RFC PATCH 2/3] nfsd: rework arguments to nfs4_set_delegation

Quoted 19 lines · Jeff Layton
Quoted 13 lines · Chuck Lever III
Quoted 5 lines · Jeff Layton

We'll need the nfs4_open to vet the filename. Change nfs4_set_delegation to take the same arguments are nfs4_open_delegation.

^are^as

Nit: Considering that in the next patch you change the synopsis of nfs4_open_delegation again but not nfs4_set_delegation, this description causes a little whiplash.

Quoted 3 lines

Yeah, I should have squashed a couple of those together. I _did_ say it was an RFC. I can resend a cleaned-up version later if you want to take this in.

I'm interested in Neil's thoughts about this approach, but I'm willing to run with it unless test results show a regression.

Quoted 33 lines
Quoted 33 lines
Quoted 33 lines

---
fs/nfsd/nfs4state.c | 8 +++++---

1 file changed, 5 insertions(+), 3 deletions(-)
fs/nfsd/nfs4state.c +0 -0
index 4f81c0bbd27b..347794028c98 100644
@@ -5260,10 +5260,12 @@ static int nfsd4_check_conflicting_opens(struct nfs4_client *clp,
 }
 

static struct nfs4_delegation *

fs/nfsd/nfs4state.c +1 -2
-nfs4_set_delegation(struct nfs4_client *clp,
-		 struct nfs4_file *fp, struct nfs4_clnt_odstate *odstate)
+nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp)
{
	int status = 0;
fs/nfsd/nfs4state.c +3 -0
+	struct nfs4_client *clp = stp->st_stid.sc_client;
+	struct nfs4_file *fp = stp->st_stid.sc_file;
+	struct nfs4_clnt_odstate *odstate = stp->st_clnt_odstate;
	struct nfs4_delegation *dp;
	struct nfsd_file *nf;
	struct file_lock *fl;
fs/nfsd/nfs4state.c +1 -1
@@ -5405,7 +5407,7 @@ nfs4_open_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp)
 		default:
 			goto out_no_deleg;
 	}
-	dp = nfs4_set_delegation(clp, stp->st_stid.sc_file, stp->st_clnt_odstate);
+	dp = nfs4_set_delegation(open, stp);
 	if (IS_ERR(dp))
 		goto out_no_deleg;
 
Quoted 12 lines
Quoted 9 lines
Quoted 3 lines
Signature · 1 line
2.36.1

--
Chuck Lever

Signature · 1 line
Jeff Layton <[email protected]>

--
Chuck Lever

author maintainer
#6 lore raw

Re: [RFC PATCH 2/3] nfsd: rework arguments to nfs4_set_delegation

Quoted 19 lines · Chuck Lever III
Quoted 19 lines · Jeff Layton
Quoted 13 lines · Chuck Lever III
Quoted 5 lines · Jeff Layton

We'll need the nfs4_open to vet the filename. Change nfs4_set_delegation to take the same arguments are nfs4_open_delegation.

^are^as

Nit: Considering that in the next patch you change the synopsis of nfs4_open_delegation again but not nfs4_set_delegation, this description causes a little whiplash.

Quoted 8 lines
Quoted 3 lines

Yeah, I should have squashed a couple of those together. I _did_ say it was an RFC. I can resend a cleaned-up version later if you want to take this in.

I'm interested in Neil's thoughts about this approach, but I'm willing to run with it unless test results show a regression.

...and there is a regression. Partial lockdep pop here:

Jul 14 14:46:54 quad3 kernel: ============================================
Jul 14 14:46:54 quad3 kernel: WARNING: possible recursive locking detected
Jul 14 14:46:54 quad3 kernel: 5.19.0-rc5+ #316 Tainted: G           OE    
Jul 14 14:46:54 quad3 kernel: --------------------------------------------
Jul 14 14:46:54 quad3 kernel: nfsd/1148 is trying to acquire lock:
Jul 14 14:46:54 quad3 kernel: ffff88812a3a7388 (&inode->i_sb->s_type->i_mutex_dir_key/1){++++}-{3:3}, at: nfsd4_process_open2+0x1890/0x2710 [nfsd]
Jul 14 14:46:54 quad3 kernel: 
                              but task is already holding lock:
Jul 14 14:46:54 quad3 kernel: ffff88812a3a7388 (&inode->i_sb->s_type->i_mutex_dir_key/1){++++}-{3:3}, at: nfsd_lookup_dentry+0x16f/0x6a0 [nfsd]
Jul 14 14:46:54 quad3 kernel: 
                              other info that might help us debug this:
Jul 14 14:46:54 quad3 kernel:  Possible unsafe locking scenario:
Jul 14 14:46:54 quad3 kernel:        CPU0
Jul 14 14:46:54 quad3 kernel:        ----
Jul 14 14:46:54 quad3 kernel:   lock(&inode->i_sb->s_type->i_mutex_dir_key/1);
Jul 14 14:46:54 quad3 kernel:   lock(&inode->i_sb->s_type->i_mutex_dir_key/1);
Jul 14 14:46:54 quad3 kernel: 
                               *** DEADLOCK ***
Jul 14 14:46:54 quad3 kernel:  May be due to missing lock nesting notation
Jul 14 14:46:54 quad3 kernel: 1 lock held by nfsd/1148:
Jul 14 14:46:54 quad3 kernel:  #0: ffff88812a3a7388 (&inode->i_sb->s_type->i_mutex_dir_key/1){++++}-{3:3}, at: nfsd_lookup_dentry+0x16f/0x6a0 [nfsd]
Jul 14 14:46:54 quad3 kernel: 

The core problem is the unclear locking in nfsd_lookup_dentry. Sometimes that returns with the i_rwsem held, but there's no clear indication of whether that's the case when the function returns. I guess fh_unlock just takes care of that (which is a little scary, tbqh).

Now that I've taken a stab at it, I don't see how we can fix this w/o taking Neil's locking cleanups series first. I think I'll pull that in and try to redo this series on top of it.

Cheers,

Signature · 1 line
Jeff Layton <[email protected]>
author maintainer
#7 lore raw

RFC PATCH 3/3 nfsd: vet the opened dentry after setting a delegation

In reply to Jeff Layton

Between opening a file and setting a delegation on it, someone could rename or unlink the dentry. If this happens, we do not want to grant a delegation on the open.

Take the i_rwsem before setting the delegation. If we're granted a lease, redo the lookup of the file being opened and validate that the resulting dentry matches the one in the open file description. We only need to do this for the CLAIM_NULL open case however.

---
 fs/nfsd/nfs4state.c | 55 ++++++++++++++++++++++++++++++++++++++++-----
 1 file changed, 50 insertions(+), 5 deletions(-)
fs/nfsd/nfs4state.c +50 -5
index 347794028c98..e5c7f6690d2d 100644
@@ -1172,6 +1172,7 @@ alloc_init_deleg(struct nfs4_client *clp, struct nfs4_file *fp,
 
 void
 nfs4_put_stid(struct nfs4_stid *s)
+	__releases(&clp->cl_lock)
 {
 	struct nfs4_file *fp = s->sc_file;
 	struct nfs4_client *clp = s->sc_client;
@@ -5259,13 +5260,41 @@ static int nfsd4_check_conflicting_opens(struct nfs4_client *clp,
 	return 0;
 }
 
+/*
+ * It's possible that between opening the dentry and setting the delegation,
+ * that it has been renamed or unlinked. Redo the lookup to validate that this
+ * hasn't happened.
+ */
+static int
+nfsd4_vet_deleg_parent(struct nfsd4_open *open, struct nfs4_file *fp, struct dentry *parent)
+{
+	struct dentry *child;
+
+	/* Only need to do this if this is an open-by-name */
+	if (!parent)
+		return 0;
+
+	child = lookup_one_len(open->op_fname, parent, open->op_fnamelen);
+	if (IS_ERR(child))
+		return PTR_ERR(child);
+	dput(child);
+
+	/* Uh-oh, there has been a rename or unlink of the file. No deleg! */
+	if (child != file_dentry(fp->fi_deleg_file->nf_file))
+		return -EAGAIN;
+
+	return 0;
+}
+
 static struct nfs4_delegation *
-nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp)
+nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp,
+		    struct svc_fh *parent_fh)
 {
 	int status = 0;
 	struct nfs4_client *clp = stp->st_stid.sc_client;
 	struct nfs4_file *fp = stp->st_stid.sc_file;
 	struct nfs4_clnt_odstate *odstate = stp->st_clnt_odstate;
+	struct dentry *parent = parent_fh ? parent_fh->fh_dentry : NULL;
 	struct nfs4_delegation *dp;
 	struct nfsd_file *nf;
 	struct file_lock *fl;
@@ -5315,11 +5344,23 @@ nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp)
 	if (!fl)
 		goto out_clnt_odstate;
 
+	if (parent)
+		inode_lock_shared_nested(d_inode(parent), I_MUTEX_PARENT);
 	status = vfs_setlease(fp->fi_deleg_file->nf_file, fl->fl_type, &fl, NULL);
 	if (fl)
 		locks_free_lock(fl);
-	if (status)
+	if (status) {
+		if (parent)
+			inode_unlock_shared(d_inode(parent));
 		goto out_clnt_odstate;
+	}
+
+	status = nfsd4_vet_deleg_parent(open, fp, parent);
+	if (parent)
+		inode_unlock_shared(d_inode(parent));
+	if (status)
+		goto out_unlock;
+
 	status = nfsd4_check_conflicting_opens(clp, fp);
 	if (status)
 		goto out_unlock;
@@ -5375,11 +5416,13 @@ static void nfsd4_open_deleg_none_ext(struct nfsd4_open *open, int status)
  * proper support for them.
  */
 static void
-nfs4_open_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp)
+nfs4_open_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp,
+		     struct svc_fh *current_fh)
 {
 	struct nfs4_delegation *dp;
 	struct nfs4_openowner *oo = openowner(stp->st_stateowner);
 	struct nfs4_client *clp = stp->st_stid.sc_client;
+	struct svc_fh *parent_fh = NULL;
 	int cb_up;
 	int status = 0;
 
@@ -5393,6 +5436,8 @@ nfs4_open_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp)
 				goto out_no_deleg;
 			break;
 		case NFS4_OPEN_CLAIM_NULL:
+			parent_fh = current_fh;
+			fallthrough;
 		case NFS4_OPEN_CLAIM_FH:
 			/*
 			 * Let's not give out any delegations till everyone's
@@ -5407,7 +5452,7 @@ nfs4_open_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp)
 		default:
 			goto out_no_deleg;
 	}
-	dp = nfs4_set_delegation(open, stp);
+	dp = nfs4_set_delegation(open, stp, parent_fh);
 	if (IS_ERR(dp))
 		goto out_no_deleg;
 
@@ -5539,7 +5584,7 @@ nfsd4_process_open2(struct svc_rqst *rqstp, struct svc_fh *current_fh, struct nf
 	* Attempt to hand out a delegation. No error return, because the
 	* OPEN succeeds even if we fail.
 	*/
-	nfs4_open_delegation(open, stp);
+	nfs4_open_delegation(open, stp, &resp->cstate.current_fh);
 nodeleg:
 	status = nfs_ok;
 	trace_nfsd_open(&stp->st_stid.sc_stateid);
Signature · 1 line
2.36.1
#8 lore raw

Re: [RFC PATCH 3/3] nfsd: vet the opened dentry after setting a delegation

Quoted 27 lines · Jeff Layton

Between opening a file and setting a delegation on it, someone could rename or unlink the dentry. If this happens, we do not want to grant a delegation on the open.

Take the i_rwsem before setting the delegation. If we're granted a lease, redo the lookup of the file being opened and validate that the resulting dentry matches the one in the open file description. We only need to do this for the CLAIM_NULL open case however.

---
fs/nfsd/nfs4state.c | 55 ++++++++++++++++++++++++++++++++++++++++-----

1 file changed, 50 insertions(+), 5 deletions(-)
fs/nfsd/nfs4state.c +0 -0
index 347794028c98..e5c7f6690d2d 100644
@@ -1172,6 +1172,7 @@ alloc_init_deleg(struct nfs4_client *clp, struct nfs4_file *fp,
 

void
nfs4_put_stid(struct nfs4_stid *s)

fs/nfsd/nfs4state.c +1 -0
+	__releases(&clp->cl_lock)
Quoted 3 lines
{
	struct nfs4_file *fp = s->sc_file;
	struct nfs4_client *clp = s->sc_client;

This hunk causes a bunch of new sparse warnings:

/home/cel/src/linux/klimt/include/linux/list.h:137:19: warning: context imbalance in 'put_clnt_odstate' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:1174:1: warning: context imbalance in 'nfs4_put_stid' - different lock contexts for basic block /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:1230:13: warning: context imbalance in 'destroy_unhashed_deleg' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:1528:17: warning: context imbalance in 'release_lock_stateid' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:1624:17: warning: context imbalance in 'release_last_closed_stateid' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:2212:22: warning: context imbalance in '__destroy_client' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:4481:17: warning: context imbalance in 'nfsd4_find_and_lock_existing_open' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:4557:25: warning: context imbalance in 'init_open_stateid' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:4606:17: warning: context imbalance in 'move_to_close_lru' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:4752:13: warning: context imbalance in 'nfsd4_cb_recall_release' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:5003:17: warning: context imbalance in 'nfs4_check_deleg' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:5392:9: warning: context imbalance in 'nfs4_set_delegation' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:5467:9: warning: context imbalance in 'nfs4_open_delegation' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:5619:17: warning: context imbalance in 'nfsd4_process_open2' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:5638:17: warning: context imbalance in 'nfsd4_cleanup_open_state' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:5934:27: warning: context imbalance in 'nfs4_laundromat' - different lock contexts for basic block /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:6160:17: warning: context imbalance in 'nfsd4_lookup_stateid' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:6374:25: warning: context imbalance in 'nfs4_preprocess_stateid_op' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:6422:9: warning: context imbalance in 'nfsd4_free_lock_stateid' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:6459:28: warning: context imbalance in 'nfsd4_free_stateid' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:6528:17: warning: context imbalance in 'nfs4_preprocess_seqid_op' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:6545:17: warning: context imbalance in 'nfs4_preprocess_confirmed_seqid_op' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:6588:9: warning: context imbalance in 'nfsd4_open_confirm' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:6657:9: warning: context imbalance in 'nfsd4_open_downgrade' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:6730:9: warning: context imbalance in 'nfsd4_close' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:6762:9: warning: context imbalance in 'nfsd4_delegreturn' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:7034:17: warning: context imbalance in 'init_lock_stateid' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:7063:17: warning: context imbalance in 'find_or_create_lock_stateid' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:7362:17: warning: context imbalance in 'nfsd4_lock' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:7535:9: warning: context imbalance in 'nfsd4_locku' - unexpected unlock

Let's repair the "/home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:1185:9:
warning: context imbalance in 'nfs4_put_stid' - unexpected unlock" message in a separate patch.

Otherwise, this seems reasonable and the surgery is not invasive. Do you have a sense of the overhead of this new check?

Quoted 108 lines
fs/nfsd/nfs4state.c +26 -0
@@ -5259,13 +5260,41 @@ static int nfsd4_check_conflicting_opens(struct nfs4_client *clp,
 	return 0;
 }
 
+/*
+ * It's possible that between opening the dentry and setting the delegation,
+ * that it has been renamed or unlinked. Redo the lookup to validate that this
+ * hasn't happened.
+ */
+static int
+nfsd4_vet_deleg_parent(struct nfsd4_open *open, struct nfs4_file *fp, struct dentry *parent)
+{
+	struct dentry *child;
+
+	/* Only need to do this if this is an open-by-name */
+	if (!parent)
+		return 0;
+
+	child = lookup_one_len(open->op_fname, parent, open->op_fnamelen);
+	if (IS_ERR(child))
+		return PTR_ERR(child);
+	dput(child);
+
+	/* Uh-oh, there has been a rename or unlink of the file. No deleg! */
+	if (child != file_dentry(fp->fi_deleg_file->nf_file))
+		return -EAGAIN;
+
+	return 0;
+}
+

static struct nfs4_delegation *

fs/nfsd/nfs4state.c +2 -1
-nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp)
+nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp,
+		    struct svc_fh *parent_fh)
{
	int status = 0;
	struct nfs4_client *clp = stp->st_stid.sc_client;
	struct nfs4_file *fp = stp->st_stid.sc_file;
	struct nfs4_clnt_odstate *odstate = stp->st_clnt_odstate;
fs/nfsd/nfs4state.c +1 -0
+	struct dentry *parent = parent_fh ? parent_fh->fh_dentry : NULL;
	struct nfs4_delegation *dp;
	struct nfsd_file *nf;
	struct file_lock *fl;
fs/nfsd/nfs4state.c +13 -1
@@ -5315,11 +5344,23 @@ nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp)
 	if (!fl)
 		goto out_clnt_odstate;
 
+	if (parent)
+		inode_lock_shared_nested(d_inode(parent), I_MUTEX_PARENT);
 	status = vfs_setlease(fp->fi_deleg_file->nf_file, fl->fl_type, &fl, NULL);
 	if (fl)
 		locks_free_lock(fl);
-	if (status)
+	if (status) {
+		if (parent)
+			inode_unlock_shared(d_inode(parent));
 		goto out_clnt_odstate;
+	}
+
+	status = nfsd4_vet_deleg_parent(open, fp, parent);
+	if (parent)
+		inode_unlock_shared(d_inode(parent));
+	if (status)
+		goto out_unlock;
+
 	status = nfsd4_check_conflicting_opens(clp, fp);
 	if (status)
 		goto out_unlock;
@@ -5375,11 +5416,13 @@ static void nfsd4_open_deleg_none_ext(struct nfsd4_open *open, int status)
 * proper support for them.
 */

static void

fs/nfsd/nfs4state.c +2 -1
-nfs4_open_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp)
+nfs4_open_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp,
+		     struct svc_fh *current_fh)
{
	struct nfs4_delegation *dp;
	struct nfs4_openowner *oo = openowner(stp->st_stateowner);
	struct nfs4_client *clp = stp->st_stid.sc_client;
fs/nfsd/nfs4state.c +1 -0
+	struct svc_fh *parent_fh = NULL;
	int cb_up;
	int status = 0;
fs/nfsd/nfs4state.c +4 -2
@@ -5393,6 +5436,8 @@ nfs4_open_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp)
 				goto out_no_deleg;
 			break;
 		case NFS4_OPEN_CLAIM_NULL:
+			parent_fh = current_fh;
+			fallthrough;
 		case NFS4_OPEN_CLAIM_FH:
 			/*
 			 * Let's not give out any delegations till everyone's
@@ -5407,7 +5452,7 @@ nfs4_open_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp)
 		default:
 			goto out_no_deleg;
 	}
-	dp = nfs4_set_delegation(open, stp);
+	dp = nfs4_set_delegation(open, stp, parent_fh);
 	if (IS_ERR(dp))
 		goto out_no_deleg;
 
@@ -5539,7 +5584,7 @@ nfsd4_process_open2(struct svc_rqst *rqstp, struct svc_fh *current_fh, struct nf
 	* Attempt to hand out a delegation. No error return, because the
 	* OPEN succeeds even if we fail.
 	*/
-	nfs4_open_delegation(open, stp);
+	nfs4_open_delegation(open, stp, &resp->cstate.current_fh);

nodeleg:

Quoted 5 lines
	status = nfs_ok;
	trace_nfsd_open(&stp->st_stid.sc_stateid);
Signature · 1 line
2.36.1

--
Chuck Lever

author maintainer
#9 lore raw

Re: [RFC PATCH 3/3] nfsd: vet the opened dentry after setting a delegation

Quoted 63 lines · Chuck Lever III
Quoted 26 lines · Jeff Layton

Between opening a file and setting a delegation on it, someone could rename or unlink the dentry. If this happens, we do not want to grant a delegation on the open.

Take the i_rwsem before setting the delegation. If we're granted a lease, redo the lookup of the file being opened and validate that the resulting dentry matches the one in the open file description. We only need to do this for the CLAIM_NULL open case however.

---
fs/nfsd/nfs4state.c | 55 ++++++++++++++++++++++++++++++++++++++++-----

1 file changed, 50 insertions(+), 5 deletions(-)
fs/nfsd/nfs4state.c +0 -0
index 347794028c98..e5c7f6690d2d 100644
@@ -1172,6 +1172,7 @@ alloc_init_deleg(struct nfs4_client *clp, struct nfs4_file *fp,
 

void
nfs4_put_stid(struct nfs4_stid *s)

fs/nfsd/nfs4state.c +1 -0
+	__releases(&clp->cl_lock)
Quoted 3 lines
{
	struct nfs4_file *fp = s->sc_file;
	struct nfs4_client *clp = s->sc_client;

This hunk causes a bunch of new sparse warnings:

/home/cel/src/linux/klimt/include/linux/list.h:137:19: warning: context imbalance in 'put_clnt_odstate' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:1174:1: warning: context imbalance in 'nfs4_put_stid' - different lock contexts for basic block /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:1230:13: warning: context imbalance in 'destroy_unhashed_deleg' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:1528:17: warning: context imbalance in 'release_lock_stateid' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:1624:17: warning: context imbalance in 'release_last_closed_stateid' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:2212:22: warning: context imbalance in '__destroy_client' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:4481:17: warning: context imbalance in 'nfsd4_find_and_lock_existing_open' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:4557:25: warning: context imbalance in 'init_open_stateid' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:4606:17: warning: context imbalance in 'move_to_close_lru' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:4752:13: warning: context imbalance in 'nfsd4_cb_recall_release' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:5003:17: warning: context imbalance in 'nfs4_check_deleg' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:5392:9: warning: context imbalance in 'nfs4_set_delegation' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:5467:9: warning: context imbalance in 'nfs4_open_delegation' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:5619:17: warning: context imbalance in 'nfsd4_process_open2' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:5638:17: warning: context imbalance in 'nfsd4_cleanup_open_state' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:5934:27: warning: context imbalance in 'nfs4_laundromat' - different lock contexts for basic block /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:6160:17: warning: context imbalance in 'nfsd4_lookup_stateid' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:6374:25: warning: context imbalance in 'nfs4_preprocess_stateid_op' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:6422:9: warning: context imbalance in 'nfsd4_free_lock_stateid' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:6459:28: warning: context imbalance in 'nfsd4_free_stateid' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:6528:17: warning: context imbalance in 'nfs4_preprocess_seqid_op' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:6545:17: warning: context imbalance in 'nfs4_preprocess_confirmed_seqid_op' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:6588:9: warning: context imbalance in 'nfsd4_open_confirm' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:6657:9: warning: context imbalance in 'nfsd4_open_downgrade' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:6730:9: warning: context imbalance in 'nfsd4_close' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:6762:9: warning: context imbalance in 'nfsd4_delegreturn' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:7034:17: warning: context imbalance in 'init_lock_stateid' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:7063:17: warning: context imbalance in 'find_or_create_lock_stateid' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:7362:17: warning: context imbalance in 'nfsd4_lock' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:7535:9: warning: context imbalance in 'nfsd4_locku' - unexpected unlock

Quoted 4 lines

Let's repair the "/home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:1185:9:
warning: context imbalance in 'nfs4_put_stid' - unexpected unlock" message in a separate patch.

Yeah, I saw that too after I sent this. That hunk doesn't belong in here. I'll drop it from my local copy.

Quoted 4 lines

Otherwise, this seems reasonable and the surgery is not invasive. Do you have a sense of the overhead of this new check?

Not really...

It's an extra (shared) rwsem acquisition, and a lookup (though that should be in cache in most cases). Neither of those sound too awful on their own, but the open codepath can be sensitive. It's probably heavily workload dependent as well (as is commonly the case). I think we'd need to take it in and do some testing to see if it harms anything.

Quoted 119 lines
Quoted 108 lines
fs/nfsd/nfs4state.c +26 -0
@@ -5259,13 +5260,41 @@ static int nfsd4_check_conflicting_opens(struct nfs4_client *clp,
 	return 0;
 }
 
+/*
+ * It's possible that between opening the dentry and setting the delegation,
+ * that it has been renamed or unlinked. Redo the lookup to validate that this
+ * hasn't happened.
+ */
+static int
+nfsd4_vet_deleg_parent(struct nfsd4_open *open, struct nfs4_file *fp, struct dentry *parent)
+{
+	struct dentry *child;
+
+	/* Only need to do this if this is an open-by-name */
+	if (!parent)
+		return 0;
+
+	child = lookup_one_len(open->op_fname, parent, open->op_fnamelen);
+	if (IS_ERR(child))
+		return PTR_ERR(child);
+	dput(child);
+
+	/* Uh-oh, there has been a rename or unlink of the file. No deleg! */
+	if (child != file_dentry(fp->fi_deleg_file->nf_file))
+		return -EAGAIN;
+
+	return 0;
+}
+

static struct nfs4_delegation *

fs/nfsd/nfs4state.c +2 -1
-nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp)
+nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp,
+		    struct svc_fh *parent_fh)
{
	int status = 0;
	struct nfs4_client *clp = stp->st_stid.sc_client;
	struct nfs4_file *fp = stp->st_stid.sc_file;
	struct nfs4_clnt_odstate *odstate = stp->st_clnt_odstate;
fs/nfsd/nfs4state.c +1 -0
+	struct dentry *parent = parent_fh ? parent_fh->fh_dentry : NULL;
	struct nfs4_delegation *dp;
	struct nfsd_file *nf;
	struct file_lock *fl;
fs/nfsd/nfs4state.c +13 -1
@@ -5315,11 +5344,23 @@ nfs4_set_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp)
 	if (!fl)
 		goto out_clnt_odstate;
 
+	if (parent)
+		inode_lock_shared_nested(d_inode(parent), I_MUTEX_PARENT);
 	status = vfs_setlease(fp->fi_deleg_file->nf_file, fl->fl_type, &fl, NULL);
 	if (fl)
 		locks_free_lock(fl);
-	if (status)
+	if (status) {
+		if (parent)
+			inode_unlock_shared(d_inode(parent));
 		goto out_clnt_odstate;
+	}
+
+	status = nfsd4_vet_deleg_parent(open, fp, parent);
+	if (parent)
+		inode_unlock_shared(d_inode(parent));
+	if (status)
+		goto out_unlock;
+
 	status = nfsd4_check_conflicting_opens(clp, fp);
 	if (status)
 		goto out_unlock;
@@ -5375,11 +5416,13 @@ static void nfsd4_open_deleg_none_ext(struct nfsd4_open *open, int status)
 * proper support for them.
 */

static void

fs/nfsd/nfs4state.c +2 -1
-nfs4_open_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp)
+nfs4_open_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp,
+		     struct svc_fh *current_fh)
{
	struct nfs4_delegation *dp;
	struct nfs4_openowner *oo = openowner(stp->st_stateowner);
	struct nfs4_client *clp = stp->st_stid.sc_client;
fs/nfsd/nfs4state.c +1 -0
+	struct svc_fh *parent_fh = NULL;
	int cb_up;
	int status = 0;
fs/nfsd/nfs4state.c +4 -2
@@ -5393,6 +5436,8 @@ nfs4_open_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp)
 				goto out_no_deleg;
 			break;
 		case NFS4_OPEN_CLAIM_NULL:
+			parent_fh = current_fh;
+			fallthrough;
 		case NFS4_OPEN_CLAIM_FH:
 			/*
 			 * Let's not give out any delegations till everyone's
@@ -5407,7 +5452,7 @@ nfs4_open_delegation(struct nfsd4_open *open, struct nfs4_ol_stateid *stp)
 		default:
 			goto out_no_deleg;
 	}
-	dp = nfs4_set_delegation(open, stp);
+	dp = nfs4_set_delegation(open, stp, parent_fh);
 	if (IS_ERR(dp))
 		goto out_no_deleg;
 
@@ -5539,7 +5584,7 @@ nfsd4_process_open2(struct svc_rqst *rqstp, struct svc_fh *current_fh, struct nf
 	* Attempt to hand out a delegation. No error return, because the
 	* OPEN succeeds even if we fail.
 	*/
-	nfs4_open_delegation(open, stp);
+	nfs4_open_delegation(open, stp, &resp->cstate.current_fh);

nodeleg:

Quoted 5 lines
	status = nfs_ok;
	trace_nfsd_open(&stp->st_stid.sc_stateid);
Signature · 1 line
2.36.1

--
Chuck Lever

Signature · 1 line
Jeff Layton <[email protected]>
#10 lore raw

Re: [RFC PATCH 3/3] nfsd: vet the opened dentry after setting a delegation

Quoted 68 lines · Jeff Layton
Quoted 63 lines · Chuck Lever III
Quoted 26 lines · Jeff Layton

Between opening a file and setting a delegation on it, someone could rename or unlink the dentry. If this happens, we do not want to grant a delegation on the open.

Take the i_rwsem before setting the delegation. If we're granted a lease, redo the lookup of the file being opened and validate that the resulting dentry matches the one in the open file description. We only need to do this for the CLAIM_NULL open case however.

---
fs/nfsd/nfs4state.c | 55 ++++++++++++++++++++++++++++++++++++++++-----

1 file changed, 50 insertions(+), 5 deletions(-)
fs/nfsd/nfs4state.c +0 -0
index 347794028c98..e5c7f6690d2d 100644
@@ -1172,6 +1172,7 @@ alloc_init_deleg(struct nfs4_client *clp, struct nfs4_file *fp,
 

void
nfs4_put_stid(struct nfs4_stid *s)

fs/nfsd/nfs4state.c +1 -0
+	__releases(&clp->cl_lock)
Quoted 3 lines
{
	struct nfs4_file *fp = s->sc_file;
	struct nfs4_client *clp = s->sc_client;

This hunk causes a bunch of new sparse warnings:

/home/cel/src/linux/klimt/include/linux/list.h:137:19: warning: context imbalance in 'put_clnt_odstate' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:1174:1: warning: context imbalance in 'nfs4_put_stid' - different lock contexts for basic block /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:1230:13: warning: context imbalance in 'destroy_unhashed_deleg' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:1528:17: warning: context imbalance in 'release_lock_stateid' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:1624:17: warning: context imbalance in 'release_last_closed_stateid' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:2212:22: warning: context imbalance in '__destroy_client' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:4481:17: warning: context imbalance in 'nfsd4_find_and_lock_existing_open' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:4557:25: warning: context imbalance in 'init_open_stateid' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:4606:17: warning: context imbalance in 'move_to_close_lru' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:4752:13: warning: context imbalance in 'nfsd4_cb_recall_release' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:5003:17: warning: context imbalance in 'nfs4_check_deleg' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:5392:9: warning: context imbalance in 'nfs4_set_delegation' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:5467:9: warning: context imbalance in 'nfs4_open_delegation' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:5619:17: warning: context imbalance in 'nfsd4_process_open2' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:5638:17: warning: context imbalance in 'nfsd4_cleanup_open_state' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:5934:27: warning: context imbalance in 'nfs4_laundromat' - different lock contexts for basic block /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:6160:17: warning: context imbalance in 'nfsd4_lookup_stateid' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:6374:25: warning: context imbalance in 'nfs4_preprocess_stateid_op' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:6422:9: warning: context imbalance in 'nfsd4_free_lock_stateid' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:6459:28: warning: context imbalance in 'nfsd4_free_stateid' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:6528:17: warning: context imbalance in 'nfs4_preprocess_seqid_op' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:6545:17: warning: context imbalance in 'nfs4_preprocess_confirmed_seqid_op' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:6588:9: warning: context imbalance in 'nfsd4_open_confirm' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:6657:9: warning: context imbalance in 'nfsd4_open_downgrade' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:6730:9: warning: context imbalance in 'nfsd4_close' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:6762:9: warning: context imbalance in 'nfsd4_delegreturn' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:7034:17: warning: context imbalance in 'init_lock_stateid' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:7063:17: warning: context imbalance in 'find_or_create_lock_stateid' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:7362:17: warning: context imbalance in 'nfsd4_lock' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:7535:9: warning: context imbalance in 'nfsd4_locku' - unexpected unlock

Quoted 8 lines
Quoted 4 lines

Let's repair the "/home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:1185:9:
warning: context imbalance in 'nfs4_put_stid' - unexpected unlock" message in a separate patch.

Yeah, I saw that too after I sent this. That hunk doesn't belong in here. I'll drop it from my local copy.

Well, I'm interested in trying to get rid of the existing sparse warnings too, since those tend to block our ability to see new warnings that arise.

If you have suggestions or proposals, please send them!

--
Chuck Lever

author maintainer
#11 lore raw

Re: [RFC PATCH 3/3] nfsd: vet the opened dentry after setting a delegation

Quoted 73 lines · Chuck Lever III
Quoted 73 lines · Jeff Layton
Quoted 67 lines · Chuck Lever III
Quoted 26 lines · Jeff Layton

Between opening a file and setting a delegation on it, someone could rename or unlink the dentry. If this happens, we do not want to grant a delegation on the open.

Take the i_rwsem before setting the delegation. If we're granted a lease, redo the lookup of the file being opened and validate that the resulting dentry matches the one in the open file description. We only need to do this for the CLAIM_NULL open case however.

---
fs/nfsd/nfs4state.c | 55 ++++++++++++++++++++++++++++++++++++++++-----

1 file changed, 50 insertions(+), 5 deletions(-)
fs/nfsd/nfs4state.c +0 -0
index 347794028c98..e5c7f6690d2d 100644
@@ -1172,6 +1172,7 @@ alloc_init_deleg(struct nfs4_client *clp, struct nfs4_file *fp,
 

void
nfs4_put_stid(struct nfs4_stid *s)

fs/nfsd/nfs4state.c +1 -0
+	__releases(&clp->cl_lock)
Quoted 3 lines
{
	struct nfs4_file *fp = s->sc_file;
	struct nfs4_client *clp = s->sc_client;

This hunk causes a bunch of new sparse warnings:

/home/cel/src/linux/klimt/include/linux/list.h:137:19: warning: context imbalance in 'put_clnt_odstate' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:1174:1: warning: context imbalance in 'nfs4_put_stid' - different lock contexts for basic block /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:1230:13: warning: context imbalance in 'destroy_unhashed_deleg' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:1528:17: warning: context imbalance in 'release_lock_stateid' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:1624:17: warning: context imbalance in 'release_last_closed_stateid' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:2212:22: warning: context imbalance in '__destroy_client' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:4481:17: warning: context imbalance in 'nfsd4_find_and_lock_existing_open' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:4557:25: warning: context imbalance in 'init_open_stateid' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:4606:17: warning: context imbalance in 'move_to_close_lru' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:4752:13: warning: context imbalance in 'nfsd4_cb_recall_release' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:5003:17: warning: context imbalance in 'nfs4_check_deleg' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:5392:9: warning: context imbalance in 'nfs4_set_delegation' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:5467:9: warning: context imbalance in 'nfs4_open_delegation' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:5619:17: warning: context imbalance in 'nfsd4_process_open2' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:5638:17: warning: context imbalance in 'nfsd4_cleanup_open_state' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:5934:27: warning: context imbalance in 'nfs4_laundromat' - different lock contexts for basic block /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:6160:17: warning: context imbalance in 'nfsd4_lookup_stateid' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:6374:25: warning: context imbalance in 'nfs4_preprocess_stateid_op' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:6422:9: warning: context imbalance in 'nfsd4_free_lock_stateid' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:6459:28: warning: context imbalance in 'nfsd4_free_stateid' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:6528:17: warning: context imbalance in 'nfs4_preprocess_seqid_op' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:6545:17: warning: context imbalance in 'nfs4_preprocess_confirmed_seqid_op' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:6588:9: warning: context imbalance in 'nfsd4_open_confirm' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:6657:9: warning: context imbalance in 'nfsd4_open_downgrade' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:6730:9: warning: context imbalance in 'nfsd4_close' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:6762:9: warning: context imbalance in 'nfsd4_delegreturn' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:7034:17: warning: context imbalance in 'init_lock_stateid' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:7063:17: warning: context imbalance in 'find_or_create_lock_stateid' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:7362:17: warning: context imbalance in 'nfsd4_lock' - unexpected unlock /home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:7535:9: warning: context imbalance in 'nfsd4_locku' - unexpected unlock

Let's repair the "/home/cel/src/linux/klimt/fs/nfsd/nfs4state.c:1185:9:
warning: context imbalance in 'nfs4_put_stid' - unexpected unlock" message in a separate patch.

Quoted 9 lines
Quoted 2 lines

Yeah, I saw that too after I sent this. That hunk doesn't belong in here. I'll drop it from my local copy.

Well, I'm interested in trying to get rid of the existing sparse warnings too, since those tend to block our ability to see new warnings that arise.

If you have suggestions or proposals, please send them!

We should definitely annotate these functions that have unbalanced locking like this one. That's the usual way to silence these sorts of warnings. I'm hoping Neil's patches will make it easier to address.

Signature · 1 line
Jeff Layton <[email protected]>
#12 lore raw
In reply to Jeff Layton
Quoted 9 lines · Jeff Layton

Here's a first stab at a patchset to close a potential race when setting a delegation on a file. Between the point where we open the file and where we set the delegation, another task or client could unlink or rename the dentry. If that occurs, we shouldn't hand out a delegation in the open response, but we don't prevent that today.

The basic idea here is to re-do the lookup after setting the delegation. If the resulting dentry is not the one we have in the open, then we can reject handing out a delegation.

I have this distinct memory of actually doing that before.

But looking through the git history all I find is 4335723e8e9f "nfsd4:
fix delegation-unlink/rename race", from 2014, which claims to fix a similar-sounding race in a different way.

How are you reproducing this?

--b.

Quoted 12 lines

Only lightly tested, so this is an RFC for now.

Jeff Layton (3): nfsd: drop fh argument from alloc_init_deleg nfsd:
rework arguments to nfs4_set_delegation nfsd: vet the opened dentry after setting a delegation

 fs/nfsd/nfs4state.c | 65

++++++++++++++++++++++++++++++++++++++------- 1 file changed, 55 insertions(+), 10 deletions(-)

-- 2.36.1

author maintainer
#13 lore raw
Quoted 13 lines · J. Bruce Fields
Quoted 9 lines · Jeff Layton

Here's a first stab at a patchset to close a potential race when setting a delegation on a file. Between the point where we open the file and where we set the delegation, another task or client could unlink or rename the dentry. If that occurs, we shouldn't hand out a delegation in the open response, but we don't prevent that today.

The basic idea here is to re-do the lookup after setting the delegation. If the resulting dentry is not the one we have in the open, then we can reject handing out a delegation.

I have this distinct memory of actually doing that before.

Quoted 8 lines

But looking through the git history all I find is 4335723e8e9f "nfsd4:
fix delegation-unlink/rename race", from 2014, which claims to fix a similar-sounding race in a different way.

How are you reproducing this?

--b.

I'm not at all, so far. This race is entirely theoretical. Probably we could reproduce it by introducing some artificial delays or something.

Quoted 12 lines
Quoted 12 lines

Only lightly tested, so this is an RFC for now.

Jeff Layton (3): nfsd: drop fh argument from alloc_init_deleg nfsd:
rework arguments to nfs4_set_delegation nfsd: vet the opened dentry after setting a delegation

 fs/nfsd/nfs4state.c | 65

++++++++++++++++++++++++++++++++++++++------- 1 file changed, 55 insertions(+), 10 deletions(-)

-- 2.36.1

Signature · 1 line
Jeff Layton <[email protected]>