Skip to contentSkip to conversation

delegation state fixes and cleanups for 3.13

PATCH In review Benny Halevy 2 participants 21 messages Oct 29, 2013 – Oct 30, 2013

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover delegation state fixes and cleanups for 3.13

commit c7342d3cf5d5677063cbc6341634ed8cea1b176f

Author: Benny Halevy <[email protected]>
Date:   Tue Oct 15 17:07:31 2013 +0300

nfsd4: fix recall_lock use in unhash_delegation

Access to dp->dl_perclnt must be synchronized by the recall_lock

commit 87174785bcec37371b718757af67dde0f6d083e3

Author: Benny Halevy <[email protected]>
Date:   Tue Oct 15 17:29:13 2013 +0300

nfsd4: need to destroy revoked delegations in destroy_client

[use list_splice_init]

commit 775a7353f9d1e60ebd9096b1a82bfb78fe7f3cc9

Author: Benny Halevy <[email protected]>
Date:   Tue Oct 15 17:41:53 2013 +0300

nfsd4: properly hash delegation in nfs4_setlease

keep all accesses via dl_perclnt under the recall_lock.

commit fdf586869a6b026893f818cb538574a7b77f5604

Author: Benny Halevy <[email protected]>
Date:   Tue Oct 15 21:08:48 2013 +0300

nfsd4: hash deleg stateid only on successful nfs4_set_delegation

We don't want the stateid to be found in the hash table before the delegation is granted.

commit 635a93446fde8d8fb1f4bb825926750d9b6aea9d

Author: Benny Halevy <[email protected]>
Date:   Tue Oct 15 23:46:55 2013 +0300

nfsd4: cleanup hash and unhash delegation

make sure all atomic ops are under the recall_lock

commit 39f26e5974f73b62144abde6793d41b07a32719a

Author: Benny Halevy <[email protected]>
Date:   Tue Oct 15 23:50:30 2013 +0300

nfsd4: move nfs4_put_file from unhash to put delegation

revoked delegations are unhashed but are kept around in nfsv4.1 and we better
hang on to dp_file in this case.

commit cd2b2cac09cbd45419ce7d0e354dd94537a4fbfb

Author: Benny Halevy <[email protected]>
Date:   Wed Oct 16 00:17:14 2013 +0300

nfsd4: hold recall_lock while accessing nfs4_delegation.dl_recall_lru

#1 lore raw

PATCH 1/7 nfsd4: fix recall_lock use in unhash_delegation

Access to dp->dl_perclnt must be synchronized by the recall_lock

---
 fs/nfsd/nfs4state.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
fs/nfsd/nfs4state.c +1 -1
index a90949a..a403502 100644
@@ -436,8 +436,8 @@ static void unhash_stid(struct nfs4_stid *s)
 static void
 unhash_delegation(struct nfs4_delegation *dp)
 {
-	list_del_init(&dp->dl_perclnt);
 	spin_lock(&recall_lock);
+	list_del_init(&dp->dl_perclnt);
 	list_del_init(&dp->dl_perfile);
 	list_del_init(&dp->dl_recall_lru);
 	spin_unlock(&recall_lock);
Signature · 1 line
1.8.3.1
#2 lore raw

Re: [PATCH 1/7] nfsd4: fix recall_lock use in unhash_delegation

Quoted 1 line · Benny Halevy

Access to dp->dl_perclnt must be synchronized by the recall_lock

Are you sure? recall_lock is for stuff that's needed in the delegation break callback (nfsd_break_deleg_cb() and any of the subsequent callback handling). I don't think that includes dl_perclnt.

--b.

Quoted 27 lines
---
 fs/nfsd/nfs4state.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
fs/nfsd/nfs4state.c +1 -1
index a90949a..a403502 100644
@@ -436,8 +436,8 @@ static void unhash_stid(struct nfs4_stid *s)
 static void
 unhash_delegation(struct nfs4_delegation *dp)
 {
-	list_del_init(&dp->dl_perclnt);
 	spin_lock(&recall_lock);
+	list_del_init(&dp->dl_perclnt);
 	list_del_init(&dp->dl_perfile);
 	list_del_init(&dp->dl_recall_lru);
 	spin_unlock(&recall_lock);
Signature · 6 lines
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#3 lore raw

Re: [PATCH 1/7] nfsd4: fix recall_lock use in unhash_delegation

Quoted 6 lines · J. Bruce Fields
Quoted 1 line · Benny Halevy

Access to dp->dl_perclnt must be synchronized by the recall_lock

Are you sure? recall_lock is for stuff that's needed in the delegation break callback (nfsd_break_deleg_cb() and any of the subsequent callback handling). I don't think that includes dl_perclnt.

I was mislead by the fact that destroy_client and nfs4_state_shutdown_net care to acquire the recall_lock for traversing the clp->cl_delegations and nn->del_recall_lru lists, respectively.

Now nfs4_state_shutdown_net, although its comment says it should be called with the state lock held (SHOULD or should, or should it be MUST? :) It doesn't seem like nfsd_shutdown_net acquires the state lock.

Therefore, we either need to grab the state lock in nfs4_state_shutdown_net which adds yet another problem to fix, or access dl_perclnt always under the spin lock.

Benny

Quoted 35 lines

--b.

Quoted 27 lines
---
 fs/nfsd/nfs4state.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
fs/nfsd/nfs4state.c +1 -1
index a90949a..a403502 100644
@@ -436,8 +436,8 @@ static void unhash_stid(struct nfs4_stid *s)
 static void
 unhash_delegation(struct nfs4_delegation *dp)
 {
-	list_del_init(&dp->dl_perclnt);
 	spin_lock(&recall_lock);
+	list_del_init(&dp->dl_perclnt);
 	list_del_init(&dp->dl_perfile);
 	list_del_init(&dp->dl_recall_lru);
 	spin_unlock(&recall_lock);
Signature · 6 lines
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#4 lore raw

Re: [PATCH 1/7] nfsd4: fix recall_lock use in unhash_delegation

Quoted 8 lines · Benny Halevy
Quoted 6 lines · J. Bruce Fields
Quoted 1 line · Benny Halevy

Access to dp->dl_perclnt must be synchronized by the recall_lock

Are you sure? recall_lock is for stuff that's needed in the delegation break callback (nfsd_break_deleg_cb() and any of the subsequent callback handling). I don't think that includes dl_perclnt.

Quoted 7 lines

I was mislead by the fact that destroy_client and nfs4_state_shutdown_net care to acquire the recall_lock for traversing the clp->cl_delegations and nn->del_recall_lru lists, respectively.

Now nfs4_state_shutdown_net, although its comment says it should be called with the state lock held (SHOULD or should, or should it be MUST? :) It doesn't seem like nfsd_shutdown_net acquires the state lock.

Good point.

Quoted 2 lines

Therefore, we either need to grab the state lock in nfs4_state_shutdown_net which adds yet another problem to fix,

That sounds simplest as the short-term fix.... What problem do you see?

--b.

commit 2c9407ae694dad8a1ad6394d9bf6077024b9fae7

Author: J. Bruce Fields <[email protected]>
Date:   Wed Oct 30 10:33:09 2013 -0400

nfsd4: nfsd_shutdown_net needs state lock

A comment claims the caller should take it, but that's not being done. Note we don't want it around the cancel_delayed_work_sync since that may wait on work which holds the client lock.

fs/nfsd/nfs4state.c +2 -1
index 21eb678..e03e8ef 100644
@@ -5124,7 +5124,6 @@ out_recovery:
 	return ret;
 }
 
-/* should be called with the state lock held */
 void
 nfs4_state_shutdown_net(struct net *net)
 {
@@ -5135,6 +5134,7 @@ nfs4_state_shutdown_net(struct net *net)
 	cancel_delayed_work_sync(&nn->laundromat_work);
 	locks_end_grace(&nn->nfsd4_manager);
 
+	nfs4_lock_state();
 	INIT_LIST_HEAD(&reaplist);
 	spin_lock(&recall_lock);
 	list_for_each_safe(pos, next, &nn->del_recall_lru) {
@@ -5149,6 +5149,7 @@ nfs4_state_shutdown_net(struct net *net)
 
 	nfsd4_client_tracking_exit(net);
 	nfs4_state_destroy_net(net);
+	nfs4_unlock_state();
 }
 
 void
#5 lore raw

Re: [PATCH 1/7] nfsd4: fix recall_lock use in unhash_delegation

Quoted 17 lines · J. Bruce Fields
Quoted 8 lines · Benny Halevy
Quoted 6 lines · J. Bruce Fields
Quoted 1 line · Benny Halevy

Access to dp->dl_perclnt must be synchronized by the recall_lock

Are you sure? recall_lock is for stuff that's needed in the delegation break callback (nfsd_break_deleg_cb() and any of the subsequent callback handling). I don't think that includes dl_perclnt.

Quoted 7 lines

I was mislead by the fact that destroy_client and nfs4_state_shutdown_net care to acquire the recall_lock for traversing the clp->cl_delegations and nn->del_recall_lru lists, respectively.

Now nfs4_state_shutdown_net, although its comment says it should be called with the state lock held (SHOULD or should, or should it be MUST? :) It doesn't seem like nfsd_shutdown_net acquires the state lock.

Quoted 6 lines

Good point.

Quoted 2 lines

Therefore, we either need to grab the state lock in nfs4_state_shutdown_net which adds yet another problem to fix,

That sounds simplest as the short-term fix.... What problem do you see?

Nothing specific, just the general use of the global state lock...

Quoted 7 lines

--b.

commit 2c9407ae694dad8a1ad6394d9bf6077024b9fae7

Author: J. Bruce Fields <[email protected]>
Date:   Wed Oct 30 10:33:09 2013 -0400
Quoted 7 lines

nfsd4: nfsd_shutdown_net needs state lock

A comment claims the caller should take it, but that's not being done. Note we don't want it around the cancel_delayed_work_sync since that may wait on work which holds the client lock.

Ack

Quoted 34 lines
fs/nfsd/nfs4state.c +2 -2
index 21eb678..e03e8ef 100644
@@ -5124,7 +5124,6 @@ out_recovery:
 	return ret;
 }
 
-/* should be called with the state lock held */
 void
 nfs4_state_shutdown_net(struct net *net)
 {
@@ -5135,6 +5134,7 @@ nfs4_state_shutdown_net(struct net *net)
 	cancel_delayed_work_sync(&nn->laundromat_work);
 	locks_end_grace(&nn->nfsd4_manager);
 
+	nfs4_lock_state();
 	INIT_LIST_HEAD(&reaplist);
 	spin_lock(&recall_lock);
 	list_for_each_safe(pos, next, &nn->del_recall_lru) {
@@ -5149,6 +5149,7 @@ nfs4_state_shutdown_net(struct net *net)
 
 	nfsd4_client_tracking_exit(net);
 	nfs4_state_destroy_net(net);
+	nfs4_unlock_state();
 }
 
 void
--

To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#6 lore raw

Re: [PATCH 1/7] nfsd4: fix recall_lock use in unhash_delegation

Quoted 36 lines · Benny Halevy
Quoted 17 lines · J. Bruce Fields
Quoted 8 lines · Benny Halevy
Quoted 6 lines · J. Bruce Fields
Quoted 1 line · Benny Halevy

Access to dp->dl_perclnt must be synchronized by the recall_lock

Are you sure? recall_lock is for stuff that's needed in the delegation break callback (nfsd_break_deleg_cb() and any of the subsequent callback handling). I don't think that includes dl_perclnt.

Quoted 7 lines

I was mislead by the fact that destroy_client and nfs4_state_shutdown_net care to acquire the recall_lock for traversing the clp->cl_delegations and nn->del_recall_lru lists, respectively.

Now nfs4_state_shutdown_net, although its comment says it should be called with the state lock held (SHOULD or should, or should it be MUST? :) It doesn't seem like nfsd_shutdown_net acquires the state lock.

Quoted 6 lines

Good point.

Quoted 2 lines

Therefore, we either need to grab the state lock in nfs4_state_shutdown_net which adds yet another problem to fix,

That sounds simplest as the short-term fix.... What problem do you see?

Nothing specific, just the general use of the global state lock...

Quoted 9 lines

--b.

commit 2c9407ae694dad8a1ad6394d9bf6077024b9fae7

Author: J. Bruce Fields <[email protected]>
Date:   Wed Oct 30 10:33:09 2013 -0400

nfsd4: nfsd_shutdown_net needs state lock

Quoted 7 lines
Quoted 5 lines

A comment claims the caller should take it, but that's not being done. Note we don't want it around the cancel_delayed_work_sync since that may wait on work which holds the client lock.

Ack

Thanks, done, and pushed out to

	git://linux-nfs.org/~bfields/linux.git for-3.13

--b.

Quoted 35 lines
Quoted 35 lines
fs/nfsd/nfs4state.c +2 -2
index 21eb678..e03e8ef 100644
@@ -5124,7 +5124,6 @@ out_recovery:
 	return ret;
 }
 
-/* should be called with the state lock held */
 void
 nfs4_state_shutdown_net(struct net *net)
 {
@@ -5135,6 +5134,7 @@ nfs4_state_shutdown_net(struct net *net)
 	cancel_delayed_work_sync(&nn->laundromat_work);
 	locks_end_grace(&nn->nfsd4_manager);
 
+	nfs4_lock_state();
 	INIT_LIST_HEAD(&reaplist);
 	spin_lock(&recall_lock);
 	list_for_each_safe(pos, next, &nn->del_recall_lru) {
@@ -5149,6 +5149,7 @@ nfs4_state_shutdown_net(struct net *net)
 
 	nfsd4_client_tracking_exit(net);
 	nfs4_state_destroy_net(net);
+	nfs4_unlock_state();
 }
 
 void
--

To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#7 lore raw

PATCH 2/7 nfsd4: need to destroy revoked delegations in destroy_client

In reply to Benny Halevy
[use list_splice_init]
---
 fs/nfsd/nfs4state.c | 7 +++++++
 1 file changed, 7 insertions(+)
fs/nfsd/nfs4state.c +7 -0
index a403502..a66b0ad 100644
@@ -1129,6 +1129,13 @@ static struct nfs4_client *alloc_client(struct xdr_netobj name)
 		dp = list_entry(reaplist.next, struct nfs4_delegation, dl_recall_lru);
 		destroy_delegation(dp);
 	}
+	spin_lock(&recall_lock);
+	list_splice_init(&clp->cl_revoked, &reaplist);
+	spin_unlock(&recall_lock);
+	while (!list_empty(&reaplist)) {
+		dp = list_entry(reaplist.next, struct nfs4_delegation, dl_recall_lru);
+		destroy_revoked_delegation(dp);
+	}
 	while (!list_empty(&clp->cl_openowners)) {
 		oo = list_entry(clp->cl_openowners.next, struct nfs4_openowner, oo_perclient);
 		release_openowner(oo);
Signature · 1 line
1.8.3.1
#8 lore raw

Re: [PATCH 2/7] nfsd4: need to destroy revoked delegations in destroy_client

Whoops, yes, looks like a good fix.

I'm not convinced of the need for the recall_lock here for reasons given before. Could you just drop the recall_lock here and resend?

--b.

Quoted 31 lines · Benny Halevy
[use list_splice_init]
---
 fs/nfsd/nfs4state.c | 7 +++++++
 1 file changed, 7 insertions(+)
fs/nfsd/nfs4state.c +7 -0
index a403502..a66b0ad 100644
@@ -1129,6 +1129,13 @@ static struct nfs4_client *alloc_client(struct xdr_netobj name)
 		dp = list_entry(reaplist.next, struct nfs4_delegation, dl_recall_lru);
 		destroy_delegation(dp);
 	}
+	spin_lock(&recall_lock);
+	list_splice_init(&clp->cl_revoked, &reaplist);
+	spin_unlock(&recall_lock);
+	while (!list_empty(&reaplist)) {
+		dp = list_entry(reaplist.next, struct nfs4_delegation, dl_recall_lru);
+		destroy_revoked_delegation(dp);
+	}
 	while (!list_empty(&clp->cl_openowners)) {
 		oo = list_entry(clp->cl_openowners.next, struct nfs4_openowner, oo_perclient);
 		release_openowner(oo);
Signature · 6 lines
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#9 lore raw

Re: [PATCH 2/7] nfsd4: need to destroy revoked delegations in destroy_client

Quoted 4 lines · bfields

Whoops, yes, looks like a good fix.

I'm not convinced of the need for the recall_lock here for reasons given before. Could you just drop the recall_lock here and resend?

Actually never mind I've done that on my local tree. I'll wait to push it out till you've had a chance to object.

--b.

Quoted 35 lines

--b.

Quoted 31 lines · Benny Halevy
[use list_splice_init]
---
 fs/nfsd/nfs4state.c | 7 +++++++
 1 file changed, 7 insertions(+)
fs/nfsd/nfs4state.c +7 -0
index a403502..a66b0ad 100644
@@ -1129,6 +1129,13 @@ static struct nfs4_client *alloc_client(struct xdr_netobj name)
 		dp = list_entry(reaplist.next, struct nfs4_delegation, dl_recall_lru);
 		destroy_delegation(dp);
 	}
+	spin_lock(&recall_lock);
+	list_splice_init(&clp->cl_revoked, &reaplist);
+	spin_unlock(&recall_lock);
+	while (!list_empty(&reaplist)) {
+		dp = list_entry(reaplist.next, struct nfs4_delegation, dl_recall_lru);
+		destroy_revoked_delegation(dp);
+	}
 	while (!list_empty(&clp->cl_openowners)) {
 		oo = list_entry(clp->cl_openowners.next, struct nfs4_openowner, oo_perclient);
 		release_openowner(oo);
Signature · 6 lines
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#10 lore raw

Re: [PATCH 2/7] nfsd4: need to destroy revoked delegations in destroy_client

Quoted 8 lines · J. Bruce Fields
Quoted 4 lines · bfields

Whoops, yes, looks like a good fix.

I'm not convinced of the need for the recall_lock here for reasons given before. Could you just drop the recall_lock here and resend?

Actually never mind I've done that on my local tree. I'll wait to push it out till you've had a chance to object.

no objection. thanks.

Quoted 43 lines

--b.

Quoted 31 lines

--b.

Quoted 27 lines · Benny Halevy
[use list_splice_init]
---
 fs/nfsd/nfs4state.c | 7 +++++++
 1 file changed, 7 insertions(+)
fs/nfsd/nfs4state.c +7 -0
index a403502..a66b0ad 100644
@@ -1129,6 +1129,13 @@ static struct nfs4_client *alloc_client(struct xdr_netobj name)
 		dp = list_entry(reaplist.next, struct nfs4_delegation, dl_recall_lru);
 		destroy_delegation(dp);
 	}
+	spin_lock(&recall_lock);
+	list_splice_init(&clp->cl_revoked, &reaplist);
+	spin_unlock(&recall_lock);
+	while (!list_empty(&reaplist)) {
+		dp = list_entry(reaplist.next, struct nfs4_delegation, dl_recall_lru);
+		destroy_revoked_delegation(dp);
+	}
 	while (!list_empty(&clp->cl_openowners)) {
 		oo = list_entry(clp->cl_openowners.next, struct nfs4_openowner, oo_perclient);
 		release_openowner(oo);
Signature · 1 line
1.8.3.1
Quoted 4 lines
Quoted 4 lines

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#11 lore raw

PATCH 3/7 nfsd4: properly hash delegation in nfs4_setlease

In reply to Benny Halevy

keep all accesses via dl_perclnt under the recall_lock.

---
 fs/nfsd/nfs4state.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)
fs/nfsd/nfs4state.c +12 -5
index a66b0ad..405649f 100644
@@ -432,6 +432,13 @@ static void unhash_stid(struct nfs4_stid *s)
 	s->sc_type = 0;
 }
 
+static void
+hash_delegation_locked(struct nfs4_delegation *dp, struct nfs4_file *fp)
+{
+	list_add(&dp->dl_perfile, &fp->fi_delegations);
+	list_add(&dp->dl_perclnt, &dp->dl_stid.sc_client->cl_delegations);
+}
+
 /* Called under the state lock. */
 static void
 unhash_delegation(struct nfs4_delegation *dp)
@@ -3035,17 +3042,17 @@ static int nfs4_setlease(struct nfs4_delegation *dp)
 	if (!fl)
 		return -ENOMEM;
 	fl->fl_file = find_readable_file(fp);
-	list_add(&dp->dl_perclnt, &dp->dl_stid.sc_client->cl_delegations);
 	status = vfs_setlease(fl->fl_file, fl->fl_type, &fl);
 	if (status) {
-		list_del_init(&dp->dl_perclnt);
 		locks_free_lock(fl);
 		return status;
 	}
 	fp->fi_lease = fl;
 	fp->fi_deleg_file = get_file(fl->fl_file);
 	atomic_set(&fp->fi_delegees, 1);
-	list_add(&dp->dl_perfile, &fp->fi_delegations);
+	spin_lock(&recall_lock);
+	hash_delegation_locked(dp, fp);
+	spin_unlock(&recall_lock);
 	return 0;
 }
 
@@ -3070,9 +3077,8 @@ static int nfs4_set_delegation(struct nfs4_delegation *dp, struct nfs4_file *fp)
 		goto out_free;
 	}
 	atomic_inc(&fp->fi_delegees);
-	list_add(&dp->dl_perfile, &fp->fi_delegations);
+	hash_delegation_locked(dp, fp);
 	spin_unlock(&recall_lock);
-	list_add(&dp->dl_perclnt, &dp->dl_stid.sc_client->cl_delegations);
 	return 0;
 out_free:
 	put_nfs4_file(fp);
@@ -4871,6 +4877,7 @@ static u64 nfsd_find_all_delegations(struct nfs4_client *clp, u64 max,
 	struct nfs4_delegation *dp, *next;
 	u64 count = 0;
 
+	lockdep_assert_held(&recall_lock);
 	list_for_each_entry_safe(dp, next, &clp->cl_delegations, dl_perclnt) {
 		if (victims)
 			list_move(&dp->dl_recall_lru, victims);
Signature · 1 line
1.8.3.1
#12 lore raw

PATCH 4/7 nfsd4: hash deleg stateid only on successful nfs4_set_delegation

In reply to Benny Halevy

We don't want the stateid to be found in the hash table before the delegation is granted.

---
 fs/nfsd/nfs4state.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
fs/nfsd/nfs4state.c +1 -1
index 405649f..64fcdd6 100644
@@ -375,7 +375,6 @@ static struct nfs4_ol_stateid * nfs4_alloc_stateid(struct nfs4_client *clp)
 	dp = delegstateid(nfs4_alloc_stid(clp, deleg_slab));
 	if (dp == NULL)
 		return dp;
-	dp->dl_stid.sc_type = NFS4_DELEG_STID;
 	/*
 	 * delegation seqid's are never incremented.  The 4.1 special
 	 * meaning of seqid 0 isn't meaningful, really, but let's avoid
@@ -3079,6 +3078,7 @@ static int nfs4_set_delegation(struct nfs4_delegation *dp, struct nfs4_file *fp)
 	atomic_inc(&fp->fi_delegees);
 	hash_delegation_locked(dp, fp);
 	spin_unlock(&recall_lock);
+	dp->dl_stid.sc_type = NFS4_DELEG_STID;
 	return 0;
 out_free:
 	put_nfs4_file(fp);
Signature · 1 line
1.8.3.1
#13 lore raw

Re: [PATCH 4/7] nfsd4: hash deleg stateid only on successful nfs4_set_delegation

Quoted 2 lines · Benny Halevy

We don't want the stateid to be found in the hash table before the delegation is granted.

With all this code and any find_stateid calls under the state lock, this isn't currently an issue, right?

The changelog needs to make clear whether this is fixing a current bug or preparing to allow dropping of the state lock somewhere. If the latter, it needs to outline what the expected change is that we're preparing for.

--b.

Quoted 33 lines
---
 fs/nfsd/nfs4state.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
fs/nfsd/nfs4state.c +1 -1
index 405649f..64fcdd6 100644
@@ -375,7 +375,6 @@ static struct nfs4_ol_stateid * nfs4_alloc_stateid(struct nfs4_client *clp)
 	dp = delegstateid(nfs4_alloc_stid(clp, deleg_slab));
 	if (dp == NULL)
 		return dp;
-	dp->dl_stid.sc_type = NFS4_DELEG_STID;
 	/*
 	 * delegation seqid's are never incremented.  The 4.1 special
 	 * meaning of seqid 0 isn't meaningful, really, but let's avoid
@@ -3079,6 +3078,7 @@ static int nfs4_set_delegation(struct nfs4_delegation *dp, struct nfs4_file *fp)
 	atomic_inc(&fp->fi_delegees);
 	hash_delegation_locked(dp, fp);
 	spin_unlock(&recall_lock);
+	dp->dl_stid.sc_type = NFS4_DELEG_STID;
 	return 0;
 out_free:
 	put_nfs4_file(fp);
Signature · 6 lines
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#14 lore raw

Re: [PATCH 4/7] nfsd4: hash deleg stateid only on successful nfs4_set_delegation

Quoted 6 lines · J. Bruce Fields
Quoted 2 lines · Benny Halevy

We don't want the stateid to be found in the hash table before the delegation is granted.

With all this code and any find_stateid calls under the state lock, this isn't currently an issue, right?

Right.

Quoted 5 lines

The changelog needs to make clear whether this is fixing a current bug or preparing to allow dropping of the state lock somewhere. If the latter, it needs to outline what the expected change is that we're preparing for.

Agreed. We can delay this patch to the state lock elimination series.

Benny

Quoted 41 lines

--b.

Quoted 33 lines
---
 fs/nfsd/nfs4state.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
fs/nfsd/nfs4state.c +1 -1
index 405649f..64fcdd6 100644
@@ -375,7 +375,6 @@ static struct nfs4_ol_stateid * nfs4_alloc_stateid(struct nfs4_client *clp)
 	dp = delegstateid(nfs4_alloc_stid(clp, deleg_slab));
 	if (dp == NULL)
 		return dp;
-	dp->dl_stid.sc_type = NFS4_DELEG_STID;
 	/*
 	 * delegation seqid's are never incremented.  The 4.1 special
 	 * meaning of seqid 0 isn't meaningful, really, but let's avoid
@@ -3079,6 +3078,7 @@ static int nfs4_set_delegation(struct nfs4_delegation *dp, struct nfs4_file *fp)
 	atomic_inc(&fp->fi_delegees);
 	hash_delegation_locked(dp, fp);
 	spin_unlock(&recall_lock);
+	dp->dl_stid.sc_type = NFS4_DELEG_STID;
 	return 0;
 out_free:
 	put_nfs4_file(fp);
Signature · 6 lines
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#15 lore raw

PATCH 5/7 nfsd4: cleanup hash and unhash delegation

In reply to Benny Halevy

make sure all atomic ops are under the recall_lock

---
 fs/nfsd/nfs4state.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
fs/nfsd/nfs4state.c +3 -2
index 64fcdd6..8840206 100644
@@ -436,6 +436,7 @@ static void unhash_stid(struct nfs4_stid *s)
 {
 	list_add(&dp->dl_perfile, &fp->fi_delegations);
 	list_add(&dp->dl_perclnt, &dp->dl_stid.sc_client->cl_delegations);
+	dp->dl_stid.sc_type = NFS4_DELEG_STID;
 }
 
 /* Called under the state lock. */
@@ -446,8 +447,9 @@ static void unhash_stid(struct nfs4_stid *s)
 	list_del_init(&dp->dl_perclnt);
 	list_del_init(&dp->dl_perfile);
 	list_del_init(&dp->dl_recall_lru);
-	spin_unlock(&recall_lock);
+	dp->dl_stid.sc_type = 0;
 	nfs4_put_deleg_lease(dp->dl_file);
+	spin_unlock(&recall_lock);
 	put_nfs4_file(dp->dl_file);
 	dp->dl_file = NULL;
 }
@@ -3078,7 +3080,6 @@ static int nfs4_set_delegation(struct nfs4_delegation *dp, struct nfs4_file *fp)
 	atomic_inc(&fp->fi_delegees);
 	hash_delegation_locked(dp, fp);
 	spin_unlock(&recall_lock);
-	dp->dl_stid.sc_type = NFS4_DELEG_STID;
 	return 0;
 out_free:
 	put_nfs4_file(fp);
Signature · 1 line
1.8.3.1
#16 lore raw

PATCH 6/7 nfsd4: move nfs4_put_file from unhash to put delegation

In reply to Benny Halevy

revoked delegations are unhashed but are kept around in nfsv4.1 and we better
hang on to dp_file in this case.

---
 fs/nfsd/nfs4state.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
fs/nfsd/nfs4state.c +2 -2
index 8840206..7f78ff5 100644
@@ -411,6 +411,8 @@ static void nfs4_free_stid(struct kmem_cache *slab, struct nfs4_stid *s)
 {
 	remove_stid(&dp->dl_stid);
 	if (atomic_dec_and_test(&dp->dl_count)) {
+		if (dp->dl_file)
+			put_nfs4_file(dp->dl_file);
 		nfs4_free_stid(deleg_slab, &dp->dl_stid);
 		num_delegations--;
 	}
@@ -450,8 +452,6 @@ static void unhash_stid(struct nfs4_stid *s)
 	dp->dl_stid.sc_type = 0;
 	nfs4_put_deleg_lease(dp->dl_file);
 	spin_unlock(&recall_lock);
-	put_nfs4_file(dp->dl_file);
-	dp->dl_file = NULL;
 }
 
 
Signature · 1 line
1.8.3.1
#17 lore raw

Re: [PATCH 6/7] nfsd4: move nfs4_put_file from unhash to put delegation

Quoted 2 lines · Benny Halevy

revoked delegations are unhashed but are kept around in nfsv4.1 and we better
hang on to dp_file in this case.

I don't agree: we shouldn't need dl_file any more once the delegation is revoked. It exists only to process delegreturns and set SEQ4_STATUS_RECALLABLE_STATE_REVOKED.

If some other code (like delegreturn) *is* attempting to use dl_file on revoked delegation, then that sounds like a delegreturn bug.

Among other things I think we'd like to be able to free up the reference and deallocate the file on disk if necessary once the delegation has been revoked.

--b.

Quoted 35 lines
---
 fs/nfsd/nfs4state.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
fs/nfsd/nfs4state.c +2 -2
index 8840206..7f78ff5 100644
@@ -411,6 +411,8 @@ static void nfs4_free_stid(struct kmem_cache *slab, struct nfs4_stid *s)
 {
 	remove_stid(&dp->dl_stid);
 	if (atomic_dec_and_test(&dp->dl_count)) {
+		if (dp->dl_file)
+			put_nfs4_file(dp->dl_file);
 		nfs4_free_stid(deleg_slab, &dp->dl_stid);
 		num_delegations--;
 	}
@@ -450,8 +452,6 @@ static void unhash_stid(struct nfs4_stid *s)
 	dp->dl_stid.sc_type = 0;
 	nfs4_put_deleg_lease(dp->dl_file);
 	spin_unlock(&recall_lock);
-	put_nfs4_file(dp->dl_file);
-	dp->dl_file = NULL;
 }
 
 
Signature · 6 lines
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#18 lore raw

Re: [PATCH 6/7] nfsd4: move nfs4_put_file from unhash to put delegation

Quoted 7 lines · J. Bruce Fields
Quoted 2 lines · Benny Halevy

revoked delegations are unhashed but are kept around in nfsv4.1 and we better
hang on to dp_file in this case.

I don't agree: we shouldn't need dl_file any more once the delegation is revoked. It exists only to process delegreturns and set SEQ4_STATUS_RECALLABLE_STATE_REVOKED.

OK, makes sense.

Quoted 3 lines

If some other code (like delegreturn) *is* attempting to use dl_file on revoked delegation, then that sounds like a delegreturn bug.

It doesn't seem so.

Quoted 4 lines

Among other things I think we'd like to be able to free up the reference and deallocate the file on disk if necessary once the delegation has been revoked.

OK, I'll deal with blocking case in my patchset, but putting the file will need to beseparated from unhashing as the latter will be done under a spin lock and put_nfs4_file -> iput may block.

Benny

Quoted 43 lines

--b.

Quoted 35 lines
---
 fs/nfsd/nfs4state.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
fs/nfsd/nfs4state.c +2 -2
index 8840206..7f78ff5 100644
@@ -411,6 +411,8 @@ static void nfs4_free_stid(struct kmem_cache *slab, struct nfs4_stid *s)
 {
 	remove_stid(&dp->dl_stid);
 	if (atomic_dec_and_test(&dp->dl_count)) {
+		if (dp->dl_file)
+			put_nfs4_file(dp->dl_file);
 		nfs4_free_stid(deleg_slab, &dp->dl_stid);
 		num_delegations--;
 	}
@@ -450,8 +452,6 @@ static void unhash_stid(struct nfs4_stid *s)
 	dp->dl_stid.sc_type = 0;
 	nfs4_put_deleg_lease(dp->dl_file);
 	spin_unlock(&recall_lock);
-	put_nfs4_file(dp->dl_file);
-	dp->dl_file = NULL;
 }
 
 
Signature · 6 lines
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#19 lore raw

PATCH 7/7 nfsd4: hold recall_lock while accessing nfs4_delegation.dl_recall_lru

In reply to Benny Halevy
---
 fs/nfsd/nfs4state.c | 20 ++++++++++++++------
 1 file changed, 14 insertions(+), 6 deletions(-)
fs/nfsd/nfs4state.c +14 -6
index 7f78ff5..d9161ea 100644
@@ -441,24 +441,29 @@ static void unhash_stid(struct nfs4_stid *s)
 	dp->dl_stid.sc_type = NFS4_DELEG_STID;
 }
 
-/* Called under the state lock. */
 static void
-unhash_delegation(struct nfs4_delegation *dp)
+unhash_delegation_locked(struct nfs4_delegation *dp)
 {
-	spin_lock(&recall_lock);
 	list_del_init(&dp->dl_perclnt);
 	list_del_init(&dp->dl_perfile);
 	list_del_init(&dp->dl_recall_lru);
 	dp->dl_stid.sc_type = 0;
 	nfs4_put_deleg_lease(dp->dl_file);
-	spin_unlock(&recall_lock);
 }
 
-
+static void
+unhash_delegation(struct nfs4_delegation *dp)
+{
+	spin_lock(&recall_lock);
+	unhash_delegation_locked(dp);
+	spin_unlock(&recall_lock);
+}
 
 static void destroy_revoked_delegation(struct nfs4_delegation *dp)
 {
+	spin_lock(&recall_lock);
 	list_del_init(&dp->dl_recall_lru);
+	spin_unlock(&recall_lock);
 	nfs4_put_delegation(dp);
 }
 
@@ -475,9 +480,11 @@ static void revoke_delegation(struct nfs4_delegation *dp)
 	if (clp->cl_minorversion == 0)
 		destroy_delegation(dp);
 	else {
-		unhash_delegation(dp);
+		spin_lock(&recall_lock);
+		unhash_delegation_locked(dp);
 		dp->dl_stid.sc_type = NFS4_REVOKED_DELEG_STID;
 		list_add(&dp->dl_recall_lru, &clp->cl_revoked);
+		spin_unlock(&recall_lock);
 	}
 }
 
@@ -2729,6 +2736,7 @@ static void nfsd_break_one_deleg(struct nfs4_delegation *dp)
 	struct nfs4_client *clp = dp->dl_stid.sc_client;
 	struct nfsd_net *nn = net_generic(clp->net, nfsd_net_id);
 
+	lockdep_assert_held(&recall_lock);
 	/* We're assuming the state code never drops its reference
 	 * without first removing the lease.  Since we're in this lease
 	 * callback (and since the lease code is serialized by the kernel
Signature · 1 line
1.8.3.1
#20 lore raw
In reply to Benny Halevy

"nfsd4: need to destroy revoked delegations in destroy_client" is the only one that looks to me like it fixes a current bug.

I understand the others may be laying the groundwork for a state lock change, but I don't have the context to review those yet....

--b.

Quoted 73 lines · Benny Halevy

commit c7342d3cf5d5677063cbc6341634ed8cea1b176f

Author: Benny Halevy <[email protected]>
Date:   Tue Oct 15 17:07:31 2013 +0300

nfsd4: fix recall_lock use in unhash_delegation

Access to dp->dl_perclnt must be synchronized by the recall_lock

commit 87174785bcec37371b718757af67dde0f6d083e3

Author: Benny Halevy <[email protected]>
Date:   Tue Oct 15 17:29:13 2013 +0300

nfsd4: need to destroy revoked delegations in destroy_client

[use list_splice_init]

commit 775a7353f9d1e60ebd9096b1a82bfb78fe7f3cc9

Author: Benny Halevy <[email protected]>
Date:   Tue Oct 15 17:41:53 2013 +0300

nfsd4: properly hash delegation in nfs4_setlease

keep all accesses via dl_perclnt under the recall_lock.

commit fdf586869a6b026893f818cb538574a7b77f5604

Author: Benny Halevy <[email protected]>
Date:   Tue Oct 15 21:08:48 2013 +0300

nfsd4: hash deleg stateid only on successful nfs4_set_delegation

We don't want the stateid to be found in the hash table before the delegation is granted.

commit 635a93446fde8d8fb1f4bb825926750d9b6aea9d

Author: Benny Halevy <[email protected]>
Date:   Tue Oct 15 23:46:55 2013 +0300

nfsd4: cleanup hash and unhash delegation

make sure all atomic ops are under the recall_lock

commit 39f26e5974f73b62144abde6793d41b07a32719a

Author: Benny Halevy <[email protected]>
Date:   Tue Oct 15 23:50:30 2013 +0300

nfsd4: move nfs4_put_file from unhash to put delegation

revoked delegations are unhashed but are kept around in nfsv4.1 and we better
hang on to dp_file in this case.

commit cd2b2cac09cbd45419ce7d0e354dd94537a4fbfb

Author: Benny Halevy <[email protected]>
Date:   Wed Oct 16 00:17:14 2013 +0300

nfsd4: hold recall_lock while accessing nfs4_delegation.dl_recall_lru

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html