Skip to contentSkip to conversation

a couple fixes in nfsd4_release_lockowner

PATCH Superseded Benny Halevy 6 participants 13 messages Dec 13, 2013 – Dec 20, 2013

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover a couple fixes in nfsd4_release_lockowner

Bruce, while working on the state mutex elimination stuff I saw these two issues in nfsd4_release_lockowner. These patches are untested but I just wanted to quickly get your take on them.

[PATCH 1/2] nfsd4: break from inner lookup loop in nfsd4_release_lockowner on first match

looks like a potential list corruption risk.

[PATCH 2/2] nfsd4: ignore nfsv4.1 lockowners in nfsd4_release_lockowner

I'm not sure that 100% needed but since we keep both nfsv4.0 and v4.1 owners hashed on the same lists we don't want a v4.0 operation to accidentally touch v4.1 state.

Benny

#1 lore raw

PATCH 1/2 nfsd4: break from inner lookup loop in nfsd4_release_lockowner on first match

Otherwise the lockowner may by added to "matches" more than once.

---
 fs/nfsd/nfs4state.c | 1 +
 1 file changed, 1 insertion(+)
fs/nfsd/nfs4state.c +1 -0
index 0874998..84007b6 100644
@@ -4660,6 +4660,7 @@ nfsd4_release_lockowner(struct svc_rqst *rqstp,
 			if (check_for_locks(stp->st_file, lo))
 				goto out;
 			list_add(&lo->lo_list, &matches);
+			break;
 		}
 	}
 	/* Clients probably won't expect us to return with some (but not all)
Signature · 1 line
1.8.3.1
#3 lore raw

PATCH 2/2 nfsd4: ignore nfsv4.1 lockowners in nfsd4_release_lockowner

In reply to Benny Halevy

RELEASE_LOCKOWNER is a NFSv4.0 operation only so it can quickly skip lockowners created by nfsv4.1 clients.

---
 fs/nfsd/nfs4state.c | 2 ++
 1 file changed, 2 insertions(+)
fs/nfsd/nfs4state.c +2 -0
index 84007b6..00424f2 100644
@@ -4652,6 +4652,8 @@ nfsd4_release_lockowner(struct svc_rqst *rqstp,
 	list_for_each_entry(sop, &nn->ownerstr_hashtbl[hashval], so_strhash) {
 		if (sop->so_is_open_owner)
 			continue;
+		if (sop->so_client->cl_minorversion)
+			continue;
 		if (!same_owner_str(sop, owner, clid))
 			continue;
 		list_for_each_entry(stp, &sop->so_stateids,
Signature · 1 line
1.8.3.1
#4 lore raw

Re: [PATCH 2/2] nfsd4: ignore nfsv4.1 lockowners in nfsd4_release_lockowner

fs/nfsd/nfs4state.c nfsd4_release_lockowner(struct svc_rqst *rqstp, · line 4659

Quoted 8 lines
fs/nfsd/nfs4state.c +2 -0
 	list_for_each_entry(sop, &nn->ownerstr_hashtbl[hashval], so_strhash) {
 		if (sop->so_is_open_owner)
 			continue;
+		if (sop->so_client->cl_minorversion)
+			continue;
 		if (!same_owner_str(sop, owner, clid))
 			continue;
 		list_for_each_entry(stp, &sop->so_stateids,

This needs at least a good comment as it's not very obvious from glancing over the code. That being said is same_owner_str so much overhead that it's really worth it?

#2 lore raw

Re: [PATCH 1/2] nfsd4: break from inner lookup loop in nfsd4_release_lockowner on first match

In reply to Benny Halevy
Quoted 1 line · Benny Halevy

Otherwise the lockowner may by added to "matches" more than once.

Whoops, thanks, looks right.

The lo = assignment should probably also be moved up out of this loop.

--b.

Quoted 21 lines
---
 fs/nfsd/nfs4state.c | 1 +
 1 file changed, 1 insertion(+)
fs/nfsd/nfs4state.c +1 -0
index 0874998..84007b6 100644
@@ -4660,6 +4660,7 @@ nfsd4_release_lockowner(struct svc_rqst *rqstp,
 			if (check_for_locks(stp->st_file, lo))
 				goto out;
 			list_add(&lo->lo_list, &matches);
+			break;
 		}
 	}
 	/* Clients probably won't expect us to return with some (but not all)
Signature · 1 line
1.8.3.1
#6 lore raw

cover a couple fixes in nfsd4_release_lockowner

In reply to Benny Halevy

changes from v1:

[PATCH v2 1/2] nfsd4: break from inner lookup loop in nfsd4_release_lockowner on first match

- initialize lo_list
- in case locks our found delete all previous matches from temporary list

[PATCH 2/2] nfsd4: ignore nfsv4.1 lockowners in nfsd4_release_lockowner

- added comment

Quoted 19 lines · Benny Halevy

Bruce, while working on the state mutex elimination stuff I saw these two issues in nfsd4_release_lockowner. These patches are untested but I just wanted to quickly get your take on them.

[PATCH 1/2] nfsd4: break from inner lookup loop in nfsd4_release_lockowner on first match

looks like a potential list corruption risk.

[PATCH 2/2] nfsd4: ignore nfsv4.1 lockowners in nfsd4_release_lockowner

I'm not sure that 100% needed but since we keep both nfsv4.0 and v4.1 owners hashed on the same lists we don't want a v4.0 operation to accidentally touch v4.1 state.

Benny
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#7 lore raw

PATCH v2 1/2 nfsd4: break from inner lookup loop in nfsd4_release_lockowner on first match

Otherwise the lockowner may by added to "matches" more than once.

---
 fs/nfsd/nfs4state.c | 17 +++++++++++------
 1 file changed, 11 insertions(+), 6 deletions(-)
fs/nfsd/nfs4state.c +11 -6
index 0874998..b04f765 100644
@@ -4192,6 +4192,7 @@ alloc_init_lock_stateowner(unsigned int strhashval, struct nfs4_client *clp, str
 	/* It is the openowner seqid that will be incremented in encode in the
 	 * case of new lockowners; so increment the lock seqid manually: */
 	lo->lo_owner.so_seqid = lock->lk_new_lock_seqid + 1;
+	INIT_LIST_HEAD(&lo->lo_list);
 	hash_lockowner(lo, strhashval, clp, open_stp);
 	return lo;
 }
@@ -4646,7 +4647,6 @@ nfsd4_release_lockowner(struct svc_rqst *rqstp,
 	if (status)
 		goto out;
 
-	status = nfserr_locks_held;
 	INIT_LIST_HEAD(&matches);
 
 	list_for_each_entry(sop, &nn->ownerstr_hashtbl[hashval], so_strhash) {
@@ -4654,25 +4654,30 @@ nfsd4_release_lockowner(struct svc_rqst *rqstp,
 			continue;
 		if (!same_owner_str(sop, owner, clid))
 			continue;
+		lo = lockowner(sop);
 		list_for_each_entry(stp, &sop->so_stateids,
 				st_perstateowner) {
-			lo = lockowner(sop);
-			if (check_for_locks(stp->st_file, lo))
-				goto out;
+			if (check_for_locks(stp->st_file, lo)) {
+				status = nfserr_locks_held;
+				goto locks_held;
+			}
 			list_add(&lo->lo_list, &matches);
+			break;
 		}
 	}
 	/* Clients probably won't expect us to return with some (but not all)
 	 * of the lockowner state released; so don't release any until all
 	 * have been checked. */
 	status = nfs_ok;
+locks_held:
 	while (!list_empty(&matches)) {
-		lo = list_entry(matches.next, struct nfs4_lockowner,
+		lo = list_first_entry(&matches, struct nfs4_lockowner,
 								lo_list);
 		/* unhash_stateowner deletes so_perclient only
 		 * for openowners. */
 		list_del(&lo->lo_list);
-		release_lockowner(lo);
+		if (status == nfs_ok)
+			release_lockowner(lo);
 	}
 out:
 	nfs4_unlock_state();
Signature · 1 line
1.8.3.1
#12 lore raw

PATCH v2 2/2 nfsd4: ignore nfsv4.1 lockowners in nfsd4_release_lockowner

In reply to Benny Halevy

RELEASE_LOCKOWNER is a NFSv4.0 operation only so it can quickly skip lockowners created by nfsv4.1 clients.

---
 fs/nfsd/nfs4state.c | 3 +++
 1 file changed, 3 insertions(+)
fs/nfsd/nfs4state.c +3 -0
index b04f765..7d79494 100644
@@ -4652,6 +4652,9 @@ nfsd4_release_lockowner(struct svc_rqst *rqstp,
 	list_for_each_entry(sop, &nn->ownerstr_hashtbl[hashval], so_strhash) {
 		if (sop->so_is_open_owner)
 			continue;
+		/* This is NFSv4.0 only operation, skip NFSv4.x lockowners */
+		if (sop->so_client->cl_minorversion)
+			continue;
 		if (!same_owner_str(sop, owner, clid))
 			continue;
 		lo = lockowner(sop);
Signature · 1 line
1.8.3.1
#8 lore raw

Re: [PATCH v2 1/2] nfsd4: break from inner lookup loop in nfsd4_release_lockowner on first match

In reply to Benny Halevy

Hi Benny,

Quoted 33 lines · Benny Halevy

Otherwise the lockowner may by added to "matches" more than once.

---
 fs/nfsd/nfs4state.c | 17 +++++++++++------
 1 file changed, 11 insertions(+), 6 deletions(-)
fs/nfsd/nfs4state.c +2 -1
index 0874998..b04f765 100644
@@ -4192,6 +4192,7 @@ alloc_init_lock_stateowner(unsigned int strhashval, struct nfs4_client *clp, str
        /* It is the openowner seqid that will be incremented in encode in the
         * case of new lockowners; so increment the lock seqid manually: */
        lo->lo_owner.so_seqid = lock->lk_new_lock_seqid + 1;
+       INIT_LIST_HEAD(&lo->lo_list);
        hash_lockowner(lo, strhashval, clp, open_stp);
        return lo;
 }
@@ -4646,7 +4647,6 @@ nfsd4_release_lockowner(struct svc_rqst *rqstp,
        if (status)
                goto out;
 
-       status = nfserr_locks_held;
        INIT_LIST_HEAD(&matches);
 
        list_for_each_entry(sop, &nn->ownerstr_hashtbl[hashval], so_strhash) {
@@ -4654,25 +4654,30 @@ nfsd4_release_lockowner(struct svc_rqst *rqstp,
                        continue;
                if (!same_owner_str(sop, owner, clid))
                        continue;
+               lo = lockowner(sop);
                list_for_each_entry(stp, &sop->so_stateids,

fs/nfsd/nfs4state.c nfsd4_release_lockowner(struct svc_rqst *rqstp, · line 4665

Quoted 10 lines
fs/nfsd/nfs4state.c +5 -3
                                st_perstateowner) {
-                       lo = lockowner(sop);
-                       if (check_for_locks(stp->st_file, lo))
-                               goto out;
+                       if (check_for_locks(stp->st_file, lo)) {
+                               status = nfserr_locks_held;
+                               goto locks_held;
+                       }
                        list_add(&lo->lo_list, &matches);
+                       break;

If so_stateids is empty, lockowner is skipped. It was skipped before the patch as well but I guess that need to be fixed, right?

Thanks,
Tao

#9 lore raw

Re: [PATCH v2 1/2] nfsd4: break from inner lookup loop in nfsd4_release_lockowner on first match

Quoted 40 lines · Peng Tao

Hi Benny,

Quoted 37 lines · Benny Halevy

Otherwise the lockowner may by added to "matches" more than once.

---
 fs/nfsd/nfs4state.c | 17 +++++++++++------
 1 file changed, 11 insertions(+), 6 deletions(-)
fs/nfsd/nfs4state.c +2 -4
index 0874998..b04f765 100644
@@ -4192,6 +4192,7 @@ alloc_init_lock_stateowner(unsigned int strhashval, struct nfs4_client *clp, str
        /* It is the openowner seqid that will be incremented in encode in the
         * case of new lockowners; so increment the lock seqid manually: */
        lo->lo_owner.so_seqid = lock->lk_new_lock_seqid + 1;
+       INIT_LIST_HEAD(&lo->lo_list);
        hash_lockowner(lo, strhashval, clp, open_stp);
        return lo;
 }
@@ -4646,7 +4647,6 @@ nfsd4_release_lockowner(struct svc_rqst *rqstp,
        if (status)
                goto out;
 
-       status = nfserr_locks_held;
        INIT_LIST_HEAD(&matches);
 
        list_for_each_entry(sop, &nn->ownerstr_hashtbl[hashval], so_strhash) {
@@ -4654,25 +4654,30 @@ nfsd4_release_lockowner(struct svc_rqst *rqstp,
                        continue;
                if (!same_owner_str(sop, owner, clid))
                        continue;
+               lo = lockowner(sop);
                list_for_each_entry(stp, &sop->so_stateids,
                                st_perstateowner) {
-                       lo = lockowner(sop);
-                       if (check_for_locks(stp->st_file, lo))
-                               goto out;
Quoted 8 lines

fs/nfsd/nfs4state.c nfsd4_release_lockowner(struct svc_rqst *rqstp, · line 4665

Quoted 6 lines
fs/nfsd/nfs4state.c +5 -0
+                       if (check_for_locks(stp->st_file, lo)) {
+                               status = nfserr_locks_held;
+                               goto locks_held;
+                       }
                        list_add(&lo->lo_list, &matches);
+                       break;

If so_stateids is empty, lockowner is skipped. It was skipped before the patch as well but I guess that need to be fixed, right?

I'm not sure that's a valid state at all.

Benny

Quoted 4 lines

Thanks,
Tao

#5 lore raw

Re: [PATCH 2/2] nfsd4: ignore nfsv4.1 lockowners in nfsd4_release_lockowner

In reply to Christoph Hellwig
Quoted 4 lines · Christoph Hellwig
Quoted 4 lines
fs/nfsd/nfs4state.c +1 -0
 	list_for_each_entry(sop, &nn->ownerstr_hashtbl[hashval], so_strhash) {
 		if (sop->so_is_open_owner)
 			continue;
+		if (sop->so_client->cl_minorversion)
Quoted 9 lines

fs/nfsd/nfs4state.c nfsd4_release_lockowner(struct svc_rqst *rqstp, · line 4659

Quoted 4 lines
fs/nfsd/nfs4state.c +1 -0
+			continue;
 		if (!same_owner_str(sop, owner, clid))
 			continue;
 		list_for_each_entry(stp, &sop->so_stateids,

This needs at least a good comment as it's not very obvious from glancing over the code. That being said is same_owner_str so much overhead that it's really worth it?

Right, this seems redundant with the cli_id comparison in same_owner_str. That could be reordered to ensure it precedes the memcmp if we think that's worthwhile.

--b.

#11 lore raw

Re: [PATCH v2 1/2] nfsd4: break from inner lookup loop in nfsd4_release_lockowner on first match

In reply to Benny Halevy
Quoted 12 lines · Benny Halevy

Otherwise the lockowner may by added to "matches" more than once.

---
 fs/nfsd/nfs4state.c | 17 +++++++++++------
 1 file changed, 11 insertions(+), 6 deletions(-)
fs/nfsd/nfs4state.c +0 -0
index 0874998..b04f765 100644

fs/nfsd/nfs4state.c alloc_init_lock_stateowner(unsigned int strhashval, struct nfs4_client *clp, str · line 4195

Quoted 4 lines
fs/nfsd/nfs4state.c +1 -0
@@ -4192,6 +4192,7 @@ alloc_init_lock_stateowner(unsigned int strhashval, struct nfs4_client *clp, str
 	/* It is the openowner seqid that will be incremented in encode in the
 	 * case of new lockowners; so increment the lock seqid manually: */
 	lo->lo_owner.so_seqid = lock->lk_new_lock_seqid + 1;
+	INIT_LIST_HEAD(&lo->lo_list);

This doesn't really fix any bug--we don't depend on this list head being initialized anywhere as far as I can see. If you think it's useful anyway fo rdebugging purposes or something, that's fine, but stick this in a separate patch from the actual bugfix.

Quoted 17 lines
fs/nfsd/nfs4state.c +1 -1
 	hash_lockowner(lo, strhashval, clp, open_stp);
 	return lo;
 }
@@ -4646,7 +4647,6 @@ nfsd4_release_lockowner(struct svc_rqst *rqstp,
 	if (status)
 		goto out;
 
-	status = nfserr_locks_held;
 	INIT_LIST_HEAD(&matches);
 
 	list_for_each_entry(sop, &nn->ownerstr_hashtbl[hashval], so_strhash) {
@@ -4654,25 +4654,30 @@ nfsd4_release_lockowner(struct svc_rqst *rqstp,
 			continue;
 		if (!same_owner_str(sop, owner, clid))
 			continue;
+		lo = lockowner(sop);
 		list_for_each_entry(stp, &sop->so_stateids,

fs/nfsd/nfs4state.c nfsd4_release_lockowner(struct svc_rqst *rqstp, · line 4665

Quoted 10 lines
fs/nfsd/nfs4state.c +5 -3
 				st_perstateowner) {
-			lo = lockowner(sop);
-			if (check_for_locks(stp->st_file, lo))
-				goto out;
+			if (check_for_locks(stp->st_file, lo)) {
+				status = nfserr_locks_held;
+				goto locks_held;
+			}
 			list_add(&lo->lo_list, &matches);
+			break;

I'm a little lost here: it looks like if sop->so_stateids has more than one entry, then we'll decide to release lo just because the first entry doesn't have any associated locks (when subsequent entries still might).

Instead of breaking at the end I think you just want to move the list_add after the loop, to ensure that we check all the stateid's.

Quoted 7 lines
fs/nfsd/nfs4state.c +1 -0
 		}
 	}
 	/* Clients probably won't expect us to return with some (but not all)
 	 * of the lockowner state released; so don't release any until all
 	 * have been checked. */
 	status = nfs_ok;
+locks_held:

fs/nfsd/nfs4state.c nfsd4_release_lockowner(struct svc_rqst *rqstp, · line 4680

Quoted 10 lines
fs/nfsd/nfs4state.c +3 -2
 	while (!list_empty(&matches)) {
-		lo = list_entry(matches.next, struct nfs4_lockowner,
+		lo = list_first_entry(&matches, struct nfs4_lockowner,
 								lo_list);
 		/* unhash_stateowner deletes so_perclient only
 		 * for openowners. */
 		list_del(&lo->lo_list);
-		release_lockowner(lo);
+		if (status == nfs_ok)
+			release_lockowner(lo);

Again, we don't depend on lo_list being initialized anywhere, so this is really a sort of cleanup unrelated to this bugfix.

And if you think it may be asking for trouble to leave lo_list on a list that doesn't exist any more, OK, but make that argument in a separate patch.

--b.

Quoted 6 lines
fs/nfsd/nfs4state.c +0 -0
 	}
 out:
 	nfs4_unlock_state();
Signature · 1 line
1.8.3.1
#10 lore raw

Re: [PATCH v2 1/2] nfsd4: break from inner lookup loop in nfsd4_release_lockowner on first match

In reply to Benny Halevy
Quoted 52 lines · Benny Halevy
Quoted 40 lines · Peng Tao

Hi Benny,

Quoted 37 lines · Benny Halevy

Otherwise the lockowner may by added to "matches" more than once.

---
 fs/nfsd/nfs4state.c | 17 +++++++++++------
 1 file changed, 11 insertions(+), 6 deletions(-)
fs/nfsd/nfs4state.c +2 -4
index 0874998..b04f765 100644
@@ -4192,6 +4192,7 @@ alloc_init_lock_stateowner(unsigned int strhashval, struct nfs4_client *clp, str
        /* It is the openowner seqid that will be incremented in encode in the
         * case of new lockowners; so increment the lock seqid manually: */
        lo->lo_owner.so_seqid = lock->lk_new_lock_seqid + 1;
+       INIT_LIST_HEAD(&lo->lo_list);
        hash_lockowner(lo, strhashval, clp, open_stp);
        return lo;
 }
@@ -4646,7 +4647,6 @@ nfsd4_release_lockowner(struct svc_rqst *rqstp,
        if (status)
                goto out;
 
-       status = nfserr_locks_held;
        INIT_LIST_HEAD(&matches);
 
        list_for_each_entry(sop, &nn->ownerstr_hashtbl[hashval], so_strhash) {
@@ -4654,25 +4654,30 @@ nfsd4_release_lockowner(struct svc_rqst *rqstp,
                        continue;
                if (!same_owner_str(sop, owner, clid))
                        continue;
+               lo = lockowner(sop);
                list_for_each_entry(stp, &sop->so_stateids,
                                st_perstateowner) {
-                       lo = lockowner(sop);
-                       if (check_for_locks(stp->st_file, lo))
-                               goto out;
Quoted 8 lines

fs/nfsd/nfs4state.c nfsd4_release_lockowner(struct svc_rqst *rqstp, · line 4665

Quoted 6 lines
fs/nfsd/nfs4state.c +5 -0
+                       if (check_for_locks(stp->st_file, lo)) {
+                               status = nfserr_locks_held;
+                               goto locks_held;
+                       }
                        list_add(&lo->lo_list, &matches);
+                       break;

If so_stateids is empty, lockowner is skipped. It was skipped before the patch as well but I guess that need to be fixed, right?

Quoted 1 line

I'm not sure that's a valid state at all.

OK. I see the comments in lookup_or_create_lock_state() that says:

/* XXX: a lockowner always has exactly one stateid: */

And lookup_or_create_lock_state() does implement that way. So so_stateid always has exactly one member for lockowner. But then the original code (before the patch) is working properly, right? The list_for_each_entry can be replaced with list_first_entry and the added break doesn't seem necessary. Or is the situation somehow obsolete?

Thanks,
Tao