Skip to contentSkip to conversation

RCU-isms in fs/nfs/delegation.c

DISCUSSION David Howells 2 participants 5 messages Apr 19, 2010 – Apr 20, 2010

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

I'm trying to redo my NFS RCU warning fixup patch on top of Paul's patches, and I've found a small potential bug: nfs_inode_reclaim_delegation() doesn't use the appropriate accessors/locks to protect NFS_I(inode)->delegation, and nor does it use such to protect *delegation that I can see. It just overwrites the record.

Furthermore, for consistency's sake, it should also protect accesses to delegation->cred within that function.

Trond: can you confirm that both NFS_I(inode)->delegation and delegation->cred should be considered RCU-protected pointers?

David

#1 lore raw
Quoted 5 lines · David Howells

I'm trying to redo my NFS RCU warning fixup patch on top of Paul's patches, and I've found a small potential bug: nfs_inode_reclaim_delegation() doesn't use the appropriate accessors/locks to protect NFS_I(inode)->delegation, and nor does it use such to protect *delegation that I can see. It just overwrites the record.

Hmm... Yes, I think that function should probably take the rcu_read_lock(), and then take the delegation->lock before modifying the delegation. Furthermore, it should probably fall back to nfs_inode_set_delegation() in case we race with a delegreturn. See below...

Quoted 5 lines

Furthermore, for consistency's sake, it should also protect accesses to delegation->cred within that function.

Trond: can you confirm that both NFS_I(inode)->delegation and delegation->cred should be considered RCU-protected pointers?

With the above changes to nfs_inode_reclaim_delegation() I don't think delegation->cred needs to be RCU-protected.

Cheers
Trond

----------------------------------------------------------------------------------------------------------
NFSv4: Fix the locking in nfs_inode_reclaim_delegation()

From: Trond Myklebust <[email protected]>

Ensure that we correctly rcu-dereference the delegation itself, and that we protect against removal while we're changing the contents.

---

 fs/nfs/delegation.c |   37 ++++++++++++++++++++++++-------------
 1 files changed, 24 insertions(+), 13 deletions(-)
fs/nfs/delegation.c +24 -13
index 1567124..f9c6b63 100644
@@ -129,21 +129,32 @@ again:
  */
 void nfs_inode_reclaim_delegation(struct inode *inode, struct rpc_cred *cred, struct nfs_openres *res)
 {
-	struct nfs_delegation *delegation = NFS_I(inode)->delegation;
-	struct rpc_cred *oldcred;
+	struct nfs_delegation *delegation;
+	struct rpc_cred *oldcred = NULL;
 
+	rcu_read_lock();
+	delegation = rcu_dereference(NFS_I(inode)->delegation);
 	if (delegation == NULL)
-		return;
-	memcpy(delegation->stateid.data, res->delegation.data,
-			sizeof(delegation->stateid.data));
-	delegation->type = res->delegation_type;
-	delegation->maxsize = res->maxsize;
-	oldcred = delegation->cred;
-	delegation->cred = get_rpccred(cred);
-	clear_bit(NFS_DELEGATION_NEED_RECLAIM, &delegation->flags);
-	NFS_I(inode)->delegation_state = delegation->type;
-	smp_wmb();
-	put_rpccred(oldcred);
+		goto out;
+	spin_lock(&delegation->lock);
+	if (delegation->inode != NULL) {
+		memcpy(delegation->stateid.data, res->delegation.data,
+				sizeof(delegation->stateid.data));
+		delegation->type = res->delegation_type;
+		delegation->maxsize = res->maxsize;
+		oldcred = delegation->cred;
+		delegation->cred = get_rpccred(cred);
+		clear_bit(NFS_DELEGATION_NEED_RECLAIM, &delegation->flags);
+		NFS_I(inode)->delegation_state = delegation->type;
+		spin_unlock(&delegation->lock);
+		put_rpccred(oldcred);
+	} else {
+		/* We appear to have raced with a delegation return. */
+		spin_unlock(&delegation->lock);
+		nfs_inode_set_delegation(inode, cred, res);
+	}
+out:
+	rcu_read_unlock();
 }
 
 static int nfs_do_return_delegation(struct inode *inode, struct nfs_delegation *delegation, int issync)
#2 lore raw

fs/nfs/delegation.c again: · line 158

Quoted 5 lines · Trond Myklebust
fs/nfs/delegation.c +4 -0
+		nfs_inode_set_delegation(inode, cred, res);
+	}
+out:
+	rcu_read_unlock();
 }

I think this is wrong. nfs_inode_set_delegation() may sleep as it calls kmalloc() with GFP_KERNEL - but you still have the RCU read lock held. I think you need to drop the RCU read lock before calling it.

David

#3 lore raw
Quoted 12 lines · David Howells

fs/nfs/delegation.c again: · line 158

Quoted 5 lines · Trond Myklebust
fs/nfs/delegation.c +4 -0
+		nfs_inode_set_delegation(inode, cred, res);
+	}
+out:
+	rcu_read_unlock();
 }

I think this is wrong. nfs_inode_set_delegation() may sleep as it calls kmalloc() with GFP_KERNEL - but you still have the RCU read lock held. I think you need to drop the RCU read lock before calling it.

Quoted 1 line

David

Agreed. Thanks for the review...

Cheers
Trond

#4 lore raw
Quoted 1 line · Trond Myklebust

Agreed. Thanks for the review...

The version of the patch I posted just before lunch is hopefully fixed correctly.

David