Skip to contentSkip to conversation

autofs4 - update autofs4 to deal with VFS locking change

RFC PATCH In review Ian Kent 2 participants 16 messages Sep 24, 2009 – Sep 28, 2009 VFS

lore
Share & tools
lore

Ian Kent

author
#0 lore raw

cover autofs4 - update autofs4 to deal with VFS locking change

A change to the VFS path walk locking is needed to resolve an issue identified by Sage Weil. This locking change requires significant changes to the autofs4 module to allow it to callback to userspace without introducing a deadlock.

To cope with the change the autofs4 module needs to redirect mount requests from ->d_revalidate() to ->lookup() if the directory inode mutex is held when a callback needs to be done. Note that we cannot redirect these requests when the mutex is not held because, to function correctly, the mutex must be held over both revalidate and lookup.

Of the patches in the series most are cleanups and refactoring done to keep the real change in "autofs4 - always use lookup for lookup" as clean as possible. Unfortuneately, there is still quite a bit left in it.

Also, I need confirmation that the patch that changes the VFS path walk locking is in fact correct, or at least like for like to what will be submitted. I had some difficulty with the original patches that were paosted. The patch in question below is "vfs: make real_lookup do dentry revalidation with i_mutex held".

I've done quite a bit of fairly heavy stress testing of the patch series and they (finally) hold up to it. Although I have also managed to uncover a locking bug in the user space daemon as a result, ;)

---

Ian Kent (10):
autofs4 - always use lookup for lookup
autofs4 - rename dentry to expiring in autofs4_lookup_expiring() autofs4 - rename dentry to active in autofs4_lookup_active() autofs4 - eliminate d_unhashed in path walk checks autofs4 - cleanup active and expire lookup
autofs4 - renamer unhashed to active in autofs4_lookup() autofs4 - use autofs_info for pending flag
autofs4 - use macro for need mount check
autofs4 - use macros for expiring list
autofs4 - use macros for active list handling

Sage Weil (1):
Subject: [PATCH] vfs: make real_lookup do dentry revalidation with i_mutex held

 fs/autofs4/autofs_i.h |   38 +++
 fs/autofs4/expire.c   |    8 -
 fs/autofs4/inode.c    |    2 
 fs/autofs4/root.c     |  616 ++++++++++++++++++++++++++++++++-----------------
 fs/namei.c            |   58 ++---
 5 files changed, 480 insertions(+), 242 deletions(-)
Signature · 1 line
Ian
author
#1 lore raw

RFC PATCH 1/11 Subject: [PATCH] vfs: make real_lookup do dentry revalidation with i_mutex held

From: Sage Weil <[email protected]>

real_lookup() is called by do_lookup() if dentry revalidation fails. If the cache is re-populated while waiting for i_mutex, it may find that a d_lookup() subsequently succeeds (see the "Uhhuh! Nasty case" comment).

Previously, real_lookup() would drop i_mutex and do_revalidate() again. If revalidate failed _again_, however, it would give up with -ENOENT. The problem here that network file systems may be invalidating dentries via server callbacks, e.g. due to concurrent access from another client, and -ENOENT is frequently the wrong answer.

This problem has been seen with both Lustre and Ceph. It seems possible to hit this case with NFS as well if the cache lifetime is very short.

Instead, we should do_revalidate() while i_mutex is still held. If revalidation fails, we can move on to a ->lookup() and ensure a correct result without worrying about any subsequent races.

Note that do_revalidate() is called with i_mutex held elsewhere. For example, do_filp_open(), lookup_create(), do_unlinkat(), do_rmdir(), and possibly others all take the directory i_mutex, and then

-> lookup_hash

        -> __lookup_hash
                -> cached_lookup
                        -> do_revalidate

so this does not introduce any new locking rules for d_revalidate implementations.

---

 fs/namei.c |   58 +++++++++++++++++++++++++++++++---------------------------
 1 files changed, 31 insertions(+), 27 deletions(-)
fs/namei.c +31 -27
index d11f404..d68ea6d 100644
@@ -477,6 +477,7 @@ static struct dentry * real_lookup(struct dentry * parent, struct qstr * name, s
 {
 	struct dentry * result;
 	struct inode *dir = parent->d_inode;
+	struct dentry *dentry;
 
 	mutex_lock(&dir->i_mutex);
 	/*
@@ -494,38 +495,41 @@ static struct dentry * real_lookup(struct dentry * parent, struct qstr * name, s
 	 * so doing d_lookup() (with seqlock), instead of lockfree __d_lookup
 	 */
 	result = d_lookup(parent, name);
-	if (!result) {
-		struct dentry *dentry;
-
-		/* Don't create child dentry for a dead directory. */
-		result = ERR_PTR(-ENOENT);
-		if (IS_DEADDIR(dir))
-			goto out_unlock;
-
-		dentry = d_alloc(parent, name);
-		result = ERR_PTR(-ENOMEM);
-		if (dentry) {
-			result = dir->i_op->lookup(dir, dentry, nd);
+	if (result) {
+		/*
+		 * The cache was re-populated while we waited on the
+		 * mutex.  We need to revalidate, this time while
+		 * holding i_mutex (to avoid another race).
+		 */
+		if (result->d_op && result->d_op->d_revalidate) {
+			result = do_revalidate(result, nd);
 			if (result)
-				dput(dentry);
-			else
-				result = dentry;
+				goto out_unlock;
+			/*
+			 * The dentry was left behind invalid.  Just
+			 * do the lookup.
+			 */
+		} else {
+			goto out_unlock;
 		}
-out_unlock:
-		mutex_unlock(&dir->i_mutex);
-		return result;
 	}
 
-	/*
-	 * Uhhuh! Nasty case: the cache was re-populated while
-	 * we waited on the semaphore. Need to revalidate.
-	 */
-	mutex_unlock(&dir->i_mutex);
-	if (result->d_op && result->d_op->d_revalidate) {
-		result = do_revalidate(result, nd);
-		if (!result)
-			result = ERR_PTR(-ENOENT);
+	/* Don't create child dentry for a dead directory. */
+	result = ERR_PTR(-ENOENT);
+	if (IS_DEADDIR(dir))
+		goto out_unlock;
+
+	dentry = d_alloc(parent, name);
+	result = ERR_PTR(-ENOMEM);
+	if (dentry) {
+		result = dir->i_op->lookup(dir, dentry, nd);
+		if (result)
+			dput(dentry);
+		else
+			result = dentry;
 	}
+out_unlock:
+	mutex_unlock(&dir->i_mutex);
 	return result;
 }
 
author
#2 lore raw

RFC PATCH 2/11 autofs4 - use macros for active list handling

In reply to Ian Kent

Define some simple macro functions for adding and deleting entries on the active (and unhashed) dentry list.

---

 fs/autofs4/autofs_i.h |    2 ++
 fs/autofs4/inode.c    |    1 +
 fs/autofs4/root.c     |   46 +++++++++++++++++++++++++++++++++++-----------
 3 files changed, 38 insertions(+), 11 deletions(-)
fs/autofs4/autofs_i.h +2 -0
index 8f7cdde..f3cf151 100644
@@ -75,6 +75,8 @@ struct autofs_info {
 	struct completion expire_complete;
 
 	struct list_head active;
+	int active_count;
+
 	struct list_head expiring;
 
 	struct autofs_sb_info *sbi;
fs/autofs4/inode.c +1 -0
index 69c8142..4670a78 100644
@@ -49,6 +49,7 @@ struct autofs_info *autofs4_init_ino(struct autofs_info *ino,
 		ino->dentry = NULL;
 		ino->size = 0;
 		INIT_LIST_HEAD(&ino->active);
+		ino->active_count = 0;
 		INIT_LIST_HEAD(&ino->expiring);
 		atomic_set(&ino->count, 0);
 	}
fs/autofs4/root.c +35 -11

70 lines not on this page · raw message

author
#3 lore raw

RFC PATCH 3/11 autofs4 - use macros for expiring list

In reply to Ian Kent

Define some simple macro functions for adding and deleting entries on the expiring dentry list.

---

 fs/autofs4/autofs_i.h |   26 ++++++++++++++++++++++++++
 fs/autofs4/root.c     |   15 +++------------
 2 files changed, 29 insertions(+), 12 deletions(-)
fs/autofs4/autofs_i.h +26 -0
index f3cf151..fe9fc23 100644
@@ -266,5 +266,31 @@ out:
 	return ret;
 }
 
+static inline void autofs4_add_expiring(struct dentry *dentry)
+{
+	struct autofs_sb_info *sbi = autofs4_sbi(dentry->d_sb);
+	struct autofs_info *ino = autofs4_dentry_ino(dentry);
+	if (ino) {
+		spin_lock(&sbi->lookup_lock);
+		if (list_empty(&ino->expiring))
+			list_add(&ino->expiring, &sbi->expiring_list);
+		spin_unlock(&sbi->lookup_lock);
+	}
+	return;
+}
+
+static inline void autofs4_del_expiring(struct dentry *dentry)
+{
+	struct autofs_sb_info *sbi = autofs4_sbi(dentry->d_sb);
+	struct autofs_info *ino = autofs4_dentry_ino(dentry);
+	if (ino) {
+		spin_lock(&sbi->lookup_lock);
+		if (!list_empty(&ino->expiring))
+			list_del_init(&ino->expiring);
+		spin_unlock(&sbi->lookup_lock);
+	}
+	return;
+}
+
 void autofs4_dentry_release(struct dentry *);
 extern void autofs4_kill_sb(struct super_block *);
fs/autofs4/root.c +3 -12
index 67d8d96..2954ac5 100644
@@ -563,10 +563,7 @@ static struct dentry *autofs4_lookup(struct inode *dir, struct dentry *dentry, s
 			 */
 			ino = autofs4_dentry_ino(expiring);
 			autofs4_expire_wait(expiring);
-			spin_lock(&sbi->lookup_lock);
-			if (!list_empty(&ino->expiring))
-				list_del_init(&ino->expiring);
-			spin_unlock(&sbi->lookup_lock);
+			autofs4_del_expiring(expiring);
 			dput(expiring);
 		}
 
@@ -732,10 +729,7 @@ static int autofs4_dir_unlink(struct inode *dir, struct dentry *dentry)
 	dir->i_mtime = CURRENT_TIME;
 
 	spin_lock(&dcache_lock);
-	spin_lock(&sbi->lookup_lock);
-	if (list_empty(&ino->expiring))
-		list_add(&ino->expiring, &sbi->expiring_list);
-	spin_unlock(&sbi->lookup_lock);
+	autofs4_add_expiring(dentry);
 	spin_lock(&dentry->d_lock);
 	__d_drop(dentry);
 	spin_unlock(&dentry->d_lock);
@@ -761,10 +755,7 @@ static int autofs4_dir_rmdir(struct inode *dir, struct dentry *dentry)
 		spin_unlock(&dcache_lock);
 		return -ENOTEMPTY;
 	}
-	spin_lock(&sbi->lookup_lock);
-	if (list_empty(&ino->expiring))
-		list_add(&ino->expiring, &sbi->expiring_list);
-	spin_unlock(&sbi->lookup_lock);
+	autofs4_add_expiring(dentry);
 	spin_lock(&dentry->d_lock);
 	__d_drop(dentry);
 	spin_unlock(&dentry->d_lock);
author
#4 lore raw

RFC PATCH 4/11 autofs4 - use macro for need mount check

In reply to Ian Kent

Define simple macro function for checking if we need to trigger a mount.

---

 fs/autofs4/root.c |   12 ++++++++++--
 1 files changed, 10 insertions(+), 2 deletions(-)
fs/autofs4/root.c +10 -2
index 2954ac5..f6e8ca9 100644
@@ -104,6 +104,14 @@ static void autofs4_del_active(struct dentry *dentry)
 	return;
 }
 
+static unsigned int autofs4_need_mount(unsigned int flags)
+{
+	unsigned int res = 0;
+	if (flags & (TRIGGER_FLAGS | TRIGGER_INTENTS))
+		res = 1;
+	return res;
+}
+
 static int autofs4_dir_open(struct inode *inode, struct file *file)
 {
 	struct dentry *dentry = file->f_path.dentry;
@@ -168,7 +176,7 @@ static int try_to_fill_dentry(struct dentry *dentry, int flags)
 		}
 	/* Trigger mount for path component or follow link */
 	} else if (dentry->d_flags & DCACHE_AUTOFS_PENDING ||
-			flags & (TRIGGER_FLAGS | TRIGGER_INTENTS) ||
+			autofs4_need_mount(flags) ||
 			current->link_count) {
 		DPRINTK("waiting for mount name=%.*s",
 			dentry->d_name.len, dentry->d_name.name);
@@ -234,7 +242,7 @@ static void *autofs4_follow_link(struct dentry *dentry, struct nameidata *nd)
 	autofs4_expire_wait(dentry);
 
 	/* We trigger a mount for almost all flags */
-	lookup_type = nd->flags & (TRIGGER_FLAGS | TRIGGER_INTENTS);
+	lookup_type = autofs4_need_mount(nd->flags);
 	if (!(lookup_type || dentry->d_flags & DCACHE_AUTOFS_PENDING))
 		goto follow;
 
author
#5 lore raw

RFC PATCH 5/11 autofs4 - use autofs_info for pending flag

In reply to Ian Kent

Eliminate the use of the d_lock spin lock by using the autofs super block info spin lock. This reduces the number of spin locks we use by one and makes the code for the following patch (to redirect ->d_revalidate() to ->lookup()) a little simpler.

---

 fs/autofs4/autofs_i.h |    3 ++-
 fs/autofs4/expire.c   |    2 +-
 fs/autofs4/root.c     |   58 +++++++++++++++++++++++++++----------------------
 3 files changed, 35 insertions(+), 28 deletions(-)
fs/autofs4/autofs_i.h +2 -1
index fe9fc23..3d283ab 100644
@@ -97,6 +97,7 @@ struct autofs_info {
 
 #define AUTOFS_INF_EXPIRING	(1<<0) /* dentry is in the process of expiring */
 #define AUTOFS_INF_MOUNTPOINT	(1<<1) /* mountpoint status for direct expire */
+#define AUTOFS_INF_PENDING	(1<<2) /* dentry pending mount */
 
 struct autofs_wait_queue {
 	wait_queue_head_t queue;
@@ -163,7 +164,7 @@ static inline int autofs4_ispending(struct dentry *dentry)
 {
 	struct autofs_info *inf = autofs4_dentry_ino(dentry);
 
-	if (dentry->d_flags & DCACHE_AUTOFS_PENDING)
+	if (inf->flags & AUTOFS_INF_PENDING)
 		return 1;
 
 	if (inf->flags & AUTOFS_INF_EXPIRING)
fs/autofs4/expire.c +1 -1
index 3da18d4..a796c94 100644
@@ -27,7 +27,7 @@ static inline int autofs4_can_expire(struct dentry *dentry,
 		return 0;
 
 	/* No point expiring a pending mount */
-	if (dentry->d_flags & DCACHE_AUTOFS_PENDING)
+	if (ino->flags & AUTOFS_INF_PENDING)
 		return 0;
 
 	if (!do_now) {
fs/autofs4/root.c +32 -26

137 lines not on this page · raw message

author
#6 lore raw

RFC PATCH 6/11 autofs4 - renamer unhashed to active in autofs4_lookup()

In reply to Ian Kent

Rename the variable unhashed to active in autofs4_lookup() to better reflect its usage.

---

 fs/autofs4/root.c |   20 ++++++++++----------
 1 files changed, 10 insertions(+), 10 deletions(-)
fs/autofs4/root.c +10 -10
index 305136b..961ff37 100644
@@ -514,7 +514,7 @@ static struct dentry *autofs4_lookup(struct inode *dir, struct dentry *dentry, s
 {
 	struct autofs_sb_info *sbi;
 	struct autofs_info *ino;
-	struct dentry *expiring, *unhashed;
+	struct dentry *expiring, *active;
 	int oz_mode;
 
 	DPRINTK("name = %.*s",
@@ -530,9 +530,9 @@ static struct dentry *autofs4_lookup(struct inode *dir, struct dentry *dentry, s
 	DPRINTK("pid = %u, pgrp = %u, catatonic = %d, oz_mode = %d",
 		 current->pid, task_pgrp_nr(current), sbi->catatonic, oz_mode);
 
-	unhashed = autofs4_lookup_active(sbi, dentry->d_parent, &dentry->d_name);
-	if (unhashed) {
-		dentry = unhashed;
+	active = autofs4_lookup_active(sbi, dentry->d_parent, &dentry->d_name);
+	if (active) {
+		dentry = active;
 		ino = autofs4_dentry_ino(dentry);
 	} else {
 		/*
@@ -600,8 +600,8 @@ static struct dentry *autofs4_lookup(struct inode *dir, struct dentry *dentry, s
 			if (sigismember (sigset, SIGKILL) ||
 			    sigismember (sigset, SIGQUIT) ||
 			    sigismember (sigset, SIGINT)) {
-			    if (unhashed)
-				dput(unhashed);
+			    if (active)
+				dput(active);
 			    return ERR_PTR(-ERESTARTNOINTR);
 			}
 		}
@@ -633,14 +633,14 @@ static struct dentry *autofs4_lookup(struct inode *dir, struct dentry *dentry, s
 		else
 			dentry = ERR_PTR(-ENOENT);
 
-		if (unhashed)
-			dput(unhashed);
+		if (active)
+			dput(active);
 
 		return dentry;
 	}
 
-	if (unhashed)
-		return unhashed;
+	if (active)
+		return active;
 
 	return NULL;
 }
author
#7 lore raw

RFC PATCH 7/11 autofs4 - cleanup active and expire lookup

In reply to Ian Kent

The lookup functions for active and expiring dentrys use parameters that can be easily obtained on entry so we change the call to to take just the dentry. This makes the subsequent change, to send all lookups to ->lookup(), a bit cleaner.

---

 fs/autofs4/root.c |   16 ++++++++++------
 1 files changed, 10 insertions(+), 6 deletions(-)
fs/autofs4/root.c +10 -6
index 961ff37..81700f4 100644
@@ -405,8 +405,11 @@ static const struct dentry_operations autofs4_dentry_operations = {
 	.d_release	= autofs4_dentry_release,
 };
 
-static struct dentry *autofs4_lookup_active(struct autofs_sb_info *sbi, struct dentry *parent, struct qstr *name)
+static struct dentry *autofs4_lookup_active(struct dentry *dentry)
 {
+	struct autofs_sb_info *sbi = autofs4_sbi(dentry->d_sb);
+	struct dentry *parent = dentry->d_parent;
+	struct qstr *name = &dentry->d_name;
 	unsigned int len = name->len;
 	unsigned int hash = name->hash;
 	const unsigned char *str = name->name;
@@ -457,8 +460,11 @@ next:
 	return NULL;
 }
 
-static struct dentry *autofs4_lookup_expiring(struct autofs_sb_info *sbi, struct dentry *parent, struct qstr *name)
+static struct dentry *autofs4_lookup_expiring(struct dentry *dentry)
 {
+	struct autofs_sb_info *sbi = autofs4_sbi(dentry->d_sb);
+	struct dentry *parent = dentry->d_parent;
+	struct qstr *name = &dentry->d_name;
 	unsigned int len = name->len;
 	unsigned int hash = name->hash;
 	const unsigned char *str = name->name;
@@ -530,7 +536,7 @@ static struct dentry *autofs4_lookup(struct inode *dir, struct dentry *dentry, s
 	DPRINTK("pid = %u, pgrp = %u, catatonic = %d, oz_mode = %d",
 		 current->pid, task_pgrp_nr(current), sbi->catatonic, oz_mode);
 
-	active = autofs4_lookup_active(sbi, dentry->d_parent, &dentry->d_name);
+	active = autofs4_lookup_active(dentry);
 	if (active) {
 		dentry = active;
 		ino = autofs4_dentry_ino(dentry);
@@ -567,9 +573,7 @@ static struct dentry *autofs4_lookup(struct inode *dir, struct dentry *dentry, s
 
 	if (!oz_mode) {
 		mutex_unlock(&dir->i_mutex);
-		expiring = autofs4_lookup_expiring(sbi,
-						   dentry->d_parent,
-						   &dentry->d_name);
+		expiring = autofs4_lookup_expiring(dentry);
 		if (expiring) {
 			/*
 			 * If we are racing with expire the request might not
author
#8 lore raw

RFC PATCH 8/11 autofs4 - eliminate d_unhashed in path walk checks

In reply to Ian Kent

We unhash the dentry (in a subsequent patch) in ->d_revalidate() in order to send mount requests to ->lookup(). But then we can not rely on d_unhased() to give reliable results because it may be called at any time by any code path. The d_unhashed() function is used by __simple_empty() in the path walking callbacks but autofs mount point dentrys should have no directories at all so a list_empty() on d_subdirs should be (and is) sufficient.

---

 fs/autofs4/root.c |    7 +++----
 1 files changed, 3 insertions(+), 4 deletions(-)
fs/autofs4/root.c +3 -4
index 81700f4..b6530f3 100644
@@ -133,7 +133,7 @@ static int autofs4_dir_open(struct inode *inode, struct file *file)
 	 * it.
 	 */
 	spin_lock(&dcache_lock);
-	if (!d_mountpoint(dentry) && __simple_empty(dentry)) {
+	if (!d_mountpoint(dentry) && list_empty(&dentry->d_subdirs)) {
 		spin_unlock(&dcache_lock);
 		return -ENOENT;
 	}
@@ -257,7 +257,7 @@ static void *autofs4_follow_link(struct dentry *dentry, struct nameidata *nd)
 	 * mount it again.
 	 */
 	if (ino->flags & AUTOFS_INF_PENDING ||
-	    (!d_mountpoint(dentry) && __simple_empty(dentry))) {
+	    (!d_mountpoint(dentry) && list_empty(&dentry->d_subdirs))) {
 		spin_unlock(&dcache_lock);
 		spin_unlock(&sbi->fs_lock);
 
@@ -340,8 +340,7 @@ static int autofs4_revalidate(struct dentry *dentry, struct nameidata *nd)
 	/* Check for a non-mountpoint directory with no contents */
 	spin_lock(&dcache_lock);
 	if (S_ISDIR(dentry->d_inode->i_mode) &&
-	    !d_mountpoint(dentry) && 
-	    __simple_empty(dentry)) {
+	    !d_mountpoint(dentry) && list_empty(&dentry->d_subdirs)) {
 		DPRINTK("dentry=%p %.*s, emptydir",
 			 dentry, dentry->d_name.len, dentry->d_name.name);
 		spin_unlock(&dcache_lock);
author
#9 lore raw

RFC PATCH 9/11 autofs4 - rename dentry to active in autofs4_lookup_active()

In reply to Ian Kent

In autofs4_lookup_active() a declaration within the list traversal loop uses a declaration that has the same name as the function parameter.

---

 fs/autofs4/root.c |   24 ++++++++++++------------
 1 files changed, 12 insertions(+), 12 deletions(-)
fs/autofs4/root.c +12 -12
index b6530f3..e8a8881 100644
@@ -419,23 +419,23 @@ static struct dentry *autofs4_lookup_active(struct dentry *dentry)
 	head = &sbi->active_list;
 	list_for_each(p, head) {
 		struct autofs_info *ino;
-		struct dentry *dentry;
+		struct dentry *active;
 		struct qstr *qstr;
 
 		ino = list_entry(p, struct autofs_info, active);
-		dentry = ino->dentry;
+		active = ino->dentry;
 
-		spin_lock(&dentry->d_lock);
+		spin_lock(&active->d_lock);
 
 		/* Already gone? */
-		if (atomic_read(&dentry->d_count) == 0)
+		if (atomic_read(&active->d_count) == 0)
 			goto next;
 
-		qstr = &dentry->d_name;
+		qstr = &active->d_name;
 
-		if (dentry->d_name.hash != hash)
+		if (active->d_name.hash != hash)
 			goto next;
-		if (dentry->d_parent != parent)
+		if (active->d_parent != parent)
 			goto next;
 
 		if (qstr->len != len)
@@ -443,15 +443,15 @@ static struct dentry *autofs4_lookup_active(struct dentry *dentry)
 		if (memcmp(qstr->name, str, len))
 			goto next;
 
-		if (d_unhashed(dentry)) {
-			dget(dentry);
-			spin_unlock(&dentry->d_lock);
+		if (d_unhashed(active)) {
+			dget(active);
+			spin_unlock(&active->d_lock);
 			spin_unlock(&sbi->lookup_lock);
 			spin_unlock(&dcache_lock);
-			return dentry;
+			return active;
 		}
 next:
-		spin_unlock(&dentry->d_lock);
+		spin_unlock(&active->d_lock);
 	}
 	spin_unlock(&sbi->lookup_lock);
 	spin_unlock(&dcache_lock);
author
#10 lore raw

RFC PATCH 10/11 autofs4 - rename dentry to expiring in autofs4_lookup_expiring()

In reply to Ian Kent

In autofs4_lookup_expiring() a declaration within the list traversal loop uses a declaration that has the same name as the function parameter.

---

 fs/autofs4/root.c |   24 ++++++++++++------------
 1 files changed, 12 insertions(+), 12 deletions(-)
fs/autofs4/root.c +12 -12
index e8a8881..a015b49 100644
@@ -474,23 +474,23 @@ static struct dentry *autofs4_lookup_expiring(struct dentry *dentry)
 	head = &sbi->expiring_list;
 	list_for_each(p, head) {
 		struct autofs_info *ino;
-		struct dentry *dentry;
+		struct dentry *expiring;
 		struct qstr *qstr;
 
 		ino = list_entry(p, struct autofs_info, expiring);
-		dentry = ino->dentry;
+		expiring = ino->dentry;
 
-		spin_lock(&dentry->d_lock);
+		spin_lock(&expiring->d_lock);
 
 		/* Bad luck, we've already been dentry_iput */
-		if (!dentry->d_inode)
+		if (!expiring->d_inode)
 			goto next;
 
-		qstr = &dentry->d_name;
+		qstr = &expiring->d_name;
 
-		if (dentry->d_name.hash != hash)
+		if (expiring->d_name.hash != hash)
 			goto next;
-		if (dentry->d_parent != parent)
+		if (expiring->d_parent != parent)
 			goto next;
 
 		if (qstr->len != len)
@@ -498,15 +498,15 @@ static struct dentry *autofs4_lookup_expiring(struct dentry *dentry)
 		if (memcmp(qstr->name, str, len))
 			goto next;
 
-		if (d_unhashed(dentry)) {
-			dget(dentry);
-			spin_unlock(&dentry->d_lock);
+		if (d_unhashed(expiring)) {
+			dget(expiring);
+			spin_unlock(&expiring->d_lock);
 			spin_unlock(&sbi->lookup_lock);
 			spin_unlock(&dcache_lock);
-			return dentry;
+			return expiring;
 		}
 next:
-		spin_unlock(&dentry->d_lock);
+		spin_unlock(&expiring->d_lock);
 	}
 	spin_unlock(&sbi->lookup_lock);
 	spin_unlock(&dcache_lock);
author
#11 lore raw

RFC PATCH 11/11 autofs4 - always use lookup for lookup

In reply to Ian Kent

We need to be able to cope with the directory mutex being held during ->d_revalidate() in some cases, but not all cases, and not necessarily by us. Because we need to release the mutex when we call back to the daemon to do perform a mount we must be sure that it is us who holds the mutex so we must redirect mount requests to ->lookup() if the mutex is held.

---

 fs/autofs4/autofs_i.h |    7 +
 fs/autofs4/expire.c   |    6 +
 fs/autofs4/inode.c    |    1 
 fs/autofs4/root.c     |  474 +++++++++++++++++++++++++++++++++----------------
 4 files changed, 330 insertions(+), 158 deletions(-)
fs/autofs4/autofs_i.h +7 -0
index 3d283ab..0118d67 100644
@@ -60,6 +60,11 @@ do {							\
 		current->pid, __func__, ##args);	\
 } while (0)
 
+struct rehash_entry {
+	struct task_struct *task;
+	struct list_head list;
+};
+
 /* Unified info structure.  This is pointed to by both the dentry and
    inode structures.  Each file in the filesystem has an instance of this
    structure.  It holds a reference to the dentry, so dentries are never
@@ -76,6 +81,7 @@ struct autofs_info {
 
 	struct list_head active;
 	int active_count;
+	struct list_head rehash_list;
 
 	struct list_head expiring;
 
@@ -98,6 +104,7 @@ struct autofs_info {
 #define AUTOFS_INF_EXPIRING	(1<<0) /* dentry is in the process of expiring */
 #define AUTOFS_INF_MOUNTPOINT	(1<<1) /* mountpoint status for direct expire */
 #define AUTOFS_INF_PENDING	(1<<2) /* dentry pending mount */
+#define AUTOFS_INF_REHASH	(1<<3) /* dentry in transit to ->lookup() */
 
 struct autofs_wait_queue {
 	wait_queue_head_t queue;
fs/autofs4/expire.c +5 -1
index a796c94..74bc9aa 100644
@@ -279,6 +279,7 @@ struct dentry *autofs4_expire_direct(struct super_block *sb,
 			root->d_mounted--;
 		}
 		ino->flags |= AUTOFS_INF_EXPIRING;
+		autofs4_add_expiring(root);
 		init_completion(&ino->expire_complete);
 		spin_unlock(&sbi->fs_lock);
 		return root;
@@ -406,6 +407,7 @@ found:
 		expired, (int)expired->d_name.len, expired->d_name.name);
 	ino = autofs4_dentry_ino(expired);
 	ino->flags |= AUTOFS_INF_EXPIRING;
+	autofs4_add_expiring(expired);
 	init_completion(&ino->expire_complete);
 	spin_unlock(&sbi->fs_lock);
 	spin_lock(&dcache_lock);
@@ -433,7 +435,7 @@ int autofs4_expire_wait(struct dentry *dentry)
 
 		DPRINTK("expire done status=%d", status);
 
-		if (d_unhashed(dentry))
+		if (d_unhashed(dentry) && IS_DEADDIR(dentry->d_inode))
 			return -EAGAIN;
 
 		return status;
@@ -473,6 +475,7 @@ int autofs4_expire_run(struct super_block *sb,
 	spin_lock(&sbi->fs_lock);
 	ino = autofs4_dentry_ino(dentry);
 	ino->flags &= ~AUTOFS_INF_EXPIRING;
+	autofs4_del_expiring(dentry);
 	complete_all(&ino->expire_complete);
 	spin_unlock(&sbi->fs_lock);
 
@@ -503,6 +506,7 @@ int autofs4_do_expire_multi(struct super_block *sb, struct vfsmount *mnt,
 			ino->flags &= ~AUTOFS_INF_MOUNTPOINT;
 		}
 		ino->flags &= ~AUTOFS_INF_EXPIRING;
+		autofs4_del_expiring(dentry);
 		complete_all(&ino->expire_complete);
 		spin_unlock(&sbi->fs_lock);
 		dput(dentry);
fs/autofs4/inode.c +1 -0
index 4670a78..d0a3de2 100644
@@ -49,6 +49,7 @@ struct autofs_info *autofs4_init_ino(struct autofs_info *ino,
 		ino->dentry = NULL;
 		ino->size = 0;
 		INIT_LIST_HEAD(&ino->active);
+		INIT_LIST_HEAD(&ino->rehash_list);
 		ino->active_count = 0;
 		INIT_LIST_HEAD(&ino->expiring);
 		atomic_set(&ino->count, 0);
fs/autofs4/root.c +317 -157

654 lines not on this page · raw message

author
#12 lore raw
In reply to Ian Kent
Quoted 4 lines · Ian Kent

A change to the VFS path walk locking is needed to resolve an issue identified by Sage Weil. This locking change requires significant changes to the autofs4 module to allow it to callback to userspace without introducing a deadlock.

Oops, meant to say this series is against the current Linus tree and not
the current mm tree. However, I don't expect there will be differences
in the autofs4 patches.

Ian

#13 lore raw
In reply to Ian Kent
Quoted 17 lines · Ian Kent

A change to the VFS path walk locking is needed to resolve an issue identified by Sage Weil. This locking change requires significant changes to the autofs4 module to allow it to callback to userspace without introducing a deadlock.

To cope with the change the autofs4 module needs to redirect mount requests from ->d_revalidate() to ->lookup() if the directory inode mutex is held when a callback needs to be done. Note that we cannot redirect these requests when the mutex is not held because, to function correctly, the mutex must be held over both revalidate and lookup.

Of the patches in the series most are cleanups and refactoring done to keep the real change in "autofs4 - always use lookup for lookup" as clean as possible. Unfortuneately, there is still quite a bit left in it.

Quoted 5 lines

Also, I need confirmation that the patch that changes the VFS path walk locking is in fact correct, or at least like for like to what will be submitted. I had some difficulty with the original patches that were paosted. The patch in question below is "vfs: make real_lookup do dentry revalidation with i_mutex held".

It looks identical to be the original two folded into one patch. I'll repost those two now, freshened against Linus' tree. The first has just the functional change, and the cleanup is in the second (as per Christoph's review).

Quoted 4 lines

I've done quite a bit of fairly heavy stress testing of the patch series and they (finally) hold up to it. Although I have also managed to uncover a locking bug in the user space daemon as a result, ;)

I'm glad some other good has come of it. Thanks, Ian, for carrying this through!

sage

Quoted 30 lines

---

Ian Kent (10):
autofs4 - always use lookup for lookup
autofs4 - rename dentry to expiring in autofs4_lookup_expiring() autofs4 - rename dentry to active in autofs4_lookup_active() autofs4 - eliminate d_unhashed in path walk checks autofs4 - cleanup active and expire lookup
autofs4 - renamer unhashed to active in autofs4_lookup() autofs4 - use autofs_info for pending flag
autofs4 - use macro for need mount check
autofs4 - use macros for expiring list
autofs4 - use macros for active list handling

Sage Weil (1):
Subject: [PATCH] vfs: make real_lookup do dentry revalidation with i_mutex held

 fs/autofs4/autofs_i.h |   38 +++
 fs/autofs4/expire.c   |    8 -
 fs/autofs4/inode.c    |    2 
 fs/autofs4/root.c     |  616 ++++++++++++++++++++++++++++++++-----------------
 fs/namei.c            |   58 ++---
 5 files changed, 480 insertions(+), 242 deletions(-)
Signature · 1 line
Ian
author
#14 lore raw
Quoted 30 lines · Sage Weil
Quoted 17 lines · Ian Kent

A change to the VFS path walk locking is needed to resolve an issue identified by Sage Weil. This locking change requires significant changes to the autofs4 module to allow it to callback to userspace without introducing a deadlock.

To cope with the change the autofs4 module needs to redirect mount requests from ->d_revalidate() to ->lookup() if the directory inode mutex is held when a callback needs to be done. Note that we cannot redirect these requests when the mutex is not held because, to function correctly, the mutex must be held over both revalidate and lookup.

Of the patches in the series most are cleanups and refactoring done to keep the real change in "autofs4 - always use lookup for lookup" as clean as possible. Unfortuneately, there is still quite a bit left in it.

Quoted 5 lines

Also, I need confirmation that the patch that changes the VFS path walk locking is in fact correct, or at least like for like to what will be submitted. I had some difficulty with the original patches that were paosted. The patch in question below is "vfs: make real_lookup do dentry revalidation with i_mutex held".

It looks identical to be the original two folded into one patch. I'll repost those two now, freshened against Linus' tree. The first has just the functional change, and the cleanup is in the second (as per Christoph's review).

Quoted 7 lines
Quoted 4 lines

I've done quite a bit of fairly heavy stress testing of the patch series and they (finally) hold up to it. Although I have also managed to uncover a locking bug in the user space daemon as a result, ;)

I'm glad some other good has come of it. Thanks, Ian, for carrying this through!

Not quite so good as I haven't fixed it yet.
But that's user space and it happens occasionally under an extended heavy load so it will take a while to work out.

Ian

author
#15 lore raw
In reply to Sage Weil
Quoted 25 lines · Sage Weil
Quoted 17 lines · Ian Kent

A change to the VFS path walk locking is needed to resolve an issue identified by Sage Weil. This locking change requires significant changes to the autofs4 module to allow it to callback to userspace without introducing a deadlock.

To cope with the change the autofs4 module needs to redirect mount requests from ->d_revalidate() to ->lookup() if the directory inode mutex is held when a callback needs to be done. Note that we cannot redirect these requests when the mutex is not held because, to function correctly, the mutex must be held over both revalidate and lookup.

Of the patches in the series most are cleanups and refactoring done to keep the real change in "autofs4 - always use lookup for lookup" as clean as possible. Unfortuneately, there is still quite a bit left in it.

Quoted 5 lines

Also, I need confirmation that the patch that changes the VFS path walk locking is in fact correct, or at least like for like to what will be submitted. I had some difficulty with the original patches that were paosted. The patch in question below is "vfs: make real_lookup do dentry revalidation with i_mutex held".

Quoted 4 lines

It looks identical to be the original two folded into one patch. I'll repost those two now, freshened against Linus' tree. The first has just the functional change, and the cleanup is in the second (as per Christoph's review).

Yes, after checking, the patch I was using is functionally the same as the combination of the two you re-posted. So all is good for may part.

Ian