autofs4 - update autofs4 to deal with VFS locking change
cover autofs4 - update autofs4 to deal with VFS locking change
A change to the VFS path walk locking is needed to resolve an issue identified by Sage Weil. This locking change requires significant changes to the autofs4 module to allow it to callback to userspace without introducing a deadlock.
To cope with the change the autofs4 module needs to redirect mount requests from ->d_revalidate() to ->lookup() if the directory inode mutex is held when a callback needs to be done. Note that we cannot redirect these requests when the mutex is not held because, to function correctly, the mutex must be held over both revalidate and lookup.
Of the patches in the series most are cleanups and refactoring done to keep the real change in "autofs4 - always use lookup for lookup" as clean as possible. Unfortuneately, there is still quite a bit left in it.
Also, I need confirmation that the patch that changes the VFS path walk locking is in fact correct, or at least like for like to what will be submitted. I had some difficulty with the original patches that were paosted. The patch in question below is "vfs: make real_lookup do dentry revalidation with i_mutex held".
I've done quite a bit of fairly heavy stress testing of the patch series and they (finally) hold up to it. Although I have also managed to uncover a locking bug in the user space daemon as a result, ;)
---
Ian Kent (10):
autofs4 - always use lookup for lookup
autofs4 - rename dentry to expiring in autofs4_lookup_expiring() autofs4 - rename dentry to active in autofs4_lookup_active() autofs4 - eliminate d_unhashed in path walk checks autofs4 - cleanup active and expire lookup
autofs4 - renamer unhashed to active in autofs4_lookup() autofs4 - use autofs_info for pending flag
autofs4 - use macro for need mount check
autofs4 - use macros for expiring list
autofs4 - use macros for active list handling
Sage Weil (1):
Subject: [PATCH] vfs: make real_lookup do dentry revalidation with i_mutex held
fs/autofs4/autofs_i.h | 38 +++ fs/autofs4/expire.c | 8 - fs/autofs4/inode.c | 2 fs/autofs4/root.c | 616 ++++++++++++++++++++++++++++++++----------------- fs/namei.c | 58 ++--- 5 files changed, 480 insertions(+), 242 deletions(-)
Signature · 1 line
Ian
RFC PATCH 1/11 Subject: [PATCH] vfs: make real_lookup do dentry revalidation with i_mutex held
From: Sage Weil <[email protected]>
real_lookup() is called by do_lookup() if dentry revalidation fails. If the cache is re-populated while waiting for i_mutex, it may find that a d_lookup() subsequently succeeds (see the "Uhhuh! Nasty case" comment).
Previously, real_lookup() would drop i_mutex and do_revalidate() again. If revalidate failed _again_, however, it would give up with -ENOENT. The problem here that network file systems may be invalidating dentries via server callbacks, e.g. due to concurrent access from another client, and -ENOENT is frequently the wrong answer.
This problem has been seen with both Lustre and Ceph. It seems possible to hit this case with NFS as well if the cache lifetime is very short.
Instead, we should do_revalidate() while i_mutex is still held. If revalidation fails, we can move on to a ->lookup() and ensure a correct result without worrying about any subsequent races.
Note that do_revalidate() is called with i_mutex held elsewhere. For example, do_filp_open(), lookup_create(), do_unlinkat(), do_rmdir(), and possibly others all take the directory i_mutex, and then
-> lookup_hash
-> __lookup_hash
-> cached_lookup
-> do_revalidateso this does not introduce any new locking rules for d_revalidate implementations.
---
fs/namei.c | 58 +++++++++++++++++++++++++++++++--------------------------- 1 files changed, 31 insertions(+), 27 deletions(-)
fs/namei.c +31 -27
@@ -477,6 +477,7 @@ static struct dentry * real_lookup(struct dentry * parent, struct qstr * name, s {struct dentry * result;struct inode *dir = parent->d_inode; +struct dentry *dentry;mutex_lock (&dir->i_mutex);/*
@@ -494,38 +495,41 @@ static struct dentry * real_lookup(struct dentry * parent, struct qstr * name, s* so doing d_lookup() (with seqlock), instead of lockfree __d_lookup */ result =d_lookup (parent, name); -if (!result) { -struct dentry *dentry; - -/* Don't create child dentry for a dead directory. */ - result =ERR_PTR (-ENOENT ); -if (IS_DEADDIR (dir)) - goto out_unlock; - - dentry =d_alloc (parent, name); - result =ERR_PTR (-ENOMEM ); -if (dentry) { - result = dir->i_op->lookup (dir, dentry, nd); +if (result) { +/* +* The cache was re-populated while we waited on the +* mutex. We need to revalidate, this time while +* holding i_mutex (to avoid another race). +*/ +if (result->d_op && result->d_op->d_revalidate) { + result =do_revalidate (result, nd);if (result) -dput (dentry); -else - result = dentry; + goto out_unlock; +/* +* The dentry was left behind invalid. Just +* do the lookup. +*/ + }else { + goto out_unlock; } -out_unlock: -mutex_unlock (&dir->i_mutex); -return result; } -/* -* Uhhuh! Nasty case: the cache was re-populated while -* we waited on the semaphore. Need to revalidate. -*/ -mutex_unlock (&dir->i_mutex); -if (result->d_op && result->d_op->d_revalidate) { - result =do_revalidate (result, nd); -if (!result) - result =ERR_PTR (-ENOENT ); +/* Don't create child dentry for a dead directory. */ + result =ERR_PTR (-ENOENT ); +if (IS_DEADDIR (dir)) + goto out_unlock; + + dentry =d_alloc (parent, name); + result =ERR_PTR (-ENOMEM ); +if (dentry) { + result = dir->i_op->lookup (dir, dentry, nd); +if (result) +dput (dentry); +else + result = dentry; } +out_unlock: +mutex_unlock (&dir->i_mutex);return result; }
RFC PATCH 2/11 autofs4 - use macros for active list handling
In reply to Ian KentDefine some simple macro functions for adding and deleting entries on the active (and unhashed) dentry list.
---
fs/autofs4/autofs_i.h | 2 ++ fs/autofs4/inode.c | 1 + fs/autofs4/root.c | 46 +++++++++++++++++++++++++++++++++++----------- 3 files changed, 38 insertions(+), 11 deletions(-)
fs/autofs4/autofs_i.h +2 -0
@@ -75,6 +75,8 @@ struct autofs_info {struct completion expire_complete;struct list_head active; +int active_count; +struct list_head expiring;struct autofs_sb_info *sbi;
fs/autofs4/inode.c +1 -0
@@ -49,6 +49,7 @@ struct autofs_info *autofs4_init_ino(struct autofs_info *ino, ino->dentry =NULL ; ino->size =0 ;INIT_LIST_HEAD (&ino->active); + ino->active_count =0 ;INIT_LIST_HEAD (&ino->expiring);atomic_set (&ino->count,0 ); }
fs/autofs4/root.c +35 -11
70 lines not on this page · raw message
RFC PATCH 3/11 autofs4 - use macros for expiring list
In reply to Ian KentDefine some simple macro functions for adding and deleting entries on the expiring dentry list.
---
fs/autofs4/autofs_i.h | 26 ++++++++++++++++++++++++++ fs/autofs4/root.c | 15 +++------------ 2 files changed, 29 insertions(+), 12 deletions(-)
fs/autofs4/autofs_i.h +26 -0
@@ -266,5 +266,31 @@ out:return ret; } +static inline void autofs4_add_expiring (struct dentry *dentry) +{ +struct autofs_sb_info *sbi =autofs4_sbi (dentry->d_sb); +struct autofs_info *ino =autofs4_dentry_ino (dentry); +if (ino) { +spin_lock (&sbi->lookup_lock); +if (list_empty (&ino->expiring)) +list_add (&ino->expiring, &sbi->expiring_list); +spin_unlock (&sbi->lookup_lock); + } +return ; +} + +static inline void autofs4_del_expiring (struct dentry *dentry) +{ +struct autofs_sb_info *sbi =autofs4_sbi (dentry->d_sb); +struct autofs_info *ino =autofs4_dentry_ino (dentry); +if (ino) { +spin_lock (&sbi->lookup_lock); +if (!list_empty (&ino->expiring)) +list_del_init (&ino->expiring); +spin_unlock (&sbi->lookup_lock); + } +return ; +} +void autofs4_dentry_release (struct dentry *);extern void autofs4_kill_sb (struct super_block *);
fs/autofs4/root.c +3 -12
@@ -563,10 +563,7 @@ static struct dentry *autofs4_lookup(struct inode *dir, struct dentry *dentry, s*/ ino =autofs4_dentry_ino (expiring);autofs4_expire_wait (expiring); -spin_lock (&sbi->lookup_lock); -if (!list_empty (&ino->expiring)) -list_del_init (&ino->expiring); -spin_unlock (&sbi->lookup_lock); +autofs4_del_expiring (expiring);dput (expiring); }
@@ -732,10 +729,7 @@ static int autofs4_dir_unlink(struct inode *dir, struct dentry *dentry) dir->i_mtime =CURRENT_TIME ;spin_lock (&dcache_lock); -spin_lock (&sbi->lookup_lock); -if (list_empty (&ino->expiring)) -list_add (&ino->expiring, &sbi->expiring_list); -spin_unlock (&sbi->lookup_lock); +autofs4_add_expiring (dentry);spin_lock (&dentry->d_lock);__d_drop (dentry);spin_unlock (&dentry->d_lock);
@@ -761,10 +755,7 @@ static int autofs4_dir_rmdir(struct inode *dir, struct dentry *dentry)spin_unlock (&dcache_lock);return -ENOTEMPTY ; } -spin_lock (&sbi->lookup_lock); -if (list_empty (&ino->expiring)) -list_add (&ino->expiring, &sbi->expiring_list); -spin_unlock (&sbi->lookup_lock); +autofs4_add_expiring (dentry);spin_lock (&dentry->d_lock);__d_drop (dentry);spin_unlock (&dentry->d_lock);
RFC PATCH 4/11 autofs4 - use macro for need mount check
In reply to Ian KentDefine simple macro function for checking if we need to trigger a mount.
---
fs/autofs4/root.c | 12 ++++++++++-- 1 files changed, 10 insertions(+), 2 deletions(-)
fs/autofs4/root.c +10 -2
@@ -104,6 +104,14 @@ static void autofs4_del_active(struct dentry *dentry)return ; } +static unsigned int autofs4_need_mount (unsigned int flags) +{ +unsigned int res =0 ; +if (flags & (TRIGGER_FLAGS |TRIGGER_INTENTS )) + res =1 ; +return res; +} +static int autofs4_dir_open (struct inode *inode,struct file *file) {struct dentry *dentry = file->f_path.dentry;
@@ -168,7 +176,7 @@ static int try_to_fill_dentry(struct dentry *dentry, int flags) }/* Trigger mount for path component or follow link */ }else if (dentry->d_flags &DCACHE_AUTOFS_PENDING || - flags & (TRIGGER_FLAGS |TRIGGER_INTENTS ) || +autofs4_need_mount (flags) || current->link_count) {DPRINTK ("waiting for mount name=%.*s" , dentry->d_name.len, dentry->d_name.name);
@@ -234,7 +242,7 @@ static void *autofs4_follow_link(struct dentry *dentry, struct nameidata *nd)autofs4_expire_wait (dentry);/* We trigger a mount for almost all flags */ - lookup_type = nd->flags & (TRIGGER_FLAGS |TRIGGER_INTENTS ); + lookup_type =autofs4_need_mount (nd->flags);if (!(lookup_type || dentry->d_flags &DCACHE_AUTOFS_PENDING )) goto follow;
RFC PATCH 5/11 autofs4 - use autofs_info for pending flag
In reply to Ian KentEliminate the use of the d_lock spin lock by using the autofs super block info spin lock. This reduces the number of spin locks we use by one and makes the code for the following patch (to redirect ->d_revalidate() to ->lookup()) a little simpler.
---
fs/autofs4/autofs_i.h | 3 ++- fs/autofs4/expire.c | 2 +- fs/autofs4/root.c | 58 +++++++++++++++++++++++++++---------------------- 3 files changed, 35 insertions(+), 28 deletions(-)
fs/autofs4/autofs_i.h +2 -1
@@ -97,6 +97,7 @@ struct autofs_info {#define AUTOFS_INF_EXPIRING (1<<0)/* dentry is in the process of expiring */ #define AUTOFS_INF_MOUNTPOINT (1<<1)/* mountpoint status for direct expire */ +#define AUTOFS_INF_PENDING (1<<2)/* dentry pending mount */ struct autofs_wait_queue {wait_queue_head_t queue;
@@ -163,7 +164,7 @@ static inline int autofs4_ispending(struct dentry *dentry) {struct autofs_info *inf =autofs4_dentry_ino (dentry); -if (dentry->d_flags &DCACHE_AUTOFS_PENDING ) +if (inf->flags &AUTOFS_INF_PENDING )return 1 ;if (inf->flags &AUTOFS_INF_EXPIRING )
fs/autofs4/expire.c +1 -1
@@ -27,7 +27,7 @@ static inline int autofs4_can_expire(struct dentry *dentry,return 0 ;/* No point expiring a pending mount */ -if (dentry->d_flags &DCACHE_AUTOFS_PENDING ) +if (ino->flags &AUTOFS_INF_PENDING )return 0 ;if (!do_now) {
fs/autofs4/root.c +32 -26
137 lines not on this page · raw message
RFC PATCH 6/11 autofs4 - renamer unhashed to active in autofs4_lookup()
In reply to Ian KentRename the variable unhashed to active in autofs4_lookup() to better reflect its usage.
---
fs/autofs4/root.c | 20 ++++++++++---------- 1 files changed, 10 insertions(+), 10 deletions(-)
fs/autofs4/root.c +10 -10
@@ -514,7 +514,7 @@ static struct dentry *autofs4_lookup(struct inode *dir, struct dentry *dentry, s {struct autofs_sb_info *sbi;struct autofs_info *ino; -struct dentry *expiring, *unhashed; +struct dentry *expiring, *active;int oz_mode; DPRINTK("name = %.*s",
@@ -530,9 +530,9 @@ static struct dentry *autofs4_lookup(struct inode *dir, struct dentry *dentry, sDPRINTK ("pid = %u, pgrp = %u, catatonic = %d, oz_mode = %d" , current->pid,task_pgrp_nr (current), sbi->catatonic, oz_mode); - unhashed =autofs4_lookup_active (sbi, dentry->d_parent, &dentry->d_name); -if (unhashed) { - dentry = unhashed; + active =autofs4_lookup_active (sbi, dentry->d_parent, &dentry->d_name); +if (active) { + dentry = active; ino =autofs4_dentry_ino (dentry); }else {/*
@@ -600,8 +600,8 @@ static struct dentry *autofs4_lookup(struct inode *dir, struct dentry *dentry, sif (sigismember (sigset,SIGKILL ) ||sigismember (sigset,SIGQUIT ) ||sigismember (sigset,SIGINT )) { -if (unhashed) -dput (unhashed); +if (active) +dput (active);return ERR_PTR (-ERESTARTNOINTR ); } }
@@ -633,14 +633,14 @@ static struct dentry *autofs4_lookup(struct inode *dir, struct dentry *dentry, selse dentry =ERR_PTR (-ENOENT ); -if (unhashed) -dput (unhashed); +if (active) +dput (active);return dentry; } -if (unhashed) -return unhashed; +if (active) +return active;return NULL ; }
RFC PATCH 7/11 autofs4 - cleanup active and expire lookup
In reply to Ian KentThe lookup functions for active and expiring dentrys use parameters that can be easily obtained on entry so we change the call to to take just the dentry. This makes the subsequent change, to send all lookups to ->lookup(), a bit cleaner.
---
fs/autofs4/root.c | 16 ++++++++++------ 1 files changed, 10 insertions(+), 6 deletions(-)
fs/autofs4/root.c +10 -6
@@ -405,8 +405,11 @@ static const struct dentry_operations autofs4_dentry_operations = { .d_release = autofs4_dentry_release, }; -static struct dentry *autofs4_lookup_active (struct autofs_sb_info *sbi,struct dentry *parent,struct qstr *name) +static struct dentry *autofs4_lookup_active (struct dentry *dentry) { +struct autofs_sb_info *sbi =autofs4_sbi (dentry->d_sb); +struct dentry *parent = dentry->d_parent; +struct qstr *name = &dentry->d_name;unsigned int len = name->len;unsigned int hash = name->hash;const unsigned char *str = name->name;
@@ -457,8 +460,11 @@ next:return NULL ; } -static struct dentry *autofs4_lookup_expiring (struct autofs_sb_info *sbi,struct dentry *parent,struct qstr *name) +static struct dentry *autofs4_lookup_expiring (struct dentry *dentry) { +struct autofs_sb_info *sbi =autofs4_sbi (dentry->d_sb); +struct dentry *parent = dentry->d_parent; +struct qstr *name = &dentry->d_name;unsigned int len = name->len;unsigned int hash = name->hash;const unsigned char *str = name->name;
@@ -530,7 +536,7 @@ static struct dentry *autofs4_lookup(struct inode *dir, struct dentry *dentry, sDPRINTK ("pid = %u, pgrp = %u, catatonic = %d, oz_mode = %d" , current->pid,task_pgrp_nr (current), sbi->catatonic, oz_mode); - active =autofs4_lookup_active (sbi, dentry->d_parent, &dentry->d_name); + active =autofs4_lookup_active (dentry);if (active) { dentry = active; ino =autofs4_dentry_ino (dentry);
@@ -567,9 +573,7 @@ static struct dentry *autofs4_lookup(struct inode *dir, struct dentry *dentry, sif (!oz_mode) {mutex_unlock (&dir->i_mutex); - expiring =autofs4_lookup_expiring (sbi, - dentry->d_parent, - &dentry->d_name); + expiring =autofs4_lookup_expiring (dentry);if (expiring) {/* * If we are racing with expire the request might not
RFC PATCH 8/11 autofs4 - eliminate d_unhashed in path walk checks
In reply to Ian KentWe unhash the dentry (in a subsequent patch) in ->d_revalidate() in order to send mount requests to ->lookup(). But then we can not rely on d_unhased() to give reliable results because it may be called at any time by any code path. The d_unhashed() function is used by __simple_empty() in the path walking callbacks but autofs mount point dentrys should have no directories at all so a list_empty() on d_subdirs should be (and is) sufficient.
---
fs/autofs4/root.c | 7 +++---- 1 files changed, 3 insertions(+), 4 deletions(-)
fs/autofs4/root.c +3 -4
@@ -133,7 +133,7 @@ static int autofs4_dir_open(struct inode *inode, struct file *file)* it. */ spin_lock (&dcache_lock); -if (!d_mountpoint (dentry) &&__simple_empty (dentry)) { +if (!d_mountpoint (dentry) &&list_empty (&dentry->d_subdirs)) {spin_unlock (&dcache_lock);return -ENOENT ; }
@@ -257,7 +257,7 @@ static void *autofs4_follow_link(struct dentry *dentry, struct nameidata *nd)* mount it again. */ if (ino->flags &AUTOFS_INF_PENDING || - (!d_mountpoint (dentry) &&__simple_empty (dentry))) { + (!d_mountpoint (dentry) &&list_empty (&dentry->d_subdirs))) {spin_unlock (&dcache_lock);spin_unlock (&sbi->fs_lock);
@@ -340,8 +340,7 @@ static int autofs4_revalidate(struct dentry *dentry, struct nameidata *nd)/* Check for a non-mountpoint directory with no contents */ spin_lock (&dcache_lock);if (S_ISDIR (dentry->d_inode->i_mode) && - !d_mountpoint (dentry) && -__simple_empty (dentry)) { + !d_mountpoint (dentry) &&list_empty (&dentry->d_subdirs)) {DPRINTK ("dentry=%p %.*s, emptydir" , dentry, dentry->d_name.len, dentry->d_name.name);spin_unlock (&dcache_lock);
RFC PATCH 9/11 autofs4 - rename dentry to active in autofs4_lookup_active()
In reply to Ian KentIn autofs4_lookup_active() a declaration within the list traversal loop uses a declaration that has the same name as the function parameter.
---
fs/autofs4/root.c | 24 ++++++++++++------------ 1 files changed, 12 insertions(+), 12 deletions(-)
fs/autofs4/root.c +12 -12
@@ -419,23 +419,23 @@ static struct dentry *autofs4_lookup_active(struct dentry *dentry) head = &sbi->active_list;list_for_each (p, head) {struct autofs_info *ino; -struct dentry *dentry; +struct dentry *active;struct qstr *qstr; ino = list_entry(p, struct autofs_info, active); - dentry = ino->dentry; + active = ino->dentry; -spin_lock (&dentry->d_lock); +spin_lock (&active->d_lock);/* Already gone? */ -if (atomic_read (&dentry->d_count) ==0 ) +if (atomic_read (&active->d_count) ==0 ) goto next; - qstr = &dentry->d_name; + qstr = &active->d_name; -if (dentry->d_name.hash != hash) +if (active->d_name.hash != hash) goto next; -if (dentry->d_parent != parent) +if (active->d_parent != parent) goto next;if (qstr->len != len)
@@ -443,15 +443,15 @@ static struct dentry *autofs4_lookup_active(struct dentry *dentry)if (memcmp (qstr->name, str, len)) goto next; -if (d_unhashed (dentry)) { -dget (dentry); -spin_unlock (&dentry->d_lock); +if (d_unhashed (active)) { +dget (active); +spin_unlock (&active->d_lock);spin_unlock (&sbi->lookup_lock);spin_unlock (&dcache_lock); -return dentry; +return active; } next: -spin_unlock (&dentry->d_lock); +spin_unlock (&active->d_lock); }spin_unlock (&sbi->lookup_lock);spin_unlock (&dcache_lock);
RFC PATCH 10/11 autofs4 - rename dentry to expiring in autofs4_lookup_expiring()
In reply to Ian KentIn autofs4_lookup_expiring() a declaration within the list traversal loop uses a declaration that has the same name as the function parameter.
---
fs/autofs4/root.c | 24 ++++++++++++------------ 1 files changed, 12 insertions(+), 12 deletions(-)
fs/autofs4/root.c +12 -12
@@ -474,23 +474,23 @@ static struct dentry *autofs4_lookup_expiring(struct dentry *dentry) head = &sbi->expiring_list;list_for_each (p, head) {struct autofs_info *ino; -struct dentry *dentry; +struct dentry *expiring;struct qstr *qstr; ino = list_entry(p, struct autofs_info, expiring); - dentry = ino->dentry; + expiring = ino->dentry; -spin_lock (&dentry->d_lock); +spin_lock (&expiring->d_lock);/* Bad luck, we've already been dentry_iput */ -if (!dentry->d_inode) +if (!expiring->d_inode) goto next; - qstr = &dentry->d_name; + qstr = &expiring->d_name; -if (dentry->d_name.hash != hash) +if (expiring->d_name.hash != hash) goto next; -if (dentry->d_parent != parent) +if (expiring->d_parent != parent) goto next;if (qstr->len != len)
@@ -498,15 +498,15 @@ static struct dentry *autofs4_lookup_expiring(struct dentry *dentry)if (memcmp (qstr->name, str, len)) goto next; -if (d_unhashed (dentry)) { -dget (dentry); -spin_unlock (&dentry->d_lock); +if (d_unhashed (expiring)) { +dget (expiring); +spin_unlock (&expiring->d_lock);spin_unlock (&sbi->lookup_lock);spin_unlock (&dcache_lock); -return dentry; +return expiring; } next: -spin_unlock (&dentry->d_lock); +spin_unlock (&expiring->d_lock); }spin_unlock (&sbi->lookup_lock);spin_unlock (&dcache_lock);
RFC PATCH 11/11 autofs4 - always use lookup for lookup
In reply to Ian KentWe need to be able to cope with the directory mutex being held during ->d_revalidate() in some cases, but not all cases, and not necessarily by us. Because we need to release the mutex when we call back to the daemon to do perform a mount we must be sure that it is us who holds the mutex so we must redirect mount requests to ->lookup() if the mutex is held.
---
fs/autofs4/autofs_i.h | 7 + fs/autofs4/expire.c | 6 + fs/autofs4/inode.c | 1 fs/autofs4/root.c | 474 +++++++++++++++++++++++++++++++++---------------- 4 files changed, 330 insertions(+), 158 deletions(-)
fs/autofs4/autofs_i.h +7 -0
@@ -60,6 +60,11 @@ do { \ current->pid, __func__, ##args); \ } while (0) +struct rehash_entry { +struct task_struct *task; +struct list_head list; +}; +/* Unified info structure. This is pointed to by both the dentry and inode structures. Each file in the filesystem has an instance of this structure. It holds a reference to the dentry, so dentries are never
@@ -76,6 +81,7 @@ struct autofs_info {struct list_head active;int active_count; +struct list_head rehash_list;struct list_head expiring;
@@ -98,6 +104,7 @@ struct autofs_info {#define AUTOFS_INF_EXPIRING (1<<0)/* dentry is in the process of expiring */ #define AUTOFS_INF_MOUNTPOINT (1<<1)/* mountpoint status for direct expire */ #define AUTOFS_INF_PENDING (1<<2)/* dentry pending mount */ +#define AUTOFS_INF_REHASH (1<<3)/* dentry in transit to ->lookup() */ struct autofs_wait_queue {wait_queue_head_t queue;
fs/autofs4/expire.c +5 -1
@@ -279,6 +279,7 @@ struct dentry *autofs4_expire_direct(struct super_block *sb, root->d_mounted--; } ino->flags |=AUTOFS_INF_EXPIRING ; +autofs4_add_expiring (root);init_completion (&ino->expire_complete);spin_unlock (&sbi->fs_lock);return root;
@@ -406,6 +407,7 @@ found: expired, (int)expired->d_name.len, expired->d_name.name); ino =autofs4_dentry_ino (expired); ino->flags |=AUTOFS_INF_EXPIRING ; +autofs4_add_expiring (expired);init_completion (&ino->expire_complete);spin_unlock (&sbi->fs_lock);spin_lock (&dcache_lock);
@@ -433,7 +435,7 @@ int autofs4_expire_wait(struct dentry *dentry)DPRINTK ("expire done status=%d" , status); -if (d_unhashed (dentry)) +if (d_unhashed (dentry) &&IS_DEADDIR (dentry->d_inode))return -EAGAIN ;return status;
@@ -473,6 +475,7 @@ int autofs4_expire_run(struct super_block *sb,spin_lock (&sbi->fs_lock); ino =autofs4_dentry_ino (dentry); ino->flags &= ~AUTOFS_INF_EXPIRING ; +autofs4_del_expiring (dentry);complete_all (&ino->expire_complete);spin_unlock (&sbi->fs_lock);
@@ -503,6 +506,7 @@ int autofs4_do_expire_multi(struct super_block *sb, struct vfsmount *mnt, ino->flags &= ~AUTOFS_INF_MOUNTPOINT ; } ino->flags &= ~AUTOFS_INF_EXPIRING ; +autofs4_del_expiring (dentry);complete_all (&ino->expire_complete);spin_unlock (&sbi->fs_lock);dput (dentry);
fs/autofs4/inode.c +1 -0
@@ -49,6 +49,7 @@ struct autofs_info *autofs4_init_ino(struct autofs_info *ino, ino->dentry =NULL ; ino->size =0 ;INIT_LIST_HEAD (&ino->active); +INIT_LIST_HEAD (&ino->rehash_list); ino->active_count =0 ;INIT_LIST_HEAD (&ino->expiring);atomic_set (&ino->count,0 );
fs/autofs4/root.c +317 -157
654 lines not on this page · raw message
Quoted 4 lines · Ian Kent
A change to the VFS path walk locking is needed to resolve an issue identified by Sage Weil. This locking change requires significant changes to the autofs4 module to allow it to callback to userspace without introducing a deadlock.
Oops, meant to say this series is against the current Linus tree and not
the current mm tree. However, I don't expect there will be differences
in the autofs4 patches.
Ian
Quoted 17 lines · Ian Kent
A change to the VFS path walk locking is needed to resolve an issue identified by Sage Weil. This locking change requires significant changes to the autofs4 module to allow it to callback to userspace without introducing a deadlock.
To cope with the change the autofs4 module needs to redirect mount requests from ->d_revalidate() to ->lookup() if the directory inode mutex is held when a callback needs to be done. Note that we cannot redirect these requests when the mutex is not held because, to function correctly, the mutex must be held over both revalidate and lookup.
Of the patches in the series most are cleanups and refactoring done to keep the real change in "autofs4 - always use lookup for lookup" as clean as possible. Unfortuneately, there is still quite a bit left in it.
Quoted 5 lines
Also, I need confirmation that the patch that changes the VFS path walk locking is in fact correct, or at least like for like to what will be submitted. I had some difficulty with the original patches that were paosted. The patch in question below is "vfs: make
real_lookupdo dentry revalidation withi_mutexheld".
It looks identical to be the original two folded into one patch. I'll repost those two now, freshened against Linus' tree. The first has just the functional change, and the cleanup is in the second (as per Christoph's review).
Quoted 4 lines
I've done quite a bit of fairly heavy stress testing of the patch series and they (finally) hold up to it. Although I have also managed to uncover a locking bug in the user space daemon as a result, ;)
I'm glad some other good has come of it. Thanks, Ian, for carrying this through!
sage
Quoted 30 lines
---
Ian Kent (10):
autofs4 - always use lookup for lookup
autofs4 - rename dentry to expiring inautofs4_lookup_expiring()autofs4 - rename dentry to active inautofs4_lookup_active()autofs4 - eliminated_unhashedin path walk checks autofs4 - cleanup active and expire lookup
autofs4 - renamer unhashed to active inautofs4_lookup()autofs4 - useautofs_infofor pending flag
autofs4 - use macro for need mount check
autofs4 - use macros for expiring list
autofs4 - use macros for active list handlingSage Weil (1):
Subject: [PATCH] vfs: makereal_lookupdo dentry revalidation withi_mutexheldfs/autofs4/autofs_i.h | 38 +++ fs/autofs4/expire.c | 8 - fs/autofs4/inode.c | 2 fs/autofs4/root.c | 616 ++++++++++++++++++++++++++++++++----------------- fs/namei.c | 58 ++--- 5 files changed, 480 insertions(+), 242 deletions(-)Signature · 1 line
Ian
Quoted 30 lines · Sage Weil
Quoted 17 lines · Ian Kent
A change to the VFS path walk locking is needed to resolve an issue identified by Sage Weil. This locking change requires significant changes to the autofs4 module to allow it to callback to userspace without introducing a deadlock.
To cope with the change the autofs4 module needs to redirect mount requests from ->d_revalidate() to ->lookup() if the directory inode mutex is held when a callback needs to be done. Note that we cannot redirect these requests when the mutex is not held because, to function correctly, the mutex must be held over both revalidate and lookup.
Of the patches in the series most are cleanups and refactoring done to keep the real change in "autofs4 - always use lookup for lookup" as clean as possible. Unfortuneately, there is still quite a bit left in it.
Quoted 5 lines
Also, I need confirmation that the patch that changes the VFS path walk locking is in fact correct, or at least like for like to what will be submitted. I had some difficulty with the original patches that were paosted. The patch in question below is "vfs: make
real_lookupdo dentry revalidation withi_mutexheld".It looks identical to be the original two folded into one patch. I'll repost those two now, freshened against Linus' tree. The first has just the functional change, and the cleanup is in the second (as per Christoph's review).
Quoted 7 lines
Quoted 4 lines
I've done quite a bit of fairly heavy stress testing of the patch series and they (finally) hold up to it. Although I have also managed to uncover a locking bug in the user space daemon as a result, ;)
I'm glad some other good has come of it. Thanks, Ian, for carrying this through!
Not quite so good as I haven't fixed it yet.
But that's user space and it happens occasionally under an extended heavy load so it will take a while to work out.
Ian
Quoted 25 lines · Sage Weil
Quoted 17 lines · Ian Kent
A change to the VFS path walk locking is needed to resolve an issue identified by Sage Weil. This locking change requires significant changes to the autofs4 module to allow it to callback to userspace without introducing a deadlock.
To cope with the change the autofs4 module needs to redirect mount requests from ->d_revalidate() to ->lookup() if the directory inode mutex is held when a callback needs to be done. Note that we cannot redirect these requests when the mutex is not held because, to function correctly, the mutex must be held over both revalidate and lookup.
Of the patches in the series most are cleanups and refactoring done to keep the real change in "autofs4 - always use lookup for lookup" as clean as possible. Unfortuneately, there is still quite a bit left in it.
Quoted 5 lines
Also, I need confirmation that the patch that changes the VFS path walk locking is in fact correct, or at least like for like to what will be submitted. I had some difficulty with the original patches that were paosted. The patch in question below is "vfs: make
real_lookupdo dentry revalidation withi_mutexheld".
Quoted 4 lines
It looks identical to be the original two folded into one patch. I'll repost those two now, freshened against Linus' tree. The first has just the functional change, and the cleanup is in the second (as per Christoph's review).
Yes, after checking, the patch I was using is functionally the same as the combination of the two you re-posted. So all is good for may part.
Ian