Skip to contentSkip to conversation

fs: dcache remove d_mounted

RFC PATCH In review Nick Piggin 2 participants 11 messages Oct 9, 2009

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

RFC PATCH 1/1 fs: dcache remove d_mounted

Hi,

I'd just like to ask you to look at autofs4 in the context of this change. I don't really know what needs to be considered there. If this is a generally visible dentry that any other users may mount filesystems on, then this might be difficult to get working here.

I'm not quite sure what games you're playing here with d_mounted... In the simplest case we might be able to just remove DCACHE_MOUNTED.

Anyway this would be great if we can make it work so I can replace the member with d_seq for my path walk patches and not bloat dentry. Can you take a look please if you have a chance?
--

XXX: breaks autofs4, have to work out what it is doing. We can get a mounted count if needed by counting the hashtable.

---
 fs/dcache.c            |    1 -
 fs/namespace.c         |   19 +++++++++++++++----
 include/linux/dcache.h |   42 +++++++++++++++++++++---------------------
 3 files changed, 36 insertions(+), 26 deletions(-)
fs/dcache.c +0 -1
Index: linux-2.6/fs/dcache.c
===================================================================
@@ -1192,7 +1192,6 @@ struct dentry *d_alloc(struct dentry * p
 	dentry->d_sb = NULL;
 	dentry->d_op = NULL;
 	dentry->d_fsdata = NULL;
-	dentry->d_mounted = 0;
 	INIT_HLIST_NODE(&dentry->d_hash);
 	INIT_LIST_HEAD(&dentry->d_lru);
 	INIT_LIST_HEAD(&dentry->d_subdirs);
fs/namespace.c +15 -4
Index: linux-2.6/fs/namespace.c
===================================================================
@@ -574,6 +574,15 @@ static void __touch_mnt_namespace(struct
 	}
 }
 
+static void dentry_reset_mounted(struct vfsmount *mnt, struct dentry *dentry)
+{
+	if (!__lookup_mnt(mnt, dentry, 0)) {
+		spin_lock(&dentry->d_lock);
+		dentry->d_flags &= ~DCACHE_MOUNTED;
+		spin_unlock(&dentry->d_lock);
+	}
+}
+
 static void detach_mnt(struct vfsmount *mnt, struct path *old_path)
 {
 	old_path->dentry = mnt->mnt_mountpoint;
@@ -582,7 +591,7 @@ static void detach_mnt(struct vfsmount *
 	mnt->mnt_mountpoint = mnt->mnt_root;
 	list_del_init(&mnt->mnt_hash);
 	list_del_init(&mnt->mnt_child);
-	old_path->dentry->d_mounted--;
+	dentry_reset_mounted(old_path->mnt, old_path->dentry);
 	WARN_ON(mnt->mnt_mounted != 1);
 	mnt->mnt_mounted--;
 }
@@ -591,8 +600,10 @@ void mnt_set_mountpoint(struct vfsmount
 			struct vfsmount *child_mnt)
 {
 	child_mnt->mnt_parent = mntget(mnt);
-	child_mnt->mnt_mountpoint = dget(dentry);
-	dentry->d_mounted++;
+	spin_lock(&dentry->d_lock);
+	child_mnt->mnt_mountpoint = dget_dlock(dentry);
+	dentry->d_flags |= DCACHE_MOUNTED;
+	spin_unlock(&dentry->d_lock);
 }
 
 static void attach_mnt(struct vfsmount *mnt, struct path *path)
@@ -1170,7 +1181,7 @@ void umount_tree(struct vfsmount *mnt, i
 		p->mnt_mounted--;
 		if (p->mnt_parent != p) {
 			p->mnt_parent->mnt_ghosts++;
-			p->mnt_mountpoint->d_mounted--;
+			dentry_reset_mounted(p->mnt_parent, p->mnt_mountpoint);
 		}
 		change_mnt_propagation(p, MS_PRIVATE);
 	}
include/linux/dcache.h +21 -21

72 lines not on this page · raw message

maintainer
#1 lore raw
Quoted 9 lines · Nick Piggin

Hi,

I'd just like to ask you to look at autofs4 in the context of this change. I don't really know what needs to be considered there. If this is a generally visible dentry that any other users may mount filesystems on, then this might be difficult to get working here.

I'm not quite sure what games you're playing here with d_mounted... In the simplest case we might be able to just remove DCACHE_MOUNTED.

Hahaha, "games", harsh but true.

My need is fairly simple really.

I must be able to stop the follow down at the mount point for some cases of a covered dentry. Which, IIRC, means that d_mountpoint() needs to be sensitive to this requirement, and that's about all.

This was always questionable, but seemed like the best way to do it at the time, without adding autofs specific code to the VFS. Since we are changing this part of the VFS now with this patch, it is a good time to fix it in a generic non-autofs specific way.

Quoted 4 lines

Anyway this would be great if we can make it work so I can replace the member with d_seq for my path walk patches and not bloat dentry. Can you take a look please if you have a chance?

Sure, let me have a look around and think about it for a while.

From a quick look it appears that all I could just change the DCACHE_MOUNTED flag and check the actual mounted status when restoring it.

Quoted 152 lines

--

XXX: breaks autofs4, have to work out what it is doing. We can get a mounted count if needed by counting the hashtable.

---
 fs/dcache.c            |    1 -
 fs/namespace.c         |   19 +++++++++++++++----
 include/linux/dcache.h |   42 +++++++++++++++++++++---------------------
 3 files changed, 36 insertions(+), 26 deletions(-)

Index: linux-2.6/fs/dcache.c
===================================================================

fs/dcache.c +0 -1
@@ -1192,7 +1192,6 @@ struct dentry *d_alloc(struct dentry * p
 	dentry->d_sb = NULL;
 	dentry->d_op = NULL;
 	dentry->d_fsdata = NULL;
-	dentry->d_mounted = 0;
 	INIT_HLIST_NODE(&dentry->d_hash);
 	INIT_LIST_HEAD(&dentry->d_lru);
 	INIT_LIST_HEAD(&dentry->d_subdirs);

Index: linux-2.6/fs/namespace.c
===================================================================

fs/namespace.c +15 -4
@@ -574,6 +574,15 @@ static void __touch_mnt_namespace(struct
 	}
 }
 
+static void dentry_reset_mounted(struct vfsmount *mnt, struct dentry *dentry)
+{
+	if (!__lookup_mnt(mnt, dentry, 0)) {
+		spin_lock(&dentry->d_lock);
+		dentry->d_flags &= ~DCACHE_MOUNTED;
+		spin_unlock(&dentry->d_lock);
+	}
+}
+
 static void detach_mnt(struct vfsmount *mnt, struct path *old_path)
 {
 	old_path->dentry = mnt->mnt_mountpoint;
@@ -582,7 +591,7 @@ static void detach_mnt(struct vfsmount *
 	mnt->mnt_mountpoint = mnt->mnt_root;
 	list_del_init(&mnt->mnt_hash);
 	list_del_init(&mnt->mnt_child);
-	old_path->dentry->d_mounted--;
+	dentry_reset_mounted(old_path->mnt, old_path->dentry);
 	WARN_ON(mnt->mnt_mounted != 1);
 	mnt->mnt_mounted--;
 }
@@ -591,8 +600,10 @@ void mnt_set_mountpoint(struct vfsmount
 			struct vfsmount *child_mnt)
 {
 	child_mnt->mnt_parent = mntget(mnt);
-	child_mnt->mnt_mountpoint = dget(dentry);
-	dentry->d_mounted++;
+	spin_lock(&dentry->d_lock);
+	child_mnt->mnt_mountpoint = dget_dlock(dentry);
+	dentry->d_flags |= DCACHE_MOUNTED;
+	spin_unlock(&dentry->d_lock);
 }
 
 static void attach_mnt(struct vfsmount *mnt, struct path *path)
@@ -1170,7 +1181,7 @@ void umount_tree(struct vfsmount *mnt, i
 		p->mnt_mounted--;
 		if (p->mnt_parent != p) {
 			p->mnt_parent->mnt_ghosts++;
-			p->mnt_mountpoint->d_mounted--;
+			dentry_reset_mounted(p->mnt_parent, p->mnt_mountpoint);
 		}
 		change_mnt_propagation(p, MS_PRIVATE);
 	}

Index: linux-2.6/include/linux/dcache.h
===================================================================

include/linux/dcache.h +21 -21
@@ -83,14 +83,13 @@ full_name_hash(const unsigned char *name
 #ifdef CONFIG_64BIT
 #define DNAME_INLINE_LEN_MIN 32 /* 192 bytes */
 #else
-#define DNAME_INLINE_LEN_MIN 40 /* 128 bytes */
+#define DNAME_INLINE_LEN_MIN 44 /* 128 bytes */
 #endif
 
 struct dentry {
 	unsigned int d_count;		/* protected by d_lock */
 	unsigned int d_flags;		/* protected by d_lock */
 	spinlock_t d_lock;		/* per dentry lock */
-	int d_mounted;
 	struct inode *d_inode;		/* Where the name belongs to - NULL is
 					 * negative */
 	/*
@@ -161,30 +160,31 @@ d_iput:		no		no       yes
 
 /* d_flags entries */
 #define DCACHE_AUTOFS_PENDING 0x0001    /* autofs: "under construction" */
-#define DCACHE_NFSFS_RENAMED  0x0002    /* this dentry has been "silly
-					 * renamed" and has to be
-					 * deleted on the last dput()
-					 */
-#define	DCACHE_DISCONNECTED 0x0004
-     /* This dentry is possibly not currently connected to the dcache tree,
-      * in which case its parent will either be itself, or will have this
-      * flag as well.  nfsd will not use a dentry with this bit set, but will
-      * first endeavour to clear the bit either by discovering that it is
-      * connected, or by performing lookup operations.   Any filesystem which
-      * supports nfsd_operations MUST have a lookup function which, if it finds
-      * a directory inode with a DCACHE_DISCONNECTED dentry, will d_move
-      * that dentry into place and return that dentry rather than the passed one,
-      * typically using d_splice_alias.
-      */
+#define DCACHE_NFSFS_RENAMED  0x0002
+     /* this dentry has been "silly renamed" and has to be deleted on the last
+      * dput() */
+
+#define	DCACHE_DISCONNECTED	0x0004
+     /* This dentry is possibly not currently connected to the dcache tree, in
+      * which case its parent will either be itself, or will have this flag as
+      * well.  nfsd will not use a dentry with this bit set, but will first
+      * endeavour to clear the bit either by discovering that it is connected,
+      * or by performing lookup operations.   Any filesystem which supports
+      * nfsd_operations MUST have a lookup function which, if it finds a
+      * directory inode with a DCACHE_DISCONNECTED dentry, will d_move that
+      * dentry into place and return that dentry rather than the passed one,
+      * typically using d_splice_alias. */
 
 #define DCACHE_REFERENCED	0x0008  /* Recently used, don't discard. */
 #define DCACHE_UNHASHED		0x0010	
-
-#define DCACHE_INOTIFY_PARENT_WATCHED	0x0020 /* Parent inode is watched by inotify */
+#define DCACHE_INOTIFY_PARENT_WATCHED 0x0020
+     /* Parent inode is watched by inotify */
 
 #define DCACHE_COOKIE		0x0040	/* For use by dcookie subsystem */
+#define DCACHE_FSNOTIFY_PARENT_WATCHED 0x0080
+     /* Parent inode is watched by some fsnotify listener */
 
-#define DCACHE_FSNOTIFY_PARENT_WATCHED	0x0080 /* Parent inode is watched by some fsnotify listener */
+#define DCACHE_MOUNTED		0x0100	/* is a mountpoint */
 
 extern seqlock_t rename_lock;
 
@@ -349,7 +349,7 @@ extern void dput(struct dentry *);
 
 static inline int d_mountpoint(struct dentry *dentry)
 {
-	return dentry->d_mounted;
+	return dentry->d_flags & DCACHE_MOUNTED;
 }
 
 extern struct vfsmount *lookup_mnt(struct path *);
#2 lore raw
Quoted 15 lines · Ian Kent
Quoted 9 lines · Nick Piggin

Hi,

I'd just like to ask you to look at autofs4 in the context of this change. I don't really know what needs to be considered there. If this is a generally visible dentry that any other users may mount filesystems on, then this might be difficult to get working here.

I'm not quite sure what games you're playing here with d_mounted... In the simplest case we might be able to just remove DCACHE_MOUNTED.

Hahaha, "games", harsh but true.

My need is fairly simple really.

Quoted 8 lines

I must be able to stop the follow down at the mount point for some cases of a covered dentry. Which, IIRC, means that d_mountpoint() needs to be sensitive to this requirement, and that's about all.

This was always questionable, but seemed like the best way to do it at the time, without adding autofs specific code to the VFS. Since we are changing this part of the VFS now with this patch, it is a good time to fix it in a generic non-autofs specific way.

I guess you could have a flag in the vfsmount which you could then set to have lookup_mnt (and hence follow_mount etc) ignore it.

Unsetting / decrementing d_mounted I guess works, but I would just be worried if other mounts can be attached to the dentry then you might ignore that other mount or even follow your autofs mount.i

If there is no way to have anything else mounted here, then there shouldn't be a problem and indeed unsetting d_mounted might be the easiest approach. However you still have to be careful of a racing lookup that has found d_mounted to be true, but is yet to look up the mount hash table -- that might be tricky and is a case where the vfsmount flag approach should work better.

Quoted 8 lines
Quoted 3 lines

Anyway this would be great if we can make it work so I can replace the member with d_seq for my path walk patches and not bloat dentry. Can you take a look please if you have a chance?

Sure, let me have a look around and think about it for a while.

Quoted 1 line

From a quick look it appears that all I could just change the

DCACHE_MOUNTED flag and check the actual mounted status when restoring it.

OK, thanks. I'll do that as an intermediate hack here, and if you find a problem with it or if we devise a better generic approach, then I'll rip it out.

Thanks,
Nick

maintainer
#3 lore raw
Quoted 31 lines · Nick Piggin
Quoted 14 lines · Ian Kent
Quoted 9 lines · Nick Piggin

Hi,

I'd just like to ask you to look at autofs4 in the context of this change. I don't really know what needs to be considered there. If this is a generally visible dentry that any other users may mount filesystems on, then this might be difficult to get working here.

I'm not quite sure what games you're playing here with d_mounted... In the simplest case we might be able to just remove DCACHE_MOUNTED.

Hahaha, "games", harsh but true.

My need is fairly simple really.

Quoted 8 lines

I must be able to stop the follow down at the mount point for some cases of a covered dentry. Which, IIRC, means that d_mountpoint() needs to be sensitive to this requirement, and that's about all.

This was always questionable, but seemed like the best way to do it at the time, without adding autofs specific code to the VFS. Since we are changing this part of the VFS now with this patch, it is a good time to fix it in a generic non-autofs specific way.

I guess you could have a flag in the vfsmount which you could then set to have lookup_mnt (and hence follow_mount etc) ignore it.

Unsetting / decrementing d_mounted I guess works, but I would just be worried if other mounts can be attached to the dentry then you might ignore that other mount or even follow your autofs mount.i

Quoted 6 lines

If there is no way to have anything else mounted here, then there shouldn't be a problem and indeed unsetting d_mounted might be the easiest approach. However you still have to be careful of a racing lookup that has found d_mounted to be true, but is yet to look up the mount hash table -- that might be tricky and is a case where the vfsmount flag approach should work better.

My original description was a bit simple mined.

This is only ever done for dentrys in the autofs fs.

Although I won't go into the ongoing and difficult problem of submounts, when this is done for the common case all user space walks are blocked waiting on the expire while the daemon does the umount. The reason it needs to be done at all is because autofs mount types AUTOFS_TYPE_DIRECT and AUTOFS_TYPE_OFFSET are such that the autofs fs is mounted on the host dentry (that may also be another autofs fs) and another mount (that is being expired in this case) is mounted on top of that. Hence path walks skips right over the top of the dentry and into the expiring mount.

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

Anyway this would be great if we can make it work so I can replace the member with d_seq for my path walk patches and not bloat dentry. Can you take a look please if you have a chance?

Quoted 8 lines
Quoted 4 lines

Sure, let me have a look around and think about it for a while.

Quoted 1 line

From a quick look it appears that all I could just change the

DCACHE_MOUNTED flag and check the actual mounted status when restoring it.

OK, thanks. I'll do that as an intermediate hack here, and if you find a problem with it or if we devise a better generic approach, then I'll rip it out.

Ummm .. I don't seem to be able to cleanly apply this patch to a linus tree or an mm tree?

Is there a git repo I can use to work on this?

Quoted 4 lines

Thanks,
Nick

#4 lore raw
Quoted 43 lines · Ian Kent
Quoted 31 lines · Nick Piggin
Quoted 14 lines · Ian Kent
Quoted 9 lines · Nick Piggin

Hi,

I'd just like to ask you to look at autofs4 in the context of this change. I don't really know what needs to be considered there. If this is a generally visible dentry that any other users may mount filesystems on, then this might be difficult to get working here.

I'm not quite sure what games you're playing here with d_mounted... In the simplest case we might be able to just remove DCACHE_MOUNTED.

Hahaha, "games", harsh but true.

My need is fairly simple really.

Quoted 8 lines

I must be able to stop the follow down at the mount point for some cases of a covered dentry. Which, IIRC, means that d_mountpoint() needs to be sensitive to this requirement, and that's about all.

This was always questionable, but seemed like the best way to do it at the time, without adding autofs specific code to the VFS. Since we are changing this part of the VFS now with this patch, it is a good time to fix it in a generic non-autofs specific way.

I guess you could have a flag in the vfsmount which you could then set to have lookup_mnt (and hence follow_mount etc) ignore it.

Unsetting / decrementing d_mounted I guess works, but I would just be worried if other mounts can be attached to the dentry then you might ignore that other mount or even follow your autofs mount.i

Quoted 6 lines

If there is no way to have anything else mounted here, then there shouldn't be a problem and indeed unsetting d_mounted might be the easiest approach. However you still have to be careful of a racing lookup that has found d_mounted to be true, but is yet to look up the mount hash table -- that might be tricky and is a case where the vfsmount flag approach should work better.

My original description was a bit simple mined.

This is only ever done for dentrys in the autofs fs.

Quoted 8 lines

Although I won't go into the ongoing and difficult problem of submounts, when this is done for the common case all user space walks are blocked waiting on the expire while the daemon does the umount. The reason it needs to be done at all is because autofs mount types AUTOFS_TYPE_DIRECT and AUTOFS_TYPE_OFFSET are such that the autofs fs is mounted on the host dentry (that may also be another autofs fs) and another mount (that is being expired in this case) is mounted on top of that. Hence path walks skips right over the top of the dentry and into the expiring mount.

OK, that makes some sense to me ;) If nothing else can meddle with the dentry (like attaching a mount to it) then I see no problem with just clearing d_mounted.

Quoted 8 lines
Quoted 8 lines
Quoted 7 lines
Quoted 3 lines

Anyway this would be great if we can make it work so I can replace the member with d_seq for my path walk patches and not bloat dentry. Can you take a look please if you have a chance?

Sure, let me have a look around and think about it for a while.

Quoted 1 line

From a quick look it appears that all I could just change the

DCACHE_MOUNTED flag and check the actual mounted status when restoring it.

Quoted 8 lines
Quoted 3 lines

OK, thanks. I'll do that as an intermediate hack here, and if you find a problem with it or if we devise a better generic approach, then I'll rip it out.

Ummm .. I don't seem to be able to cleanly apply this patch to a linus tree or an mm tree?

Is there a git repo I can use to work on this?

Yeah I will redo it against mainline and send it out again. I will get around to doing a git tree of the vfs scalability stuff after I get the series in a bit more reasonable shape... I'll need you to look at that too because autofs4 does a lot of meddling with dcache_lock ;)

maintainer
#5 lore raw
Quoted 69 lines · Nick Piggin
Quoted 41 lines · Ian Kent
Quoted 30 lines · Nick Piggin
Quoted 14 lines · Ian Kent
Quoted 9 lines · Nick Piggin

Hi,

I'd just like to ask you to look at autofs4 in the context of this change. I don't really know what needs to be considered there. If this is a generally visible dentry that any other users may mount filesystems on, then this might be difficult to get working here.

I'm not quite sure what games you're playing here with d_mounted... In the simplest case we might be able to just remove DCACHE_MOUNTED.

Hahaha, "games", harsh but true.

My need is fairly simple really.

Quoted 8 lines

I must be able to stop the follow down at the mount point for some cases of a covered dentry. Which, IIRC, means that d_mountpoint() needs to be sensitive to this requirement, and that's about all.

This was always questionable, but seemed like the best way to do it at the time, without adding autofs specific code to the VFS. Since we are changing this part of the VFS now with this patch, it is a good time to fix it in a generic non-autofs specific way.

I guess you could have a flag in the vfsmount which you could then set to have lookup_mnt (and hence follow_mount etc) ignore it.

Unsetting / decrementing d_mounted I guess works, but I would just be worried if other mounts can be attached to the dentry then you might ignore that other mount or even follow your autofs mount.i

Quoted 6 lines

If there is no way to have anything else mounted here, then there shouldn't be a problem and indeed unsetting d_mounted might be the easiest approach. However you still have to be careful of a racing lookup that has found d_mounted to be true, but is yet to look up the mount hash table -- that might be tricky and is a case where the vfsmount flag approach should work better.

My original description was a bit simple mined.

This is only ever done for dentrys in the autofs fs.

Quoted 8 lines

Although I won't go into the ongoing and difficult problem of submounts, when this is done for the common case all user space walks are blocked waiting on the expire while the daemon does the umount. The reason it needs to be done at all is because autofs mount types AUTOFS_TYPE_DIRECT and AUTOFS_TYPE_OFFSET are such that the autofs fs is mounted on the host dentry (that may also be another autofs fs) and another mount (that is being expired in this case) is mounted on top of that. Hence path walks skips right over the top of the dentry and into the expiring mount.

OK, that makes some sense to me ;) If nothing else can meddle with the dentry (like attaching a mount to it) then I see no problem with just clearing d_mounted.

Quoted 13 lines
Quoted 10 lines
Quoted 7 lines
Quoted 3 lines

Anyway this would be great if we can make it work so I can replace the member with d_seq for my path walk patches and not bloat dentry. Can you take a look please if you have a chance?

Sure, let me have a look around and think about it for a while.

Quoted 1 line

From a quick look it appears that all I could just change the

DCACHE_MOUNTED flag and check the actual mounted status when restoring it.

OK, thanks. I'll do that as an intermediate hack here, and if you find a problem with it or if we devise a better generic approach, then I'll rip it out.

Ummm .. I don't seem to be able to cleanly apply this patch to a linus tree or an mm tree?

Quoted 7 lines
Quoted 1 line

Is there a git repo I can use to work on this?

Yeah I will redo it against mainline and send it out again. I will get around to doing a git tree of the vfs scalability stuff after I get the series in a bit more reasonable shape... I'll need you to look at that too because autofs4 does a lot of meddling with dcache_lock ;)

Hehe, sorry.

np, just point me at it and I'll do what I can.

I'm open to advice on where and why I don't need to use the dcache_lock. I think there are case where I unnecessarily take it, especially in the changes that I have just posted to support the Sage Weil change to the taking of the directory i_mutex over revalidate in real_lookup(). Which are present in the mm kernel atm, if the latest one has been posted yet that is.

Ian

#6 lore raw
Quoted 10 lines · Ian Kent
Quoted 5 lines · Nick Piggin

Yeah I will redo it against mainline and send it out again. I will get around to doing a git tree of the vfs scalability stuff after I get the series in a bit more reasonable shape... I'll need you to look at that too because autofs4 does a lot of meddling with dcache_lock ;)

Hehe, sorry.

np, just point me at it and I'll do what I can.

I will do so, I just want to get a chance to clean it up and document it all a bit better before I ask filesystem maintainres to take a really good look.

Quoted 6 lines

I'm open to advice on where and why I don't need to use the dcache_lock. I think there are case where I unnecessarily take it, especially in the changes that I have just posted to support the Sage Weil change to the taking of the directory i_mutex over revalidate in real_lookup(). Which are present in the mm kernel atm, if the latest one has been posted yet that is.

Well, after my patches there is no dcache_lock, so we need to be sure that we _don't_ need dcache_lock :)

Basically what I have done for autofs4 is that I've tried to preserve the same guarantees (from the dcache) that it looks like you need, when replacing each instance of dcache_lock.

I have then replaced each dcache_lock with a new global autofs4_lock, because there is so much state in there and so many places where you take dcache_lock that I can't tell what autofs internals are being protected with it.

I.... have it "working", but the reality is that there will be bugs especially in my initial conversions of something hard like autofs4. So just informed reviewing and questions/comments are going to be important.

Thanks,
Nick

maintainer
#7 lore raw
Quoted 23 lines · Nick Piggin
Quoted 9 lines · Ian Kent
Quoted 5 lines · Nick Piggin

Yeah I will redo it against mainline and send it out again. I will get around to doing a git tree of the vfs scalability stuff after I get the series in a bit more reasonable shape... I'll need you to look at that too because autofs4 does a lot of meddling with dcache_lock ;)

Hehe, sorry.

np, just point me at it and I'll do what I can.

I will do so, I just want to get a chance to clean it up and document it all a bit better before I ask filesystem maintainres to take a really good look.

Quoted 6 lines

I'm open to advice on where and why I don't need to use the dcache_lock. I think there are case where I unnecessarily take it, especially in the changes that I have just posted to support the Sage Weil change to the taking of the directory i_mutex over revalidate in real_lookup(). Which are present in the mm kernel atm, if the latest one has been posted yet that is.

Quoted 2 lines

Well, after my patches there is no dcache_lock, so we need to be sure that we _don't_ need dcache_lock :)

Oh wow, that's a fairly big change, it will certainly need a bit of though on my part. How does this design protect directory list changes during traversals?

fyi, there are plans to retire autofs and rename autofs4 to autofs, but that is potentially very disruptive so I'm not sure yet when I will propose it. In any case, once the real_lookup() locking change goes in autofs will unavoidably deadlock, so it's got to go.

Quoted 4 lines

Basically what I have done for autofs4 is that I've tried to preserve the same guarantees (from the dcache) that it looks like you need, when replacing each instance of dcache_lock.

Right, I do sometimes do things that are probably not OK in general file systems because of the fact that the daemon, a single process, is the only one that can do certain things, like create or remove a directory, for example. OTOH, taking that further, I probably often don't really need to take the dcache_lock as often as I do.

Quoted 5 lines

I have then replaced each dcache_lock with a new global autofs4_lock, because there is so much state in there and so many places where you take dcache_lock that I can't tell what autofs internals are being protected with it.

This may offer an opportunity to simplify that locking a lot, if I analyse why I'm taking it, and relate that back to the way the locking is now meant to work. So the documentation you mentioned will be essential.

Quoted 5 lines

I.... have it "working", but the reality is that there will be bugs especially in my initial conversions of something hard like autofs4. So just informed reviewing and questions/comments are going to be important.

Bugs .. surely not !!!

Ian

#8 lore raw
Quoted 10 lines · Ian Kent
Quoted 9 lines · Nick Piggin
Quoted 7 lines · Ian Kent

Nick Piggin wrote:
I'm open to advice on where and why I don't need to use the dcache_lock. I think there are case where I unnecessarily take it, especially in the changes that I have just posted to support the Sage Weil change to the taking of the directory i_mutex over revalidate in real_lookup(). Which are present in the mm kernel atm, if the latest one has been posted yet that is.

Quoted 6 lines
Quoted 2 lines

Well, after my patches there is no dcache_lock, so we need to be sure that we _don't_ need dcache_lock :)

Oh wow, that's a fairly big change, it will certainly need a bit of though on my part. How does this design protect directory list changes during traversals?

It depends on the traversal exactly, but yes basically we lose the ability to (easily) freeze the entire tree. parent and its children can be protected with the parent's d_lock.

Basically all properties of a given dentry including its immediate parent and children are protected with d_lock in fact.

You can get more creative as needed after that. Retry on rename, lock multiple.

Quoted 4 lines

fyi, there are plans to retire autofs and rename autofs4 to autofs, but that is potentially very disruptive so I'm not sure yet when I will propose it. In any case, once the real_lookup() locking change goes in autofs will unavoidably deadlock, so it's got to go.

No compaints here :) I only looked at autofs4 so far.

Quoted 11 lines
Quoted 3 lines

Basically what I have done for autofs4 is that I've tried to preserve the same guarantees (from the dcache) that it looks like you need, when replacing each instance of dcache_lock.

Right, I do sometimes do things that are probably not OK in general file systems because of the fact that the daemon, a single process, is the only one that can do certain things, like create or remove a directory, for example. OTOH, taking that further, I probably often don't really need to take the dcache_lock as often as I do.

Quoted 1 line
Quoted 8 lines
Quoted 4 lines

I have then replaced each dcache_lock with a new global autofs4_lock, because there is so much state in there and so many places where you take dcache_lock that I can't tell what autofs internals are being protected with it.

This may offer an opportunity to simplify that locking a lot, if I analyse why I'm taking it, and relate that back to the way the locking is now meant to work. So the documentation you mentioned will be essential.

Cool, well I look forward to your reviewing the changes when I get a bit further down the track. In the meantime, here is the d_mounted patch diffed against rc3.

--

Rather than keep a d_mounted count in the dentry (which is only used to speculatively take a look in the mount hash table if it is non-zero), set a dentry flag instead. The flag can be cleared by checking the hash table to see if there are any mounts left. It is not time critical because it is performed at detach time.

This saves 4 bytes on 32-bit, nothing on 64-bit but it does provide a hole which I might use later.

---
 fs/autofs4/expire.c    |    8 ++++++--
 fs/dcache.c            |    1 -
 fs/namespace.c         |   19 +++++++++++++++----
 include/linux/dcache.h |   42 +++++++++++++++++++++---------------------
 4 files changed, 42 insertions(+), 28 deletions(-)
fs/dcache.c +0 -1
Index: linux-2.6/fs/dcache.c
===================================================================
@@ -947,7 +947,6 @@ struct dentry *d_alloc(struct dentry * p
 	dentry->d_sb = NULL;
 	dentry->d_op = NULL;
 	dentry->d_fsdata = NULL;
-	dentry->d_mounted = 0;
 	INIT_HLIST_NODE(&dentry->d_hash);
 	INIT_LIST_HEAD(&dentry->d_lru);
 	INIT_LIST_HEAD(&dentry->d_subdirs);
fs/namespace.c +15 -4
Index: linux-2.6/fs/namespace.c
===================================================================
@@ -467,6 +467,15 @@ static void __touch_mnt_namespace(struct
 	}
 }
 
+static void dentry_reset_mounted(struct vfsmount *mnt, struct dentry *dentry)
+{
+	if (!__lookup_mnt(mnt, dentry, 0)) {
+		spin_lock(&dentry->d_lock);
+		dentry->d_flags &= ~DCACHE_MOUNTED;
+		spin_unlock(&dentry->d_lock);
+	}
+}
+
 static void detach_mnt(struct vfsmount *mnt, struct path *old_path)
 {
 	old_path->dentry = mnt->mnt_mountpoint;
@@ -475,15 +484,17 @@ static void detach_mnt(struct vfsmount *
 	mnt->mnt_mountpoint = mnt->mnt_root;
 	list_del_init(&mnt->mnt_child);
 	list_del_init(&mnt->mnt_hash);
-	old_path->dentry->d_mounted--;
+	dentry_reset_mounted(old_path->mnt, old_path->dentry);
 }
 
 void mnt_set_mountpoint(struct vfsmount *mnt, struct dentry *dentry,
 			struct vfsmount *child_mnt)
 {
 	child_mnt->mnt_parent = mntget(mnt);
-	child_mnt->mnt_mountpoint = dget(dentry);
-	dentry->d_mounted++;
+	spin_lock(&dentry->d_lock);
+	child_mnt->mnt_mountpoint = dget_dlock(dentry);
+	dentry->d_flags |= DCACHE_MOUNTED;
+	spin_unlock(&dentry->d_lock);
 }
 
 static void attach_mnt(struct vfsmount *mnt, struct path *path)
@@ -1015,7 +1026,7 @@ void umount_tree(struct vfsmount *mnt, i
 		list_del_init(&p->mnt_child);
 		if (p->mnt_parent != p) {
 			p->mnt_parent->mnt_ghosts++;
-			p->mnt_mountpoint->d_mounted--;
+			dentry_reset_mounted(p->mnt_parent, p->mnt_mountpoint);
 		}
 		change_mnt_propagation(p, MS_PRIVATE);
 	}
include/linux/dcache.h +21 -21

72 lines not on this page · raw message

fs/autofs4/expire.c +6 -2
Index: linux-2.6/fs/autofs4/expire.c
===================================================================
@@ -276,7 +276,9 @@ struct dentry *autofs4_expire_direct(str
 		struct autofs_info *ino = autofs4_dentry_ino(root);
 		if (d_mountpoint(root)) {
 			ino->flags |= AUTOFS_INF_MOUNTPOINT;
-			root->d_mounted--;
+			spin_lock(&root->d_lock);
+			root->d_flags &= ~DCACHE_MOUNTED;
+			spin_unlock(&root->d_lock);
 		}
 		ino->flags |= AUTOFS_INF_EXPIRING;
 		init_completion(&ino->expire_complete);
@@ -499,7 +501,9 @@ int autofs4_do_expire_multi(struct super
 
 		spin_lock(&sbi->fs_lock);
 		if (ino->flags & AUTOFS_INF_MOUNTPOINT) {
-			sb->s_root->d_mounted++;
+			spin_lock(&sb->s_root->d_lock);
+			sb->s_root->d_flags |= DCACHE_MOUNTED;
+			spin_unlock(&sb->s_root->d_lock);
 			ino->flags &= ~AUTOFS_INF_MOUNTPOINT;
 		}
 		ino->flags &= ~AUTOFS_INF_EXPIRING;
maintainer
#9 lore raw
Quoted 4 lines · Nick Piggin

Cool, well I look forward to your reviewing the changes when I get a bit further down the track. In the meantime, here is the d_mounted patch diffed against rc3.

The autofs4 bit of the patch below isn't quite how this is supposed to work. I'll have a look and see if I can make the appropriate change.

Quoted 185 lines

--

Rather than keep a d_mounted count in the dentry (which is only used to speculatively take a look in the mount hash table if it is non-zero), set a dentry flag instead. The flag can be cleared by checking the hash table to see if there are any mounts left. It is not time critical because it is performed at detach time.

This saves 4 bytes on 32-bit, nothing on 64-bit but it does provide a hole which I might use later.

---
 fs/autofs4/expire.c    |    8 ++++++--
 fs/dcache.c            |    1 -
 fs/namespace.c         |   19 +++++++++++++++----
 include/linux/dcache.h |   42 +++++++++++++++++++++---------------------
 4 files changed, 42 insertions(+), 28 deletions(-)

Index: linux-2.6/fs/dcache.c
===================================================================

fs/dcache.c +0 -1
@@ -947,7 +947,6 @@ struct dentry *d_alloc(struct dentry * p
 	dentry->d_sb = NULL;
 	dentry->d_op = NULL;
 	dentry->d_fsdata = NULL;
-	dentry->d_mounted = 0;
 	INIT_HLIST_NODE(&dentry->d_hash);
 	INIT_LIST_HEAD(&dentry->d_lru);
 	INIT_LIST_HEAD(&dentry->d_subdirs);

Index: linux-2.6/fs/namespace.c
===================================================================

fs/namespace.c +15 -4
@@ -467,6 +467,15 @@ static void __touch_mnt_namespace(struct
 	}
 }
 
+static void dentry_reset_mounted(struct vfsmount *mnt, struct dentry *dentry)
+{
+	if (!__lookup_mnt(mnt, dentry, 0)) {
+		spin_lock(&dentry->d_lock);
+		dentry->d_flags &= ~DCACHE_MOUNTED;
+		spin_unlock(&dentry->d_lock);
+	}
+}
+
 static void detach_mnt(struct vfsmount *mnt, struct path *old_path)
 {
 	old_path->dentry = mnt->mnt_mountpoint;
@@ -475,15 +484,17 @@ static void detach_mnt(struct vfsmount *
 	mnt->mnt_mountpoint = mnt->mnt_root;
 	list_del_init(&mnt->mnt_child);
 	list_del_init(&mnt->mnt_hash);
-	old_path->dentry->d_mounted--;
+	dentry_reset_mounted(old_path->mnt, old_path->dentry);
 }
 
 void mnt_set_mountpoint(struct vfsmount *mnt, struct dentry *dentry,
 			struct vfsmount *child_mnt)
 {
 	child_mnt->mnt_parent = mntget(mnt);
-	child_mnt->mnt_mountpoint = dget(dentry);
-	dentry->d_mounted++;
+	spin_lock(&dentry->d_lock);
+	child_mnt->mnt_mountpoint = dget_dlock(dentry);
+	dentry->d_flags |= DCACHE_MOUNTED;
+	spin_unlock(&dentry->d_lock);
 }
 
 static void attach_mnt(struct vfsmount *mnt, struct path *path)
@@ -1015,7 +1026,7 @@ void umount_tree(struct vfsmount *mnt, i
 		list_del_init(&p->mnt_child);
 		if (p->mnt_parent != p) {
 			p->mnt_parent->mnt_ghosts++;
-			p->mnt_mountpoint->d_mounted--;
+			dentry_reset_mounted(p->mnt_parent, p->mnt_mountpoint);
 		}
 		change_mnt_propagation(p, MS_PRIVATE);
 	}

Index: linux-2.6/include/linux/dcache.h
===================================================================

include/linux/dcache.h +21 -21
@@ -83,14 +83,13 @@ full_name_hash(const unsigned char *name
 #ifdef CONFIG_64BIT
 #define DNAME_INLINE_LEN_MIN 32 /* 192 bytes */
 #else
-#define DNAME_INLINE_LEN_MIN 40 /* 128 bytes */
+#define DNAME_INLINE_LEN_MIN 44 /* 128 bytes */
 #endif
 
 struct dentry {
 	atomic_t d_count;
 	unsigned int d_flags;		/* protected by d_lock */
 	spinlock_t d_lock;		/* per dentry lock */
-	int d_mounted;
 	struct inode *d_inode;		/* Where the name belongs to - NULL is
 					 * negative */
 	/*
@@ -161,30 +160,31 @@ d_iput:		no		no		no       yes
 
 /* d_flags entries */
 #define DCACHE_AUTOFS_PENDING 0x0001    /* autofs: "under construction" */
-#define DCACHE_NFSFS_RENAMED  0x0002    /* this dentry has been "silly
-					 * renamed" and has to be
-					 * deleted on the last dput()
-					 */
-#define	DCACHE_DISCONNECTED 0x0004
-     /* This dentry is possibly not currently connected to the dcache tree,
-      * in which case its parent will either be itself, or will have this
-      * flag as well.  nfsd will not use a dentry with this bit set, but will
-      * first endeavour to clear the bit either by discovering that it is
-      * connected, or by performing lookup operations.   Any filesystem which
-      * supports nfsd_operations MUST have a lookup function which, if it finds
-      * a directory inode with a DCACHE_DISCONNECTED dentry, will d_move
-      * that dentry into place and return that dentry rather than the passed one,
-      * typically using d_splice_alias.
-      */
+#define DCACHE_NFSFS_RENAMED  0x0002
+     /* this dentry has been "silly renamed" and has to be deleted on the last
+      * dput() */
+
+#define	DCACHE_DISCONNECTED	0x0004
+     /* This dentry is possibly not currently connected to the dcache tree, in
+      * which case its parent will either be itself, or will have this flag as
+      * well.  nfsd will not use a dentry with this bit set, but will first
+      * endeavour to clear the bit either by discovering that it is connected,
+      * or by performing lookup operations.   Any filesystem which supports
+      * nfsd_operations MUST have a lookup function which, if it finds a
+      * directory inode with a DCACHE_DISCONNECTED dentry, will d_move that
+      * dentry into place and return that dentry rather than the passed one,
+      * typically using d_splice_alias. */
 
 #define DCACHE_REFERENCED	0x0008  /* Recently used, don't discard. */
 #define DCACHE_UNHASHED		0x0010	
-
-#define DCACHE_INOTIFY_PARENT_WATCHED	0x0020 /* Parent inode is watched by inotify */
+#define DCACHE_INOTIFY_PARENT_WATCHED 0x0020
+     /* Parent inode is watched by inotify */
 
 #define DCACHE_COOKIE		0x0040	/* For use by dcookie subsystem */
+#define DCACHE_FSNOTIFY_PARENT_WATCHED 0x0080
+     /* Parent inode is watched by some fsnotify listener */
 
-#define DCACHE_FSNOTIFY_PARENT_WATCHED	0x0080 /* Parent inode is watched by some fsnotify listener */
+#define DCACHE_MOUNTED		0x0100	/* is a mountpoint */
 
 extern spinlock_t dcache_lock;
 extern seqlock_t rename_lock;
@@ -372,7 +372,7 @@ extern void dput(struct dentry *);
 
 static inline int d_mountpoint(struct dentry *dentry)
 {
-	return dentry->d_mounted;
+	return dentry->d_flags & DCACHE_MOUNTED;
 }
 
 extern struct vfsmount *lookup_mnt(struct path *);

Index: linux-2.6/fs/autofs4/expire.c
===================================================================

fs/autofs4/expire.c +6 -2
@@ -276,7 +276,9 @@ struct dentry *autofs4_expire_direct(str
 		struct autofs_info *ino = autofs4_dentry_ino(root);
 		if (d_mountpoint(root)) {
 			ino->flags |= AUTOFS_INF_MOUNTPOINT;
-			root->d_mounted--;
+			spin_lock(&root->d_lock);
+			root->d_flags &= ~DCACHE_MOUNTED;
+			spin_unlock(&root->d_lock);
 		}
 		ino->flags |= AUTOFS_INF_EXPIRING;
 		init_completion(&ino->expire_complete);
@@ -499,7 +501,9 @@ int autofs4_do_expire_multi(struct super
 
 		spin_lock(&sbi->fs_lock);
 		if (ino->flags & AUTOFS_INF_MOUNTPOINT) {
-			sb->s_root->d_mounted++;
+			spin_lock(&sb->s_root->d_lock);
+			sb->s_root->d_flags |= DCACHE_MOUNTED;
+			spin_unlock(&sb->s_root->d_lock);
 			ino->flags &= ~AUTOFS_INF_MOUNTPOINT;
 		}
 		ino->flags &= ~AUTOFS_INF_EXPIRING;
maintainer
#10 lore raw
In reply to Nick Piggin
Quoted 6 lines · Nick Piggin

I guess you could have a flag in the vfsmount which you could then set to have lookup_mnt (and hence follow_mount etc) ignore it.

Unsetting / decrementing d_mounted I guess works, but I would just be worried if other mounts can be attached to the dentry then you might ignore that other mount or even follow your autofs mount.i

The other thing worth mentioning is that these mounts are entirely managed by the user space daemon, no-one else can mount within the autofs fs (well almost), unless a privileged user space process deliberately uses ioctls to change that.

Ian