Skip to contentSkip to conversation

find_fh_dentry returned a DISCONNECTED directory

PATCH J. Bruce Fields 3 participants 18 messages Feb 13, 2014 – Feb 14, 2014

Subsystems (5)
lore
Share & tools
lore
#0 lore raw

Yesterday you passed on a report of this printk from nfsdfh.c firing:

	printk("nfsd: find_fh_dentry returned a DISCONNECTED directory: %pd2\n",
	                                dentry);

I think the dentry probably comes from the FILEID_ROOT case of:

	if (fileid_type == FILEID_ROOT)
		dentry = dget(exp->ex_path.dentry);
	else {
		dentry = exportfs_decode_fh(exp->ex_path.mnt, fid,
				data_left, fileid_type,
				nfsd_acceptable, exp);
        }

In that case the dentry was found using ordinary filesystem lookups, so doesn't go through the same DISCONNECTED-clearing logic as in the case of lookups by filehandle.

Probably they have an export root that's not a filesystem root, and the lookups happened in the right order?

I suspect that's fine, and that the printk is just stupid, but maybe we should clear DISCONNECTED when possible on normal lookups. The following is my attempt, though I'm not sure if d_alloc is the right place to do this. In any case it might help confirm this is what's happening.

So if you pass along this patch to the person who was seeing that printk I'd be interested in the results.

--b.

commit 49fc5637ca951d2add808a695768512f8b5bd7ad

Author: J. Bruce Fields <[email protected]>
Date:   Thu Feb 13 14:04:43 2014 -0500

dcache: clear DCACHE_DISCONNECTED on child when cleared on parent

Currently DCACHE_DISCONNECTED is only cleared by the filehandle lookup code. That's OK, it's only the filehandle code that really cares about this flag. But it might be kinder to also clear DCACHE_DISCONNECTED on regular lookups when we notice it's appropriate.

fs/dcache.c +11 -0
index 265e0ce..c4a9478 100644
@@ -1534,6 +1534,16 @@ struct dentry *__d_alloc(struct super_block *sb, const struct qstr *name)
 	return dentry;
 }
 
+static void d_inherit_connected(struct dentry *dentry)
+{
+	/*
+	 * If the parent is connected, then the child is too.  (But if
+	 * DISCONNECTED is set, that doesn't tell us anything certain.)
+	 */
+	if (!(dentry->d_parent->d_flags & DCACHE_DISCONNECTED))
+		dentry->d_flags &= ~DCACHE_DISCONNECTED;
+}
+
 /**
  * d_alloc	-	allocate a dcache entry
  * @parent: parent of entry to allocate
@@ -1556,6 +1566,7 @@ struct dentry *d_alloc(struct dentry * parent, const struct qstr *name)
 	 */
 	__dget_dlock(parent);
 	dentry->d_parent = parent;
+	d_inherit_connected(dentry);
 	list_add(&dentry->d_u.d_child, &parent->d_subdirs);
 	spin_unlock(&parent->d_lock);
 
#1 lore raw
Quoted 22 lines · J. Bruce Fields

Yesterday you passed on a report of this printk from nfsdfh.c firing:

	printk("nfsd: find_fh_dentry returned a DISCONNECTED directory: %pd2\n",
	                                dentry);

I think the dentry probably comes from the FILEID_ROOT case of:

	if (fileid_type == FILEID_ROOT)
		dentry = dget(exp->ex_path.dentry);
	else {
		dentry = exportfs_decode_fh(exp->ex_path.mnt, fid,
				data_left, fileid_type,
				nfsd_acceptable, exp);
        }

In that case the dentry was found using ordinary filesystem lookups, so doesn't go through the same DISCONNECTED-clearing logic as in the case of lookups by filehandle.

Probably they have an export root that's not a filesystem root, and the lookups happened in the right order?

Quoted 8 lines

I suspect that's fine, and that the printk is just stupid, but maybe we should clear DISCONNECTED when possible on normal lookups. The following is my attempt, though I'm not sure if d_alloc is the right place to do this. In any case it might help confirm this is what's happening.

So if you pass along this patch to the person who was seeing that printk I'd be interested in the results.

I have been reading through the dentry code for other reasons and your patch definitely won't change anything. __d_alloc sets d_flags = 0. Therefore d_alloc always returns with d_flags == 0.

Eric

Quoted 49 lines

--b.

commit 49fc5637ca951d2add808a695768512f8b5bd7ad

Author: J. Bruce Fields <[email protected]>
Date:   Thu Feb 13 14:04:43 2014 -0500

dcache: clear DCACHE_DISCONNECTED on child when cleared on parent

Currently DCACHE_DISCONNECTED is only cleared by the filehandle lookup code. That's OK, it's only the filehandle code that really cares about this flag. But it might be kinder to also clear DCACHE_DISCONNECTED on regular lookups when we notice it's appropriate.

fs/dcache.c +11 -1
index 265e0ce..c4a9478 100644
@@ -1534,6 +1534,16 @@ struct dentry *__d_alloc(struct super_block *sb, const struct qstr *name)
 	return dentry;
 }
 
+static void d_inherit_connected(struct dentry *dentry)
+{
+	/*
+	 * If the parent is connected, then the child is too.  (But if
+	 * DISCONNECTED is set, that doesn't tell us anything certain.)
+	 */
+	if (!(dentry->d_parent->d_flags & DCACHE_DISCONNECTED))
+		dentry->d_flags &= ~DCACHE_DISCONNECTED;
+}
+
 /**
  * d_alloc	-	allocate a dcache entry
  * @parent: parent of entry to allocate
@@ -1556,6 +1566,7 @@ struct dentry *d_alloc(struct dentry * parent, const struct qstr *name)
 	 */
 	__dget_dlock(parent);
 	dentry->d_parent = parent;
+	d_inherit_connected(dentry);
 	list_add(&dentry->d_u.d_child, &parent->d_subdirs);
 	spin_unlock(&parent->d_lock);
 
--

To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#2 lore raw
Quoted 30 lines · Eric W. Biederman
Quoted 28 lines · J. Bruce Fields

Yesterday you passed on a report of this printk from nfsdfh.c firing:

	printk("nfsd: find_fh_dentry returned a DISCONNECTED directory: %pd2\n",
	                                dentry);

I think the dentry probably comes from the FILEID_ROOT case of:

	if (fileid_type == FILEID_ROOT)
		dentry = dget(exp->ex_path.dentry);
	else {
		dentry = exportfs_decode_fh(exp->ex_path.mnt, fid,
				data_left, fileid_type,
				nfsd_acceptable, exp);
        }

In that case the dentry was found using ordinary filesystem lookups, so doesn't go through the same DISCONNECTED-clearing logic as in the case of lookups by filehandle.

Probably they have an export root that's not a filesystem root, and the lookups happened in the right order?

I suspect that's fine, and that the printk is just stupid, but maybe we should clear DISCONNECTED when possible on normal lookups. The following is my attempt, though I'm not sure if d_alloc is the right place to do this. In any case it might help confirm this is what's happening.

Quoted 6 lines
Quoted 2 lines

So if you pass along this patch to the person who was seeing that printk I'd be interested in the results.

I have been reading through the dentry code for other reasons and your patch definitely won't change anything. __d_alloc sets d_flags = 0. Therefore d_alloc always returns with d_flags == 0.

You're right, of course. I wasn't thinking straight.

So the only dentries with DISCONNECTED set are those created with d_obtain_alias, which is normally only used when you're looking up by filehandle.

Except btrfs has a weird use in get_default_root(). So maybe they were running into the dentry that created?

So btrfs should probably be using something else, I'm not sure what.

--b.

#3 lore raw
Quoted 40 lines · J. Bruce Fields
Quoted 30 lines · Eric W. Biederman
Quoted 28 lines · J. Bruce Fields

Yesterday you passed on a report of this printk from nfsdfh.c firing:

	printk("nfsd: find_fh_dentry returned a DISCONNECTED directory: %pd2\n",
	                                dentry);

I think the dentry probably comes from the FILEID_ROOT case of:

	if (fileid_type == FILEID_ROOT)
		dentry = dget(exp->ex_path.dentry);
	else {
		dentry = exportfs_decode_fh(exp->ex_path.mnt, fid,
				data_left, fileid_type,
				nfsd_acceptable, exp);
        }

In that case the dentry was found using ordinary filesystem lookups, so doesn't go through the same DISCONNECTED-clearing logic as in the case of lookups by filehandle.

Probably they have an export root that's not a filesystem root, and the lookups happened in the right order?

I suspect that's fine, and that the printk is just stupid, but maybe we should clear DISCONNECTED when possible on normal lookups. The following is my attempt, though I'm not sure if d_alloc is the right place to do this. In any case it might help confirm this is what's happening.

Quoted 6 lines
Quoted 2 lines

So if you pass along this patch to the person who was seeing that printk I'd be interested in the results.

I have been reading through the dentry code for other reasons and your patch definitely won't change anything. __d_alloc sets d_flags = 0. Therefore d_alloc always returns with d_flags == 0.

You're right, of course. I wasn't thinking straight.

Quoted 8 lines

So the only dentries with DISCONNECTED set are those created with d_obtain_alias, which is normally only used when you're looking up by filehandle.

Except btrfs has a weird use in get_default_root(). So maybe they were running into the dentry that created?

So btrfs should probably be using something else, I'm not sure what.

The nfs client also has the case where it uses DISCONNECTED dentries for directories that are not root on the server. Which seems very similiar to the btrfs case.

Usually I would expect some of these things to pass through d_materialise_unique and become connected. But I don't know much about the nfsd usage and how things connected.

Eric

#4 lore raw
Quoted 46 lines · Eric W. Biederman
Quoted 44 lines · J. Bruce Fields
Quoted 30 lines · Eric W. Biederman
Quoted 28 lines · J. Bruce Fields

Yesterday you passed on a report of this printk from nfsdfh.c firing:

	printk("nfsd: find_fh_dentry returned a DISCONNECTED directory: %pd2\n",
	                                dentry);

I think the dentry probably comes from the FILEID_ROOT case of:

	if (fileid_type == FILEID_ROOT)
		dentry = dget(exp->ex_path.dentry);
	else {
		dentry = exportfs_decode_fh(exp->ex_path.mnt, fid,
				data_left, fileid_type,
				nfsd_acceptable, exp);
        }

In that case the dentry was found using ordinary filesystem lookups, so doesn't go through the same DISCONNECTED-clearing logic as in the case of lookups by filehandle.

Probably they have an export root that's not a filesystem root, and the lookups happened in the right order?

I suspect that's fine, and that the printk is just stupid, but maybe we should clear DISCONNECTED when possible on normal lookups. The following is my attempt, though I'm not sure if d_alloc is the right place to do this. In any case it might help confirm this is what's happening.

Quoted 6 lines
Quoted 2 lines

So if you pass along this patch to the person who was seeing that printk I'd be interested in the results.

I have been reading through the dentry code for other reasons and your patch definitely won't change anything. __d_alloc sets d_flags = 0. Therefore d_alloc always returns with d_flags == 0.

You're right, of course. I wasn't thinking straight.

So the only dentries with DISCONNECTED set are those created with d_obtain_alias, which is normally only used when you're looking up by filehandle.

Quoted 8 lines
Quoted 4 lines

Except btrfs has a weird use in get_default_root(). So maybe they were running into the dentry that created?

So btrfs should probably be using something else, I'm not sure what.

The nfs client also has the case where it uses DISCONNECTED dentries for directories that are not root on the server. Which seems very similiar to the btrfs case.

I don't think there's any reason for those to be flagged DISCONNECTED either.

--b.

Quoted 7 lines

Usually I would expect some of these things to pass through d_materialise_unique and become connected. But I don't know much about the nfsd usage and how things connected.

Eric

#5 lore raw
Quoted 50 lines · J. Bruce Fields
Quoted 49 lines · Eric W. Biederman
Quoted 47 lines · J. Bruce Fields
Quoted 30 lines · Eric W. Biederman
Quoted 28 lines · J. Bruce Fields

Yesterday you passed on a report of this printk from nfsdfh.c firing:

	printk("nfsd: find_fh_dentry returned a DISCONNECTED directory: %pd2\n",
	                                dentry);

I think the dentry probably comes from the FILEID_ROOT case of:

	if (fileid_type == FILEID_ROOT)
		dentry = dget(exp->ex_path.dentry);
	else {
		dentry = exportfs_decode_fh(exp->ex_path.mnt, fid,
				data_left, fileid_type,
				nfsd_acceptable, exp);
        }

In that case the dentry was found using ordinary filesystem lookups, so doesn't go through the same DISCONNECTED-clearing logic as in the case of lookups by filehandle.

Probably they have an export root that's not a filesystem root, and the lookups happened in the right order?

I suspect that's fine, and that the printk is just stupid, but maybe we should clear DISCONNECTED when possible on normal lookups. The following is my attempt, though I'm not sure if d_alloc is the right place to do this. In any case it might help confirm this is what's happening.

Quoted 6 lines
Quoted 2 lines

So if you pass along this patch to the person who was seeing that printk I'd be interested in the results.

I have been reading through the dentry code for other reasons and your patch definitely won't change anything. __d_alloc sets d_flags = 0. Therefore d_alloc always returns with d_flags == 0.

You're right, of course. I wasn't thinking straight.

So the only dentries with DISCONNECTED set are those created with d_obtain_alias, which is normally only used when you're looking up by filehandle.

Except btrfs has a weird use in get_default_root(). So maybe they were running into the dentry that created?

Quoted 8 lines
Quoted 5 lines
Quoted 1 line

So btrfs should probably be using something else, I'm not sure what.

The nfs client also has the case where it uses DISCONNECTED dentries for directories that are not root on the server. Which seems very similiar to the btrfs case.

I don't think there's any reason for those to be flagged DISCONNECTED either.

The only practical difference between the two cases is how quickly it is desirable to connect the entries.

The disconnected dentries processed by exportfs are dentries that we want to connect immediately, and it is an error/problem to have the disconnected after processing.

The dentries that are the roots of file systems we want to connect them if we get the chance with d_materialise_unique but we don't care if they go long periods without being connected.

I believe we want both groups of dentries on the s_anon list so that if they remain disconnected when the filesystem is unmounted we can find them and deal with them.

I can see distinguishing between dentries that are supposed to be disconnected for a short time, and dentries that are supposed to be disconnected indefinitely but we currently (as of 3.14-rc1) don't have that distinction.

But a blanket statement that the long term disconnected dentries are doing it wrong seems off base.

If those dentires can tolerate not being on the s_anon list d_alloc_pseudo or d_make_root looks like it will serve just as well from the perspective of d_materialise_unique, but that leaves me with the queasy feeling that we will leak dentries and inodes when we unmount the filesystems in question, if those dentries have never been attached.

Eric

#6 lore raw
Quoted 68 lines · Eric W. Biederman
Quoted 50 lines · J. Bruce Fields
Quoted 49 lines · Eric W. Biederman
Quoted 47 lines · J. Bruce Fields
Quoted 30 lines · Eric W. Biederman
Quoted 28 lines · J. Bruce Fields

Yesterday you passed on a report of this printk from nfsdfh.c firing:

	printk("nfsd: find_fh_dentry returned a DISCONNECTED directory: %pd2\n",
	                                dentry);

I think the dentry probably comes from the FILEID_ROOT case of:

	if (fileid_type == FILEID_ROOT)
		dentry = dget(exp->ex_path.dentry);
	else {
		dentry = exportfs_decode_fh(exp->ex_path.mnt, fid,
				data_left, fileid_type,
				nfsd_acceptable, exp);
        }

In that case the dentry was found using ordinary filesystem lookups, so doesn't go through the same DISCONNECTED-clearing logic as in the case of lookups by filehandle.

Probably they have an export root that's not a filesystem root, and the lookups happened in the right order?

I suspect that's fine, and that the printk is just stupid, but maybe we should clear DISCONNECTED when possible on normal lookups. The following is my attempt, though I'm not sure if d_alloc is the right place to do this. In any case it might help confirm this is what's happening.

Quoted 6 lines
Quoted 2 lines

So if you pass along this patch to the person who was seeing that printk I'd be interested in the results.

I have been reading through the dentry code for other reasons and your patch definitely won't change anything. __d_alloc sets d_flags = 0. Therefore d_alloc always returns with d_flags == 0.

You're right, of course. I wasn't thinking straight.

So the only dentries with DISCONNECTED set are those created with d_obtain_alias, which is normally only used when you're looking up by filehandle.

Except btrfs has a weird use in get_default_root(). So maybe they were running into the dentry that created?

Quoted 8 lines
Quoted 5 lines
Quoted 1 line

So btrfs should probably be using something else, I'm not sure what.

The nfs client also has the case where it uses DISCONNECTED dentries for directories that are not root on the server. Which seems very similiar to the btrfs case.

I don't think there's any reason for those to be flagged DISCONNECTED either.

The only practical difference between the two cases is how quickly it is desirable to connect the entries.

The disconnected dentries processed by exportfs are dentries that we want to connect immediately, and it is an error/problem to have the disconnected after processing.

Quoted 7 lines

The dentries that are the roots of file systems we want to connect them if we get the chance with d_materialise_unique but we don't care if they go long periods without being connected.

I believe we want both groups of dentries on the s_anon list so that if they remain disconnected when the filesystem is unmounted we can find them and deal with them.

Note it's IS_ROOT(), not DCACHE_DISCONNECTED, that determines whether a hashed dentry is on s_anon or not. (See 7632e465feb182cadc3c9aa1282a057201818a8c for more detailed discussion.)

Quoted 4 lines

I can see distinguishing between dentries that are supposed to be disconnected for a short time, and dentries that are supposed to be disconnected indefinitely but we currently (as of 3.14-rc1) don't have that distinction.

I believe we do: DCACHE_DISCONNECTED is for the former, and the latter should be IS_ROOT(), !DCACHE_DISCONNECTED.

DCACHE_DISCONNECTED was intended to be used only for dentries created while performing lookup-by-filehandle which have not yet been confirmed to be linked back to filesystem root by a chain of ->d_parent pointers.

Quoted 5 lines

But a blanket statement that the long term disconnected dentries are doing it wrong seems off base.

If those dentires can tolerate not being on the s_anon list d_alloc_pseudo or d_make_root looks like it will serve just as well

The difference is that d_alloc_pseudo and d_make_root unconditionally create new dentries, whereas d_materialise_unique lets us reuse an existing dentry.

(In the NFS client case, I believe this happens for example when you mount export A from a server, then export B from the same server, and then one day you look up A/foo and find out that it's the same directory as B's root. You don't want to duplicate the inode or give it multiple dentries, so you instead reuse the existing IS_ROOT dentry for B to represent A/foo.

In the btrfs case I guess it's a similar situation but with two subvolumes instead of two exports?)

Quoted 4 lines

from
the perspective of d_materialise_unique, but that leaves me with the queasy feeling that we will leak dentries and inodes when we unmount the filesystems in question, if those dentries have never been attached.

In the NFS server (lookup-by-filehandle) case, I believe dentries in the process of being reconnected are all children of some IS_ROOT dentry which is on the s_anon list, so everyone's accounted for.

Thanks for looking at this as I've found myself easily confused by it all.... (And judging by some of the code in the tree I'm not alone.)

--b.

#7 lore raw

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Quoted 133 lines · J. Bruce Fields
Quoted 74 lines · Eric W. Biederman
Quoted 58 lines · J. Bruce Fields
Quoted 56 lines · Eric W. Biederman
Quoted 53 lines · J. Bruce Fields
Quoted 32 lines · Eric W. Biederman
Quoted 30 lines · J. Bruce Fields

Yesterday you passed on a report of this printk from nfsdfh.c firing:

printk("nfsd: find_fh_dentry returned a DISCONNECTED
directory: %pd2\n", dentry);

I think the dentry probably comes from the FILEID_ROOT case of:

if (fileid_type == FILEID_ROOT) dentry = dget(exp->ex_path.dentry); else { dentry = exportfs_decode_fh(exp->ex_path.mnt, fid, data_left, fileid_type, nfsd_acceptable, exp); }

In that case the dentry was found using ordinary filesystem lookups, so doesn't go through the same DISCONNECTED-clearing logic as in the case of lookups by filehandle.

Probably they have an export root that's not a filesystem root, and the lookups happened in the right order?

I suspect that's fine, and that the printk is just stupid, but maybe we should clear DISCONNECTED when possible on normal lookups. The following is my attempt, though I'm not sure if d_alloc is the right place to do this. In any case it might help confirm this is what's happening.

Quoted 7 lines
Quoted 2 lines

So if you pass along this patch to the person who was seeing that printk I'd be interested in the results.

I have been reading through the dentry code for other reasons and your patch definitely won't change anything. __d_alloc sets d_flags = 0. Therefore d_alloc always returns with d_flags == 0.

You're right, of course. I wasn't thinking straight.

So the only dentries with DISCONNECTED set are those created with d_obtain_alias, which is normally only used when you're looking up by filehandle.

Except btrfs has a weird use in get_default_root(). So maybe they were running into the dentry that created?

So btrfs should probably be using something else, I'm not sure what.

Quoted 6 lines
Quoted 3 lines

The nfs client also has the case where it uses DISCONNECTED dentries for directories that are not root on the server. Which seems very similiar to the btrfs case.

I don't think there's any reason for those to be flagged DISCONNECTED either.

The only practical difference between the two cases is how quickly it is desirable to connect the entries.

The disconnected dentries processed by exportfs are dentries that we want to connect immediately, and it is an error/problem to have the disconnected after processing.

Quoted 7 lines

The dentries that are the roots of file systems we want to connect them if we get the chance with d_materialise_unique but we don't care if they go long periods without being connected.

I believe we want both groups of dentries on the s_anon list so that if they remain disconnected when the filesystem is unmounted we can find them and deal with them.

Note it's IS_ROOT(), not DCACHE_DISCONNECTED, that determines whether a hashed dentry is on s_anon or not. (See 7632e465feb182cadc3c9aa1282a057201818a8c for more detailed discussion.)

Quoted 4 lines

I can see distinguishing between dentries that are supposed to be disconnected for a short time, and dentries that are supposed to be disconnected indefinitely but we currently (as of 3.14-rc1) don't have that distinction.

I believe we do: DCACHE_DISCONNECTED is for the former, and the latter should be IS_ROOT(), !DCACHE_DISCONNECTED.

DCACHE_DISCONNECTED was intended to be used only for dentries created while performing lookup-by-filehandle which have not yet been confirmed to be linked back to filesystem root by a chain of ->d_parent pointers.

Quoted 6 lines

But a blanket statement that the long term disconnected dentries are doing it wrong seems off base.

If those dentires can tolerate not being on the s_anon list d_alloc_pseudo or d_make_root looks like it will serve just as well

The difference is that d_alloc_pseudo and d_make_root unconditionally create new dentries, whereas d_materialise_unique lets us reuse an existing dentry.

(In the NFS client case, I believe this happens for example when you mount export A from a server, then export B from the same server, and then one day you look up A/foo and find out that it's the same directory as B's root. You don't want to duplicate the inode or give it multiple dentries, so you instead reuse the existing IS_ROOT dentry for B to represent A/foo.

In the btrfs case I guess it's a similar situation but with two subvolumes instead of two exports?)

Quoted 4 lines

from the perspective of d_materialise_unique, but that leaves me with the queasy feeling that we will leak dentries and inodes when we unmount the filesystems in question, if those dentries have never been attached.

In the NFS server (lookup-by-filehandle) case, I believe dentries in the process of being reconnected are all children of some IS_ROOT dentry which is on the s_anon list, so everyone's accounted for.

Quoted 4 lines

Thanks for looking at this as I've found myself easily confused by it all.... (And judging by some of the code in the tree I'm not alone.)

Ok using d_make_root makes us leak inodes on umount (I don't know why, I'm not drunk enough yet.) I haven't tried it yet but I'm almost 100% positive if I just clear out DISCONNECTED we will BUG_ON() in d_splice_alias() when we go to walk into the subvol we've mounted with the default subvol option. So what I think I need to do is use use d_materialise_unique in our lookup function instead of d_splice_alias() and keep using d_obtain_alias() in get_default_root() and just clear DISOCONNECTED on success. Does this sound crazy to anybody else? Thanks,

Josef
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBAgAGBQJS/kaBAAoJEANb+wAKly3BXp8P/RiWEPYsJXwmD9N2JamQZawa p5Fh507ZX3PRaOAZ6zgWB2uN4xRQtqFpsIjoZl6c2NoxnR4L89d4x/ExBF39WE2t PmK3Ot/KJ6GXBVHSyexWJMix8R8u40eKHJcyywqj35HhHML47Ll1fy6k2vmRno5q FVP+E0kLiBOH0W2ae8o7kPImLp2Ue7moDvExlqKTI3jeRPQI5u+/lH4uj8oVQXx4 /HNoZ7+htzi0Eb+mA+ve0k2/efiUlPloerE0oFUaCrrOF0HMLAVoXxhiDR7eDM96 nZ1LHZ422DOBRCBQ6kpDo7iAebQyRKx6rdkdr/n5/5Bs0iTevg3bWy8WEbMP+NvW OR+mQ4r3X8J7t91Dp15OJuiEPhyH6lb2McPcxq/ozRDcGR0enZ1iYGl/GNw02eaw 1/JS4+nwokLCcEvTiW16n8sLGv+iU4fXawisjdKs76KQGO+rzdOd2msYjnOF8ZgT YTO4k689qlWJu6TtY8iC0fRStIksTAMesMmCoCBl+2zkGsHMS9tMQt3kzIF91UP4 WTuZOBxdqByKmQiWy0INKhYXOSVxAUs27JuaHPIBUz65fDDu80IW+2Ih1sro73y/ +t9+7vSueviUub1azUocEdYxuc8mDQ7totpzrJBYHCF7C1T2DZV5nEpGUHD/d19R 3QwsV2oPiu230jh+nJR2
=VIUq
-----END PGP SIGNATURE-----

#8 lore raw
Quoted 139 lines · Josef Bacik

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Quoted 133 lines · J. Bruce Fields
Quoted 74 lines · Eric W. Biederman
Quoted 58 lines · J. Bruce Fields
Quoted 56 lines · Eric W. Biederman
Quoted 53 lines · J. Bruce Fields
Quoted 32 lines · Eric W. Biederman
Quoted 30 lines · J. Bruce Fields

Yesterday you passed on a report of this printk from nfsdfh.c firing:

printk("nfsd: find_fh_dentry returned a DISCONNECTED
directory: %pd2\n", dentry);

I think the dentry probably comes from the FILEID_ROOT case of:

if (fileid_type == FILEID_ROOT) dentry = dget(exp->ex_path.dentry); else { dentry = exportfs_decode_fh(exp->ex_path.mnt, fid, data_left, fileid_type, nfsd_acceptable, exp); }

In that case the dentry was found using ordinary filesystem lookups, so doesn't go through the same DISCONNECTED-clearing logic as in the case of lookups by filehandle.

Probably they have an export root that's not a filesystem root, and the lookups happened in the right order?

I suspect that's fine, and that the printk is just stupid, but maybe we should clear DISCONNECTED when possible on normal lookups. The following is my attempt, though I'm not sure if d_alloc is the right place to do this. In any case it might help confirm this is what's happening.

Quoted 7 lines
Quoted 2 lines

So if you pass along this patch to the person who was seeing that printk I'd be interested in the results.

I have been reading through the dentry code for other reasons and your patch definitely won't change anything. __d_alloc sets d_flags = 0. Therefore d_alloc always returns with d_flags == 0.

You're right, of course. I wasn't thinking straight.

So the only dentries with DISCONNECTED set are those created with d_obtain_alias, which is normally only used when you're looking up by filehandle.

Except btrfs has a weird use in get_default_root(). So maybe they were running into the dentry that created?

So btrfs should probably be using something else, I'm not sure what.

Quoted 6 lines
Quoted 3 lines

The nfs client also has the case where it uses DISCONNECTED dentries for directories that are not root on the server. Which seems very similiar to the btrfs case.

I don't think there's any reason for those to be flagged DISCONNECTED either.

The only practical difference between the two cases is how quickly it is desirable to connect the entries.

The disconnected dentries processed by exportfs are dentries that we want to connect immediately, and it is an error/problem to have the disconnected after processing.

Quoted 7 lines

The dentries that are the roots of file systems we want to connect them if we get the chance with d_materialise_unique but we don't care if they go long periods without being connected.

I believe we want both groups of dentries on the s_anon list so that if they remain disconnected when the filesystem is unmounted we can find them and deal with them.

Note it's IS_ROOT(), not DCACHE_DISCONNECTED, that determines whether a hashed dentry is on s_anon or not. (See 7632e465feb182cadc3c9aa1282a057201818a8c for more detailed discussion.)

Quoted 4 lines

I can see distinguishing between dentries that are supposed to be disconnected for a short time, and dentries that are supposed to be disconnected indefinitely but we currently (as of 3.14-rc1) don't have that distinction.

I believe we do: DCACHE_DISCONNECTED is for the former, and the latter should be IS_ROOT(), !DCACHE_DISCONNECTED.

DCACHE_DISCONNECTED was intended to be used only for dentries created while performing lookup-by-filehandle which have not yet been confirmed to be linked back to filesystem root by a chain of ->d_parent pointers.

Quoted 6 lines

But a blanket statement that the long term disconnected dentries are doing it wrong seems off base.

If those dentires can tolerate not being on the s_anon list d_alloc_pseudo or d_make_root looks like it will serve just as well

The difference is that d_alloc_pseudo and d_make_root unconditionally create new dentries, whereas d_materialise_unique lets us reuse an existing dentry.

(In the NFS client case, I believe this happens for example when you mount export A from a server, then export B from the same server, and then one day you look up A/foo and find out that it's the same directory as B's root. You don't want to duplicate the inode or give it multiple dentries, so you instead reuse the existing IS_ROOT dentry for B to represent A/foo.

In the btrfs case I guess it's a similar situation but with two subvolumes instead of two exports?)

Quoted 4 lines

from the perspective of d_materialise_unique, but that leaves me with the queasy feeling that we will leak dentries and inodes when we unmount the filesystems in question, if those dentries have never been attached.

In the NFS server (lookup-by-filehandle) case, I believe dentries in the process of being reconnected are all children of some IS_ROOT dentry which is on the s_anon list, so everyone's accounted for.

Quoted 7 lines
Quoted 4 lines

Thanks for looking at this as I've found myself easily confused by it all.... (And judging by some of the code in the tree I'm not alone.)

Ok using d_make_root makes us leak inodes on umount (I don't know why, I'm not drunk enough yet.)

Oh, well, a little early in the morning, but that's a solveable problem.

Quoted 4 lines

I haven't tried it yet but I'm almost 100%
positive if I just clear out DISCONNECTED we will BUG_ON() in d_splice_alias() when we go to walk into the subvol we've mounted with the default subvol option.

I'd expect it not to BUG_ON(), but instead to fall into the !new case and create a second alias for that directory. Which probably actually causes the same problems d_make_root does.

Quoted 5 lines

So what I think I need to do is use use d_materialise_unique in our lookup function instead of d_splice_alias() and keep using d_obtain_alias() in get_default_root() and just clear DISOCONNECTED on success. Does this sound crazy to anybody else? Thanks,

Probably not crazy.

--b.

Quoted 20 lines

Josef
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBAgAGBQJS/kaBAAoJEANb+wAKly3BXp8P/RiWEPYsJXwmD9N2JamQZawa p5Fh507ZX3PRaOAZ6zgWB2uN4xRQtqFpsIjoZl6c2NoxnR4L89d4x/ExBF39WE2t PmK3Ot/KJ6GXBVHSyexWJMix8R8u40eKHJcyywqj35HhHML47Ll1fy6k2vmRno5q FVP+E0kLiBOH0W2ae8o7kPImLp2Ue7moDvExlqKTI3jeRPQI5u+/lH4uj8oVQXx4 /HNoZ7+htzi0Eb+mA+ve0k2/efiUlPloerE0oFUaCrrOF0HMLAVoXxhiDR7eDM96 nZ1LHZ422DOBRCBQ6kpDo7iAebQyRKx6rdkdr/n5/5Bs0iTevg3bWy8WEbMP+NvW OR+mQ4r3X8J7t91Dp15OJuiEPhyH6lb2McPcxq/ozRDcGR0enZ1iYGl/GNw02eaw 1/JS4+nwokLCcEvTiW16n8sLGv+iU4fXawisjdKs76KQGO+rzdOd2msYjnOF8ZgT YTO4k689qlWJu6TtY8iC0fRStIksTAMesMmCoCBl+2zkGsHMS9tMQt3kzIF91UP4 WTuZOBxdqByKmQiWy0INKhYXOSVxAUs27JuaHPIBUz65fDDu80IW+2Ih1sro73y/ +t9+7vSueviUub1azUocEdYxuc8mDQ7totpzrJBYHCF7C1T2DZV5nEpGUHD/d19R 3QwsV2oPiu230jh+nJR2
=VIUq
-----END PGP SIGNATURE-----

#9 lore raw

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Quoted 155 lines · J. Bruce Fields

On Fri, Feb 14, 2014 at 11:38:25AM -0500, Josef Bacik wrote:

Quoted 143 lines · J. Bruce Fields
Quoted 143 lines
Quoted 143 lines
Quoted 85 lines · Eric W. Biederman
Quoted 63 lines · J. Bruce Fields
Quoted 61 lines · Eric W. Biederman
Quoted 58 lines · J. Bruce Fields
Quoted 37 lines · Eric W. Biederman
Quoted 31 lines · J. Bruce Fields

Yesterday you passed on a report of this printk from nfsdfh.c firing:

printk("nfsd: find_fh_dentry returned a DISCONNECTED directory: %pd2\n", dentry);

I think the dentry probably comes from the FILEID_ROOT case of:

if (fileid_type == FILEID_ROOT) dentry = dget(exp->ex_path.dentry); else { dentry = exportfs_decode_fh(exp->ex_path.mnt, fid, data_left, fileid_type, nfsd_acceptable, exp); }

In that case the dentry was found using ordinary filesystem lookups, so doesn't go through the same DISCONNECTED-clearing logic as in the case of lookups by filehandle.

Probably they have an export root that's not a filesystem root, and the lookups happened in the right order?

I suspect that's fine, and that the printk is just stupid, but maybe we should clear DISCONNECTED when possible on normal lookups. The following is my attempt, though I'm not sure if d_alloc is the right place to do this. In any case it might help confirm this is what's happening.

Quoted 3 lines

So if you pass along this patch to the person who was seeing that printk I'd be interested in the results.

Quoted 5 lines

I have been reading through the dentry code for other reasons and your patch definitely won't change anything. __d_alloc sets d_flags = 0. Therefore d_alloc always returns with d_flags == 0.

You're right, of course. I wasn't thinking straight.

So the only dentries with DISCONNECTED set are those created with d_obtain_alias, which is normally only used when you're looking up by filehandle.

Except btrfs has a weird use in get_default_root(). So maybe they were running into the dentry that created?

So btrfs should probably be using something else, I'm not sure what.

Quoted 7 lines
Quoted 4 lines

The nfs client also has the case where it uses DISCONNECTED dentries for directories that are not root on the server. Which seems very similiar to the btrfs case.

I don't think there's any reason for those to be flagged DISCONNECTED either.

The only practical difference between the two cases is how quickly it is desirable to connect the entries.

The disconnected dentries processed by exportfs are dentries that we want to connect immediately, and it is an error/problem to have the disconnected after processing.

The dentries that are the roots of file systems we want to connect them if we get the chance with d_materialise_unique but we don't care if they go long periods without being connected.

Quoted 4 lines

I believe we want both groups of dentries on the s_anon list so that if they remain disconnected when the filesystem is unmounted we can find them and deal with them.

Note it's IS_ROOT(), not DCACHE_DISCONNECTED, that determines whether a hashed dentry is on s_anon or not. (See
7632e465feb182cadc3c9aa1282a057201818a8c for more detailed
discussion.)

Quoted 4 lines

I can see distinguishing between dentries that are supposed to be disconnected for a short time, and dentries that are supposed to be disconnected indefinitely but we currently (as of 3.14-rc1) don't have that distinction.

I believe we do: DCACHE_DISCONNECTED is for the former, and the latter should be IS_ROOT(), !DCACHE_DISCONNECTED.

DCACHE_DISCONNECTED was intended to be used only for dentries created while performing lookup-by-filehandle which have not yet been confirmed to be linked back to filesystem root by a chain of ->d_parent pointers.

Quoted 6 lines

But a blanket statement that the long term disconnected dentries are doing it wrong seems off base.

If those dentires can tolerate not being on the s_anon list
d_alloc_pseudo or d_make_root looks like it will serve
just as well

The difference is that d_alloc_pseudo and d_make_root unconditionally create new dentries, whereas d_materialise_unique lets us reuse an existing dentry.

(In the NFS client case, I believe this happens for example when you mount export A from a server, then export B from the same server, and then one day you look up A/foo and find out that it's the same directory as B's root. You don't want to duplicate the inode or give it multiple dentries, so you instead reuse the existing IS_ROOT dentry for B to represent A/foo.

In the btrfs case I guess it's a similar situation but with two subvolumes instead of two exports?)

Quoted 4 lines

from the perspective of d_materialise_unique, but that leaves me with the queasy feeling that we will leak dentries and inodes when we unmount the filesystems in question, if those dentries have never been attached.

In the NFS server (lookup-by-filehandle) case, I believe dentries in the process of being reconnected are all children of some IS_ROOT dentry which is on the s_anon list, so everyone's accounted for.

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

Thanks for looking at this as I've found myself easily confused by it all.... (And judging by some of the code in the tree I'm not alone.)

Ok using d_make_root makes us leak inodes on umount (I don't know why, I'm not drunk enough yet.)

Quoted 11 lines
Quoted 2 lines

Oh, well, a little early in the morning, but that's a solveable problem.

I haven't tried it yet but I'm almost 100% positive if I just clear out DISCONNECTED we will BUG_ON() in d_splice_alias() when we go to walk into the subvol we've mounted with the default subvol option.

Quoted 3 lines

I'd expect it not to BUG_ON(), but instead to fall into the !new case and create a second alias for that directory. Which probably actually causes the same problems d_make_root does.

(weird that thunderbird is treating my replies as the most recent reply, but whatever)

So neither of these cases happened, I just didn't see _anything_ in the directory when I mounted the fs. Once I mounted the parent subvol and walked into the default subvol I could see everything properly. This is with clearing DISCONNECTED and still using d_splice_alias() in lookup.

Once I switched d_splice_alias() to d_materialise_unique() in btrfs_lookup() everything went back to working as normal. So I'm calling it a win and forgetting we ever had this conversation. Thanks,

Josef
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBAgAGBQJS/kwfAAoJEANb+wAKly3BHSIP/jx01XDxT98Tk4TCRz2R12Cb HmgWZ8etu82MPp/KChuXnidxJU00H7BOfaY4vs1q8XFz+MZg3ZCiazp5zu2RVPct yjMYRY7XPEQcce0Oj5h40dfhclJ61eSupuUd7u+gNpNErsUdaFCpbtXGqVeGAuI4 lMbi8Tqd5YKNa3e+bs8OHEGRG+pQv1Ak2D+dcS+KDn3/Np38wC9eGRsQsZocubQf EpjERcX3R8ug3k5C/sX15SkpVfHWZS2ydL0ypnOVAI5XZD9ufT+HyyAqhgFZFPlK WhDTWH54gnq6e2RxU3bKN+q7VzwBD2L/82wHk0/bykXDL+56JRJ0MMptBimWt2M2 QH7+aUN3pM/hm0TiW3qC6WkfM4WBdEME/Mzf8s5vVJT4gQj7u0Xh3RlOX0Y+OEwV MvICzlXNNT9jbp7EHHzDeJuSbvpLmvVOL24Tf9UmjYO2WXHIJK2bSW1ai2V5v+zx jZ+NeAg8tfkpxKANbDxtOAzrehHTk8gPimsO3QXFlRvUyUggZdd524Chl07dxnQZ JNvn4C9JrRkYE32d4dKrjmJvnQXsFKw1KlCvp14nLpQP6q5+J70KRo85NtcpK3fo tUU24jzCD7m6W+Y5tT5rJgDLz7IAmTZ/1WTI9bc9YMk61ci7GlLE0g4hYbTWfS5h Q0A5+xddyQT/I5HtCoPZ
=dBiR
-----END PGP SIGNATURE-----

#10 lore raw
Quoted 10 lines · Josef Bacik

So neither of these cases happened, I just didn't see _anything_ in the directory when I mounted the fs. Once I mounted the parent subvol and walked into the default subvol I could see everything properly. This is with clearing DISCONNECTED and still using d_splice_alias() in lookup.

Once I switched d_splice_alias() to d_materialise_unique() in btrfs_lookup() everything went back to working as normal. So I'm calling it a win and forgetting we ever had this conversation. Thanks,

Yes. d_splice_alias won't find anything unless DCACHE_DISCONNECTED is set, and dmaterialse_unique is definitely what you want.

Eric

#11 lore raw
In reply to Josef Bacik
Quoted 9 lines · Josef Bacik

Ok using d_make_root makes us leak inodes on umount (I don't know why, I'm not drunk enough yet.) I haven't tried it yet but I'm almost 100% positive if I just clear out DISCONNECTED we will BUG_ON() in d_splice_alias() when we go to walk into the subvol we've mounted with the default subvol option. So what I think I need to do is use use d_materialise_unique in our lookup function instead of d_splice_alias() and keep using d_obtain_alias() in get_default_root() and just clear DISOCONNECTED on success. Does this sound crazy to anybody else? Thanks,

That sounds exactly right.

Eric

#12 lore raw
In reply to J. Bruce Fields
Quoted 10 lines · J. Bruce Fields
Quoted 4 lines · Eric W. Biederman

I believe we want both groups of dentries on the s_anon list so that if they remain disconnected when the filesystem is unmounted we can find them and deal with them.

Note it's IS_ROOT(), not DCACHE_DISCONNECTED, that determines whether a hashed dentry is on s_anon or not. (See 7632e465feb182cadc3c9aa1282a057201818a8c for more detailed discussion.)

Interesting. It remains the case that d_obtain_alias that sets DCACHE_DISCONNECTED is the only way to get on the s_anon list.

Quoted 1 line

From the rest of the conversation below I think what is needed is

d_obtain_alias_root. A function like d_obtain_alias but that does not
set DCACHE_DISCONNECTED, that places IS_ROOT dentries on the s_anon list and that is usually expected to not find an alias.

Quoted 5 lines
Quoted 4 lines

I can see distinguishing between dentries that are supposed to be disconnected for a short time, and dentries that are supposed to be disconnected indefinitely but we currently (as of 3.14-rc1) don't have that distinction.

Quoted 7 lines

I believe we do: DCACHE_DISCONNECTED is for the former, and the latter should be IS_ROOT(), !DCACHE_DISCONNECTED.

DCACHE_DISCONNECTED was intended to be used only for dentries created while performing lookup-by-filehandle which have not yet been confirmed to be linked back to filesystem root by a chain of ->d_parent pointers.

Given the current usages that was not at all clear from reading through
the code. So I suggest d_obtain_alias_root to sort the last of it out.

Quoted 9 lines
Quoted 5 lines

But a blanket statement that the long term disconnected dentries are doing it wrong seems off base.

If those dentires can tolerate not being on the s_anon list d_alloc_pseudo or d_make_root looks like it will serve just as well

The difference is that d_alloc_pseudo and d_make_root unconditionally create new dentries, whereas d_materialise_unique lets us reuse an existing dentry.

Aliasing differences aside when dentries are allocated my point is that d_materialise_uniuqe does not care afterwards if DCACHE_DISCONNECTED is set or not, and d_materialise_unique will perform the connection if that is possible later on.

Quoted 9 lines

(In the NFS client case, I believe this happens for example when you mount export A from a server, then export B from the same server, and then one day you look up A/foo and find out that it's the same directory as B's root. You don't want to duplicate the inode or give it multiple dentries, so you instead reuse the existing IS_ROOT dentry for B to represent A/foo.

In the btrfs case I guess it's a similar situation but with two subvolumes instead of two exports?)

That is what it sounds like. And d_materialise_unique will connect them in either case.

So we just need d_obtain_alias_root to allocate these weird oddball dentries and we should be good.

I suspect the code would be much clearer if we were to do a mass s/DCACHE_DISCONNECTED/DCACHE_CONNECTING/ to make it clear that the flag is not intended to cover the general case of when dentries are floating around without parents.

Quoted 8 lines
Quoted 4 lines

from
the perspective of d_materialise_unique, but that leaves me with the queasy feeling that we will leak dentries and inodes when we unmount the filesystems in question, if those dentries have never been attached.

In the NFS server (lookup-by-filehandle) case, I believe dentries in the process of being reconnected are all children of some IS_ROOT dentry which is on the s_anon list, so everyone's accounted for.

My concern there is for all of the other cases. Because so far today only DCACHE_DISCONNECTED dentries land on the s_anon list and that means if we don't use d_obtain_alias and we can have dentry leaks and unmount.

Quoted 2 lines

Thanks for looking at this as I've found myself easily confused by it all.... (And judging by some of the code in the tree I'm not alone.)

I am in the final stages of putting together some patches so that filesystems don't need to like to the vfs to preserve vfs invariants about mountpoints. In particular I am implemeting automatic detaching of mountpionts on unlink and d_invalidate. Once that is done and everyone is dropping directory dentries instead of sticking to the silly 2.2 era logic that present resides in d_invalidate some of these other dcache uses should become a little less mysterious.

Eric

#13 lore raw
Quoted 8 lines · Eric W. Biederman
Quoted 6 lines · J. Bruce Fields
Quoted 4 lines · Eric W. Biederman

I believe we want both groups of dentries on the s_anon list so that if they remain disconnected when the filesystem is unmounted we can find them and deal with them.

Quoted 7 lines
Quoted 4 lines

Note it's IS_ROOT(), not DCACHE_DISCONNECTED, that determines whether a hashed dentry is on s_anon or not. (See 7632e465feb182cadc3c9aa1282a057201818a8c for more detailed discussion.)

Interesting. It remains the case that d_obtain_alias that sets DCACHE_DISCONNECTED is the only way to get on the s_anon list.

You're right, I forgot that.

Quoted 4 lines
Quoted 1 line

From the rest of the conversation below I think what is needed is

d_obtain_alias_root. A function like d_obtain_alias but that does not
set DCACHE_DISCONNECTED, that places IS_ROOT dentries on the s_anon list and that is usually expected to not find an alias.

Sounds good.

Quoted 4 lines

I suspect the code would be much clearer if we were to do a mass s/DCACHE_DISCONNECTED/DCACHE_CONNECTING/ to make it clear that the flag is not intended to cover the general case of when dentries are floating around without parents.

I'm all for it.

bfields@pepper:linux-2.6$ git grep DCACHE_DISCONNECTED|wc -l
24

So it's not so terrible. All but a few are in dcache.c and expfs.c.

Quoted 7 lines

I am in the final stages of putting together some patches so that filesystems don't need to like to the vfs to preserve vfs invariants about mountpoints. In particular I am implemeting automatic detaching of mountpionts on unlink and d_invalidate. Once that is done and everyone is dropping directory dentries instead of sticking to the silly 2.2 era logic that present resides in d_invalidate some of these other dcache uses should become a little less mysterious.

OK.

--b.

#14 lore raw
Quoted 17 lines · J. Bruce Fields
Quoted 8 lines · Eric W. Biederman
Quoted 6 lines · J. Bruce Fields
Quoted 4 lines · Eric W. Biederman

I believe we want both groups of dentries on the s_anon list so that if they remain disconnected when the filesystem is unmounted we can find them and deal with them.

Quoted 7 lines
Quoted 4 lines

Note it's IS_ROOT(), not DCACHE_DISCONNECTED, that determines whether a hashed dentry is on s_anon or not. (See 7632e465feb182cadc3c9aa1282a057201818a8c for more detailed discussion.)

Interesting. It remains the case that d_obtain_alias that sets DCACHE_DISCONNECTED is the only way to get on the s_anon list.

Quoted 8 lines

You're right, I forgot that.

Quoted 4 lines
Quoted 1 line

From the rest of the conversation below I think what is needed is

d_obtain_alias_root. A function like d_obtain_alias but that does not
set DCACHE_DISCONNECTED, that places IS_ROOT dentries on the s_anon list and that is usually expected to not find an alias.

Sounds good.

So, something like this.

(Untested, and will also screw up nilfs2 until we fix d_splice_alias not to require DCACHE_DISCONNECTED dentries.)

--b.

commit c2bc8557e935b24caa2174403b093bc00dc62112

Author: J. Bruce Fields <[email protected]>
Date:   Fri Feb 14 17:35:37 2014 -0500

dcache: d_obtain_alias callers don't all want DISCONNECTED

There are a few d_obtain_alias callers that are using it to get the root of a filesystem which may already have an alias somewhere else.

This is not the same as the filehandle-lookup case, and none of them actually need DCACHE_DISCONNECTED set.

In the btrfs case this was causing a spurious printk from nfsd/nfsfh.c:fh_verify when it found an unexpected DCACHE_DISCONNECTED dentry.

Which isn't really a serious problem, but it would really be clearer if we reserved DCACHE_DISCONNECTED for those cases where it's actually needed.

fs/btrfs/super.c +1 -1
index 97cc241..07ce96b 100644
@@ -925,7 +925,7 @@ setup_root:
 		return dget(sb->s_root);
 	}
 
-	return d_obtain_alias(inode);
+	return d_obtain_alias_root(inode);
 }
 
 static int btrfs_fill_super(struct super_block *sb,
fs/ceph/super.c +1 -1
index 2df963f..9666950 100644
@@ -732,7 +732,7 @@ static struct dentry *open_root_dentry(struct ceph_fs_client *fsc,
 				goto out;
 			}
 		} else {
-			root = d_obtain_alias(inode);
+			root = d_obtain_alias_root(inode);
 		}
 		ceph_init_dentry(root);
 		dout("open_root_inode success, root dentry is %p\n", root);
fs/dcache.c +49 -20

90 lines not on this page · raw message

fs/nfs/getroot.c +1 -1
index 66984a9..7b32522 100644
@@ -112,7 +112,7 @@ struct dentry *nfs_get_root(struct super_block *sb, struct nfs_fh *mntfh,
 	 * if the dentry tree reaches them; however if the dentry already
 	 * exists, we'll pick it up at this point and use it as the root
 	 */
-	ret = d_obtain_alias(inode);
+	ret = d_obtain_alias_root(inode);
 	if (IS_ERR(ret)) {
 		dprintk("nfs_get_root: get root dentry failed\n");
 		goto out;
fs/nilfs2/super.c +1 -1
index 7ac2a12..a518a73 100644
@@ -942,7 +942,7 @@ static int nilfs_get_root_dentry(struct super_block *sb,
 			iput(inode);
 		}
 	} else {
-		dentry = d_obtain_alias(inode);
+		dentry = d_obtain_alias_root(inode);
 		if (IS_ERR(dentry)) {
 			ret = PTR_ERR(dentry);
 			goto failed_dentry;
include/linux/dcache.h +1 -0
index bf72e9a..2973aab 100644
@@ -247,6 +247,7 @@ extern struct dentry * d_splice_alias(struct inode *, struct dentry *);
 extern struct dentry * d_add_ci(struct dentry *, struct inode *, struct qstr *);
 extern struct dentry *d_find_any_alias(struct inode *inode);
 extern struct dentry * d_obtain_alias(struct inode *);
+extern struct dentry * d_obtain_alias_root(struct inode *);
 extern void shrink_dcache_sb(struct super_block *);
 extern void shrink_dcache_parent(struct dentry *);
 extern void shrink_dcache_for_umount(struct super_block *);
#15 lore raw
In reply to J. Bruce Fields

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Quoted 43 lines · J. Bruce Fields
Quoted 29 lines · Eric W. Biederman
Quoted 27 lines · J. Bruce Fields

Yesterday you passed on a report of this printk from nfsdfh.c firing:

printk("nfsd: find_fh_dentry returned a DISCONNECTED directory:
%pd2\n", dentry);

I think the dentry probably comes from the FILEID_ROOT case of:

if (fileid_type == FILEID_ROOT) dentry = dget(exp->ex_path.dentry); else { dentry = exportfs_decode_fh(exp->ex_path.mnt, fid, data_left, fileid_type, nfsd_acceptable, exp); }

In that case the dentry was found using ordinary filesystem lookups, so doesn't go through the same DISCONNECTED-clearing logic as in the case of lookups by filehandle.

Probably they have an export root that's not a filesystem root, and the lookups happened in the right order?

I suspect that's fine, and that the printk is just stupid, but maybe we should clear DISCONNECTED when possible on normal lookups. The following is my attempt, though I'm not sure if d_alloc is the right place to do this. In any case it might help confirm this is what's happening.

Quoted 6 lines
Quoted 2 lines

So if you pass along this patch to the person who was seeing that printk I'd be interested in the results.

I have been reading through the dentry code for other reasons and your patch definitely won't change anything. __d_alloc sets d_flags = 0. Therefore d_alloc always returns with d_flags == 0.

You're right, of course. I wasn't thinking straight.

So the only dentries with DISCONNECTED set are those created with d_obtain_alias, which is normally only used when you're looking up by filehandle.

Quoted 6 lines

Except btrfs has a weird use in get_default_root(). So maybe they were running into the dentry that created?

So btrfs should probably be using something else, I'm not sure what.

Urgh sorry for the wild goose chase Bruce, I'll fix this up,

Josef
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBAgAGBQJS/iVdAAoJEANb+wAKly3BfE0P/2U3wV18Dr4IRzZN/sOIlv4s 4ni9wPbGdOocySmFTvqAFDxixRzbfW+5JCigbFuTZFHpuJTffnCBwoXJVoIrmnWw dfStqJBOG/yrAKNK+2gV1W8CYdNwKD5/QYt/9MNj+Txn2PQXiNJIPPaiKiiJfyQz Ht+ABvyTDb00YcsAss5ldbYDVCvZ6UzfWZZ9j2lfDbAL4zsF2p3YUcANTiNKPe01 5crJ1//5bvlFNFAgLqJNJSq1JlHdbD3ANr86Cv1ikDercmvyIeuUY1Yk5kpH/+p+ BqQATQ3qvJ9C6U/sbZyBaNa86Jt7NyxgD4HgmyeTQHtk7H42Jv2HSrnPy31MYF44 VksMTkaSNFrR6RUI88X82JHuax/bhT8MBjBuXbrD50AnFXzZuUT70lcUGDfIoUni +BgHzaCa06DHgzZCEmnP0/KEy6f5Zty9eEMMgqL30WtZGsKPZA98RsobuV3IVw67 dW6bnmdx9kIZRhyUM/hh8Cqyep7Y54lxBzUtG7KMYZDNP9tZsjCtWxWIq+j9htYW RJ6KPs9rnjdJoRp7Ca8jGaTbuOa8OmhS62C9sFmmiXbuh40KNbSKea+rO63roUs5 5MwqYjqv8OvHNukxtRQyX2FxZwJojx8IUXf18urJnzvffCOAw2tg47GRkbRKVE51 WqW/C6HcSxxOOUr92Lup
=ZYeX
-----END PGP SIGNATURE-----

#16 lore raw
In reply to J. Bruce Fields

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Quoted 43 lines · J. Bruce Fields
Quoted 29 lines · Eric W. Biederman
Quoted 27 lines · J. Bruce Fields

Yesterday you passed on a report of this printk from nfsdfh.c firing:

printk("nfsd: find_fh_dentry returned a DISCONNECTED directory:
%pd2\n", dentry);

I think the dentry probably comes from the FILEID_ROOT case of:

if (fileid_type == FILEID_ROOT) dentry = dget(exp->ex_path.dentry); else { dentry = exportfs_decode_fh(exp->ex_path.mnt, fid, data_left, fileid_type, nfsd_acceptable, exp); }

In that case the dentry was found using ordinary filesystem lookups, so doesn't go through the same DISCONNECTED-clearing logic as in the case of lookups by filehandle.

Probably they have an export root that's not a filesystem root, and the lookups happened in the right order?

I suspect that's fine, and that the printk is just stupid, but maybe we should clear DISCONNECTED when possible on normal lookups. The following is my attempt, though I'm not sure if d_alloc is the right place to do this. In any case it might help confirm this is what's happening.

Quoted 6 lines
Quoted 2 lines

So if you pass along this patch to the person who was seeing that printk I'd be interested in the results.

I have been reading through the dentry code for other reasons and your patch definitely won't change anything. __d_alloc sets d_flags = 0. Therefore d_alloc always returns with d_flags == 0.

You're right, of course. I wasn't thinking straight.

So the only dentries with DISCONNECTED set are those created with d_obtain_alias, which is normally only used when you're looking up by filehandle.

Quoted 6 lines

Except btrfs has a weird use in get_default_root(). So maybe they were running into the dentry that created?

So btrfs should probably be using something else, I'm not sure what.

Course now that I look at it I'm not sure what to do. We know the location of the inode we want to use as our root dentry, but we could already have a dentry for this in cache which is why we use d_obtain_alias(). If we use d_make_root() and then wander into the directory later we end up with inodes left over. So should we just build a path to the location and do the path lookup stuff so we have a valid dentry? Thanks,

Josef
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBAgAGBQJS/jKfAAoJEANb+wAKly3B1q4P/1wgkd/w6QI6A3c40Qw5Oyfk gTOFcSrCnGpd59wOzCu3B8fKuVygBweJ0vwYMlnNgGjCzqidTt37LCD+rRb1iz0w BZVSvWS5v8qeaW+qQJJSQH83NX8v+L6zc/5D1stwPIfctXf0WvSijmacV2/BGRgJ PqZVXd4yQhd24cqonXUzqNswiFOmPUPs8xrSz3NaR4GcRFVMLuRuubpf/pWkotmI mldm8p+SCrDiQHgXEYYHKW1rHKjxKU4GIc9+Dsx0Jjnv26ITiooM8pY2tWW8w98S CQv54VQGzzAmyNHQxmv/3sxPg9K4CL+8t7Z4nLQ043K9U9itd+YU/v/vvnzAXvob KX5amGZc0cWyYHu9e/9i4HXOJmrmEUdwl5megeI47V/LStzkRUggeha+tRzm4xQW +nT4dpueUK1ezDDPvJdOF9id9WNWaGwxKqg6Mqj9+tsnmOwYll2zaNuoOVUZacWY Qmv+1JS6gdE8/qLTVjp9I4nTdGN9Cr/sE1EwBVM/GFbCQzAxe4m4uKnXtJ28+Noo LOg/u27GSssO6oUFA8Mj6dnTEOKtm5QOMDD+4lxRSyWuAHCOvNg6I4SRykHb1o1I OexIp6DK1Siofj6mAFGG5AZunXYH8uR8aOH4Ctva04vC1D6TlZsI5ycgE6V51NUk Rv4ICvuwjuu3nZRLLKve
=KK7m
-----END PGP SIGNATURE-----

#17 lore raw
Quoted 52 lines · Josef Bacik

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Quoted 46 lines · J. Bruce Fields
Quoted 29 lines · Eric W. Biederman
Quoted 27 lines · J. Bruce Fields

Yesterday you passed on a report of this printk from nfsdfh.c firing:

printk("nfsd: find_fh_dentry returned a DISCONNECTED directory:
%pd2\n", dentry);

I think the dentry probably comes from the FILEID_ROOT case of:

if (fileid_type == FILEID_ROOT) dentry = dget(exp->ex_path.dentry); else { dentry = exportfs_decode_fh(exp->ex_path.mnt, fid, data_left, fileid_type, nfsd_acceptable, exp); }

In that case the dentry was found using ordinary filesystem lookups, so doesn't go through the same DISCONNECTED-clearing logic as in the case of lookups by filehandle.

Probably they have an export root that's not a filesystem root, and the lookups happened in the right order?

I suspect that's fine, and that the printk is just stupid, but maybe we should clear DISCONNECTED when possible on normal lookups. The following is my attempt, though I'm not sure if d_alloc is the right place to do this. In any case it might help confirm this is what's happening.

Quoted 6 lines
Quoted 2 lines

So if you pass along this patch to the person who was seeing that printk I'd be interested in the results.

I have been reading through the dentry code for other reasons and your patch definitely won't change anything. __d_alloc sets d_flags = 0. Therefore d_alloc always returns with d_flags == 0.

You're right, of course. I wasn't thinking straight.

So the only dentries with DISCONNECTED set are those created with d_obtain_alias, which is normally only used when you're looking up by filehandle.

Except btrfs has a weird use in get_default_root(). So maybe they were running into the dentry that created?

Quoted 8 lines
Quoted 3 lines

So btrfs should probably be using something else, I'm not sure what.

Course now that I look at it I'm not sure what to do. We know the location of the inode we want to use as our root dentry, but we could already have a dentry for this in cache which is why we use d_obtain_alias().

That's exactly the sort of situation that d_obtain_alias() is meant to help you with.

It's just that you don't want DCACHE_DISCONNECTED set. I'm not even sure if that causes any actual bug, but it seems confusing at least.

Maybe it would be worth making a d_obtain_alias_for_root() or something for yours and nfs's use.

--b.

Quoted 24 lines

If we use d_make_root() and then wander into the
directory later we end up with inodes left over. So should we just build a path to the location and do the path lookup stuff so we have a valid dentry? Thanks,

Josef
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBAgAGBQJS/jKfAAoJEANb+wAKly3B1q4P/1wgkd/w6QI6A3c40Qw5Oyfk gTOFcSrCnGpd59wOzCu3B8fKuVygBweJ0vwYMlnNgGjCzqidTt37LCD+rRb1iz0w BZVSvWS5v8qeaW+qQJJSQH83NX8v+L6zc/5D1stwPIfctXf0WvSijmacV2/BGRgJ PqZVXd4yQhd24cqonXUzqNswiFOmPUPs8xrSz3NaR4GcRFVMLuRuubpf/pWkotmI mldm8p+SCrDiQHgXEYYHKW1rHKjxKU4GIc9+Dsx0Jjnv26ITiooM8pY2tWW8w98S CQv54VQGzzAmyNHQxmv/3sxPg9K4CL+8t7Z4nLQ043K9U9itd+YU/v/vvnzAXvob KX5amGZc0cWyYHu9e/9i4HXOJmrmEUdwl5megeI47V/LStzkRUggeha+tRzm4xQW +nT4dpueUK1ezDDPvJdOF9id9WNWaGwxKqg6Mqj9+tsnmOwYll2zaNuoOVUZacWY Qmv+1JS6gdE8/qLTVjp9I4nTdGN9Cr/sE1EwBVM/GFbCQzAxe4m4uKnXtJ28+Noo LOg/u27GSssO6oUFA8Mj6dnTEOKtm5QOMDD+4lxRSyWuAHCOvNg6I4SRykHb1o1I OexIp6DK1Siofj6mAFGG5AZunXYH8uR8aOH4Ctva04vC1D6TlZsI5ycgE6V51NUk Rv4ICvuwjuu3nZRLLKve
=KK7m
-----END PGP SIGNATURE-----