Skip to contentSkip to conversation

nfs-utils crossmnt/fsid bug

DISCUSSION J. Bruce Fields 3 participants 9 messages Sep 5, 2007 – Dec 11, 2007 2 messages not in the archive

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

With an exports entry like

/exports *.citi.umich.edu(rw,crossmnt,fsid=0)

and some other stuff mounted under /exports, I get an odd situation where listing /exports/submnt/ shows me the contents of /exports/.

I looks to me like what's happening is that the new code to automatically export filesystems under crossmnt exports is exporting those directories with the fsid option turned on, with the result that the root of each of those subdirectories ends up with the same filehandle as the root of /exports, and the hilarity ensues. But I haven't tried a patch yet.

Does that make sense? I assume we should just clear the fsid flag on submounts, as it's never going to make sense, and leave the logic otherwise intact.

--b.

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

#1 lore raw
Quoted 14 lines · On Wednesday September 5, [email protected] wrote:

With an exports entry like

/exports *.citi.umich.edu(rw,crossmnt,fsid=0)

and some other stuff mounted under /exports, I get an odd situation where listing /exports/submnt/ shows me the contents of /exports/.

I looks to me like what's happening is that the new code to automatically export filesystems under crossmnt exports is exporting those directories with the fsid option turned on, with the result that the root of each of those subdirectories ends up with the same filehandle as the root of /exports, and the hilarity ensues. But I haven't tried a patch yet.

Quoted 5 lines

Does that make sense? I assume we should just clear the fsid flag on submounts, as it's never going to make sense, and leave the logic otherwise intact.

--b.

Thanks for noticing and reporting this.

Does this (untested, but it compiles) patch seem right?

NeilBrown

--------
When exported a filesystems with option inherited (by the crossmnt option) from a higherlevel filesystem, ignore filesystem specific options like FSID and explicit UUID.

utils/mountd/cache.c +12 -3
index fd317cd..ff56b9f 100644
@@ -565,20 +565,29 @@ static int dump_to_cache(FILE *f, char *domain, char *path, struct exportent *ex
 	qword_print(f, path);
 	qword_printint(f, time(0)+30*60);
 	if (exp) {
-		qword_printint(f, exp->e_flags);
+		int different_fs = 0;
+		struct stat stb1, stb2;
+		if (stat(path, &stb1) == 0 &&
+		    stat(exp->e_path, &stb2) == 0 &&
+		    stb1.st_dev != stb2.st_dev)
+			different_fs = 1;
+		if (different_fs)
+			qword_printint(f, exp->e_flags & ~NFSEXP_FSID);
+		else
+			qword_printint(f, exp->e_flags);
 		qword_printint(f, exp->e_anonuid);
 		qword_printint(f, exp->e_anongid);
 		qword_printint(f, exp->e_fsid);
 		write_fsloc(f, exp, path);
 		write_secinfo(f, exp);
 #if USE_BLKID
- 		if (exp->e_uuid == NULL) {
+ 		if (exp->e_uuid == NULL || different_fs) {
  			char u[16];
  			if (get_uuid(path, NULL, 16, u)) {
  				qword_print(f, "uuid");
  				qword_printhex(f, u, 16);
  			}
- 		} else if (exp->e_uuid) {
+ 		} else {
  			qword_print(f, "uuid");
  			qword_printhex(f, exp->e_uuid, 16);
  		}

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
_______________________________________________
Please note that [email protected] is being discontinued. Please subscribe to [email protected] instead.
http://vger.kernel.org/vger-lists.html#linux-nfs

A message that is not in the archive (sent off-list or not yet received): 18246.16373.177320.570040-wvvUuzkyo1EYVZTmpyfIwg@public.gmane.org

#3 lore raw
Quoted 23 lines · Neil Brown
Quoted 14 lines · On Wednesday September 5, [email protected] wrote:

With an exports entry like

/exports *.citi.umich.edu(rw,crossmnt,fsid=0)

and some other stuff mounted under /exports, I get an odd situation where listing /exports/submnt/ shows me the contents of /exports/.

I looks to me like what's happening is that the new code to automatically export filesystems under crossmnt exports is exporting those directories with the fsid option turned on, with the result that the root of each of those subdirectories ends up with the same filehandle as the root of /exports, and the hilarity ensues. But I haven't tried a patch yet.

Quoted 5 lines

Does that make sense? I assume we should just clear the fsid flag on submounts, as it's never going to make sense, and leave the logic otherwise intact.

--b.

Thanks for noticing and reporting this.

Quoted 8 lines

Does this (untested, but it compiles) patch seem right?

NeilBrown

--------
When exported a filesystems with option inherited (by the crossmnt option) from a higherlevel filesystem, ignore filesystem specific options like FSID and explicit UUID.

Looks right to me. I'm confused about uuid's--how does the non-explicit UUID case work?

--b.

Quoted 30 lines
utils/mountd/cache.c +11 -2
index fd317cd..ff56b9f 100644
@@ -565,20 +565,29 @@ static int dump_to_cache(FILE *f, char *domain, char *path, struct exportent *ex
 	qword_print(f, path);
 	qword_printint(f, time(0)+30*60);
 	if (exp) {
-		qword_printint(f, exp->e_flags);
+		int different_fs = 0;
+		struct stat stb1, stb2;
+		if (stat(path, &stb1) == 0 &&
+		    stat(exp->e_path, &stb2) == 0 &&
+		    stb1.st_dev != stb2.st_dev)
+			different_fs = 1;
+		if (different_fs)
+			qword_printint(f, exp->e_flags & ~NFSEXP_FSID);
+		else
+			qword_printint(f, exp->e_flags);
 		qword_printint(f, exp->e_anonuid);
 		qword_printint(f, exp->e_anongid);
 		qword_printint(f, exp->e_fsid);
 		write_fsloc(f, exp, path);
 		write_secinfo(f, exp);
 #if USE_BLKID
- 		if (exp->e_uuid == NULL) {
+ 		if (exp->e_uuid == NULL || different_fs) {

utils/mountd/cache.c static int dump_to_cache(FILE *f, char *domain, char *path, struct exportent *ex · line 593

Quoted 10 lines
utils/mountd/cache.c +1 -1
  			char u[16];
  			if (get_uuid(path, NULL, 16, u)) {
  				qword_print(f, "uuid");
  				qword_printhex(f, u, 16);
  			}
- 		} else if (exp->e_uuid) {
+ 		} else {
  			qword_print(f, "uuid");
  			qword_printhex(f, exp->e_uuid, 16);
  		}

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
_______________________________________________
Please note that [email protected] is being discontinued. Please subscribe to [email protected] instead.
http://vger.kernel.org/vger-lists.html#linux-nfs

#4 lore raw
Quoted 6 lines · On Monday November 26, [email protected] wrote:
Quoted 3 lines

When exported a filesystems with option inherited (by the crossmnt option) from a higherlevel filesystem, ignore filesystem specific options like FSID and explicit UUID.

Looks right to me. I'm confused about uuid's--how does the non-explicit UUID case work?

Thanks, though on reflection I don't think we need to do the 'stat's. Just checking the path names should be sufficient, but I will double check.

non-explicit UUIDs are derived using libblkid, typically a UUID out of the filesystem that is being exported.
The only problem is that it appears that libblkid isn't as efficient as one might like and if you have hundreds of mounts, mountd can slow down. I still need to look into that.

NeilBrown

-------------------------------------------------------------------------
SF.Net email is sponsored by: The Future of Linux Business White Paper from Novell. From the desktop to the data center, Linux is going mainstream. Let it simplify your IT future.
http://altfarm.mediaplex.com/ad/ck/8857-50307-18918-4
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
_______________________________________________
Please note that [email protected] is being discontinued. Please subscribe to [email protected] instead.
http://vger.kernel.org/vger-lists.html#linux-nfs

#5 lore raw
In reply to a message not in the archive
Quoted 19 lines · Neil Brown
Quoted 18 lines · On Wednesday September 5, [email protected] wrote:

With an exports entry like

/exports *.citi.umich.edu(rw,crossmnt,fsid=0)

and some other stuff mounted under /exports, I get an odd situation where listing /exports/submnt/ shows me the contents of /exports/.

I looks to me like what's happening is that the new code to automatically export filesystems under crossmnt exports is exporting those directories with the fsid option turned on, with the result that the root of each of those subdirectories ends up with the same filehandle as the root of /exports, and the hilarity ensues. But I haven't tried a patch yet.

Does that make sense? I assume we should just clear the fsid flag on submounts, as it's never going to make sense, and leave the logic otherwise intact.

Quoted 5 lines
Quoted 1 line

--b.

Thanks for noticing and reporting this.

Does this (untested, but it compiles) patch seem right?

In testing security negotiation, I ran into this problem. I've tried this patch and it doesn't seem to be working right. My exports file is:

/export/home    gss/krb5i(nohide,insecure,no_subtree_check,no_root_squash,async,rw)
/export/home    gss/krb5p(nohide,insecure,no_subtree_check,no_root_squash,async,rw)
/export         gss/krb5(fsid=0,insecure,no_subtree_check,no_root_squash,async,rw,crossmnt)
/export         gss/krb5i(fsid=0,insecure,no_subtree_check,no_root_squash,async,rw,crossmnt)
Quoted 1 line

From the client I execute:

mount -overs=4,sec=krb5 elm3a19.beaverton.ibm.com:/ /mnt ls -l /mnt/home

I added some debug logging, and I see the different_fs path being taken on the /export/home. I'm wondering, could there be a kernel part to this also?

Thanks

Frank Filz

-------------------------------------------------------------------------
SF.Net email is sponsored by: The Future of Linux Business White Paper from Novell. From the desktop to the data center, Linux is going mainstream. Let it simplify your IT future.
http://altfarm.mediaplex.com/ad/ck/8857-50307-18918-4
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
_______________________________________________
Please note that [email protected] is being discontinued. Please subscribe to [email protected] instead.
http://vger.kernel.org/vger-lists.html#linux-nfs

#6 lore raw
Quoted 10 lines · On Monday December 3, [email protected] wrote:
Quoted 1 line · Neil Brown

Does this (untested, but it compiles) patch seem right?

In testing security negotiation, I ran into this problem. I've tried this patch and it doesn't seem to be working right. My exports file is:

/export/home    gss/krb5i(nohide,insecure,no_subtree_check,no_root_squash,async,rw)
/export/home    gss/krb5p(nohide,insecure,no_subtree_check,no_root_squash,async,rw)
/export         gss/krb5(fsid=0,insecure,no_subtree_check,no_root_squash,async,rw,crossmnt)
/export         gss/krb5i(fsid=0,insecure,no_subtree_check,no_root_squash,async,rw,crossmnt)

You don't need the patch for this exports file. It is only relevant when the underlying filesystem is not explicitly exported, so the export flags a deduced implicitly.

So - this is a separate issue.

Quoted 5 lines

From the client I execute:

mount -overs=4,sec=krb5 elm3a19.beaverton.ibm.com:/ /mnt ls -l /mnt/home

... and what happens?

NeilBrown

-------------------------------------------------------------------------
SF.Net email is sponsored by: The Future of Linux Business White Paper from Novell. From the desktop to the data center, Linux is going mainstream. Let it simplify your IT future.
http://altfarm.mediaplex.com/ad/ck/8857-50307-18918-4
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
_______________________________________________
Please note that [email protected] is being discontinued. Please subscribe to [email protected] instead.
http://vger.kernel.org/vger-lists.html#linux-nfs

A message that is not in the archive (sent off-list or not yet received): 18260.43206.943331.889058-wvvUuzkyo1EYVZTmpyfIwg@public.gmane.org

#8 lore raw
Quoted 12 lines · Neil Brown
Quoted 10 lines · On Monday December 3, [email protected] wrote:
Quoted 1 line · Neil Brown

Does this (untested, but it compiles) patch seem right?

In testing security negotiation, I ran into this problem. I've tried this patch and it doesn't seem to be working right. My exports file is:

/export/home    gss/krb5i(nohide,insecure,no_subtree_check,no_root_squash,async,rw)
/export/home    gss/krb5p(nohide,insecure,no_subtree_check,no_root_squash,async,rw)
/export         gss/krb5(fsid=0,insecure,no_subtree_check,no_root_squash,async,rw,crossmnt)
/export         gss/krb5i(fsid=0,insecure,no_subtree_check,no_root_squash,async,rw,crossmnt)
Quoted 5 lines

You don't need the patch for this exports file. It is only relevant when the underlying filesystem is not explicitly exported, so the export flags a deduced implicitly.

So - this is a separate issue.

Ah, ok.

Quoted 6 lines
Quoted 4 lines

From the client I execute:

mount -overs=4,sec=krb5 elm3a19.beaverton.ibm.com:/ /mnt ls -l /mnt/home

... and what happens?

This is what I get:

# mount -overs=4,sec=krb5 elm3a19.beaverton.ibm.com:/ /mnt
# ls -l /mnt
total 32
-rw-r--r--    1 nobody   nobody           32 Nov  7 15:00 foo
drwxr-xr-x   19 nobody   nobody         4096 Dec  3 14:40 home
drwxr-xr-x    2 nobody   nobody         4096 Nov  2 17:51 homw2
# ls -l /mnt/home
total 32
-rw-r--r--    1 nobody   nobody           32 Nov  7 15:00 foo
drwxr-xr-x   19 nobody   nobody         4096 Dec  3 14:40 home
drwxr-xr-x    2 nobody   nobody         4096 Nov  2 17:51 homw2
# ls -l /mnt/homw2  
total 0
-rw-r--r--    1 nobody   nobody            0 Nov  2 17:51 foo
# umount /mnt

This is from an AIX client, however, with the new format exports with the sec= export option, the AIX client works just fine (given the patch I submitted previously).

Frank Filz

-------------------------------------------------------------------------
SF.Net email is sponsored by: The Future of Linux Business White Paper from Novell. From the desktop to the data center, Linux is going mainstream. Let it simplify your IT future.
http://altfarm.mediaplex.com/ad/ck/8857-50307-18918-4
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
_______________________________________________
Please note that [email protected] is being discontinued. Please subscribe to [email protected] instead.
http://vger.kernel.org/vger-lists.html#linux-nfs

#9 lore raw

I've done some more exploration, I think the exports are getting set wrong. First, here is the working case with the new exports format (cut from an annotated /var/log/messages):

START====================================================== ======= cat /etc/exports
/export *(sec=krb5,fsid=0,insecure,no_subtree_check,no_root_squash,async)
/export/home *(sec=krb5,nohide,insecure,no_subtree_check,no_root_squash,async)
======= mount -overs=4,sec=krb5 elm3a19.beaverton.ibm.com:/ /mnt

Dec  7 10:47:20 elm3a19 mountd[4691]: cache_export_ent domain=*, exp path=/export, path=/export 
Dec  7 10:47:20 elm3a19 mountd[4691]: dump_to_cache path=/export, exp path=/export 
======= ls -l /mnt
Dec  7 10:47:22 elm3a19 mountd[4691]: nfsd_export domain=*, path=/export/home, exp path=/export/home 
Dec  7 10:47:22 elm3a19 mountd[4691]: dump_to_cache path=/export/home, exp path=/export/home 
Dec  7 10:47:23 elm3a19 mountd[4691]: cache_export_ent domain=*, exp path=/export/home, path=/export/home 
Dec  7 10:47:23 elm3a19 mountd[4691]: dump_to_cache path=/export/home, exp path=/export/home 
======= umount /mnt
==========================
Dec  7 10:47:28 elm3a19 rpc.idmapd[2471]: nfsdcb: authbuf=gss/krb5 authtype=user
Dec  7 10:47:28 elm3a19 rpc.idmapd[2471]: nfsdcb: authbuf=gss/krb5 authtype=group

======= ls -l /mnt/home
======= umount /mnt END=======================================================

And then here is the non-working form with the old exports format:

START====================================================== ======= cat /etc/exports
/export/home gss/krb5i(nohide,insecure,no_subtree_check,no_root_squash,async,rw)
/export/home gss/krb5p(nohide,insecure,no_subtree_check,no_root_squash,async,rw)
/export gss/krb5(fsid=0,insecure,no_subtree_check,no_root_squash,async,rw,crossmnt)
/export gss/krb5i(fsid=0,insecure,no_subtree_check,no_root_squash,async,rw,crossmnt)

======= mount -overs=4,sec=krb5 elm3a19.beaverton.ibm.com:/ /mnt

Dec  7 10:46:53 elm3a19 mountd[4691]: cache_export_ent domain=gss/krb5, exp path=/export, path=/export 
Dec  7 10:46:53 elm3a19 mountd[4691]: dump_to_cache path=/export, exp path=/export 
======= ls -l /mnt
Dec  7 10:46:55 elm3a19 mountd[4691]: nfsd_export domain=gss/krb5, path=/export/home, exp path=/export 
Dec  7 10:46:55 elm3a19 mountd[4691]: dump_to_cache path=/export/home, exp path=/export 
Dec  7 10:46:56 elm3a19 mountd[4691]: cache_export_ent domain=gss/krb5, exp path=/export, path=/export/home 
Dec  7 10:46:56 elm3a19 mountd[4691]: dump_to_cache path=/export, exp path=/export 
Dec  7 10:46:56 elm3a19 mountd[4691]: dump_to_cache path=/export/home, exp path=/export 

======= umount /mnt
==========================
======= ls -l /mnt/home
======= umount /mnt END=======================================================

I see that the way mountd caches the /exports/home export is different between the two cases, with the non-working old exports format case caching a path of /export/home but using the /export export.

p.s. I am using nfs-utils-1.1.1 for this, and applied the previously mentioned crossmount patch.

Oh, and just for completeness, here is implicit exporting via crossmnt, with the same result:

START====================================================== ======= cat /etc/exports
/export gss/krb5(fsid=0,insecure,no_subtree_check,no_root_squash,async,rw,crossmnt) /export gss/krb5i(fsid=0,insecure,no_subtree_check,no_root_squash,async,rw,crossmnt)

======= mount -overs=4,sec=krb5 elm3a19.beaverton.ibm.com:/ /mnt

Dec  7 11:00:22 elm3a19 mountd[4691]: cache_export_ent domain=gss/krb5, exp path=/export, path=/export 
Dec  7 11:00:22 elm3a19 mountd[4691]: dump_to_cache path=/export, exp path=/export 
Dec  7 11:00:22 elm3a19 rpc.idmapd[2471]: nfsdcb: authbuf=gss/krb5 authtype=user
Dec  7 11:00:22 elm3a19 rpc.idmapd[2471]: nfsdcb: authbuf=gss/krb5 authtype=group
======= ls -l /mnt
Dec  7 11:00:24 elm3a19 mountd[4691]: nfsd_export domain=gss/krb5, path=/export/home, exp path=/export 
Dec  7 11:00:24 elm3a19 mountd[4691]: dump_to_cache path=/export/home, exp path=/export 
Dec  7 11:00:25 elm3a19 mountd[4691]: cache_export_ent domain=gss/krb5, exp path=/export, path=/export/home 
Dec  7 11:00:25 elm3a19 mountd[4691]: dump_to_cache path=/export, exp path=/export 
Dec  7 11:00:25 elm3a19 mountd[4691]: dump_to_cache path=/export/home, exp path=/export 

======= umount /mnt
==========================
======= ls -l /mnt/home
======= umount /mnt END=======================================================

Looks the same as with the explicit exporting.

Any suggestions or thoughts?

Thanks

Frank Filz

-------------------------------------------------------------------------
SF.Net email is sponsored by:
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
_______________________________________________
Please note that [email protected] is being discontinued. Please subscribe to [email protected] instead.
http://vger.kernel.org/vger-lists.html#linux-nfs

#10 lore raw
In reply to Frank Filz

Ok, I've looked at this issue some more. The crossmnt and nohide options work correctly so long as security negotiation isn't required to cross the mount point (whether the mount command provides the correct set of security flavors or not). If the crossmnt export option is provided in this case (whether or not nohide is specified on the child export), an ls of the /mnt/home ends up with the results from an ls of /mnt. If the crossmnt option is not specified on the parent but nohide is specified on the child, an ls -l of /mnt results in slightly different contents while an ls -l of /mnt/home results in the contents of the hidden directory on the parent export.

These behaviors seem undesirable, but perhaps not worth eliminating since they do represent something unsupported by the old format which is deprecated in favor of the new format where everything works just fine.

Frank Filz

-------------------------------------------------------------------------
SF.Net email is sponsored by:
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
_______________________________________________
Please note that [email protected] is being discontinued. Please subscribe to [email protected] instead.
http://vger.kernel.org/vger-lists.html#linux-nfs