NFS and non-blkdev-backed FS?

When I NFS-export UBIFS, I see the following warning:
exportfs: Warning: /mnt/ubifs requires fsid= for NFS export

And the client won't mount UBIFS:
mount.nfs: access denied by server while mounting gollum:/mnt/ubifs

The problem is solved by adding 'fsid=5' to /etc/exports. And BTW, exactly the same happens to JFFS2.

This looks strange to me, probably because I misunderstand something.

AFAIU, NFS uses major/minor numbers of block device to create FSID and uses it as part of file handle. And the idea behind this is that if the server rebooted unexpectedly, and then exported a _different_ file-system to the same path, the client would notice this because the FSID part of file handle would change.

Well, I do not see how it solves the problem if the hard drive has been changed, though.

Since UBIFS (and JFFS2) does not work on top of block devices, NFS cannot compose FSID from major/minor numbers and exportfs yells.

But I do not like the idea of adding explicit fsid=5 to /etc/exports, because it does not seem to solve the above problem, surely?

UBIFS (as e.g. XFS) has a 16-byte UUID in superblock which is supposed to uniquely identify the file-system. I'd like to use this UUID as NFS FSID instead, however, I failed to find how to do this. Any hints how could I do this?

Thanks.

Best Regards,
Artem Bityutskiy (Артём Битюцкий)
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

You would need a new export operations method for initializating the the fsid part of the file handle. This method would be calked from fh_compose or rather the new fsid generating factored out from that.

Note that it would be good to use this for block bases filesystems, too so there should be some way to override the default fsids for those, too.

By adding a 'get_fsid()' method to the export_ops structure...

David Woodhouse                            Open Source Technology Centre
[email protected]                              Intel Corporation

If the uuid can be found by libuuid, then it will automatically be used as the default identifier by rpc.mount (man 5 exports). There should be no need for a new export_ops method.

Cheers
Trond

We'll need it anyway, for exports not starting at the root of the volume, and we'll need a method to be called when exporting a filesystem to reject it if the filehandle couldn't fit into the space available for the given nfs protocol version.

So far we've been using the pair (fsid:fileid) to label the export inode. Why does using a UUID as the fsid change that?

As for the problem of dealing with a uuid-based filehandle not fitting into the NFSv2 maximum of 32-bytes, why is that any different from the general problem of defining alternative globally unique identifiers, and why do we now think we can do that better in the kernel rather than relying on the previous strategy of defining such identifiers in userland?

That approach is painful -- it's simple enough to make the export work, and it returns an appropriate root fh -- but then falls over immediately when the client gives that fh back to us in an FSINFO request and userspace mountd doesn't understand it. A hack to prepopulate the svc_export_cache too 'fixes' it, but won't survive reboots.

The problem with libuuid is that it assumes that there's a block device, and a 1:1 mapping between file system and block device. That often isn't true.

However, there _is_ a f_fsid field in struct statfs64, which we could happily use if libuuid fails.

See git.infradead.org/users/dwmw2/nfs-utils.git -- if the file system fills in f_fsid, that should be all that's needed.

David Woodhouse                            Open Source Technology Centre
[email protected]                              Intel Corporation

Fix a couple of bugs which show up if you try to explicitly set a 16-byte UUID when exporting a file system. First, exportfs cuts the first two bytes off the UUID and writes something invalid to etab. Second, mountd writes the _ascii_ form of the UUID to the kernel, instead of converting it to hex.

---
 support/nfs/exports.c |    2 +-
 utils/mountd/cache.c  |    4 +++-
 2 files changed, 4 insertions(+), 2 deletions(-)

index 525e5b1..334c08e 100644
@@ -595,7 +595,7 @@ bad_option:
 				if (opt[5]!='\0' && *oe == '\0') 
 					ep->e_flags |= NFSEXP_FSID;
 				else if (valid_uuid(opt+5))
-					ep->e_uuid = strdup(opt+7);
+					ep->e_uuid = strdup(opt+5);
 				else {
 					xlog(L_ERROR, "%s: %d: bad fsid \"%s\"\n",
 					     flname, flline, opt);	

index f555dcc..cd377ce 100644
@@ -590,8 +590,10 @@ static int dump_to_cache(FILE *f, char *domain, char *path, struct exportent *ex
  				qword_printhex(f, u, 16);
  			}
  		} else {
+			char u[16];
+			get_uuid(NULL, exp->e_uuid, 16, u);
  			qword_print(f, "uuid");
- 			qword_printhex(f, exp->e_uuid, 16);
+ 			qword_printhex(f, u, 16);
  		}
 #endif
 	}

1.5.5.1


-- 
David Woodhouse                            Open Source Technology Centre
[email protected]                              Intel Corporation

---
 utils/mountd/cache.c |   74 +++++++++++++++++++++++++++-----------------------
 1 files changed, 40 insertions(+), 34 deletions(-)

index cd377ce..0136eca 100644
@@ -14,6 +14,7 @@
 #include <sys/types.h>
 #include <sys/select.h>
 #include <sys/stat.h>
+#include <sys/vfs.h>
 #include <time.h>
 #include <netinet/in.h>
 #include <arpa/inet.h>

@@ -165,6 +166,41 @@ void auth_unix_gid(FILE *f)
 }
 
 #if USE_BLKID
+static const char *get_uuid_blkdev(char *path)
+{
+	static blkid_cache cache = NULL;
+	struct stat stb;
+	char *devname;
+	blkid_tag_iterate iter;
+	blkid_dev dev;
+	const char *type;
+	const char *val = NULL;
+
+	if (cache == NULL)
+		blkid_get_cache(&cache, NULL);
+
+	if (stat(path, &stb) != 0)
+		return NULL;
+	devname = blkid_devno_to_devname(stb.st_dev);
+	if (!devname)
+		return NULL;
+	dev = blkid_get_dev(cache, devname, BLKID_DEV_NORMAL);
+	free(devname);
+	if (!dev)
+		return NULL;
+	iter = blkid_tag_iterate_begin(dev);
+	if (!iter)
+		return NULL;
+	while (blkid_tag_next(iter, &type, &val) == 0)
+		if (strcmp(type, "UUID") == 0)
+			break;
+	blkid_tag_iterate_end(iter);
+	return val;
+}
+#else
+#define get_uuid_blkdev(path) (NULL)
+#endif
+
 int get_uuid(char *path, char *uuid, int uuidlen, char *u)
 {
 	/* extract hex digits from uuidstr and compose a uuid

@@ -172,35 +208,12 @@ int get_uuid(char *path, char *uuid, int uuidlen, char *u)
 	 * a smaller uuid.  Then compare with uuid
 	 */
 	int i = 0;
-	const char *val;
+	const char *val = NULL;
+	char fsid_val[17];
 
 	if (path) {
-		static blkid_cache cache = NULL;
-		struct stat stb;
-		char *devname;
-		blkid_tag_iterate iter;
-		blkid_dev dev;
-		const char *type;
-		if (cache == NULL)
-			blkid_get_cache(&cache, NULL);
-
-		if (stat(path, &stb) != 0)
-			return 0;
-		devname = blkid_devno_to_devname(stb.st_dev);
-		if (!devname)
-			return 0;
-		dev = blkid_get_dev(cache, devname, BLKID_DEV_NORMAL);
-		free(devname);
-		if (!dev)
-			return 0;
-		iter = blkid_tag_iterate_begin(dev);
-		if (!iter)
-			return 0;
-		while (blkid_tag_next(iter, &type, &val) == 0)
-			if (strcmp(type, "UUID") == 0)
-				break;
-		blkid_tag_iterate_end(iter);
-		if (!type)
+		val = get_uuid_blkdev(path);
+		if (!val)
 			return 0;
 	} else {
 		val = uuid;

@@ -227,7 +240,6 @@ int get_uuid(char *path, char *uuid, int uuidlen, char *u)
 	}
 	return 1;
 }
-#endif
 
 /* Iterate through /etc/mtab, finding mountpoints
  * at or below a given path

@@ -437,7 +449,6 @@ void nfsd_fh(FILE *f)
 				if (!is_mountpoint(path))
 					continue;
 			check_uuid:
-#if USE_BLKID
 				if (exp->m_export.e_uuid)
 					get_uuid(NULL, exp->m_export.e_uuid,
 						 uuidlen, u);

@@ -448,9 +459,6 @@ void nfsd_fh(FILE *f)
 				if (memcmp(u, fhuuid, uuidlen) != 0)
 					continue;
 				break;
-#else
-				continue;
-#endif
 			}
 			if (use_ipaddr) {
 				if (he == NULL) {

@@ -582,7 +590,6 @@ static int dump_to_cache(FILE *f, char *domain, char *path, struct exportent *ex
 		qword_printint(f, exp->e_fsid);
 		write_fsloc(f, exp, path);
 		write_secinfo(f, exp);
-#if USE_BLKID
  		if (exp->e_uuid == NULL || different_fs) {
  			char u[16];
  			if (get_uuid(path, NULL, 16, u)) {

@@ -595,7 +602,6 @@ static int dump_to_cache(FILE *f, char *domain, char *path, struct exportent *ex
  			qword_print(f, "uuid");
  			qword_printhex(f, u, 16);
  		}
-#endif
 	}
 	return qword_eol(f);
 }

1.5.5.1


-- 
David Woodhouse                            Open Source Technology Centre
[email protected]                              Intel Corporation



--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

---
 utils/mountd/cache.c |   13 +++++++++++--
 1 files changed, 11 insertions(+), 2 deletions(-)

index 0136eca..f29b885 100644
@@ -213,8 +213,17 @@ int get_uuid(char *path, char *uuid, int uuidlen, char *u)
 
 	if (path) {
 		val = get_uuid_blkdev(path);
-		if (!val)
-			return 0;
+		if (!val) {
+			struct statfs64 st;
+
+			if (statfs64(path, &st))
+				return 0;
+			if (!st.f_fsid.__val[0] && !st.f_fsid.__val[1])
+				return 0;
+			snprintf(fsid_val, 17, "%08x%08x",
+				 st.f_fsid.__val[0], st.f_fsid.__val[1]);
+			val = fsid_val;
+		}
 	} else {
 		val = uuid;
 	}

1.5.5.1


-- 
David Woodhouse                            Open Source Technology Centre
[email protected]                              Intel Corporation



--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

---
 utils/exportfs/exportfs.c |   10 +++++++++-
 1 files changed, 9 insertions(+), 1 deletions(-)

index 18dfe5a..fec2571 100644
@@ -12,6 +12,7 @@
 #include <config.h>
 #endif
 
+#include <sys/vfs.h>
 #include <unistd.h>
 #include <stdlib.h>
 #include <string.h>

@@ -388,6 +389,8 @@ validate_export(nfs_export *exp)
 	 */
 	struct stat stb;
 	char *path = exp->m_export.e_path;
+	struct statfs64 stf;
+	int fs_has_fsid = 0;
 
 	if (stat(path, &stb) < 0) {
 		fprintf(stderr, "exportfs: Warning: %s does not exist\n",

@@ -403,7 +406,12 @@ validate_export(nfs_export *exp)
 	if (!can_test())
 		return;
 
-	if ((exp->m_export.e_flags & NFSEXP_FSID) || exp->m_export.e_uuid) {
+	if (!statfs64(path, &stf) &&
+	    (stf.f_fsid.__val[0] || stf.f_fsid.__val[1]))
+		fs_has_fsid = 1;
+
+	if ((exp->m_export.e_flags & NFSEXP_FSID) || exp->m_export.e_uuid ||
+	    fs_has_fsid) {
 		if ( !test_export(path, 1)) {
 			fprintf(stderr, "exportfs: Warning: %s does not "
 				"support NFS export.\n",

1.5.5.1



-- 
David Woodhouse                            Open Source Technology Centre
[email protected]                              Intel Corporation



--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

After the reboot, the client would not notice that that the server had exported a different filesystem to the same path because the client has already done the lookup; he has no reason to do it again. (If the original file is still exported under a different name, the client will continue accessing the old file. If the original filesystem isn't exported at all anymore, the client's access via the old file handle fails and the client repeats the lookup and continues accessing the new file as if nothing had changed).

The point of the FSID is simply that the file handle has to completely identify the file so the server knows which file to access. An NFS client request to read a file doesn't come with a file name; the file handle is all there is.

If the filesystem lives on a disk device and you trust the disk device's major/minor number not to change across a reboot, major/minor number makes a good FSID. If not, you need something else, such as the filesystem UUID you propose.

Having the system administrator manually identify the filesystem in a special namespace via fsid=5 in /etc/exports obviously doesn't have the same level of integrity, but it does solve the problem of not having any way to identify the filesystem at all.

Committed...

steved
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

Committed...

steved.
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

Committed...

steved.
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

Committed...

steved.