Skip to contentSkip to conversation

exportfs: add support for "nowcc" option

PATCH In review Jeff Layton 3 participants 10 messages Sep 3, 2015 – Sep 4, 2015 1 message not in the archive

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 exportfs: add support for "nowcc" option

This is the userland companion patch for the patch to add a "nowcc" option to knfsd. This just adds the necessary code to allow userspace to set that option.

---
 support/include/nfs/export.h |  3 ++-
 support/nfs/exports.c        |  5 +++++
 utils/exportfs/exportfs.c    |  2 ++
 utils/exportfs/exports.man   | 14 ++++++++++++++
 4 files changed, 23 insertions(+), 1 deletion(-)
support/include/nfs/export.h +2 -1
index 1194255899bd..68bee5c2f305 100644
@@ -26,7 +26,8 @@
 #define	NFSEXP_CROSSMOUNT	0x4000
 #define NFSEXP_NOACL		0x8000 /* reserved for possible ACL related use */
 #define NFSEXP_V4ROOT		0x10000
-#define NFSEXP_PNFS            0x20000
+#define NFSEXP_PNFS		0x20000
+#define NFSEXP_NOWCC		0x40000
 /*
  * All flags supported by the kernel before addition of the
  * export_features interface:
support/nfs/exports.c +5 -0
index 0aea6f154f09..791b5f756c9c 100644
@@ -276,6 +276,7 @@ putexportent(struct exportent *ep)
 	if (ep->e_flags & NFSEXP_NOREADDIRPLUS)
 		fprintf(fp, "nordirplus,");
 	fprintf(fp, "%spnfs,", (ep->e_flags & NFSEXP_PNFS)? "" : "no_");
+	fprintf(fp, "%swcc,", (ep->e_flags & NFSEXP_NOWCC) ? "no" : "");
 	if (ep->e_flags & NFSEXP_FSID) {
 		fprintf(fp, "fsid=%d,", ep->e_fsid);
 	}
@@ -586,6 +587,10 @@ parseopts(char *cp, struct exportent *ep, int warn, int *had_subtree_opt_ptr)
 			setflags(NFSEXP_PNFS, active, ep);
 		else if (!strcmp(opt, "no_pnfs"))
 			clearflags(NFSEXP_PNFS, active, ep);
+		else if (!strcmp(opt, "wcc"))
+			clearflags(NFSEXP_NOWCC, active, ep);
+		else if (!strcmp(opt, "nowcc"))
+			setflags(NFSEXP_NOWCC, active, ep);
 		else if (strncmp(opt, "anonuid=", 8) == 0) {
 			char *oe;
 			ep->e_anonuid = strtol(opt+8, &oe, 10);
utils/exportfs/exportfs.c +2 -0
index 87582316b086..a2afc80ee6ea 100644
@@ -823,6 +823,8 @@ dump(int verbose, int export_format)
 				c = dumpopt(c, "no_acl");
 			if (ep->e_flags & NFSEXP_PNFS)
 				c = dumpopt(c, "pnfs");
+			if (ep->e_flags & NFSEXP_NOWCC)
+				c = dumpopt(c, "nowcc");
 			if (ep->e_flags & NFSEXP_FSID)
 				c = dumpopt(c, "fsid=%d", ep->e_fsid);
 			if (ep->e_uuid)
utils/exportfs/exports.man +14 -0
index 93092463153b..9213a228bb04 100644
@@ -417,6 +417,20 @@ devices. The default can be explicitly requested with the
 .I no_pnfs
 option.
 
+.TP
+.IR nowcc
+This option disables the collection and reporting of WCC (weak cache
+consistency) data in NFSv3 requests.  RFC 1813 recommends that all
+servers report this information to the clients as it allows the client
+to avoid some extra round trips to the server.  In some underlying
+filesystems however, collecting this data can be cost prohibitive and
+atomicity is difficult to guarantee.  Also, in some client use-cases WCC
+data is ignored or not terribly useful.  The default is to always report
+weak cache consistency data to the client in NFSv3 requests, and can be
+explicitly requested with the
+.I wcc
+option.
+
 .SS User ID Mapping
 .PP
 .B nfsd
Signature · 1 line
2.4.3
#1 lore raw
Quoted 68 lines · Jeff Layton

This is the userland companion patch for the patch to add a "nowcc" option to knfsd. This just adds the necessary code to allow userspace to set that option.

---
 support/include/nfs/export.h |  3 ++-
 support/nfs/exports.c        |  5 +++++
 utils/exportfs/exportfs.c    |  2 ++
 utils/exportfs/exports.man   | 14 ++++++++++++++
 4 files changed, 23 insertions(+), 1 deletion(-)
support/include/nfs/export.h +2 -1
index 1194255899bd..68bee5c2f305 100644
@@ -26,7 +26,8 @@
 #define	NFSEXP_CROSSMOUNT	0x4000
 #define NFSEXP_NOACL		0x8000 /* reserved for possible ACL related use */
 #define NFSEXP_V4ROOT		0x10000
-#define NFSEXP_PNFS            0x20000
+#define NFSEXP_PNFS		0x20000
+#define NFSEXP_NOWCC		0x40000
 /*
  * All flags supported by the kernel before addition of the
  * export_features interface:
support/nfs/exports.c +5 -0
index 0aea6f154f09..791b5f756c9c 100644
@@ -276,6 +276,7 @@ putexportent(struct exportent *ep)
 	if (ep->e_flags & NFSEXP_NOREADDIRPLUS)
 		fprintf(fp, "nordirplus,");
 	fprintf(fp, "%spnfs,", (ep->e_flags & NFSEXP_PNFS)? "" : "no_");
+	fprintf(fp, "%swcc,", (ep->e_flags & NFSEXP_NOWCC) ? "no" : "");
 	if (ep->e_flags & NFSEXP_FSID) {
 		fprintf(fp, "fsid=%d,", ep->e_fsid);
 	}
@@ -586,6 +587,10 @@ parseopts(char *cp, struct exportent *ep, int warn, int *had_subtree_opt_ptr)
 			setflags(NFSEXP_PNFS, active, ep);
 		else if (!strcmp(opt, "no_pnfs"))
 			clearflags(NFSEXP_PNFS, active, ep);
+		else if (!strcmp(opt, "wcc"))
+			clearflags(NFSEXP_NOWCC, active, ep);
+		else if (!strcmp(opt, "nowcc"))
+			setflags(NFSEXP_NOWCC, active, ep);
 		else if (strncmp(opt, "anonuid=", 8) == 0) {
 			char *oe;
 			ep->e_anonuid = strtol(opt+8, &oe, 10);
utils/exportfs/exportfs.c +2 -0
index 87582316b086..a2afc80ee6ea 100644
@@ -823,6 +823,8 @@ dump(int verbose, int export_format)
 				c = dumpopt(c, "no_acl");
 			if (ep->e_flags & NFSEXP_PNFS)
 				c = dumpopt(c, "pnfs");
+			if (ep->e_flags & NFSEXP_NOWCC)
+				c = dumpopt(c, "nowcc");
 			if (ep->e_flags & NFSEXP_FSID)
 				c = dumpopt(c, "fsid=%d", ep->e_fsid);
 			if (ep->e_uuid)
utils/exportfs/exports.man +0 -0
index 93092463153b..9213a228bb04 100644
@@ -417,6 +417,20 @@ devices. The default can be explicitly requested with the
 .I no_pnfs

utils/exportfs/exports.man devices. The default can be explicitly requested with the · line 427

Quoted 10 lines
utils/exportfs/exports.man +8 -0
 option.
 
+.TP
+.IR nowcc
+This option disables the collection and reporting of WCC (weak cache
+consistency) data in NFSv3 requests.  RFC 1813 recommends that all
+servers report this information to the clients as it allows the client
+to avoid some extra round trips to the server.  In some underlying
+filesystems however, collecting this data can be cost prohibitive and
+atomicity is difficult to guarantee.

Which filesystems?

utils/exportfs/exports.man devices. The default can be explicitly requested with the · line 433

Quoted 7 lines

Also, in some client use-cases WCC

utils/exportfs/exports.man +6 -0
+data is ignored or not terribly useful.  The default is to always report
+weak cache consistency data to the client in NFSv3 requests, and can be
+explicitly requested with the
+.I wcc
+option.
+

This is a little vague. Can we give users any better guidelines?

--b.

#2 lore raw
Quoted 71 lines · On Thu, 3 Sep 2015 14:31:03 -0400 "J. Bruce Fields" <[email protected]>…
Quoted 70 lines · Jeff Layton

This is the userland companion patch for the patch to add a "nowcc" option to knfsd. This just adds the necessary code to allow userspace to set that option.

---
 support/include/nfs/export.h |  3 ++-
 support/nfs/exports.c        |  5 +++++
 utils/exportfs/exportfs.c    |  2 ++
 utils/exportfs/exports.man   | 14 ++++++++++++++
 4 files changed, 23 insertions(+), 1 deletion(-)
support/include/nfs/export.h +2 -1
index 1194255899bd..68bee5c2f305 100644
@@ -26,7 +26,8 @@
 #define	NFSEXP_CROSSMOUNT	0x4000
 #define NFSEXP_NOACL		0x8000 /* reserved for possible ACL related use */
 #define NFSEXP_V4ROOT		0x10000
-#define NFSEXP_PNFS            0x20000
+#define NFSEXP_PNFS		0x20000
+#define NFSEXP_NOWCC		0x40000
 /*
  * All flags supported by the kernel before addition of the
  * export_features interface:
support/nfs/exports.c +5 -0
index 0aea6f154f09..791b5f756c9c 100644
@@ -276,6 +276,7 @@ putexportent(struct exportent *ep)
 	if (ep->e_flags & NFSEXP_NOREADDIRPLUS)
 		fprintf(fp, "nordirplus,");
 	fprintf(fp, "%spnfs,", (ep->e_flags & NFSEXP_PNFS)? "" : "no_");
+	fprintf(fp, "%swcc,", (ep->e_flags & NFSEXP_NOWCC) ? "no" : "");
 	if (ep->e_flags & NFSEXP_FSID) {
 		fprintf(fp, "fsid=%d,", ep->e_fsid);
 	}
@@ -586,6 +587,10 @@ parseopts(char *cp, struct exportent *ep, int warn, int *had_subtree_opt_ptr)
 			setflags(NFSEXP_PNFS, active, ep);
 		else if (!strcmp(opt, "no_pnfs"))
 			clearflags(NFSEXP_PNFS, active, ep);
+		else if (!strcmp(opt, "wcc"))
+			clearflags(NFSEXP_NOWCC, active, ep);
+		else if (!strcmp(opt, "nowcc"))
+			setflags(NFSEXP_NOWCC, active, ep);
 		else if (strncmp(opt, "anonuid=", 8) == 0) {
 			char *oe;
 			ep->e_anonuid = strtol(opt+8, &oe, 10);
utils/exportfs/exportfs.c +2 -0
index 87582316b086..a2afc80ee6ea 100644
@@ -823,6 +823,8 @@ dump(int verbose, int export_format)
 				c = dumpopt(c, "no_acl");
 			if (ep->e_flags & NFSEXP_PNFS)
 				c = dumpopt(c, "pnfs");
+			if (ep->e_flags & NFSEXP_NOWCC)
+				c = dumpopt(c, "nowcc");
 			if (ep->e_flags & NFSEXP_FSID)
 				c = dumpopt(c, "fsid=%d", ep->e_fsid);
 			if (ep->e_uuid)
utils/exportfs/exports.man +0 -0
index 93092463153b..9213a228bb04 100644
@@ -417,6 +417,20 @@ devices. The default can be explicitly requested with the
 .I no_pnfs
 option.
 
Quoted 11 lines

utils/exportfs/exports.man devices. The default can be explicitly requested with the

Quoted 8 lines
utils/exportfs/exports.man +8 -0
+.TP
+.IR nowcc
+This option disables the collection and reporting of WCC (weak cache
+consistency) data in NFSv3 requests.  RFC 1813 recommends that all
+servers report this information to the clients as it allows the client
+to avoid some extra round trips to the server.  In some underlying
+filesystems however, collecting this data can be cost prohibitive and
+atomicity is difficult to guarantee.

Which filesystems?

NFS is the main one that comes to mind, but this is also the case with many clustered filesystems (GFS2, ceph, OCFS2, etc...). You generally need to do some network activity to ensure that your getattr is consistent and that can be quite expensive.

Quoted 10 lines

utils/exportfs/exports.man devices. The default can be explicitly requested with the · line 433

Quoted 7 lines

Also, in some client use-cases WCC

utils/exportfs/exports.man +6 -0
+data is ignored or not terribly useful.  The default is to always report
+weak cache consistency data to the client in NFSv3 requests, and can be
+explicitly requested with the
+.I wcc
+option.
+

This is a little vague. Can we give users any better guidelines?

Sure, I guess we could mention that if you're primarily exporting to clients that are doing direct I/O or are setting up a DS for flexfiles then you might want to enable this option. Would that clarify things?

Signature · 1 line
Jeff Layton <[email protected]>
#3 lore raw
Quoted 102 lines · Jeff Layton
Quoted 71 lines · On Thu, 3 Sep 2015 14:31:03 -0400 "J. Bruce Fields" <[email protected]>…
Quoted 70 lines · Jeff Layton

This is the userland companion patch for the patch to add a "nowcc" option to knfsd. This just adds the necessary code to allow userspace to set that option.

---
 support/include/nfs/export.h |  3 ++-
 support/nfs/exports.c        |  5 +++++
 utils/exportfs/exportfs.c    |  2 ++
 utils/exportfs/exports.man   | 14 ++++++++++++++
 4 files changed, 23 insertions(+), 1 deletion(-)
support/include/nfs/export.h +2 -1
index 1194255899bd..68bee5c2f305 100644
@@ -26,7 +26,8 @@
 #define	NFSEXP_CROSSMOUNT	0x4000
 #define NFSEXP_NOACL		0x8000 /* reserved for possible ACL related use */
 #define NFSEXP_V4ROOT		0x10000
-#define NFSEXP_PNFS            0x20000
+#define NFSEXP_PNFS		0x20000
+#define NFSEXP_NOWCC		0x40000
 /*
  * All flags supported by the kernel before addition of the
  * export_features interface:
support/nfs/exports.c +5 -0
index 0aea6f154f09..791b5f756c9c 100644
@@ -276,6 +276,7 @@ putexportent(struct exportent *ep)
 	if (ep->e_flags & NFSEXP_NOREADDIRPLUS)
 		fprintf(fp, "nordirplus,");
 	fprintf(fp, "%spnfs,", (ep->e_flags & NFSEXP_PNFS)? "" : "no_");
+	fprintf(fp, "%swcc,", (ep->e_flags & NFSEXP_NOWCC) ? "no" : "");
 	if (ep->e_flags & NFSEXP_FSID) {
 		fprintf(fp, "fsid=%d,", ep->e_fsid);
 	}
@@ -586,6 +587,10 @@ parseopts(char *cp, struct exportent *ep, int warn, int *had_subtree_opt_ptr)
 			setflags(NFSEXP_PNFS, active, ep);
 		else if (!strcmp(opt, "no_pnfs"))
 			clearflags(NFSEXP_PNFS, active, ep);
+		else if (!strcmp(opt, "wcc"))
+			clearflags(NFSEXP_NOWCC, active, ep);
+		else if (!strcmp(opt, "nowcc"))
+			setflags(NFSEXP_NOWCC, active, ep);
 		else if (strncmp(opt, "anonuid=", 8) == 0) {
 			char *oe;
 			ep->e_anonuid = strtol(opt+8, &oe, 10);
utils/exportfs/exportfs.c +2 -0
index 87582316b086..a2afc80ee6ea 100644
@@ -823,6 +823,8 @@ dump(int verbose, int export_format)
 				c = dumpopt(c, "no_acl");
 			if (ep->e_flags & NFSEXP_PNFS)
 				c = dumpopt(c, "pnfs");
+			if (ep->e_flags & NFSEXP_NOWCC)
+				c = dumpopt(c, "nowcc");
 			if (ep->e_flags & NFSEXP_FSID)
 				c = dumpopt(c, "fsid=%d", ep->e_fsid);
 			if (ep->e_uuid)
utils/exportfs/exports.man +0 -0
index 93092463153b..9213a228bb04 100644
@@ -417,6 +417,20 @@ devices. The default can be explicitly requested with the
 .I no_pnfs
 option.
 
Quoted 11 lines

utils/exportfs/exports.man devices. The default can be explicitly requested with the

Quoted 8 lines
utils/exportfs/exports.man +8 -0
+.TP
+.IR nowcc
+This option disables the collection and reporting of WCC (weak cache
+consistency) data in NFSv3 requests.  RFC 1813 recommends that all
+servers report this information to the clients as it allows the client
+to avoid some extra round trips to the server.  In some underlying
+filesystems however, collecting this data can be cost prohibitive and
+atomicity is difficult to guarantee.

Which filesystems?

NFS is the main one that comes to mind, but this is also the case with many clustered filesystems (GFS2, ceph, OCFS2, etc...). You generally need to do some network activity to ensure that your getattr is consistent and that can be quite expensive.

Quoted 10 lines

utils/exportfs/exports.man devices. The default can be explicitly requested with the · line 433

Quoted 7 lines

Also, in some client use-cases WCC

utils/exportfs/exports.man +6 -0
+data is ignored or not terribly useful.  The default is to always report
+weak cache consistency data to the client in NFSv3 requests, and can be
+explicitly requested with the
+.I wcc
+option.
+

This is a little vague. Can we give users any better guidelines?

Quoted 3 lines

Sure, I guess we could mention that if you're primarily exporting to clients that are doing direct I/O or are setting up a DS for flexfiles then you might want to enable this option. Would that clarify things?

Yes, and maybe mention the distributed filesystem case (I assume the getattr's close to free on most disk filesystems).

--b.

#4 lore raw
Quoted 75 lines · J. Bruce Fields
Quoted 74 lines · Jeff Layton
Quoted 71 lines · On Thu, 3 Sep 2015 14:31:03 -0400 "J. Bruce Fields" <[email protected]>…
Quoted 70 lines · Jeff Layton

This is the userland companion patch for the patch to add a "nowcc" option to knfsd. This just adds the necessary code to allow userspace to set that option.

---
 support/include/nfs/export.h |  3 ++-
 support/nfs/exports.c        |  5 +++++
 utils/exportfs/exportfs.c    |  2 ++
 utils/exportfs/exports.man   | 14 ++++++++++++++
 4 files changed, 23 insertions(+), 1 deletion(-)
support/include/nfs/export.h +2 -1
index 1194255899bd..68bee5c2f305 100644
@@ -26,7 +26,8 @@
 #define	NFSEXP_CROSSMOUNT	0x4000
 #define NFSEXP_NOACL		0x8000 /* reserved for possible ACL related use */
 #define NFSEXP_V4ROOT		0x10000
-#define NFSEXP_PNFS            0x20000
+#define NFSEXP_PNFS		0x20000
+#define NFSEXP_NOWCC		0x40000
 /*
  * All flags supported by the kernel before addition of the
  * export_features interface:
support/nfs/exports.c +5 -0
index 0aea6f154f09..791b5f756c9c 100644
@@ -276,6 +276,7 @@ putexportent(struct exportent *ep)
 	if (ep->e_flags & NFSEXP_NOREADDIRPLUS)
 		fprintf(fp, "nordirplus,");
 	fprintf(fp, "%spnfs,", (ep->e_flags & NFSEXP_PNFS)? "" : "no_");
+	fprintf(fp, "%swcc,", (ep->e_flags & NFSEXP_NOWCC) ? "no" : "");
 	if (ep->e_flags & NFSEXP_FSID) {
 		fprintf(fp, "fsid=%d,", ep->e_fsid);
 	}
@@ -586,6 +587,10 @@ parseopts(char *cp, struct exportent *ep, int warn, int *had_subtree_opt_ptr)
 			setflags(NFSEXP_PNFS, active, ep);
 		else if (!strcmp(opt, "no_pnfs"))
 			clearflags(NFSEXP_PNFS, active, ep);
+		else if (!strcmp(opt, "wcc"))
+			clearflags(NFSEXP_NOWCC, active, ep);
+		else if (!strcmp(opt, "nowcc"))
+			setflags(NFSEXP_NOWCC, active, ep);
 		else if (strncmp(opt, "anonuid=", 8) == 0) {
 			char *oe;
 			ep->e_anonuid = strtol(opt+8, &oe, 10);
utils/exportfs/exportfs.c +2 -0
index 87582316b086..a2afc80ee6ea 100644
@@ -823,6 +823,8 @@ dump(int verbose, int export_format)
 				c = dumpopt(c, "no_acl");
 			if (ep->e_flags & NFSEXP_PNFS)
 				c = dumpopt(c, "pnfs");
+			if (ep->e_flags & NFSEXP_NOWCC)
+				c = dumpopt(c, "nowcc");
 			if (ep->e_flags & NFSEXP_FSID)
 				c = dumpopt(c, "fsid=%d", ep->e_fsid);
 			if (ep->e_uuid)
utils/exportfs/exports.man +0 -0
index 93092463153b..9213a228bb04 100644
@@ -417,6 +417,20 @@ devices. The default can be explicitly requested with the
 .I no_pnfs
 option.
 
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

utils/exportfs/exports.man devices. The default can be explicitly requested with the

Quoted 8 lines
utils/exportfs/exports.man +8 -0
+.TP
+.IR nowcc
+This option disables the collection and reporting of WCC (weak cache
+consistency) data in NFSv3 requests.  RFC 1813 recommends that all
+servers report this information to the clients as it allows the client
+to avoid some extra round trips to the server.  In some underlying
+filesystems however, collecting this data can be cost prohibitive and
+atomicity is difficult to guarantee.

By the way, independently of this change, maybe we should by default disable the pre-op attribute on distributed filesystems? As I understand it we're just depending on the i_mutex for atomicity, which can't be right in that case.

--b.

#5 lore raw
Quoted 90 lines · On Thu, 3 Sep 2015 15:42:55 -0400 "J. Bruce Fields" <[email protected]>…
Quoted 75 lines · J. Bruce Fields
Quoted 74 lines · Jeff Layton
Quoted 71 lines · On Thu, 3 Sep 2015 14:31:03 -0400 "J. Bruce Fields" <[email protected]>…
Quoted 70 lines · Jeff Layton

This is the userland companion patch for the patch to add a "nowcc" option to knfsd. This just adds the necessary code to allow userspace to set that option.

---
 support/include/nfs/export.h |  3 ++-
 support/nfs/exports.c        |  5 +++++
 utils/exportfs/exportfs.c    |  2 ++
 utils/exportfs/exports.man   | 14 ++++++++++++++
 4 files changed, 23 insertions(+), 1 deletion(-)
support/include/nfs/export.h +2 -1
index 1194255899bd..68bee5c2f305 100644
@@ -26,7 +26,8 @@
 #define	NFSEXP_CROSSMOUNT	0x4000
 #define NFSEXP_NOACL		0x8000 /* reserved for possible ACL related use */
 #define NFSEXP_V4ROOT		0x10000
-#define NFSEXP_PNFS            0x20000
+#define NFSEXP_PNFS		0x20000
+#define NFSEXP_NOWCC		0x40000
 /*
  * All flags supported by the kernel before addition of the
  * export_features interface:
support/nfs/exports.c +5 -0
index 0aea6f154f09..791b5f756c9c 100644
@@ -276,6 +276,7 @@ putexportent(struct exportent *ep)
 	if (ep->e_flags & NFSEXP_NOREADDIRPLUS)
 		fprintf(fp, "nordirplus,");
 	fprintf(fp, "%spnfs,", (ep->e_flags & NFSEXP_PNFS)? "" : "no_");
+	fprintf(fp, "%swcc,", (ep->e_flags & NFSEXP_NOWCC) ? "no" : "");
 	if (ep->e_flags & NFSEXP_FSID) {
 		fprintf(fp, "fsid=%d,", ep->e_fsid);
 	}
@@ -586,6 +587,10 @@ parseopts(char *cp, struct exportent *ep, int warn, int *had_subtree_opt_ptr)
 			setflags(NFSEXP_PNFS, active, ep);
 		else if (!strcmp(opt, "no_pnfs"))
 			clearflags(NFSEXP_PNFS, active, ep);
+		else if (!strcmp(opt, "wcc"))
+			clearflags(NFSEXP_NOWCC, active, ep);
+		else if (!strcmp(opt, "nowcc"))
+			setflags(NFSEXP_NOWCC, active, ep);
 		else if (strncmp(opt, "anonuid=", 8) == 0) {
 			char *oe;
 			ep->e_anonuid = strtol(opt+8, &oe, 10);
utils/exportfs/exportfs.c +2 -0
index 87582316b086..a2afc80ee6ea 100644
@@ -823,6 +823,8 @@ dump(int verbose, int export_format)
 				c = dumpopt(c, "no_acl");
 			if (ep->e_flags & NFSEXP_PNFS)
 				c = dumpopt(c, "pnfs");
+			if (ep->e_flags & NFSEXP_NOWCC)
+				c = dumpopt(c, "nowcc");
 			if (ep->e_flags & NFSEXP_FSID)
 				c = dumpopt(c, "fsid=%d", ep->e_fsid);
 			if (ep->e_uuid)
utils/exportfs/exports.man +0 -0
index 93092463153b..9213a228bb04 100644
@@ -417,6 +417,20 @@ devices. The default can be explicitly requested with the
 .I no_pnfs
 option.
 
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

utils/exportfs/exports.man devices. The default can be explicitly requested with the

Quoted 8 lines
utils/exportfs/exports.man +8 -0
+.TP
+.IR nowcc
+This option disables the collection and reporting of WCC (weak cache
+consistency) data in NFSv3 requests.  RFC 1813 recommends that all
+servers report this information to the clients as it allows the client
+to avoid some extra round trips to the server.  In some underlying
+filesystems however, collecting this data can be cost prohibitive and
+atomicity is difficult to guarantee.

By the way, independently of this change, maybe we should by default disable the pre-op attribute on distributed filesystems? As I understand it we're just depending on the i_mutex for atomicity, which can't be right in that case.

Quoted 1 line

--b.

Right. In the case of something like a clustered or network filesystem you really can't guarantee atomicity, so why even try?

So, that is another option...we could add a flags field to the export_operations struct and use one flag to indicate whether to send wcc attrs. That'd give us the ability to add other flags in the future as well.

The reason I went with the export option is that I didn't really want to add the flags field, and I figured the flexibility of being able to do that on any filesystem might be helpful.

That said, I'm open to doing it that way instead of (or in addition to) the export option if you think it would be better.

Signature · 1 line
Jeff Layton <[email protected]>
#6 lore raw
Quoted 103 lines · Jeff Layton
Quoted 90 lines · On Thu, 3 Sep 2015 15:42:55 -0400 "J. Bruce Fields" <[email protected]>…
Quoted 75 lines · J. Bruce Fields
Quoted 74 lines · Jeff Layton
Quoted 71 lines · On Thu, 3 Sep 2015 14:31:03 -0400 "J. Bruce Fields" <[email protected]>…
Quoted 70 lines · Jeff Layton

This is the userland companion patch for the patch to add a "nowcc" option to knfsd. This just adds the necessary code to allow userspace to set that option.

---
 support/include/nfs/export.h |  3 ++-
 support/nfs/exports.c        |  5 +++++
 utils/exportfs/exportfs.c    |  2 ++
 utils/exportfs/exports.man   | 14 ++++++++++++++
 4 files changed, 23 insertions(+), 1 deletion(-)
support/include/nfs/export.h +2 -1
index 1194255899bd..68bee5c2f305 100644
@@ -26,7 +26,8 @@
 #define	NFSEXP_CROSSMOUNT	0x4000
 #define NFSEXP_NOACL		0x8000 /* reserved for possible ACL related use */
 #define NFSEXP_V4ROOT		0x10000
-#define NFSEXP_PNFS            0x20000
+#define NFSEXP_PNFS		0x20000
+#define NFSEXP_NOWCC		0x40000
 /*
  * All flags supported by the kernel before addition of the
  * export_features interface:
support/nfs/exports.c +5 -0
index 0aea6f154f09..791b5f756c9c 100644
@@ -276,6 +276,7 @@ putexportent(struct exportent *ep)
 	if (ep->e_flags & NFSEXP_NOREADDIRPLUS)
 		fprintf(fp, "nordirplus,");
 	fprintf(fp, "%spnfs,", (ep->e_flags & NFSEXP_PNFS)? "" : "no_");
+	fprintf(fp, "%swcc,", (ep->e_flags & NFSEXP_NOWCC) ? "no" : "");
 	if (ep->e_flags & NFSEXP_FSID) {
 		fprintf(fp, "fsid=%d,", ep->e_fsid);
 	}
@@ -586,6 +587,10 @@ parseopts(char *cp, struct exportent *ep, int warn, int *had_subtree_opt_ptr)
 			setflags(NFSEXP_PNFS, active, ep);
 		else if (!strcmp(opt, "no_pnfs"))
 			clearflags(NFSEXP_PNFS, active, ep);
+		else if (!strcmp(opt, "wcc"))
+			clearflags(NFSEXP_NOWCC, active, ep);
+		else if (!strcmp(opt, "nowcc"))
+			setflags(NFSEXP_NOWCC, active, ep);
 		else if (strncmp(opt, "anonuid=", 8) == 0) {
 			char *oe;
 			ep->e_anonuid = strtol(opt+8, &oe, 10);
utils/exportfs/exportfs.c +2 -0
index 87582316b086..a2afc80ee6ea 100644
@@ -823,6 +823,8 @@ dump(int verbose, int export_format)
 				c = dumpopt(c, "no_acl");
 			if (ep->e_flags & NFSEXP_PNFS)
 				c = dumpopt(c, "pnfs");
+			if (ep->e_flags & NFSEXP_NOWCC)
+				c = dumpopt(c, "nowcc");
 			if (ep->e_flags & NFSEXP_FSID)
 				c = dumpopt(c, "fsid=%d", ep->e_fsid);
 			if (ep->e_uuid)
utils/exportfs/exports.man +0 -0
index 93092463153b..9213a228bb04 100644
@@ -417,6 +417,20 @@ devices. The default can be explicitly requested with the
 .I no_pnfs
 option.
 
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

utils/exportfs/exports.man devices. The default can be explicitly requested with the

Quoted 8 lines
utils/exportfs/exports.man +8 -0
+.TP
+.IR nowcc
+This option disables the collection and reporting of WCC (weak cache
+consistency) data in NFSv3 requests.  RFC 1813 recommends that all
+servers report this information to the clients as it allows the client
+to avoid some extra round trips to the server.  In some underlying
+filesystems however, collecting this data can be cost prohibitive and
+atomicity is difficult to guarantee.

By the way, independently of this change, maybe we should by default disable the pre-op attribute on distributed filesystems? As I understand it we're just depending on the i_mutex for atomicity, which can't be right in that case.

Quoted 1 line

--b.

Right. In the case of something like a clustered or network filesystem you really can't guarantee atomicity, so why even try?

So, that is another option...we could add a flags field to the export_operations struct and use one flag to indicate whether to send wcc attrs. That'd give us the ability to add other flags in the future as well.

Quoted 6 lines

The reason I went with the export option is that I didn't really want to add the flags field, and I figured the flexibility of being able to do that on any filesystem might be helpful.

That said, I'm open to doing it that way instead of (or in addition to) the export option if you think it would be better.

Well, it's sounding so far like:

- nobody will want this on a non-distributed filesystem (hard to see what the benefit would be; and it might cost).
- people probably *do* want this on any distributed filesystem.
- we could save users some trouble by not giving them another export option which they might just set wrong.

Eh, but I think I'm wrong about #2. Dropping the pre-op attribute is arguably necessary for correctness, but dropping the post-op attribute as well isn't necessary and might hurt, right? OK.

--b.

#7 lore raw
Quoted 111 lines · J. Bruce Fields
Quoted 103 lines · Jeff Layton
Quoted 90 lines · On Thu, 3 Sep 2015 15:42:55 -0400 "J. Bruce Fields" <[email protected]>…
Quoted 75 lines · J. Bruce Fields
Quoted 74 lines · Jeff Layton
Quoted 71 lines · On Thu, 3 Sep 2015 14:31:03 -0400 "J. Bruce Fields" <[email protected]>…
Quoted 70 lines · Jeff Layton

This is the userland companion patch for the patch to add a "nowcc" option to knfsd. This just adds the necessary code to allow userspace to set that option.

---

support/include/nfs/export.h |  3 ++-
support/nfs/exports.c        |  5 +++++
utils/exportfs/exportfs.c    |  2 ++
utils/exportfs/exports.man   | 14 ++++++++++++++
4 files changed, 23 insertions(+), 1 deletion(-)
support/include/nfs/export.h +2 -1
index 1194255899bd..68bee5c2f305 100644
@@ -26,7 +26,8 @@
 #define	NFSEXP_CROSSMOUNT	0x4000
 #define NFSEXP_NOACL		0x8000 /* reserved for possible ACL related use */
 #define NFSEXP_V4ROOT		0x10000
-#define NFSEXP_PNFS            0x20000
+#define NFSEXP_PNFS		0x20000
+#define NFSEXP_NOWCC		0x40000
 /*
 * All flags supported by the kernel before addition of the
 * export_features interface:
support/nfs/exports.c +5 -0
index 0aea6f154f09..791b5f756c9c 100644
@@ -276,6 +276,7 @@ putexportent(struct exportent *ep)
 	if (ep->e_flags & NFSEXP_NOREADDIRPLUS)
 		fprintf(fp, "nordirplus,");
 	fprintf(fp, "%spnfs,", (ep->e_flags & NFSEXP_PNFS)? "" : "no_");
+	fprintf(fp, "%swcc,", (ep->e_flags & NFSEXP_NOWCC) ? "no" : "");
 	if (ep->e_flags & NFSEXP_FSID) {
 		fprintf(fp, "fsid=%d,", ep->e_fsid);
 	}
@@ -586,6 +587,10 @@ parseopts(char *cp, struct exportent *ep, int warn, int *had_subtree_opt_ptr)
 			setflags(NFSEXP_PNFS, active, ep);
 		else if (!strcmp(opt, "no_pnfs"))
 			clearflags(NFSEXP_PNFS, active, ep);
+		else if (!strcmp(opt, "wcc"))
+			clearflags(NFSEXP_NOWCC, active, ep);
+		else if (!strcmp(opt, "nowcc"))
+			setflags(NFSEXP_NOWCC, active, ep);
 		else if (strncmp(opt, "anonuid=", 8) == 0) {
 			char *oe;
 			ep->e_anonuid = strtol(opt+8, &oe, 10);
utils/exportfs/exportfs.c +2 -0
index 87582316b086..a2afc80ee6ea 100644
@@ -823,6 +823,8 @@ dump(int verbose, int export_format)
 				c = dumpopt(c, "no_acl");
 			if (ep->e_flags & NFSEXP_PNFS)
 				c = dumpopt(c, "pnfs");
+			if (ep->e_flags & NFSEXP_NOWCC)
+				c = dumpopt(c, "nowcc");
 			if (ep->e_flags & NFSEXP_FSID)
 				c = dumpopt(c, "fsid=%d", ep->e_fsid);
 			if (ep->e_uuid)
utils/exportfs/exports.man +0 -0
index 93092463153b..9213a228bb04 100644
@@ -417,6 +417,20 @@ devices. The default can be explicitly requested with the

.I no_pnfs
option.

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

utils/exportfs/exports.man

Quoted 8 lines

+.TP
+.IR nowcc

utils/exportfs/exports.man +6 -0
+This option disables the collection and reporting of WCC (weak cache
+consistency) data in NFSv3 requests.  RFC 1813 recommends that all
+servers report this information to the clients as it allows the client
+to avoid some extra round trips to the server.  In some underlying
+filesystems however, collecting this data can be cost prohibitive and
+atomicity is difficult to guarantee.

By the way, independently of this change, maybe we should by default disable the pre-op attribute on distributed filesystems? As I understand it we're just depending on the i_mutex for atomicity, which can't be right in that case.

Quoted 1 line

--b.

Right. In the case of something like a clustered or network filesystem you really can't guarantee atomicity, so why even try?

So, that is another option...we could add a flags field to the export_operations struct and use one flag to indicate whether to send wcc attrs. That'd give us the ability to add other flags in the future as well.

Quoted 6 lines

The reason I went with the export option is that I didn't really want to add the flags field, and I figured the flexibility of being able to do that on any filesystem might be helpful.

That said, I'm open to doing it that way instead of (or in addition to) the export option if you think it would be better.

Quoted 7 lines

Well, it's sounding so far like:

- nobody will want this on a non-distributed filesystem (hard to see what the benefit would be; and it might cost).
- people probably *do* want this on any distributed filesystem.
- we could save users some trouble by not giving them another export option which they might just set wrong.

Here's another vote for having the setting automatically determined.

Quoted 3 lines

Eh, but I think I'm wrong about #2. Dropping the pre-op attribute is arguably necessary for correctness, but dropping the post-op attribute as well isn't necessary and might hurt, right? OK.

--
Chuck Lever

#8 lore raw
In reply to J. Bruce Fields
Quoted 119 lines · On Thu, 3 Sep 2015 16:32:00 -0400 "J. Bruce Fields" <[email protected]>…
Quoted 103 lines · Jeff Layton
Quoted 90 lines · On Thu, 3 Sep 2015 15:42:55 -0400 "J. Bruce Fields" <[email protected]>…
Quoted 75 lines · J. Bruce Fields
Quoted 74 lines · Jeff Layton
Quoted 71 lines · On Thu, 3 Sep 2015 14:31:03 -0400 "J. Bruce Fields" <[email protected]>…
Quoted 70 lines · Jeff Layton

This is the userland companion patch for the patch to add a "nowcc" option to knfsd. This just adds the necessary code to allow userspace to set that option.

---
 support/include/nfs/export.h |  3 ++-
 support/nfs/exports.c        |  5 +++++
 utils/exportfs/exportfs.c    |  2 ++
 utils/exportfs/exports.man   | 14 ++++++++++++++
 4 files changed, 23 insertions(+), 1 deletion(-)
support/include/nfs/export.h +2 -1
index 1194255899bd..68bee5c2f305 100644
@@ -26,7 +26,8 @@
 #define	NFSEXP_CROSSMOUNT	0x4000
 #define NFSEXP_NOACL		0x8000 /* reserved for possible ACL related use */
 #define NFSEXP_V4ROOT		0x10000
-#define NFSEXP_PNFS            0x20000
+#define NFSEXP_PNFS		0x20000
+#define NFSEXP_NOWCC		0x40000
 /*
  * All flags supported by the kernel before addition of the
  * export_features interface:
support/nfs/exports.c +5 -0
index 0aea6f154f09..791b5f756c9c 100644
@@ -276,6 +276,7 @@ putexportent(struct exportent *ep)
 	if (ep->e_flags & NFSEXP_NOREADDIRPLUS)
 		fprintf(fp, "nordirplus,");
 	fprintf(fp, "%spnfs,", (ep->e_flags & NFSEXP_PNFS)? "" : "no_");
+	fprintf(fp, "%swcc,", (ep->e_flags & NFSEXP_NOWCC) ? "no" : "");
 	if (ep->e_flags & NFSEXP_FSID) {
 		fprintf(fp, "fsid=%d,", ep->e_fsid);
 	}
@@ -586,6 +587,10 @@ parseopts(char *cp, struct exportent *ep, int warn, int *had_subtree_opt_ptr)
 			setflags(NFSEXP_PNFS, active, ep);
 		else if (!strcmp(opt, "no_pnfs"))
 			clearflags(NFSEXP_PNFS, active, ep);
+		else if (!strcmp(opt, "wcc"))
+			clearflags(NFSEXP_NOWCC, active, ep);
+		else if (!strcmp(opt, "nowcc"))
+			setflags(NFSEXP_NOWCC, active, ep);
 		else if (strncmp(opt, "anonuid=", 8) == 0) {
 			char *oe;
 			ep->e_anonuid = strtol(opt+8, &oe, 10);
utils/exportfs/exportfs.c +2 -0
index 87582316b086..a2afc80ee6ea 100644
@@ -823,6 +823,8 @@ dump(int verbose, int export_format)
 				c = dumpopt(c, "no_acl");
 			if (ep->e_flags & NFSEXP_PNFS)
 				c = dumpopt(c, "pnfs");
+			if (ep->e_flags & NFSEXP_NOWCC)
+				c = dumpopt(c, "nowcc");
 			if (ep->e_flags & NFSEXP_FSID)
 				c = dumpopt(c, "fsid=%d", ep->e_fsid);
 			if (ep->e_uuid)
utils/exportfs/exports.man +0 -0
index 93092463153b..9213a228bb04 100644
@@ -417,6 +417,20 @@ devices. The default can be explicitly requested with the
 .I no_pnfs
 option.
 
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

utils/exportfs/exports.man devices. The default can be explicitly requested with the · line 427

Quoted 8 lines
utils/exportfs/exports.man +8 -0
+.TP
+.IR nowcc
+This option disables the collection and reporting of WCC (weak cache
+consistency) data in NFSv3 requests.  RFC 1813 recommends that all
+servers report this information to the clients as it allows the client
+to avoid some extra round trips to the server.  In some underlying
+filesystems however, collecting this data can be cost prohibitive and
+atomicity is difficult to guarantee.

By the way, independently of this change, maybe we should by default disable the pre-op attribute on distributed filesystems? As I understand it we're just depending on the i_mutex for atomicity, which can't be right in that case.

Quoted 1 line

--b.

Right. In the case of something like a clustered or network filesystem you really can't guarantee atomicity, so why even try?

So, that is another option...we could add a flags field to the export_operations struct and use one flag to indicate whether to send wcc attrs. That'd give us the ability to add other flags in the future as well.

Quoted 6 lines

The reason I went with the export option is that I didn't really want to add the flags field, and I figured the flexibility of being able to do that on any filesystem might be helpful.

That said, I'm open to doing it that way instead of (or in addition to) the export option if you think it would be better.

Well, it's sounding so far like:

- nobody will want this on a non-distributed filesystem (hard to see what the benefit would be; and it might cost).
- people probably *do* want this on any distributed filesystem.
- we could save users some trouble by not giving them another export option which they might just set wrong.

Quoted 4 lines

Eh, but I think I'm wrong about #2. Dropping the pre-op attribute is arguably necessary for correctness, but dropping the post-op attribute as well isn't necessary and might hurt, right? OK.

Oh, hmmm.

The pre-op attrs are generally pretty lightweight. It just scrapes the info out of the in-core inode. The post-op attrs require a vfs_getattr call though, and that's where the expensive part comes in. I guess I looked at them as a unit. You either want both pre and post attrs or none. The patch disables both when you add the nowcc option.

So I guess this means you're leaning toward a flags field in the export operations instead of an export option? If so, I'm fine with that and I can respin the patch accordingly.

There is probably some micro-optimization in turning them off on local filesystems when you are doing DIO or having it act as a DS for flexfiles, but I doubt it's significant enough to bother with.

Signature · 1 line
Jeff Layton <[email protected]>

A message that is not in the archive (sent off-list or not yet received): CAHQdGtS=ZCnG+YgNZAT0+DqpBQs30OQOZLsvTEDNsxA6x7EUEg@mail.gmail.com

#10 lore raw
Quoted 155 lines · On Thu, 3 Sep 2015 15:27:52 -0700 Trond Myklebust…
Quoted 8 lines · Jeff Layton
Quoted 4 lines · On Thu, 3 Sep 2015 16:32:00 -0400 "J. Bruce Fields" <[email protected]>…
Quoted 3 lines · Jeff Layton

On Thu, 3 Sep 2015 15:42:55 -0400
"J. Bruce Fields" <[email protected]> wrote:

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines · J. Bruce Fields
Quoted 6 lines · Jeff Layton
Quoted 5 lines
Quoted 2 lines · On Thu, 3 Sep 2015 14:31:03 -0400 "J. Bruce Fields" <[email protected]>…
Quoted 1 line · Jeff Layton

This is the userland companion patch for the patch to add a

"nowcc"

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

option to knfsd. This just adds the necessary code to allow

userspace

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

Review

Quoted 11 lines

to set that option.

---
 support/include/nfs/export.h |  3 ++-
 support/nfs/exports.c        |  5 +++++
 utils/exportfs/exportfs.c    |  2 ++
 utils/exportfs/exports.man   | 14 ++++++++++++++
 4 files changed, 23 insertions(+), 1 deletion(-)
(quoted diff) +0 -0

b/support/include/nfs/export.h

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

support/include/nfs/export.h line 27

Quoted 6 lines

index 1194255899bd..68bee5c2f305 100644

support/include/nfs/export.h +0 -0
@@ -26,7 +26,8 @@
 #define  NFSEXP_CROSSMOUNT       0x4000
 #define NFSEXP_NOACL             0x8000 /* reserved for

possible ACL related use */

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
support/include/nfs/export.h +2 -1
 #define NFSEXP_V4ROOT            0x10000
-#define NFSEXP_PNFS            0x20000
+#define NFSEXP_PNFS              0x20000
+#define NFSEXP_NOWCC             0x40000
 /*
  * All flags supported by the kernel before addition of the
  * export_features interface:
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

support/nfs/exports.c putexportent(struct exportent *ep) · line 278

Quoted 8 lines
support/nfs/exports.c +0 -0
index 0aea6f154f09..791b5f756c9c 100644
@@ -276,6 +276,7 @@ putexportent(struct exportent *ep)
  if (ep->e_flags & NFSEXP_NOREADDIRPLUS)
          fprintf(fp, "nordirplus,");
  fprintf(fp, "%spnfs,", (ep->e_flags & NFSEXP_PNFS)? "" :

"no_");

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

support/nfs/exports.c putexportent(struct exportent *ep) · line 279

Quoted 1 line
support/nfs/exports.c +1 -0
+ fprintf(fp, "%swcc,", (ep->e_flags & NFSEXP_NOWCC) ? "no"

: "");

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

support/nfs/exports.c parseopts(char *cp, struct exportent

Quoted 4 lines
support/nfs/exports.c +0 -0
  if (ep->e_flags & NFSEXP_FSID) {
          fprintf(fp, "fsid=%d,", ep->e_fsid);
  }
@@ -586,6 +587,10 @@ parseopts(char *cp, struct exportent

*ep, int warn, int *had_subtree_opt_ptr)

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

Review

Quoted 11 lines
support/nfs/exports.c +4 -0
                  setflags(NFSEXP_PNFS, active, ep);
          else if (!strcmp(opt, "no_pnfs"))
                  clearflags(NFSEXP_PNFS, active, ep);
+         else if (!strcmp(opt, "wcc"))
+                 clearflags(NFSEXP_NOWCC, active, ep);
+         else if (!strcmp(opt, "nowcc"))
+                 setflags(NFSEXP_NOWCC, active, ep);
          else if (strncmp(opt, "anonuid=", 8) == 0) {
                  char *oe;
                  ep->e_anonuid = strtol(opt+8, &oe, 10);
(quoted diff) +0 -0

b/utils/exportfs/exportfs.c

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

utils/exportfs/exportfs.c dump(int verbose, int export_format) · line 829

Quoted 11 lines

index 87582316b086..a2afc80ee6ea 100644

utils/exportfs/exportfs.c +2 -0
@@ -823,6 +823,8 @@ dump(int verbose, int export_format)
                          c = dumpopt(c, "no_acl");
                  if (ep->e_flags & NFSEXP_PNFS)
                          c = dumpopt(c, "pnfs");
+                 if (ep->e_flags & NFSEXP_NOWCC)
+                         c = dumpopt(c, "nowcc");
                  if (ep->e_flags & NFSEXP_FSID)
                          c = dumpopt(c, "fsid=%d",

ep->e_fsid);

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

Review

Quoted 2 lines
utils/exportfs/exportfs.c +0 -0
                  if (ep->e_uuid)
(quoted diff) +0 -0

b/utils/exportfs/exports.man

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

utils/exportfs/exports.man devices. The default can be explicitly

Quoted 4 lines

index 93092463153b..9213a228bb04 100644

utils/exportfs/exports.man +0 -0
@@ -417,6 +417,20 @@ devices. The default can be explicitly

requested with the

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

utils/exportfs/exports.man devices. The default can be explicitly · line 422

Quoted 6 lines
utils/exportfs/exports.man +3 -0
 .I no_pnfs
 option.
 
+.TP
+.IR nowcc
+This option disables the collection and reporting of WCC

(weak cache

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

utils/exportfs/exports.man devices. The default can be explicitly · line 423

Quoted 1 line
utils/exportfs/exports.man +1 -0
+consistency) data in NFSv3 requests.  RFC 1813 recommends

that all

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

utils/exportfs/exports.man devices. The default can be explicitly · line 424

Quoted 1 line
utils/exportfs/exports.man +1 -0
+servers report this information to the clients as it

allows the client

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

utils/exportfs/exports.man devices. The default can be explicitly · line 425

Quoted 1 line
utils/exportfs/exports.man +1 -0
+to avoid some extra round trips to the server.  In some

underlying

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

utils/exportfs/exports.man devices. The default can be explicitly · line 426

Quoted 1 line
utils/exportfs/exports.man +1 -0
+filesystems however, collecting this data can be cost

prohibitive and

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line

utils/exportfs/exports.man devices. The default can be explicitly · line 427

Quoted 1 line
utils/exportfs/exports.man +1 -0
+atomicity is difficult to guarantee.

By the way, independently of this change, maybe we should by default disable the pre-op attribute on distributed filesystems? As I understand it we're just depending on the i_mutex for atomicity,

which

Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
Quoted 3 lines

can't be right in that case.

--b.

Right. In the case of something like a clustered or network filesystem you really can't guarantee atomicity, so why even try?

So, that is another option...we could add a flags field to the export_operations struct and use one flag to indicate whether to send wcc attrs. That'd give us the ability to add other flags in the future as well.

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

The reason I went with the export option is that I didn't really want to add the flags field, and I figured the flexibility of being able to do that on any filesystem might be helpful.

That said, I'm open to doing it that way instead of (or in addition

to)

Quoted 23 lines
Quoted 10 lines
Quoted 1 line

the export option if you think it would be better.

Well, it's sounding so far like:

- nobody will want this on a non-distributed filesystem (hard to see what the benefit would be; and it might cost).
- people probably *do* want this on any distributed filesystem.
- we could save users some trouble by not giving them another export option which they might just set wrong.

Quoted 4 lines

Eh, but I think I'm wrong about #2. Dropping the pre-op attribute is arguably necessary for correctness, but dropping the post-op attribute as well isn't necessary and might hurt, right? OK.

Oh, hmmm.

The pre-op attrs are generally pretty lightweight. It just scrapes the info out of the in-core inode. The post-op attrs require a vfs_getattr call though, and that's where the expensive part comes in. I guess I looked at them as a unit. You either want both pre and post attrs or none. The patch disables both when you add the nowcc option.

Quoted 7 lines
Quoted 3 lines

So I guess this means you're leaning toward a flags field in the export operations instead of an export option? If so, I'm fine with that and I can respin the patch accordingly.

Agreed that this is desirable. Which filesystems are going to turn this on by default?

Good question. I'll certainly do it for NFS in the reexporting code that we're working on, but I'd be a little leery of adding this to others without some OK by their maintainers and some testing. I'd expect that ceph, lustre, gfs2, and ocfs2 would probably be good places to consider enabling it though.

What I'll probably do is add the flag and drop a note to those maintainers mentioning that they may want to consider enabling it. I'll also plan to document this in Documentation/filesystems/nfs/Exporting to help them make an informed decision about it.

Quoted 10 lines
Quoted 3 lines

There is probably some micro-optimization in turning them off on local filesystems when you are doing DIO or having it act as a DS for flexfiles, but I doubt it's significant enough to bother with.

Disagree. There are quite a few workloads where the wcc mainly results in unnecessary cache flushes on the client due to out of order RPC replies. I.e. the sysadmin knows there is only one client accessing the file at a time, but the OOO replies lead to clients believing there are multiple accesses to the same file.

Ahh...apples and oranges. I was referring to the performance impact of doing the actual vfs_getattr call on the underlying fs, not the effect on the client. For filesystems that have a cheap getattr routine, there really is little harm to the server in sending the data along.

OOO replies are a different problem entirely though. It seems like that might be better addressed by a "nowcc" mount option on the client? Having to set that on the server seems a bit too coarse-grained if the admin has to make that decision based on workload.

Signature · 1 line
Jeff Layton <[email protected]>