Skip to contentSkip to conversation

nfs-utils: signed filehandle support

PATCH Superseded Benjamin Coddington 2 participants 4 messages Jan 21 – Jan 22

Subsystems (2)

v2 of 7 Superseded Latest is v7 →

lore
Share & tools
lore
#0 lore raw

v2 cover nfs-utils: signed filehandle support

Here are two patches allowing userspace to set a secret key for kNFSD to sign filehandles, and also set the option to sign filehandles for an export.

The secret key passed to the server is the first 128 bits of a sha1 hash of the contents of a file configured via the nfs.conf server section "fh_key_file". Exports that have the option "sign_fh" set will cause the server to use this key to append an 8-byte siphash of the filehandle onto each filehandle.

This version of the userspace patches correspond with the v2 of the kernel changes which have been posted here:
https://lore.kernel.org/linux-nfs/[email protected]

This work is based on a branch that includes Jeff Layton's patch for min-threads:
https://lore.kernel.org/linux-nfs/[email protected]/

Comments and critique welcomed.

Changes since v1:
- fh_key hash is passed as an optional attr to threads verb
- include missing hash_fh_key_file() function hunk
- wording updates (thanks NeilBrown)

Benjamin Coddington (2):
exportfs: Add support for export option sign_fh nfsdctl/rpc.nfsd: Add support for passing encrypted filehandle key

 configure.ac                 |  4 +-
 nfs.conf                     |  1 +
 support/include/nfs/export.h |  2 +-
 support/include/nfslib.h     |  2 +
 support/nfs/Makefile.am      |  4 +-
 support/nfs/exports.c        |  4 ++
 support/nfs/fh_key_file.c    | 80 ++++++++++++++++++++++++++++++++++++
 systemd/nfs.conf.man         |  1 +
 utils/exportfs/exportfs.c    |  2 +
 utils/exportfs/exports.man   |  9 ++++
 utils/nfsd/nfsd.c            | 16 +++++++-
 utils/nfsd/nfssvc.c          | 26 ++++++++++++
 utils/nfsd/nfssvc.h          |  1 +
 utils/nfsdctl/nfsd_netlink.h |  2 +
 utils/nfsdctl/nfsdctl.c      | 35 ++++++++++++++--
 15 files changed, 179 insertions(+), 10 deletions(-)
 create mode 100644 support/nfs/fh_key_file.c

base-commit: 612e407c46b848932c32be00b835a7b5317e3d08
prerequisite-patch-id: cc4d768b1f6935b3c94ae87bd0389270717bc5b0

Signature · 1 line
2.50.1
#1 lore raw

PATCH v2 1/2 exportfs: Add support for export option sign_fh

If configured with "sign_fh", exports will be flagged to signal that filehandles should be signed with a Message Authentication Code (MAC).

---
 support/include/nfs/export.h | 2 +-
 support/nfs/exports.c        | 4 ++++
 utils/exportfs/exportfs.c    | 2 ++
 utils/exportfs/exports.man   | 9 +++++++++
 4 files changed, 16 insertions(+), 1 deletion(-)
support/include/nfs/export.h +1 -1
index be5867cffc3c..ef3f3e7ea684 100644
@@ -19,7 +19,7 @@
 #define NFSEXP_GATHERED_WRITES	0x0020
 #define NFSEXP_NOREADDIRPLUS	0x0040
 #define NFSEXP_SECURITY_LABEL	0x0080
-/* 0x100 unused */
+#define NFSEXP_SIGN_FH		0x0100
 #define NFSEXP_NOHIDE		0x0200
 #define NFSEXP_NOSUBTREECHECK	0x0400
 #define NFSEXP_NOAUTHNLM	0x0800
support/nfs/exports.c +4 -0
index 21ec6486ba3d..6b4ca87ee957 100644
@@ -310,6 +310,8 @@ putexportent(struct exportent *ep)
 		fprintf(fp, "nordirplus,");
 	if (ep->e_flags & NFSEXP_SECURITY_LABEL)
 		fprintf(fp, "security_label,");
+	if (ep->e_flags & NFSEXP_SIGN_FH)
+		fprintf(fp, "sign_fh,");
 	fprintf(fp, "%spnfs,", (ep->e_flags & NFSEXP_PNFS)? "" : "no_");
 	if (ep->e_flags & NFSEXP_FSID) {
 		fprintf(fp, "fsid=%d,", ep->e_fsid);
@@ -676,6 +678,8 @@ parseopts(char *cp, struct exportent *ep, int *had_subtree_opt_ptr)
 			setflags(NFSEXP_NOREADDIRPLUS, active, ep);
 		else if (!strcmp(opt, "security_label"))
 			setflags(NFSEXP_SECURITY_LABEL, active, ep);
+		else if (!strcmp(opt, "sign_fh"))
+			setflags(NFSEXP_SIGN_FH, active, ep);
 		else if (!strcmp(opt, "nohide"))
 			setflags(NFSEXP_NOHIDE, active, ep);
 		else if (!strcmp(opt, "hide"))
utils/exportfs/exportfs.c +2 -0
index 748c38e3e966..54ce62c5ce9a 100644
@@ -718,6 +718,8 @@ dump(int verbose, int export_format)
 				c = dumpopt(c, "nordirplus");
 			if (ep->e_flags & NFSEXP_SECURITY_LABEL)
 				c = dumpopt(c, "security_label");
+			if (ep->e_flags & NFSEXP_SIGN_FH)
+				c = dumpopt(c, "sign_fh");
 			if (ep->e_flags & NFSEXP_NOACL)
 				c = dumpopt(c, "no_acl");
 			if (ep->e_flags & NFSEXP_PNFS)
utils/exportfs/exports.man +9 -0
index 39dc30fb8290..bd6669f431ba 100644
@@ -351,6 +351,15 @@ file.  If you put neither option,
 .B exportfs
 will warn you that the change has occurred.
 
+.TP
+.IR sign_fh
+This option enforces signing filehandles on the export.  If the server has
+been configured with a secret key for such purpose, filehandles will include
+a hash to verify the filehandle was created by the server in order to guard
+against filehandle guessing attacks which can bypass path-name based access
+restrictions.  Note that for NFSv2 and NFSv3, some exported filesystems may
+exceed the maximum filehandle size when the signing hash is added.
+
 .TP
 .IR insecure_locks
 .TP
Signature · 1 line
2.50.1
#3 lore raw

PATCH v2 2/2 nfsdctl/rpc.nfsd: Add support for passing encrypted filehandle key

In reply to Benjamin Coddington

If fh-key-file=<path> is set in the nfsd section of nfs.conf, the "nfsdctl autostart" command will hash the contents of the file with libuuid's uuid_generate_sha1 and send the first 16 bytes into the kernel via NFSD_CMD_FH_KEY_SET.

If fh-key-file=<path> is set in the nfsd section nfs.conf, rpc.nfsd will also hash the contents of the file with libuuid's uuid_generate_sha1 and write the resulting uuid into nfsdfs's ./fh_key_file entry.

A compatible kernel can use this key to sign filehandles.

---
 configure.ac                 |  4 +-
 nfs.conf                     |  1 +
 support/include/nfslib.h     |  2 +
 support/nfs/Makefile.am      |  4 +-
 support/nfs/fh_key_file.c    | 80 ++++++++++++++++++++++++++++++++++++
 systemd/nfs.conf.man         |  1 +
 utils/nfsd/nfsd.c            | 16 +++++++-
 utils/nfsd/nfssvc.c          | 26 ++++++++++++
 utils/nfsd/nfssvc.h          |  1 +
 utils/nfsdctl/nfsd_netlink.h |  2 +
 utils/nfsdctl/nfsdctl.c      | 35 ++++++++++++++--
 11 files changed, 163 insertions(+), 9 deletions(-)
 create mode 100644 support/nfs/fh_key_file.c
configure.ac +2 -2
index 33866e869666..db027ddbd995 100644
@@ -265,9 +265,9 @@ AC_ARG_ENABLE(nfsdctl,
 		AC_CHECK_DECLS([NFSD_A_SERVER_MIN_THREADS], , ,
 			       [#include <linux/nfsd_netlink.h>])
 
-		# ensure we have the pool-mode commands
+		# ensure we have the fh-key commands
 		AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#include <linux/nfsd_netlink.h>]],
-				                   [[int foo = NFSD_CMD_POOL_MODE_GET;]])],
+				                   [[int foo = NFSD_CMD_FH_KEY_SET;]])],
 				   [AC_DEFINE([USE_SYSTEM_NFSD_NETLINK_H], 1,
 					      ["Use system's linux/nfsd_netlink.h"])])
 		AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#include <linux/lockd_netlink.h>]],
nfs.conf +1 -0
index 3cca68c3530d..39068c19d7df 100644
@@ -76,6 +76,7 @@
 # vers4.2=y
 rdma=y
 rdma-port=20049
+# fh-key-file=/etc/nfs_fh.key
 
 [statd]
 # debug=0
support/include/nfslib.h +2 -0
index eff2a486307f..c8601a156cba 100644
@@ -22,6 +22,7 @@
 #include <paths.h>
 #include <rpcsvc/nfs_prot.h>
 #include <nfs/nfs.h>
+#include <uuid/uuid.h>
 #include "xlog.h"
 
 #ifndef _PATH_EXPORTS
@@ -132,6 +133,7 @@ struct rmtabent *	fgetrmtabent(FILE *fp, int log, long *pos);
 void			fputrmtabent(FILE *fp, struct rmtabent *xep, long *pos);
 void			fendrmtabent(FILE *fp);
 void			frewindrmtabent(FILE *fp);
+int				hash_fh_key_file(const char *fh_key_file, uuid_t hash);
 
 _Bool state_setup_basedir(const char *, const char *);
 int setup_state_path_names(const char *, const char *, const char *, const char *, struct state_paths *);
support/nfs/Makefile.am +2 -2
index 2e1577cc12df..775bccc6c5ea 100644
@@ -7,8 +7,8 @@ libnfs_la_SOURCES = exports.c rmtab.c xio.c rpcmisc.c rpcdispatch.c \
 		   xcommon.c wildmat.c mydaemon.c \
 		   rpc_socket.c getport.c \
 		   svc_socket.c cacheio.c closeall.c nfs_mntent.c \
-		   svc_create.c atomicio.c strlcat.c strlcpy.c
-libnfs_la_LIBADD = libnfsconf.la
+		   svc_create.c atomicio.c strlcat.c strlcpy.c fh_key_file.c
+libnfs_la_LIBADD = libnfsconf.la -luuid
 libnfs_la_CPPFLAGS = $(AM_CPPFLAGS) $(CPPFLAGS) -I$(top_srcdir)/support/reexport
 
 libnfsconf_la_SOURCES = conffile.c xlog.c
support/nfs/fh_key_file.c new file +80 -0

80 lines not on this page · raw message

systemd/nfs.conf.man +1 -0
index 484de2c086db..1fb5653042d3 100644
@@ -176,6 +176,7 @@ Recognized values:
 .BR vers4.1 ,
 .BR vers4.2 ,
 .BR rdma ,
+.BR fh-key-file ,
 
 Version and protocol values are Boolean values as described above,
 and are also used by
utils/nfsd/nfsd.c +15 -1
index a405649976c2..08a7eaed4906 100644
@@ -69,7 +69,7 @@ int
 main(int argc, char **argv)
 {
 	int	count = NFSD_NPROC, c, i, j, error = 0, portnum, fd, found_one;
-	char *p, *progname, *port, *rdma_port = NULL;
+	char *p, *progname, *port, *fh_key_file, *rdma_port = NULL;
 	char **haddr = NULL;
 	char *scope = NULL;
 	int hcounter = 0;
@@ -134,6 +134,8 @@ main(int argc, char **argv)
 		}
 	}
 
+	fh_key_file = conf_get_str("nfsd", "fh-key-file");
+
 	/* We assume the kernel will default all minor versions to 'on',
 	 * and allow the config file to disable some.
 	 */
@@ -380,6 +382,18 @@ main(int argc, char **argv)
 		goto set_threads;
 	}
 
+	if (fh_key_file) {
+		error = nfssvc_setfh_key(fh_key_file);
+		if (error) {
+			/* Common case: key is already set */
+			if (error != -EEXIST) {
+				xlog(L_ERROR, "Unable to set fh_key_file, error %d", error);
+				goto out;
+			}
+			xlog(L_NOTICE, "fh_key_file already set");
+		}
+	}
+
 	/*
 	 * Must set versions before the fd's so that the right versions get
 	 * registered with rpcbind. Note that on older kernels w/o the right
utils/nfsd/nfssvc.c +26 -0
index 9650cecee986..4f3088b74285 100644
@@ -34,6 +34,7 @@
 #define NFSD_PORTS_FILE   NFSD_FS_DIR "/portlist"
 #define NFSD_VERS_FILE    NFSD_FS_DIR "/versions"
 #define NFSD_THREAD_FILE  NFSD_FS_DIR "/threads"
+#define NFSD_FH_KEY_FILE  NFSD_FS_DIR "/fh_key"
 
 /*
  * declaring a common static scratch buffer here keeps us from having to
@@ -414,6 +415,31 @@ out:
 	return;
 }
 
+int
+nfssvc_setfh_key(const char *fh_key_file)
+{
+	char uuid_str[37];
+	uuid_t hash;
+	int fd, ret;
+
+	ret = hash_fh_key_file(fh_key_file, hash);
+	if (ret)
+		return ret;
+
+	uuid_unparse(hash, uuid_str);
+
+	fd = open(NFSD_FH_KEY_FILE, O_WRONLY);
+	if (fd < 0)
+		return fd;
+
+	ret = write(fd, uuid_str, 37);
+	close(fd);
+	if (ret < 0)
+		return -errno;
+
+	return 0;
+}
+
 int
 nfssvc_threads(const int nrservs)
 {
utils/nfsd/nfssvc.h +1 -0
index 4d53af1a8bc3..463110cac804 100644
@@ -30,3 +30,4 @@ void	nfssvc_setvers(unsigned int ctlbits, unsigned int minorvers4,
 		       unsigned int minorvers4set, int force4dot0);
 int	nfssvc_threads(int nrservs);
 void	nfssvc_get_minormask(unsigned int *mask);
+int nfssvc_setfh_key(const char *fh_key_file);
utils/nfsdctl/nfsd_netlink.h +2 -0
index 887cbd12b695..f7e7f5576774 100644
@@ -34,6 +34,7 @@ enum {
 	NFSD_A_SERVER_GRACETIME,
 	NFSD_A_SERVER_LEASETIME,
 	NFSD_A_SERVER_SCOPE,
+	NFSD_A_SERVER_FH_KEY,
 
 	__NFSD_A_SERVER_MAX,
 	NFSD_A_SERVER_MAX = (__NFSD_A_SERVER_MAX - 1)
@@ -88,6 +89,7 @@ enum {
 	NFSD_CMD_LISTENER_GET,
 	NFSD_CMD_POOL_MODE_SET,
 	NFSD_CMD_POOL_MODE_GET,
+	NFSD_CMD_FH_KEY_SET,
 
 	__NFSD_CMD_MAX,
 	NFSD_CMD_MAX = (__NFSD_CMD_MAX - 1)
utils/nfsdctl/nfsdctl.c +31 -4
index 67cf27b04ca3..1d673d7bdc90 100644
@@ -29,6 +29,7 @@
 
 #include <readline/readline.h>
 #include <readline/history.h>
+#include <uuid/uuid.h>
 
 #ifdef USE_SYSTEM_NFSD_NETLINK_H
 #include <linux/nfsd_netlink.h>
@@ -42,6 +43,7 @@
 #include "lockd_netlink.h"
 #endif
 
+#include "nfslib.h"
 #include "nfsdctl.h"
 #include "conffile.h"
 #include "xlog.h"
@@ -524,8 +526,10 @@ out:
 }
 
 static int threads_doit(struct nl_sock *sock, int cmd, int grace, int lease,
-			int pool_count, int *pool_threads, char *scope, int minthreads)
+			int pool_count, int *pool_threads, char *scope, int minthreads,
+			uuid_t fh_key)
 {
+	struct nl_data *fh_key_data = NULL;
 	struct genlmsghdr *ghdr;
 	struct nlmsghdr *nlh;
 	struct nl_msg *msg;
@@ -550,6 +554,15 @@ static int threads_doit(struct nl_sock *sock, int cmd, int grace, int lease,
 		if (minthreads >= 0)
 			nla_put_u32(msg, NFSD_A_SERVER_MIN_THREADS, minthreads);
 #endif
+		if (!uuid_is_null(fh_key)) {
+			fh_key_data = nl_data_alloc(fh_key, sizeof(uuid_t));
+			if (!fh_key_data) {
+				xlog(L_ERROR, "failed to allocate netlink data");
+				ret = 1;
+				goto out;
+			}
+			nla_put_data(msg, NFSD_A_SERVER_FH_KEY, fh_key_data);
+		}
 		for (i = 0; i < pool_count; ++i)
 			nla_put_u32(msg, NFSD_A_SERVER_THREADS, pool_threads[i]);
 	}
@@ -584,6 +597,8 @@ static int threads_doit(struct nl_sock *sock, int cmd, int grace, int lease,
 out_cb:
 	nl_cb_put(cb);
 out:
+	if (fh_key_data)
+		nl_data_free(fh_key_data);
 	nlmsg_free(msg);
 	return ret;
 }
@@ -612,6 +627,7 @@ static int threads_func(struct nl_sock *sock, int argc, char **argv)
 	int *pool_threads = NULL;
 	int minthreads = -1;
 	int opt, pools = 0;
+	uuid_t fh_key;
 
 	optind = 1;
 	while ((opt = getopt_long(argc, argv, "h", threads_options, NULL)) != -1) {
@@ -654,7 +670,9 @@ static int threads_func(struct nl_sock *sock, int argc, char **argv)
 			}
 		}
 	}
-	return threads_doit(sock, cmd, 0, 0, pools, pool_threads, NULL, minthreads);
+	uuid_clear(fh_key);
+	return threads_doit(sock, cmd, 0, 0, pools, pool_threads, NULL,
+				minthreads, fh_key);
 }
 
 /*
@@ -1611,8 +1629,9 @@ static int autostart_func(struct nl_sock *sock, int argc, char ** argv)
 	int *threads, grace, lease, idx, ret, opt, pools, minthreads;
 	struct conf_list *thread_str;
 	struct conf_list_node *n;
-	char *scope, *pool_mode;
+	char *scope, *pool_mode, *fh_key_file;
 	bool failed_listeners = false;
+	uuid_t fh_key;
 
 	optind = 1;
 	while ((opt = getopt_long(argc, argv, "h", help_only_options, NULL)) != -1) {
@@ -1687,12 +1706,20 @@ static int autostart_func(struct nl_sock *sock, int argc, char ** argv)
 		threads[0] = DEFAULT_AUTOSTART_THREADS;
 	}
 
+	uuid_clear(fh_key);
+	fh_key_file = conf_get_str("nfsd", "fh-key-file");
+	if (fh_key_file) {
+		ret = hash_fh_key_file(fh_key_file, fh_key);
+		if (ret)
+			return ret;
+	}
+
 	lease = conf_get_num("nfsd", "lease-time", 0);
 	scope = conf_get_str("nfsd", "scope");
 	minthreads = conf_get_num("nfsd", "min-threads", 0);
 
 	ret = threads_doit(sock, NFSD_CMD_THREADS_SET, grace, lease, pools,
-			   threads, scope, minthreads);
+			   threads, scope, minthreads, fh_key);
 out:
 	free(threads);
 	return ret;
Signature · 1 line
2.50.1
maintainer
#2 lore raw

Re: [PATCH v2 1/2] exportfs: Add support for export option sign_fh

In reply to Benjamin Coddington
Quoted 16 lines · Benjamin Coddington

If configured with "sign_fh", exports will be flagged to signal that filehandles should be signed with a Message Authentication Code (MAC).

---
 support/include/nfs/export.h | 2 +-
 support/nfs/exports.c        | 4 ++++
 utils/exportfs/exportfs.c    | 2 ++
 utils/exportfs/exports.man   | 9 +++++++++
 4 files changed, 16 insertions(+), 1 deletion(-)
support/include/nfs/export.h +0 -0
index be5867cffc3c..ef3f3e7ea684 100644

support/include/nfs/export.h line 25

Quoted 8 lines
support/include/nfs/export.h +1 -1
@@ -19,7 +19,7 @@
 #define NFSEXP_GATHERED_WRITES	0x0020
 #define NFSEXP_NOREADDIRPLUS	0x0040
 #define NFSEXP_SECURITY_LABEL	0x0080
-/* 0x100 unused */
+#define NFSEXP_SIGN_FH		0x0100
 #define NFSEXP_NOHIDE		0x0200
 #define NFSEXP_NOSUBTREECHECK	0x0400
 #define NFSEXP_NOAUTHNLM	0x0800

Side note: it would be good to convert nfs-utils to use UAPI header for this instead of relying on a separate copy in the tree.

Quoted 45 lines
support/nfs/exports.c +4 -0
index 21ec6486ba3d..6b4ca87ee957 100644
@@ -310,6 +310,8 @@ putexportent(struct exportent *ep)
 		fprintf(fp, "nordirplus,");
 	if (ep->e_flags & NFSEXP_SECURITY_LABEL)
 		fprintf(fp, "security_label,");
+	if (ep->e_flags & NFSEXP_SIGN_FH)
+		fprintf(fp, "sign_fh,");
 	fprintf(fp, "%spnfs,", (ep->e_flags & NFSEXP_PNFS)? "" : "no_");
 	if (ep->e_flags & NFSEXP_FSID) {
 		fprintf(fp, "fsid=%d,", ep->e_fsid);
@@ -676,6 +678,8 @@ parseopts(char *cp, struct exportent *ep, int *had_subtree_opt_ptr)
 			setflags(NFSEXP_NOREADDIRPLUS, active, ep);
 		else if (!strcmp(opt, "security_label"))
 			setflags(NFSEXP_SECURITY_LABEL, active, ep);
+		else if (!strcmp(opt, "sign_fh"))
+			setflags(NFSEXP_SIGN_FH, active, ep);
 		else if (!strcmp(opt, "nohide"))
 			setflags(NFSEXP_NOHIDE, active, ep);
 		else if (!strcmp(opt, "hide"))
utils/exportfs/exportfs.c +2 -0
index 748c38e3e966..54ce62c5ce9a 100644
@@ -718,6 +718,8 @@ dump(int verbose, int export_format)
 				c = dumpopt(c, "nordirplus");
 			if (ep->e_flags & NFSEXP_SECURITY_LABEL)
 				c = dumpopt(c, "security_label");
+			if (ep->e_flags & NFSEXP_SIGN_FH)
+				c = dumpopt(c, "sign_fh");
 			if (ep->e_flags & NFSEXP_NOACL)
 				c = dumpopt(c, "no_acl");
 			if (ep->e_flags & NFSEXP_PNFS)
utils/exportfs/exports.man +2 -0
index 39dc30fb8290..bd6669f431ba 100644
@@ -351,6 +351,15 @@ file.  If you put neither option,
 .B exportfs
 will warn you that the change has occurred.
 
+.TP
+.IR sign_fh

utils/exportfs/exports.man file. If you put neither option, · line 365

Quoted 10 lines
utils/exportfs/exports.man +7 -0
+This option enforces signing filehandles on the export.  If the server has
+been configured with a secret key for such purpose, filehandles will include
+a hash to verify the filehandle was created by the server in order to guard
+against filehandle guessing attacks which can bypass path-name based access
+restrictions.  Note that for NFSv2 and NFSv3, some exported filesystems may
+exceed the maximum filehandle size when the signing hash is added.
+
 .TP
 .IR insecure_locks
 .TP

Looks good overall though.

Signature · 1 line
Jeff Layton <[email protected]>