exportfs/exportd/mountd: allow them to use netlink for up/downcalls
Subsystems (2)
Jeff Layton
maintainerv2 cover exportfs/exportd/mountd: allow them to use netlink for up/downcalls
Minor revision to rebase onto recent upstream changes. Original cover letter follows:
This adds support for the new netlink-based upcalls and downcalls in mountd, exportd and exportfs. With this, mountd is no longer reliant on /proc for sunrpc cache upcalls.
There are also a few bugfixes and cleanups for existing code and documentation too.
---
Changes in v2:
- Consolidate UAPI header updates into one patch
- Rebase onto nfs-utils-2-9-1-rc2
- Link to v1: https://lore.kernel.org/r/[email protected]
---
Jeff Layton (16):
nfsdctl: move *_netlink.h to support/include/
support/export: remove unnecessary static variables in nfsd_fh expkey lookup
exportfs: remove obsolete legacy mode documentation from manpage support/include: update netlink headers for all cache upcalls
build: add libnl3 and netlink header support for exportd and mountd
xlog: claim D_FAC3 as D_NETLINK
exportd/mountd: add netlink support for svc_export cache exportd/mountd: add netlink support for the nfsd.fh cache exportd/mountd: add netlink support for the auth.unix.ip cache exportd/mountd: add netlink support for the auth.unix.gid cache mountd/exportd: only use /proc interfaces if netlink setup fails support/export: check for pending requests after opening netlink sockets exportd/mountd: use cache type from notifications to target scanning
exportfs: add netlink support for cache flush with /proc fallback
exportfs: use netlink to probe kernel support, skip export_test mountd/exportd/exportfs: add --no-netlink option to disable netlink
configure.ac | 33 +-
support/export/Makefile.am | 5 +-
support/export/cache.c | 1892 +++++++++++++++++---
support/export/cache_flush.c | 166 ++
support/include/Makefile.am | 7 +-
{utils/nfsdctl => support/include}/lockd_netlink.h | 0
support/include/nfsd_netlink.h | 240 +++
support/include/sunrpc_netlink.h | 84 +
support/include/xlog.h | 2 +-
support/nfs/cacheio.c | 49 +-
utils/exportd/Makefile.am | 2 +-
utils/exportd/exportd.c | 10 +-
utils/exportd/exportd.man | 12 +-
utils/exportfs/Makefile.am | 6 +-
utils/exportfs/exportfs.c | 55 +-
utils/exportfs/exportfs.man | 79 +-
utils/mountd/Makefile.am | 2 +-
utils/mountd/mountd.c | 9 +-
utils/mountd/mountd.man | 9 +
utils/nfsdctl/nfsd_netlink.h | 99 -
20 files changed, 2350 insertions(+), 411 deletions(-)---
base-commit: a06a3251c2eb1316f781149f8b7f9acd9d41e7fc
change-id: 20260316-exportd-netlink-a53bf66ae034
Best regards,
Signature · 1 line
Jeff Layton <[email protected]>
Jeff Layton
maintainerPATCH v2 1/16 nfsdctl: move *_netlink.h to support/include/
Move lockd_netlink.h and nfsd_netlink.h from utils/nfsdctl/ to support/include/ so they are available to other components.
The support/include/ directory is already in the default include path (via AC_CONFIG_HEADERS), so no source-level #include changes are needed.
---
support/include/Makefile.am | 6 ++++--
{utils/nfsdctl => support/include}/lockd_netlink.h | 0
{utils/nfsdctl => support/include}/nfsd_netlink.h | 0
3 files changed, 4 insertions(+), 2 deletions(-)support/include/Makefile.am +4 -2
@@ -4,13 +4,16 @@ SUBDIRS = nfs rpcsvc sys noinst_HEADERS = \ cld.h \ + conffile.h \ exportfs.h \ ha-callout.h \ junction.h \ + lockd_netlink.h \ misc.h \ nfs_mntent.h \ nfs_paths.h \ nfs_ucred.h \ + nfsd_netlink.h \ nfsd_path.h \ nfslib.h \ nfsrpc.h \
@@ -26,7 +29,6 @@ noinst_HEADERS = \ xlog.h \ xmalloc.h \ xcommon.h \ - xstat.h \ - conffile.h + xstat.h MAINTAINERCLEANFILES = Makefile.in
support/include/lockd_netlink.h renamed from utils/nfsdctl/lockd_netlink.h +0 -0
support/include/nfsd_netlink.h renamed from utils/nfsdctl/nfsd_netlink.h +0 -0
Signature · 1 line
2.53.0
Jeff Layton
maintainerPATCH v2 2/16 support/export: remove unnecessary static variables in nfsd_fh expkey lookup
In reply to Jeff LaytonThe prev and mnt variables in the CROSSMOUNT submount iteration loop only need to persist across iterations of the inner for loop, not across calls to the function. Convert them from static to local variables scoped to the outer for loop.
--- support/export/cache.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
support/export/cache.c +3 -3
@@ -810,7 +810,10 @@ static int nfsd_handle_fh(int f, char *bp, int blen)/* Now determine export point for this fsid/domain */ for (i=0 ; i <MCL_MAXTYPES ; i++) { +nfs_export *prev =NULL ;nfs_export *next_exp; +void *mnt =NULL ; +for (exp = exportlist[i].p_head; exp; exp = next_exp) {char *path;
@@ -820,9 +823,6 @@ static int nfsd_handle_fh(int f, char *bp, int blen) }if (exp->m_export.e_flags &NFSEXP_CROSSMOUNT ) { -static nfs_export *prev =NULL ; -static void *mnt =NULL ; -if (prev == exp) {/* try a submount */ path =next_mnt (&mnt, exp->m_export.e_path);
Signature · 1 line
2.53.0
Jeff Layton
maintainerPATCH v2 3/16 exportfs: remove obsolete legacy mode documentation from manpage
In reply to Jeff LaytonThe exportfs manpage described a "legacy mode" for 2.4 and earlier kernels that used the nfsservctl() syscall to inject exports directly into the kernel. This syscall was removed from the kernel in Linux 3.1 and the corresponding nfs-utils code was dropped long ago. Remove the outdated text and simplify the description.
--- utils/exportfs/exportfs.man | 29 ++--------------------------- 1 file changed, 2 insertions(+), 27 deletions(-)
utils/exportfs/exportfs.man +2 -27
@@ -53,39 +53,14 @@ by using the command. .PP .B exportfs -and its partner program -.B rpc.mountd -work in one of two modes: a legacy mode which applies to 2.4 and -earlier versions of the Linux kernel, and a new mode which applies to -2.6 and later versions, providing the -.B nfsd -virtual filesystem has been mounted at -.I /proc/fs/nfsd -or -.IR /proc/fs/nfs . -On 2.6 kernels, if this filesystem is not mounted, the legacy mode is used. -.PP -In the new mode, -.B exportfs -does not give any information to the kernel, but provides it only to +does not give any information to the kernel directly, but provides it +only to .B rpc.mountd through the .I /var/lib/nfs/etab file. .B rpc.mountd then manages kernel requests for information about exports, as needed. -.PP -In the legacy mode, -exports which identify a specific host, rather than a subnet or netgroup, -are entered directly into the kernel's export table, -as well as being written to -.IR /var/lib/nfs/etab . -Further, exports listed in -.I /var/lib/nfs/rmtab -which match a non host-specific export request will cause an -appropriate export entry for the host given in -.I rmtab -to be added to the kernel's export table. .SH OPTIONS .TP .B \-d kind " or " \-\-debug kind
Signature · 1 line
2.53.0
Jeff Layton
maintainerPATCH v2 4/16 support/include: update netlink headers for all cache upcalls
In reply to Jeff LaytonSync the bundled nfsd_netlink.h with the current kernel UAPI header. This brings in:
- NFSD_CACHE_TYPE_EXPKEY in the cache-type enum
- NFSD_A_SVC_EXPORT_FLAGS and NFSD_A_SVC_EXPORT_FSID attributes
- NFSD_A_EXPKEY_* and NFSD_A_EXPKEY_REQS_* attribute enums
- NFSD_CMD_EXPKEY_GET_REQS and NFSD_CMD_EXPKEY_SET_REQS commands
- NFSD_CMD_CACHE_FLUSH command
Add a new bundled sunrpc_netlink.h header for the sunrpc genl family, which handles auth.unix.ip and auth.unix.gid cache upcalls via netlink.
--- support/include/Makefile.am | 1 + support/include/nfsd_netlink.h | 141 +++++++++++++++++++++++++++++++++++++++ support/include/sunrpc_netlink.h | 84 +++++++++++++++++++++++ 3 files changed, 226 insertions(+)
support/include/Makefile.am +1 -0
@@ -15,6 +15,7 @@ noinst_HEADERS = \ nfs_ucred.h \ nfsd_netlink.h \ nfsd_path.h \ + sunrpc_netlink.h \ nfslib.h \ nfsrpc.h \ nls.h \
support/include/nfsd_netlink.h +141 -0
162 lines not on this page · raw message
support/include/sunrpc_netlink.h new file +84 -0
84 lines not on this page · raw message
Signature · 1 line
2.53.0
Jeff Layton
maintainerPATCH v2 5/16 build: add libnl3 and netlink header support for exportd and mountd
In reply to Jeff LaytonAdd libnl3 dependency and netlink header detection for exportd and mountd builds. Update the nfsd_netlink.h sentinel to check for NFSD_A_EXPKEY_SEQNO to ensure the system header includes expkey support. Add AC_COMPILE_IFELSE check for sunrpc_netlink.h using SUNRPC_CMD_CACHE_NOTIFY as sentinel.
--- configure.ac | 33 +++++++++++++++++++-------------- support/export/Makefile.am | 3 ++- utils/exportd/Makefile.am | 2 +- utils/mountd/Makefile.am | 2 +- 4 files changed, 23 insertions(+), 17 deletions(-)
configure.ac +19 -14
@@ -252,27 +252,32 @@ AC_ARG_ENABLE(nfsdcltrack, enable_nfsdcltrack=$enableval, enable_nfsdcltrack="no") +PKG_CHECK_MODULES(LIBNL3, libnl-3.0 >= 3.1) +PKG_CHECK_MODULES(LIBNLGENL3, libnl-genl-3.0 >= 3.1) + +AC_CHECK_HEADERS(linux/nfsd_netlink.h) + +# ensure we have the expkey attributes +AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#include <linux/nfsd_netlink.h>]], + [[int foo = NFSD_A_EXPKEY_SEQNO;]])], + [AC_DEFINE([USE_SYSTEM_NFSD_NETLINK_H], 1, + ["Use system's linux/nfsd_netlink.h"])]) +AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#include <linux/lockd_netlink.h>]], + [[int foo = LOCKD_CMD_SERVER_GET;]])], + [AC_DEFINE([USE_SYSTEM_LOCKD_NETLINK_H], 1, + ["Use system's linux/lockd_netlink.h"])]) +AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#include <linux/sunrpc_netlink.h>]], + [[int foo = SUNRPC_CMD_CACHE_NOTIFY;]])], + [AC_DEFINE([USE_SYSTEM_SUNRPC_NETLINK_H], 1, + ["Use system's linux/sunrpc_netlink.h"])]) + AC_ARG_ENABLE(nfsdctl, [AS_HELP_STRING([--disable-nfsdctl],[disable nfsdctl program for controlling nfsd@<:@default=no@:>@])], enable_nfsdctl=$enableval, enable_nfsdctl="yes") AM_CONDITIONAL(CONFIG_NFSDCTL, [test "$enable_nfsdctl" = "yes" ]) if test "$enable_nfsdctl" = yes; then - PKG_CHECK_MODULES(LIBNL3, libnl-3.0 >= 3.1) - PKG_CHECK_MODULES(LIBNLGENL3, libnl-genl-3.0 >= 3.1) PKG_CHECK_MODULES(LIBREADLINE, readline) - AC_CHECK_HEADERS(linux/nfsd_netlink.h) - - # ensure we have the MIN_THREADS attribute - AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#include <linux/nfsd_netlink.h>]], - [[int foo = NFSD_A_SERVER_MIN_THREADS; - int bar = NFSD_A_SERVER_FH_KEY;]])], - [AC_DEFINE([USE_SYSTEM_NFSD_NETLINK_H], 1, - ["Use system's linux/nfsd_netlink.h"])]) - AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#include <linux/lockd_netlink.h>]], - [[int foo = LOCKD_CMD_SERVER_GET;]])], - [AC_DEFINE([USE_SYSTEM_LOCKD_NETLINK_H], 1, - ["Use system's linux/lockd_netlink.h"])]) fi AC_ARG_ENABLE(nfsv4server,
support/export/Makefile.am +2 -1
@@ -14,7 +14,8 @@ libexport_a_SOURCES = client.c export.c hostname.c \ xtab.c mount_clnt.c mount_xdr.c \ cache.c auth.c v4root.c fsloc.c \ v4clients.c -libexport_a_CPPFLAGS = $(AM_CPPFLAGS) $(CPPFLAGS) -I$(top_srcdir)/support/reexport +libexport_a_CPPFLAGS = $(AM_CPPFLAGS) $(CPPFLAGS) -I$(top_srcdir)/support/reexport \ + $(LIBNL3_CFLAGS) $(LIBNLGENL3_CFLAGS) BUILT_SOURCES = $(GENFILES)
utils/exportd/Makefile.am +1 -1
@@ -18,7 +18,7 @@ exportd_LDADD = ../../support/export/libexport.a \ ../../support/misc/libmisc.a \ ../../support/reexport/libreexport.a \ $(OPTLIBS) $(LIBBLKID) $(LIBPTHREAD) \ - -luuid + -luuid $(LIBNL3_LIBS) $(LIBNLGENL3_LIBS) exportd_CPPFLAGS = $(AM_CPPFLAGS) $(CPPFLAGS) \ -I$(top_srcdir)/support/export
utils/mountd/Makefile.am +1 -1
@@ -20,7 +20,7 @@ mountd_LDADD = ../../support/export/libexport.a \ ../../support/reexport/libreexport.a \ $(OPTLIBS) \ $(LIBBSD) $(LIBWRAP) $(LIBNSL) $(LIBBLKID) -luuid $(LIBTIRPC) \ - $(LIBPTHREAD) + $(LIBPTHREAD) $(LIBNL3_LIBS) $(LIBNLGENL3_LIBS) mountd_CPPFLAGS = $(AM_CPPFLAGS) $(CPPFLAGS) \ -I$(top_builddir)/support/include \
Signature · 1 line
2.53.0
Jeff Layton
maintainerPATCH v2 6/16 xlog: claim D_FAC3 as D_NETLINK
In reply to Jeff LaytonNothing uses the D_FAC* debugging bits, so claim D_FAC3 as D_NETLINK for logging about netlink activity.
--- support/include/xlog.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
support/include/xlog.h +1 -1
@@ -25,7 +25,7 @@#define D_GENERAL 0x0001/* general debug info */ #define D_CALL 0x0002#define D_AUTH 0x0004 -#define D_FAC3 0x0008 +#define D_NETLINK 0x0008#define D_FAC4 0x0010#define D_FAC5 0x0020#define D_PARSE 0x0040
Signature · 1 line
2.53.0
Jeff Layton
maintainerPATCH v2 7/16 exportd/mountd: add netlink support for svc_export cache
In reply to Jeff LaytonAdd netlink-based svc_export cache handling alongside the existing procfs channel mechanism. When the kernel sends an NFSD_CMD_SVC_EXPORT_NOTIFY multicast notification on the "exportd" genetlink group, userspace responds by issuing NFSD_CMD_SVC_EXPORT_GET_REQS to retrieve all pending export requests, resolving each one via lookup_export(), and sending back the results with NFSD_CMD_SVC_EXPORT_SET_REQS.
Two netlink sockets are used: a non-blocking notification socket that joins the multicast group and is integrated into the existing select() loop, and a blocking command socket for the GET_REQS dump and SET_REQS response. The notification socket is added to the fd_set in cache_set_fds() and checked in cache_process_req().
If the kernel does not support the nfsd genetlink family or the "exportd" multicast group (older kernels), the netlink setup silently fails and the daemon falls back to procfs-only operation. Both mechanisms can coexist safely since the kernel's CACHE_PENDING check prevents duplicate processing.
--- support/export/cache.c | 560 ++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 558 insertions(+), 2 deletions(-)
support/export/cache.c +558 -2
@@ -36,6 +36,18 @@#include "reexport.h" #include "fsloc.h" +#include <netlink/genl/genl.h> +#include <netlink/genl/ctrl.h> +#include <netlink/msg.h> +#include <netlink/attr.h> +#include <linux/netlink.h> + +#ifdef USE_SYSTEM_NFSD_NETLINK_H +#include <linux/nfsd_netlink.h> +#else +#include "nfsd_netlink.h" +#endif +#ifdef USE_BLKID #include "blkid/blkid.h" #endif
@@ -1052,6 +1064,541 @@ static void write_xprtsec(char **bp, int *blen, struct exportent *ep)qword_addint (bp, blen, p->info->number); } +/* +* Netlink-based svc_export cache support. +* +* The kernel can notify userspace of pending export cache requests via +* the "exportd" genetlink multicast group. We listen for +* NFSD_CMD_SVC_EXPORT_NOTIFY, then issue NFSD_CMD_SVC_EXPORT_GET_REQS +* to retrieve pending requests, resolve them, and respond with +* NFSD_CMD_SVC_EXPORT_SET_REQS. +*/ +static nfs_export *lookup_export (char *dom,char *path,struct addrinfo *ai); +static struct nl_msg *cache_nl_new_msg (int family,int cmd,int flags); + +static struct nl_sock *nfsd_nl_notify_sock;/* multicast notifications */ +static struct nl_sock *nfsd_nl_cmd_sock;/* GET_REQS / SET_REQS commands */ +static int nfsd_nl_family; + +#define NL_BUFFER_SIZE 65536 + +struct export_req { +char *client; +char *path; +}; + +static struct nl_sock *nl_sock_setup (void ) +{ +struct nl_sock *sock; +int val =1 ; + + sock =nl_socket_alloc (); +if (!sock) +return NULL ; + +if (genl_connect (sock)) { +nl_socket_free (sock); +return NULL ; + } + +nl_socket_set_buffer_size (sock,NL_BUFFER_SIZE ,NL_BUFFER_SIZE ); +setsockopt (nl_socket_get_fd (sock),SOL_NETLINK ,NETLINK_EXT_ACK , + &val,sizeof (val)); + +return sock; +} + +static int cache_nfsd_nl_open (void ) +{ +int grp; + + nfsd_nl_family =0 ; + + nfsd_nl_cmd_sock =nl_sock_setup (); +if (!nfsd_nl_cmd_sock) { +xlog (D_NETLINK ,"cache_nfsd_nl_open: failed to allocate command socket" ); +return -ENOMEM ; + } + + nfsd_nl_family =genl_ctrl_resolve (nfsd_nl_cmd_sock,NFSD_FAMILY_NAME ); +if (nfsd_nl_family <0 ) { +xlog (D_NETLINK ,"cache_nfsd_nl_open: nfsd netlink family not found" ); + goto out_free_cmd; + } + + grp =genl_ctrl_resolve_grp (nfsd_nl_cmd_sock,NFSD_FAMILY_NAME , +NFSD_MCGRP_EXPORTD ); +if (grp <0 ) { +xlog (D_NETLINK ,"cache_nfsd_nl_open: exportd multicast group not found" ); + goto out_free_cmd; + } + + nfsd_nl_notify_sock =nl_sock_setup (); +if (!nfsd_nl_notify_sock) { +xlog (D_NETLINK ,"cache_nfsd_nl_open: failed to allocate notify socket" ); + goto out_free_cmd; + } + +nl_socket_disable_seq_check (nfsd_nl_notify_sock); + +if (nl_socket_add_membership (nfsd_nl_notify_sock, grp)) { +xlog (L_WARNING ,"cache_nfsd_nl_open: failed to join exportd multicast group" ); + goto out_free_notify; + } + +nl_socket_set_nonblocking (nfsd_nl_notify_sock); +xlog (D_NETLINK ,"cache_nfsd_nl_open: listening for export notifications" ); +return 0 ; + +out_free_notify: +nl_socket_free (nfsd_nl_notify_sock); + nfsd_nl_notify_sock =NULL ; +out_free_cmd: +nl_socket_free (nfsd_nl_cmd_sock); + nfsd_nl_cmd_sock =NULL ; + nfsd_nl_family =0 ; +return -ENOENT ; +} + +static int nfsd_nl_notify_handler (struct nl_msg *UNUSED (msg ),void *UNUSED (arg )) +{ +return NL_OK ; +} + +static void cache_nfsd_nl_drain (void ) +{ +struct nl_cb *cb; + + cb =nl_cb_alloc (NL_CB_DEFAULT ); +if (!cb) +return ; + +nl_cb_set (cb,NL_CB_VALID ,NL_CB_CUSTOM , nfsd_nl_notify_handler,NULL ); +nl_recvmsgs (nfsd_nl_notify_sock, cb); +nl_cb_put (cb); +} + +struct get_export_reqs_data { +struct export_req *reqs; +int nreqs; +int maxreqs; +int err; +}; + +static int get_export_reqs_cb (struct nl_msg *msg,void *arg) +{ +struct get_export_reqs_data *data = arg; +struct genlmsghdr *gnlh =nlmsg_data (nlmsg_hdr (msg)); +struct nlattr *attr; +int rem; + +nla_for_each_attr (attr,genlmsg_attrdata (gnlh,0 ), +genlmsg_attrlen (gnlh,0 ), rem) { +struct nlattr *tb[NFSD_A_SVC_EXPORT_PATH +1 ]; +struct export_req *req; + +if (nla_type (attr) !=NFSD_A_SVC_EXPORT_REQS_REQUESTS ) +continue ; + +if (nla_parse_nested (tb,NFSD_A_SVC_EXPORT_PATH , attr, +NULL )) +continue ; + +if (!tb[NFSD_A_SVC_EXPORT_CLIENT ] || + !tb[NFSD_A_SVC_EXPORT_PATH ]) +continue ; + +if (data->nreqs >= data->maxreqs) { +int newmax = data->maxreqs ? data->maxreqs *2 :16 ; +struct export_req *tmp; + + tmp =realloc (data->reqs, + newmax *sizeof (*tmp)); +if (!tmp) { + data->err = -ENOMEM ; +return NL_STOP ; + } + data->reqs = tmp; + data->maxreqs = newmax; + } + + req = &data->reqs[data->nreqs++]; + req->client =strdup (nla_get_string (tb[NFSD_A_SVC_EXPORT_CLIENT ])); + req->path =strdup (nla_get_string (tb[NFSD_A_SVC_EXPORT_PATH ])); + +if (!req->client || !req->path) { + data->err = -ENOMEM ; +return NL_STOP ; + } + } + +return NL_OK ; +} + +static int nl_finish_cb (struct nl_msg *UNUSED (msg ),void *arg) +{ +int *done = arg; + + *done =1 ; +return NL_STOP ; +} + +static int nl_error_cb (struct sockaddr_nl *UNUSED (nla ), +struct nlmsgerr *UNUSED (nlerr ),void *arg) +{ +int *done = arg; + + *done =1 ; +return NL_STOP ; +} + +static int cache_nl_get_export_reqs (struct export_req **reqs_out,int *nreqs_out) +{ +struct get_export_reqs_data data = { }; +struct nl_msg *msg; +struct nl_cb *cb; +int done =0 ; +int ret; + + msg =cache_nl_new_msg (nfsd_nl_family, +NFSD_CMD_SVC_EXPORT_GET_REQS ,NLM_F_DUMP ); +if (!msg) +return -ENOMEM ; + + cb =nl_cb_alloc (NL_CB_DEFAULT ); +if (!cb) { +nlmsg_free (msg); +return -ENOMEM ; + } + +nl_cb_set (cb,NL_CB_VALID ,NL_CB_CUSTOM , get_export_reqs_cb, &data); +nl_cb_set (cb,NL_CB_FINISH ,NL_CB_CUSTOM , nl_finish_cb, &done); +nl_cb_err (cb,NL_CB_CUSTOM , nl_error_cb, &done); + + ret =nl_send_auto (nfsd_nl_cmd_sock, msg); +nlmsg_free (msg); +if (ret <0 ) { +nl_cb_put (cb); +return ret; + } + +while (!done) { + ret =nl_recvmsgs (nfsd_nl_cmd_sock, cb); +if (ret <0 ) +break ; + } + +nl_cb_put (cb); + +if (data.err) { +int i; +for (i =0 ; i < data.nreqs; i++) { +free (data.reqs[i].client); +free (data.reqs[i].path); + } +free (data.reqs); +return data.err; + } + + *reqs_out = data.reqs; + *nreqs_out = data.nreqs; +return 0 ; +} + +static int nfsd_nl_add_fsloc (struct nl_msg *msg,struct exportent *ep) +{ +struct servers *servers; +struct nlattr *fslocs; +int i; + +if (ep->e_fslocmethod ==FSLOC_NONE ) +return 0 ; + + servers =replicas_lookup (ep->e_fslocmethod, ep->e_fslocdata); +if (!servers) +return 0 ; + +if (servers->h_num <0 ) { +release_replicas (servers); +return 0 ; + } + + fslocs =nla_nest_start (msg,NFSD_A_SVC_EXPORT_FSLOCATIONS ); +if (!fslocs) { +release_replicas (servers); +return -1 ; + } + +for (i =0 ; i < servers->h_num; i++) { +struct nlattr *loc; + + loc =nla_nest_start (msg,NFSD_A_FSLOCATIONS_LOCATION ); +if (!loc) { +release_replicas (servers); +return -1 ; + } + +if (nla_put_string (msg,NFSD_A_FSLOCATION_HOST , + servers->h_mp[i]->h_host) <0 || +nla_put_string (msg,NFSD_A_FSLOCATION_PATH , + servers->h_mp[i]->h_path) <0 ) { +nla_nest_cancel (msg, fslocs); +release_replicas (servers); +return -1 ; + } +nla_nest_end (msg, loc); + } + +nla_nest_end (msg, fslocs); +release_replicas (servers); +return 0 ; +} + +static int nfsd_nl_add_secinfo (struct nl_msg *msg,struct exportent *ep) +{ +struct sec_entry *p; + +for (p = ep->e_secinfo; p->flav; p++) + ;/* count */ +if (p == ep->e_secinfo) +return 0 ; + +fix_pseudoflavor_flags (ep); +for (p = ep->e_secinfo; p->flav; p++) { +struct nlattr *sec; + + sec =nla_nest_start (msg,NFSD_A_SVC_EXPORT_SECINFO ); +if (!sec) +return -1 ; + +if (nla_put_u32 (msg,NFSD_A_AUTH_FLAVOR_PSEUDOFLAVOR , + p->flav->fnum) <0 || +nla_put_u32 (msg,NFSD_A_AUTH_FLAVOR_FLAGS , + p->flags) <0 ) +return -1 ; +nla_nest_end (msg, sec); + } + +return 0 ; +} + +static int nfsd_nl_add_xprtsec (struct nl_msg *msg,struct exportent *ep) +{ +struct xprtsec_entry *p; + +for (p = ep->e_xprtsec; p->info; p++) + ; +if (p == ep->e_xprtsec) +return 0 ; + +for (p = ep->e_xprtsec; p->info; p++) +if (nla_put_u32 (msg,NFSD_A_SVC_EXPORT_XPRTSEC , + p->info->number) <0 ) +return -1 ; + +return 0 ; +} + +static int nfsd_nl_add_export (struct nl_msg *msg,char *domain,char *path, +struct exportent *exp,int ttl) +{ +struct nlattr *nest; +time_t now =time (0 ); +char u[16 ]; + +if (ttl <=1 ) + ttl = default_ttl; + + nest =nla_nest_start (msg,NFSD_A_SVC_EXPORT_REQS_REQUESTS ); +if (!nest) +return -1 ; + +if (nla_put_string (msg,NFSD_A_SVC_EXPORT_CLIENT , domain) <0 || +nla_put_string (msg,NFSD_A_SVC_EXPORT_PATH , path) <0 || +nla_put_u64 (msg,NFSD_A_SVC_EXPORT_EXPIRY , now + ttl) <0 ) + goto nla_failure; + +if (!exp) { +if (nla_put_flag (msg,NFSD_A_SVC_EXPORT_NEGATIVE ) <0 ) + goto nla_failure; + }else { +if (nla_put_u32 (msg,NFSD_A_SVC_EXPORT_ANON_UID , + exp->e_anonuid) <0 || +nla_put_u32 (msg,NFSD_A_SVC_EXPORT_ANON_GID , + exp->e_anongid) <0 || +nla_put_u32 (msg,NFSD_A_SVC_EXPORT_FLAGS , + exp->e_flags) <0 || +nla_put_s32 (msg,NFSD_A_SVC_EXPORT_FSID , + exp->e_fsid) <0 ) + goto nla_failure; + +if (nfsd_nl_add_fsloc (msg, exp)) + goto nla_failure; + +if (exp->e_uuid) { +get_uuid (exp->e_uuid,16 , u); +if (nla_put (msg,NFSD_A_SVC_EXPORT_UUID , +16 , u) <0 ) + goto nla_failure; + }else if (uuid_by_path (path,0 ,16 , u)) { +if (nla_put (msg,NFSD_A_SVC_EXPORT_UUID , +16 , u) <0 ) + goto nla_failure; + } + +if (nfsd_nl_add_secinfo (msg, exp)) + goto nla_failure; + +if (nfsd_nl_add_xprtsec (msg, exp)) + goto nla_failure; + } + +nla_nest_end (msg, nest); +return 0 ; + +nla_failure: +nla_nest_cancel (msg, nest); +return -1 ; +} + +static struct nl_msg *cache_nl_new_msg (int family,int cmd,int flags) +{ +struct nl_msg *msg; + + msg =nlmsg_alloc (); +if (!msg) +return NULL ; + +if (!genlmsg_put (msg,NL_AUTO_PID ,NL_AUTO_SEQ , family, +0 , flags, cmd,0 )) { +nlmsg_free (msg); +return NULL ; + } +return msg; +} + +static int cache_nl_set_reqs (struct nl_msg *msg) +{ +struct nl_cb *cb; +int done =0 ; +int ret; + + cb =nl_cb_alloc (NL_CB_DEFAULT ); +if (!cb) +return -ENOMEM ; + +nl_cb_set (cb,NL_CB_ACK ,NL_CB_CUSTOM , nl_finish_cb, &done); +nl_cb_err (cb,NL_CB_CUSTOM , nl_error_cb, &done); + + ret =nl_send_auto (nfsd_nl_cmd_sock, msg); +if (ret <0 ) { +nl_cb_put (cb); +return ret; + } + +while (!done) { + ret =nl_recvmsgs (nfsd_nl_cmd_sock, cb); +if (ret <0 ) +break ; + } + +nl_cb_put (cb); +if (ret <0 ) +xlog (L_WARNING ,"%s: SET_REQS failed: %d" , __func__, ret); +return ret; +} + +static void cache_nl_process_export (void ) +{ +struct export_req *reqs =NULL ; +int nreqs =0 ; +struct nl_msg *msg; +int i; + +/* Drain pending notifications */ +cache_nfsd_nl_drain (); + +/* Fetch all pending requests from the kernel */ +if (cache_nl_get_export_reqs (&reqs, &nreqs)) { +xlog (L_WARNING ,"cache_nl_process_export: failed to get export requests" ); +return ; + } + +if (!nreqs) +return ; + +xlog (D_CALL ,"cache_nl_process_export: %d pending export requests" , nreqs); + +/* Build the SET_REQS response */ + msg =cache_nl_new_msg (nfsd_nl_family, +NFSD_CMD_SVC_EXPORT_SET_REQS ,0 ); +if (!msg) + goto out_free; + +auth_reload (); + +for (i =0 ; i < nreqs; i++) { +char *dom = reqs[i].client; +char *path = reqs[i].path; +struct addrinfo *ai =NULL ; +nfs_export *found =NULL ; +struct exportent *epp =NULL ; +int ttl =0 ; + +if (is_ipaddr_client (dom)) { + ai =lookup_client_addr (dom); +if (!ai) +xlog (D_AUTH ,"cache_nl_process_export: " +"failed to resolve client %s" , dom); + } + +if (ai || !is_ipaddr_client (dom)) + found =lookup_export (dom, path, ai); + +if (found) { +char *mp = found->m_export.e_mountpoint; + +if (mp && !*mp) + mp = found->m_export.e_path; +if (mp && !is_mountpoint (mp)) { +xlog (L_WARNING , +"Cannot export path '%s': not a mountpoint" , + path); + ttl =60 ; + }else { + epp = &found->m_export; + } + } + +if (nfsd_nl_add_export (msg, dom, path, epp, ttl) <0 ) { +cache_nl_set_reqs (msg); +nlmsg_free (msg); + msg =cache_nl_new_msg (nfsd_nl_family, +NFSD_CMD_SVC_EXPORT_SET_REQS ,0 ); +if (!msg) { +nfs_freeaddrinfo (ai); + goto out_free; + } +if (nfsd_nl_add_export (msg, dom, path, + epp, ttl) <0 ) +xlog (L_WARNING ,"%s: skipping oversized " +"entry for %s" , __func__, path); + } + +nfs_freeaddrinfo (ai); + } + +cache_nl_set_reqs (msg); +nlmsg_free (msg); + +out_free: +for (i =0 ; i < nreqs; i++) { +free (reqs[i].client); +free (reqs[i].path); + } +free (reqs); +} +static int can_reexport_via_fsidnum (struct exportent *exp,struct statfs *st) {if (st->f_type !=0x6969 /* NFS_SUPER_MAGIC */ )
@@ -1612,7 +2159,7 @@ extern int manage_gids;* cache_open - prepare communications channels with kernel RPC caches * */ -void cache_open (void ) +void cache_open (void ) {int i;
@@ -1623,6 +2170,7 @@ void cache_open(void)sprintf (path,"/proc/net/rpc/%s/channel" , cachelist[i].cache_name); cachelist[i].f =open (path,O_RDWR ); } +cache_nfsd_nl_open (); }/**
@@ -1636,13 +2184,15 @@ void cache_set_fds(fd_set *fdset)if (cachelist[i].f >=0 )FD_SET (cachelist[i].f, fdset); } +if (nfsd_nl_notify_sock) +FD_SET (nl_socket_get_fd (nfsd_nl_notify_sock), fdset); }/** * cache_process_req - process any active cache file descriptors during service loop iteration * @fdset: pointer to fd_set to examine for activity */ -int cache_process_req (fd_set *readfds) +int cache_process_req (fd_set *readfds) {int i;int cnt =0 ;
@@ -1654,6 +2204,12 @@ int cache_process_req(fd_set *readfds)FD_CLR (cachelist[i].f, readfds); } } +if (nfsd_nl_notify_sock && +FD_ISSET (nl_socket_get_fd (nfsd_nl_notify_sock), readfds)) { + cnt++; +cache_nl_process_export (); +FD_CLR (nl_socket_get_fd (nfsd_nl_notify_sock), readfds); + }return cnt; }
Signature · 1 line
2.53.0
Jeff Layton
maintainerPATCH v2 8/16 exportd/mountd: add netlink support for the nfsd.fh cache
In reply to Jeff LaytonRefactor cache_nl_set_reqs() to take a socket parameter so it can be reused for multiple genl families.
Rename cache_nl_process() to cache_nl_process_export() and factor out the notification drain and auth_reload() into a new cache_nfsd_nl_process() wrapper.
Add the expkey (nfsd.fh) netlink cache handler, which uses the same nfsd genl family as svc_export. The handler resolves fsid-based lookups including CROSSMOUNT submount iteration and responds via NFSD_CMD_EXPKEY_SET_REQS.
--- support/export/cache.c | 323 +++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 314 insertions(+), 9 deletions(-)
support/export/cache.c +314 -9
@@ -1477,7 +1477,7 @@ static struct nl_msg *cache_nl_new_msg(int family, int cmd, int flags)return msg; } -static int cache_nl_set_reqs (struct nl_msg *msg) +static int cache_nl_set_reqs (struct nl_sock *sock,struct nl_msg *msg) {struct nl_cb *cb;int done =0 ;
@@ -1490,14 +1490,14 @@ static int cache_nl_set_reqs(struct nl_msg *msg)nl_cb_set (cb,NL_CB_ACK ,NL_CB_CUSTOM , nl_finish_cb, &done);nl_cb_err (cb,NL_CB_CUSTOM , nl_error_cb, &done); - ret =nl_send_auto (nfsd_nl_cmd_sock, msg); + ret =nl_send_auto (sock, msg);if (ret <0 ) {nl_cb_put (cb);return ret; }while (!done) { - ret =nl_recvmsgs (nfsd_nl_cmd_sock, cb); + ret =nl_recvmsgs (sock, cb);if (ret <0 )break ; }
@@ -1515,9 +1515,6 @@ static void cache_nl_process_export(void)struct nl_msg *msg;int i; -/* Drain pending notifications */ -cache_nfsd_nl_drain (); -/* Fetch all pending requests from the kernel */ if (cache_nl_get_export_reqs (&reqs, &nreqs)) {xlog (L_WARNING ,"cache_nl_process_export: failed to get export requests" );
@@ -1571,7 +1568,7 @@ static void cache_nl_process_export(void) }if (nfsd_nl_add_export (msg, dom, path, epp, ttl) <0 ) { -cache_nl_set_reqs (msg); +cache_nl_set_reqs (nfsd_nl_cmd_sock, msg);nlmsg_free (msg); msg =cache_nl_new_msg (nfsd_nl_family,NFSD_CMD_SVC_EXPORT_SET_REQS ,0 );
@@ -1588,7 +1585,7 @@ static void cache_nl_process_export(void)nfs_freeaddrinfo (ai); } -cache_nl_set_reqs (msg); +cache_nl_set_reqs (nfsd_nl_cmd_sock, msg);nlmsg_free (msg); out_free:
@@ -1599,6 +1596,314 @@ out_free:free (reqs); } +/* +* Netlink-based expkey (nfsd.fh) cache support. +* +* Uses the same nfsd genl family as svc_export. The kernel sends +* NFSD_CMD_CACHE_NOTIFY with NFSD_CACHE_TYPE_EXPKEY to signal +* pending expkey cache requests. +*/ +struct expkey_req { +char *client; +int fsidtype; +char *fsid; +int fsidlen; +}; + +struct get_expkey_reqs_data { +struct expkey_req *reqs; +int nreqs; +int maxreqs; +int err; +}; + +static int get_expkey_reqs_cb (struct nl_msg *msg,void *arg) +{ +struct get_expkey_reqs_data *data = arg; +struct genlmsghdr *gnlh =nlmsg_data (nlmsg_hdr (msg)); +struct nlattr *attr; +int rem; + +nla_for_each_attr (attr,genlmsg_attrdata (gnlh,0 ), +genlmsg_attrlen (gnlh,0 ), rem) { +struct nlattr *tb[NFSD_A_EXPKEY_PATH +1 ]; +struct expkey_req *req; + +if (nla_type (attr) !=NFSD_A_EXPKEY_REQS_REQUESTS ) +continue ; + +if (nla_parse_nested (tb,NFSD_A_EXPKEY_PATH , attr,NULL )) +continue ; + +if (!tb[NFSD_A_EXPKEY_CLIENT ] || + !tb[NFSD_A_EXPKEY_FSIDTYPE ] || + !tb[NFSD_A_EXPKEY_FSID ]) +continue ; + +if (data->nreqs >= data->maxreqs) { +int newmax = data->maxreqs ? data->maxreqs *2 :16 ; +struct expkey_req *tmp; + + tmp =realloc (data->reqs, newmax *sizeof (*tmp)); +if (!tmp) { + data->err = -ENOMEM ; +return NL_STOP ; + } + data->reqs = tmp; + data->maxreqs = newmax; + } + + req = &data->reqs[data->nreqs++]; + req->client =strdup (nla_get_string (tb[NFSD_A_EXPKEY_CLIENT ])); + req->fsidtype =nla_get_u8 (tb[NFSD_A_EXPKEY_FSIDTYPE ]); + req->fsidlen =nla_len (tb[NFSD_A_EXPKEY_FSID ]); + req->fsid =malloc (req->fsidlen); + +if (!req->client || !req->fsid) { + data->err = -ENOMEM ; +return NL_STOP ; + } +memcpy (req->fsid,nla_data (tb[NFSD_A_EXPKEY_FSID ]), + req->fsidlen); + } + +return NL_OK ; +} + +static int cache_nl_get_expkey_reqs (struct expkey_req **reqs_out, +int *nreqs_out) +{ +struct get_expkey_reqs_data data = { }; +struct nl_msg *msg; +struct nl_cb *cb; +int done =0 ; +int ret; + + msg =cache_nl_new_msg (nfsd_nl_family, +NFSD_CMD_EXPKEY_GET_REQS ,NLM_F_DUMP ); +if (!msg) +return -ENOMEM ; + + cb =nl_cb_alloc (NL_CB_DEFAULT ); +if (!cb) { +nlmsg_free (msg); +return -ENOMEM ; + } + +nl_cb_set (cb,NL_CB_VALID ,NL_CB_CUSTOM , get_expkey_reqs_cb, &data); +nl_cb_set (cb,NL_CB_FINISH ,NL_CB_CUSTOM , nl_finish_cb, &done); +nl_cb_err (cb,NL_CB_CUSTOM , nl_error_cb, &done); + + ret =nl_send_auto (nfsd_nl_cmd_sock, msg); +nlmsg_free (msg); +if (ret <0 ) { +nl_cb_put (cb); +return ret; + } + +while (!done) { + ret =nl_recvmsgs (nfsd_nl_cmd_sock, cb); +if (ret <0 ) +break ; + } + +nl_cb_put (cb); + +if (data.err) { +int i; +for (i =0 ; i < data.nreqs; i++) { +free (data.reqs[i].client); +free (data.reqs[i].fsid); + } +free (data.reqs); +return data.err; + } + + *reqs_out = data.reqs; + *nreqs_out = data.nreqs; +return 0 ; +} + +static int nfsd_nl_add_expkey (struct nl_msg *msg,char *dom,int fsidtype, +char *fsid,int fsidlen,char *found_path) +{ +struct nlattr *nest; + + nest =nla_nest_start (msg,NFSD_A_EXPKEY_REQS_REQUESTS ); +if (!nest) +return -1 ; + +if (nla_put_string (msg,NFSD_A_EXPKEY_CLIENT , dom) <0 || +nla_put_u8 (msg,NFSD_A_EXPKEY_FSIDTYPE , fsidtype) <0 || +nla_put (msg,NFSD_A_EXPKEY_FSID , fsidlen, fsid) <0 || +nla_put_u64 (msg,NFSD_A_EXPKEY_EXPIRY ,0x7fffffff ) <0 ) + goto nla_failure; + +if (found_path) { +if (nla_put_string (msg,NFSD_A_EXPKEY_PATH , + found_path) <0 ) + goto nla_failure; + }else { +if (nla_put_flag (msg,NFSD_A_EXPKEY_NEGATIVE ) <0 ) + goto nla_failure; + } + +nla_nest_end (msg, nest); +return 0 ; + +nla_failure: +nla_nest_cancel (msg, nest); +return -1 ; +} + +static void cache_nl_process_expkey (void ) +{ +struct expkey_req *reqs =NULL ; +int nreqs =0 ; +struct nl_msg *msg; +int i; + +if (cache_nl_get_expkey_reqs (&reqs, &nreqs)) { +xlog (L_WARNING ,"cache_nl_process_expkey: failed to get expkey requests" ); +return ; + } + +if (!nreqs) +return ; + +xlog (D_CALL ,"cache_nl_process_expkey: %d pending expkey requests" , nreqs); + + msg =cache_nl_new_msg (nfsd_nl_family,NFSD_CMD_EXPKEY_SET_REQS ,0 ); +if (!msg) + goto out_free; + +for (i =0 ; i < nreqs; i++) { +char *dom = reqs[i].client; +int fsidtype = reqs[i].fsidtype; +char *fsid = reqs[i].fsid; +int fsidlen = reqs[i].fsidlen; +struct parsed_fsid parsed; +struct addrinfo *ai =NULL ; +struct exportent *found =NULL ; +char *found_path =NULL ; +nfs_export *exp; +int j; + +if (parse_fsid (fsidtype, fsidlen, fsid, &parsed)) + goto do_add_expkey; + +if (is_ipaddr_client (dom)) { + ai =lookup_client_addr (dom); +if (!ai) + goto do_add_expkey; + } + +for (j =0 ; j <MCL_MAXTYPES ; j++) { +nfs_export *prev =NULL ; +nfs_export *next_exp; +void *mnt =NULL ; + +for (exp = exportlist[j].p_head; exp; + exp = next_exp) { +char *path; + +if (exp->m_export.e_flags & +NFSEXP_CROSSMOUNT ) { +if (prev == exp) { + path =next_mnt (&mnt, + exp->m_export.e_path); +if (!path) { + next_exp = exp->m_next; + prev =NULL ; +continue ; + } + next_exp = exp; + }else { + prev = exp; + mnt =NULL ; + path = exp->m_export.e_path; + next_exp = exp; + } + }else { + path = exp->m_export.e_path; + next_exp = exp->m_next; + } + +if (!is_ipaddr_client (dom) && + !namelist_client_matches (exp, dom)) +continue ; + +switch (match_fsid (&parsed, exp, path)) { +case 0 : +continue ; +case -1 : +continue ; + } + +if (is_ipaddr_client (dom) && + !ipaddr_client_matches (exp, ai)) +continue ; + +if (!found || +subexport (&exp->m_export, found)) { + found = &exp->m_export; +free (found_path); + found_path =strdup (path); +if (!found_path) + goto do_add_expkey; + } + } + } + +do_add_expkey: +if (nfsd_nl_add_expkey (msg, dom, fsidtype, fsid, + fsidlen, found_path) <0 ) { +cache_nl_set_reqs (nfsd_nl_cmd_sock, msg); +nlmsg_free (msg); + msg =cache_nl_new_msg (nfsd_nl_family, +NFSD_CMD_EXPKEY_SET_REQS ,0 ); +if (!msg) { +free (found_path); +nfs_freeaddrinfo (ai); + goto out_free; + } +if (nfsd_nl_add_expkey (msg, dom, fsidtype, fsid, + fsidlen, found_path) <0 ) +xlog (L_WARNING ,"%s: skipping oversized " +"entry" , __func__); + } +if (!found) +xlog (D_AUTH ,"denied access to %s" , + *dom =='$' ? dom +1 : dom); +free (found_path); +nfs_freeaddrinfo (ai); + } + +cache_nl_set_reqs (nfsd_nl_cmd_sock, msg); +nlmsg_free (msg); + +out_free: +for (i =0 ; i < nreqs; i++) { +free (reqs[i].client); +free (reqs[i].fsid); + } +free (reqs); +} + +static void cache_nfsd_nl_process (void ) +{ +/* Drain pending nfsd notifications */ +cache_nfsd_nl_drain (); + +auth_reload (); + +/* Handle any pending svc_export requests */ +cache_nl_process_export (); + +/* Handle any pending expkey requests */ +cache_nl_process_expkey (); +} +static int can_reexport_via_fsidnum (struct exportent *exp,struct statfs *st) {if (st->f_type !=0x6969 /* NFS_SUPER_MAGIC */ )
@@ -2207,7 +2512,7 @@ int cache_process_req(fd_set *readfds)if (nfsd_nl_notify_sock &&FD_ISSET (nl_socket_get_fd (nfsd_nl_notify_sock), readfds)) { cnt++; -cache_nl_process_export (); +cache_nfsd_nl_process ();FD_CLR (nl_socket_get_fd (nfsd_nl_notify_sock), readfds); }return cnt;
Signature · 1 line
2.53.0
Jeff Layton
maintainerPATCH v2 9/16 exportd/mountd: add netlink support for the auth.unix.ip cache
In reply to Jeff LaytonRefactor cache_nfsd_nl_open() into a generic cache_genl_open() helper that takes the family name, multicast group name, and output pointers for the command socket, notification socket, and resolved family ID. Convert cache_nfsd_nl_open() to use it.
Add the sunrpc genl family socket setup for handling auth.unix.ip and auth.unix.gid cache upcalls. The sunrpc family is resolved at startup and silently falls back if the kernel doesn't support it.
Add the ip_map (auth.unix.ip) netlink cache handler. For each pending request, the handler resolves the IP address using client_resolve() and responds via SUNRPC_CMD_IP_MAP_SET_REQS with the domain name or a negative entry.
Wire the sunrpc notification socket into cache_open(), cache_set_fds(), and cache_process_req().
--- support/export/cache.c | 366 +++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 342 insertions(+), 24 deletions(-)
support/export/cache.c +342 -24
@@ -48,6 +48,12 @@#include "nfsd_netlink.h" #endif +#ifdef USE_SYSTEM_SUNRPC_NETLINK_H +#include <linux/sunrpc_netlink.h> +#else +#include "sunrpc_netlink.h" +#endif +#ifdef USE_BLKID #include "blkid/blkid.h" #endif
@@ -1108,58 +1114,71 @@ static struct nl_sock *nl_sock_setup(void)return sock; } -static int cache_nfsd_nl_open (void ) +static int cache_genl_open (const char *family_name,const char *mcgrp_name, +struct nl_sock **cmd_sock, +struct nl_sock **notify_sock,int *family_out) {int grp; - nfsd_nl_family =0 ; + *family_out =0 ; - nfsd_nl_cmd_sock =nl_sock_setup (); -if (!nfsd_nl_cmd_sock) { -xlog (D_NETLINK ,"cache_nfsd_nl_open: failed to allocate command socket" ); + *cmd_sock =nl_sock_setup (); +if (!*cmd_sock) { +xlog (D_NETLINK ,"%s: failed to allocate command socket" , + family_name);return -ENOMEM ; } - nfsd_nl_family =genl_ctrl_resolve (nfsd_nl_cmd_sock,NFSD_FAMILY_NAME ); -if (nfsd_nl_family <0 ) { -xlog (D_NETLINK ,"cache_nfsd_nl_open: nfsd netlink family not found" ); + *family_out =genl_ctrl_resolve (*cmd_sock, family_name); +if (*family_out <0 ) { +xlog (D_NETLINK ,"%s: netlink family not found" , family_name); goto out_free_cmd; } - grp =genl_ctrl_resolve_grp (nfsd_nl_cmd_sock,NFSD_FAMILY_NAME , -NFSD_MCGRP_EXPORTD ); + grp =genl_ctrl_resolve_grp (*cmd_sock, family_name, mcgrp_name);if (grp <0 ) { -xlog (D_NETLINK ,"cache_nfsd_nl_open: exportd multicast group not found" ); +xlog (D_NETLINK ,"%s: %s multicast group not found" , + family_name, mcgrp_name); goto out_free_cmd; } - nfsd_nl_notify_sock =nl_sock_setup (); -if (!nfsd_nl_notify_sock) { -xlog (D_NETLINK ,"cache_nfsd_nl_open: failed to allocate notify socket" ); + *notify_sock =nl_sock_setup (); +if (!*notify_sock) { +xlog (D_NETLINK ,"%s: failed to allocate notify socket" , + family_name); goto out_free_cmd; } -nl_socket_disable_seq_check (nfsd_nl_notify_sock); +nl_socket_disable_seq_check (*notify_sock); -if (nl_socket_add_membership (nfsd_nl_notify_sock, grp)) { -xlog (L_WARNING ,"cache_nfsd_nl_open: failed to join exportd multicast group" ); +if (nl_socket_add_membership (*notify_sock, grp)) { +xlog (L_WARNING ,"%s: failed to join %s multicast group" , + family_name, mcgrp_name); goto out_free_notify; } -nl_socket_set_nonblocking (nfsd_nl_notify_sock); -xlog (D_NETLINK ,"cache_nfsd_nl_open: listening for export notifications" ); +nl_socket_set_nonblocking (*notify_sock); +xlog (D_NETLINK ,"%s: listening for %s notifications" , + family_name, mcgrp_name);return 0 ; out_free_notify: -nl_socket_free (nfsd_nl_notify_sock); - nfsd_nl_notify_sock =NULL ; +nl_socket_free (*notify_sock); + *notify_sock =NULL ; out_free_cmd: -nl_socket_free (nfsd_nl_cmd_sock); - nfsd_nl_cmd_sock =NULL ; - nfsd_nl_family =0 ; +nl_socket_free (*cmd_sock); + *cmd_sock =NULL ; + *family_out =0 ;return -ENOENT ; } +static int cache_nfsd_nl_open (void ) +{ +return cache_genl_open (NFSD_FAMILY_NAME ,NFSD_MCGRP_EXPORTD , + &nfsd_nl_cmd_sock, &nfsd_nl_notify_sock, + &nfsd_nl_family); +} +static int nfsd_nl_notify_handler (struct nl_msg *UNUSED (msg ),void *UNUSED (arg )) {return NL_OK ;
@@ -1904,6 +1923,296 @@ static void cache_nfsd_nl_process(void)cache_nl_process_expkey (); } +/* +* Netlink-based sunrpc cache support. +* +* The sunrpc genl family handles auth.unix.ip and auth.unix.gid caches. +* A SUNRPC_CMD_CACHE_NOTIFY on the "exportd" multicast group signals +* pending cache requests. +*/ +static struct nl_sock *sunrpc_nl_notify_sock; +static struct nl_sock *sunrpc_nl_cmd_sock; +static int sunrpc_nl_family; + +static int cache_sunrpc_nl_open (void ) +{ +return cache_genl_open (SUNRPC_FAMILY_NAME ,SUNRPC_MCGRP_EXPORTD , + &sunrpc_nl_cmd_sock, &sunrpc_nl_notify_sock, + &sunrpc_nl_family); +} + +static void cache_sunrpc_nl_drain (void ) +{ +struct nl_cb *cb; + + cb =nl_cb_alloc (NL_CB_DEFAULT ); +if (!cb) +return ; + +nl_cb_set (cb,NL_CB_VALID ,NL_CB_CUSTOM , nfsd_nl_notify_handler,NULL ); +nl_recvmsgs (sunrpc_nl_notify_sock, cb); +nl_cb_put (cb); +} + +/* +* ip_map (auth.unix.ip) netlink handler +*/ +struct ip_map_req { +char *class; +char *addr; +}; + +struct get_ip_map_reqs_data { +struct ip_map_req *reqs; +int nreqs; +int maxreqs; +int err; +}; + +static int get_ip_map_reqs_cb (struct nl_msg *msg,void *arg) +{ +struct get_ip_map_reqs_data *data = arg; +struct genlmsghdr *gnlh =nlmsg_data (nlmsg_hdr (msg)); +struct nlattr *attr; +int rem; + +nla_for_each_attr (attr,genlmsg_attrdata (gnlh,0 ), +genlmsg_attrlen (gnlh,0 ), rem) { +struct nlattr *tb[SUNRPC_A_IP_MAP_EXPIRY +1 ]; +struct ip_map_req *req; + +if (nla_type (attr) !=SUNRPC_A_IP_MAP_REQS_REQUESTS ) +continue ; + +if (nla_parse_nested (tb,SUNRPC_A_IP_MAP_EXPIRY , attr, +NULL )) +continue ; + +if (!tb[SUNRPC_A_IP_MAP_CLASS ] || + !tb[SUNRPC_A_IP_MAP_ADDR ]) +continue ; + +if (data->nreqs >= data->maxreqs) { +int newmax = data->maxreqs ? data->maxreqs *2 :16 ; +struct ip_map_req *tmp; + + tmp =realloc (data->reqs, newmax *sizeof (*tmp)); +if (!tmp) { + data->err = -ENOMEM ; +return NL_STOP ; + } + data->reqs = tmp; + data->maxreqs = newmax; + } + + req = &data->reqs[data->nreqs++]; + req->class =strdup (nla_get_string (tb[SUNRPC_A_IP_MAP_CLASS ])); + req->addr =strdup (nla_get_string (tb[SUNRPC_A_IP_MAP_ADDR ])); + +if (!req->class || !req->addr) { + data->err = -ENOMEM ; +return NL_STOP ; + } + } + +return NL_OK ; +} + +static int cache_nl_get_ip_map_reqs (struct ip_map_req **reqs_out, +int *nreqs_out) +{ +struct get_ip_map_reqs_data data = { }; +struct nl_msg *msg; +struct nl_cb *cb; +int done =0 ; +int ret; + + msg =cache_nl_new_msg (sunrpc_nl_family, +SUNRPC_CMD_IP_MAP_GET_REQS ,NLM_F_DUMP ); +if (!msg) +return -ENOMEM ; + + cb =nl_cb_alloc (NL_CB_DEFAULT ); +if (!cb) { +nlmsg_free (msg); +return -ENOMEM ; + } + +nl_cb_set (cb,NL_CB_VALID ,NL_CB_CUSTOM , get_ip_map_reqs_cb, &data); +nl_cb_set (cb,NL_CB_FINISH ,NL_CB_CUSTOM , nl_finish_cb, &done); +nl_cb_err (cb,NL_CB_CUSTOM , nl_error_cb, &done); + + ret =nl_send_auto (sunrpc_nl_cmd_sock, msg); +nlmsg_free (msg); +if (ret <0 ) { +nl_cb_put (cb); +return ret; + } + +while (!done) { + ret =nl_recvmsgs (sunrpc_nl_cmd_sock, cb); +if (ret <0 ) +break ; + } + +nl_cb_put (cb); + +if (data.err) { +int i; +for (i =0 ; i < data.nreqs; i++) { +free (data.reqs[i].class); +free (data.reqs[i].addr); + } +free (data.reqs); +return data.err; + } + + *reqs_out = data.reqs; + *nreqs_out = data.nreqs; +return 0 ; +} + +static int nl_add_ip_map (struct nl_msg *msg,char *class,char *addr, +char *domain) +{ +struct nlattr *nest; +time_t now =time (0 ); + + nest =nla_nest_start (msg,SUNRPC_A_IP_MAP_REQS_REQUESTS ); +if (!nest) +return -1 ; + +if (nla_put_string (msg,SUNRPC_A_IP_MAP_CLASS , class) <0 || +nla_put_string (msg,SUNRPC_A_IP_MAP_ADDR , addr) <0 || +nla_put_u64 (msg,SUNRPC_A_IP_MAP_EXPIRY , + now + default_ttl) <0 ) + goto nla_failure; + +if (domain) { +if (nla_put_string (msg,SUNRPC_A_IP_MAP_DOMAIN , + domain) <0 ) + goto nla_failure; + }else { +if (nla_put_flag (msg,SUNRPC_A_IP_MAP_NEGATIVE ) <0 ) + goto nla_failure; + } + +nla_nest_end (msg, nest); +return 0 ; + +nla_failure: +nla_nest_cancel (msg, nest); +return -1 ; +} + +static void cache_nl_process_ip_map (void ) +{ +struct ip_map_req *reqs =NULL ; +int nreqs =0 ; +struct nl_msg *msg; +int i; + +if (cache_nl_get_ip_map_reqs (&reqs, &nreqs)) { +xlog (L_WARNING ,"cache_nl_process_ip_map: failed to get ip_map requests" ); +return ; + } + +if (!nreqs) +return ; + +xlog (D_CALL ,"cache_nl_process_ip_map: %d pending ip_map requests" , + nreqs); + + msg =cache_nl_new_msg (sunrpc_nl_family, +SUNRPC_CMD_IP_MAP_SET_REQS ,0 ); +if (!msg) + goto out_free; + +for (i =0 ; i < nreqs; i++) { +char *class = reqs[i].class; +char *ipaddr = reqs[i].addr; +char *client =NULL ; +char *domain =NULL ; +char *dom_alloc =NULL ; +struct addrinfo *tmp =NULL ; + +if (strcmp (class,"nfsd" ) ==0 ) { + tmp =host_pton (ipaddr); +if (tmp) { +struct addrinfo *ai; + + ai =client_resolve (tmp->ai_addr); +if (ai) { + client =client_compose (ai); +nfs_freeaddrinfo (ai); + } + } + +if (use_ipaddr && client) { + dom_alloc =malloc (strlen (ipaddr) +2 ); +if (dom_alloc) { + dom_alloc[0 ] ='$' ; +strcpy (dom_alloc +1 , ipaddr); + domain = dom_alloc; + } + }else if (client) { + domain = *client ? client :"DEFAULT" ; + } + } + +if (nl_add_ip_map (msg, class, ipaddr, domain) <0 ) { +cache_nl_set_reqs (sunrpc_nl_cmd_sock, msg); +nlmsg_free (msg); + msg =cache_nl_new_msg (sunrpc_nl_family, +SUNRPC_CMD_IP_MAP_SET_REQS ,0 ); +if (!msg) { +free (dom_alloc); +free (client); +nfs_freeaddrinfo (tmp); + goto out_free; + } +if (nl_add_ip_map (msg, class, ipaddr, domain) <0 ) +xlog (L_WARNING ,"%s: skipping oversized " +"entry for %s" , __func__, ipaddr); + } + +if (tmp && !client) +xlog (D_AUTH ,"failed authentication for IP %s" , + ipaddr); +else if (client && !use_ipaddr) +xlog (D_AUTH ,"successful authentication for IP %s as %s" , + ipaddr, *client ? client :"DEFAULT" ); +else if (client) +xlog (D_AUTH ,"successful authentication for IP %s" , + ipaddr); + +free (dom_alloc); +free (client); +nfs_freeaddrinfo (tmp); + } + +cache_nl_set_reqs (sunrpc_nl_cmd_sock, msg); +nlmsg_free (msg); + +out_free: +for (i =0 ; i < nreqs; i++) { +free (reqs[i].class); +free (reqs[i].addr); + } +free (reqs); +} + +static void cache_sunrpc_nl_process (void ) +{ +/* Drain pending sunrpc notifications */ +cache_sunrpc_nl_drain (); + +auth_reload (); + +/* Handle any pending ip_map requests */ +cache_nl_process_ip_map (); +} +static int can_reexport_via_fsidnum (struct exportent *exp,struct statfs *st) {if (st->f_type !=0x6969 /* NFS_SUPER_MAGIC */ )
@@ -2476,6 +2785,7 @@ void cache_open(void) cachelist[i].f =open (path,O_RDWR ); }cache_nfsd_nl_open (); +cache_sunrpc_nl_open (); }/**
@@ -2491,6 +2801,8 @@ void cache_set_fds(fd_set *fdset) }if (nfsd_nl_notify_sock)FD_SET (nl_socket_get_fd (nfsd_nl_notify_sock), fdset); +if (sunrpc_nl_notify_sock) +FD_SET (nl_socket_get_fd (sunrpc_nl_notify_sock), fdset); }/**
@@ -2515,6 +2827,12 @@ int cache_process_req(fd_set *readfds)cache_nfsd_nl_process ();FD_CLR (nl_socket_get_fd (nfsd_nl_notify_sock), readfds); } +if (sunrpc_nl_notify_sock && +FD_ISSET (nl_socket_get_fd (sunrpc_nl_notify_sock), readfds)) { + cnt++; +cache_sunrpc_nl_process (); +FD_CLR (nl_socket_get_fd (sunrpc_nl_notify_sock), readfds); + }return cnt; }
Signature · 1 line
2.53.0
Jeff Layton
maintainerPATCH v2 10/16 exportd/mountd: add netlink support for the auth.unix.gid cache
In reply to Jeff LaytonAdd the unix_gid (auth.unix.gid) netlink cache handler. For each pending request, the handler resolves the UID to a group list using getpwuid() and getgrouplist(), and responds via SUNRPC_CMD_UNIX_GID_SET_REQS.
The handler is only active when manage_gids is set, matching the behavior of the existing procfs handler.
--- support/export/cache.c | 236 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 236 insertions(+)
support/export/cache.c +236 -0
@@ -110,6 +110,7 @@ static bool path_lookup_error(int err)#define INITIAL_MANAGED_GROUPS 100extern int use_ipaddr; +extern int manage_gids;static void auth_unix_ip (int f) {
@@ -2202,6 +2203,237 @@ out_free:free (reqs); } +/* +* unix_gid (auth.unix.gid) netlink handler +*/ +struct unix_gid_req { +uid_t uid; +}; + +struct get_unix_gid_reqs_data { +struct unix_gid_req *reqs; +int nreqs; +int maxreqs; +int err; +}; + +static int get_unix_gid_reqs_cb (struct nl_msg *msg,void *arg) +{ +struct get_unix_gid_reqs_data *data = arg; +struct genlmsghdr *gnlh =nlmsg_data (nlmsg_hdr (msg)); +struct nlattr *attr; +int rem; + +nla_for_each_attr (attr,genlmsg_attrdata (gnlh,0 ), +genlmsg_attrlen (gnlh,0 ), rem) { +struct nlattr *tb[SUNRPC_A_UNIX_GID_EXPIRY +1 ]; +struct unix_gid_req *req; + +if (nla_type (attr) !=SUNRPC_A_UNIX_GID_REQS_REQUESTS ) +continue ; + +if (nla_parse_nested (tb,SUNRPC_A_UNIX_GID_EXPIRY , attr, +NULL )) +continue ; + +if (!tb[SUNRPC_A_UNIX_GID_UID ]) +continue ; + +if (data->nreqs >= data->maxreqs) { +int newmax = data->maxreqs ? data->maxreqs *2 :16 ; +struct unix_gid_req *tmp; + + tmp =realloc (data->reqs, newmax *sizeof (*tmp)); +if (!tmp) { + data->err = -ENOMEM ; +return NL_STOP ; + } + data->reqs = tmp; + data->maxreqs = newmax; + } + + req = &data->reqs[data->nreqs++]; + req->uid =nla_get_u32 (tb[SUNRPC_A_UNIX_GID_UID ]); + } + +return NL_OK ; +} + +static int cache_nl_get_unix_gid_reqs (struct unix_gid_req **reqs_out, +int *nreqs_out) +{ +struct get_unix_gid_reqs_data data = { }; +struct nl_msg *msg; +struct nl_cb *cb; +int done =0 ; +int ret; + + msg =cache_nl_new_msg (sunrpc_nl_family, +SUNRPC_CMD_UNIX_GID_GET_REQS ,NLM_F_DUMP ); +if (!msg) +return -ENOMEM ; + + cb =nl_cb_alloc (NL_CB_DEFAULT ); +if (!cb) { +nlmsg_free (msg); +return -ENOMEM ; + } + +nl_cb_set (cb,NL_CB_VALID ,NL_CB_CUSTOM , get_unix_gid_reqs_cb, + &data); +nl_cb_set (cb,NL_CB_FINISH ,NL_CB_CUSTOM , nl_finish_cb, &done); +nl_cb_err (cb,NL_CB_CUSTOM , nl_error_cb, &done); + + ret =nl_send_auto (sunrpc_nl_cmd_sock, msg); +nlmsg_free (msg); +if (ret <0 ) { +nl_cb_put (cb); +return ret; + } + +while (!done) { + ret =nl_recvmsgs (sunrpc_nl_cmd_sock, cb); +if (ret <0 ) +break ; + } + +nl_cb_put (cb); + +if (data.err) { +free (data.reqs); +return data.err; + } + + *reqs_out = data.reqs; + *nreqs_out = data.nreqs; +return 0 ; +} + +static int nl_add_unix_gid (struct nl_msg *msg,uid_t uid,gid_t *groups, +int ngroups) +{ +struct nlattr *nest; +time_t now =time (0 ); +int i; + + nest =nla_nest_start (msg,SUNRPC_A_UNIX_GID_REQS_REQUESTS ); +if (!nest) +return -1 ; + +if (nla_put_u32 (msg,SUNRPC_A_UNIX_GID_UID , uid) <0 || +nla_put_u64 (msg,SUNRPC_A_UNIX_GID_EXPIRY , now + default_ttl) <0 ) + goto nla_failure; + +if (ngroups >=0 ) { +for (i =0 ; i < ngroups; i++) +if (nla_put_u32 (msg,SUNRPC_A_UNIX_GID_GIDS , groups[i]) <0 ) + goto nla_failure; + }else { +if (nla_put_flag (msg,SUNRPC_A_UNIX_GID_NEGATIVE ) <0 ) + goto nla_failure; + } + +nla_nest_end (msg, nest); +return 0 ; +nla_failure: +nla_nest_cancel (msg, nest); +return -1 ; +} + +static void cache_nl_process_unix_gid (void ) +{ +struct unix_gid_req *reqs =NULL ; +int nreqs =0 ; +struct nl_msg *msg; +static gid_t *groups =NULL ; +static int groups_len =0 ; +int i; + +if (cache_nl_get_unix_gid_reqs (&reqs, &nreqs)) { +xlog (L_WARNING ,"cache_nl_process_unix_gid: failed to get unix_gid requests" ); +return ; + } + +if (!nreqs) +return ; + +xlog (D_CALL ,"cache_nl_process_unix_gid: %d pending unix_gid requests" , + nreqs); + +if (groups_len ==0 ) { + groups =malloc (sizeof (gid_t) *INITIAL_MANAGED_GROUPS ); +if (!groups) + goto out_free; + groups_len =INITIAL_MANAGED_GROUPS ; + } + + msg =cache_nl_new_msg (sunrpc_nl_family, +SUNRPC_CMD_UNIX_GID_SET_REQS ,0 ); +if (!msg) + goto out_free; + +for (i =0 ; i < nreqs; i++) { +uid_t uid = reqs[i].uid; +struct passwd *pw; +int ngroups; +int rv; +int ret; + + ngroups = groups_len; + pw =getpwuid (uid); +if (!pw) { + rv =-1 ; + }else { + rv =getgrouplist (pw->pw_name, pw->pw_gid, + groups, &ngroups); +if (rv ==-1 && ngroups >= groups_len) { +gid_t *more_groups; + + more_groups =realloc (groups, +sizeof (gid_t) * ngroups); +if (!more_groups) { + rv =-1 ; + }else { + groups = more_groups; + groups_len = ngroups; + rv =getgrouplist (pw->pw_name, + pw->pw_gid, + groups, &ngroups); + } + } + } + +if (rv >=0 ) + ret =nl_add_unix_gid (msg, uid, groups, ngroups); +else + ret =nl_add_unix_gid (msg, uid,NULL ,-1 ); + +if (ret <0 ) { +/* Flush current message and retry with a fresh one */ +cache_nl_set_reqs (sunrpc_nl_cmd_sock, msg); +nlmsg_free (msg); + msg =cache_nl_new_msg (sunrpc_nl_family, +SUNRPC_CMD_UNIX_GID_SET_REQS ,0 ); +if (!msg) + goto out_free; + +if (rv >=0 ) + ret =nl_add_unix_gid (msg, uid, groups, ngroups); +else + ret =nl_add_unix_gid (msg, uid,NULL ,-1 ); +if (ret <0 ) +xlog (L_WARNING ,"%s: skipping oversized entry for uid %u" , + __func__, uid); + } + } + +cache_nl_set_reqs (sunrpc_nl_cmd_sock, msg); +nlmsg_free (msg); + +out_free: +free (reqs); +} +static void cache_sunrpc_nl_process (void ) {/* Drain pending sunrpc notifications */
@@ -2211,6 +2443,10 @@ static void cache_sunrpc_nl_process(void)/* Handle any pending ip_map requests */ cache_nl_process_ip_map (); + +/* Handle any pending unix_gid requests */ +if (manage_gids) +cache_nl_process_unix_gid (); }static int can_reexport_via_fsidnum (struct exportent *exp,struct statfs *st)
Signature · 1 line
2.53.0
Jeff Layton
maintainerPATCH v2 11/16 mountd/exportd: only use /proc interfaces if netlink setup fails
In reply to Jeff LaytonNow that all of the exportd upcalls are converted to use netlink, don't bother opening /proc channel files if that succeeds.
--- support/export/cache.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-)
support/export/cache.c +10 -2
@@ -3013,6 +3013,16 @@ void cache_open(void) {int i; +if (cache_nfsd_nl_open () ==0 ) { +if (cache_sunrpc_nl_open () ==0 ) +return ; +xlog (L_NOTICE ,"sunrpc netlink family unavailable, falling back to /proc" ); +nl_socket_free (nfsd_nl_notify_sock); + nfsd_nl_notify_sock =NULL ; +nl_socket_free (nfsd_nl_cmd_sock); + nfsd_nl_cmd_sock =NULL ; + } +for (i=0 ; cachelist[i].cache_name; i++ ) {char path[100 ];if (!manage_gids && cachelist[i].cache_handle == auth_unix_gid)
@@ -3020,8 +3030,6 @@ void cache_open(void)sprintf (path,"/proc/net/rpc/%s/channel" , cachelist[i].cache_name); cachelist[i].f =open (path,O_RDWR ); } -cache_nfsd_nl_open (); -cache_sunrpc_nl_open (); }/**
Signature · 1 line
2.53.0
Jeff Layton
maintainerPATCH v2 12/16 support/export: check for pending requests after opening netlink sockets
In reply to Jeff LaytonIt's possible for there to be requests queued before mountd comes online. Always check for already-pending requests after opening netlink sockets.
--- support/export/cache.c | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-)
support/export/cache.c +12 -1
@@ -3014,8 +3014,19 @@ void cache_open(void)int i;if (cache_nfsd_nl_open () ==0 ) { -if (cache_sunrpc_nl_open () ==0 ) +if (cache_sunrpc_nl_open () ==0 ) { +/* +* Check for pending requests, in case any +* were queued before we opened the socket. +*/ +auth_reload (); +cache_nl_process_export (); +cache_nl_process_expkey (); +cache_nl_process_ip_map (); +if (manage_gids) +cache_nl_process_unix_gid ();return ; + }xlog (L_NOTICE ,"sunrpc netlink family unavailable, falling back to /proc" );nl_socket_free (nfsd_nl_notify_sock); nfsd_nl_notify_sock =NULL ;
Signature · 1 line
2.53.0
Jeff Layton
maintainerPATCH v2 13/16 exportd/mountd: use cache type from notifications to target scanning
In reply to Jeff LaytonParse the NFSD_A_CACHE_NOTIFY_CACHE_TYPE and SUNRPC_A_CACHE_NOTIFY_CACHE_TYPE attributes from netlink multicast notifications to determine which specific caches have pending requests. Only issue GET_REQS for the caches that actually need servicing, avoiding unnecessary netlink round-trips.
If the notification can't be parsed, fall back to scanning all caches for that family.
--- support/export/cache.c | 81 ++++++++++++++++++++++++++++++++++++++++---------- 1 file changed, 66 insertions(+), 15 deletions(-)
support/export/cache.c +66 -15
@@ -1180,22 +1180,45 @@ static int cache_nfsd_nl_open(void) &nfsd_nl_family); } -static int nfsd_nl_notify_handler (struct nl_msg *UNUSED (msg ),void *UNUSED (arg )) +static int nl_seq_check_handler (struct nl_msg *UNUSED (msg ),void *UNUSED (arg )) {return NL_OK ; } -static void cache_nfsd_nl_drain (void ) +static int nfsd_notify_handler (struct nl_msg *msg,void *arg) { +unsigned int *cache_mask = arg; +struct genlmsghdr *gnlh =nlmsg_data (nlmsg_hdr (msg)); +struct nlattr *tb[NFSD_A_CACHE_NOTIFY_MAX +1 ]; + +if (nla_parse (tb,NFSD_A_CACHE_NOTIFY_MAX , +genlmsg_attrdata (gnlh,0 ), +genlmsg_attrlen (gnlh,0 ),NULL ) ==0 && + tb[NFSD_A_CACHE_NOTIFY_CACHE_TYPE ]) + *cache_mask |=nla_get_u32 (tb[NFSD_A_CACHE_NOTIFY_CACHE_TYPE ]); +else + *cache_mask = ~0U ; + +xlog (D_NETLINK ,"nfsd_notify_handler: cache_mask=%x" , *cache_mask); +return NL_OK ; +} + +static unsigned int cache_nfsd_nl_drain (void ) +{ +unsigned int cache_mask =0 ;struct nl_cb *cb; cb =nl_cb_alloc (NL_CB_DEFAULT );if (!cb) -return ; +return ~0U ; -nl_cb_set (cb,NL_CB_VALID ,NL_CB_CUSTOM , nfsd_nl_notify_handler,NULL ); +nl_cb_set (cb,NL_CB_SEQ_CHECK ,NL_CB_CUSTOM , + nl_seq_check_handler,NULL ); +nl_cb_set (cb,NL_CB_VALID ,NL_CB_CUSTOM , nfsd_notify_handler, + &cache_mask);nl_recvmsgs (nfsd_nl_notify_sock, cb);nl_cb_put (cb); +return cache_mask; }struct get_export_reqs_data {
@@ -1552,8 +1575,6 @@ static void cache_nl_process_export(void)if (!msg) goto out_free; -auth_reload (); -for (i =0 ; i < nreqs; i++) {char *dom = reqs[i].client;char *path = reqs[i].path;
@@ -1912,16 +1933,20 @@ out_free:static void cache_nfsd_nl_process (void ) { +unsigned int cache_mask; +/* Drain pending nfsd notifications */ -cache_nfsd_nl_drain (); + cache_mask =cache_nfsd_nl_drain ();auth_reload ();/* Handle any pending svc_export requests */ -cache_nl_process_export (); +if (cache_mask &NFSD_CACHE_TYPE_SVC_EXPORT ) +cache_nl_process_export ();/* Handle any pending expkey requests */ -cache_nl_process_expkey (); +if (cache_mask &NFSD_CACHE_TYPE_EXPKEY ) +cache_nl_process_expkey (); }/*
@@ -1942,17 +1967,40 @@ static int cache_sunrpc_nl_open(void) &sunrpc_nl_family); } -static void cache_sunrpc_nl_drain (void ) +static int sunrpc_notify_handler (struct nl_msg *msg,void *arg) { +unsigned int *cache_mask = arg; +struct genlmsghdr *gnlh =nlmsg_data (nlmsg_hdr (msg)); +struct nlattr *tb[SUNRPC_A_CACHE_NOTIFY_MAX +1 ]; + +if (nla_parse (tb,SUNRPC_A_CACHE_NOTIFY_MAX , +genlmsg_attrdata (gnlh,0 ), +genlmsg_attrlen (gnlh,0 ),NULL ) ==0 && + tb[SUNRPC_A_CACHE_NOTIFY_CACHE_TYPE ]) + *cache_mask |=nla_get_u32 (tb[SUNRPC_A_CACHE_NOTIFY_CACHE_TYPE ]); +else + *cache_mask = ~0U ; + +xlog (D_NETLINK ,"sunrpc_notify_handler: cache_mask=%x" , *cache_mask); +return NL_OK ; +} + +static unsigned int cache_sunrpc_nl_drain (void ) +{ +unsigned int cache_mask =0 ;struct nl_cb *cb; cb =nl_cb_alloc (NL_CB_DEFAULT );if (!cb) -return ; +return ~0U ; -nl_cb_set (cb,NL_CB_VALID ,NL_CB_CUSTOM , nfsd_nl_notify_handler,NULL ); +nl_cb_set (cb,NL_CB_SEQ_CHECK ,NL_CB_CUSTOM , + nl_seq_check_handler,NULL ); +nl_cb_set (cb,NL_CB_VALID ,NL_CB_CUSTOM , sunrpc_notify_handler, + &cache_mask);nl_recvmsgs (sunrpc_nl_notify_sock, cb);nl_cb_put (cb); +return cache_mask; }/*
@@ -2436,16 +2484,19 @@ out_free:static void cache_sunrpc_nl_process (void ) { +unsigned int cache_mask; +/* Drain pending sunrpc notifications */ -cache_sunrpc_nl_drain (); + cache_mask =cache_sunrpc_nl_drain ();auth_reload ();/* Handle any pending ip_map requests */ -cache_nl_process_ip_map (); +if (cache_mask &SUNRPC_CACHE_TYPE_IP_MAP ) +cache_nl_process_ip_map ();/* Handle any pending unix_gid requests */ -if (manage_gids) +if (manage_gids && (cache_mask &SUNRPC_CACHE_TYPE_UNIX_GID ))cache_nl_process_unix_gid (); }
Signature · 1 line
2.53.0
Jeff Layton
maintainerPATCH v2 14/16 exportfs: add netlink support for cache flush with /proc fallback
In reply to Jeff LaytonMove cache_flush() from support/nfs/cacheio.c to a new file support/export/cache_flush.c, which has access to libnl.
The new implementation tries netlink first:
- Sends SUNRPC_CMD_CACHE_FLUSH to flush auth.unix.ip and auth.unix.gid
- Sends NFSD_CMD_CACHE_FLUSH to flush nfsd.fh and nfsd.export
- Flushes sunrpc caches before nfsd caches to maintain dependency order
If the sunrpc genl family cannot be resolved (older kernel), falls back to the original /proc/net/rpc/*/flush write path.
--- support/export/Makefile.am | 2 +- support/export/cache_flush.c | 164 +++++++++++++++++++++++++++++++++++++++++++ support/nfs/cacheio.c | 49 +------------ 3 files changed, 166 insertions(+), 49 deletions(-)
support/export/Makefile.am +1 -1
@@ -12,7 +12,7 @@ EXTRA_DIST = mount.x noinst_LIBRARIES = libexport.a libexport_a_SOURCES = client.c export.c hostname.c \ xtab.c mount_clnt.c mount_xdr.c \ - cache.c auth.c v4root.c fsloc.c \ + cache.c cache_flush.c auth.c v4root.c fsloc.c \ v4clients.c libexport_a_CPPFLAGS = $(AM_CPPFLAGS) $(CPPFLAGS) -I$(top_srcdir)/support/reexport \ $(LIBNL3_CFLAGS) $(LIBNLGENL3_CFLAGS)
support/export/cache_flush.c new file +164 -0
164 lines not on this page · raw message
support/nfs/cacheio.c +1 -48
52 lines not on this page · raw message
Signature · 1 line
2.53.0
Jeff Layton
maintainerPATCH v2 15/16 exportfs: use netlink to probe kernel support, skip export_test
In reply to Jeff LaytonRefactor can_test() to first try resolving the sunrpc genl family via netlink. If successful, the kernel supports the new netlink-based cache interfaces and we can skip the advisory export_test() probe -- mountd and exportd will validate exports at cache-fill time via check_export().
can_test() now returns:
2 = netlink available (skip export_test) 1 = /proc available (keep export_test as before) 0 = neither available
Update validate_export() to check this return value and skip the export_test() calls when netlink is available.
Link exportfs against libnl3/libnl-genl-3 (already a mandatory build dependency for nfs-utils).
--- utils/exportfs/Makefile.am | 6 ++++-- utils/exportfs/exportfs.c | 42 +++++++++++++++++++++++++++++++++++++++++- 2 files changed, 45 insertions(+), 3 deletions(-)
utils/exportfs/Makefile.am +4 -2
@@ -11,8 +11,10 @@ exportfs_LDADD = ../../support/export/libexport.a \ ../../support/nfs/libnfs.la \ ../../support/misc/libmisc.a \ ../../support/reexport/libreexport.a \ - $(LIBWRAP) $(LIBNSL) $(LIBPTHREAD) + $(LIBWRAP) $(LIBNSL) $(LIBPTHREAD) \ + $(LIBNL3_LIBS) $(LIBNLGENL3_LIBS) -exportfs_CPPFLAGS = $(AM_CPPFLAGS) $(CPPFLAGS) -I$(top_srcdir)/support/reexport +exportfs_CPPFLAGS = $(AM_CPPFLAGS) $(CPPFLAGS) -I$(top_srcdir)/support/reexport \ + $(LIBNL3_CFLAGS) $(LIBNLGENL3_CFLAGS) MAINTAINERCLEANFILES = Makefile.in
utils/exportfs/exportfs.c +41 -1
73 lines not on this page · raw message
Signature · 1 line
2.53.0
Jeff Layton
maintainerPATCH v2 16/16 mountd/exportd/exportfs: add --no-netlink option to disable netlink
In reply to Jeff LaytonAdd a --no-netlink command-line option (short: -L) and no-netlink= config file setting that disables netlink support, forcing the use of /proc interfaces instead.
For mountd and exportd, the setting lives in their respective [mountd] and [exportd] sections of nfs.conf. exportfs will look in either stanza for the setting.
This is useful for debugging, testing the /proc fallback path, or working around kernel netlink issues.
--- support/export/cache.c | 3 ++- support/export/cache_flush.c | 4 ++- utils/exportd/exportd.c | 10 ++++++-- utils/exportd/exportd.man | 12 +++++++-- utils/exportfs/exportfs.c | 13 +++++++++- utils/exportfs/exportfs.man | 58 ++++++++++++++++++++++++++++++++++++++------ utils/mountd/mountd.c | 9 ++++++- utils/mountd/mountd.man | 9 +++++++ 8 files changed, 102 insertions(+), 16 deletions(-)
support/export/cache.c +2 -1
@@ -111,6 +111,7 @@ static bool path_lookup_error(int err)extern int use_ipaddr;extern int manage_gids; +extern int no_netlink;static void auth_unix_ip (int f) {
@@ -3064,7 +3065,7 @@ void cache_open(void) {int i; -if (cache_nfsd_nl_open () ==0 ) { +if (!no_netlink &&cache_nfsd_nl_open () ==0 ) {if (cache_sunrpc_nl_open () ==0 ) {/* * Check for pending requests, in case any
support/export/cache_flush.c +3 -1
@@ -20,6 +20,8 @@#include "nfslib.h" #include "xlog.h" +extern int no_netlink; +#include <netlink/genl/genl.h> #include <netlink/genl/ctrl.h> #include <netlink/msg.h>
@@ -155,7 +157,7 @@ static void cache_proc_flush(void)void cache_flush (void ) { -if (cache_nl_flush () ==0 ) { +if (!no_netlink &&cache_nl_flush () ==0 ) {xlog (D_NETLINK ,"cache flush via netlink succeeded" );return ; }
utils/exportd/exportd.c +8 -2
@@ -32,6 +32,7 @@ static int num_threads = 1;#define MAX_THREADS 64int manage_gids; +int no_netlink;int use_ipaddr =-1 ; static struct option longopts[] =
@@ -40,13 +41,14 @@ static struct option longopts[] = { "debug", 1, 0, 'd' }, { "help", 0, 0, 'h' }, { "manage-gids", 0, 0, 'g' }, + { "no-netlink", 0, 0, 'L' }, { "num-threads", 1, 0, 't' }, { "log-auth", 0, 0, 'l' }, { "cache-use-ipaddr", 0, 0, 'i' }, { "ttl", 0, 0, 'T' }, { NULL, 0, 0, 0 } }; -static char shortopts[] ="d:fghs:t:liT:" ; +static char shortopts[] ="d:fghs:t:liLT:" ;/* * Signal handlers.
@@ -109,7 +111,7 @@ usage(const char *prog, int n)"Usage: %s [-f|--foreground] [-h|--help] [-d kind|--debug kind]\n" " [-g|--manage-gids] [-l|--log-auth] [-i|--cache-use-ipaddr] [-T|--ttl ttl]\n" " [-s|--state-directory-path path]\n" -" [-t num|--num-threads=num]\n", prog); +" [-t num|--num-threads=num] [-L|--no-netlink]\n", prog);exit (n); }
@@ -124,6 +126,7 @@ read_exportd_conf(char *progname, char **argv)xlog_set_debug (progname); manage_gids =conf_get_bool ("exportd" ,"manage-gids" , manage_gids); + no_netlink =conf_get_bool ("exportd" ,"no-netlink" , no_netlink); num_threads =conf_get_num ("exportd" ,"threads" , num_threads);if (conf_get_bool ("mountd" ,"cache-use-ipaddr" ,0 )) use_ipaddr =2 ;
@@ -171,6 +174,9 @@ main(int argc, char **argv)case 'g' : manage_gids =1 ;break ; +case 'L' : + no_netlink =1 ; +break ;case 'h' :usage (progname,0 );break ;
utils/exportd/exportd.man +10 -2
@@ -106,6 +106,13 @@ the server. Note that the 'primary' group id is not affected so a .B newgroup command on the client will still be effective. This function requires a Linux Kernel with version at least 2.6.21. +.TP +.B \-L " or " \-\-no-netlink +Disable the use of netlink for kernel communication and force the use +of the legacy +.I /proc/net/rpc +interfaces instead. This can be useful for debugging or working around +kernel netlink issues. .SH CONFIGURATION FILE Many of the options that can be set on the command line can also be controlled through values set in the
@@ -120,8 +127,9 @@ Values recognized in the section include .B cache\-use\-ipaddr , .BR ttl , -.BR manage-gids ", and" -.B debug +.BR manage-gids , +.BR no\-netlink ", and" +.B debug which each have the same effect as the option with the same name. .SH FILES .TP 2.5i
utils/exportfs/exportfs.c +12 -1
@@ -49,6 +49,8 @@#include "sunrpc_netlink.h" #endif +int no_netlink; +static void export_all (int verbose);static void exportfs (char *arg,char *options,int verbose);static void unexportfs (char *arg,int verbose);
@@ -109,8 +111,11 @@ read_exportfs_conf(char **argv)xlog_set_debug ("exportfs" );/* NOTE: following uses "mountd" section of nfs.conf !!!! */ + no_netlink =conf_get_bool ("mountd" ,"no-netlink" , no_netlink); s =conf_get_str ("mountd" ,"state-directory-path" );/* Also look in the exportd section */ +if (!no_netlink) + no_netlink =conf_get_bool ("exportd" ,"no-netlink" , no_netlink);if (s ==NULL ) s =conf_get_str ("exportd" ,"state-directory-path" );if (s && !state_setup_basedir (argv[0 ], s))
@@ -145,7 +150,7 @@ main(int argc, char **argv)nfsd_path_init (); -while ((c =getopt (argc, argv,"ad:fhio:ruvs" )) !=EOF ) { +while ((c =getopt (argc, argv,"ad:fhiLo:ruvs" )) !=EOF ) {switch (c) {case 'a' : f_all =1 ;
@@ -162,6 +167,9 @@ main(int argc, char **argv)case 'i' : f_ignore =1 ;break ; +case 'L' : + no_netlink =1 ; +break ;case 'o' : options = optarg;break ;
@@ -500,6 +508,8 @@ static int can_test(void)size_t bufsiz =sizeof (buf);/* Try netlink first: resolve sunrpc genl family */ +if (no_netlink) + goto try_proc; sock =nl_socket_alloc ();if (sock) {if (genl_connect (sock) ==0 ) {
@@ -513,6 +523,7 @@ static int can_test(void) }/* Fallback: /proc probe */ +try_proc: fd =open ("/proc/net/rpc/auth.unix.ip/channel" ,O_WRONLY );if (fd <0 )return 0 ;
utils/exportfs/exportfs.man +50 -8
81 lines not on this page · raw message
utils/mountd/mountd.c +8 -1
@@ -41,6 +41,7 @@ static struct nfs_fh_len *get_rootfh(struct svc_req *, dirpath *, nfs_export **,int reverse_resolve =0 ;int manage_gids; +int no_netlink;int apply_root_cred;int use_ipaddr =-1 ;
@@ -72,6 +73,7 @@ static struct option longopts[] = { "num-threads", 1, 0, 't' }, { "reverse-lookup", 0, 0, 'r' }, { "manage-gids", 0, 0, 'g' }, + { "no-netlink", 0, 0, 'L' }, { "no-udp", 0, 0, 'u' }, { "log-auth", 0, 0, 'l'}, { "cache-use-ipaddr", 0, 0, 'i'},
@@ -667,6 +669,7 @@ read_mountd_conf(char **argv)xlog_set_debug ("mountd" ); manage_gids =conf_get_bool ("mountd" ,"manage-gids" , manage_gids); + no_netlink =conf_get_bool ("mountd" ,"no-netlink" , no_netlink); descriptors =conf_get_num ("mountd" ,"descriptors" , descriptors); port =conf_get_num ("mountd" ,"port" , port); num_threads =conf_get_num ("mountd" ,"threads" , num_threads);
@@ -734,6 +737,9 @@ main(int argc, char **argv)case 'g' : manage_gids =1 ;break ; +case 'L' : + no_netlink =1 ; +break ;case 'o' : descriptors =atoi (optarg);if (descriptors <=0 ) {
@@ -951,6 +957,7 @@ usage(const char *prog, int n)" [-N version|--no-nfs-version version] [-n|--no-tcp]\n" " [-H prog |--ha-callout prog] [-r |--reverse-lookup]\n" " [-s|--state-directory-path path] [-g|--manage-gids]\n" -" [-t num|--num-threads=num] [-u|--no-udp]\n", prog); +" [-t num|--num-threads=num] [-u|--no-udp]\n" +" [-L|--no-netlink]\n", prog);exit (n); }
utils/mountd/mountd.man +9 -0
@@ -284,6 +284,14 @@ the server. Note that the 'primary' group id is not affected so a command on the client will still be effective. This function requires a Linux Kernel with version at least 2.6.21. +.TP +.B \-L " or " \-\-no-netlink +Disable the use of netlink for kernel communication and force the use +of the legacy +.I /proc/net/rpc +interfaces instead. This can be useful for debugging or working around +kernel netlink issues. + .SH CONFIGURATION FILE Many of the options that can be set on the command line can also be controlled through values set in the
@@ -297,6 +305,7 @@ Values recognized in the .B [mountd] section include .BR manage-gids , +.BR no\-netlink , .BR cache\-use\-ipaddr , .BR descriptors , .BR port ,
Signature · 1 line
2.53.0
Quoted 11 lines · Jeff Layton
Minor revision to rebase onto recent upstream changes. Original cover letter follows:
This adds support for the new netlink-based upcalls and downcalls in mountd, exportd and exportfs. With this, mountd is no longer reliant on /proc for sunrpc cache upcalls.
There are also a few bugfixes and cleanups for existing code and documentation too.
Signed-off-by Jeff Layton <[email protected]>
Committed... (tag: nfs-utils-2.9.1-rc3)
steved.
Quoted 51 lines
---
Changes in v2:
- Consolidate UAPI header updates into one patch
- Rebase onto nfs-utils-2-9-1-rc2
- Link to v1: https://lore.kernel.org/r/[email protected]---
Jeff Layton (16):
nfsdctl: move *_netlink.h to support/include/
support/export: remove unnecessary static variables innfsd_fhexpkey lookup
exportfs: remove obsolete legacy mode documentation from manpage support/include: update netlink headers for all cache upcalls
build: add libnl3 and netlink header support for exportd and mountd
xlog: claimD_FAC3asD_NETLINK
exportd/mountd: add netlink support forsvc_exportcache exportd/mountd: add netlink support for the nfsd.fh cache exportd/mountd: add netlink support for the auth.unix.ip cache exportd/mountd: add netlink support for the auth.unix.gid cache mountd/exportd: only use /proc interfaces if netlink setup fails support/export: check for pending requests after opening netlink sockets exportd/mountd: use cache type from notifications to target scanning
exportfs: add netlink support for cache flush with /proc fallback
exportfs: use netlink to probe kernel support, skipexport_testmountd/exportd/exportfs: add --no-netlink option to disable netlinkconfigure.ac | 33 +- support/export/Makefile.am | 5 +- support/export/cache.c | 1892 +++++++++++++++++--- support/export/cache_flush.c | 166 ++ support/include/Makefile.am | 7 +- {utils/nfsdctl => support/include}/lockd_netlink.h | 0 support/include/nfsd_netlink.h | 240 +++ support/include/sunrpc_netlink.h | 84 + support/include/xlog.h | 2 +- support/nfs/cacheio.c | 49 +- utils/exportd/Makefile.am | 2 +- utils/exportd/exportd.c | 10 +- utils/exportd/exportd.man | 12 +- utils/exportfs/Makefile.am | 6 +- utils/exportfs/exportfs.c | 55 +- utils/exportfs/exportfs.man | 79 +- utils/mountd/Makefile.am | 2 +- utils/mountd/mountd.c | 9 +- utils/mountd/mountd.man | 9 + utils/nfsdctl/nfsd_netlink.h | 99 - 20 files changed, 2350 insertions(+), 411 deletions(-)---
base-commit:a06a3251c2eb1316f781149f8b7f9acd9d41e7fc
change-id: 20260316-exportd-netlink-a53bf66ae034Best regards,