Address netns refcount issues for localio
Subsystems (3)
cover Address netns refcount issues for localio
Following are revised versions of 6 patches from the v14 localio series.
The issue addressed is net namespace refcounting.
We don't want to keep a long-term counted reference in the client because that prevents a server container from completely shutting down.
So we avoid taking a reference at all and rely on the per-cpu reference to the server being sufficient to keep the net-ns active. This involves allowing the net-ns exit code to iterate all active clients and clear their ->net pointers (which they need to find the per-cpu-refcount for the nfs_serv).
So:
- embed nfs_uuid_t in nfs_client. This provides a list_head that can be used to find the client. It does add the actual uuid to nfs_client so it is bigger than needed. If that is really a problem we can find a fix.
- When the nfs server confirms that the uuid is local, it moves the nfs_uuid_t onto a per-net-ns list.
- When the net-ns is shutting down - in a "pre_exit" handler, all these nfS_uuid_t have their ->net cleared. There is an rcu_synchronize() call between pre_exit() handlers and exit() handlers so and caller that sees ->net as not NULL can safely check the ->counter
- We now pass the nfs_uuid_t to nfsd_open_local_fh() so it can safely look at ->net in a private rcu_read_lock() section.
I have compile tested this code but nothing more.
Thanks,
NeilBrown
[PATCH 14/25] nfs_common: add NFS LOCALIO auxiliary protocol [PATCH 15/25] nfs_common: introduce nfs_localio_ctx struct and [PATCH 16/25] nfsd: add localio support
[PATCH 17/25] nfsd: implement server support for NFS_LOCALIO_PROGRAM [PATCH 19/25] nfs: add localio support
[PATCH 23/25] nfs: implement client support for NFS_LOCALIO_PROGRAM
PATCH 14/25 nfs_common: add NFS LOCALIO auxiliary protocol enablement
From: Mike Snitzer <[email protected]>
fs/nfs_common/nfslocalio.c provides interfaces that enable an NFS client to generate a nonce (single-use UUID) and associated nfs_uuid_t struct, register it with nfs_common for subsequent lookup and verification by the NFS server and if matched the NFS server populates members in the nfs_uuid_t struct.
When the server populates the structure, it is moved onto a private list allowing the server to invalidate the ->net pointer when the name space is shut down. A maybe_get_net() call under rcu_read_lock() can be used to stablish the net pointer active use.
nfs_common's nfs_uuids list is the basis for localio enablement, as such it has members that point to nfsd memory for direct use by the client (e.g. 'net' is the server's network namespace, through it the client can access nn->nfsd_serv with proper rcu read access).
--- fs/nfs_common/Makefile | 3 ++ fs/nfs_common/nfslocalio.c | 108 +++++++++++++++++++++++++++++++++++++ include/linux/nfslocalio.h | 34 ++++++++++++ 3 files changed, 145 insertions(+) create mode 100644 fs/nfs_common/nfslocalio.c create mode 100644 include/linux/nfslocalio.h
fs/nfs_common/Makefile +3 -0
@@ -6,6 +6,9 @@ obj-$(CONFIG_NFS_ACL_SUPPORT) += nfs_acl.o nfs_acl-objs := nfsacl.o +obj-$(CONFIG_NFS_COMMON_LOCALIO_SUPPORT) += nfs_localio.o +nfs_localio-objs := nfslocalio.o + obj-$(CONFIG_GRACE_PERIOD) += grace.o obj-$(CONFIG_NFS_V4_2_SSC_HELPER) += nfs_ssc.o
fs/nfs_common/nfslocalio.c new file +108 -0
108 lines not on this page · raw message
include/linux/nfslocalio.h new file +34 -0
@@ -0,0 +1,34 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* Copyright (C) 2024 Mike Snitzer <[email protected]> +*/ +#ifndef __LINUX_NFSLOCALIO_H +#define __LINUX_NFSLOCALIO_H + +#include <linux/list.h> +#include <linux/uuid.h> +#include <linux/sunrpc/svcauth.h> +#include <linux/nfs.h> +#include <net/net_namespace.h> + +/* +* Useful to allow a client to negotiate if localio +* possible with its server. +* +* See Documentation/filesystems/nfs/localio.rst for more detail. +*/ +typedef struct { +uuid_t uuid; +struct list_head list; +struct net *net;/* nfsd's network namespace */ +struct auth_domain *dom;/* auth_domain for localio */ +}nfs_uuid_t ; + +void nfs_uuid_begin (nfs_uuid_t *); +void nfs_uuid_end (nfs_uuid_t *); +void nfs_uuid_is_local (const uuid_t *,struct list_head *, +struct net *,struct auth_domain *); +void nfs_uuid_invalidate_clients (struct list_head *list); +void nfs_uuid_invalidate_one_client (nfs_uuid_t *nfs_uuid); + +#endif /* __LINUX_NFSLOCALIO_H */
Signature · 1 line
2.44.0
PATCH 15/25 nfs_common: introduce nfs_localio_ctx struct and interfaces
In reply to NeilBrownFrom: Mike Snitzer <[email protected]>
Introduce struct nfs_localio_ctx and the interfaces nfs_localio_ctx_alloc() and nfs_localio_ctx_free(). The next commit will introduce nfsd_open_local_fh() which returns a nfs_localio_ctx structure.
Also, expose localio's required NFSD symbols to NFS client:
- Cache nfsd_open_local_fh symbol and other required NFSD symbols in a globally accessible 'nfs_to' nfs_to_nfsd_t struct. Add interfaces get_nfs_to_nfsd_symbols() and put_nfs_to_nfsd_symbols() to allow each NFS client to take a reference on NFSD symbols.
- Apologies for the DEFINE_NFS_TO_NFSD_SYMBOL macro that makes defining get_##NFSD_SYMBOL() and put_##NFSD_SYMBOL() functions far simpler (and avoids cut-n-paste bugs, which is what motivated the development and use of a macro for this). But as C macros go it is a very simple one and there are many like it all over the kernel.
- Given the unique nature of NFS LOCALIO being an optional feature that when used requires NFS share access to NFSD memory: a unique bridging of NFSD resources to NFS (via nfs_common) is needed. But that bridge must be dynamic, hence the use of symbol_request() and symbol_put(). Proposed ideas to accomolish the same without using symbol_{request,put} would be far more tedious to implement and very likely no easier to review. Anyway: sorry NeilBrown...
- Despite the use of indirect function calls, caching these nfsd symbols for use by the client offers a ~10% performance win (compared to always doing get+call+put) for high IOPS workloads.
- Introduce nfsd_file_file() wrapper that provides access to nfsd_file's backing file. Keeps nfsd_file structure opaque to NFS client (as suggested by Jeff Layton).
- The addition of nfsd_file_get, nfsd_file_put and nfsd_file_file symbols prepares for the NFS client to use nfsd_file for localio.
--- fs/nfs_common/nfslocalio.c | 159 +++++++++++++++++++++++++++++++++++++ fs/nfsd/filecache.c | 25 ++++++ fs/nfsd/filecache.h | 1 + fs/nfsd/nfssvc.c | 5 ++ include/linux/nfslocalio.h | 38 +++++++++ 5 files changed, 228 insertions(+)
fs/nfs_common/nfslocalio.c +159 -0
@@ -106,3 +106,162 @@ void nfs_uuid_invalidate_one_client(nfs_uuid_t *nfs_uuid) } }EXPORT_SYMBOL_GPL (nfs_uuid_invalidate_one_client); + +/* +* The nfs localio code needs to call into nfsd using various symbols (below), +* but cannot be statically linked, because that will make the nfs module +* depend on the nfsd module. +* +* Instead, do dynamic linking to the nfsd module (via nfs_common module). The +* nfs_common module will only hold a reference on nfsd when localio is in use. +* This allows some sanity checking, like giving up on localio if nfsd isn't loaded. +*/ +static DEFINE_SPINLOCK (nfs_to_nfsd_lock); +nfs_to_nfsd_t nfs_to; +EXPORT_SYMBOL_GPL (nfs_to); + +/* Macro to define nfs_to get and put methods, avoids copy-n-paste bugs */ +#define DEFINE_NFS_TO_NFSD_SYMBOL (NFSD_SYMBOL ) \ +static nfs_to_##NFSD_SYMBOL##_t get_##NFSD_SYMBOL(void) \ +{ \ + return symbol_request(NFSD_SYMBOL); \ +} \ +static void put_##NFSD_SYMBOL(void) \ +{ \ + symbol_put(NFSD_SYMBOL); \ + nfs_to.NFSD_SYMBOL = NULL; \ +} + +/* The nfs localio code needs to call into nfsd to map filehandle -> struct nfsd_file */ +extern struct nfs_localio_ctx * +nfsd_open_local_fh (nfs_uuid_t *,struct rpc_clnt *, +const struct cred *,const struct nfs_fh *,const fmode_t ); +DEFINE_NFS_TO_NFSD_SYMBOL (nfsd_open_local_fh); + +/* The nfs localio code needs to call into nfsd to acquire the nfsd_file */ +extern struct nfsd_file *nfsd_file_get (struct nfsd_file *nf); +DEFINE_NFS_TO_NFSD_SYMBOL (nfsd_file_get); + +/* The nfs localio code needs to call into nfsd to release the nfsd_file */ +extern void nfsd_file_put (struct nfsd_file *nf); +DEFINE_NFS_TO_NFSD_SYMBOL (nfsd_file_put); + +/* The nfs localio code needs to call into nfsd to access the nf->nf_file */ +extern struct file *nfsd_file_file (struct nfsd_file *nf); +DEFINE_NFS_TO_NFSD_SYMBOL (nfsd_file_file); + +/* The nfs localio code needs to call into nfsd to release nn->nfsd_serv */ +extern void nfsd_serv_put (struct nfsd_net *nn); +DEFINE_NFS_TO_NFSD_SYMBOL (nfsd_serv_put); +#undef DEFINE_NFS_TO_NFSD_SYMBOL + +static struct kmem_cache *nfs_localio_ctx_cache; + +struct nfs_localio_ctx *nfs_localio_ctx_alloc (void ) +{ +return kmem_cache_alloc (nfs_localio_ctx_cache, +GFP_KERNEL | __GFP_ZERO); +} +EXPORT_SYMBOL_GPL (nfs_localio_ctx_alloc); + +void nfs_localio_ctx_free (struct nfs_localio_ctx *localio) +{ +if (localio->nf) + nfs_to.nfsd_file_put (localio->nf); +if (localio->nn) + nfs_to.nfsd_serv_put (localio->nn); +kmem_cache_free (nfs_localio_ctx_cache, localio); +} +EXPORT_SYMBOL_GPL (nfs_localio_ctx_free); + +bool get_nfs_to_nfsd_symbols (void ) +{ +spin_lock (&nfs_to_nfsd_lock); + +/* Only get symbols on first reference */ +if (refcount_read (&nfs_to.ref) ==0 ) +refcount_set (&nfs_to.ref,1 ); +else { +refcount_inc (&nfs_to.ref); +spin_unlock (&nfs_to_nfsd_lock); +return true; + } + + nfs_to.nfsd_open_local_fh =get_nfsd_open_local_fh (); +if (!nfs_to.nfsd_open_local_fh) + goto out_nfsd_open_local_fh; + + nfs_to.nfsd_file_get =get_nfsd_file_get (); +if (!nfs_to.nfsd_file_get) + goto out_nfsd_file_get; + + nfs_to.nfsd_file_put =get_nfsd_file_put (); +if (!nfs_to.nfsd_file_put) + goto out_nfsd_file_put; + + nfs_to.nfsd_file_file =get_nfsd_file_file (); +if (!nfs_to.nfsd_file_file) + goto out_nfsd_file_file; + + nfs_to.nfsd_serv_put =get_nfsd_serv_put (); +if (!nfs_to.nfsd_serv_put) + goto out_nfsd_serv_put; + +spin_unlock (&nfs_to_nfsd_lock); +return true; + +out_nfsd_serv_put: +put_nfsd_file_file (); +out_nfsd_file_file: +put_nfsd_file_put (); +out_nfsd_file_put: +put_nfsd_file_get (); +out_nfsd_file_get: +put_nfsd_open_local_fh (); +out_nfsd_open_local_fh: +spin_unlock (&nfs_to_nfsd_lock); +return false; +} +EXPORT_SYMBOL_GPL (get_nfs_to_nfsd_symbols); + +void put_nfs_to_nfsd_symbols (void ) +{ +spin_lock (&nfs_to_nfsd_lock); + +if (!refcount_dec_and_test (&nfs_to.ref)) + goto out; + +put_nfsd_open_local_fh (); +put_nfsd_file_get (); +put_nfsd_file_put (); +put_nfsd_file_file (); +put_nfsd_serv_put (); +out: +spin_unlock (&nfs_to_nfsd_lock); +} +EXPORT_SYMBOL_GPL (put_nfs_to_nfsd_symbols); + +static int __init nfslocalio_init (void ) +{ +refcount_set (&nfs_to.ref,0 ); + + nfs_to.nfsd_open_local_fh =NULL ; + nfs_to.nfsd_file_get =NULL ; + nfs_to.nfsd_file_put =NULL ; + nfs_to.nfsd_file_file =NULL ; + nfs_to.nfsd_serv_put =NULL ; + + nfs_localio_ctx_cache =KMEM_CACHE (nfs_localio_ctx,0 ); +if (!nfs_localio_ctx_cache) +return -ENOMEM ; + +return 0 ; +} + +static void __exit nfslocalio_exit(void) +{ +kmem_cache_destroy (nfs_localio_ctx_cache); +} + +module_init (nfslocalio_init); +module_exit (nfslocalio_exit);
fs/nfsd/filecache.c +25 -0
@@ -39,6 +39,7 @@#include <linux/fsnotify.h> #include <linux/seq_file.h> #include <linux/rhashtable.h> +#include <linux/nfslocalio.h> #include "vfs.h" #include "nfsd.h"
@@ -345,6 +346,10 @@ nfsd_file_get(struct nfsd_file *nf)return nf;return NULL ; } +EXPORT_SYMBOL_GPL (nfsd_file_get); + +/* Compile time type checking, not used by anything */ +static nfs_to_nfsd_file_get_t __maybe_unused nfsd_file_get_typecheck = nfsd_file_get;/** * nfsd_file_put - put the reference to a nfsd_file
@@ -389,6 +394,26 @@ nfsd_file_put(struct nfsd_file *nf)if (refcount_dec_and_test (&nf->nf_ref))nfsd_file_free (nf); } +EXPORT_SYMBOL_GPL (nfsd_file_put); + +/* Compile time type checking, not used by anything */ +static nfs_to_nfsd_file_put_t __maybe_unused nfsd_file_put_typecheck = nfsd_file_put; + +/** +* nfsd_file_file - get the backing file of an nfsd_file +* @nf: nfsd_file of which to access the backing file. +* +* Return backing file for @nf. +*/ +struct file * +nfsd_file_file (struct nfsd_file *nf) +{ +return nf->nf_file; +} +EXPORT_SYMBOL_GPL (nfsd_file_file); + +/* Compile time type checking, not used by anything */ +static nfs_to_nfsd_file_file_t __maybe_unused nfsd_file_file_typecheck = nfsd_file_file;static void nfsd_file_dispose_list (struct list_head *dispose)
fs/nfsd/filecache.h +1 -0
@@ -56,6 +56,7 @@ int nfsd_file_cache_start_net(struct net *net);void nfsd_file_cache_shutdown_net (struct net *net);void nfsd_file_put (struct nfsd_file *nf);struct nfsd_file *nfsd_file_get (struct nfsd_file *nf); +struct file *nfsd_file_file (struct nfsd_file *nf);void nfsd_file_close_inode_sync (struct inode *inode);void nfsd_file_net_dispose (struct nfsd_net *nn);bool nfsd_file_is_cached (struct inode *inode);
fs/nfsd/nfssvc.c +5 -0
@@ -19,6 +19,7 @@#include <linux/sunrpc/svc_xprt.h> #include <linux/lockd/bind.h> #include <linux/nfsacl.h> +#include <linux/nfslocalio.h> #include <linux/seq_file.h> #include <linux/inetdevice.h> #include <net/addrconf.h>
@@ -201,6 +202,10 @@ void nfsd_serv_put(struct nfsd_net *nn) {percpu_ref_put (&nn->nfsd_serv_ref); } +EXPORT_SYMBOL_GPL (nfsd_serv_put); + +/* Compile time type checking, not used by anything */ +static nfs_to_nfsd_serv_put_t __maybe_unused nfsd_serv_put_typecheck = nfsd_serv_put;static void nfsd_serv_done (struct percpu_ref *ref) {
include/linux/nfslocalio.h +38 -0
@@ -7,6 +7,8 @@#include <linux/list.h> #include <linux/uuid.h> +#include <linux/refcount.h> +#include <linux/sunrpc/clnt.h> #include <linux/sunrpc/svcauth.h> #include <linux/nfs.h> #include <net/net_namespace.h>
@@ -31,4 +33,40 @@ void nfs_uuid_is_local(const uuid_t *, struct list_head *,void nfs_uuid_invalidate_clients (struct list_head *list);void nfs_uuid_invalidate_one_client (nfs_uuid_t *nfs_uuid); +struct nfsd_file ; +struct nfsd_net ; + +struct nfs_localio_ctx { +struct nfsd_file *nf; +struct nfsd_net *nn; +}; + +typedef struct nfs_localio_ctx * +(*nfs_to_nfsd_open_local_fh_t )(nfs_uuid_t *, +struct rpc_clnt *,const struct cred *, +const struct nfs_fh *,const fmode_t ); +typedef struct nfsd_file * (*nfs_to_nfsd_file_get_t )(struct nfsd_file *); +typedef void (*nfs_to_nfsd_file_put_t )(struct nfsd_file *); +typedef struct file * (*nfs_to_nfsd_file_file_t )(struct nfsd_file *); +typedef unsigned int (*nfs_to_nfsd_net_id_value_t )(void ); +typedef void (*nfs_to_nfsd_serv_put_t )(struct nfsd_net *); + +typedef struct { +refcount_t ref; +nfs_to_nfsd_open_local_fh_t nfsd_open_local_fh; +nfs_to_nfsd_file_get_t nfsd_file_get; +nfs_to_nfsd_file_put_t nfsd_file_put; +nfs_to_nfsd_file_file_t nfsd_file_file; +nfs_to_nfsd_net_id_value_t nfsd_net_id_value; +nfs_to_nfsd_serv_put_t nfsd_serv_put; +}nfs_to_nfsd_t ; + +extern nfs_to_nfsd_t nfs_to; + +bool get_nfs_to_nfsd_symbols (void ); +void put_nfs_to_nfsd_symbols (void ); + +struct nfs_localio_ctx *nfs_localio_ctx_alloc (void ); +void nfs_localio_ctx_free (struct nfs_localio_ctx *); +#endif /* __LINUX_NFSLOCALIO_H */
Signature · 1 line
2.44.0
PATCH 16/25 nfsd: add localio support
In reply to NeilBrownFrom: Weston Andros Adamson <[email protected]>
Add server support for bypassing NFS for localhost reads, writes, and commits. This is only useful when both the client and server are running on the same host.
If nfsd_open_local_fh() fails then the NFS client will both retry and fallback to normal network-based read, write and commit operations if localio is no longer supported.
Care is taken to ensure the same NFS security mechanisms are used (authentication, etc) regardless of whether localio or regular NFS access is used. The auth_domain established as part of the traditional NFS client access to the NFS server is also used for localio. Store auth_domain for localio in nfsd_uuid_t and transfer it to the client if it is local to the server.
Relative to containers, localio gives the client access to the network namespace the server has. This is required to allow the client to access the server's per-namespace nfsd_net struct.
CONFIG_NFSD_LOCALIO controls the server enablement for localio. A later commit will add CONFIG_NFS_LOCALIO to allow the client enablement.
This commit also introduces the use of nfsd's percpu_ref to interlock nfsd_destroy_serv and nfsd_open_local_fh, to ensure nn->nfsd_serv is not destroyed while in use by nfsd_open_local_fh, and warrants a more detailed explanation:
nfsd_open_local_fh uses nfsd_serv_try_get before opening its file handle and then the reference must be dropped by the caller using nfsd_serv_put (via nfs_localio_ctx_free).
Verified to fix an easy to hit crash that would occur if an nfsd instance running in a container, with a localio client mounted, is shutdown. Upon restart of the container and associated nfsd the client would go on to crash due to NULL pointer dereference that occurred due to the nfs client's localio attempting to nfsd_open_local_fh(), using nn->nfsd_serv, without having a proper reference on nn->nfsd_serv.
--- fs/Kconfig | 3 ++ fs/nfsd/Kconfig | 16 +++++++ fs/nfsd/Makefile | 1 + fs/nfsd/filecache.c | 2 +- fs/nfsd/localio.c | 114 ++++++++++++++++++++++++++++++++++++++++++++ fs/nfsd/trace.h | 3 +- fs/nfsd/vfs.h | 8 ++++ 7 files changed, 145 insertions(+), 2 deletions(-) create mode 100644 fs/nfsd/localio.c
fs/Kconfig +3 -0
@@ -377,6 +377,9 @@ config NFS_ACL_SUPPORT tristate select FS_POSIX_ACL +config NFS_COMMON_LOCALIO_SUPPORT + bool + config NFS_COMMON bool depends on NFSD || NFS_FS || LOCKD
fs/nfsd/Kconfig +16 -0
@@ -90,6 +90,22 @@ config NFSD_V4 If unsure, say N. +config NFSD_LOCALIO + bool "NFS server support for the LOCALIO auxiliary protocol" + depends on NFSD + select NFS_COMMON_LOCALIO_SUPPORT + default n + help + Some NFS servers support an auxiliary NFS LOCALIO protocol + that is not an official part of the NFS protocol. + + This option enables support for the LOCALIO protocol in the + kernel's NFS server. Enable this to permit local NFS clients + to bypass the network when issuing reads and writes to the + local NFS server. + + If unsure, say N. + config NFSD_PNFS bool
fs/nfsd/Makefile +1 -0
@@ -23,3 +23,4 @@ nfsd-$(CONFIG_NFSD_PNFS) += nfs4layouts.o nfsd-$(CONFIG_NFSD_BLOCKLAYOUT) += blocklayout.o blocklayoutxdr.o nfsd-$(CONFIG_NFSD_SCSILAYOUT) += blocklayout.o blocklayoutxdr.o nfsd-$(CONFIG_NFSD_FLEXFILELAYOUT) += flexfilelayout.o flexfilelayoutxdr.o +nfsd-$(CONFIG_NFSD_LOCALIO) += localio.o
fs/nfsd/filecache.c +1 -1
@@ -53,7 +53,7 @@#define NFSD_FILE_CACHE_UP (0)/* We only care about NFSD_MAY_READ/WRITE for this cache */ -#define NFSD_FILE_MAY_MASK (NFSD_MAY_READ|NFSD_MAY_WRITE) +#define NFSD_FILE_MAY_MASK (NFSD_MAY_READ|NFSD_MAY_WRITE|NFSD_MAY_LOCALIO) static DEFINE_PER_CPU(unsigned long, nfsd_file_cache_hits); static DEFINE_PER_CPU(unsigned long, nfsd_file_acquisitions);
fs/nfsd/localio.c new file +114 -0
114 lines not on this page · raw message
fs/nfsd/trace.h +2 -1
@@ -86,7 +86,8 @@ DEFINE_NFSD_XDR_ERR_EVENT(cant_encode); { NFSD_MAY_NOT_BREAK_LEASE, "NOT_BREAK_LEASE" }, \ { NFSD_MAY_BYPASS_GSS, "BYPASS_GSS" }, \ { NFSD_MAY_READ_IF_EXEC, "READ_IF_EXEC" }, \ - { NFSD_MAY_64BIT_COOKIE, "64BIT_COOKIE" }) + { NFSD_MAY_64BIT_COOKIE, "64BIT_COOKIE" }, \ + { NFSD_MAY_LOCALIO, "LOCALIO" }) TRACE_EVENT(nfsd_compound, TP_PROTO(
fs/nfsd/vfs.h +8 -0
@@ -8,6 +8,7 @@#include <linux/fs.h> #include <linux/posix_acl.h> +#include <linux/nfslocalio.h> #include "nfsfh.h" #include "nfsd.h"
@@ -33,6 +34,8 @@#define NFSD_MAY_64BIT_COOKIE 0x1000/* 64 bit readdir cookies for >= NFSv3 */ +#define NFSD_MAY_LOCALIO 0x2000/* for tracing, reflects when localio used */ +#define NFSD_MAY_CREATE (NFSD_MAY_EXEC|NFSD_MAY_WRITE)#define NFSD_MAY_REMOVE (NFSD_MAY_EXEC|NFSD_MAY_WRITE|NFSD_MAY_TRUNC)
@@ -158,6 +161,11 @@ __be32 nfsd_permission(struct svc_cred *cred, struct svc_export *exp,void nfsd_filp_close (struct file *fp); +struct nfs_localio_ctx * +nfsd_open_local_fh (nfs_uuid_t *, +struct rpc_clnt *,const struct cred *, +const struct nfs_fh *,const fmode_t ); +static inline int fh_want_write (struct svc_fh *fh) {int ret;
Signature · 1 line
2.44.0
PATCH 17/25 nfsd: implement server support for NFS_LOCALIO_PROGRAM
In reply to NeilBrownFrom: Mike Snitzer <[email protected]>
The LOCALIO auxiliary RPC protocol consists of a single "UUID_IS_LOCAL" RPC method that allows the Linux NFS client to verify the local Linux NFS server can see the nonce (single-use UUID) the client generated and made available in nfs_common. The server expects this protocol to use the same transport as NFS and NFSACL for its RPCs. This protocol isn't part of an IETF standard, nor does it need to be considering it is Linux-to-Linux auxiliary RPC protocol that amounts to an implementation detail.
The UUID_IS_LOCAL method encodes the client generated uuid_t in terms of the fixed UUID_SIZE (16 bytes). The fixed size opaque encode and decode XDR methods are used instead of the less efficient variable sized methods.
The RPC program number for the NFS_LOCALIO_PROGRAM is 400122 (as assigned by IANA, see https://www.iana.org/assignments/rpc-program-numbers/ ):
Linux Kernel Organization 400122 nfslocalio
After a successful handshake the client will hold a non-counted reference to the server's network namespace. On namespace shutdown these non-counted references will be invalidated.
--- fs/nfsd/localio.c | 77 +++++++++++++++++++++++++++++++++++++++++++++ fs/nfsd/netns.h | 2 ++ fs/nfsd/nfsctl.c | 16 ++++++++++ fs/nfsd/nfsd.h | 4 +++ fs/nfsd/nfssvc.c | 23 +++++++++++++- include/linux/nfs.h | 7 +++++ 6 files changed, 128 insertions(+), 1 deletion(-)
fs/nfsd/localio.c +77 -0
@@ -13,12 +13,15 @@#include <linux/nfs.h> #include <linux/nfs_common.h> #include <linux/nfslocalio.h> +#include <linux/nfs_fs.h> +#include <linux/nfs_xdr.h> #include <linux/string.h> #include "nfsd.h" #include "vfs.h" #include "netns.h" #include "filecache.h" +#include "cache.h" /** * nfsd_open_local_fh - lookup a local filehandle @nfs_fh and map to nfsd_file
@@ -112,3 +115,77 @@ EXPORT_SYMBOL_GPL(nfsd_open_local_fh);/* Compile time type checking, not used by anything */ static nfs_to_nfsd_open_local_fh_t __maybe_unused nfsd_open_local_fh_typecheck = nfsd_open_local_fh; + +/* +* UUID_IS_LOCAL XDR functions +*/ + +static __be32 localio_proc_null (struct svc_rqst *rqstp) +{ +return rpc_success; +} + +struct localio_uuidarg { +uuid_t uuid; +}; + +static __be32 localio_proc_uuid_is_local (struct svc_rqst *rqstp) +{ +struct localio_uuidarg *argp = rqstp->rq_argp; +struct net *net =SVC_NET (rqstp); +struct nfsd_net *nn =net_generic (net, nfsd_net_id); + +nfs_uuid_is_local (&argp->uuid, &nn->local_clients, + net, rqstp->rq_client); + +return rpc_success; +} + +static bool localio_decode_uuidarg (struct svc_rqst *rqstp, +struct xdr_stream *xdr) +{ +struct localio_uuidarg *argp = rqstp->rq_argp; +u8 uuid[UUID_SIZE ]; + +if (decode_opaque_fixed (xdr, uuid,UUID_SIZE )) +return false; +import_uuid (&argp->uuid, uuid); + +return true; +} + +static const struct svc_procedure localio_procedures1[] = { + [LOCALIOPROC_NULL ] = { + .pc_func = localio_proc_null, + .pc_decode = nfssvc_decode_voidarg, + .pc_encode = nfssvc_encode_voidres, + .pc_argsize =sizeof (struct nfsd_voidargs ), + .pc_ressize =sizeof (struct nfsd_voidres ), + .pc_cachetype =RC_NOCACHE , + .pc_xdrressize =0 , + .pc_name ="NULL" , + }, + [LOCALIOPROC_UUID_IS_LOCAL ] = { + .pc_func = localio_proc_uuid_is_local, + .pc_decode = localio_decode_uuidarg, + .pc_encode = nfssvc_encode_voidres, + .pc_argsize =sizeof (struct localio_uuidarg ), + .pc_argzero =sizeof (struct localio_uuidarg ), + .pc_ressize =sizeof (struct nfsd_voidres ), + .pc_cachetype =RC_NOCACHE , + .pc_name ="UUID_IS_LOCAL" , + }, +}; + +#define LOCALIO_NR_PROCEDURES ARRAY_SIZE(localio_procedures1) +static DEFINE_PER_CPU_ALIGNED(unsigned long, + localio_count[LOCALIO_NR_PROCEDURES]); +const struct svc_version localio_version1 = { + .vs_vers =1 , + .vs_nproc =LOCALIO_NR_PROCEDURES , + .vs_proc = localio_procedures1, + .vs_dispatch = nfsd_dispatch, + .vs_count = localio_count, + .vs_xdrsize =XDR_QUADLEN (UUID_SIZE ), + .vs_hidden = true, +};
fs/nfsd/netns.h +2 -0
@@ -216,6 +216,8 @@ struct nfsd_net {/* last time an admin-revoke happened for NFSv4.0 */ time64_t nfs40_last_revoke; +/* Local clients to be invalidated when net is shut down */ +struct list_head local_clients; };/* Simple check to find out if a given net was properly initialized */
fs/nfsd/nfsctl.c +16 -0
@@ -18,6 +18,7 @@#include <linux/sunrpc/svc.h> #include <linux/module.h> #include <linux/fsnotify.h> +#include <linux/nfslocalio.h> #include "idmap.h" #include "nfsd.h"
@@ -2257,6 +2258,7 @@ static __net_init int nfsd_net_init(struct net *net)get_random_bytes (&nn->siphash_key,sizeof (nn->siphash_key));seqlock_init (&nn->writeverf_lock);nfsd_proc_stat_init (net); +INIT_LIST_HEAD (&nn->local_clients);return 0 ;
@@ -2268,6 +2270,19 @@ static __net_init int nfsd_net_init(struct net *net)return retval; } +/** +* nfsd_net_pre_exit - Disconnect localio clients from net namespace +* @net: a network namespace that is about to be destroyed +* +* This invalidated ->net pointers held by localio clients +* while they can still safely access nn->counter. +*/ +static __net_exit void nfsd_net_pre_exit(struct net *net) +{ +struct nfsd_net *nn =net_generic (net, nfsd_net_id); + +nfs_uuid_invalidate_clients (&nn->local_clients); +}/** * nfsd_net_exit - Release the nfsd_net portion of a net namespace * @net: a network namespace that is about to be destroyed
@@ -2285,6 +2300,7 @@ static __net_exit void nfsd_net_exit(struct net *net)static struct pernet_operations nfsd_net_ops = { .init = nfsd_net_init, + .pre_exit = nfsd_net_pre_exit, .exit = nfsd_net_exit, .id = &nfsd_net_id, .size =sizeof (struct nfsd_net ),
fs/nfsd/nfsd.h +4 -0
@@ -146,6 +146,10 @@ extern const struct svc_version nfsd_acl_version3;#endif #endif +#if IS_ENABLED (CONFIG_NFSD_LOCALIO ) +extern const struct svc_version localio_version1; +#endif +struct nfsd_net ;enum vers_op {NFSD_SET ,NFSD_CLEAR ,NFSD_TEST ,NFSD_AVAIL };
fs/nfsd/nfssvc.c +22 -1
@@ -80,6 +80,15 @@ DEFINE_SPINLOCK(nfsd_drc_lock);unsigned long nfsd_drc_max_mem;unsigned long nfsd_drc_mem_used; +#if IS_ENABLED (CONFIG_NFSD_LOCALIO ) +static const struct svc_version *localio_versions[] = { + [1 ] = &localio_version1, +}; + +#define NFSD_LOCALIO_NRVERS ARRAY_SIZE(localio_versions) + +#endif /* CONFIG_NFSD_LOCALIO */ + #if defined(CONFIG_NFSD_V2_ACL) || defined(CONFIG_NFSD_V3_ACL) static const struct svc_version *nfsd_acl_version[] = { # if defined(CONFIG_NFSD_V2_ACL)
@@ -128,6 +137,18 @@ struct svc_program nfsd_programs[] = { .pg_rpcbind_set = nfsd_acl_rpcbind_set, },#endif /* defined(CONFIG_NFSD_V2_ACL) || defined(CONFIG_NFSD_V3_ACL) */ +#if IS_ENABLED (CONFIG_NFSD_LOCALIO ) + { + .pg_prog = NFS_LOCALIO_PROGRAM, + .pg_nvers = NFSD_LOCALIO_NRVERS, + .pg_vers = localio_versions, + .pg_name = "nfslocalio", + .pg_class = "nfsd", + .pg_authenticate = svc_set_client, + .pg_init_request = svc_generic_init_request, + .pg_rpcbind_set = svc_generic_rpcbind_set, + } +#endif /* IS_ENABLED(CONFIG_NFSD_LOCALIO) */ };bool nfsd_support_version (int vers)
@@ -949,7 +970,7 @@ nfsd(void *vrqstp) }/** -* nfsd_dispatch - Process an NFS or NFSACL Request +* nfsd_dispatch - Process an NFS or NFSACL or LOCALIO Request * @rqstp: incoming request * * This RPC dispatcher integrates the NFS server's duplicate reply cache.
include/linux/nfs.h +7 -0
@@ -13,6 +13,13 @@#include <linux/crc32.h> #include <uapi/linux/nfs.h> +/* The localio program is entirely private to Linux and is +* NOT part of the uapi. +*/ +#define NFS_LOCALIO_PROGRAM 400122 +#define LOCALIOPROC_NULL 0 +#define LOCALIOPROC_UUID_IS_LOCAL 1 +/* * This is the kernel NFS client file handle representation */
Signature · 1 line
2.44.0
PATCH 19/25 nfs: add localio support
In reply to NeilBrownFrom: Weston Andros Adamson <[email protected]>
Add client support for bypassing NFS for localhost reads, writes, and commits. This is only useful when the client and the server are running on the same host.
nfs_local_probe() is stubbed out, later commits will enable client and server handshake via a Linux-only LOCALIO auxiliary RPC protocol.
This has dynamic binding with the nfsd module (via nfs_localio module which is part of nfs_common). Localio will only work if nfsd is already loaded.
The "localio_enabled" nfs kernel module parameter can be used to disable and enable the ability to use localio support.
CONFIG_NFS_LOCALIO controls the client enablement.
Lastly, localio uses an nfsd_file to initiate all IO. To make proper use of nfsd_file (and nfsd's filecache) its lifetime (duration before nfsd_file_put is called) must extend until after commit, read and write operations. So rather than immediately call nfsd_file_put() in nfs_local_open_fh(), nfsd_file_put() isn't called until nfs_local_pgio_release() for read/write and not until nfs_local_release_commit_data() for commit. The same applies to the reference held on nfsd's nn->nfsd_serv. Both object lifetimes and associated references are managed through calls to nfs_localio_ctx_free().
--- fs/nfs/Kconfig | 16 + fs/nfs/Makefile | 1 + fs/nfs/client.c | 10 + fs/nfs/internal.h | 45 +++ fs/nfs/localio.c | 605 ++++++++++++++++++++++++++++++++++++++ fs/nfs/nfstrace.h | 61 ++++ fs/nfs/pagelist.c | 4 + fs/nfs/write.c | 3 + include/linux/nfs.h | 2 + include/linux/nfs_fs_sb.h | 9 + 10 files changed, 756 insertions(+) create mode 100644 fs/nfs/localio.c
fs/nfs/Kconfig +16 -0
@@ -87,6 +87,22 @@ config NFS_V4 If unsure, say Y. +config NFS_LOCALIO + bool "NFS client support for the LOCALIO auxiliary protocol" + depends on NFS_FS + select NFS_COMMON_LOCALIO_SUPPORT + default n + help + Some NFS servers support an auxiliary NFS LOCALIO protocol + that is not an official part of the NFS protocol. + + This option enables support for the LOCALIO protocol in the + kernel's NFS client. Enable this to permit local NFS clients + to bypass the network when issuing reads and writes to the + local NFS server. + + If unsure, say N. + config NFS_SWAP bool "Provide swap over NFS support" default n
fs/nfs/Makefile +1 -0
@@ -13,6 +13,7 @@ nfs-y := client.o dir.o file.o getroot.o inode.o super.o \ nfs-$(CONFIG_ROOT_NFS) += nfsroot.o nfs-$(CONFIG_SYSCTL) += sysctl.o nfs-$(CONFIG_NFS_FSCACHE) += fscache.o +nfs-$(CONFIG_NFS_LOCALIO) += localio.o obj-$(CONFIG_NFS_V2) += nfsv2.o nfsv2-y := nfs2super.o proc.o nfs2xdr.o
fs/nfs/client.c +10 -0
@@ -178,6 +178,13 @@ struct nfs_client *nfs_alloc_client(const struct nfs_client_initdata *cl_init) clp->cl_max_connect = cl_init->max_connect ? cl_init->max_connect :1 ; clp->cl_net =get_net (cl_init->net); +#if IS_ENABLED (CONFIG_NFS_LOCALIO ) +seqlock_init (&clp->cl_boot_lock); +ktime_get_real_ts64 (&clp->cl_nfssvc_boot); + clp->cl_uuid.net =NULL ; +spin_lock_init (&clp->cl_localio_lock); +#endif /* CONFIG_NFS_LOCALIO */ + clp->cl_principal ="*" ; clp->cl_xprtsec = cl_init->xprtsec;return clp;
@@ -233,6 +240,8 @@ static void pnfs_init_server(struct nfs_server *server)*/ void nfs_free_client (struct nfs_client *clp) { +nfs_local_disable (clp); +/* -EIO all pending I/O */ if (!IS_ERR (clp->cl_rpcclient))rpc_shutdown_client (clp->cl_rpcclient);
@@ -424,6 +433,7 @@ struct nfs_client *nfs_get_client(const struct nfs_client_initdata *cl_init)list_add_tail (&new->cl_share_link, &nn->nfs_client_list);spin_unlock (&nn->nfs_client_lock); +nfs_local_probe (new);return rpc_ops->init_client (new, cl_init); }
fs/nfs/internal.h +45 -0
51 lines not on this page · raw message
fs/nfs/localio.c new file +605 -0
605 lines not on this page · raw message
fs/nfs/nfstrace.h +61 -0
67 lines not on this page · raw message
fs/nfs/pagelist.c +4 -0
@@ -762,6 +762,10 @@ int nfs_initiate_pgio(struct rpc_clnt *clnt, struct nfs_pgio_header *hdr, hdr->args.count, (unsigned long long)hdr->args.offset); +if (localio) +return nfs_local_doio (NFS_SERVER (hdr->inode)->nfs_client, + localio, hdr, call_ops); + task =rpc_run_task (&task_setup_data);if (IS_ERR (task))return PTR_ERR (task);
fs/nfs/write.c +3 -0
@@ -1693,6 +1693,9 @@ int nfs_initiate_commit(struct rpc_clnt *clnt, struct nfs_commit_data *data,dprintk ("NFS: initiated commit call\n" ); +if (localio) +return nfs_local_commit (localio, data, call_ops, how); + task =rpc_run_task (&task_setup_data);if (IS_ERR (task))return PTR_ERR (task);
include/linux/nfs.h +2 -0
@@ -8,6 +8,8 @@#ifndef _LINUX_NFS_H#define _LINUX_NFS_H +#include <linux/cred.h> +#include <linux/sunrpc/auth.h> #include <linux/sunrpc/msg_prot.h> #include <linux/string.h> #include <linux/crc32.h>
include/linux/nfs_fs_sb.h +9 -0
@@ -8,6 +8,7 @@#include <linux/wait.h> #include <linux/nfs_xdr.h> #include <linux/sunrpc/xprt.h> +#include <linux/nfslocalio.h> #include <linux/atomic.h> #include <linux/refcount.h>
@@ -49,6 +50,7 @@ struct nfs_client {#define NFS_CS_DS 7/* - Server is a DS */ #define NFS_CS_REUSEPORT 8/* - reuse src port on reconnect */ #define NFS_CS_PNFS 9/* - Server used for pnfs */ +#define NFS_CS_LOCAL_IO 10/* - client is local */ struct sockaddr_storage cl_addr;/* server identifier */ size_t cl_addrlen;char * cl_hostname;/* hostname of server */
@@ -125,6 +127,13 @@ struct nfs_client {struct net *cl_net;struct list_head pending_cb_stateids;struct rcu_head rcu; + +#if IS_ENABLED (CONFIG_NFS_LOCALIO ) +struct timespec64 cl_nfssvc_boot; +seqlock_t cl_boot_lock; +nfs_uuid_t cl_uuid; +spinlock_t cl_localio_lock; +#endif /* CONFIG_NFS_LOCALIO */ };/*
Signature · 1 line
2.44.0
PATCH 23/25 nfs: implement client support for NFS_LOCALIO_PROGRAM
In reply to NeilBrownFrom: Mike Snitzer <[email protected]>
The LOCALIO auxiliary RPC protocol consists of a single "UUID_IS_LOCAL" RPC method that allows the Linux NFS client to verify the local Linux NFS server can see the nonce (single-use UUID) the client generated and made available in nfs_common for subsequent lookup and verification by the NFS server. If matched, the NFS server populates members in the nfs_uuid_t struct. The NFS client then transfers these nfs_uuid_t struct member pointers to the nfs_client struct and cleans up the nfs_uuid_t struct. See: fs/nfs/localio.c:nfs_local_probe()
This protocol isn't part of an IETF standard, nor does it need to be considering it is Linux-to-Linux auxiliary RPC protocol that amounts to an implementation detail.
Localio is only supported when UNIX-style authentication (AUTH_UNIX, aka AUTH_SYS) is used (enforced by fs/nfs/localio.c:nfs_local_probe()).
The UUID_IS_LOCAL method encodes the client generated uuid_t in terms of the fixed UUID_SIZE (16 bytes). The fixed size opaque encode and decode XDR methods are used instead of the less efficient variable sized methods.
Having a nonce (single-use uuid) is better than using the same uuid for the life of the server, and sending it proactively by client rather than reactively by the server is also safer.
[NeilBrown factored out and simplified a single localio protocol and proposed making the uuid short-lived]
--- fs/nfs/client.c | 6 ++- fs/nfs/localio.c | 132 +++++++++++++++++++++++++++++++++++++++++++++-- 2 files changed, 132 insertions(+), 6 deletions(-)
fs/nfs/client.c +4 -2
@@ -433,8 +433,10 @@ struct nfs_client *nfs_get_client(const struct nfs_client_initdata *cl_init)list_add_tail (&new->cl_share_link, &nn->nfs_client_list);spin_unlock (&nn->nfs_client_lock); -nfs_local_probe (new); -return rpc_ops->init_client (new, cl_init); + new = rpc_ops->init_client (new, cl_init); +if (!IS_ERR (new)) +nfs_local_probe (new); +return new; }spin_unlock (&nn->nfs_client_lock);
fs/nfs/localio.c +128 -4
163 lines not on this page · raw message
Signature · 1 line
2.44.0
Quoted 27 lines · NeilBrown
Following are revised versions of 6 patches from the v14 localio series.
The issue addressed is net namespace refcounting.
We don't want to keep a long-term counted reference in the client because that prevents a server container from completely shutting down.
So we avoid taking a reference at all and rely on the per-cpu reference to the server being sufficient to keep the net-ns active. This involves allowing the net-ns exit code to iterate all active clients and clear their ->net pointers (which they need to find the per-cpu-refcount for the
nfs_serv).So:
- embednfs_uuid_tinnfs_client. This provides alist_headthat can be used to find the client. It does add the actual uuid tonfs_clientso it is bigger than needed. If that is really a problem we can find a fix.- When the nfs server confirms that the uuid is local, it moves the
nfs_uuid_tonto a per-net-ns list.- When the net-ns is shutting down - in a "
pre_exit" handler, all thesenfS_uuid_thave their ->net cleared. There is anrcu_synchronize()call betweenpre_exit()handlers andexit()handlers so and caller that sees ->net as not NULL can safely check the ->counter
Quoted 4 lines
- We now pass the
nfs_uuid_ttonfsd_open_local_fh()so it can safely look at ->net in a privatercu_read_lock()section.I have compile tested this code but nothing more.
Wow, nicely done. I will go over it in much more detail and test.
For the benefit of others, here is a single incremental diff relative to v14:
fs/nfs/client.c +1 -2
@@ -181,8 +181,7 @@ struct nfs_client *nfs_alloc_client(const struct nfs_client_initdata *cl_init)#if IS_ENABLED (CONFIG_NFS_LOCALIO )seqlock_init (&clp->cl_boot_lock);ktime_get_real_ts64 (&clp->cl_nfssvc_boot); - clp->cl_nfssvc_net =NULL ; - clp->cl_nfssvc_dom =NULL ; + clp->cl_uuid.net =NULL ;spin_lock_init (&clp->cl_localio_lock);#endif /* CONFIG_NFS_LOCALIO */
fs/nfs/localio.c +11 -38
114 lines not on this page · raw message
fs/nfs_common/nfslocalio.c +55 -21
124 lines not on this page · raw message
fs/nfsd/localio.c +18 -7
@@ -42,13 +42,14 @@* nfsd_serv_put (via nfs_localio_ctx_free). */ struct nfs_localio_ctx * -nfsd_open_local_fh (struct net *cl_nfssvc_net,struct auth_domain *cl_nfssvc_dom, +nfsd_open_local_fh (nfs_uuid_t *uuid,struct rpc_clnt *rpc_clnt,const struct cred *cred,const struct nfs_fh *nfs_fh,const fmode_t fmode) {int mayflags =NFSD_MAY_LOCALIO ;int status =0 ; -struct nfsd_net *nn; +struct nfsd_net *nn =NULL ; +struct net *net;struct svc_cred rq_cred;struct svc_fh fh;struct nfs_localio_ctx *localio;
@@ -64,12 +65,20 @@ nfsd_open_local_fh(struct net *cl_nfssvc_net, struct auth_domain *cl_nfssvc_dom,/* * Not running in nfsd context, so must safely get reference on nfsd_serv. * But the server may already be shutting down, if so disallow new localio. +* uuid->net is NOT a counted reference, but rcu_read_lock() ensures that if +* uuid->net is not NULL, then nfsd_serv_try_get() is safe and if that succeeds +* we will have an implied reference to the net. */ - nn =net_generic (cl_nfssvc_net, nfsd_net_id); -if (unlikely (!nfsd_serv_try_get (nn))) { +rcu_read_lock (); + net =READ_ONCE (uuid->net); +if (net) + nn =net_generic (net, nfsd_net_id); +if (unlikely (!nn || !nfsd_serv_try_get (nn))) { +rcu_read_unlock (); status = -ENXIO ; goto out_nfsd_serv; } +rcu_read_unlock ();/* nfs_fh -> svc_fh */ fh_init (&fh,NFS4_FHSIZE );
@@ -83,7 +92,7 @@ nfsd_open_local_fh(struct net *cl_nfssvc_net, struct auth_domain *cl_nfssvc_dom,svcauth_map_clnt_to_svc_cred_local (rpc_clnt, cred, &rq_cred); - beres =nfsd_file_acquire_local (cl_nfssvc_net, &rq_cred, cl_nfssvc_dom, + beres =nfsd_file_acquire_local (uuid->net, &rq_cred, uuid->dom, &fh, mayflags, &localio->nf);if (beres) { status =nfs_stat_to_errno (be32_to_cpu (beres));
@@ -123,9 +132,11 @@ struct localio_uuidarg {static __be32 localio_proc_uuid_is_local (struct svc_rqst *rqstp) {struct localio_uuidarg *argp = rqstp->rq_argp; +struct net *net =SVC_NET (rqstp); +struct nfsd_net *nn =net_generic (net, nfsd_net_id); - (void )nfs_uuid_is_local (&argp->uuid,SVC_NET (rqstp), - rqstp->rq_client); +nfs_uuid_is_local (&argp->uuid, &nn->local_clients, + net, rqstp->rq_client);return rpc_success; }
fs/nfsd/netns.h +2 -0
@@ -216,6 +216,8 @@ struct nfsd_net {/* last time an admin-revoke happened for NFSv4.0 */ time64_t nfs40_last_revoke; +/* Local clients to be invalidated when net is shut down */ +struct list_head local_clients; };/* Simple check to find out if a given net was properly initialized */
fs/nfsd/nfsctl.c +16 -0
@@ -18,6 +18,7 @@#include <linux/sunrpc/svc.h> #include <linux/module.h> #include <linux/fsnotify.h> +#include <linux/nfslocalio.h> #include "idmap.h" #include "nfsd.h"
@@ -2257,6 +2258,7 @@ static __net_init int nfsd_net_init(struct net *net)get_random_bytes (&nn->siphash_key,sizeof (nn->siphash_key));seqlock_init (&nn->writeverf_lock);nfsd_proc_stat_init (net); +INIT_LIST_HEAD (&nn->local_clients);return 0 ;
@@ -2268,6 +2270,19 @@ static __net_init int nfsd_net_init(struct net *net)return retval; } +/** +* nfsd_net_pre_exit - Disconnect localio clients from net namespace +* @net: a network namespace that is about to be destroyed +* +* This invalidated ->net pointers held by localio clients +* while they can still safely access nn->counter. +*/ +static __net_exit void nfsd_net_pre_exit(struct net *net) +{ +struct nfsd_net *nn =net_generic (net, nfsd_net_id); + +nfs_uuid_invalidate_clients (&nn->local_clients); +}/** * nfsd_net_exit - Release the nfsd_net portion of a net namespace * @net: a network namespace that is about to be destroyed
@@ -2285,6 +2300,7 @@ static __net_exit void nfsd_net_exit(struct net *net)static struct pernet_operations nfsd_net_ops = { .init = nfsd_net_init, + .pre_exit = nfsd_net_pre_exit, .exit = nfsd_net_exit, .id = &nfsd_net_id, .size =sizeof (struct nfsd_net ),
fs/nfsd/vfs.h +2 -1
@@ -8,6 +8,7 @@#include <linux/fs.h> #include <linux/posix_acl.h> +#include <linux/nfslocalio.h> #include "nfsfh.h" #include "nfsd.h"
@@ -161,7 +162,7 @@ __be32 nfsd_permission(struct svc_cred *cred, struct svc_export *exp,void nfsd_filp_close (struct file *fp);struct nfs_localio_ctx * -nfsd_open_local_fh (struct net *,struct auth_domain *, +nfsd_open_local_fh (nfs_uuid_t *,struct rpc_clnt *,const struct cred *,const struct nfs_fh *,const fmode_t );
include/linux/nfs_fs_sb.h +1 -2
@@ -131,8 +131,7 @@ struct nfs_client {#if IS_ENABLED (CONFIG_NFS_LOCALIO )struct timespec64 cl_nfssvc_boot;seqlock_t cl_boot_lock; -struct net __rcu * cl_nfssvc_net; -struct auth_domain __rcu * cl_nfssvc_dom; +nfs_uuid_t cl_uuid;spinlock_t cl_localio_lock;#endif /* CONFIG_NFS_LOCALIO */ };
include/linux/nfslocalio.h +5 -2
@@ -28,7 +28,10 @@ typedef struct {void nfs_uuid_begin (nfs_uuid_t *);void nfs_uuid_end (nfs_uuid_t *); -bool nfs_uuid_is_local (const uuid_t *,struct net *,struct auth_domain *); +void nfs_uuid_is_local (const uuid_t *,struct list_head *, +struct net *,struct auth_domain *); +void nfs_uuid_invalidate_clients (struct list_head *list); +void nfs_uuid_invalidate_one_client (nfs_uuid_t *nfs_uuid);struct nfsd_file ;struct nfsd_net ;
@@ -39,7 +42,7 @@ struct nfs_localio_ctx { };typedef struct nfs_localio_ctx * -(*nfs_to_nfsd_open_local_fh_t )(struct net *,struct auth_domain *, +(*nfs_to_nfsd_open_local_fh_t )(nfs_uuid_t *,struct rpc_clnt *,const struct cred *,const struct nfs_fh *,const fmode_t );typedef struct nfsd_file * (*nfs_to_nfsd_file_get_t )(struct nfsd_file *);
Quoted 35 lines · NeilBrown
Following are revised versions of 6 patches from the v14 localio series.
The issue addressed is net namespace refcounting.
We don't want to keep a long-term counted reference in the client because that prevents a server container from completely shutting down.
So we avoid taking a reference at all and rely on the per-cpu reference to the server being sufficient to keep the net-ns active. This involves allowing the net-ns exit code to iterate all active clients and clear their ->net pointers (which they need to find the per-cpu-refcount for the
nfs_serv).So:
- embednfs_uuid_tinnfs_client. This provides alist_headthat can be used to find the client. It does add the actual uuid tonfs_clientso it is bigger than needed. If that is really a problem we can find a fix.- When the nfs server confirms that the uuid is local, it moves the
nfs_uuid_tonto a per-net-ns list.- When the net-ns is shutting down - in a "
pre_exit" handler, all thesenfS_uuid_thave their ->net cleared. There is anrcu_synchronize()call betweenpre_exit()handlers andexit()handlers so and caller that sees ->net as not NULL can safely check the ->counter- We now pass the
nfs_uuid_ttonfsd_open_local_fh()so it can safely look at ->net in a privatercu_read_lock()section.I have compile tested this code but nothing more.
Thanks,
NeilBrown
Quoted 6 lines
[PATCH 14/25]
nfs_common: add NFS LOCALIO auxiliary protocol [PATCH 15/25]nfs_common: introducenfs_localio_ctxstruct and [PATCH 16/25] nfsd: add localio support
[PATCH 17/25] nfsd: implement server support forNFS_LOCALIO_PROGRAM[PATCH 19/25] nfs: add localio support
[PATCH 23/25] nfs: implement client support forNFS_LOCALIO_PROGRAM
Hey Neil,
I attempted to test the kernel with your changes but it crashed with:
[ 55.422564] list_add corruption. next is NULL. [ 55.423523] ------------[ cut here ]------------ [ 55.424423] kernel BUG at lib/list_debug.c:27! [ 55.425291] Oops: invalid opcode: 0000 [#1] PREEMPT SMP NOPTI [ 55.426367] CPU: 29 UID: 0 PID: 5251 Comm: nfsd Kdump: loaded Not tainted 6.11.0-rc4.snitm+ #147 [ 55.427991] Hardware name: Red Hat KVM/RHEL-AV, BIOS 1.16.0-4.module+el8.9.0+19570+14a90618 04/01/2014 [ 55.429697] RIP: 0010:__list_add_valid_or_report+0x55/0xa0 [ 55.430741] Code: 4c 39 cf 74 4f b8 01 00 00 00 5d c3 cc cc cc cc 48 c7 c7 98 1d a5 82 e8 d9 6d 93 ff 0f 0b 48 c7 c7 c0 1d a5 82 e8 cb 6d 93 ff <0f> 0b 4c 89 c1 48 c7 c7 e8 1d a5 82 e8 ba 6d 93 ff 0f 0b 48 89 d1 [ 55.434167] RSP: 0018:ffff8881441a7d50 EFLAGS: 00010296 [ 55.435141] RAX: 0000000000000022 RBX: ffff888107b50370 RCX: 0000000000000000 [ 55.436455] RDX: ffff888473caf800 RSI: ffff888473ca18c0 RDI: ffff888473ca18c0 [ 55.437770] RBP: ffff8881441a7d50 R08: 0000000000000022 R09: ffff8881441a7be8 [ 55.439098] R10: ffff8881441a7be0 R11: ffffffff8333f328 R12: ffff888107b50380 [ 55.440419] R13: ffff888103b15080 R14: ffff888107bb5d00 R15: 0000000000000000 [ 55.441737] FS: 0000000000000000(0000) GS:ffff888473c80000(0000) knlGS:0000000000000000 [ 55.443228] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 55.444297] CR2: 000055958fab3488 CR3: 000000010615e000 CR4: 0000000000350ef0 [ 55.445615] Call Trace: [ 55.446090] <TASK> [ 55.446498] ? show_regs+0x6d/0x80 [ 55.447149] ? die+0x3c/0xa0 [ 55.447698] ? do_trap+0xcf/0xf0 [ 55.448316] ? do_error_trap+0x75/0xa0 [ 55.449026] ? __list_add_valid_or_report+0x55/0xa0 [ 55.449938] ? exc_invalid_op+0x57/0x80 [ 55.450660] ? __list_add_valid_or_report+0x55/0xa0 [ 55.451646] ? asm_exc_invalid_op+0x1f/0x30 [ 55.452438] ? __list_add_valid_or_report+0x55/0xa0 [ 55.453355] nfs_uuid_is_local+0xba/0x110 [ 55.454115] localio_proc_uuid_is_local+0x64/0x80 [nfsd] [ 55.455145] nfsd_dispatch+0xc2/0x210 [nfsd] [ 55.455977] svc_process_common+0x2e6/0x6e0 [ 55.456761] ? __pfx_nfsd_dispatch+0x10/0x10 [nfsd] [ 55.457697] svc_process+0x13e/0x1e0 [ 55.458377] svc_recv+0x89e/0xa70 [ 55.459012] ? __pfx_nfsd+0x10/0x10 [nfsd] [ 55.459806] nfsd+0xa5/0x100 [nfsd] [ 55.460486] kthread+0xe5/0x120 [ 55.461090] ? __pfx_kthread+0x10/0x10 [ 55.461801] ret_from_fork+0x3d/0x60 [ 55.462476] ? __pfx_kthread+0x10/0x10 [ 55.463184] ret_from_fork_asm+0x1a/0x30 [ 55.463923] </TASK>
I'll triple check my melding of your changes and mine in ~7 hours.. I may have missed something.
Note this is _not_ with your other incremental patch (that uses __module_get) -- only because I didn't get to that yet.
Mike
Quoted 50 lines · Mike Snitzer
Quoted 35 lines · NeilBrown
Following are revised versions of 6 patches from the v14 localio series.
The issue addressed is net namespace refcounting.
We don't want to keep a long-term counted reference in the client because that prevents a server container from completely shutting down.
So we avoid taking a reference at all and rely on the per-cpu reference to the server being sufficient to keep the net-ns active. This involves allowing the net-ns exit code to iterate all active clients and clear their ->net pointers (which they need to find the per-cpu-refcount for the
nfs_serv).So:
- embednfs_uuid_tinnfs_client. This provides alist_headthat can be used to find the client. It does add the actual uuid tonfs_clientso it is bigger than needed. If that is really a problem we can find a fix.- When the nfs server confirms that the uuid is local, it moves the
nfs_uuid_tonto a per-net-ns list.- When the net-ns is shutting down - in a "
pre_exit" handler, all thesenfS_uuid_thave their ->net cleared. There is anrcu_synchronize()call betweenpre_exit()handlers andexit()handlers so and caller that sees ->net as not NULL can safely check the ->counter- We now pass the
nfs_uuid_ttonfsd_open_local_fh()so it can safely look at ->net in a privatercu_read_lock()section.I have compile tested this code but nothing more.
Thanks,
NeilBrownQuoted 6 lines
[PATCH 14/25]
nfs_common: add NFS LOCALIO auxiliary protocol [PATCH 15/25]nfs_common: introducenfs_localio_ctxstruct and [PATCH 16/25] nfsd: add localio support
[PATCH 17/25] nfsd: implement server support forNFS_LOCALIO_PROGRAM[PATCH 19/25] nfs: add localio support
[PATCH 23/25] nfs: implement client support forNFS_LOCALIO_PROGRAMHey Neil,
I attempted to test the kernel with your changes but it crashed with:
[ 55.422564] list_add corruption. next is NULL. [ 55.423523] ------------[ cut here ]------------ [ 55.424423] kernel BUG at lib/list_debug.c:27!
...
Quoted 2 lines
I'll triple check my melding of your changes and mine in ~7 hours.. I may have missed something.
Good news, I was just missing your fs/nfsd/nfsctl.c changes.. works well with those ;)
Seeing a very slight drop in performance (with my well-worn smoke test that does 20 secs of aio directio 128K reads with 24 threads on my testbed). But I mean slight (~2-3%). Not worried about it, correctness trumps performance, but I think reclaiming that performance will be easy (by avoiding the need for KMEM_CACHE nfs_localio_ctx_{alloc,free}).
Quoted 2 lines
Note this is _not_ with your other incremental patch (that uses
__module_get) -- only because I didn't get to that yet.
Moving on to incorporating your nfsd __module_get now. Then on to the work of switching from nfs_localio_ctx back to returning nfsd_file.
Mike