nfsd/sunrpc: convert nfsstat server-side interfaces to use netlink
Subsystems (4)
Jeff Layton
maintainerv4 cover nfsd/sunrpc: convert nfsstat server-side interfaces to use netlink
This version just fixes an error handling bug that Sashiko noticed.
The nfsstat tool currently scrapes /proc/net/rpc/nfsd for server statistics. This procfs interface has several limitations: the counters are global (not network-namespace-aware), the format is fragile to parse, and it cannot be extended without breaking existing parsers.
This series adds per-network-namespace procedure call counts to the sunrpc layer and exposes them through a new netlink handler in the nfsd generic netlink family, allowing nfsstat to retrieve server statistics via netlink with a procfs fallback for older kernels.
Patch 1 adds per-netns per-procedure percpu call count arrays to struct svc_stat, allocated alongside the nfsd_net lifecycle.
Patch 2 switches svc_seq_show() to read from the per-netns counters, making /proc/net/rpc/nfsd namespace-aware. Note that this is a behavior change, but I think it's a desirable one.
Patch 3 implements the server-stats-get netlink dump handler, streaming reply-cache, filehandle, IO, network, and RPC statistics plus per-version and per-operation procedure counts.
Patch 4 removes the now-unused global svc_version vs_count percpu arrays from nfsd, lockd, and the NFS client callback service.
I'll also be sending an nfs-utils patch to convert nfsstat to use the new interfaces.
---
Changes in v4:
- Ensure error is returned when nla_put_*() calls fail
- Link to v3: https://lore.kernel.org/r/[email protected]
Changes in v3:
- Only increment per-net stats for the "primary" program
- Link to v2: https://lore.kernel.org/r/[email protected]
---
Jeff Layton (4):
sunrpc: add per-netns per-procedure call counts to svc_stat
sunrpc: use per-net counts in svc_seq_show()
nfsd: implement server-stats-get netlink handler
sunrpc: remove unused svc_version vs_count field
Documentation/netlink/specs/nfsd.yaml | 105 +++++++++++++++++ fs/lockd/svc4proc.c | 4 - fs/lockd/svcproc.c | 7 -- fs/nfs/callback_xdr.c | 6 - fs/nfsd/localio.c | 3 - fs/nfsd/netlink.c | 5 + fs/nfsd/netlink.h | 2 + fs/nfsd/nfs2acl.c | 3 - fs/nfsd/nfs3acl.c | 3 - fs/nfsd/nfs3proc.c | 3 - fs/nfsd/nfs4proc.c | 3 - fs/nfsd/nfsctl.c | 214 +++++++++++++++++++++++++++++++++- fs/nfsd/nfsproc.c | 3 - include/linux/sunrpc/stats.h | 6 + include/linux/sunrpc/svc.h | 1 - include/uapi/linux/nfsd_netlink.h | 35 ++++++ net/sunrpc/stats.c | 56 ++++++++- net/sunrpc/svc.c | 9 +- 18 files changed, 428 insertions(+), 40 deletions(-)
---
base-commit: c13952992d5459add7732eb4876eebfeec03af58
change-id: 20260316-exportd-netlink-1c9fb52536e3
Best regards,
Signature · 1 line
Jeff Layton <[email protected]>
Jeff Layton
maintainerPATCH v4 1/4 sunrpc: add per-netns per-procedure call counts to svc_stat
The existing per-procedure call counts live in global svc_version->vs_count[] arrays which are not network-namespace-aware. Add per-netns equivalents in struct svc_stat so the upcoming netlink stats interface can return namespace-scoped statistics.
Add a vs_count pointer array to struct svc_stat, along with svc_stat_alloc_counts() and svc_stat_free_counts() helpers to manage per-version percpu call count arrays.
Increment the per-net counter alongside the global one in svc_generic_init_request(). Call the alloc/free helpers from nfsd_net_init() and nfsd_net_exit().
--- fs/nfsd/nfsctl.c | 8 ++++++- include/linux/sunrpc/stats.h | 6 +++++ net/sunrpc/stats.c | 54 ++++++++++++++++++++++++++++++++++++++++++++ net/sunrpc/svc.c | 8 +++++++ 4 files changed, 75 insertions(+), 1 deletion(-)
fs/nfsd/nfsctl.c +7 -1
@@ -2514,9 +2514,12 @@ static __net_init int nfsd_net_init(struct net *net)memset (&nn->nfsd_svcstats,0 ,sizeof (nn->nfsd_svcstats)); nn->nfsd_svcstats.program = &nfsd_programs[0 ]; + retval =svc_stat_alloc_counts (&nn->nfsd_svcstats); +if (retval) + goto out_proc_error;if (!nfsd_proc_stat_init (net)) { retval = -ENOMEM ; - goto out_proc_error; + goto out_svcstats_error; }for (i =0 ; i <sizeof (nn->nfsd_versions); i++)
@@ -2534,6 +2537,8 @@ static __net_init int nfsd_net_init(struct net *net)#endif return 0 ; +out_svcstats_error: +svc_stat_free_counts (&nn->nfsd_svcstats); out_proc_error:percpu_counter_destroy_many (nn->counter,NFSD_STATS_COUNTERS_NUM ); out_repcache_error:
@@ -2574,6 +2579,7 @@ static __net_exit void nfsd_net_exit(struct net *net)kfree_sensitive (nn->fh_key);nfsd_net_cb_shutdown (nn);nfsd_proc_stat_shutdown (net); +svc_stat_free_counts (&nn->nfsd_svcstats);percpu_counter_destroy_many (nn->counter,NFSD_STATS_COUNTERS_NUM );nfsd_idmap_shutdown (net);nfsd_export_shutdown (net);
include/linux/sunrpc/stats.h +6 -0
@@ -37,9 +37,15 @@ struct svc_stat { rpcbadfmt, rpcbadauth, rpcbadclnt; + +/* Per-version per-procedure call counts (per-cpu, per-netns) */ +unsigned long __percpu **vs_count; };struct net ; +int svc_stat_alloc_counts (struct svc_stat *statp); +void svc_stat_free_counts (struct svc_stat *statp); +#ifdef CONFIG_PROC_FS int rpc_proc_init (struct net *);void rpc_proc_exit (struct net *);
net/sunrpc/stats.c +54 -0
60 lines not on this page · raw message
net/sunrpc/svc.c +8 -0
@@ -1438,6 +1438,14 @@ svc_generic_init_request(struct svc_rqst *rqstp,/* Bump per-procedure stats counter */ this_cpu_inc (versp->vs_count[rqstp->rq_proc]); +/* Bump per-net per-procedure stats counter */ +if (rqstp->rq_server->sv_stats && + rqstp->rq_server->sv_stats->program == progp && + rqstp->rq_server->sv_stats->vs_count && + rqstp->rq_server->sv_stats->vs_count[rqstp->rq_vers]) +this_cpu_inc (rqstp->rq_server->sv_stats->vs_count + [rqstp->rq_vers][rqstp->rq_proc]); + ret->dispatch = versp->vs_dispatch;return rpc_success; err_bad_vers:
Signature · 1 line
2.54.0
Jeff Layton
maintainerPATCH v4 2/4 sunrpc: use per-net counts in svc_seq_show()
In reply to Jeff LaytonUpdate svc_seq_show() to read from the per-netns statp->vs_count[] arrays instead of the global svc_version->vs_count[].
The only caller is nfsd, which always allocates vs_count via svc_stat_alloc_counts() in nfsd_net_init(), so the per-netns arrays are always available.
This makes /proc/net/rpc/nfsd report per-network-namespace procedure call counts.
--- net/sunrpc/stats.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
net/sunrpc/stats.c +1 -1
@@ -108,7 +108,7 @@ void svc_seq_show(struct seq_file *seq, const struct svc_stat *statp)for (j =0 ; j < vers->vs_nproc; j++) { count =0 ;for_each_possible_cpu (k) - count +=per_cpu (vers->vs_count[j], k); + count +=per_cpu (statp->vs_count[i][j], k);seq_printf (seq," %lu" , count); }seq_putc (seq,'\n' );
Signature · 1 line
2.54.0
Jeff Layton
maintainerPATCH v4 3/4 nfsd: implement server-stats-get netlink handler
In reply to Jeff LaytonImplement nfsd_nl_server_stats_get_dumpit() which exposes the NFS server statistics currently available via /proc/net/rpc/nfsd through the nfsd generic netlink family.
The handler uses a dump operation to stream statistics across multiple netlink messages:
- First message: all scalar stats (reply cache, filehandle, IO, network, RPC) plus per-version procedure counts (proc2/3/4-ops) using per-netns vs_count arrays.
- Subsequent messages: NFSv4 per-operation counts (proc4ops-ops), one entry per message, using cb->args[0] to track the current operation index across dump calls.
This allows nfsstat to retrieve server statistics via netlink with a procfs fallback for older kernels.
--- Documentation/netlink/specs/nfsd.yaml | 105 +++++++++++++++++ fs/nfsd/netlink.c | 5 + fs/nfsd/netlink.h | 2 + fs/nfsd/nfsctl.c | 206 ++++++++++++++++++++++++++++++++++ include/uapi/linux/nfsd_netlink.h | 35 ++++++ 5 files changed, 353 insertions(+)
Documentation/netlink/specs/nfsd.yaml +105 -0
@@ -330,6 +330,86 @@ attribute-sets: of which client holds the state. Intended for use after all clients have been unexported from a given path, enabling the underlying filesystem to be unmounted. + - + name: server-proc-entry + attributes: + - + name: op + type: u32 + - + name: count + type: u64 + - + name: pad + type: pad + - + name: server-stats + attributes: + - + name: rc-hits + type: u64 + - + name: rc-misses + type: u64 + - + name: rc-nocache + type: u64 + - + name: pad + type: pad + - + name: fh-stale + type: u64 + - + name: io-read + type: u64 + - + name: io-write + type: u64 + - + name: netcnt + type: u32 + - + name: netudpcnt + type: u32 + - + name: nettcpcnt + type: u32 + - + name: nettcpconn + type: u32 + - + name: rpccnt + type: u32 + - + name: rpcbadfmt + type: u32 + - + name: rpcbadauth + type: u32 + - + name: rpcbadclnt + type: u32 + - + name: proc2-ops + type: nest + nested-attributes: server-proc-entry + multi-attr: true + - + name: proc3-ops + type: nest + nested-attributes: server-proc-entry + multi-attr: true + - + name: proc4-ops + type: nest + nested-attributes: server-proc-entry + multi-attr: true + - + name: proc4ops-ops + type: nest + nested-attributes: server-proc-entry + multi-attr: true operations: list:
@@ -516,6 +596,31 @@ operations: request: attributes: - path + - + name: server-stats-get + doc: dump NFS server statistics + attribute-set: server-stats + dump: + reply: + attributes: + - rc-hits + - rc-misses + - rc-nocache + - fh-stale + - io-read + - io-write + - netcnt + - netudpcnt + - nettcpcnt + - nettcpconn + - rpccnt + - rpcbadfmt + - rpcbadauth + - rpcbadclnt + - proc2-ops + - proc3-ops + - proc4-ops + - proc4ops-ops mcast-groups: list:
fs/nfsd/netlink.c +5 -0
@@ -225,6 +225,11 @@ static const struct genl_split_ops nfsd_nl_ops[] = { .maxattr = NFSD_A_UNLOCK_EXPORT_PATH, .flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DO, }, + { + .cmd = NFSD_CMD_SERVER_STATS_GET, + .dumpit = nfsd_nl_server_stats_get_dumpit, + .flags = GENL_CMD_CAP_DUMP, + }, };static const struct genl_multicast_group nfsd_nl_mcgrps[] = {
fs/nfsd/netlink.h +2 -0
@@ -42,6 +42,8 @@ int nfsd_nl_cache_flush_doit(struct sk_buff *skb, struct genl_info *info);int nfsd_nl_unlock_ip_doit (struct sk_buff *skb,struct genl_info *info);int nfsd_nl_unlock_filesystem_doit (struct sk_buff *skb,struct genl_info *info);int nfsd_nl_unlock_export_doit (struct sk_buff *skb,struct genl_info *info); +int nfsd_nl_server_stats_get_dumpit (struct sk_buff *skb, +struct netlink_callback *cb);enum {NFSD_NLGRP_NONE ,
fs/nfsd/nfsctl.c +206 -0
212 lines not on this page · raw message
include/uapi/linux/nfsd_netlink.h +35 -0
@@ -225,6 +225,40 @@ enum {NFSD_A_UNLOCK_EXPORT_MAX = (__NFSD_A_UNLOCK_EXPORT_MAX -1 ) }; +enum { +NFSD_A_SERVER_PROC_ENTRY_OP =1 , +NFSD_A_SERVER_PROC_ENTRY_COUNT , +NFSD_A_SERVER_PROC_ENTRY_PAD , + + __NFSD_A_SERVER_PROC_ENTRY_MAX, +NFSD_A_SERVER_PROC_ENTRY_MAX = (__NFSD_A_SERVER_PROC_ENTRY_MAX -1 ) +}; + +enum { +NFSD_A_SERVER_STATS_RC_HITS =1 , +NFSD_A_SERVER_STATS_RC_MISSES , +NFSD_A_SERVER_STATS_RC_NOCACHE , +NFSD_A_SERVER_STATS_PAD , +NFSD_A_SERVER_STATS_FH_STALE , +NFSD_A_SERVER_STATS_IO_READ , +NFSD_A_SERVER_STATS_IO_WRITE , +NFSD_A_SERVER_STATS_NETCNT , +NFSD_A_SERVER_STATS_NETUDPCNT , +NFSD_A_SERVER_STATS_NETTCPCNT , +NFSD_A_SERVER_STATS_NETTCPCONN , +NFSD_A_SERVER_STATS_RPCCNT , +NFSD_A_SERVER_STATS_RPCBADFMT , +NFSD_A_SERVER_STATS_RPCBADAUTH , +NFSD_A_SERVER_STATS_RPCBADCLNT , +NFSD_A_SERVER_STATS_PROC2_OPS , +NFSD_A_SERVER_STATS_PROC3_OPS , +NFSD_A_SERVER_STATS_PROC4_OPS , +NFSD_A_SERVER_STATS_PROC4OPS_OPS , + + __NFSD_A_SERVER_STATS_MAX, +NFSD_A_SERVER_STATS_MAX = (__NFSD_A_SERVER_STATS_MAX -1 ) +}; +enum {NFSD_CMD_RPC_STATUS_GET =1 ,NFSD_CMD_THREADS_SET ,
@@ -244,6 +278,7 @@ enum {NFSD_CMD_UNLOCK_IP ,NFSD_CMD_UNLOCK_FILESYSTEM ,NFSD_CMD_UNLOCK_EXPORT , +NFSD_CMD_SERVER_STATS_GET , __NFSD_CMD_MAX,NFSD_CMD_MAX = (__NFSD_CMD_MAX -1 )
Signature · 1 line
2.54.0
Jeff Layton
maintainerPATCH v4 4/4 sunrpc: remove unused svc_version vs_count field
In reply to Jeff LaytonNow that svc_seq_show() and the nfsd netlink stats handler both use the per-netns svc_stat vs_count arrays, the global per-version vs_count percpu counters are no longer read by anything. Remove the vs_count field from struct svc_version and all the associated DEFINE_PER_CPU_ALIGNED arrays and initializers across nfsd, lockd, and the NFS client callback service.
--- fs/lockd/svc4proc.c | 4 ---- fs/lockd/svcproc.c | 7 ------- fs/nfs/callback_xdr.c | 6 ------ fs/nfsd/localio.c | 3 --- fs/nfsd/nfs2acl.c | 3 --- fs/nfsd/nfs3acl.c | 3 --- fs/nfsd/nfs3proc.c | 3 --- fs/nfsd/nfs4proc.c | 3 --- fs/nfsd/nfsproc.c | 3 --- include/linux/sunrpc/svc.h | 1 - net/sunrpc/svc.c | 3 --- 11 files changed, 39 deletions(-)
fs/lockd/svc4proc.c +0 -4
@@ -1420,14 +1420,10 @@ union nlm4svc_xdrstore {struct nlm4_shareres_wrapper shareres; }; -static DEFINE_PER_CPU_ALIGNED(unsigned long, - nlm4svc_call_counters[ARRAY_SIZE(nlm4svc_procedures)]); -const struct svc_version nlmsvc_version4 = { .vs_vers =4 , .vs_nproc =ARRAY_SIZE (nlm4svc_procedures), .vs_proc = nlm4svc_procedures, - .vs_count = nlm4svc_call_counters, .vs_dispatch = nlmsvc_dispatch, .vs_xdrsize =sizeof (union nlm4svc_xdrstore ), };
fs/lockd/svcproc.c +0 -7
@@ -1433,25 +1433,18 @@ union nlmsvc_xdrstore {* NLMv1 defines only procedures 1 - 15. Linux lockd also implements * procedures 0 (NULL) and 16 (SM_NOTIFY). */ -static DEFINE_PER_CPU_ALIGNED(unsigned long, nlm1svc_call_counters[17]); -const struct svc_version nlmsvc_version1 = { .vs_vers =1 , .vs_nproc =17 , .vs_proc = nlmsvc_procedures, - .vs_count = nlm1svc_call_counters, .vs_dispatch = nlmsvc_dispatch, .vs_xdrsize =sizeof (union nlmsvc_xdrstore ), }; -static DEFINE_PER_CPU_ALIGNED(unsigned long, - nlm3svc_call_counters[ARRAY_SIZE(nlmsvc_procedures)]); -const struct svc_version nlmsvc_version3 = { .vs_vers =3 , .vs_nproc =ARRAY_SIZE (nlmsvc_procedures), .vs_proc = nlmsvc_procedures, - .vs_count = nlm3svc_call_counters, .vs_dispatch = nlmsvc_dispatch, .vs_xdrsize =sizeof (union nlmsvc_xdrstore ), };
fs/nfs/callback_xdr.c +0 -6
@@ -1090,26 +1090,20 @@ static const struct svc_procedure nfs4_callback_procedures1[] = { } }; -static DEFINE_PER_CPU_ALIGNED(unsigned long, - nfs4_callback_count1[ARRAY_SIZE(nfs4_callback_procedures1)]);const struct svc_version nfs4_callback_version1 = { .vs_vers =1 , .vs_nproc =ARRAY_SIZE (nfs4_callback_procedures1), .vs_proc = nfs4_callback_procedures1, - .vs_count = nfs4_callback_count1, .vs_xdrsize =NFS4_CALLBACK_XDRSIZE , .vs_dispatch = nfs_callback_dispatch, .vs_hidden = true, .vs_need_cong_ctrl = true, }; -static DEFINE_PER_CPU_ALIGNED(unsigned long, - nfs4_callback_count4[ARRAY_SIZE(nfs4_callback_procedures1)]);const struct svc_version nfs4_callback_version4 = { .vs_vers =4 , .vs_nproc =ARRAY_SIZE (nfs4_callback_procedures1), .vs_proc = nfs4_callback_procedures1, - .vs_count = nfs4_callback_count4, .vs_xdrsize =NFS4_CALLBACK_XDRSIZE , .vs_dispatch = nfs_callback_dispatch, .vs_hidden = true,
fs/nfsd/localio.c +0 -3
@@ -210,14 +210,11 @@ static const struct svc_procedure localio_procedures1[] = { };#define LOCALIO_NR_PROCEDURES ARRAY_SIZE(localio_procedures1) -static DEFINE_PER_CPU_ALIGNED(unsigned long, - localio_count[LOCALIO_NR_PROCEDURES]);const struct svc_version localio_version1 = { .vs_vers =1 , .vs_nproc =LOCALIO_NR_PROCEDURES , .vs_proc = localio_procedures1, .vs_dispatch = nfsd_dispatch, - .vs_count = localio_count, .vs_xdrsize =XDR_QUADLEN (UUID_SIZE ), .vs_hidden = true, };
fs/nfsd/nfs2acl.c +0 -3
@@ -389,13 +389,10 @@ static const struct svc_procedure nfsd_acl_procedures2[5] = { }, }; -static DEFINE_PER_CPU_ALIGNED(unsigned long, - nfsd_acl_count2[ARRAY_SIZE(nfsd_acl_procedures2)]);const struct svc_version nfsd_acl_version2 = { .vs_vers =2 , .vs_nproc =ARRAY_SIZE (nfsd_acl_procedures2), .vs_proc = nfsd_acl_procedures2, - .vs_count = nfsd_acl_count2, .vs_dispatch = nfsd_dispatch, .vs_xdrsize =NFS3_SVC_XDRSIZE , };
fs/nfsd/nfs3acl.c +0 -3
@@ -278,13 +278,10 @@ static const struct svc_procedure nfsd_acl_procedures3[3] = { }, }; -static DEFINE_PER_CPU_ALIGNED(unsigned long, - nfsd_acl_count3[ARRAY_SIZE(nfsd_acl_procedures3)]);const struct svc_version nfsd_acl_version3 = { .vs_vers =3 , .vs_nproc =ARRAY_SIZE (nfsd_acl_procedures3), .vs_proc = nfsd_acl_procedures3, - .vs_count = nfsd_acl_count3, .vs_dispatch = nfsd_dispatch, .vs_xdrsize =NFS3_SVC_XDRSIZE , };
fs/nfsd/nfs3proc.c +0 -3
@@ -1068,13 +1068,10 @@ static const struct svc_procedure nfsd_procedures3[22] = { }, }; -static DEFINE_PER_CPU_ALIGNED(unsigned long, - nfsd_count3[ARRAY_SIZE(nfsd_procedures3)]);const struct svc_version nfsd_version3 = { .vs_vers =3 , .vs_nproc =ARRAY_SIZE (nfsd_procedures3), .vs_proc = nfsd_procedures3, .vs_dispatch = nfsd_dispatch, - .vs_count = nfsd_count3, .vs_xdrsize =NFS3_SVC_XDRSIZE , };
fs/nfsd/nfs4proc.c +0 -3
@@ -4151,13 +4151,10 @@ static const struct svc_procedure nfsd_procedures4[2] = { }, }; -static DEFINE_PER_CPU_ALIGNED(unsigned long, - nfsd_count4[ARRAY_SIZE(nfsd_procedures4)]);const struct svc_version nfsd_version4 = { .vs_vers =4 , .vs_nproc =ARRAY_SIZE (nfsd_procedures4), .vs_proc = nfsd_procedures4, - .vs_count = nfsd_count4, .vs_dispatch = nfsd_dispatch, .vs_xdrsize =NFS4_SVC_XDRSIZE , .vs_rpcb_optnl = true,
fs/nfsd/nfsproc.c +0 -3
@@ -845,13 +845,10 @@ static const struct svc_procedure nfsd_procedures2[18] = { }, }; -static DEFINE_PER_CPU_ALIGNED(unsigned long, - nfsd_count2[ARRAY_SIZE(nfsd_procedures2)]);const struct svc_version nfsd_version2 = { .vs_vers =2 , .vs_nproc =ARRAY_SIZE (nfsd_procedures2), .vs_proc = nfsd_procedures2, - .vs_count = nfsd_count2, .vs_dispatch = nfsd_dispatch, .vs_xdrsize =NFS2_SVC_XDRSIZE , };
include/linux/sunrpc/svc.h +0 -1
@@ -408,7 +408,6 @@ struct svc_version {u32 vs_vers;/* version number */ u32 vs_nproc;/* number of procedures */ const struct svc_procedure *vs_proc;/* per-procedure info */ -unsigned long __percpu *vs_count;/* call counts */ u32 vs_xdrsize;/* xdrsize needed for this version */ /* Don't register with rpcbind */
net/sunrpc/svc.c +0 -3
@@ -1435,9 +1435,6 @@ svc_generic_init_request(struct svc_rqst *rqstp,memset (rqstp->rq_argp,0 , procp->pc_argzero);memset (rqstp->rq_resp,0 , procp->pc_ressize); -/* Bump per-procedure stats counter */ -this_cpu_inc (versp->vs_count[rqstp->rq_proc]); -/* Bump per-net per-procedure stats counter */ if (rqstp->rq_server->sv_stats && rqstp->rq_server->sv_stats->program == progp &&
Signature · 1 line
2.54.0
Chuck Lever
maintainerRe: [PATCH v4 1/4] sunrpc: add per-netns per-procedure call counts to svc_stat
In reply to Jeff LaytonHey Jeff -
Quoted 8 lines · Jeff Layton
The existing per-procedure call counts live in global svc_version->vs_count[] arrays which are not network-namespace-aware. Add per-netns equivalents in struct
svc_statso the upcoming netlink stats interface can return namespace-scoped statistics.Add a
vs_countpointer array to structsvc_stat, along withsvc_stat_alloc_counts()andsvc_stat_free_counts()helpers to manage per-version percpu call count arrays.
This patch appears to break the build for an NFSv3-only server configured with CONFIG_PROC_FS=n.
svc_stat_alloc_counts() and svc_stat_free_counts() are defined in net/sunrpc/stats.c, which the Makefile builds only under PROC_FS:
sunrpc-$(CONFIG_PROC_FS) += stats.o
nfsd_net_init()/nfsd_net_exit() call both helpers unconditionally, and the declarations in <linux/sunrpc/stats.h> sit outside the #ifdef CONFIG_PROC_FS block with no !PROC_FS stubs. CONFIG_NFSD does not depend on PROC_FS -- only NFSD_V4 does -- so NFSD=y with NFSD_V2/V3 and PROC_FS=n is a valid Kconfig combination. The result is an undefined reference to svc_stat_alloc_counts at vmlinux link (NFSD=y), or a modpost failure for NFSD=m since both symbols are EXPORT_SYMBOL_GPL.
net/sunrpc/svc.c is always built, and might be a more natural home for the new functions.
Signature · 1 line
Chuck Lever
Chuck Lever
maintainerRe: [PATCH v4 3/4] nfsd: implement server-stats-get netlink handler
In reply to Jeff LaytonHey Jeff -
Quoted 20 lines · Jeff Layton
Implement
nfsd_nl_server_stats_get_dumpit()which exposes the NFS server statistics currently available via/proc/net/rpc/nfsdthrough the nfsd generic netlink family.The handler uses a dump operation to stream statistics across multiple netlink messages:
- First message: all scalar stats (reply cache, filehandle, IO, network, RPC) plus per-version procedure counts (proc2/3/4-ops) using per-netns
vs_countarrays.- Subsequent messages: NFSv4 per-operation counts (proc4ops-ops), one entry per message, using cb->args[0] to track the current operation index across dump calls.
This allows nfsstat to retrieve server statistics via netlink with a procfs fallback for older kernels.
Assisted-by Claude:claude-opus-4-6Signed-off-by Jeff Layton <[email protected]>
Quoted 7 lines
--- Documentation/netlink/specs/nfsd.yaml | 105 +++++++++++++++++ fs/nfsd/netlink.c | 5 + fs/nfsd/netlink.h | 2 + fs/nfsd/nfsctl.c | 206 ++++++++++++++++++++++++++++++++++ include/uapi/linux/nfsd_netlink.h | 35 ++++++ 5 files changed, 353 insertions(+)
The procfs output in nfsd_show() emits one more counter that this dump handler drops: wdeleg_getattr (NFSD_STATS_WDELEG_GETATTR), printed at the end of the CONFIG_NFSD_V4 block in fs/nfsd/stats.c, right after proc4ops. The netlink dump stops after proc4ops-ops and sets cb->args[0] = -1, so the write-delegation GETATTR-conflict counter never goes out on the wire.
There's no schema slot for it either: the server-stats attribute-set ends at proc4ops-ops, so a consumer that prefers netlink and only falls back to procfs on an old kernel loses this counter on every kernel new enough to support server-stats-get.
I Suggest:
- add a wdeleg-getattr (u64) attribute to the server-stats attribute-set in nfsd.yaml and list it under the dump reply attributes;
- add NFSD_A_SERVER_STATS_WDELEG_GETATTR to the uapi enum, after PROC4OPS_OPS and before __NFSD_A_SERVER_STATS_MAX so existing values don't shift;
- emit it as a single scalar in the start == 0 message, wrapped in CONFIG_NFSD_V4 to match the procfs side.
Signature · 1 line
Chuck Lever
Jeff Layton
maintainerRe: [PATCH v4 1/4] sunrpc: add per-netns per-procedure call counts to svc_stat
In reply to Chuck LeverQuoted 29 lines · Chuck Lever
Hey Jeff -
Quoted 8 lines · Jeff Layton
The existing per-procedure call counts live in global svc_version->vs_count[] arrays which are not network-namespace-aware. Add per-netns equivalents in struct
svc_statso the upcoming netlink stats interface can return namespace-scoped statistics.Add a
vs_countpointer array to structsvc_stat, along withsvc_stat_alloc_counts()andsvc_stat_free_counts()helpers to manage per-version percpu call count arrays.This patch appears to break the build for an NFSv3-only server configured with
CONFIG_PROC_FS=n.
svc_stat_alloc_counts()andsvc_stat_free_counts()are defined innet/sunrpc/stats.c, which theMakefilebuilds only underPROC_FS:sunrpc-$(CONFIG_PROC_FS) += stats.onfsd_net_init()/nfsd_net_exit() call both helpers unconditionally, and the declarations in <
linux/sunrpc/stats.h> sit outside the #ifdefCONFIG_PROC_FSblock with no !PROC_FS stubs.CONFIG_NFSDdoes not depend onPROC_FS-- onlyNFSD_V4does -- so NFSD=y with NFSD_V2/V3 and PROC_FS=n is a validKconfigcombination. The result is an undefined reference tosvc_stat_alloc_countsat vmlinux link (NFSD=y), or a modpost failure for NFSD=m since both symbols areEXPORT_SYMBOL_GPL.
Quoted 2 lines
net/sunrpc/svc.cis always built, and might be a more natural home for the new functions.
Thanks. Looks pretty simple to fix. I'll plan to send a v5.
Signature · 1 line
Jeff Layton <[email protected]>
Jeff Layton
maintainerRe: [PATCH v4 3/4] nfsd: implement server-stats-get netlink handler
In reply to Chuck LeverQuoted 49 lines · Chuck Lever
Hey Jeff -
Quoted 20 lines · Jeff Layton
Implement
nfsd_nl_server_stats_get_dumpit()which exposes the NFS server statistics currently available via/proc/net/rpc/nfsdthrough the nfsd generic netlink family.The handler uses a dump operation to stream statistics across multiple netlink messages:
- First message: all scalar stats (reply cache, filehandle, IO, network, RPC) plus per-version procedure counts (proc2/3/4-ops) using per-netns
vs_countarrays.- Subsequent messages: NFSv4 per-operation counts (proc4ops-ops), one entry per message, using cb->args[0] to track the current operation index across dump calls.
This allows nfsstat to retrieve server statistics via netlink with a procfs fallback for older kernels.
Assisted-by Claude:claude-opus-4-6Signed-off-by Jeff Layton <[email protected]>Quoted 7 lines
--- Documentation/netlink/specs/nfsd.yaml | 105 +++++++++++++++++ fs/nfsd/netlink.c | 5 + fs/nfsd/netlink.h | 2 + fs/nfsd/nfsctl.c | 206 ++++++++++++++++++++++++++++++++++ include/uapi/linux/nfsd_netlink.h | 35 ++++++ 5 files changed, 353 insertions(+)The procfs output in
nfsd_show()emits one more counter that this dump handler drops:wdeleg_getattr(NFSD_STATS_WDELEG_GETATTR), printed at the end of theCONFIG_NFSD_V4block infs/nfsd/stats.c, right after proc4ops. The netlink dump stops after proc4ops-ops and sets cb->args[0] = -1, so the write-delegation GETATTR-conflict counter never goes out on the wire.There's no schema slot for it either: the server-stats attribute-set ends at proc4ops-ops, so a consumer that prefers netlink and only falls back to procfs on an old kernel loses this counter on every kernel new enough to support server-stats-get.
I Suggest:
- add a wdeleg-getattr (u64) attribute to the server-stats attribute-set in
nfsd.yamland list it under the dump reply attributes;
Quoted 7 lines
- add
NFSD_A_SERVER_STATS_WDELEG_GETATTRto the uapi enum, afterPROC4OPS_OPSand before__NFSD_A_SERVER_STATS_MAXso existing values don't shift;- emit it as a single scalar in the start == 0 message, wrapped in
CONFIG_NFSD_V4to match the procfs side.
This omission was intentional. The stated purpose of this patchset was to convert nfsstat to use netlink. nfsstat does not access or display this field today. Without something that consumes this stat, I don't see the point of adding it here. We can always add it later when/if it becomes useful to userland.
Is there something that accesses this today that I'm not aware of? If not, is there some future intent to add this to nfsstat?
Signature · 1 line
Jeff Layton <[email protected]>
Chuck Lever
maintainerRe: [PATCH v4 3/4] nfsd: implement server-stats-get netlink handler
Quoted 58 lines · Jeff Layton
Quoted 49 lines · Chuck Lever
Hey Jeff -
Quoted 20 lines · Jeff Layton
Implement
nfsd_nl_server_stats_get_dumpit()which exposes the NFS server statistics currently available via/proc/net/rpc/nfsdthrough the nfsd generic netlink family.The handler uses a dump operation to stream statistics across multiple netlink messages:
- First message: all scalar stats (reply cache, filehandle, IO, network, RPC) plus per-version procedure counts (proc2/3/4-ops) using per-netns
vs_countarrays.- Subsequent messages: NFSv4 per-operation counts (proc4ops-ops), one entry per message, using cb->args[0] to track the current operation index across dump calls.
This allows nfsstat to retrieve server statistics via netlink with a procfs fallback for older kernels.
Assisted-by Claude:claude-opus-4-6Signed-off-by Jeff Layton <[email protected]>Quoted 7 lines
--- Documentation/netlink/specs/nfsd.yaml | 105 +++++++++++++++++ fs/nfsd/netlink.c | 5 + fs/nfsd/netlink.h | 2 + fs/nfsd/nfsctl.c | 206 ++++++++++++++++++++++++++++++++++ include/uapi/linux/nfsd_netlink.h | 35 ++++++ 5 files changed, 353 insertions(+)The procfs output in
nfsd_show()emits one more counter that this dump handler drops:wdeleg_getattr(NFSD_STATS_WDELEG_GETATTR), printed at the end of theCONFIG_NFSD_V4block infs/nfsd/stats.c, right after proc4ops. The netlink dump stops after proc4ops-ops and sets cb->args[0] = -1, so the write-delegation GETATTR-conflict counter never goes out on the wire.There's no schema slot for it either: the server-stats attribute-set ends at proc4ops-ops, so a consumer that prefers netlink and only falls back to procfs on an old kernel loses this counter on every kernel new enough to support server-stats-get.
I Suggest:
- add a wdeleg-getattr (u64) attribute to the server-stats attribute-set in
nfsd.yamland list it under the dump reply attributes;Quoted 7 lines
- add
NFSD_A_SERVER_STATS_WDELEG_GETATTRto the uapi enum, afterPROC4OPS_OPSand before__NFSD_A_SERVER_STATS_MAXso existing values don't shift;- emit it as a single scalar in the start == 0 message, wrapped in
CONFIG_NFSD_V4to match the procfs side.
Quoted 5 lines
This omission was intentional. The stated purpose of this patchset was to convert nfsstat to use netlink. nfsstat does not access or display this field today. Without something that consumes this stat, I don't see the point of adding it here. We can always add it later when/if it becomes useful to userland.
If we were adding a greenfield kernel-userspace API, I would agree. However, this API is meant to be compatible with the existing procfs API, so it should match to prevent a regression (I'm not aware of a consumer, this is only a general guideline).
For future reference: since a missing feature cannot be expressed in a patch's diff body, your intention to exclude this field could be mentioned in the commit message.
Quoted 2 lines
Is there something that accesses this today that I'm not aware of? If not, is there some future intent to add this to nfsstat?
I think nfsstat should display the wdeleg-getattr field. It has simply been forgotten.
Signature · 1 line
Chuck Lever
Jeff Layton
maintainerRe: [PATCH v4 3/4] nfsd: implement server-stats-get netlink handler
Quoted 76 lines · Chuck Lever
Quoted 58 lines · Jeff Layton
Quoted 49 lines · Chuck Lever
Hey Jeff -
Quoted 20 lines · Jeff Layton
Implement
nfsd_nl_server_stats_get_dumpit()which exposes the NFS server statistics currently available via/proc/net/rpc/nfsdthrough the nfsd generic netlink family.The handler uses a dump operation to stream statistics across multiple netlink messages:
- First message: all scalar stats (reply cache, filehandle, IO, network, RPC) plus per-version procedure counts (proc2/3/4-ops) using per-netns
vs_countarrays.- Subsequent messages: NFSv4 per-operation counts (proc4ops-ops), one entry per message, using cb->args[0] to track the current operation index across dump calls.
This allows nfsstat to retrieve server statistics via netlink with a procfs fallback for older kernels.
Assisted-by Claude:claude-opus-4-6Signed-off-by Jeff Layton <[email protected]>Quoted 7 lines
--- Documentation/netlink/specs/nfsd.yaml | 105 +++++++++++++++++ fs/nfsd/netlink.c | 5 + fs/nfsd/netlink.h | 2 + fs/nfsd/nfsctl.c | 206 ++++++++++++++++++++++++++++++++++ include/uapi/linux/nfsd_netlink.h | 35 ++++++ 5 files changed, 353 insertions(+)The procfs output in
nfsd_show()emits one more counter that this dump handler drops:wdeleg_getattr(NFSD_STATS_WDELEG_GETATTR), printed at the end of theCONFIG_NFSD_V4block infs/nfsd/stats.c, right after proc4ops. The netlink dump stops after proc4ops-ops and sets cb->args[0] = -1, so the write-delegation GETATTR-conflict counter never goes out on the wire.There's no schema slot for it either: the server-stats attribute-set ends at proc4ops-ops, so a consumer that prefers netlink and only falls back to procfs on an old kernel loses this counter on every kernel new enough to support server-stats-get.
I Suggest:
- add a wdeleg-getattr (u64) attribute to the server-stats attribute-set in
nfsd.yamland list it under the dump reply attributes;Quoted 7 lines
- add
NFSD_A_SERVER_STATS_WDELEG_GETATTRto the uapi enum, afterPROC4OPS_OPSand before__NFSD_A_SERVER_STATS_MAXso existing values don't shift;- emit it as a single scalar in the start == 0 message, wrapped in
CONFIG_NFSD_V4to match the procfs side.Quoted 5 lines
This omission was intentional. The stated purpose of this patchset was to convert nfsstat to use netlink. nfsstat does not access or display this field today. Without something that consumes this stat, I don't see the point of adding it here. We can always add it later when/if it becomes useful to userland.
If we were adding a greenfield kernel-userspace API, I would agree. However, this API is meant to be compatible with the existing procfs API, so it should match to prevent a regression (I'm not aware of a consumer, this is only a general guideline).
For future reference: since a missing feature cannot be expressed in a patch's diff body, your intention to exclude this field could be mentioned in the commit message.
Quoted 6 lines
Quoted 2 lines
Is there something that accesses this today that I'm not aware of? If not, is there some future intent to add this to nfsstat?
I think nfsstat should display the wdeleg-getattr field. It has simply been forgotten.
Ok. I'll plan to add it to the interface and send it in v5. nfs-utils will need a follow-on patch to display the value. No promises on when I'll do that, but it shouldn't be too hard.
Signature · 1 line
Jeff Layton <[email protected]>
Jeff Layton
maintainerRe: [PATCH v4 3/4] nfsd: implement server-stats-get netlink handler
Quoted 80 lines · Jeff Layton
Quoted 79 lines · Chuck Lever
Quoted 58 lines · Jeff Layton
Quoted 49 lines · Chuck Lever
Hey Jeff -
Quoted 20 lines · Jeff Layton
Implement
nfsd_nl_server_stats_get_dumpit()which exposes the NFS server statistics currently available via/proc/net/rpc/nfsdthrough the nfsd generic netlink family.The handler uses a dump operation to stream statistics across multiple netlink messages:
- First message: all scalar stats (reply cache, filehandle, IO, network, RPC) plus per-version procedure counts (proc2/3/4-ops) using per-netns
vs_countarrays.- Subsequent messages: NFSv4 per-operation counts (proc4ops-ops), one entry per message, using cb->args[0] to track the current operation index across dump calls.
This allows nfsstat to retrieve server statistics via netlink with a procfs fallback for older kernels.
Assisted-by Claude:claude-opus-4-6Signed-off-by Jeff Layton <[email protected]>Quoted 7 lines
--- Documentation/netlink/specs/nfsd.yaml | 105 +++++++++++++++++ fs/nfsd/netlink.c | 5 + fs/nfsd/netlink.h | 2 + fs/nfsd/nfsctl.c | 206 ++++++++++++++++++++++++++++++++++ include/uapi/linux/nfsd_netlink.h | 35 ++++++ 5 files changed, 353 insertions(+)The procfs output in
nfsd_show()emits one more counter that this dump handler drops:wdeleg_getattr(NFSD_STATS_WDELEG_GETATTR), printed at the end of theCONFIG_NFSD_V4block infs/nfsd/stats.c, right after proc4ops. The netlink dump stops after proc4ops-ops and sets cb->args[0] = -1, so the write-delegation GETATTR-conflict counter never goes out on the wire.There's no schema slot for it either: the server-stats attribute-set ends at proc4ops-ops, so a consumer that prefers netlink and only falls back to procfs on an old kernel loses this counter on every kernel new enough to support server-stats-get.
I Suggest:
- add a wdeleg-getattr (u64) attribute to the server-stats attribute-set in
nfsd.yamland list it under the dump reply attributes;Quoted 7 lines
- add
NFSD_A_SERVER_STATS_WDELEG_GETATTRto the uapi enum, afterPROC4OPS_OPSand before__NFSD_A_SERVER_STATS_MAXso existing values don't shift;- emit it as a single scalar in the start == 0 message, wrapped in
CONFIG_NFSD_V4to match the procfs side.Quoted 5 lines
This omission was intentional. The stated purpose of this patchset was to convert nfsstat to use netlink. nfsstat does not access or display this field today. Without something that consumes this stat, I don't see the point of adding it here. We can always add it later when/if it becomes useful to userland.
If we were adding a greenfield kernel-userspace API, I would agree. However, this API is meant to be compatible with the existing procfs API, so it should match to prevent a regression (I'm not aware of a consumer, this is only a general guideline).
For future reference: since a missing feature cannot be expressed in a patch's diff body, your intention to exclude this field could be mentioned in the commit message.
Quoted 2 lines
Is there something that accesses this today that I'm not aware of? If not, is there some future intent to add this to nfsstat?
Quoted 7 lines
Quoted 3 lines
I think nfsstat should display the wdeleg-getattr field. It has simply been forgotten.
Ok. I'll plan to add it to the interface and send it in v5. nfs-utils will need a follow-on patch to display the value. No promises on when I'll do that, but it shouldn't be too hard.
I have a drafted patch, but I'm not thrilled with it. wdeleg_getattr is weird in that it's not a statistic about server-side protocol operations. So, it ends up sort of sitting off on its own in nfsstat.
What would you (and Dai) think about instead keeping and exposing server-side callback call counts? Then we could just add a new section to nfsstat server-side output with all of the callback operation counts in it. wdeleg_getattr would just be the count for CB_GETATTRs.
Thoughts?
Signature · 1 line
Jeff Layton <[email protected]>
Chuck Lever
maintainerRe: [PATCH v4 3/4] nfsd: implement server-stats-get netlink handler
Quoted 20 lines · Jeff Layton
Quoted 7 lines · Jeff Layton
Quoted 6 lines · Chuck Lever
Quoted 2 lines · Jeff Layton
Is there something that accesses this today that I'm not aware of? If not, is there some future intent to add this to nfsstat?
Quoted 7 lines
Quoted 3 lines
I think nfsstat should display the wdeleg-getattr field. It has simply been forgotten.
Ok. I'll plan to add it to the interface and send it in v5. nfs-utils will need a follow-on patch to display the value. No promises on when I'll do that, but it shouldn't be too hard.
I have a drafted patch, but I'm not thrilled with it.
wdeleg_getattris weird in that it's not a statistic about server-side protocol operations. So, it ends up sort of sitting off on its own in nfsstat.
Quoted 6 lines
What would you (and Dai) think about instead keeping and exposing server-side callback call counts? Then we could just add a new section to nfsstat server-side output with all of the callback operation counts in it.
wdeleg_getattrwould just be the count forCB_GETATTRs.Thoughts?
That sounds ideal.
Signature · 1 line
Chuck Lever