Skip to contentSkip to conversation

SUNRPC: dispatch ready transports round-robin across clients, classes from BPF

RFC PATCH In review Benjamin Coddington 3 participants 17 messages Oct 7 – Oct 8

Subsystems (8)
lore
Share & tools
lore
#0 lore raw

v2 cover SUNRPC: dispatch ready transports round-robin across clients, classes from BPF

From: Benjamin Coddington <[email protected]>

This is v2 of [3], with the change I floated in that thread: the kernel no longer decides which transports belong together. By default nothing changes. An administrator who wants peers grouped loads a small BPF program that returns a class for each accepted connection, and the pool takes turns across classes.

What's different from v1
------------------------

The table I described on [3] became a hook. Instead of a kernel table of address prefixes, a netlink interface for it and an nfs-utils side, sunrpc registers a struct_ops with one callback: given the accepted svc_xprt, return a class. 0 means the anonymous client (shared with everything unclassified, today's order among them); any other number is a class every transport returning it shares; a flag bit on top means "and one client per peer address within that class". So "all movers are one class, everybody else per host" is two entries in a prefix map, and the posted v1 behaviour is one entry.

The hook runs once per accepted connection, in process context, after TCP and RDMA have both set the peer address. Nothing runs per request. No program loaded means every transport is on the
anonymous client, and with patch 6 the pool uses its old flat queue until a classifier attaches, so the dispatch path is today's code for anyone who hasn't asked for anything. That's the answer to Chuck's cost question from [3]: measured on bare metal the two-level queue cost about half a microsecond a dispatch (1.7% at 484k IOPS on one client's eight connections); with no program it now costs a static branch on enqueue and an empty-queue test on dequeue.

Attaching is a struct_ops link, bound to the attaching task's network namespace, one per namespace. A second attach gets -EBUSY, a link update replaces the program, closing the link removes it, and a namespace that exits leaves its link with nothing behind. Transports keep the class they got at accept; a new program only affects new connections. The reference program and the attach tests are in the BPF selftests (patch 7). svc-classify (patch 8) attaches the prefix classifier and manages its map by address prefix, so an administrator never touches bpftool. The page in patch 9, Documentation/filesystems/nfs/rpc-server-clients.rst, has the class word, the install procedure, and six class maps with the share each one gives its clients.

Still not in here: moving a transport between clients, so a v4.1 clientid key, which Chuck asked about on [3], is not in this version. A multi-homed host or an IPv6 host with a prefix's worth of addresses is handled by the prefix map instead.

Also from the [3] review: control events (accept, close, handshake) go ahead of data within a client (patch 5), and the dequeue tracepoint prints the class (patch 4).

Numbers
-------

The dispatch numbers are the v1 series' on a two-socket box (2 x Xeon 6542Y, 96 threads, no KASAN), from [3]. The dispatch code is the same
here: v2 puts the classifier in front of it, and the flat queue behind a static branch when nothing is attached. Loopback, NFSv3, 16 threads, 10 ms injected service time with 50% jitter. A is v7.2, B the series. v4.1 is within 2% in every cell.

One interactive client (bursts of 32) against one host with K backlogged connections, burst completion p50 in ms, floor 37:

    K             4      8     16     32
    A            70    195    351    671
    B            56     62     61     63

    N (K=16)      1      8     32     64     96    128
    A            21     98    350    690   1035   1365
    B            11     31     62    103    144    179

    M hosts x 4   1      2      4      8
    A            70    192    351    670
    B            57     81    121    192

    S: 2x8 vs K, share   4      8     16     32
    A                 41.0   20.0   11.1    5.9
    B                 50.3   50.0   50.0   50.0

A real NFSv3 client walking 500 files against a 16-connection
aggressor: 37.8 s on A, 2.5 s on B, 0.12 s alone on both.

Classes. The v1 kernel keyed on source address, so on [3] I emulated "all movers are one class" by giving six movers one address. With this series that's one map entry (the movers' prefix -> 1, everything else per address), and the six-address column is what "everything per address" gives. Six movers at 4 or 8 connections each against customers, customer share of dispatches:

                                 A           per address   movers one class
    one customer, 1 conn x 16    4.0 / 2.0   14.3 / 14.3   49.9 / 49.9
    one customer, 4 x 4         14.3 / 7.7   14.3 / 14.3   49.8 / 49.7
    four customers, 4 x 4 each  40 / 25      40 / 40       80 / 80

Burst of 32 against M busy hosts, p50 ms: hosts on their own addresses 57 81 121 192, as one class 56 62 62 62. A: 71 193 348 673 either way.

Cost. With a classifier attached, dispatch takes two or three more lock-free queue operations per request. On the same box that's about half a microsecond when the enqueue and the dequeue run on different cores, fio 4k O_DIRECT randread over loopback, 5 x 20s:

                                   A        B
    nconnect=1, 16 jobs        152.8k   152.2k   -0.4%
    nconnect=8, 16 jobs        483.8k   475.5k   -1.7%
    nconnect=8, 64 jobs        584.2k   581.0k   -0.5%

The serial walk and 4 jobs on one connection don't move. With no classifier attached the pool is on its old flat queue (patch 6). I only have my VM for that path so far (KASAN, 10 vCPUs, loopback, the same fio), and there the no-program kernel measures the same as v7.2 within the run-to-run spread: 16 jobs 94.9k vs 95.5k, 64 jobs 138.6k vs 139.0k. So the static branch adds nothing I can see, but that's a VM number.

Not tested: RDMA. The hook sits in the accept path TCP and RDMA share, but I could not bring a soft-RoCE listener up on the VM.

Daire - the same nine patches on v7.2 (plus the wake fix) are at
https://github.com/bcodding/linux branch nfsd-clientq-v2-7.2 if you want to run it; the loader builds from tools/net/sunrpc/svc-classify in that tree.

The series is on nfsd-testing at 56589cdb5881, which already has the svc_clean_up_xprts() wake fix from [3].

  patch 1  track clients by class, no change to dispatch
  patch 2  dispatch round-robin across clients
  patch 3  the struct_ops hook
  patch 4  class in the svc_xprt_dequeue tracepoint
  patch 5  control events ahead of data within a client
  patch 6  flat queue while no classifier is attached
  patch 7  selftests: the prefix program and the attach tests
  patch 8  tools/net/sunrpc/svc-classify: the prefix classifier and its

loader (load/replace/unload, add/del/list by prefix, status)
patch 9 Documentation: the model, the class word, installing a
classifier with svc-classify, six class maps and the share each gives, how it works underneath

[1] https://lore.kernel.org/linux-nfs/[email protected]
[2] https://lore.kernel.org/linux-nfs/[email protected]
[3] https://lore.kernel.org/linux-nfs/[email protected]

Benjamin Coddington (9):
SUNRPC: track service clients by class
SUNRPC: dispatch ready transports round-robin across clients
SUNRPC: add a BPF struct_ops hook to classify accepted transports
SUNRPC: report the transport's class in the svc_xprt_dequeue
tracepoint
SUNRPC: dispatch control events ahead of data within a client
SUNRPC: keep the single transport queue while no classifier is
attached
selftests/bpf: add svc_classifier tests
tools/net/sunrpc: add svc-classify, a prefix classifier and its loader
Documentation: describe RPC server transport classes and the BPF
classifier

 Documentation/filesystems/nfs/index.rst       |   1 +
 .../filesystems/nfs/rpc-server-clients.rst    | 394 ++++++++++++++++++
 include/linux/sunrpc/svc.h                    |  72 +++-
 include/linux/sunrpc/svc_xprt.h               |  30 ++
 include/trace/events/sunrpc.h                 |  11 +-
 net/sunrpc/Kconfig                            |  12 +
 net/sunrpc/Makefile                           |   1 +
 net/sunrpc/netns.h                            |   4 +
 net/sunrpc/sunrpc_syms.c                      |   4 +
 net/sunrpc/svc.c                              |  36 ++
 net/sunrpc/svc_classify.c                     | 210 ++++++++++
 net/sunrpc/svc_xprt.c                         | 281 ++++++++++++-
 tools/net/sunrpc/svc-classify/.gitignore      |   1 +
 tools/net/sunrpc/svc-classify/Makefile        |  65 +++
 tools/net/sunrpc/svc-classify/svc-classify.c  | 357 ++++++++++++++++
 .../sunrpc/svc-classify/svc_classify.bpf.c    |  83 ++++
 tools/testing/selftests/bpf/config            |   2 +
 .../selftests/bpf/prog_tests/svc_classifier.c | 104 +++++
 .../selftests/bpf/progs/bpf_svc_classifier.c  |  80 ++++
 19 files changed, 1724 insertions(+), 24 deletions(-)
 create mode 100644 Documentation/filesystems/nfs/rpc-server-clients.rst
 create mode 100644 net/sunrpc/svc_classify.c
 create mode 100644 tools/net/sunrpc/svc-classify/.gitignore
 create mode 100644 tools/net/sunrpc/svc-classify/Makefile
 create mode 100644 tools/net/sunrpc/svc-classify/svc-classify.c
 create mode 100644 tools/net/sunrpc/svc-classify/svc_classify.bpf.c
 create mode 100644 tools/testing/selftests/bpf/prog_tests/svc_classifier.c
 create mode 100644 tools/testing/selftests/bpf/progs/bpf_svc_classifier.c
Signature · 1 line
2.53.0
#1 lore raw

RFC PATCH v2 1/9 SUNRPC: track service clients by class

From: Benjamin Coddington <[email protected]>

A pool dispatches its ready transports in FIFO order, one RPC per turn, so a peer's share of the service grows with the number of connections it holds: a client with K connections is served K times as often as a client with one. Dispatching per client instead needs the service to know which transports belong together, and which transports belong together is a question for the administrator, not the kernel: one site wants every host served alike, another wants a set of bulk movers to share one turn among them.

Add struct svc_client, one per class of transports in a network namespace of a service, or one per peer address within a class, found through a small hash table under a spinlock. The class of an accepted transport comes from svc_classify(), which for now returns SVC_CLASS_NONE, so every transport stays on the service's anonymous client; a later patch lets an administrator's BPF program supply it. The table is touched only when a connection is accepted and when a client's last transport is freed; dispatch will follow the transport's xpt_client pointer, which is set before the transport can be enqueued (it is still XPT_BUSY from svc_xprt_init()) and is kept alive by the transport's reference. Each client carries a per-pool lwq of its ready transports and a node for the pool's queue of clients, unused as yet.

Listeners, UDP sockets, transports of no class, and accepted transports for which no client can be allocated are bound to the anonymous client.

Dispatch is unchanged by this patch.

---
 include/linux/sunrpc/svc.h      |  43 +++++++++++++
 include/linux/sunrpc/svc_xprt.h |   8 +++
 net/sunrpc/svc.c                |  34 ++++++++++
 net/sunrpc/svc_xprt.c           | 108 ++++++++++++++++++++++++++++++++
 4 files changed, 193 insertions(+)
include/linux/sunrpc/svc.h +43 -0
index dfadea50e0c6..e45f4fdea20b 100644
@@ -58,6 +58,44 @@ enum {
 	SP_TASK_STARTING,	/* Task has started but not added to idle yet */
 };
 
+/*
+ * A group of transports that share turns at the service: those of one
+ * class in one network namespace, or of one peer address within a class.
+ * Ready transports are queued per client and per pool, and a pool
+ * dispatches round-robin across its queued clients, so a client's share
+ * of the service does not grow with its connection count.
+ */
+struct svc_client_pool {
+	struct lwq		cp_xprts;	/* ready transports */
+	struct lwq_node		cp_ready;	/* link in svc_pool.sp_clients */
+	unsigned long		cp_flags;
+	struct svc_client	*cp_client;
+};
+
+enum {
+	SVC_CP_QUEUED,	/* on sp_clients, or held by the thread that took it */
+};
+
+struct svc_client {
+	struct hlist_node	cl_hash;
+	refcount_t		cl_ref;
+	struct net		*cl_net;
+	u32			cl_class;
+	struct sockaddr_storage	cl_addr;	/* with SVC_CLASS_PER_ADDR; port ignored */
+	struct svc_client_pool	cl_pool[];	/* one per pool */
+};
+
+#define SVC_CLIENT_HASH_BITS	8
+
+/*
+ * Classes returned by svc_classify().  SVC_CLASS_NONE leaves the transport
+ * on the service's anonymous client; any other value names a client shared
+ * by every transport of that class, or, with SVC_CLASS_PER_ADDR set, one
+ * client per peer address within the class.
+ */
+#define SVC_CLASS_NONE		0
+#define SVC_CLASS_PER_ADDR	BIT_U32(31)
+
 
 struct svc_rqst;
 
@@ -96,6 +134,10 @@ struct svc_serv {
 	struct svc_pool *	sv_pools;	/* array of thread pools */
 	int			(*sv_threadfn)(void *data);
 
+	spinlock_t		sv_client_lock;	/* protects sv_client_hash */
+	struct hlist_head	sv_client_hash[1 << SVC_CLIENT_HASH_BITS];
+	struct svc_client	*sv_anon_client; /* transports without a peer */
+
 #if defined(CONFIG_SUNRPC_BACKCHANNEL)
 	struct lwq		sv_cb_list;	/* queue for callback requests
 						 * that arrive over the same
@@ -503,6 +545,7 @@ void		   svc_reserve(struct svc_rqst *rqstp, int space);
 void		   svc_pool_wake_idle_thread(struct svc_pool *pool);
 struct svc_pool   *svc_pool_for_cpu(struct svc_serv *serv);
 unsigned int	   svc_serv_nrpools(const struct svc_serv *serv);
+struct svc_client *svc_client_alloc(unsigned int nrpools, gfp_t gfp);
 char *		   svc_print_addr(struct svc_rqst *, char *, size_t);
 const char *	   svc_proc_name(const struct svc_rqst *rqstp);
 int		   svc_encode_result_payload(struct svc_rqst *rqstp,
include/linux/sunrpc/svc_xprt.h +8 -0
index 7176c42f19d7..d4f6dd5c8db3 100644
@@ -63,6 +63,7 @@ struct svc_xprt {
 	unsigned long		xpt_flags;
 
 	struct svc_serv		*xpt_server;	/* service for transport */
+	struct svc_client	*xpt_client;	/* peer this transport belongs to */
 	atomic_t		xpt_reserved;	/* outq space rsvd, UDP only */
 	atomic_t		xpt_nr_rqsts;	/* Number of requests */
 	struct mutex		xpt_mutex;	/* to serialize sending data */
@@ -177,6 +178,13 @@ int	svc_xprt_create(struct svc_serv *serv, const char *xprt_name,
 void	svc_xprt_destroy_all(struct svc_serv *serv, struct net *net,
 			     bool unregister);
 void	svc_xprt_received(struct svc_xprt *xprt);
+
+/* The class of an accepted transport; SVC_CLASS_* in svc.h. */
+static inline u32 svc_classify(const struct svc_xprt *xprt)
+{
+	return SVC_CLASS_NONE;
+}
+
 void	svc_xprt_enqueue(struct svc_xprt *xprt);
 void	svc_xprt_put(struct svc_xprt *xprt);
 void	svc_xprt_copy_addrs(struct svc_rqst *rqstp, struct svc_xprt *xprt);
net/sunrpc/svc.c +34 -0
index 0a2c040ad096..cb2b7caaa627 100644
@@ -387,6 +387,30 @@ static void svc_pool_destroy_counters(struct svc_pool *pool)
 	percpu_counter_destroy(&pool->sp_threads_woken);
 }
 
+/**
+ * svc_client_alloc - allocate a service client
+ * @nrpools: number of thread pools in the service
+ * @gfp: allocation flags
+ *
+ * Return: the client, holding one reference, or %NULL.
+ */
+struct svc_client *svc_client_alloc(unsigned int nrpools, gfp_t gfp)
+{
+	struct svc_client *cl;
+	unsigned int i;
+
+	cl = kzalloc(struct_size(cl, cl_pool, nrpools), gfp);
+	if (!cl)
+		return NULL;
+	refcount_set(&cl->cl_ref, 1);
+	INIT_HLIST_NODE(&cl->cl_hash);
+	for (i = 0; i < nrpools; i++) {
+		lwq_init(&cl->cl_pool[i].cp_xprts);
+		cl->cl_pool[i].cp_client = cl;
+	}
+	return cl;
+}
+
 /*
  * Create an RPC service
  */
@@ -432,8 +456,16 @@ __svc_create(struct svc_program *prog, int nprogs, struct svc_stat *stats,
 
 	__svc_init_bc(serv);
 
+	spin_lock_init(&serv->sv_client_lock);
+	serv->sv_anon_client = svc_client_alloc(npools, GFP_KERNEL);
+	if (!serv->sv_anon_client) {
+		kfree(serv);
+		return NULL;
+	}
+
 	serv->sv_pools = kzalloc_objs(struct svc_pool, npools);
 	if (!serv->sv_pools) {
+		kfree(serv->sv_anon_client);
 		kfree(serv);
 		return NULL;
 	}
@@ -459,6 +491,7 @@ __svc_create(struct svc_program *prog, int nprogs, struct svc_stat *stats,
 	while (i--)
 		svc_pool_destroy_counters(&serv->sv_pools[i]);
 	kfree(serv->sv_pools);
+	kfree(serv->sv_anon_client);
 	kfree(serv);
 	return NULL;
 }
@@ -545,6 +578,7 @@ svc_destroy(struct svc_serv **servp)
 	if (serv->sv_is_pooled)
 		svc_pool_map_put();
 
+	kfree(serv->sv_anon_client);
 	kfree(serv->sv_pools);
 	kfree(serv);
 }
net/sunrpc/svc_xprt.c +108 -0

140 lines not on this page · raw message

Signature · 1 line
2.53.0
#2 lore raw

RFC PATCH v2 2/9 SUNRPC: dispatch ready transports round-robin across clients

In reply to Benjamin Coddington

From: Benjamin Coddington <[email protected]>

A pool keeps its ready transports in one FIFO and serves one RPC per turn, so a peer is served in proportion to how many of its connections are backlogged. A client with K connections, or a deep NFSv4.1 slot table spread over nconnect transports, takes K turns for every one a single-connection client gets, and the single-connection client's latency grows with everyone else's connection count.

Queue ready transports on their client instead, and have the pool dispatch round-robin across clients: take the client at the head of the pool's client queue, dispatch one of its transports, and put the client back at the tail if it has more. Every client with work queued gets one dispatch per round however many connections it holds, and a client's wait is bounded by the number of busy clients rather than by the number of busy connections. Within a client, transports are still served in FIFO order.

Enqueue stays lockless: the transport goes on the client's per-pool lwq, and the client goes on the pool's lwq the first time one of its transports becomes ready, guarded by a per-pool SVC_CP_QUEUED bit that the dequeuing thread clears when it finds the client empty. The bit is cleared before the client's queue is re-tested, so an enqueue that saw it set and did not queue the client is caught by the re-test. Dequeue takes one more lwq spinlock than before. Control events (XPT_CONN, XPT_CLOSE, XPT_HANDSHAKE) take turns like data: a close queued behind k of its own client's transports is dispatched k rounds later, where it used to wait behind every queued transport in the pool.

Network namespace teardown walks the pool's clients: those of the namespace being destroyed are drained and their transports deleted; the anonymous client, which may hold listeners from several namespaces, is filtered per transport; the rest are re-queued and a thread is woken.

While nothing returns a class from svc_classify(), every transport is on the anonymous client, a pool's client queue holds one entry, and dispatch is in today's FIFO order.

---
 include/linux/sunrpc/svc.h |  2 +-
 net/sunrpc/svc.c           |  2 +-
 net/sunrpc/svc_xprt.c      | 97 +++++++++++++++++++++++++++++---------
 3 files changed, 77 insertions(+), 24 deletions(-)
include/linux/sunrpc/svc.h +1 -1
index e45f4fdea20b..a27dc2942167 100644
@@ -38,7 +38,7 @@ struct svc_pool {
 	unsigned int		sp_nrthreads;	/* # of threads currently running in pool */
 	unsigned int		sp_nrthrmin;	/* Min number of threads to run per pool */
 	unsigned int		sp_nrthrmax;	/* Max requested number of threads in pool */
-	struct lwq		sp_xprts;	/* pending transports */
+	struct lwq		sp_clients;	/* clients with ready transports */
 	struct list_head	sp_all_threads;	/* all server threads */
 	struct llist_head	sp_idle_threads; /* idle server threads */
 
net/sunrpc/svc.c +1 -1
index cb2b7caaa627..4adf113d067b 100644
@@ -477,7 +477,7 @@ __svc_create(struct svc_program *prog, int nprogs, struct svc_stat *stats,
 				i, serv->sv_name);
 
 		pool->sp_id = i;
-		lwq_init(&pool->sp_xprts);
+		lwq_init(&pool->sp_clients);
 		INIT_LIST_HEAD(&pool->sp_all_threads);
 		init_llist_head(&pool->sp_idle_threads);
 
net/sunrpc/svc_xprt.c +75 -22

141 lines not on this page · raw message

Signature · 1 line
2.53.0
#3 lore raw

RFC PATCH v2 3/9 SUNRPC: add a BPF struct_ops hook to classify accepted transports

In reply to Benjamin Coddington

From: Benjamin Coddington <[email protected]>

Which transports should share turns at a service is site policy: one server wants every host served alike, another wants a set of bulk movers to count as one peer while its interactive clients are served per host, and a server behind a NAT or a re-export gateway wants the gateway's address split some other way. The kernel has no good basis for choosing, and the address table this started as would need its own configuration interface and a userspace tool to fill it.

Let the administrator supply the classification instead. Add struct svc_classifier, a BPF struct_ops with one callback that is handed each accepted svc_xprt, in process context with its peer and local addresses set, and returns an SVC_CLASS_* value: SVC_CLASS_NONE to leave the transport on the anonymous client, a class number shared by every transport in that class, or a class number with SVC_CLASS_PER_ADDR set for one client per peer address within the class. A program keyed on an address-prefix map does the whole job in a few lines. The callback runs under rcu_read_lock(), so sleepable programs are refused.

Attaching a classifier (a struct_ops link) binds it to the attaching task's network namespace, one per namespace and one namespace per instance; a second attach fails with -EBUSY, a link update replaces it, detaching or closing the link removes it, and a namespace that exits leaves its link with nothing behind it. The classifier applies to every RPC service in the namespace; a program that cares which one reads xpt_server->sv_name. Transports keep the class they were given at accept, so a change of classifier affects new connections only.

With no classifier attached, or with CONFIG_SUNRPC_BPF_CLASSIFY off, svc_classify() returns SVC_CLASS_NONE and nothing changes. The struct_ops registers from sunrpc's module init; a module without BTF registers nothing, and the BPF core says so when the configuration promised module BTF.

---
 include/linux/sunrpc/svc.h      |  24 ++++
 include/linux/sunrpc/svc_xprt.h |  15 +++
 net/sunrpc/Kconfig              |  12 ++
 net/sunrpc/Makefile             |   1 +
 net/sunrpc/netns.h              |   4 +
 net/sunrpc/sunrpc_syms.c        |   4 +
 net/sunrpc/svc_classify.c       | 200 ++++++++++++++++++++++++++++++++
 7 files changed, 260 insertions(+)
 create mode 100644 net/sunrpc/svc_classify.c
include/linux/sunrpc/svc.h +24 -0
index a27dc2942167..5d8e35c5379e 100644
@@ -96,6 +96,30 @@ struct svc_client {
 #define SVC_CLASS_NONE		0
 #define SVC_CLASS_PER_ADDR	BIT_U32(31)
 
+#define SVC_CLASSIFIER_NAME_MAX	16
+
+struct svc_xprt;
+
+/**
+ * struct svc_classifier - BPF struct_ops that classifies accepted transports
+ * @name: a label for the instance, set by the program and not interpreted
+ * @classify: called once per accepted transport, in process context, under
+ *	rcu_read_lock(), with the peer and local addresses set; returns an
+ *	SVC_CLASS_* value
+ *
+ * Attaching one binds it to the attaching task's network namespace, where
+ * it classifies every service's transports until it is detached; one per
+ * namespace.  Transports keep the class they were given when accepted.
+ */
+struct svc_classifier {
+	/* private: */
+	struct net	*net;
+
+	/* public: */
+	char		name[SVC_CLASSIFIER_NAME_MAX];
+	u32		(*classify)(const struct svc_xprt *xprt);
+};
+
 
 struct svc_rqst;
 
include/linux/sunrpc/svc_xprt.h +15 -0
index d4f6dd5c8db3..10c18e948b42 100644
@@ -180,11 +180,26 @@ void	svc_xprt_destroy_all(struct svc_serv *serv, struct net *net,
 void	svc_xprt_received(struct svc_xprt *xprt);
 
 /* The class of an accepted transport; SVC_CLASS_* in svc.h. */
+#if IS_ENABLED(CONFIG_SUNRPC_BPF_CLASSIFY)
+u32	svc_classify(const struct svc_xprt *xprt);
+void	svc_classifier_exit_net(struct net *net);
+int	svc_classifier_init_module(void);
+#else
 static inline u32 svc_classify(const struct svc_xprt *xprt)
 {
 	return SVC_CLASS_NONE;
 }
 
+static inline void svc_classifier_exit_net(struct net *net)
+{
+}
+
+static inline int svc_classifier_init_module(void)
+{
+	return 0;
+}
+#endif
+
 void	svc_xprt_enqueue(struct svc_xprt *xprt);
 void	svc_xprt_put(struct svc_xprt *xprt);
 void	svc_xprt_copy_addrs(struct svc_rqst *rqstp, struct svc_xprt *xprt);
net/sunrpc/Kconfig +12 -0
index e7808e5714dc..2b96ac8d1ba2 100644
@@ -16,6 +16,18 @@ config SUNRPC_SWAP
 	bool
 	depends on SUNRPC
 
+config SUNRPC_BPF_CLASSIFY
+	bool "BPF classification of RPC service transports"
+	depends on SUNRPC && BPF_SYSCALL && BPF_JIT
+	default y
+	help
+	  Let a BPF struct_ops program attached in a network namespace say
+	  which accepted transports of an RPC service (NFS, NLM, the NFSv4
+	  callback service) share turns at the service's threads.  With no
+	  program attached, dispatch is unchanged.
+
+	  If unsure, say Y.
+
 config RPCSEC_GSS_KRB5
 	tristate "Secure RPC: Kerberos V mechanism"
 	depends on SUNRPC && CRYPTO
net/sunrpc/Makefile +1 -0
index 96727df3aa85..fdcb860b616e 100644
@@ -19,3 +19,4 @@ sunrpc-$(CONFIG_SUNRPC_DEBUG) += debugfs.o
 sunrpc-$(CONFIG_SUNRPC_BACKCHANNEL) += backchannel_rqst.o
 sunrpc-$(CONFIG_PROC_FS) += stats.o
 sunrpc-$(CONFIG_SYSCTL) += sysctl.o
+sunrpc-$(CONFIG_SUNRPC_BPF_CLASSIFY) += svc_classify.o
net/sunrpc/netns.h +4 -0
index 4efb5f28d881..ed01bc5296dd 100644
@@ -34,6 +34,10 @@ struct sunrpc_net {
 	atomic_t pipe_users;
 	struct proc_dir_entry *use_gssp_proc;
 	struct proc_dir_entry *gss_krb5_enctypes;
+
+#if IS_ENABLED(CONFIG_SUNRPC_BPF_CLASSIFY)
+	struct svc_classifier __rcu *svc_classifier;
+#endif
 };
 
 extern unsigned int sunrpc_net_id;
net/sunrpc/sunrpc_syms.c +4 -0
index 1a3884a0376a..4fdb676baa30 100644
@@ -74,6 +74,7 @@ static __net_exit void sunrpc_exit_net(struct net *net)
 {
 	struct sunrpc_net *sn = net_generic(net, sunrpc_net_id);
 
+	svc_classifier_exit_net(net);
 	rpc_pipefs_exit_net(net);
 	unix_gid_cache_destroy(net);
 	ip_map_cache_destroy(net);
@@ -122,6 +123,9 @@ init_sunrpc(void)
 #endif
 	svc_init_xprt_sock();	/* svc sock transport */
 	init_socket_xprt();	/* clnt sock transport */
+	err = svc_classifier_init_module();
+	if (err)
+		pr_warn("RPC: no BPF transport classifier (%d)\n", err);
 	return 0;
 
 out6:
net/sunrpc/svc_classify.c new file +200 -0

200 lines not on this page · raw message

Signature · 1 line
2.53.0
#4 lore raw

RFC PATCH v2 4/9 SUNRPC: report the transport's class in the svc_xprt_dequeue tracepoint

In reply to Benjamin Coddington

From: Benjamin Coddington <[email protected]>

With dispatch round-robin across clients, which client a transport was served as matters as much as its address. Print the class the transport was bound to at accept: 0 for the anonymous client, the class number otherwise, with "+addr" when the client is one address within the class. The peer address is already in the event.

---
 include/trace/events/sunrpc.h | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)
include/trace/events/sunrpc.h +9 -2
index 77efe63d2c14..107fbed5d556 100644
@@ -2042,20 +2042,27 @@ TRACE_EVENT(svc_xprt_dequeue,
 
 	TP_STRUCT__entry(
 		SVC_XPRT_ENDPOINT_FIELDS(rqst->rq_xprt)
+		__field(u32, class)
+		__field(bool, per_addr)
 		__field(unsigned long, wakeup)
 		__field(unsigned long, qtime)
 	),
 
 	TP_fast_assign(
 		ktime_t ktime = ktime_get();
+		u32 class = rqst->rq_xprt->xpt_client->cl_class;
 
 		SVC_XPRT_ENDPOINT_ASSIGNMENTS(rqst->rq_xprt);
+		__entry->class = class & ~SVC_CLASS_PER_ADDR;
+		__entry->per_addr = class & SVC_CLASS_PER_ADDR;
 		__entry->wakeup = ktime_to_us(ktime_sub(ktime, rqst->rq_qtime));
 		__entry->qtime = ktime_to_us(ktime_sub(ktime, rqst->rq_xprt->xpt_qtime));
 	),
 
-	TP_printk(SVC_XPRT_ENDPOINT_FORMAT " wakeup-us=%lu qtime-us=%lu",
-		SVC_XPRT_ENDPOINT_VARARGS, __entry->wakeup, __entry->qtime)
+	TP_printk(SVC_XPRT_ENDPOINT_FORMAT " class=%u%s wakeup-us=%lu qtime-us=%lu",
+		  SVC_XPRT_ENDPOINT_VARARGS,
+		  __entry->class, __entry->per_addr ? "+addr" : "",
+		  __entry->wakeup, __entry->qtime)
 );
 
 DECLARE_EVENT_CLASS(svc_xprt_event,
Signature · 1 line
2.53.0
#5 lore raw

RFC PATCH v2 5/9 SUNRPC: dispatch control events ahead of data within a client

In reply to Benjamin Coddington

From: Benjamin Coddington <[email protected]>

A transport that needs a connection accepted, closed, or a handshake run waits its turn like a transport with data, behind every transport of its client queued before it, and a client with many connections can hold a close back for k rounds of all the busy clients. None of those events costs the service a request's worth of work, and a close frees resources the sooner it is served.

Give each client a second per-pool queue for transports with XPT_CONN, XPT_CLOSE or XPT_HANDSHAKE set when they are enqueued, and have the dispatcher serve that queue before the client's data queue. The client's turn in the round-robin is unchanged; only which of its transports goes first. A transport that is already queued with data when it is closed is served in its original order, as before.

---
 include/linux/sunrpc/svc.h      |  2 ++
 include/linux/sunrpc/svc_xprt.h |  3 ++
 net/sunrpc/svc.c                |  1 +
 net/sunrpc/svc_xprt.c           | 60 +++++++++++++++++++++++++++------
 4 files changed, 56 insertions(+), 10 deletions(-)
include/linux/sunrpc/svc.h +2 -0
index 5d8e35c5379e..02a8eb5d3357 100644
@@ -66,6 +66,7 @@ enum {
  * of the service does not grow with its connection count.
  */
 struct svc_client_pool {
+	struct lwq		cp_ctrl;	/* ready transports with a control event */
 	struct lwq		cp_xprts;	/* ready transports */
 	struct lwq_node		cp_ready;	/* link in svc_pool.sp_clients */
 	unsigned long		cp_flags;
@@ -74,6 +75,7 @@ struct svc_client_pool {
 
 enum {
 	SVC_CP_QUEUED,	/* on sp_clients, or held by the thread that took it */
+	SVC_CP_DATA_NEXT, /* last turn served a control event; data is owed */
 };
 
 struct svc_client {
include/linux/sunrpc/svc_xprt.h +3 -0
index 10c18e948b42..e05d7c8610ff 100644
@@ -116,6 +116,9 @@ enum {
 				 */
 };
 
+/* events a client's queue serves ahead of its data */
+#define SVC_XPT_CTRL_EVENTS	(BIT(XPT_CONN) | BIT(XPT_CLOSE) | BIT(XPT_HANDSHAKE))
+
 /*
  * Maximum number of "tmp" connections - those without XPT_PEER_VALID -
  * permitted on any service.
net/sunrpc/svc.c +1 -0
index 4adf113d067b..92c6737e5905 100644
@@ -405,6 +405,7 @@ struct svc_client *svc_client_alloc(unsigned int nrpools, gfp_t gfp)
 	refcount_set(&cl->cl_ref, 1);
 	INIT_HLIST_NODE(&cl->cl_hash);
 	for (i = 0; i < nrpools; i++) {
+		lwq_init(&cl->cl_pool[i].cp_ctrl);
 		lwq_init(&cl->cl_pool[i].cp_xprts);
 		cl->cl_pool[i].cp_client = cl;
 	}
net/sunrpc/svc_xprt.c +50 -10

113 lines not on this page · raw message

Signature · 1 line
2.53.0
#6 lore raw

RFC PATCH v2 6/9 SUNRPC: keep the single transport queue while no classifier is attached

In reply to Benjamin Coddington

From: Benjamin Coddington <[email protected]>

With no classifier attached every transport is on the anonymous client and a pool's client queue holds one entry, so dispatch is in today's order but takes one more lwq dequeue and a re-queue per request than today's code. That is about half a microsecond a dispatch on current hardware, paid by every server whether or not it has asked for anything.

Keep the pool's flat transport queue, sp_xprts, and use it while no classifier is attached anywhere, switched by a static key that attaching a classifier raises and detaching it drops. The dispatcher looks first at whichever queue is not in use, so transports queued before a switch are served promptly after it and nothing is lost in either direction; the thread's sleep test covers both queues, and network namespace teardown cleans both. With the key off the enqueue and dequeue paths are today's, plus one empty test of the client queue at dequeue.

---
 include/linux/sunrpc/svc.h      |  1 +
 include/linux/sunrpc/svc_xprt.h |  4 +++
 net/sunrpc/svc.c                |  1 +
 net/sunrpc/svc_classify.c       | 20 ++++++++++----
 net/sunrpc/svc_xprt.c           | 46 ++++++++++++++++++++++++++++++---
 5 files changed, 63 insertions(+), 9 deletions(-)
include/linux/sunrpc/svc.h +1 -0
index 02a8eb5d3357..49c1cebe5702 100644
@@ -38,6 +38,7 @@ struct svc_pool {
 	unsigned int		sp_nrthreads;	/* # of threads currently running in pool */
 	unsigned int		sp_nrthrmin;	/* Min number of threads to run per pool */
 	unsigned int		sp_nrthrmax;	/* Max requested number of threads in pool */
+	struct lwq		sp_xprts;	/* ready transports, no classifier */
 	struct lwq		sp_clients;	/* clients with ready transports */
 	struct list_head	sp_all_threads;	/* all server threads */
 	struct llist_head	sp_idle_threads; /* idle server threads */
include/linux/sunrpc/svc_xprt.h +4 -0
index e05d7c8610ff..ef21b2e57b98 100644
@@ -8,6 +8,7 @@
 #ifndef SUNRPC_SVC_XPRT_H
 #define SUNRPC_SVC_XPRT_H
 
+#include <linux/jump_label.h>
 #include <linux/sunrpc/svc.h>
 
 struct module;
@@ -182,6 +183,9 @@ void	svc_xprt_destroy_all(struct svc_serv *serv, struct net *net,
 			     bool unregister);
 void	svc_xprt_received(struct svc_xprt *xprt);
 
+/* Set while a classifier is attached anywhere: dispatch by client. */
+DECLARE_STATIC_KEY_FALSE(svc_classify_key);
+
 /* The class of an accepted transport; SVC_CLASS_* in svc.h. */
 #if IS_ENABLED(CONFIG_SUNRPC_BPF_CLASSIFY)
 u32	svc_classify(const struct svc_xprt *xprt);
net/sunrpc/svc.c +1 -0
index 92c6737e5905..5f6a0fc06e66 100644
@@ -478,6 +478,7 @@ __svc_create(struct svc_program *prog, int nprogs, struct svc_stat *stats,
 				i, serv->sv_name);
 
 		pool->sp_id = i;
+		lwq_init(&pool->sp_xprts);
 		lwq_init(&pool->sp_clients);
 		INIT_LIST_HEAD(&pool->sp_all_threads);
 		init_llist_head(&pool->sp_idle_threads);
net/sunrpc/svc_classify.c +15 -5
index aa63e0a3cb95..8d93d63157f5 100644
@@ -36,16 +36,17 @@ u32 svc_classify(const struct svc_xprt *xprt)
 	return class;
 }
 
-/* Caller holds svc_classifier_lock. */
-static void svc_classifier_detach(struct svc_classifier *c)
+/* Caller holds svc_classifier_lock; the static key is dropped after. */
+static bool svc_classifier_detach(struct svc_classifier *c)
 {
 	struct sunrpc_net *sn;
 
 	if (!c->net)
-		return;
+		return false;
 	sn = net_generic(c->net, sunrpc_net_id);
 	RCU_INIT_POINTER(sn->svc_classifier, NULL);
 	c->net = NULL;
+	return true;
 }
 
 /**
@@ -58,13 +59,16 @@ void svc_classifier_exit_net(struct net *net)
 {
 	struct sunrpc_net *sn = net_generic(net, sunrpc_net_id);
 	struct svc_classifier *c;
+	bool detached = false;
 
 	spin_lock(&svc_classifier_lock);
 	c = rcu_dereference_protected(sn->svc_classifier,
 				      lockdep_is_held(&svc_classifier_lock));
 	if (c)
-		svc_classifier_detach(c);
+		detached = svc_classifier_detach(c);
 	spin_unlock(&svc_classifier_lock);
+	if (detached)
+		static_branch_dec(&svc_classify_key);
 }
 
 static int svc_classifier_reg(void *kdata, struct bpf_link *link)
@@ -86,14 +90,20 @@ static int svc_classifier_reg(void *kdata, struct bpf_link *link)
 		rcu_assign_pointer(sn->svc_classifier, c);
 	}
 	spin_unlock(&svc_classifier_lock);
+	if (!err)
+		static_branch_inc(&svc_classify_key);
 	return err;
 }
 
 static void svc_classifier_unreg(void *kdata, struct bpf_link *link)
 {
+	bool detached;
+
 	spin_lock(&svc_classifier_lock);
-	svc_classifier_detach(kdata);
+	detached = svc_classifier_detach(kdata);
 	spin_unlock(&svc_classifier_lock);
+	if (detached)
+		static_branch_dec(&svc_classify_key);
 	synchronize_rcu();
 }
 
net/sunrpc/svc_xprt.c +42 -4

91 lines not on this page · raw message

Signature · 1 line
2.53.0
#7 lore raw

RFC PATCH v2 7/9 selftests/bpf: add svc_classifier tests

In reply to Benjamin Coddington

From: Benjamin Coddington <[email protected]>

A classifier for RPC service transports that looks the peer address up in an LPM trie keyed by {family, address}, the reference for the sunrpc svc_classifier struct_ops, and tests of the attach model: a second attach in the same network namespace fails with -EBUSY, a link update replaces the classifier, a detached instance can be attached again, a classifier attached inside a namespace does not occupy the initial one, and a namespace that exits leaves its link to detach into nothing. Skipped where the struct_ops is not registered.

---
 tools/testing/selftests/bpf/config            |   2 +
 .../selftests/bpf/prog_tests/svc_classifier.c | 104 ++++++++++++++++++
 .../selftests/bpf/progs/bpf_svc_classifier.c  |  80 ++++++++++++++
 3 files changed, 186 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/svc_classifier.c
 create mode 100644 tools/testing/selftests/bpf/progs/bpf_svc_classifier.c
tools/testing/selftests/bpf/config +2 -0
index ea7044f30adc..2cb7080d7da2 100644
@@ -114,6 +114,7 @@ CONFIG_IP_NF_IPTABLES=y
 CONFIG_IP6_NF_IPTABLES=y
 CONFIG_IP6_NF_FILTER=y
 CONFIG_NF_NAT=y
+CONFIG_NFS_FS=y
 CONFIG_PACKET=y
 CONFIG_RC_CORE=y
 CONFIG_SAMPLES=y
@@ -133,6 +134,7 @@ CONFIG_TCP_CONG_BBR=y
 CONFIG_INFINIBAND=y
 CONFIG_SMC=y
 CONFIG_SMC_HS_CTRL_BPF=y
+CONFIG_SUNRPC_BPF_CLASSIFY=y
 CONFIG_DIBS=y
 CONFIG_DIBS_LO=y
 CONFIG_PM_WAKELOCKS=y
tools/testing/selftests/bpf/prog_tests/svc_classifier.c new file +104 -0

104 lines not on this page · raw message

tools/testing/selftests/bpf/progs/bpf_svc_classifier.c new file +80 -0

80 lines not on this page · raw message

Signature · 1 line
2.53.0
#8 lore raw

RFC PATCH v2 8/9 tools/net/sunrpc: add svc-classify, a prefix classifier and its loader

In reply to Benjamin Coddington

From: Benjamin Coddington <[email protected]>

An administrator who wants RPC service transports dispatched per client needs a classifier attached in the service's network namespace and a way to fill its map. Add svc-classify: the prefix classifier from the BPF selftests, with its own type definitions in place of vmlinux.h, embedded in a loader as a skeleton so the binary stands alone. load attaches it and pins the link and the map, replace swaps the program and keeps the map, unload detaches; add, del and list manage the map with address prefixes and class words written as N, N+addr or addr; status shows the link. The Makefile builds the in-tree libbpf and bpftool it needs.

---
 tools/net/sunrpc/svc-classify/.gitignore      |   1 +
 tools/net/sunrpc/svc-classify/Makefile        |  65 ++++
 tools/net/sunrpc/svc-classify/svc-classify.c  | 357 ++++++++++++++++++
 .../sunrpc/svc-classify/svc_classify.bpf.c    |  83 ++++
 4 files changed, 506 insertions(+)
 create mode 100644 tools/net/sunrpc/svc-classify/.gitignore
 create mode 100644 tools/net/sunrpc/svc-classify/Makefile
 create mode 100644 tools/net/sunrpc/svc-classify/svc-classify.c
 create mode 100644 tools/net/sunrpc/svc-classify/svc_classify.bpf.c
tools/net/sunrpc/svc-classify/.gitignore new file +1 -0
new file mode 100644
index 000000000000..567609b1234a
@@ -0,0 +1 @@
+build/
tools/net/sunrpc/svc-classify/Makefile new file +65 -0

65 lines not on this page · raw message

tools/net/sunrpc/svc-classify/svc-classify.c new file +357 -0

357 lines not on this page · raw message

tools/net/sunrpc/svc-classify/svc_classify.bpf.c new file +83 -0

83 lines not on this page · raw message

Signature · 1 line
2.53.0
#11 lore raw

RFC PATCH v2 9/9 Documentation: describe RPC server transport classes and the BPF classifier

In reply to Benjamin Coddington

From: Benjamin Coddington <[email protected]>

Dispatch round-robin across clients changes how a service shares its threads, and the classes come from a BPF program the administrator loads, so the administrator needs the model, the class word, and the procedure. Add a page with those: install a classifier with svc-classify (build, attach, fill the map, check, change, remove, keep across reboots), six class maps and the share each gives its clients, the tracepoint that shows the class, and how the hook, the program and the attach model work underneath.

---
 Documentation/filesystems/nfs/index.rst       |   1 +
 .../filesystems/nfs/rpc-server-clients.rst    | 394 ++++++++++++++++++
 2 files changed, 395 insertions(+)
 create mode 100644 Documentation/filesystems/nfs/rpc-server-clients.rst
Documentation/filesystems/nfs/index.rst +1 -0
index a29a212b5b4d..57a61ce0533d 100644
@@ -16,3 +16,4 @@ NFS
    nfsd-io-modes
    knfsd-stats
    reexport
+   rpc-server-clients
Documentation/filesystems/nfs/rpc-server-clients.rst new file +394 -0

394 lines not on this page · raw message

Signature · 1 line
2.53.0
maintainer
#14 lore raw
In reply to Benjamin Coddington
Quoted 5 lines · Benjamin Coddington

This is v2 of [3], with the change I floated in that thread: the kernel no longer decides which transports belong together. By default nothing changes. An administrator who wants peers grouped loads a small BPF program that returns a class for each accepted connection, and the pool takes turns across classes.

As a quick response to this posting: I'd like to hear opinions from others on this approach rather than just fetching it straight into nfsd-testing.

Having patches to mull over is great. Thanks Ben!

Signature · 1 line
Chuck Lever (Come to NFS bake-a-thon! https://nfsv4bat.org)
#15 lore raw
Quoted 11 lines · Chuck Lever
Quoted 5 lines · Benjamin Coddington

This is v2 of [3], with the change I floated in that thread: the kernel no longer decides which transports belong together. By default nothing changes. An administrator who wants peers grouped loads a small BPF program that returns a class for each accepted connection, and the pool takes turns across classes.

As a quick response to this posting: I'd like to hear opinions from others on this approach rather than just fetching it straight into nfsd-testing.

Having patches to mull over is great. Thanks Ben!

No problem, thanks for all the feedback and discussion so far. One point of trouble on the bpf-list side is their CI bot can't apply these patches because there are conflicts from the nfsd-testing base. It might be that the bpf folks won't even bother to review if their CI can't build it..

If this goes further, I'll rebase a next version onto something the bpf-list tooling can handle, then back to nfsd-testing if it looks like its mergeable.

Ben

maintainer
#9 lore raw

Re: [PATCH RFC v2 8/9] tools/net/sunrpc: add svc-classify, a prefix classifier and its loader

In reply to Benjamin Coddington
Quoted 545 lines · Benjamin Coddington

From: Benjamin Coddington <[email protected]>

An administrator who wants RPC service transports dispatched per client needs a classifier attached in the service's network namespace and a way to fill its map. Add svc-classify: the prefix classifier from the BPF selftests, with its own type definitions in place of vmlinux.h, embedded in a loader as a skeleton so the binary stands alone. load attaches it and pins the link and the map, replace swaps the program and keeps the map, unload detaches; add, del and list manage the map with address prefixes and class words written as N, N+addr or addr; status shows the link. The Makefile builds the in-tree libbpf and bpftool it needs.

---
 tools/net/sunrpc/svc-classify/.gitignore      |   1 +
 tools/net/sunrpc/svc-classify/Makefile        |  65 ++++
 tools/net/sunrpc/svc-classify/svc-classify.c  | 357 ++++++++++++++++++
 .../sunrpc/svc-classify/svc_classify.bpf.c    |  83 ++++
 4 files changed, 506 insertions(+)
 create mode 100644 tools/net/sunrpc/svc-classify/.gitignore
 create mode 100644 tools/net/sunrpc/svc-classify/Makefile
 create mode 100644 tools/net/sunrpc/svc-classify/svc-classify.c
 create mode 100644 tools/net/sunrpc/svc-classify/svc_classify.bpf.c
tools/net/sunrpc/svc-classify/.gitignore new file +1 -0
new file mode 100644
index 000000000000..567609b1234a
@@ -0,0 +1 @@
+build/
tools/net/sunrpc/svc-classify/Makefile new file +65 -0
new file mode 100644
index 000000000000..9d86bc88417b
@@ -0,0 +1,65 @@
+# SPDX-License-Identifier: GPL-2.0
+OUTPUT ?= $(CURDIR)/build/
+override OUTPUT := $(abspath $(OUTPUT))/
+$(shell mkdir -p $(OUTPUT))
+
+include ../../../build/Build.include
+include ../../../scripts/Makefile.arch
+include ../../../scripts/Makefile.include
+
+TOOLSDIR := $(abspath ../../..)
+BPFDIR := $(TOOLSDIR)/lib/bpf
+BPFTOOLDIR := $(TOOLSDIR)/bpf/bpftool
+APIDIR := $(TOOLSDIR)/include/uapi
+INCLUDE_DIR := $(OUTPUT)include
+BPFOBJ := $(OUTPUT)libbpf/libbpf.a
+DEFAULT_BPFTOOL := $(OUTPUT)sbin/bpftool
+BPFTOOL ?= $(DEFAULT_BPFTOOL)
+CLANG ?= clang
+msg = $(if $(Q),@printf '  %-8s %s\n' "$(1)" "$(3)";)
+
+prefix ?= /usr/local
+sbindir ?= $(prefix)/sbin
+INSTALL ?= install
+
+CFLAGS += -g -O2 -Wall -I$(INCLUDE_DIR) -I$(OUTPUT)
+LDLIBS += -lelf -lz
+BPF_CFLAGS := -g -O2 -target bpf -D__TARGET_ARCH_$(SRCARCH) \
+	      -I$(INCLUDE_DIR) -I$(APIDIR) -Wall
+
+all: $(OUTPUT)svc-classify
+
+$(OUTPUT)libbpf $(OUTPUT)bpftool:
+	$(Q)mkdir -p $@
+
+$(BPFOBJ): $(wildcard $(BPFDIR)/*.[ch] $(BPFDIR)/Makefile) | $(OUTPUT)libbpf
+	$(Q)$(MAKE) $(submake_extras) -C $(BPFDIR) OUTPUT=$(OUTPUT)libbpf/ \
+		    DESTDIR=$(OUTPUT) prefix= all install_headers
+
+$(DEFAULT_BPFTOOL): $(wildcard $(BPFTOOLDIR)/*.[ch] $(BPFTOOLDIR)/Makefile) \
+		    $(BPFOBJ) | $(OUTPUT)bpftool
+	$(Q)$(MAKE) $(submake_extras) -C $(BPFTOOLDIR) OUTPUT=$(OUTPUT)bpftool/ \
+		    LIBBPF_OUTPUT=$(OUTPUT)libbpf/ LIBBPF_DESTDIR=$(OUTPUT) \
+		    prefix= DESTDIR=$(OUTPUT) install-bin
+
+$(OUTPUT)svc_classify.bpf.o: svc_classify.bpf.c $(BPFOBJ)
+	$(call msg,CLNG-BPF,,$(notdir $@))
+	$(Q)$(CLANG) $(BPF_CFLAGS) -c $< -o $@
+
+$(OUTPUT)svc_classify.skel.h: $(OUTPUT)svc_classify.bpf.o $(BPFTOOL)
+	$(call msg,GEN-SKEL,,$(notdir $@))
+	$(Q)$(BPFTOOL) gen skeleton $< name svc_classify > $@
+
+$(OUTPUT)svc-classify: svc-classify.c $(OUTPUT)svc_classify.skel.h $(BPFOBJ)
+	$(call msg,CC,,$(notdir $@))
+	$(Q)$(CC) $(CFLAGS) -o $@ $< $(BPFOBJ) $(LDLIBS)
+
+install: $(OUTPUT)svc-classify
+	$(Q)$(INSTALL) -d $(DESTDIR)$(sbindir)
+	$(Q)$(INSTALL) -m 755 $(OUTPUT)svc-classify $(DESTDIR)$(sbindir)/
+
+clean:
+	$(Q)rm -rf $(OUTPUT)
+
+.PHONY: all install clean
+.DELETE_ON_ERROR:
tools/net/sunrpc/svc-classify/svc-classify.c new file +357 -0
new file mode 100644
index 000000000000..f722b891be2c
@@ -0,0 +1,357 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * svc-classify - attach the prefix classifier to the RPC services of this
+ * network namespace and manage its prefix map.
+ *
+ *   svc-classify [-p DIR] load             attach; pin the link and the map
+ *   svc-classify [-p DIR] replace          replace the program, keep the map
+ *   svc-classify [-p DIR] unload           detach
+ *   svc-classify [-p DIR] add PREFIX CLASS
+ *   svc-classify [-p DIR] del PREFIX
+ *   svc-classify [-p DIR] list
+ *   svc-classify [-p DIR] status
+ *
+ * PREFIX is a.b.c.d[/len], x:y::z[/len], any, any4 or any6.  CLASS is N
+ * (one client for every peer that matches), N+addr (one client per peer
+ * address within class N), addr (the same as 0+addr) or 0 (the anonymous
+ * client).  DIR, where the link and the map are pinned, defaults to
+ * /sys/fs/bpf/svc_classify.
+ */
+#include <arpa/inet.h>
+#include <errno.h>
+#include <limits.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/stat.h>
+#include <unistd.h>
+
+#include <bpf/bpf.h>
+#include <bpf/libbpf.h>
+
+#include "svc_classify.skel.h"
+
+#define PIN_DIR_DEFAULT	"/sys/fs/bpf/svc_classify"
+#define SVC_CLASS_PER_ADDR	(1U << 31)
+
+struct prefix_key {
+	__u32 prefixlen;
+	__u8 family;
+	__u8 addr[16];
+};
+
+static const char *progname;
+static char pin_link[PATH_MAX], pin_map[PATH_MAX];
+static const char *pin_dir = PIN_DIR_DEFAULT;
+
+static void usage(void)
+{
+	fprintf(stderr,
+		"usage: %s [-p DIR] load | replace | unload | list | status |\n"
+		"       add PREFIX CLASS | del PREFIX\n"
+		"PREFIX: a.b.c.d[/len], x:y::z[/len], any, any4, any6\n"
+		"CLASS:  N, N+addr, addr, 0\n", progname);
+	exit(2);
+}
+
+static int do_load(void)
+{
+	struct svc_classify *skel;
+	struct bpf_link *link;
+	int err;
+
+	if (access(pin_link, F_OK) == 0) {
+		fprintf(stderr, "%s: already loaded (%s exists)\n", progname,
+			pin_link);
+		return 1;
+	}
+	mkdir(pin_dir, 0700);
+
+	skel = svc_classify__open_and_load();
+	if (!skel) {
+		fprintf(stderr, "%s: load: %s\n", progname, strerror(errno));
+		rmdir(pin_dir);
+		return 1;
+	}
+	link = bpf_map__attach_struct_ops(skel->maps.prefix);
+	if (!link) {
+		fprintf(stderr, "%s: attach: %s\n", progname, strerror(errno));
+		rmdir(pin_dir);
+		return 1;
+	}
+	err = bpf_link__pin(link, pin_link);
+	if (!err)
+		err = bpf_map__pin(skel->maps.prefixes, pin_map);
+	if (err) {
+		fprintf(stderr, "%s: pin: %s\n", progname, strerror(-err));
+		bpf_link__destroy(link);
+		unlink(pin_link);
+		rmdir(pin_dir);
+		return 1;
+	}
+	return 0;
+}
+
+static int do_replace(void)
+{
+	struct svc_classify *skel;
+	struct bpf_link *link;
+	int link_fd, err;
+
+	link_fd = bpf_obj_get(pin_link);
+	if (link_fd < 0) {
+		fprintf(stderr, "%s: not loaded (%s)\n", progname,
+			strerror(errno));
+		return 1;
+	}
+	skel = svc_classify__open();
+	if (!skel) {
+		fprintf(stderr, "%s: open: %s\n", progname, strerror(errno));
+		return 1;
+	}
+	/*
+	 * Reuse the pinned map so the entries survive.  A build whose map
+	 * definition differs fails here; unload and load then.
+	 */
+	err = bpf_map__set_pin_path(skel->maps.prefixes, pin_map);
+	if (!err)
+		err = svc_classify__load(skel);
+	if (err) {
+		fprintf(stderr, "%s: load: %s\n", progname, strerror(-err));
+		return 1;
+	}
+	/*
+	 * bpf_link__update_map() wants the link that attached the map, not
+	 * one reopened from a pin.  bpf_map__attach_struct_ops() writes the
+	 * program into the new map's value before it tries to create a link,
+	 * and the link is refused with EBUSY while the old one is attached;
+	 * the written map is what the link update needs.
+	 */
+	link = bpf_map__attach_struct_ops(skel->maps.prefix);
+	if (link) {
+		bpf_link__destroy(link);
+		fprintf(stderr, "%s: nothing attached in this namespace; unload and load\n",
+			progname);
+		return 1;
+	}
+	if (errno != EBUSY) {
+		fprintf(stderr, "%s: attach: %s\n", progname, strerror(errno));
+		return 1;
+	}
+	err = bpf_link_update(link_fd, bpf_map__fd(skel->maps.prefix), NULL);
+	if (err) {
+		fprintf(stderr, "%s: replace: %s\n", progname, strerror(errno));
+		return 1;
+	}
+	return 0;
+}
+
+static int do_unload(void)
+{
+	int ret = 0;
+
+	if (unlink(pin_link) && errno != ENOENT) {
+		perror(pin_link);
+		ret = 1;
+	}
+	if (unlink(pin_map) && errno != ENOENT) {
+		perror(pin_map);
+		ret = 1;
+	}
+	rmdir(pin_dir);
+	return ret;
+}
+
+static int parse_prefix(const char *s, struct prefix_key *key)
+{
+	char buf[64], *slash;
+	int bits, max;
+
+	memset(key, 0, sizeof(*key));
+	if (!strcmp(s, "any"))
+		return 0;
+	if (!strcmp(s, "any4")) {
+		key->family = AF_INET;
+		key->prefixlen = 8;
+		return 0;
+	}
+	if (!strcmp(s, "any6")) {
+		key->family = AF_INET6;
+		key->prefixlen = 8;
+		return 0;
+	}
+	if (strlen(s) >= sizeof(buf))
+		return -1;
+	strcpy(buf, s);
+	slash = strchr(buf, '/');
+	if (slash)
+		*slash++ = '\0';
+	if (inet_pton(AF_INET, buf, key->addr) == 1) {
+		key->family = AF_INET;
+		max = 32;
+	} else if (inet_pton(AF_INET6, buf, key->addr) == 1) {
+		key->family = AF_INET6;
+		max = 128;
+	} else {
+		return -1;
+	}
+	bits = max;
+	if (slash) {
+		char *end;
+
+		if (*slash < '0' || *slash > '9')
+			return -1;
+		bits = strtoul(slash, &end, 10);
+		if (*end || bits > max)
+			return -1;
+	}
+	key->prefixlen = 8 + bits;
+	return 0;
+}
+
+static int parse_class(const char *s, __u32 *class)
+{
+	unsigned long n;
+	char *end;
+
+	if (!strcmp(s, "addr")) {
+		*class = SVC_CLASS_PER_ADDR;
+		return 0;
+	}
+	n = strtoul(s, &end, 0);
+	if (end == s || n >= SVC_CLASS_PER_ADDR)
+		return -1;
+	*class = n;
+	if (!strcmp(end, "+addr"))
+		*class |= SVC_CLASS_PER_ADDR;
+	else if (*end)
+		return -1;
+	return 0;
+}
+
+static void print_entry(const struct prefix_key *key, __u32 class)
+{
+	char addr[INET6_ADDRSTRLEN];
+
+	if (key->prefixlen == 0)
+		printf("any");
+	else if (key->prefixlen == 8)
+		printf("any%d", key->family == AF_INET ? 4 : 6);
+	else
+		printf("%s/%u", inet_ntop(key->family, key->addr, addr,
+					  sizeof(addr)), key->prefixlen - 8);
+	printf(" -> %u%s\n", class & ~SVC_CLASS_PER_ADDR,
+	       class & SVC_CLASS_PER_ADDR ? "+addr" : "");
+}
+
+static int open_map(void)
+{
+	int fd = bpf_obj_get(pin_map);
+
+	if (fd < 0) {
+		fprintf(stderr, "%s: not loaded (%s: %s)\n", progname, pin_map,
+			strerror(errno));
+		exit(1);
+	}
+	return fd;
+}
+
+static int do_add(const char *prefix, const char *cls)
+{
+	struct prefix_key key;
+	__u32 class;
+
+	if (parse_prefix(prefix, &key) || parse_class(cls, &class))
+		usage();
+	if (bpf_map_update_elem(open_map(), &key, &class, BPF_ANY)) {
+		fprintf(stderr, "%s: add: %s\n", progname, strerror(errno));
+		return 1;
+	}
+	return 0;
+}
+
+static int do_del(const char *prefix)
+{
+	struct prefix_key key;
+
+	if (parse_prefix(prefix, &key))
+		usage();
+	if (bpf_map_delete_elem(open_map(), &key)) {
+		fprintf(stderr, "%s: del: %s\n", progname, strerror(errno));
+		return 1;
+	}
+	return 0;
+}
+
+static int do_list(void)
+{
+	struct prefix_key key, next;
+	__u32 class;
+	int fd;
+
+	fd = open_map();
+	if (bpf_map_get_next_key(fd, NULL, &next))
+		return 0;
+	do {
+		key = next;
+		if (!bpf_map_lookup_elem(fd, &key, &class))
+			print_entry(&key, class);
+	} while (!bpf_map_get_next_key(fd, &key, &next));
+	return 0;
+}
+
+static int do_status(void)
+{
+	struct bpf_link_info info = {};
+	__u32 len = sizeof(info);
+	int fd;
+
+	fd = bpf_obj_get(pin_link);
+	if (fd < 0) {
+		printf("not loaded\n");
+		return 1;
+	}
+	if (bpf_link_get_info_by_fd(fd, &info, &len))
+		printf("loaded\n");
+	else
+		printf("loaded, link id %u, struct_ops map id %u\n", info.id,
+		       info.struct_ops.map_id);
+	return 0;
+}
+
+int main(int argc, char **argv)
+{
+	const char *cmd;
+	int opt;
+
+	progname = argv[0];
+	while ((opt = getopt(argc, argv, "p:")) != -1) {
+		if (opt != 'p')
+			usage();
+		pin_dir = optarg;
+	}
+	argc -= optind;
+	argv += optind;
+	if (argc < 1)
+		usage();
+	cmd = argv[0];
+	snprintf(pin_link, sizeof(pin_link), "%s/link", pin_dir);
+	snprintf(pin_map, sizeof(pin_map), "%s/prefixes", pin_dir);
+
+	if (!strcmp(cmd, "load") && argc == 1)
+		return do_load();
+	if (!strcmp(cmd, "replace") && argc == 1)
+		return do_replace();
+	if (!strcmp(cmd, "unload") && argc == 1)
+		return do_unload();
+	if (!strcmp(cmd, "add") && argc == 3)
+		return do_add(argv[1], argv[2]);
+	if (!strcmp(cmd, "del") && argc == 2)
+		return do_del(argv[1]);
+	if (!strcmp(cmd, "list") && argc == 1)
+		return do_list();
+	if (!strcmp(cmd, "status") && argc == 1)
+		return do_status();
+	usage();
+	return 2;
+}
tools/net/sunrpc/svc-classify/svc_classify.bpf.c new file +73 -0
new file mode 100644
index 000000000000..3d012f102ec3
@@ -0,0 +1,83 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * A svc_classifier for RPC service transports: the peer address is looked
+ * up in an LPM trie keyed by {family, address}; a hit returns the class
+ * word stored there, a miss returns 0, the anonymous client.  A key of
+ * prefixlen 0 is the default for every family, 8 the default for one.
+ */
+#include <linux/types.h>
+#include <linux/bpf.h>
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+#include <bpf/bpf_core_read.h>
+
+char _license[] SEC("license") = "GPL";
+
+#define AF_INET			2
+#define AF_INET6		10
+
+#define SVC_CLASSIFIER_NAME_MAX	16
+
+struct sockaddr_storage___local {
+	unsigned short ss_family;
+	char __data[126];
+};
+
+struct svc_xprt___local {
+	struct sockaddr_storage___local xpt_remote;
+} __attribute__((preserve_access_index));
+
+struct svc_classifier___local {
+	char name[SVC_CLASSIFIER_NAME_MAX];
+	__u32 (*classify)(const struct svc_xprt___local *xprt);
+};
+
+struct prefix_key {
+	__u32 prefixlen;	/* 8 (the family byte) + address bits */
+	__u8 family;
+	__u8 addr[16];
+};
+
+struct {
+	__uint(type, BPF_MAP_TYPE_LPM_TRIE);
+	__type(key, struct prefix_key);
+	__type(value, __u32);
+	__uint(max_entries, 1024);
+	__uint(map_flags, BPF_F_NO_PREALLOC);
+} prefixes SEC(".maps");
+
+SEC("struct_ops/classify")
+__u32 BPF_PROG(prefix_classify, const struct svc_xprt___local *xprt)
+{
+	struct sockaddr_storage___local ss;
+	struct prefix_key key = {};
+	__u32 *class;
+
+	if (bpf_core_read(&ss, sizeof(ss), &xprt->xpt_remote))
+		return 0;
+
+	key.family = ss.ss_family;
+	switch (ss.ss_family) {
+	case AF_INET:
+		/* struct sockaddr_in: port at 2, address at 4 */
+		__builtin_memcpy(key.addr, &ss.__data[2], 4);
+		key.prefixlen = 8 + 32;
+		break;
+	case AF_INET6:
+		/* struct sockaddr_in6: port at 2, flowinfo at 4, address at 8 */
+		__builtin_memcpy(key.addr, &ss.__data[6], 16);
+		key.prefixlen = 8 + 128;
+		break;
+	default:
+		return 0;
+	}

tools/net/sunrpc/svc-classify/svc_classify.bpf.c line 83

Quoted 10 lines
tools/net/sunrpc/svc-classify/svc_classify.bpf.c new file +10 -0
+
+	class = bpf_map_lookup_elem(&prefixes, &key);
+	return class ? *class : 0;
+}
+
+SEC(".struct_ops.link")
+struct svc_classifier___local prefix = {
+	.name		= "prefix",
+	.classify	= (void *)prefix_classify,
+};

The userland programs in the kernel are often not packaged, so it seems to me that svc-classify would be better integrated into nfs-utils. That would allow you to more easily bundle it with the systemd service file.

It might even make sense to integrate this functionality into nfsdctl, but I guess the classifier affects all rpc_serv objects as well (e.g. NLM), so it may be best to keep it as its own thing.

Signature · 1 line
Jeff Layton <[email protected]>
maintainer
#12 lore raw

Re: [PATCH RFC v2 9/9] Documentation: describe RPC server transport classes and the BPF classifier

In reply to Benjamin Coddington
Quoted 119 lines · Benjamin Coddington

From: Benjamin Coddington <[email protected]>

Dispatch round-robin across clients changes how a service shares its threads, and the classes come from a BPF program the administrator loads, so the administrator needs the model, the class word, and the procedure. Add a page with those: install a classifier with svc-classify (build, attach, fill the map, check, change, remove, keep across reboots), six class maps and the share each gives its clients, the tracepoint that shows the class, and how the hook, the program and the attach model work underneath.

---
 Documentation/filesystems/nfs/index.rst       |   1 +
 .../filesystems/nfs/rpc-server-clients.rst    | 394 ++++++++++++++++++
 2 files changed, 395 insertions(+)
 create mode 100644 Documentation/filesystems/nfs/rpc-server-clients.rst
Documentation/filesystems/nfs/index.rst +1 -0
index a29a212b5b4d..57a61ce0533d 100644
@@ -16,3 +16,4 @@ NFS
    nfsd-io-modes
    knfsd-stats
    reexport
+   rpc-server-clients
Documentation/filesystems/nfs/rpc-server-clients.rst new file +86 -0
new file mode 100644
index 000000000000..b08351a88314
@@ -0,0 +1,394 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+======================================================
+RPC server: per-client dispatch and transport classes
+======================================================
+
+An RPC service thread pool serves its ready transports in FIFO order,
+one request per turn.  A peer with many connections (``nconnect``, a
+deep NFSv4.1 slot table, a data mover with a thread per file) takes a
+turn per connection, and a peer with one connection waits behind all
+of them.
+
+With a classifier attached, the pool instead serves its *clients* round
+robin: each client with work queued gets one request per round, however
+many connections it holds.  Which transports form a client is decided
+by a BPF program the administrator loads, once per accepted connection.
+With no program loaded nothing changes.
+
+This document describes the model, how to install a classifier with
+the ``svc-classify`` tool, what some class maps do, and how it works
+underneath.
+
+Clients, classes and turns
+==========================
+
+A *client* is a set of transports that share one turn.  Every service
+(nfsd, lockd, the NFSv4 callback service) keeps its own clients, found
+by *class* and network namespace, or by class, namespace and peer
+address.
+
+The classifier returns a 32-bit class word for each accepted transport:
+
+``0``
+    ``SVC_CLASS_NONE``: the transport stays on the service's *anonymous
+    client*.
+
+``N``, 1 to 2^31 - 1
+    one client for every transport of class ``N``.
+
+``N | SVC_CLASS_PER_ADDR``
+    one client per peer address within class ``N``.  ``N`` may be 0, so
+    ``SVC_CLASS_PER_ADDR`` alone means one client per peer address.
+
+``SVC_CLASS_PER_ADDR`` is bit 31.  ``svc-classify`` and this document
+write a class word with the bit set as ``N+addr``; ``svc-classify``
+also accepts ``addr`` for ``0+addr``.
+
+Dispatch rules:
+
+- A pool serves the clients that have transports queued round robin,
+  one request per client per round.
+- Within a client, transports are served in FIFO order, except that a
+  transport needing a connection accepted, closed, or a TLS handshake
+  run goes ahead of transports with data, though not two such turns in
+  a row while data waits.
+- A client's share of the pool does not grow with its connection count.
+  A client with one connection and a client with thirty get the same
+  number of turns while both have work queued.
+
+Things that follow from the rules and are easy to get wrong:
+
+- The anonymous client is one client.  Once any classifier is attached,
+  every transport whose class is ``0`` shares a single turn with every
+  other such transport of that service.  Within that one client the old
+  FIFO order applies, so those peers share the turn in proportion to
+  their connection counts.  A map that classifies only the hosts it
+  cares about and leaves the rest at ``0`` gives "the rest" one turn
+  in total (see example 5 below).
+- Per-address clients ignore the port.  Only IPv4 and IPv6 peers can be
+  keyed by address; a transport whose peer is anything else is left on
+  the anonymous client.
+- Listeners and UDP sockets are never classified; UDP traffic is served
+  from the anonymous client.
+- A transport keeps the class it was given when accepted.  Changing the
+  map, or replacing or removing the classifier, affects connections
+  accepted afterwards.  A connection that must be reclassified has to
+  reconnect.
+- nfsd has one service per network namespace, so its anonymous client
+  is per namespace.  lockd and the NFSv4 callback service are one
+  service for all namespaces; their anonymous clients span namespaces.
+- With no classifier attached anywhere, the pool uses a single FIFO;
+  the per-request cost of the feature is then a static branch on
+  enqueue and one empty-queue test on dequeue.
+  ``CONFIG_SUNRPC_BPF_CLASSIFY`` (default y) builds the hook; without
+  it there is no classification.
+

Documentation/filesystems/nfs/rpc-server-clients.rst line 96

Quoted 10 lines
Documentation/filesystems/nfs/rpc-server-clients.rst new file +10 -0
+Installing a classifier
+=======================
+
+There is one tool to do it with: ``svc-classify``, in
+``tools/net/sunrpc/svc-classify`` of the kernel source.  It carries the
+classifier program inside the binary, attaches it, and manages the
+class map by address prefix.  ``bpftool`` is not needed, and its
+``struct_ops`` subcommands cannot be used in its place (see "How it
+works").
+

"There is one tool to do it with: ..."

The obviously LLM-written documentation gives me the Ick and I stopped reading. There is also a lot of info in this doc that is not particularly helpful, like "the things that follow from the rules and are easy to get wrong".

If you want humans to actually read this, then it probably needs to be written by a human. If you think the LLM slop is actually useful for other LLMs, then maybe put those bits at the end in a clearly- delineated section.

Quoted 298 lines
Documentation/filesystems/nfs/rpc-server-clients.rst new file +298 -0
+What you need
+-------------
+
+- A kernel with ``CONFIG_SUNRPC_BPF_CLASSIFY`` (default y) and BTF:
+  ``CONFIG_DEBUG_INFO_BTF``, and ``CONFIG_DEBUG_INFO_BTF_MODULES`` when
+  ``sunrpc`` is a module.
+- ``/sys/fs/bpf`` mounted (systemd mounts it).
+- To build the tool: ``clang``, and the ``libelf`` and ``zlib``
+  development files.  The tool builds the libbpf and bpftool it needs
+  from the kernel source tree.
+
+Build and install the tool
+--------------------------
+
+In the kernel source tree::
+
+    $ make -C tools/net/sunrpc/svc-classify
+    # make -C tools/net/sunrpc/svc-classify install
+
+The binary goes to ``/usr/local/sbin/svc-classify`` (``prefix=`` and
+``sbindir=`` override that).  At run time it needs only libelf and
+zlib.
+
+Attach it
+---------
+
+As root, in the network namespace the service runs in (on a host that
+is the initial namespace; in a container, ``nsenter`` or
+``ip netns exec`` into it first)::
+
+    # svc-classify load
+    # svc-classify status
+    loaded, link id 46, struct_ops map id 140
+
+The classifier is now attached to every RPC service in the namespace,
+with an empty map: every connection accepted from now on returns class
+``0`` and stays on the anonymous client, so nothing has changed yet.
+One classifier per namespace: a second ``load`` is refused.
+
+Install the class map
+---------------------
+
+Add one line per address prefix.  The longest matching prefix wins; a
+peer that matches nothing gets class ``0``::
+
+    # svc-classify add 192.0.2.0/24 1
+    # svc-classify add any addr
+    # svc-classify list
+    192.0.2.0/24 -> 1
+    any -> 0+addr
+
+``PREFIX`` is ``a.b.c.d[/len]``, ``x:y::z[/len]``, ``any`` (every
+address of either family), ``any4`` or ``any6``.  ``CLASS`` is ``N``
+(one client for every peer that matches), ``N+addr`` (one client per
+peer address, within class ``N``), ``addr`` (one client per peer
+address, the same as ``0+addr``), or ``0`` (the anonymous client).
+
+Entries apply to connections accepted after they are added; to
+reclassify existing connections, have the clients reconnect (restarting
+the service does that for all of them).  ``svc-classify del PREFIX``
+removes an entry.
+
+Check it
+--------
+
+``svc-classify status`` and ``svc-classify list`` show the link and the
+map.  The ``sunrpc:svc_xprt_dequeue`` tracepoint shows the class every
+transport is dispatched as (see "Observing"), which is the check that
+the map does what was meant.
+
+Change or remove it
+-------------------
+
+- ``svc-classify add`` and ``del`` change the map at any time.
+- ``svc-classify replace`` installs a new build of the tool's program
+  on the attached link and keeps the map, provided the new build's map
+  definition is unchanged; otherwise unload and load.
+- ``svc-classify unload`` detaches the classifier.  Connections accepted
+  afterwards are anonymous; when no classifier is attached in any
+  namespace, the service is back on its single FIFO.
+
+Across reboots
+--------------
+
+Nothing persists: the link and the map live in ``/sys/fs/bpf`` and are
+gone at boot.  Run the ``load`` and ``add`` commands before the service
+starts, for example from a unit ordered before ``nfs-server.service``::
+
+    [Unit]
+    Description=RPC service transport classifier
+    Before=nfs-server.service
+
+    [Service]
+    Type=oneshot
+    RemainAfterExit=yes
+    ExecStart=/usr/local/sbin/svc-classify load
+    ExecStart=/usr/local/sbin/svc-classify add 192.0.2.0/24 1
+    ExecStart=/usr/local/sbin/svc-classify add any addr
+    ExecStop=/usr/local/sbin/svc-classify unload
+
+    [Install]
+    WantedBy=nfs-server.service
+
+Loading after the service is up works too; it only misses the
+connections already accepted.
+
+Examples
+========
+
+Each example gives the ``svc-classify add`` lines, the hosts they are
+applied to, and the share of the pool each client gets while all of
+them have work queued.
+
+1. Every host its own client
+----------------------------
+
+::
+
+    # svc-classify add any addr
+
+Three hosts: one with eight connections, one with one, one with two.
+Each host is a client, and each gets a third of the turns.  Without a
+classifier they would be served in proportion to their connections,
+8:1:2.
+
+2. A set of movers as one client, everyone else per host
+--------------------------------------------------------
+
+::
+
+    # svc-classify add 192.0.2.0/24 1
+    # svc-classify add any addr
+
+Three movers in ``192.0.2.0/24`` with four connections each, and two
+other hosts, one with one connection and one with two.  The movers are
+one client; each of the other hosts is a client of its own.  Each of
+the three clients gets a third of the pool; the movers share theirs by
+connection count, a ninth each.  Without a classifier the movers'
+twelve connections would take twelve fifteenths of the pool and the
+one-connection host one fifteenth.
+
+3. Two mover groups, one turn each
+----------------------------------
+
+::
+
+    # svc-classify add 192.0.2.0/25 1
+    # svc-classify add 192.0.2.128/25 2
+    # svc-classify add any addr
+
+Two hosts in ``192.0.2.0/25`` and two in ``192.0.2.128/25``, four
+connections each, and one other host with one connection.  Each group
+is a client and the other host is a client: three clients, a third
+each.  Within a group the two hosts split the group's third by
+connection count, a sixth each.
+
+4. Per host inside a campus, the rest of the world as one client
+----------------------------------------------------------------
+
+::
+
+    # svc-classify add 198.51.100.0/24 1+addr
+    # svc-classify add any 2
+
+Two hosts in ``198.51.100.0/24`` and two hosts outside it.  Each campus
+host is a client of its own (class 1, one client per address); the
+outside hosts together are one client (class 2).  Three clients, a
+third each; the outside hosts split their third by connection count.
+
+5. Leaving hosts unclassified
+-----------------------------
+
+::
+
+    # svc-classify add 203.0.113.0/24 0
+    # svc-classify add any addr
+
+Two hosts in ``203.0.113.0/24`` and two hosts elsewhere.  The two in
+``203.0.113.0/24`` return ``0`` and so share the anonymous client, one
+turn between them; the two elsewhere are a client each.  Three
+clients, a third each; the anonymous pair split their third by
+connection count.
+
+The common mistake is the map with only the movers in it::
+
+    # svc-classify add 192.0.2.0/24 1
+
+Three movers in ``192.0.2.0/24`` and two other hosts, one with one
+connection and one with two.  There are exactly two clients: the
+movers, and everyone else on the anonymous client.  The movers get
+half the pool.  The other half goes to the two other hosts in
+proportion to their connections, a sixth and a third of the pool.  Add
+``any addr`` to serve them per host.
+
+6. IPv6
+-------
+
+::
+
+    # svc-classify add 2001:db8:1::/48 1
+    # svc-classify add any addr
+
+Entries are per family; ``any`` covers both families, ``any6`` IPv6
+only.  Two hosts in ``2001:db8:1::/48`` and one host elsewhere: the two
+are one client, the other is a client of its own, half the pool each.
+
+Observing
+=========
+
+The ``sunrpc:svc_xprt_dequeue`` tracepoint reports the class of every
+transport as it is dispatched::
+
+    svc_xprt_dequeue: server=127.0.0.1:3049 client=127.0.1.1:38209 xpt_id=1164 flags=BUSY|DATA|TEMP|CACHE_AUTH|LOCAL|CONG_CTRL class=1 wakeup-us=30 qtime-us=4
+    svc_xprt_dequeue: server=127.0.0.1:3049 client=127.0.2.1:45965 xpt_id=1162 flags=BUSY|DATA|TEMP|CACHE_AUTH|LOCAL|CONG_CTRL class=0+addr wakeup-us=60 qtime-us=10
+    svc_xprt_dequeue: server=[::1]:3049 client=[2001:db8:1::1]:44401 xpt_id=1152 flags=BUSY|DATA|TEMP|CACHE_AUTH|LOCAL|CONG_CTRL class=1 wakeup-us=62 qtime-us=7
+    svc_xprt_dequeue: server=[::]:3049 client=(einval) xpt_id=2 flags=BUSY|CONN|CHNGBUF|LISTENER|CACHE_AUTH|CONG_CTRL|RPCB_UNREG class=0 wakeup-us=25 qtime-us=25
+
+``class=0`` is the anonymous client; ``+addr`` marks a per-address
+client; listeners show ``class=0`` and ``LISTENER`` in their flags.
+``bpftool link show`` lists the attached classifier's link, and
+``bpftool map dump pinned /sys/fs/bpf/svc_classify/prefixes`` the map
+with its raw keys.
+
+How it works
+============
+
+The classifier
+--------------
+
+.. kernel-doc:: include/linux/sunrpc/svc.h
+   :identifiers: svc_classifier
+
+The callback runs once per accepted transport, in process context,
+under ``rcu_read_lock()``; sleepable programs are refused at load.
+Only the base BPF helpers are available (map lookups, the usual).  The
+program is handed the ``struct svc_xprt`` and may read its fields
+directly:
+
+- ``xpt_remote`` and ``xpt_remotelen``: the peer address;
+- ``xpt_local``: the address the connection arrived on;
+- ``xpt_net``: the network namespace;
+- ``xpt_server->sv_name``: which service, ``"nfsd"``, ``"lockd"``,
+  ``"NFSv4 callback"``;
+- ``xpt_class->xcl_name``: the transport class, ``"tcp"``, ``"rdma"``.
+
+A classifier applies to every service in its namespace.  A program that
+wants to treat services differently reads ``xpt_server->sv_name``.
+
+The program ``svc-classify`` carries,
+``tools/net/sunrpc/svc-classify/svc_classify.bpf.c``, is the prefix
+classifier from the BPF selftests
+(``tools/testing/selftests/bpf/progs/bpf_svc_classifier.c``).  Its one
+map is an LPM trie keyed by ``{prefixlen, family, addr[16]}`` with
+``prefixlen`` counting the family byte plus the address bits, and the
+class word as the value; a miss returns ``0``.  Nothing else in the
+program is specific to this policy.  A classifier keyed on the local
+address, the service name, or anything else the ``svc_xprt`` shows is
+the same program with a different lookup, and ``svc-classify replace``
+installs a rebuilt one as long as its map definition is unchanged.
+
+Attaching
+---------
+
+The classifier is a ``struct_ops`` map.  Creating its link attaches the
+classifier to the network namespace of the task that creates the link:
+
+- one classifier per namespace; a second attach fails with ``-EBUSY``;
+- ``BPF_LINK_UPDATE`` on the link replaces the program;
+- closing or detaching the link, or unpinning its last reference,
+  removes the classifier;
+- a namespace that exits leaves its link attached to nothing;
+  closing it is harmless.
+
+``svc-classify load`` creates the link with
+``bpf_map__attach_struct_ops()`` and pins it, with the map, under
+``/sys/fs/bpf/svc_classify`` (``-p DIR`` chooses another directory,
+for a second namespace); ``replace`` is ``bpf_link__update_map()`` on
+the pinned link; ``unload`` unpins both.
+
+``bpftool struct_ops`` (``register``, ``dump``, ``unregister``) resolves
+the struct_ops type in the kernel's own BTF only, so when ``sunrpc`` is
+a module those subcommands do not find ``svc_classifier`` maps, and
+``register`` fails after creating the link.  Use ``svc-classify``.
+
+Limits
+======
+
+- A transport cannot move between clients; a client identity that is
+  only known after the connection is accepted (an NFSv4.1 client id,
+  say) cannot be used.  NFSv4.1 sessions over ``nconnect`` are one
+  client by peer address.
+- RDMA transports are classified like TCP, by the peer address of the
+  connection; UDP is not classified.
+- The cost with no classifier attached is a static branch on enqueue
+  and one empty-queue test on dequeue.  With one attached, dispatch
+  takes two or three more lock-free queue operations per request than
+  before: the client's queue, the pool's queue of clients, and a
+  requeue when the client has more.
Signature · 1 line
Jeff Layton <[email protected]>
maintainer
#16 lore raw
In reply to Benjamin Coddington
Quoted 180 lines · Benjamin Coddington

From: Benjamin Coddington <[email protected]>

This is v2 of [3], with the change I floated in that thread: the kernel no longer decides which transports belong together. By default nothing changes. An administrator who wants peers grouped loads a small BPF program that returns a class for each accepted connection, and the pool takes turns across classes.

What's different from v1
------------------------

The table I described on [3] became a hook. Instead of a kernel table of address prefixes, a netlink interface for it and an nfs-utils side, sunrpc registers a struct_ops with one callback: given the accepted svc_xprt, return a class. 0 means the anonymous client (shared with everything unclassified, today's order among them); any other number is a class every transport returning it shares; a flag bit on top means "and one client per peer address within that class". So "all movers are one class, everybody else per host" is two entries in a prefix map, and the posted v1 behaviour is one entry.

The hook runs once per accepted connection, in process context, after TCP and RDMA have both set the peer address. Nothing runs per request. No program loaded means every transport is on the
anonymous client, and with patch 6 the pool uses its old flat queue until a classifier attaches, so the dispatch path is today's code for anyone who hasn't asked for anything. That's the answer to Chuck's cost question from [3]: measured on bare metal the two-level queue cost about half a microsecond a dispatch (1.7% at 484k IOPS on one client's eight connections); with no program it now costs a static branch on enqueue and an empty-queue test on dequeue.

Attaching is a struct_ops link, bound to the attaching task's network namespace, one per namespace. A second attach gets -EBUSY, a link update replaces the program, closing the link removes it, and a namespace that exits leaves its link with nothing behind. Transports keep the class they got at accept; a new program only affects new connections. The reference program and the attach tests are in the BPF selftests (patch 7). svc-classify (patch 8) attaches the prefix classifier and manages its map by address prefix, so an administrator never touches bpftool. The page in patch 9, Documentation/filesystems/nfs/rpc-server-clients.rst, has the class word, the install procedure, and six class maps with the share each one gives its clients.

Still not in here: moving a transport between clients, so a v4.1 clientid key, which Chuck asked about on [3], is not in this version. A multi-homed host or an IPv6 host with a prefix's worth of addresses is handled by the prefix map instead.

Also from the [3] review: control events (accept, close, handshake) go ahead of data within a client (patch 5), and the dequeue tracepoint prints the class (patch 4).

Numbers
-------

The dispatch numbers are the v1 series' on a two-socket box (2 x Xeon 6542Y, 96 threads, no KASAN), from [3]. The dispatch code is the same
here: v2 puts the classifier in front of it, and the flat queue behind a static branch when nothing is attached. Loopback, NFSv3, 16 threads, 10 ms injected service time with 50% jitter. A is v7.2, B the series. v4.1 is within 2% in every cell.

One interactive client (bursts of 32) against one host with K backlogged connections, burst completion p50 in ms, floor 37:

    K             4      8     16     32
    A            70    195    351    671
    B            56     62     61     63

    N (K=16)      1      8     32     64     96    128
    A            21     98    350    690   1035   1365
    B            11     31     62    103    144    179

    M hosts x 4   1      2      4      8
    A            70    192    351    670
    B            57     81    121    192

    S: 2x8 vs K, share   4      8     16     32
    A                 41.0   20.0   11.1    5.9
    B                 50.3   50.0   50.0   50.0

A real NFSv3 client walking 500 files against a 16-connection
aggressor: 37.8 s on A, 2.5 s on B, 0.12 s alone on both.

Classes. The v1 kernel keyed on source address, so on [3] I emulated "all movers are one class" by giving six movers one address. With this series that's one map entry (the movers' prefix -> 1, everything else per address), and the six-address column is what "everything per address" gives. Six movers at 4 or 8 connections each against customers, customer share of dispatches:

                                 A           per address   movers one class
    one customer, 1 conn x 16    4.0 / 2.0   14.3 / 14.3   49.9 / 49.9
    one customer, 4 x 4         14.3 / 7.7   14.3 / 14.3   49.8 / 49.7
    four customers, 4 x 4 each  40 / 25      40 / 40       80 / 80

Burst of 32 against M busy hosts, p50 ms: hosts on their own addresses 57 81 121 192, as one class 56 62 62 62. A: 71 193 348 673 either way.

Cost. With a classifier attached, dispatch takes two or three more lock-free queue operations per request. On the same box that's about half a microsecond when the enqueue and the dequeue run on different cores, fio 4k O_DIRECT randread over loopback, 5 x 20s:

                                   A        B
    nconnect=1, 16 jobs        152.8k   152.2k   -0.4%
    nconnect=8, 16 jobs        483.8k   475.5k   -1.7%
    nconnect=8, 64 jobs        584.2k   581.0k   -0.5%

The serial walk and 4 jobs on one connection don't move. With no classifier attached the pool is on its old flat queue (patch 6). I only have my VM for that path so far (KASAN, 10 vCPUs, loopback, the same fio), and there the no-program kernel measures the same as v7.2 within the run-to-run spread: 16 jobs 94.9k vs 95.5k, 64 jobs 138.6k vs 139.0k. So the static branch adds nothing I can see, but that's a VM number.

Not tested: RDMA. The hook sits in the accept path TCP and RDMA share, but I could not bring a soft-RoCE listener up on the VM.

Daire - the same nine patches on v7.2 (plus the wake fix) are at
https://github.com/bcodding/linux branch nfsd-clientq-v2-7.2 if you want to run it; the loader builds from tools/net/sunrpc/svc-classify in that tree.

The series is on nfsd-testing at 56589cdb5881, which already has the svc_clean_up_xprts() wake fix from [3].

  patch 1  track clients by class, no change to dispatch
  patch 2  dispatch round-robin across clients
  patch 3  the struct_ops hook
  patch 4  class in the svc_xprt_dequeue tracepoint
  patch 5  control events ahead of data within a client
  patch 6  flat queue while no classifier is attached
  patch 7  selftests: the prefix program and the attach tests
  patch 8  tools/net/sunrpc/svc-classify: the prefix classifier and its

loader (load/replace/unload, add/del/list by prefix, status)
patch 9 Documentation: the model, the class word, installing a
classifier with svc-classify, six class maps and the share each gives, how it works underneath

[1] https://lore.kernel.org/linux-nfs/[email protected]
[2] https://lore.kernel.org/linux-nfs/[email protected]
[3] https://lore.kernel.org/linux-nfs/[email protected]

Benjamin Coddington (9):
SUNRPC: track service clients by class
SUNRPC: dispatch ready transports round-robin across clients
SUNRPC: add a BPF struct_ops hook to classify accepted transports
SUNRPC: report the transport's class in the svc_xprt_dequeue
tracepoint
SUNRPC: dispatch control events ahead of data within a client
SUNRPC: keep the single transport queue while no classifier is
attached
selftests/bpf: add svc_classifier tests
tools/net/sunrpc: add svc-classify, a prefix classifier and its loader
Documentation: describe RPC server transport classes and the BPF
classifier

 Documentation/filesystems/nfs/index.rst       |   1 +
 .../filesystems/nfs/rpc-server-clients.rst    | 394 ++++++++++++++++++
 include/linux/sunrpc/svc.h                    |  72 +++-
 include/linux/sunrpc/svc_xprt.h               |  30 ++
 include/trace/events/sunrpc.h                 |  11 +-
 net/sunrpc/Kconfig                            |  12 +
 net/sunrpc/Makefile                           |   1 +
 net/sunrpc/netns.h                            |   4 +
 net/sunrpc/sunrpc_syms.c                      |   4 +
 net/sunrpc/svc.c                              |  36 ++
 net/sunrpc/svc_classify.c                     | 210 ++++++++++
 net/sunrpc/svc_xprt.c                         | 281 ++++++++++++-
 tools/net/sunrpc/svc-classify/.gitignore      |   1 +
 tools/net/sunrpc/svc-classify/Makefile        |  65 +++
 tools/net/sunrpc/svc-classify/svc-classify.c  | 357 ++++++++++++++++
 .../sunrpc/svc-classify/svc_classify.bpf.c    |  83 ++++
 tools/testing/selftests/bpf/config            |   2 +
 .../selftests/bpf/prog_tests/svc_classifier.c | 104 +++++
 .../selftests/bpf/progs/bpf_svc_classifier.c  |  80 ++++
 19 files changed, 1724 insertions(+), 24 deletions(-)
Quoted 8 lines
 create mode 100644 Documentation/filesystems/nfs/rpc-server-clients.rst
 create mode 100644 net/sunrpc/svc_classify.c
 create mode 100644 tools/net/sunrpc/svc-classify/.gitignore
 create mode 100644 tools/net/sunrpc/svc-classify/Makefile
 create mode 100644 tools/net/sunrpc/svc-classify/svc-classify.c
 create mode 100644 tools/net/sunrpc/svc-classify/svc_classify.bpf.c
 create mode 100644 tools/testing/selftests/bpf/prog_tests/svc_classifier.c
 create mode 100644 tools/testing/selftests/bpf/progs/bpf_svc_classifier.c

My Comments:

Conceptually, I like this idea, and I personally like that I now have a clear (to me) example of how to plug eBPF programs into kernel code, so thanks for that!

I think the userland bits should be made part of nfs-utils, and the LLM-slop docs need to be rewritten for humans.

Signature · 1 line
Jeff Layton <[email protected]>
#10 lore raw

Re: [PATCH RFC v2 8/9] tools/net/sunrpc: add svc-classify, a prefix classifier and its loader

In reply to Jeff Layton
Quoted 7 lines · Jeff Layton

The userland programs in the kernel are often not packaged, so it seems to me that svc-classify would be better integrated into nfs-utils. That would allow you to more easily bundle it with the systemd service file.

It might even make sense to integrate this functionality into nfsdctl, but I guess the classifier affects all rpc_serv objects as well (e.g. NLM), so it may be best to keep it as its own thing.

Technically we don't need this classifer, we could just expect that if you want to use this feature you must write your own. I added it to make things easier for potential testers - but it is limited. For example, it doesn't classify on anything other than src address.

Ben

#13 lore raw

Re: [PATCH RFC v2 9/9] Documentation: describe RPC server transport classes and the BPF classifier

In reply to Jeff Layton
Quoted 124 lines · Jeff Layton
Quoted 123 lines · Benjamin Coddington

From: Benjamin Coddington <[email protected]>

Dispatch round-robin across clients changes how a service shares its threads, and the classes come from a BPF program the administrator loads, so the administrator needs the model, the class word, and the procedure. Add a page with those: install a classifier with svc-classify (build, attach, fill the map, check, change, remove, keep across reboots), six class maps and the share each gives its clients, the tracepoint that shows the class, and how the hook, the program and the attach model work underneath.

---
 Documentation/filesystems/nfs/index.rst       |   1 +
 .../filesystems/nfs/rpc-server-clients.rst    | 394 ++++++++++++++++++
 2 files changed, 395 insertions(+)
 create mode 100644 Documentation/filesystems/nfs/rpc-server-clients.rst
Documentation/filesystems/nfs/index.rst +1 -0
index a29a212b5b4d..57a61ce0533d 100644
@@ -16,3 +16,4 @@ NFS
    nfsd-io-modes
    knfsd-stats
    reexport
+   rpc-server-clients
Documentation/filesystems/nfs/rpc-server-clients.rst new file +90 -0
new file mode 100644
index 000000000000..b08351a88314
@@ -0,0 +1,394 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+======================================================
+RPC server: per-client dispatch and transport classes
+======================================================
+
+An RPC service thread pool serves its ready transports in FIFO order,
+one request per turn.  A peer with many connections (``nconnect``, a
+deep NFSv4.1 slot table, a data mover with a thread per file) takes a
+turn per connection, and a peer with one connection waits behind all
+of them.
+
+With a classifier attached, the pool instead serves its *clients* round
+robin: each client with work queued gets one request per round, however
+many connections it holds.  Which transports form a client is decided
+by a BPF program the administrator loads, once per accepted connection.
+With no program loaded nothing changes.
+
+This document describes the model, how to install a classifier with
+the ``svc-classify`` tool, what some class maps do, and how it works
+underneath.
+
+Clients, classes and turns
+==========================
+
+A *client* is a set of transports that share one turn.  Every service
+(nfsd, lockd, the NFSv4 callback service) keeps its own clients, found
+by *class* and network namespace, or by class, namespace and peer
+address.
+
+The classifier returns a 32-bit class word for each accepted transport:
+
+``0``
+    ``SVC_CLASS_NONE``: the transport stays on the service's *anonymous
+    client*.
+
+``N``, 1 to 2^31 - 1
+    one client for every transport of class ``N``.
+
+``N | SVC_CLASS_PER_ADDR``
+    one client per peer address within class ``N``.  ``N`` may be 0, so
+    ``SVC_CLASS_PER_ADDR`` alone means one client per peer address.
+
+``SVC_CLASS_PER_ADDR`` is bit 31.  ``svc-classify`` and this document
+write a class word with the bit set as ``N+addr``; ``svc-classify``
+also accepts ``addr`` for ``0+addr``.
+
+Dispatch rules:
+
+- A pool serves the clients that have transports queued round robin,
+  one request per client per round.
+- Within a client, transports are served in FIFO order, except that a
+  transport needing a connection accepted, closed, or a TLS handshake
+  run goes ahead of transports with data, though not two such turns in
+  a row while data waits.
+- A client's share of the pool does not grow with its connection count.
+  A client with one connection and a client with thirty get the same
+  number of turns while both have work queued.
+
+Things that follow from the rules and are easy to get wrong:
+
+- The anonymous client is one client.  Once any classifier is attached,
+  every transport whose class is ``0`` shares a single turn with every
+  other such transport of that service.  Within that one client the old
+  FIFO order applies, so those peers share the turn in proportion to
+  their connection counts.  A map that classifies only the hosts it
+  cares about and leaves the rest at ``0`` gives "the rest" one turn
+  in total (see example 5 below).
+- Per-address clients ignore the port.  Only IPv4 and IPv6 peers can be
+  keyed by address; a transport whose peer is anything else is left on
+  the anonymous client.
+- Listeners and UDP sockets are never classified; UDP traffic is served
+  from the anonymous client.
+- A transport keeps the class it was given when accepted.  Changing the
+  map, or replacing or removing the classifier, affects connections
+  accepted afterwards.  A connection that must be reclassified has to
+  reconnect.
+- nfsd has one service per network namespace, so its anonymous client
+  is per namespace.  lockd and the NFSv4 callback service are one
+  service for all namespaces; their anonymous clients span namespaces.
+- With no classifier attached anywhere, the pool uses a single FIFO;
+  the per-request cost of the feature is then a static branch on
+  enqueue and one empty-queue test on dequeue.
+  ``CONFIG_SUNRPC_BPF_CLASSIFY`` (default y) builds the hook; without
+  it there is no classification.
+
+Installing a classifier
+=======================
+
+There is one tool to do it with: ``svc-classify``, in
Quoted 8 lines

Documentation/filesystems/nfs/rpc-server-clients.rst line 96

Quoted 6 lines
Documentation/filesystems/nfs/rpc-server-clients.rst new file +6 -0
+``tools/net/sunrpc/svc-classify`` of the kernel source.  It carries the
+classifier program inside the binary, attaches it, and manages the
+class map by address prefix.  ``bpftool`` is not needed, and its
+``struct_ops`` subcommands cannot be used in its place (see "How it
+works").
+

"There is one tool to do it with: ..."

barf.

Quoted 9 lines

The obviously LLM-written documentation gives me the Ick and I stopped reading. There is also a lot of info in this doc that is not particularly helpful, like "the things that follow from the rules and are easy to get wrong".

If you want humans to actually read this, then it probably needs to be written by a human. If you think the LLM slop is actually useful for other LLMs, then maybe put those bits at the end in a clearly- delineated section.

Yeah, you're right. This really got spit out of the AI tubes without a lot of refinement by me - thanks for reading even this far.

Ben