Skip to contentSkip to conversation

Deliver TLS session tags to upper-layer consumers (NFSD)

PATCH In review Chuck Lever 4 participants 15 messages Nov 13 – Nov 22

Subsystems (11)
lore
Share & tools
lore

Chuck Lever

author maintainer
#0 lore raw

v3 cover Deliver TLS session tags to upper-layer consumers (NFSD)

NFSD controls remote access to its exports today by IP address and hostname. Neither of these are cryptographically secure. For instance, a MitM can replace the response to a server's plaintext DNS query. As a result, a malicious NFS client can pretend to be a client whose hostname matches one that has access to exports it would otherwise be denied access to.

So instead, we want access-control decisions to be based on cryptographically-vetted objects such as TLS peer-certificates. In-kernel x.509 parsing would duplicate work mature userspace libraries already do, so let's make tlshd evaluate peer certificates in user space.

The certificates are evaluated against admin-defined policy, such as whether the certificate's Issuer matches a particular string. tlshd then marks NFS TLS sessions with those policy matches. Admins also provide a set of policy names on each NFSD export (allow_tags). Computing the intersection with these per-export tags is quick and completely separate from the details of parsing x.509 certificates.

A yet-to-be-posted patch series introduces tag set support to the exportfs command in nfs-utils. tlshd 1.5.0, current as of this writing, already implements TLS session tagging.

---
Changes in v3:
- Replace the sloppy cover letter.
- Avoid a double free in the stringset capacity KUnit case (sashiko).
- Add list markers to the stringset kernel-doc Return: lists (sashiko).
- Move the duplicate session-tag KUnit changes into patch 5 (sashiko).
- Link to v2: https://patch.msgid.link/[email protected]

Changes in v2:
- Drop v1's 1/9, sent ahead of the set per Jeff and now merged.
- Adopt the pinned hr_file/fput lifetime model.
- Adapt to the errno-valued handshake status (hp_done takes int).
- Rename "tagset" to "stringset" throughout (Hannes)
- Warn and return false when a query precedes stringset_finalize().
- Add a patch with KUnit tests for stringset.
- Split the hp_done session-tags argument into its own patch.
- Collect the session tags before taking the completion gate.
- Pass ownership of the tag set to handshake_finish_complete().
- Add handshake_session_tag_len(), shared with the NFSD parser.
- Add KUnit tests for session-tag collection.
- Check the file reference count in the completion-gate KUnit case.
- Rename the ACCEPT cap attribute MAX_TAGS to MAX_SESSION_TAGS.
- Publish the saved tags with smp_wmb() and test_bit_acquire().
- Deny a tagged export on the non-mTLS arms of check_xprtsec_policy().
- Extract nfsd_nl_parse_allow_tags() from the netlink export parser.
- Build stringset only when a consumer selects CONFIG_STRINGSET.
- Free the session tags as soon as hp_done() returns.
- Report a rejected session tag or allow-tag through the extack.
- Show allow_tags in /proc/fs/nfsd/exports.
- Record the client address and XID in nfsd_export_tags_denied.
- Link to v1: https://patch.msgid.link/[email protected]

---
Chuck Lever (10):
handshake: Add tags to "done" downcall
lib: Add a "stringset" data structure
lib: Add KUnit tests for the stringset data structure
handshake: Add a session-tags argument to the TLS done callback
handshake: Pick up session tags passed during the DONE downcall
handshake: Add a kunit test for the completion gate
handshake: Advertise the session-tag cap to user space
SUNRPC: Copy the TLS session tags when they are available
NFSD: Implement export tagging
NFSD: Add allow_tags to the netlink export interface

 Documentation/core-api/index.rst           |   1 +
 Documentation/core-api/stringset.rst       | 243 ++++++++++++++++++++++
 Documentation/netlink/specs/handshake.yaml |  19 ++
 Documentation/netlink/specs/nfsd.yaml      |  10 +
 Documentation/networking/tls-handshake.rst |  62 +++++-
 MAINTAINERS                                |   8 +
 drivers/nvme/host/tcp.c                    |   3 +-
 drivers/nvme/target/tcp.c                  |   3 +-
 fs/nfsd/Kconfig                            |   1 +
 fs/nfsd/export.c                           | 194 +++++++++++++++++-
 fs/nfsd/export.h                           |   9 +
 fs/nfsd/netlink.c                          |   4 +-
 fs/nfsd/netlink.h                          |   3 +-
 fs/nfsd/trace.h                            |  27 +++
 include/linux/stringset.h                  | 191 ++++++++++++++++++
 include/linux/sunrpc/svc_xprt.h            |   7 +-
 include/net/handshake.h                    |  50 ++++-
 include/uapi/linux/handshake.h             |   4 +
 include/uapi/linux/nfsd_netlink.h          |   1 +
 lib/Kconfig                                |  12 ++
 lib/Kconfig.debug                          |   6 +
 lib/Makefile                               |   1 +
 lib/stringset.c                            | 190 ++++++++++++++++++
 lib/tests/Makefile                         |   1 +
 lib/tests/stringset_kunit.c                | 311 +++++++++++++++++++++++++++++
 net/Kconfig                                |   1 +
 net/handshake/genl.c                       |   5 +-
 net/handshake/handshake-test.c             | 307 ++++++++++++++++++++++++++++
 net/handshake/handshake.h                  |   9 +
 net/handshake/netlink.c                    | 110 +++++++++-
 net/handshake/request.c                    |  61 +++++-
 net/handshake/tlshd.c                      |  10 +-
 net/sunrpc/Kconfig                         |   1 +
 net/sunrpc/svc_xprt.c                      |  11 +-
 net/sunrpc/svcauth_unix.c                  |  14 +-
 net/sunrpc/svcsock.c                       |  63 +++++-
 net/sunrpc/xprtsock.c                      |   4 +-
 tools/net/ynl/Makefile.deps                |   3 +-
 38 files changed, 1914 insertions(+), 46 deletions(-)

---
base-commit: d1f4b7c6b997c29de67ad7178ababac2328cb1b4
change-id: 20260512-tls-session-tags-9d0042583f44

Best regards,

Signature · 1 line
Chuck Lever <[email protected]>
author maintainer
#1 lore raw

PATCH v3 1/10 handshake: Add tags to "done" downcall

tlshd has no way to report certificate characteristics to the kernel. An Upper Layer Protocol (such as NFS) therefore cannot authorize access based on what a client's certificate contains.

For example, an export might be restricted to clients that present certificates whose issuer DN contains "O=Oracle". tlshd can parse incoming certificates, and add an "oraclegroup" tag to handshakes where a client presents a certificate with "O=Oracle" somewhere in its Issuer field. NFSD can then be configured to look for that tag and permit access only when it is present. NFSD needs no knowledge of x.509 certificates.

Plumb in the netlink protocol elements for tlshd to return a list of tags to the kernel when a handshake succeeds. Subsequent patches add tag extraction and storage in the handshake layer.

---
 Documentation/netlink/specs/handshake.yaml | 15 +++++++++++++++
 include/uapi/linux/handshake.h             |  3 +++
 net/handshake/genl.c                       |  5 +++--
 3 files changed, 21 insertions(+), 2 deletions(-)
Documentation/netlink/specs/handshake.yaml +15 -0
index 0498ada2dd97..17521b159e89 100644
@@ -18,6 +18,10 @@ definitions:
     value: 4095
     header: linux/err.h
     scope: kernel
+  -
+    name: session-tag-max-len
+    type: const
+    value: 255
   -
     type: enum
     name: handler-class
@@ -96,6 +100,16 @@ attribute-sets:
         name: remote-auth
         type: s32
         multi-attr: true
+      -
+        name: tag
+        type: string
+        doc: >-
+          A tag the handshake agent assigns to this handshake. The
+          payload may omit the trailing NUL. An embedded NUL is
+          rejected.
+        checks:
+          max-len: session-tag-max-len
+        multi-attr: true
 
 operations:
   list:
@@ -133,6 +147,7 @@ operations:
             - status
             - sockfd
             - remote-auth
+            - tag
 
 mcast-groups:
   list:
include/uapi/linux/handshake.h +3 -0
index f907aa082520..bbb9ea2fa4fb 100644
@@ -10,6 +10,8 @@
 #define HANDSHAKE_FAMILY_NAME		"handshake"
 #define HANDSHAKE_FAMILY_VERSION	1
 
+#define HANDSHAKE_SESSION_TAG_MAX_LEN	255
+
 enum handshake_handler_class {
 	HANDSHAKE_HANDLER_CLASS_NONE,
 	HANDSHAKE_HANDLER_CLASS_TLSHD,
@@ -59,6 +61,7 @@ enum {
 	HANDSHAKE_A_DONE_STATUS = 1,
 	HANDSHAKE_A_DONE_SOCKFD,
 	HANDSHAKE_A_DONE_REMOTE_AUTH,
+	HANDSHAKE_A_DONE_TAG,
 
 	__HANDSHAKE_A_DONE_MAX,
 	HANDSHAKE_A_DONE_MAX = (__HANDSHAKE_A_DONE_MAX - 1)
net/handshake/genl.c +3 -2
index 26e3efec64ca..7150b32a54c4 100644
@@ -18,10 +18,11 @@ static const struct nla_policy handshake_accept_nl_policy[HANDSHAKE_A_ACCEPT_HAN
 };
 
 /* HANDSHAKE_CMD_DONE - do */
-static const struct nla_policy handshake_done_nl_policy[HANDSHAKE_A_DONE_REMOTE_AUTH + 1] = {
+static const struct nla_policy handshake_done_nl_policy[HANDSHAKE_A_DONE_TAG + 1] = {
 	[HANDSHAKE_A_DONE_STATUS] = NLA_POLICY_MAX(NLA_U32, MAX_ERRNO),
 	[HANDSHAKE_A_DONE_SOCKFD] = { .type = NLA_S32, },
 	[HANDSHAKE_A_DONE_REMOTE_AUTH] = { .type = NLA_S32, },
+	[HANDSHAKE_A_DONE_TAG] = { .type = NLA_STRING, .len = HANDSHAKE_SESSION_TAG_MAX_LEN, },
 };
 
 /* Ops table for handshake */
@@ -37,7 +38,7 @@ static const struct genl_split_ops handshake_nl_ops[] = {
 		.cmd		= HANDSHAKE_CMD_DONE,
 		.doit		= handshake_nl_done_doit,
 		.policy		= handshake_done_nl_policy,
-		.maxattr	= HANDSHAKE_A_DONE_REMOTE_AUTH,
+		.maxattr	= HANDSHAKE_A_DONE_TAG,
 		.flags		= GENL_ADMIN_PERM | GENL_CMD_CAP_DO,
 	},
 };
Signature · 1 line
2.56.0
#2 lore raw

Re: [PATCH net-next v3 01/10] handshake: Add tags to "done" downcall

Thank you for your contribution! Sashiko AI review found 1 potential issue to consider: - [Medium] handshake: HANDSHAKE_A_DONE_TAG uses NLA_STRING instead of NLA_NUL_STRING --

Thank you for your contribution! Sashiko AI review found 1 potential issue to consider:
- [Medium] handshake: HANDSHAKE_A_DONE_TAG uses NLA_STRING instead of NLA_NUL_STRING
--

commit 686d8603f3072dc2bd0a821dcea3a0351fd8b262
Author: Chuck Lever <[email protected]>

handshake: Add tags to "done" downcall

This commit adds the HANDSHAKE_A_DONE_TAG netlink attribute to the handshake "done" downcall so userspace agents can return session tags to the kernel.

Quoted 4 lines
Documentation/netlink/specs/handshake.yaml +0 -0
index 0498ada2dd97a..17521b159e893 100644

[ ... ]

Quoted 14 lines
Documentation/netlink/specs/handshake.yaml +10 -0
@@ -96,6 +100,16 @@ attribute-sets:
         name: remote-auth
         type: s32
         multi-attr: true
+      -
+        name: tag
+        type: string
+        doc: >-
+          A tag the handshake agent assigns to this handshake. The
+          payload may omit the trailing NUL. An embedded NUL is
+          rejected.
+        checks:
+          max-len: session-tag-max-len
+        multi-attr: true

[ ... ]

Quoted 5 lines
net/handshake/genl.c +0 -0
index 26e3efec64ca7..7150b32a54c45 100644

net/handshake/genl.c static const struct nla_policy handshake_accept_nl_policy[HANDSHAKE_A_ACCEPT_HAN · line 26

Quoted 10 lines
net/handshake/genl.c +2 -1
@@ -18,10 +18,11 @@ static const struct nla_policy handshake_accept_nl_policy[HANDSHAKE_A_ACCEPT_HAN
 };
 
 /* HANDSHAKE_CMD_DONE - do */
-static const struct nla_policy handshake_done_nl_policy[HANDSHAKE_A_DONE_REMOTE_AUTH + 1] = {
+static const struct nla_policy handshake_done_nl_policy[HANDSHAKE_A_DONE_TAG + 1] = {
 	[HANDSHAKE_A_DONE_STATUS] = NLA_POLICY_MAX(NLA_U32, MAX_ERRNO),
 	[HANDSHAKE_A_DONE_SOCKFD] = { .type = NLA_S32, },
 	[HANDSHAKE_A_DONE_REMOTE_AUTH] = { .type = NLA_S32, },
+	[HANDSHAKE_A_DONE_TAG] = { .type = NLA_STRING, .len = HANDSHAKE_SESSION_TAG_MAX_LEN, },
 };

[Severity: Medium]
Should HANDSHAKE_A_DONE_TAG require NUL termination with NLA_NUL_STRING instead of NLA_STRING?

In non-legacy genetlink specifications, string attributes are required to be NUL-terminated. Here, tools/net/ynl/pyynl/ynl_gen_c.py emitted NLA_STRING only because TypeMultiAttr inherits Type.attr_policy() and delegates _attr_policy() instead of attr_policy(), bypassing TypeString.attr_policy():

tools/net/ynl/pyynl/ynl_gen_c.py:TypeMultiAttr._attr_policy() {
def _attr_policy(self, policy):
return self.base_type._attr_policy(policy)
}

Documenting in handshake.yaml that the payload may omit the trailing NUL bakes this generator quirk into the uAPI: if TypeMultiAttr is later fixed to emit NLA_NUL_STRING for multi-attr strings, regenerating the policy with tools/net/ynl/ynl-regen.sh will break userspace agents that omit the trailing NUL.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
author maintainer
#3 lore raw

PATCH v3 2/10 lib: Add a "stringset" data structure

In reply to Chuck Lever

Access control mechanisms sometimes need to match metadata tags between a session and a resource. A stringset provides efficient membership testing and set intersection operations for this purpose.

Set intersection, not membership, is what motivates a purpose-built structure. xarray and rbtree answer a membership query in O(log M), so intersecting by probing costs O(N log M). rhashtable probes in O(1) expected, but pays a hash per element and gives up the ordering a merge relies on. Two sorted arrays instead merge in a single O(N + M) pass.

The implementation therefore uses a sorted array of string pointers. Beyond the intersection cost, a sorted array needs no per-element node allocation. An rbtree does, so it carries more memory overhead for the small sets typical of resource tagging.

The API follows a build-then-query pattern: callers initialize, allocate capacity, add strings, then finalize before querying. Once finalized, the stringset is suitable for concurrent read access.

Build the library only when a consumer selects CONFIG_STRINGSET. Consumers of the new APIs arrive in subsequent patches.

---
 Documentation/core-api/index.rst     |   1 +
 Documentation/core-api/stringset.rst | 243 +++++++++++++++++++++++++++++++++++
 MAINTAINERS                          |   7 +
 include/linux/stringset.h            | 191 +++++++++++++++++++++++++++
 lib/Kconfig                          |  12 ++
 lib/Makefile                         |   1 +
 lib/stringset.c                      | 190 +++++++++++++++++++++++++++
 7 files changed, 645 insertions(+)
Documentation/core-api/index.rst +1 -0
index 92f91c6a0d79..929bf11828c3 100644
@@ -45,6 +45,7 @@ Library functionality that is used throughout the kernel.
    idr
    circular-buffers
    rbtree
+   stringset
    generic-radix-tree
    packing
    this_cpu_ops
Documentation/core-api/stringset.rst new file +243 -0

243 lines not on this page · raw message

MAINTAINERS +7 -0
index c583367c88bc..9b058bb60b0f 100644
@@ -27022,6 +27022,13 @@ S:	Maintained
 F:	Documentation/networking/device_drivers/ethernet/stmicro/
 F:	drivers/net/ethernet/stmicro/stmmac/
 
+STRINGSET DATA STRUCTURE
+M:	Chuck Lever <[email protected]>
+S:	Maintained
+F:	Documentation/core-api/stringset.rst
+F:	include/linux/stringset.h
+F:	lib/stringset.c
+
 SUN HAPPY MEAL ETHERNET DRIVER
 M:	Sean Anderson <[email protected]>
 S:	Maintained
include/linux/stringset.h new file +191 -0

191 lines not on this page · raw message

lib/Kconfig +12 -0
index 1e42f5167c68..c8ba3f7e13d8 100644
@@ -363,6 +363,18 @@ config ASSOCIATIVE_ARRAY
 
 	  for more information.
 
+config STRINGSET
+	bool
+	help
+	  A sorted set of strings, built once and then queried for
+	  membership and intersection.
+
+	  See:
+
+		Documentation/core-api/stringset.rst
+
+	  for more information.
+
 config CLOSURES
 	bool
 
lib/Makefile +1 -0
index a2afe954805c..cbed3e2e7cd3 100644
@@ -61,6 +61,7 @@ obj-y += bcd.o sort.o parser.o debug_locks.o random32.o \
 	 generic-radix-tree.o bitmap-str.o
 obj-y += string_helpers.o
 obj-y += hexdump.o
+obj-$(CONFIG_STRINGSET) += stringset.o
 obj-$(CONFIG_TEST_HEXDUMP) += test_hexdump.o
 obj-y += kstrtox.o
 obj-$(CONFIG_FIND_BIT_BENCHMARK) += find_bit_benchmark.o
lib/stringset.c new file +190 -0

190 lines not on this page · raw message

Signature · 1 line
2.56.0
author maintainer
#4 lore raw

PATCH v3 3/10 lib: Add KUnit tests for the stringset data structure

In reply to Chuck Lever

The stringset data structure carries invariants that are easy to regress silently: stringset_add() transfers ownership of the passed pointer, stringset_finalize() both sorts and drops duplicate strings, queries are valid only after finalization, and stringset_copy() must produce a set that survives destruction of its source. None of these are exercised today.

Add a KUnit suite that drives the supported usage pattern and checks the observable results: capacity bounds and the once-only allocation rule, the post-finalize sort and dedup, binary-search membership at the array boundaries and in gaps, intersection over overlapping, disjoint, and empty sets, and copy independence. Built with KASAN, the dedup and copy cases become use-after-free detectors. Under kmemleak, the ownership case catches a transfer that never happened.

The misuse paths guarded by WARN_ON_ONCE() are intentionally left
out: tripping them taints the kernel, and KUnit has no facility to mark a warning as expected.

---
 MAINTAINERS                 |   1 +
 lib/Kconfig.debug           |   6 +
 lib/tests/Makefile          |   1 +
 lib/tests/stringset_kunit.c | 311 ++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 319 insertions(+)
MAINTAINERS +1 -0
index 9b058bb60b0f..71a4ffe39c53 100644
@@ -27028,6 +27028,7 @@ S:	Maintained
 F:	Documentation/core-api/stringset.rst
 F:	include/linux/stringset.h
 F:	lib/stringset.c
+F:	lib/tests/stringset_kunit.c
 
 SUN HAPPY MEAL ETHERNET DRIVER
 M:	Sean Anderson <[email protected]>
lib/Kconfig.debug +6 -0
index 2d22638ec13a..4ae7922d90ef 100644
@@ -2531,6 +2531,12 @@ config STRING_HELPERS_KUNIT_TEST
 	depends on KUNIT
 	default KUNIT_ALL_TESTS
 
+config STRINGSET_KUNIT_TEST
+	tristate "KUnit test the stringset data structure at runtime" if !KUNIT_ALL_TESTS
+	depends on KUNIT
+	select STRINGSET
+	default KUNIT_ALL_TESTS
+
 config STRING_KUNIT_BENCH
        bool "Benchmark string functions at runtime"
        depends on STRING_KUNIT_TEST
lib/tests/Makefile +1 -0
index df0537e1b91f..aabc70c40840 100644
@@ -54,6 +54,7 @@ CFLAGS_stackinit_kunit.o += $(call cc-disable-warning, switch-unreachable)
 obj-$(CONFIG_STACKINIT_KUNIT_TEST) += stackinit_kunit.o
 obj-$(CONFIG_STRING_KUNIT_TEST) += string_kunit.o
 obj-$(CONFIG_STRING_HELPERS_KUNIT_TEST) += string_helpers_kunit.o
+obj-$(CONFIG_STRINGSET_KUNIT_TEST) += stringset_kunit.o
 obj-$(CONFIG_USERCOPY_KUNIT_TEST) += usercopy_kunit.o
 obj-$(CONFIG_UTIL_MACROS_KUNIT) += util_macros_kunit.o
 obj-$(CONFIG_RATELIMIT_KUNIT_TEST) += test_ratelimit.o
lib/tests/stringset_kunit.c new file +311 -0

311 lines not on this page · raw message

Signature · 1 line
2.56.0
author maintainer
#5 lore raw

PATCH v3 4/10 handshake: Add a session-tags argument to the TLS done callback

In reply to Chuck Lever

Session tags arriving in the DONE downcall have to reach handshake consumers for authorization decisions. The completion callback is the only path to a consumer, so carrying the tags there requires a new argument on tls_done_func_t. That signature change ripples through every existing TLS handshake consumer in the tree.

Add the @tags parameter ahead of the functional change. No consumer reads it yet and the handshake core passes NULL, so behavior is unchanged. Isolating the mechanical signature churn confines the tag-collection patch that follows to the handshake core.

---
 Documentation/networking/tls-handshake.rst |  3 ++-
 drivers/nvme/host/tcp.c                    |  3 ++-
 drivers/nvme/target/tcp.c                  |  3 ++-
 include/net/handshake.h                    | 16 +++++++++++++++-
 net/handshake/tlshd.c                      |  5 +++--
 net/sunrpc/svcsock.c                       |  4 +++-
 net/sunrpc/xprtsock.c                      |  4 +++-
 7 files changed, 30 insertions(+), 8 deletions(-)
Documentation/networking/tls-handshake.rst +2 -1
index 4f7bc1087df9..7e9f4bd659b4 100644
@@ -169,7 +169,8 @@ The synopsis of this function is:
 .. code-block:: c
 
   typedef void	(*tls_done_func_t)(void *data, int status,
-                                   key_serial_t peerid);
+                                   key_serial_t peerid,
+                                   const struct stringset *tags);
 
 The consumer provides a cookie in the @ta_data field of the
 tls_handshake_args structure that is returned in the @data parameter of
drivers/nvme/host/tcp.c +2 -1
index 7ecc6c007f8d..5956b176e936 100644
@@ -1920,7 +1920,8 @@ static void nvme_tcp_set_queue_io_cpu(struct nvme_tcp_queue *queue)
 		qid, queue->io_cpu);
 }
 
-static void nvme_tcp_tls_done(void *data, int status, key_serial_t pskid)
+static void nvme_tcp_tls_done(void *data, int status, key_serial_t pskid,
+			      const struct stringset *tags)
 {
 	struct nvme_tcp_queue *queue = data;
 	struct nvme_tcp_ctrl *ctrl = queue->ctrl;
drivers/nvme/target/tcp.c +2 -1
index 5105317cc61e..40f0cd9d9960 100644
@@ -1871,7 +1871,8 @@ static int nvmet_tcp_tls_key_lookup(struct nvmet_tcp_queue *queue,
 }
 
 static void nvmet_tcp_tls_handshake_done(void *data, int status,
-					 key_serial_t peerid)
+					 key_serial_t peerid,
+					 const struct stringset *tags)
 {
 	struct nvmet_tcp_queue *queue = data;
 
include/net/handshake.h +15 -1
index 8ebd4f9ed26e..c227d42bd767 100644
@@ -10,6 +10,8 @@
 #ifndef _NET_HANDSHAKE_H
 #define _NET_HANDSHAKE_H
 
+#include <linux/stringset.h>
+
 enum {
 	TLS_NO_KEYRING = 0,
 	TLS_NO_PEERID = 0,
@@ -17,8 +19,20 @@ enum {
 	TLS_NO_PRIVKEY = 0,
 };
 
+/**
+ * typedef tls_done_func_t - TLS handshake completion callback
+ * @data: opaque context pointer set via tls_handshake_args.ta_data
+ * @status: zero on success, otherwise a negative errno
+ * @peerid: serial number of peer identity key, or TLS_NO_PEERID
+ * @tags: session tags assigned by the handshake agent
+ *
+ * The @tags pointer is valid only for the duration of this callback.
+ * A consumer requiring persistent access must copy the set with
+ * stringset_copy().
+ */
 typedef void	(*tls_done_func_t)(void *data, int status,
-				   key_serial_t peerid);
+				   key_serial_t peerid,
+				   const struct stringset *tags);
 
 struct tls_handshake_args {
 	struct socket		*ta_sock;
net/handshake/tlshd.c +3 -2
index 7567150c2a4f..997edc3fe31c 100644
@@ -26,7 +26,8 @@
 
 struct tls_handshake_req {
 	void			(*th_consumer_done)(void *data, int status,
-						    key_serial_t peerid);
+						    key_serial_t peerid,
+						    const struct stringset *tags);
 	void			*th_consumer_data;
 
 	int			th_type;
@@ -105,7 +106,7 @@ static void tls_handshake_done(struct handshake_req *req,
 		set_bit(HANDSHAKE_F_REQ_SESSION, &req->hr_flags);
 
 	treq->th_consumer_done(treq->th_consumer_data, status,
-			       treq->th_peerid[0]);
+			       treq->th_peerid[0], NULL);
 }
 
 #if IS_ENABLED(CONFIG_KEYS)
net/sunrpc/svcsock.c +3 -1
index 498c5096eef9..f21522dd51f2 100644
@@ -414,13 +414,15 @@ static void svc_tcp_kill_temp_xprt(struct svc_xprt *xprt)
  * @data: address of xprt to wake
  * @status: status of handshake
  * @peerid: serial number of key containing the remote peer's identity
+ * @tags: session tags assigned by the handshake agent
  *
  * If a security policy is specified as an export option, we don't
  * have a specific export here to check. So we set a "TLS session
  * is present" flag on the xprt and let an upper layer enforce local
  * security policy.
  */
-static void svc_tcp_handshake_done(void *data, int status, key_serial_t peerid)
+static void svc_tcp_handshake_done(void *data, int status, key_serial_t peerid,
+				   const struct stringset *tags)
 {
 	struct svc_xprt *xprt = data;
 	struct svc_sock *svsk = container_of(xprt, struct svc_sock, sk_xprt);
net/sunrpc/xprtsock.c +3 -1
index 027b37de694b..ade473f1a312 100644
@@ -2593,9 +2593,11 @@ static int xs_tcp_tls_finish_connecting(struct rpc_xprt *lower_xprt,
  * @data: address of xprt to wake
  * @status: status of handshake
  * @peerid: serial number of key containing the remote's identity
+ * @tags: session tags assigned by the handshake agent
  *
  */
-static void xs_tls_handshake_done(void *data, int status, key_serial_t peerid)
+static void xs_tls_handshake_done(void *data, int status, key_serial_t peerid,
+				  const struct stringset *tags)
 {
 	struct rpc_xprt *lower_xprt = data;
 	struct sock_xprt *lower_transport =
Signature · 1 line
2.56.0
author maintainer
#6 lore raw

PATCH v3 5/10 handshake: Pick up session tags passed during the DONE downcall

In reply to Chuck Lever

Upper Layer Protocols such as NFS have no way to base access control on TLS certificate characteristics. Parsing x.509 in the kernel duplicates work user space libraries already do. The tlshd daemon can evaluate certificates against administrator-defined policies and assign tags indicating which policies a certificate satisfies.

Collect session tags from the DONE netlink downcall and pass them to handshake consumers via their completion callback. NFSD, for example, can then restrict export access to connections whose certificate earned a particular tag, without parsing the certificate itself. The tags are valid only while the callback runs. Release them as soon as the callback returns, so a long-lived transport such as an NFS connection does not hold them for the life of the socket.

The genl family runs DONE with parallel_ops, so concurrent downcalls for the same socket can both reach the same handshake_req. Split handshake_complete() into a try/finish pair. The handler collects tags into a private stringset before taking the completion gate, and the winner hands the set to handshake_finish_complete(). A duplicate downcall is refused with -EBUSY. handshake_req_cancel() returns false once the gate is taken, and svc_tcp_handshake() then waits uninterruptibly for its callback. Collecting the tags first keeps the handler's GFP_KERNEL allocations out of that wait.

The DONE downcall can now fail, where before it always returned success. A tag the parser rejects completes the handshake with -EIO and returns the parse error to tlshd. The extended ACK names the rejected tag attribute. The request is complete on that path, so a retry collects -EBUSY.

When the count of distinct tags exceeds HANDSHAKE_MAX_SESSIONTAGS, the handler truncates the list, logs pr_warn_once(), and returns success. Failing the DONE with -E2BIG would signal the daemon definitively, but it would also tear down an otherwise usable handshake.

---
 Documentation/networking/tls-handshake.rst |  51 +++++++
 include/net/handshake.h                    |  31 +++-
 net/Kconfig                                |   1 +
 net/handshake/handshake-test.c             | 225 +++++++++++++++++++++++++++++
 net/handshake/handshake.h                  |   9 ++
 net/handshake/netlink.c                    | 110 +++++++++++++-
 net/handshake/request.c                    |  59 +++++++-
 net/handshake/tlshd.c                      |   2 +-
 8 files changed, 476 insertions(+), 12 deletions(-)
Documentation/networking/tls-handshake.rst +51 -0
index 7e9f4bd659b4..eb8cfe3367cb 100644
@@ -201,6 +201,10 @@ The @peerid parameter contains the serial number of a key containing the
 remote peer's identity or the value TLS_NO_PEERID if the session is not
 authenticated.
 
+The @tags parameter points to a stringset containing session metadata
+assigned by the handshake agent. See the "Session Tags" section
+below for details on lifetime and safe access patterns.
+
 A best practice is to close and destroy the socket immediately if the
 handshake failed.
 
@@ -221,3 +225,50 @@ received message data is TLS record data or session metadata.
 See tls.rst for details on how a kTLS consumer recognizes incoming
 (decrypted) application data, alerts, and handshake packets once the
 socket has been promoted to use the TLS ULP.
+
+
+Session Tags
+============
+
+When a TLS handshake completes successfully, the handshake agent may
+assign metadata tags to the session. These tags enable kernel consumers
+to make authorization decisions based on certificate characteristics
+without parsing x.509 certificates directly.
+
+The handshake agent evaluates the peer's certificate against
+administrator-defined policies and assigns tags indicating which
+policies the certificate satisfies. For example, an administrator
+might configure the agent to assign an "internal-servers" tag when a
+certificate's Issuer DN matches a particular corporate CA.
+
+Tags are delivered to the consumer via the @tags parameter of the
+completion callback. The stringset is valid only for the duration of the
+callback. Consumers needing persistent access must copy the stringset
+using stringset_copy() before returning.
+
+To check whether a session has a particular tag:
+
+.. code-block:: c
+
+  if (stringset_is_member(tags, "internal-servers")) {
+          /* Certificate matched the internal-servers policy */
+  }
+
+To check whether a session has any tag from a small fixed set:
+
+.. code-block:: c
+
+  if (stringset_is_member(tags, "admin") ||
+      stringset_is_member(tags, "operator")) {
+          /* Certificate matched admin or operator policy */
+  }
+
+When the required set is dynamic (e.g., parsed from an export option),
+construct a stringset and use stringset_intersection() to test for any
+overlap. See Documentation/core-api/stringset.rst for the
+initialization, add, and finalize sequence.
+
+If the handshake failed or no tags were assigned, the stringset is
+empty. The handshake layer always delivers a finalized stringset to
+the callback, so consumers may call stringset_is_member() and
+stringset_intersection() unconditionally without a separate guard.
include/net/handshake.h +30 -1
index c227d42bd767..ad96e92af04d 100644
@@ -10,8 +10,36 @@
 #ifndef _NET_HANDSHAKE_H
 #define _NET_HANDSHAKE_H
 
+#include <linux/errno.h>
 #include <linux/stringset.h>
 
+/*
+ * Bounds the cost of stringset_intersection() in consumer
+ * authorization checks.
+ */
+#define HANDSHAKE_MAX_SESSIONTAGS	64
+
+/**
+ * handshake_session_tag_len - Content length of a session tag payload
+ * @data: start of an NLA_STRING tag payload
+ * @len: payload length in bytes, including any trailing NUL
+ *
+ * A tag payload may arrive with or without a single trailing NUL, and
+ * the terminator is not part of the content. NLA_STRING cannot express
+ * "no NUL except as terminator," so callers validate the payload here.
+ *
+ * Return: content length in bytes, or -EINVAL if the payload is empty
+ * or carries an embedded NUL.
+ */
+static inline int handshake_session_tag_len(const char *data, size_t len)
+{
+	if (len > 0 && data[len - 1] == '\0')
+		len--;
+	if (len == 0 || memchr(data, '\0', len))
+		return -EINVAL;
+	return len;
+}
+
 enum {
 	TLS_NO_KEYRING = 0,
 	TLS_NO_PEERID = 0,
@@ -24,7 +52,8 @@ enum {
  * @data: opaque context pointer set via tls_handshake_args.ta_data
  * @status: zero on success, otherwise a negative errno
  * @peerid: serial number of peer identity key, or TLS_NO_PEERID
- * @tags: session tags assigned by the handshake agent
+ * @tags: session tags assigned by the handshake agent; never NULL,
+ *        always finalized, and empty when no tags were assigned
  *
  * The @tags pointer is valid only for the duration of this callback.
  * A consumer requiring persistent access must copy the set with
net/Kconfig +1 -0
index 76ab44aa439a..aab2d2f03d4d 100644
@@ -93,6 +93,7 @@ source "net/xdp/Kconfig"
 config NET_HANDSHAKE
 	bool
 	depends on SUNRPC || NVME_TARGET_TCP || NVME_TCP
+	select STRINGSET
 	default y
 
 config NET_HANDSHAKE_KUNIT_TEST
net/handshake/handshake-test.c +225 -0

244 lines not on this page · raw message

net/handshake/handshake.h +9 -0
index da61cadd1ad3..11f57d09b511 100644
@@ -10,6 +10,8 @@
 #ifndef _INTERNAL_HANDSHAKE_H
 #define _INTERNAL_HANDSHAKE_H
 
+#include <linux/stringset.h>
+
 /* Per-net namespace context */
 struct handshake_net {
 	spinlock_t		hn_lock;	/* protects next 3 fields */
@@ -36,6 +38,7 @@ struct handshake_req {
 	struct file			*hr_file;
 	struct sock			*hr_sk;
 	void				(*hr_odestruct)(struct sock *sk);
+	struct stringset		hr_tags;
 
 	/* Always the last field */
 	char				hr_priv[];
@@ -77,6 +80,9 @@ int handshake_genl_notify(struct net *net, const struct handshake_proto *proto,
 struct nlmsghdr *handshake_genl_put(struct sk_buff *msg,
 				    struct genl_info *info);
 struct handshake_net *handshake_pernet(struct net *net);
+#if IS_ENABLED(CONFIG_KUNIT)
+int handshake_get_sessiontags(struct stringset *tags, struct genl_info *info);
+#endif
 
 /* request.c */
 struct handshake_req *handshake_req_alloc(const struct handshake_proto *proto,
@@ -88,6 +94,9 @@ struct handshake_req *handshake_req_hash_lookup(struct sock *sk);
 struct handshake_req *handshake_req_next(struct handshake_net *hn, int class);
 int handshake_req_submit(struct socket *sock, struct handshake_req *req,
 			 gfp_t flags);
+bool handshake_try_complete(struct handshake_req *req);
+void handshake_finish_complete(struct handshake_req *req, int status,
+			       struct genl_info *info, struct stringset *tags);
 void handshake_complete(struct handshake_req *req, int status,
 			struct genl_info *info);
 bool handshake_req_cancel(struct sock *sk);
net/handshake/netlink.c +108 -2

135 lines not on this page · raw message

net/handshake/request.c +51 -8

83 lines not on this page · raw message

net/handshake/tlshd.c +1 -1
index 997edc3fe31c..a9c5e5bedced 100644
@@ -106,7 +106,7 @@ static void tls_handshake_done(struct handshake_req *req,
 		set_bit(HANDSHAKE_F_REQ_SESSION, &req->hr_flags);
 
 	treq->th_consumer_done(treq->th_consumer_data, status,
-			       treq->th_peerid[0], NULL);
+			       treq->th_peerid[0], &req->hr_tags);
 }
 
 #if IS_ENABLED(CONFIG_KEYS)
Signature · 1 line
2.56.0
author maintainer
#7 lore raw

PATCH v3 6/10 handshake: Add a kunit test for the completion gate

In reply to Chuck Lever

The genl family runs DONE with parallel_ops, so duplicate or concurrent DONE downcalls for the same socket can both reach the same handshake_req. The split try/finish pair guarantees that exactly one caller drives completion to the consumer's hp_done callback. Ownership of the collected session tags can then pass to the request.

Exercise the gate's exclusivity with a kunit case: only the first handshake_try_complete() wins, and a later handshake_complete() on the same request leaves hp_done fired exactly once.

---
 net/handshake/handshake-test.c | 82 ++++++++++++++++++++++++++++++++++++++++++
 net/handshake/request.c        |  2 ++
 2 files changed, 84 insertions(+)
net/handshake/handshake-test.c +82 -0
index b4a56c4350bd..b6992d1fa4ce 100644
@@ -465,6 +465,84 @@ static void handshake_req_cancel_test3(struct kunit *test)
 	fput(filp);
 }
 
+static int handshake_try_complete_done_count;
+
+static void test_done_count_func(struct handshake_req *req, int status,
+				 struct genl_info *info)
+{
+	handshake_try_complete_done_count++;
+}
+
+static struct handshake_proto handshake_req_alloc_proto_count = {
+	.hp_handler_class	= HANDSHAKE_HANDLER_CLASS_TLSHD,
+	.hp_accept		= test_accept_func,
+	.hp_done		= test_done_count_func,
+};
+
+static void handshake_try_complete_test1(struct kunit *test)
+{
+	unsigned long fcount_before;
+	struct handshake_req *req, *next;
+	struct handshake_net *hn;
+	struct socket *sock;
+	struct file *filp;
+	struct net *net;
+	bool first, second;
+	int err;
+
+	/* Arrange */
+	handshake_try_complete_done_count = 0;
+
+	req = handshake_req_alloc(&handshake_req_alloc_proto_count, GFP_KERNEL);
+	KUNIT_ASSERT_NOT_NULL(test, req);
+
+	err = __sock_create(&init_net, PF_INET, SOCK_STREAM, IPPROTO_TCP,
+			    &sock, 1);
+	KUNIT_ASSERT_EQ(test, err, 0);
+
+	filp = sock_alloc_file(sock, O_NONBLOCK, NULL);
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, filp);
+	KUNIT_ASSERT_NOT_NULL(test, sock->sk);
+	sock->file = filp;
+
+	fcount_before = file_count(filp);
+	err = handshake_req_submit(sock, req, GFP_KERNEL);
+	KUNIT_ASSERT_EQ(test, err, 0);
+	KUNIT_EXPECT_EQ(test, file_count(filp), fcount_before + 1);
+
+	net = sock_net(sock->sk);
+	hn = handshake_pernet(net);
+	KUNIT_ASSERT_NOT_NULL(test, hn);
+
+	/* Pretend to accept this request */
+	next = handshake_req_next(hn, HANDSHAKE_HANDLER_CLASS_TLSHD);
+	KUNIT_ASSERT_PTR_EQ(test, req, next);
+	/* Simulate FD_PREPARE() consuming the file reference handed
+	 * off by handshake_req_next(); see handshake_nl_accept_doit().
+	 */
+	fput(filp);
+
+	/* Act */
+	first = handshake_try_complete(req);
+	second = handshake_try_complete(req);
+	handshake_finish_complete(req, 0, NULL, NULL);
+	/* handshake_finish_complete() drops the request's pin on the file. */
+	KUNIT_EXPECT_EQ(test, file_count(filp), fcount_before);
+	/* handshake_complete() re-enters the gate via
+	 * handshake_try_complete(). With the gate already taken,
+	 * hp_done must not fire a second time.
+	 */
+	handshake_complete(req, 0, NULL);
+
+	/* Assert */
+	KUNIT_EXPECT_TRUE(test, first);
+	KUNIT_EXPECT_FALSE(test, second);
+	KUNIT_EXPECT_EQ(test, handshake_try_complete_done_count, 1);
+	KUNIT_EXPECT_EQ(test, file_count(filp), fcount_before);
+
+	fput(filp);
+}
+
 static struct handshake_req *handshake_req_destroy_test;
 
 static void test_destroy_func(struct handshake_req *req)
@@ -751,6 +829,10 @@ static struct kunit_case handshake_api_test_cases[] = {
 		.name			= "req_cancel after done",
 		.run_case		= handshake_req_cancel_test3,
 	},
+	{
+		.name			= "try_complete gate is exclusive",
+		.run_case		= handshake_try_complete_test1,
+	},
 	{
 		.name			= "req_destroy works",
 		.run_case		= handshake_req_destroy_test1,
net/handshake/request.c +2 -0
index 72515e5e8cc2..31a81b7cb7e9 100644
@@ -340,6 +340,7 @@ bool handshake_try_complete(struct handshake_req *req)
 {
 	return !test_and_set_bit(HANDSHAKE_F_REQ_COMPLETED, &req->hr_flags);
 }
+EXPORT_SYMBOL_IF_KUNIT(handshake_try_complete);
 
 /**
  * handshake_finish_complete - Deliver completion to the consumer
@@ -372,6 +373,7 @@ void handshake_finish_complete(struct handshake_req *req, int status,
 
 	fput(file);
 }
+EXPORT_SYMBOL_IF_KUNIT(handshake_finish_complete);
 
 void handshake_complete(struct handshake_req *req, int status,
 			struct genl_info *info)
Signature · 1 line
2.56.0
author maintainer
#8 lore raw

PATCH v3 7/10 handshake: Advertise the session-tag cap to user space

In reply to Chuck Lever

The kernel caps the number of session tags it accepts in a DONE downcall at HANDSHAKE_MAX_SESSIONTAGS. tlshd has no way to learn this cap. The value lives in include/net/handshake.h, not in the UAPI header, so a daemon cannot size its tag list to it. An over-cap list is truncated with one pr_warn_once per boot, which is recoverable but easy to miss.

Carry the cap on every ACCEPT reply as HANDSHAKE_A_ACCEPT_MAX_SESSION_TAGS, a u32 attribute populated by the kernel. User space reads the value at ACCEPT time and can gate its DONE-side tag list against it. A list longer than the cap then becomes a user-space policy choice instead of a kernel-side truncation.

An attribute lets the kernel raise the cap in a later release without a new UAPI constant.

---
 Documentation/netlink/specs/handshake.yaml | 4 ++++
 Documentation/networking/tls-handshake.rst | 8 ++++++++
 include/uapi/linux/handshake.h             | 1 +
 net/handshake/tlshd.c                      | 5 +++++
 4 files changed, 18 insertions(+)
Documentation/netlink/specs/handshake.yaml +4 -0
index 17521b159e89..f86490cd14c6 100644
@@ -85,6 +85,9 @@ attribute-sets:
       -
         name: keyring
         type: s32
+      -
+        name: max-session-tags
+        type: u32
   -
     name: done
     attributes:
@@ -136,6 +139,7 @@ operations:
             - certificate
             - peername
             - keyring
+            - max-session-tags
     -
       name: done
       doc: Handler reports handshake completion
Documentation/networking/tls-handshake.rst +8 -0
index eb8cfe3367cb..ca8f06973fac 100644
@@ -272,3 +272,11 @@ If the handshake failed or no tags were assigned, the stringset is
 empty. The handshake layer always delivers a finalized stringset to
 the callback, so consumers may call stringset_is_member() and
 stringset_intersection() unconditionally without a separate guard.
+
+The stringset delivered to the consumer may contain fewer tags than
+the handshake agent assigned. The kernel caps the per-DONE tag
+count at HANDSHAKE_MAX_SESSIONTAGS, and individual tags within
+the cap may be dropped under memory pressure. The kernel reports
+the cap in every ACCEPT reply as
+HANDSHAKE_A_ACCEPT_MAX_SESSION_TAGS, so the agent can size its
+DONE-side tag list to it.
include/uapi/linux/handshake.h +1 -0
index bbb9ea2fa4fb..a22393527342 100644
@@ -52,6 +52,7 @@ enum {
 	HANDSHAKE_A_ACCEPT_CERTIFICATE,
 	HANDSHAKE_A_ACCEPT_PEERNAME,
 	HANDSHAKE_A_ACCEPT_KEYRING,
+	HANDSHAKE_A_ACCEPT_MAX_SESSION_TAGS,
 
 	__HANDSHAKE_A_ACCEPT_MAX,
 	HANDSHAKE_A_ACCEPT_MAX = (__HANDSHAKE_A_ACCEPT_MAX - 1)
net/handshake/tlshd.c +5 -0
index a9c5e5bedced..ec050058805a 100644
@@ -238,6 +238,11 @@ static int tls_handshake_accept(struct handshake_req *req,
 			goto out_cancel;
 	}
 
+	ret = nla_put_u32(msg, HANDSHAKE_A_ACCEPT_MAX_SESSION_TAGS,
+			  HANDSHAKE_MAX_SESSIONTAGS);
+	if (ret < 0)
+		goto out_cancel;
+
 	ret = nla_put_u32(msg, HANDSHAKE_A_ACCEPT_AUTH_MODE,
 			  treq->th_auth_mode);
 	if (ret < 0)
Signature · 1 line
2.56.0
author maintainer
#9 lore raw

PATCH v3 8/10 SUNRPC: Copy the TLS session tags when they are available

In reply to Chuck Lever

When a server handshake completes successfully, tlshd might provide a set of TLS session tags. SUNRPC can save these within the svc_xprt. NFSD can later use them to authorize or reject operations that target NFS exports whose allow_tags list intersects those tags.

A second handshake on the same transport would destroy the saved tags while other workers read them. svcauth_tls_accept() refuses AUTH_TLS on a transport that already carries a TLS session, and svc_tcp_handshake() rechecks the session flag under XPT_BUSY to close the race with a handshake that completes concurrently.

---
 include/linux/sunrpc/svc_xprt.h |  7 ++++-
 net/sunrpc/Kconfig              |  1 +
 net/sunrpc/svc_xprt.c           | 11 +++++---
 net/sunrpc/svcauth_unix.c       | 14 +++++++++-
 net/sunrpc/svcsock.c            | 59 ++++++++++++++++++++++++++++++++++-------
 5 files changed, 78 insertions(+), 14 deletions(-)
include/linux/sunrpc/svc_xprt.h +6 -1
index 2af222f3ea2c..e52d5d4e7941 100644
@@ -9,6 +9,7 @@
 #define SUNRPC_SVC_XPRT_H
 
 #include <linux/sunrpc/svc.h>
+#include <linux/stringset.h>
 
 struct module;
 
@@ -82,6 +83,7 @@ struct svc_xprt {
 	const struct cred	*xpt_cred;
 	struct rpc_xprt		*xpt_bc_xprt;	/* NFSv4.1 backchannel */
 	struct rpc_xprt_switch	*xpt_bc_xps;	/* NFSv4.1 backchannel */
+	struct stringset	xpt_handshake_tags;	/* TLS session tags */
 };
 
 /* flag bits for xpt_flags */
@@ -101,7 +103,10 @@ enum {
 	XPT_KILL_TEMP,		/* call xpo_kill_temp_xprt before closing */
 	XPT_CONG_CTRL,		/* has congestion control */
 	XPT_HANDSHAKE,		/* xprt requests a handshake */
-	XPT_TLS_SESSION,	/* transport-layer security established */
+	XPT_TLS_SESSION,	/* transport-layer security established;
+				 * also gates xpt_handshake_tags
+				 * visibility and destruction
+				 */
 	XPT_PEER_AUTH,		/* peer has been authenticated */
 	XPT_PEER_VALID,		/* peer has presented a filehandle that
 				 * it has access to.  It is NOT counted
net/sunrpc/Kconfig +1 -0
index e7808e5714dc..5a7250b42a24 100644
@@ -2,6 +2,7 @@
 config SUNRPC
 	tristate
 	depends on MULTIUSER
+	select STRINGSET
 
 config SUNRPC_GSS
 	tristate
net/sunrpc/svc_xprt.c +8 -3
index 08cfe8a31189..7a8cfc7c3d75 100644
@@ -170,6 +170,8 @@ static void svc_xprt_free(struct kref *kref)
 	struct svc_xprt *xprt =
 		container_of(kref, struct svc_xprt, xpt_ref);
 	struct module *owner = xprt->xpt_class->xcl_owner;
+
+	stringset_destroy(&xprt->xpt_handshake_tags);
 	if (test_bit(XPT_CACHE_AUTH, &xprt->xpt_flags))
 		svcauth_unix_info_release(xprt);
 	put_cred(xprt->xpt_cred);
@@ -190,9 +192,12 @@ void svc_xprt_put(struct svc_xprt *xprt)
 }
 EXPORT_SYMBOL_GPL(svc_xprt_put);
 
-/*
- * Called by transport drivers to initialize the transport independent
- * portion of the transport instance.
+/**
+ * svc_xprt_init - initialize transport-independent fields of an xprt
+ * @net: Network namespace
+ * @xcl: Transport class
+ * @xprt: Transport to be initialized
+ * @serv: RPC service
  */
 void svc_xprt_init(struct net *net, struct svc_xprt_class *xcl,
 		   struct svc_xprt *xprt, struct svc_serv *serv)
net/sunrpc/svcauth_unix.c +13 -1
index 1380d0946401..f5ed20e267b9 100644
@@ -1100,7 +1100,8 @@ struct auth_ops svcauth_null = {
  *
  * Return values:
  *   %SVC_OK: Both credential and verifier are valid
- *   %SVC_DENIED: Credential or verifier is not valid
+ *   %SVC_DENIED: Credential or verifier is not valid, or a TLS
+ *                session already exists on this transport
  *   %SVC_GARBAGE: Failed to decode credential or verifier
  *   %SVC_CLOSE: Temporary failure
  *
@@ -1138,6 +1139,17 @@ svcauth_tls_accept(struct svc_rqst *rqstp)
 		return SVC_DENIED;
 	}
 
+	/*
+	 * AUTH_TLS initiates a handshake, and a second handshake
+	 * would destroy xpt_handshake_tags while other workers read
+	 * them. This test can pass before a concurrent handshake
+	 * completes; svc_tcp_handshake() rechecks under XPT_BUSY.
+	 */
+	if (test_bit(XPT_TLS_SESSION, &xprt->xpt_flags)) {
+		rqstp->rq_auth_stat = rpc_autherr_badcred;
+		return SVC_DENIED;
+	}
+
 	/* Signal that mapping to nobody uid/gid is required */
 	cred->cr_uid = INVALID_UID;
 	cred->cr_gid = INVALID_GID;
net/sunrpc/svcsock.c +50 -9

91 lines not on this page · raw message

Signature · 1 line
2.56.0
author maintainer
#10 lore raw

PATCH v3 9/10 NFSD: Implement export tagging

In reply to Chuck Lever

NFSD treats TLS peer identity as a boolean: a peer is authenticated or it is not. Some deployments need finer authorization. A single certificate may authenticate several distinct actors, and an administrator may wish to grant each a different level of access.

Once a TLS handshake completes, tlshd hands the kernel a list of tags for the session. When an export configures an allow_tags list, grant access only when the session carries at least one matching tag. An export with no allow_tags list continues to admit any authenticated peer. Report each denial through the nfsd_export_tags_denied tracepoint, with the client's address and the request's XID.

tlshd issues session tags only for an mTLS handshake. Reject an allow_tags list paired with any xprtsec mode other than mtls. The administrator then states the combination explicitly, where a default xprtsec setting would expose the export to plaintext or anonymous-TLS peers.

__fh_verify() does not call check_xprtsec_policy() for an NLM or LOCALIO request. lockd does not support xprtsec, and a LOCALIO request does not have an svc_rqst. The tag check is part of check_xprtsec_policy(), so with this patch an allow_tags list does not restrict NLM or LOCALIO requests. NLM and LOCALIO requests already bypass an xprtsec=mtls restriction in the same way.

A kernel that predates this patch ignores allow_tags and exports the filesystem without the restriction. List an export's tags in /proc/fs/nfsd/exports so user space can confirm the restriction took effect.

---
 fs/nfsd/Kconfig         |   1 +
 fs/nfsd/export.c        | 114 +++++++++++++++++++++++++++++++++++++++++++++---
 fs/nfsd/export.h        |   9 ++++
 fs/nfsd/trace.h         |  27 ++++++++++++
 include/net/handshake.h |   5 +++
 5 files changed, 151 insertions(+), 5 deletions(-)
fs/nfsd/Kconfig +1 -0
index ffb76761d6a8..14f694323dcb 100644
@@ -12,6 +12,7 @@ config NFSD
 	select SUNRPC
 	select EXPORTFS
 	select NFS_COMMON
+	select STRINGSET
 	select NFS_ACL_SUPPORT if NFSD_V2_ACL
 	select NFS_ACL_SUPPORT if NFSD_V3_ACL
 	depends on MULTIUSER
fs/nfsd/export.c +109 -5

206 lines not on this page · raw message

fs/nfsd/export.h +9 -0
index 117fb28db1e0..e7ebf5abc594 100644
@@ -7,6 +7,7 @@
 
 #include <linux/sunrpc/cache.h>
 #include <linux/percpu_counter.h>
+#include <linux/stringset.h>
 #include <uapi/linux/nfsd/export.h>
 #include <linux/nfs4.h>
 
@@ -47,6 +48,13 @@ struct exp_flavor_info {
 	u32	flags;
 };
 
+/*
+ * Bounds the per-request cost of the stringset_intersection() in
+ * check_xprtsec_policy(). Independent of the per-handshake session
+ * tag cap, HANDSHAKE_MAX_SESSIONTAGS.
+ */
+#define NFSD_MAX_ALLOW_TAGS	64
+
 /* Per-export stats */
 enum {
 	EXP_STATS_FH_STALE,
@@ -78,6 +86,7 @@ struct svc_export {
 	struct rcu_head		ex_rcu;
 	unsigned long		ex_xprtsec_modes;
 	struct export_stats	*ex_stats;
+	struct stringset	ex_allow_tags;
 };
 
 /* an "export key" (expkey) maps a filehandlefragement to an
fs/nfsd/trace.h +27 -0
index 2ae7f150a72c..e5c179fe28f8 100644
@@ -467,6 +467,33 @@ TRACE_EVENT(nfsd_export_update,
 	)
 );
 
+TRACE_EVENT(nfsd_export_tags_denied,
+	TP_PROTO(
+		const struct svc_rqst *rqstp,
+		const struct svc_export *exp
+	),
+	TP_ARGS(rqstp, exp),
+	TP_STRUCT__entry(
+		__field(u32, xid)
+		__sockaddr(client, rqstp->rq_xprt->xpt_remotelen)
+		__string(path, exp->ex_path.dentry->d_name.name)
+		__string(auth_domain, exp->ex_client->name)
+	),
+	TP_fast_assign(
+		const struct svc_xprt *xprt = rqstp->rq_xprt;
+
+		__entry->xid = be32_to_cpu(rqstp->rq_xid);
+		__assign_sockaddr(client, &xprt->xpt_remote,
+				  xprt->xpt_remotelen);
+		__assign_str(path);
+		__assign_str(auth_domain);
+	),
+	TP_printk("xid=0x%08x client=%pISpc path=%s domain=%s",
+		__entry->xid, __get_sockaddr(client),
+		__get_str(path), __get_str(auth_domain)
+	)
+);
+
 DECLARE_EVENT_CLASS(nfsd_io_class,
 	TP_PROTO(struct svc_rqst *rqstp,
 		 struct svc_fh	*fhp,
include/net/handshake.h +5 -0
index ad96e92af04d..d04f0c0b9cba 100644
@@ -12,10 +12,15 @@
 
 #include <linux/errno.h>
 #include <linux/stringset.h>
+#include <uapi/linux/handshake.h>
 
 /*
  * Bounds the cost of stringset_intersection() in consumer
  * authorization checks.
+ *
+ * The per-tag byte limit, HANDSHAKE_SESSION_TAG_MAX_LEN, is
+ * enforced by the netlink policy generated from
+ * Documentation/netlink/specs/handshake.yaml.
  */
 #define HANDSHAKE_MAX_SESSIONTAGS	64
 
Signature · 1 line
2.56.0
author maintainer
#11 lore raw

PATCH v3 10/10 NFSD: Add allow_tags to the netlink export interface

In reply to Chuck Lever

The legacy exportfs cache path accepts an allow_tags clause that restricts an export to mTLS sessions carrying at least one matching session tag. The netlink-based svc_export interface had no such attribute, so administrators configuring exports via netlink could not request tag enforcement. nfsd_nl_parse_one_export() left ex_allow_tags empty, and check_xprtsec_policy() then granted access to any authenticated peer.

Extend the svc-export attribute set with allow-tags and parse it in nfsd_nl_parse_one_export(). Apply the same xprtsec=mtls consistency check as svc_export_parse() so the netlink path rejects a contradictory security policy with an explicit error at configuration time rather than caching an export that can never grant access.

A bare -EINVAL does not tell an administrator which tag in a request was malformed, or that the export's xprtsec setting conflicts with allow-tags. Pass the request's extended ACK into nfsd_nl_parse_one_export(). The parser reports a malformed tag as the bad attribute, and too many tags or an xprtsec setting other than mtls alone as an error message.

---
 Documentation/netlink/specs/nfsd.yaml | 10 +++++
 fs/nfsd/export.c                      | 80 +++++++++++++++++++++++++++++++++--
 fs/nfsd/netlink.c                     |  4 +-
 fs/nfsd/netlink.h                     |  3 +-
 include/uapi/linux/nfsd_netlink.h     |  1 +
 tools/net/ynl/Makefile.deps           |  3 +-
 6 files changed, 94 insertions(+), 7 deletions(-)
Documentation/netlink/specs/nfsd.yaml +10 -0
index 9ae37bf3ea71..e58856b34b02 100644
@@ -7,6 +7,10 @@ uapi-header: linux/nfsd_netlink.h
 doc: NFSD configuration over generic netlink.
 
 definitions:
+  -
+    name: handshake-session-tag-max-len
+    type: const
+    header: linux/handshake.h
   -
     type: flags
     name: cache-type
@@ -297,6 +301,12 @@ attribute-sets:
       -
         name: fsid
         type: s32
+      -
+        name: allow-tags
+        type: string
+        checks:
+          max-len: handshake-session-tag-max-len
+        multi-attr: true
   -
     name: svc-export-reqs
     attributes:
fs/nfsd/export.c +76 -4

122 lines not on this page · raw message

fs/nfsd/netlink.c +3 -1
index 88a4a4ffcb7f..146c65575741 100644
@@ -10,6 +10,7 @@
 #include "netlink.h"
 
 #include <uapi/linux/nfsd_netlink.h>
+#include <linux/handshake.h>
 
 /* Common nested types */
 const struct nla_policy nfsd_auth_flavor_nl_policy[NFSD_A_AUTH_FLAVOR_FLAGS + 1] = {
@@ -41,7 +42,7 @@ const struct nla_policy nfsd_sock_nl_policy[NFSD_A_SOCK_TRANSPORT_NAME + 1] = {
 	[NFSD_A_SOCK_TRANSPORT_NAME] = { .type = NLA_NUL_STRING, },
 };
 
-const struct nla_policy nfsd_svc_export_nl_policy[NFSD_A_SVC_EXPORT_FSID + 1] = {
+const struct nla_policy nfsd_svc_export_nl_policy[NFSD_A_SVC_EXPORT_ALLOW_TAGS + 1] = {
 	[NFSD_A_SVC_EXPORT_SEQNO] = { .type = NLA_U64, },
 	[NFSD_A_SVC_EXPORT_CLIENT] = { .type = NLA_NUL_STRING, },
 	[NFSD_A_SVC_EXPORT_PATH] = { .type = NLA_NUL_STRING, },
@@ -55,6 +56,7 @@ const struct nla_policy nfsd_svc_export_nl_policy[NFSD_A_SVC_EXPORT_FSID + 1] =
 	[NFSD_A_SVC_EXPORT_XPRTSEC] = NLA_POLICY_MASK(NLA_U32, 0x7),
 	[NFSD_A_SVC_EXPORT_FLAGS] = NLA_POLICY_MASK(NLA_U32, 0x3ffff),
 	[NFSD_A_SVC_EXPORT_FSID] = { .type = NLA_S32, },
+	[NFSD_A_SVC_EXPORT_ALLOW_TAGS] = { .type = NLA_STRING, .len = HANDSHAKE_SESSION_TAG_MAX_LEN, },
 };
 
 const struct nla_policy nfsd_version_nl_policy[NFSD_A_VERSION_ENABLED + 1] = {
fs/nfsd/netlink.h +2 -1
index 027e2953db26..098514e7d1bf 100644
@@ -11,6 +11,7 @@
 #include <net/genetlink.h>
 
 #include <uapi/linux/nfsd_netlink.h>
+#include <linux/handshake.h>
 
 /* Common nested types */
 extern const struct nla_policy nfsd_auth_flavor_nl_policy[NFSD_A_AUTH_FLAVOR_FLAGS + 1];
@@ -18,7 +19,7 @@ extern const struct nla_policy nfsd_expkey_nl_policy[NFSD_A_EXPKEY_PATH + 1];
 extern const struct nla_policy nfsd_fslocation_nl_policy[NFSD_A_FSLOCATION_PATH + 1];
 extern const struct nla_policy nfsd_fslocations_nl_policy[NFSD_A_FSLOCATIONS_LOCATION + 1];
 extern const struct nla_policy nfsd_sock_nl_policy[NFSD_A_SOCK_TRANSPORT_NAME + 1];
-extern const struct nla_policy nfsd_svc_export_nl_policy[NFSD_A_SVC_EXPORT_FSID + 1];
+extern const struct nla_policy nfsd_svc_export_nl_policy[NFSD_A_SVC_EXPORT_ALLOW_TAGS + 1];
 extern const struct nla_policy nfsd_version_nl_policy[NFSD_A_VERSION_ENABLED + 1];
 
 int nfsd_nl_rpc_status_get_dumpit(struct sk_buff *skb,
include/uapi/linux/nfsd_netlink.h +1 -0
index c125af0cc6a5..e4bd1fe69fff 100644
@@ -185,6 +185,7 @@ enum {
 	NFSD_A_SVC_EXPORT_XPRTSEC,
 	NFSD_A_SVC_EXPORT_FLAGS,
 	NFSD_A_SVC_EXPORT_FSID,
+	NFSD_A_SVC_EXPORT_ALLOW_TAGS,
 
 	__NFSD_A_SVC_EXPORT_MAX,
 	NFSD_A_SVC_EXPORT_MAX = (__NFSD_A_SVC_EXPORT_MAX - 1)
tools/net/ynl/Makefile.deps +2 -1
index f8850444d240..3a3fc10a77d6 100644
@@ -33,7 +33,8 @@ CFLAGS_netdev:=$(call get_hdr_inc,_LINUX_NETDEV_H,netdev.h)
 CFLAGS_nl80211:=$(call get_hdr_inc,__LINUX_NL802121_H,nl80211.h)
 CFLAGS_nlctrl:=$(call get_hdr_inc,__LINUX_GENERIC_NETLINK_H,genetlink.h) \
 	$(call get_hdr_inc,__LINUX_NETLINK_H,netlink.h)
-CFLAGS_nfsd:=$(call get_hdr_inc,_LINUX_NFSD_NETLINK_H,nfsd_netlink.h)
+CFLAGS_nfsd:=$(call get_hdr_inc,_LINUX_NFSD_NETLINK_H,nfsd_netlink.h) \
+	$(call get_hdr_inc,_LINUX_HANDSHAKE_H,handshake.h)
 CFLAGS_ovpn:=$(call get_hdr_inc,_LINUX_OVPN_H,ovpn.h)
 CFLAGS_ovs_datapath:=$(call get_hdr_inc,__LINUX_OPENVSWITCH_H,openvswitch.h)
 CFLAGS_ovs_flow:=$(call get_hdr_inc,__LINUX_OPENVSWITCH_H,openvswitch.h)
Signature · 1 line
2.56.0
#12 lore raw

Re: [PATCH net-next v3 10/10] NFSD: Add allow_tags to the netlink export interface

Thank you for your contribution! Sashiko AI review found 1 potential issue to consider: - [Medium] nfsd: NLA_STRING policy in fs/nfsd/netlink.c disagrees with nfsd.yaml --

Thank you for your contribution! Sashiko AI review found 1 potential issue to consider:
- [Medium] nfsd: NLA_STRING policy in fs/nfsd/netlink.c disagrees with nfsd.yaml
--

commit 9462d94731ddc9a3ecf082d3129a435c7feed8da
Author: Chuck Lever <[email protected]>

NFSD: Add allow_tags to the netlink export interface

This commit adds the allow-tags attribute to the NFSD svc-export netlink interface, parses and validates session tags in nfsd_nl_parse_one_export(), and reports validation errors through the netlink extended ACK.

Quoted 4 lines
Documentation/netlink/specs/nfsd.yaml +0 -0
index 9ae37bf3ea717..e58856b34b020 100644

[ ... ]

Quoted 10 lines
Documentation/netlink/specs/nfsd.yaml +6 -0
@@ -297,6 +301,12 @@ attribute-sets:
       -
         name: fsid
         type: s32
+      -
+        name: allow-tags
+        type: string
+        checks:
+          max-len: handshake-session-tag-max-len
+        multi-attr: true

[ ... ]

Quoted 4 lines
fs/nfsd/netlink.c +0 -0
index 88a4a4ffcb7fd..146c655757410 100644

[ ... ]

fs/nfsd/netlink.c const struct nla_policy nfsd_svc_export_nl_policy[NFSD_A_SVC_EXPORT_FSID + 1] = · line 60

Quoted 6 lines
fs/nfsd/netlink.c +1 -0
@@ -55,6 +56,7 @@ const struct nla_policy nfsd_svc_export_nl_policy[NFSD_A_SVC_EXPORT_FSID + 1] =
 	[NFSD_A_SVC_EXPORT_XPRTSEC] = NLA_POLICY_MASK(NLA_U32, 0x7),
 	[NFSD_A_SVC_EXPORT_FLAGS] = NLA_POLICY_MASK(NLA_U32, 0x3ffff),
 	[NFSD_A_SVC_EXPORT_FSID] = { .type = NLA_S32, },
+	[NFSD_A_SVC_EXPORT_ALLOW_TAGS] = { .type = NLA_STRING, .len = HANDSHAKE_SESSION_TAG_MAX_LEN, },
 };

[Severity: Medium]
Should nfsd_svc_export_nl_policy[NFSD_A_SVC_EXPORT_ALLOW_TAGS] match the NLA_NUL_STRING policy generated from Documentation/netlink/specs/nfsd.yaml?

fs/nfsd/netlink.c is generated by tools/net/ynl/ynl-regen.sh, which emits NLA_NUL_STRING for string attributes without unterminated-ok:

tools/net/ynl/pyynl/ynl_gen_c.py:TypeString.attr_policy()
if self.checks.get('unterminated-ok', False):
policy = 'NLA_STRING'
else:
policy = 'NLA_NUL_STRING'

Because fs/nfsd/netlink.c was edited to use NLA_STRING here, running ynl-regen.sh will overwrite it with NLA_NUL_STRING and silently change the kernel uAPI validation policy to reject non-NUL-terminated tag payloads.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=10
reviewer
#13 lore raw
In reply to Chuck Lever
Quoted 20 lines · Chuck Lever

NFSD controls remote access to its exports today by IP address and hostname. Neither of these are cryptographically secure. For instance, a MitM can replace the response to a server's plaintext DNS query. As a result, a malicious NFS client can pretend to be a client whose hostname matches one that has access to exports it would otherwise be denied access to.

So instead, we want access-control decisions to be based on cryptographically-vetted objects such as TLS peer-certificates. In-kernel x.509 parsing would duplicate work mature userspace libraries already do, so let's make tlshd evaluate peer certificates in user space.

The certificates are evaluated against admin-defined policy, such as whether the certificate's Issuer matches a particular string. tlshd then marks NFS TLS sessions with those policy matches. Admins also provide a set of policy names on each NFSD export (allow_tags). Computing the intersection with these per-export tags is quick and completely separate from the details of parsing x.509 certificates.

Quoted 3 lines

A yet-to-be-posted patch series introduces tag set support to the exportfs command in nfs-utils. tlshd 1.5.0, current as of this writing, already implements TLS session tagging.

Hi Chuck, all,

FYI, while looking over this patchset I noticed that CI flags shadow len variables in ynl generated C code.

I believe this is a bug in ynl-gen and have posted a patch to try to address it.

- [PATCH net-next] tools: ynl-gen: avoid shadow local variable declarations
https://lore.kernel.org/netdev/[email protected]/

...

maintainer
#14 lore raw
In reply to Chuck Lever
Quoted 20 lines · On Fri, 13 Nov 2026 14:21:58 -0500 Chuck Lever wrote:

NFSD controls remote access to its exports today by IP address and hostname. Neither of these are cryptographically secure. For instance, a MitM can replace the response to a server's plaintext DNS query. As a result, a malicious NFS client can pretend to be a client whose hostname matches one that has access to exports it would otherwise be denied access to.

So instead, we want access-control decisions to be based on cryptographically-vetted objects such as TLS peer-certificates. In-kernel x.509 parsing would duplicate work mature userspace libraries already do, so let's make tlshd evaluate peer certificates in user space.

The certificates are evaluated against admin-defined policy, such as whether the certificate's Issuer matches a particular string. tlshd then marks NFS TLS sessions with those policy matches. Admins also provide a set of policy names on each NFSD export (allow_tags). Computing the intersection with these per-export tags is quick and completely separate from the details of parsing x.509 certificates.

Quoted 3 lines

A yet-to-be-posted patch series introduces tag set support to the exportfs command in nfs-utils. tlshd 1.5.0, current as of this writing, already implements TLS session tagging.

I submitted this for AI review now, I think due to the problem Simon flagged CI didn't just gobble it up automatically. The AI review will not be emailed to the list automatically but hopefully it will be ready here by the time you read this:
https://netdev-ai.bots.linux.dev/sashiko/#/patchset/[email protected]

The patches look reasonable to me, overall. At a high level tho - I don't have any familiarity with "tag" based policies. Are you confident it's flexible and convenient enough to survive as uAPI?

Naively I'd think it creates unnecessarily wide API on both ends (handshake and server). My knee jerk design would be to populate the full policy in the server, and only carry some "policy ID" thru the handshake. Number of policy IDs would equal to the number of tag combinations (which result in different behavior).

Any examples of APIs like the one you're adding working well? Anyone willing to review this series, please??