Skip to contentSkip to conversation

MPTCP KTLS support

RFC Superseded Geliang Tang 3 participants 25 messages May 29 – May 31

Subsystems (5)

v22 of 25 Superseded Latest is v25 →

lore
Share & tools
lore

Geliang Tang

author reviewer
#0 lore raw

v22 cover MPTCP KTLS support

From: Geliang Tang <[email protected]>

v22:
- patches 1-4 (new):
- Add standalone fixes for TLS/TCP ULP issues, according to Sashiko comments on the previous version, and placed at the beginning of the series.
- patch 5 (merge of v21 01+02+05):
- Add sk->sk_socket NULL check in tls_build_proto()
- Update update_sk_prot() to use smp_store_release() for both sk->sk_prot

    and sk->sk_socket->ops

- patch 6 (was v21 03):
- "tls: introduce tls protocol ops structure"
- Add skb_copy_bits callback to struct tls_prot_ops, and initialize it in tls_tcp_ops as skb_copy_bits
- patch 9:
- Add mptcp_skb_copy_bits(), implemented by Gang Yan, to handle fragmented MPTCP skbs when copying TLS record headers
- patch 20:
- Enable multipath testing.
- Depends on: mptcp: address stall under memory pressure, v10

Based-on: <[email protected]>

v21:
- patch 1:
- Rename struct tls_proto to struct tls_prot (including all related functions and variables) for clarity
- Add missing rcu_barrier() in tls_prot_cleanup() to ensure all RCU callbacks complete before module unload
- Fix module reference leak by adding module_put(prot->ops->owner) in the cleanup loop of tls_prot_cleanup()
- patch 2 (new):
- split from patch 1.
- patch 5 (new):
- Add memory barriers in tls_sk_poll() and tls_sw_sock_is_readable() to pair with smp_store_release() in update_sk_prot()
- Replace direct tcp_poll() call with indirect call via

    tls_ctx->prot->ops->poll for proper MPTCP support

- patch 11:
- Expand kernel config: add CONFIG_INET_ESP, CONFIG_INET_ESPINTCP, CONFIG_INET6_ESP, CONFIG_INET6_ESPINTCP to support espintcp ULP testing
- patch 12:
- In ulp_sock_pair(), accept EOPNOTSUPP (in addition to ENOENT) when
setting TCP_ULP fails, as MPTCP sockets may return this error
- patch 16 (new):
- Add TEST_F_TIMEOUT(tls, multi_chunk_sendfile, 240) to prevent timeout
failures on MPTCP configurations
-Link: https://patchwork.kernel.org/project/mptcp/cover/[email protected]/

v20:
- patch 1:
- Use kfree_rcu() instead of synchronous kfree() in tls_proto_cleanup()
and tls_proto_put() to prevent use-after-free for RCU readers
- Add duplicate check under tls_proto_lock before inserting new tls_proto
to list to prevent race condition
- Fix reference counting: drop the initial reference from tls_build_proto()
after successful tls_ctx_create() to prevent refcount leak
- patch 2:
- Replace spin_lock() with spin_lock_bh() in tls_register_prot_ops()
and tls_unregister_prot_ops() to avoid Lockdep splat (softirq context
usage)
- patch 3:
- Add memory barrier ordering in tls_sk_poll() to prevent NULL pointer
dereference during concurrent setsockopt(TCP_ULP)
- Move try_module_get() inside tls_prot_ops_find() while holding the lock
to prevent ops structure from being freed before module reference is
acquired
- patch 8:
- Add missing fallback check __mptcp_check_fallback(msk) before calling mptcp_setsockopt_tcp_ulp() to prevent TLS being attached to fallback
sockets
- Re-check fallback status inside mptcp_setsockopt_tcp_ulp() after acquiring socket lock to eliminate TOCTOU race
- patch 10:
- Remove unused _metadata parameter from is_mptcp_enable()
-Link: https://patchwork.kernel.org/project/mptcp/cover/[email protected]/

v19:
- Remove RCU from tls_proto and tls_prot_ops management
Replace RCU with spinlock (tls_proto_lock) for all list operations.
Remove rcu_head from struct tls_proto, eliminate call_rcu() and
synchronize_rcu() calls. Use list_del + kfree directly in
tls_proto_cleanup() and tls_proto_put().
- Add memory barrier in update_sk_prot()
Use smp_store_release() when setting sk->sk_socket->ops to ensure it is
not visible before icsk_ulp_data. Add explanatory comment to prevent
NULL pointer dereference in concurrent tls_sk_poll() due to CPU
reordering.
- Add module owner field to tls_prot_ops
Add .owner = THIS_MODULE to tls_tcp_ops (and tls_mptcp_ops in MPTCP
side). Add try_module_get(ops->owner) in tls_build_proto() before
caching ops pointer. Add module_put(proto->ops->owner) in
tls_proto_put() when releasing proto.
- Fix tls_toe_sk_destruct use-after-free
Restore sk->sk_prot before calling original destructor to prevent UAF
when __sk_destruct() accesses sk->sk_prot->destroy.
- Optimize indirect call performance
Delay inq and get_skb_seq indirect calls in tls_read_flush_backlog()
and tls_rx_msg_size() to avoid unnecessary retpoline overhead on fast
path.
- Fix shutdown_reuse test flakiness
Replace MPTCP-specific wait with generic bind retry loop. Retry bind()
on EINVAL up to 1000 times (1 second) to handle asynchronous state
transitions for both TCP and MPTCP.
- Remove redundant tls_ctx->proto->ops NULL checks
Rely on memory barrier to guarantee ordering, eliminate unnecessary
NULL checks that could introduce other issues.
- Link: https://patchwork.kernel.org/project/mptcp/cover/[email protected]/

v18:
- add ip_ver field to tls_proto and pass it to tls_proto_find to
distinguish IPv4/IPv6 instances
- convert tls_proto_mutex to spinlock, and convert tls_proto_cleanup
to use call_rcu and rcu_barrier for softirq context compatibility
- add tls_proto_put() for atomic refcount decrement + list_del +
call_rcu, and simplify tls_ctx_free and tls_init error paths to use it
- use GFP_ATOMIC for tls_proto allocation in tls_build_proto
- reorder tls_register cleanup labels to unwind correctly on error
- implement new mptcp_inq
- Link: https://patchwork.kernel.org/project/mptcp/cover/[email protected]/

v17:
- remove owner from tls_prot_ops since both TCP and MPTCP are
built-in and cannot be unloaded
- add rcu_head to tls_proto for delayed freeing
- replace synchronize_rcu with call_rcu in tls_ctx_free
- remove module refcounting (try_module_get / module_put) from
tls_build_proto and tls_init
- simplify tls_proto_cleanup to directly free all protos without
refcnt check
- change tls_proto refcnt from 2 to 1 (only socket reference, no
list reference)
- move synchronize_rcu outside spinlock in tls_unregister_prot_ops
- use kzalloc_obj instead of kzalloc to fix the checkpatch warning
- update commit logs
- Link: https://patchwork.kernel.org/project/mptcp/cover/[email protected]/

v16:
- drop rcu_head from struct tls_proto, use refcnt for lifecycle
management.
- add back TLS_NUM_PROTS to handle IPv4/IPv6 separately.
- add .owner field to tls_tcp_ops and tls_mptcp_ops (THIS_MODULE).
- add module refcounting (try_module_get / module_put) in tls_build_proto
and tls_init.
- add missing NULL check for tls_ctx->proto->ops in tls_sk_poll.
- add RCU read lock protection in tls_register_prot_ops.
- add error handling for tls_register_prot_ops calls in tls_register
(with rollback on failure).
- adjust MPTCP cleanup: move tcp_cleanup_ulp from mptcp_destroy_common
to mptcp_destroy.
- remove increase_rlimit from selftest and fix fd check.
- Link: https://patchwork.kernel.org/project/mptcp/cover/[email protected]/

v15:
- patch 1: add proto parameter for tls_toe_bypass.
- patch 1: add a proto null-check in update_sk_prot.
- patch 1: hold mutex_lock in tls_proto_cleanup.
- patch 14: raise the limit of file descriptor values to 4096 to avoid
test failures.
- Link: https://patchwork.kernel.org/project/mptcp/cover/[email protected]/

v14:
- address review comments from sashiko
- patch 1: add rcu for tls_proto, add tls_proto_cleanup.
- patch 2: add unregister helper.
- patch 3: add tls_prot_ops pointer to tls_proto, instead of
tls_context
- patch 5: update mptcp_get_skb_seq, using map_seq - offset, then the
patch "tls: add skb offset check for mptcp" can be dropped.
- patch 7: check len < 0.
- patch 8: call tcp_cleanup_ulp in mptcp_destroy_common.
- patch 9: replace all "tls" as "espintcp" in sock_test_tcpulp.
- patch 10: add is_mptcp_enable helper.
- Link: https://patchwork.kernel.org/project/mptcp/cover/[email protected]/

v13:
- patch 1: Add new patch "add per-protocol cache" to address AI review.
- patch 2: Hold RCU read lock in tls_prot_ops_find().
- patch 3: Set icsk_ulp_data to NULL in error path.
- patch 6: Use spin_is_locked() instead of lockdep_is_held() to fix
build errors.
- patch 9: Drop tcp_sock_set_ulp().
- patch 11: Remove the "return" statement in ulp_sock_pair and check
the return values of socket().
- patch 14: Update wait_for_tcp_close().
- patch 16: Add a max argument to init() and set it to '0' to disable
multipath testing, so that this series does not depend on the "mptcp:
fix stall because of data_ready" series. Multipath testing will be
re-enabled together with that series later, as a squash-to patch.
- Link: https://patchwork.kernel.org/project/mptcp/cover/[email protected]/

v12:
- Thanks for the help from Paolo and Gang Yan, I finally solved the
deadlock issue in read_sock. As a result, the patch "mptcp: avoid
sleeping in read_sock path under softirq" in v11 has been dropped,
and instead a lock_is_held interface has been added to struct
tls_prot_ops. When MPTCP implements this interface, it not only
checks sock_owned_by_user_nocheck(sk) as TCP does, but also needs
to check whether the MPTCP data lock is held.
- Update selftests to make them more stable.
- Fix shellcheck errors for the selftests.
- Link: https://patchwork.kernel.org/project/mptcp/cover/[email protected]/

v11:
- Fix memory leak errors reported by CI. In v10, these occurred in
the shutdown_reuse test and "usleep(500000)" caused the memory
leaks. In v11, a dedicated helper wait_for_tcp_close() has been
added to provide an appropriate delay.
- Drop the code that used mptcp_data_trylock() in mptcp_move_skbs()
to fix a deadlock issue, as that deadlock no longer occurs in v11.
- Do not add "mptcp" variable for the "tls_err" tests, adding it for
the "tls" tests is sufficient.
- No longer increase timeout values for poll/epoll tests, as they
are no longer needed.
- Add ns1 definition in mptcp_tls.sh to fix "ns1 is referenced but
not assigned" error.
- Link: https://patchwork.kernel.org/project/mptcp/cover/[email protected]/

v10:
- Address comments by ai review:
- patch 2: call tls_ctx_free(sk, ctx) and clear icsk_ulp_data before
goto out.
- patch 3: update commit log as "validate each SKB's offset except
the first".
- patch 5: add sock_owned_by_user() checks.
- patch 7: disable device offload for MPTCP sockets.
- patch 9: use TCP_ULP_NAME_MAX in mptcp_setsockopt_tcp_ulp(), drop
SOL_TLS in mptcp_supported_sockopt().
- Make .get_skb_off optional instead of mandatory, TCP does not need
to define it.
- Test "espintcp" ULP instead of "smc" in patch 10. "smc" ULP is
removed recently.
- With Gang Yan's "mptcp: fix stall because of data_ready" v3, mptcp
tls selftests can run without failures. Now add them in this set.
- Link: https://patchwork.kernel.org/project/mptcp/cover/[email protected]/

v9:
- add a new patch to "add MPTCP SKB offset check in strp queue walk",
thanks to Gang Yan for the fix.
- add a new patch to "avoid deadlocks in read_sock path", replacing the "in_softirq()" check used in v8.
- update the selftests.
- Link: https://patchwork.kernel.org/project/mptcp/cover/[email protected]/

v8:
- do not hold tls_prot_ops_lock in tls_init(); otherwise, a deadlock
occurs.
- change return value of mptcp_stream_is_readable() as 'bool' to fix the
"expected restricted __poll_t" warning reported by CI.
- fixed other CI checkpatch warnings regarding excessively long lines.
- Link: https://patchwork.kernel.org/project/mptcp/cover/[email protected]/

v7:
- Passing an MPTCP socket to tcp_sock_rate_check_app_limited() causes a
crash. In v7, an MPTCP version of check_app_limited() is implemented,
which calls tcp_sock_rate_check_app_limited() for each subflow.
- Register tls_tcp_ops and tls_mptcp_ops in tls_register() rather than in tls_init().
- Set ctx->ops in tls_init() instead of in do_tls_setsockopt_conf().
- Keep tls_device.c unchanged. MPTCP TLS_HW mode has not been implemented
yet, so EOPNOTSUPP is returned in this case.
- Also add TCP TLS tests in mptcp_join.sh.
- Link: https://patchwork.kernel.org/project/mptcp/cover/[email protected]/

v6:
- register each ops as Matt suggested.
- drop sk_is_msk().
- add tcp_sock_get_ulp/tcp_sock_set_ulp helpers.
- set another ULP in sock_test_tcpulp as Matt suggested.
- add tls tests using multiple subflows in mptcp_join.sh.
- Link: https://patchwork.kernel.org/project/mptcp/cover/[email protected]/

v5:
- As suggested by Mat and Matt, this set introduces struct tls_prot_ops
for TLS.
- Includes Gang Yan's patches to add MPTCP support to the TLS selftests.
- Link: https://patchwork.kernel.org/project/mptcp/cover/[email protected]/

v4:
- split "tls: add MPTCP protocol support" into smaller, more
focused patches.
- a new mptcp_inq helper has been implemented instead of directly
using mptcp_inq_hint to fix the issue mentioned in [1].
- add sk_is_msk helper.
- the 'expect' parameter will no longer be added to sock_test_tcpulp.
Instead, SOCK_TEST_TCPULP items causing the tests failure will be
directly removed.
- remove the "TCP KTLS" tests, keeping only the MPTCP-related ones.
- Link: https://patchwork.kernel.org/project/mptcp/cover/[email protected]/

[1]
https://patchwork.kernel.org/project/mptcp/patch/ce74452f4c095a1761ef493b767b4bd9f9c14359.1764333805.git.tanggeliang@kylinos.cn/

v3:
- mptcp_read_sock() and mptcp_poll() are not exported, as mptcp_sockopt
test does not use read_sock/poll interfaces. They will be exported when
new tests are added in the future.
- call mptcp_inq_hint in tls_device_rx_resync_new_rec(), tls_device_core_ctrl_rx_resync() and tls_read_flush_backlog() too.
- update selftests.
- Link: https://patchwork.kernel.org/project/mptcp/cover/[email protected]/

v2:
- fix disconnect.
- update selftests.

This series adds KTLS support for MPTCP. Since the ULP of msk is not being used, ULP KTLS can be directly configured onto msk without affecting its communication.

Gang Yan (1):
mptcp: update mptcp_check_readable helper

Geliang Tang (19):
tls: add clone callback to prevent NULL accept crash
tls: init mixed SW/TOE proto to prevent NULL call
tls: add ESTABLISHED state check for TOE bypass
tcp: add socket lock to TCP_ULP getsockopt
tls: add per-protocol cache for MPTCP support
tls: introduce tls protocol ops structure
tls: store protocol ops pointer in tls_proto
mptcp: implement mptcp-specific tls protocol ops
tls: disable device offload for mptcp sockets
mptcp: update ulp getsockopt for tls support
mptcp: enable ulp setsockopt for tls support
selftests: mptcp: connect: use espintcp for ulp test
selftests: tls: add mptcp variant for testing
selftests: tls: increase pollin timeouts for mptcp
selftests: tls: increase nonblocking data size for mptcp
selftests: tls: retry bind on EINVAL in shutdown_reuse
selftests: tls: set timeout for multi_chunk_sendfile
selftests: tls: add mptcp test cases
selftests: mptcp: cover mptcp tls tests

 include/linux/tcp.h                           |   2 +
 include/net/mptcp.h                           |   2 +
 include/net/tcp.h                             |   1 +
 include/net/tls.h                             |  35 ++
 include/net/tls_toe.h                         |   2 +-
 net/ipv4/tcp.c                                |  53 ++-
 net/mptcp/protocol.c                          | 161 ++++++++-
 net/mptcp/protocol.h                          |   1 +
 net/mptcp/sockopt.c                           |  60 +++-
 net/tls/tls.h                                 |   2 +-
 net/tls/tls_device.c                          |   6 +
 net/tls/tls_main.c                            | 317 +++++++++++++++---
 net/tls/tls_strp.c                            |  33 +-
 net/tls/tls_sw.c                              |  27 +-
 net/tls/tls_toe.c                             |   6 +-
 tools/testing/selftests/net/mptcp/.gitignore  |   1 +
 tools/testing/selftests/net/mptcp/Makefile    |   2 +
 tools/testing/selftests/net/mptcp/config      |   9 +
 .../selftests/net/mptcp/mptcp_connect.c       |   4 +-
 .../testing/selftests/net/mptcp/mptcp_tls.sh  |  61 ++++
 tools/testing/selftests/net/mptcp/tls.c       |   1 +
 tools/testing/selftests/net/tls.c             | 159 ++++++++-
 22 files changed, 840 insertions(+), 105 deletions(-)
 create mode 100755 tools/testing/selftests/net/mptcp/mptcp_tls.sh
 create mode 120000 tools/testing/selftests/net/mptcp/tls.c
Signature · 1 line
2.53.0
author reviewer
#1 lore raw

RFC v22 1/20 tls: add clone callback to prevent NULL accept crash

From: Geliang Tang <[email protected]>

When a TLS ULP context is attached to a listening socket (e.g., by calling setsockopt(TCP_ULP, "tls") before accept()), the subsequent accept() triggers inet_clone_ulp() which accesses ulp_ops->clone. Since tcp_tls_ulp_ops lacked a .clone member, this resulted in a NULL pointer access and kernel panic.

Add an empty tls_clone() function and assign it to the .clone field. The existing ESTABLISHED state check in tls_init() already prevents TLS on listen sockets, so this serves as a safety net for any remaining paths.

---
 net/tls/tls_main.c | 7 +++++++
 1 file changed, 7 insertions(+)
net/tls/tls_main.c +7 -0
index fd39acf41a61..51ae8925d891 100644
@@ -1082,6 +1082,12 @@ static int tls_init(struct sock *sk)
 	return rc;
 }
 
+static void tls_clone(const struct request_sock *req,
+		      struct sock *newsk,
+		      const gfp_t priority)
+{
+}
+
 static void tls_update(struct sock *sk, struct proto *p,
 		       void (*write_space)(struct sock *sk))
 {
@@ -1231,6 +1237,7 @@ static struct tcp_ulp_ops tcp_tls_ulp_ops __read_mostly = {
 	.name			= "tls",
 	.owner			= THIS_MODULE,
 	.init			= tls_init,
+	.clone			= tls_clone,
 	.update			= tls_update,
 	.get_info		= tls_get_info,
 	.get_info_size		= tls_get_info_size,
Signature · 1 line
2.53.0
author reviewer
#2 lore raw

RFC v22 2/20 tls: init mixed SW/TOE proto to prevent NULL call

In reply to Geliang Tang

From: Geliang Tang <[email protected]>

When a TOE socket falls back to software offload, update_sk_prot() accesses prot->prots[TLS_SW][TLS_HW_RECORD] and proto_ops[TLS_SW][TLS_HW_RECORD]. These entries were never initialized, causing NULL function pointers and kernel panic. Add the missing entries for [TLS_SW][TLS_HW_RECORD] and [TLS_HW_RECORD][TLS_SW] in both build_protos() and build_proto_ops().

---
 net/tls/tls_main.c | 8 ++++++++
 1 file changed, 8 insertions(+)
net/tls/tls_main.c +8 -0
index 51ae8925d891..6d1272dc3c34 100644
@@ -965,6 +965,9 @@ static void build_proto_ops(struct proto_ops ops[TLS_NUM_CONFIG][TLS_NUM_CONFIG]
 #endif
 #ifdef CONFIG_TLS_TOE
 	ops[TLS_HW_RECORD][TLS_HW_RECORD] = *base;
+
+	ops[TLS_SW][TLS_HW_RECORD] = ops[TLS_SW][TLS_SW];
+	ops[TLS_HW_RECORD][TLS_SW] = ops[TLS_BASE][TLS_SW];
 #endif
 }
 
@@ -1041,6 +1044,11 @@ static void build_protos(struct proto prot[TLS_NUM_CONFIG][TLS_NUM_CONFIG],
 	prot[TLS_HW_RECORD][TLS_HW_RECORD] = *base;
 	prot[TLS_HW_RECORD][TLS_HW_RECORD].hash		= tls_toe_hash;
 	prot[TLS_HW_RECORD][TLS_HW_RECORD].unhash	= tls_toe_unhash;
+
+	prot[TLS_SW][TLS_HW_RECORD] = prot[TLS_SW][TLS_SW];
+	prot[TLS_HW_RECORD][TLS_SW] = prot[TLS_BASE][TLS_SW];
+	prot[TLS_SW][TLS_HW_RECORD].hash = NULL;
+	prot[TLS_HW_RECORD][TLS_SW].unhash = NULL;
 #endif
 }
 
Signature · 1 line
2.53.0
author reviewer
#3 lore raw

RFC v22 3/20 tls: add ESTABLISHED state check for TOE bypass

In reply to Geliang Tang

From: Geliang Tang <[email protected]>

When CONFIG_TLS_TOE is enabled, tls_init() calls tls_toe_bypass() before checking the socket state. This allows a listening socket (TCP_LISTEN) to attach a TOE TLS context via setsockopt(TCP_ULP, "tls") after listen().

Later when accept() clones the socket, inet_clone_ulp() dereferences the ULP's clone callback (which is NULL for TLS), causing a NULL pointer access and kernel panic.

Move the state check before tls_toe_bypass() to ensure TOE bypass is only attempted on established sockets. This prevents unsafe attachment to listen sockets and aligns TOE behavior with the existing restriction applied to software TLS.

---
 net/tls/tls_main.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
net/tls/tls_main.c +5 -5
index 6d1272dc3c34..8809b056290a 100644
@@ -1059,11 +1059,6 @@ static int tls_init(struct sock *sk)
 
 	tls_build_proto(sk);
 
-#ifdef CONFIG_TLS_TOE
-	if (tls_toe_bypass(sk))
-		return 0;
-#endif
-
 	/* The TLS ulp is currently supported only for TCP sockets
 	 * in ESTABLISHED state.
 	 * Supporting sockets in LISTEN state will require us
@@ -1073,6 +1068,11 @@ static int tls_init(struct sock *sk)
 	if (sk->sk_state != TCP_ESTABLISHED)
 		return -ENOTCONN;
 
+#ifdef CONFIG_TLS_TOE
+	if (tls_toe_bypass(sk))
+		return 0;
+#endif
+
 	/* allocate tls context */
 	write_lock_bh(&sk->sk_callback_lock);
 	ctx = tls_ctx_create(sk);
Signature · 1 line
2.53.0
author reviewer
#4 lore raw

RFC v22 4/20 tcp: add socket lock to TCP_ULP getsockopt

In reply to Geliang Tang

From: Geliang Tang <[email protected]>

When handling TCP_ULP getsockopt, do_tcp_getsockopt() accesses icsk->icsk_ulp_ops without holding the socket lock. A concurrent connect(AF_UNSPEC) (or other operation that calls tcp_cleanup_ulp()) can set icsk_ulp_ops to NULL and release the module reference. If this occurs between the NULL check and the dereference of ->name, the kernel may access a NULL pointer or a stale pointer, leading to a crash or use-after-free.

Extract the TCP_ULP getsockopt logic into a helper tcp_sock_get_ulp() and wrap the call with lock_sock()/release_sock() to ensure the ULP operation is protected against concurrent modification.

---
 net/ipv4/tcp.c | 44 +++++++++++++++++++++++++++++---------------
 1 file changed, 29 insertions(+), 15 deletions(-)
net/ipv4/tcp.c +29 -15
index a058f350a759..c01f97d5dfe8 100644
@@ -4481,6 +4481,28 @@ struct sk_buff *tcp_get_timestamping_opt_stats(const struct sock *sk,
 	return stats;
 }
 
+static int tcp_sock_get_ulp(struct sock *sk, sockptr_t optval,
+			    sockptr_t optlen)
+{
+	struct inet_connection_sock *icsk = inet_csk(sk);
+	int len;
+
+	if (copy_from_sockptr(&len, optlen, sizeof(int)))
+		return -EFAULT;
+	len = min_t(unsigned int, len, TCP_ULP_NAME_MAX);
+	if (!icsk->icsk_ulp_ops) {
+		len = 0;
+		if (copy_to_sockptr(optlen, &len, sizeof(int)))
+			return -EFAULT;
+		return 0;
+	}
+	if (copy_to_sockptr(optlen, &len, sizeof(int)))
+		return -EFAULT;
+	if (copy_to_sockptr(optval, icsk->icsk_ulp_ops->name, len))
+		return -EFAULT;
+	return 0;
+}
+
 int do_tcp_getsockopt(struct sock *sk, int level,
 		      int optname, sockptr_t optval, sockptr_t optlen)
 {
@@ -4589,22 +4611,14 @@ int do_tcp_getsockopt(struct sock *sk, int level,
 			return -EFAULT;
 		return 0;
 
-	case TCP_ULP:
-		if (copy_from_sockptr(&len, optlen, sizeof(int)))
-			return -EFAULT;
-		len = min_t(unsigned int, len, TCP_ULP_NAME_MAX);
-		if (!icsk->icsk_ulp_ops) {
-			len = 0;
-			if (copy_to_sockptr(optlen, &len, sizeof(int)))
-				return -EFAULT;
-			return 0;
-		}
-		if (copy_to_sockptr(optlen, &len, sizeof(int)))
-			return -EFAULT;
-		if (copy_to_sockptr(optval, icsk->icsk_ulp_ops->name, len))
-			return -EFAULT;
-		return 0;
+	case TCP_ULP: {
+		int err;
 
+		lock_sock(sk);
+		err = tcp_sock_get_ulp(sk, optval, optlen);
+		release_sock(sk);
+		return err;
+	}
 	case TCP_FASTOPEN_KEY: {
 		u64 key[TCP_FASTOPEN_KEY_BUF_LENGTH / sizeof(u64)];
 		unsigned int key_len;
Signature · 1 line
2.53.0
author reviewer
#5 lore raw

RFC v22 5/20 tls: add per-protocol cache for MPTCP support

In reply to Geliang Tang

From: Geliang Tang <[email protected]>

The TLS ULP uses global static arrays to cache base protocol operations. When MPTCP sockets enable TLS, they overwrite these global caches when building for MPTCP, causing active TCP TLS sockets to use MPTCP-specific ops. This leads to type confusion and kernel panics.

Fix by replacing the global caches with a per-protocol linked list. Each protocol (TCP, MPTCP, etc.) now has its own cached operations, stored in struct tls_prot and referenced from tls_context.

Add a struct tls_prot *prot parameter to tls_ctx_create() and tls_toe_bypass(), and store the pointer in tls_context. This allows protocol-specific TLS operation tables (e.g., for MPTCP) to be passed down.

Also add WRITE_ONCE(sk->sk_prot, ctx->sk_proto) in tls_toe_sk_destruct() to restore the original socket protocol before freeing the TLS context. This is necessary because the socket may have been switched to a TLS protocol variant; on destruction we must revert it to avoid use-after-free or stale function pointers.

Add smp_rmb() barriers in tls_sk_poll() and tls_sw_sock_is_readable() to ensure proper ordering when reading the TLS context. These pair with the smp_store_release() in update_sk_prot().

Also add a NULL check for tls_ctx in tls_sw_sock_is_readable() to safely return false when the context is not yet initialized.

---
 include/net/tls.h     |  12 ++++
 include/net/tls_toe.h |   2 +-
 net/tls/tls.h         |   2 +-
 net/tls/tls_main.c    | 158 +++++++++++++++++++++++++++++++-----------
 net/tls/tls_sw.c      |  18 ++++-
 net/tls/tls_toe.c     |   6 +-
 6 files changed, 151 insertions(+), 47 deletions(-)
include/net/tls.h +12 -0
index ebd2550280ae..7ff4d4566d41 100644
@@ -220,6 +220,16 @@ struct tls_prot_info {
 	u16 tail_size;
 };
 
+struct tls_prot {
+	struct rcu_head			rcu;
+	refcount_t			refcnt;
+	struct list_head		list;
+	int				ip_ver;
+	const struct proto		*prot;
+	struct proto prots[TLS_NUM_CONFIG][TLS_NUM_CONFIG];
+	struct proto_ops proto_ops[TLS_NUM_CONFIG][TLS_NUM_CONFIG];
+};
+
 struct tls_context {
 	/* read-only cache line */
 	struct tls_prot_info prot_info;
@@ -257,6 +267,8 @@ struct tls_context {
 	struct proto *sk_proto;
 	struct sock *sk;
 
+	struct tls_prot *prot;
+
 	void (*sk_destruct)(struct sock *sk);
 
 	union tls_crypto_context crypto_send;
include/net/tls_toe.h +1 -1
index b3aa7593ce2c..f1de7d2498cf 100644
@@ -69,7 +69,7 @@ struct tls_toe_device {
 	struct kref kref;
 };
 
-int tls_toe_bypass(struct sock *sk);
+int tls_toe_bypass(struct sock *sk, struct tls_prot *prot);
 int tls_toe_hash(struct sock *sk);
 void tls_toe_unhash(struct sock *sk);
 
net/tls/tls.h +1 -1
index 12f44cb649c9..b9a41e1b8f8c 100644
@@ -136,7 +136,7 @@ struct tls_rec {
 int __net_init tls_proc_init(struct net *net);
 void __net_exit tls_proc_fini(struct net *net);
 
-struct tls_context *tls_ctx_create(struct sock *sk);
+struct tls_context *tls_ctx_create(struct sock *sk, struct tls_prot *prot);
 void tls_ctx_free(struct sock *sk, struct tls_context *ctx);
 void update_sk_prot(struct sock *sk, struct tls_context *ctx);
 
net/tls/tls_main.c +117 -41

240 lines not on this page · raw message

net/tls/tls_sw.c +16 -2
index 964ebc268ee4..8c5eeb92d017 100644
@@ -2450,11 +2450,25 @@ int tls_sw_read_sock(struct sock *sk, read_descriptor_t *desc,
 
 bool tls_sw_sock_is_readable(struct sock *sk)
 {
-	struct tls_context *tls_ctx = tls_get_ctx(sk);
-	struct tls_sw_context_rx *ctx = tls_sw_ctx_rx(tls_ctx);
+	struct tls_sw_context_rx *ctx;
+	struct tls_context *tls_ctx;
 	bool ingress_empty = true;
 	struct sk_psock *psock;
 
+	/* Paired with smp_store_release() in update_sk_prot()
+	 * for sk->sk_prot. Orders the read of icsk_ulp_data
+	 * after the read of sk->sk_prot in sk_is_readable().
+	 * Prevents seeing a new sk->sk_prot but a stale
+	 * icsk_ulp_data.
+	 */
+	smp_rmb();
+
+	tls_ctx = tls_get_ctx(sk);
+	if (!tls_ctx)
+		return false;
+
+	ctx = tls_sw_ctx_rx(tls_ctx);
+
 	rcu_read_lock();
 	psock = sk_psock(sk);
 	if (psock)
net/tls/tls_toe.c +4 -2
index 825669e1ab47..e92853c106da 100644
@@ -48,13 +48,15 @@ static void tls_toe_sk_destruct(struct sock *sk)
 	struct inet_connection_sock *icsk = inet_csk(sk);
 	struct tls_context *ctx = tls_get_ctx(sk);
 
+	WRITE_ONCE(sk->sk_prot, ctx->sk_proto);
+
 	ctx->sk_destruct(sk);
 	/* Free ctx */
 	rcu_assign_pointer(icsk->icsk_ulp_data, NULL);
 	tls_ctx_free(sk, ctx);
 }
 
-int tls_toe_bypass(struct sock *sk)
+int tls_toe_bypass(struct sock *sk, struct tls_prot *prot)
 {
 	struct tls_toe_device *dev;
 	struct tls_context *ctx;
@@ -63,7 +65,7 @@ int tls_toe_bypass(struct sock *sk)
 	spin_lock_bh(&device_spinlock);
 	list_for_each_entry(dev, &device_list, dev_list) {
 		if (dev->feature && dev->feature(dev)) {
-			ctx = tls_ctx_create(sk);
+			ctx = tls_ctx_create(sk, prot);
 			if (!ctx)
 				goto out;
 
Signature · 1 line
2.53.0
author reviewer
#6 lore raw

RFC v22 6/20 tls: introduce tls protocol ops structure

In reply to Geliang Tang

From: Geliang Tang <[email protected]>

To extend MPTCP support based on TCP TLS, a tls_prot_ops structure has been introduced for TLS, encapsulating TCP-specific helpers within this structure.

Add registering, validating and finding functions for this structure to add, validate and find a tls_prot_ops on the global list tls_prot_ops_list.

Register TCP-specific structure tls_tcp_ops in tls_register().

---
 include/net/tls.h  |  22 ++++++++++
 net/tls/tls_main.c | 106 ++++++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 127 insertions(+), 1 deletion(-)
include/net/tls.h +22 -0
index 7ff4d4566d41..e13894c054ad 100644
@@ -220,6 +220,28 @@ struct tls_prot_info {
 	u16 tail_size;
 };
 
+struct tls_prot_ops {
+	struct module		*owner;
+	int			protocol;
+	struct list_head	list;
+
+	int (*inq)(struct sock *sk);
+	int (*sendmsg_locked)(struct sock *sk, struct msghdr *msg,
+			      size_t size);
+	struct sk_buff *(*recv_skb)(struct sock *sk, u32 *off);
+	bool (*lock_is_held)(struct sock *sk);
+	int (*read_sock)(struct sock *sk, read_descriptor_t *desc,
+			 sk_read_actor_t recv_actor);
+	void (*read_done)(struct sock *sk, size_t len);
+	u32 (*get_skb_seq)(struct sk_buff *skb);
+	int (*skb_copy_bits)(const struct sk_buff *skb, int offset,
+			     void *to, int len);
+	__poll_t (*poll)(struct file *file, struct socket *sock,
+			 struct poll_table_struct *wait);
+	bool (*epollin_ready)(const struct sock *sk);
+	void (*check_app_limited)(struct sock *sk);
+};
+
 struct tls_prot {
 	struct rcu_head			rcu;
 	refcount_t			refcnt;
net/tls/tls_main.c +105 -1

142 lines not on this page · raw message

Signature · 1 line
2.53.0
author reviewer
#7 lore raw

RFC v22 7/20 tls: store protocol ops pointer in tls_proto

In reply to Geliang Tang

From: Geliang Tang <[email protected]>

A pointer to struct tls_prot_ops, named 'ops', has been added to struct tls_prot. The places originally calling TLS-specific helpers have now been modified to indirectly invoke them via 'ops' pointer in tls_prot.

In tls_build_proto(), prot->ops is assigned either 'tls_mptcp_ops' or 'tls_tcp_ops' based on the socket protocol.

---
 include/net/tls.h  |  1 +
 net/tls/tls_main.c | 19 +++++++++++++++----
 net/tls/tls_strp.c | 33 ++++++++++++++++++++++-----------
 net/tls/tls_sw.c   |  9 ++++++---
 4 files changed, 44 insertions(+), 18 deletions(-)
include/net/tls.h +1 -0
index e13894c054ad..9ea199328070 100644
@@ -248,6 +248,7 @@ struct tls_prot {
 	struct list_head		list;
 	int				ip_ver;
 	const struct proto		*prot;
+	const struct tls_prot_ops	*ops;
 	struct proto prots[TLS_NUM_CONFIG][TLS_NUM_CONFIG];
 	struct proto_ops proto_ops[TLS_NUM_CONFIG][TLS_NUM_CONFIG];
 };
net/tls/tls_main.c +15 -4
index 2d9e10507f4b..3c19cb04dae7 100644
@@ -156,6 +156,7 @@ static void tls_prot_cleanup(void)
 	spin_lock_bh(&tls_prot_lock);
 	list_for_each_entry_safe(prot, tmp, &tls_prot_list, list) {
 		list_del_rcu(&prot->list);
+		module_put(prot->ops->owner);
 		call_rcu(&prot->rcu, tls_prot_free);
 	}
 	spin_unlock_bh(&tls_prot_lock);
@@ -248,13 +249,13 @@ int tls_push_sg(struct sock *sk,
 	ctx->splicing_pages = true;
 	while (1) {
 		/* is sending application-limited? */
-		tcp_rate_check_app_limited(sk);
+		ctx->prot->ops->check_app_limited(sk);
 		p = sg_page(sg);
 retry:
 		bvec_set_page(&bvec, p, size, offset);
 		iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, &bvec, 1, size);
 
-		ret = tcp_sendmsg_locked(sk, &msg, size);
+		ret = ctx->prot->ops->sendmsg_locked(sk, &msg, size);
 
 		if (ret != size) {
 			if (ret > 0) {
@@ -380,6 +381,7 @@ static void tls_prot_put(struct tls_prot *prot)
 		spin_lock_bh(&tls_prot_lock);
 		list_del_rcu(&prot->list);
 		spin_unlock_bh(&tls_prot_lock);
+		module_put(prot->ops->owner);
 		call_rcu(&prot->rcu, tls_prot_free);
 	}
 }
@@ -492,14 +494,14 @@ static __poll_t tls_sk_poll(struct file *file, struct socket *sock,
 	 */
 	smp_rmb();
 
-	mask = tcp_poll(file, sock, wait);
+	tls_ctx = tls_get_ctx(sk);
+	mask = tls_ctx->prot->ops->poll(file, sock, wait);
 
 	state = inet_sk_state_load(sk);
 	shutdown = READ_ONCE(sk->sk_shutdown);
 	if (unlikely(state != TCP_ESTABLISHED || shutdown & RCV_SHUTDOWN))
 		return mask;
 
-	tls_ctx = tls_get_ctx(sk);
 	ctx = tls_sw_ctx_rx(tls_ctx);
 	psock = sk_psock_get(sk);
 
@@ -1060,6 +1062,7 @@ static struct tls_prot *tls_build_proto(struct sock *sk)
 	int ip_ver = sk->sk_family == AF_INET6 ? TLSV6 : TLSV4;
 	struct proto *proto = READ_ONCE(sk->sk_prot);
 	struct tls_prot *prot, *cache;
+	struct tls_prot_ops *ops;
 
 	if (!sk->sk_socket)
 		return NULL;
@@ -1076,8 +1079,16 @@ static struct tls_prot *tls_build_proto(struct sock *sk)
 		return cache;
 	}
 
+	ops = tls_prot_ops_find(sk->sk_protocol);
+	if (!ops || !try_module_get(ops->owner)) {
+		spin_unlock_bh(&tls_prot_lock);
+		kfree(prot);
+		return NULL;
+	}
+
 	prot->ip_ver = ip_ver;
 	prot->prot = proto;
+	prot->ops = ops;
 	refcount_set(&prot->refcnt, 1);
 	build_protos(prot->prots, proto);
 	build_proto_ops(prot->proto_ops,
net/tls/tls_strp.c +22 -11
index c72e88317627..c88cb966649b 100644
@@ -120,6 +120,7 @@ struct sk_buff *tls_strp_msg_detach(struct tls_sw_context_rx *ctx)
 int tls_strp_msg_cow(struct tls_sw_context_rx *ctx)
 {
 	struct tls_strparser *strp = &ctx->strp;
+	struct tls_context *tls_ctx = tls_get_ctx(strp->sk);
 	struct sk_buff *skb;
 
 	if (strp->copy_mode)
@@ -132,7 +133,7 @@ int tls_strp_msg_cow(struct tls_sw_context_rx *ctx)
 	tls_strp_anchor_free(strp);
 	strp->anchor = skb;
 
-	tcp_read_done(strp->sk, strp->stm.full_len);
+	tls_ctx->prot->ops->read_done(strp->sk, strp->stm.full_len);
 	strp->copy_mode = 1;
 
 	return 0;
@@ -376,6 +377,7 @@ static int tls_strp_copyin(read_descriptor_t *desc, struct sk_buff *in_skb,
 
 static int tls_strp_read_copyin(struct tls_strparser *strp)
 {
+	struct tls_context *ctx = tls_get_ctx(strp->sk);
 	read_descriptor_t desc;
 
 	desc.arg.data = strp;
@@ -383,13 +385,14 @@ static int tls_strp_read_copyin(struct tls_strparser *strp)
 	desc.count = 1; /* give more than one skb per call */
 
 	/* sk should be locked here, so okay to do read_sock */
-	tcp_read_sock(strp->sk, &desc, tls_strp_copyin);
+	ctx->prot->ops->read_sock(strp->sk, &desc, tls_strp_copyin);
 
 	return desc.error;
 }
 
 static int tls_strp_read_copy(struct tls_strparser *strp, bool qshort)
 {
+	struct tls_context *ctx = tls_get_ctx(strp->sk);
 	struct skb_shared_info *shinfo;
 	struct page *page;
 	int need_spc, len;
@@ -398,7 +401,7 @@ static int tls_strp_read_copy(struct tls_strparser *strp, bool qshort)
 	 * to read the data out. Otherwise the connection will stall.
 	 * Without pressure threshold of INT_MAX will never be ready.
 	 */
-	if (likely(qshort && !tcp_epollin_ready(strp->sk, INT_MAX)))
+	if (likely(qshort && !ctx->prot->ops->epollin_ready(strp->sk)))
 		return 0;
 
 	shinfo = skb_shinfo(strp->anchor);
@@ -434,12 +437,13 @@ static int tls_strp_read_copy(struct tls_strparser *strp, bool qshort)
 static bool tls_strp_check_queue_ok(struct tls_strparser *strp)
 {
 	unsigned int len = strp->stm.offset + strp->stm.full_len;
+	struct tls_context *ctx = tls_get_ctx(strp->sk);
 	struct sk_buff *first, *skb;
 	u32 seq;
 
 	first = skb_shinfo(strp->anchor)->frag_list;
 	skb = first;
-	seq = TCP_SKB_CB(first)->seq;
+	seq = ctx->prot->ops->get_skb_seq(first);
 
 	/* Make sure there's no duplicate data in the queue,
 	 * and the decrypted status matches.
@@ -449,7 +453,7 @@ static bool tls_strp_check_queue_ok(struct tls_strparser *strp)
 		len -= skb->len;
 		skb = skb->next;
 
-		if (TCP_SKB_CB(skb)->seq != seq)
+		if (ctx->prot->ops->get_skb_seq(skb) != seq)
 			return false;
 		if (skb_cmp_decrypted(first, skb))
 			return false;
@@ -460,11 +464,11 @@ static bool tls_strp_check_queue_ok(struct tls_strparser *strp)
 
 static void tls_strp_load_anchor_with_queue(struct tls_strparser *strp, int len)
 {
-	struct tcp_sock *tp = tcp_sk(strp->sk);
+	struct tls_context *ctx = tls_get_ctx(strp->sk);
 	struct sk_buff *first;
 	u32 offset;
 
-	first = tcp_recv_skb(strp->sk, tp->copied_seq, &offset);
+	first = ctx->prot->ops->recv_skb(strp->sk, &offset);
 	if (WARN_ON_ONCE(!first))
 		return;
 
@@ -483,6 +487,7 @@ static void tls_strp_load_anchor_with_queue(struct tls_strparser *strp, int len)
 
 bool tls_strp_msg_load(struct tls_strparser *strp, bool force_refresh)
 {
+	struct tls_context *ctx = tls_get_ctx(strp->sk);
 	struct strp_msg *rxm;
 	struct tls_msg *tlm;
 
@@ -490,7 +495,8 @@ bool tls_strp_msg_load(struct tls_strparser *strp, bool force_refresh)
 	DEBUG_NET_WARN_ON_ONCE(!strp->stm.full_len);
 
 	if (!strp->copy_mode && force_refresh) {
-		if (unlikely(tcp_inq(strp->sk) < strp->stm.full_len)) {
+		if (unlikely(ctx->prot->ops->inq(strp->sk) <
+			     strp->stm.full_len)) {
 			WRITE_ONCE(strp->msg_ready, 0);
 			memset(&strp->stm, 0, sizeof(strp->stm));
 			return false;
@@ -511,9 +517,10 @@ bool tls_strp_msg_load(struct tls_strparser *strp, bool force_refresh)
 /* Called with lock held on lower socket */
 static int tls_strp_read_sock(struct tls_strparser *strp)
 {
+	struct tls_context *ctx = tls_get_ctx(strp->sk);
 	int sz, inq;
 
-	inq = tcp_inq(strp->sk);
+	inq = ctx->prot->ops->inq(strp->sk);
 	if (inq < 1)
 		return 0;
 
@@ -556,6 +563,8 @@ void tls_strp_check_rcv(struct tls_strparser *strp)
 /* Lower sock lock held */
 void tls_strp_data_ready(struct tls_strparser *strp)
 {
+	struct tls_context *ctx = tls_get_ctx(strp->sk);
+
 	/* This check is needed to synchronize with do_tls_strp_work.
 	 * do_tls_strp_work acquires a process lock (lock_sock) whereas
 	 * the lock held here is bh_lock_sock. The two locks can be
@@ -563,7 +572,7 @@ void tls_strp_data_ready(struct tls_strparser *strp)
 	 * allows a thread in BH context to safely check if the process
 	 * lock is held. In this case, if the lock is held, queue work.
 	 */
-	if (sock_owned_by_user_nocheck(strp->sk)) {
+	if (ctx->prot->ops->lock_is_held(strp->sk)) {
 		queue_work(tls_strp_wq, &strp->work);
 		return;
 	}
@@ -583,10 +592,12 @@ static void tls_strp_work(struct work_struct *w)
 
 void tls_strp_msg_done(struct tls_strparser *strp)
 {
+	struct tls_context *ctx = tls_get_ctx(strp->sk);
+
 	WARN_ON(!strp->stm.full_len);
 
 	if (likely(!strp->copy_mode))
-		tcp_read_done(strp->sk, strp->stm.full_len);
+		ctx->prot->ops->read_done(strp->sk, strp->stm.full_len);
 	else
 		tls_strp_flush_anchor_copy(strp);
 
net/tls/tls_sw.c +6 -3
index 8c5eeb92d017..437662598756 100644
@@ -1987,7 +1987,8 @@ tls_read_flush_backlog(struct sock *sk, struct tls_prot_info *prot,
 		return false;
 
 	max_rec = prot->overhead_size - prot->tail_size + TLS_MAX_PAYLOAD_SIZE;
-	if (done - *flushed_at < SZ_128K && tcp_inq(sk) > max_rec)
+	if (done - *flushed_at < SZ_128K &&
+	    tls_get_ctx(sk)->prot->ops->inq(sk) > max_rec)
 		return false;
 
 	*flushed_at = done;
@@ -2499,7 +2500,8 @@ int tls_rx_msg_size(struct tls_strparser *strp, struct sk_buff *skb)
 	}
 
 	/* Linearize header to local buffer */
-	ret = skb_copy_bits(skb, strp->stm.offset, header, prot->prepend_size);
+	ret = tls_ctx->prot->ops->skb_copy_bits(skb, strp->stm.offset, header,
+						prot->prepend_size);
 	if (ret < 0)
 		goto read_failure;
 
@@ -2530,7 +2532,8 @@ int tls_rx_msg_size(struct tls_strparser *strp, struct sk_buff *skb)
 	}
 
 	tls_device_rx_resync_new_rec(strp->sk, data_len + TLS_HEADER_SIZE,
-				     TCP_SKB_CB(skb)->seq + strp->stm.offset);
+				     tls_ctx->prot->ops->get_skb_seq(skb) +
+				     strp->stm.offset);
 	return data_len + TLS_HEADER_SIZE;
 
 read_failure:
Signature · 1 line
2.53.0
author reviewer
#8 lore raw

RFC v22 8/20 mptcp: update mptcp_check_readable helper

In reply to Geliang Tang

From: Gang Yan <[email protected]>

This patch makes mptcp_check_readable() aligned with TCP, and renames it to mptcp_stream_is_readable(). It will be used in the case of KTLS, because 'prot' will be modified, tls_sw_sock_is_readable() is expected to be called from prot->sock_is_readable().

---
 net/mptcp/protocol.c | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)
net/mptcp/protocol.c +6 -3
index 7fe618e22d1b..b7976a551af0 100644
@@ -3401,9 +3401,11 @@ void __mptcp_unaccepted_force_close(struct sock *sk)
 	__mptcp_destroy_sock(sk);
 }
 
-static __poll_t mptcp_check_readable(struct sock *sk)
+static bool mptcp_stream_is_readable(struct sock *sk)
 {
-	return mptcp_epollin_ready(sk) ? EPOLLIN | EPOLLRDNORM : 0;
+	if (mptcp_epollin_ready(sk))
+		return true;
+	return sk_is_readable(sk);
 }
 
 static void mptcp_check_listen_stop(struct sock *sk)
@@ -4467,7 +4469,8 @@ static __poll_t mptcp_poll(struct file *file, struct socket *sock,
 		mask |= EPOLLIN | EPOLLRDNORM | EPOLLRDHUP;
 
 	if (state != TCP_SYN_SENT && state != TCP_SYN_RECV) {
-		mask |= mptcp_check_readable(sk);
+		if (mptcp_stream_is_readable(sk))
+			mask |= EPOLLIN | EPOLLRDNORM;
 		if (shutdown & SEND_SHUTDOWN)
 			mask |= EPOLLOUT | EPOLLWRNORM;
 		else
Signature · 1 line
2.53.0
author reviewer
#9 lore raw

RFC v22 9/20 mptcp: implement mptcp-specific tls protocol ops

In reply to Geliang Tang

From: Geliang Tang <[email protected]>

This patch implements the MPTCP-specific struct tls_prot_ops, named 'tls_mptcp_ops'.

Passing an MPTCP socket to tcp_sock_rate_check_app_limited() can trigger a crash. Here, an MPTCP version of check_app_limited() is implemented, which calls tcp_sock_rate_check_app_limited() for each subflow.

When MPTCP implements lock_is_held interface, it not only checks sock_owned_by_user_nocheck(sk) as TCP does, but also needs to check whether the MPTCP data lock is held. This is required because TLS may call lock_is_held from softirq context with bh_lock_sock held. Checking both conditions ensures TLS always defers to workqueue when the MPTCP data lock is held, avoiding deadlock.

Implement mptcp_skb_copy_bits() to handle fragmented MPTCP skbs when copying TLS record headers.

---
 include/net/mptcp.h  |   2 +
 include/net/tcp.h    |   1 +
 net/ipv4/tcp.c       |   9 ++-
 net/mptcp/protocol.c | 151 +++++++++++++++++++++++++++++++++++++++++--
 net/mptcp/protocol.h |   1 +
 net/tls/tls_main.c   |  13 ++++
 6 files changed, 171 insertions(+), 6 deletions(-)
include/net/mptcp.h +2 -0
index aef2dbeb847b..75a30c70c6e1 100644
@@ -128,6 +128,8 @@ struct mptcp_pm_ops {
 	void (*release)(struct mptcp_sock *msk);
 } ____cacheline_aligned_in_smp;
 
+extern struct tls_prot_ops tls_mptcp_ops;
+
 #ifdef CONFIG_MPTCP
 void mptcp_init(void);
 
include/net/tcp.h +1 -0
index f063eccbbba3..1c8201f69ef1 100644
@@ -849,6 +849,7 @@ static inline int tcp_bound_to_half_wnd(struct tcp_sock *tp, int pktsize)
 
 /* tcp.c */
 void tcp_get_info(struct sock *, struct tcp_info *);
+void tcp_sock_rate_check_app_limited(struct tcp_sock *tp);
 void tcp_rate_check_app_limited(struct sock *sk);
 
 /* Read 'sendfile()'-style from a TCP socket */
net/ipv4/tcp.c +7 -2
index c01f97d5dfe8..413807e1be75 100644
@@ -1097,9 +1097,9 @@ int tcp_sendmsg_fastopen(struct sock *sk, struct msghdr *msg, int *copied,
 }
 
 /* If a gap is detected between sends, mark the socket application-limited. */
-void tcp_rate_check_app_limited(struct sock *sk)
+void tcp_sock_rate_check_app_limited(struct tcp_sock *tp)
 {
-	struct tcp_sock *tp = tcp_sk(sk);
+	struct sock *sk = (struct sock *)tp;
 
 	if (/* We have less than one packet to send. */
 	    tp->write_seq - tp->snd_nxt < tp->mss_cache &&
@@ -1112,6 +1112,11 @@ void tcp_rate_check_app_limited(struct sock *sk)
 		tp->app_limited =
 			(tp->delivered + tcp_packets_in_flight(tp)) ? : 1;
 }
+
+void tcp_rate_check_app_limited(struct sock *sk)
+{
+	tcp_sock_rate_check_app_limited(tcp_sk(sk));
+}
 EXPORT_SYMBOL_GPL(tcp_rate_check_app_limited);
 
 int tcp_sendmsg_locked(struct sock *sk, struct msghdr *msg, size_t size)
net/mptcp/protocol.c +147 -4

184 lines not on this page · raw message

net/mptcp/protocol.h +1 -0
index 2321ad4d845d..1d8fcd742eb9 100644
@@ -380,6 +380,7 @@ struct mptcp_sock {
 
 #define mptcp_data_lock(sk) spin_lock_bh(&(sk)->sk_lock.slock)
 #define mptcp_data_unlock(sk) spin_unlock_bh(&(sk)->sk_lock.slock)
+#define mptcp_data_is_locked(sk) spin_is_locked(&(sk)->sk_lock.slock)
 
 #define mptcp_for_each_subflow(__msk, __subflow)			\
 	list_for_each_entry(__subflow, &((__msk)->conn_list), node)
net/tls/tls_main.c +13 -0
index 3c19cb04dae7..67541b8880d7 100644
@@ -1450,6 +1450,12 @@ static int __init tls_register(void)
 	if (err)
 		goto err_strp;
 
+#ifdef CONFIG_MPTCP
+	err = tls_register_prot_ops(&tls_mptcp_ops);
+	if (err)
+		goto err_tcp;
+#endif
+
 	err = tls_device_init();
 	if (err)
 		goto err_ops;
@@ -1458,6 +1464,10 @@ static int __init tls_register(void)
 
 	return 0;
 err_ops:
+#ifdef CONFIG_MPTCP
+	tls_unregister_prot_ops(&tls_mptcp_ops);
+err_tcp:
+#endif
 	tls_unregister_prot_ops(&tls_tcp_ops);
 err_strp:
 	tls_strp_dev_exit();
@@ -1469,6 +1479,9 @@ static int __init tls_register(void)
 static void __exit tls_unregister(void)
 {
 	tcp_unregister_ulp(&tcp_tls_ulp_ops);
+#ifdef CONFIG_MPTCP
+	tls_unregister_prot_ops(&tls_mptcp_ops);
+#endif
 	tls_unregister_prot_ops(&tls_tcp_ops);
 	tls_prot_cleanup();
 	tls_strp_dev_exit();
Signature · 1 line
2.53.0
author reviewer
#10 lore raw

RFC v22 10/20 tls: disable device offload for mptcp sockets

In reply to Geliang Tang

From: Geliang Tang <[email protected]>

MPTCP TLS hardware offload is not yet implemented. Return -EOPNOTSUPP when attempting to enable device offload on MPTCP sockets.

---
 net/tls/tls_device.c | 6 ++++++
 1 file changed, 6 insertions(+)
net/tls/tls_device.c +6 -0
index 741aef09bfd3..06f45edffb5f 100644
@@ -1074,6 +1074,9 @@ int tls_set_device_offload(struct sock *sk)
 	ctx = tls_get_ctx(sk);
 	prot = &ctx->prot_info;
 
+	if (sk->sk_protocol == IPPROTO_MPTCP)
+		return -EOPNOTSUPP;
+
 	if (ctx->priv_ctx_tx)
 		return -EEXIST;
 
@@ -1196,6 +1199,9 @@ int tls_set_device_offload_rx(struct sock *sk, struct tls_context *ctx)
 	struct net_device *netdev;
 	int rc = 0;
 
+	if (sk->sk_protocol == IPPROTO_MPTCP)
+		return -EOPNOTSUPP;
+
 	if (ctx->crypto_recv.info.version != TLS_1_2_VERSION)
 		return -EOPNOTSUPP;
 
Signature · 1 line
2.53.0
author reviewer
#11 lore raw

RFC v22 11/20 mptcp: update ulp getsockopt for tls support

In reply to Geliang Tang

From: Geliang Tang <[email protected]>

Export tcp_sock_get_ulp() helper and use it in MPTCP to get the ULP name of the MPTCP socket itself, replacing the previous approach that returned the ULP of the first subflow via mptcp_getsockopt_first_sf_only().

Add mptcp_getsockopt_tcp_ulp() which acquires the socket lock before calling the helper, ensuring safe access to icsk_ulp_ops.

---
 include/linux/tcp.h |  2 ++
 net/ipv4/tcp.c      |  4 ++--
 net/mptcp/sockopt.c | 20 ++++++++++++++++++++
 3 files changed, 24 insertions(+), 2 deletions(-)
include/linux/tcp.h +2 -0
index 8a6807082672..662a7de907e6 100644
@@ -653,6 +653,8 @@ void tcp_sock_set_quickack(struct sock *sk, int val);
 int tcp_sock_set_syncnt(struct sock *sk, int val);
 int tcp_sock_set_user_timeout(struct sock *sk, int val);
 int tcp_sock_set_maxseg(struct sock *sk, int val);
+int tcp_sock_get_ulp(struct sock *sk, sockptr_t optval,
+		     sockptr_t optlen);
 
 static inline bool dst_tcp_usec_ts(const struct dst_entry *dst)
 {
net/ipv4/tcp.c +2 -2
index 413807e1be75..ed9f8c27ace1 100644
@@ -4486,8 +4486,8 @@ struct sk_buff *tcp_get_timestamping_opt_stats(const struct sock *sk,
 	return stats;
 }
 
-static int tcp_sock_get_ulp(struct sock *sk, sockptr_t optval,
-			    sockptr_t optlen)
+int tcp_sock_get_ulp(struct sock *sk, sockptr_t optval,
+		     sockptr_t optlen)
 {
 	struct inet_connection_sock *icsk = inet_csk(sk);
 	int len;
net/mptcp/sockopt.c +20 -0
index fcf6feb2a9eb..b7eccade1f90 100644
@@ -1408,6 +1408,25 @@ static int mptcp_put_int_option(struct mptcp_sock *msk, char __user *optval,
 	return 0;
 }
 
+static int mptcp_getsockopt_tcp_ulp(struct sock *sk,
+				    char __user *optval,
+				    int __user *optlen)
+{
+	int ret, len;
+
+	if (copy_from_sockptr(&len, USER_SOCKPTR(optlen), sizeof(int)))
+		return -EFAULT;
+
+	if (len < 0)
+		return -EINVAL;
+
+	lock_sock(sk);
+	ret = tcp_sock_get_ulp(sk, USER_SOCKPTR(optval),
+			       USER_SOCKPTR(optlen));
+	release_sock(sk);
+	return ret;
+}
+
 static int mptcp_getsockopt_sol_tcp(struct mptcp_sock *msk, int optname,
 				    char __user *optval, int __user *optlen)
 {
@@ -1415,6 +1434,7 @@ static int mptcp_getsockopt_sol_tcp(struct mptcp_sock *msk, int optname,
 
 	switch (optname) {
 	case TCP_ULP:
+		return mptcp_getsockopt_tcp_ulp(sk, optval, optlen);
 	case TCP_CONGESTION:
 	case TCP_INFO:
 	case TCP_CC_INFO:
Signature · 1 line
2.53.0
author reviewer
#12 lore raw

RFC v22 12/20 mptcp: enable ulp setsockopt for tls support

In reply to Geliang Tang

From: Geliang Tang <[email protected]>

Allow MPTCP sockets to set the TCP_ULP socket option to enable TLS. Add mptcp_setsockopt_tcp_ulp() which validates the socket state (must not be CLOSE or LISTEN), only accepts "tls" as the ULP name, and then calls tcp_set_ulp().

Include TCP_ULP in the list of supported options in supported_sockopt(), and handle it in setsockopt_sol_tcp() instead of returning -EOPNOTSUPP.

Call tcp_cleanup_ulp() in mptcp_destroy_common() to release ULP module's reference count.

---
 net/mptcp/protocol.c |  1 +
 net/mptcp/sockopt.c  | 40 +++++++++++++++++++++++++++++++++++++++-
 2 files changed, 40 insertions(+), 1 deletion(-)
net/mptcp/protocol.c +1 -0
index f9dbe3a1e57b..26a9b0df5808 100644
@@ -3765,6 +3765,7 @@ static void mptcp_destroy(struct sock *sk)
 	/* allow the following to close even the initial subflow */
 	msk->free_first = 1;
 	mptcp_destroy_common(msk);
+	tcp_cleanup_ulp(sk);
 	sk_sockets_allocated_dec(sk);
 }
 
net/mptcp/sockopt.c +39 -1
index b7eccade1f90..d692bca17adf 100644
@@ -12,6 +12,7 @@
 #include <net/protocol.h>
 #include <net/tcp.h>
 #include <net/mptcp.h>
+#include <net/tls.h>
 #include "protocol.h"
 
 #define MIN_INFO_OPTLEN_SIZE		16
@@ -577,6 +578,7 @@ static bool mptcp_supported_sockopt(int level, int optname)
 		case TCP_FASTOPEN_CONNECT:
 		case TCP_FASTOPEN_KEY:
 		case TCP_FASTOPEN_NO_COOKIE:
+		case TCP_ULP:
 			return true;
 		}
 
@@ -830,6 +832,42 @@ static int mptcp_setsockopt_all_sf(struct mptcp_sock *msk, int level,
 	return ret;
 }
 
+static int mptcp_setsockopt_tcp_ulp(struct sock *sk, sockptr_t optval,
+				    unsigned int optlen)
+{
+	struct mptcp_sock *msk = mptcp_sk(sk);
+	char name[TCP_ULP_NAME_MAX];
+	int err = 0;
+	size_t len;
+	int val;
+
+	if (optlen < 1)
+		return -EINVAL;
+
+	len = min_t(long, TCP_ULP_NAME_MAX - 1, optlen);
+	val = strncpy_from_sockptr(name, optval, len);
+	if (val < 0)
+		return -EFAULT;
+	name[val] = 0;
+
+	if (strcmp(name, "tls"))
+		return -EOPNOTSUPP;
+
+	sockopt_lock_sock(sk);
+	if (__mptcp_check_fallback(msk)) {
+		err = -EOPNOTSUPP;
+		goto out;
+	}
+	if ((1 << sk->sk_state) & (TCPF_CLOSE | TCPF_LISTEN)) {
+		err = -ENOTCONN;
+		goto out;
+	}
+	err = tcp_set_ulp(sk, name);
+out:
+	sockopt_release_sock(sk);
+	return err;
+}
+
 static int mptcp_setsockopt_sol_tcp(struct mptcp_sock *msk, int optname,
 				    sockptr_t optval, unsigned int optlen)
 {
@@ -838,7 +876,7 @@ static int mptcp_setsockopt_sol_tcp(struct mptcp_sock *msk, int optname,
 
 	switch (optname) {
 	case TCP_ULP:
-		return -EOPNOTSUPP;
+		return mptcp_setsockopt_tcp_ulp(sk, optval, optlen);
 	case TCP_CONGESTION:
 		return mptcp_setsockopt_sol_tcp_congestion(msk, optval, optlen);
 	case TCP_DEFER_ACCEPT:
Signature · 1 line
2.53.0
author reviewer
#13 lore raw

RFC v22 13/20 selftests: mptcp: connect: use espintcp for ulp test

In reply to Geliang Tang

From: Geliang Tang <[email protected]>

With KTLS being implemented, "tls" should no longer be used in sock_test_tcpulp(), it breaks mptcp_connect.sh tests. Another ULP name, "espintcp", is set instead in this patch.

---
 tools/testing/selftests/net/mptcp/config          | 4 ++++
 tools/testing/selftests/net/mptcp/mptcp_connect.c | 4 ++--
 2 files changed, 6 insertions(+), 2 deletions(-)
tools/testing/selftests/net/mptcp/config +4 -0
index 59051ee2a986..f48bd5183fb3 100644
@@ -34,3 +34,7 @@ CONFIG_NFT_SOCKET=m
 CONFIG_NFT_TPROXY=m
 CONFIG_SYN_COOKIES=y
 CONFIG_VETH=y
+CONFIG_INET_ESP=y
+CONFIG_INET_ESPINTCP=y
+CONFIG_INET6_ESP=y
+CONFIG_INET6_ESPINTCP=y
tools/testing/selftests/net/mptcp/mptcp_connect.c +2 -2
index cbe573c4ab3a..299a7a02d6f5 100644
@@ -285,11 +285,11 @@ static void sock_test_tcpulp(int sock, int proto, unsigned int line)
 	if (buflen > 0) {
 		if (strcmp(buf, "mptcp") != 0)
 			xerror("unexpected ULP '%s' for proto %d at line %u", buf, proto, line);
-		ret = do_ulp_so(sock, "tls");
+		ret = do_ulp_so(sock, "espintcp");
 		if (ret == 0)
 			X("setsockopt");
 	} else if (proto == IPPROTO_MPTCP) {
-		ret = do_ulp_so(sock, "tls");
+		ret = do_ulp_so(sock, "espintcp");
 		if (ret != -1)
 			X("setsockopt");
 	}
Signature · 1 line
2.53.0
author reviewer
#14 lore raw

RFC v22 14/20 selftests: tls: add mptcp variant for testing

In reply to Geliang Tang

From: Geliang Tang <[email protected]>

To enable easy MPTCP socket creation in MPTCP TLS tests, two protocol parameters (cli_proto and srv_proto) have been added to ulp_sock_pair(). These are passed as third arguments of socket(): 0 creates TCP sockets, IPPROTO_MPTCP creates MPTCP sockets.

A new variant "mptcp" is added both in FIXTURE_VARIANT(tls) to control whether to create MPTCP sockets or not for tests.

Add is_mptcp_enable() helper to check MPTCP support. Used in FIXTURE_SETUP(tls) to skip MPTCP variants when MPTCP is not enabled.

Also accept EOPNOTSUPP when setting TCP_ULP on MPTCP sockets, as they may return this error in addition to ENOENT.

---
 tools/testing/selftests/net/tls.c | 46 +++++++++++++++++++++++++++----
 1 file changed, 40 insertions(+), 6 deletions(-)
tools/testing/selftests/net/tls.c +40 -6
index 9b9a3cb2700d..fed83918cd9d 100644
@@ -26,6 +26,10 @@
 #define TLS_PAYLOAD_MAX_LEN 16384
 #define SOL_TLS 282
 
+#ifndef IPPROTO_MPTCP
+#define IPPROTO_MPTCP 262
+#endif
+
 static int fips_enabled;
 
 struct tls_crypto_info_keys {
@@ -108,8 +112,9 @@ static void memrnd(void *s, size_t n)
 		*byte++ = rand();
 }
 
-static void ulp_sock_pair(struct __test_metadata *_metadata,
-			  int *fd, int *cfd, bool *notls)
+static void __ulp_sock_pair(struct __test_metadata *_metadata,
+			    int *fd, int *cfd, bool *notls,
+			    int cli_proto, int srv_proto)
 {
 	struct sockaddr_in addr;
 	socklen_t len;
@@ -122,8 +127,8 @@ static void ulp_sock_pair(struct __test_metadata *_metadata,
 	addr.sin_addr.s_addr = htonl(INADDR_ANY);
 	addr.sin_port = 0;
 
-	*fd = socket(AF_INET, SOCK_STREAM, 0);
-	sfd = socket(AF_INET, SOCK_STREAM, 0);
+	*fd = socket(AF_INET, SOCK_STREAM, cli_proto);
+	sfd = socket(AF_INET, SOCK_STREAM, srv_proto);
 
 	ret = bind(sfd, &addr, sizeof(addr));
 	ASSERT_EQ(ret, 0);
@@ -143,7 +148,7 @@ static void ulp_sock_pair(struct __test_metadata *_metadata,
 
 	ret = setsockopt(*fd, IPPROTO_TCP, TCP_ULP, "tls", sizeof("tls"));
 	if (ret != 0) {
-		ASSERT_EQ(errno, ENOENT);
+		ASSERT_TRUE(errno == ENOENT || errno == EOPNOTSUPP);
 		*notls = true;
 		printf("Failure setting TCP_ULP, testing without tls\n");
 		return;
@@ -153,6 +158,12 @@ static void ulp_sock_pair(struct __test_metadata *_metadata,
 	ASSERT_EQ(ret, 0);
 }
 
+static void ulp_sock_pair(struct __test_metadata *_metadata,
+			  int *fd, int *cfd, bool *notls)
+{
+	__ulp_sock_pair(_metadata, fd, cfd, notls, 0, 0);
+}
+
 /* Produce a basic cmsg */
 static int tls_send_cmsg(int fd, unsigned char record_type,
 			 void *data, size_t len, int flags)
@@ -310,6 +321,7 @@ FIXTURE_VARIANT(tls)
 	uint16_t tls_version;
 	uint16_t cipher_type;
 	bool nopad, fips_non_compliant;
+	bool mptcp;
 };
 
 FIXTURE_VARIANT_ADD(tls, 12_aes_gcm)
@@ -395,6 +407,23 @@ FIXTURE_VARIANT_ADD(tls, 12_aria_gcm_256)
 	.cipher_type = TLS_CIPHER_ARIA_GCM_256,
 };
 
+static bool is_mptcp_enable(void)
+{
+	char buf[16] = { 0 };
+	ssize_t n;
+	int fd;
+
+	fd = open("/proc/sys/net/mptcp/enabled", O_RDONLY);
+	if (fd < 0)
+		return false;
+
+	n = read(fd, buf, sizeof(buf) - 1);
+	close(fd);
+	if (n <= 0)
+		return false;
+	return (atoi(buf) == 1);
+}
+
 FIXTURE_SETUP(tls)
 {
 	struct tls_crypto_info_keys tls12;
@@ -404,10 +433,15 @@ FIXTURE_SETUP(tls)
 	if (fips_enabled && variant->fips_non_compliant)
 		SKIP(return, "Unsupported cipher in FIPS mode");
 
+	if (variant->mptcp && !is_mptcp_enable())
+		SKIP(return, "no MPTCP support");
+
 	tls_crypto_info_init(variant->tls_version, variant->cipher_type,
 			     &tls12, 0);
 
-	ulp_sock_pair(_metadata, &self->fd, &self->cfd, &self->notls);
+	__ulp_sock_pair(_metadata, &self->fd, &self->cfd, &self->notls,
+			variant->mptcp ? IPPROTO_MPTCP : 0,
+			variant->mptcp ? IPPROTO_MPTCP : 0);
 
 	if (self->notls)
 		return;
Signature · 1 line
2.53.0
author reviewer
#15 lore raw

RFC v22 15/20 selftests: tls: increase pollin timeouts for mptcp

In reply to Geliang Tang

From: Geliang Tang <[email protected]>

MPTCP requires longer timeouts in pollin test due to subflow establishment delays and slower state transitions. Increase timeout values to prevent false failures:

# RUN tls.13_sm4_ccm_mptcp.pollin ...
# tls.c:1411:pollin:Expected poll(&fd, 1, 20) (0) == 1 (1) # tls.c:1412:pollin:Expected fd.revents & POLLIN (0) == 1 (1) # pollin: Test failed
# FAIL tls.13_sm4_ccm_mptcp.pollin
not ok 357 tls.13_sm4_ccm_mptcp.pollin

---
 tools/testing/selftests/net/tls.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
tools/testing/selftests/net/tls.c +3 -2
index fed83918cd9d..5a6c5ee2757f 100644
@@ -1362,6 +1362,7 @@ TEST_F(tls, bidir)
 
 TEST_F(tls, pollin)
 {
+	int timeout = variant->mptcp ? 100 : 20;
 	char const *test_str = "test_poll";
 	struct pollfd fd = { 0, 0, 0 };
 	char buf[10];
@@ -1371,11 +1372,11 @@ TEST_F(tls, pollin)
 	fd.fd = self->cfd;
 	fd.events = POLLIN;
 
-	EXPECT_EQ(poll(&fd, 1, 20), 1);
+	EXPECT_EQ(poll(&fd, 1, timeout), 1);
 	EXPECT_EQ(fd.revents & POLLIN, 1);
 	EXPECT_EQ(recv(self->cfd, buf, send_len, MSG_WAITALL), send_len);
 	/* Test timing out */
-	EXPECT_EQ(poll(&fd, 1, 20), 0);
+	EXPECT_EQ(poll(&fd, 1, timeout), 0);
 }
 
 TEST_F(tls, poll_wait)
Signature · 1 line
2.53.0
author reviewer
#16 lore raw

RFC v22 16/20 selftests: tls: increase nonblocking data size for mptcp

In reply to Geliang Tang

From: Geliang Tang <[email protected]>

Increase the data size in nonblocking tests to accommodate MPTCP's multi-subflow behavior and ensure sufficient data for testing, avoiding the following errors:

# RUN tls.12_aria_gcm_mptcp.nonblocking ...
# tls.c:1534:nonblocking:Expected 0 (0) != eagain (0) # nonblocking: Test failed

---
 tools/testing/selftests/net/tls.c | 3 +++
 1 file changed, 3 insertions(+)
tools/testing/selftests/net/tls.c +3 -0
index 5a6c5ee2757f..64aad661a1b4 100644
@@ -1468,6 +1468,9 @@ TEST_F(tls, nonblocking)
 	int flags;
 	int res;
 
+	if (variant->mptcp)
+		data *= 4;
+
 	flags = fcntl(self->fd, F_GETFL, 0);
 	fcntl(self->fd, F_SETFL, flags | O_NONBLOCK);
 	fcntl(self->cfd, F_SETFL, flags | O_NONBLOCK);
Signature · 1 line
2.53.0
author reviewer
#17 lore raw

RFC v22 17/20 selftests: tls: retry bind on EINVAL in shutdown_reuse

In reply to Geliang Tang

From: Geliang Tang <[email protected]>

In the shutdown_reuse test, after shutdown and close, bind() may fail with EINVAL for MPTCP sockets due to asynchronous state transition of the top-level MPTCP socket. The subflow may have reached TCP_CLOSE, but the MPTCP socket state hasn't been updated yet.

Retry bind() on EINVAL for up to 1000 iterations (1 second) to allow the MPTCP socket to complete its state transition.

This fixes the following intermittent failures:

# RUN tls.12_aes_gcm_mptcp.shutdown_reuse ...
# tls.c:1790:shutdown_reuse:Expected ret (-1) == 0 (0) # shutdown_reuse: Test failed
# FAIL tls.12_aes_gcm_mptcp.shutdown_reuse
not ok 14 tls.12_aes_gcm_mptcp.shutdown_reuse
# RUN tls.13_aes_gcm_mptcp.shutdown_reuse ...
# tls.c:1790:shutdown_reuse:Expected ret (-1) == 0 (0) # shutdown_reuse: Test failed
# FAIL tls.13_aes_gcm_mptcp.shutdown_reuse
not ok 15 tls.13_aes_gcm_mptcp.shutdown_reuse
# RUN tls.12_chacha_mptcp.shutdown_reuse ...
# OK tls.12_chacha_mptcp.shutdown_reuse
ok 16 tls.12_chacha_mptcp.shutdown_reuse
# RUN tls.13_chacha_mptcp.shutdown_reuse ...
# OK tls.13_chacha_mptcp.shutdown_reuse
ok 17 tls.13_chacha_mptcp.shutdown_reuse
# RUN tls.13_sm4_gcm_mptcp.shutdown_reuse ...
# tls.c:1790:shutdown_reuse:Expected ret (-1) == 0 (0) # shutdown_reuse: Test failed
# FAIL tls.13_sm4_gcm_mptcp.shutdown_reuse
not ok 18 tls.13_sm4_gcm_mptcp.shutdown_reuse

This is only done for MPTCP variants to avoid slowing down plain TCP tests.

---
 tools/testing/selftests/net/tls.c | 7 +++++++
 1 file changed, 7 insertions(+)
tools/testing/selftests/net/tls.c +7 -0
index 64aad661a1b4..4ae7505846f9 100644
@@ -1744,6 +1744,7 @@ TEST_F(tls, shutdown_unsent)
 TEST_F(tls, shutdown_reuse)
 {
 	struct sockaddr_in addr;
+	int i = 0;
 	int ret;
 
 	shutdown(self->fd, SHUT_RDWR);
@@ -1754,7 +1755,13 @@ TEST_F(tls, shutdown_reuse)
 	addr.sin_addr.s_addr = htonl(INADDR_ANY);
 	addr.sin_port = 0;
 
+retry:
 	ret = bind(self->fd, &addr, sizeof(addr));
+	if (variant->mptcp &&
+	    ret < 0 && errno == EINVAL && i++ < 1000) {
+		usleep(1000);
+		goto retry;
+	}
 	EXPECT_EQ(ret, 0);
 	ret = listen(self->fd, 10);
 	EXPECT_EQ(ret, -1);
Signature · 1 line
2.53.0
author reviewer
#18 lore raw

RFC v22 18/20 selftests: tls: set timeout for multi_chunk_sendfile

In reply to Geliang Tang

From: Geliang Tang <[email protected]>

The multi_chunk_sendfile test can time out with the default 30-second limit when run on certain configurations (e.g., MPTCP). Add a 240-second timeout using TEST_F_TIMEOUT to prevent failure:

# multi_chunk_sendfile: Test terminated by timeout 30
# FAIL tls.13_chacha_mptcp.multi_chunk_sendfile
not ok 204 tls.13_chacha_mptcp.multi_chunk_sendfile

---
 tools/testing/selftests/net/tls.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tools/testing/selftests/net/tls.c +1 -1
index 4ae7505846f9..6f51da27ef1e 100644
@@ -536,7 +536,7 @@ static void chunked_sendfile(struct __test_metadata *_metadata,
 	close(fd);
 }
 
-TEST_F(tls, multi_chunk_sendfile)
+TEST_F_TIMEOUT(tls, multi_chunk_sendfile, 240)
 {
 	chunked_sendfile(_metadata, self, 4096, 4096);
 	chunked_sendfile(_metadata, self, 4096, 0);
Signature · 1 line
2.53.0
maintainer
#19 lore raw

Re: [RFC mptcp-next v22 18/20] selftests: tls: set timeout for multi_chunk_sendfile

Quoted 23 lines · Geliang Tang

From: Geliang Tang <[email protected]>

The multi_chunk_sendfile test can time out with the default 30-second limit when run on certain configurations (e.g., MPTCP). Add a 240-second timeout using TEST_F_TIMEOUT to prevent failure:

# multi_chunk_sendfile: Test terminated by timeout 30
# FAIL tls.13_chacha_mptcp.multi_chunk_sendfile
not ok 204 tls.13_chacha_mptcp.multi_chunk_sendfile

---
 tools/testing/selftests/net/tls.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tools/testing/selftests/net/tls.c +0 -0
index 4ae7505846f9..6f51da27ef1e 100644

tools/testing/selftests/net/tls.c static void chunked_sendfile(struct __test_metadata *_metadata, · line 542

Quoted 8 lines
tools/testing/selftests/net/tls.c +1 -1
@@ -536,7 +536,7 @@ static void chunked_sendfile(struct __test_metadata *_metadata,
 	close(fd);
 }
 
-TEST_F(tls, multi_chunk_sendfile)
+TEST_F_TIMEOUT(tls, multi_chunk_sendfile, 240)
 {
 	chunked_sendfile(_metadata, self, 4096, 4096);
 	chunked_sendfile(_metadata, self, 4096, 0);

Is this still required? in my tests since v8 of the stall patch I could not observe any long timeout anymore.

/P

author reviewer
#20 lore raw

Re: [RFC mptcp-next v22 18/20] selftests: tls: set timeout for multi_chunk_sendfile

Hi Paolo,

Quoted 32 lines · Paolo Abeni
Quoted 31 lines · Geliang Tang

From: Geliang Tang <[email protected]>

The multi_chunk_sendfile test can time out with the default 30- second
limit when run on certain configurations (e.g., MPTCP). Add a 240- second
timeout using TEST_F_TIMEOUT to prevent failure:

 # multi_chunk_sendfile: Test terminated by timeout 30 #          FAIL  tls.13_chacha_mptcp.multi_chunk_sendfile not ok 204 tls.13_chacha_mptcp.multi_chunk_sendfile

---
 tools/testing/selftests/net/tls.c | 2 +-

 1 file changed, 1 insertion(+), 1 deletion(-)
tools/testing/selftests/net/tls.c +0 -0
index 4ae7505846f9..6f51da27ef1e 100644
@@ -536,7 +536,7 @@ static void chunked_sendfile(struct
 __test_metadata *_metadata,
  	close(fd);
  }


-TEST_F(tls, multi_chunk_sendfile)

Quoted 8 lines
Quoted 4 lines

+TEST_F_TIMEOUT(tls, multi_chunk_sendfile, 240) {
  chunked_sendfile(_metadata, self, 4096, 4096); chunked_sendfile(_metadata, self, 4096, 0);

Is this still required? in my tests since v8 of the stall patch I could
not observe any long timeout anymore.

Thank you for the reminder. I will double-check.

-Geliang

Quoted 2 lines

/P

author reviewer
#21 lore raw

Re: [RFC mptcp-next v22 18/20] selftests: tls: set timeout for multi_chunk_sendfile

Hi Paolo,

Quoted 45 lines · Geliang Tang

Hi Paolo,

Quoted 33 lines · Paolo Abeni
Quoted 32 lines · Geliang Tang

From: Geliang Tang <[email protected]>

The multi_chunk_sendfile test can time out with the default 30- second
limit when run on certain configurations (e.g., MPTCP). Add a 240-
second
timeout using TEST_F_TIMEOUT to prevent failure:

 # multi_chunk_sendfile: Test terminated by timeout 30 #          FAIL  tls.13_chacha_mptcp.multi_chunk_sendfile not ok 204 tls.13_chacha_mptcp.multi_chunk_sendfile

---
 tools/testing/selftests/net/tls.c | 2 +-

 1 file changed, 1 insertion(+), 1 deletion(-)
tools/testing/selftests/net/tls.c +0 -0
index 4ae7505846f9..6f51da27ef1e 100644
@@ -536,7 +536,7 @@ static void chunked_sendfile(struct
 __test_metadata *_metadata,
  	close(fd);
  }


-TEST_F(tls, multi_chunk_sendfile)

Quoted 8 lines
Quoted 4 lines

+TEST_F_TIMEOUT(tls, multi_chunk_sendfile, 240) {
  chunked_sendfile(_metadata, self, 4096, 4096); chunked_sendfile(_metadata, self, 4096, 0);

Is this still required? in my tests since v8 of the stall patch I could
not observe any long timeout anymore.

Quoted 1 line

Thank you for the reminder. I will double-check.

I ran MPTCP TLS tests on the latest v11 of 'mptcp: address stall under memory pressure', and still infrequently encountered this timeout issue in debug mode, so we still need this patch.

Thanks,
-Geliang

Quoted 5 lines

-Geliang

Quoted 2 lines

/P

author reviewer
#22 lore raw

RFC v22 19/20 selftests: tls: add mptcp test cases

In reply to Geliang Tang

From: Geliang Tang <[email protected]>

This patch introduces MPTCP test cases for the TLS fixture. These "mptcp" variants are configured to create MPTCP sockets specifically for MPTCP TLS testing purposes.

---
 tools/testing/selftests/net/tls.c | 96 +++++++++++++++++++++++++++++++
 1 file changed, 96 insertions(+)
tools/testing/selftests/net/tls.c +96 -0
index 6f51da27ef1e..83cdb06da587 100644
@@ -407,6 +407,102 @@ FIXTURE_VARIANT_ADD(tls, 12_aria_gcm_256)
 	.cipher_type = TLS_CIPHER_ARIA_GCM_256,
 };
 
+FIXTURE_VARIANT_ADD(tls, 12_aes_gcm_mptcp)
+{
+	.tls_version = TLS_1_2_VERSION,
+	.cipher_type = TLS_CIPHER_AES_GCM_128,
+	.mptcp = true,
+};
+
+FIXTURE_VARIANT_ADD(tls, 13_aes_gcm_mptcp)
+{
+	.tls_version = TLS_1_3_VERSION,
+	.cipher_type = TLS_CIPHER_AES_GCM_128,
+	.mptcp = true,
+};
+
+FIXTURE_VARIANT_ADD(tls, 12_chacha_mptcp)
+{
+	.tls_version = TLS_1_2_VERSION,
+	.cipher_type = TLS_CIPHER_CHACHA20_POLY1305,
+	.fips_non_compliant = true,
+	.mptcp = true,
+};
+
+FIXTURE_VARIANT_ADD(tls, 13_chacha_mptcp)
+{
+	.tls_version = TLS_1_3_VERSION,
+	.cipher_type = TLS_CIPHER_CHACHA20_POLY1305,
+	.fips_non_compliant = true,
+	.mptcp = true,
+};
+
+FIXTURE_VARIANT_ADD(tls, 13_sm4_gcm_mptcp)
+{
+	.tls_version = TLS_1_3_VERSION,
+	.cipher_type = TLS_CIPHER_SM4_GCM,
+	.fips_non_compliant = true,
+	.mptcp = true,
+};
+
+FIXTURE_VARIANT_ADD(tls, 13_sm4_ccm_mptcp)
+{
+	.tls_version = TLS_1_3_VERSION,
+	.cipher_type = TLS_CIPHER_SM4_CCM,
+	.fips_non_compliant = true,
+	.mptcp = true,
+};
+
+FIXTURE_VARIANT_ADD(tls, 12_aes_ccm_mptcp)
+{
+	.tls_version = TLS_1_2_VERSION,
+	.cipher_type = TLS_CIPHER_AES_CCM_128,
+	.mptcp = true,
+};
+
+FIXTURE_VARIANT_ADD(tls, 13_aes_ccm_mptcp)
+{
+	.tls_version = TLS_1_3_VERSION,
+	.cipher_type = TLS_CIPHER_AES_CCM_128,
+	.mptcp = true,
+};
+
+FIXTURE_VARIANT_ADD(tls, 12_aes_gcm_256_mptcp)
+{
+	.tls_version = TLS_1_2_VERSION,
+	.cipher_type = TLS_CIPHER_AES_GCM_256,
+	.mptcp = true,
+};
+
+FIXTURE_VARIANT_ADD(tls, 13_aes_gcm_256_mptcp)
+{
+	.tls_version = TLS_1_3_VERSION,
+	.cipher_type = TLS_CIPHER_AES_GCM_256,
+	.mptcp = true,
+};
+
+FIXTURE_VARIANT_ADD(tls, 13_nopad_mptcp)
+{
+	.tls_version = TLS_1_3_VERSION,
+	.cipher_type = TLS_CIPHER_AES_GCM_128,
+	.nopad = true,
+	.mptcp = true,
+};
+
+FIXTURE_VARIANT_ADD(tls, 12_aria_gcm_mptcp)
+{
+	.tls_version = TLS_1_2_VERSION,
+	.cipher_type = TLS_CIPHER_ARIA_GCM_128,
+	.mptcp = true,
+};
+
+FIXTURE_VARIANT_ADD(tls, 12_aria_gcm_256_mptcp)
+{
+	.tls_version = TLS_1_2_VERSION,
+	.cipher_type = TLS_CIPHER_ARIA_GCM_256,
+	.mptcp = true,
+};
+
 static bool is_mptcp_enable(void)
 {
 	char buf[16] = { 0 };
Signature · 1 line
2.53.0
author reviewer
#23 lore raw

RFC v22 20/20 selftests: mptcp: cover mptcp tls tests

In reply to Geliang Tang

From: Geliang Tang <[email protected]>

The mptcp tests for tls.c is available now, this patch adds mptcp_tls.sh to test it in the MPTCP CI by default.

---
 tools/testing/selftests/net/mptcp/.gitignore  |  1 +
 tools/testing/selftests/net/mptcp/Makefile    |  2 +
 tools/testing/selftests/net/mptcp/config      |  5 ++
 .../testing/selftests/net/mptcp/mptcp_tls.sh  | 61 +++++++++++++++++++
 tools/testing/selftests/net/mptcp/tls.c       |  1 +
 5 files changed, 70 insertions(+)
 create mode 100755 tools/testing/selftests/net/mptcp/mptcp_tls.sh
 create mode 120000 tools/testing/selftests/net/mptcp/tls.c
tools/testing/selftests/net/mptcp/.gitignore +1 -0
index 833279fb34e2..f6defec6eeb5 100644
@@ -4,4 +4,5 @@ mptcp_diag
 mptcp_inq
 mptcp_sockopt
 pm_nl_ctl
+tls
 *.pcap
tools/testing/selftests/net/mptcp/Makefile +2 -0
index 22ba0da2adb8..f7c959a25b3b 100644
@@ -14,6 +14,7 @@ TEST_PROGS := \
 	mptcp_connect_splice.sh \
 	mptcp_join.sh \
 	mptcp_sockopt.sh \
+	mptcp_tls.sh \
 	pm_netlink.sh \
 	simult_flows.sh \
 	userspace_pm.sh \
@@ -25,6 +26,7 @@ TEST_GEN_FILES := \
 	mptcp_inq \
 	mptcp_sockopt \
 	pm_nl_ctl \
+	tls \
 # end of TEST_GEN_FILES
 
 TEST_FILES := \
tools/testing/selftests/net/mptcp/config +5 -0
index f48bd5183fb3..bfc26bde0501 100644
@@ -38,3 +38,8 @@ CONFIG_INET_ESP=y
 CONFIG_INET_ESPINTCP=y
 CONFIG_INET6_ESP=y
 CONFIG_INET6_ESPINTCP=y
+CONFIG_TLS=m
+CONFIG_CRYPTO_ARIA=m
+CONFIG_CRYPTO_CCM=m
+CONFIG_CRYPTO_CHACHA20POLY1305=m
+CONFIG_CRYPTO_SM4_GENERIC=m
tools/testing/selftests/net/mptcp/mptcp_tls.sh new file +61 -0

61 lines not on this page · raw message

tools/testing/selftests/net/mptcp/tls.c new file +1 -0
new file mode 120000
index 000000000000..724b1f047c89
@@ -0,0 +1 @@
+../tls.c
\ No newline at end of file
Signature · 1 line
2.53.0
#24 lore raw
In reply to Geliang Tang

Hi Geliang,

Thank you for your modifications, that's great!

Our CI did some validations and here is its report:

- KVM Validation: normal (except selftest_mptcp_join): Unstable: 1 failed test(s): selftest_simult_flows ⚠️
- KVM Validation: normal (only selftest_mptcp_join): Success! ✅
- KVM Validation: debug (except selftest_mptcp_join): Success! ✅
- KVM Validation: debug (only selftest_mptcp_join): Success! ✅
- KVM Validation: btf-normal (only bpftest_all): Success! ✅
- KVM Validation: btf-debug (only bpftest_all): Success! ✅
- Task: https://github.com/multipath-tcp/mptcp_net-next/actions/runs/26634797129

Initiator: Patchew Applier
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/1a53bf73533f
Patchwork: https://patchwork.kernel.org/project/mptcp/list/?series=1102837

If there are some issues, you can reproduce them using the same environment as the one used by the CI thanks to a docker image, e.g.:

    $ cd [kernel source code]
    $ docker run -v "${PWD}:${PWD}:rw" -w "${PWD}" --privileged --rm -it \
        --pull always mptcp/mptcp-upstream-virtme-docker:latest \
        auto-normal

For more details:

https://github.com/multipath-tcp/mptcp-upstream-virtme-docker

Please note that despite all the efforts that have been already done to have a stable tests suite when executed on a public CI like here, it is possible some reported issues are not due to your modifications. Still, do not hesitate to help us improve that ;-)

Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (NGI0 Core)