MPTCP KTLS support
Subsystems (6)
Geliang Tang
reviewercover MPTCP KTLS support
From: Geliang Tang <[email protected]>
Prior to this work, MPTCP did not support TLS. The two protocols conflicted because both MPTCP and TLS use the ULP (Upper Layer Protocol) infrastructure in the Linux kernel. ULP settings, including TLS configuration, were disabled in MPTCP. If an application attempted to set TLS for an MPTCP socket, the system would return an error code indicating EOPNOTSUPP (Operation not supported).
This series adds KTLS support for MPTCP. Since no ULP is currently attached to the MPTCP socket (msk), KTLS can be configured directly on the msk rather than on individual subflows. This does not affect its existing communication, and leverages HMAC-based authentication to ensure subflow security.
RFC versions of this series have gone through many iterations on MPTCP mailing list, mainly to address Sashiko's review comments. It is now mostly stable.
A follow-up series will add MPTCP support to the TLS selftests (tools/testing/selftests/net/tls.c). All existing TCP test cases have already been verified to pass over MPTCP as well.
The primary validation use case for this work is NVMe over MPTCP with KTLS. NVMe over TCP is a storage protocol that transports NVMe commands over TCP. By combining it with MPTCP, multipath capabilities for storage traffic are gained. By adding KTLS, the storage traffic is secured with encryption. Although NVMe over MPTCP is still under active development, I have already verified that KTLS operates correctly on top of it.
All feedback is welcome.
Gang Yan (1):
mptcp: update mptcp_check_readable helper
Geliang Tang (16):
tls: make tls_ctx_create and update_sk_prot static
tls: factor out __tls_build_proto for mptcp support
tls: add protocol dimension to tls operation cache
mptcp: add sendmsg_locked to proto_ops
tls: use sendmsg_locked from the underlying socket
mptcp: implement peek_len for proto_ops
tls: replace tcp_inq with socket peek_len
tls: store original read_sock for non-tcp sockets
tls: introduce tls protocol ops structure
tls: use protocol ops via tls_context
mptcp: implement mptcp-specific tls protocol ops
tls: add mptcp support for sk_poll
tls: disable device offload for mptcp sockets
mptcp: implement ulp getsockopt for tls support
mptcp: implement ulp setsockopt for tls support
selftests: mptcp: connect: use espintcp for ulp test
include/net/mptcp.h | 11 + include/net/tcp.h | 1 + include/net/tls.h | 19 ++ net/ipv4/tcp.c | 9 +- net/mptcp/protocol.c | 180 +++++++++++++- net/mptcp/protocol.h | 1 + net/mptcp/sockopt.c | 68 +++++- net/tls/tls.h | 2 - net/tls/tls_device.c | 10 +- net/tls/tls_main.c | 227 +++++++++++++++--- net/tls/tls_strp.c | 35 ++- net/tls/tls_sw.c | 10 +- tools/testing/selftests/net/mptcp/config | 4 + .../selftests/net/mptcp/mptcp_connect.c | 4 +- 14 files changed, 516 insertions(+), 65 deletions(-)
Signature · 1 line
2.53.0
Geliang Tang
reviewerRFC 1/17 tls: make tls_ctx_create and update_sk_prot static
From: Geliang Tang <[email protected]>
The TLS TOE (TCP offload engine) support has been removed. As a result, tls_ctx_create() and update_sk_prot() are no longer used outside of tls_main.c.
Make them static and remove their prototypes from tls.h. This avoids exporting unnecessary symbols and cleans up the internal API.
--- net/tls/tls.h | 2 -- net/tls/tls_main.c | 4 ++-- 2 files changed, 2 insertions(+), 4 deletions(-)
net/tls/tls.h +0 -2
@@ -136,9 +136,7 @@ struct tls_rec {int __net_init tls_proc_init (struct net *net); void __net_exit tls_proc_fini(struct net *net); -struct tls_context *tls_ctx_create (struct sock *sk);void tls_ctx_free (struct sock *sk,struct tls_context *ctx); -void update_sk_prot (struct sock *sk,struct tls_context *ctx);int wait_on_pending_writer (struct sock *sk,long *timeo);void tls_err_abort (struct sock *sk,int err);
net/tls/tls_main.c +2 -2
@@ -126,7 +126,7 @@ static struct proto_ops tls_proto_ops[TLS_NUM_PROTS][TLS_NUM_CONFIG][TLS_NUM_CONstatic void build_protos (struct proto prot[TLS_NUM_CONFIG ][TLS_NUM_CONFIG ],const struct proto *base); -void update_sk_prot (struct sock *sk,struct tls_context *ctx) +static void update_sk_prot (struct sock *sk,struct tls_context *ctx) {int ip_ver = sk->sk_family ==AF_INET6 ?TLSV6 :TLSV4 ;
@@ -913,7 +913,7 @@ static int tls_disconnect(struct sock *sk, int flags)return -EOPNOTSUPP ; } -struct tls_context *tls_ctx_create (struct sock *sk) +static struct tls_context *tls_ctx_create (struct sock *sk) {struct inet_connection_sock *icsk =inet_csk (sk);struct tls_context *ctx;
Signature · 1 line
2.53.0
Geliang Tang
reviewerRFC 2/17 tls: factor out __tls_build_proto for mptcp support
In reply to Geliang TangFrom: Geliang Tang <[email protected]>
tls_build_proto() contains duplicated logic for building IPv4 and IPv6 TLS protocol caches.
Factor out the common code into a new helper __tls_build_proto(), which takes the saved protocol pointer, mutex, and IP family as parameters. This prepares for adding MPTCP support by reducing the amount of duplicated code needed when introducing additional protocol variants.
No functional change intended.
--- net/tls/tls_main.c | 46 ++++++++++++++++++++++++---------------------- 1 file changed, 24 insertions(+), 22 deletions(-)
net/tls/tls_main.c +24 -22
@@ -968,35 +968,37 @@ static void build_proto_ops(struct proto_ops ops[TLS_NUM_CONFIG][TLS_NUM_CONFIG]#endif } -static void tls_build_proto (struct sock *sk) +static void __tls_build_proto (struct sock *sk, +const struct proto *saved_prot, +struct mutex *prot_mutex, +int family) {int ip_ver = sk->sk_family ==AF_INET6 ?TLSV6 :TLSV4 ;struct proto *prot =READ_ONCE (sk->sk_prot); -/* Build IPv6 TLS whenever the address of tcpv6 _prot changes */ -if (ip_ver ==TLSV6 && -unlikely (prot !=smp_load_acquire (&saved_tcpv6_prot))) { -mutex_lock (&tcpv6_prot_mutex); -if (likely (prot != saved_tcpv6_prot)) { -build_protos (tls_prots[TLSV6 ], prot); -build_proto_ops (tls_proto_ops[TLSV6 ], - sk->sk_socket->ops); -smp_store_release (&saved_tcpv6_prot, prot); +if (ip_ver == family) { +/* smp_load_acquire pairs with smp_store_release below */ +if (unlikely (prot !=smp_load_acquire (&saved_prot))) { +mutex_lock (prot_mutex); +if (likely (prot != saved_prot)) { +build_protos (tls_prots[family], prot); +build_proto_ops (tls_proto_ops[family], + sk->sk_socket->ops); +/* pairs with smp_load_acquire above */ +smp_store_release (&saved_prot, prot); + } +mutex_unlock (prot_mutex); } -mutex_unlock (&tcpv6_prot_mutex); } +} -if (ip_ver ==TLSV4 && -unlikely (prot !=smp_load_acquire (&saved_tcpv4_prot))) { -mutex_lock (&tcpv4_prot_mutex); -if (likely (prot != saved_tcpv4_prot)) { -build_protos (tls_prots[TLSV4 ], prot); -build_proto_ops (tls_proto_ops[TLSV4 ], - sk->sk_socket->ops); -smp_store_release (&saved_tcpv4_prot, prot); - } -mutex_unlock (&tcpv4_prot_mutex); - } +static void tls_build_proto (struct sock *sk) +{ +/* Build IPv6 TLS whenever the address of tcpv6 _prot changes */ +__tls_build_proto (sk, saved_tcpv6_prot, &tcpv6_prot_mutex, +TLSV6 ); +__tls_build_proto (sk, saved_tcpv4_prot, &tcpv4_prot_mutex, +TLSV4 ); } static void build_protos(struct proto prot[TLS_NUM_CONFIG][TLS_NUM_CONFIG],
Signature · 1 line
2.53.0
Geliang Tang
reviewerRFC 3/17 tls: add protocol dimension to tls operation cache
In reply to Geliang TangFrom: Geliang Tang <[email protected]>
The current TLS operation cache is indexed solely by IP version (IPv4/IPv6). This was sufficient when only TCP was supported. Rename TLS_NUM_PROTS to TLS_NUM_FAMILY to accurately reflect that it represents the number of address families.
With the introduction of MPTCP, both TCP and MPTCP sockets within the same IP version now share the same cache entries. When an MPTCP socket enables TLS, it overwrites the cache with MPTCP-specific operations, causing existing TCP TLS sockets to use the wrong ops, leading to type confusion and kernel panics.
Fix by extending the cache arrays with a protocol dimension to separate TCP and MPTCP. Introduce TLSTCP and TLSMPTCP enum values, along with separate saved protocol pointers and mutexes for MPTCP. update_sk_prot() and __tls_build_proto() now select the appropriate cache based on sk->sk_protocol.
--- net/tls/tls_main.c | 40 +++++++++++++++++++++++++++++----------- 1 file changed, 29 insertions(+), 11 deletions(-)
net/tls/tls_main.c +29 -11
@@ -53,7 +53,13 @@ MODULE_ALIAS_TCP_ULP("tls");enum {TLSV4 ,TLSV6 , -TLS_NUM_PROTS , +TLS_NUM_FAMILY , +}; + +enum { +TLSTCP , +TLSMPTCP , +TLS_NUM_PROTO , };#define CHECK_CIPHER_DESC (cipher,ci) \
@@ -117,23 +123,30 @@ CHECK_CIPHER_DESC(TLS_CIPHER_SM4_CCM, tls12_crypto_info_sm4_ccm);CHECK_CIPHER_DESC (TLS_CIPHER_ARIA_GCM_128 , tls12_crypto_info_aria_gcm_128);CHECK_CIPHER_DESC (TLS_CIPHER_ARIA_GCM_256 , tls12_crypto_info_aria_gcm_256); +static const struct proto *saved_mptcpv6_prot; +static DEFINE_MUTEX (mptcpv6_prot_mutex);static const struct proto *saved_tcpv6_prot;static DEFINE_MUTEX (tcpv6_prot_mutex); +static const struct proto *saved_mptcpv4_prot; +static DEFINE_MUTEX (mptcpv4_prot_mutex);static const struct proto *saved_tcpv4_prot;static DEFINE_MUTEX (tcpv4_prot_mutex); -static struct proto tls_prots[TLS_NUM_PROTS ][TLS_NUM_CONFIG ][TLS_NUM_CONFIG ]; -static struct proto_ops tls_proto_ops[TLS_NUM_PROTS ][TLS_NUM_CONFIG ][TLS_NUM_CONFIG ]; +static struct proto +tls_prots[TLS_NUM_FAMILY ][TLS_NUM_PROTO ][TLS_NUM_CONFIG ][TLS_NUM_CONFIG ]; +static struct proto_ops +tls_proto_ops[TLS_NUM_FAMILY ][TLS_NUM_PROTO ][TLS_NUM_CONFIG ][TLS_NUM_CONFIG ];static void build_protos (struct proto prot[TLS_NUM_CONFIG ][TLS_NUM_CONFIG ],const struct proto *base);static void update_sk_prot (struct sock *sk,struct tls_context *ctx) { +int proto = sk->sk_protocol ==IPPROTO_MPTCP ?TLSMPTCP :TLSTCP ;int ip_ver = sk->sk_family ==AF_INET6 ?TLSV6 :TLSV4 ;WRITE_ONCE (sk->sk_prot, - &tls_prots[ip_ver][ctx->tx_conf][ctx->rx_conf]); + &tls_prots[ip_ver][proto][ctx->tx_conf][ctx->rx_conf]);WRITE_ONCE (sk->sk_socket->ops, - &tls_proto_ops[ip_ver][ctx->tx_conf][ctx->rx_conf]); + &tls_proto_ops[ip_ver][proto][ctx->tx_conf][ctx->rx_conf]); }int wait_on_pending_writer (struct sock *sk,long *timeo)
@@ -971,18 +984,19 @@ static void build_proto_ops(struct proto_ops ops[TLS_NUM_CONFIG][TLS_NUM_CONFIG]static void __tls_build_proto (struct sock *sk,const struct proto *saved_prot,struct mutex *prot_mutex, -int family) +int family,int protocol) { +int proto = sk->sk_protocol ==IPPROTO_MPTCP ?TLSMPTCP :TLSTCP ;int ip_ver = sk->sk_family ==AF_INET6 ?TLSV6 :TLSV4 ;struct proto *prot =READ_ONCE (sk->sk_prot); -if (ip_ver == family) { +if (ip_ver == family && proto == protocol) {/* smp_load_acquire pairs with smp_store_release below */ if (unlikely (prot !=smp_load_acquire (&saved_prot))) {mutex_lock (prot_mutex);if (likely (prot != saved_prot)) { -build_protos (tls_prots[family], prot); -build_proto_ops (tls_proto_ops[family], +build_protos (tls_prots[family][protocol], prot); +build_proto_ops (tls_proto_ops[family][protocol], sk->sk_socket->ops);/* pairs with smp_load_acquire above */ smp_store_release (&saved_prot, prot);
@@ -995,10 +1009,14 @@ static void __tls_build_proto(struct sock *sk,static void tls_build_proto (struct sock *sk) {/* Build IPv6 TLS whenever the address of tcpv6 _prot changes */ +__tls_build_proto (sk, saved_mptcpv6_prot, &mptcpv6_prot_mutex, +TLSV6 ,TLSMPTCP );__tls_build_proto (sk, saved_tcpv6_prot, &tcpv6_prot_mutex, -TLSV6 ); +TLSV6 ,TLSTCP ); +__tls_build_proto (sk, saved_mptcpv4_prot, &mptcpv4_prot_mutex, +TLSV4 ,TLSMPTCP );__tls_build_proto (sk, saved_tcpv4_prot, &tcpv4_prot_mutex, -TLSV4 ); +TLSV4 ,TLSTCP ); } static void build_protos(struct proto prot[TLS_NUM_CONFIG][TLS_NUM_CONFIG],
Signature · 1 line
2.53.0
Geliang Tang
reviewerRFC 4/17 mptcp: add sendmsg_locked to proto_ops
In reply to Geliang TangFrom: Geliang Tang <[email protected]>
MPTCP currently provides a standard sendmsg() implementation which acquires and releases the socket lock internally. However, certain upper layers (e.g., TLS) need to call the sendmsg method while the socket lock is already held.
Split the existing mptcp_sendmsg() into mptcp_sendmsg_locked() which assumes the caller holds the socket lock, and a tiny wrapper mptcp_sendmsg() that acquires the lock and calls the locked version.
Expose .sendmsg_locked in both mptcp_stream_ops and mptcp_v6_stream_ops.
--- net/mptcp/protocol.c | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-)
net/mptcp/protocol.c +14 -4
@@ -1967,7 +1967,7 @@ static void mptcp_rps_record_subflows(const struct mptcp_sock *msk) } } -static int mptcp_sendmsg (struct sock *sk,struct msghdr *msg,size_t len) +static int mptcp_sendmsg_locked (struct sock *sk,struct msghdr *msg,size_t len) {struct mptcp_sock *msk =mptcp_sk (sk);struct page_frag *pfrag;
@@ -1979,8 +1979,6 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) msg->msg_flags &=MSG_MORE |MSG_DONTWAIT |MSG_NOSIGNAL |MSG_FASTOPEN |MSG_EOR ; -lock_sock (sk); -mptcp_rps_record_subflows (msk); if (unlikely(inet_test_bit(DEFER_CONNECT, sk) ||
@@ -2096,7 +2094,6 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) } out: -release_sock (sk);return copied; do_error:
@@ -2107,6 +2104,17 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) goto out; } +static int mptcp_sendmsg (struct sock *sk,struct msghdr *msg,size_t len) +{ +int ret; + +lock_sock (sk); + ret =mptcp_sendmsg_locked (sk, msg, len); +release_sock (sk); + +return ret; +} +static void mptcp_rcv_space_adjust (struct mptcp_sock *msk,int copied);static void mptcp_eat_recv_skb (struct sock *sk,struct sk_buff *skb)
@@ -4703,6 +4711,7 @@ static const struct proto_ops mptcp_stream_ops = { .set_rcvlowat = mptcp_set_rcvlowat, .read_sock = mptcp_read_sock, .splice_read = mptcp_splice_read, + .sendmsg_locked = mptcp_sendmsg_locked, };static struct inet_protosw mptcp_protosw = {
@@ -4815,6 +4824,7 @@ static const struct proto_ops mptcp_v6_stream_ops = { .set_rcvlowat = mptcp_set_rcvlowat, .read_sock = mptcp_read_sock, .splice_read = mptcp_splice_read, + .sendmsg_locked = mptcp_sendmsg_locked, };static struct proto mptcp_v6_prot;
Signature · 1 line
2.53.0
Geliang Tang
reviewerRFC 5/17 tls: use sendmsg_locked from the underlying socket
In reply to Geliang TangFrom: Geliang Tang <[email protected]>
TLS offload (device and sw) may call tcp_sendmsg_locked() directly when pushing TLS records. This assumes the underlying socket is always a TCP socket. With MPTCP, the socket can be an MPTCP socket, which does not directly expose a sendmsg_locked method via its proto_ops.
Replace the hard-coded tcp_sendmsg_locked() call with sk->sk_socket->ops->sendmsg_locked(). This enables TLS to work transparently over any socket that implements .sendmsg_locked, including MPTCP after the previous commit.
The change is safe because both TCP and MPTCP now provide a conformant .sendmsg_locked implementation.
--- net/tls/tls_main.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
net/tls/tls_main.c +1 -1
@@ -205,7 +205,7 @@ int tls_push_sg(struct sock *sk,bvec_set_page (&bvec, p, size, offset);iov_iter_bvec (&msg.msg_iter,ITER_SOURCE , &bvec,1 , size); - ret =tcp_sendmsg_locked (sk, &msg, size); + ret = sk->sk_socket->ops->sendmsg_locked (sk, &msg, size);if (ret != size) {if (ret >0 ) {
Signature · 1 line
2.53.0
Geliang Tang
reviewerRFC 6/17 mptcp: implement peek_len for proto_ops
In reply to Geliang TangFrom: Geliang Tang <[email protected]>
The TLS stack uses tcp_inq() to query the amount of data available in the receive queue without consuming it. For MPTCP sockets, this information is not directly available from a TCP subflow; it must be computed from the MPTCP receive queue and the current mapping.
Introduce mptcp_peek_len() which returns the number of bytes that can be peeked from the MPTCP socket. It reuses the existing mptcp_inq() helper (used by ioctl SIOCINQ). The implementation considers the first skb in the receive queue, the current ack_seq, and handles the FIN case.
Assign .peek_len in both mptcp_stream_ops and mptcp_v6_stream_ops so that upper layers (e.g., TLS) can obtain the correct in-queue byte count for an MPTCP connection.
--- net/mptcp/protocol.c | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+)
net/mptcp/protocol.c +34 -0
@@ -4689,6 +4689,38 @@ static ssize_t mptcp_splice_read(struct socket *sock, loff_t *ppos,return ret; } +static int mptcp_inq (struct sock *sk) +{ +const struct mptcp_sock *msk =mptcp_sk (sk); +const struct sk_buff *skb; + +if ((1 << sk->sk_state) & (TCPF_SYN_SENT |TCPF_SYN_RECV )) +return 0 ; + + skb =skb_peek (&sk->sk_receive_queue); +if (skb) { +u64 answ =READ_ONCE (msk->ack_seq) -MPTCP_SKB_CB (skb)->map_seq; + +if (answ >=INT_MAX ) + answ =INT_MAX ; + +/* Subtract 1, if FIN was received */ +if (answ && + (sk->sk_state ==TCP_CLOSE || + (sk->sk_shutdown &RCV_SHUTDOWN ))) + answ--; + +return (int )answ; + } + +return 0 ; +} + +static int mptcp_peek_len (struct socket *sock) +{ +return mptcp_inq (sock->sk); +} +static const struct proto_ops mptcp_stream_ops = { .family =PF_INET , .owner =THIS_MODULE ,
@@ -4712,6 +4744,7 @@ static const struct proto_ops mptcp_stream_ops = { .read_sock = mptcp_read_sock, .splice_read = mptcp_splice_read, .sendmsg_locked = mptcp_sendmsg_locked, + .peek_len = mptcp_peek_len, };static struct inet_protosw mptcp_protosw = {
@@ -4825,6 +4858,7 @@ static const struct proto_ops mptcp_v6_stream_ops = { .read_sock = mptcp_read_sock, .splice_read = mptcp_splice_read, .sendmsg_locked = mptcp_sendmsg_locked, + .peek_len = mptcp_peek_len, };static struct proto mptcp_v6_prot;
Signature · 1 line
2.53.0
Geliang Tang
reviewerRFC 7/17 tls: replace tcp_inq with socket peek_len
In reply to Geliang TangFrom: Geliang Tang <[email protected]>
TLS (device, strparser, and software) calls tcp_inq() directly to determine how much data is still pending in the socket receive queue. This breaks when the underlying socket is not TCP (e.g., MPTCP).
Switch all occurrences of tcp_inq(sk) to sk->sk_socket->ops-> peek_len(sk->sk_socket). This operation is implemented for both TCP and MPTCP (after the previous commits), making TLS transparently usable over MPTCP connections.
The change is straightforward: every place where TLS needed the available in-queue bytes now uses the protocol-specific peek_len method instead of assuming a TCP socket.
--- net/tls/tls_device.c | 4 ++-- net/tls/tls_strp.c | 6 ++++-- net/tls/tls_sw.c | 4 +++- 3 files changed, 9 insertions(+), 5 deletions(-)
net/tls/tls_device.c +2 -2
@@ -805,7 +805,7 @@ void tls_device_rx_resync_new_rec(struct sock *sk, u32 rcd_len, u32 seq)/* head of next rec is already in, note that the sock_inq will * include the currently parsed message when called from parser */ - sock_data =tcp_inq (sk); + sock_data = sk->sk_socket->ops->peek_len (sk->sk_socket);if (sock_data > rcd_len) {trace_tls_device_rx_resync_nh_delay (sk, sock_data, rcd_len);
@@ -864,7 +864,7 @@ static void tls_device_core_ctrl_rx_resync(struct tls_context *tls_ctx, rxm =strp_msg (skb);/* head of next rec is already in, parser will sync for us */ -if (tcp_inq (sk) > rxm->full_len) { +if (sk->sk_socket->ops->peek_len (sk->sk_socket) > rxm->full_len) {trace_tls_device_rx_resync_nh_schedule (sk); ctx->resync_nh_do_now =1 ; }else {
net/tls/tls_strp.c +4 -2
@@ -484,12 +484,14 @@ bool tls_strp_msg_load(struct tls_strparser *strp, bool force_refresh) {struct strp_msg *rxm;struct tls_msg *tlm; +int inq;DEBUG_NET_WARN_ON_ONCE (!strp->msg_ready);DEBUG_NET_WARN_ON_ONCE (!strp->stm.full_len);if (!strp->copy_mode && force_refresh) { -if (unlikely (tcp_inq (strp->sk) < strp->stm.full_len)) { + inq = strp->sk->sk_socket->ops->peek_len (strp->sk->sk_socket); +if (unlikely (inq < strp->stm.full_len)) {WRITE_ONCE (strp->msg_ready,0 ); strp->msg_announced =0 ;memset (&strp->stm,0 ,sizeof (strp->stm));
@@ -513,7 +515,7 @@ static int tls_strp_read_sock(struct tls_strparser *strp) {int sz, inq; - inq =tcp_inq (strp->sk); + inq = strp->sk->sk_socket->ops->peek_len (strp->sk->sk_socket);if (inq <1 )return 0 ;
net/tls/tls_sw.c +3 -1
@@ -1706,12 +1706,14 @@ tls_read_flush_backlog(struct sock *sk, struct tls_prot_info *prot, size_t *flushed_at) {size_t max_rec; +int inq;if (len_left <= decrypted)return false; + inq = sk->sk_socket->ops->peek_len (sk->sk_socket); max_rec = prot->overhead_size - prot->tail_size +TLS_MAX_PAYLOAD_SIZE ; -if (done - *flushed_at <SZ_128K &&tcp_inq (sk) > max_rec) +if (done - *flushed_at <SZ_128K && inq > max_rec)return false; *flushed_at = done;
Signature · 1 line
2.53.0
Geliang Tang
reviewerRFC 8/17 tls: store original read_sock for non-tcp sockets
In reply to Geliang TangFrom: Geliang Tang <[email protected]>
TLS strparser uses tcp_read_sock() to copy data from the underlying socket. This assumes the socket is always TCP, which fails when TLS is used over MPTCP.
Store the original socket's read_sock method (sk->sk_socket->ops-> read_sock) in a new .sk_read_sock callback inside struct tls_context. Then in tls_strp_read_copyin(), call this stored callback instead of the hard-coded tcp_read_sock().
With this change, TLS strparser works transparently over any socket that implements .read_sock (including MPTCP, which already provides mptcp_read_sock). Behavior for plain TCP remains unchanged.
--- include/net/tls.h | 2 ++ net/tls/tls_main.c | 1 + net/tls/tls_strp.c | 3 ++- 3 files changed, 5 insertions(+), 1 deletion(-)
include/net/tls.h +2 -0
@@ -262,6 +262,8 @@ struct tls_context {struct sock *sk;void (*sk_destruct)(struct sock *sk); +int (*sk_read_sock)(struct sock *sk,read_descriptor_t *desc, +sk_read_actor_t recv_actor);union tls_crypto_context crypto_send;union tls_crypto_context crypto_recv;
net/tls/tls_main.c +1 -0
@@ -1086,6 +1086,7 @@ static int tls_init(struct sock *sk) ctx->tx_conf =TLS_BASE ; ctx->rx_conf =TLS_BASE ; ctx->tx_max_payload_len =TLS_MAX_PAYLOAD_SIZE ; + ctx->sk_read_sock = sk->sk_socket->ops->read_sock;update_sk_prot (sk, ctx); out:write_unlock_bh (&sk->sk_callback_lock);
net/tls/tls_strp.c +2 -1
@@ -375,6 +375,7 @@ static int tls_strp_copyin(read_descriptor_t *desc, struct sk_buff *in_skb,static int tls_strp_read_copyin (struct tls_strparser *strp) { +struct tls_context *ctx =tls_get_ctx (strp->sk);read_descriptor_t desc; desc.arg.data = strp;
@@ -382,7 +383,7 @@ static int tls_strp_read_copyin(struct tls_strparser *strp) desc.count =1 ;/* give more than one skb per call */ /* sk should be locked here, so okay to do read_sock */ -tcp_read_sock (strp->sk, &desc, tls_strp_copyin); + ctx->sk_read_sock (strp->sk, &desc, tls_strp_copyin);return desc.error; }
Signature · 1 line
2.53.0
Geliang Tang
reviewerRFC 9/17 tls: introduce tls protocol ops structure
In reply to Geliang TangFrom: Geliang Tang <[email protected]>
To extend MPTCP support based on TCP TLS, a tls_prot_ops structure has been introduced for TLS, encapsulating TCP-specific helpers within this structure.
Add registering, validating and finding functions for this structure to add, validate and find a tls_prot_ops on the global list tls_prot_ops_list.
Register TCP-specific structure tls_tcp_ops in tls_register().
--- include/net/tls.h | 15 +++++++ net/tls/tls_main.c | 101 ++++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 115 insertions(+), 1 deletion(-)
include/net/tls.h +15 -0
@@ -224,6 +224,21 @@ struct tls_prot_info {u16 tail_size; }; +struct tls_prot_ops { +struct module *owner; +struct list_head list; +int protocol; + +struct sk_buff *(*recv_skb)(struct sock *sk,u32 *off); +bool (*lock_is_held)(struct sock *sk); +void (*read_done)(struct sock *sk,size_t len); +u32 (*get_skb_seq)(struct sk_buff *skb); +int (*skb_get_header)(const struct sk_buff *skb,int offset, +void *to,int len); +bool (*epollin_ready)(const struct sock *sk); +void (*check_app_limited)(struct sock *sk); +}; +struct tls_context {/* read-only cache line */ struct tls_prot_info prot_info;
net/tls/tls_main.c +100 -1
137 lines not on this page · raw message
Signature · 1 line
2.53.0
Geliang Tang
reviewerRFC 10/17 tls: use protocol ops via tls_context
In reply to Geliang TangFrom: Geliang Tang <[email protected]>
Currently, TLS code directly calls TCP-specific functions (e.g., tcp_rate_check_app_limited, tcp_read_done, tcp_epollin_ready, etc.) and accesses TCP-specific fields (e.g., TCP_SKB_CB(skb)->seq). This makes it hard to support other protocols like MPTCP.
Introduce a struct tls_prot_ops pointer in tls_context, which is initialized during tls_init based on the socket protocol (TCP or MPTCP). All protocol-dependent operations are now invoked via this ops pointer, allowing each protocol to provide its own implementation.
Also add proper module reference counting for the ops owner.
--- include/net/tls.h | 2 ++ net/tls/tls_main.c | 15 ++++++++++++++- net/tls/tls_strp.c | 26 ++++++++++++++++++-------- net/tls/tls_sw.c | 6 ++++-- 4 files changed, 38 insertions(+), 11 deletions(-)
include/net/tls.h +2 -0
@@ -280,6 +280,8 @@ struct tls_context {int (*sk_read_sock)(struct sock *sk,read_descriptor_t *desc,sk_read_actor_t recv_actor); +const struct tls_prot_ops *ops; +union tls_crypto_context crypto_send;union tls_crypto_context crypto_recv;
net/tls/tls_main.c +14 -1
@@ -201,7 +201,7 @@ int tls_push_sg(struct sock *sk, ctx->splicing_pages = true;while (1 ) {/* is sending application-limited? */ -tcp_rate_check_app_limited (sk); + ctx->ops->check_app_limited (sk); p =sg_page (sg); retry:bvec_set_page (&bvec, p, size, offset);
@@ -340,6 +340,11 @@ void tls_ctx_free(struct sock *sk, struct tls_context *ctx)if (!ctx)return ; +if (ctx->ops) { +module_put (ctx->ops->owner); + ctx->ops =NULL ; + } +memzero_explicit (&ctx->crypto_send,sizeof (ctx->crypto_send));memzero_explicit (&ctx->crypto_recv,sizeof (ctx->crypto_recv));mutex_destroy (&ctx->tx_lock);
@@ -1079,6 +1084,7 @@ static struct tls_prot_ops *tls_prot_ops_find(int protocol)static int tls_init (struct sock *sk) { +struct tls_prot_ops *ops;struct tls_context *ctx;int rc =0 ;
@@ -1101,10 +1107,17 @@ static int tls_init(struct sock *sk) goto out; } + ops =tls_prot_ops_find (sk->sk_protocol); +if (!ops || !try_module_get (ops->owner)) { + rc = -EINVAL ; + goto out; + } + ctx->tx_conf =TLS_BASE ; ctx->rx_conf =TLS_BASE ; ctx->tx_max_payload_len =TLS_MAX_PAYLOAD_SIZE ; ctx->sk_read_sock = sk->sk_socket->ops->read_sock; + ctx->ops = ops;update_sk_prot (sk, ctx); out:write_unlock_bh (&sk->sk_callback_lock);
net/tls/tls_strp.c +18 -8
@@ -120,6 +120,7 @@ struct sk_buff *tls_strp_msg_detach(struct tls_sw_context_rx *ctx)int tls_strp_msg_cow (struct tls_sw_context_rx *ctx) {struct tls_strparser *strp = &ctx->strp; +struct tls_context *tls_ctx =tls_get_ctx (strp->sk);struct sk_buff *skb;if (strp->copy_mode)
@@ -132,7 +133,7 @@ int tls_strp_msg_cow(struct tls_sw_context_rx *ctx)tls_strp_anchor_free (strp); strp->anchor = skb; -tcp_read_done (strp->sk, strp->stm.full_len); + tls_ctx->ops->read_done (strp->sk, strp->stm.full_len); strp->copy_mode =1 ;return 0 ;
@@ -390,6 +391,7 @@ static int tls_strp_read_copyin(struct tls_strparser *strp)static int tls_strp_read_copy (struct tls_strparser *strp,bool qshort) { +struct tls_context *ctx =tls_get_ctx (strp->sk);struct skb_shared_info *shinfo;struct page *page;int need_spc, len;
@@ -398,7 +400,7 @@ static int tls_strp_read_copy(struct tls_strparser *strp, bool qshort)* to read the data out. Otherwise the connection will stall. * Without pressure threshold of INT_MAX will never be ready. */ -if (likely (qshort && !tcp_epollin_ready (strp->sk,INT_MAX ))) +if (likely (qshort && !ctx->ops->epollin_ready (strp->sk)))return 0 ; shinfo =skb_shinfo (strp->anchor);
@@ -434,12 +436,13 @@ static int tls_strp_read_copy(struct tls_strparser *strp, bool qshort)static bool tls_strp_check_queue_ok (struct tls_strparser *strp) {unsigned int len = strp->stm.offset + strp->stm.full_len; +struct tls_context *ctx =tls_get_ctx (strp->sk);struct sk_buff *first, *skb;u32 seq; first =skb_shinfo (strp->anchor)->frag_list; skb = first; - seq =TCP_SKB_CB (first)->seq; + seq = ctx->ops->get_skb_seq (first);/* Make sure there's no duplicate data in the queue, * and the decrypted status matches.
@@ -449,7 +452,7 @@ static bool tls_strp_check_queue_ok(struct tls_strparser *strp) len -= skb->len; skb = skb->next; -if (TCP_SKB_CB (skb)->seq != seq) +if (ctx->ops->get_skb_seq (skb) != seq)return false;if (skb_cmp_decrypted (first, skb))return false;
@@ -460,11 +463,11 @@ static bool tls_strp_check_queue_ok(struct tls_strparser *strp)static void tls_strp_load_anchor_with_queue (struct tls_strparser *strp,int len) { -struct tcp_sock *tp =tcp_sk (strp->sk); +struct tls_context *ctx =tls_get_ctx (strp->sk);struct sk_buff *first;u32 offset; - first =tcp_recv_skb (strp->sk, tp->copied_seq, &offset); + first = ctx->ops->recv_skb (strp->sk, &offset);if (WARN_ON_ONCE (!first))return ;
@@ -565,6 +568,11 @@ void tls_strp_check_rcv(struct tls_strparser *strp, bool announce)/* Lower sock lock held */ void tls_strp_data_ready (struct tls_strparser *strp) { +struct tls_context *ctx =tls_get_ctx (strp->sk); + +if (!ctx) +return ; +/* This check is needed to synchronize with do_tls_strp_work. * do_tls_strp_work acquires a process lock (lock_sock) whereas * the lock held here is bh_lock_sock. The two locks can be
@@ -572,7 +580,7 @@ void tls_strp_data_ready(struct tls_strparser *strp)* allows a thread in BH context to safely check if the process * lock is held. In this case, if the lock is held, queue work. */ -if (sock_owned_by_user_nocheck (strp->sk)) { +if (ctx->ops->lock_is_held (strp->sk)) {queue_work (tls_strp_wq, &strp->work);return ; }
@@ -597,10 +605,12 @@ static void tls_strp_work(struct work_struct *w)*/ void tls_strp_msg_consume (struct tls_strparser *strp) { +struct tls_context *ctx =tls_get_ctx (strp->sk); +WARN_ON (!strp->stm.full_len);if (likely (!strp->copy_mode)) -tcp_read_done (strp->sk, strp->stm.full_len); + ctx->ops->read_done (strp->sk, strp->stm.full_len);else tls_strp_flush_anchor_copy (strp);
net/tls/tls_sw.c +4 -2
@@ -2171,7 +2171,8 @@ int tls_rx_msg_size(struct tls_strparser *strp, struct sk_buff *skb) }/* Linearize header to local buffer */ - ret =skb_copy_bits (skb, strp->stm.offset, header, prot->prepend_size); + ret = tls_ctx->ops->skb_get_header (skb, strp->stm.offset, header, + prot->prepend_size);if (ret <0 ) goto read_failure;
@@ -2202,7 +2203,8 @@ int tls_rx_msg_size(struct tls_strparser *strp, struct sk_buff *skb) }tls_device_rx_resync_new_rec (strp->sk, data_len +TLS_HEADER_SIZE , -TCP_SKB_CB (skb)->seq + strp->stm.offset); + tls_ctx->ops->get_skb_seq (skb) + + strp->stm.offset);return data_len +TLS_HEADER_SIZE ; read_failure:
Signature · 1 line
2.53.0
Geliang Tang
reviewerRFC 11/17 mptcp: implement mptcp-specific tls protocol ops
In reply to Geliang TangFrom: Geliang Tang <[email protected]>
This patch implements the MPTCP-specific struct tls_prot_ops, named 'tls_mptcp_ops'.
Passing an MPTCP socket to tcp_sock_rate_check_app_limited() can trigger a crash. Here, an MPTCP version of check_app_limited() is implemented, which calls tcp_sock_rate_check_app_limited() for each subflow.
When MPTCP implements lock_is_held interface, it not only checks sock_owned_by_user_nocheck(sk) as TCP does, but also needs to check whether the MPTCP data lock is held. This is required because TLS may call lock_is_held from softirq context with bh_lock_sock held. Checking both conditions ensures TLS always defers to workqueue when the MPTCP data lock is held, avoiding deadlock.
Implement mptcp_skb_get_header() to handle fragmented MPTCP skbs when copying TLS record headers.
In tls_strp_read_sock(), tls_strp_load_anchor_with_queue() first attaches the skbs from TCP/MPTCP to the frag_list of strp->anchor. In TCP, this is fine because the skb data is contiguous; however, in MPTCP, each skb has its own offset, causing the data to be non-contiguous. As a result, during the subsequent tls_rx_msg_size() process, skb_copy_bits() may access across skbs. In MPTCP, the offset of the second skb is ignored, leading to data access errors. Therefore, mptcp_skb_get_header() can effectively handle this problem and obtain the correct TLS header.
In the later process, tls_strp_check_queue_ok() handles the copy_mode scenario. When an MPTCP skb has a non-zero offset, it falls back to copy_mode, copying the valid data from each skb one by one into anchor->frag_list, thus resolving the offset issue. Hence, the impact of the offset within the TLS module is completely eliminated.
--- include/net/mptcp.h | 2 + include/net/tcp.h | 1 + net/ipv4/tcp.c | 9 +++- net/mptcp/protocol.c | 113 +++++++++++++++++++++++++++++++++++++++++++ net/mptcp/protocol.h | 1 + net/tls/tls_main.c | 13 +++++ 6 files changed, 137 insertions(+), 2 deletions(-)
include/net/mptcp.h +2 -0
@@ -132,6 +132,8 @@ struct mptcp_pm_ops {void (*release)(struct mptcp_sock *msk); }____cacheline_aligned_in_smp ; +extern struct tls_prot_ops tls_mptcp_ops; +#ifdef CONFIG_MPTCP void mptcp_init (void );
include/net/tcp.h +1 -0
@@ -849,6 +849,7 @@ static inline int tcp_bound_to_half_wnd(struct tcp_sock *tp, int pktsize)/* tcp.c */ void tcp_get_info (struct sock *,struct tcp_info *); +void tcp_sock_rate_check_app_limited (struct tcp_sock *tp);void tcp_rate_check_app_limited (struct sock *sk);/* Read 'sendfile()'-style from a TCP socket */
net/ipv4/tcp.c +7 -2
@@ -1096,9 +1096,9 @@ int tcp_sendmsg_fastopen(struct sock *sk, struct msghdr *msg, int *copied, }/* If a gap is detected between sends, mark the socket application-limited. */ -void tcp_rate_check_app_limited (struct sock *sk) +void tcp_sock_rate_check_app_limited (struct tcp_sock *tp) { -struct tcp_sock *tp =tcp_sk (sk); +struct sock *sk = (struct sock *)tp; if (/* We have less than one packet to send. */ tp->write_seq - tp->snd_nxt < tp->mss_cache &&
@@ -1111,6 +1111,11 @@ void tcp_rate_check_app_limited(struct sock *sk) tp->app_limited = (tp->delivered +tcp_packets_in_flight (tp)) ? :1 ; } + +void tcp_rate_check_app_limited (struct sock *sk) +{ +tcp_sock_rate_check_app_limited (tcp_sk (sk)); +}EXPORT_SYMBOL_GPL (tcp_rate_check_app_limited);int tcp_sendmsg_locked (struct sock *sk,struct msghdr *msg,size_t size)
net/mptcp/protocol.c +113 -0
122 lines not on this page · raw message
net/mptcp/protocol.h +1 -0
@@ -380,6 +380,7 @@ struct mptcp_sock {#define mptcp_data_lock (sk) spin_lock_bh(&(sk)->sk_lock.slock)#define mptcp_data_unlock (sk) spin_unlock_bh(&(sk)->sk_lock.slock) +#define mptcp_data_is_locked (sk) spin_is_locked(&(sk)->sk_lock.slock)#define mptcp_for_each_subflow (__msk, __subflow) \ list_for_each_entry(__subflow, &((__msk)->conn_list), node)
net/tls/tls_main.c +13 -0
@@ -1366,6 +1366,12 @@ static int __init tls_register(void)if (err) goto err_strp; +#ifdef CONFIG_MPTCP + err =tls_register_prot_ops (&tls_mptcp_ops); +if (err) + goto err_tcp; +#endif + err =tls_device_init ();if (err) goto err_ops;
@@ -1374,6 +1380,10 @@ static int __init tls_register(void)return 0 ; err_ops: +#ifdef CONFIG_MPTCP +tls_unregister_prot_ops (&tls_mptcp_ops); +err_tcp: +#endif tls_unregister_prot_ops (&tls_tcp_ops); err_strp:tls_strp_dev_exit ();
@@ -1385,6 +1395,9 @@ static int __init tls_register(void) static void __exit tls_unregister(void) {tcp_unregister_ulp (&tcp_tls_ulp_ops); +#ifdef CONFIG_MPTCP +tls_unregister_prot_ops (&tls_mptcp_ops); +#endif tls_unregister_prot_ops (&tls_tcp_ops);tls_strp_dev_exit ();tls_device_cleanup ();
Signature · 1 line
2.53.0
Geliang Tang
reviewerRFC 12/17 tls: add mptcp support for sk_poll
In reply to Geliang TangFrom: Geliang Tang <[email protected]>
The tls_sk_poll() function currently uses tcp_poll() unconditionally to obtain the base poll mask, which works only for TCP. This prevents TLS over MPTCP from working correctly with poll().
Make the poll function protocol-aware by selecting the appropriate poll function based on sk->sk_protocol. For TCP it calls tcp_poll(), for MPTCP it calls mptcp_poll() (guarded by CONFIG_MPTCP). Any other protocol returns 0.
Also export mptcp_poll() symbol so that the TLS module can use it.
--- include/net/mptcp.h | 9 +++++++++ net/mptcp/protocol.c | 5 +++-- net/tls/tls_main.c | 17 ++++++++++++++++- 3 files changed, 28 insertions(+), 3 deletions(-)
include/net/mptcp.h +9 -0
@@ -239,6 +239,9 @@ static inline __be32 mptcp_reset_option(const struct sk_buff *skb) }void mptcp_active_detect_blackhole (struct sock *sk,bool expired); + +__poll_t mptcp_poll (struct file *file,struct socket *sock, +struct poll_table_struct *wait);#else static inline void mptcp_init (void )
@@ -316,6 +319,12 @@ static inline struct request_sock *mptcp_subflow_reqsk_alloc(const struct requesstatic inline __be32 mptcp_reset_option (const struct sk_buff *skb) {return htonl (0u ); }static inline void mptcp_active_detect_blackhole (struct sock *sk,bool expired) { } + +static inline __poll_t mptcp_poll (struct file *file,struct socket *sock, +struct poll_table_struct *wait) +{ +return 0 ; +}#endif /* CONFIG_MPTCP */ #if IS_ENABLED (CONFIG_MPTCP_IPV6 )
net/mptcp/protocol.c +3 -2
@@ -4446,8 +4446,8 @@ static __poll_t mptcp_check_writeable(struct mptcp_sock *msk)return 0 ; } -static __poll_t mptcp_poll (struct file *file,struct socket *sock, -struct poll_table_struct *wait) +__poll_t mptcp_poll (struct file *file,struct socket *sock, +struct poll_table_struct *wait) {struct sock *sk = sock->sk;struct mptcp_sock *msk;
@@ -4494,6 +4494,7 @@ static __poll_t mptcp_poll(struct file *file, struct socket *sock,return mask; } +EXPORT_SYMBOL_GPL (mptcp_poll);static struct sk_buff *mptcp_recv_skb (struct sock *sk,u32 *off) {
net/tls/tls_main.c +16 -1
@@ -416,6 +416,21 @@ static void tls_sk_proto_close(struct sock *sk, long timeout)tls_ctx_free (sk, ctx); } +static __poll_t tls_proto_poll (struct file *file,struct socket *sock, +struct poll_table_struct *wait) +{ +switch (sock->sk->sk_protocol) { +case IPPROTO_TCP : +return tcp_poll (file, sock, wait); +#ifdef CONFIG_MPTCP +case IPPROTO_MPTCP : +return mptcp_poll (file, sock, wait); +#endif +default : +return 0 ; + } +} +static __poll_t tls_sk_poll (struct file *file,struct socket *sock,struct poll_table_struct *wait) {
@@ -426,7 +441,7 @@ static __poll_t tls_sk_poll(struct file *file, struct socket *sock,u8 shutdown;int state; - mask =tcp_poll (file, sock, wait); + mask =tls_proto_poll (file, sock, wait); state =inet_sk_state_load (sk); shutdown =READ_ONCE (sk->sk_shutdown);
Signature · 1 line
2.53.0
Geliang Tang
reviewerRFC 13/17 tls: disable device offload for mptcp sockets
In reply to Geliang TangFrom: Geliang Tang <[email protected]>
MPTCP TLS hardware offload is not yet implemented. Return -EOPNOTSUPP when attempting to enable device offload on MPTCP sockets.
--- net/tls/tls_device.c | 6 ++++++ 1 file changed, 6 insertions(+)
net/tls/tls_device.c +6 -0
@@ -1074,6 +1074,9 @@ int tls_set_device_offload(struct sock *sk) ctx =tls_get_ctx (sk); prot = &ctx->prot_info; +if (sk->sk_protocol ==IPPROTO_MPTCP ) +return -EOPNOTSUPP ; +if (ctx->priv_ctx_tx)return -EEXIST ;
@@ -1196,6 +1199,9 @@ int tls_set_device_offload_rx(struct sock *sk, struct tls_context *ctx)struct net_device *netdev;int rc =0 ; +if (sk->sk_protocol ==IPPROTO_MPTCP ) +return -EOPNOTSUPP ; +if (ctx->crypto_recv.info.version !=TLS_1_2_VERSION )return -EOPNOTSUPP ;
Signature · 1 line
2.53.0
Geliang Tang
reviewerRFC 14/17 mptcp: update mptcp_check_readable helper
In reply to Geliang TangFrom: Gang Yan <[email protected]>
This patch makes mptcp_check_readable() aligned with TCP, and renames it to mptcp_stream_is_readable(). It will be used in the case of KTLS, because 'prot' will be modified, tls_sw_sock_is_readable() is expected to be called from prot->sock_is_readable().
--- net/mptcp/protocol.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)
net/mptcp/protocol.c +6 -3
@@ -3410,9 +3410,11 @@ void __mptcp_unaccepted_force_close(struct sock *sk)__mptcp_destroy_sock (sk); } -static __poll_t mptcp_check_readable (struct sock *sk) +static bool mptcp_stream_is_readable (struct sock *sk) { -return mptcp_epollin_ready (sk) ?EPOLLIN |EPOLLRDNORM :0 ; +if (mptcp_epollin_ready (sk)) +return true; +return sk_is_readable (sk); }static void mptcp_check_listen_stop (struct sock *sk)
@@ -4476,7 +4478,8 @@ __poll_t mptcp_poll(struct file *file, struct socket *sock, mask |=EPOLLIN |EPOLLRDNORM |EPOLLRDHUP ;if (state !=TCP_SYN_SENT && state !=TCP_SYN_RECV ) { - mask |=mptcp_check_readable (sk); +if (mptcp_stream_is_readable (sk)) + mask |=EPOLLIN |EPOLLRDNORM ;if (shutdown &SEND_SHUTDOWN ) mask |=EPOLLOUT |EPOLLWRNORM ;else
Signature · 1 line
2.53.0
Geliang Tang
reviewerRFC 15/17 mptcp: implement ulp getsockopt for tls support
In reply to Geliang TangFrom: Geliang Tang <[email protected]>
Add mptcp_getsockopt_tcp_ulp() to handle TCP_ULP getsockopt on MPTCP sockets. The helper reads the user length once, checks for negative value, takes the socket lock, caps the length to TCP_ULP_NAME_MAX, and copies the ULP name (or sets the length to zero if no ULP is attached) to userspace. The lock ensures safe access to icsk->icsk_ulp_ops.
--- net/mptcp/sockopt.c | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+)
net/mptcp/sockopt.c +34 -0
@@ -1408,6 +1408,39 @@ static int mptcp_put_int_option(struct mptcp_sock *msk, char __user *optval,return 0 ; } +static int mptcp_getsockopt_tcp_ulp (struct sock *sk, +char __user *optval, +int __user *optlen) +{ +struct inet_connection_sock *icsk =inet_csk (sk); +int ret =0 , len; + +if (copy_from_sockptr (&len,USER_SOCKPTR (optlen),sizeof (int ))) +return -EFAULT ; + +if (len <0 ) +return -EINVAL ; + +lock_sock (sk); + len = min_t(unsigned int, len, TCP_ULP_NAME_MAX); +if (!icsk->icsk_ulp_ops) { + len =0 ; +if (copy_to_sockptr (USER_SOCKPTR (optlen), &len,sizeof (int ))) + ret = -EFAULT ; + goto out; + } +if (copy_to_sockptr (USER_SOCKPTR (optlen), &len,sizeof (int ))) { + ret = -EFAULT ; + goto out; + } +if (copy_to_sockptr (USER_SOCKPTR (optval), icsk->icsk_ulp_ops->name, + len)) + ret = -EFAULT ; +out: +release_sock (sk); +return ret; +} +static int mptcp_getsockopt_sol_tcp (struct mptcp_sock *msk,int optname,char __user *optval,int __user *optlen) {
@@ -1415,6 +1448,7 @@ static int mptcp_getsockopt_sol_tcp(struct mptcp_sock *msk, int optname,switch (optname) {case TCP_ULP : +return mptcp_getsockopt_tcp_ulp (sk, optval, optlen);case TCP_CONGESTION :case TCP_INFO :case TCP_CC_INFO :
Signature · 1 line
2.53.0
Geliang Tang
reviewerRFC 16/17 mptcp: implement ulp setsockopt for tls support
In reply to Geliang TangFrom: Geliang Tang <[email protected]>
Allow MPTCP sockets to set the TCP_ULP socket option to enable TLS. Add mptcp_setsockopt_tcp_ulp() which validates the socket state (must not be CLOSE or LISTEN), only accepts "tls" as the ULP name, and then calls tcp_set_ulp().
Include TCP_ULP in the list of supported options in supported_sockopt(), and handle it in setsockopt_sol_tcp() instead of returning -EOPNOTSUPP.
Call tcp_cleanup_ulp() in mptcp_destroy_common() to release ULP module's reference count.
--- net/mptcp/protocol.c | 1 + net/mptcp/sockopt.c | 34 +++++++++++++++++++++++++++++++++- 2 files changed, 34 insertions(+), 1 deletion(-)
net/mptcp/protocol.c +1 -0
@@ -3765,6 +3765,7 @@ static void mptcp_destroy(struct sock *sk)/* allow the following to close even the initial subflow */ msk->free_first =1 ;mptcp_destroy_common (msk); +tcp_cleanup_ulp (sk);sk_sockets_allocated_dec (sk); }
net/mptcp/sockopt.c +33 -1
@@ -577,6 +577,7 @@ static bool mptcp_supported_sockopt(int level, int optname)case TCP_FASTOPEN_CONNECT :case TCP_FASTOPEN_KEY :case TCP_FASTOPEN_NO_COOKIE : +case TCP_ULP :return true; }
@@ -830,6 +831,37 @@ static int mptcp_setsockopt_all_sf(struct mptcp_sock *msk, int level,return ret; } +static int mptcp_setsockopt_tcp_ulp (struct sock *sk,sockptr_t optval, +unsigned int optlen) +{ +char name[TCP_ULP_NAME_MAX ]; +int err =0 ; +size_t len; +int val; + +if (optlen <1 ) +return -EINVAL ; + + len =min_t (long,TCP_ULP_NAME_MAX -1 , optlen); + val =strncpy_from_sockptr (name, optval, len); +if (val <0 ) +return -EFAULT ; + name[val] =0 ; + +if (strcmp (name,"tls" )) +return -EOPNOTSUPP ; + +sockopt_lock_sock (sk); +if ((1 << sk->sk_state) & (TCPF_CLOSE |TCPF_LISTEN )) { + err = -ENOTCONN ; + goto out; + } + err =tcp_set_ulp (sk, name); +out: +sockopt_release_sock (sk); +return err; +} +static int mptcp_setsockopt_sol_tcp (struct mptcp_sock *msk,int optname,sockptr_t optval,unsigned int optlen) {
@@ -838,7 +870,7 @@ static int mptcp_setsockopt_sol_tcp(struct mptcp_sock *msk, int optname,switch (optname) {case TCP_ULP : -return -EOPNOTSUPP ; +return mptcp_setsockopt_tcp_ulp (sk, optval, optlen);case TCP_CONGESTION :return mptcp_setsockopt_sol_tcp_congestion (msk, optval, optlen);case TCP_DEFER_ACCEPT :
Signature · 1 line
2.53.0
Geliang Tang
reviewerRFC 17/17 selftests: mptcp: connect: use espintcp for ulp test
In reply to Geliang TangFrom: Geliang Tang <[email protected]>
With KTLS being implemented, "tls" should no longer be used in sock_test_tcpulp(), it breaks mptcp_connect.sh tests. Another ULP name, "espintcp", is set instead in this patch.
--- tools/testing/selftests/net/mptcp/config | 4 ++++ tools/testing/selftests/net/mptcp/mptcp_connect.c | 4 ++-- 2 files changed, 6 insertions(+), 2 deletions(-)
tools/testing/selftests/net/mptcp/config +4 -0
@@ -34,3 +34,7 @@ CONFIG_NFT_SOCKET=m CONFIG_NFT_TPROXY=m CONFIG_SYN_COOKIES=y CONFIG_VETH=y +CONFIG_INET_ESP=y +CONFIG_INET_ESPINTCP=y +CONFIG_INET6_ESP=y +CONFIG_INET6_ESPINTCP=y
tools/testing/selftests/net/mptcp/mptcp_connect.c +2 -2
@@ -285,11 +285,11 @@ static void sock_test_tcpulp(int sock, int proto, unsigned int line)if (buflen >0 ) {if (strcmp (buf,"mptcp" ) !=0 )xerror ("unexpected ULP '%s' for proto %d at line %u" , buf, proto, line); - ret =do_ulp_so (sock,"tls" ); + ret =do_ulp_so (sock,"espintcp" );if (ret ==0 )X ("setsockopt" ); }else if (proto ==IPPROTO_MPTCP ) { - ret =do_ulp_so (sock,"tls" ); + ret =do_ulp_so (sock,"espintcp" );if (ret !=-1 )X ("setsockopt" ); }
Signature · 1 line
2.53.0
Hi Geliang,
Thank you for your modifications, that's great!
Our CI did some validations and here is its report:
- KVM Validation: normal (except selftest_mptcp_join): Success! ✅
- KVM Validation: normal (only selftest_mptcp_join): Success! ✅
- KVM Validation: debug (except selftest_mptcp_join): Notice: Call Traces at boot time, rebooted and continued ⚠️
- KVM Validation: debug (only selftest_mptcp_join): Notice: Call Traces at boot time, rebooted and continued ⚠️
- KVM Validation: btf-normal (only bpftest_all): Success! ✅
- KVM Validation: btf-debug (only bpftest_all): Notice: Call Traces at boot time, rebooted and continued ⚠️
- Task: https://github.com/multipath-tcp/mptcp_net-next/actions/runs/27948260475
Initiator: Patchew Applier
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/4fba43978071
Patchwork: https://patchwork.kernel.org/project/mptcp/list/?series=1114686
If there are some issues, you can reproduce them using the same environment as the one used by the CI thanks to a docker image, e.g.:
$ cd [kernel source code]
$ docker run -v "${PWD}:${PWD}:rw" -w "${PWD}" --privileged --rm -it \
--pull always mptcp/mptcp-upstream-virtme-docker:latest \
auto-normalFor more details:
https://github.com/multipath-tcp/mptcp-upstream-virtme-docker
Please note that despite all the efforts that have been already done to have a stable tests suite when executed on a public CI like here, it is possible some reported issues are not due to your modifications. Still, do not hesitate to help us improve that ;-)
Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (NGI0 Core)
Jakub Kicinski
maintainerQuoted 1 line · On Mon, 22 Jun 2026 18:43:20 +0800 Geliang Tang wrote:
Subject: [RFC net-next 00/17] MPTCP KTLS support
Please no. We have a ton of unfixed bugs and may have to revert some of the features we dropped back in. I'd prefer to avoid large new bug surfaces until we reach an LTS release.
Quoted 6 lines · Jakub Kicinski
Quoted 1 line · On Mon, 22 Jun 2026 18:43:20 +0800 Geliang Tang wrote:
Subject: [RFC net-next 00/17] MPTCP KTLS support
Please no. We have a ton of unfixed bugs and may have to revert some of the features we dropped back in. I'd prefer to avoid large new bug surfaces until we reach an LTS release.
Is this specifically for kTLS? Or the whole networking subsystem?
Asking because if it is the latter you probably want to announce this new rule offically and update Documentation/process/maintainer-netdev.rst
Regards,
Cong
Geliang Tang
reviewerQuoted 7 lines · Jakub Kicinski
Quoted 1 line · On Mon, 22 Jun 2026 18:43:20 +0800 Geliang Tang wrote:
Subject: [RFC net-next 00/17] MPTCP KTLS support
Please no. We have a ton of unfixed bugs and may have to revert some of
the features we dropped back in. I'd prefer to avoid large new bug surfaces until we reach an LTS release.
Sure, I can wait. During this time, I'll go over the implementation more carefully to make sure there are no issues on the MPTCP side.
Thanks,
-Geliang
Jakub Kicinski
maintainerQuoted 11 lines · On Fri, 26 Jun 2026 15:56:53 +0800 Geliang Tang wrote:
Quoted 7 lines · Jakub Kicinski
Quoted 1 line · On Mon, 22 Jun 2026 18:43:20 +0800 Geliang Tang wrote:
Subject: [RFC net-next 00/17] MPTCP KTLS support
Please no. We have a ton of unfixed bugs and may have to revert some of
the features we dropped back in. I'd prefer to avoid large new bug surfaces until we reach an LTS release.Sure, I can wait. During this time, I'll go over the implementation more carefully to make sure there are no issues on the MPTCP side.
The two things that I'd be most interested in understanding better are
1) perf impact on all the pointer chasing and indirect calls you're adding
2) what's your plan on implementing HW offload? I understand that _you_ may not care, but if you follow the ML you'll discover that sooner or later some well meaning person will slop code such support. We have pretty solid (by kernel standards) documentation for the offload. Please read if you haven't already. And then you should hopefully realize the huge issue with the layering you chose...