Skip to contentSkip to conversation

mptcp: add MSG_ZEROCOPY support

PATCH Superseded Geliang Tang 2 participants 12 messages Jul 16

Subsystems (4)

v6 of 7 Superseded Latest is v7 →

lore
Share & tools
lore

Geliang Tang

author reviewer
#0 lore raw

v6 cover mptcp: add MSG_ZEROCOPY support

From: Geliang Tang <[email protected]>

This series adds MSG_ZEROCOPY support for MPTCP sockets, allowing userspace to transmit data without intermediate kernel copies, and provides corresponding selftests to exercise the new path.

Patch 1 fixes an uninitialized memory leak in the IPv4 error queue handling for zerocopy completion notifications by adding an early check for SO_EE_ORIGIN_ZEROCOPY.

Patch 2 is a small cleanup along the mptcp_sendmsg path.

Patch 3 widens the offset field in struct mptcp_data_frag from u16 to u32 to support page sizes up to 4GB on architectures with large page sizes (256KB on PowerPC and Hexagon).

Patch 4 aligns struct mptcp_data_frag to 8 bytes on 32-bit architectures to prevent unaligned 64-bit access faults on ARM, MIPS, and SPARC.

Patch 5 removes the redundant orig_offset parameter from mptcp_carve_data_frag() since it merely duplicates pfrag->offset.

Patch 6 implements the core MSG_ZEROCOPY support. The design follows TCP's reference-counting model, with a single ubuf_info per sendmsg, tracked by three reference buckets: one held by sendmsg itself, one per MPTCP dfrag in the retransmission queue, and one per subflow SKB. Completion is reported only after all bytes are acknowledged at both the MPTCP and subflow levels. The feature is silently downgraded to a regular copy in cases where zero-copy cannot be safely performed (MSG_FASTOPEN, fallback mode, or memory pressure).

Patch 7 handles SO_ZEROCOPY in setsockopt.

Patch 8 adds a new 'zerocopy' I/O mode to the mptcp_connect selftest to enable testing and performance validation of the zero-copy send path.

Patch 9 adds a wrapper script to run the zerocopy selftest as part of the regular test suite, ensuring the zero-copy path is exercised in CI.

Patch 10 is a cleanup for mptcp_connect.c that closes listensock deterministically to avoid double-close or leaks across repeated iterations.

v6:
- Add patch 2 to widen offset field in struct mptcp_data_frag from u16 to u32 to support page sizes up to 4GB on architectures with large page sizes (256KB on PowerPC and Hexagon). This prevents offset wrap-around when page fragment offset exceeds 65535.
- Add patch 5 to fix uninitialized memory leak in IPv4 error queue handling for zerocopy completion notifications. Add early check for SO_EE_ORIGIN_ZEROCOPY in ipv4_datagram_support_cmsg() to prevent reading uninitialized memory from ip_hdr(skb)->saddr.
- Refactor zerocopy completion notification draining logic in selftests. Extract process_zc_cmsg() helper to process individual control messages. Extract drain_errqueue() helper to handle error queue draining. Simplify wait_for_zc_completions() main loop from 6 levels to 3 levels. Move variables to their actual usage scope for better maintainability.
- Use batched polling (200ms intervals with 5s total timeout) to drain completion notifications for large transfers that may generate multiple completion events.
- Parse error queue messages to verify all sent data has been ACKed before breaking the completion loop.
- Reset msg_controllen each iteration of the inner reap loop to prevent truncation of control messages.
- Use >= instead of > in the MPTFO size adjustment so file_size reaches 0 when the whole file is sent via fast open.

v5:
- A new patch to align struct mptcp_data_frag to 8 bytes on 32-bit.
- Prevent coalescing of dfrags with different ubuf instances into the same subflow skb to avoid premature completion notifications.
- Return -EFAULT instead of -EAGAIN when iov_iter_get_pages2() returns 0
to properly signal iterator exhaustion.
- Account for dfrag metadata overhead in sk_wmem_queued_add() to prevent bypassing socket memory limits.
- Widen data_len field from u16 to u32 to prevent truncation on 64KB-page kernels (ARM64, PPC64).
- In selftests, use batched polling (200ms intervals with 5s total timeout)
to drain completion notifications for large transfers that may generate multiple completion events.
- https://patchwork.kernel.org/project/mptcp/cover/[email protected]/

v4:
- two more cleanups, patch 2 and patch 7.
- net/mptcp/protocol.c, mptcp_sendmsg_frag(): pull the type-mismatch
check out of the if (can_coalesce) block and use the standard
skb_zcopy_pure() helper; mptcp_sendmsg_zerocopy_iter() now
calls iov_iter_revert() before put_page() in the kzalloc failure
path so a retry starts from the same iterator offset.
- tools/testing/selftests/net/mptcp/mptcp_connect.c, copyfd_io_zc():
reset msg_controllen each iteration of the inner reap loop using a
do {} while block; use >= instead of > in the MPTFO size
adjustment so file_size reaches 0 when the whole file is sent via
fast open.
- https://patchwork.kernel.org/project/mptcp/cover/[email protected]/

v3:
- Force fresh skb when mixing zerocopy and kernel-copy fragments.
- Simplify zero-copy availability check by using SOCK_ZEROCOPY flag
directly.
- Validate SO_ZEROCOPY value in setsockopt and avoid affecting
fallback subflow.
- In selftests, drain MPTFO partial data, use poll+recvmsg for completion notifications with timeout, and verify getsockopt round-trip.
- https://patchwork.kernel.org/project/mptcp/cover/[email protected]/

v2:
- patch 2, never mix PURE_ZEROCOPY frags with kernel-copy frags in the
same skb; handle fallback, return values.
- patch 3, a new patch to handle SO_ZEROCOPY in setsockopt.
- patch 4, set SO_ZEROCOPY, handle listen_mode, error queue.
- https://patchwork.kernel.org/project/mptcp/cover/[email protected]/

v1:
- https://patchwork.kernel.org/project/mptcp/cover/[email protected]/

Geliang Tang (10):
ipv4: fix uninitialized memory in zerocopy cmsg
mptcp: use local variable tp in sendmsg_frag
mptcp: widen offset field in mptcp_data_frag to u32
mptcp: align struct mptcp_data_frag to 8 bytes on 32-bit
mptcp: remove redundant orig_offset in carve_data_frag
mptcp: add MSG_ZEROCOPY support
mptcp: handle SO_ZEROCOPY in setsockopt
selftests: mptcp: connect: add zerocopy io mode
selftests: mptcp: connect: cover zerocopy mode
selftests: mptcp: connect: close listensock deterministically

 net/ipv4/ip_sockglue.c                        |   6 +
 net/mptcp/protocol.c                          | 139 ++++++++++--
 net/mptcp/protocol.h                          |   7 +-
 net/mptcp/sockopt.c                           |  17 +-
 tools/testing/selftests/net/mptcp/Makefile    |   1 +
 .../selftests/net/mptcp/mptcp_connect.c       | 203 +++++++++++++++++-
 .../net/mptcp/mptcp_connect_zerocopy.sh       |   5 +
 7 files changed, 347 insertions(+), 31 deletions(-)
 create mode 100755 tools/testing/selftests/net/mptcp/mptcp_connect_zerocopy.sh
Signature · 1 line
2.53.0
author reviewer
#1 lore raw

PATCH v6 1/10 ipv4: fix uninitialized memory in zerocopy cmsg

From: Geliang Tang <[email protected]>

When SOF_TIMESTAMPING_OPT_CMSG is enabled and applications read MSG_ZEROCOPY completion notifications from the error queue, the ipv4_datagram_support_cmsg() function attempts to read ip_hdr(skb)->saddr to populate IP_PKTINFO cmsg data. However, zerocopy completion notification skbs don't have IP headers, causing this read to access uninitialized memory which is then leaked to userspace via put_cmsg().

The issue occurs because the function checks PKTINFO_SKB_CB(skb)->ipi_ifindex (which aliases the ops pointer in ubuf_info_msgzc stored in skb->cb), and if non-NULL, proceeds to read ip_hdr(skb)->saddr without verifying that the skb actually has an IP header.

Fix this by adding an early check for SO_EE_ORIGIN_ZEROCOPY and returning false, since zerocopy completion notifications don't have IP headers and IP_PKTINFO cmsg is not applicable to them.

Fixes f2457533c4d3 ("ipv4: support IP_PKTINFO cmsg for SO_TIMESTAMPING")
---
 net/ipv4/ip_sockglue.c | 6 ++++++
 1 file changed, 6 insertions(+)
net/ipv4/ip_sockglue.c +6 -0
index a55ef327ec93..59513f15e522 100644
@@ -504,6 +504,12 @@ static bool ipv4_datagram_support_cmsg(const struct sock *sk,
 	if (ee_origin == SO_EE_ORIGIN_LOCAL)
 		return false;
 
+	/* Zerocopy completion notifications don't have IP headers,
+	 * so IP_PKTINFO cmsg is not applicable.
+	 */
+	if (ee_origin == SO_EE_ORIGIN_ZEROCOPY)
+		return false;
+
 	/* Support IP_PKTINFO on tstamp packets if requested, to correlate
 	 * timestamp with egress dev. Not possible for packets without iif
 	 * or without payload (SOF_TIMESTAMPING_OPT_TSONLY).
Signature · 1 line
2.53.0
author reviewer
#2 lore raw

PATCH v6 2/10 mptcp: use local variable tp in sendmsg_frag

In reply to Geliang Tang

From: Geliang Tang <[email protected]>

In mptcp_sendmsg_frag(), tcp_sk(ssk) is called multiple times. Introduce a local variable 'tp' to hold the tcp_sock pointer and use it throughout the function. This avoids repeated dereferencing and improves code readability, with no functional change.

---
 net/mptcp/protocol.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
net/mptcp/protocol.c +4 -3
index ffcf5a1788f6..1cab07b1b9b1 100644
@@ -1375,6 +1375,7 @@ static int mptcp_sendmsg_frag(struct sock *sk, struct sock *ssk,
 	u64 data_seq = dfrag->data_seq + info->sent;
 	int offset = dfrag->offset + info->sent;
 	struct mptcp_sock *msk = mptcp_sk(sk);
+	struct tcp_sock *tp = tcp_sk(ssk);
 	bool zero_window_probe = false;
 	struct mptcp_ext *mpext = NULL;
 	bool can_coalesce = false;
@@ -1410,14 +1411,14 @@ static int mptcp_sendmsg_frag(struct sock *sk, struct sock *ssk,
 		mpext = mptcp_get_ext(skb);
 		if (!mptcp_skb_can_collapse_to(data_seq, skb, mpext)) {
 			TCP_SKB_CB(skb)->eor = 1;
-			tcp_mark_push(tcp_sk(ssk), skb);
+			tcp_mark_push(tp, skb);
 			goto alloc_skb;
 		}
 
 		i = skb_shinfo(skb)->nr_frags;
 		can_coalesce = skb_can_coalesce(skb, i, dfrag->page, offset);
 		if (!can_coalesce && i >= READ_ONCE(net_hotdata.sysctl_max_skb_frags)) {
-			tcp_mark_push(tcp_sk(ssk), skb);
+			tcp_mark_push(tp, skb);
 			goto alloc_skb;
 		}
 
@@ -1471,7 +1472,7 @@ static int mptcp_sendmsg_frag(struct sock *sk, struct sock *ssk,
 	skb->truesize += copy;
 	sk_wmem_queued_add(ssk, copy);
 	sk_mem_charge(ssk, copy);
-	WRITE_ONCE(tcp_sk(ssk)->write_seq, tcp_sk(ssk)->write_seq + copy);
+	WRITE_ONCE(tp->write_seq, tp->write_seq + copy);
 	TCP_SKB_CB(skb)->end_seq += copy;
 	tcp_skb_pcount_set(skb, 0);
 
Signature · 1 line
2.53.0
author reviewer
#3 lore raw

PATCH v6 3/10 mptcp: widen offset field in mptcp_data_frag to u32

In reply to Geliang Tang

From: Geliang Tang <[email protected]>

The offset field in struct mptcp_data_frag is declared as u16, which can only represent values up to 65535. On architectures with page sizes larger than 64KB (such as 256KB on PowerPC and Hexagon), the offset calculation in mptcp_carve_data_frag() can exceed this limit, causing the u16 field to wrap around.

When offset wraps, mptcp_sendmsg() calculates the destination address using the wrapped value:

	offset = dfrag->offset + dfrag->data_len;
	page_address(dfrag->page) + offset

This could cause the function to write new network payload into the start of the page fragment, potentially overwriting the previously allocated struct mptcp_data_frag and corrupting kernel memory.

Widen the offset field from u16 to u32 to support page sizes up to 4GB. Also change the corresponding local variable in mptcp_sendmsg_frag() from int to size_t to prevent overflow when adding offset and data_len.

This is a correctness fix for architectures with 256KB pages and has no effect on systems with smaller page sizes.

---
 net/mptcp/protocol.c | 2 +-
 net/mptcp/protocol.h | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
net/mptcp/protocol.c +1 -1
index 1cab07b1b9b1..3acd61ae6160 100644
@@ -1373,7 +1373,7 @@ static int mptcp_sendmsg_frag(struct sock *sk, struct sock *ssk,
 			      struct mptcp_sendmsg_info *info)
 {
 	u64 data_seq = dfrag->data_seq + info->sent;
-	int offset = dfrag->offset + info->sent;
+	size_t offset = dfrag->offset + info->sent;
 	struct mptcp_sock *msk = mptcp_sk(sk);
 	struct tcp_sock *tp = tcp_sk(ssk);
 	bool zero_window_probe = false;
net/mptcp/protocol.h +1 -1
index da40c6f3705f..7d624cb0fa7d 100644
@@ -263,7 +263,7 @@ struct mptcp_data_frag {
 	struct list_head list;
 	u64 data_seq;
 	u16 data_len;
-	u16 offset;
+	u32 offset;		/* u16 wraps on 256KB-page kernels */
 	u8 overhead;
 	u8 eor;			/* currently using 1 bit */
 	u16 already_sent;
Signature · 1 line
2.53.0
author reviewer
#4 lore raw

PATCH v6 4/10 mptcp: align struct mptcp_data_frag to 8 bytes on 32-bit

In reply to Geliang Tang

From: Geliang Tang <[email protected]>

mptcp_carve_data_frag() places a struct mptcp_data_frag inside a page-frag region, using ALIGN(orig_offset, sizeof(long)) to compute the offset where the struct lives. On 64-bit Linux sizeof(long) is 8 and this works as intended. On 32-bit Linux sizeof(long) is 4, which only guarantees 4-byte alignment for the struct.

struct mptcp_data_frag contains a u64 data_seq member, which requires 8-byte alignment. 4-byte-aligned 64-bit loads fault on 32-bit architectures that do not handle unaligned 64-bit accesses in hardware (armv7, mips, sparc, ...). The in-source comment near BUILD_BUG_ON that states "ALIGN(1, sizeof(long)) - 1, so 8-1" is misleading because it implicitly assumes sizeof(long) is always 8.

Switch both the ALIGN() and the matching BUILD_BUG_ON to sizeof(u64) so the 8-byte alignment is enforced on every architecture, and the overhead u8 budget still fits the worst-case 7-byte padding plus the struct size. This is a no-op on 64-bit and a correctness fix on 32-bit.

---
 net/mptcp/protocol.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
net/mptcp/protocol.c +3 -3
index 3acd61ae6160..7ad3a4a09e3b 100644
@@ -1237,7 +1237,7 @@ static struct mptcp_data_frag *
 mptcp_carve_data_frag(const struct mptcp_sock *msk, struct page_frag *pfrag,
 		      int orig_offset)
 {
-	int offset = ALIGN(orig_offset, sizeof(long));
+	int offset = ALIGN(orig_offset, sizeof(u64));
 	struct mptcp_data_frag *dfrag;
 
 	dfrag = (struct mptcp_data_frag *)(page_to_virt(pfrag->page) + offset);
@@ -4789,9 +4789,9 @@ void __init mptcp_proto_init(void)
 	BUILD_BUG_ON(sizeof(struct mptcp_skb_cb) > sizeof_field(struct sk_buff, cb));
 
 	/* struct mptcp_data_frag: 'overhead' corresponds to the alignment
-	 * (ALIGN(1, sizeof(long)) - 1, so 8-1) + the struct's size
+	 * (ALIGN(1, sizeof(u64)) - 1, so 8-1) + the struct's size
 	 */
-	BUILD_BUG_ON(ALIGN(1, sizeof(long)) - 1 + sizeof(struct mptcp_data_frag)
+	BUILD_BUG_ON(ALIGN(1, sizeof(u64)) - 1 + sizeof(struct mptcp_data_frag)
 		     > U8_MAX);
 }
 
Signature · 1 line
2.53.0
author reviewer
#5 lore raw

PATCH v6 5/10 mptcp: remove redundant orig_offset in carve_data_frag

In reply to Geliang Tang

From: Geliang Tang <[email protected]>

mptcp_carve_data_frag() already receives struct page_frag via its 'pfrag' parameter, making the separate 'orig_offset' parameter redundant since it merely duplicates pfrag->offset.

Simplify the function signature by removing this unnecessary parameter and directly accessing the offset from struct pfrag internally.

---
 net/mptcp/protocol.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)
net/mptcp/protocol.c +5 -4
index 7ad3a4a09e3b..85bbcb16c87e 100644
@@ -1234,12 +1234,13 @@ static bool mptcp_page_frag_refill(struct sock *sk, struct page_frag *pfrag)
 }
 
 static struct mptcp_data_frag *
-mptcp_carve_data_frag(const struct mptcp_sock *msk, struct page_frag *pfrag,
-		      int orig_offset)
+mptcp_carve_data_frag(const struct mptcp_sock *msk, struct page_frag *pfrag)
 {
-	int offset = ALIGN(orig_offset, sizeof(u64));
+	int orig_offset = pfrag->offset;
 	struct mptcp_data_frag *dfrag;
+	int offset;
 
+	offset = ALIGN(orig_offset, sizeof(u64));
 	dfrag = (struct mptcp_data_frag *)(page_to_virt(pfrag->page) + offset);
 	dfrag->data_len = 0;
 	dfrag->data_seq = msk->write_seq;
@@ -2031,7 +2032,7 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)
 			if (!mptcp_page_frag_refill(sk, pfrag))
 				goto wait_for_memory;
 
-			dfrag = mptcp_carve_data_frag(msk, pfrag, pfrag->offset);
+			dfrag = mptcp_carve_data_frag(msk, pfrag);
 			frag_truesize = dfrag->overhead;
 		}
 
Signature · 1 line
2.53.0
author reviewer
#6 lore raw

PATCH v6 6/10 mptcp: add MSG_ZEROCOPY support

In reply to Geliang Tang

From: Geliang Tang <[email protected]>

This patch enables MSG_ZEROCOPY support for MPTCP sockets, allowing user-space to transmit data without intermediate kernel copies.

The implementation follows the same lifecycle model as TCP:

A single ubuf_info_msgzc is allocated per sendmsg via msg_zerocopy_realloc(). Its reference count consists of three parts:
one 'caller' reference held until sendmsg returns, one reference per MPTCP dfrag in the retransmission queue (released when the dfrag is acked and cleared), and one reference per subflow skb that consumes dfrag bytes (released by TCP's skb_zcopy_clear() when the skb is acked). Completion is reported only after all bytes are acknowledged at both MPTCP and subflow levels.

User pages are pinned via iov_iter_get_pages2() and stored directly in a newly allocated mptcp_data_frag. The dfrag takes one reference on the ubuf_info and one on the page. Subflow skbs take additional page references via get_page() and are marked with skb_zcopy_set() and SKBFL_PURE_ZEROCOPY, mirroring TCP's behavior and bypassing kernel memory accounting for those pages.

The feature is silently downgraded to regular copy when MSG_FASTOPEN is set or the application has not opted in via setsockopt(SO_ZEROCOPY). If msg_zerocopy_realloc() fails (e.g. RLIMIT_MEMLOCK pressure), sendmsg(2) aborts with -ENOBUFS.

The implementation prevents coalescing of dfrags with different ubuf instances into the same subflow skb to avoid premature completion notifications. The data_len field is widened from u16 to u32 to prevent truncation on 64KB-page kernels (ARM64, PPC64). The dfrag metadata overhead is accounted for in sk_wmem_queued_add() to prevent bypassing socket memory limits. When iov_iter_get_pages2() returns 0, the function returns -EFAULT to properly signal iterator exhaustion.

---
 net/mptcp/protocol.c | 117 +++++++++++++++++++++++++++++++++++++++----
 net/mptcp/protocol.h |   5 +-
 2 files changed, 111 insertions(+), 11 deletions(-)
net/mptcp/protocol.c +108 -9
index 85bbcb16c87e..9cb32bcfa58b 100644
@@ -1118,9 +1118,11 @@ static bool mptcp_frag_can_collapse_to(const struct mptcp_sock *msk,
 		df->data_seq + df->data_len == msk->write_seq;
 }
 
-static void dfrag_uncharge(struct sock *sk, int len)
+static void dfrag_uncharge(struct sock *sk, int len,
+			   struct mptcp_data_frag *dfrag)
 {
-	sk_mem_uncharge(sk, len);
+	if (!dfrag->ubuf)
+		sk_mem_uncharge(sk, len);
 	sk_wmem_queued_add(sk, -len);
 }
 
@@ -1129,7 +1131,13 @@ static void dfrag_clear(struct sock *sk, struct mptcp_data_frag *dfrag)
 	int len = dfrag->data_len + dfrag->overhead;
 
 	list_del(&dfrag->list);
-	dfrag_uncharge(sk, len);
+	dfrag_uncharge(sk, len, dfrag);
+	if (dfrag->ubuf) {
+		net_zcopy_put(dfrag->ubuf);
+		put_page(dfrag->page);
+		kfree(dfrag);
+		return;
+	}
 	put_page(dfrag->page);
 }
 
@@ -1174,7 +1182,7 @@ static void __mptcp_clean_una(struct sock *sk)
 		dfrag->data_len -= delta;
 		dfrag->already_sent -= delta;
 
-		dfrag_uncharge(sk, delta);
+		dfrag_uncharge(sk, delta, dfrag);
 	}
 
 	/* all retransmitted data acked, recovery completed */
@@ -1249,6 +1257,7 @@ mptcp_carve_data_frag(const struct mptcp_sock *msk, struct page_frag *pfrag)
 	dfrag->already_sent = 0;
 	dfrag->page = pfrag->page;
 	dfrag->eor = 0;
+	dfrag->ubuf = NULL;
 
 	return dfrag;
 }
@@ -1257,7 +1266,7 @@ struct mptcp_sendmsg_info {
 	int mss_now;
 	int size_goal;
 	u16 limit;
-	u16 sent;
+	u32 sent;		/* matches dfrag->already_sent width */
 	unsigned int flags;
 	bool data_lock_held;
 };
@@ -1416,6 +1425,16 @@ static int mptcp_sendmsg_frag(struct sock *sk, struct sock *ssk,
 			goto alloc_skb;
 		}
 
+		if (skb_zcopy_pure(skb) != !!dfrag->ubuf) {
+			tcp_mark_push(tp, skb);
+			goto alloc_skb;
+		}
+		if (dfrag->ubuf && skb_zcopy(skb) &&
+		    skb_zcopy(skb) != dfrag->ubuf) {
+			tcp_mark_push(tp, skb);
+			goto alloc_skb;
+		}
+
 		i = skb_shinfo(skb)->nr_frags;
 		can_coalesce = skb_can_coalesce(skb, i, dfrag->page, offset);
 		if (!can_coalesce && i >= READ_ONCE(net_hotdata.sysctl_max_skb_frags)) {
@@ -1456,7 +1475,7 @@ static int mptcp_sendmsg_frag(struct sock *sk, struct sock *ssk,
 	}
 
 	copy = min_t(size_t, copy, info->limit - info->sent);
-	if (!sk_wmem_schedule(ssk, copy)) {
+	if (!dfrag->ubuf && !sk_wmem_schedule(ssk, copy)) {
 		tcp_remove_empty_skb(ssk);
 		return -ENOMEM;
 	}
@@ -1466,13 +1485,19 @@ static int mptcp_sendmsg_frag(struct sock *sk, struct sock *ssk,
 	} else {
 		get_page(dfrag->page);
 		skb_fill_page_desc(skb, i, dfrag->page, offset, copy);
+
+		if (dfrag->ubuf && !skb_zcopy(skb)) {
+			skb_zcopy_set(skb, dfrag->ubuf, NULL);
+			skb_shinfo(skb)->flags |= SKBFL_PURE_ZEROCOPY;
+		}
 	}
 
 	skb->len += copy;
 	skb->data_len += copy;
 	skb->truesize += copy;
 	sk_wmem_queued_add(ssk, copy);
-	sk_mem_charge(ssk, copy);
+	if (!skb_zcopy_pure(skb))
+		sk_mem_charge(ssk, copy);
 	WRITE_ONCE(tp->write_seq, tp->write_seq + copy);
 	TCP_SKB_CB(skb)->end_seq += copy;
 	tcp_skb_pcount_set(skb, 0);
@@ -1969,22 +1994,81 @@ static void mptcp_rps_record_subflows(const struct mptcp_sock *msk)
 	}
 }
 
+static int mptcp_sendmsg_zerocopy_iter(struct sock *sk, struct msghdr *msg,
+				       struct ubuf_info *ubuf, u32 copy_limit,
+				       ssize_t *copied, long *timeo)
+{
+	struct mptcp_sock *msk = mptcp_sk(sk);
+	struct mptcp_data_frag *dfrag;
+	struct page *pages[1] = {};
+	size_t pg_off = 0;
+	ssize_t pg_len;
+
+	pg_len = iov_iter_get_pages2(&msg->msg_iter, pages,
+				     copy_limit, 1, &pg_off);
+	if (pg_len < 0)
+		return pg_len;
+	/* pg_len == 0 means the iterator is exhausted */
+	if (pg_len == 0)
+		return -EFAULT;
+
+	dfrag = kzalloc_obj(*dfrag, GFP_KERNEL_ACCOUNT);
+	if (!dfrag) {
+		iov_iter_revert(&msg->msg_iter, pg_len);
+		put_page(pages[0]);
+		return -ENOMEM;
+	}
+	dfrag->data_len = pg_len;
+	dfrag->data_seq = msk->write_seq;
+	dfrag->offset = pg_off;
+	dfrag->page = pages[0];
+	dfrag->ubuf = ubuf;
+	dfrag->overhead = sizeof(struct mptcp_data_frag);
+
+	/* one ref for the dfrag; released in dfrag_clear */
+	net_zcopy_get(ubuf);
+
+	*copied += pg_len;
+	WRITE_ONCE(msk->write_seq, msk->write_seq + pg_len);
+	sk_wmem_queued_add(sk, pg_len + dfrag->overhead);
+
+	list_add_tail(&dfrag->list, &msk->rtx_queue);
+	if (!msk->first_pending)
+		msk->first_pending = dfrag;
+
+	return 0;
+}
+
 static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)
 {
 	struct mptcp_sock *msk = mptcp_sk(sk);
+	struct ubuf_info *ubuf = NULL;
 	struct page_frag *pfrag;
 	size_t copied = 0;
+	bool zc = false;
 	int ret = 0;
 	long timeo;
 
 	/* silently ignore everything else */
 	msg->msg_flags &= MSG_MORE | MSG_DONTWAIT | MSG_NOSIGNAL |
-			  MSG_FASTOPEN | MSG_EOR;
+			  MSG_FASTOPEN | MSG_EOR | MSG_ZEROCOPY;
 
 	lock_sock(sk);
 
 	mptcp_rps_record_subflows(msk);
 
+	if (msg->msg_flags & MSG_FASTOPEN || !sock_flag(sk, SOCK_ZEROCOPY))
+		msg->msg_flags &= ~MSG_ZEROCOPY;
+
+	if ((msg->msg_flags & MSG_ZEROCOPY) && len) {
+		ubuf = msg_zerocopy_realloc(sk, len, NULL, false);
+		if (!ubuf) {
+			ret = -ENOBUFS;
+			goto do_error;
+		}
+		zc = true;
+	}
+
 	if (unlikely(inet_test_bit(DEFER_CONNECT, sk) ||
 		     msg->msg_flags & MSG_FASTOPEN)) {
 		int copied_syn = 0;
@@ -2023,6 +2107,15 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)
 		if (!copy_limit)
 			goto wait_for_memory;
 
+		if (zc) {
+			ret = mptcp_sendmsg_zerocopy_iter(sk, msg,
+							  ubuf, copy_limit,
+							  &copied, &timeo);
+			if (ret)
+				goto do_error;
+			continue;
+		}
+
 		/* reuse tail pfrag, if possible, or carve a new one from the
 		 * page allocator
 		 */
@@ -2098,6 +2191,8 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)
 	}
 
 out:
+	if (zc)
+		net_zcopy_put(ubuf);
 	release_sock(sk);
 	return copied;
 
@@ -2106,6 +2201,10 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)
 		goto out;
 
 	copied = sk_stream_error(sk, msg->msg_flags, ret);
+	if (zc) {
+		net_zcopy_put_abort(ubuf, true);
+		zc = false;
+	}
 	goto out;
 }
 
@@ -2960,7 +3059,7 @@ static void __mptcp_retrans(struct sock *sk)
 
 		retrans_seq += len;
 		msk->bytes_retrans += len;
-		dfrag->already_sent = max_t(u16, dfrag->already_sent,
+		dfrag->already_sent = max_t(u32, dfrag->already_sent,
 					    retrans_seq - dfrag->data_seq);
 
 		/* With csum enabled retransmission can send new data. */
net/mptcp/protocol.h +3 -2
index 7d624cb0fa7d..830733327e66 100644
@@ -262,12 +262,13 @@ struct mptcp_pm_addr_entry {
 struct mptcp_data_frag {
 	struct list_head list;
 	u64 data_seq;
-	u16 data_len;
+	u32 data_len;		/* u16 truncates on 64KB-page kernels */
 	u32 offset;		/* u16 wraps on 256KB-page kernels */
 	u8 overhead;
 	u8 eor;			/* currently using 1 bit */
-	u16 already_sent;
+	u32 already_sent;	/* u16 overflows when data_len > 65535 */
 	struct page *page;
+	struct ubuf_info *ubuf;
 };
 
 /* Arbitrary compromise between as low as possible to react timely to subflow
Signature · 1 line
2.53.0
author reviewer
#7 lore raw

PATCH v6 7/10 mptcp: handle SO_ZEROCOPY in setsockopt

In reply to Geliang Tang

From: Geliang Tang <[email protected]>

SO_ZEROCOPY is currently rejected by MPTCP setsockopt with -EOPNOTSUPP, even though the matching MSG_ZEROCOPY path in mptcp_sendmsg() is wired up. As a result MSG_ZEROCOPY silently falls back to a copy on MPTCP sockets because the opt-in flag is never set.

Expose SO_ZEROCOPY through the SOL_SOCKET setsockopt dispatcher. The new mptcp_setsockopt_sol_socket_zerocopy() helper validates that val is 0 or 1 (matching standard TCP sk_setsockopt) and flips SOCK_ZEROCOPY on the msk only. mptcp_sendmsg() consults the same flag directly, so opt-in is honoured on every data path without any cross-socket propagation.

---
 net/mptcp/sockopt.c | 17 ++++++++++++++++-
 1 file changed, 16 insertions(+), 1 deletion(-)
net/mptcp/sockopt.c +16 -1
index fcf6feb2a9eb..abcc7d4452b4 100644
@@ -175,6 +175,19 @@ static int mptcp_setsockopt_sol_socket_tstamp(struct mptcp_sock *msk, int optnam
 	return 0;
 }
 
+static int mptcp_setsockopt_sol_socket_zerocopy(struct mptcp_sock *msk, int val)
+{
+	struct sock *sk = (struct sock *)msk;
+
+	if (val != 0 && val != 1)
+		return -EINVAL;
+
+	lock_sock(sk);
+	sock_valbool_flag(sk, SOCK_ZEROCOPY, val);
+	release_sock(sk);
+	return 0;
+}
+
 static int mptcp_setsockopt_sol_socket_int(struct mptcp_sock *msk, int optname,
 					   sockptr_t optval,
 					   unsigned int optlen)
@@ -203,6 +216,8 @@ static int mptcp_setsockopt_sol_socket_int(struct mptcp_sock *msk, int optname,
 	case SO_TIMESTAMPNS_OLD:
 	case SO_TIMESTAMPNS_NEW:
 		return mptcp_setsockopt_sol_socket_tstamp(msk, optname, val);
+	case SO_ZEROCOPY:
+		return mptcp_setsockopt_sol_socket_zerocopy(msk, val);
 	}
 
 	return -ENOPROTOOPT;
@@ -342,6 +357,7 @@ static int mptcp_setsockopt_sol_socket(struct mptcp_sock *msk, int optname,
 	case SO_TIMESTAMP_NEW:
 	case SO_TIMESTAMPNS_OLD:
 	case SO_TIMESTAMPNS_NEW:
+	case SO_ZEROCOPY:
 		return mptcp_setsockopt_sol_socket_int(msk, optname, optval,
 						       optlen);
 	case SO_TIMESTAMPING_OLD:
@@ -387,7 +403,6 @@ static int mptcp_setsockopt_sol_socket(struct mptcp_sock *msk, int optname,
 	 * SO_CNX_ADVICE is currently unsupported, could possibly be relevant,
 	 * but likely needs careful design
 	 *
-	 * SO_ZEROCOPY is currently unsupported, TODO in sndmsg
 	 * SO_TXTIME is currently unsupported
 	 */
 
Signature · 1 line
2.53.0
author reviewer
#8 lore raw

PATCH v6 8/10 selftests: mptcp: connect: add zerocopy io mode

In reply to Geliang Tang

From: Geliang Tang <[email protected]>

Add a new I/O mode 'zerocopy' to the mptcp_connect selftest, which uses sendmsg() with the MSG_ZEROCOPY flag to transmit data. This enables testing and performance validation of the zero-copy send path for both TCP and MPTCP connections.

This mode is useful for benchmarking and for verifying correctness of MSG_ZEROCOPY handling in the MPTCP stack.

The completion notification draining uses batched polling to handle large transfers that may generate multiple completion events across different TCP segments.

---
 .../selftests/net/mptcp/mptcp_connect.c       | 179 +++++++++++++++++-
 1 file changed, 178 insertions(+), 1 deletion(-)
tools/testing/selftests/net/mptcp/mptcp_connect.c +178 -1
index ea4cb6c1bd5e..63eb6e4dd4a7 100644
@@ -33,6 +33,7 @@
 #include <linux/tcp.h>
 #include <linux/time_types.h>
 #include <linux/sockios.h>
+#include <linux/errqueue.h>
 #include <linux/compiler.h>
 
 extern int optind;
@@ -53,6 +54,7 @@ enum cfg_mode {
 	CFG_MODE_MMAP,
 	CFG_MODE_SENDFILE,
 	CFG_MODE_SPLICE,
+	CFG_MODE_ZEROCOPY,
 };
 
 enum cfg_peek {
@@ -125,7 +127,7 @@ static void die_usage(void)
 	fprintf(stderr, "\t-j     -- add additional sleep at connection start and tear down "
 		"-- for MPJ tests\n");
 	fprintf(stderr, "\t-l     -- listens mode, accepts incoming connection\n");
-	fprintf(stderr, "\t-m [poll|mmap|sendfile|splice] -- use poll(default)/mmap+write/sendfile/splice\n");
+	fprintf(stderr, "\t-m [poll|mmap|sendfile|splice|zerocopy] -- use poll(default)/mmap+write/sendfile/splice/zerocopy\n");
 	fprintf(stderr, "\t-M mark -- set socket packet mark\n");
 	fprintf(stderr, "\t-o option -- test sockopt <option>\n");
 	fprintf(stderr, "\t-p num -- use port num\n");
@@ -998,6 +1000,167 @@ static int copyfd_io_splice(int infd, int peerfd, int outfd, unsigned int size,
 	return err;
 }
 
+static bool process_zc_cmsg(struct msghdr *errmsg, unsigned int size)
+{
+	struct sock_extended_err *err;
+	uint32_t bytes_completed = 0;
+	struct cmsghdr *cm;
+
+	for (cm = CMSG_FIRSTHDR(errmsg); cm; cm = CMSG_NXTHDR(errmsg, cm)) {
+		if (cm->cmsg_level != SOL_IP && cm->cmsg_level != SOL_IPV6)
+			continue;
+		if (cm->cmsg_type != IP_RECVERR &&
+		    cm->cmsg_type != IPV6_RECVERR)
+			continue;
+
+		err = (struct sock_extended_err *)CMSG_DATA(cm);
+		if (err->ee_origin != SO_EE_ORIGIN_ZEROCOPY)
+			continue;
+
+		bytes_completed += err->ee_data - err->ee_info + 1;
+		if (bytes_completed >= size)
+			return true;
+	}
+
+	return false;
+}
+
+static bool drain_errqueue(int peerfd, unsigned int size)
+{
+	size_t ctl_len = CMSG_SPACE(sizeof(struct sock_extended_err));
+	struct msghdr errmsg = {};
+	bool done = false;
+	char *ctl;
+
+	ctl = malloc(ctl_len);
+	if (!ctl) {
+		perror("malloc ctl");
+		return false;
+	}
+
+	errmsg.msg_control = ctl;
+	errmsg.msg_controllen = ctl_len;
+
+	while (1) {
+		errmsg.msg_controllen = ctl_len;
+		if (recvmsg(peerfd, &errmsg, MSG_ERRQUEUE) < 0)
+			break;
+
+		if (process_zc_cmsg(&errmsg, size)) {
+			done = true;
+			break;
+		}
+	}
+
+	free(ctl);
+	return done;
+}
+
+static void wait_for_zc_completions(int peerfd, unsigned int size)
+{
+	struct pollfd pfd = { .fd = peerfd, .events = POLLERR };
+	int total_ms = 5000;
+	int batch_ms = 200;
+	int max, i = 0;
+
+	if (cfg_time > 0)
+		total_ms = cfg_time;
+
+	max = total_ms / batch_ms;
+	if (max < 1)
+		max = 1;
+
+	while (i < max) {
+		if (poll(&pfd, 1, batch_ms) <= 0)
+			break;
+
+		i++;
+
+		if (drain_errqueue(peerfd, size))
+			break;
+	}
+}
+
+static int copyfd_io_zc(int infd, int peerfd, int outfd, unsigned int size,
+			bool *in_closed_after_out, struct wstate *winfo)
+{
+	struct msghdr msg = {};
+	int sndbuf = size * 2;
+	struct iovec iov;
+	socklen_t len;
+	int on = 1;
+	char *buf;
+	int err;
+
+	if (spool_buf(peerfd, winfo) < 0) {
+		perror("spool_buf");
+		return 1;
+	}
+
+	buf = malloc(size);
+	if (!buf) {
+		perror("malloc");
+		return 1;
+	}
+
+	if (read(infd, buf, size) != size) {
+		perror("read input");
+		err = 1;
+		goto free_bufs;
+	}
+
+	iov.iov_base = buf;
+	iov.iov_len = size;
+	msg.msg_iov = &iov;
+	msg.msg_iovlen = 1;
+
+	if (setsockopt(peerfd, SOL_SOCKET, SO_SNDBUF, &sndbuf, sizeof(sndbuf)))
+		perror("setsockopt SO_SNDBUF");
+
+	len = sizeof(on);
+	if (setsockopt(peerfd, SOL_SOCKET, SO_ZEROCOPY, &on, len)) {
+		perror("setsockopt SO_ZEROCOPY");
+		err = 1;
+		goto free_bufs;
+	}
+
+	on = -1;
+	if (getsockopt(peerfd, SOL_SOCKET, SO_ZEROCOPY, &on, &len) ||
+	    on != 1) {
+		perror("getsockopt SO_ZEROCOPY");
+		err = 1;
+		goto free_bufs;
+	}
+
+	if (listen_mode) {
+		if (do_recvfile(peerfd, outfd) < 0) {
+			err = 1;
+			goto free_bufs;
+		}
+	}
+
+	err = sendmsg(peerfd, &msg, MSG_ZEROCOPY);
+	if (err < 0 || (unsigned int)err != size) {
+		perror("sendmsg MSG_ZEROCOPY");
+		err = 1;
+		goto free_bufs;
+	}
+	err = 0;
+
+	if (!listen_mode) {
+		shut_wr(peerfd);
+		if (do_recvfile(peerfd, outfd) < 0)
+			err = 1;
+		*in_closed_after_out = true;
+	}
+
+	wait_for_zc_completions(peerfd, size);
+
+free_bufs:
+	free(buf);
+	return err;
+}
+
 static int copyfd_io(int infd, int peerfd, int outfd, bool close_peerfd, struct wstate *winfo)
 {
 	bool in_closed_after_out = false;
@@ -1038,6 +1201,17 @@ static int copyfd_io(int infd, int peerfd, int outfd, bool close_peerfd, struct
 				       &in_closed_after_out, winfo);
 		break;
 
+	case CFG_MODE_ZEROCOPY:
+		file_size = get_infd_size(infd);
+		if (file_size < 0)
+			return file_size;
+		if (cfg_sockopt_types.mptfo && winfo->total_len &&
+		    file_size >= winfo->total_len)
+			file_size -= winfo->total_len;
+		ret = copyfd_io_zc(infd, peerfd, outfd, file_size,
+				   &in_closed_after_out, winfo);
+		break;
+
 	default:
 		fprintf(stderr, "Invalid mode %d\n", cfg_mode);
 
@@ -1453,6 +1627,8 @@ int parse_mode(const char *mode)
 		return CFG_MODE_SENDFILE;
 	if (!strcasecmp(mode, "splice"))
 		return CFG_MODE_SPLICE;
+	if (!strcasecmp(mode, "zerocopy"))
+		return CFG_MODE_ZEROCOPY;
 
 	fprintf(stderr, "Unknown test mode: %s\n", mode);
 	fprintf(stderr, "Supported modes are:\n");
@@ -1460,6 +1636,7 @@ int parse_mode(const char *mode)
 	fprintf(stderr, "\t\t\"mmap\" - send entire input file (mmap+write), then read response (-l will read input first)\n");
 	fprintf(stderr, "\t\t\"sendfile\" - send entire input file (sendfile), then read response (-l will read input first)\n");
 	fprintf(stderr, "\t\t\"splice\" - send entire input file (splice), then read response (-l will read input first)\n");
+	fprintf(stderr, "\t\t\"zerocopy\" - send entire input file (zerocopy), then read response (-l will read input first)\n");
 
 	die_usage();
 
Signature · 1 line
2.53.0
author reviewer
#9 lore raw

PATCH v6 9/10 selftests: mptcp: connect: cover zerocopy mode

In reply to Geliang Tang

From: Geliang Tang <[email protected]>

Add a new test script mptcp_connect_zerocopy.sh to run the mptcp_connect selftest with the zerocopy I/O mode. This ensures the MSG_ZEROCOPY path is exercised in the regular test suite.

The script follows the same pattern as the existing mmap, sendfile, and splice test wrappers, invoking mptcp_connect.sh with the -m zerocopy option.

---
 tools/testing/selftests/net/mptcp/Makefile                  | 1 +
 tools/testing/selftests/net/mptcp/mptcp_connect_zerocopy.sh | 5 +++++
 2 files changed, 6 insertions(+)
 create mode 100755 tools/testing/selftests/net/mptcp/mptcp_connect_zerocopy.sh
tools/testing/selftests/net/mptcp/Makefile +1 -0
index 22ba0da2adb8..9e47ecfe58e2 100644
@@ -12,6 +12,7 @@ TEST_PROGS := \
 	mptcp_connect_mmap.sh \
 	mptcp_connect_sendfile.sh \
 	mptcp_connect_splice.sh \
+	mptcp_connect_zerocopy.sh \
 	mptcp_join.sh \
 	mptcp_sockopt.sh \
 	pm_netlink.sh \
tools/testing/selftests/net/mptcp/mptcp_connect_zerocopy.sh new file +5 -0
new file mode 100755
index 000000000000..0220d29ad68e
@@ -0,0 +1,5 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+
+MPTCP_LIB_KSFT_TEST="$(basename "${0}" .sh)" \
+	"$(dirname "${0}")/mptcp_connect.sh" -m zerocopy "${@}"
Signature · 1 line
2.53.0
author reviewer
#10 lore raw

PATCH v6 10/10 selftests: mptcp: connect: close listensock deterministically

In reply to Geliang Tang

From: Geliang Tang <[email protected]>

In main_loop_s(), listensock was closed nondeterministically inside maybe_close() during the accept loop, and error paths closed it without tracking, leading to double-close or leaks across repeated iterations (via goto again).

Fix by making maybe_close() return a bool indicating whether it closed the fd, and track that per iteration. Funnel poll/accept error paths through a single 'goto out' label and close the socket there only if not already closed. Reset 'closed' at the 'again:' label so each iteration starts with a fresh tracking state. This ensures exactly one close per iteration in all cases.

---
 .../selftests/net/mptcp/mptcp_connect.c       | 24 +++++++++++++------
 1 file changed, 17 insertions(+), 7 deletions(-)
tools/testing/selftests/net/mptcp/mptcp_connect.c +17 -7
index 63eb6e4dd4a7..1394ecaf56d3 100644
@@ -1340,12 +1340,15 @@ static void check_getpeername_connect(int fd)
 			cfg_host, a, cfg_port, b);
 }
 
-static void maybe_close(int fd)
+static bool maybe_close(int fd)
 {
 	unsigned int r = rand();
 
-	if (!(cfg_join || cfg_remove || cfg_repeat > 1) && (r & 1))
+	if (!(cfg_join || cfg_remove || cfg_repeat > 1) && (r & 1)) {
 		close(fd);
+		return true;
+	}
+	return false;
 }
 
 int main_loop_s(int listensock)
@@ -1355,27 +1358,30 @@ int main_loop_s(int listensock)
 	struct pollfd polls;
 	socklen_t salen;
 	int remotesock;
+	bool closed;
 	int err = 0;
 	int fd = 0;
 
 again:
+	closed = false;
 	polls.fd = listensock;
 	polls.events = POLLIN;
 
 	switch (poll(&polls, 1, poll_timeout)) {
 	case -1:
 		perror("poll");
-		return 1;
+		err = 1;
+		goto out;
 	case 0:
 		fprintf(stderr, "%s: timed out\n", __func__);
-		close(listensock);
-		return 2;
+		err = 2;
+		goto out;
 	}
 
 	salen = sizeof(ss);
 	remotesock = accept(listensock, (struct sockaddr *)&ss, &salen);
 	if (remotesock >= 0) {
-		maybe_close(listensock);
+		closed = maybe_close(listensock);
 		check_sockaddr(pf, &ss, salen);
 		check_getpeername(remotesock, &ss, salen);
 
@@ -1391,7 +1397,8 @@ int main_loop_s(int listensock)
 		err = copyfd_io(fd, remotesock, 1, true, &winfo);
 	} else {
 		perror("accept");
-		return 1;
+		err = 1;
+		goto out;
 	}
 
 	if (cfg_input)
@@ -1400,6 +1407,9 @@ int main_loop_s(int listensock)
 	if (!err && --cfg_repeat > 0)
 		goto again;
 
+out:
+	if (!closed)
+		close(listensock);
 	return err;
 }
 
Signature · 1 line
2.53.0
#11 lore raw
In reply to Geliang Tang

Hi Geliang,

Thank you for your modifications, that's great!

Our CI did some validations and here is its report:

- KVM Validation: normal (except selftest_mptcp_join): Success! ✅
- KVM Validation: normal (only selftest_mptcp_join): Success! ✅
- KVM Validation: debug (except selftest_mptcp_join): Success! ✅
- KVM Validation: debug (only selftest_mptcp_join): Success! ✅
- KVM Validation: btf-normal (only bpftest_all): Success! ✅
- KVM Validation: btf-debug (only bpftest_all): Success! ✅
- Task: https://github.com/multipath-tcp/mptcp_net-next/actions/runs/29482890291

Initiator: Patchew Applier
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/6890940ec099
Patchwork: https://patchwork.kernel.org/project/mptcp/list/?series=1128612

If there are some issues, you can reproduce them using the same environment as the one used by the CI thanks to a docker image, e.g.:

    $ cd [kernel source code]
    $ docker run -v "${PWD}:${PWD}:rw" -w "${PWD}" --privileged --rm -it \
        --pull always mptcp/mptcp-upstream-virtme-docker:latest \
        auto-normal

For more details:

https://github.com/multipath-tcp/mptcp-upstream-virtme-docker

Please note that despite all the efforts that have been already done to have a stable tests suite when executed on a public CI like here, it is possible some reported issues are not due to your modifications. Still, do not hesitate to help us improve that ;-)

Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (NGI0 Core)