Skip to contentSkip to conversation

implement .splice_eof

PATCH RESEND Superseded Geliang Tang 3 participants 6 messages Mar 17 – Mar 26

Subsystems (4)

v2 of 4 Superseded Latest is v4 →

lore
Share & tools
lore

Geliang Tang

author reviewer
#0 lore raw

v2 cover implement .splice_eof

From: Geliang Tang <[email protected]>

RESEND:
- to trigger ai review.

v2:
- add sk_state check in mptcp_splice_eof. (Matt)
- call mptcp_rps_record_subflows in mptcp_splice_eof. (Matt)
- update commit logs.

This set implements .splice_eof for MPTCP and tests it.

v1:
- Link: https://patchwork.kernel.org/project/mptcp/cover/[email protected]/

Geliang Tang (3):
tcp: export do_tcp_splice_eof
mptcp: implement .splice_eof
selftests: mptcp: connect: trigger splice_eof

 include/net/tcp.h                             |  1 +
 net/ipv4/tcp.c                                |  8 +++++--
 net/mptcp/protocol.c                          | 23 +++++++++++++++++++
 .../selftests/net/mptcp/mptcp_connect.c       |  2 +-
 4 files changed, 31 insertions(+), 3 deletions(-)
Signature · 1 line
2.53.0
author reviewer
#1 lore raw

PATCH RESEND v2 1/3 tcp: export do_tcp_splice_eof

From: Geliang Tang <[email protected]>

Extract a do_tcp_splice_eof() helper from tcp_splice_eof() and export it to net/tcp.h, so that it can be used in MPTCP.

---
 include/net/tcp.h | 1 +
 net/ipv4/tcp.c    | 8 ++++++--
 2 files changed, 7 insertions(+), 2 deletions(-)
include/net/tcp.h +1 -0
index f87bdacb5a69..af2e4b61bf23 100644
@@ -370,6 +370,7 @@ int tcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t size);
 int tcp_sendmsg_locked(struct sock *sk, struct msghdr *msg, size_t size);
 int tcp_sendmsg_fastopen(struct sock *sk, struct msghdr *msg, int *copied,
 			 size_t size, struct ubuf_info *uarg);
+void do_tcp_splice_eof(struct sock *sk);
 void tcp_splice_eof(struct socket *sock);
 int tcp_send_mss(struct sock *sk, int *size_goal, int flags);
 int tcp_wmem_schedule(struct sock *sk, int copy);
net/ipv4/tcp.c +6 -2
index dfd677c689ef..2ffad80c1c28 100644
@@ -1469,9 +1469,8 @@ int tcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t size)
 }
 EXPORT_SYMBOL(tcp_sendmsg);
 
-void tcp_splice_eof(struct socket *sock)
+void do_tcp_splice_eof(struct sock *sk)
 {
-	struct sock *sk = sock->sk;
 	struct tcp_sock *tp = tcp_sk(sk);
 	int mss_now, size_goal;
 
@@ -1483,6 +1482,11 @@ void tcp_splice_eof(struct socket *sock)
 	tcp_push(sk, 0, mss_now, tp->nonagle, size_goal);
 	release_sock(sk);
 }
+
+void tcp_splice_eof(struct socket *sock)
+{
+	do_tcp_splice_eof(sock->sk);
+}
 EXPORT_IPV6_MOD_GPL(tcp_splice_eof);
 
 /*
Signature · 1 line
2.53.0
author reviewer
#2 lore raw

PATCH RESEND v2 2/3 mptcp: implement .splice_eof

In reply to Geliang Tang

From: Geliang Tang <[email protected]>

This patch implements the .splice_eof interface for MPTCP, namely mptcp_splice_eof(), which calls do_tcp_splice_eof() for each active subflow when a sendfile() operation reaches end-of-file.

do_tcp_splice_eof() flushes any remaining data in the TCP send queue. MPTCP operates over multiple TCP subflows, and each subflow may have pending data in its send buffer that needs to be properly finalized when splicing data through an MPTCP socket.

sock_splice_eof() calls the .splice_eof interface from struct proto_ops. To maintain consistency with regular TCP behavior, the .splice_eof interface of mptcp_stream_ops is set to inet_splice_eof, which will switch to the protocol-specific implementation (sk->sk_prot->splice_eof)
- for MPTCP, that is mptcp_splice_eof().

This is an improvement; nothing was broken before. MPTCP previously did not handle the splice EOF notification, while TCP did. Without .splice_eof() support, the queue is not flushed immediately when sendfile() reaches EOF, but it will eventually be flushed after a small delay.

---
 net/mptcp/protocol.c | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)
net/mptcp/protocol.c +23 -0
index b5676b37f8f4..ee6d5b4edd1e 100644
@@ -4020,6 +4020,27 @@ static int mptcp_connect(struct sock *sk, struct sockaddr_unsized *uaddr,
 	return 0;
 }
 
+static void mptcp_splice_eof(struct socket *sock)
+{
+	struct mptcp_subflow_context *subflow;
+	struct sock *sk = sock->sk, *ssk;
+	struct mptcp_sock *msk;
+
+	msk = mptcp_sk(sk);
+
+	lock_sock(sk);
+	mptcp_rps_record_subflows(msk);
+	mptcp_for_each_subflow(msk, subflow) {
+		ssk = mptcp_subflow_tcp_sock(subflow);
+
+		if (ssk->sk_state == TCP_CLOSE)
+			continue;
+
+		do_tcp_splice_eof(ssk);
+	}
+	release_sock(sk);
+}
+
 static struct proto mptcp_prot = {
 	.name		= "MPTCP",
 	.owner		= THIS_MODULE,
@@ -4051,6 +4072,7 @@ static struct proto mptcp_prot = {
 	.obj_size	= sizeof(struct mptcp_sock),
 	.slab_flags	= SLAB_TYPESAFE_BY_RCU,
 	.no_autobind	= true,
+	.splice_eof	= mptcp_splice_eof,
 };
 
 static int mptcp_bind(struct socket *sock, struct sockaddr_unsized *uaddr, int addr_len)
@@ -4542,6 +4564,7 @@ static const struct proto_ops mptcp_stream_ops = {
 	.set_rcvlowat	   = mptcp_set_rcvlowat,
 	.read_sock	   = mptcp_read_sock,
 	.splice_read	   = mptcp_splice_read,
+	.splice_eof	   = inet_splice_eof,
 };
 
 static struct inet_protosw mptcp_protosw = {
Signature · 1 line
2.53.0
maintainer
#3 lore raw

Re: [RESEND PATCH mptcp-next v2 2/3] mptcp: implement .splice_eof

Hi Geliang,

Quoted 17 lines · Geliang Tang

From: Geliang Tang <[email protected]>

This patch implements the .splice_eof interface for MPTCP, namely mptcp_splice_eof(), which calls do_tcp_splice_eof() for each active subflow when a sendfile() operation reaches end-of-file.

do_tcp_splice_eof() flushes any remaining data in the TCP send queue. MPTCP operates over multiple TCP subflows, and each subflow may have pending data in its send buffer that needs to be properly finalized when splicing data through an MPTCP socket.

sock_splice_eof() calls the .splice_eof interface from struct proto_ops. To maintain consistency with regular TCP behavior, the .splice_eof interface of mptcp_stream_ops is set to inet_splice_eof, which will switch to the protocol-specific implementation (sk->sk_prot->splice_eof)
- for MPTCP, that is mptcp_splice_eof().

Quoted 4 lines

This is an improvement; nothing was broken before. MPTCP previously did not handle the splice EOF notification, while TCP did. Without .splice_eof() support, the queue is not flushed immediately when sendfile() reaches EOF, but it will eventually be flushed after a small delay.

Thank you for this patch!

Do you mind checking the comments from sashiko please?

https://sashiko.dev/#/patchset/cover.1773740361.git.tanggeliang%40kylinos.cn

They look plausible, but better to double-check.

Cheers,
Matt

Signature · 1 line
Sponsored by the NGI0 Core fund.
author reviewer
#4 lore raw

PATCH RESEND v2 3/3 selftests: mptcp: connect: trigger splice_eof

In reply to Geliang Tang

From: Geliang Tang <[email protected]>

Increase the sendfile count by one to ensure the transmission size exceeds the actual data length. This triggers the splice_eof path in the kernel, allowing the newly implemented MPTCP splice_eof interface to be exercised during testing.

The change from 'count' to 'count + 1' forces the sendfile operation to attempt sending one more byte than available, which activates the end-of-file handling in the splicing logic and ensures coverage of the related MPTCP code paths.

---
 tools/testing/selftests/net/mptcp/mptcp_connect.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tools/testing/selftests/net/mptcp/mptcp_connect.c +1 -1
index cbe573c4ab3a..2aaf3ed11315 100644
@@ -870,7 +870,7 @@ static int do_sendfile(int infd, int outfd, unsigned int count,
 	while (count > 0) {
 		ssize_t r;
 
-		r = sendfile(outfd, infd, NULL, count);
+		r = sendfile(outfd, infd, NULL, count + 1);
 		if (r < 0) {
 			perror("sendfile");
 			return 3;
Signature · 1 line
2.53.0
#5 lore raw
In reply to Geliang Tang

Hi Geliang,

Thank you for your modifications, that's great!

Our CI did some validations and here is its report:

- KVM Validation: normal (except selftest_mptcp_join): Success! ✅
- KVM Validation: normal (only selftest_mptcp_join): Success! ✅
- KVM Validation: debug (except selftest_mptcp_join): Unstable: 1 failed test(s): packetdrill_sockopts 🔴
- KVM Validation: debug (only selftest_mptcp_join): Success! ✅
- KVM Validation: btf-normal (only bpftest_all): Success! ✅
- KVM Validation: btf-debug (only bpftest_all): Success! ✅
- Task: https://github.com/multipath-tcp/mptcp_net-next/actions/runs/23189071186

Initiator: Patchew Applier
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/e9dd5ccdcdfc
Patchwork: https://patchwork.kernel.org/project/mptcp/list/?series=1067870

If there are some issues, you can reproduce them using the same environment as the one used by the CI thanks to a docker image, e.g.:

    $ cd [kernel source code]
    $ docker run -v "${PWD}:${PWD}:rw" -w "${PWD}" --privileged --rm -it \
        --pull always mptcp/mptcp-upstream-virtme-docker:latest \
        auto-normal

For more details:

https://github.com/multipath-tcp/mptcp-upstream-virtme-docker

Please note that despite all the efforts that have been already done to have a stable tests suite when executed on a public CI like here, it is possible some reported issues are not due to your modifications. Still, do not hesitate to help us improve that ;-)

Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (NGI0 Core)