Skip to contentSkip to conversation

tcp: support preloading data on a listening socket

PATCH Superseded Jeremy Harris 3 participants 12 messages May 16, 2025

Subsystems (6)

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

cover tcp: support preloading data on a listening socket

Support write to a listen TCP socket, for immediate transmission on passive connection establishments.

On a normal connection transmission is triggered by the receipt of the 3rd-ack. On a fastopen (with accepted cookie) connection the data is sent in the synack packet.

The data preload is done using a sendmsg with a newly-defined flag (MSG_PRELOAD); the amount of data limited to a single linear sk_buff. Note that this definition is the last-but-two bit available if "int" is 32 bits.

Testing:

A) packetdrill scripts for
- normal non-TFO
- normal TFO
- synack lost
- 3rd-ack acks only the SYN
- 3rd-ack acks partial data
(NB: packetdrill can only check the data size, not actual content)

B) Application use, running the application testsuite and manual check of specific cases via packet capture

C) Daily-driver laptop use (not expected to trigger the feature; only regression-test)

Jeremy Harris (6):
tcp: support writing to a socket in listening state
tcp: copy write-data from listen socket to accept child socket
tcp: fastopen: add write-data to fastopen synack packet
tcp: transmit any pending data on receipt of 3rd-ack
tcp: fastopen: retransmit data when only the SYN of a synack-with-data
is acked
tcp: fastopen: extend retransmit-queue trimming to handle linear
sk_buff

 include/linux/socket.h                        |  1 +
 net/ipv4/tcp.c                                | 17 ++++--
 net/ipv4/tcp_fastopen.c                       |  3 +-
 net/ipv4/tcp_input.c                          | 15 ++++-
 net/ipv4/tcp_ipv4.c                           |  4 +-
 net/ipv4/tcp_minisocks.c                      | 58 +++++++++++++++++--
 net/ipv4/tcp_output.c                         | 50 ++++++++++++++--
 .../perf/trace/beauty/include/linux/socket.h  |  1 +
 tools/perf/trace/beauty/msg_flags.c           |  3 +
 9 files changed, 135 insertions(+), 17 deletions(-)

base-commit: 2da35e4b4df99d3dd29bacf0c054e6988013d4ec

Signature · 1 line
2.49.0
#1 lore raw

PATCH 1/6 tcp: support writing to a socket in listening state

In the tcp sendmsg handler, permit a write in LISTENING state if a MSG_PRELOAD flag is used. Copy from iovec to a linear sk_buff for placement on the socket write queue.

---
 include/linux/socket.h                         |  1 +
 net/ipv4/tcp.c                                 | 17 +++++++++++++----
 tools/perf/trace/beauty/include/linux/socket.h |  1 +
 tools/perf/trace/beauty/msg_flags.c            |  3 +++
 4 files changed, 18 insertions(+), 4 deletions(-)
include/linux/socket.h +1 -0
index 3b262487ec06..b41f4cd4dc97 100644
@@ -330,6 +330,7 @@ struct ucred {
 #define MSG_SOCK_DEVMEM 0x2000000	/* Receive devmem skbs as cmsg */
 #define MSG_ZEROCOPY	0x4000000	/* Use user data in kernel path */
 #define MSG_SPLICE_PAGES 0x8000000	/* Splice the pages from the iterator in sendmsg() */
+#define MSG_PRELOAD	0x10000000	/* Preload tx data while listening */
 #define MSG_FASTOPEN	0x20000000	/* Send data in TCP SYN */
 #define MSG_CMSG_CLOEXEC 0x40000000	/* Set close_on_exec for file
 					   descriptor received through
net/ipv4/tcp.c +13 -4
index b7b6ab41b496..72b5d7cad351 100644
@@ -1136,12 +1136,13 @@ int tcp_sendmsg_locked(struct sock *sk, struct msghdr *msg, size_t size)
 
 	tcp_rate_check_app_limited(sk);  /* is sending application-limited? */
 
-	/* Wait for a connection to finish. One exception is TCP Fast Open
+	/* Wait for a connection to finish. Exceptions are TCP Fast Open
 	 * (passive side) where data is allowed to be sent before a connection
-	 * is fully established.
+	 * is fully established, and a message marked as preload which is
+	 * allowed to be placed in the send queue of a listening socket.
 	 */
 	if (((1 << sk->sk_state) & ~(TCPF_ESTABLISHED | TCPF_CLOSE_WAIT)) &&
-	    !tcp_passive_fastopen(sk)) {
+	    !tcp_passive_fastopen(sk) && !(flags & MSG_PRELOAD)) {
 		err = sk_stream_wait_connect(sk, &timeo);
 		if (err != 0)
 			goto do_error;
@@ -1226,7 +1227,13 @@ int tcp_sendmsg_locked(struct sock *sk, struct msghdr *msg, size_t size)
 		if (copy > msg_data_left(msg))
 			copy = msg_data_left(msg);
 
-		if (zc == 0) {
+		if (unlikely(flags & MSG_PRELOAD)) {
+			copy = min_t(int, copy, skb_tailroom(skb));
+			err = skb_add_data_nocache(sk, skb, &msg->msg_iter,
+						   copy);
+			if (err)
+				goto do_error;
+		} else if (zc == 0) {
 			bool merge = true;
 			int i = skb_shinfo(skb)->nr_frags;
 			struct page_frag *pfrag = sk_page_frag(sk);
@@ -1330,6 +1337,8 @@ int tcp_sendmsg_locked(struct sock *sk, struct msghdr *msg, size_t size)
 		if (!msg_data_left(msg)) {
 			if (unlikely(flags & MSG_EOR))
 				TCP_SKB_CB(skb)->eor = 1;
+			if (unlikely(flags & MSG_PRELOAD))
+				goto out_nopush;
 			goto out;
 		}
 
tools/perf/trace/beauty/include/linux/socket.h +1 -0
index c3322eb3d686..e9ea498169f3 100644
@@ -330,6 +330,7 @@ struct ucred {
 #define MSG_SOCK_DEVMEM 0x2000000	/* Receive devmem skbs as cmsg */
 #define MSG_ZEROCOPY	0x4000000	/* Use user data in kernel path */
 #define MSG_SPLICE_PAGES 0x8000000	/* Splice the pages from the iterator in sendmsg() */
+#define MSG_PRELOAD	0x10000000	/* Preload tx data while listening */
 #define MSG_FASTOPEN	0x20000000	/* Send data in TCP SYN */
 #define MSG_CMSG_CLOEXEC 0x40000000	/* Set close_on_exec for file
 					   descriptor received through
tools/perf/trace/beauty/msg_flags.c +3 -0
index 2da581ff0c80..27e40da9b02d 100644
@@ -20,6 +20,9 @@
 #ifndef MSG_SPLICE_PAGES
 #define MSG_SPLICE_PAGES	0x8000000
 #endif
+#ifndef MSG_PRELOAD
+#define MSG_PRELOAD		0x10000000
+#endif
 #ifndef MSG_FASTOPEN
 #define MSG_FASTOPEN		0x20000000
 #endif
Signature · 1 line
2.49.0
#2 lore raw

PATCH 2/6 tcp: copy write-data from listen socket to accept child socket

In reply to Jeremy Harris

Set the request_sock flag for fastopen earlier, making it available to the af_ops SYN-handler function.

In that function copy data from the listen socket write queue into an sk_buff, allocating if needed and adding to the write queue of the newly-created child socket.
Set sequence number values depending on the fastopen status.

---
 net/ipv4/tcp_fastopen.c  |  3 ++-
 net/ipv4/tcp_ipv4.c      |  4 +--
 net/ipv4/tcp_minisocks.c | 58 ++++++++++++++++++++++++++++++++++++----
 3 files changed, 57 insertions(+), 8 deletions(-)
net/ipv4/tcp_fastopen.c +2 -1
index 9b83d639b5ac..03a86d0b87ba 100644
@@ -245,6 +245,8 @@ static struct sock *tcp_fastopen_create_child(struct sock *sk,
 	struct sock *child;
 	bool own_req;
 
+	tcp_rsk(req)->tfo_listener = true;
+
 	child = inet_csk(sk)->icsk_af_ops->syn_recv_sock(sk, skb, req, NULL,
 							 NULL, &own_req);
 	if (!child)
@@ -261,7 +263,6 @@ static struct sock *tcp_fastopen_create_child(struct sock *sk,
 	tp = tcp_sk(child);
 
 	rcu_assign_pointer(tp->fastopen_rsk, req);
-	tcp_rsk(req)->tfo_listener = true;
 
 	/* RFC1323: The window in SYN & SYN/ACK segments is never
 	 * scaled. So correct it appropriately.
net/ipv4/tcp_ipv4.c +2 -2
index 6a14f9e6fef6..e488effdbdb2 100644
@@ -1747,8 +1747,8 @@ EXPORT_IPV6_MOD(tcp_v4_conn_request);
 
 
 /*
- * The three way handshake has completed - we got a valid synack -
- * now create the new socket.
+ * The three way handshake has completed - we got a valid synack
+ * (or a FASTOPEN syn) - now create the new socket.
  */
 struct sock *tcp_v4_syn_recv_sock(const struct sock *sk, struct sk_buff *skb,
 				  struct request_sock *req,
net/ipv4/tcp_minisocks.c +53 -5

76 lines not on this page · raw message

Signature · 1 line
2.49.0
#6 lore raw

PATCH 4/6 tcp: transmit any pending data on receipt of 3rd-ack

In reply to Jeremy Harris

For the non-fastopen case of prelaod, when the 3rd-ack arrives there will be data on the write queue. Transmit it immediately by allowing the SYN_SENT state to run the xmit-recovery code.

---
 net/ipv4/tcp_input.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
net/ipv4/tcp_input.c +2 -1
index 8ec92dec321a..345a08baaf02 100644
@@ -3900,7 +3900,8 @@ static void tcp_xmit_recovery(struct sock *sk, int rexmit)
 {
 	struct tcp_sock *tp = tcp_sk(sk);
 
-	if (rexmit == REXMIT_NONE || sk->sk_state == TCP_SYN_SENT)
+	if ((rexmit == REXMIT_NONE && sk->sk_state != TCP_SYN_RECV) ||
+	    sk->sk_state == TCP_SYN_SENT)
 		return;
 
 	if (unlikely(rexmit == REXMIT_NEW)) {
Signature · 1 line
2.49.0
#7 lore raw

PATCH 5/6 tcp: fastopen: retransmit data when only the SYN of a synack-with-data is acked

In reply to Jeremy Harris

A corner-case for the 3rd-ack after a data-on-synack is for only the SYN to be acked. Handle this by, in ack processing, when in SYN_RECV state (the state is not yet updated to ESTABLISHED) marking the retransmit-queue sk_buff as having been lost.

---
 net/ipv4/tcp_input.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)
net/ipv4/tcp_input.c +12 -0
index 345a08baaf02..a53021edddd5 100644
@@ -4069,6 +4069,18 @@ static int tcp_ack(struct sock *sk, const struct sk_buff *skb, int flag)
 				      &rexmit);
 	}
 
+	/* On receiving a 3rd-ack, if we never sent a packet via
+	 * the normal means (which counts them), yet there is data
+	 * remaining for retransmit, it was data-on-synack not acked;
+	 * mark the skb for retransmission.
+	 */
+	if (sk->sk_state == TCP_SYN_RECV && tp->segs_out == 0) {
+		struct sk_buff *skb = tcp_rtx_queue_head(sk);
+
+		if (skb)
+			tcp_mark_skb_lost(sk, skb);
+	}
+
 	/* If needed, reset TLP/RTO timer when RACK doesn't set. */
 	if (flag & FLAG_SET_XMIT_TIMER)
 		tcp_set_xmit_timer(sk);
Signature · 1 line
2.49.0
#5 lore raw

PATCH 3/6 tcp: fastopen: add write-data to fastopen synack packet

In reply to Jeremy Harris

While building the synack packet, for a fastopen socket copy data from write queue to the packet.
Move the data from write queue to retransmit queue.

---
 net/ipv4/tcp_output.c | 32 ++++++++++++++++++++++++++++++--
 1 file changed, 30 insertions(+), 2 deletions(-)
net/ipv4/tcp_output.c +30 -2
index 3ac8d2d17e1f..c50553c1c795 100644
@@ -3702,7 +3702,7 @@ int tcp_send_synack(struct sock *sk)
 
 /**
  * tcp_make_synack - Allocate one skb and build a SYNACK packet.
- * @sk: listener socket
+ * @sk: listener socket (or child socket for fastopen)
  * @dst: dst entry attached to the SYNACK. It is consumed and caller
  *       should not use it again.
  * @req: request_sock pointer
@@ -3719,6 +3719,7 @@ struct sk_buff *tcp_make_synack(const struct sock *sk, struct dst_entry *dst,
 	struct inet_request_sock *ireq = inet_rsk(req);
 	const struct tcp_sock *tp = tcp_sk(sk);
 	struct tcp_out_options opts;
+	struct sock *fastopen_sk = (struct sock *)sk;
 	struct tcp_key key = {};
 	struct sk_buff *skb;
 	int tcp_header_size;
@@ -3748,7 +3749,7 @@ struct sk_buff *tcp_make_synack(const struct sock *sk, struct dst_entry *dst,
 		 * cpu might call us concurrently.
 		 * sk->sk_wmem_alloc in an atomic, we can promote to rw.
 		 */
-		skb_set_owner_w(skb, (struct sock *)sk);
+		skb_set_owner_w(skb, fastopen_sk);
 		break;
 	}
 	skb_dst_set(skb, dst);
@@ -3831,6 +3832,33 @@ struct sk_buff *tcp_make_synack(const struct sock *sk, struct dst_entry *dst,
 	th->window = htons(min(req->rsk_rcv_wnd, 65535U));
 	tcp_options_write(th, NULL, tcp_rsk(req), &opts, &key);
 	th->doff = (tcp_header_size >> 2);
+
+	/* If this is a FASTOPEN, and there is write-data on the accept socket,
+	 * re-copy it to the synack segment. If not FASTOPEN. any data waits
+	 * until 3rd-ack arrival.
+	 */
+
+	if (synack_type == TCP_SYNACK_FASTOPEN &&
+	    !skb_queue_empty(&sk->sk_write_queue)) {
+		struct sk_buff *a_skb = tcp_write_queue_tail(sk);
+		int copy = min_t(int, a_skb->len, skb_tailroom(skb));
+
+		skb_put_data(skb, a_skb->data, copy);
+		TCP_SKB_CB(skb)->end_seq += copy;
+
+		tcp_skb_pcount_set(a_skb, 1);
+		WRITE_ONCE(tcp_sk(fastopen_sk)->write_seq,
+			   TCP_SKB_CB(a_skb)->end_seq);
+
+		skb_set_delivery_time(a_skb, now, SKB_CLOCK_MONOTONIC);
+
+		/* Move the data to the retransmit queue.
+		 * Code elsewhere implies this is a full child socket and
+		 * can be treated as writeable - permitting the cast.
+		 */
+		tcp_event_new_data_sent(fastopen_sk, a_skb);
+	}
+
 	TCP_INC_STATS(sock_net(sk), TCP_MIB_OUTSEGS);
 
 	/* Okay, we have all we need - do the md5 hash if needed */
Signature · 1 line
2.49.0
#8 lore raw

PATCH 6/6 tcp: fastopen: extend retransmit-queue trimming to handle linear sk_buff

In reply to Jeremy Harris

A corner-case for the 3rd-ack after a data-on-synack is for some but not all of the data to be acked. Support this by adding to the retransmit-queue trim routine to handle a linear sk_buff.

---
 net/ipv4/tcp_output.c | 18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)
net/ipv4/tcp_output.c +16 -2
index c50553c1c795..bff5934ff04b 100644
@@ -1708,8 +1708,22 @@ static int __pskb_trim_head(struct sk_buff *skb, int len)
 	struct skb_shared_info *shinfo;
 	int i, k, eat;
 
-	DEBUG_NET_WARN_ON_ONCE(skb_headlen(skb));
-	eat = len;
+	eat = skb_headlen(skb);
+	if (unlikely(eat)) {
+		if (len < eat)
+			eat = len;
+		skb->head += eat;
+		skb->len -= eat;
+		if (skb->data_len)
+			skb->data_len -= eat;
+
+		eat = len - eat;
+		if (eat == 0)
+			return len;
+	} else {
+		eat = len;
+	}
+
 	k = 0;
 	shinfo = skb_shinfo(skb);
 	for (i = 0; i < shinfo->nr_frags; i++) {
Signature · 1 line
2.49.0
#9 lore raw
In reply to Jeremy Harris
Quoted 2 lines · Jeremy Harris

Support write to a listen TCP socket, for immediate transmission on passive connection establishments.

This series should have included "net-next" in their Subjects.

Signature · 2 lines
Apologies,
  Jeremy
#3 lore raw

Re: [PATCH 2/6] tcp: copy write-data from listen socket to accept child socket

In reply to Jeremy Harris
Quoted 8 lines · Jeremy Harris

Set the request_sock flag for fastopen earlier, making it available to the af_ops SYN-handler function.

In that function copy data from the listen socket write queue into an sk_buff, allocating if needed and adding to the write queue of the newly-created child socket.
Set sequence number values depending on the fastopen status.

I do not see any locking. I think you should run a local KASAN/syzbot instance and you will be shocked.

Honestly we need to be convinced of why adding code in sendmsg() fast path is worth this.

maintainer
#10 lore raw
In reply to Jeremy Harris
Quoted 4 lines · Jeremy Harris

Support write to a listen TCP socket, for immediate transmission on passive connection establishments.

Quoted 8 lines

On a normal connection transmission is triggered by the receipt of the 3rd-ack. On a fastopen (with accepted cookie) connection the data is sent in the synack packet.

The data preload is done using a sendmsg with a newly-defined flag (MSG_PRELOAD); the amount of data limited to a single linear sk_buff. Note that this definition is the last-but-two bit available if "int" is 32 bits.

Can you please add a bit more context, like:

+ What is the motivating use case? (Accelerating Exim?) Is this targeted for connections using encryption (like TLS/SSL), or just plain-text connections?

+ What are the exact performance improvements you are seeing in your benchmarks that (a) motivate this, and (b) justify any performance impact on the TCP stack?

+ Regarding "Support write to a listen TCP socket, for immediate transmission on passive connection establishments.": can you please make it explicitly clear whether the data written to the listening socket is saved and transmitted on all future successful passive sockets that are created for the listener, or is just transmitted on the next connection that is created?

thanks,
neal

#11 lore raw

Hi Neal,

Thanks for the initial review.

Quoted 9 lines · Neal Cardwell
Quoted 8 lines · Jeremy Harris

Support write to a listen TCP socket, for immediate transmission on passive connection establishments.

On a normal connection transmission is triggered by the receipt of the 3rd-ack. On a fastopen (with accepted cookie) connection the data is sent in the synack packet.

Quoted 8 lines
Quoted 4 lines

The data preload is done using a sendmsg with a newly-defined flag (MSG_PRELOAD); the amount of data limited to a single linear sk_buff. Note that this definition is the last-but-two bit available if "int" is 32 bits.

Can you please add a bit more context, like:

+ What is the motivating use case? (Accelerating Exim?)

Accelerating any server-first ULP, SMTP being the major use I know of (and yes, Exim is my primary testcase and is operational against a test kernel with this patch series).

One caveat: the initial server data cannot change from one passive connection to another.

Quoted 3 lines

Is this
targeted for connections using encryption (like TLS/SSL), or just plain-text connections?

TLS-on-connect cannot benefit, being client-first. SMTP that uses STARTTLS can take advantage of it, as can plaintext SMTP.

I would not expect https to be able to use it.

Quoted 3 lines

+ What are the exact performance improvements you are seeing in your benchmarks that (a) motivate this, and (b) justify any performance impact on the TCP stack?

Because of the lack of userland roundtrip needed for the initial server data, there is a latency benefit. This is better for the TFO-C case, but also significant for the non-TFO case.

Packet capture (laptop, loopback, TFO-C case) for initial SYN to first client data packet (5 samples):

- baseline   TFO_C      1064 1470 1455 1547 1595  usec
- patched    non-TFO     140  150  159  144  153  usec
- patched    TFO_C       142  149  149  125  125  usec

One fewer packet is sent by the server in most packet captures, sometimes one fewer in each direction. There is one less application kernel entry/exit on the server.

I'm hoping those differences will add up to both less cpu time (on both endpoints) and less wire-time. However, I have not run benchmarks looking for a change in peak rate of connection-handling.

In summary, this is the mirror of TCP Fast Open client data: the latency benefit is probably the most useful aspect.

Quoted 5 lines

+ Regarding "Support write to a listen TCP socket, for immediate transmission on passive connection establishments.": can you please make it explicitly clear whether the data written to the listening socket is saved and transmitted on all future successful passive sockets that are created for the listener,

This. The data is copied for each future passive socket from this listener,

Quoted 2 lines

or is just transmitted on
the next connection that is created?

(and not this option).

I'll copy these comments in any future v2.
As Eric says, I should run KASAN/syzbot first.

Signature · 2 lines
Cheers,
  Jeremy
#4 lore raw

Re: [PATCH 2/6] tcp: copy write-data from listen socket to accept child socket

In reply to Eric Dumazet
Quoted 2 lines · Eric Dumazet

I do not see any locking. I think you should run a local KASAN/syzbot instance and you will be shocked.

Thanks for the suggestion; I'll look into doing that.

Signature · 2 lines
Cheers,
  Jeremy