TCP: Fast-Open observability
Subsystems (3)
cover TCP: Fast-Open observability
Whether TCP Fast Open was used for a connection is not reliably observable by an accepting application when the SYN passed no data.
Fix this by noting during SYN receive processing that an acceptable Fast Open option was used, and provide this to userland via getsockopt TCP_INFO.
Jeremy Harris (2):
TCP: note received valid-cookie Fast Open option
TCP: pass accepted-TFO indication through getsockopt
include/linux/tcp.h | 3 ++- include/uapi/linux/tcp.h | 1 + net/ipv4/tcp.c | 2 ++ net/ipv4/tcp_fastopen.c | 1 + 4 files changed, 6 insertions(+), 1 deletion(-)
base-commit: 24e31e4747698dc50d408f0082d7eb9b9520d2f6
Signature · 1 line
2.49.0
PATCH RESEND 1/2 TCP: note received valid-cookie Fast Open option
--- include/linux/tcp.h | 3 ++- net/ipv4/tcp_fastopen.c | 1 + 2 files changed, 3 insertions(+), 1 deletion(-)
include/linux/tcp.h +2 -1
@@ -385,7 +385,8 @@ struct tcp_sock { syn_fastopen:1, /* SYN includes Fast Open option */ syn_fastopen_exp:1,/* SYN includes Fast Open exp. option */ syn_fastopen_ch:1, /* Active TFO re-enabling probe */ - syn_data_acked:1;/* data in SYN is acked by SYN-ACK */ + syn_data_acked:1,/* data in SYN is acked by SYN-ACK */ + syn_fastopen_in:1; /* Received SYN includes Fast Open option */u8 keepalive_probes;/* num of allowed keep alive probes */ u32 tcp_tx_delay;/* delay (in usec) added to TX packets */
net/ipv4/tcp_fastopen.c +1 -0
@@ -401,6 +401,7 @@ struct sock *tcp_try_fastopen(struct sock *sk, struct sk_buff *skb, }NET_INC_STATS (sock_net (sk),LINUX_MIB_TCPFASTOPENPASSIVE ); +tcp_sk (child)->syn_fastopen_in =1 ;return child; } NET_INC_STATS(sock_net(sk),
Signature · 1 line
2.49.0
Paolo Abeni
maintainerRe: [RESEND PATCH 1/2] TCP: note received valid-cookie Fast Open option
Quoted 1 line · Jeremy Harris
Signed-off-by Jeremy Harris <[email protected]>
The commit description is missing.
Quoted 10 lines
--- include/linux/tcp.h | 3 ++- net/ipv4/tcp_fastopen.c | 1 + 2 files changed, 3 insertions(+), 1 deletion(-)include/linux/tcp.h +0
-0
include/linux/tcp.h struct tcp_sock { · line 389
Quoted 6 lines
include/linux/tcp.h +2
-1@@ -385,7 +385,8 @@ struct tcp_sock { syn_fastopen:1, /* SYN includes Fast Open option */ syn_fastopen_exp:1,/* SYN includes Fast Open exp. option */ syn_fastopen_ch:1, /* Active TFO re-enabling probe */ - syn_data_acked:1;/* data in SYN is acked by SYN-ACK */ + syn_data_acked:1,/* data in SYN is acked by SYN-ACK */ + syn_fastopen_in:1; /* Received SYN includes Fast Open option */
Worth mentioning in the commit message that this will fill a bit hole.
net/ipv4/tcp_fastopen.c struct sock *tcp_try_fastopen(struct sock *sk, struct sk_buff *skb, · line 404
Quoted 12 lines
include/linux/tcp.h +0
-0u8 keepalive_probes;/* num of allowed keep alive probes */ u32 tcp_tx_delay;/* delay (in usec) added to TX packets */ net/ipv4/tcp_fastopen.c +1
-0@@ -401,6 +401,7 @@ struct sock *tcp_try_fastopen(struct sock *sk, struct sk_buff *skb, }NET_INC_STATS (sock_net (sk),LINUX_MIB_TCPFASTOPENPASSIVE ); +tcp_sk (child)->syn_fastopen_in =1 ;
Likely you need to reset the bit on disconnect().
/P
Re: [RESEND PATCH 1/2] TCP: note received valid-cookie Fast Open option
Thanks for the review. I'm preparing a v2, and a patch to "ss" in iproute2.
Quoted 1 line · Paolo Abeni
The commit description is missing.
Ack.
Quoted 3 lines
include/linux/tcp.h struct tcp_sock { · line 389
Quoted 1 line
include/linux/tcp.h +1
-0+ syn_fastopen_in:1 ;/* Received SYN includes Fast Open option */ Worth mentioning in the commit message that this will fill a bit hole.
Will do.
Quoted 7 lines
net/ipv4/tcp_fastopen.c struct sock *tcp_try_fastopen(struct sock *sk, struct sk_buff *skb, · line 404
Quoted 5 lines
net/ipv4/tcp_fastopen.c +1
-0@@ -401,6 +401,7 @@ struct sock *tcp_try_fastopen(struct sock *sk, struct sk_buff *skb, }NET_INC_STATS (sock_net (sk),LINUX_MIB_TCPFASTOPENPASSIVE ); +tcp_sk (child)->syn_fastopen_in =1 ;Likely you need to reset the bit on
disconnect().
Added.
Signature · 2 lines
Cheers, Jeremy
Neal Cardwell
maintainerRe: [RESEND PATCH 1/2] TCP: note received valid-cookie Fast Open option
In reply to Jeremy HarrisQuoted 2 lines · Jeremy Harris
Signed-off-by Jeremy Harris <[email protected]>
The suggested commit title is:
TCP: note received valid-cookie Fast Open option
The "TCP:" prefix is not the typical prefix for Linux TCP changes. A "tcp:" is much more common.
Please follow the convention that we try to adhere to for TCP TFO changes by using something like:
tcp: fastopen: note received valid-cookie Fast Open option
Quoted 10 lines
--- include/linux/tcp.h | 3 ++- net/ipv4/tcp_fastopen.c | 1 + 2 files changed, 3 insertions(+), 1 deletion(-)include/linux/tcp.h +0
-0
include/linux/tcp.h struct tcp_sock { · line 389
Quoted 6 lines
include/linux/tcp.h +2
-1@@ -385,7 +385,8 @@ struct tcp_sock { syn_fastopen:1, /* SYN includes Fast Open option */ syn_fastopen_exp:1,/* SYN includes Fast Open exp. option */ syn_fastopen_ch:1, /* Active TFO re-enabling probe */ - syn_data_acked:1;/* data in SYN is acked by SYN-ACK */ + syn_data_acked:1,/* data in SYN is acked by SYN-ACK */ + syn_fastopen_in:1; /* Received SYN includes Fast Open option */
IMHO this field name and comment are slightly misleading.
Sometimes when a SYN is received with a TFO option the server will fail to create a child because the TFO cookie is incorrect.
When this bit is set, we know not only that the "Received SYN includes Fast Open option", but we also know that the TFO cookie was correct and a child socket was created.
So I would suggest a more specific comment and field name, like:
syn_fastopen_child:1; /* created TFO passive child socket */
thanks,
neal
Re: [RESEND PATCH 1/2] TCP: note received valid-cookie Fast Open option
Thanks for the review. I'm preparing a v2, and a patch to "ss" in iproute2.
Quoted 7 lines
The "TCP:" prefix is not the typical prefix for Linux TCP changes. A "tcp:" is much more common.
Please follow the convention that we try to adhere to for TCP TFO changes by using something like:
tcp: fastopen: note received valid-cookie Fast Open option
Will change for v2.
Quoted 9 lines
include/linux/tcp.h struct tcp_sock { · line 389
Quoted 3 lines
include/linux/tcp.h +2
-1- syn_data_acked:1 ;/* data in SYN is acked by SYN-ACK */ + syn_data_acked:1 ,/* data in SYN is acked by SYN-ACK */ + syn_fastopen_in:1; /* Received SYN includes Fast Open option */IMHO this field name and comment are slightly misleading.
Sometimes when a SYN is received with a TFO option the server will fail to create a child because the TFO cookie is incorrect.
Quoted 7 lines
When this bit is set, we know not only that the "Received SYN includes Fast Open option", but we also know that the TFO cookie was correct and a child socket was created.
So I would suggest a more specific comment and field name, like:
syn_fastopen_child:1; /* created TFO passive child socket */
Done.
Signature · 2 lines
Cheers, Jeremy
PATCH RESEND 2/2 TCP: pass accepted-TFO indication through getsockopt
In reply to Jeremy Harris--- include/uapi/linux/tcp.h | 1 + net/ipv4/tcp.c | 2 ++ 2 files changed, 3 insertions(+)
include/uapi/linux/tcp.h +1 -0
@@ -184,6 +184,7 @@ enum tcp_fastopen_client_fail {#define TCPI_OPT_ECN_SEEN 16/* we received at least one packet with ECT */ #define TCPI_OPT_SYN_DATA 32/* SYN-ACK acked data in SYN sent or rcvd */ #define TCPI_OPT_USEC_TS 64/* usec timestamps */ +#define TCPI_OPT_TFO_SEEN 128/* we accepted a Fast Open option on SYN */ /* * Sender's congestion state indicating normal or abnormal situations
net/ipv4/tcp.c +2 -0
@@ -4164,6 +4164,8 @@ void tcp_get_info(struct sock *sk, struct tcp_info *info) info->tcpi_options |=TCPI_OPT_SYN_DATA ;if (tp->tcp_usec_ts) info->tcpi_options |=TCPI_OPT_USEC_TS ; +if (tp->syn_fastopen_in) + info->tcpi_options |=TCPI_OPT_TFO_SEEN ; info->tcpi_rto =jiffies_to_usecs (icsk->icsk_rto); info->tcpi_ato = jiffies_to_usecs(min_t(u32, icsk->icsk_ack.ato,
Signature · 1 line
2.49.0
Re: [RESEND PATCH 2/2] TCP: pass accepted-TFO indication through getsockopt
From: Jeremy Harris <[email protected]>
Date: Wed, 16 Apr 2025 10:15:38 +0100
Quoted 11 lines
Signed-off-by Jeremy Harris <[email protected]>--- include/uapi/linux/tcp.h | 1 + net/ipv4/tcp.c | 2 ++ 2 files changed, 3 insertions(+)include/uapi/linux/tcp.h +0
-0
include/uapi/linux/tcp.h enum tcp_fastopen_client_fail { · line 187
Quoted 4 lines
include/uapi/linux/tcp.h +1
-0@@ -184,6 +184,7 @@ enum tcp_fastopen_client_fail {#define TCPI_OPT_ECN_SEEN 16/* we received at least one packet with ECT */ #define TCPI_OPT_SYN_DATA 32/* SYN-ACK acked data in SYN sent or rcvd */ #define TCPI_OPT_USEC_TS 64/* usec timestamps */ +#define TCPI_OPT_TFO_SEEN 128/* we accepted a Fast Open option on SYN */
This is the last bit of tcpi_options.
We can add another field at the end of the struct if necessary in the future, but from the cover letter and commit message, I didn't get why this info is needed at per-socket granurality.
Also, This can be retrieved with BPF.
Quoted 18 lines
include/uapi/linux/tcp.h +0
-0/* * Sender's congestion state indicating normal or abnormal situations net/ipv4/tcp.c +2
-0@@ -4164,6 +4164,8 @@ void tcp_get_info(struct sock *sk, struct tcp_info *info) info->tcpi_options |=TCPI_OPT_SYN_DATA ;if (tp->tcp_usec_ts) info->tcpi_options |=TCPI_OPT_USEC_TS ; +if (tp->syn_fastopen_in) + info->tcpi_options |=TCPI_OPT_TFO_SEEN ; info->tcpi_rto =jiffies_to_usecs (icsk->icsk_rto); info->tcpi_ato = jiffies_to_usecs(min_t(u32, icsk->icsk_ack.ato,Signature · 1 line
2.49.0
Re: [RESEND PATCH 2/2] TCP: pass accepted-TFO indication through getsockopt
Quoted 7 lines · Kuniyuki Iwashima
Quoted 1 line
+#define
TCPI_OPT_TFO_SEEN128 /* we accepted a Fast Open option on SYN */This is the last bit of
tcpi_options.We can add another field at the end of the struct if necessary in the future, but from the cover letter and commit message, I didn't get why this info is needed at per-socket granurality.
An application is operating at socket granularity. The provision is so that an application can tell what is going on with each one of its comms channels independently, rather than (for instance) the rolled-up information on the entire system that the existing statistics counter offers.
Quoted 1 line
Also, This can be retrieved with BPF.
If I understand correctly, BPF is aimed at sysadminss probing or modifying system operations rather than production use by an application. While your point is true, I don't think BPF is a good fit.
Signature · 2 lines
Cheers, Jeremy
Paolo Abeni
maintainerRe: [RESEND PATCH 2/2] TCP: pass accepted-TFO indication through getsockopt
In reply to Jeremy HarrisQuoted 1 line · Jeremy Harris
Signed-off-by Jeremy Harris <[email protected]>
The commit message is missing.
Quoted 22 lines
--- include/uapi/linux/tcp.h | 1 + net/ipv4/tcp.c | 2 ++ 2 files changed, 3 insertions(+)include/uapi/linux/tcp.h +1
-0@@ -184,6 +184,7 @@ enum tcp_fastopen_client_fail {#define TCPI_OPT_ECN_SEEN 16/* we received at least one packet with ECT */ #define TCPI_OPT_SYN_DATA 32/* SYN-ACK acked data in SYN sent or rcvd */ #define TCPI_OPT_USEC_TS 64/* usec timestamps */ +#define TCPI_OPT_TFO_SEEN 128/* we accepted a Fast Open option on SYN */ /* * Sender's congestion state indicating normal or abnormal situations net/ipv4/tcp.c +0
-0
net/ipv4/tcp.c void tcp_get_info(struct sock *sk, struct tcp_info *info) · line 4168
Quoted 5 lines
net/ipv4/tcp.c +2
-0@@ -4164,6 +4164,8 @@ void tcp_get_info(struct sock *sk, struct tcp_info *info) info->tcpi_options |=TCPI_OPT_SYN_DATA ;if (tp->tcp_usec_ts) info->tcpi_options |=TCPI_OPT_USEC_TS ; +if (tp->syn_fastopen_in) + info->tcpi_options |=TCPI_OPT_TFO_SEEN ;
I guess a paired iproute2 change is needed to really observe the new info.
And it would be nice to leverage such thing to later add a paired self-test for this feature.
Thanks,
Paolo
Neal Cardwell
maintainerRe: [RESEND PATCH 2/2] TCP: pass accepted-TFO indication through getsockopt
In reply to Jeremy HarrisQuoted 2 lines · Jeremy Harris
Signed-off-by Jeremy Harris <[email protected]>
Regarding the proposed commit title:
TCP: pass accepted-TFO indication through getsockopt
Please use something more like:
tcp: fastopen: pass TFO child indication through getsockopt
Quoted 10 lines
--- include/uapi/linux/tcp.h | 1 + net/ipv4/tcp.c | 2 ++ 2 files changed, 3 insertions(+)include/uapi/linux/tcp.h +0
-0
include/uapi/linux/tcp.h enum tcp_fastopen_client_fail { · line 187
Quoted 4 lines
include/uapi/linux/tcp.h +1
-0@@ -184,6 +184,7 @@ enum tcp_fastopen_client_fail {#define TCPI_OPT_ECN_SEEN 16/* we received at least one packet with ECT */ #define TCPI_OPT_SYN_DATA 32/* SYN-ACK acked data in SYN sent or rcvd */ #define TCPI_OPT_USEC_TS 64/* usec timestamps */ +#define TCPI_OPT_TFO_SEEN 128/* we accepted a Fast Open option on SYN */
IMHO this bit name is slightly misleading, and does not match the comment.
Sometimes when a SYN is received with a TFO option the server will fail to create a child because the TFO cookie is incorrect. In such a case, a TFO option is "seen", but TFO is not *used* because the TFO cookie is incorrect; so this "TFO_SEEN" bit would be 0 even though a TFO option was "seen". IMHO that is confusing/misleading.
When this bit is set, we know not only that the "Received SYN includes Fast Open option" (comment from the previous patch), but we also know that the TFO cookie was correct and a child socket was created.
So I would suggest a more specific bit name, something like:
+#define TCPI_OPT_TFO_CHILD 128 /* child from a Fast Open option on SYN */
+ if (tp->syn_fastopen_child)
+ info->tcpi_options |= TCPI_OPT_TFO_CHILD;
thanks,
neal
Re: [RESEND PATCH 2/2] TCP: pass accepted-TFO indication through getsockopt
Thanks for the review. I'm preparing a v2, and a patch to "ss" in iproute2.
Quoted 5 lines · Neal Cardwell
TCP: pass accepted-TFO indication through getsockopt
Please use something more like:
tcp: fastopen: pass TFO child indication through getsockopt
Will change for v2.
Quoted 16 lines
Quoted 1 line
+#define
TCPI_OPT_TFO_SEEN128 /* we accepted a Fast Open option on SYN */IMHO this bit name is slightly misleading, and does not match the comment.
Sometimes when a SYN is received with a TFO option the server will fail to create a child because the TFO cookie is incorrect. In such a case, a TFO option is "seen", but TFO is not *used* because the TFO cookie is incorrect; so this "
TFO_SEEN" bit would be 0 even though a TFO option was "seen". IMHO that is confusing/misleading.When this bit is set, we know not only that the "Received SYN includes Fast Open option" (comment from the previous patch), but we also know that the TFO cookie was correct and a child socket was created.
So I would suggest a more specific bit name, something like:
Review
Quoted 4 lines
(quoted diff) +3
-0+#define TCPI_OPT_TFO_CHILD 128 /* child from a Fast Open option on SYN */ + if (tp->syn_fastopen_child) + info->tcpi_options |= TCPI_OPT_TFO_CHILD;
Done.
Signature · 2 lines
Cheers, Jeremy