Skip to contentSkip to conversation

tcp: Enable header prediction for active open connections with MD5.

PATCH In review Kuniyuki Iwashima 4 participants 6 messages Aug 3, 2023

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 tcp: Enable header prediction for active open connections with MD5.

TCP socket saves the minimum required header length in tcp_header_len of struct tcp_sock, and later the value is used in __tcp_fast_path_on() to generate a part of TCP header in tcp_sock(sk)->pred_flags.

In tcp_rcv_established(), if the incoming packet has the same pattern with pred_flags, we enter the fast path and skip full option parsing.

The MD5 option is parsed in tcp_v[46]_rcv(), so we need not parse it again later in tcp_rcv_established() unless other options exist. Thus, MD5 should add TCPOLEN_MD5SIG_ALIGNED to tcp_header_len and avoid the slow path.

For passive open connections with MD5, we add TCPOLEN_MD5SIG_ALIGNED to tcp_header_len in tcp_create_openreq_child() after 3WHS.

On the other hand, we do it in tcp_connect_init() for active open connections. However, the value is overwritten while processing SYN+ACK or crossed SYN in tcp_rcv_synsent_state_process().

1) SYN+ACK

    tcp_rcv_synsent_state_process
      tp->tcp_header_len = sizeof(struct tcphdr) or
                           sizeof(struct tcphdr) + TCPOLEN_TSTAMP_ALIGNED
      tcp_finish_connect
        __tcp_fast_path_on
      tcp_send_ack

2) Crossed SYN and the following ACK

    tcp_rcv_synsent_state_process
      tp->tcp_header_len = sizeof(struct tcphdr) or
                           sizeof(struct tcphdr) + TCPOLEN_TSTAMP_ALIGNED
      tcp_set_state(sk, TCP_SYN_RECV)
      tcp_send_synack

    -- ACK received --
    tcp_v4_rcv
      tcp_v4_do_rcv
        tcp_rcv_state_process
          tcp_fast_path_on
            __tcp_fast_path_on

So these two cases will have the wrong value in pred_flags and never go into the fast path.

Let's add TCPOLEN_MD5SIG_ALIGNED in tcp_rcv_synsent_state_process() to enable header prediction for active open connections.

Fixes cfb6eeb4c860 ("[TCP]: MD5 Signature Option (RFC2385) support.")

---
v2:
* Update function graph

v1: https://lore.kernel.org/all/[email protected]/

---
 net/ipv4/tcp_input.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
net/ipv4/tcp_input.c +10 -0
index 57c8af1859c1..4d274b511d97 100644
@@ -6291,6 +6291,11 @@ static int tcp_rcv_synsent_state_process(struct sock *sk, struct sk_buff *skb,
 			tp->tcp_header_len = sizeof(struct tcphdr);
 		}
 
+#ifdef CONFIG_TCP_MD5SIG
+		if (tp->af_specific->md5_lookup(sk, sk))
+			tp->tcp_header_len += TCPOLEN_MD5SIG_ALIGNED;
+#endif
+
 		tcp_sync_mss(sk, icsk->icsk_pmtu_cookie);
 		tcp_initialize_rcv_mss(sk);
 
@@ -6368,6 +6373,11 @@ static int tcp_rcv_synsent_state_process(struct sock *sk, struct sk_buff *skb,
 			tp->tcp_header_len = sizeof(struct tcphdr);
 		}
 
+#ifdef CONFIG_TCP_MD5SIG
+		if (tp->af_specific->md5_lookup(sk, sk))
+			tp->tcp_header_len += TCPOLEN_MD5SIG_ALIGNED;
+#endif
+
 		WRITE_ONCE(tp->rcv_nxt, TCP_SKB_CB(skb)->seq + 1);
 		WRITE_ONCE(tp->copied_seq, tp->rcv_nxt);
 		tp->rcv_wup = TCP_SKB_CB(skb)->seq + 1;
Signature · 1 line
2.30.2
#1 lore raw
Quoted 44 lines · Kuniyuki Iwashima

TCP socket saves the minimum required header length in tcp_header_len of struct tcp_sock, and later the value is used in __tcp_fast_path_on() to generate a part of TCP header in tcp_sock(sk)->pred_flags.

In tcp_rcv_established(), if the incoming packet has the same pattern with pred_flags, we enter the fast path and skip full option parsing.

The MD5 option is parsed in tcp_v[46]_rcv(), so we need not parse it again later in tcp_rcv_established() unless other options exist. Thus, MD5 should add TCPOLEN_MD5SIG_ALIGNED to tcp_header_len and avoid the slow path.

For passive open connections with MD5, we add TCPOLEN_MD5SIG_ALIGNED to tcp_header_len in tcp_create_openreq_child() after 3WHS.

On the other hand, we do it in tcp_connect_init() for active open connections. However, the value is overwritten while processing SYN+ACK or crossed SYN in tcp_rcv_synsent_state_process().

1) SYN+ACK

    tcp_rcv_synsent_state_process
      tp->tcp_header_len = sizeof(struct tcphdr) or
                           sizeof(struct tcphdr) + TCPOLEN_TSTAMP_ALIGNED
      tcp_finish_connect
        __tcp_fast_path_on
      tcp_send_ack

2) Crossed SYN and the following ACK

    tcp_rcv_synsent_state_process
      tp->tcp_header_len = sizeof(struct tcphdr) or
                           sizeof(struct tcphdr) + TCPOLEN_TSTAMP_ALIGNED
      tcp_set_state(sk, TCP_SYN_RECV)
      tcp_send_synack

    -- ACK received --
    tcp_v4_rcv
      tcp_v4_do_rcv
        tcp_rcv_state_process
          tcp_fast_path_on
            __tcp_fast_path_on
Quoted 5 lines

So these two cases will have the wrong value in pred_flags and never go into the fast path.

Let's add TCPOLEN_MD5SIG_ALIGNED in tcp_rcv_synsent_state_process() to enable header prediction for active open connections.

I do not think we want to slow down fast path (no MD5), for 'header prediction' of MD5 flows,
considering how slow MD5 is anyway (no GSO/GRO), and add yet another ugly #ifdef CONFIG_TCP_MD5SIG
in already convoluted code base.

The case of cross-syn is kind of hilarious, if you ask me.

Quoted 2 lines
Fixes cfb6eeb4c860 ("[TCP]: MD5 Signature Option (RFC2385) support.")

This would be net-next material anyway, unless you show a huge improvement after this patch,
which I doubt very much.
Or maybe the real intent is not fully expressed in your changelog ?

maintainer
#2 lore raw
Quoted 48 lines · Eric Dumazet
Quoted 47 lines · Kuniyuki Iwashima

TCP socket saves the minimum required header length in tcp_header_len of struct tcp_sock, and later the value is used in __tcp_fast_path_on() to generate a part of TCP header in tcp_sock(sk)->pred_flags.

In tcp_rcv_established(), if the incoming packet has the same pattern with pred_flags, we enter the fast path and skip full option parsing.

The MD5 option is parsed in tcp_v[46]_rcv(), so we need not parse it again later in tcp_rcv_established() unless other options exist. Thus, MD5 should add TCPOLEN_MD5SIG_ALIGNED to tcp_header_len and avoid the slow path.

For passive open connections with MD5, we add TCPOLEN_MD5SIG_ALIGNED to tcp_header_len in tcp_create_openreq_child() after 3WHS.

On the other hand, we do it in tcp_connect_init() for active open connections. However, the value is overwritten while processing SYN+ACK or crossed SYN in tcp_rcv_synsent_state_process().

1) SYN+ACK

    tcp_rcv_synsent_state_process
      tp->tcp_header_len = sizeof(struct tcphdr) or
                           sizeof(struct tcphdr) + TCPOLEN_TSTAMP_ALIGNED
      tcp_finish_connect
        __tcp_fast_path_on
      tcp_send_ack

2) Crossed SYN and the following ACK

    tcp_rcv_synsent_state_process
      tp->tcp_header_len = sizeof(struct tcphdr) or
                           sizeof(struct tcphdr) + TCPOLEN_TSTAMP_ALIGNED
      tcp_set_state(sk, TCP_SYN_RECV)
      tcp_send_synack

    -- ACK received --
    tcp_v4_rcv
      tcp_v4_do_rcv
        tcp_rcv_state_process
          tcp_fast_path_on
            __tcp_fast_path_on

So these two cases will have the wrong value in pred_flags and never go into the fast path.

Quoted 8 lines
Quoted 2 lines

Let's add TCPOLEN_MD5SIG_ALIGNED in tcp_rcv_synsent_state_process() to enable header prediction for active open connections.

I do not think we want to slow down fast path (no MD5), for 'header prediction' of MD5 flows,
considering how slow MD5 is anyway (no GSO/GRO), and add yet another ugly #ifdef CONFIG_TCP_MD5SIG
in already convoluted code base.

Somewhat related, do you know how much header prediction makes a difference for plain TCP? IIRC quite some time ago there was the idea to remove header prediction completely to simplify the code overall - then reverted because indeed caused regression in RR test-case. Do you know if that is still true? would it make sense to re-evaluate that thing (HP removal) again?

Thanks!

Paolo

#3 lore raw
Quoted 59 lines · Paolo Abeni
Quoted 48 lines · Eric Dumazet
Quoted 47 lines · Kuniyuki Iwashima

TCP socket saves the minimum required header length in tcp_header_len of struct tcp_sock, and later the value is used in __tcp_fast_path_on() to generate a part of TCP header in tcp_sock(sk)->pred_flags.

In tcp_rcv_established(), if the incoming packet has the same pattern with pred_flags, we enter the fast path and skip full option parsing.

The MD5 option is parsed in tcp_v[46]_rcv(), so we need not parse it again later in tcp_rcv_established() unless other options exist. Thus, MD5 should add TCPOLEN_MD5SIG_ALIGNED to tcp_header_len and avoid the slow path.

For passive open connections with MD5, we add TCPOLEN_MD5SIG_ALIGNED to tcp_header_len in tcp_create_openreq_child() after 3WHS.

On the other hand, we do it in tcp_connect_init() for active open connections. However, the value is overwritten while processing SYN+ACK or crossed SYN in tcp_rcv_synsent_state_process().

1) SYN+ACK

    tcp_rcv_synsent_state_process
      tp->tcp_header_len = sizeof(struct tcphdr) or
                           sizeof(struct tcphdr) + TCPOLEN_TSTAMP_ALIGNED
      tcp_finish_connect
        __tcp_fast_path_on
      tcp_send_ack

2) Crossed SYN and the following ACK

    tcp_rcv_synsent_state_process
      tp->tcp_header_len = sizeof(struct tcphdr) or
                           sizeof(struct tcphdr) + TCPOLEN_TSTAMP_ALIGNED
      tcp_set_state(sk, TCP_SYN_RECV)
      tcp_send_synack

    -- ACK received --
    tcp_v4_rcv
      tcp_v4_do_rcv
        tcp_rcv_state_process
          tcp_fast_path_on
            __tcp_fast_path_on

So these two cases will have the wrong value in pred_flags and never go into the fast path.

Quoted 8 lines
Quoted 2 lines

Let's add TCPOLEN_MD5SIG_ALIGNED in tcp_rcv_synsent_state_process() to enable header prediction for active open connections.

I do not think we want to slow down fast path (no MD5), for 'header prediction' of MD5 flows,
considering how slow MD5 is anyway (no GSO/GRO), and add yet another ugly #ifdef CONFIG_TCP_MD5SIG
in already convoluted code base.

Quoted 7 lines

Somewhat related, do you know how much header prediction makes a difference for plain TCP? IIRC quite some time ago there was the idea to remove header prediction completely to simplify the code overall - then reverted because indeed caused regression in RR test-case. Do you know if that is still true? would it make sense to re-evaluate that thing (HP removal) again?

I think Florian did this, he might recall the details.

#4 lore raw
Quoted 17 lines · Eric Dumazet
Quoted 8 lines · Paolo Abeni
Quoted 5 lines · Eric Dumazet

I do not think we want to slow down fast path (no MD5), for 'header prediction' of MD5 flows,
considering how slow MD5 is anyway (no GSO/GRO), and add yet another ugly #ifdef CONFIG_TCP_MD5SIG
in already convoluted code base.

Quoted 7 lines

Somewhat related, do you know how much header prediction makes a difference for plain TCP? IIRC quite some time ago there was the idea to remove header prediction completely to simplify the code overall - then reverted because indeed caused regression in RR test-case. Do you know if that is still true? would it make sense to re-evaluate that thing (HP removal) again?

Quoted 1 line

I think Florian did this, he might recall the details.

Memory is hazy here, 31770e34e43d ("tcp: Revert "tcp: remove header prediction"") has some clues.

One would need to refactor tcp_ack so that all the extra accesses are only done if the packet matches expected next sequence without advancing the window. Not sure its worth doing, one would start to add tcp header prediction v2, with little or no significant LOC savings.

#5 lore raw
In reply to Eric Dumazet

From: Eric Dumazet <[email protected]>
Date: Thu, 3 Aug 2023 08:44:14 +0200

Quoted 57 lines · Kuniyuki Iwashima
Quoted 44 lines

TCP socket saves the minimum required header length in tcp_header_len of struct tcp_sock, and later the value is used in __tcp_fast_path_on() to generate a part of TCP header in tcp_sock(sk)->pred_flags.

In tcp_rcv_established(), if the incoming packet has the same pattern with pred_flags, we enter the fast path and skip full option parsing.

The MD5 option is parsed in tcp_v[46]_rcv(), so we need not parse it again later in tcp_rcv_established() unless other options exist. Thus, MD5 should add TCPOLEN_MD5SIG_ALIGNED to tcp_header_len and avoid the slow path.

For passive open connections with MD5, we add TCPOLEN_MD5SIG_ALIGNED to tcp_header_len in tcp_create_openreq_child() after 3WHS.

On the other hand, we do it in tcp_connect_init() for active open connections. However, the value is overwritten while processing SYN+ACK or crossed SYN in tcp_rcv_synsent_state_process().

1) SYN+ACK

    tcp_rcv_synsent_state_process
      tp->tcp_header_len = sizeof(struct tcphdr) or
                           sizeof(struct tcphdr) + TCPOLEN_TSTAMP_ALIGNED
      tcp_finish_connect
        __tcp_fast_path_on
      tcp_send_ack

2) Crossed SYN and the following ACK

    tcp_rcv_synsent_state_process
      tp->tcp_header_len = sizeof(struct tcphdr) or
                           sizeof(struct tcphdr) + TCPOLEN_TSTAMP_ALIGNED
      tcp_set_state(sk, TCP_SYN_RECV)
      tcp_send_synack

    -- ACK received --
    tcp_v4_rcv
      tcp_v4_do_rcv
        tcp_rcv_state_process
          tcp_fast_path_on
            __tcp_fast_path_on
Quoted 5 lines

So these two cases will have the wrong value in pred_flags and never go into the fast path.

Let's add TCPOLEN_MD5SIG_ALIGNED in tcp_rcv_synsent_state_process() to enable header prediction for active open connections.

I do not think we want to slow down fast path (no MD5), for 'header prediction' of MD5 flows,
considering how slow MD5 is anyway (no GSO/GRO), and add yet another ugly #ifdef CONFIG_TCP_MD5SIG
in already convoluted code base.

Quoted 8 lines

The case of cross-syn is kind of hilarious, if you ask me.

Quoted 2 lines
Fixes cfb6eeb4c860 ("[TCP]: MD5 Signature Option (RFC2385) support.")

This would be net-next material anyway, unless you show a huge improvement after this patch,
which I doubt very much.

I just noticed the MD5 does not add TCPOLEN_MD5SIG_ALIGNED and thought it would benefit from the fast path, but sorry, you were correct.

I have tested with slightly modified netperf that uses MD5 for each flow and found that the patch does not improve MD5 perf at all. Rather, without all TCPOLEN_MD5SIG_ALIGNED addition in tcp_connect_init() and tcp_create_openreq_child(), the slow path became the faster path for MD5.

On c5.4xlarge EC2 instance (16 vCPU, 32 GiB mem)

  $ for i in {1..10}; do
  ./super_netperf $(nproc) -H localhost -l 10 -- -m 256 -M 256 2>/dev/null;
  done

- 36e68eadd303 : 10.376 Gbps
- all fast path : 10.374 Gbps
- all slow path : 10.394 Gbps

I'll post v3 that disable fast path for MD5 instead.

Thanks!