tcp: rehash onto different local ECMP path on retransmit timeout
Subsystems (4)
v3 cover tcp: rehash onto different local ECMP path on retransmit timeout
Make TCP retransmission timeouts select a different ECMP path for IPv6.
Currently sk_rethink_txhash() changes the socket's txhash on RTO, but the cached route is reused and the new hash is not propagated into the ECMP path selection logic. This series adds sk_dst_reset() alongside sk_rethink_txhash() to force a fresh route lookup, and sets fl6->mp_hash from sk_txhash so fib6_select_path() picks a path based on the new hash.
Five selftest scenarios verify the behavior across connection setup and established flows, forward and reverse path failures, and PLB:
- SYN retransmission (forward path blocked during setup)
- SYN/ACK retransmission (reverse path blocked during setup)
- Midstream RTO (forward path blocked on established connection)
- Midstream ACK rehash (reverse path blocked on established connection)
- PLB rehash (ECN-driven congestion on established connection)
Changes since v2: https://lore.kernel.org/netdev/[email protected]/
- Retitle "ECMP" to "local ECMP" to distinguish from remote ECMP
(Neal Cardwell)
- Add fl6->mp_hash propagation in inet6_sk_rebuild_header() (af_inet6.c), covering the dst rebuild path used on established sockets
- Remove incorrect ir_iif update from tcp_check_req() in tcp_minisocks.c;
the SYN/ACK rehash is already handled by tcp_rtx_synack() re-rolling
txhash which feeds into inet6_csk_route_req()'s mp_hash
(Eric Dumazet)
- Add ACK rehash and PLB rehash selftests
- Improve selftest reliability
Changes since v1: https://lore.kernel.org/netdev/[email protected]/
- Use tcp_rsk(req)->txhash instead of jhash_1word(req->num_retrans, ...)
for ECMP path selection in inet6_csk_route_req(), making the request
socket path consistent with the established socket path (Eric Dumazet)
- Add comments explaining the >> 1 shift for 31-bit mp_hash range
- Use socat -u (unidirectional) in selftest to avoid SIGPIPE race
- Increase tcp_syn_retries and tcp_syn_linear_timeouts to 25 for
better rehash coverage
Neil Spring (2):
tcp: rehash onto different local ECMP path on retransmit timeout
selftests: net: add local ECMP rehash test
net/ipv4/tcp_input.c | 4 +- net/ipv4/tcp_plb.c | 1 + net/ipv4/tcp_timer.c | 1 + net/ipv6/af_inet6.c | 3 + net/ipv6/inet6_connection_sock.c | 6 + tools/testing/selftests/net/Makefile | 1 + tools/testing/selftests/net/ecmp_rehash.sh | 569 +++++++++++++++++++++ 7 files changed, 584 insertions(+), 1 deletion(-) create mode 100755 tools/testing/selftests/net/ecmp_rehash.sh
Signature · 1 line
2.52.0
PATCH v3 1/2 tcp: rehash onto different local ECMP path on retransmit timeout
Currently sk_rethink_txhash() re-rolls the socket's txhash on RTO, PLB, and spurious-retransmission events, but the cached route is reused and the new hash is not propagated into the ECMP path selection logic. Two changes are needed to make rehash select a different local ECMP path:
1. Add sk_dst_reset() alongside sk_rethink_txhash() in tcp_write_timeout(), tcp_rcv_spurious_retrans(), and tcp_plb_check_rehash() so the cached dst is invalidated and the next transmit triggers a fresh route lookup.
2. Set fl6->mp_hash from sk_txhash (or tcp_rsk(req)->txhash for SYN/ACK retransmits) in inet6_sk_rebuild_header(), inet6_csk_route_req(), and inet6_csk_route_socket() so fib6_select_path() picks a path based on the new hash.
It is necessary to update mp_hash explicitly because the default ECMP hash derives from fl6->flowlabel via np->flow_label, which is not updated from sk_txhash (REPFLOW is off by default). ip6_make_flowlabel() cannot help either, as it runs after the route lookup.
--- net/ipv4/tcp_input.c | 4 +++- net/ipv4/tcp_plb.c | 1 + net/ipv4/tcp_timer.c | 1 + net/ipv6/af_inet6.c | 3 +++ net/ipv6/inet6_connection_sock.c | 6 ++++++ 5 files changed, 14 insertions(+), 1 deletion(-)
net/ipv4/tcp_input.c +3 -1
@@ -5020,8 +5020,10 @@ static void tcp_rcv_spurious_retrans(struct sock *sk, skb->protocol == htons(ETH_P_IPV6) && (tcp_sk(sk)->inet_conn.icsk_ack.lrcv_flowlabel != ntohl(ip6_flowlabel(ipv6_hdr(skb)))) && - sk_rethink_txhash(sk)) + sk_rethink_txhash(sk)) {NET_INC_STATS (sock_net (sk),LINUX_MIB_TCPDUPLICATEDATAREHASH ); +sk_dst_reset (sk); + }/* Save last flowlabel after a spurious retrans. */ tcp_save_lrcv_flowlabel (sk, skb);
net/ipv4/tcp_plb.c +1 -0
@@ -79,6 +79,7 @@ void tcp_plb_check_rehash(struct sock *sk, struct tcp_plb_state *plb)return ;sk_rethink_txhash (sk); +sk_dst_reset (sk); plb->consec_cong_rounds =0 ;WRITE_ONCE (tcp_sk (sk)->plb_rehash,tcp_sk (sk)->plb_rehash +1 );NET_INC_STATS (sock_net (sk),LINUX_MIB_TCPPLBREHASH );
net/ipv4/tcp_timer.c +1 -0
@@ -300,6 +300,7 @@ static int tcp_write_timeout(struct sock *sk)if (sk_rethink_txhash (sk)) {WRITE_ONCE (tp->timeout_rehash, tp->timeout_rehash +1 );__NET_INC_STATS (sock_net (sk),LINUX_MIB_TCPTIMEOUTREHASH ); +sk_dst_reset (sk); }return 0 ;
net/ipv6/af_inet6.c +3 -0
@@ -823,6 +823,9 @@ int inet6_sk_rebuild_header(struct sock *sk) fl6->flowi6_uid =sk_uid (sk);security_sk_classify_flow (sk,flowi6_to_flowi_common (fl6)); +/* >> 1 for 31-bit mp_hash range matching nhc_upper_bound. */ + fl6->mp_hash = sk->sk_txhash >>1 ; +rcu_read_lock (); final_p =fl6_update_dst (fl6,rcu_dereference (np->opt), &np->final);rcu_read_unlock ();
net/ipv6/inet6_connection_sock.c +6 -0
@@ -48,6 +48,9 @@ struct dst_entry *inet6_csk_route_req(const struct sock *sk, fl6->flowi6_uid =sk_uid (sk);security_req_classify_flow (req,flowi6_to_flowi_common (fl6)); +/* >> 1 for 31-bit mp_hash range matching nhc_upper_bound. */ + fl6->mp_hash =tcp_rsk (req)->txhash >>1 ; +if (!dst) { dst =ip6_dst_lookup_flow (sock_net (sk), sk, fl6, final_p);if (IS_ERR (dst))
@@ -70,6 +73,9 @@ struct dst_entry *inet6_csk_route_socket(struct sock *sk, fl6->saddr = np->saddr; fl6->flowlabel = np->flow_label;IP6_ECN_flow_xmit (sk, fl6->flowlabel); + +/* >> 1 for 31-bit mp_hash range matching nhc_upper_bound. */ + fl6->mp_hash = sk->sk_txhash >>1 ; fl6->flowi6_oif = sk->sk_bound_dev_if; fl6->flowi6_mark = sk->sk_mark; fl6->fl6_sport = inet->inet_sport;
Signature · 1 line
2.52.0
Quoted 36 lines · Neil Spring
Currently
sk_rethink_txhash()re-rolls the socket's txhash on RTO, PLB, and spurious-retransmission events, but the cached route is reused and the new hash is not propagated into the ECMP path selection logic. Two changes are needed to make rehash select a different local ECMP path:1. Add
sk_dst_reset()alongsidesk_rethink_txhash()intcp_write_timeout(),tcp_rcv_spurious_retrans(), andtcp_plb_check_rehash()so the cached dst is invalidated and the next transmit triggers a fresh route lookup.2. Set
fl6->mp_hashfromsk_txhash(or tcp_rsk(req)->txhash for SYN/ACK retransmits) ininet6_sk_rebuild_header(),inet6_csk_route_req(), andinet6_csk_route_socket()sofib6_select_path()picks a path based on the new hash.It is necessary to update
mp_hashexplicitly because the default ECMP hash derives fromfl6->flowlabelvianp->flow_label, which is not updated fromsk_txhash(REPFLOW is off by default).ip6_make_flowlabel()cannot help either, as it runs after the route lookup.Signed-off-by Neil Spring <[email protected]>--- net/ipv4/tcp_input.c | 4 +++- net/ipv4/tcp_plb.c | 1 + net/ipv4/tcp_timer.c | 1 + net/ipv6/af_inet6.c | 3 +++ net/ipv6/inet6_connection_sock.c | 6 ++++++ 5 files changed, 14 insertions(+), 1 deletion(-)net/ipv4/tcp_input.c +0
-0
net/ipv4/tcp_input.c static void tcp_rcv_spurious_retrans(struct sock *sk, · line 5025
Quoted 7 lines
net/ipv4/tcp_input.c +2
-1@@ -5020,8 +5020,10 @@ static void tcp_rcv_spurious_retrans(struct sock *sk, skb->protocol == htons(ETH_P_IPV6) && (tcp_sk(sk)->inet_conn.icsk_ack.lrcv_flowlabel != ntohl(ip6_flowlabel(ipv6_hdr(skb)))) && - sk_rethink_txhash(sk)) + sk_rethink_txhash(sk)) {NET_INC_STATS (sock_net (sk),LINUX_MIB_TCPDUPLICATEDATAREHASH ); +sk_dst_reset (sk);
I think it would be nice to use __sk_dst_reset() in TCP, to reflect the fact that we own the socket lock.
PATCH v3 2/2 selftests: net: add local ECMP rehash test
In reply to Neil SpringAdd ecmp_rehash.sh to exercise TCP local ECMP path re-selection on retransmission timeout and PLB. The topology is two namespaces connected by two parallel veth pairs with a 2-way ECMP route. When a TCP path is blocked (via tc drop) or congested (via netem ECN marking), the kernel rehashes the connection via sk_rethink_txhash() + sk_dst_reset(), causing the next route lookup to pick the other local ECMP nexthop through the new fl6->mp_hash propagation.
Five tests cover the code paths modified in the preceding patch:
test_ecmp_syn_rehash: Block both forward paths during connection setup. With tcp_syn_retries=25, SYNs retry at 1-second intervals; verify that both interfaces carry SYN packets, proving tcp_write_timeout() rehashed the SYN to a new path.
test_ecmp_synack_rehash: Block the server's return paths so SYN/ACKs are dropped. Each duplicate SYN arriving at the server triggers tcp_rtx_synack() which re-rolls tcp_rsk(req)->txhash, so the retransmitted SYN/ACK selects a different return path via inet6_csk_route_req().
test_ecmp_midstream_rehash: Establish a data transfer, then block the active forward path. Verify data appears on the alternate path after RTO triggers sk_dst_reset() in tcp_write_timeout() and the retransmit rebuilds the route through inet6_sk_rebuild_header() with the new mp_hash. Sets tcp_retries1=255 to suppress the pre-existing __dst_negative_advice() fallback, isolating the patch's sk_dst_reset() as the sole dst invalidation mechanism.
test_ecmp_midstream_ack_rehash: Establish a data transfer, then block the receiver's return paths so ACKs are dropped. The sender times out and retransmits with a new flowlabel; the receiver detects the changed flowlabel via tcp_rcv_spurious_retrans() and rehashes its own txhash so that its ACKs try a different ECMP return path.
test_ecmp_plb_rehash: Establish a data transfer using DCTCP, then apply ECN marking (netem loss 100% ecn) on the active path. Sustained ECN congestion triggers PLB (Protective Load Balancing) which calls sk_rethink_txhash() + sk_dst_reset(), moving traffic to the alternate path. Verifies that TCPPLBRehash increments and TcpTimeoutRehash does not, confirming the rehash is PLB-driven, not RTO-driven.
--- tools/testing/selftests/net/Makefile | 1 + tools/testing/selftests/net/ecmp_rehash.sh | 569 +++++++++++++++++++++ 2 files changed, 570 insertions(+) create mode 100755 tools/testing/selftests/net/ecmp_rehash.sh
tools/testing/selftests/net/Makefile +1 -0
@@ -26,6 +26,7 @@ TEST_PROGS := \ cmsg_time.sh \ double_udp_encap.sh \ drop_monitor_tests.sh \ + ecmp_rehash.sh \ fcnal-ipv4.sh \ fcnal-ipv6.sh \ fcnal-other.sh \
tools/testing/selftests/net/ecmp_rehash.sh new file +569 -0
569 lines not on this page · raw message
Signature · 1 line
2.52.0