Skip to contentSkip to conversation

tcp: rehash onto different local ECMP path on retransmit timeout

PATCH Superseded Neil Spring 2 participants 4 messages May 5 – May 6

Subsystems (4)

v3 of 14 Superseded Latest is v14 →

lore
Share & tools
lore
#0 lore raw

v3 cover tcp: rehash onto different local ECMP path on retransmit timeout

Make TCP retransmission timeouts select a different ECMP path for IPv6.

Currently sk_rethink_txhash() changes the socket's txhash on RTO, but the cached route is reused and the new hash is not propagated into the ECMP path selection logic. This series adds sk_dst_reset() alongside sk_rethink_txhash() to force a fresh route lookup, and sets fl6->mp_hash from sk_txhash so fib6_select_path() picks a path based on the new hash.

Five selftest scenarios verify the behavior across connection setup and established flows, forward and reverse path failures, and PLB:

- SYN retransmission (forward path blocked during setup)
- SYN/ACK retransmission (reverse path blocked during setup)
- Midstream RTO (forward path blocked on established connection)
- Midstream ACK rehash (reverse path blocked on established connection)
- PLB rehash (ECN-driven congestion on established connection)

Changes since v2: https://lore.kernel.org/netdev/[email protected]/
- Retitle "ECMP" to "local ECMP" to distinguish from remote ECMP
(Neal Cardwell)
- Add fl6->mp_hash propagation in inet6_sk_rebuild_header() (af_inet6.c), covering the dst rebuild path used on established sockets
- Remove incorrect ir_iif update from tcp_check_req() in tcp_minisocks.c;
the SYN/ACK rehash is already handled by tcp_rtx_synack() re-rolling
txhash which feeds into inet6_csk_route_req()'s mp_hash
(Eric Dumazet)
- Add ACK rehash and PLB rehash selftests
- Improve selftest reliability

Changes since v1: https://lore.kernel.org/netdev/[email protected]/
- Use tcp_rsk(req)->txhash instead of jhash_1word(req->num_retrans, ...)
for ECMP path selection in inet6_csk_route_req(), making the request
socket path consistent with the established socket path (Eric Dumazet)
- Add comments explaining the >> 1 shift for 31-bit mp_hash range
- Use socat -u (unidirectional) in selftest to avoid SIGPIPE race
- Increase tcp_syn_retries and tcp_syn_linear_timeouts to 25 for
better rehash coverage

Neil Spring (2):
tcp: rehash onto different local ECMP path on retransmit timeout
selftests: net: add local ECMP rehash test

 net/ipv4/tcp_input.c                       |   4 +-
 net/ipv4/tcp_plb.c                         |   1 +
 net/ipv4/tcp_timer.c                       |   1 +
 net/ipv6/af_inet6.c                        |   3 +
 net/ipv6/inet6_connection_sock.c           |   6 +
 tools/testing/selftests/net/Makefile       |   1 +
 tools/testing/selftests/net/ecmp_rehash.sh | 569 +++++++++++++++++++++
 7 files changed, 584 insertions(+), 1 deletion(-)
 create mode 100755 tools/testing/selftests/net/ecmp_rehash.sh
Signature · 1 line
2.52.0
#1 lore raw

PATCH v3 1/2 tcp: rehash onto different local ECMP path on retransmit timeout

Currently sk_rethink_txhash() re-rolls the socket's txhash on RTO, PLB, and spurious-retransmission events, but the cached route is reused and the new hash is not propagated into the ECMP path selection logic. Two changes are needed to make rehash select a different local ECMP path:

1. Add sk_dst_reset() alongside sk_rethink_txhash() in tcp_write_timeout(), tcp_rcv_spurious_retrans(), and tcp_plb_check_rehash() so the cached dst is invalidated and the next transmit triggers a fresh route lookup.

2. Set fl6->mp_hash from sk_txhash (or tcp_rsk(req)->txhash for SYN/ACK retransmits) in inet6_sk_rebuild_header(), inet6_csk_route_req(), and inet6_csk_route_socket() so fib6_select_path() picks a path based on the new hash.

It is necessary to update mp_hash explicitly because the default ECMP hash derives from fl6->flowlabel via np->flow_label, which is not updated from sk_txhash (REPFLOW is off by default). ip6_make_flowlabel() cannot help either, as it runs after the route lookup.

---
 net/ipv4/tcp_input.c             | 4 +++-
 net/ipv4/tcp_plb.c               | 1 +
 net/ipv4/tcp_timer.c             | 1 +
 net/ipv6/af_inet6.c              | 3 +++
 net/ipv6/inet6_connection_sock.c | 6 ++++++
 5 files changed, 14 insertions(+), 1 deletion(-)
net/ipv4/tcp_input.c +3 -1
index 7995a89bafc9..126dffd675c9 100644
@@ -5020,8 +5020,10 @@ static void tcp_rcv_spurious_retrans(struct sock *sk,
 	    skb->protocol == htons(ETH_P_IPV6) &&
 	    (tcp_sk(sk)->inet_conn.icsk_ack.lrcv_flowlabel !=
 	     ntohl(ip6_flowlabel(ipv6_hdr(skb)))) &&
-	    sk_rethink_txhash(sk))
+	    sk_rethink_txhash(sk)) {
 		NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPDUPLICATEDATAREHASH);
+		sk_dst_reset(sk);
+	}
 
 	/* Save last flowlabel after a spurious retrans. */
 	tcp_save_lrcv_flowlabel(sk, skb);
net/ipv4/tcp_plb.c +1 -0
index c11a0cd3f8fe..0c067a54c57a 100644
@@ -79,6 +79,7 @@ void tcp_plb_check_rehash(struct sock *sk, struct tcp_plb_state *plb)
 		return;
 
 	sk_rethink_txhash(sk);
+	sk_dst_reset(sk);
 	plb->consec_cong_rounds = 0;
 	WRITE_ONCE(tcp_sk(sk)->plb_rehash, tcp_sk(sk)->plb_rehash + 1);
 	NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPPLBREHASH);
net/ipv4/tcp_timer.c +1 -0
index 322db13333c7..d92f3ba9de81 100644
@@ -300,6 +300,7 @@ static int tcp_write_timeout(struct sock *sk)
 	if (sk_rethink_txhash(sk)) {
 		WRITE_ONCE(tp->timeout_rehash, tp->timeout_rehash + 1);
 		__NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPTIMEOUTREHASH);
+		sk_dst_reset(sk);
 	}
 
 	return 0;
net/ipv6/af_inet6.c +3 -0
index 0a88b376141d..90ff4448aa56 100644
@@ -823,6 +823,9 @@ int inet6_sk_rebuild_header(struct sock *sk)
 	fl6->flowi6_uid = sk_uid(sk);
 	security_sk_classify_flow(sk, flowi6_to_flowi_common(fl6));
 
+	/* >> 1 for 31-bit mp_hash range matching nhc_upper_bound. */
+	fl6->mp_hash = sk->sk_txhash >> 1;
+
 	rcu_read_lock();
 	final_p = fl6_update_dst(fl6, rcu_dereference(np->opt), &np->final);
 	rcu_read_unlock();
net/ipv6/inet6_connection_sock.c +6 -0
index 37534e116899..fc4b75de6af8 100644
@@ -48,6 +48,9 @@ struct dst_entry *inet6_csk_route_req(const struct sock *sk,
 	fl6->flowi6_uid = sk_uid(sk);
 	security_req_classify_flow(req, flowi6_to_flowi_common(fl6));
 
+	/* >> 1 for 31-bit mp_hash range matching nhc_upper_bound. */
+	fl6->mp_hash = tcp_rsk(req)->txhash >> 1;
+
 	if (!dst) {
 		dst = ip6_dst_lookup_flow(sock_net(sk), sk, fl6, final_p);
 		if (IS_ERR(dst))
@@ -70,6 +73,9 @@ struct dst_entry *inet6_csk_route_socket(struct sock *sk,
 	fl6->saddr = np->saddr;
 	fl6->flowlabel = np->flow_label;
 	IP6_ECN_flow_xmit(sk, fl6->flowlabel);
+
+	/* >> 1 for 31-bit mp_hash range matching nhc_upper_bound. */
+	fl6->mp_hash = sk->sk_txhash >> 1;
 	fl6->flowi6_oif = sk->sk_bound_dev_if;
 	fl6->flowi6_mark = sk->sk_mark;
 	fl6->fl6_sport = inet->inet_sport;
Signature · 1 line
2.52.0
#2 lore raw
Quoted 36 lines · Neil Spring

Currently sk_rethink_txhash() re-rolls the socket's txhash on RTO, PLB, and spurious-retransmission events, but the cached route is reused and the new hash is not propagated into the ECMP path selection logic. Two changes are needed to make rehash select a different local ECMP path:

1. Add sk_dst_reset() alongside sk_rethink_txhash() in tcp_write_timeout(), tcp_rcv_spurious_retrans(), and tcp_plb_check_rehash() so the cached dst is invalidated and the next transmit triggers a fresh route lookup.

2. Set fl6->mp_hash from sk_txhash (or tcp_rsk(req)->txhash for SYN/ACK retransmits) in inet6_sk_rebuild_header(), inet6_csk_route_req(), and inet6_csk_route_socket() so fib6_select_path() picks a path based on the new hash.

It is necessary to update mp_hash explicitly because the default ECMP hash derives from fl6->flowlabel via np->flow_label, which is not updated from sk_txhash (REPFLOW is off by default). ip6_make_flowlabel() cannot help either, as it runs after the route lookup.

---
 net/ipv4/tcp_input.c             | 4 +++-
 net/ipv4/tcp_plb.c               | 1 +
 net/ipv4/tcp_timer.c             | 1 +
 net/ipv6/af_inet6.c              | 3 +++
 net/ipv6/inet6_connection_sock.c | 6 ++++++
 5 files changed, 14 insertions(+), 1 deletion(-)
net/ipv4/tcp_input.c +0 -0
index 7995a89bafc9..126dffd675c9 100644

net/ipv4/tcp_input.c static void tcp_rcv_spurious_retrans(struct sock *sk, · line 5025

Quoted 7 lines
net/ipv4/tcp_input.c +2 -1
@@ -5020,8 +5020,10 @@ static void tcp_rcv_spurious_retrans(struct sock *sk,
            skb->protocol == htons(ETH_P_IPV6) &&
            (tcp_sk(sk)->inet_conn.icsk_ack.lrcv_flowlabel !=
             ntohl(ip6_flowlabel(ipv6_hdr(skb)))) &&
-           sk_rethink_txhash(sk))
+           sk_rethink_txhash(sk)) {
                NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPDUPLICATEDATAREHASH);
+               sk_dst_reset(sk);

I think it would be nice to use __sk_dst_reset() in TCP, to reflect the fact that we own the socket lock.

#3 lore raw

PATCH v3 2/2 selftests: net: add local ECMP rehash test

In reply to Neil Spring

Add ecmp_rehash.sh to exercise TCP local ECMP path re-selection on retransmission timeout and PLB. The topology is two namespaces connected by two parallel veth pairs with a 2-way ECMP route. When a TCP path is blocked (via tc drop) or congested (via netem ECN marking), the kernel rehashes the connection via sk_rethink_txhash() + sk_dst_reset(), causing the next route lookup to pick the other local ECMP nexthop through the new fl6->mp_hash propagation.

Five tests cover the code paths modified in the preceding patch:

test_ecmp_syn_rehash: Block both forward paths during connection setup. With tcp_syn_retries=25, SYNs retry at 1-second intervals; verify that both interfaces carry SYN packets, proving tcp_write_timeout() rehashed the SYN to a new path.

test_ecmp_synack_rehash: Block the server's return paths so SYN/ACKs are dropped. Each duplicate SYN arriving at the server triggers tcp_rtx_synack() which re-rolls tcp_rsk(req)->txhash, so the retransmitted SYN/ACK selects a different return path via inet6_csk_route_req().

test_ecmp_midstream_rehash: Establish a data transfer, then block the active forward path. Verify data appears on the alternate path after RTO triggers sk_dst_reset() in tcp_write_timeout() and the retransmit rebuilds the route through inet6_sk_rebuild_header() with the new mp_hash. Sets tcp_retries1=255 to suppress the pre-existing __dst_negative_advice() fallback, isolating the patch's sk_dst_reset() as the sole dst invalidation mechanism.

test_ecmp_midstream_ack_rehash: Establish a data transfer, then block the receiver's return paths so ACKs are dropped. The sender times out and retransmits with a new flowlabel; the receiver detects the changed flowlabel via tcp_rcv_spurious_retrans() and rehashes its own txhash so that its ACKs try a different ECMP return path.

test_ecmp_plb_rehash: Establish a data transfer using DCTCP, then apply ECN marking (netem loss 100% ecn) on the active path. Sustained ECN congestion triggers PLB (Protective Load Balancing) which calls sk_rethink_txhash() + sk_dst_reset(), moving traffic to the alternate path. Verifies that TCPPLBRehash increments and TcpTimeoutRehash does not, confirming the rehash is PLB-driven, not RTO-driven.

---
 tools/testing/selftests/net/Makefile       |   1 +
 tools/testing/selftests/net/ecmp_rehash.sh | 569 +++++++++++++++++++++
 2 files changed, 570 insertions(+)
 create mode 100755 tools/testing/selftests/net/ecmp_rehash.sh
tools/testing/selftests/net/Makefile +1 -0
index baa30287cf22..6ec1b24218ad 100644
@@ -26,6 +26,7 @@ TEST_PROGS := \
 	cmsg_time.sh \
 	double_udp_encap.sh \
 	drop_monitor_tests.sh \
+	ecmp_rehash.sh \
 	fcnal-ipv4.sh \
 	fcnal-ipv6.sh \
 	fcnal-other.sh \
tools/testing/selftests/net/ecmp_rehash.sh new file +569 -0

569 lines not on this page · raw message

Signature · 1 line
2.52.0