Skip to contentSkip to conversation

tcp: rehash onto different local ECMP path on retransmit timeout

PATCH Superseded Neil Spring 2 participants 4 messages May 7 – May 12

Subsystems (4)

v4 of 14 Superseded Latest is v14 →

lore
Share & tools
lore
#0 lore raw

v4 cover tcp: rehash onto different local ECMP path on retransmit timeout

Make TCP retransmission timeouts select a different ECMP path for IPv6.

Currently sk_rethink_txhash() changes the socket's txhash on RTO, but the cached route is reused and the new hash is not propagated into the ECMP path selection logic. This series adds __sk_dst_reset() alongside sk_rethink_txhash() to force a fresh route lookup, and sets fl6->mp_hash from sk_txhash so fib6_select_path() picks a path based on the new hash.

Five selftest scenarios verify the behavior across connection setup and established flows, forward and reverse path failures, and PLB:

- SYN retransmission (forward path blocked during setup)
- SYN/ACK retransmission (reverse path blocked during setup)
- Midstream RTO (forward path blocked on established connection)
- Midstream ACK rehash (reverse path blocked on established connection)
- PLB rehash (ECN-driven congestion on established connection)

Changes since v3: https://lore.kernel.org/netdev/[email protected]/
- Use __sk_dst_reset() instead of sk_dst_reset() since the socket lock
is held in all three call sites (Eric Dumazet)
- Guard __sk_dst_reset() with sk->sk_family == AF_INET6 since IPv4 ECMP
does not use sk_txhash for path selection
- Guard __sk_dst_reset() in tcp_plb_check_rehash() with the return value
of sk_rethink_txhash()
- Move tcp_rsk(req)->txhash initialization before route_req() in tcp_conn_request() to avoid reading uninitialized memory
- Add CONFIG_TCP_CONG_DCTCP=m to selftests/net/config for PLB test
- Skip PLB test gracefully if DCTCP is not available
- Save and restore original congestion control algorithm in PLB test
- Default get_netstat_counter() to 0 when counter is not found
- Skip all tests if tcp_syn_linear_timeouts is not available
- Replace bash/pipe data sources with socat OPEN:/dev/zero for
cleaner process cleanup
- Fix shellcheck warnings

Changes since v2: https://lore.kernel.org/netdev/[email protected]/
- Retitle "ECMP" to "local ECMP" to distinguish from remote ECMP
(Neal Cardwell)
- Add fl6->mp_hash propagation in inet6_sk_rebuild_header() (af_inet6.c), covering the dst rebuild path used on established sockets
- Remove incorrect ir_iif update from tcp_check_req() in tcp_minisocks.c;
the SYN/ACK rehash is already handled by tcp_rtx_synack() re-rolling
txhash which feeds into inet6_csk_route_req()'s mp_hash
(Eric Dumazet)
- Add ACK rehash and PLB rehash selftests
- Improve selftest reliability

Changes since v1: https://lore.kernel.org/netdev/[email protected]/
- Use tcp_rsk(req)->txhash instead of jhash_1word(req->num_retrans, ...)
for ECMP path selection in inet6_csk_route_req(), making the request
socket path consistent with the established socket path (Eric Dumazet)
- Add comments explaining the >> 1 shift for 31-bit mp_hash range
- Use socat -u (unidirectional) in selftest to avoid SIGPIPE race
- Increase tcp_syn_retries and tcp_syn_linear_timeouts to 25 for
better rehash coverage

Neil Spring (2):
tcp: rehash onto different local ECMP path on retransmit timeout
selftests: net: add local ECMP rehash test

 net/ipv4/tcp_input.c                       |   6 +-
 net/ipv4/tcp_plb.c                         |   7 +-
 net/ipv4/tcp_timer.c                       |   4 +
 net/ipv6/af_inet6.c                        |   3 +
 net/ipv6/inet6_connection_sock.c           |   6 +
 tools/testing/selftests/net/Makefile       |   1 +
 tools/testing/selftests/net/config         |   1 +
 tools/testing/selftests/net/ecmp_rehash.sh | 582 +++++++++++++++++++++
 8 files changed, 607 insertions(+), 3 deletions(-)
 create mode 100755 tools/testing/selftests/net/ecmp_rehash.sh
Signature · 1 line
2.53.0-Meta
#1 lore raw

PATCH v4 1/2 tcp: rehash onto different local ECMP path on retransmit timeout

Currently sk_rethink_txhash() re-rolls the socket's txhash on RTO, PLB, and spurious-retransmission events, but the cached route is reused and the new hash is not propagated into the ECMP path selection logic. Two changes are needed to make rehash select a different local ECMP path:

1. Add __sk_dst_reset() alongside sk_rethink_txhash() in tcp_write_timeout(), tcp_rcv_spurious_retrans(), and tcp_plb_check_rehash() so the cached dst is invalidated and the next transmit triggers a fresh route lookup.

2. Set fl6->mp_hash from sk_txhash (or tcp_rsk(req)->txhash for SYN/ACK retransmits) in inet6_sk_rebuild_header(), inet6_csk_route_req(), and inet6_csk_route_socket() so fib6_select_path() picks a path based on the new hash.

It is necessary to update mp_hash explicitly because the default ECMP hash derives from fl6->flowlabel via np->flow_label, which is not updated from sk_txhash (REPFLOW is off by default). ip6_make_flowlabel() cannot help either, as it runs after the route lookup.

The dst reset is guarded by sk->sk_family == AF_INET6 since IPv4 ECMP does not currently use sk_txhash for path selection.

tcp_rsk(req)->txhash initialization is moved before route_req() in tcp_conn_request() so that inet6_csk_route_req() reads a valid hash on the initial SYN/ACK.

---
 net/ipv4/tcp_input.c             | 6 ++++--
 net/ipv4/tcp_plb.c               | 7 ++++++-
 net/ipv4/tcp_timer.c             | 4 ++++
 net/ipv6/af_inet6.c              | 3 +++
 net/ipv6/inet6_connection_sock.c | 6 ++++++
 5 files changed, 23 insertions(+), 3 deletions(-)
net/ipv4/tcp_input.c +4 -2
index 7995a89bafc9..8f602a665b71 100644
@@ -5020,8 +5020,10 @@ static void tcp_rcv_spurious_retrans(struct sock *sk,
 	    skb->protocol == htons(ETH_P_IPV6) &&
 	    (tcp_sk(sk)->inet_conn.icsk_ack.lrcv_flowlabel !=
 	     ntohl(ip6_flowlabel(ipv6_hdr(skb)))) &&
-	    sk_rethink_txhash(sk))
+	    sk_rethink_txhash(sk)) {
 		NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPDUPLICATEDATAREHASH);
+		__sk_dst_reset(sk);
+	}
 
 	/* Save last flowlabel after a spurious retrans. */
 	tcp_save_lrcv_flowlabel(sk, skb);
@@ -7636,6 +7638,7 @@ int tcp_conn_request(struct request_sock_ops *rsk_ops,
 	tcp_rsk(req)->af_specific = af_ops;
 	tcp_rsk(req)->ts_off = 0;
 	tcp_rsk(req)->req_usec_ts = false;
+	tcp_rsk(req)->txhash = net_tx_rndhash();
 #if IS_ENABLED(CONFIG_MPTCP)
 	tcp_rsk(req)->is_mptcp = 0;
 #endif
@@ -7717,7 +7720,6 @@ int tcp_conn_request(struct request_sock_ops *rsk_ops,
 	}
 #endif
 	tcp_rsk(req)->snt_isn = isn;
-	tcp_rsk(req)->txhash = net_tx_rndhash();
 	tcp_rsk(req)->syn_tos = TCP_SKB_CB(skb)->ip_dsfield;
 	tcp_openreq_init_rwin(req, sk, dst);
 	sk_rx_queue_set(req_to_sk(req), skb);
net/ipv4/tcp_plb.c +6 -1
index c11a0cd3f8fe..accdd83dfc3d 100644
@@ -78,7 +78,12 @@ void tcp_plb_check_rehash(struct sock *sk, struct tcp_plb_state *plb)
 	if (plb->pause_until)
 		return;
 
-	sk_rethink_txhash(sk);
+	if (sk_rethink_txhash(sk)) {
+#if IS_ENABLED(CONFIG_IPV6)
+		if (sk->sk_family == AF_INET6)
+			__sk_dst_reset(sk);
+#endif
+	}
 	plb->consec_cong_rounds = 0;
 	WRITE_ONCE(tcp_sk(sk)->plb_rehash, tcp_sk(sk)->plb_rehash + 1);
 	NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPPLBREHASH);
net/ipv4/tcp_timer.c +4 -0
index 322db13333c7..24c1c19eda6e 100644
@@ -300,6 +300,10 @@ static int tcp_write_timeout(struct sock *sk)
 	if (sk_rethink_txhash(sk)) {
 		WRITE_ONCE(tp->timeout_rehash, tp->timeout_rehash + 1);
 		__NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPTIMEOUTREHASH);
+#if IS_ENABLED(CONFIG_IPV6)
+		if (sk->sk_family == AF_INET6)
+			__sk_dst_reset(sk);
+#endif
 	}
 
 	return 0;
net/ipv6/af_inet6.c +3 -0
index 0a88b376141d..90ff4448aa56 100644
@@ -823,6 +823,9 @@ int inet6_sk_rebuild_header(struct sock *sk)
 	fl6->flowi6_uid = sk_uid(sk);
 	security_sk_classify_flow(sk, flowi6_to_flowi_common(fl6));
 
+	/* >> 1 for 31-bit mp_hash range matching nhc_upper_bound. */
+	fl6->mp_hash = sk->sk_txhash >> 1;
+
 	rcu_read_lock();
 	final_p = fl6_update_dst(fl6, rcu_dereference(np->opt), &np->final);
 	rcu_read_unlock();
net/ipv6/inet6_connection_sock.c +6 -0
index 37534e116899..fc4b75de6af8 100644
@@ -48,6 +48,9 @@ struct dst_entry *inet6_csk_route_req(const struct sock *sk,
 	fl6->flowi6_uid = sk_uid(sk);
 	security_req_classify_flow(req, flowi6_to_flowi_common(fl6));
 
+	/* >> 1 for 31-bit mp_hash range matching nhc_upper_bound. */
+	fl6->mp_hash = tcp_rsk(req)->txhash >> 1;
+
 	if (!dst) {
 		dst = ip6_dst_lookup_flow(sock_net(sk), sk, fl6, final_p);
 		if (IS_ERR(dst))
@@ -70,6 +73,9 @@ struct dst_entry *inet6_csk_route_socket(struct sock *sk,
 	fl6->saddr = np->saddr;
 	fl6->flowlabel = np->flow_label;
 	IP6_ECN_flow_xmit(sk, fl6->flowlabel);
+
+	/* >> 1 for 31-bit mp_hash range matching nhc_upper_bound. */
+	fl6->mp_hash = sk->sk_txhash >> 1;
 	fl6->flowi6_oif = sk->sk_bound_dev_if;
 	fl6->flowi6_mark = sk->sk_mark;
 	fl6->fl6_sport = inet->inet_sport;
Signature · 1 line
2.53.0-Meta
#3 lore raw

PATCH v4 2/2 selftests: net: add local ECMP rehash test

In reply to Neil Spring

Add ecmp_rehash.sh with five scenarios verifying that TCP rehash selects a different local ECMP path for IPv6:

- SYN retransmission (forward path blocked during setup)
- SYN/ACK retransmission (reverse path blocked during setup)
- Midstream RTO (forward path blocked on established connection)
- Midstream ACK rehash (reverse path blocked on established connection)
- PLB rehash (ECN-driven congestion on established connection)

---
 tools/testing/selftests/net/Makefile       |   1 +
 tools/testing/selftests/net/config         |   1 +
 tools/testing/selftests/net/ecmp_rehash.sh | 582 +++++++++++++++++++++
 3 files changed, 584 insertions(+)
 create mode 100755 tools/testing/selftests/net/ecmp_rehash.sh
tools/testing/selftests/net/Makefile +1 -0
index baa30287cf22..6ec1b24218ad 100644
@@ -26,6 +26,7 @@ TEST_PROGS := \
 	cmsg_time.sh \
 	double_udp_encap.sh \
 	drop_monitor_tests.sh \
+	ecmp_rehash.sh \
 	fcnal-ipv4.sh \
 	fcnal-ipv6.sh \
 	fcnal-other.sh \
tools/testing/selftests/net/config +1 -0
index 94d722770420..20fce6e4500b 100644
@@ -122,6 +122,7 @@ CONFIG_PSAMPLE=m
 CONFIG_RPS=y
 CONFIG_SYSFS=y
 CONFIG_TAP=m
+CONFIG_TCP_CONG_DCTCP=m
 CONFIG_TCP_MD5SIG=y
 CONFIG_TEST_BLACKHOLE_DEV=m
 CONFIG_TEST_BPF=m
tools/testing/selftests/net/ecmp_rehash.sh new file +582 -0

582 lines not on this page · raw message

Signature · 1 line
2.53.0-Meta
maintainer
#2 lore raw
In reply to Neil Spring

This is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.
---
tcp: rehash onto different local ECMP path on retransmit timeout

This commit updates ECMP path selection logic to use the socket's txhash on RTO, PLB, and spurious retransmissions by invalidating the cached dst and setting fl6->mp_hash for IPv6. It also moves txhash initialization earlier in tcp_conn_request().

Quoted 4 lines
net/ipv6/inet6_connection_sock.c +0 -0
index 37534e1168992..fc4b75de6af8e 100644

[ ... ]

net/ipv6/inet6_connection_sock.c struct dst_entry *inet6_csk_route_socket(struct sock *sk, · line 81

Quoted 10 lines
net/ipv6/inet6_connection_sock.c +3 -0
@@ -70,6 +73,9 @@ struct dst_entry *inet6_csk_route_socket(struct sock *sk,
 	fl6->saddr = np->saddr;
 	fl6->flowlabel = np->flow_label;
 	IP6_ECN_flow_xmit(sk, fl6->flowlabel);
+
+	/* >> 1 for 31-bit mp_hash range matching nhc_upper_bound. */
+	fl6->mp_hash = sk->sk_txhash >> 1;
 	fl6->flowi6_oif = sk->sk_bound_dev_if;
 	fl6->flowi6_mark = sk->sk_mark;
 	fl6->fl6_sport = inet->inet_sport;

Does this leave active IPv6 connections and syncookie-created sockets vulnerable to spontaneous ECMP path flapping upon natural route invalidation?

The patch updates inet6_sk_rebuild_header() and inet6_csk_route_socket() to use sk_txhash for ECMP path selection, but it appears to miss two critical paths:

1. tcp_v6_connect() (active open): fl6.mp_hash is 0 during ip6_dst_lookup_flow(). sk_set_txhash(sk) is called only after the route is cached.

2. cookie_v6_check() (IPv6 syncookies): fl6.mp_hash is 0 during ip6_dst_lookup_flow(), despite treq->txhash having been generated in cookie_tcp_reqsk_alloc().

In both cases, the initial dst is selected via the standard 5-tuple hash, but the socket is assigned a random txhash. If the route is later naturally invalidated (e.g., via PMTU discovery), the rebuild functions will apply sk_txhash >> 1. Because this new hash is uncorrelated with the 5-tuple, the connection could unpredictably flap to a different ECMP path, potentially breaking connections passing through stateful firewalls/NATs and causing severe packet reordering.

Signature · 1 line
pw-bot: cr