tcp: rehash onto different local ECMP path on retransmit timeout
Subsystems (7)
v11 cover tcp: rehash onto different local ECMP path on retransmit timeout
Currently sk_rethink_txhash() re-rolls the socket's txhash on RTO, PLB, and spurious-retransmission events, but the new hash is not propagated into the IPv6 ECMP path selection logic. The cached route is reused and fib6_select_path() is never re-invoked, so the connection uses the same local ECMP decision.
This series adds the two missing pieces:
1. __sk_dst_reset() alongside sk_rethink_txhash() so the cached dst is invalidated and the next transmit triggers a fresh route lookup.
2. fl6->mp_hash set from sk_txhash before each route lookup so fib6_select_path() picks a path based on the (potentially re-rolled) hash. This only applies to fib_multipath_hash_policy 0 (the default L3 policy). Policy 0 is already asymmetric for TCP: with auto_flowlabels enabled (the default), each socket gets a per-socket flowlabel, so the hash is already per-connection and unidirectional. Policies 1-3 exist for operators who need deterministic symmetric hashing (e.g., for stateful middleboxes) and are left unchanged.
Patch 1 is the kernel change; patch 2 adds selftests covering SYN rehash, SYN/ACK rehash, midstream RTO rehash, midstream ACK rehash (spurious retransmission), PLB rehash, a policy 1 negative test, a flowlabel leak regression test, two dst rebuild consistency tests (normal and syncookie) verifying that natural route invalidation does not cause unintended path changes, and a syncookie server path consistency test verifying that the SYN-ACK and post-cookie ACKs use the same ECMP nexthop.
Changes since v10: https://lore.kernel.org/netdev/[email protected]/
Patch 1:
- Fix build without CONFIG_SYN_COOKIES
- Leave IPv4 syncookie txhash unmodified (net_tx_rndhash())
- Document the IPv6 TCP policy 0 behavior change in ip-sysctl.rst
Patch 2:
- Correct runtime estimate from ~15s to ~60s
- Build DCTCP as =y instead of =m to avoid module load races
- Fix false failure of the midstream ACK test by limiting the send
buffer to avoid a closed receive window; window probes do not
cause rehash
Changes since v9: https://lore.kernel.org/netdev/[email protected]/
Patch 1:
- Split cookie_init_sequence() into pure computation and a new cookie_record_sent() helper for the side effects; call
cookie_record_sent() after route_req() succeeds so the overflow
timestamp and SYNCOOKIESSENT counter are not bumped when no
SYN-ACK is sent
Patch 2:
- Make midstream ACK rehash test more reliable by blocking the unused
path first
- Fix port overlap when ECMP_REBUILD_ROUNDS exceeds the default
Changes since v8: https://lore.kernel.org/netdev/[email protected]/
Patch 1:
- Fix REPFLOW flowlabel reflection for syncookie SYN-ACKs: pass 0 as
tw_isn to route_req() so tcp_v6_init_req() saves ireq->pktopts
Patch 2:
- Give midstream and ACK rehash attempt helpers distinct failure
messages (no TX activity vs no data on alternate path vs counter
not incrementing) instead of a single generic error
- Drop unused ns_server parameter from ecmp_dst_rebuild_check()
- Clean up server socat before break on setup failure in the dst
rebuild loop
Changes since v7: https://lore.kernel.org/netdev/[email protected]/
Patch 1:
- Remove #if IS_ENABLED(CONFIG_IPV6) guards around __sk_dst_reset()
in tcp_plb.c and tcp_timer.c (Eric Dumazet)
- Guard mp_hash in inet6_csk_route_socket() on sk_protocol == IPPROTO_TCP instead of txhash != 0, since non-TCP callers like L2TP set sk_txhash
in __ip6_datagram_connect() and should retain flow-key-based ECMP
- Use the syncookie (ISN) as txhash for both the SYN-ACK route lookup
and cookie_v6_check() socket creation, so the server's ECMP selection is consistent across the stateless SYN-ACK and the subsequent full socket. Move cookie_init_sequence() before route_req() in tcp_conn_request()
so the SYN-ACK dst is computed with the cookie-derived txhash; derive txhash from snt_isn in cookie_tcp_reqsk_init() to match
Patch 2:
- Invalidate dst via dummy route add/del instead of route replace to
avoid a transient single-nexthop state during multipath replacement
- Add syncookie server path consistency test verifying the SYN-ACK and post-cookie ACKs use the same ECMP path
- Strengthen policy 1 negative test to wait for multiple rehash attempts
and verify SYNs landed on exactly one interface
Changes since v6: https://lore.kernel.org/netdev/[email protected]/
- Guard mp_hash assignment so that non-TCP callers of inet6_csk_route_socket() fall through to rt6_multipath_hash()
(superseded in v8 by sk_protocol == IPPROTO_TCP guard)
- Initialize txhash in bpf_sk_assign_tcp_reqsk() to avoid reading uninitialized slab memory in inet6_csk_route_req()
- Check post-rebuild busywait return status to avoid silent false pass
Changes since v5: https://lore.kernel.org/netdev/[email protected]/
- Improve selftest reliability: suppress __dst_negative_advice() via tcp_retries1=255 in dst rebuild tests so a real RTO cannot trigger
an unintended rehash; add internal retry to midstream and ACK
rehash tests to tolerate probabilistic ECMP path selection; fix
midstream baseline capture to account for packets that bypass tc
filters during the prio qdisc's TCQ_F_CAN_BYPASS window
- Increase ECMP_REBUILD_ROUNDS default to 10 for reliable regression detection with 2-way ECMP; replace sleep with busywait
- Use tcp_allowed_congestion_control instead of changing the host's
default congestion control for PLB test
- Use (txhash >> 1) ?: 1 to guarantee non-zero mp_hash, since zero
falls back to rt6_multipath_hash()
Changes since v4: https://lore.kernel.org/netdev/[email protected]/
- Condition fl6->mp_hash on fib_multipath_hash_policy == 0 to preserve deterministic hash policies 1-3 (e.g., symmetric 5-tuple for policy 1)
- Set fl6->mp_hash in tcp_v6_connect() and cookie_v6_check() for
initial route lookup consistency; move sk_set_txhash() earlier
(Jakub Kicinski)
- Add policy 1 negative test; improve sysctl save/restore
- Add flowlabel leak test confirming mp_hash does not alter the
on-wire IPv6 flow label
- Add dst rebuild consistency tests (normal and syncookie) verifying
that route table changes do not cause unintended ECMP path changes
Changes since v3: https://lore.kernel.org/netdev/[email protected]/
- Use __sk_dst_reset() instead of sk_dst_reset() since the socket lock
is held in all three call sites (Eric Dumazet)
- Guard __sk_dst_reset() with sk->sk_family == AF_INET6 since IPv4 ECMP
does not use sk_txhash for path selection
- Guard __sk_dst_reset() in tcp_plb_check_rehash() with the return value
of sk_rethink_txhash()
- Move tcp_rsk(req)->txhash initialization before route_req() in tcp_conn_request() to avoid reading uninitialized memory
- Add CONFIG_TCP_CONG_DCTCP=m to selftests/net/config for PLB test
- Skip PLB test gracefully if DCTCP is not available
- Save and restore original congestion control algorithm in PLB test
- Default get_netstat_counter() to 0 when counter is not found
- Skip all tests if tcp_syn_linear_timeouts is not available
- Replace bash/pipe data sources with socat OPEN:/dev/zero for
cleaner process cleanup
- Fix shellcheck warnings
Changes since v2: https://lore.kernel.org/netdev/[email protected]/
- Retitle "ECMP" to "local ECMP" to distinguish from remote ECMP
(Neal Cardwell)
- Add fl6->mp_hash propagation in inet6_sk_rebuild_header() (af_inet6.c), covering the dst rebuild path used on established sockets
- Remove incorrect ir_iif update from tcp_check_req() in tcp_minisocks.c;
the SYN/ACK rehash is already handled by tcp_rtx_synack() re-rolling
txhash which feeds into inet6_csk_route_req()'s mp_hash
(Eric Dumazet)
- Add ACK rehash and PLB rehash selftests
- Improve selftest reliability
Changes since v1: https://lore.kernel.org/netdev/[email protected]/
- Use tcp_rsk(req)->txhash instead of jhash_1word(req->num_retrans, ...)
for ECMP path selection in inet6_csk_route_req(), making the request
socket path consistent with the established socket path (Eric Dumazet)
- Add comments explaining the >> 1 shift for 31-bit mp_hash range
- Use socat -u (unidirectional) in selftest to avoid SIGPIPE race
- Increase tcp_syn_retries and tcp_syn_linear_timeouts to 25 for
better rehash coverage
Neil Spring (2):
tcp: rehash onto different local ECMP path on retransmit timeout
selftests: net: add local ECMP rehash test
Documentation/networking/ip-sysctl.rst | 5 +- include/net/tcp.h | 20 +- net/core/filter.c | 1 + net/ipv4/syncookies.c | 11 +- net/ipv4/tcp_input.c | 15 +- net/ipv4/tcp_plb.c | 5 +- net/ipv4/tcp_timer.c | 2 + net/ipv6/af_inet6.c | 3 + net/ipv6/inet6_connection_sock.c | 8 + net/ipv6/syncookies.c | 4 + net/ipv6/tcp_ipv6.c | 13 +- tools/testing/selftests/net/Makefile | 1 + tools/testing/selftests/net/config | 1 + tools/testing/selftests/net/ecmp_rehash.sh | 1066 ++++++++++++++++++++ 14 files changed, 1140 insertions(+), 15 deletions(-) create mode 100755 tools/testing/selftests/net/ecmp_rehash.sh
base-commit: c1e5127b577c6b88fa48e532616932ae978528d5 --
2.53.0-Meta
PATCH v11 1/2 tcp: rehash onto different local ECMP path on retransmit timeout
Currently sk_rethink_txhash() re-rolls the socket's txhash on RTO, PLB, and spurious-retransmission events, but the cached route is reused and the new hash is not propagated into the ECMP path selection logic. Two changes are needed to make rehash select a different local ECMP path:
1. Add __sk_dst_reset() alongside sk_rethink_txhash() in tcp_write_timeout(), tcp_rcv_spurious_retrans(), and tcp_plb_check_rehash() so the cached dst is invalidated and the next transmit triggers a fresh route lookup.
2. Set fl6->mp_hash from sk_txhash (or tcp_rsk(req)->txhash for SYN/ACK retransmits and syncookies) in tcp_v6_connect(), inet6_sk_rebuild_header(), inet6_csk_route_req(), inet6_csk_route_socket(), and cookie_v6_check() so fib6_select_path() picks a path based on the new hash.
The mp_hash override only applies to fib_multipath_hash_policy 0 (the default L3 policy). Policy 0 is already asymmetric for TCP: with auto_flowlabels enabled (the default), each socket gets a per-socket flowlabel derived from sk_txhash, so the hash is already per-connection and unidirectional. Overriding with txhash is consistent with that existing behavior while making rehash effective. Policies 1-3 exist for operators who need deterministic symmetric hashing (e.g., for stateful middleboxes or same-path debugging), and are left unchanged.
The mp_hash assignment in inet6_csk_route_socket() is guarded by sk_protocol == IPPROTO_TCP so that non-TCP callers (e.g., L2TP via inet6_csk_xmit) fall through to rt6_multipath_hash() and retain their existing flow-key-based ECMP behavior. The expression uses (txhash >> 1) ?: 1 so that the rare txhash == 1 still produces a valid non-zero mp_hash.
Setting mp_hash explicitly is necessary because the default ECMP hash derives from fl6->flowlabel via np->flow_label, which is not updated from sk_txhash (REPFLOW is off by default). ip6_make_flowlabel() cannot help either, as it runs after the route lookup.
sk_set_txhash() is moved before ip6_dst_lookup_flow() in tcp_v6_connect() so the initial ECMP path is selected by the same txhash that subsequent route rebuilds will use. This avoids unintended path changes when the cached dst is naturally invalidated (e.g., by PMTU discovery or route changes).
The dst reset in tcp_write_timeout() and tcp_plb_check_rehash() is guarded by sk->sk_family == AF_INET6 since IPv4 ECMP does not currently use sk_txhash for path selection. For IPv4-mapped IPv6 sockets this produces a redundant dst reset on a cold path (RTO/PLB); the subsequent IPv4 route lookup returns the same result.
For syncookies, cookie_init_sequence() computes the cookie value before route_req() and sets txhash so the SYN-ACK selects the same ECMP path that cookie_v6_check() will use when the full socket is created. cookie_tcp_reqsk_init() derives txhash from the cookie for IPv6 sockets so the ECMP path matches the SYN-ACK; IPv4 sockets retain net_tx_rndhash() since IPv4 ECMP does not use sk_txhash and the v4 cookie has mssind bits that would bias queue distribution.
cookie_init_sequence() is split from the former version that also called tcp_synq_overflow() and incremented SYNCOOKIESSENT; those side effects are now in cookie_record_sent(), called after route_req() succeeds so they are not triggered when no SYN-ACK is sent. cookie_record_sent() is guarded by CONFIG_SYN_COOKIES to match the guard on tcp_synq_overflow(). route_req() receives 0 as tw_isn for the syncookie path so that tcp_v6_init_req() still saves ireq->pktopts for REPFLOW flowlabel reflection and IPv6 cmsg options. The ecn_ok clear for syncookies without timestamps stays after tcp_ecn_create_request() so it takes precedence.
bpf_sk_assign_tcp_reqsk() is updated to initialize txhash via net_tx_rndhash() to avoid reading uninitialized slab memory.
--- Documentation/networking/ip-sysctl.rst | 5 ++++- include/net/tcp.h | 20 ++++++++++++++------ net/core/filter.c | 1 + net/ipv4/syncookies.c | 11 ++++++++++- net/ipv4/tcp_input.c | 15 +++++++++++---- net/ipv4/tcp_plb.c | 5 ++++- net/ipv4/tcp_timer.c | 2 ++ net/ipv6/af_inet6.c | 3 +++ net/ipv6/inet6_connection_sock.c | 8 ++++++++ net/ipv6/syncookies.c | 4 ++++ net/ipv6/tcp_ipv6.c | 13 +++++++++++-- 11 files changed, 72 insertions(+), 15 deletions(-)
Documentation/networking/ip-sysctl.rst +4 -1
@@ -2444,7 +2444,10 @@ fib_multipath_hash_policy - INTEGER Possible values: - - 0 - Layer 3 (source and destination addresses plus flow label) + - 0 - Layer 3 (source and destination addresses plus flow label). + For IPv6 TCP, each connection selects its own ECMP path, + which may change after a retransmission timeout to recover + from path failure. - 1 - Layer 4 (standard 5-tuple) - 2 - Layer 3 or inner Layer 3 if present - 3 - Custom multipath hash. Fields used for multipath hash calculation
include/net/tcp.h +14 -6
@@ -2540,22 +2540,30 @@ extern const struct tcp_request_sock_ops tcp_request_sock_ipv6_ops;#ifdef CONFIG_SYN_COOKIES static inline __u32 cookie_init_sequence (const struct tcp_request_sock_ops *ops, -const struct sock *sk,struct sk_buff *skb, -__u16 *mss) +struct sk_buff *skb,__u16 *mss) { -tcp_synq_overflow (sk); -__NET_INC_STATS (sock_net (sk),LINUX_MIB_SYNCOOKIESSENT );return ops->cookie_init_seq (skb, mss); }#else static inline __u32 cookie_init_sequence (const struct tcp_request_sock_ops *ops, -const struct sock *sk,struct sk_buff *skb, -__u16 *mss) +struct sk_buff *skb,__u16 *mss) {return 0 ; }#endif +#ifdef CONFIG_SYN_COOKIES +static inline void cookie_record_sent (const struct sock *sk) +{ +tcp_synq_overflow (sk); +__NET_INC_STATS (sock_net (sk),LINUX_MIB_SYNCOOKIESSENT ); +} +#else +static inline void cookie_record_sent (const struct sock *sk) +{ +} +#endif + struct tcp_key { union { struct {
net/core/filter.c +1 -0
@@ -12301,6 +12301,7 @@ __bpf_kfunc int bpf_sk_assign_tcp_reqsk(struct __sk_buff *s, struct sock *sk, treq->req_usec_ts = !!attrs->usec_ts_ok; treq->ts_off = tsoff; + treq->txhash =net_tx_rndhash ();skb_orphan (skb); skb->sk =req_to_sk (req);
net/ipv4/syncookies.c +10 -1
@@ -280,9 +280,18 @@ static int cookie_tcp_reqsk_init(struct sock *sk, struct sk_buff *skb, treq->snt_synack =0 ; treq->snt_tsval_first =0 ; treq->tfo_listener = false; - treq->txhash =net_tx_rndhash (); treq->rcv_isn =ntohl (th->seq) -1 ; treq->snt_isn =ntohl (th->ack_seq) -1 ; +if (sk->sk_family ==AF_INET6 ) { +/* Use the cookie as txhash so the ECMP path matches +* the SYN-ACK, where txhash was also set to the +* cookie. The original request socket (and its +* txhash) was freed after sending the SYN-ACK. +*/ + treq->txhash = treq->snt_isn; + }else { + treq->txhash =net_tx_rndhash (); + } treq->syn_tos =TCP_SKB_CB (skb)->ip_dsfield;#if IS_ENABLED (CONFIG_MPTCP )
net/ipv4/tcp_input.c +11 -4
@@ -5020,8 +5020,10 @@ static void tcp_rcv_spurious_retrans(struct sock *sk, skb->protocol == htons(ETH_P_IPV6) && (tcp_sk(sk)->inet_conn.icsk_ack.lrcv_flowlabel != ntohl(ip6_flowlabel(ipv6_hdr(skb)))) && - sk_rethink_txhash(sk)) + sk_rethink_txhash(sk)) {NET_INC_STATS (sock_net (sk),LINUX_MIB_TCPDUPLICATEDATAREHASH ); +__sk_dst_reset (sk); + }/* Save last flowlabel after a spurious retrans. */ tcp_save_lrcv_flowlabel (sk, skb);
@@ -7636,6 +7638,7 @@ int tcp_conn_request(struct request_sock_ops *rsk_ops,tcp_rsk (req)->af_specific = af_ops;tcp_rsk (req)->ts_off =0 ;tcp_rsk (req)->req_usec_ts = false; +tcp_rsk (req)->txhash =net_tx_rndhash ();#if IS_ENABLED (CONFIG_MPTCP )tcp_rsk (req)->is_mptcp =0 ;#endif
@@ -7659,7 +7662,12 @@ int tcp_conn_request(struct request_sock_ops *rsk_ops,/* Note: tcp_v6_init_req() might override ir_iif for link locals */ inet_rsk (req)->ir_iif =inet_request_bound_dev_if (sk, skb); - dst = af_ops->route_req (sk, skb, &fl, req, isn); +if (want_cookie) { + isn =cookie_init_sequence (af_ops, skb, &req->mss); +tcp_rsk (req)->txhash = isn; + } + + dst = af_ops->route_req (sk, skb, &fl, req, want_cookie ?0 : isn);if (!dst) goto drop_and_free;
@@ -7699,7 +7707,7 @@ int tcp_conn_request(struct request_sock_ops *rsk_ops,tcp_ecn_create_request (req, skb, sk, dst);if (want_cookie) { - isn =cookie_init_sequence (af_ops, sk, skb, &req->mss); +cookie_record_sent (sk);if (!tmp_opt.tstamp_ok)inet_rsk (req)->ecn_ok =0 ; }
@@ -7717,7 +7725,6 @@ int tcp_conn_request(struct request_sock_ops *rsk_ops, }#endif tcp_rsk (req)->snt_isn = isn; -tcp_rsk (req)->txhash =net_tx_rndhash ();tcp_rsk (req)->syn_tos =TCP_SKB_CB (skb)->ip_dsfield;tcp_openreq_init_rwin (req, sk, dst);sk_rx_queue_set (req_to_sk (req), skb);
net/ipv4/tcp_plb.c +4 -1
@@ -78,7 +78,10 @@ void tcp_plb_check_rehash(struct sock *sk, struct tcp_plb_state *plb)if (plb->pause_until)return ; -sk_rethink_txhash (sk); +if (sk_rethink_txhash (sk)) { +if (sk->sk_family ==AF_INET6 ) +__sk_dst_reset (sk); + } plb->consec_cong_rounds =0 ;WRITE_ONCE (tcp_sk (sk)->plb_rehash,tcp_sk (sk)->plb_rehash +1 );NET_INC_STATS (sock_net (sk),LINUX_MIB_TCPPLBREHASH );
net/ipv4/tcp_timer.c +2 -0
@@ -300,6 +300,8 @@ static int tcp_write_timeout(struct sock *sk)if (sk_rethink_txhash (sk)) {WRITE_ONCE (tp->timeout_rehash, tp->timeout_rehash +1 );__NET_INC_STATS (sock_net (sk),LINUX_MIB_TCPTIMEOUTREHASH ); +if (sk->sk_family ==AF_INET6 ) +__sk_dst_reset (sk); }return 0 ;
net/ipv6/af_inet6.c +3 -0
@@ -823,6 +823,9 @@ int inet6_sk_rebuild_header(struct sock *sk) fl6->flowi6_uid =sk_uid (sk);security_sk_classify_flow (sk,flowi6_to_flowi_common (fl6)); +if (ip6_multipath_hash_policy (sock_net (sk)) ==0 && sk->sk_txhash) + fl6->mp_hash = (sk->sk_txhash >>1 ) ?:1 ; +rcu_read_lock (); final_p =fl6_update_dst (fl6,rcu_dereference (np->opt), &np->final);rcu_read_unlock ();
net/ipv6/inet6_connection_sock.c +8 -0
@@ -48,6 +48,10 @@ struct dst_entry *inet6_csk_route_req(const struct sock *sk, fl6->flowi6_uid =sk_uid (sk);security_req_classify_flow (req,flowi6_to_flowi_common (fl6)); +if (ip6_multipath_hash_policy (sock_net (sk)) ==0 && +tcp_rsk (req)->txhash) + fl6->mp_hash = (tcp_rsk (req)->txhash >>1 ) ?:1 ; +if (!dst) { dst =ip6_dst_lookup_flow (sock_net (sk), sk, fl6, final_p);if (IS_ERR (dst))
@@ -70,6 +74,10 @@ struct dst_entry *inet6_csk_route_socket(struct sock *sk, fl6->saddr = np->saddr; fl6->flowlabel = np->flow_label;IP6_ECN_flow_xmit (sk, fl6->flowlabel); + +if (sk->sk_protocol ==IPPROTO_TCP && +ip6_multipath_hash_policy (sock_net (sk)) ==0 && sk->sk_txhash) + fl6->mp_hash = (sk->sk_txhash >>1 ) ?:1 ; fl6->flowi6_oif = sk->sk_bound_dev_if; fl6->flowi6_mark = sk->sk_mark; fl6->fl6_sport = inet->inet_sport;
net/ipv6/syncookies.c +4 -0
@@ -245,6 +245,10 @@ struct sock *cookie_v6_check(struct sock *sk, struct sk_buff *skb) fl6.flowi6_uid =sk_uid (sk);security_req_classify_flow (req,flowi6_to_flowi_common (&fl6)); +if (ip6_multipath_hash_policy (net) ==0 && +tcp_rsk (req)->txhash) + fl6.mp_hash = (tcp_rsk (req)->txhash >>1 ) ?:1 ; + dst =ip6_dst_lookup_flow (net, sk, &fl6, final_p);if (IS_ERR (dst)) {SKB_DR_SET (reason,IP_OUTNOROUTES );
net/ipv6/tcp_ipv6.c +11 -2
@@ -258,6 +258,8 @@ static int tcp_v6_connect(struct sock *sk, struct sockaddr_unsized *uaddr,if (!ipv6_addr_any (&sk->sk_v6_rcv_saddr)) saddr = &sk->sk_v6_rcv_saddr; +sk_set_txhash (sk); + fl6->flowi6_proto =IPPROTO_TCP ; fl6->daddr = sk->sk_v6_daddr; fl6->saddr = saddr ? *saddr : np->saddr;
@@ -275,6 +277,15 @@ static int tcp_v6_connect(struct sock *sk, struct sockaddr_unsized *uaddr,security_sk_classify_flow (sk,flowi6_to_flowi_common (fl6)); +/* Non-zero mp_hash bypasses rt6_multipath_hash() in +* fib6_select_path(), letting txhash control ECMP path +* selection so that sk_rethink_txhash() rehashes onto a +* different path. Policies 1-3 derive a deterministic +* hash from the flow keys and must not be overridden. +*/ +if (ip6_multipath_hash_policy (net) ==0 && sk->sk_txhash) + fl6->mp_hash = (sk->sk_txhash >>1 ) ?:1 ; + dst =ip6_dst_lookup_flow (net, sk, fl6, final_p);if (IS_ERR (dst)) { err =PTR_ERR (dst);
@@ -313,8 +324,6 @@ static int tcp_v6_connect(struct sock *sk, struct sockaddr_unsized *uaddr,if (err) goto late_failure; -sk_set_txhash (sk); -if (likely (!tp->repair)) {union tcp_seq_and_ts_off st;
Signature · 1 line
2.53.0-Meta
[email protected]
AI reviewShould this txhash assignment be guarded so it only applies to IPv6?
Quoted 4 lines
net/ipv4/tcp_input.c +0
-0
[ ... ]
Quoted 5 lines
net/ipv4/tcp_input.c +1
-0@@ -7633,6 +7635,7 @@ int tcp_conn_request(struct request_sock_ops *rsk_ops,tcp_rsk (req)->af_specific = af_ops;tcp_rsk (req)->ts_off =0 ;tcp_rsk (req)->req_usec_ts = false; +tcp_rsk (req)->txhash =net_tx_rndhash ();
[ ... ]
net/ipv4/tcp_input.c int tcp_conn_request(struct request_sock_ops *rsk_ops, · line 7665
Quoted 9 lines
net/ipv4/tcp_input.c +4
-1@@ -7656,7 +7659,12 @@ int tcp_conn_request(struct request_sock_ops *rsk_ops,/* Note: tcp_v6_init_req() might override ir_iif for link locals */ inet_rsk (req)->ir_iif =inet_request_bound_dev_if (sk, skb); - dst = af_ops->route_req (sk, skb, &fl, req, isn); +if (want_cookie) { + isn =cookie_init_sequence (af_ops, skb, &req->mss); +tcp_rsk (req)->txhash = isn; + }
Should this txhash assignment be guarded so it only applies to IPv6?
For the want_cookie path here, isn is the SYN cookie. For IPv4 that cookie comes from __cookie_v4_init_sequence() -> secure_tcp_syn_cookie(), whose low bits encode the mss index (mssind). tcp_make_synack() then does:
net/ipv4/tcp_output.c:tcp_make_synack()
skb_set_hash(skb, READ_ONCE(tcp_rsk(req)->txhash), PKT_HASH_TYPE_L4);
so for IPv4 the SYN-ACK skb->hash becomes the mssind-biased cookie value instead of the uniformly-random net_tx_rndhash() the old code assigned below. skb->hash drives TX queue selection (netdev_pick_tx/XPS), so under an IPv4 SYN flood the SYN-ACK transmit-queue distribution would be biased by mssind.
This seems to be the same bias the commit message says the v4 path should avoid:
IPv4 sockets retain net_tx_rndhash() since IPv4 ECMP does not use sk_txhash and the v4 cookie has mssind bits that would bias queue distribution.
That net_tx_rndhash() guard was applied in cookie_tcp_reqsk_init() (the full-socket path), but the parallel assignment here in tcp_conn_request() is not guarded by sk_family / AF_INET6. For IPv6 the cookie-as-txhash looks intentional (it keeps the SYN-ACK on the same ECMP path that cookie_v6_check() later picks), so only the IPv4 case appears affected.
Quoted 5 lines
net/ipv4/tcp_input.c +2
-0+ + dst = af_ops->route_req (sk, skb, &fl, req, want_cookie ?0 : isn);if (!dst) goto drop_and_free;
[ ... ]
net/ipv4/tcp_input.c int tcp_conn_request(struct request_sock_ops *rsk_ops, · line 7725
Quoted 6 lines
net/ipv4/tcp_input.c +0
-1@@ -7714,7 +7722,6 @@ int tcp_conn_request(struct request_sock_ops *rsk_ops, }#endif tcp_rsk (req)->snt_isn = isn; -tcp_rsk (req)->txhash =net_tx_rndhash ();tcp_rsk (req)->syn_tos =TCP_SKB_CB (skb)->ip_dsfield;
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/26839485409
[email protected]
AI reviewThank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:
New issues:
- [High] Missing mp_hash initialization in tcp_v6_send_response() causes TCP control packets (RST, TIME_WAIT ACKs) to use a different ECMP path than data packets.
- [High] Unconditional txhash randomization in BPF syncookies breaks symmetric ECMP routing for IPv6.
- [Medium] IPv4-mapped IPv6 connections incorrectly apply IPv6-specific cookie txhash logic, biasing queue distribution.
Pre-existing issues:
- [Critical] bpf_sk_assign_tcp_reqsk() partially initializes tcp_request_sock, leading to uninitialized slab memory usage and wild pointer dereferences. --
commit e48c642c65c933dd8659dee905bbd3b1d86dd5a3
Author: Neil Spring <[email protected]>
tcp: rehash onto different local ECMP path on retransmit timeout
This patch updates TCP to select a different local ECMP path upon retransmit timeout, path MTU discovery, or spurious retransmissions by propagating the socket's txhash into the IPv6 multipath hash logic. It ensures the cached destination is invalidated so the next transmit triggers a fresh route lookup using the new hash. Syncookies are also updated to derive txhash from the cookie for IPv6 to maintain ECMP path consistency.
net/core/filter.c __bpf_kfunc int bpf_sk_assign_tcp_reqsk(struct __sk_buff *s, struct sock *sk, · line 12305
Quoted 9 lines
net/core/filter.c +1
-0@@ -12301,6 +12301,7 @@ __bpf_kfunc int bpf_sk_assign_tcp_reqsk(struct __sk_buff *s, struct sock *sk, treq->req_usec_ts = !!attrs->usec_ts_ok; treq->ts_off = tsoff; + treq->txhash =net_tx_rndhash ();
[Severity: Critical]
This is a pre-existing issue, but since inet_reqsk_alloc() allocates the request socket via a slab cache without zeroing memory, do the other fields of tcp_request_sock (such as used_tcp_ao and af_specific) remain uninitialized garbage?
If so, when tcp_create_openreq_child() is called later and dereferences treq->af_specific->md5_lookup, could this result in a kernel panic from evaluating the uninitialized pointers?
[Severity: High]
Does initializing txhash to a random value here break symmetric ECMP routing for IPv6 BPF syncookies?
Since standard IPv6 syncookies set txhash to the cookie to maintain path symmetry with the SYN-ACK, would this random initialization cause data packets to route over a completely different ECMP path than the SYN/SYN-ACK exchange?
Quoted 1 line
net/core/filter.c +0
-0skb_orphan (skb);
[ ... ]
Quoted 10 lines
net/ipv4/syncookies.c +0
-1@@ -280,9 +280,18 @@ static int cookie_tcp_reqsk_init(struct sock *sk, struct sk_buff *skb, treq->snt_synack =0 ; treq->snt_tsval_first =0 ; treq->tfo_listener = false; - treq->txhash =net_tx_rndhash (); treq->rcv_isn =ntohl (th->seq) -1 ; treq->snt_isn =ntohl (th->ack_seq) -1 ;
net/ipv4/syncookies.c static int cookie_tcp_reqsk_init(struct sock *sk, struct sk_buff *skb, · line 294
Quoted 10 lines
net/ipv4/syncookies.c +10
-0+if (sk->sk_family ==AF_INET6 ) { +/* Use the cookie as txhash so the ECMP path matches +* the SYN-ACK, where txhash was also set to the +* cookie. The original request socket (and its +* txhash) was freed after sending the SYN-ACK. +*/ + treq->txhash = treq->snt_isn; + }else { + treq->txhash =net_tx_rndhash (); + }
[Severity: Medium]
Can dual-stack listeners (IPv4-mapped IPv6) be negatively affected by this check?
Since sk->sk_family is AF_INET6 for these listeners even when accepting IPv4 connections, would IPv4 connections incorrectly execute treq->txhash = treq->snt_isn?
As noted in the commit message, the IPv4 cookie contains mssind bits that could bias queue distribution if used as a hash.
Quoted 1 line
net/ipv4/syncookies.c +0
-0treq->syn_tos =TCP_SKB_CB (skb)->ip_dsfield;
[ ... ]
Quoted 6 lines
net/ipv6/tcp_ipv6.c +0
-0@@ -275,6 +277,15 @@ static int tcp_v6_connect(struct sock *sk, struct sockaddr_unsized *uaddr,security_sk_classify_flow (sk,flowi6_to_flowi_common (fl6));
net/ipv6/tcp_ipv6.c static int tcp_v6_connect(struct sock *sk, struct sockaddr_unsized *uaddr, · line 288
Quoted 10 lines
net/ipv6/tcp_ipv6.c +9
-0+/* Non-zero mp_hash bypasses rt6_multipath_hash() in +* fib6_select_path(), letting txhash control ECMP path +* selection so that sk_rethink_txhash() rehashes onto a +* different path. Policies 1-3 derive a deterministic +* hash from the flow keys and must not be overridden. +*/ +if (ip6_multipath_hash_policy (net) ==0 && sk->sk_txhash) + fl6->mp_hash = (sk->sk_txhash >>1 ) ?:1 ; +
[Severity: High]
Does the missing mp_hash initialization in tcp_v6_send_response() cause TCP control packets (like RST and TIME_WAIT ACKs) to use a different ECMP path than data packets?
If tcp_v6_send_response() doesn't set fl6.mp_hash, it seems fib6_select_path() will fall back to rt6_multipath_hash(). If the data path was rehashed to recover from a failure, wouldn't the RSTs stay on the broken default path and be dropped?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
PATCH v11 2/2 selftests: net: add local ECMP rehash test
In reply to Neil SpringAdd ecmp_rehash.sh with ten scenarios verifying that TCP rehash selects a different local ECMP path for IPv6:
- SYN retransmission (forward path blocked during setup)
- SYN/ACK retransmission (reverse path blocked during setup)
- Midstream RTO (forward path blocked on established connection)
- Midstream ACK rehash (reverse path blocked on established connection)
- PLB rehash (ECN-driven congestion on established connection)
- Hash policy 1 negative test (rehash attempted but path unchanged)
- No flowlabel leak (client mp_hash does not alter on-wire flowlabel)
- Dst rebuild consistency (dst invalidation does not change path)
- Dst rebuild consistency with syncookies (server socket created via cookie_v6_check instead of the normal three-way handshake)
- Syncookie server path consistency (SYN-ACK and post-cookie ACKs use the same ECMP path)
The policy 1 test verifies that fib_multipath_hash_policy=1 computes a deterministic 5-tuple hash, so txhash re-rolls do not change the ECMP path while TcpTimeoutRehash still increments.
The flowlabel leak test sets auto_flowlabels=0 on the client and installs tc filters on client egress that drop TCP packets with nonzero flowlabel, confirming that the client's fl6->mp_hash does not leak into the on-wire IPv6 flow label.
The dst rebuild tests stream data, invalidate the cached dst by adding and removing a dummy route (bumping the fib6_node sernum), and verify that traffic stays on the same path. The sernum change causes ip6_dst_check() to fail on the next transmit, triggering a fresh route lookup via inet6_csk_route_socket(). ECMP_REBUILD_ROUNDS=10 repeats the check to reduce the probability of a buggy kernel passing by chance with 2-way ECMP.
The syncookie server path consistency test verifies that the server's SYN-ACK and subsequent ACKs use the same ECMP path. With syncookies, the request socket is freed after the SYN-ACK, so cookie_tcp_reqsk_init() must derive the same txhash (from the cookie) that was used for the SYN-ACK's route lookup.
--- tools/testing/selftests/net/Makefile | 1 + tools/testing/selftests/net/config | 1 + tools/testing/selftests/net/ecmp_rehash.sh | 1066 ++++++++++++++++++++ 3 files changed, 1068 insertions(+) create mode 100755 tools/testing/selftests/net/ecmp_rehash.sh
tools/testing/selftests/net/Makefile +1 -0
@@ -26,6 +26,7 @@ TEST_PROGS := \ cmsg_time.sh \ double_udp_encap.sh \ drop_monitor_tests.sh \ + ecmp_rehash.sh \ fcnal-ipv4.sh \ fcnal-ipv6.sh \ fcnal-other.sh \
tools/testing/selftests/net/config +1 -0
@@ -122,6 +122,7 @@ CONFIG_PSAMPLE=m CONFIG_RPS=y CONFIG_SYSFS=y CONFIG_TAP=m +CONFIG_TCP_CONG_DCTCP=y CONFIG_TCP_MD5SIG=y CONFIG_TEST_BLACKHOLE_DEV=m CONFIG_TEST_BPF=m
tools/testing/selftests/net/ecmp_rehash.sh new file +1066 -0
1,066 lines not on this page · raw message
Signature · 1 line
2.53.0-Meta
[email protected]
AI reviewRe: [PATCH v11 2/2] selftests: net: add local ECMP rehash test
These per-device tc filter counters match all TCP IPv6 traffic on the device, with no isolation by port.
Quoted 6 lines
tools/testing/selftests/net/ecmp_rehash.sh new file +0
-0@@ -0,0 +1,1066 @@
[ ... ]
tools/testing/selftests/net/ecmp_rehash.sh line 59
Quoted 11 lines
tools/testing/selftests/net/ecmp_rehash.sh new file +11
-0+tc_filter_pkt_count() +{ + local ns=$1; shift + local dev=$1; shift + + ip netns exec "$ns" tc -s filter show dev "$dev" parent 1: 2>/dev/null | + awk '/Sent .* pkt/ { + for (i=1; i<=NF; i++) + if ($i == "pkt") { print $(i-1); exit } + }' +}
These per-device tc filter counters match all TCP IPv6 traffic on the device, with no isolation by port.
[ ... ]
Quoted 2 lines
tools/testing/selftests/net/ecmp_rehash.sh new file +2
-0+test_ecmp_syncookie_path_consistency() +{
[ ... ]
tools/testing/selftests/net/ecmp_rehash.sh line 1017
Quoted 13 lines
tools/testing/selftests/net/ecmp_rehash.sh new file +13
-0+ for r in $(seq 1 "$ECMP_REBUILD_ROUNDS"); do + local port=$((PORT + 3 * ECMP_REBUILD_ROUNDS + r)) + + ip netns exec "$NS2" socat -u \ + "TCP6-LISTEN:$port,bind=[fd00:ff::2],reuseaddr" \ + - >/dev/null & + local server_pid=$! + + wait_local_port_listen "$NS2" "$port" tcp + + local srv_base0 srv_base1 + srv_base0=$(tc_filter_pkt_count "$NS2" veth0b) + srv_base1=$(tc_filter_pkt_count "$NS2" veth1b)
The baseline is snapshotted here, after the new listener starts, which handles leakage that arrives before this point.
[ ... ]
Quoted 5 lines
tools/testing/selftests/net/ecmp_rehash.sh new file +5
-0+ local srv_tcp0 srv_tcp1 + srv_tcp0=$(tc_filter_pkt_count "$NS2" veth0b) + srv_tcp1=$(tc_filter_pkt_count "$NS2" veth1b) + local srv_delta0=$(( ${srv_tcp0:-0} - ${srv_base0:-0} )) + local srv_delta1=$(( ${srv_tcp1:-0} - ${srv_base1:-0} ))
tools/testing/selftests/net/ecmp_rehash.sh line 1052
Quoted 10 lines
tools/testing/selftests/net/ecmp_rehash.sh new file +10
-0+ + if [ "$srv_delta0" -gt 0 ] && [ "$srv_delta1" -gt 0 ]; then + path_splits=$((path_splits + 1)) + fi + + kill "$client_pid" 2>/dev/null + wait "$client_pid" 2>/dev/null + kill "$server_pid" 2>/dev/null + wait "$server_pid" 2>/dev/null + done
The client and server from each round are killed at the end of the round, but the loop moves straight on to the next round.
Can asynchronous teardown traffic (FIN/RST/ACK) from the killed client or server of a previous round still be in flight after srv_base0 and srv_base1 are snapshotted for the next round? Since the counters match all TCP IPv6 traffic on the device rather than the current round's port, would a late teardown packet on one path plus the new round's traffic on the other path increment both srv_delta0 and srv_delta1, satisfying the srv_delta0 > 0 && srv_delta1 > 0 check and recording a spurious split?
Would a short quiescence wait or filtering the tc counters by the current round's port make this check more robust against late packets?
tools/testing/selftests/net/ecmp_rehash.sh line 1056
Quoted 3 lines
tools/testing/selftests/net/ecmp_rehash.sh new file +3
-0+ if [ "$path_splits" -gt 0 ]; then + check_err 1 "$path_splits/$ECMP_REBUILD_ROUNDS had split server path" + fi
The same pattern appears in ecmp_dst_rebuild_loop(), which also relies on these global per-device counters across rounds.
This was raised by an automated review on an earlier version (v9, https://lore.kernel.org/r/[email protected])
and the counter structure is unchanged in the current version. It affects test reliability rather than kernel correctness.
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/26839485409
Re: [PATCH v11 2/2] selftests: net: add local ECMP rehash test
Quoted 79 lines · On Tue, Jun 2, 2026 at 11:43 AM <[email protected]> wrote:
Quoted 8 lines
tools/testing/selftests/net/ecmp_rehash.sh new file +0
-0@@ -0,0 +1,1066 @@[ ... ]
tools/testing/selftests/net/ecmp_rehash.sh line 59
Quoted 11 lines
tools/testing/selftests/net/ecmp_rehash.sh new file +11
-0+tc_filter_pkt_count() +{ + local ns=$1; shift + local dev=$1; shift + + ip netns exec "$ns" tc -s filter show dev "$dev" parent 1: 2>/dev/null | + awk '/Sent .* pkt/ { + for (i=1; i<=NF; i++) + if ($i == "pkt") { print $(i-1); exit } + }' +}These per-device tc filter counters match all TCP IPv6 traffic on the device, with no isolation by port.
[ ... ]
Quoted 2 lines
tools/testing/selftests/net/ecmp_rehash.sh new file +2
-0+test_ecmp_syncookie_path_consistency() +{[ ... ]
tools/testing/selftests/net/ecmp_rehash.sh line 1017
Quoted 13 lines
tools/testing/selftests/net/ecmp_rehash.sh new file +13
-0+ for r in $(seq 1 "$ECMP_REBUILD_ROUNDS"); do + local port=$((PORT + 3 * ECMP_REBUILD_ROUNDS + r)) + + ip netns exec "$NS2" socat -u \ + "TCP6-LISTEN:$port,bind=[fd00:ff::2],reuseaddr" \ + - >/dev/null & + local server_pid=$! + + wait_local_port_listen "$NS2" "$port" tcp + + local srv_base0 srv_base1 + srv_base0=$(tc_filter_pkt_count "$NS2" veth0b) + srv_base1=$(tc_filter_pkt_count "$NS2" veth1b)The baseline is snapshotted here, after the new listener starts, which handles leakage that arrives before this point.
[ ... ]
Quoted 5 lines
tools/testing/selftests/net/ecmp_rehash.sh new file +5
-0+ local srv_tcp0 srv_tcp1 + srv_tcp0=$(tc_filter_pkt_count "$NS2" veth0b) + srv_tcp1=$(tc_filter_pkt_count "$NS2" veth1b) + local srv_delta0=$(( ${srv_tcp0:-0} - ${srv_base0:-0} )) + local srv_delta1=$(( ${srv_tcp1:-0} - ${srv_base1:-0} ))tools/testing/selftests/net/ecmp_rehash.sh line 1052
Quoted 10 lines
tools/testing/selftests/net/ecmp_rehash.sh new file +10
-0+ + if [ "$srv_delta0" -gt 0 ] && [ "$srv_delta1" -gt 0 ]; then + path_splits=$((path_splits + 1)) + fi + + kill "$client_pid" 2>/dev/null + wait "$client_pid" 2>/dev/null + kill "$server_pid" 2>/dev/null + wait "$server_pid" 2>/dev/null + doneThe client and server from each round are killed at the end of the round, but the loop moves straight on to the next round.
Can asynchronous teardown traffic (FIN/RST/ACK) from the killed client or server of a previous round still be in flight after
srv_base0andsrv_base1are snapshotted for the next round? Since the counters match all TCP IPv6 traffic on the device rather than the current round's port, would a late teardown packet on one path plus the new round's traffic on the other path increment bothsrv_delta0andsrv_delta1, satisfying thesrv_delta0> 0 &&srv_delta1> 0 check and recording a spurious split?
Quoted 2 lines
Would a short quiescence wait or filtering the tc counters by the current round's port make this check more robust against late packets?
Yes, thanks; fixed locally and will appear in v12. Each round now waits for all active TCP sockets on the port to close before proceeding to the next round, preventing late FIN/RST packets from polluting the next round's tc filter baseline.
Quoted 6 lines
tools/testing/selftests/net/ecmp_rehash.sh line 1056
Quoted 3 lines
tools/testing/selftests/net/ecmp_rehash.sh new file +3
-0+ if [ "$path_splits" -gt 0 ]; then + check_err 1 "$path_splits/$ECMP_REBUILD_ROUNDS had split server path" + fiThe same pattern appears in
ecmp_dst_rebuild_loop(), which also relies on these global per-device counters across rounds.
This was less of an issue since ecmp_dst_rebuild_loop had a threshold number of packets that a stray connection termination packet would not reach, but the same wait doesn't hurt.
Quoted 4 lines
This was raised by an automated review on an earlier version (v9, https://lore.kernel.org/r/[email protected] ) and the counter structure is unchanged in the current version. It affects test reliability rather than kernel correctness.
Apologies, my error.
-neil
Quoted 7 lines
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.mdCI run summary: https://github.com/kernel-patches/bpf/actions/runs/26839485409