inet: fix lockless ehash walk races
Subsystems (6)
v5 cover inet: fix lockless ehash walk races
Fix two issues in lockless ehash walks. Reject unhashed sockets after taking a reference in established lookups, and restore the acquire/release ordering around refcount publication and acquisition.
Changes in v5:
- 2/2: add acquire ordering to bpf_sk_assign() for BTF socket pointers (Kuniyuki)
- v4: https://lore.kernel.org/r/[email protected]
Changes in v4:
- 2/2: add acquire ordering to inet_twsk_purge() and update the changelog (Sashiko)
- v3: https://lore.kernel.org/r/[email protected]
Changes in v3:
- 1/2: clarify the request reference comment (Eric)
- 2/2: drop the sk_clone() change, keep the TCP Fast Open update in place, add release ordering to TIME_WAIT, and rework the changelog (Eric)
- 2/2: note that backports to trees without the refcount helpers need adjustment (Eric)
- v2: https://lore.kernel.org/r/[email protected]
Changes in v2:
- 1/2: split the unhashed checks and __reqsk_free() to reqsk_put() conversions from the ordering changes, set the Fixes tag to commit 079096f103fa ("tcp/dccp: install syn_recv requests into ehash table"), mention the TIME_WAIT hashdance and request replacement cases, clarify request reference ownership, and drop the full socket race discussion (Eric)
- 2/2: new, split out the acquire/release changes, set the Fixes tag to commit fb041bb7c0a9 ("locking/refcount: Consolidate implementations of refcount_t"), cover sk_clone(), and leave the already ordered TIME_WAIT path unchanged (Eric)
- v1: https://lore.kernel.org/r/ath58Egc-oI6p3Jl@v4bel
Hyunwoo Kim (2):
inet: reject unhashed sockets during established lookups
net: restore refcount ordering for lockless socket access
net/core/filter.c | 2 +- net/ipv4/inet_connection_sock.c | 16 ++++++++++------ net/ipv4/inet_hashtables.c | 7 ++++--- net/ipv4/inet_timewait_sock.c | 4 ++-- net/ipv4/syncookies.c | 4 +++- net/ipv4/tcp_fastopen.c | 2 +- net/ipv4/tcp_input.c | 4 +++- net/ipv6/inet6_hashtables.c | 6 ++++-- 8 files changed, 28 insertions(+), 17 deletions(-)
Signature · 1 line
2.43.0
PATCH v5 1/2 inet: reject unhashed sockets during established lookups
TCP established ehash lookups are lockless. The socket caches involved use SLAB_TYPESAFE_BY_RCU, so memory can be reused before an RCU reader that retained its old address has finished.
After a tuple match, __inet_lookup_established() and __inet6_lookup_established() acquire a reference and repeat the tuple check. They do not check whether the new object is hashed. If the address is reused for a socket with the same tuple, the lookup can acquire a reference to the new object and return it while its nulls node is unhashed.
A PoC that drives reuse with a SYN cookie request socket reproduced the crash. The new request socket is unhashed and has a NULL rsk_listener. A lookup that had loaded the address of the old request socket acquired the reference published by tcp_get_cookie_sock() and returned the new object. tcp_v4_rcv() then read sk_state through the NULL listener.
The resulting crash was:
BUG: kernel NULL pointer dereference, address: 0000000000000012 #PF: supervisor read access in kernel mode #PF: error_code(0x0000) - not-present page ... Oops: Oops: 0000 [#1] SMP NOPTI CPU: 4 UID: 0 PID: 0 Comm: swapper/4 Not tainted 7.3.0-00591-g0e135910deec #1 PREEMPT(lazy) ... RIP: 0010:tcp_v4_rcv+0x44e/0x1280 ... R10: 0000000000000001 R11: 00000000024ca8c0 R12: 0000000000000000 ... CR2: 0000000000000012 CR3: 0000000103d50004 CR4: 0000000000770ef0 ... Call Trace: <IRQ> ? raw_local_deliver+0xae/0x240 ip_protocol_deliver_rcu+0x2d/0x180 ip_local_deliver_finish+0x84/0x100 ip_sublist_rcv_finish+0x5f/0x70 ip_sublist_rcv+0x12a/0x1d0 ip_list_rcv+0xfd/0x130 __netif_receive_skb_list_core+0x223/0x250 netif_receive_skb_list_internal+0x1d2/0x2c0 ? try_fill_recv+0x39/0x7b0 napi_complete_done+0x74/0x170 virtnet_poll+0x445/0x600 __napi_poll+0x27/0x1e0 net_rx_action+0x2fe/0x3b0 handle_softirqs+0xc2/0x2c0 __irq_exit_rcu+0x74/0xd0 common_interrupt+0x7f/0x90 </IRQ> ... Kernel panic - not syncing: Fatal exception in interrupt
After taking a reference, check whether the nulls node is unhashed in both IPv4 and IPv6 established lookups. If it is unhashed or the tuple no longer matches, drop the reference and restart the lookup.
Lookups racing with the TIME_WAIT hashdance, or with a request being replaced by its child, now restart and find the new socket instead of returning the old one.
Request sockets that were never inserted into ehash can have a nonzero refcount, and a stale lookup can hold a transient reference. Therefore, when SYN cookie, Fast Open, and request migration paths fail after setting a nonzero refcount, release the references held by those paths with reqsk_put() instead of freeing the request directly.
--- net/ipv4/inet_connection_sock.c | 10 +++++++--- net/ipv4/inet_hashtables.c | 5 +++-- net/ipv4/syncookies.c | 2 ++ net/ipv4/tcp_input.c | 4 +++- net/ipv6/inet6_hashtables.c | 4 +++- 5 files changed, 18 insertions(+), 7 deletions(-)
net/ipv4/inet_connection_sock.c +7 -3
@@ -1136,10 +1136,14 @@ static void reqsk_timer_handler(struct timer_list *t)*/ if (nreq) {__NET_INC_STATS (net,LINUX_MIB_TCPMIGRATEREQFAILURE ); +/* The timer was not armed, drop its reference here. */ +reqsk_put (nreq); no_ownership:reqsk_migrate_reset (nreq);reqsk_queue_removed (queue, nreq); -__reqsk_free (nreq); +/* Drop the temporary reference and the one reserved for ehash. */ +reqsk_put (nreq); +reqsk_put (nreq); } drop:
@@ -1447,7 +1451,7 @@ struct sock *inet_csk_complete_hashdance(struct sock *sk, struct sock *child,__NET_INC_STATS (sock_net (sk),LINUX_MIB_TCPMIGRATEREQFAILURE );reqsk_migrate_reset (nreq); -__reqsk_free (nreq); +reqsk_put (nreq); }else if (inet_csk_reqsk_queue_add (sk, req, child)) {return child; }
@@ -1502,7 +1506,7 @@ void inet_csk_listen_stop(struct sock *sk)__NET_INC_STATS (sock_net (nsk),LINUX_MIB_TCPMIGRATEREQFAILURE );reqsk_migrate_reset (nreq); -__reqsk_free (nreq); +reqsk_put (nreq); }rcu_read_unlock ();
net/ipv4/inet_hashtables.c +3 -2
@@ -552,8 +552,9 @@ struct sock *__inet_lookup_established(const struct net *net,if (likely (inet_match (net, sk, acookie, ports, dif, sdif))) {if (unlikely (!refcount_inc_not_zero (&sk->sk_refcnt))) goto out; -if (unlikely (!inet_match (net, sk, acookie, - ports, dif, sdif))) { +if (unlikely (hlist_nulls_unhashed_lockless (&sk->sk_nulls_node) || + !inet_match (net, sk, acookie, ports, dif, + sdif))) {sock_gen_put (sk); goto begin; }
net/ipv4/syncookies.c +2 -0
@@ -218,6 +218,8 @@ struct sock *tcp_get_cookie_sock(struct sock *sk, struct sk_buff *skb,bh_unlock_sock (child);sock_put (child); +reqsk_put (req); +return NULL ; }__reqsk_free (req);
net/ipv4/tcp_input.c +3 -1
@@ -7777,7 +7777,9 @@ int tcp_conn_request(struct request_sock_ops *rsk_ops,if (!inet_csk_reqsk_queue_add (sk, req, fastopen_sk)) {bh_unlock_sock (fastopen_sk);sock_put (fastopen_sk); - goto drop_and_free; +/* tcp_disconnect() already dropped one reqsk ref. */ +reqsk_put (req); + goto drop; }READ_ONCE (sk->sk_data_ready)(sk);bh_unlock_sock (fastopen_sk);
net/ipv6/inet6_hashtables.c +3 -1
@@ -108,7 +108,9 @@ struct sock *__inet6_lookup_established(const struct net *net,if (unlikely (!refcount_inc_not_zero (&sk->sk_refcnt))) goto out; -if (unlikely (!inet6_match (net, sk, saddr, daddr, ports, dif, sdif))) { +if (unlikely (hlist_nulls_unhashed_lockless (&sk->sk_nulls_node) || + !inet6_match (net, sk, saddr, daddr, ports, dif, + sdif))) {sock_gen_put (sk); goto begin; }
Signature · 1 line
2.43.0
[email protected]
AI reviewRe: [PATCH net v5 1/2] inet: reject unhashed sockets during established lookups
Thank you for your contribution! Sashiko AI review found more severe issue(s) in other patches of this series. This mail is only an FYI, in case you want to address the nit picks below when you…
Thank you for your contribution! Sashiko AI review found more severe issue(s) in other patches of this series. This mail is only an FYI, in case you want to address the nit picks below when you respin.
Low priority / nit picks for your consideration:
- [Low] In this commit by itself, the new hlist_nulls_unhashed_lockless(&sk->sk_nulls_node) check in…
--
inet: reject unhashed sockets during established lookups
This commit adds a hlist_nulls_unhashed_lockless() check after the reference is taken in __inet_lookup_established() and __inet6_lookup_established(). A lookup that races with SLAB_TYPESAFE_BY_RCU reuse can no longer return a request socket that was never hashed. It also changes the SYN cookie, Fast Open and request migration failure paths to drop references with reqsk_put() instead of calling __reqsk_free().
net/ipv4/inet_hashtables.c struct sock *__inet_lookup_established(const struct net *net, · line 557
Quoted 13 lines
net/ipv4/inet_hashtables.c +3
-2@@ -552,8 +552,9 @@ struct sock *__inet_lookup_established(const struct net *net,if (likely (inet_match (net, sk, acookie, ports, dif, sdif))) {if (unlikely (!refcount_inc_not_zero (&sk->sk_refcnt))) goto out; -if (unlikely (!inet_match (net, sk, acookie, - ports, dif, sdif))) { +if (unlikely (hlist_nulls_unhashed_lockless (&sk->sk_nulls_node) || + !inet_match (net, sk, acookie, ports, dif, + sdif))) {
[Severity: Low]
If this commit is applied by itself, is the new unhashed check ordered after the refcount increment?
refcount_inc_not_zero() here is a relaxed cmpxchg. The writers that make never-hashed request sockets referenceable also use a plain refcount_set():
net/ipv4/syncookies.c:tcp_get_cookie_sock() {
...
if (child) {
refcount_set(&req->rsk_refcnt, 1);
...
}The same applies to tcp_fastopen_create_child(), and to the migration clones in reqsk_timer_handler(), inet_csk_complete_hashdance() and inet_csk_listen_stop().
On weakly ordered CPUs, can the pprev load in hlist_nulls_unhashed_lockless() and the inet_match() reload be satisfied before the increment? They could then see the old, hashed incarnation, while the increment succeeds on the new, unhashed SYN cookie request whose rsk_listener is NULL. That is the tcp_v4_rcv() NULL dereference this commit describes. __inet6_lookup_established() has the same pattern.
The next patch in the series, "net: restore refcount ordering for lockless socket access", resolves this. It switches both lookups to refcount_inc_not_zero_acquire(), and switches tcp_get_cookie_sock(), tcp_fastopen_create_child() and the three migration writers to refcount_set_release().
Both patches are tagged Cc: stable. Should the two be backported together, so stable trees don't pick up this one without the ordering fix?
Quoted 3 lines
net/ipv4/inet_hashtables.c +0
-0sock_gen_put (sk); goto begin; }
[ ... ]
Signature · 1 line
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20261029214219.1035176-1-imv4bel%40gmail.com
PATCH v5 2/2 net: restore refcount ordering for lockless socket access
In reply to Hyunwoo KimBefore commit fb041bb7c0a9 ("locking/refcount: Consolidate implementations of refcount_t"), default builds implemented refcount_inc_not_zero() using atomic_add_unless(), which provided full memory ordering on success. The commit removed that ordering on weakly ordered architectures, allowing accesses after a successful reference acquisition to be reordered before it.
Use refcount_inc_not_zero_acquire() in the IPv4 and IPv6 established lookups and in inet_twsk_purge(), where the socket is validated again after taking a reference.
bpf_sk_assign() can receive a socket pointer obtained through BTF without a reference. Use acquire ordering when it takes its own reference for a refcounted socket.
Use refcount_set_release() when setting refcounts in the SYN cookie, TCP Fast Open, request migration, and TIME_WAIT paths. Keep the existing TIME_WAIT smp_wmb(), which separately orders the refcount store before ehash publication.
--- net/core/filter.c | 2 +- net/ipv4/inet_connection_sock.c | 6 +++--- net/ipv4/inet_hashtables.c | 2 +- net/ipv4/inet_timewait_sock.c | 4 ++-- net/ipv4/syncookies.c | 2 +- net/ipv4/tcp_fastopen.c | 2 +- net/ipv6/inet6_hashtables.c | 2 +- 7 files changed, 10 insertions(+), 10 deletions(-)
net/core/filter.c +1 -1
@@ -7989,7 +7989,7 @@ BPF_CALL_3(bpf_sk_assign, struct sk_buff *, skb, struct sock *, sk, u64, flags)if (sk_unhashed (sk))return -EOPNOTSUPP ;if (sk_is_refcounted (sk) && -unlikely (!refcount_inc_not_zero (&sk->sk_refcnt))) +unlikely (!refcount_inc_not_zero_acquire (&sk->sk_refcnt)))return -ENOENT ;skb_orphan (skb);
net/ipv4/inet_connection_sock.c +3 -3
@@ -1056,8 +1056,8 @@ static void reqsk_timer_handler(struct timer_list *t)* hold another count to prevent use-after-free and * call reqsk_put() just before return. */ -refcount_set (&nreq->rsk_refcnt,2 +1 );timer_setup (&nreq->rsk_timer, reqsk_timer_handler,TIMER_PINNED ); +refcount_set_release (&nreq->rsk_refcnt,2 +1 );reqsk_queue_migrated (&inet_csk (nsk)->icsk_accept_queue, req); req = nreq;
@@ -1441,7 +1441,7 @@ struct sock *inet_csk_complete_hashdance(struct sock *sk, struct sock *child, goto child_put; } -refcount_set (&nreq->rsk_refcnt,1 ); +refcount_set_release (&nreq->rsk_refcnt,1 );if (inet_csk_reqsk_queue_add (sk, nreq, child)) {__NET_INC_STATS (sock_net (sk),LINUX_MIB_TCPMIGRATEREQSUCCESS );reqsk_migrate_reset (req);
@@ -1494,7 +1494,7 @@ void inet_csk_listen_stop(struct sock *sk)if (nsk) { nreq =inet_reqsk_clone (req, nsk);if (nreq) { -refcount_set (&nreq->rsk_refcnt,1 ); +refcount_set_release (&nreq->rsk_refcnt,1 );rcu_read_lock ();if (inet_csk_reqsk_queue_add (nsk, nreq, child)) {
net/ipv4/inet_hashtables.c +1 -1
@@ -550,7 +550,7 @@ struct sock *__inet_lookup_established(const struct net *net, if (sk->sk_hash != hash) continue; if (likely(inet_match(net, sk, acookie, ports, dif, sdif))) { - if (unlikely(!refcount_inc_not_zero(&sk->sk_refcnt))) + if (unlikely(!refcount_inc_not_zero_acquire(&sk->sk_refcnt))) goto out; if (unlikely(hlist_nulls_unhashed_lockless(&sk->sk_nulls_node) || !inet_match(net, sk, acookie, ports, dif,
net/ipv4/inet_timewait_sock.c +2 -2
@@ -141,7 +141,7 @@ void inet_twsk_hashdance_schedule(struct inet_timewait_sock *tw,* Also note that after this point, we lost our implicit reference * so we are not allowed to use tw anymore. */ -refcount_set (&tw->tw_refcnt,3 ); +refcount_set_release (&tw->tw_refcnt,3 );/* Ensure tw_refcnt has been set before tw is published. * smp_wmb() provides the necessary memory barrier to enforce this
@@ -328,7 +328,7 @@ void inet_twsk_purge(struct inet_hashinfo *hashinfo)if (check_net (sock_net (sk)))continue ; -if (unlikely (!refcount_inc_not_zero (&sk->sk_refcnt))) +if (unlikely (!refcount_inc_not_zero_acquire (&sk->sk_refcnt)))continue ;if (check_net (sock_net (sk))) {
net/ipv4/syncookies.c +1 -1
@@ -205,7 +205,7 @@ struct sock *tcp_get_cookie_sock(struct sock *sk, struct sk_buff *skb, child = icsk->icsk_af_ops->syn_recv_sock (sk, skb, req, dst,NULL , &own_req,NULL );if (child) { -refcount_set (&req->rsk_refcnt,1 ); +refcount_set_release (&req->rsk_refcnt,1 );sock_rps_save_rxhash (child, skb);if (rsk_drop_req (req)) {
net/ipv4/tcp_fastopen.c +1 -1
@@ -364,7 +364,7 @@ static struct sock *tcp_fastopen_create_child(struct sock *sk,tcp_reset_xmit_timer (child,ICSK_TIME_RETRANS , req->timeout, false); -refcount_set (&req->rsk_refcnt,2 ); +refcount_set_release (&req->rsk_refcnt,2 );sk_mark_napi_id_set (child, skb);
net/ipv6/inet6_hashtables.c +1 -1
@@ -105,7 +105,7 @@ struct sock *__inet6_lookup_established(const struct net *net,continue ;if (!inet6_match (net, sk, saddr, daddr, ports, dif, sdif))continue ; -if (unlikely (!refcount_inc_not_zero (&sk->sk_refcnt))) +if (unlikely (!refcount_inc_not_zero_acquire (&sk->sk_refcnt))) goto out; if (unlikely(hlist_nulls_unhashed_lockless(&sk->sk_nulls_node) ||
Signature · 1 line
2.43.0
Re: [PATCH net v5 2/2] net: restore refcount ordering for lockless socket access
Quoted 20 lines · Hyunwoo Kim
Before commit
fb041bb7c0a9("locking/refcount: Consolidate implementations ofrefcount_t"), default builds implementedrefcount_inc_not_zero()usingatomic_add_unless(), which provided full memory ordering on success. The commit removed that ordering on weakly ordered architectures, allowing accesses after a successful reference acquisition to be reordered before it.Use
refcount_inc_not_zero_acquire()in the IPv4 and IPv6 established lookups and ininet_twsk_purge(), where the socket is validated again after taking a reference.
bpf_sk_assign()can receive a socket pointer obtained through BTF without a reference. Use acquire ordering when it takes its own reference for a refcounted socket.Use
refcount_set_release()when setting refcounts in the SYN cookie, TCP Fast Open, request migration, andTIME_WAITpaths. Keep the existingTIME_WAITsmp_wmb(), which separately orders the refcount store before ehash publication.
Quoted 4 lines
Fixes fb041bb7c0a9 ("locking/refcount: Consolidate implementations of refcount_t")Fixes 27e5203bd9c5 ("bpf: Change bpf_sk_assign to accept ARG_PTR_TO_BTF_ID_SOCK_COMMON")Signed-off-by Hyunwoo Kim <[email protected]>
Thanks !
[email protected]
AI reviewRe: [PATCH net v5 2/2] net: restore refcount ordering for lockless socket access
In reply to Hyunwoo KimThank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider.
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider.
Critical: 0 · High: 2 · Medium: 0 · Low: 0
If you believe none of the issues are legitimate, please reply to each issue of Medium severity or higher and explain why the AI is wrong. If any of them are legitimate and you plan to respin, please let patchwork know by including "pw-bot: cr" as a separate line at the end of your reply (one such reply per series is enough).
- [High] The bpf_sk_assign() change in net/core/filter.c does not fix the problem that its Fixes: 27e5203bd9c5 tag points at.
- [High] The patch is titled "restore refcount ordering for lockless socket access" and converts lockless lookups "where the socket is…
--
net: restore refcount ordering for lockless socket access
This patch switches the IPv4/IPv6 established lookups, inet_twsk_purge() and bpf_sk_assign() to refcount_inc_not_zero_acquire(). It also uses refcount_set_release() when setting refcounts in the SYN cookie, TCP Fast Open, request migration and TIME_WAIT paths.
net/core/filter.c BPF_CALL_3(bpf_sk_assign, struct sk_buff *, skb, struct sock *, sk, u64, flags) · line 8028
Quoted 13 lines
net/core/filter.c +1
-1@@ -8022,7 +8022,7 @@ BPF_CALL_3(bpf_sk_assign, struct sk_buff *, skb, struct sock *, sk, u64, flags)if (sk_unhashed (sk))return -EOPNOTSUPP ;if (sk_is_refcounted (sk) && -unlikely (!refcount_inc_not_zero (&sk->sk_refcnt))) +unlikely (!refcount_inc_not_zero_acquire (&sk->sk_refcnt)))return -ENOENT ;skb_orphan (skb);
[Severity: High]
Does acquire ordering here fix the unreferenced BTF pointer case that the
Acquire only orders the accesses that come after the increment. In bpf_sk_assign(), every check on the socket runs before it:
if (unlikely(dev_net(skb->dev) != sock_net(sk)))
return -ENETUNREACH;
if (sk_unhashed(sk))
return -EOPNOTSUPP;
if (sk_is_refcounted(sk) &&
unlikely(!refcount_inc_not_zero_acquire(&sk->sk_refcnt)))
return -ENOENT;
skb_orphan(skb);
skb->sk = sk;
skb->destructor = sock_pfree;After the increment there are only stores. Nothing is checked again.
The verifier's btf_id_sock_common_types accepts plain, untrusted PTR_TO_BTF_ID. tcp_prot and tcpv6_prot use SLAB_TYPESAFE_BY_RCU, and sk_destruct() frees sockets without SOCK_RCU_FREE right away through __sk_destruct().
Given that, can this sequence happen?
- socket A passes the netns, hashed and sk_is_refcounted() checks
- A is freed and its slot is reused as socket B (maybe in another netns, or not hashed yet), and B's refcount is set
- refcount_inc_not_zero_acquire() succeeds on B
- B is attached to the skb without being checked
inet_steal_sock()/skb_steal_sock() then deliver to the prefetched socket without checking the netns. That would get around the -ENETUNREACH check.
sock_pfree() and skb_steal_sock() also call sk_is_refcounted() again later. If the reused object's result differs, could the reference taken here leak?
This race doesn't depend on memory ordering, and it existed even when the increment was fully ordered. The established lookups changed by this patch re-check with inet_match()/inet6_match() and hlist_nulls_unhashed_lockless() after the increment, then call sock_gen_put() and retry. bpf_sk_assign() has no such step.
Would it work better to redo the netns, hashed and refcounted checks after taking the reference and call sock_gen_put() on a mismatch? Rejecting unreferenced pointers here would be another option.
net/ipv4/inet_hashtables.c struct sock *__inet_lookup_established(const struct net *net, · line 556
Quoted 13 lines
net/ipv4/inet_hashtables.c +1
-1@@ -550,7 +550,7 @@ struct sock *__inet_lookup_established(const struct net *net, if (sk->sk_hash != hash) continue; if (likely(inet_match(net, sk, acookie, ports, dif, sdif))) { - if (unlikely(!refcount_inc_not_zero(&sk->sk_refcnt))) + if (unlikely(!refcount_inc_not_zero_acquire(&sk->sk_refcnt))) goto out; if (unlikely(hlist_nulls_unhashed_lockless(&sk->sk_nulls_node) || !inet_match(net, sk, acookie, ports, dif,
[Severity: High]
The commit message says the change targets lookups "where the socket is validated again after taking a reference". Should the MPTCP token lookups be converted too? They follow the same pattern but still use plain refcount_inc_not_zero():
net/mptcp/token.c:mptcp_token_get_sock() {
...
if (!refcount_inc_not_zero(&sk->sk_refcnt))
goto not_found;
if (READ_ONCE(msk->token) != token ||
!net_eq(sock_net(sk), net)) {
sock_put(sk);
goto again;
}
...
}mptcp_token_iter_next() does the same thing: it increments and then checks net_eq() again.
mptcp_prot uses SLAB_TYPESAFE_BY_RCU, and mptcp_sk_clone_init() clears SOCK_RCU_FREE on established msks:
sock_reset_flag(nsk, SOCK_RCU_FREE);
So an msk can be freed and its slot reused while an RCU reader still holds the address.
On weakly ordered CPUs, can the token and sk_net loads in the re-check run before the increment? If so, they could see the old object's values, and the lookup would return a referenced msk for a different connection or netns.
Callers include the userspace PM netlink operations that look up by token, MP_JOIN handling in subflow.c and syncookies.c, and the diag and PM iterators. HMAC validation would probably reject most of the MP_JOIN cases.
mptcp_subflow_reqsk_clone() already uses refcount_inc_not_zero_acquire() on the same msk objects.
Signature · 1 line
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20261029214219.1035176-1-imv4bel%40gmail.com
Re: [PATCH net v5 2/2] net: restore refcount ordering for lockless socket access
Quoted 86 lines · On Mon, Nov 02, 2026 at 10:06:21PM +0000, [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider.
Critical: 0 · High: 2 · Medium: 0 · Low: 0
If you believe none of the issues are legitimate, please reply to each issue of Medium severity or higher and explain why the AI is wrong. If any of them are legitimate and you plan to respin, please let patchwork know by including "pw-bot: cr" as a separate line at the end of your reply (one such reply per series is enough).
- [High] The
bpf_sk_assign()change innet/core/filter.cdoes not fix the problem that its Fixes:27e5203bd9c5tag points at.
- [High] The patch is titled "restore refcount ordering for lockless socket access" and converts lockless lookups "where the socket is…--
net: restore refcount ordering for lockless socket access
This patch switches the IPv4/IPv6 established lookups,
inet_twsk_purge()andbpf_sk_assign()torefcount_inc_not_zero_acquire(). It also usesrefcount_set_release()when setting refcounts in the SYN cookie, TCP Fast Open, request migration andTIME_WAITpaths.net/core/filter.c BPF_CALL_3(bpf_sk_assign, struct sk_buff *, skb, struct sock *, sk, u64, flags) · line 8028
Quoted 13 lines
net/core/filter.c +1
-1@@ -8022,7 +8022,7 @@ BPF_CALL_3(bpf_sk_assign, struct sk_buff *, skb, struct sock *, sk, u64, flags)if (sk_unhashed (sk))return -EOPNOTSUPP ;if (sk_is_refcounted (sk) && -unlikely (!refcount_inc_not_zero (&sk->sk_refcnt))) +unlikely (!refcount_inc_not_zero_acquire (&sk->sk_refcnt)))return -ENOENT ;skb_orphan (skb);[Severity: High]
Does acquire ordering here fix the unreferenced BTF pointer case that theFixes 27e5203bd9c5 tag refers to?Acquire only orders the accesses that come after the increment. In
bpf_sk_assign(), every check on the socket runs before it:if (unlikely(dev_net(skb->dev) != sock_net(sk))) return -ENETUNREACH; if (sk_unhashed(sk)) return -EOPNOTSUPP; if (sk_is_refcounted(sk) && unlikely(!refcount_inc_not_zero_acquire(&sk->sk_refcnt))) return -ENOENT; skb_orphan(skb); skb->sk = sk; skb->destructor = sock_pfree;After the increment there are only stores. Nothing is checked again.
The verifier's
btf_id_sock_common_typesaccepts plain, untrustedPTR_TO_BTF_ID.tcp_protandtcpv6_protuseSLAB_TYPESAFE_BY_RCU, andsk_destruct()frees sockets withoutSOCK_RCU_FREEright away through__sk_destruct().Given that, can this sequence happen?
- socket A passes the netns, hashed and
sk_is_refcounted()checks
- A is freed and its slot is reused as socket B (maybe in another netns, or not hashed yet), and B's refcount is set
-refcount_inc_not_zero_acquire()succeeds on B
- B is attached to the skb without being checkedinet_steal_sock()/skb_steal_sock() then deliver to the prefetched socket without checking the netns. That would get around the
-ENETUNREACHcheck.
sock_pfree()andskb_steal_sock()also callsk_is_refcounted()again later. If the reused object's result differs, could the reference taken here leak?This race doesn't depend on memory ordering, and it existed even when the increment was fully ordered. The established lookups changed by this patch re-check with inet_match()/inet6_match() and
hlist_nulls_unhashed_lockless()after the increment, then callsock_gen_put()and retry.bpf_sk_assign()has no such step.
Quoted 3 lines
Would it work better to redo the netns, hashed and refcounted checks after taking the reference and call
sock_gen_put()on a mismatch? Rejecting unreferenced pointers here would be another option.
This is beyond the ordering fix and would complicate the patch. I'll address it in a follow-up patch.
Quoted 60 lines
Quoted 6 lines
net/ipv4/inet_hashtables.c +0
-0net/ipv4/inet_hashtables.c struct sock *__inet_lookup_established(const struct net *net, · line 556
Quoted 8 lines
net/ipv4/inet_hashtables.c +1
-1@@ -550,7 +550,7 @@ struct sock *__inet_lookup_established(const struct net *net, if (sk->sk_hash != hash) continue; if (likely(inet_match(net, sk, acookie, ports, dif, sdif))) { - if (unlikely(!refcount_inc_not_zero(&sk->sk_refcnt))) + if (unlikely(!refcount_inc_not_zero_acquire(&sk->sk_refcnt))) goto out; if (unlikely(hlist_nulls_unhashed_lockless(&sk->sk_nulls_node) || !inet_match(net, sk, acookie, ports, dif,[Severity: High]
The commit message says the change targets lookups "where the socket is validated again after taking a reference". Should the MPTCP token lookups be converted too? They follow the same pattern but still use plainrefcount_inc_not_zero():net/mptcp/token.c:mptcp_token_get_sock() { ... if (!refcount_inc_not_zero(&sk->sk_refcnt)) goto not_found; if (READ_ONCE(msk->token) != token || !net_eq(sock_net(sk), net)) { sock_put(sk); goto again; } ... }
mptcp_token_iter_next()does the same thing: it increments and then checksnet_eq()again.
mptcp_protusesSLAB_TYPESAFE_BY_RCU, andmptcp_sk_clone_init()clearsSOCK_RCU_FREEon established msks:sock_reset_flag(nsk, SOCK_RCU_FREE);So an msk can be freed and its slot reused while an RCU reader still holds the address.
On weakly ordered CPUs, can the token and
sk_netloads in the re-check run before the increment? If so, they could see the old object's values, and the lookup would return a referenced msk for a different connection or netns.Callers include the userspace PM netlink operations that look up by token,
MP_JOINhandling insubflow.candsyncookies.c, and the diag and PM iterators. HMAC validation would probably reject most of theMP_JOINcases.
mptcp_subflow_reqsk_clone()already usesrefcount_inc_not_zero_acquire()on the same msk objects.Signature · 1 line
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20261029214219.1035176-1-imv4bel%40gmail.com
Paolo Abeni
maintainerRe: [PATCH net v5 2/2] net: restore refcount ordering for lockless socket access
In reply to [email protected]Quoted 49 lines · On 11/2/26 23:06, [email protected] wrote:
net/ipv4/inet_hashtables.c struct sock *__inet_lookup_established(const struct net *net, · line 556
Quoted 13 lines
net/ipv4/inet_hashtables.c +1
-1@@ -550,7 +550,7 @@ struct sock *__inet_lookup_established(const struct net *net, if (sk->sk_hash != hash) continue; if (likely(inet_match(net, sk, acookie, ports, dif, sdif))) { - if (unlikely(!refcount_inc_not_zero(&sk->sk_refcnt))) + if (unlikely(!refcount_inc_not_zero_acquire(&sk->sk_refcnt))) goto out; if (unlikely(hlist_nulls_unhashed_lockless(&sk->sk_nulls_node) || !inet_match(net, sk, acookie, ports, dif,[Severity: High]
The commit message says the change targets lookups "where the socket is validated again after taking a reference". Should the MPTCP token lookups be converted too? They follow the same pattern but still use plainrefcount_inc_not_zero():net/mptcp/token.c:mptcp_token_get_sock() { ... if (!refcount_inc_not_zero(&sk->sk_refcnt)) goto not_found; if (READ_ONCE(msk->token) != token || !net_eq(sock_net(sk), net)) { sock_put(sk); goto again; } ... }
mptcp_token_iter_next()does the same thing: it increments and then checksnet_eq()again.
mptcp_protusesSLAB_TYPESAFE_BY_RCU, andmptcp_sk_clone_init()clearsSOCK_RCU_FREEon established msks:sock_reset_flag(nsk, SOCK_RCU_FREE);So an msk can be freed and its slot reused while an RCU reader still holds the address.
On weakly ordered CPUs, can the token and
sk_netloads in the re-check run before the increment? If so, they could see the old object's values, and the lookup would return a referenced msk for a different connection or netns.
Quoted 7 lines
Callers include the userspace PM netlink operations that look up by token,
MP_JOINhandling insubflow.candsyncookies.c, and the diag and PM iterators. HMAC validation would probably reject most of theMP_JOINcases.
mptcp_subflow_reqsk_clone()already usesrefcount_inc_not_zero_acquire()on the same msk objects.
Even this part is better handled with a separate patch.
/P
Eric Dumazet
maintainerRe: [PATCH net v5 2/2] net: restore refcount ordering for lockless socket access
In reply to Hyunwoo KimQuoted 21 lines · Hyunwoo Kim
Before commit
fb041bb7c0a9("locking/refcount: Consolidate implementations ofrefcount_t"), default builds implementedrefcount_inc_not_zero()usingatomic_add_unless(), which provided full memory ordering on success. The commit removed that ordering on weakly ordered architectures, allowing accesses after a successful reference acquisition to be reordered before it.Use
refcount_inc_not_zero_acquire()in the IPv4 and IPv6 established lookups and ininet_twsk_purge(), where the socket is validated again after taking a reference.
bpf_sk_assign()can receive a socket pointer obtained through BTF without a reference. Use acquire ordering when it takes its own reference for a refcounted socket.Use
refcount_set_release()when setting refcounts in the SYN cookie, TCP Fast Open, request migration, andTIME_WAITpaths. Keep the existingTIME_WAITsmp_wmb(), which separately orders the refcount store before ehash publication.
Quoted 5 lines
Fixes fb041bb7c0a9 ("locking/refcount: Consolidate implementations of refcount_t")Fixes 27e5203bd9c5 ("bpf: Change bpf_sk_assign to accept ARG_PTR_TO_BTF_ID_SOCK_COMMON")Signed-off-by Hyunwoo Kim <[email protected]>---
Lack of links to prior versions is rather annoying.
Re: [PATCH net v5 2/2] net: restore refcount ordering for lockless socket access
Quoted 22 lines · Eric Dumazet
Quoted 21 lines · Hyunwoo Kim
Before commit
fb041bb7c0a9("locking/refcount: Consolidate implementations ofrefcount_t"), default builds implementedrefcount_inc_not_zero()usingatomic_add_unless(), which provided full memory ordering on success. The commit removed that ordering on weakly ordered architectures, allowing accesses after a successful reference acquisition to be reordered before it.Use
refcount_inc_not_zero_acquire()in the IPv4 and IPv6 established lookups and ininet_twsk_purge(), where the socket is validated again after taking a reference.
bpf_sk_assign()can receive a socket pointer obtained through BTF without a reference. Use acquire ordering when it takes its own reference for a refcounted socket.Use
refcount_set_release()when setting refcounts in the SYN cookie, TCP Fast Open, request migration, andTIME_WAITpaths. Keep the existingTIME_WAITsmp_wmb(), which separately orders the refcount store before ehash publication.
Quoted 7 lines
Quoted 5 lines
Fixes fb041bb7c0a9 ("locking/refcount: Consolidate implementations of refcount_t")Fixes 27e5203bd9c5 ("bpf: Change bpf_sk_assign to accept ARG_PTR_TO_BTF_ID_SOCK_COMMON")Signed-off-by Hyunwoo Kim <[email protected]>---
Lack of links to prior versions is rather annoying.
Is the cover letter alone insufficient?
Should the changelog be in each patch?
Paolo Abeni
maintainerRe: [PATCH net v5 2/2] net: restore refcount ordering for lockless socket access
Quoted 29 lines · Hyunwoo Kim
Quoted 28 lines · Eric Dumazet
Quoted 26 lines · Hyunwoo Kim
Before commit
fb041bb7c0a9("locking/refcount: Consolidate implementations ofrefcount_t"), default builds implementedrefcount_inc_not_zero()usingatomic_add_unless(), which provided full memory ordering on success. The commit removed that ordering on weakly ordered architectures, allowing accesses after a successful reference acquisition to be reordered before it.Use
refcount_inc_not_zero_acquire()in the IPv4 and IPv6 established lookups and ininet_twsk_purge(), where the socket is validated again after taking a reference.
bpf_sk_assign()can receive a socket pointer obtained through BTF without a reference. Use acquire ordering when it takes its own reference for a refcounted socket.Use
refcount_set_release()when setting refcounts in the SYN cookie, TCP Fast Open, request migration, andTIME_WAITpaths. Keep the existingTIME_WAITsmp_wmb(), which separately orders the refcount store before ehash publication.Fixes fb041bb7c0a9 ("locking/refcount: Consolidate implementations of refcount_t")Fixes 27e5203bd9c5 ("bpf: Change bpf_sk_assign to accept ARG_PTR_TO_BTF_ID_SOCK_COMMON")Signed-off-by Hyunwoo Kim <[email protected]>---
Quoted 5 lines
Quoted 1 line
Lack of links to prior versions is rather annoying.
Is the cover letter alone insufficient?
Should the changelog be in each patch?
Ideally the cover letter should include a very quick/short overview of the changes, and a per patch changelog should include a more detailed list (no entry added when the patch is unmodified).
Thanks,
/P
Re: [PATCH net v5 2/2] net: restore refcount ordering for lockless socket access
Quoted 32 lines · Paolo Abeni
Quoted 31 lines · Hyunwoo Kim
Quoted 29 lines · Eric Dumazet
Quoted 21 lines · Hyunwoo Kim
Before commit
fb041bb7c0a9("locking/refcount: Consolidate implementations ofrefcount_t"), default builds implementedrefcount_inc_not_zero()usingatomic_add_unless(), which provided full memory ordering on success. The commit removed that ordering on weakly ordered architectures, allowing accesses after a successful reference acquisition to be reordered before it.Use
refcount_inc_not_zero_acquire()in the IPv4 and IPv6 established lookups and ininet_twsk_purge(), where the socket is validated again after taking a reference.
bpf_sk_assign()can receive a socket pointer obtained through BTF without a reference. Use acquire ordering when it takes its own reference for a refcounted socket.Use
refcount_set_release()when setting refcounts in the SYN cookie, TCP Fast Open, request migration, andTIME_WAITpaths. Keep the existingTIME_WAITsmp_wmb(), which separately orders the refcount store before ehash publication.Quoted 5 lines
Fixes fb041bb7c0a9 ("locking/refcount: Consolidate implementations of refcount_t")Fixes 27e5203bd9c5 ("bpf: Change bpf_sk_assign to accept ARG_PTR_TO_BTF_ID_SOCK_COMMON")Signed-off-by Hyunwoo Kim <[email protected]>---
Lack of links to prior versions is rather annoying.
Quoted 7 lines
Quoted 3 lines
Is the cover letter alone insufficient?
Should the changelog be in each patch?
Ideally the cover letter should include a very quick/short overview of the changes, and a per patch changelog should include a more detailed list (no entry added when the patch is unmodified).
Understood. I'll do that from now on.
Best regards,
Hyunwoo Kim
This series was applied to netdev/net.git (main) by Paolo Abeni <[email protected]>:
Hello:
This series was applied to netdev/net.git (main)
by Paolo Abeni <[email protected]>:
Quoted 10 lines · On Fri, 30 Oct 2026 06:42:15 +0900 you wrote:
Fix two issues in lockless ehash walks. Reject unhashed sockets after taking a reference in established lookups, and restore the acquire/release ordering around refcount publication and acquisition.
Changes in v5:
- 2/2: add acquire ordering tobpf_sk_assign()for BTF socket pointers (Kuniyuki)
- v4: https://lore.kernel.org/r/[email protected][...]
Here is the summary with links:
- [net,v5,1/2] inet: reject unhashed sockets during established lookups
https://git.kernel.org/netdev/net/c/4e2b3ef19e0e - [net,v5,2/2] net: restore refcount ordering for lockless socket access https://git.kernel.org/netdev/net/c/5b5d8f351dd9
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html