net/rds: make connection lifetime reference-counted
Subsystems (4)
Allison Henderson
maintainerv6 cover net/rds: make connection lifetime reference-counted
Hi all,
This is v6 of the connection-lifetime set (v1 at [1], v2 at [2], v3 at [3], v4 at [6], v5 at [7]), following
"net/rds: own the fastpath locks across connection teardown", which is in net-next. It is targeted at net-next: although the series fixes real use-after-frees (one syzbot report and one report from Chengfeng Ye below), it does so by reworking connection lifetime rather than patching the individual crash sites, and that rework is too invasive for net.
rds_conn_destroy() frees the connection, its paths and its workqueues on the spot, relying on the documented assumption that "no one else is referencing the connection". That assumption stopped being true a long time ago: connections are destroyed on netns teardown and on a protocol-version mismatch as well as on rmmod, while pointers to them still live in socket rs_conn caches, congestion-map conn lists, CM callbacks and workers, and - for as long as an application leaves data unread - in every rds_incoming sitting on a receive queue. No single
Fixes: commit covers the rot, so the series carries Reported-by tags where there are concrete reports instead.
Patch 1 fixes rds_ib_conn_free() re-enabling interrupts under a caller's irqsave lock.
Patch 2 makes the passive-connection exits of __rds_conn_create() undo conn_alloc() the same way the lost-race exit does, through one helper (a refactor: the passive twin only exists for IB, so nothing leaked there). Both are first in the series because the reference-counted teardown calls conn_free() from more places.
Patch 3 guards the five work-arming sites that never tested rds_destroy_pending(): two IB completion paths, the IB recv refill, the TCP accept kick and the multipath reconnect in sendmsg.
Patch 4 gives the connection its own destroy-in-progress marker. With f97d8c7bab78 in net-next there is no single-connection destroy left for the old predicate to miss, but the later patches need the precise answer: the IB unload re-sweep hands a connection to rds_conn_destroy() more than once, and the passive-twin creation must refuse a parent whose destroy has begun.
Based on UEK commits:
e2f5005adf63 net/rds: Add krefs to struct rds_connection
https://github.com/oracle/linux-uek/commit/e2f5005adf63
6c53ef92f46e net/rds: Merge uses of conn->c_destroy_in_prog & RDS_DESTROY_PENDING
https://github.com/oracle/linux-uek/commit/6c53ef92f46e Patch 5 splits rds_conn_destroy() into a synchronous quiesce and a kref-governed free.
Based on UEK commits:
2c8569e4c880 ("net/rds: Add krefs to struct rds_connection").
https://github.com/oracle/linux-uek/commit/2c8569e4c880 Patch 6 makes each transport's exit path wait for its connections to actually be freed before the module goes away. The wait is
unbounded and warns every ten seconds: a connection reference can be held for an application-controlled time (unread data), so a timeout would only move the use-after-free from freed memory to unloaded module text. rmmod blocking while data is queued and unread is the historical RDS contract. For IB the wait re-sweeps the nodev list, since device connections migrate to it asynchronously.
Based on UEK commits:
ece4b4e39afa ("net/rds: wait_event_timeout until zero connections during rmmod")
https://github.com/oracle/linux-uek/commit/ece4b4e39afa
905ec90e6166 ("net/rds: Each RDS transport should keep its own connection count")
https://github.com/oracle/linux-uek/commit/905ec90e6166
Patch 7 unlinks each transport node before its destroy, so a free deferred past the teardown loop cannot write into the loop's stack-local list head. IB marks a gathered node as claimed by the sweep, since its connect and shutdown paths move the node too.
Patch 8 hands out real references everywhere a connection pointer previously escaped bare, RCU-annotates parent->c_passive, refuses to revive a passive connection whose destroy has begun, and serializes the SIOCRDSSETTOS check with the rs_conn cache.
Based on UEK commits:
2c8569e4c880 ("net/rds: Add krefs to struct rds_connection")
https://github.com/oracle/linux-uek/commit/2c8569e4c880
0e9e3a72b7f7 ("net/rds: rds_sendmsg must use rs_conn only when not being destroyed").
https://github.com/oracle/linux-uek/commit/0e9e3a72b7f7 Patch 9 has the quiesce purge cp_send_queue under cp_lock, as every adder to that queue holds it. No sender can be in flight at any of today's destroy triggers (netns teardown, module unload), so this is hygiene rather than a race fix, and the v5 refusals in the senders that guarded against that impossible state are gone.
Patch 10 pins the connection across the RDMA-CM event handler and rejects a connect request for a connection whose destroy has already quiesced it.
Patch 11 drops the now-unused global rds_conn_count.
Based on UEK commits:
905ec90e6166 ("net/rds: Each RDS transport should keep its own connection count").
https://github.com/oracle/linux-uek/commit/905ec90e6166
Patch 12 makes struct rds_incoming hold a reference on i_conn, the
fix for the KASAN use-after-free Chengfeng Ye reported [4].
Based on UEK commits:
99b9a3715419 ("net/rds: fix crash by expanding kref coverage to rds_incoming.i_conn").
https://github.com/oracle/linux-uek/commit/99b9a3715419
Patches 5, 8, 6 and 12 are ports of the connection kref work Sharath Srinivasan did for Oracle UEK, adapted to the upstream code.
The series has been validated with the RDS selftests over loopback-TCP and RXE-RDMA, plus churn tests that delete network namespaces and unload the modules under live rds-stress traffic.
Changes since v5 [7]:
- Rebased; the version-mismatch destroy is now an rds_conn_drop() in net-next (f97d8c7bab78), so patch 4's Fixes: tag is gone and its changelog describes what still needs the per-connection flag, and the CM-callback deadlock reports against earlier versions no longer apply.
- Patch 2: described as the refactor it is (the passive twin is IB-only, single path); Fixes: tag dropped.
- Patch 9: reduced to the cp_lock purge; the send-side refusals and the negative *queued signalling are dropped, since no sender can be in flight at a destroy trigger.
- Patch 6: wait comment describes the initial-sweep-plus-resweep contract; changelog notes the guarantee is complete only once incs hold references.
- Patch 8: c_destroy_in_prog read through rds_destroy_pending() in __rds_conn_create(); the c_passive puts documented as possibly the last; READ_ONCE() on the unlocked rs_tos sample; rs_conn comment notes the rds_release() exception.
- Patch 10: stale forward-reference comment fixed.
Changes since v4 [6]:
- Patch 7: the IB sweep marks each gathered node with a new i_ib_node_detached flag instead of relying on list emptiness. A node parked on the sweep's stack list is not empty, so a concurrent rds_ib_add_conn() would have unlinked it from under the sweep's lockless walk; rds_ib_add_conn(), rds_ib_remove_conn() and rds_ib_conn_free() now leave a claimed node alone.
Changes since v3 [3]:
- Rebased; the version-mismatch destroy that patch 10 also had to cope with is now an rds_conn_drop() in net (f97d8c7bab78), and patch 10's changelog says what remains for it to cover.
- The uninitialised transport pointer in the CM event handler that the second v2 review pass raised turned out to be the bug Aohan Mei already posted a fix for (v2 at [5], stalled after review); it is carried forward separately for net rather than added here.
- Patch 7: the teardown walks take a reference on each gathered connection, so a connection destroyed earlier and freed by a pending holder cannot vanish under the iterator, and the two IB list movers tolerate a node the sweep already unlinked instead of BUG_ON()ing; the nodev sweep gathers entry by entry, so the resweep from the unload wait no longer relies on list_splice_init().
- Patch 8: SIOCRDSSETTOS check-then-act closed - the install in rds_sendmsg() re-checks the socket's ToS under rs_lock; rs_conn documented as a referenced, rs_lock-serialised cache; contract comment restored to rds_conn_lookup().
- Patch 9: rds_send_probe() gets the same rds_destroy_pending() test under cp_lock as rds_send_queue_rm() (a probe queued after the purge pinned the connection for good).
- v3 patch 10 (rds_tcp_accept_one() destroy check) dropped: the check was not serialised against the destroy, and the race it aimed at is not reachable - every TCP destroy path stops the listener, flushing the accept work, first. A comment now records that ordering.
- Changelog and comment corrections from the second v2 review pass (self-requeue exemption in patch 3, c_refcount comment in patch 5, the uninterruptible wait and the transport-text wake in patch 6, "lock-free" wording in patch 11, cross-netns comment in patch 12).
Changes since v2 [2]:
- New patches 1 and 2: pre-existing rds_ib_conn_free() interrupt state clobber and passive-path transport data leak, surfaced by review of the teardown changes.
- Patch 6: rds_ib_destroy_nodev_conns() uses list_splice_init(), so the resweep from the unload wait cannot splice a stale list head.
- Patch 8: SIOCRDSGETTOS reads rs_tos under rs_lock like SETTOS.
- New patch 9: rds_send_queue_rm() refuses a connection whose destroy has begun, under cp_lock, and the quiesce purges cp_send_queue under cp_lock (list corruption against an in-flight sender, and a message that would pin the connection forever).
- New patch 10: rds_tcp_accept_one() does not install a socket on a connection whose destroy has begun (socket left pointing at a freed path).
Changes since v1 [1]:
- New patch 1: guard the five work-arming sites that never tested rds_destroy_pending(); patch 2's changelog and comments narrowed to what it actually newly covers.
- Patch 4 moved ahead of the reference holders, so no bisect point has references without the unload wait; wait made unbounded with a periodic warning instead of a 10 s timeout; IB exit re-sweeps the nodev list for connections that detach from their device late.
- New patch 5: transport nodes unlinked before destroy (stack list head use-after-free from a deferred conn_free).
- Patch 6: c_passive RCU-annotated; a destroyed passive child is refused by __rds_conn_create() and clears the parent's pointer itself; SIOCRDSSETTOS uses rs_lock; lookup comment reworded; explicit not-for-stable note.
- New patch 7: reference across the CM event handler, and a destroy-pending re-check in rds_ib_cm_handle_connect().
- Patch 9: changelog states what a lingering inc keeps alive and that it blocks module unload.
- Changelog corrections throughout (netns teardown paths named as the non-rmmod destroyers, stale rds_conn_path_destroy() reference).
[1] https://lore.kernel.org/netdev/[email protected]/
[2] https://lore.kernel.org/netdev/[email protected]/
[3] https://lore.kernel.org/netdev/[email protected]/
[4] https://lore.kernel.org/netdev/[email protected]/
[5] https://lore.kernel.org/netdev/[email protected]/
[6] https://lore.kernel.org/netdev/[email protected]/
[7] https://lore.kernel.org/netdev/[email protected]/
Allison
Allison Henderson (8):
net/rds: ib: don't enable interrupts in rds_ib_conn_free() net/rds: free every path's transport data on the passive create paths net/rds: guard every work-requeueing site with rds_destroy_pending() net/rds: make rds_destroy_pending() report a connection's own destroy net/rds: unlink transport nodes before a possibly deferred connection
free
net/rds: take cp_lock to purge cp_send_queue in the quiesce net/rds: pin the connection across RDMA-CM event handling net/rds: drop rds_conn_count in favor of t_conn_count
Sharath Srinivasan (4):
net/rds: split connection destroy into quiesce and kref-governed free net/rds: wait for connections to be freed on transport unload net/rds: hold connection references in lookup, sockets and c_passive net/rds: hold a connection reference from struct rds_incoming
net/rds/af_rds.c | 24 ++- net/rds/connection.c | 330 +++++++++++++++++++++++++++++++++------ net/rds/ib.c | 22 ++- net/rds/ib.h | 4 + net/rds/ib_cm.c | 29 +++- net/rds/ib_rdma.c | 67 ++++++-- net/rds/ib_recv.c | 6 +- net/rds/ib_send.c | 18 ++- net/rds/loop.c | 61 ++++++-- net/rds/message.c | 16 +- net/rds/rdma_transport.c | 16 +- net/rds/rds.h | 46 +++++- net/rds/recv.c | 26 ++- net/rds/send.c | 68 +++++++- net/rds/tcp.c | 53 ++++++- net/rds/tcp_listen.c | 21 ++- 16 files changed, 690 insertions(+), 117 deletions(-)
Signature · 1 line
2.25.1
Allison Henderson
maintainerPATCH v6 1/12 net/rds: ib: don't enable interrupts in rds_ib_conn_free()
rds_ib_conn_free() unlinks the connection from its device or nodev list under spin_lock_irq()/spin_unlock_irq(). It is not only called from the rmmod path, though: __rds_conn_create() calls trans->conn_free() to undo a lost creation race while it still holds rds_conn_lock, taken with spin_lock_irqsave(). The unconditional spin_unlock_irq() then re-enables interrupts with rds_conn_lock held and leaves them enabled when the caller's spin_unlock_irqrestore() runs, defeating the irqsave the caller relied on.
Use the irqsave/irqrestore pair, as rds_tcp_conn_free() and rds_loop_conn_free() already do.
--- net/rds/ib_cm.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)
net/rds/ib_cm.c +7 -2
@@ -1271,6 +1271,7 @@ void rds_ib_conn_free(void *arg) {struct rds_ib_connection *ic = arg;spinlock_t *lock_ptr; +unsigned long flags;rdsdebug ("ic %p\n" , ic);
@@ -1278,12 +1279,16 @@ void rds_ib_conn_free(void *arg)* Conn is either on a dev's list or on the nodev list. * A race with shutdown() or connect() would cause problems * (since rds_ibdev would change) but that should never happen. +* +* Callers may hold rds_conn_lock with interrupts disabled +* (__rds_conn_create() undoing a lost creation race), so do not +* re-enable interrupts unconditionally here. */ lock_ptr = ic->rds_ibdev ? &ic->rds_ibdev->spinlock : &ib_nodev_conns_lock; -spin_lock_irq (lock_ptr); +spin_lock_irqsave (lock_ptr, flags);list_del (&ic->ib_node); -spin_unlock_irq (lock_ptr); +spin_unlock_irqrestore (lock_ptr, flags);rds_ib_recv_free_caches (ic);
Signature · 1 line
2.25.1
[email protected]
AI reviewRe: [PATCH net-next v6 01/12] net/rds: ib: don't enable interrupts in rds_ib_conn_free()
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 9 lines
rds_ib_conn_free()unlinks the connection from its device or nodev list under spin_lock_irq()/spin_unlock_irq(). It is not only called from the rmmod path, though:__rds_conn_create()callstrans->conn_free()to undo a lost creation race while it still holdsrds_conn_lock, taken withspin_lock_irqsave(). The unconditionalspin_unlock_irq()then re-enables interrupts withrds_conn_lockheld and leaves them enabled when the caller'sspin_unlock_irqrestore()runs, defeating the irqsave the caller relied on.
Quoted 6 lines
Use the irqsave/irqrestore pair, as
rds_tcp_conn_free()andrds_loop_conn_free()already do.Fixes 745cbccac3fe ("RDS: Rewrite connection cleanup, fixing oops on rmmod")Assisted-by Claude-Code:claude-fable-5Signed-off-by Allison Henderson <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Allison Henderson
maintainerPATCH v6 2/12 net/rds: undo conn_alloc() the same way on every __rds_conn_create() exit
In reply to Allison Hendersontrans->conn_alloc() may allocate transport data for every path of a multipath connection - rds_tcp_conn_alloc() does - which is why the lost-creation-race exit of __rds_conn_create() loops over all npaths when it frees the connection it just built. The passive-connection exit right above it frees only path 0.
That is not a leak today: a passive twin is only created for an IB loopback connection (an incoming TCP connect to a local address is refused with -EOPNOTSUPP before it gets here), and the IB transport is not multipath, so npaths is 1 on that exit. But the two exits express the same "undo conn_alloc()" step in two different ways, and the following patches add another exit of the same kind. Move the loop into a helper and use it everywhere, so that the step cannot silently diverge if a multipath transport ever grows a passive twin.
--- net/rds/connection.c | 31 ++++++++++++++++++------------- 1 file changed, 18 insertions(+), 13 deletions(-)
net/rds/connection.c +18 -13
@@ -161,6 +161,22 @@ static void __rds_conn_path_init(struct rds_connection *conn, cp->cp_flags =0 ; } +/* Undo trans->conn_alloc(): it may have allocated transport data for +* every path of a multipath connection, not just for path 0. +*/ +static void rds_conn_free_transport_data (struct rds_connection *conn, +int npaths) +{ +struct rds_conn_path *cp; +int i; + +for (i =0 ; i < npaths; i++) { + cp = &conn->c_path[i]; +if (cp->cp_transport_data) + conn->c_trans->conn_free (cp->cp_transport_data); + } +} +/* * There is only every one 'conn' for a given pair of addresses in the * system at a time. They contain messages to be retransmitted and so
@@ -316,7 +332,7 @@ static struct rds_connection *__rds_conn_create(struct net *net,if (parent) {/* Creating passive conn */ if (parent->c_passive) { - trans->conn_free (conn->c_path[0 ].cp_transport_data); +rds_conn_free_transport_data (conn, npaths); free_cp = conn->c_path;kmem_cache_free (rds_conn_slab, conn); conn = parent->c_passive;
@@ -332,18 +348,7 @@ static struct rds_connection *__rds_conn_create(struct net *net, found =rds_conn_lookup (net, head, laddr, faddr, trans, tos, dev_if);if (found) { -struct rds_conn_path *cp; -int i; - -for (i =0 ; i < npaths; i++) { - cp = &conn->c_path[i]; -/* The ->conn_alloc invocation may have -* allocated resource for all paths, so all -* of them may have to be freed here. -*/ -if (cp->cp_transport_data) - trans->conn_free (cp->cp_transport_data); - } +rds_conn_free_transport_data (conn, npaths); free_cp = conn->c_path;kmem_cache_free (rds_conn_slab, conn); conn = found;
Signature · 1 line
2.25.1
[email protected]
AI reviewRe: [PATCH net-next v6 02/12] net/rds: undo conn_alloc() the same way on every __rds_conn_create() exit
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 15 lines
trans->conn_alloc()may allocate transport data for every path of a multipath connection -rds_tcp_conn_alloc()does - which is why the lost-creation-race exit of__rds_conn_create()loops over all npaths when it frees the connection it just built. The passive-connection exit right above it frees only path 0.That is not a leak today: a passive twin is only created for an IB loopback connection (an incoming TCP connect to a local address is refused with
-EOPNOTSUPPbefore it gets here), and the IB transport is not multipath, so npaths is 1 on that exit. But the two exits express the same "undoconn_alloc()" step in two different ways, and the following patches add another exit of the same kind. Move the loop into a helper and use it everywhere, so that the step cannot silently diverge if a multipath transport ever grows a passive twin.
Quoted 2 lines
Assisted-by Claude-Code:claude-fable-5Signed-off-by Allison Henderson <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
Allison Henderson
maintainerPATCH v6 3/12 net/rds: guard every work-requeueing site with rds_destroy_pending()
In reply to Allison Hendersonrds_conn_destroy() cancels the path works and then destroys the per-path workqueue. The sites that can re-arm those works are supposed to test rds_destroy_pending() under rcu_read_lock() first, paired with the synchronize_rcu() in the destroy path, so that no new work can be queued once the cancellation has begun.
Five arming sites never got that guard:
- rds_ib_send_cqe_handler() and rds_ib_send_add_credits() re-arm cp_send_w when a send completion or a credit update clears RDS_LL_SEND_FULL,
- rds_ib_recv_refill() re-arms cp_recv_w when the recv ring runs low,
- rds_tcp_accept_one() kicks cp_recv_w on the freshly accepted socket, and
- rds_sendmsg() arms cp_conn_w for a multipath connection whose path 0 is not up yet.
The IB completion sites are reachable from soft-irq at any point before the QP is drained, so a completion landing in the window between the cancel and destroy_workqueue() in rds_conn_path_destroy() re-arms a work on a workqueue that is about to be destroyed: with delay 0 the work is queued directly on the freed workqueue, and with delay 1 the timer survives destroy_workqueue() unseen and fires afterwards, queueing from a timer_list that lives in the freed c_path array.
Wrap all five sites in the same rcu_read_lock() + rds_destroy_pending() pattern the other arming sites already use.
The four self-requeues in rds_send_worker() and rds_recv_worker() are left alone on purpose: they run from inside the work item itself, and cancel_delayed_work_sync() disables the work for the duration of the cancel, so a requeue issued by the still-running callback is dropped and none can follow once the cancel has returned.
With the predicate as it stands the guards cover the netns teardown and module unload cases; the following patch extends it to the destroy of a single connection.
--- net/rds/ib_recv.c | 6 +++++- net/rds/ib_send.c | 18 ++++++++++++++---- net/rds/send.c | 11 ++++++++--- net/rds/tcp_listen.c | 10 +++++++--- 4 files changed, 34 insertions(+), 11 deletions(-)
net/rds/ib_recv.c +5 -1
@@ -458,7 +458,11 @@ void rds_ib_recv_refill(struct rds_connection *conn, int prefill, gfp_t gfp) (must_wake || (can_wait && rds_ib_ring_low(&ic->i_recv_ring)) || rds_ib_ring_empty(&ic->i_recv_ring))) { -queue_delayed_work (conn->c_path->cp_wq, &conn->c_recv_w,1 ); +rcu_read_lock (); +if (!rds_destroy_pending (conn)) +queue_delayed_work (conn->c_path->cp_wq, + &conn->c_recv_w,1 ); +rcu_read_unlock (); }if (can_wait)cond_resched ();
net/rds/ib_send.c +14 -4
@@ -298,8 +298,13 @@ void rds_ib_send_cqe_handler(struct rds_ib_connection *ic, struct ib_wc *wc)rds_ib_sub_signaled (ic, nr_sig);if (test_and_clear_bit (RDS_LL_SEND_FULL , &conn->c_flags) || -test_bit (0 , &conn->c_map_queued)) -queue_delayed_work (conn->c_path->cp_wq, &conn->c_send_w,0 ); +test_bit (0 , &conn->c_map_queued)) { +rcu_read_lock (); +if (!rds_destroy_pending (conn)) +queue_delayed_work (conn->c_path->cp_wq, + &conn->c_send_w,0 ); +rcu_read_unlock (); + }/* We expect errors as the qp is drained during shutdown */ if (wc->status !=IB_WC_SUCCESS &&rds_conn_up (conn)) {
@@ -420,8 +425,13 @@ void rds_ib_send_add_credits(struct rds_connection *conn, unsigned int credits) test_bit(RDS_LL_SEND_FULL, &conn->c_flags) ? ", ll_send_full" : "");atomic_add (IB_SET_SEND_CREDITS (credits), &ic->i_credits); -if (test_and_clear_bit (RDS_LL_SEND_FULL , &conn->c_flags)) -queue_delayed_work (conn->c_path->cp_wq, &conn->c_send_w,0 ); +if (test_and_clear_bit (RDS_LL_SEND_FULL , &conn->c_flags)) { +rcu_read_lock (); +if (!rds_destroy_pending (conn)) +queue_delayed_work (conn->c_path->cp_wq, + &conn->c_send_w,0 ); +rcu_read_unlock (); + }WARN_ON (IB_GET_SEND_CREDITS (credits) >=16384 );
net/rds/send.c +8 -3
@@ -1378,9 +1378,14 @@ int rds_sendmsg(struct socket *sock, struct msghdr *msg, size_t payload_len)* outstanding. */ if (!test_and_set_bit (RDS_RECONNECT_PENDING , - &conn->c_path[0 ].cp_flags)) -queue_delayed_work (conn->c_path[0 ].cp_wq, - &conn->c_path[0 ].cp_conn_w,0 ); + &conn->c_path[0 ].cp_flags)) { +rcu_read_lock (); +if (!rds_destroy_pending (conn)) +queue_delayed_work (conn->c_path[0 ].cp_wq, + &conn->c_path[0 ].cp_conn_w, +0 ); +rcu_read_unlock (); + }rds_send_ping (conn,0 ); }
net/rds/tcp_listen.c +7 -3
@@ -316,10 +316,14 @@ int rds_tcp_accept_one(struct rds_tcp_net *rtn)*/ if (READ_ONCE (sk->sk_state) ==TCP_CLOSE_WAIT ||READ_ONCE (sk->sk_state) ==TCP_LAST_ACK || -READ_ONCE (sk->sk_state) ==TCP_CLOSE ) +READ_ONCE (sk->sk_state) ==TCP_CLOSE ) {rds_conn_path_drop (cp,0 ); -else -queue_delayed_work (cp->cp_wq, &cp->cp_recv_w,0 ); + }else { +rcu_read_lock (); +if (!rds_destroy_pending (cp->cp_conn)) +queue_delayed_work (cp->cp_wq, &cp->cp_recv_w,0 ); +rcu_read_unlock (); + }sock_put (sk);
Signature · 1 line
2.25.1
[email protected]
AI reviewRe: [PATCH net-next v6 03/12] net/rds: guard every work-requeueing site with rds_destroy_pending()
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 18 lines
rds_conn_destroy()cancels the path works and then destroys the per-path workqueue. The sites that can re-arm those works are supposed to testrds_destroy_pending()underrcu_read_lock()first, paired with thesynchronize_rcu()in the destroy path, so that no new work can be queued once the cancellation has begun.Five arming sites never got that guard:
-
rds_ib_send_cqe_handler()andrds_ib_send_add_credits()re-armcp_send_wwhen a send completion or a credit update clearsRDS_LL_SEND_FULL,
-rds_ib_recv_refill()re-armscp_recv_wwhen the recv ring runs low,
-rds_tcp_accept_one()kickscp_recv_won the freshly accepted socket, and
[ ... ]
single connection.
Quoted 3 lines
Fixes ebeeb1ad9b8a ("rds: tcp: use rds_destroy_pending() to synchronize netns/module teardown and rds connection/workq management")Assisted-by Claude-Code:claude-fable-5Signed-off-by Allison Henderson <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
Allison Henderson
maintainerPATCH v6 4/12 net/rds: make rds_destroy_pending() report a connection's own destroy
In reply to Allison Hendersonrds_conn_destroy() cancels the path works and then destroys the per-path workqueue. However, nothing currently stops the work-requeueing sites from queueing new work on the connection while that happens. The existing code would suggest that this protection is supposed to come from rds_destroy_pending(), since all of those sites - apart from the workers' own self-requeues, which the sync cancel in the destroy path already rejects - guard the queueing with rds_destroy_pending() under rcu_read_lock() (the last stragglers were converted by the previous patch), and rds_conn_destroy() already issues a synchronize_rcu() after unhashing the connection. But the predicate only tests for the two global teardown cases (netns destruction via check_net(), module unload via ->t_unloading). Because the conn itself lacks any indication that a destroy is in progress, the predicate does not cover the destruction of a single connection outside these two cases.
Today every rds_conn_destroy() does happen on one of those two global paths - the last single-connection caller, the protocol-version mismatch in rds_ib_cm_connect_complete(), was turned into a drop by commit f97d8c7bab78 ("rds: ib: use rds_conn_drop() on protocol version mismatch") - so the predicate is currently never wrong. It is also never precise: it answers "is this connection's world going away", not "is this connection being destroyed", and the following patches need the second answer. Once the free is deferred to the last reference, a connection can be handed to rds_conn_destroy() more than once (the IB unload path re-sweeps its list until every connection is gone) and must recognise its own destroy in progress; the passive-twin creation must refuse a parent whose destroy has begun; and a future single-connection destroy - a hot-unplugged IB device, or the asynchronous teardown the Oracle UEK kernel has - would re-open the window below. While a single-connection destroy runs, a concurrent rds_cong_queue_updates() can
still find the connection on the congestion map's m_conn_list (the conn is only removed from it after the paths are torn down) and call queue_delayed_work() on a cp_wq that destroy_workqueue() has already freed. Additionally, the other requeueing sites can likewise re-arm works that live in the about-to-be-freed connection unless rds_destroy_pending() has something to guard it with.
The version-mismatch path used to be covered: commit c90ecbfaf50d2 ("rds: Use atomic flag to track connections being destroyed") introduced the RDS_DESTROY_PENDING cp_flags bit for exactly this, and after commit ebeeb1ad9b8ad ("rds: tcp: use rds_destroy_pending() to synchronize netns/module teardown and rds connection/workq management") it was set right before that rds_conn_destroy() call and tested via rds_ib_is_unloading(). Commit cdc306a5c9cd3 ("rds: make v3.1 as compat version") then removed the last set_bit while leaving the test behind, so the bit has been dead ever since and per-conn destroy has run unguarded.
Record the destroy on the connection itself: set conn->c_destroy_in_prog before the unhash + synchronize_rcu() sequence in rds_conn_destroy() and test it first in rds_destroy_pending(). The existing rcu_read_lock() around every check-and-queue site pairs with that synchronize_rcu(): once it returns, every new reader observes the flag and refuses to queue, and anything queued before it is flushed or cancelled by the existing teardown. Drop the now-unreferenced RDS_DESTROY_PENDING bit and its dead test.
In the Oracle UEK kernel the equivalent conn->c_destroy_in_prog flag is part of the larger connection refcounting rework ("net/rds: Add krefs to struct rds_connection"), including ("net/rds: Merge uses of conn->c_destroy_in_prog & RDS_DESTROY_PENDING"). This ports the missing pieces of the requeue guard, which stand on their own.
--- net/rds/connection.c | 8 ++++++++ net/rds/ib.c | 5 +---- net/rds/rds.h | 12 ++++++++++-- 3 files changed, 19 insertions(+), 6 deletions(-)
net/rds/connection.c +8 -0
@@ -579,6 +579,14 @@ void rds_conn_destroy(struct rds_connection *conn)"%pI4\n" , conn, &conn->c_laddr, &conn->c_faddr); +/* Make rds_destroy_pending() true for this conn. Together with +* the synchronize_rcu() below this stops the work-requeueing +* sites (which all test rds_destroy_pending() under +* rcu_read_lock()) from queueing new work on the path +* workqueues once we start cancelling and destroying them. +*/ +WRITE_ONCE (conn->c_destroy_in_prog, true); +/* Ensure conn will not be scheduled for reconnect */ spin_lock_irq (&rds_conn_lock);hlist_del_init_rcu (&conn->c_hash_node);
net/rds/ib.c +1 -4
@@ -525,10 +525,7 @@ static void rds_ib_set_unloading(void)static bool rds_ib_is_unloading (struct rds_connection *conn) { -struct rds_conn_path *cp = &conn->c_path[0 ]; - -return (test_bit (RDS_DESTROY_PENDING , &cp->cp_flags) || -atomic_read (&rds_ib_unloading) !=0 ); +return atomic_read (&rds_ib_unloading) !=0 ; }void rds_ib_exit (void )
net/rds/rds.h +10 -2
@@ -89,7 +89,6 @@ enum {#define RDS_RECONNECT_PENDING 1#define RDS_IN_XMIT 2#define RDS_RECV_REFILL 3 -#define RDS_DESTROY_PENDING 4/* Max number of multipaths per RDS connection. Must be a power of 2 */ #define RDS_MPATH_WORKERS 8
@@ -148,6 +147,14 @@ struct rds_connection { c_pad_to_32:29 ;int c_npaths;bool c_with_sport_idx; +/* Set once, by rds_conn_destroy(), before it cancels the path +* works; read through rds_destroy_pending(). A site that arms +* a path work must test the predicate and queue the work inside +* one rcu_read_lock() section: the synchronize_rcu() that +* follows the store is what keeps a queue issued after the +* cancellation from landing on a destroyed workqueue. +*/ +bool c_destroy_in_prog;struct rds_connection *c_passive;struct rds_transport *c_trans;
@@ -994,7 +1001,8 @@ void __rds_put_mr_final(struct kref *kref);static inline bool rds_destroy_pending (struct rds_connection *conn) { -return !check_net (rds_conn_net (conn)) || +return READ_ONCE (conn->c_destroy_in_prog) || + !check_net (rds_conn_net (conn)) || (conn->c_trans->t_unloading && conn->c_trans->t_unloading (conn)); }
Signature · 1 line
2.25.1
[email protected]
AI reviewRe: [PATCH net-next v6 04/12] net/rds: make rds_destroy_pending() report a connection's own destroy
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 18 lines
rds_conn_destroy()cancels the path works and then destroys the per-path workqueue. However, nothing currently stops the work-requeueing sites from queueing new work on the connection while that happens. The existing code would suggest that this protection is supposed to come fromrds_destroy_pending(), since all of those sites - apart from the workers' own self-requeues, which the sync cancel in the destroy path already rejects - guard the queueing withrds_destroy_pending()underrcu_read_lock()(the last stragglers were converted by the previous patch), andrds_conn_destroy()already issues asynchronize_rcu()after unhashing the connection. But the predicate only tests for the two global teardown cases (netns destruction viacheck_net(), module unload via ->t_unloading). Because the conn itself lacks any indication that a destroy is in progress, the predicate does not cover the destruction of a single connection outside these two cases.
[ ... ]
missing pieces of the requeue guard, which stand on their own.
Quoted 3 lines
Suggested-by Sharath Srinivasan <[email protected]>Assisted-by Claude-Code:claude-fable-5Signed-off-by Allison Henderson <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
[email protected]
AI reviewRe: [PATCH net-next v6 04/12] net/rds: make rds_destroy_pending() report a connection's own destroy
In reply to Allison HendersonThank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider.
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider.
Critical: 0 · High: 0 (1) · Medium: 1 · Low: 1
If you believe none of the issues are legitimate, please reply to each issue of Medium severity or higher and explain why the AI is wrong. If any of them are legitimate and you plan to respin, please let patchwork know by including "pw-bot: cr" as a separate line at the end of your reply (one such reply per series is enough).
- [Medium] The invariant this patch documents is contradicted by in-tree arming sites, and the gap covers exactly the future case the commit…
- [Low] Commit message factual error: the justification "Today every rds_conn_destroy() does happen on one of those two global paths ... so…
Pre-existing issues:
- [High] This problem wasn't introduced by this patch, but it is a permanent, repeatable leak of 7 ordered workqueues per loopback…
--
net/rds: make rds_destroy_pending() report a connection's own destroy
This adds a per-connection c_destroy_in_prog flag, set by rds_conn_destroy() before the unhash plus synchronize_rcu() sequence, and tests it first in rds_destroy_pending() so the predicate reports a single connection's own destroy. The now-unreferenced RDS_DESTROY_PENDING cp_flags bit and its dead test in rds_ib_is_unloading() are removed.
A few questions below, first about the commit message, then about the new comments and the surrounding teardown code.
Quoted 5 lines
Today every
rds_conn_destroy()does happen on one of those two global paths - the last single-connection caller, the protocol-version mismatch inrds_ib_cm_connect_complete(), was turned into a drop by commitf97d8c7bab78("rds: ib: userds_conn_drop()on protocol version mismatch") - so the predicate is currently never wrong.
[Severity: Low]
Is "the predicate is currently never wrong" true for the loopback transport at module unload? rds_conn_exit() runs the pernet unregister before the loop transport marks itself unloading:
net/rds/connection.c:rds_conn_exit() {
rds_loop_net_exit(); /* unregister pernet callback */
rds_loop_exit();
...
}rds_loop_net_exit() -> unregister_pernet_device() runs the .exit hook for every net still on net_namespace_list, including init_net, so rds_loop_exit_net() -> rds_loop_kill_conns() -> rds_conn_destroy() runs with check_net() still true for those conns.
rds_loop_set_unloading() is the only writer of the flag that t_unloading reports for the loop transport, and it is not called until rds_loop_exit(), which runs afterwards:
net/rds/loop.c:rds_loop_exit() {
rds_loop_set_unloading();
synchronize_rcu();
...
}So for those loop connections the pre-patch rds_destroy_pending() is false exactly while rds_conn_destroy() cancels the path works and calls destroy_workqueue() on cp_wq, which would make this patch also close a narrow real window rather than being purely preparatory. Would it make sense either to qualify the claim by naming this loop ordering, or to set the loop unloading flag before unregister_pernet_device()?
Quoted 6 lines
net/rds/connection.c +0
-0@@ -579,6 +579,14 @@ void rds_conn_destroy(struct rds_connection *conn) "%pI4\n", conn, &conn->c_laddr,
net/rds/connection.c void rds_conn_destroy(struct rds_connection *conn) · line 589
Quoted 10 lines
net/rds/connection.c +8
-0&conn->c_faddr); +/* Make rds_destroy_pending() true for this conn. Together with +* the synchronize_rcu() below this stops the work-requeueing +* sites (which all test rds_destroy_pending() under +* rcu_read_lock()) from queueing new work on the path +* workqueues once we start cancelling and destroying them. +*/ +WRITE_ONCE (conn->c_destroy_in_prog, true); +
[Severity: Medium]
Is the parenthetical "which all test rds_destroy_pending() under rcu_read_lock()" accurate? Two classes of arming sites do not.
rds_conn_path_drop() short-circuits the predicate when destroy is true:
net/rds/connection.c:rds_conn_path_drop() {
rcu_read_lock();
if (!destroy && rds_destroy_pending(cp->cp_conn)) {
rcu_read_unlock();
return;
}
queue_work(cp->cp_wq, &cp->cp_down_w);
rcu_read_unlock();
}rds_ib_dev_shutdown() uses exactly that form for every connection on a device's conn_list:
net/rds/ib.c:rds_ib_dev_shutdown() {
list_for_each_entry(ic, &rds_ibdev->conn_list, ib_node)
rds_conn_path_drop(&ic->conn->c_path[0], true);
}That is the hot-unplugged IB device case the commit message names as a reason for the new flag, and the new flag does not cover it, while rds_conn_path_destroy() later does:
net/rds/connection.c:rds_conn_path_destroy() {
if (cp->cp_wq != rds_wq) {
destroy_workqueue(cp->cp_wq);
cp->cp_wq = NULL;
}
...
}The second class is the workers' own requeues, which use neither the predicate nor an RCU section:
net/rds/threads.c:rds_send_worker() {
case -EAGAIN:
rds_stats_inc(s_send_immediate_retry);
queue_delayed_work(cp->cp_wq, &cp->cp_send_w, 0);
...
}The same shape appears for cp_recv_w in rds_recv_worker(). The commit message calls out the self-requeue exception, but the new comment here and the new comment on c_destroy_in_prog state the rule without it.
Would it be better to weaken both comments so they name the destroy == true and self-requeue exceptions, or to serialize rds_conn_path_drop(cp, true) against the workqueue teardown so the stated invariant actually holds?
[Severity: High]
This isn't a bug introduced by this patch, but since rds_conn_destroy() is being touched here: does this path leak the per-path workqueues of a loopback-substituted connection?
__rds_conn_create() computes npaths from the caller's transport, then substitutes the loop transport without recomputing it:
net/rds/connection.c:__rds_conn_create() {
int npaths = (trans->t_mp_capable ? RDS_MPATH_WORKERS : 1);
...
trans = &rds_loop_transport;
...
conn->c_trans = trans;
...
for (i = 0; i < npaths; i++) {
...
conn->c_path[i].cp_wq =
alloc_ordered_workqueue("krds_cp_wq#%lu/%d", 0,
rds_conn_count, i);
...
}rds_tcp_transport sets both t_prefer_loopback and t_mp_capable, so npaths is 8 and eight ordered workqueues are allocated. rds_loop_transport sets neither, so the destroy side recomputes 1:
net/rds/connection.c:rds_conn_destroy() {
int npaths = (conn->c_trans->t_mp_capable ? RDS_MPATH_WORKERS : 1);
...
for (i = 0; i < npaths; i++) {
cp = &conn->c_path[i];
rds_conn_path_destroy(cp);
...
kfree(conn->c_path);
...
}Paths 1 through 7 never reach destroy_workqueue(cp->cp_wq), and kfree(conn->c_path) then drops the only pointers to those seven workqueue_structs, which stay on the global workqueue list.
Note that just fixing npaths does not appear to be enough, because rds_loop_conn_alloc() only fills c_path[0].cp_transport_data and rds_conn_path_destroy() returns early:
net/rds/connection.c:rds_conn_path_destroy() {
if (!cp->cp_transport_data)
return;
...
}so the cp_wq teardown would also have to move out of that guard.
Trigger: in a netns, create an AF_RDS socket, bind a local address and sendmsg() to a local RDS address so the loop substitution applies, then destroy the netns (rds_loop_exit_net() -> rds_loop_kill_conns() -> rds_conn_destroy()) or unload rds. Each such connection destroy leaks seven workqueues.
This also survives to the end of the series: rds_conn_destroy_fini() uses the same npaths formula and rds_conn_path_free() keeps the !cp->cp_transport_data early return.
Quoted 3 lines
net/rds/connection.c +0
-0/* Ensure conn will not be scheduled for reconnect */ spin_lock_irq (&rds_conn_lock);hlist_del_init_rcu (&conn->c_hash_node);
[ ... ]
Quoted 6 lines
net/rds/rds.h +0
-0@@ -148,6 +147,14 @@ struct rds_connection { c_pad_to_32:29 ;
net/rds/rds.h struct rds_connection { · line 157
Quoted 10 lines
net/rds/rds.h +8
-0int c_npaths;bool c_with_sport_idx; +/* Set once, by rds_conn_destroy(), before it cancels the path +* works; read through rds_destroy_pending(). A site that arms +* a path work must test the predicate and queue the work inside +* one rcu_read_lock() section: the synchronize_rcu() that +* follows the store is what keeps a queue issued after the +* cancellation from landing on a destroyed workqueue. +*/ +bool c_destroy_in_prog;
This is the second half of the comment question above: the rule stated here as unconditional is not followed by rds_conn_path_drop(cp, true) or by the rds_send_worker()/rds_recv_worker() retry requeues.
[ ... ]
Signature · 1 line
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20260922085410.391323-1-achender%40kernel.org
Allison Henderson
maintainerPATCH v6 5/12 net/rds: split connection destroy into quiesce and kref-governed free
In reply to Allison HendersonFrom: Sharath Srinivasan <[email protected]>
rds_conn_destroy() tears down the transport state and immediately frees the connection, along with its paths and its workqueues. This relies on the assumption (documented in the rds_conn_destroy() comments) that "no one else is referencing the connection", which "we can only ensure ... in the rmmod path". However, the callers stopped honoring that long ago. Today, connections are also destroyed on network namespace teardown (rds_tcp_kill_sock() and rds_loop_kill_conns()) and, one at a time, when a peer negotiates an unsupported protocol version (rds_ib_cm_connect_complete()).
This leaves loose ends since references to these destroyed connections still exist. Sockets cache their connections in rs_conn, and congestion updates will walk the maps' m_conn_list. The CM callbacks and workers may also still hold the pointer.
Prepare to close those holes by making the connection refcounted:
- kref_init() the connection in __rds_conn_create(); the initial reference belongs to whoever is responsible for destroying the connection.
- rds_conn_destroy() still quiesces synchronously exactly as before (workers cancelled, paths dropped and shut down, queued messages purged, congestion list removal), but the frees - the transport's conn_free, the path workqueues, the c_path array and the connection slab object - move to rds_conn_destroy_fini(), which runs when the last reference is dropped via rds_conn_put().
- Export rds_conn_get()/rds_conn_put(), plus an inline rds_conn_get_unless_zero() for holders that may find a connection already on its way out, for the reference holders introduced in the following patches.
With no additional reference holders yet, this only sets up the refcounting framework and is functionally equivalent to the current code (the initial reference is the only one), so every free still completes inside rds_conn_destroy(). The next two patches make a deferred free safe - the transport unload wait and the unlinking of the transport nodes ahead of the destroy - and only then do the patches that follow take references at the places that today rely on bare pointers.
Based on the Oracle UEK commit "net/rds: Add krefs to struct rds_connection".
[achender: substantial reimplementation for net-next: UEK's
rds_conn_destroy_init()/_fini() split redone against upstream's rds_conn_destroy()/rds_conn_path_destroy() (no heartbeat/reap/trace infrastructure, no rds_net, single conn hash); destroy keeps its one-call external interface; holder coverage split out into follow-up patches; rewrite commit message]
--- net/rds/connection.c | 96 +++++++++++++++++++++++++++++++++++--------- net/rds/rds.h | 13 ++++++ 2 files changed, 91 insertions(+), 18 deletions(-)
net/rds/connection.c +78 -18
@@ -231,6 +231,7 @@ static struct rds_connection *__rds_conn_create(struct net *net, goto out; } +kref_init (&conn->c_refcount);INIT_HLIST_NODE (&conn->c_hash_node); conn->c_laddr = *laddr; conn->c_isv6 = !ipv6_addr_v4mapped (laddr);
@@ -471,9 +472,10 @@ void rds_conn_shutdown(struct rds_conn_path *cp)* Quiesce the reconnect timer before bailing * out, though. When a pending destroy did * suppress the queue, no later pass runs, and -* rds_conn_path_destroy() is about to flush -* cp_down_w and free the path: it must not -* find cp_conn_w still armed. A successor +* rds_conn_path_quiesce() is about to flush +* cp_down_w, ahead of the path's deferred +* free: it must not find cp_conn_w still +* armed. A successor * pass, when there is one, re-arms the * reconnect from its own tail. */
@@ -520,10 +522,12 @@ void rds_conn_shutdown(struct rds_conn_path *cp) conn->c_trans->conn_slots_available (conn, false); } -/* destroy a single rds_conn_path. rds_conn_destroy() iterates over -* all paths using rds_conn_path_destroy() +/* quiesce a single rds_conn_path: shut it down and tear down any +* queued messages. rds_conn_destroy() iterates over all paths using +* rds_conn_path_quiesce(); the transport state and the workqueue are +* freed later, from rds_conn_path_free(). */ -static void rds_conn_path_destroy (struct rds_conn_path *cp) +static void rds_conn_path_quiesce (struct rds_conn_path *cp) {struct rds_message *rm, *rtmp;
@@ -552,6 +556,16 @@ static void rds_conn_path_destroy(struct rds_conn_path *cp)WARN_ON (delayed_work_pending (&cp->cp_recv_w));WARN_ON (delayed_work_pending (&cp->cp_conn_w));WARN_ON (work_pending (&cp->cp_down_w)); +} + +/* free a quiesced rds_conn_path's transport state and workqueue; runs +* from rds_conn_destroy_fini() once the last connection reference is +* dropped. +*/ +static void rds_conn_path_free (struct rds_conn_path *cp) +{ +if (!cp->cp_transport_data) +return ;if (cp->cp_wq != rds_wq) {destroy_workqueue (cp->cp_wq);
@@ -561,16 +575,52 @@ static void rds_conn_path_destroy(struct rds_conn_path *cp) cp->cp_conn->c_trans->conn_free (cp->cp_transport_data); } +/* Free a connection. This runs from rds_conn_put() when the last +* reference is dropped, after rds_conn_destroy() has quiesced the +* connection and dropped the initial reference. +*/ +static void rds_conn_destroy_fini (struct kref *kref) +{ + struct rds_connection *conn = container_of(kref, struct rds_connection, + c_refcount); +int npaths = (conn->c_trans->t_mp_capable ?RDS_MPATH_WORKERS :1 ); +unsigned long flags; +int i; + +for (i =0 ; i < npaths; i++) +rds_conn_path_free (&conn->c_path[i]); + +kfree (conn->c_path); +kmem_cache_free (rds_conn_slab, conn); + +spin_lock_irqsave (&rds_conn_lock, flags); + rds_conn_count--; +spin_unlock_irqrestore (&rds_conn_lock, flags); +} + +void rds_conn_get (struct rds_connection *conn) +{ +kref_get (&conn->c_refcount); +} +EXPORT_SYMBOL_GPL (rds_conn_get); + +void rds_conn_put (struct rds_connection *conn) +{ +kref_put (&conn->c_refcount, rds_conn_destroy_fini); +} +EXPORT_SYMBOL_GPL (rds_conn_put); +/* * Stop and free a connection. * -* This can only be used in very limited circumstances. It assumes that once -* the conn has been shutdown that no one else is referencing the connection. -* We can only ensure this in the rmmod path in the current code. +* Quiesces the connection synchronously (workers cancelled, transport +* connections shut down, queued messages dropped) and drops the +* initial reference. The memory - including the transport's +* per-connection state and the path workqueues - is freed once the +* last rds_conn_put() runs, which may be after this returns. */ void rds_conn_destroy (struct rds_connection *conn) { -unsigned long flags;int i;struct rds_conn_path *cp;int npaths = (conn->c_trans->t_mp_capable ?RDS_MPATH_WORKERS :1 );
@@ -584,11 +634,23 @@ void rds_conn_destroy(struct rds_connection *conn)* sites (which all test rds_destroy_pending() under * rcu_read_lock()) from queueing new work on the path * workqueues once we start cancelling and destroying them. +* +* Now that the transport state stays discoverable (e.g. on the +* transports' connection lists) until the final rds_conn_put(), +* a conn can be handed to rds_conn_destroy() more than once - +* e.g. dropped for a protocol version mismatch and then found +* again at module unload. Only the first caller proceeds; the +* unhash also happens under rds_conn_lock, so a looked-up conn +* can never be quiesced twice. */ +spin_lock_irq (&rds_conn_lock); +if (conn->c_destroy_in_prog) { +spin_unlock_irq (&rds_conn_lock); +return ; + }WRITE_ONCE (conn->c_destroy_in_prog, true);/* Ensure conn will not be scheduled for reconnect */ -spin_lock_irq (&rds_conn_lock);hlist_del_init_rcu (&conn->c_hash_node);spin_unlock_irq (&rds_conn_lock);synchronize_rcu ();
@@ -596,7 +658,7 @@ void rds_conn_destroy(struct rds_connection *conn)/* shut the connection down */ for (i =0 ; i < npaths; i++) { cp = &conn->c_path[i]; -rds_conn_path_destroy (cp); +rds_conn_path_quiesce (cp);BUG_ON (!list_empty (&cp->cp_retrans)); }
@@ -607,12 +669,10 @@ void rds_conn_destroy(struct rds_connection *conn)*/ rds_cong_remove_conn (conn); -kfree (conn->c_path); -kmem_cache_free (rds_conn_slab, conn); - -spin_lock_irqsave (&rds_conn_lock, flags); - rds_conn_count--; -spin_unlock_irqrestore (&rds_conn_lock, flags); +/* drop the initial reference; the connection is freed from +* rds_conn_destroy_fini() once every holder has dropped theirs +*/ +rds_conn_put (conn); }EXPORT_SYMBOL_GPL (rds_conn_destroy);
net/rds/rds.h +13 -0
@@ -137,6 +137,12 @@ struct rds_conn_path {/* One rds_connection per RDS address pair */ struct rds_connection {struct hlist_node c_hash_node; +/* rds_conn_destroy() quiesces the connection synchronously; +* freeing it - the connection memory, the path workqueues and +* the transport's per-connection state - is deferred until the +* last reference is dropped via rds_conn_put(). +*/ +struct kref c_refcount;struct in6_addr c_laddr;struct in6_addr c_faddr;int c_dev_if;/* ifindex used for this conn */
@@ -826,6 +832,13 @@ struct rds_connection *rds_conn_create_outgoing(struct net *net, u8 tos, gfp_t gfp, int dev_if);void rds_conn_shutdown (struct rds_conn_path *cpath);void rds_conn_destroy (struct rds_connection *conn); +void rds_conn_get (struct rds_connection *conn); +void rds_conn_put (struct rds_connection *conn); +/* take a reference unless the connection is already being freed */ +static inline bool rds_conn_get_unless_zero (struct rds_connection *conn) +{ +return kref_get_unless_zero (&conn->c_refcount); +}void rds_conn_drop (struct rds_connection *conn);void rds_conn_path_drop (struct rds_conn_path *cpath,bool destroy);void rds_conn_connect_if_down (struct rds_connection *conn);
Signature · 1 line
2.25.1
[email protected]
AI reviewRe: [PATCH net-next v6 05/12] net/rds: split connection destroy into quiesce and kref-governed free
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 19 lines
From: Sharath Srinivasan <[email protected]>
rds_conn_destroy()tears down the transport state and immediately frees the connection, along with its paths and its workqueues. This relies on the assumption (documented in therds_conn_destroy()comments) that "no one else is referencing the connection", which "we can only ensure ... in the rmmod path". However, the callers stopped honoring that long ago. Today, connections are also destroyed on network namespace teardown (rds_tcp_kill_sock()andrds_loop_kill_conns()) and, one at a time, when a peer negotiates an unsupported protocol version (rds_ib_cm_connect_complete()).This leaves loose ends since references to these destroyed connections still exist. Sockets cache their connections in
rs_conn, and congestion updates will walk the maps'm_conn_list. The CM [ ... ]
infrastructure, nords_net, single conn hash); destroy keeps its one-call external interface; holder coverage split out into follow-up patches; rewrite commit message]
Quoted 2 lines
Assisted-by Claude-Code:claude-fable-5Signed-off-by Allison Henderson <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5
[email protected]
AI reviewRe: [PATCH net-next v6 05/12] net/rds: split connection destroy into quiesce and kref-governed free
In reply to Allison HendersonThank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider.
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider.
Critical: 0 · High: 0 (1) · Medium: 2 · Low: 1
If you believe none of the issues are legitimate, please reply to each issue of Medium severity or higher and explain why the AI is wrong. If any of them are legitimate and you plan to respin, please let patchwork know by including "pw-bot: cr" as a separate line at the end of your reply (one such reply per series is enough).
- [Medium] The changelog justifies the refcounting rework (and the new one-shot c_destroy_in_prog guard) with a call path that does not…
- [Medium] The new c_destroy_in_prog test in rds_conn_destroy() (net/rds/connection.c:646-651) makes a second or concurrent caller…
- [Low] The rds.h comment documenting c_destroy_in_prog (net/rds/rds.h:156-162) states it is "Set once, by rds_conn_destroy(),…
Pre-existing issues:
- [High] Per-path workqueue leak for loopback-converted connections:
__rds_conn_create() computes npaths from the *incoming* transport…
--
net/rds: split connection destroy into quiesce and kref-governed free
This patch adds a kref to struct rds_connection and splits rds_conn_destroy() into a synchronous quiesce plus a deferred free (rds_conn_destroy_fini()) that runs when the last reference is dropped. rds_conn_get()/rds_conn_put() are exported and a new one-shot c_destroy_in_prog test under rds_conn_lock makes repeated destroy calls return early. No Link: tags are present in the commit header.
A few questions below.
Quoted 4 lines
net/rds/connection.c +0
-0
[ ... ]
Quoted 18 lines
net/rds/connection.c +10
-0@@ -552,6 +556,16 @@ static void rds_conn_path_destroy(struct rds_conn_path *cp)WARN_ON (delayed_work_pending (&cp->cp_recv_w));WARN_ON (delayed_work_pending (&cp->cp_conn_w));WARN_ON (work_pending (&cp->cp_down_w)); +} + +/* free a quiesced rds_conn_path's transport state and workqueue; runs +* from rds_conn_destroy_fini() once the last connection reference is +* dropped. +*/ +static void rds_conn_path_free (struct rds_conn_path *cp) +{ +if (!cp->cp_transport_data) +return ;if (cp->cp_wq != rds_wq) {destroy_workqueue (cp->cp_wq);
net/rds/connection.c static void rds_conn_path_destroy(struct rds_conn_path *cp) · line 577
Quoted 3 lines
net/rds/connection.c +0
-0@@ -561,16 +575,52 @@ static void rds_conn_path_destroy(struct rds_conn_path *cp) cp->cp_conn->c_trans->conn_free (cp->cp_transport_data); }
[Severity: High]
This isn't a leak introduced by this patch - the old rds_conn_path_destroy() already had the same !cp->cp_transport_data early return ahead of destroy_workqueue() - but the patch moves destroy_workqueue() into rds_conn_path_free(), reached from a freshly recomputed npaths loop, so it may be worth addressing here. Does this leak the per-path workqueues of a loopback-converted connection?
__rds_conn_create() computes npaths from the incoming transport:
net/rds/connection.c:__rds_conn_create() {
int npaths = (trans->t_mp_capable ? RDS_MPATH_WORKERS : 1);
...
}For an outgoing RDS/TCP connection to a local address that is 8, and 8 ordered workqueues get allocated:
for (i = 0; i < npaths; i++) {
...
conn->c_path[i].cp_wq =
alloc_ordered_workqueue("krds_cp_wq#%lu/%d", 0,
rds_conn_count, i);But trans is replaced before it is stored in conn->c_trans:
if (likely(is_outgoing)) {
...
trans = &rds_loop_transport;rds_loop_transport has t_mp_capable == 0, so both rds_conn_destroy() and the new rds_conn_destroy_fini() recompute npaths as 1 and only visit path 0. And even if paths 1..7 were visited, rds_loop_conn_alloc() sets cp_transport_data only on path 0, so rds_conn_path_free() would take the !cp->cp_transport_data return before reaching destroy_workqueue(). kfree(conn->c_path) then drops the last pointers to the remaining 7 workqueues, once per destroyed connection, on every netns teardown (rds_loop_kill_conns()) or module unload (rds_loop_exit()).
For contrast, the allocation rollback in __rds_conn_create() iterates the entry-time npaths and destroys every workqueue that is not rds_wq. Should the teardown path use the same bound, e.g. by recording npaths in the connection?
Quoted 56 lines
net/rds/connection.c +40
-4+/* Free a connection. This runs from rds_conn_put() when the last +* reference is dropped, after rds_conn_destroy() has quiesced the +* connection and dropped the initial reference. +*/ +static void rds_conn_destroy_fini (struct kref *kref) +{ + struct rds_connection *conn = container_of(kref, struct rds_connection, + c_refcount); +int npaths = (conn->c_trans->t_mp_capable ?RDS_MPATH_WORKERS :1 ); +unsigned long flags; +int i; + +for (i =0 ; i < npaths; i++) +rds_conn_path_free (&conn->c_path[i]); + +kfree (conn->c_path); +kmem_cache_free (rds_conn_slab, conn); + +spin_lock_irqsave (&rds_conn_lock, flags); + rds_conn_count--; +spin_unlock_irqrestore (&rds_conn_lock, flags); +} + +void rds_conn_get (struct rds_connection *conn) +{ +kref_get (&conn->c_refcount); +} +EXPORT_SYMBOL_GPL (rds_conn_get); + +void rds_conn_put (struct rds_connection *conn) +{ +kref_put (&conn->c_refcount, rds_conn_destroy_fini); +} +EXPORT_SYMBOL_GPL (rds_conn_put); +/* * Stop and free a connection. * -* This can only be used in very limited circumstances. It assumes that once -* the conn has been shutdown that no one else is referencing the connection. -* We can only ensure this in the rmmod path in the current code. +* Quiesces the connection synchronously (workers cancelled, transport +* connections shut down, queued messages dropped) and drops the +* initial reference. The memory - including the transport's +* per-connection state and the path workqueues - is freed once the +* last rds_conn_put() runs, which may be after this returns. */ void rds_conn_destroy (struct rds_connection *conn) { -unsigned long flags;int i;struct rds_conn_path *cp;int npaths = (conn->c_trans->t_mp_capable ?RDS_MPATH_WORKERS :1 );@@ -584,11 +634,23 @@ void rds_conn_destroy(struct rds_connection *conn)* sites (which all test rds_destroy_pending() under * rcu_read_lock()) from queueing new work on the path
net/rds/connection.c void rds_conn_destroy(struct rds_connection *conn) · line 645
Quoted 10 lines
net/rds/connection.c +8
-0* workqueues once we start cancelling and destroying them. +* +* Now that the transport state stays discoverable (e.g. on the +* transports' connection lists) until the final rds_conn_put(), +* a conn can be handed to rds_conn_destroy() more than once - +* e.g. dropped for a protocol version mismatch and then found +* again at module unload. Only the first caller proceeds; the +* unhash also happens under rds_conn_lock, so a looked-up conn +* can never be quiesced twice. */
[Severity: Medium]
Is the protocol-version-mismatch example still reachable? The changelog says:
"Today, connections are also destroyed on network namespace teardown (rds_tcp_kill_sock() and rds_loop_kill_conns()) and, one at a time, when a peer negotiates an unsupported protocol version (rds_ib_cm_connect_complete())."
In this tree that branch drops the connection rather than destroying it:
net/rds/ib_cm.c:rds_ib_cm_connect_complete() {
...
if (conn->c_version < RDS_PROTOCOL_VERSION) {
...
rds_conn_drop(conn);
return;
}
...
}git grep rds_conn_destroy net/rds/ finds call sites only in ib_rdma.c (rds_ib_destroy_nodev_conns), loop.c and tcp.c - none in ib_cm.c. Commit f97d8c7bab78 ("rds: ib: use rds_conn_drop() on protocol version mismatch") made that change, and the preceding patch in this series says the same thing ("the last single-connection caller ... was turned into a drop by commit f97d8c7bab78").
The same example is now also in the permanent comment above ("e.g. dropped for a protocol version mismatch and then found again at module unload"). Could the changelog and the comment use an example that can still happen?
net/rds/connection.c void rds_conn_destroy(struct rds_connection *conn) · line 651
Quoted 6 lines
net/rds/connection.c +5
-0+spin_lock_irq (&rds_conn_lock); +if (conn->c_destroy_in_prog) { +spin_unlock_irq (&rds_conn_lock); +return ; + }WRITE_ONCE (conn->c_destroy_in_prog, true);
[Severity: Medium]
The flag is tested and set at the very top of rds_conn_destroy(), ahead of synchronize_rcu(), the per-path rds_conn_path_quiesce() calls and rds_cong_remove_conn(). So a caller that finds the flag already set returns before anything has been quiesced, and there is no completion or state wait on the early-return path:
if (conn->c_destroy_in_prog) {
spin_unlock_irq(&rds_conn_lock);
return;
}That window contains a full synchronize_rcu() plus cancel_delayed_work_sync(cp_send_w), cancel_delayed_work_sync(cp_recv_w) and flush_work(cp_down_w) in rds_conn_path_quiesce().
The function's own new block comment states the contract without qualification:
* Quiesces the connection synchronously (workers cancelled, transport
* connections shut down, queued messages dropped) and drops the
* initial reference.
and so does the new c_refcount comment in rds.h ("rds_conn_destroy() quiesces the connection synchronously"). Neither holds on the early-return path, and the void return gives the caller no way to tell.
I could not find a second destroy caller today - all five sites detach the transport node under the transport lock first, and netns exit and module unload are excluded by pernet_ops_rwsem - so should the documentation say the quiesce and the reference drop only happen for the first caller? If a second caller ever does appear, would rds_tcp_kill_sock()/rds_loop_kill_conns() proceed while the transport socket and the workers are still live?
[Severity: Low]
This isn't a bug, but the existing comment on c_destroy_in_prog in rds.h now describes fewer access rules than the code relies on:
/* Set once, by rds_conn_destroy(), before it cancels the path * works; read through rds_destroy_pending(). ... */
The new "only the first caller proceeds" guarantee depends on the plain read plus WRITE_ONCE() being an atomic test-and-set under rds_conn_lock. Should the comment mention that? A future writer following only the documented rule would store the flag without rds_conn_lock, as the code did before this patch, and a reader following "read through rds_destroy_pending()" would get a different predicate, since rds_destroy_pending() is also true for netns teardown and for ->t_unloading.
Quoted 6 lines
net/rds/connection.c +1
-1+/* Ensure conn will not be scheduled for reconnect */ -spin_lock_irq (&rds_conn_lock);hlist_del_init_rcu (&conn->c_hash_node);spin_unlock_irq (&rds_conn_lock);synchronize_rcu ();
[ ... ]
Quoted 14 lines
net/rds/rds.h +6
-0@@ -137,6 +137,12 @@ struct rds_conn_path {/* One rds_connection per RDS address pair */ struct rds_connection {struct hlist_node c_hash_node; +/* rds_conn_destroy() quiesces the connection synchronously; +* freeing it - the connection memory, the path workqueues and +* the transport's per-connection state - is deferred until the +* last reference is dropped via rds_conn_put(). +*/ +struct kref c_refcount;
[ ... ]
Signature · 1 line
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20260922085410.391323-1-achender%40kernel.org
Allison Henderson
maintainerPATCH v6 6/12 net/rds: wait for connections to be freed on transport unload
In reply to Allison HendersonFrom: Sharath Srinivasan <[email protected]>
Since connection free became asynchronous, rds_conn_destroy() only quiesces the connection; the actual free - including the transport's conn_free, which lives in the transport module - runs when the last reference is dropped. The transports' exit paths destroy all of their connections and then proceed to unload, so a free that is still pending (a racing lookup-style holder, or simply the destroyer's own put not yet run when destroy was invoked from another context earlier) would execute transport module code after that module's text is gone.
Count each transport's live connections in t_conn_count (incremented when a connection is published in __rds_conn_create(), decremented as the last step of rds_conn_destroy_fini()) and make the transport exit paths - rds_ib_exit(), rds_tcp_exit() and rds_loop_exit() - wait for the count to drop to zero after destroying their connections.
A bound socket holds a module reference on its own transport (rds_trans_get_preferred()), but that does not bound the wait: once the following patches make incs hold a connection reference, an unread datagram pins its connection for as long as the application leaves it on the receive queue, and rds_find_bound() does not filter on transport, so that socket may well belong to a different transport than the connection and pin nothing that stops this unload. Proceeding after a timeout would turn a leak into a use-after-free in this module's text, so the wait is unbounded: it polls the count, warns every ten seconds naming the transport and the number of connections outstanding, and returns only when the count reaches zero. rmmod therefore blocks while data is queued and unread, which is the historical RDS contract - teardown does not discard queued data.
The wait is deliberately not interruptible: the exit function has already passed the point of no return, and a killed rmmod would leave the module half torn down and unloadable, which is worse than a blocked one. The polling wakes every 100 ms, so the hung task detector does not fire.
The wake at the end of rds_conn_destroy_fini() can run from a thread executing transport module text, but never as that thread's last use of it: a CM event handler's reference is always dropped before the rdma_destroy_id() in the connection's own shutdown returns, and rds_rdma_exit() stops the listener - waiting for any handler running on it - before rds_ib_exit() starts to wait.
At this point in the series the only reference holder is the destroyer itself, so the count reaches zero as soon as the sweep has run; the guarantee only becomes load-bearing once the following patches hand references to sockets and incs.
rds_ib_exit() has one more wrinkle. rds_ib_dev_shutdown() only drops the connections still attached to a device, and each of them moves itself to ib_nodev_conns from its own shutdown work, asynchronously. A connection that has not migrated by the time rds_ib_destroy_nodev_conns() sweeps the list would never be destroyed, and would hold the count up for good. So the wait takes a resweep callback, which rds_ib_exit() points at rds_ib_destroy_nodev_conns(): late arrivals get destroyed on the next poll instead of being waited on.
In rds_ib_exit(), tearing down the last connection can also drop the final reference on a device, which defers rds_ib_dev_free() - again this module's text - to rds_wq. Flush the workqueue once after the connections are gone; rds_ib_dev_free() queues nothing further on rds_wq, so a single pass drains it.
Based on the Oracle UEK commits "net/rds: wait_event_timeout until zero connections during rmmod" and "net/rds:
Each RDS transport should keep its own connection count".
[achender: reimplementation for net-next: t_conn_count did not exist
upstream and is introduced here; single global waitqueue instead of per-transport (the loop transport never goes through rds_trans_register()); unbounded wait with a periodic warning in place of UEK's wait_event_timeout() + WARN_ON(), plus the resweep for IB's asynchronous device detach; also cover rds_loop_exit(); rewrite commit message]
--- net/rds/connection.c | 52 ++++++++++++++++++++++++++++++++++++++++++++ net/rds/ib.c | 17 +++++++++++++++ net/rds/ib_rdma.c | 2 +- net/rds/loop.c | 2 ++ net/rds/rds.h | 14 ++++++++++++ net/rds/tcp.c | 1 + 6 files changed, 87 insertions(+), 1 deletion(-)
net/rds/connection.c +52 -0
@@ -48,6 +48,8 @@/* converting this to RCU is a chore for another day.. */ static DEFINE_SPINLOCK (rds_conn_lock);static unsigned long rds_conn_count; +/* woken whenever a transport's t_conn_count drops to zero */ +static DECLARE_WAIT_QUEUE_HEAD (rds_conn_freed_waitq);static struct hlist_head rds_conn_hash[RDS_CONNECTION_HASH_ENTRIES ];static struct kmem_cache *rds_conn_slab;
@@ -341,6 +343,7 @@ static struct rds_connection *__rds_conn_create(struct net *net, parent->c_passive = conn;rds_cong_add_conn (conn); rds_conn_count++; +atomic_inc (&conn->c_trans->t_conn_count); } }else {/* Creating normal conn */
@@ -359,6 +362,7 @@ static struct rds_connection *__rds_conn_create(struct net *net,hlist_add_head_rcu (&conn->c_hash_node, head);rds_cong_add_conn (conn); rds_conn_count++; +atomic_inc (&conn->c_trans->t_conn_count); } }spin_unlock_irqrestore (&rds_conn_lock, flags);
@@ -584,6 +588,7 @@ static void rds_conn_destroy_fini(struct kref *kref) struct rds_connection *conn = container_of(kref, struct rds_connection, c_refcount);int npaths = (conn->c_trans->t_mp_capable ?RDS_MPATH_WORKERS :1 ); +struct rds_transport *trans = conn->c_trans;unsigned long flags;int i;
@@ -596,7 +601,54 @@ static void rds_conn_destroy_fini(struct kref *kref)spin_lock_irqsave (&rds_conn_lock, flags); rds_conn_count--;spin_unlock_irqrestore (&rds_conn_lock, flags); + +/* only after everything the transport module owns has been +* freed above may its unload proceed +*/ +if (!atomic_dec_return (&trans->t_conn_count)) +wake_up_all (&rds_conn_freed_waitq); +} + +/* Wait for all of @trans's connections to be freed; the free runs +* asynchronously once rds_conn_destroy() has quiesced a connection. +* Called on transport module unload after an initial sweep has +* destroyed the transport's connections; @resweep, when given, is +* called on every poll to destroy connections that were still +* detaching from a device when the sweep ran (IB). A connection +* reference can be held for an application-controlled time - once +* incs hold one, an unread datagram pins the inc that carries it, and +* thus the connection - so the wait is unbounded: the frees that run +* after unload call into this module's text (conn_free, inc_free) and +* free into its slabs, so proceeding while any remain would be a +* use-after-free, not a leak. Warn periodically so a stuck count is +* diagnosable, but never stop waiting. This matches the historical +* RDS contract that teardown does not discard queued data. +*/ +void rds_conn_wait_conns_freed (struct rds_transport *trans, +void (*resweep)(void )) +{ +unsigned long warn_interval = +msecs_to_jiffies (RDS_CONN_FREE_WARN_INTERVAL_MS ); +unsigned long warn_at = jiffies + warn_interval; + +while (!wait_event_timeout (rds_conn_freed_waitq, + !atomic_read (&trans->t_conn_count), +msecs_to_jiffies (RDS_CONN_FREE_POLL_MS ))) { +/* A transport whose teardown is asynchronous (IB moves a +* connection off its device from the shutdown work) gives +* us a resweep to destroy what has arrived since. +*/ +if (resweep) +resweep (); +if (time_after_eq (jiffies, warn_at)) { +pr_warn ("RDS/%s: still waiting for %d connection(s) to be freed before unload\n" , + trans->t_name, +atomic_read (&trans->t_conn_count)); + warn_at = jiffies + warn_interval; + } + } } +EXPORT_SYMBOL_GPL (rds_conn_wait_conns_freed);void rds_conn_get (struct rds_connection *conn) {
net/rds/ib.c +17 -0
@@ -537,7 +537,24 @@ void rds_ib_exit(void)rds_info_deregister_func (RDS6_INFO_IB_CONNECTIONS , rds6_ib_ic_info);#endif rds_ib_unregister_client (); + +/* rds_ib_dev_shutdown() only dropped the connections still +* attached to a device; each moves itself to ib_nodev_conns +* from its shutdown work. Destroy what is there now and keep +* sweeping the list while the wait sees connections outstanding, +* so a late arrival is destroyed rather than waited on forever. +*/ rds_ib_destroy_nodev_conns (); +rds_conn_wait_conns_freed (&rds_ib_transport, + rds_ib_destroy_nodev_conns); + +/* Tearing down the last connection may have dropped the final +* reference on a device, deferring rds_ib_dev_free() to rds_wq. +* Drain it before the module goes away; it queues nothing +* further on rds_wq. +*/ +flush_workqueue (rds_wq); +rds_ib_sysctl_exit ();rds_ib_recv_exit ();rds_trans_unregister (&rds_ib_transport);
net/rds/ib_rdma.c +1 -1
@@ -165,7 +165,7 @@ void rds_ib_destroy_nodev_conns(void)/* avoid calling conn_destroy with irqs off */ spin_lock_irq (&ib_nodev_conns_lock); -list_splice (&ib_nodev_conns, &tmp_list); +list_splice_init (&ib_nodev_conns, &tmp_list);spin_unlock_irq (&ib_nodev_conns_lock);list_for_each_entry_safe (ic, _ic, &tmp_list, ib_node)
net/rds/loop.c +2 -0
@@ -195,6 +195,8 @@ void rds_loop_exit(void)WARN_ON (lc->conn->c_passive);rds_conn_destroy (lc->conn); } + +rds_conn_wait_conns_freed (&rds_loop_transport,NULL ); }static void rds_loop_kill_conns (struct net *net)
net/rds/rds.h +14 -0
@@ -557,6 +557,12 @@ struct rds_transport { unsigned int t_prefer_loopback:1, t_mp_capable:1;unsigned int t_type; +/* Connections of this transport not yet freed; freeing runs +* asynchronously once rds_conn_destroy() has quiesced a +* connection, so transport module unload has to wait for this +* to reach zero (rds_conn_wait_conns_freed()). +*/ +atomic_t t_conn_count;int (*laddr_check)(struct net *net,const struct in6_addr *addr,__u32 scope_id);
@@ -839,6 +845,14 @@ static inline bool rds_conn_get_unless_zero(struct rds_connection *conn) {return kref_get_unless_zero (&conn->c_refcount); } + +/* transport unload waits for its connections to be freed, polling at +* the first interval and warning at the second +*/ +#define RDS_CONN_FREE_POLL_MS 100 +#define RDS_CONN_FREE_WARN_INTERVAL_MS 10000 +void rds_conn_wait_conns_freed (struct rds_transport *trans, +void (*resweep)(void ));void rds_conn_drop (struct rds_connection *conn);void rds_conn_path_drop (struct rds_conn_path *cpath,bool destroy);void rds_conn_connect_if_down (struct rds_connection *conn);
net/rds/tcp.c +1 -0
@@ -805,6 +805,7 @@ static void rds_tcp_exit(void)#endif unregister_pernet_device (&rds_tcp_net_ops);rds_tcp_destroy_conns (); +rds_conn_wait_conns_freed (&rds_tcp_transport,NULL );rds_trans_unregister (&rds_tcp_transport);rds_tcp_recv_exit ();kmem_cache_destroy (rds_tcp_conn_slab);
Signature · 1 line
2.25.1
[email protected]
AI reviewRe: [PATCH net-next v6 06/12] net/rds: wait for connections to be freed on transport unload
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 19 lines
From: Sharath Srinivasan <[email protected]>
Since connection free became asynchronous,
rds_conn_destroy()only quiesces the connection; the actual free - including the transport'sconn_free, which lives in the transport module - runs when the last reference is dropped. The transports' exit paths destroy all of their connections and then proceed to unload, so a free that is still pending (a racing lookup-style holder, or simply the destroyer's own put not yet run when destroy was invoked from another context earlier) would execute transport module code after that module's text is gone.Count each transport's live connections in
t_conn_count(incremented when a connection is published in__rds_conn_create(), decremented as the last step ofrds_conn_destroy_fini()) and make the transport exit paths -rds_ib_exit(),rds_tcp_exit()andrds_loop_exit()- wait for [ ... ]
place of UEK'swait_event_timeout()+WARN_ON(), plus the resweep for IB's asynchronous device detach; also coverrds_loop_exit(); rewrite commit message]
Quoted 2 lines
Assisted-by Claude-Code:claude-fable-5Signed-off-by Allison Henderson <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=6
[email protected]
AI reviewRe: [PATCH net-next v6 06/12] net/rds: wait for connections to be freed on transport unload
In reply to Allison HendersonThank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider.
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider.
Critical: 0 · High: 0 · Medium: 1 · Low: 0
If you believe none of the issues are legitimate, please reply to each issue of Medium severity or higher and explain why the AI is wrong. If any of them are legitimate and you plan to respin, please let patchwork know by including "pw-bot: cr" as a separate line at the end of your reply (one such reply per series is enough).
- [Medium] At this commit only, the combination of list_splice_init() in net/rds/ib_rdma.c:rds_ib_destroy_nodev_conns() (line 168) and the…
--
net/rds: wait for connections to be freed on transport unload
This adds a per-transport live connection count (t_conn_count) that is incremented when a connection is published in __rds_conn_create() and decremented at the end of rds_conn_destroy_fini(). The transport exit paths then poll that count until it reaches zero, with an optional resweep callback used by the IB transport for connections that are still detaching from a device.
[ ... ]
Quoted 8 lines
net/rds/ib.c +0
-0@@ -537,7 +537,24 @@ void rds_ib_exit(void)rds_info_deregister_func (RDS6_INFO_IB_CONNECTIONS , rds6_ib_ic_info);#endif rds_ib_unregister_client ();
net/rds/ib.c void rds_ib_exit(void) · line 549
Quoted 10 lines
net/rds/ib.c +9
-0+ +/* rds_ib_dev_shutdown() only dropped the connections still +* attached to a device; each moves itself to ib_nodev_conns +* from its shutdown work. Destroy what is there now and keep +* sweeping the list while the wait sees connections outstanding, +* so a late arrival is destroyed rather than waited on forever. +*/ rds_ib_destroy_nodev_conns (); +rds_conn_wait_conns_freed (&rds_ib_transport, + rds_ib_destroy_nodev_conns);
[Severity: Medium]
Passing rds_ib_destroy_nodev_conns as the resweep means the list is emptied again on every 100 ms poll for the whole duration of the unload wait, rather than just once. Combined with the list_splice_init() change below, can that widen a window where rds_ib_add_conn() takes BUG()?
At this commit rds_ib_add_conn() still asserts that the list head is non-empty:
net/rds/ib_rdma.c:rds_ib_add_conn() {
/* conn was previously on the nodev_conns_list */
spin_lock_irq(&ib_nodev_conns_lock);
BUG_ON(list_empty(&ib_nodev_conns));
BUG_ON(list_empty(&ic->ib_node));
list_del(&ic->ib_node);
...
}A connect that was already in flight when the unload started can reach that assert while the sweep still holds the claimed nodes on its stack-local tmp_list:
rds_ib_cm_handle_event() (ADDR_RESOLVED/ROUTE_RESOLVED)
rds_ib_cm_initiate_connect()
rds_ib_setup_qp()
rds_ib_add_conn() <- list_empty(&ib_nodev_conns) == trueThe sweep itself calls rds_conn_destroy(), which unhashes the connection and then blocks in synchronize_rcu() before driving the path to RDS_CONN_DISCONNECTING, so there is real time spent with the head empty and the connections still alive.
Quoted 16 lines
net/rds/ib.c +8
-0+ +/* Tearing down the last connection may have dropped the final +* reference on a device, deferring rds_ib_dev_free() to rds_wq. +* Drain it before the module goes away; it queues nothing +* further on rds_wq. +*/ +flush_workqueue (rds_wq); +rds_ib_sysctl_exit ();rds_ib_recv_exit ();rds_trans_unregister (&rds_ib_transport);net/rds/ib_rdma.c +0
-0
net/rds/ib_rdma.c void rds_ib_destroy_nodev_conns(void) · line 171
Quoted 8 lines
net/rds/ib_rdma.c +1
-1@@ -165,7 +165,7 @@ void rds_ib_destroy_nodev_conns(void)/* avoid calling conn_destroy with irqs off */ spin_lock_irq (&ib_nodev_conns_lock); -list_splice (&ib_nodev_conns, &tmp_list); +list_splice_init (&ib_nodev_conns, &tmp_list);spin_unlock_irq (&ib_nodev_conns_lock);list_for_each_entry_safe (ic, _ic, &tmp_list, ib_node)
[Severity: Medium]
With the old plain list_splice(), ib_nodev_conns was left stale and non-empty, so the BUG_ON(list_empty(&ib_nodev_conns)) in rds_ib_add_conn() could not fire during a sweep. Does re-initializing the head here make that assert reachable for an ic whose node the sweep has already moved onto tmp_list?
Worth noting: the next patch in this series, "net/rds: unlink transport nodes before a possibly deferred connection free", removes both BUG_ONs from rds_ib_add_conn() and replaces the splice with a per-node i_ib_node_detached claim, so the end state of the series does not have this window - it exists only at this intermediate commit, which makes it a bisection hazard. Could the assert removal be folded in here, or the list_splice_init() change moved to the later patch, so no commit in the series has the reachable BUG()?
[ ... ]
Signature · 1 line
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20260922085410.391323-1-achender%40kernel.org
Allison Henderson
maintainerPATCH v6 7/12 net/rds: unlink transport nodes before a possibly deferred connection free
In reply to Allison HendersonThe transport teardown helpers - rds_tcp_destroy_conns(), rds_tcp_kill_sock(), rds_ib_destroy_nodev_conns(), rds_loop_exit() and rds_loop_kill_conns() - gather the per-connection transport nodes onto a list head on their own stack and call rds_conn_destroy() for each. The node is unlinked much later, by the transport's conn_free():rds_tcp_conn_free() and rds_loop_conn_free() list_del() it, and rds_ib_conn_free() does so unconditionally.
That is fine for as long as rds_conn_destroy() frees the connection before it returns, which is still the case at this point in the
series: the initial reference is the only one. The following patches hand out references that outlive the teardown loop - a socket's cached rs_conn, an inc parked on a receive queue - and with those, a conn_free() deferred until after the helper has returned would list_del() the node from a stack frame that no longer exists. Make the helpers ready for that first.
Unlink each node under the transport lock right before its rds_conn_destroy(), so that nothing is left on the stack list for a later free to touch. TCP marks the node detached, as rds_tcp_kill_sock() already does for the secondary paths of a multipath connection; loopback uses list_del_init() and has its conn_free() skip a node that is already empty.
IB needs an explicit flag, i_ib_node_detached, because its node has other movers: a connect worker moves it from the nodev list to a device's list in rds_ib_add_conn(), and a shutdown moves it back in rds_ib_remove_conn(). Either can run while the sweep holds the node on its stack list, and "the node is linked" cannot tell that list from the nodev list - an add_conn() that went by list emptiness would unlink the node from under the sweep's lockless walk. So the sweep sets the flag when it gathers the node, under ib_nodev_conns_lock, and from then on add_conn(), remove_conn() and conn_free() leave the node alone; the node belongs to the sweep, and its walk needs no lock. Those movers used to assert that the node is linked (and rds_ib_add_conn() that the nodev list is non-empty); a connect or shutdown worker can still be running for a connection the sweep has claimed, and such a connection is about to be destroyed anyway, so the assertions go.
The walk itself must not lose the entries either. A connection that was destroyed earlier - dropped for a protocol version mismatch, then found again at module unload - is kept alive only by whatever reference is still pending, and that can be dropped at any point during the walk, freeing the transport node the iterator is about to read. So the gather takes a reference on each connection it moves onto the stack list, under the transport lock, and drops it after the destroy; a connection whose free is already running gets no reference and is left where it is, since that free unlinks the node itself once the lock is released. The tmp_list gathering itself remains: it is what keeps rds_conn_destroy() from being called with the transport lock held.
--- net/rds/ib.h | 4 +++ net/rds/ib_cm.c | 4 ++- net/rds/ib_rdma.c | 67 ++++++++++++++++++++++++++++++++++++----------- net/rds/loop.c | 59 ++++++++++++++++++++++++++++++++--------- net/rds/tcp.c | 52 +++++++++++++++++++++++++++++++----- 5 files changed, 152 insertions(+), 34 deletions(-)
net/rds/ib.h +4 -0
@@ -143,6 +143,10 @@ struct rds_ib_device;struct rds_ib_connection {struct list_head ib_node; +/* set under ib_nodev_conns_lock once a transport teardown has +* claimed ib_node; from then on only the teardown touches it +*/ +bool i_ib_node_detached;struct rds_ib_device *rds_ibdev;struct rds_connection *conn;
net/rds/ib_cm.c +3 -1
@@ -1287,7 +1287,9 @@ void rds_ib_conn_free(void *arg) lock_ptr = ic->rds_ibdev ? &ic->rds_ibdev->spinlock : &ib_nodev_conns_lock;spin_lock_irqsave (lock_ptr, flags); -list_del (&ic->ib_node); +/* a transport teardown that gathered us first owns the node */ +if (!ic->i_ib_node_detached) +list_del (&ic->ib_node);spin_unlock_irqrestore (lock_ptr, flags);rds_ib_recv_free_caches (ic);
net/rds/ib_rdma.c +52 -15
95 lines not on this page · raw message
net/rds/loop.c +47 -12
88 lines not on this page · raw message
net/rds/tcp.c +46 -6
79 lines not on this page · raw message
Signature · 1 line
2.25.1
[email protected]
AI reviewRe: [PATCH net-next v6 07/12] net/rds: unlink transport nodes before a possibly deferred connection free
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 19 lines
The transport teardown helpers -
rds_tcp_destroy_conns(),rds_tcp_kill_sock(),rds_ib_destroy_nodev_conns(),rds_loop_exit()andrds_loop_kill_conns()- gather the per-connection transport nodes onto a list head on their own stack and callrds_conn_destroy()for each. The node is unlinked much later, by the transport'sconn_free():rds_tcp_conn_free()andrds_loop_conn_free()list_del()it, andrds_ib_conn_free()does so unconditionally.That is fine for as long as
rds_conn_destroy()frees the connection before it returns, which is still the case at this point in the
series: the initial reference is the only one. The following patches hand out references that outlive the teardown loop - a socket's cachedrs_conn, an inc parked on a receive queue - and with those, aconn_free()deferred until after the helper has returned wouldlist_del()the node from a stack frame that no longer exists. Make [ ... ]
what keepsrds_conn_destroy()from being called with the transport lock held.
Quoted 2 lines
Assisted-by Claude-Code:claude-fable-5Signed-off-by Allison Henderson <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=7
[email protected]
AI reviewRe: [PATCH net-next v6 07/12] net/rds: unlink transport nodes before a possibly deferred connection free
In reply to Allison HendersonThank you for your contribution! Sashiko AI review found more severe issue(s) in other patches of this series. This mail is only an FYI, in case you want to address the nit picks below when you…
Thank you for your contribution! Sashiko AI review found more severe issue(s) in other patches of this series. This mail is only an FYI, in case you want to address the nit picks below when you respin.
Low priority / nit picks for your consideration:
- [Low] Stale/unreachable justification for the new per-connection reference in rds_ib_destroy_nodev_conns(): both the commit message ("A…
--
net/rds: unlink transport nodes before a possibly deferred connection free
The RDS transport teardown helpers now unlink each per-connection transport node under the transport lock right before calling rds_conn_destroy(), so a conn_free() that runs after the helper has returned does not touch a list head on a dead stack frame. IB gets an explicit i_ib_node_detached flag, and the gather loops take a reference on each connection they move onto the stack list.
Quoted 6 lines
The walk itself must not lose the entries either. A connection that was destroyed earlier - dropped for a protocol version mismatch, then found again at module unload - is kept alive only by whatever reference is still pending, and that can be dropped at any point during the walk, freeing the transport node the iterator is about to read.
[Severity: Low]
Is this scenario reachable in this tree? A grep for rds_conn_destroy() callers at this commit finds only the five teardown helpers:
net/rds/ib_rdma.c: rds_ib_destroy_nodev_conns() net/rds/loop.c: rds_loop_destroy_gathered_conns() net/rds/tcp.c: rds_tcp_destroy_gathered_conns()
There is no protocol-version-mismatch caller of rds_conn_destroy() any more - net/rds/ib_cm.c has none, that path was converted to rds_conn_drop() upstream by f97d8c7bab78 - so a connection cannot have been "destroyed earlier ... for a protocol version mismatch".
The same commit message also says two paragraphs earlier:
Quoted 3 lines
That is fine for as long as
rds_conn_destroy()frees the connection before it returns, which is still the case at this point in the
series: the initial reference is the only one.
and the sibling patch "net/rds: make rds_destroy_pending() report a connection's own destroy" states that "Today every rds_conn_destroy() does happen on one of those two global paths".
Could this paragraph be reworded to say that the reference is preparation for the later patches that hand out references, rather than describing a use-after-free that exists today? As written, and with no
Fixes: tag, it reads like a fix for a live bug.
Quoted 4 lines
net/rds/ib_rdma.c +0
-0
[ ... ]
net/rds/ib_rdma.c void rds_ib_destroy_nodev_conns(void) · line 181
Quoted 11 lines
net/rds/ib_rdma.c +6
-1@@ -163,13 +173,40 @@ void rds_ib_destroy_nodev_conns(void)struct rds_ib_connection *ic, *_ic;LIST_HEAD (tmp_list); -/* avoid calling conn_destroy with irqs off */ +struct rds_connection *conn; + +/* Gather the connections and take a reference on each, so that +* none is freed under the walk below (a connection destroyed +* earlier, for a protocol version mismatch, can be on this list +* with only a socket's reference still pending). One whose free
[Severity: Low]
The same claim is repeated here in the comment on rds_ib_destroy_nodev_conns(), so the stale justification ends up in the source tree and stays there through the rest of the series. Since nothing calls rds_conn_destroy() for a protocol version mismatch, and rds_ib_destroy_nodev_conns() is only reached from rds_ib_exit() and its resweep, is there a caller that can drop a pending reference under this walk today?
Would it be clearer to describe the rds_conn_get_unless_zero() here as guarding against the deferred frees introduced later in the series?
Quoted 32 lines
net/rds/ib_rdma.c +25
-3+* is already running gets no reference: its free unlinks the +* node itself, under this lock, once we drop it. Marking the +* node detached claims it for this sweep: rds_ib_add_conn(), +* rds_ib_remove_conn() and rds_ib_conn_free() leave a claimed +* node alone, so the walk over tmp_list below needs no lock. +* Avoid calling conn_destroy with irqs off. +*/ spin_lock_irq (&ib_nodev_conns_lock); -list_splice_init (&ib_nodev_conns, &tmp_list); +list_for_each_entry_safe (ic, _ic, &ib_nodev_conns, ib_node) { +if (rds_conn_get_unless_zero (ic->conn)) { + ic->i_ib_node_detached = true; +list_move_tail (&ic->ib_node, &tmp_list); + } + }spin_unlock_irq (&ib_nodev_conns_lock); -list_for_each_entry_safe (ic, _ic, &tmp_list, ib_node) -rds_conn_destroy (ic->conn); +/* rds_conn_destroy() can return before the connection is freed, +* and it is the free - rds_ib_conn_free() - that would unlink +* ib_node. tmp_list lives on this stack frame, so take each node +* off it before its destroy; the free then leaves it alone. +*/ +list_for_each_entry_safe (ic, _ic, &tmp_list, ib_node) { + conn = ic->conn; +list_del_init (&ic->ib_node); + +rds_conn_destroy (conn); +rds_conn_put (conn); + } }
[ ... ]
Signature · 1 line
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20260922085410.391323-1-achender%40kernel.org
Allison Henderson
maintainerPATCH v6 8/12 net/rds: hold connection references in lookup, sockets and c_passive
In reply to Allison HendersonFrom: Sharath Srinivasan <[email protected]>
Hand out real references everywhere a struct rds_connection pointer previously escaped bare:
- rds_conn_lookup() takes a reference on the connection it returns (kref_get_unless_zero(), so that it only ever hands out a live reference), and
__rds_conn_create() returns the connection with a reference held for the caller on every path: lookup hit, fresh creation, lost creation race, and the passive-loopback lookup, which now also holds the parent while it dereferences parent->c_passive.
- The rs->rs_conn sendmsg cache owns a reference, which is dropped when the cache is replaced or the socket is released. rds_sendmsg() itself holds a reference for the duration of the call, during which reads and updates of rs_conn are serialized by rs_lock. So neither a concurrent rds_conn_destroy() nor another sender replacing the cache can free the connection under a sender. The connection may still be destroyed while a send is in flight - when its device is removed or its netns is torn down - but it is only quiesced; the free is held off by the sender's reference. A cached connection whose destruction has begun is no longer reused. Instead, sendmsg drops it and looks up or creates a live one, so a socket cannot get stuck returning -EAGAIN forever against a quiesced connection. Both ToS ioctls use the same lock now (they used the unrelated global rds_sock_lock before), and since the create in rds_sendmsg() samples rs_tos without the lock, the install re-checks under it that the new connection's ToS still matches the socket's and returns -EAGAIN if a SIOCRDSSETTOS slipped in between, rather than sending on, and caching, a connection with the old ToS.
- parent->c_passive owns a reference, dropped when the parent is destroyed. The pointer is read under rcu_read_lock() and written under rds_conn_lock, so it is RCU-annotated and accessed through rcu_dereference()/rcu_assign_pointer(). A passive connection whose own destroy has begun is neither handed out nor left dangling in the parent: __rds_conn_create() refuses it, and the child's destroy clears the parent's pointer and drops that reference itself, so a quiesced passive conn cannot be revived by a later connect request.
Serializing the rs_conn cache under rs_lock also resolves a syzbot-reported KCSAN data race between concurrent rds_sendmsg() calls on the same socket, each installing the connection it created into rs->rs_conn with a plain store:
BUG: KCSAN: data-race in rds_sendmsg / rds_sendmsg
write to 0xffff888101dec818 of 8 bytes by task 30904 on cpu 0:
rds_sendmsg+0xc1f/0x1580 net/rds/send.c:1332
write to 0xffff888101dec818 of 8 bytes by task 30905 on cpu 1:
rds_sendmsg+0xc1f/0x1580 net/rds/send.c:1332
value changed: 0x0000000000000000 -> 0xffff88811b61faf0
cm_id->context still carries no reference of its own after this patch; the following patch pins the connection for the duration of the CM event handler.
rds_tcp_accept_one() needs no destroy check of its own even though it now gets a referenced connection back from rds_conn_create(): TCP connections are only destroyed on netns teardown and module unload, and both stop the listener - flushing the accept work and clearing the listen socket the accept tests first - before destroying anything.
This is not a stable candidate on its own: it depends on the connection reference counting introduced by the preceding patches, and the race it closes needs a connection destroyed under a live socket, which takes netns teardown, module unload or device removal.
Based on the Oracle UEK commits "net/rds: Add krefs to struct rds_connection" and "net/rds: rds_sendmsg must use rs_conn only when not being destroyed".
[achender: substantial reimplementation for net-next: upstream has no
conn reaper, per-conn workers hold no references (destroy cancels them synchronously before the final put), and the sendmsg cache is serialized with rs_lock instead of UEK's socket flag; rewrite commit message]
--- net/rds/af_rds.c | 24 ++++++-- net/rds/connection.c | 127 +++++++++++++++++++++++++++++++++++++++++-- net/rds/ib_cm.c | 9 ++- net/rds/loop.c | 2 +- net/rds/rds.h | 7 ++- net/rds/send.c | 56 +++++++++++++++++-- net/rds/tcp_listen.c | 11 +++- 7 files changed, 215 insertions(+), 21 deletions(-)
net/rds/af_rds.c +19 -5
@@ -80,6 +80,14 @@ static int rds_release(struct socket *sock)rds_notify_queue_get (rs,NULL );rds_notify_msg_zcopy_purge (&rs->rs_zcookie_queue); +/* drop the cached connection reference; no sendmsg can race +* with us here, the socket is going away +*/ +if (rs->rs_conn) { +rds_conn_put (rs->rs_conn); + rs->rs_conn =NULL ; + } +spin_lock_bh (&rds_sock_lock);list_del_init (&rs->rs_item);spin_unlock_bh (&rds_sock_lock);
@@ -255,6 +263,7 @@ static int rds_ioctl(struct socket *sock, unsigned int cmd, unsigned long arg) {struct rds_sock *rs =rds_sk_to_rs (sock->sk);rds_tos_t utos, tos =0 ; +unsigned long flags;switch (cmd) {case SIOCRDSSETTOS :
@@ -267,18 +276,23 @@ static int rds_ioctl(struct socket *sock, unsigned int cmd, unsigned long arg)else return -ENOIOCTLCMD ; -spin_lock_bh (&rds_sock_lock); +/* rs_conn is serialized by rs_lock (see rds_sendmsg()); +* hold it across the "no connection yet" check and the +* rs_tos store so a racing sendmsg cannot cache a conn +* whose c_tos then disagrees with rs_tos. +*/ +spin_lock_irqsave (&rs->rs_lock, flags);if (rs->rs_tos || rs->rs_conn) { -spin_unlock_bh (&rds_sock_lock); +spin_unlock_irqrestore (&rs->rs_lock, flags);return -EINVAL ; } rs->rs_tos = tos; -spin_unlock_bh (&rds_sock_lock); +spin_unlock_irqrestore (&rs->rs_lock, flags);break ;case SIOCRDSGETTOS : -spin_lock_bh (&rds_sock_lock); +spin_lock_irqsave (&rs->rs_lock, flags); tos = rs->rs_tos; -spin_unlock_bh (&rds_sock_lock); +spin_unlock_irqrestore (&rs->rs_lock, flags);if (put_user (tos, (rds_tos_t __user *)arg))return -EFAULT ;break ;
net/rds/connection.c +122 -5
192 lines not on this page · raw message
net/rds/ib_cm.c +8 -1
@@ -924,8 +924,15 @@ int rds_ib_cm_handle_connect(struct rdma_cm_id *cm_id,rds_ib_conn_error (conn,"rdma_accept failed\n" ); out: -if (conn) +if (conn) {mutex_unlock (&conn->c_cm_lock); +/* Drop the reference rds_conn_create() handed us. The +* conn stays reachable through cm_id->context without a +* reference of its own for now; the CM event handler is +* given one of its own by a following patch. +*/ +rds_conn_put (conn); + }if (err)rdma_reject (cm_id, &err,sizeof (int ),IB_CM_REJ_CONSUMER_DEFINED );
net/rds/loop.c +1 -1
@@ -170,7 +170,7 @@ static void rds_loop_destroy_gathered_conns(struct list_head *tmp_list)list_for_each_entry_safe (lc, _lc, tmp_list, loop_node) { conn = lc->conn; -WARN_ON (conn->c_passive); +WARN_ON (rcu_access_pointer (conn->c_passive));spin_lock_irq (&loop_conns_lock);list_del_init (&lc->loop_node);
net/rds/rds.h +5 -2
@@ -161,7 +161,7 @@ struct rds_connection {* cancellation from landing on a destroyed workqueue. */ bool c_destroy_in_prog; -struct rds_connection *c_passive; +struct rds_connection __rcu *c_passive;struct rds_transport *c_trans;struct rds_cong_map *c_lcong;
@@ -669,7 +669,10 @@ struct rds_sock {/* * rds_sendmsg caches the conn it used the last time around. -* This helps avoid costly lookups. +* This helps avoid costly lookups. The cache owns a connection +* reference, dropped when it is replaced or the socket is +* released, and is read and written under rs_lock - except by +* rds_release(), which runs once no one else can reach the socket. */ struct rds_connection *rs_conn;
net/rds/send.c +50 -6
96 lines not on this page · raw message
net/rds/tcp_listen.c +10 -1
@@ -153,7 +153,7 @@ int rds_tcp_accept_one(struct rds_tcp_net *rtn) {struct socket *listen_sock = rtn->rds_tcp_listen_sock;struct socket *new_sock =NULL ; -struct rds_connection *conn; +struct rds_connection *conn =NULL ;int ret;struct inet_sock *inet;struct rds_tcp_connection *rs_tcp =NULL ;
@@ -229,6 +229,7 @@ int rds_tcp_accept_one(struct rds_tcp_net *rtn)if (IS_ERR (conn)) { ret =PTR_ERR (conn); + conn =NULL ; goto out; }/* An incoming SYN request came in, and TCP just accepted it.
@@ -277,6 +278,12 @@ int rds_tcp_accept_one(struct rds_tcp_net *rtn) cp = rs_tcp->t_cpath; conn_state =rds_conn_path_state (cp);WARN_ON (conn_state ==RDS_CONN_UP ); +/* A connection whose destroy has begun cannot be found here: +* TCP connections are only destroyed on netns teardown and on +* module unload, and both run rds_tcp_listen_stop() - which +* flushes this work and clears the listen socket that the top +* of this function tests - before any connection is destroyed. +*/ if (conn_state !=RDS_CONN_CONNECTING && conn_state !=RDS_CONN_ERROR ) {rds_conn_path_drop (cp,0 ); goto rst_nsk;
@@ -347,6 +354,8 @@ int rds_tcp_accept_one(struct rds_tcp_net *rtn)mutex_unlock (&rs_tcp->t_conn_path_lock);if (new_sock)sock_release (new_sock); +if (conn) +rds_conn_put (conn);mutex_unlock (&rtn->rds_tcp_accept_lock);
Signature · 1 line
2.25.1
[email protected]
AI reviewRe: [PATCH net-next v6 08/12] net/rds: hold connection references in lookup, sockets and c_passive
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 13 lines
From: Sharath Srinivasan <[email protected]>
Hand out real references everywhere a struct
rds_connectionpointer previously escaped bare:-
rds_conn_lookup()takes a reference on the connection it returns (kref_get_unless_zero(), so that it only ever hands out a live reference), and
__rds_conn_create()returns the connection with a reference held for the caller on every path: lookup hit, fresh creation, lost creation race, and the passive-loopback lookup, which now also holds the parent while it dereferencesparent->c_passive.
Quoted 8 lines
- The
rs->rs_connsendmsg cache owns a reference, which is dropped when the cache is replaced or the socket is released.
[ ... ]
them synchronously before the final put), and the sendmsg cache is serialized withrs_lockinstead of UEK's socket flag; rewrite commit message]Assisted-by Claude-Code:claude-fable-5Signed-off-by Allison Henderson <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=8
[email protected]
AI reviewRe: [PATCH net-next v6 08/12] net/rds: hold connection references in lookup, sockets and c_passive
In reply to Allison HendersonThank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider.
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider.
Critical: 0 · High: 0 · Medium: 2 (1) · Low: 1
If you believe none of the issues are legitimate, please reply to each issue of Medium severity or higher and explain why the AI is wrong. If any of them are legitimate and you plan to respin, please let patchwork know by including "pw-bot: cr" as a separate line at the end of your reply (one such reply per series is enough).
- [Medium] The commit message's central justification for the new rs->rs_conn reference is factually wrong about reachability.
- [Medium] Concern as filed: rds_sendmsg()'s new connection reference can be the final reference at either exit put (net/rds/send.c:1516 success…
- [Low] The comment newly added to rds_tcp_accept_one() (net/rds/tcp_listen.c, around line 281) misattributes behaviour: it…
Pre-existing issues:
- [Medium] Concern as filed: this patch makes it an explicitly supported state for rds_sendmsg() to keep running on a connection whose…
--
net/rds: hold connection references in lookup, sockets and c_passive
This patch hands out real references wherever a struct rds_connection pointer previously escaped bare: rds_conn_lookup() and __rds_conn_create() return a referenced connection, the rs->rs_conn sendmsg cache owns a reference and is read/written under rs_lock, and parent->c_passive owns an RCU-annotated reference. Serializing the rs_conn cache also resolves a syzbot-reported KCSAN data race between concurrent rds_sendmsg() calls.
A few questions on the commit message and on the new reference handling.
[Severity: Medium]
The commit message says:
The connection may still be destroyed while a send is in flight - when its device is removed or its netns is torn down - but it is only quiesced; the free is held off by the sender's reference.
and later:
the race it closes needs a connection destroyed under a live socket, which takes netns teardown, module unload or device removal.
Are either of the two named triggers reachable at this commit?
For device removal, rds_ib_remove_one() only calls rds_ib_dev_shutdown(), which drops paths rather than destroying connections:
net/rds/ib.c:rds_ib_dev_shutdown() {
...
list_for_each_entry(ic, &rds_ibdev->conn_list, ib_node)
rds_conn_path_drop(&ic->conn->c_path[0], true);
}The only IB caller of rds_conn_destroy() is rds_ib_destroy_nodev_conns() from rds_ib_exit(), i.e. module unload.
For netns teardown, rds_tcp_kill_sock() and rds_loop_kill_conns() run from pernet ->exit hooks, which only run once the netns refcount reaches zero, and a userspace RDS socket holds a netns reference from sk_alloc(). rds_sendmsg() also only ever caches a conn whose c_net matches the socket's netns, given rds_conn_create_outgoing(sock_net(sock->sk), ...) and the net == rds_conn_net(conn) test in rds_conn_lookup().
Module unload is blocked for as long as a bound socket exists:
net/rds/transport.c:rds_trans_get_preferred() {
if (trans && (trans->laddr_check(net, addr, scope_id) == 0) &&
(!trans->t_owner || try_module_get(trans->t_owner))) {and that module reference is released only by rds_trans_put() in rds_release().
Would it be more accurate to describe what is fixed here as the KCSAN data race on the plain rs->rs_conn stores plus the -EAGAIN-forever behaviour against a quiesced cached conn, rather than a free under a sender?
Quoted 4 lines
net/rds/send.c +0
-0
[ ... ]
Quoted 17 lines
net/rds/send.c +9
-4@@ -1340,21 +1341,59 @@ int rds_sendmsg(struct socket *sock, struct msghdr *msg, size_t payload_len) rm->m_daddr = daddr;/* rds_conn_create has a spinlock that runs with IRQ off. -* Caching the conn in the socket helps a lot. */ -if (rs->rs_conn &&ipv6_addr_equal (&rs->rs_conn->c_faddr, &daddr) && - rs->rs_tos == rs->rs_conn->c_tos) { - conn = rs->rs_conn; +* Caching the conn in the socket helps a lot. +* +* The cached rs_conn holds a connection reference; take one of +* our own for the duration of this call (dropped on both exit +* paths), so that neither a concurrent sender replacing the +* cache nor rds_conn_destroy() can free the connection under +* us. A cached connection whose destruction has begun is not +* reused: dropping it here lets the next sendmsg look up or +* create a live one instead of returning -EAGAIN forever.
net/rds/send.c int rds_sendmsg(struct socket *sock, struct msghdr *msg, size_t payload_len) · line 1362
Quoted 10 lines
net/rds/send.c +9
-0+*/ +spin_lock_irqsave (&rs->rs_lock, flags); + conn = rs->rs_conn; +if (conn &&ipv6_addr_equal (&conn->c_faddr, &daddr) && + rs->rs_tos == conn->c_tos && !rds_destroy_pending (conn)) { +rds_conn_get (conn); }else { + conn =NULL ; + } +spin_unlock_irqrestore (&rs->rs_lock, flags);
[Severity: Medium]
This is a pre-existing issue and not introduced by this patch, but the patch does make it an explicitly supported state for rds_sendmsg() to keep running on a connection whose rds_conn_destroy() is in progress ("it is only quiesced; the free is held off by the sender's reference").
Is the quiesce synchronized against a sender that is already past this rds_destroy_pending() test?
rds_conn_path_quiesce() walks and unlinks cp_send_queue with no cp_lock held, and BUG_ON()s on any message a socket still has linked:
net/rds/connection.c:rds_conn_path_quiesce() {
/* tear down queued messages */
list_for_each_entry_safe(rm, rtmp,
&cp->cp_send_queue,
m_conn_item) {
list_del_init(&rm->m_conn_item);
BUG_ON(!list_empty(&rm->m_sock_item));Meanwhile rds_sendmsg() tests rds_destroy_pending(conn) once and can then block for an unbounded time in rds_cong_wait() and in the wait_event_interruptible_timeout() retry loop before rds_send_queue_rm() inserts the message under cp_lock, with no re-test at insertion.
The unlocked traversal is addressed later in the same series by "net/rds:
take cp_lock to purge cp_send_queue in the quiesce", which splices the queue under cp_lock. The remaining window - a message landing on cp_send_queue after the purge - still needs a destroy running under a live sender, which does not look reachable at this commit for the reasons above.
[ ... ]
Quoted 10 lines
net/rds/send.c +2
-0@@ -1474,6 +1513,8 @@ int rds_sendmsg(struct socket *sock, struct msghdr *msg, size_t payload_len)kfree (vct.vec[ind].iov);kfree (vct.vec); +rds_conn_put (conn); +return payload_len; out:
net/rds/send.c int rds_sendmsg(struct socket *sock, struct msghdr *msg, size_t payload_len) · line 1527
Quoted 6 lines
net/rds/send.c +3
-0@@ -1481,6 +1522,9 @@ int rds_sendmsg(struct socket *sock, struct msghdr *msg, size_t payload_len)kfree (vct.vec[ind].iov);kfree (vct.vec); +if (conn) +rds_conn_put (conn); +
[Severity: Medium]
Can either of these puts (or the rds_conn_put(old) in the install path above) be the final reference while a message that this same call queued is still linked on cp->cp_send_queue?
At this commit rds_send_queue_rm() stores the connection and path pointers into the message but takes only a message reference:
net/rds/send.c:rds_send_queue_rm() {
...
rds_message_populate_header(&rm->m_inc.i_hdr, sport, dport, 0);
rm->m_inc.i_conn = conn;
rm->m_inc.i_conn_path = cp;
rds_message_addref(rm);and the socket side later dereferences both:
net/rds/send.c:rds_send_drop_to() {
conn = rm->m_inc.i_conn;
if (conn->c_trans->t_mp_capable)
cp = rm->m_inc.i_conn_path;
else
cp = &conn->c_path[0];
spin_lock_irqsave(&cp->cp_lock, flags);after rds_conn_destroy_fini() has done kfree(conn->c_path) and kmem_cache_free(rds_conn_slab, conn).
A later patch in this series, "net/rds: hold a connection reference from struct rds_incoming" (fe1de9d527be), adds rds_conn_get(conn) in rds_send_queue_rm() immediately before the i_conn store, released from rds_message_put(), which closes this gap. Hitting it at this commit also requires rds_conn_destroy() to run while a socket still has a message queued on that connection, which does not look reachable here.
Would it make sense to order the rds_incoming reference patch ahead of this one, or to note the dependency in the commit message?
Quoted 4 lines
net/rds/tcp_listen.c +0
-0
[ ... ]
net/rds/tcp_listen.c int rds_tcp_accept_one(struct rds_tcp_net *rtn) · line 286
Quoted 10 lines
net/rds/tcp_listen.c +6
-0@@ -277,6 +278,12 @@ int rds_tcp_accept_one(struct rds_tcp_net *rtn) cp = rs_tcp->t_cpath; conn_state =rds_conn_path_state (cp);WARN_ON (conn_state ==RDS_CONN_UP ); +/* A connection whose destroy has begun cannot be found here: +* TCP connections are only destroyed on netns teardown and on +* module unload, and both run rds_tcp_listen_stop() - which +* flushes this work and clears the listen socket that the top +* of this function tests - before any connection is destroyed. +*/
[Severity: Low]
The comment credits rds_tcp_listen_stop() with clearing the listen socket, but that helper is handed the socket by value and never touches rtn->rds_tcp_listen_sock:
net/rds/tcp_listen.c:rds_tcp_listen_stop() {
...
flush_workqueue(rds_wq);
flush_work(acceptor);
sock_release(sock);
}The pointer is cleared by the caller, just before it calls the helper:
net/rds/tcp.c:rds_tcp_kill_sock() {
...
rtn->rds_tcp_listen_sock = NULL;
rds_tcp_listen_stop(lsock, &rtn->rds_tcp_accept_w);The clear-then-flush-then-destroy ordering the safety argument relies on does hold, but it comes from rds_tcp_kill_sock(). Could the comment name that function instead?
Quoted 3 lines
net/rds/tcp_listen.c +1
-0+if (conn_state !=RDS_CONN_CONNECTING && conn_state !=RDS_CONN_ERROR ) {rds_conn_path_drop (cp,0 ); goto rst_nsk;
[ ... ]
Signature · 1 line
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20260922085410.391323-1-achender%40kernel.org
Allison Henderson
maintainerPATCH v6 9/12 net/rds: take cp_lock to purge cp_send_queue in the quiesce
In reply to Allison Hendersonrds_conn_path_quiesce() empties cp_send_queue by walking it with no lock held, while every path that adds to that queue - rds_send_queue_rm(), rds_send_probe(), the retransmit requeue in rds_send_path_reset() - does so under cp_lock. The unlocked walk was justified by the destroy running with nothing else alive: at every destroy trigger there is - netns teardown and module unload - no socket can still be sending on the connection, since a bound socket pins its transport module and a namespace closes its sockets before its RDS connections are torn down.
That argument still holds, but it is an argument about the callers, not a property of the code, and the following patches make a connection outlive its destroy in more situations. Splice the queue away under cp_lock and drop the message references outside it, so the purge is correct against a concurrent adder regardless.
--- net/rds/connection.c | 15 +++++++++++---- net/rds/send.c | 3 +-- 2 files changed, 12 insertions(+), 6 deletions(-)
net/rds/connection.c +11 -4
@@ -606,6 +606,8 @@ void rds_conn_shutdown(struct rds_conn_path *cp)static void rds_conn_path_quiesce (struct rds_conn_path *cp) {struct rds_message *rm, *rtmp; +unsigned long flags; +LIST_HEAD (purge);if (!cp->cp_transport_data)return ;
@@ -617,10 +619,15 @@ static void rds_conn_path_quiesce(struct rds_conn_path *cp)rds_conn_path_drop (cp, true);flush_work (&cp->cp_down_w); -/* tear down queued messages */ -list_for_each_entry_safe (rm, rtmp, - &cp->cp_send_queue, - m_conn_item) { +/* Tear down queued messages. Every path that adds to +* cp_send_queue does so under cp_lock; take it here too rather +* than rely on the argument that nothing can be adding at this +* point. +*/ +spin_lock_irqsave (&cp->cp_lock, flags); +list_splice_init (&cp->cp_send_queue, &purge); +spin_unlock_irqrestore (&cp->cp_lock, flags); +list_for_each_entry_safe (rm, rtmp, &purge, m_conn_item) {list_del_init (&rm->m_conn_item);BUG_ON (!list_empty (&rm->m_sock_item));rds_message_put (rm);
net/rds/send.c +1 -2
@@ -1366,8 +1366,7 @@ int rds_sendmsg(struct socket *sock, struct msghdr *msg, size_t payload_len) conn =rds_conn_create_outgoing (sock_net (sock->sk), &rs->rs_bound_addr, &daddr, - rs->rs_transport, -READ_ONCE (rs->rs_tos), + rs->rs_transport, rs->rs_tos, sock->sk->sk_allocation, scope_id);if (IS_ERR (conn)) {
Signature · 1 line
2.25.1
[email protected]
AI reviewRe: [PATCH net-next v6 09/12] net/rds: take cp_lock to purge cp_send_queue in the quiesce
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [High] rds: BUG_ON panic in rds_conn_path_quiesce() from concurrent sendmsg - [Low] rds: lockless read of…
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] rds: BUG_ON panic in rds_conn_path_quiesce() from concurrent sendmsg
- [Low] rds: lockless read of rs->rs_tos introduces KCSAN data race
--
commit 9489b74a07c0708162de3f8489ff10b9010c01f0
Author: Allison Henderson <[email protected]>
net/rds: take cp_lock to purge cp_send_queue in the quiesce
This commit modifies rds_conn_path_quiesce() to take the cp_lock when purging the cp_send_queue. It splices the queue to a local list under the lock to safely handle concurrent additions during the teardown walk.
Quoted 12 lines
net/rds/connection.c +1
-4@@ -617,10 +619,15 @@ static void rds_conn_path_quiesce(struct rds_conn_path *cp)rds_conn_path_drop (cp, true);flush_work (&cp->cp_down_w); -/* tear down queued messages */ -list_for_each_entry_safe (rm, rtmp, - &cp->cp_send_queue, - m_conn_item) { +/* Tear down queued messages. Every path that adds to
net/rds/connection.c static void rds_conn_path_quiesce(struct rds_conn_path *cp) · line 632
Quoted 10 lines
net/rds/connection.c +8
-0+* cp_send_queue does so under cp_lock; take it here too rather +* than rely on the argument that nothing can be adding at this +* point. +*/ +spin_lock_irqsave (&cp->cp_lock, flags); +list_splice_init (&cp->cp_send_queue, &purge); +spin_unlock_irqrestore (&cp->cp_lock, flags); +list_for_each_entry_safe (rm, rtmp, &purge, m_conn_item) {list_del_init (&rm->m_conn_item);BUG_ON (!list_empty (&rm->m_sock_item));
[Severity: High]
If a concurrent sender is executing rds_sendmsg(), can it add a message to both the socket queue and connection queue via rds_send_queue_rm() after the rds_destroy_pending() check?
If the newly added message is spliced here into the purge list, it will still be present on the socket queue (m_sock_item). Will that cause this BUG_ON() to trigger a kernel panic when the list_empty() check evaluates to false?
Quoted 1 line
net/rds/connection.c +0
-0rds_message_put (rm);
[ ... ]
net/rds/send.c int rds_sendmsg(struct socket *sock, struct msghdr *msg, size_t payload_len) · line 1369
Quoted 10 lines
net/rds/send.c +1
-2@@ -1366,8 +1366,7 @@ int rds_sendmsg(struct socket *sock, struct msghdr *msg, size_t payload_len) conn = rds_conn_create_outgoing(sock_net(sock->sk), &rs->rs_bound_addr, &daddr, - rs->rs_transport, - READ_ONCE(rs->rs_tos), + rs->rs_transport, rs->rs_tos,
[Severity: Low]
Does removing the READ_ONCE() wrapper here introduce a KCSAN data race warning?
Since rs_tos is read outside of the rs_lock and could be modified concurrently by SIOCRDSSETTOS, it seems a lockless read without READ_ONCE() might be unsafe.
Quoted 3 lines
net/rds/send.c +0
-0sock->sk->sk_allocation, scope_id);if (IS_ERR (conn)) {
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=9
[email protected]
AI reviewRe: [PATCH net-next v6 09/12] net/rds: take cp_lock to purge cp_send_queue in the quiesce
In reply to Allison HendersonThank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider.
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider.
Critical: 0 · High: 1 · Medium: 1 · Low: 0
If you believe none of the issues are legitimate, please reply to each issue of Medium severity or higher and explain why the AI is wrong. If any of them are legitimate and you plan to respin, please let patchwork know by including "pw-bot: cr" as a separate line at the end of your reply (one such reply per series is enough).
- [High] The commit message asserts that splicing cp_send_queue under cp_lock makes the purge "correct against a concurrent adder…
- [Medium] Undisclosed, unrelated hunk: the patch reverts READ_ONCE(rs->rs_tos) back to a plain rs->rs_tos load at the…
--
net/rds: take cp_lock to purge cp_send_queue in the quiesce
rds_conn_path_quiesce() used to walk cp_send_queue with no lock while every adder links onto that list under cp_lock. This patch splices the queue away under cp_lock and drops the message references outside it. A second hunk in net/rds/send.c changes the rs_tos argument passed to rds_conn_create_outgoing().
Quoted 23 lines
net/rds/connection.c +4
-4@@ -606,6 +606,8 @@ void rds_conn_shutdown(struct rds_conn_path *cp)static void rds_conn_path_quiesce (struct rds_conn_path *cp) {struct rds_message *rm, *rtmp; +unsigned long flags; +LIST_HEAD (purge);if (!cp->cp_transport_data)return ;@@ -617,10 +619,15 @@ void rds_conn_shutdown(struct rds_conn_path *cp)rds_conn_path_drop (cp, true);flush_work (&cp->cp_down_w); -/* tear down queued messages */ -list_for_each_entry_safe (rm, rtmp, - &cp->cp_send_queue, - m_conn_item) { +/* Tear down queued messages. Every path that adds to +* cp_send_queue does so under cp_lock; take it here too rather
net/rds/connection.c void rds_conn_shutdown(struct rds_conn_path *cp) · line 633
Quoted 10 lines
net/rds/connection.c +7
-0+* than rely on the argument that nothing can be adding at this +* point. +*/ +spin_lock_irqsave (&cp->cp_lock, flags); +list_splice_init (&cp->cp_send_queue, &purge); +spin_unlock_irqrestore (&cp->cp_lock, flags); +list_for_each_entry_safe (rm, rtmp, &purge, m_conn_item) {list_del_init (&rm->m_conn_item);BUG_ON (!list_empty (&rm->m_sock_item));rds_message_put (rm);
[Severity: High]
The commit message says the purge is now "correct against a concurrent adder regardless", and the new comment in rds_conn_path_quiesce() repeats that. Does taking cp_lock around list_splice_init() actually give that property? The splice makes the list manipulation atomic, but it does not establish any state that stops an adder, so there seem to be two cases left.
First, if the adder wins the race just before the splice. rds_send_queue_rm() links the socket item first, under rs_lock:
net/rds/send.c:rds_send_queue_rm() {
...
list_add_tail(&rm->m_sock_item, &rs->rs_send_queue);
...
spin_lock(&cp->cp_lock);
rm->m_inc.i_hdr.h_sequence = cpu_to_be64(cp->cp_next_tx_seq++);
list_add_tail(&rm->m_conn_item, &cp->cp_send_queue);
...
}So a message contributed by a live sender is spliced with m_sock_item still on rs->rs_send_queue, and the retained assertion in the purge loop
BUG_ON(!list_empty(&rm->m_sock_item));
fires for exactly the message the locking was added to cope with. Does this turn a racy walk into a deterministic BUG() in that case?
Second, if the adder wins just after the splice. list_splice_init() is a one-shot snapshot, and neither rds_send_queue_rm() nor rds_send_probe() re-tests rds_destroy_pending() under cp_lock. The only destroy test before queueing is in rds_sendmsg(), taken outside cp_lock and with a potentially blocking rds_cong_wait() in between:
net/rds/send.c:rds_sendmsg() {
...
if (rds_destroy_pending(conn)) {
ret = -EAGAIN;
goto out;
}
...
ret = rds_cong_wait(conn->c_fcong, dport, nonblock, rs);
...
while (!rds_send_queue_rm(rs, conn, cpath, rm, rs->rs_bound_port,
dport, &queued)) {
...
}Is anything revisiting a message linked after the unlock? rds_conn_destroy() runs the quiesce once under c_destroy_in_prog, and rds_conn_path_free() / rds_conn_destroy_fini() neither drain nor warn about cp_send_queue, so the connection's reference on that rds_message looks leaked. For a rds_send_probe() message that reference is the only owner. For a sendmsg message, rm->m_conn_path keeps pointing into the c_path array that rds_conn_destroy_fini() kfree()s, which would be a use-after-free when the socket later closes and rds_send_drop_to() takes rm->m_conn_path->cp_lock.
Would mutual exclusion rather than atomicity be needed here - adders testing the connection's destroy state under cp_lock, or the quiesce publishing a "queue closed" marker under cp_lock - plus handling rather than asserting on a message still owned by a socket? At minimum, would a WARN_ON(!list_empty(&cp->cp_send_queue)) re-check after the splice make a missed message diagnosable?
For what it is worth, with the destroy triggers that exist at this commit and at the end of the series (netns teardown and transport module unload, both of which exclude a live sending socket), I could not construct a sending socket that reaches this today, so the trigger appears unproven in the current tree. The code in rds_conn_path_quiesce() is unchanged at the end of the series.
net/rds/send.c int rds_sendmsg(struct socket *sock, struct msghdr *msg, size_t payload_len) · line 1369
Quoted 11 lines
net/rds/send.c +1
-2@@ -1366,8 +1366,7 @@ int rds_sendmsg(struct socket *sock, struct msghdr *msg, size_t payload_len) conn = rds_conn_create_outgoing(sock_net(sock->sk), &rs->rs_bound_addr, &daddr, - rs->rs_transport, - READ_ONCE(rs->rs_tos), + rs->rs_transport, rs->rs_tos,
[Severity: Medium]
Is this hunk intentional? The subject and the changelog describe only the cp_lock and list_splice change in rds_conn_path_quiesce() and never mention net/rds/send.c or rs_tos, but this reverts the READ_ONCE() that the immediately preceding patch of the series, "net/rds: hold connection references in lookup, sockets and c_passive", added here. That patch's message states that "the create in rds_sendmsg() samples rs_tos without the lock".
The comment it added is still right below, and now documents a lockless read that the code no longer marks:
net/rds/send.c:rds_sendmsg() {
...
/* rs_tos was sampled without rs_lock for the create above,
* and SIOCRDSSETTOS only refuses a change once rs_conn is
* set, so it can have changed underneath us.
...
}The concurrent writer is the SIOCRDSSETTOS path in rds_ioctl(), a plain store under rs_lock reachable from unprivileged userspace, and every other rs_tos access in net/rds/ is taken under rs_lock, so after this change this is the only unmarked side of a plain/plain concurrent access.
rs_tos is a u8 so it cannot tear, and the sampled value is re-validated under rs_lock a few lines further down before the connection is installed, so the observable effect looks limited to a KCSAN report plus a hunk the changelog does not describe. The plain read is still there at the end of the series, so no later patch restores it. Would restoring the READ_ONCE(), or splitting the removal into its own patch with a justification, be preferable?
Signature · 1 line
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20260922085410.391323-1-achender%40kernel.org
Allison Henderson
maintainerPATCH v6 10/12 net/rds: pin the connection across RDMA-CM event handling
In reply to Allison Hendersonrds_rdma_cm_event_handler_cmn() picks the connection up from cm_id->context, which carries no reference, and holds c_cm_lock - a mutex that lives in the connection's path array - across the transport callbacks. Before this series that was already a use-after-free whenever a callback destroyed the connection, since rds_conn_destroy() freed it synchronously and the handler's mutex_unlock() ran on freed memory; the one such callback, rds_ib_cm_connect_complete() on a protocol version below 3.1, has meanwhile been switched to rds_conn_drop() by commit f97d8c7bab78 ("rds: ib: use rds_conn_drop() on protocol version mismatch"), which also removed the deadlock that destroy took on c_cm_lock.
Now that a connection is freed by its last reference, the remaining exposure is a callback that drops the last reference other than the handler's - which has none - and the reference handed out by rds_conn_create() to rds_ib_cm_handle_connect() and dropped at its end. Take a reference for the duration of the handler, and ignore the event if the connection is already being freed: its cm_id teardown is what stops event delivery, so an event that still arrives belongs to a connection whose shutdown has run and whose memory is on its way out.
rds_ib_cm_handle_connect() has the mirror-image hole: a connection whose destroy has already quiesced it sits in RDS_CONN_DOWN with no cm_id, which is exactly the state the DOWN -> CONNECTING transition claims. A connect request arriving then would install a new cm_id and QP on a connection that is only waiting for its last reference to go away, and nothing would tear them down again. Re-check rds_destroy_pending() under c_cm_lock and reject the request instead.
--- net/rds/ib_cm.c | 11 +++++++++-- net/rds/rdma_transport.c | 16 +++++++++++++++- 2 files changed, 24 insertions(+), 3 deletions(-)
net/rds/ib_cm.c +9 -2
@@ -874,6 +874,13 @@ int rds_ib_cm_handle_connect(struct rdma_cm_id *cm_id,* see the comment above rds_queue_reconnect() */ mutex_lock (&conn->c_cm_lock); +/* A destroy that has already quiesced this conn leaves it in +* RDS_CONN_DOWN with no cm_id, exactly what the transition +* below would happily claim; nothing would tear the new cm_id +* and QP down again before the conn is freed. Reject instead. +*/ +if (rds_destroy_pending (conn)) + goto out;if (!rds_conn_transition (conn,RDS_CONN_DOWN ,RDS_CONN_CONNECTING )) {if (rds_conn_state (conn) ==RDS_CONN_UP ) {rdsdebug ("incoming connect while connecting\n" );
@@ -928,8 +935,8 @@ int rds_ib_cm_handle_connect(struct rdma_cm_id *cm_id,mutex_unlock (&conn->c_cm_lock);/* Drop the reference rds_conn_create() handed us. The * conn stays reachable through cm_id->context without a -* reference of its own for now; the CM event handler is -* given one of its own by a following patch. +* reference of its own; rds_rdma_cm_event_handler_cmn() +* takes one for the duration of each event it handles. */ rds_conn_put (conn); }
net/rds/rdma_transport.c +15 -1
@@ -63,6 +63,18 @@ static int rds_rdma_cm_event_handler_cmn(struct rdma_cm_id *cm_id,if (cm_id->device->node_type ==RDMA_NODE_IB_CA ) trans = &rds_ib_transport; +/* cm_id->context carries no reference of its own. Pin the +* connection for the duration of the handler: what the callbacks +* below do may drop the last reference other than ours, and the +* mutex released at out: lives in the connection's path array. +* A connection already being freed gets no events handled. +*/ +if (conn && !rds_conn_get_unless_zero (conn)) { +rdsdebug ("conn %p id %p is being freed, ignoring event\n" , + conn, cm_id); +return 0 ; + } +/* Prevent shutdown from tearing down the connection * while we're executing. */ if (conn) {
@@ -171,8 +183,10 @@ static int rds_rdma_cm_event_handler_cmn(struct rdma_cm_id *cm_id, } out: -if (conn) +if (conn) {mutex_unlock (&conn->c_cm_lock); +rds_conn_put (conn); + }rdsdebug ("id %p event %u (%s) handling ret %d\n" , cm_id, event->event,rdma_event_msg (event->event), ret);
Signature · 1 line
2.25.1
[email protected]
AI reviewRe: [PATCH net-next v6 10/12] net/rds: pin the connection across RDMA-CM event handling
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 19 lines
rds_rdma_cm_event_handler_cmn()picks the connection up fromcm_id->context, which carries no reference, and holdsc_cm_lock- a mutex that lives in the connection's path array - across the transport callbacks. Before this series that was already a use-after-free whenever a callback destroyed the connection, sincerds_conn_destroy()freed it synchronously and the handler'smutex_unlock()ran on freed memory; the one such callback,rds_ib_cm_connect_complete()on a protocol version below 3.1, has meanwhile been switched tords_conn_drop()by commitf97d8c7bab78("rds: ib: userds_conn_drop()on protocol version mismatch"), which also removed the deadlock that destroy took onc_cm_lock.Now that a connection is freed by its last reference, the remaining exposure is a callback that drops the last reference other than the handler's - which has none - and the reference handed out by [ ... ]
away, and nothing would tear them down again. Re-checkrds_destroy_pending()underc_cm_lockand reject the request instead.
Quoted 2 lines
Assisted-by Claude-Code:claude-fable-5Signed-off-by Allison Henderson <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=10
[email protected]
AI reviewRe: [PATCH net-next v6 10/12] net/rds: pin the connection across RDMA-CM event handling
In reply to Allison HendersonThank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider.
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider.
Critical: 0 · High: 0 · Medium: 1 · Low: 0
If you believe none of the issues are legitimate, please reply to each issue of Medium severity or higher and explain why the AI is wrong. If any of them are legitimate and you plan to respin, please let patchwork know by including "pw-bot: cr" as a separate line at the end of your reply (one such reply per series is enough).
- [Medium] The rationale attached to the new reference pin in rds_rdma_cm_event_handler_cmn() (net/rds/rdma_transport.c:66-72) and…
--
net/rds: pin the connection across RDMA-CM event handling
This takes a reference on the connection found via cm_id->context for the duration of rds_rdma_cm_event_handler_cmn(), ignoring the event if the connection is already being freed, and re-checks rds_destroy_pending() under c_cm_lock in rds_ib_cm_handle_connect() so a connect request cannot install a new cm_id and QP on a connection that is only waiting for its last reference.
Quoted 4 lines
net/rds/ib_cm.c +0
-0
[ ... ]
Quoted 20 lines
net/rds/ib_cm.c +2
-2@@ -928,8 +935,8 @@ int rds_ib_cm_handle_connect(struct rdma_cm_id *cm_id,mutex_unlock (&conn->c_cm_lock);/* Drop the reference rds_conn_create() handed us. The * conn stays reachable through cm_id->context without a -* reference of its own for now; the CM event handler is -* given one of its own by a following patch. +* reference of its own; rds_rdma_cm_event_handler_cmn() +* takes one for the duration of each event it handles. */ rds_conn_put (conn); }net/rds/rdma_transport.c +1
-0@@ -63,6 +63,18 @@ static int rds_rdma_cm_event_handler_cmn(struct rdma_cm_id *cm_id,if (cm_id->device->node_type ==RDMA_NODE_IB_CA ) trans = &rds_ib_transport; +/* cm_id->context carries no reference of its own. Pin the
net/rds/rdma_transport.c static int rds_rdma_cm_event_handler_cmn(struct rdma_cm_id *cm_id, · line 76
Quoted 10 lines
net/rds/rdma_transport.c +10
-0+* connection for the duration of the handler: what the callbacks +* below do may drop the last reference other than ours, and the +* mutex released at out: lives in the connection's path array. +* A connection already being freed gets no events handled. +*/ +if (conn && !rds_conn_get_unless_zero (conn)) { +rdsdebug ("conn %p id %p is being freed, ignoring event\n" , + conn, cm_id); +return 0 ; + }
[Severity: Medium]
The get_unless_zero/put pair itself looks balanced, but is the stated reason for it accurate? The comment says "what the callbacks below do may drop the last reference other than ours", and the changelog says the same:
Now that a connection is freed by its last reference, the remaining exposure is a callback that drops the last reference other than the handler's
Walking the callbacks dispatched from the switch below, none of them puts a reference on the connection the handler just pinned.
cm_connect_complete -> rds_ib_cm_connect_complete() only drops the connection on a version mismatch, and after commit f97d8c7bab78 that is rds_conn_drop(), which takes no reference action:
net/rds/connection.c:rds_conn_path_drop() {
atomic_set(&cp->cp_state, RDS_CONN_ERROR);
rcu_read_lock();
if (!destroy && rds_destroy_pending(cp->cp_conn)) {
rcu_read_unlock();
return;
}
queue_work(cp->cp_wq, &cp->cp_down_w);
rcu_read_unlock();
}cm_initiate_connect -> rds_ib_cm_initiate_connect() contains no put, and the remaining switch arms only call rds_conn_drop().
cm_handle_connect -> rds_ib_cm_handle_connect() does call rds_conn_put(), but that releases the reference rds_conn_create() handed to that same function, and it runs after its own mutex_unlock(&conn->c_cm_lock). On RDMA_CM_EVENT_CONNECT_REQUEST the handler's conn is NULL anyway, since the listener cm_id is created with a NULL context:
net/rds/rdma_transport.c:rds_rdma_listen() {
cm_id = rdma_create_id(&init_net, handler, NULL,
RDMA_PS_TCP, IB_QPT_RC);
}which is what BUG_ON(cm_id->context) in rds_ib_cm_handle_connect() asserts, so nothing was pinned on that path.
That leaves holders outside the handler as the ones that can reach zero concurrently - rds_conn_destroy() dropping the initial reference, a socket dropping rs_conn, a c_passive drop, or an inc drop - which is a different invariant than the one written down. Could the comment and the changelog name the holder that can actually race here, or say plainly that the pin is defensive? As written, a later reader auditing the callbacks will find no put and may conclude the pin is dead code, and the same wording is what a backporter would use to decide whether this is a fix or hardening.
I also checked the end of the series (fe1de9d527be84f2ab7fed636cc4915635b09f0d) and the comment is unchanged there, so no later patch adjusts it.
Signature · 1 line
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20260922085410.391323-1-achender%40kernel.org
Allison Henderson
maintainerPATCH v6 11/12 net/rds: drop rds_conn_count in favor of t_conn_count
In reply to Allison HendersonThe previous patch gave each transport its own connection count in t_conn_count, incremented and decremented at exactly the points where the global rds_conn_count is. That leaves rds_conn_count with a single remaining consumer: the seed of the per-path workqueue names in __rds_conn_create().
Switch the name seed to t_conn_count, as UEK does, and remove rds_conn_count. The numbering becomes per-transport instead of global, so connections of different transports can now receive the same seed; workqueue names carry no uniqueness requirement, and the seed was already reused as the count rose and fell. Removing the counter also removes the rds_conn_lock round-trip that rds_conn_destroy_fini() took solely to decrement it. (The free path still sleeps in destroy_workqueue(), so the last reference has to be dropped from process context as before; only that one lock goes.)
Based on the Oracle UEK commit "net/rds: Each RDS transport should keep its own connection count".
--- net/rds/connection.c | 13 +++---------- 1 file changed, 3 insertions(+), 10 deletions(-)
net/rds/connection.c +3 -10
@@ -47,7 +47,6 @@/* converting this to RCU is a chore for another day.. */ static DEFINE_SPINLOCK (rds_conn_lock); -static unsigned long rds_conn_count;/* woken whenever a transport's t_conn_count drops to zero */ static DECLARE_WAIT_QUEUE_HEAD (rds_conn_freed_waitq);static struct hlist_head rds_conn_hash[RDS_CONNECTION_HASH_ENTRIES ];
@@ -332,12 +331,13 @@ static struct rds_connection *__rds_conn_create(struct net *net,init_waitqueue_head (&conn->c_hs_waitq);for (i =0 ; i < npaths; i++) { +int seq =atomic_read (&trans->t_conn_count); +__rds_conn_path_init (conn, &conn->c_path[i], is_outgoing); conn->c_path[i].cp_index = i; conn->c_path[i].cp_wq = -alloc_ordered_workqueue ("krds_cp_wq#%lu/%d" ,0 , - rds_conn_count, i); +alloc_ordered_workqueue ("krds_cp_wq#%d/%d" ,0 , seq, i);if (!conn->c_path[i].cp_wq) conn->c_path[i].cp_wq = rds_wq; }
@@ -408,7 +408,6 @@ static struct rds_connection *__rds_conn_create(struct net *net,rds_conn_get (conn);/* caller */ rcu_assign_pointer (parent->c_passive, conn);rds_cong_add_conn (conn); - rds_conn_count++;atomic_inc (&conn->c_trans->t_conn_count); } }else {
@@ -431,7 +430,6 @@ static struct rds_connection *__rds_conn_create(struct net *net,rds_conn_get (conn);hlist_add_head_rcu (&conn->c_hash_node, head);rds_cong_add_conn (conn); - rds_conn_count++;atomic_inc (&conn->c_trans->t_conn_count); } }
@@ -668,7 +666,6 @@ static void rds_conn_destroy_fini(struct kref *kref) c_refcount);int npaths = (conn->c_trans->t_mp_capable ?RDS_MPATH_WORKERS :1 );struct rds_transport *trans = conn->c_trans; -unsigned long flags;int i;for (i =0 ; i < npaths; i++)
@@ -677,10 +674,6 @@ static void rds_conn_destroy_fini(struct kref *kref)kfree (conn->c_path);kmem_cache_free (rds_conn_slab, conn); -spin_lock_irqsave (&rds_conn_lock, flags); - rds_conn_count--; -spin_unlock_irqrestore (&rds_conn_lock, flags); -/* only after everything the transport module owns has been * freed above may its unload proceed */
Signature · 1 line
2.25.1
[email protected]
AI reviewRe: [PATCH net-next v6 11/12] net/rds: drop rds_conn_count in favor of t_conn_count
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 16 lines
The previous patch gave each transport its own connection count in
t_conn_count, incremented and decremented at exactly the points where the globalrds_conn_countis. That leavesrds_conn_countwith a single remaining consumer: the seed of the per-path workqueue names in__rds_conn_create().Switch the name seed to
t_conn_count, as UEK does, and removerds_conn_count. The numbering becomes per-transport instead of global, so connections of different transports can now receive the same seed; workqueue names carry no uniqueness requirement, and the seed was already reused as the count rose and fell. Removing the counter also removes therds_conn_lockround-trip thatrds_conn_destroy_fini()took solely to decrement it. (The free path still sleeps indestroy_workqueue(), so the last reference has to be dropped from process context as before; only that one lock goes.)
Quoted 5 lines
Based on the Oracle UEK commit "net/rds: Each RDS transport should keep its own connection count".
Assisted-by Claude-Code:claude-fable-5Signed-off-by Allison Henderson <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=11
Allison Henderson
maintainerPATCH v6 12/12 net/rds: hold a connection reference from struct rds_incoming
In reply to Allison HendersonFrom: Sharath Srinivasan <[email protected]>
struct rds_incoming->i_conn stores a pointer to the connection a message it belongs to, for both received messages and messages the socket sends. But without taking a reference, nothing keeps that connection alive. Embedded as the messages m_inc, an inc routinely outlives the connection it points at, by sitting in the socket's receive queue until the application reads it, while the connection is destroyed by device removal, netns teardown or module unload - and every dereference of i_conn after that point touches freed memory.
Chengfeng Ye reported one way to reach it, where the socket info callbacks walk a receive queue after rmmod freed the connections:
BUG: KASAN: slab-use-after-free in rds6_inc_info_copy+0x459/0x530 [rds] Read of size 1 at addr ffff888106031c50 by task poc/101 Call Trace: rds6_inc_info_copy+0x459/0x530 [rds] rds6_sock_inc_info+0x2b9/0x3c0 [rds] rds_info_getsockopt+0x19d/0x380 [rds] do_sock_getsockopt+0x2ac/0x480 __sys_getsockopt+0x128/0x210 Freed by task 102: kmem_cache_free+0x1b5/0x3d0 rds_conn_destroy+0x484/0x600 [rds] rds_loop_exit_net+0x32/0x50 [rds] unregister_pernet_device+0x2c/0x50 rds_conn_exit+0x13/0xa0 [rds] rds_exit+0x1a/0xc40 [rds] __do_sys_delete_module+0x30a/0x4d0
Closing the socket gets there too, with no reader of i_conn other than RDS itself: freeing an IB inc dereferences i_conn to hand the inc and its fragments back to the connection's recycle cache, so draining the receive queue of a socket whose connection is gone crashes in the transport:
panic
...
rds_ib_recv_cache_put (net/rds/ib_recv.c:703)
rds_ib_inc_free (net/rds/ib_recv.c:207)
rds_clear_recv_queue (net/rds/recv.c:909)
rds_release (net/rds/af_rds.c:212)
__sock_release (net/socket.c:649)
sock_close (net/socket.c:1336)
with the freed connection confirmed by its now-zero reference count:
-trace[9]["inc"].i_conn.c_refcount
(struct kref){
.refcount = (refcount_t){
.refs = (atomic_t){
.counter = (int)0,
},
},
}Now that connections are reference counted, make every holder of i_conn own a reference. The pointer is assigned in six places - rds_inc_init() and rds_inc_path_init() for received messages, rds_recv_incoming() when it re-points an inc at the connection it arrived on, and rds_send_queue_rm(), rds_send_probe() and the congestion-map path of rds_send_xmit() for m_inc - and each of them now takes a reference. The references are dropped from rds_inc_put() and rds_message_put(), which are the points where the last user of the pointer goes away. rds_recv_incoming() takes the new reference before dropping the old one, so re-pointing an inc at the connection it already refers to cannot free it. rds_inc_put() drops its reference through a local copy, since inc_free() may free the memory the inc lives in.
This keeps a connection allocated for as long as messages that arrived over it are queued on sockets, which is longer than before. What lingers is the quiesced connection, its per-path workqueues (idle, every work cancelled) and the transport's per-connection state, including an IB connection's receive caches: rds_conn_destroy() still quiesces the connection synchronously, so nothing runs on any of it. It also means the transport cannot unload while such a datagram is unread - rds_conn_wait_conns_freed() waits, warning every ten seconds, until the application reads or closes. That is the intended
behaviour: teardown does not discard queued data, and unloading over a live reference would be a use-after-free in the transport's text.
The final rds_conn_put() runs the free path, which destroys the per-path workqueues and therefore may sleep, so the last reference has to be dropped from process context. It always is. While a connection is alive its hash-table entry holds the initial reference, so a put from a completion handler or tasklet can never be the last one; that reference is dropped by rds_conn_destroy(), from process context, after the connection has been quiesced and its queued messages freed. The references that survive that point are the ones held by incs on socket receive queues and by messages on socket send queues, and those are dropped from recvmsg and from close - process context in both cases.
This is not a stable candidate: reaching the use-after-free requires freeing a connection out from under a live socket, which needs CAP_SYS_MODULE, netns teardown or physical device removal, and the fix depends on the connection reference counting introduced earlier in this series.
Based on the Oracle UEK commit "net/rds: fix crash by expanding kref coverage to rds_incoming.i_conn".
[achender: port to net-next: same six assignment sites, but upstream
splits the message free across rds_message_unpin_worker(), so the m_inc reference is dropped from a shared rds_message_free() helper that both paths call; rds_recv_incoming() takes the new reference before dropping the old; rewrite commit message]
--- net/rds/message.c | 16 ++++++++++++++-- net/rds/recv.c | 26 +++++++++++++++++++++++--- net/rds/send.c | 4 ++++ 3 files changed, 41 insertions(+), 5 deletions(-)
net/rds/message.c +14 -2
@@ -182,6 +182,18 @@ static void rds_message_purge(struct rds_message *rm)kref_put (&rm->atomic.op_rdma_mr->r_kref, __rds_put_mr_final); } +static void rds_message_free (struct rds_message *rm) +{ +/* get in rds_send_queue_rm(), rds_send_probe() or the congestion +* map path of rds_send_xmit(). Messages that were never queued on +* a connection have no reference to drop. +*/ +if (rm->m_inc.i_conn) +rds_conn_put (rm->m_inc.i_conn); + +kfree (rm); +} +static void rds_message_unpin_worker (struct work_struct *work) { struct rds_message *rm = container_of(work, struct rds_message,
@@ -192,7 +204,7 @@ static void rds_message_unpin_worker(struct work_struct *work)if (rm->atomic.op_unpin_deferred)rds_atomic_op_unpin_page (&rm->atomic); -kfree (rm); +rds_message_free (rm); }void rds_message_put (struct rds_message *rm)
@@ -217,7 +229,7 @@ void rds_message_put(struct rds_message *rm)return ; } -kfree (rm); +rds_message_free (rm); } }EXPORT_SYMBOL_GPL (rds_message_put);
net/rds/recv.c +23 -3
@@ -46,6 +46,7 @@ void rds_inc_init(struct rds_incoming *inc, struct rds_connection *conn, {refcount_set (&inc->i_refcount,1 );INIT_LIST_HEAD (&inc->i_item); +rds_conn_get (conn);/* put in rds_inc_put() */ inc->i_conn = conn; inc->i_conn_path =NULL ; inc->i_saddr = *saddr;
@@ -61,6 +62,7 @@ void rds_inc_path_init(struct rds_incoming *inc, struct rds_conn_path *cp, {refcount_set (&inc->i_refcount,1 );INIT_LIST_HEAD (&inc->i_item); +rds_conn_get (cp->cp_conn);/* put in rds_inc_put() */ inc->i_conn = cp->cp_conn; inc->i_conn_path = cp; inc->i_saddr = *saddr;
@@ -81,9 +83,19 @@ void rds_inc_put(struct rds_incoming *inc) {rdsdebug ("put inc %p ref %d\n" , inc,refcount_read (&inc->i_refcount));if (refcount_dec_and_test (&inc->i_refcount)) { +struct rds_connection *conn = inc->i_conn; +BUG_ON (!list_empty (&inc->i_item)); - inc->i_conn->c_trans->inc_free (inc); +/* inc_free() can free the memory @inc lives in, so the +* connection reference has to be dropped through the +* copy taken above. +*/ + conn->c_trans->inc_free (inc); +/* get in rds_inc_init(), rds_inc_path_init() or +* rds_recv_incoming() +*/ +rds_conn_put (conn); } }EXPORT_SYMBOL_GPL (rds_inc_put);
@@ -325,6 +337,13 @@ void rds_recv_incoming(struct rds_connection *conn, struct in6_addr *saddr,unsigned long flags;struct rds_conn_path *cp; +/* every caller initialized @inc with rds_inc_init() or +* rds_inc_path_init() first, so i_conn already holds a reference. +* Take the new one before dropping the old, so that re-pointing an +* inc at the connection it already refers to cannot free it. +*/ +rds_conn_get (conn); +rds_conn_put (inc->i_conn); inc->i_conn = conn; inc->i_rx_jiffies = jiffies;if (conn->c_trans->t_mp_capable)
@@ -406,8 +425,9 @@ void rds_recv_incoming(struct rds_connection *conn, struct in6_addr *saddr,* rds_find_bound() uses a global (netns-agnostic) hash table. * An RDS connection created in netns A can match a socket bound * in the init netns, delivering inc cross-netns with inc->i_conn -* pointing into netns A. When cleanup_net() then frees that conn, -* any subsequent dereference of inc->i_conn is a use-after-free. +* pointing into netns A. Cross-netns delivery is wrong on its +* own, and the inc's connection reference would keep a +* connection of a dead netns around, with a stale c_net. * Drop the inc if the receiving socket lives in a different netns. */ if (!net_eq (sock_net (rds_rs_to_sk (rs)),rds_conn_net (conn))) {
net/rds/send.c +4 -0
@@ -290,6 +290,8 @@ int rds_send_xmit(struct rds_conn_path *cp) } rm->data.op_active =1 ; rm->m_inc.i_conn_path = cp; +/* put in rds_message_put() */ +rds_conn_get (cp->cp_conn); rm->m_inc.i_conn = cp->cp_conn; cp->cp_xmit_rm = rm;
@@ -947,6 +949,7 @@ static int rds_send_queue_rm(struct rds_sock *rs, struct rds_connection *conn,/* The code ordering is a little weird, but we're trying to minimize the time we hold c_lock */ rds_message_populate_header (&rm->m_inc.i_hdr, sport, dport,0 ); +rds_conn_get (conn);/* put in rds_message_put() */ rm->m_inc.i_conn = conn; rm->m_inc.i_conn_path = cp;rds_message_addref (rm);
@@ -1570,6 +1573,7 @@ rds_send_probe(struct rds_conn_path *cp, __be16 sport,list_add_tail (&rm->m_conn_item, &cp->cp_send_queue);set_bit (RDS_MSG_ON_CONN , &rm->m_flags);rds_message_addref (rm); +rds_conn_get (cp->cp_conn);/* put in rds_message_put() */ rm->m_inc.i_conn = cp->cp_conn; rm->m_inc.i_conn_path = cp;
Signature · 1 line
2.25.1
[email protected]
AI reviewRe: [PATCH net-next v6 12/12] net/rds: hold a connection reference from struct rds_incoming
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 14 lines
From: Sharath Srinivasan <[email protected]>
struct
rds_incoming->i_connstores a pointer to the connection a message it belongs to, for both received messages and messages the socket sends. But without taking a reference, nothing keeps that connection alive. Embedded as the messagesm_inc, an inc routinely outlives the connection it points at, by sitting in the socket's receive queue until the application reads it, while the connection is destroyed by device removal, netns teardown or module unload - and every dereference ofi_connafter that point touches freed memory.Chengfeng Ye reported one way to reach it, where the socket info callbacks walk a receive queue after rmmod freed the connections:
Quoted 7 lines
BUG: KASAN: slab-use-after-free in rds6_inc_info_copy+0x459/0x530 [rds][ ... ]
m_increference is dropped from a sharedrds_message_free()helper that both paths call;rds_recv_incoming()takes the new reference before dropping the old; rewrite commit message]Assisted-by Claude-Code:claude-opus-5Signed-off-by Allison Henderson <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=12