RDMA/ucma: reject a second join of the same multicast address
Subsystems (2)
PATCH 1/1 RDMA/ucma: reject a second join of the same multicast address
ucma_process_join() lets one cm_id join the same multicast address twice, creating two ucma_multicast objects (A and B) and, via rdma_join_multicast(), two cma_multicast entries that both point back at the ucma object.
rdma_leave_multicast() matches a leave only by sockaddr and destroys the first entry it finds. So LEAVE_MCAST on A destroys B's cma_multicast and frees ucma object A, while A's cma_multicast stays on the list with a context pointing at freed memory. The next event for it is a use-after-free:
BUG: KASAN: slab-use-after-free in ucma_create_uevent+0xb11/0xb70 [rdma_ucm] Workqueue: rdma_cm cma_iboe_join_work_handler [rdma_cm] Call Trace: ucma_create_uevent+0xb11/0xb70 [rdma_ucm] ucma_event_handler+0xd5/0xa10 [rdma_ucm] cma_cm_event_handler+0x98/0x310 [rdma_cm] cma_iboe_join_work_handler+0xac/0x170 [rdma_cm] process_scheduled_works+0xb59/0x1890 worker_thread+0x470/0xe40 kthread+0x33a/0x450 ret_from_fork+0x59/0x90 ret_from_fork_asm+0x1a/0x30
Any user with access to /dev/infiniband/rdma_cm can trigger it.
Reject a second join of an address already joined on the same cm_id. The incomplete fix 662ade4de9ff ("RDMA/ucma: Serialize join and leave on copy_to_user failure") left this path open.
---
Found during a backport review.
Reproduced on debian13 6.12.88 debug (KASAN=y) over Soft-RoCE: two joins of one address on a cm_id, then LEAVE -> the KASAN use-after-free above within one run. With the fix the second join returns -EADDRINUSE and the run is clean.
drivers/infiniband/core/ucma.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)
drivers/infiniband/core/ucma.c +8 -1
@@ -1509,7 +1509,7 @@ {struct rdma_ucm_create_id_resp resp;struct ucma_context *ctx; -struct ucma_multicast *mc; +struct ucma_multicast *mc, *tmp;struct sockaddr *addr;int ret;u8 join_state;
@@ -1544,6 +1544,13 @@memcpy (&mc->addr, addr, cmd->addr_size);xa_lock (&multicast_table); +/* leave matches by address only; refuse a duplicate */ +list_for_each_entry (tmp, &ctx->mc_list, list) +if (!memcmp (&tmp->addr, addr, cmd->addr_size)) { + ret = -EADDRINUSE ; + goto err_free_mc; + } +if (__xa_alloc (&multicast_table, &mc->id,NULL , xa_limit_32b,GFP_KERNEL )) { ret = -ENOMEM ;
[email protected]
AI reviewThank you for your contribution! Sashiko AI review found 2 potential issues to consider: - [Critical] RDMA/ucma: UAF race between ucma_leave_multicast() and ucma_process_join() - [High] RDMA/ucma…
Thank you for your contribution! Sashiko AI review found 2 potential issues to consider:
- [Critical] RDMA/ucma: UAF race between ucma_leave_multicast() and ucma_process_join()
- [High] RDMA/ucma: TOCTOU race in ucma_process_join() across __xa_alloc() lock drop
--
commit 92134efcefe26ef534f470da9d6476bb2827da65
Author: Andrey Misiurov <[email protected]>
RDMA/ucma: reject a second join of the same multicast address
This commit rejects duplicate multicast joins for the same address on a ucma_context by checking ctx->mc_list under multicast_table's XArray lock before allocating an ID.
Quoted 14 lines
drivers/infiniband/core/ucma.c +1
-1@@ -1509,7 +1509,7 @@ static ssize_t ucma_process_join(struct ucma_file *file, {struct rdma_ucm_create_id_resp resp;struct ucma_context *ctx; -struct ucma_multicast *mc; +struct ucma_multicast *mc, *tmp;struct sockaddr *addr;int ret;u8 join_state;
drivers/infiniband/core/ucma.c static ssize_t ucma_process_join(struct ucma_file *file, · line 1552
Quoted 9 lines
drivers/infiniband/core/ucma.c +6
-0@@ -1544,6 +1544,13 @@ static ssize_t ucma_process_join(struct ucma_file *file,memcpy (&mc->addr, addr, cmd->addr_size);xa_lock (&multicast_table); +/* leave matches by address only; refuse a duplicate */ +list_for_each_entry (tmp, &ctx->mc_list, list) +if (!memcmp (&tmp->addr, addr, cmd->addr_size)) { + ret = -EADDRINUSE ; + goto err_free_mc; + }
[Severity: Critical]
Can a concurrent ucma_leave_multicast() and ucma_process_join() for the same address bypass this check and cause a use-after-free of the leaving ucma_multicast?
ucma_leave_multicast() removes the old entry from ctx->mc_list and drops multicast_table's xa_lock before acquiring ctx->mutex to call rdma_leave_multicast(). A join entering in that window will not see the address on ctx->mc_list and can take ctx->mutex first:
CPU0 (leave mc_A) CPU1 (re-join same addr, mc_B)ucma_leave_multicast()
xa_lock(&multicast_table)
list_del(&mc_A->list)
xa_unlock(&multicast_table)
ucma_process_join()
xa_lock(&multicast_table)
/* mc_A already unlinked */
list_add_tail(&mc_B->list,
&ctx->mc_list)
xa_unlock(&multicast_table)
mutex_lock(&ctx->mutex)
rdma_join_multicast(..., mc_B)
/* prepends cma_mc_B to
id_priv->mc_list */
mutex_unlock(&ctx->mutex)
mutex_lock(&ctx->mutex)
rdma_leave_multicast()
/* matches by addr from head,
destroys cma_mc_B instead
of cma_mc_A */
mutex_unlock(&ctx->mutex)
kfree(mc_A)
cma_iboe_join_work_handler()
...
ucma_create_uevent()
/* dereferences mc_A */
<-- use-after-freeBecause rdma_join_multicast() inserts at the head of id_priv->mc_list with list_add(), CPU0's rdma_leave_multicast() destroys CPU1's newly joined cma_multicast entry while leaving cma_mc_A active and pointing to freed mc_A. Checking and updating ctx->mc_list (or serializing the join and leave operations) under ctx->mutex would close this window.
drivers/infiniband/core/ucma.c line 1556
Quoted 4 lines
drivers/infiniband/core/ucma.c +1
-0+if (__xa_alloc (&multicast_table, &mc->id,NULL , xa_limit_32b,GFP_KERNEL )) { ret = -ENOMEM ;
[Severity: High]
Can two concurrent joins of the same address both pass the duplicate check when __xa_alloc() drops multicast_table's xa_lock to allocate an XArray node?
Because __xa_alloc() is called with GFP_KERNEL before mc is added to ctx->mc_list, __xas_nomem() will drop and reacquire multicast_table's xa_lock if it needs to allocate a radix_tree_node:
CPU0 CPU1ucma_process_join()
xa_lock(&multicast_table)
/* checks ctx->mc_list: empty */
__xa_alloc(..., GFP_KERNEL)
__xas_nomem()
xas_unlock_type()
ucma_process_join()
xa_lock(&multicast_table)
/* checks ctx->mc_list: still empty */
__xa_alloc(...)
list_add_tail(&mc1->list,
&ctx->mc_list)
xa_unlock(&multicast_table)
xas_lock_type()
list_add_tail(&mc0->list,
&ctx->mc_list) xa_unlock(&multicast_table)
Both joins then succeed for the same address on the same cm_id, recreating the duplicate cma_multicast state and subsequent use-after-free on leave. Inserting mc into ctx->mc_list before __xa_alloc() (and unlinking it if __xa_alloc() fails), or serializing under ctx->mutex, avoids this race.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1