Skip to contentSkip to conversation

RDMA/rxe: fix responder UAF on IB_QP_MAX_DEST_RD_ATOMIC modify_qp

PATCH Superseded Ibrahim Hashimov 3 participants 6 messages Jul 8 – Jul 12

Subsystems (3)

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 RDMA/rxe: fix responder UAF on IB_QP_MAX_DEST_RD_ATOMIC modify_qp

rxe_qp_from_attr()'s IB_QP_MAX_DEST_RD_ATOMIC branch frees and reallocates qp->resp.resources[] (the rd_atomic resource array used by the responder to track in-flight RDMA READ/ATOMIC/FLUSH requests) completely outside of the IB_QP_STATE handling above it. Unlike every other place that tears this array down -- rxe_qp_reset(), reached only under IB_QP_STATE, always calls rxe_disable_task(&qp->recv_task) / rxe_disable_task(&qp->send_task) to drain the responder and requester tasks before touching per-QP state, then re-enables them -- this branch runs with the responder task (rxe_receiver(), scheduled as recv_task on the rxe_wq workqueue) fully live and unlocked. A userspace modify_qp() that sets only IB_QP_MAX_DEST_RD_ATOMIC (no state change, so __qp_chk_state()/ib_modify_qp_is_ok() never runs and qp->state_lock is never taken here) can therefore race the responder in two ways:

1. free_rd_atomic_resources() calls kfree(qp->resp.resources) and alloc_rd_atomic_resources() kzalloc_objs()'s a new array while rxe_prepare_res()/find_resource() in rxe_resp.c are concurrently walking &qp->resp.resources[i] with no lock held -- a straight free-vs-read race on the array itself.

2. free_rd_atomic_resources() only NULLs qp->resp.resources; it never clears qp->resp.res, the raw pointer *into* that array that rxe_resp.c caches across a multi-packet RDMA READ/ATOMIC/FLUSH reply (set at rxe_resp.c read/atomic/flush-reply sites, cleared only on the normal completion paths). If a modify_qp() races a resource still referenced by qp->resp.res, the array is freed out from under the cached pointer and the next reply packet dereferences it -- independent of the kfree/kzalloc_objs() window in (1).

Reproduced with KASAN: a single process driving one RC QP pair in rxe loopback, one thread pumping large multi-packet IBV_WR_RDMA_READs against qpB while a second thread hammers ibv_modify_qp(qpB, IB_QP_MAX_DEST_RD_ATOMIC), reliably (~11s) produces

  BUG: KASAN: slab-use-after-free in rxe_receiver+0x4f78/0x89e0 [rdma_rxe]
  Workqueue: rxe_wq do_work [rdma_rxe]

with the freed kmalloc-1k object being the rd_atomic resource array freed by the modify_qp() thread while the recv_task kworker reads it. An identical run modifying only IB_QP_MIN_RNR_TIMER (no resource free) is clean.

Fix both races the same way rxe_qp_reset() already handles tearing down this exact array: quiesce the responder task around the free/realloc by calling rxe_disable_task(&qp->recv_task) before free_rd_atomic_resources() and rxe_enable_task(&qp->recv_task) after alloc_rd_atomic_resources(), so rxe_receiver() cannot observe the array mid-free/mid-realloc. And close the still-open window for (2) at the source: have free_rd_atomic_resources() clear qp->resp.res along with qp->resp.resources, exactly like the existing completion paths in rxe_resp.c (check_rkey()/duplicate_request()/RESPST_CLEANUP) already do when a resource's lifetime ends, so a drained-and-resumed responder restarts at RESPST_CHK_PSN against the fresh array instead of replaying a stale reference into the old one.

Only qp->recv_task is drained: qp->resp.resources / qp->resp.res are touched exclusively by the responder (rxe_resp.c); the requester (send_task / rxe_sender()) never reads them, so there is no need to widen this beyond what rxe_qp_reset() would drain for the equivalent state.

Verified on the same v6.19 KASAN stand: with this fix applied, the identical differential reproducer drives sustained MAX_DEST_RD_ATOMIC storms against qpB well past the ~11s pre-fix time-to-first-splat with zero KASAN reports, versus reliably tripping the slab-use-after-free in rxe_receiver() described above before the fix.

Fixes 8700e3e7c485 ("Soft RoCE driver")
Assisted-by AuditCode-AI:2026.07
---
 drivers/infiniband/sw/rxe/rxe_qp.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
drivers/infiniband/sw/rxe/rxe_qp.c +8 -1
index f3dff1aea96a..646957707765 100644
@@ -172,6 +172,7 @@ static void free_rd_atomic_resources(struct rxe_qp *qp)
 		}
 		kfree(qp->resp.resources);
 		qp->resp.resources = NULL;
+		qp->resp.res = NULL;
 	}
 }
 
@@ -709,9 +710,15 @@ int rxe_qp_from_attr(struct rxe_qp *qp, struct ib_qp_attr *attr, int mask,
 
 		qp->attr.max_dest_rd_atomic = max_dest_rd_atomic;
 
+		/*
+		 * Not gated by IB_QP_STATE above: quiesce the responder task
+		 * the same way rxe_qp_reset() does before touching this
+		 * array, so rxe_receiver() can't race the free/realloc.
+		 */
+		rxe_disable_task(&qp->recv_task);
 		free_rd_atomic_resources(qp);
-
 		err = alloc_rd_atomic_resources(qp, max_dest_rd_atomic);
+		rxe_enable_task(&qp->recv_task);
 		if (err)
 			return err;
 	}
Signature · 1 line
2.50.1 (Apple Git-155)
#1 lore raw
Quoted 29 lines · Ibrahim Hashimov

rxe_qp_from_attr()'s IB_QP_MAX_DEST_RD_ATOMIC branch frees and reallocates qp->resp.resources[] (the rd_atomic resource array used by the responder to track in-flight RDMA READ/ATOMIC/FLUSH requests) completely outside of the IB_QP_STATE handling above it. Unlike every other place that tears this array down -- rxe_qp_reset(), reached only under IB_QP_STATE, always calls rxe_disable_task(&qp->recv_task) / rxe_disable_task(&qp->send_task) to drain the responder and requester tasks before touching per-QP state, then re-enables them -- this branch runs with the responder task (rxe_receiver(), scheduled as recv_task on the rxe_wq workqueue) fully live and unlocked. A userspace modify_qp() that sets only IB_QP_MAX_DEST_RD_ATOMIC (no state change, so __qp_chk_state()/ib_modify_qp_is_ok() never runs and qp->state_lock is never taken here) can therefore race the responder in two ways:

1. free_rd_atomic_resources() calls kfree(qp->resp.resources) and alloc_rd_atomic_resources() kzalloc_objs()'s a new array while rxe_prepare_res()/find_resource() in rxe_resp.c are concurrently walking &qp->resp.resources[i] with no lock held -- a straight free-vs-read race on the array itself.

2. free_rd_atomic_resources() only NULLs qp->resp.resources; it never clears qp->resp.res, the raw pointer *into* that array that rxe_resp.c caches across a multi-packet RDMA READ/ATOMIC/FLUSH reply (set at rxe_resp.c read/atomic/flush-reply sites, cleared only on the normal completion paths). If a modify_qp() races a resource still referenced by qp->resp.res, the array is freed out from under the cached pointer and the next reply packet dereferences it -- independent of the kfree/kzalloc_objs() window in (1).

Quoted 7 lines

Reproduced with KASAN: a single process driving one RC QP pair in rxe loopback, one thread pumping large multi-packet IBV_WR_RDMA_READs against qpB while a second thread hammers ibv_modify_qp(qpB, IB_QP_MAX_DEST_RD_ATOMIC), reliably (~11s) produces

   BUG: KASAN: slab-use-after-free in rxe_receiver+0x4f78/0x89e0 [rdma_rxe]
   Workqueue: rxe_wq do_work [rdma_rxe]

Can you share all the stack trace with us? Thanks a lot.

Quoted 55 lines

with the freed kmalloc-1k object being the rd_atomic resource array freed by the modify_qp() thread while the recv_task kworker reads it. An identical run modifying only IB_QP_MIN_RNR_TIMER (no resource free) is clean.

Fix both races the same way rxe_qp_reset() already handles tearing down this exact array: quiesce the responder task around the free/realloc by calling rxe_disable_task(&qp->recv_task) before free_rd_atomic_resources() and rxe_enable_task(&qp->recv_task) after alloc_rd_atomic_resources(), so rxe_receiver() cannot observe the array mid-free/mid-realloc. And close the still-open window for (2) at the source: have free_rd_atomic_resources() clear qp->resp.res along with qp->resp.resources, exactly like the existing completion paths in rxe_resp.c (check_rkey()/duplicate_request()/RESPST_CLEANUP) already do when a resource's lifetime ends, so a drained-and-resumed responder restarts at RESPST_CHK_PSN against the fresh array instead of replaying a stale reference into the old one.

Only qp->recv_task is drained: qp->resp.resources / qp->resp.res are touched exclusively by the responder (rxe_resp.c); the requester (send_task / rxe_sender()) never reads them, so there is no need to widen this beyond what rxe_qp_reset() would drain for the equivalent state.

Verified on the same v6.19 KASAN stand: with this fix applied, the identical differential reproducer drives sustained MAX_DEST_RD_ATOMIC storms against qpB well past the ~11s pre-fix time-to-first-splat with zero KASAN reports, versus reliably tripping the slab-use-after-free in rxe_receiver() described above before the fix.

Fixes 8700e3e7c485 ("Soft RoCE driver")
Assisted-by AuditCode-AI:2026.07
---
  drivers/infiniband/sw/rxe/rxe_qp.c | 9 ++++++++-
  1 file changed, 8 insertions(+), 1 deletion(-)
drivers/infiniband/sw/rxe/rxe_qp.c +1 -0
index f3dff1aea96a..646957707765 100644
@@ -172,6 +172,7 @@ static void free_rd_atomic_resources(struct rxe_qp *qp)
  		}
  		kfree(qp->resp.resources);
  		qp->resp.resources = NULL;
+		qp->resp.res = NULL;
  	}
  }
 
@@ -709,9 +710,15 @@ int rxe_qp_from_attr(struct rxe_qp *qp, struct ib_qp_attr *attr, int mask,
 
  		qp->attr.max_dest_rd_atomic = max_dest_rd_atomic;
 

drivers/infiniband/sw/rxe/rxe_qp.c int rxe_qp_from_attr(struct rxe_qp *qp, struct ib_qp_attr *attr, int mask, · line 721

Quoted 10 lines
drivers/infiniband/sw/rxe/rxe_qp.c +7 -1
+		/*
+		 * Not gated by IB_QP_STATE above: quiesce the responder task
+		 * the same way rxe_qp_reset() does before touching this
+		 * array, so rxe_receiver() can't race the free/realloc.
+		 */
+		rxe_disable_task(&qp->recv_task);
  		free_rd_atomic_resources(qp);
-
  		err = alloc_rd_atomic_resources(qp, max_dest_rd_atomic);
+		rxe_enable_task(&qp->recv_task);

If alloc_rd_atomic_resources fails, that is, qp->resp.resources is NULL.
After rxe_enable_task(&qp->recv_task); qp->resp.resources(NULL) will be used in resp. This will cause problems.

drivers/infiniband/sw/rxe/rxe_resp.c:656: res = &qp->resp.resources[qp->resp.res_head];
drivers/infiniband/sw/rxe/rxe_resp.c:1325: struct resp_res *res = &qp->resp.resources[i];

Zhu Yanjun

Quoted 3 lines
drivers/infiniband/sw/rxe/rxe_qp.c +0 -0
  		if (err)
  			return err;
  	}
#2 lore raw

Hi Zhu Yanjun, thanks for the review.

Quoted 3 lines
Quoted 2 lines
BUG: KASAN: slab-use-after-free in rxe_receiver+0x4f78/0x89e0 [rdma_rxe]
Workqueue: rxe_wq do_work [rdma_rxe]

Can you share all the stack trace with us? Thanks a lot.

Sure. The read side -- the recv_task kworker running rxe_receiver() on the freed rd_atomic resource array -- from the pre-fix v6.19 kernel (CONFIG_KASAN generic, rxe loopback):

  BUG: KASAN: slab-use-after-free in rxe_receiver+0x4f78/0x89e0 [rdma_rxe]
  Read of size 4 at addr ffff88800b79f84c by task kworker/u8:2/51
  CPU: 0 UID: 0 PID: 51 Comm: kworker/u8:2 Not tainted 6.19.0 #1
  Workqueue: rxe_wq do_work [rdma_rxe]
  Call Trace:
   <TASK>
   dump_stack_lvl+0x4d/0x70
   print_report+0x170/0x4f3
   kasan_report+0xda/0x110
   rxe_receiver+0x4f78/0x89e0 [rdma_rxe]
   do_work+0x144/0x470 [rdma_rxe]
   process_one_work+0x611/0xe80
   worker_thread+0x52e/0xdc0
   kthread+0x30c/0x630
   ret_from_fork+0x2fd/0x3e0
   ret_from_fork_asm+0x1a/0x30
   </TASK>

  The buggy address belongs to the object at ffff88800b79f800
   which belongs to the cache kmalloc-1k of size 1024
  The buggy address is located 76 bytes inside of
   freed 1024-byte region [ffff88800b79f800, ffff88800b79fc00)

The freed kmalloc-1k object is qp->resp.resources[] (the rd_atomic array); it is freed by the concurrent ib_modify_qp(IB_QP_MAX_DEST_RD_ATOMIC) thread in free_rd_atomic_resources() <- rxe_qp_from_attr(), while this recv_task kworker reads it. This was a kasan_multi_shot run and the per-object Allocated-by/Freed-by backtraces were lost in the splat storm -- I can send a single-shot capture that includes those two stacks if it would help.

Quoted 5 lines

If alloc_rd_atomic_resources fails, that is, qp->resp.resources is NULL. After rxe_enable_task(&qp->recv_task); qp->resp.resources(NULL) will be used in resp. This will cause problems.
drivers/infiniband/sw/rxe/rxe_resp.c:656: res = &qp->resp.resources[qp->resp.res_head];
drivers/infiniband/sw/rxe/rxe_resp.c:1325: struct resp_res *res = &qp->resp.resources[i];

Good catch -- you're right. Re-enabling recv_task before the error check resumes the responder against a NULL qp->resp.resources on the ENOMEM path. v2 moves rxe_enable_task() below the error check, so the responder is re-enabled only once a fresh array has been installed and stays quiesced on failure:

	rxe_disable_task(&qp->recv_task);
	free_rd_atomic_resources(qp);
	err = alloc_rd_atomic_resources(qp, max_dest_rd_atomic);
	if (err)
		return err;
	rxe_enable_task(&qp->recv_task);

rxe_disable_task()/rxe_enable_task() are state-based (TASK_STATE_DRAINED /IDLE), not refcounted, so leaving recv_task drained on the failed modify is safe and recoverable: a subsequent successful modify_qp or a QP destroy both handle the DRAINED state.

I'll send this as [PATCH v2].

Thanks,
Ibrahim

#3 lore raw

PATCH v2 1/1 RDMA/rxe: fix responder UAF on IB_QP_MAX_DEST_RD_ATOMIC modify_qp

In reply to Ibrahim Hashimov

rxe_qp_from_attr()'s IB_QP_MAX_DEST_RD_ATOMIC branch frees and reallocates qp->resp.resources[] (the rd_atomic resource array used by the responder to track in-flight RDMA READ/ATOMIC/FLUSH requests) completely outside of the IB_QP_STATE handling above it. Unlike every other place that tears this array down -- rxe_qp_reset(), reached only under IB_QP_STATE, always calls rxe_disable_task(&qp->recv_task) / rxe_disable_task(&qp->send_task) to drain the responder and requester tasks before touching per-QP state, then re-enables them -- this branch runs with the responder task (rxe_receiver(), scheduled as recv_task on the rxe_wq workqueue) fully live and unlocked. A userspace modify_qp() that sets only IB_QP_MAX_DEST_RD_ATOMIC (no state change, so __qp_chk_state()/ib_modify_qp_is_ok() never runs and qp->state_lock is never taken here) can therefore race the responder in two ways:

1. free_rd_atomic_resources() calls kfree(qp->resp.resources) and alloc_rd_atomic_resources() kzalloc_objs()'s a new array while rxe_prepare_res()/find_resource() in rxe_resp.c are concurrently walking &qp->resp.resources[i] with no lock held -- a straight free-vs-read race on the array itself.

2. free_rd_atomic_resources() only NULLs qp->resp.resources; it never clears qp->resp.res, the raw pointer *into* that array that rxe_resp.c caches across a multi-packet RDMA READ/ATOMIC/FLUSH reply (set at rxe_resp.c read/atomic/flush-reply sites, cleared only on the normal completion paths). If a modify_qp() races a resource still referenced by qp->resp.res, the array is freed out from under the cached pointer and the next reply packet dereferences it -- independent of the kfree/kzalloc_objs() window in (1).

Reproduced with KASAN: a single process driving one RC QP pair in rxe loopback, one thread pumping large multi-packet IBV_WR_RDMA_READs against qpB while a second thread hammers ibv_modify_qp(qpB, IB_QP_MAX_DEST_RD_ATOMIC), reliably (~11s) produces

  BUG: KASAN: slab-use-after-free in rxe_receiver+0x4f78/0x89e0 [rdma_rxe]
  Workqueue: rxe_wq do_work [rdma_rxe]

with the freed kmalloc-1k object being the rd_atomic resource array freed by the modify_qp() thread while the recv_task kworker reads it. An identical run modifying only IB_QP_MIN_RNR_TIMER (no resource free) is clean.

Fix both races the same way rxe_qp_reset() already handles tearing down this exact array: quiesce the responder task around the free/realloc by calling rxe_disable_task(&qp->recv_task) before free_rd_atomic_resources() and rxe_enable_task(&qp->recv_task) only after alloc_rd_atomic_resources() has succeeded, so rxe_receiver() cannot observe the array mid-free/ mid-realloc. On the alloc-failure path the responder is deliberately left quiesced: qp->resp.resources is NULL at that point and rxe_prepare_res()/find_resource() would dereference it, so recv_task must not be re-enabled until a fresh array has been installed. And close the still-open window for (2) at the source: have free_rd_atomic_resources() clear qp->resp.res along with qp->resp.resources, exactly like the existing completion paths in rxe_resp.c (check_rkey()/duplicate_request()/RESPST_CLEANUP) already do when a resource's lifetime ends, so a drained-and-resumed responder restarts at RESPST_CHK_PSN against the fresh array instead of replaying a stale reference into the old one.

Only qp->recv_task is drained: qp->resp.resources / qp->resp.res are touched exclusively by the responder (rxe_resp.c); the requester (send_task / rxe_sender()) never reads them, so there is no need to widen this beyond what rxe_qp_reset() would drain for the equivalent state.

Verified on the same v6.19 KASAN stand: with this fix applied, the identical differential reproducer drives sustained MAX_DEST_RD_ATOMIC storms against qpB well past the ~11s pre-fix time-to-first-splat with zero KASAN reports, versus reliably tripping the slab-use-after-free in rxe_receiver() described above before the fix.

Fixes 8700e3e7c485 ("Soft RoCE driver")
Assisted-by AuditCode-AI:2026.07

---
v2: address Zhu Yanjun's review of v1
(https://lore.kernel.org/linux-rdma/[email protected]/):
only re-enable recv_task after alloc_rd_atomic_resources() succeeds, so the responder is not resumed against a NULL qp->resp.resources on the ENOMEM path (rxe_prepare_res()/find_resource() would dereference it). No change to the successful path; fix description updated accordingly.

 drivers/infiniband/sw/rxe/rxe_qp.c | 16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)
drivers/infiniband/sw/rxe/rxe_qp.c +15 -1
index f3dff1aea96a..e39fb144cbbb 100644
@@ -172,6 +172,7 @@ static void free_rd_atomic_resources(struct rxe_qp *qp)
 		}
 		kfree(qp->resp.resources);
 		qp->resp.resources = NULL;
+		qp->resp.res = NULL;
 	}
 }
 
@@ -709,11 +710,24 @@ int rxe_qp_from_attr(struct rxe_qp *qp, struct ib_qp_attr *attr, int mask,
 
 		qp->attr.max_dest_rd_atomic = max_dest_rd_atomic;
 
+		/*
+		 * This branch is not gated by IB_QP_STATE, so the responder
+		 * task is live here. Quiesce it the way rxe_qp_reset() does
+		 * before swapping the rd_atomic resource array, so
+		 * rxe_receiver() cannot race the free/realloc.
+		 */
+		rxe_disable_task(&qp->recv_task);
 		free_rd_atomic_resources(qp);
-
 		err = alloc_rd_atomic_resources(qp, max_dest_rd_atomic);
+		/*
+		 * On failure the responder stays quiesced: qp->resp.resources
+		 * is NULL now, and rxe_prepare_res()/find_resource() would
+		 * dereference it, so do not re-enable recv_task until a fresh
+		 * array has been installed.
+		 */
 		if (err)
 			return err;
+		rxe_enable_task(&qp->recv_task);
 	}
 
 	if (mask & IB_QP_EN_SQD_ASYNC_NOTIFY)
Signature · 1 line
2.50.1 (Apple Git-155)
Reviewed-by on v2 1/1
#4 lore raw
Quoted 111 lines · Ibrahim Hashimov

rxe_qp_from_attr()'s IB_QP_MAX_DEST_RD_ATOMIC branch frees and reallocates qp->resp.resources[] (the rd_atomic resource array used by the responder to track in-flight RDMA READ/ATOMIC/FLUSH requests) completely outside of the IB_QP_STATE handling above it. Unlike every other place that tears this array down -- rxe_qp_reset(), reached only under IB_QP_STATE, always calls rxe_disable_task(&qp->recv_task) / rxe_disable_task(&qp->send_task) to drain the responder and requester tasks before touching per-QP state, then re-enables them -- this branch runs with the responder task (rxe_receiver(), scheduled as recv_task on the rxe_wq workqueue) fully live and unlocked. A userspace modify_qp() that sets only IB_QP_MAX_DEST_RD_ATOMIC (no state change, so __qp_chk_state()/ib_modify_qp_is_ok() never runs and qp->state_lock is never taken here) can therefore race the responder in two ways:

1. free_rd_atomic_resources() calls kfree(qp->resp.resources) and alloc_rd_atomic_resources() kzalloc_objs()'s a new array while rxe_prepare_res()/find_resource() in rxe_resp.c are concurrently walking &qp->resp.resources[i] with no lock held -- a straight free-vs-read race on the array itself.

2. free_rd_atomic_resources() only NULLs qp->resp.resources; it never clears qp->resp.res, the raw pointer *into* that array that rxe_resp.c caches across a multi-packet RDMA READ/ATOMIC/FLUSH reply (set at rxe_resp.c read/atomic/flush-reply sites, cleared only on the normal completion paths). If a modify_qp() races a resource still referenced by qp->resp.res, the array is freed out from under the cached pointer and the next reply packet dereferences it -- independent of the kfree/kzalloc_objs() window in (1).

Reproduced with KASAN: a single process driving one RC QP pair in rxe loopback, one thread pumping large multi-packet IBV_WR_RDMA_READs against qpB while a second thread hammers ibv_modify_qp(qpB, IB_QP_MAX_DEST_RD_ATOMIC), reliably (~11s) produces

   BUG: KASAN: slab-use-after-free in rxe_receiver+0x4f78/0x89e0 [rdma_rxe]
   Workqueue: rxe_wq do_work [rdma_rxe]

with the freed kmalloc-1k object being the rd_atomic resource array freed by the modify_qp() thread while the recv_task kworker reads it. An identical run modifying only IB_QP_MIN_RNR_TIMER (no resource free) is clean.

Fix both races the same way rxe_qp_reset() already handles tearing down this exact array: quiesce the responder task around the free/realloc by calling rxe_disable_task(&qp->recv_task) before free_rd_atomic_resources() and rxe_enable_task(&qp->recv_task) only after alloc_rd_atomic_resources() has succeeded, so rxe_receiver() cannot observe the array mid-free/ mid-realloc. On the alloc-failure path the responder is deliberately left quiesced: qp->resp.resources is NULL at that point and rxe_prepare_res()/find_resource() would dereference it, so recv_task must not be re-enabled until a fresh array has been installed. And close the still-open window for (2) at the source: have free_rd_atomic_resources() clear qp->resp.res along with qp->resp.resources, exactly like the existing completion paths in rxe_resp.c (check_rkey()/duplicate_request()/RESPST_CLEANUP) already do when a resource's lifetime ends, so a drained-and-resumed responder restarts at RESPST_CHK_PSN against the fresh array instead of replaying a stale reference into the old one.

Only qp->recv_task is drained: qp->resp.resources / qp->resp.res are touched exclusively by the responder (rxe_resp.c); the requester (send_task / rxe_sender()) never reads them, so there is no need to widen this beyond what rxe_qp_reset() would drain for the equivalent state.

Verified on the same v6.19 KASAN stand: with this fix applied, the identical differential reproducer drives sustained MAX_DEST_RD_ATOMIC storms against qpB well past the ~11s pre-fix time-to-first-splat with zero KASAN reports, versus reliably tripping the slab-use-after-free in rxe_receiver() described above before the fix.

Fixes 8700e3e7c485 ("Soft RoCE driver")
Assisted-by AuditCode-AI:2026.07

---
v2: address Zhu Yanjun's review of v1
(https://lore.kernel.org/linux-rdma/[email protected]/):
only re-enable recv_task after alloc_rd_atomic_resources() succeeds, so the responder is not resumed against a NULL qp->resp.resources on the ENOMEM path (rxe_prepare_res()/find_resource() would dereference it). No change to the successful path; fix description updated accordingly.

  drivers/infiniband/sw/rxe/rxe_qp.c | 16 +++++++++++++++-
  1 file changed, 15 insertions(+), 1 deletion(-)
drivers/infiniband/sw/rxe/rxe_qp.c +8 -1
index f3dff1aea96a..e39fb144cbbb 100644
@@ -172,6 +172,7 @@ static void free_rd_atomic_resources(struct rxe_qp *qp)
  		}
  		kfree(qp->resp.resources);
  		qp->resp.resources = NULL;
+		qp->resp.res = NULL;
  	}
  }
 
@@ -709,11 +710,24 @@ int rxe_qp_from_attr(struct rxe_qp *qp, struct ib_qp_attr *attr, int mask,
 
  		qp->attr.max_dest_rd_atomic = max_dest_rd_atomic;
 
+		/*
+		 * This branch is not gated by IB_QP_STATE, so the responder
+		 * task is live here. Quiesce it the way rxe_qp_reset() does
+		 * before swapping the rd_atomic resource array, so
+		 * rxe_receiver() cannot race the free/realloc.
+		 */
+		rxe_disable_task(&qp->recv_task);
  		free_rd_atomic_resources(qp);
-

drivers/infiniband/sw/rxe/rxe_qp.c int rxe_qp_from_attr(struct rxe_qp *qp, struct ib_qp_attr *attr, int mask, · line 730

Quoted 10 lines
drivers/infiniband/sw/rxe/rxe_qp.c +7 -0
  		err = alloc_rd_atomic_resources(qp, max_dest_rd_atomic);
+		/*
+		 * On failure the responder stays quiesced: qp->resp.resources
+		 * is NULL now, and rxe_prepare_res()/find_resource() would
+		 * dereference it, so do not re-enable recv_task until a fresh
+		 * array has been installed.
+		 */
  		if (err)
  			return err;
+		rxe_enable_task(&qp->recv_task);

Thanks a lot. I am fine with this commit. Please Leon and Jason comment on this commit.

Zhu Yanjun

Quoted 3 lines
drivers/infiniband/sw/rxe/rxe_qp.c +0 -0
  	}
 
  	if (mask & IB_QP_EN_SQD_ASYNC_NOTIFY)
#5 lore raw
Quoted 114 lines · yanjun.zhu
Quoted 113 lines · Ibrahim Hashimov

rxe_qp_from_attr()'s IB_QP_MAX_DEST_RD_ATOMIC branch frees and reallocates qp->resp.resources[] (the rd_atomic resource array used by the responder to track in-flight RDMA READ/ATOMIC/FLUSH requests) completely outside of the IB_QP_STATE handling above it. Unlike every other place that tears this array down -- rxe_qp_reset(), reached only under IB_QP_STATE, always calls rxe_disable_task(&qp->recv_task) / rxe_disable_task(&qp->send_task) to drain the responder and requester tasks before touching per-QP state, then re-enables them -- this branch runs with the responder task (rxe_receiver(), scheduled as recv_task on the rxe_wq workqueue) fully live and unlocked. A userspace modify_qp() that sets only IB_QP_MAX_DEST_RD_ATOMIC (no state change, so __qp_chk_state()/ib_modify_qp_is_ok() never runs and qp->state_lock is never taken here) can therefore race the responder in two ways:

1. free_rd_atomic_resources() calls kfree(qp->resp.resources) and alloc_rd_atomic_resources() kzalloc_objs()'s a new array while rxe_prepare_res()/find_resource() in rxe_resp.c are concurrently walking &qp->resp.resources[i] with no lock held -- a straight free-vs-read race on the array itself.

2. free_rd_atomic_resources() only NULLs qp->resp.resources; it never clears qp->resp.res, the raw pointer *into* that array that rxe_resp.c caches across a multi-packet RDMA READ/ATOMIC/FLUSH reply (set at rxe_resp.c read/atomic/flush-reply sites, cleared only on the normal completion paths). If a modify_qp() races a resource still referenced by qp->resp.res, the array is freed out from under the cached pointer and the next reply packet dereferences it -- independent of the kfree/kzalloc_objs() window in (1).

Reproduced with KASAN: a single process driving one RC QP pair in rxe loopback, one thread pumping large multi-packet IBV_WR_RDMA_READs against qpB while a second thread hammers ibv_modify_qp(qpB, IB_QP_MAX_DEST_RD_ATOMIC), reliably (~11s) produces

   BUG: KASAN: slab-use-after-free in rxe_receiver+0x4f78/0x89e0 [rdma_rxe]
   Workqueue: rxe_wq do_work [rdma_rxe]

with the freed kmalloc-1k object being the rd_atomic resource array freed by the modify_qp() thread while the recv_task kworker reads it. An identical run modifying only IB_QP_MIN_RNR_TIMER (no resource free) is clean.

Fix both races the same way rxe_qp_reset() already handles tearing down this exact array: quiesce the responder task around the free/realloc by calling rxe_disable_task(&qp->recv_task) before free_rd_atomic_resources() and rxe_enable_task(&qp->recv_task) only after alloc_rd_atomic_resources() has succeeded, so rxe_receiver() cannot observe the array mid-free/ mid-realloc. On the alloc-failure path the responder is deliberately left quiesced: qp->resp.resources is NULL at that point and rxe_prepare_res()/find_resource() would dereference it, so recv_task must not be re-enabled until a fresh array has been installed. And close the still-open window for (2) at the source: have free_rd_atomic_resources() clear qp->resp.res along with qp->resp.resources, exactly like the existing completion paths in rxe_resp.c (check_rkey()/duplicate_request()/RESPST_CLEANUP) already do when a resource's lifetime ends, so a drained-and-resumed responder restarts at RESPST_CHK_PSN against the fresh array instead of replaying a stale reference into the old one.

Only qp->recv_task is drained: qp->resp.resources / qp->resp.res are touched exclusively by the responder (rxe_resp.c); the requester (send_task / rxe_sender()) never reads them, so there is no need to widen this beyond what rxe_qp_reset() would drain for the equivalent state.

Verified on the same v6.19 KASAN stand: with this fix applied, the identical differential reproducer drives sustained MAX_DEST_RD_ATOMIC storms against qpB well past the ~11s pre-fix time-to-first-splat with zero KASAN reports, versus reliably tripping the slab-use-after-free in rxe_receiver() described above before the fix.

Fixes 8700e3e7c485 ("Soft RoCE driver")
Assisted-by AuditCode-AI:2026.07

---
v2: address Zhu Yanjun's review of v1
(https://lore.kernel.org/linux-rdma/[email protected]/):
only re-enable recv_task after alloc_rd_atomic_resources() succeeds, so the responder is not resumed against a NULL qp->resp.resources on the ENOMEM path (rxe_prepare_res()/find_resource() would dereference it). No change to the successful path; fix description updated accordingly.

  drivers/infiniband/sw/rxe/rxe_qp.c | 16 +++++++++++++++-
  1 file changed, 15 insertions(+), 1 deletion(-)
drivers/infiniband/sw/rxe/rxe_qp.c +12 -1
index f3dff1aea96a..e39fb144cbbb 100644
@@ -172,6 +172,7 @@ static void free_rd_atomic_resources(struct rxe_qp *qp)
  		}
  		kfree(qp->resp.resources);
  		qp->resp.resources = NULL;
+		qp->resp.res = NULL;
  	}
  }
@@ -709,11 +710,24 @@ int rxe_qp_from_attr(struct rxe_qp *qp, struct ib_qp_attr *attr, int mask,
  		qp->attr.max_dest_rd_atomic = max_dest_rd_atomic;
+		/*
+		 * This branch is not gated by IB_QP_STATE, so the responder
+		 * task is live here. Quiesce it the way rxe_qp_reset() does
+		 * before swapping the rd_atomic resource array, so
+		 * rxe_receiver() cannot race the free/realloc.
+		 */
+		rxe_disable_task(&qp->recv_task);
  		free_rd_atomic_resources(qp);
-
  		err = alloc_rd_atomic_resources(qp, max_dest_rd_atomic);
+		/*
+		 * On failure the responder stays quiesced: qp->resp.resources
+		 * is NULL now, and rxe_prepare_res()/find_resource() would
+		 * dereference it, so do not re-enable recv_task until a fresh
Quoted 8 lines

drivers/infiniband/sw/rxe/rxe_qp.c int rxe_qp_from_attr(struct rxe_qp *qp, struct ib_qp_attr *attr, int mask, · line 730

Quoted 5 lines
drivers/infiniband/sw/rxe/rxe_qp.c +3 -0
+		 * array has been installed.
+		 */
  		if (err)
  			return err;
+		rxe_enable_task(&qp->recv_task);

Thanks a lot. I am fine with this commit. Please Leon and Jason comment on this commit.

1. Please send patches as a standalone series, not as a reply to the previous one.

2. Write commit messages in a clear and sensible way so they are easy for humans to understand. Keep them short and direct, and describe the function call flow instead of producing AI prose.

Thanks

Quoted 8 lines

Zhu Yanjun

Quoted 2 lines
drivers/infiniband/sw/rxe/rxe_qp.c +0 -0
  	}
  	if (mask & IB_QP_EN_SQD_ASYNC_NOTIFY)