Skip to contentSkip to conversation

RDMA/erdma: Use IRQ-safe XArray helpers for QP and CQ tables

PATCH Applied Cheng Xu 3 participants 4 messages Nov 17 – Nov 18

Subsystems (2)
lore
Share & tools
lore

Cheng Xu

author
#0 lore raw

PATCH 1/1 RDMA/erdma: Use IRQ-safe XArray helpers for QP and CQ tables

[ Upstream commit 00baeade709fb66da647e8327e8398bb532e30f7 ]

Locked QP and CQ lookups from EQ interrupts can deadlock with create-path XArray updates. If an interrupt arrives while the create path holds the plain xa_lock, the lookup spins forever trying to acquire the same lock.

Use IRQ-safe XArray helpers for all QP and CQ create-path updates, including the GSI QP store and error paths. Initialize both arrays with XA_FLAGS_LOCK_IRQ so sleeping allocations preserve interrupt state.

[ The GSI QP create path is not present in 6.6.y, so omit that hunk. ]

Fixes 98df2aee1459 ("RDMA/erdma: Hold CQ references when processing EQ events")
Fixes a52eeff32024 ("RDMA/erdma: Hold QP references for AE and CM processing")
---
 drivers/infiniband/hw/erdma/erdma_main.c  |  4 ++--
 drivers/infiniband/hw/erdma/erdma_verbs.c | 16 ++++++++--------
 2 files changed, 10 insertions(+), 10 deletions(-)
drivers/infiniband/hw/erdma/erdma_main.c +2 -2
index 0880c79a978c..6dc6c7f4fb2e 100644
@@ -519,8 +519,8 @@ static int erdma_ib_device_add(struct pci_dev *pdev)
 	INIT_LIST_HEAD(&dev->cep_list);
 
 	spin_lock_init(&dev->lock);
-	xa_init_flags(&dev->qp_xa, XA_FLAGS_ALLOC1);
-	xa_init_flags(&dev->cq_xa, XA_FLAGS_ALLOC1);
+	xa_init_flags(&dev->qp_xa, XA_FLAGS_ALLOC1 | XA_FLAGS_LOCK_IRQ);
+	xa_init_flags(&dev->cq_xa, XA_FLAGS_ALLOC1 | XA_FLAGS_LOCK_IRQ);
 	dev->next_alloc_cqn = 1;
 	dev->next_alloc_qpn = 1;
 
drivers/infiniband/hw/erdma/erdma_verbs.c +8 -8
index a9b2d0890570..94b3d4aea123 100644
@@ -945,9 +945,9 @@ int erdma_create_qp(struct ib_qp *ibqp, struct ib_qp_init_attr *attrs,
 	kref_init(&qp->ref);
 	init_completion(&qp->safe_free);
 
-	ret = xa_alloc_cyclic(&dev->qp_xa, &qp->ibqp.qp_num, qp,
-			      XA_LIMIT(1, dev->attrs.max_qp - 1),
-			      &dev->next_alloc_qpn, GFP_KERNEL);
+	ret = xa_alloc_cyclic_irq(&dev->qp_xa, &qp->ibqp.qp_num, qp,
+				  XA_LIMIT(1, dev->attrs.max_qp - 1),
+				  &dev->next_alloc_qpn, GFP_KERNEL);
 	if (ret < 0) {
 		ret = -ENOMEM;
 		goto err_out;
@@ -1003,7 +1003,7 @@ int erdma_create_qp(struct ib_qp *ibqp, struct ib_qp_init_attr *attrs,
 	else
 		free_kernel_qp(qp);
 err_out_xa:
-	xa_erase(&dev->qp_xa, QP_ID(qp));
+	xa_erase_irq(&dev->qp_xa, QP_ID(qp));
 err_out:
 	return ret;
 }
@@ -1682,9 +1682,9 @@ int erdma_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
 	refcount_set(&cq->refcount, 1);
 	init_completion(&cq->free);
 
-	ret = xa_alloc_cyclic(&dev->cq_xa, &cq->cqn, cq,
-			      XA_LIMIT(1, dev->attrs.max_cq - 1),
-			      &dev->next_alloc_cqn, GFP_KERNEL);
+	ret = xa_alloc_cyclic_irq(&dev->cq_xa, &cq->cqn, cq,
+				  XA_LIMIT(1, dev->attrs.max_cq - 1),
+				  &dev->next_alloc_cqn, GFP_KERNEL);
 	if (ret < 0)
 		return ret;
 
@@ -1731,7 +1731,7 @@ int erdma_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
 	}
 
 err_out_xa:
-	xa_erase(&dev->cq_xa, cq->cqn);
+	xa_erase_irq(&dev->cq_xa, cq->cqn);
 
 	return ret;
 }
Signature · 1 line
2.31.1
#1 lore raw
Sashiko has reviewed this patch and found no issues.
Quoted 11 lines

[ Upstream commit 00baeade709fb66da647e8327e8398bb532e30f7 ]

Locked QP and CQ lookups from EQ interrupts can deadlock with create-path XArray updates. If an interrupt arrives while the create path holds the plain xa_lock, the lookup spins forever trying to acquire the same lock.

Use IRQ-safe XArray helpers for all QP and CQ create-path updates, including the GSI QP store and error paths. Initialize both arrays with XA_FLAGS_LOCK_IRQ so sleeping allocations preserve interrupt state.

Quoted 7 lines

[ The GSI QP create path is not present in 6.6.y, so omit that hunk. ]

Fixes 98df2aee1459 ("RDMA/erdma: Hold CQ references when processing EQ events")
Fixes a52eeff32024 ("RDMA/erdma: Hold QP references for AE and CM processing")

Sashiko has reviewed this patch and found no issues.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
maintainer Applied
#2 lore raw
In reply to Cheng Xu
Quoted 2 lines

Use IRQ-safe XArray helpers for all QP and CQ create-path updates, including the GSI QP store and error paths.

Queued for 6.12 and 6.6, thanks.

6.1 has the same bug and is not covered. If you want it there, please send a 6.1.y backport.

Signature · 2 lines
Thanks,
Sasha
author
#3 lore raw
Quoted 8 lines · Sasha Levin
Quoted 2 lines

Use IRQ-safe XArray helpers for all QP and CQ create-path updates, including the GSI QP store and error paths.

Queued for 6.12 and 6.6, thanks.

6.1 has the same bug and is not covered. If you want it there, please send a 6.1.y backport.

Thanks. I have sent the 6.1.y backport.

Cheng Xu