Skip to contentSkip to conversation

RDMA/mana_ib: Add kernel RC and fast registration support

PATCH Superseded Konstantin Taranov 2 participants 21 messages Oct 1 – Oct 2

Subsystems (5)

v1 of 11 Superseded Latest is v11 →

lore
Share & tools
lore
#0 lore raw

cover RDMA/mana_ib: Add kernel RC and fast registration support

From: Konstantin Taranov <[email protected]>

Add kernel RC QPs and fast-registration MRs to the MANA RDMA driver for kernel consumers such as NVMe/RDMA and NFS/RDMA.

Implement QP creation and state management, work request posting, completion processing, error handling, and queue draining. Kernel RC QPs require extended-WQE and power-of-two-SQ hardware support.

Tested with krping, NVMe/RDMA, and NFS/RDMA, including RC send and receive error recovery with krping and NVMe/RDMA.

Konstantin Taranov (10):
RDMA/mana_ib: Allocate and map fast-registration MRs RDMA/mana: Create and destroy kernel RC QPs net/mana: Extend GDMA encoding for new RDMA WQEs RDMA/mana_ib: Maintain kernel RC QP state RDMA/mana_ib: Post receive WRs on kernel RC QPs RDMA/mana_ib: Post send and memory-management WRs on RC QPs RDMA/mana_ib: Poll RC completions using PSN and FSN progress RDMA/mana_ib: Flush and notify CQs when kernel QPs enter ERR RDMA/mana_ib: Handle error CQEs for RC QPs RDMA/mana_ib: Drain kernel receive and send queues

 drivers/infiniband/hw/mana/cq.c               | 240 ++++++++++-
 drivers/infiniband/hw/mana/device.c           |   4 +
 drivers/infiniband/hw/mana/main.c             |   8 +-
 drivers/infiniband/hw/mana/mana_ib.h          | 135 +++++-
 drivers/infiniband/hw/mana/mr.c               |  84 +++-
 drivers/infiniband/hw/mana/qp.c               | 323 ++++++++++++++-
 drivers/infiniband/hw/mana/shadow_queue.h     |  22 +
 drivers/infiniband/hw/mana/wr.c               | 385 ++++++++++++++++++
 .../net/ethernet/microsoft/mana/gdma_main.c   |  49 ++-
 include/net/mana/gdma.h                       |  38 +-
 10 files changed, 1253 insertions(+), 35 deletions(-)
Signature · 1 line
2.43.0
#1 lore raw

PATCH 1/10 RDMA/mana_ib: Allocate and map fast-registration MRs

From: Konstantin Taranov <[email protected]>

Implement alloc_mr and map_mr_sg for memory-registration MRs.

---
 drivers/infiniband/hw/mana/device.c  |  2 +
 drivers/infiniband/hw/mana/main.c    |  3 +-
 drivers/infiniband/hw/mana/mana_ib.h | 46 +++++++++++++++
 drivers/infiniband/hw/mana/mr.c      | 84 +++++++++++++++++++++++++++-
 include/net/mana/gdma.h              | 15 +++++
 5 files changed, 148 insertions(+), 2 deletions(-)
drivers/infiniband/hw/mana/device.c +2 -0
index 62c7803ca6c6..cdaada7f6e8a 100644
@@ -19,6 +19,7 @@ static const struct ib_device_ops mana_ib_dev_ops = {
 
 	.add_gid = mana_ib_gd_add_gid,
 	.alloc_mw = mana_ib_alloc_mw,
+	.alloc_mr = mana_ib_alloc_mr,
 	.alloc_pd = mana_ib_alloc_pd,
 	.alloc_ucontext = mana_ib_alloc_ucontext,
 	.create_ah = mana_ib_create_ah,
@@ -40,6 +41,7 @@ static const struct ib_device_ops mana_ib_dev_ops = {
 	.get_dma_mr = mana_ib_get_dma_mr,
 	.get_link_layer = mana_ib_get_link_layer,
 	.get_port_immutable = mana_ib_get_port_immutable,
+	.map_mr_sg = mana_ib_map_mr_sg,
 	.mmap = mana_ib_mmap,
 	.modify_qp = mana_ib_modify_qp,
 	.modify_wq = mana_ib_modify_wq,
drivers/infiniband/hw/mana/main.c +2 -1
index 672bd3f650dd..ceb4ec58023a 100644
@@ -133,7 +133,7 @@ int mana_ib_alloc_pd(struct ib_pd *ibpd, struct ib_udata *udata)
 			     sizeof(resp));
 
 	if (!udata)
-		flags |= GDMA_PD_FLAG_ALLOW_GPA_MR;
+		flags |= GDMA_PD_FLAG_ALLOW_GPA_MR | GDMA_PD_FLAG_ALLOW_FMR_MR;
 
 	req.flags = flags;
 	err = mana_gd_send_request(gc, sizeof(req), &req, sizeof(resp), &resp);
@@ -647,6 +647,7 @@ int mana_ib_query_device(struct ib_device *ibdev, struct ib_device_attr *props,
 	props->max_ah = INT_MAX;
 	props->max_pkeys = 1;
 	props->local_ca_ack_delay = MANA_CA_ACK_DELAY;
+	props->max_fast_reg_page_list_len = MANA_MAX_FMR_PAGES;
 	if (!mana_ib_is_rnic(dev))
 		props->raw_packet_caps = IB_RAW_PACKET_CAP_IP_CSUM;
 
drivers/infiniband/hw/mana/mana_ib.h +46 -0

76 lines not on this page · raw message

drivers/infiniband/hw/mana/mr.c +83 -1

119 lines not on this page · raw message

include/net/mana/gdma.h +15 -0
index 2a28fcaa637d..332f394b82e1 100644
@@ -865,6 +865,8 @@ enum gdma_mr_access_flags {
 	GDMA_ACCESS_FLAG_REMOTE_WRITE = BIT_ULL(3),
 	GDMA_ACCESS_FLAG_REMOTE_ATOMIC = BIT_ULL(4),
 	GDMA_ACCESS_FLAG_BIND_MW = BIT_ULL(5),
+	GDMA_ACCESS_FLAG_INVALIDATE = BIT_ULL(6),
+	GDMA_ACCESS_FLAG_FMR = BIT_ULL(7),
 };
 
 /* GDMA_CREATE_DMA_REGION */
@@ -917,6 +919,7 @@ struct gdma_destroy_dma_region_req {
 
 enum gdma_pd_flags {
 	GDMA_PD_FLAG_ALLOW_GPA_MR = BIT(0),
+	GDMA_PD_FLAG_ALLOW_FMR_MR = BIT(1),
 	GDMA_PD_FLAG_SHORT_PDN = BIT(2),
 };
 
@@ -953,6 +956,11 @@ enum gdma_mr_type {
 	 * address that is set up in the MST
 	 */
 	GDMA_MR_TYPE_GVA = 2,
+	/* Fast Memory - MRs are initially put in the free state,
+	 * and the specified number of PTEs are reserved for future fast
+	 * memory registrations
+	 */
+	GDMA_MR_TYPE_FMR = 3,
 	/* Guest zero-based address MRs */
 	GDMA_MR_TYPE_ZBVA = 4,
 	/* Device address MRs */
@@ -982,6 +990,9 @@ struct gdma_create_mr_params {
 			u64 length;
 			enum gdma_mr_access_flags access_flags;
 		} da;
+		struct {
+			u32 rsvd_pte_count;
+		} fmr;
 	};
 };
 
@@ -1006,6 +1017,10 @@ struct gdma_create_mr_request {
 			u64 offset;
 			enum gdma_mr_access_flags access_flags;
 		} __packed da;
+		struct {
+			u32 page_size;
+			u32 rsvd_pte_count;
+		} __packed fmr;
 	} __packed;
 	u32 reserved_2;
 	union {
Signature · 1 line
2.43.0
#2 lore raw

Re: [PATCH rdma-next 01/10] RDMA/mana_ib: Allocate and map fast-registration MRs

Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 5 lines

From: Konstantin Taranov <[email protected]>

Implement alloc_mr and map_mr_sg for memory-registration MRs.

Sashiko has reviewed this patch and found no issues. It looks great!

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
#3 lore raw

PATCH 2/10 RDMA/mana: Create and destroy kernel RC QPs

In reply to Konstantin Taranov

From: Konstantin Taranov <[email protected]>

Implement kernel RC QP creation and destruction. This feature requires extended-WQE and power-of-two-SQ support from the HW.

---
 drivers/infiniband/hw/mana/main.c    |   3 +-
 drivers/infiniband/hw/mana/mana_ib.h |   6 ++
 drivers/infiniband/hw/mana/qp.c      | 145 ++++++++++++++++++++++++++-
 3 files changed, 152 insertions(+), 2 deletions(-)
drivers/infiniband/hw/mana/main.c +2 -1
index ceb4ec58023a..fed20197addf 100644
@@ -632,7 +632,8 @@ int mana_ib_query_device(struct ib_device *ibdev, struct ib_device_attr *props,
 	props->max_qp = dev->adapter_caps.max_qp_count;
 	props->max_qp_wr = dev->adapter_caps.max_qp_wr;
 	props->device_cap_flags = IB_DEVICE_RC_RNR_NAK_GEN;
-	props->max_send_sge = dev->adapter_caps.max_send_sge_count;
+	/* Subtract 1 from max_send_sge to account for the reserved SGE */
+	props->max_send_sge = dev->adapter_caps.max_send_sge_count - 1;
 	props->max_recv_sge = dev->adapter_caps.max_recv_sge_count;
 	props->max_sge_rd = dev->adapter_caps.max_recv_sge_count;
 	props->max_cq = dev->adapter_caps.max_cq_count;
drivers/infiniband/hw/mana/mana_ib.h +6 -0
index ceb7830b8da1..61af35bf0e40 100644
@@ -265,6 +265,10 @@ struct mana_ib_qp {
 	u32 sq_psn;
 	bool sq_sig_all;
 
+	/* Serializes receive WR posting. */
+	spinlock_t rq_lock;
+	/* Serializes send and memory-management WR posting. */
+	spinlock_t sq_lock;
 	/* Serializes QP modification and error-list transitions. */
 	struct mutex modify_lock;
 
@@ -272,6 +276,7 @@ struct mana_ib_qp {
 	struct list_head recv_err_node;
 	struct shadow_queue shadow_rq;
 	struct shadow_queue shadow_sq;
+	struct shadow_queue shadow_mmq;
 
 	refcount_t		refcount;
 	struct completion	free;
@@ -313,6 +318,7 @@ enum mana_ib_adapter_features {
 	MANA_IB_FEATURE_DEV_COUNTERS_SUPPORT = BIT(5),
 	MANA_IB_FEATURE_MULTI_PORTS_SUPPORT = BIT(6),
 	MANA_IB_FEATURE_MSN_IN_WQE_SUPPORT = BIT(7),
+	MANA_IB_FEATURE_EXTENDED_WQE_FORMAT = BIT(9),
 	MANA_IB_FEATURE_RC_QP_SQ_POW2_SUPPORT = BIT(14),
 	MANA_IB_FEATURE_MULTI_PORT_GSI_SUPPORT = BIT(15),
 };
drivers/infiniband/hw/mana/qp.c +144 -1

182 lines not on this page · raw message

Signature · 1 line
2.43.0
#4 lore raw

Re: [PATCH rdma-next 02/10] RDMA/mana: Create and destroy kernel RC QPs

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] mana: integer underflow when querying device max_send_sge bypasses IB core limits --

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] mana: integer underflow when querying device max_send_sge bypasses IB core limits --

commit 38c0e6ff844f0113447371575b0813a46aa399ab
Author: Konstantin Taranov <[email protected]>

RDMA/mana: Create and destroy kernel RC QPs

This commit implements kernel RC QP creation and destruction in the mana driver. The feature adds support for extended-WQE and power-of-two-SQ queues requiring support from the underlying hardware.

drivers/infiniband/hw/mana/main.c int mana_ib_query_device(struct ib_device *ibdev, struct ib_device_attr *props, · line 641

Quoted 10 lines
drivers/infiniband/hw/mana/main.c +2 -1
@@ -637,7 +637,8 @@ int mana_ib_query_device(struct ib_device *ibdev, struct ib_device_attr *props,
 	props->max_qp = dev->adapter_caps.max_qp_count;
 	props->max_qp_wr = dev->adapter_caps.max_qp_wr;
 	props->device_cap_flags = IB_DEVICE_RC_RNR_NAK_GEN;
-	props->max_send_sge = dev->adapter_caps.max_send_sge_count;
+	/* Subtract 1 from max_send_sge to account for the reserved SGE */
+	props->max_send_sge = dev->adapter_caps.max_send_sge_count - 1;

[Severity: Medium]
Is it possible for dev->adapter_caps.max_send_sge_count to be 0 when queried from the device firmware?

If the device reports 0 for this capability, this unchecked subtraction could wrap around and expose UINT_MAX as the maximum send SGE limit to the IB core and userspace.

Could this lead to integer overflows in mana_ib_wqe_size() and under-allocated DMA queues if a user requests an overly large SGE count based on this inflated limit, ultimately causing out-of-bounds hardware memory accesses?

Quoted 3 lines
drivers/infiniband/hw/mana/main.c +0 -0
 	props->max_recv_sge = dev->adapter_caps.max_recv_sge_count;
 	props->max_sge_rd = dev->adapter_caps.max_recv_sge_count;
 	props->max_cq = dev->adapter_caps.max_cq_count;
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
#5 lore raw

PATCH 3/10 net/mana: Extend GDMA encoding for new RDMA WQEs

In reply to Konstantin Taranov

From: Konstantin Taranov <[email protected]>

Extend mana_gd_post_work_request() to support extra-large OOB data and fixed-size WQEs. These formats are supported only for power-of-two WQE sizes.

For fixed-size WQEs, use num_pad_sge to specify the padding size. Encode the WQE header fields required by RC QPs based on the WR flags.

---
 .../net/ethernet/microsoft/mana/gdma_main.c   | 49 ++++++++++++++++---
 include/net/mana/gdma.h                       | 21 ++++++--
 2 files changed, 57 insertions(+), 13 deletions(-)
drivers/net/ethernet/microsoft/mana/gdma_main.c +41 -8
index 05ba7055405d..444101dcbb64 100644
@@ -1699,8 +1699,27 @@ static u32 mana_gd_write_client_oob(const struct gdma_wqe_request *wqe_req,
 	header->num_sge = wqe_req->num_sge;
 	header->inline_oob_size_div4 = client_oob_size / sizeof(u32);
 
+	if (client_oob_size == INLINE_OOB_EXTRA_LARGE_SIZE) {
+		/* Extra large oob is encoded as 7 */
+		header->inline_oob_size_div4 = 7;
+		/* Extra large oob has 8 byte padding  */
+		client_oob_size += 8;
+	}
+
+	if (wqe_req->wqe_size_in_bu) {
+		header->num_sge = (wqe_req->wqe_size_in_bu * GDMA_WQE_BU_SIZE
+			- sizeof(struct gdma_wqe) - client_oob_size) / sizeof(struct gdma_sge);
+		header->num_pad_sge = header->num_sge - wqe_req->num_sge;
+	}
+
+	header->check_rsn = !!(wqe_req->flags & GDMA_WR_CHECK_RSN);
+	header->check_sn = !!(wqe_req->flags & GDMA_WR_CHECK_SN);
+	header->fence = !!(wqe_req->flags & GDMA_WR_FENCE);
+	header->fence_extended = !!(wqe_req->flags & GDMA_WR_FENCE_EXTENDED);
+	header->generation1 = !!(wqe_req->flags & GDMA_WR_GENERATION_1);
+
 	if (oob_in_sgl) {
-		WARN_ON_ONCE(wqe_req->num_sge < 2);
+		WARN_ON_ONCE(wqe_req->num_sge < 1);
 
 		header->client_oob_in_sgl = 1;
 
@@ -1714,8 +1733,10 @@ static u32 mana_gd_write_client_oob(const struct gdma_wqe_request *wqe_req,
 	/* The size of gdma_wqe + client_oob_size must be less than or equal
 	 * to one Basic Unit (i.e. 32 bytes), so the pointer can't go beyond
 	 * the queue memory buffer boundary.
+	 * The exception is when the WQE size is fixed and a power of 2, as
+	 * such a wqe cannot go beyond the queue memory buffer boundary.
 	 */
-	ptr = wqe_ptr + sizeof(header);
+	ptr = wqe_ptr + sizeof(struct gdma_wqe);
 
 	if (wqe_req->inline_oob_data && wqe_req->inline_oob_size > 0) {
 		memcpy(ptr, wqe_req->inline_oob_data, wqe_req->inline_oob_size);
@@ -1724,8 +1745,8 @@ static u32 mana_gd_write_client_oob(const struct gdma_wqe_request *wqe_req,
 			memset(ptr + wqe_req->inline_oob_size, 0,
 			       client_oob_size - wqe_req->inline_oob_size);
 	}
-
-	return sizeof(header) + client_oob_size;
+	/* The hw requires that (gdma_wqe + client_oob_size) is sge size aligned */
+	return ALIGN(sizeof(struct gdma_wqe) + client_oob_size, sizeof(struct gdma_sge));
 }
 
 static void mana_gd_write_sgl(struct gdma_queue *wq, u32 sgl_offset,
@@ -1784,7 +1805,8 @@ int mana_gd_post_work_request(struct gdma_queue *wq,
 		return -EINVAL;
 
 	if (wq->type == GDMA_RQ) {
-		if (client_oob_size != 0)
+		if (client_oob_size != 0 &&
+		    client_oob_size != INLINE_OOB_SMALL_SIZE)
 			return -EINVAL;
 
 		client_oob_size = INLINE_OOB_SMALL_SIZE;
@@ -1792,15 +1814,26 @@ int mana_gd_post_work_request(struct gdma_queue *wq,
 		max_wqe_size = GDMA_MAX_RQE_SIZE;
 	} else {
 		if (client_oob_size != INLINE_OOB_SMALL_SIZE &&
-		    client_oob_size != INLINE_OOB_LARGE_SIZE)
+		    client_oob_size != INLINE_OOB_LARGE_SIZE &&
+		    client_oob_size != INLINE_OOB_EXTRA_LARGE_SIZE)
 			return -EINVAL;
 
 		max_wqe_size = GDMA_MAX_SQE_SIZE;
 	}
 
 	sgl_data_size = sizeof(struct gdma_sge) * wqe_req->num_sge;
-	wqe_size = ALIGN(sizeof(struct gdma_wqe) + client_oob_size +
-			 sgl_data_size, GDMA_WQE_BU_SIZE);
+	if (wqe_req->wqe_size_in_bu)
+		wqe_size = wqe_req->wqe_size_in_bu * GDMA_WQE_BU_SIZE;
+	else
+		wqe_size = ALIGN(sizeof(struct gdma_wqe) + client_oob_size +
+				 sgl_data_size, GDMA_WQE_BU_SIZE);
+	/* Ensure the WQE size is a power of 2 when a fixed wqe size or extra large OOB are used.
+	 * Otherwise, the WQE may cross the queue memory buffer boundary, which is not allowed.
+	 */
+	if (wqe_req->wqe_size_in_bu || client_oob_size == INLINE_OOB_EXTRA_LARGE_SIZE) {
+		if (!is_power_of_2(wqe_size))
+			return -EINVAL;
+	}
 	if (wqe_size > max_wqe_size)
 		return -EINVAL;
 
include/net/mana/gdma.h +16 -5
index 332f394b82e1..508da28591ae 100644
@@ -56,6 +56,11 @@ enum gdma_work_request_flags {
 	GDMA_WR_OOB_IN_SGL		= BIT(0),
 	GDMA_WR_PAD_BY_SGE0		= BIT(1),
 	GDMA_WR_IB_SGL			= BIT(2),
+	GDMA_WR_CHECK_SN                = BIT(3),
+	GDMA_WR_CHECK_RSN               = BIT(4),
+	GDMA_WR_FENCE                   = BIT(5),
+	GDMA_WR_FENCE_EXTENDED          = BIT(6),
+	GDMA_WR_GENERATION_1            = BIT(7),
 };
 
 enum gdma_eqe_type {
@@ -219,8 +224,8 @@ struct gdma_wqe_request {
 		struct ib_sge *ib_sgl;
 	};
 	u32 num_sge;
-
-	u32 inline_oob_size;
+	u16 wqe_size_in_bu;
+	u16 inline_oob_size;
 	const void *inline_oob_data;
 
 	u32 flags;
@@ -538,7 +543,9 @@ void mana_gd_ring_dim(struct gdma_queue *cq, u32 mod_usec, bool mod_usec_vld,
 		      u32 mod_comps, bool mod_comps_vld);
 
 struct gdma_wqe {
-	u32 reserved	:24;
+	u32 num_pad_sge	:5;
+	u32 generation1	:1;
+	u32 reserved	:18;
 	u32 last_vbytes	:8;
 
 	union {
@@ -548,9 +555,13 @@ struct gdma_wqe {
 			u32 num_sge		:8;
 			u32 inline_oob_size_div4:3;
 			u32 client_oob_in_sgl	:1;
-			u32 reserved1		:4;
+			u32 check_rsn		:1;
+			u32 fence		:1;
+			u32 fence_extended	:1;
+			u32 reserved1		:1;
 			u32 client_data_unit	:14;
-			u32 reserved2		:2;
+			u32 check_sn		:1;
+			u32 reserved2		:1;
 		};
 	};
 }; /* HW DATA */
Signature · 1 line
2.43.0
#6 lore raw

Re: [PATCH rdma-next 03/10] net/mana: Extend GDMA encoding for new RDMA WQEs

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] rdma/mana: Missing fixed WQE size validation causes ring buffer overflow --

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] rdma/mana: Missing fixed WQE size validation causes ring buffer overflow --

commit 8f045a852a650adae362c74de90d65ccaf6541d9
Author: Konstantin Taranov <[email protected]>

net/mana: Extend GDMA encoding for new RDMA WQEs

This commit extends mana_gd_post_work_request() to support extra-large OOB data and fixed-size WQEs. It encodes WQE header fields based on WR flags and uses num_pad_sge to specify padding size for fixed-size WQEs.

Quoted 9 lines
drivers/net/ethernet/microsoft/mana/gdma_main.c +1 -0
index 05ba7055405da..444101dcbb647 100644
@@ -1699,8 +1699,27 @@ static u32 mana_gd_write_client_oob(const struct gdma_wqe_request *wqe_req,
 	header->num_sge = wqe_req->num_sge;
 	header->inline_oob_size_div4 = client_oob_size / sizeof(u32);
 
+	if (client_oob_size == INLINE_OOB_EXTRA_LARGE_SIZE) {

drivers/net/ethernet/microsoft/mana/gdma_main.c static u32 mana_gd_write_client_oob(const struct gdma_wqe_request *wqe_req, · line 1712

Quoted 10 lines
drivers/net/ethernet/microsoft/mana/gdma_main.c +10 -0
+		/* Extra large oob is encoded as 7 */
+		header->inline_oob_size_div4 = 7;
+		/* Extra large oob has 8 byte padding  */
+		client_oob_size += 8;
+	}
+
+	if (wqe_req->wqe_size_in_bu) {
+		header->num_sge = (wqe_req->wqe_size_in_bu * GDMA_WQE_BU_SIZE
+			- sizeof(struct gdma_wqe) - client_oob_size) / sizeof(struct gdma_sge);
+		header->num_pad_sge = header->num_sge - wqe_req->num_sge;

[Severity: High]
Does this calculation risk silently underflowing if the fixed WQE size is too small?

If a caller provides an undersized fixed WQE size, this subtraction could underflow, which might corrupt the hardware descriptor bitfields and lead to an out-of-bounds write of SGEs.

Quoted 3 lines
drivers/net/ethernet/microsoft/mana/gdma_main.c +3 -0
+	}
+
+	header->check_rsn = !!(wqe_req->flags & GDMA_WR_CHECK_RSN);

[ ... ]

Quoted 7 lines
drivers/net/ethernet/microsoft/mana/gdma_main.c +2 -1
@@ -1792,15 +1814,26 @@ int mana_gd_post_work_request(struct gdma_queue *wq,
 		max_wqe_size = GDMA_MAX_RQE_SIZE;
 	} else {
 		if (client_oob_size != INLINE_OOB_SMALL_SIZE &&
-		    client_oob_size != INLINE_OOB_LARGE_SIZE)
+		    client_oob_size != INLINE_OOB_LARGE_SIZE &&
+		    client_oob_size != INLINE_OOB_EXTRA_LARGE_SIZE)

drivers/net/ethernet/microsoft/mana/gdma_main.c int mana_gd_post_work_request(struct gdma_queue *wq, · line 1826

Quoted 10 lines
drivers/net/ethernet/microsoft/mana/gdma_main.c +2 -2
 			return -EINVAL;
 
 		max_wqe_size = GDMA_MAX_SQE_SIZE;
 	}
 
 	sgl_data_size = sizeof(struct gdma_sge) * wqe_req->num_sge;
-	wqe_size = ALIGN(sizeof(struct gdma_wqe) + client_oob_size +
-			 sgl_data_size, GDMA_WQE_BU_SIZE);
+	if (wqe_req->wqe_size_in_bu)
+		wqe_size = wqe_req->wqe_size_in_bu * GDMA_WQE_BU_SIZE;

[Severity: High]
Is it possible for the user-provided fixed WQE size to be accepted here without verifying that it is actually large enough to contain the requested SGEs and OOB payload?

If a caller creates a queue pair with a small max_send_sge (fixing the WQE size) but posts a send work request with num_sge larger than max_send_sge, could this bypass validation?

If this validation is missing, it appears it would cause the integer underflow mentioned above in mana_gd_write_client_oob() and lead to an out-of-bounds memory write during mana_gd_write_sgl().

Quoted 3 lines
drivers/net/ethernet/microsoft/mana/gdma_main.c +3 -0
+	else
+		wqe_size = ALIGN(sizeof(struct gdma_wqe) + client_oob_size +
+				 sgl_data_size, GDMA_WQE_BU_SIZE);
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
#7 lore raw

PATCH 4/10 RDMA/mana_ib: Maintain kernel RC QP state

In reply to Konstantin Taranov

From: Konstantin Taranov <[email protected]>

Update mana_ib_modify_qp_state() to support RC QPs. Initialize the packet, message, and frame counters for use by work requests.

Arm the requester receive queue at the initial PSN so that a CQE is generated when the specified PSN is acknowledged, indicating send completions.

---
 drivers/infiniband/hw/mana/mana_ib.h |  5 +++++
 drivers/infiniband/hw/mana/qp.c      | 23 ++++++++++++++++++++---
 include/net/mana/gdma.h              |  2 ++
 3 files changed, 27 insertions(+), 3 deletions(-)
drivers/infiniband/hw/mana/mana_ib.h +5 -0
index 61af35bf0e40..f8d9feee0abe 100644
@@ -263,7 +263,12 @@ struct mana_ib_qp {
 	/* The port on the IB device, starting with 1 */
 	u32 port;
 	u32 sq_psn;
+	u32 sq_ssn;
+	u32 sq_fsn;
+	u16 sq_rsn;
+	bool pending_mmq_fence;
 	bool sq_sig_all;
+	enum ib_mtu mtu;
 
 	/* Serializes receive WR posting. */
 	spinlock_t rq_lock;
drivers/infiniband/hw/mana/qp.c +20 -3
index 1dc5b078b4ff..f5611fd30efe 100644
@@ -744,8 +744,10 @@ static void mana_remove_qp_from_cqs(struct mana_ib_qp *qp, bool reset)
 	spin_lock_irqsave(&send_cq->cq_lock, flags);
 	list_del_init(&qp->send_err_node);
 	/* Keep shadow reset serialized with hardware polling and SW flushing. */
-	if (reset)
+	if (reset) {
 		reset_shadow_queue(&qp->shadow_sq);
+		reset_shadow_queue(&qp->shadow_mmq);
+	}
 	spin_unlock_irqrestore(&send_cq->cq_lock, flags);
 
 	spin_lock_irqsave(&recv_cq->cq_lock, flags);
@@ -1028,11 +1030,16 @@ static int mana_ib_gd_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr,
 static void mana_ib_modify_qp_state(struct ib_qp *ibqp, struct ib_qp_attr *attr,
 				    int attr_mask, struct ib_udata *udata)
 {
+	struct mana_ib_dev *mdev = container_of(ibqp->device, struct mana_ib_dev, ib_dev);
 	struct mana_ib_qp *qp = container_of(ibqp, struct mana_ib_qp, ibqp);
+	struct gdma_queue *rq;
 
 	if (udata)
 		return;
 
+	if (attr_mask & IB_QP_PATH_MTU)
+		qp->mtu = attr->path_mtu;
+
 	if (attr_mask & IB_QP_STATE) {
 		switch (attr->qp_state) {
 		case IB_QPS_RESET:
@@ -1043,8 +1050,19 @@ static void mana_ib_modify_qp_state(struct ib_qp *ibqp, struct ib_qp_attr *attr,
 		}
 	}
 
-	if (attr_mask & IB_QP_SQ_PSN)
+	if (attr_mask & IB_QP_SQ_PSN) {
+		qp->sq_ssn = 1;
+		qp->sq_rsn = 1;
+		qp->sq_fsn = 1;
 		qp->sq_psn = attr->sq_psn;
+		qp->pending_mmq_fence = false;
+
+		if (qp->ibqp.qp_type == IB_QPT_RC) {
+			rq = qp->rc_qp.queues[MANA_RC_RECV_QUEUE_REQUESTER].kmem;
+			mana_gd_wq_ring_doorbell_ext(mdev_to_gc(mdev), rq, attr->sq_psn,
+						     SET_ARM_BIT, MANA_PSN_CLIENT_OFFSET);
+		}
+	}
 }
 
 int mana_ib_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr,
@@ -1057,7 +1075,6 @@ int mana_ib_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr,
 
 	switch (ibqp->qp_type) {
 	case IB_QPT_RC:
-	case IB_QPT_UC:
 	case IB_QPT_UD:
 	case IB_QPT_GSI:
 		ret = mana_ib_gd_modify_qp(ibqp, attr, attr_mask, udata);
include/net/mana/gdma.h +2 -0
index 508da28591ae..fd0ec8fc44d5 100644
@@ -605,6 +605,8 @@ struct gdma_cqe {
 
 #define GDMA_EQE_OWNER_BITS 3
 
+#define MANA_PSN_CLIENT_OFFSET 0x8
+
 union gdma_eqe_info {
 	u32 as_uint32;
 
Signature · 1 line
2.43.0
#8 lore raw

Re: [PATCH rdma-next 04/10] RDMA/mana_ib: Maintain kernel RC QP state

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] RDMA/mana_ib: Broken UC QP modification due to removed case --

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] RDMA/mana_ib: Broken UC QP modification due to removed case
--

commit 5fb460641c0259509af070b4f53dae8c74239937
Author: Konstantin Taranov <[email protected]>

RDMA/mana_ib: Maintain kernel RC QP state

Update mana_ib_modify_qp_state() to support RC QPs and initialize the packet, message, and frame counters for use by work requests. It also arms the requester receive queue at the initial PSN.

Quoted 4 lines
drivers/infiniband/hw/mana/qp.c +0 -0
index edaf03372cd81..8c2ec4d091f0d 100644

[ ... ]

drivers/infiniband/hw/mana/qp.c int mana_ib_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, · line 1081

Quoted 8 lines
drivers/infiniband/hw/mana/qp.c +0 -1
@@ -1058,7 +1076,6 @@ int mana_ib_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr,
 
 	switch (ibqp->qp_type) {
 	case IB_QPT_RC:
-	case IB_QPT_UC:
 	case IB_QPT_UD:
 	case IB_QPT_GSI:
 		ret = mana_ib_gd_modify_qp(ibqp, attr, attr_mask, udata);

[Severity: High]
Does removing IB_QPT_UC break Unreliable Connection (UC) QP functionality?

By removing this case, any userspace or kernel client attempting to modify a UC QP state will hit the default switch case in mana_ib_modify_qp(), which returns -EOPNOTSUPP. This prevents UC QPs from being transitioned to active states (INIT, RTR, RTS).

Was this case removed unintentionally, or is UC QP modification handled elsewhere now?

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
#9 lore raw

PATCH 5/10 RDMA/mana_ib: Post receive WRs on kernel RC QPs

In reply to Konstantin Taranov

From: Konstantin Taranov <[email protected]>

Implement receive request posting for RC QPs. When posting a batch of WRs, ring the doorbell periodically to prevent the u8 counter from overflowing.

---
 drivers/infiniband/hw/mana/wr.c | 37 +++++++++++++++++++++++++++++++++
 1 file changed, 37 insertions(+)
drivers/infiniband/hw/mana/wr.c +37 -0
index 250d673f58f2..5d42af91fce4 100644
@@ -70,6 +70,41 @@ static int mana_ib_post_recv_ud(struct mana_ib_qp *qp, const struct ib_recv_wr *
 	return err;
 }
 
+static int mana_ib_post_recv_rc(struct mana_ib_qp *qp, const struct ib_recv_wr *wr,
+				const struct ib_recv_wr **bad_wr)
+{
+	struct mana_ib_dev *mdev = container_of(qp->ibqp.device, struct mana_ib_dev, ib_dev);
+	struct mana_ib_queue *ib_rq = mana_qp_get_rq(qp);
+	unsigned long flags;
+	u8 wqe_cnt = 0;
+	int err = 0;
+
+	spin_lock_irqsave(&qp->rq_lock, flags);
+
+	for (; wr; wr = wr->next) {
+		err = mana_ib_post_rq(qp, wr);
+		if (unlikely(err)) {
+			*bad_wr = wr;
+			break;
+		}
+		wqe_cnt++;
+		if (wqe_cnt == 0xFF) {
+			mana_gd_wq_ring_doorbell_ext(mdev_to_gc(mdev), ib_rq->kmem,
+						     ib_rq->kmem->head * GDMA_WQE_BU_SIZE,
+						     wqe_cnt, 0);
+			wqe_cnt = 0;
+		}
+	}
+
+	if (wqe_cnt)
+		mana_gd_wq_ring_doorbell_ext(mdev_to_gc(mdev), ib_rq->kmem,
+					     ib_rq->kmem->head * GDMA_WQE_BU_SIZE,
+					     wqe_cnt, 0);
+
+	spin_unlock_irqrestore(&qp->rq_lock, flags);
+	return err;
+}
+
 int mana_ib_post_recv(struct ib_qp *ibqp, const struct ib_recv_wr *wr,
 		      const struct ib_recv_wr **bad_wr)
 {
@@ -79,6 +114,8 @@ int mana_ib_post_recv(struct ib_qp *ibqp, const struct ib_recv_wr *wr,
 	case IB_QPT_UD:
 	case IB_QPT_GSI:
 		return mana_ib_post_recv_ud(qp, wr, bad_wr);
+	case IB_QPT_RC:
+		return mana_ib_post_recv_rc(qp, wr, bad_wr);
 	default:
 		/* Unsupported QP type */
 		*bad_wr = wr;
Signature · 1 line
2.43.0
#10 lore raw

Re: [PATCH rdma-next 05/10] RDMA/mana_ib: Post receive WRs on kernel RC QPs

Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 7 lines

From: Konstantin Taranov <[email protected]>

Implement receive request posting for RC QPs. When posting a batch of WRs, ring the doorbell periodically to prevent the u8 counter from overflowing.

Sashiko has reviewed this patch and found no issues. It looks great!

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5
#11 lore raw

PATCH 6/10 RDMA/mana_ib: Post send and memory-management WRs on RC QPs

In reply to Konstantin Taranov

From: Konstantin Taranov <[email protected]>

Implement send work request posting for RC QPs. Post communication WRs to the send queue and memory WRs to the memory management queue.

Advertise the memory management extensions on adapters that support extended WQEs.

---
 drivers/infiniband/hw/mana/main.c    |   2 +
 drivers/infiniband/hw/mana/mana_ib.h |   1 +
 drivers/infiniband/hw/mana/wr.c      | 348 +++++++++++++++++++++++++++
 3 files changed, 351 insertions(+)
drivers/infiniband/hw/mana/main.c +2 -0
index fed20197addf..9a8ae38b865a 100644
@@ -632,6 +632,8 @@ int mana_ib_query_device(struct ib_device *ibdev, struct ib_device_attr *props,
 	props->max_qp = dev->adapter_caps.max_qp_count;
 	props->max_qp_wr = dev->adapter_caps.max_qp_wr;
 	props->device_cap_flags = IB_DEVICE_RC_RNR_NAK_GEN;
+	if (dev->adapter_caps.feature_flags & MANA_IB_FEATURE_EXTENDED_WQE_FORMAT)
+		props->device_cap_flags |= IB_DEVICE_MEM_MGT_EXTENSIONS;
 	/* Subtract 1 from max_send_sge to account for the reserved SGE */
 	props->max_send_sge = dev->adapter_caps.max_send_sge_count - 1;
 	props->max_recv_sge = dev->adapter_caps.max_recv_sge_count;
drivers/infiniband/hw/mana/mana_ib.h +1 -0
index f8d9feee0abe..c7968d3c7ecd 100644
@@ -323,6 +323,7 @@ enum mana_ib_adapter_features {
 	MANA_IB_FEATURE_DEV_COUNTERS_SUPPORT = BIT(5),
 	MANA_IB_FEATURE_MULTI_PORTS_SUPPORT = BIT(6),
 	MANA_IB_FEATURE_MSN_IN_WQE_SUPPORT = BIT(7),
+	MANA_IB_FEATURE_FMR_IN_OOB_SUPPORT = BIT(8),
 	MANA_IB_FEATURE_EXTENDED_WQE_FORMAT = BIT(9),
 	MANA_IB_FEATURE_RC_QP_SQ_POW2_SUPPORT = BIT(14),
 	MANA_IB_FEATURE_MULTI_PORT_GSI_SUPPORT = BIT(15),
drivers/infiniband/hw/mana/wr.c +348 -0

366 lines not on this page · raw message

Signature · 1 line
2.43.0
#12 lore raw

Re: [PATCH rdma-next 06/10] RDMA/mana_ib: Post send and memory-management WRs on RC QPs

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] rdma/mana_ib: WQE ring buffer overwrite due to missing SQ SGE bounds check --

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] rdma/mana_ib: WQE ring buffer overwrite due to missing SQ SGE bounds check --

commit 7f1f7264400f5f00c6c885de8b54ac6900f87a1f
Author: Konstantin Taranov <[email protected]>

RDMA/mana_ib: Post send and memory-management WRs on RC QPs

This commit implements send work request posting for RC QPs in the mana_ib driver. It posts communication work requests to the send queue and memory work requests to the memory management queue.

Quoted 4 lines
drivers/infiniband/hw/mana/wr.c +0 -0
index 5d42af91fce4a..6cc67537c6557 100644

[ ... ]

Quoted 7 lines
drivers/infiniband/hw/mana/wr.c +3 -0
@@ -224,6 +289,287 @@ static int mana_ib_post_send_ud(struct mana_ib_qp *qp, const struct ib_send_wr *
 	return err;
 }
 
+static int mana_ib_post_sq(struct mana_ib_qp *qp, const struct ib_send_wr *wr)
+{
+	struct gdma_queue *sq = qp->rc_qp.queues[MANA_RC_SEND_QUEUE_REQUESTER].kmem;

[ ... ]

drivers/infiniband/hw/mana/wr.c static int mana_ib_post_send_ud(struct mana_ib_qp *qp, const struct ib_send_wr * · line 308

Quoted 7 lines
drivers/infiniband/hw/mana/wr.c +7 -0
+	int err, i;
+
+	if (shadow_queue_full(&qp->shadow_sq))
+		return -ENOMEM;
+
+	if (wr->num_sge > MAX_TX_WQE_SGL_ENTRIES)
+		return -EINVAL;

[Severity: High]
Does this bounds check in mana_ib_post_sq() allow out-of-bounds memory writes if a kernel RDMA client provides a wr->num_sge that is larger than the QP's negotiated maximum capacity?

Other RDMA core providers typically validate num_sge against the QP's specific limits because clients can misuse the API.

[ ... ]

drivers/infiniband/hw/mana/wr.c static int mana_ib_post_send_ud(struct mana_ib_qp *qp, const struct ib_send_wr * · line 353

Quoted 5 lines
drivers/infiniband/hw/mana/wr.c +5 -0
+	wqe_req.inline_oob_size = INLINE_OOB_EXTRA_LARGE_SIZE;
+	wqe_req.inline_oob_data = &send_oob;
+	wqe_req.client_data_unit = ib_mtu_enum_to_int(qp->mtu);
+	wqe_req.flags |= GDMA_WR_GENERATION_1;
+	wqe_req.wqe_size_in_bu = qp->rc_qp.wqe_size_in_bu;

Because the WQE size is explicitly fixed to the QP's configured size here, if wr->num_sge is larger than the QP's negotiated capacity but still less than or equal to MAX_TX_WQE_SGL_ENTRIES, mana_gd_post_work_request() will use wqe_size_in_bu to calculate the ring buffer slot size.

Meanwhile, mana_gd_write_ib_sgl() will write the full wr->num_sge entries into the queue. This can overwrite the adjacent WQE in the ring buffer, leading to hardware memory corruption.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=6
#13 lore raw

PATCH 7/10 RDMA/mana_ib: Poll RC completions using PSN and FSN progress

In reply to Konstantin Taranov

From: Konstantin Taranov <[email protected]>

Decode RC receive completions, armed-PSN notifications, and MMQ completions. Retire requester and MMQ shadow entries using their sequence numbers, honor cross-queue fence dependencies, and suppress successful WCs for unsignaled work requests.

On an ARMED completion, rearm the next signal PSN to generate the next send CQE.

---
 drivers/infiniband/hw/mana/cq.c           | 149 ++++++++++++++++++++--
 drivers/infiniband/hw/mana/mana_ib.h      |  31 +++++
 drivers/infiniband/hw/mana/shadow_queue.h |  22 ++++
 3 files changed, 192 insertions(+), 10 deletions(-)
drivers/infiniband/hw/mana/cq.c +139 -10
index 3b8044b7dc8f..29eb12c1848e 100644
@@ -5,6 +5,12 @@
 
 #include "mana_ib.h"
 
+#define PSN_SIGN_BIT 0x800000
+#define PSN_GE(PSN1, PSN2) ((((PSN1) - (PSN2)) & PSN_SIGN_BIT) == 0)
+#define PSN_GT(PSN1, PSN2) PSN_GE(PSN1, (PSN2) + 1)
+#define PSN_LE(PSN1, PSN2) PSN_GE(PSN2, PSN1)
+#define PSN_LT(PSN1, PSN2) PSN_GT(PSN2, PSN1)
+
 static enum ib_wc_status vendor_error_to_wc_error(uint32_t vendor_error)
 {
 	switch (vendor_error) {
@@ -311,10 +317,10 @@ static struct ib_wc *mana_fill_wc(struct mana_ib_qp *qp,
 	return wc;
 }
 
-static void mana_complete_send(struct mana_ib_qp *qp,
+static void mana_complete_send(struct mana_ib_qp *qp, bool mmq,
 			       struct mana_cq_poll *poll, u32 vendor_error)
 {
-	struct shadow_queue *shadow = &qp->shadow_sq;
+	struct shadow_queue *shadow = mmq ? &qp->shadow_mmq : &qp->shadow_sq;
 	struct shadow_wqe_header *wqe = shadow_queue_get_next_to_consume(shadow);
 	struct gdma_queue *queue;
 
@@ -324,8 +330,19 @@ static void mana_complete_send(struct mana_ib_qp *qp,
 	if (vendor_error || !(wqe->flags & MANA_WQ_NO_SIGNAL_WC))
 		mana_fill_wc(qp, poll, wqe, wqe->send_opcode, vendor_error);
 
-	queue = mana_qp_get_sq(qp)->kmem;
-	queue->tail += wqe->wqe_size_in_bu;
+	if (mmq) {
+		queue = qp->rc_qp.queues[MANA_RC_SEND_QUEUE_MMQ].kmem;
+		queue->tail += wqe->wqe_size_in_bu;
+	} else {
+		queue = mana_qp_get_sq(qp)->kmem;
+		if (wqe->send_opcode == IB_WC_RDMA_READ) {
+			qp->rc_qp.queues[MANA_RC_RECV_QUEUE_REQUESTER].kmem->tail +=
+				wqe->wqe_size_in_bu;
+			queue->tail += qp->rc_qp.wqe_size_in_bu;
+		} else {
+			queue->tail += wqe->wqe_size_in_bu;
+		}
+	}
 	shadow_queue_advance_consumer(shadow);
 }
 
@@ -354,7 +371,7 @@ static void handle_ud_sq_cqe(struct mana_ib_qp *qp, struct mana_rdma_cqe *rdma_c
 		return;
 
 	for (; to_complete; to_complete--)
-		mana_complete_send(qp, poll, VENDOR_ERR_OK);
+		mana_complete_send(qp, false, poll, VENDOR_ERR_OK);
 }
 
 static void handle_rq_cqe(struct mana_ib_qp *qp, struct gdma_comp *cqe,
@@ -382,6 +399,21 @@ static void handle_rq_cqe(struct mana_ib_qp *qp, struct gdma_comp *cqe,
 		wc->src_qp = rdma_cqe->ud_recv.src_qpn;
 		wc->wc_flags |= IB_WC_GRH;
 		break;
+	case CQE_TYPE_RC_WRITE_IMM:
+		wc->opcode = IB_WC_RECV_RDMA_WITH_IMM;
+		fallthrough;
+	case CQE_TYPE_RC_SEND_IMM:
+		wc->wc_flags |= IB_WC_WITH_IMM;
+		wc->ex.imm_data = cpu_to_be32(rdma_cqe->rc_recv.imm_data);
+		fallthrough;
+	case CQE_TYPE_RC_SEND:
+		wc->byte_len = rdma_cqe->rc_recv.msg_len;
+		break;
+	case CQE_TYPE_RC_SEND_INV:
+		wc->wc_flags |= IB_WC_WITH_INVALIDATE;
+		wc->byte_len = rdma_cqe->rc_recv.msg_len;
+		wc->ex.invalidate_rkey = rdma_cqe->rc_recv.imm_data;
+		break;
 	default:
 		break;
 	}
@@ -390,17 +422,104 @@ static void handle_rq_cqe(struct mana_ib_qp *qp, struct gdma_comp *cqe,
 	shadow_queue_advance_consumer(&qp->shadow_rq);
 }
 
+/*
+ * A fenced WQE can acknowledge the other send queue. Retire those dependencies
+ * first, without losing the fence when the caller's WC array fills up.
+ */
+static bool mana_advance_send(struct mana_ib_qp *qp, bool mmq, u32 seq,
+			      struct mana_cq_poll *poll)
+{
+	struct shadow_queue *shadow = mmq ? &qp->shadow_mmq : &qp->shadow_sq;
+	struct shadow_queue *other = mmq ? &qp->shadow_sq : &qp->shadow_mmq;
+	struct shadow_wqe_header *shadow_wqe;
+
+	while ((shadow_wqe = shadow_queue_get_next_to_consume(shadow))) {
+		if (PSN_LT(seq, mmq ? shadow_wqe->fsn : shadow_wqe->wqe_offset_or_psn))
+			break;
+		if (poll->produced == poll->budget)
+			return false;
+		if (shadow_wqe->flags & MANA_WQ_FENCE_WC) {
+			u32 dependency = mmq ? shadow_wqe->wqe_offset_or_psn : shadow_wqe->fsn;
+			struct shadow_wqe_header *other_wqe;
+
+			while ((other_wqe = shadow_queue_get_next_to_consume(other))) {
+				if (PSN_LT(dependency, mmq ? other_wqe->wqe_offset_or_psn :
+							     other_wqe->fsn))
+					break;
+				mana_complete_send(qp, !mmq, poll, VENDOR_ERR_OK);
+				if (poll->produced == poll->budget)
+					return false;
+			}
+		}
+
+		mana_complete_send(qp, mmq, poll, VENDOR_ERR_OK);
+	}
+
+	return true;
+}
+
+static bool handle_mmq_cqe(struct mana_ib_qp *qp, u32 fsn,
+			   struct mana_cq_poll *poll)
+{
+	return mana_advance_send(qp, true, fsn, poll);
+}
+
+static bool handle_psn_cqe(struct mana_ib_qp *qp, u32 psn,
+			   struct mana_cq_poll *poll)
+{
+	struct mana_ib_dev *mdev = container_of(qp->ibqp.device, struct mana_ib_dev, ib_dev);
+	struct shadow_wqe_header *shadow_wqe;
+	struct gdma_queue *recv_q;
+	bool consumed;
+	u32 arm_psn;
+
+	if (qp->ibqp.qp_type != IB_QPT_RC)
+		return true;
+
+	recv_q = qp->rc_qp.queues[MANA_RC_RECV_QUEUE_REQUESTER].kmem;
+
+	consumed = mana_advance_send(qp, false, psn, poll);
+	if (consumed) {
+		shadow_wqe = shadow_queue_get_next_to_signal(&qp->shadow_sq);
+		arm_psn = shadow_wqe ? shadow_wqe->wqe_offset_or_psn : psn + 1;
+		mana_gd_wq_ring_doorbell_ext(mdev_to_gc(mdev), recv_q, arm_psn,
+					     SET_ARM_BIT, MANA_PSN_CLIENT_OFFSET);
+	}
+
+	return consumed;
+}
+
+static bool next_shadow_wqe_is_sq(struct shadow_wqe_header *shadow_wqe_sq,
+				  struct shadow_wqe_header *shadow_wqe_mq)
+{
+	if (!shadow_wqe_mq)
+		return true;
+
+	if (!shadow_wqe_sq)
+		return false;
+
+	return (PSN_LE(shadow_wqe_sq->wqe_offset_or_psn, shadow_wqe_mq->wqe_offset_or_psn) &&
+		PSN_LE(shadow_wqe_sq->fsn, shadow_wqe_mq->fsn));
+}
+
 static bool mana_handle_cqe(struct mana_ib_cq *cq, struct mana_ib_dev *mdev,
 			    struct mana_cq_poll *poll)
 {
 	struct gdma_comp *cqe = &cq->pending_cqe;
 	struct mana_rdma_cqe *rdma_cqe = (struct mana_rdma_cqe *)cqe->cqe_data;
 	struct mana_ib_qp *qp = mana_get_qp_ref(mdev, cqe->wq_num, cqe->is_sq);
+	bool consumed = true;
 
 	if (!qp)
 		return true;
 
 	switch (rdma_cqe->cqe_type) {
+	case CQE_TYPE_ARMED_CMPL:
+		consumed = handle_psn_cqe(qp, rdma_cqe->rc_armed_completion.psn, poll);
+		break;
+	case CQE_TYPE_LWR:
+		consumed = handle_mmq_cqe(qp, rdma_cqe->rc_mm.fsn, poll);
+		break;
 	case CQE_TYPE_UD_SEND:
 		if (cqe->is_sq) {
 			handle_ud_sq_cqe(qp, rdma_cqe, poll);
@@ -408,6 +527,10 @@ static bool mana_handle_cqe(struct mana_ib_cq *cq, struct mana_ib_dev *mdev,
 		}
 		fallthrough;
 	case CQE_TYPE_UD_SEND_IMM:
+	case CQE_TYPE_RC_SEND:
+	case CQE_TYPE_RC_SEND_IMM:
+	case CQE_TYPE_RC_SEND_INV:
+	case CQE_TYPE_RC_WRITE_IMM:
 		handle_rq_cqe(qp, cqe, poll);
 		break;
 	default:
@@ -416,21 +539,27 @@ static bool mana_handle_cqe(struct mana_ib_cq *cq, struct mana_ib_dev *mdev,
 		break;
 	}
 	mana_put_qp_ref(qp);
-	return true;
+	return consumed;
 }
 
 static void mana_flush_completions(struct mana_ib_cq *cq, struct mana_cq_poll *poll)
 {
-	struct shadow_wqe_header *wqe;
+	struct shadow_wqe_header *sq_wqe, *mmq_wqe, *wqe;
 	struct mana_ib_qp *qp;
+	bool next_is_sq;
 
 	if (poll->produced >= poll->budget)
 		return;
 
 	list_for_each_entry(qp, &cq->send_err_qp_list, send_err_node) {
-		while (poll->produced < poll->budget &&
-		       shadow_queue_get_next_to_consume(&qp->shadow_sq))
-			mana_complete_send(qp, poll, VENDOR_ERR_SW_FLUSHED);
+		while (poll->produced < poll->budget) {
+			sq_wqe = shadow_queue_get_next_to_consume(&qp->shadow_sq);
+			mmq_wqe = shadow_queue_get_next_to_consume(&qp->shadow_mmq);
+			if (!sq_wqe && !mmq_wqe)
+				break;
+			next_is_sq = next_shadow_wqe_is_sq(sq_wqe, mmq_wqe);
+			mana_complete_send(qp, !next_is_sq, poll, VENDOR_ERR_SW_FLUSHED);
+		}
 		if (poll->produced == poll->budget)
 			return;
 	}
drivers/infiniband/hw/mana/mana_ib.h +31 -0
index c7968d3c7ecd..1fc816792fc6 100644
@@ -687,6 +687,12 @@ enum mana_ib_error_code {
 enum mana_ib_cqe_type {
 	CQE_TYPE_UD_SEND = 1,
 	CQE_TYPE_UD_SEND_IMM = 2,
+	CQE_TYPE_RC_SEND = 3,
+	CQE_TYPE_RC_SEND_IMM = 4,
+	CQE_TYPE_RC_SEND_INV = 5,
+	CQE_TYPE_RC_WRITE_IMM = 6,
+	CQE_TYPE_ARMED_CMPL = 7,
+	CQE_TYPE_LWR = 8,
 }; /* HW DATA */
 
 struct mana_rdma_cqe {
@@ -710,6 +716,31 @@ struct mana_rdma_cqe {
 			u32 imm_data;
 			u32 rx_wqe_offset;
 		} ud_recv;
+		struct {
+			u32 cqe_type		: 8;
+			u32 reserved1		: 24;
+			u32 msg_len;
+			u32 psn			: 24;
+			u32 reserved2		: 8;
+			u32 imm_data;
+			u32 rx_wqe_offset;
+		} rc_recv;
+		struct {
+			u32 cqe_type		: 8;
+			u32 reserved1		: 24;
+			u32 msn			: 24;
+			u32 syndrome		: 8;
+			u32 psn			: 24;
+			u32 reserved2		: 8;
+		} rc_armed_completion;
+		struct {
+			u32 cqe_type		: 8;
+			u32 reserved1		: 24;
+			u32 psn			: 24;
+			u32 reserved2		: 8;
+			u32 fsn			: 24;
+			u32 reserved3		: 8;
+		} rc_mm;
 	};
 }; /* HW DATA */
 
drivers/infiniband/hw/mana/shadow_queue.h +22 -0
index b5d6bb28bd45..9abf390cf3a3 100644
@@ -26,6 +26,8 @@ struct shadow_queue {
 	u64 prod_idx;
 	/* Unmasked consumer index, Incremented on cq polling */
 	u64 cons_idx;
+	/* Unmasked index of next-to-signal shadow WQE */
+	u64 next_to_signal_idx;
 	/* queue size in wqes */
 	u32 length;
 	/* distance between elements in bytes */
@@ -50,6 +52,7 @@ static inline void reset_shadow_queue(struct shadow_queue *queue)
 {
 	queue->prod_idx = 0;
 	queue->cons_idx = 0;
+	queue->next_to_signal_idx = 0;
 }
 
 static inline void destroy_shadow_queue(struct shadow_queue *queue)
@@ -105,4 +108,23 @@ static inline void shadow_queue_advance_consumer(struct shadow_queue *queue)
 	smp_store_release(&queue->cons_idx, queue->cons_idx + 1);
 }
 
+static inline struct shadow_wqe_header *
+shadow_queue_get_next_to_signal(struct shadow_queue *queue)
+{
+	struct shadow_wqe_header *wqe = NULL;
+	/* Inspect only entries that have been fully published by posting. */
+	u64 prod_idx = smp_load_acquire(&queue->prod_idx);
+
+	queue->next_to_signal_idx = max(queue->next_to_signal_idx,
+					queue->cons_idx);
+	while (queue->next_to_signal_idx < prod_idx) {
+		wqe = shadow_queue_get_element(queue, queue->next_to_signal_idx);
+		queue->next_to_signal_idx++;
+		if (!(wqe->flags & MANA_WQ_NO_SIGNAL_WC))
+			return wqe;
+	}
+
+	return NULL;
+}
+
 #endif
Signature · 1 line
2.43.0
#14 lore raw

Re: [PATCH rdma-next 07/10] RDMA/mana_ib: Poll RC completions using PSN and FSN progress

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] rdma/mana_ib: interrupt storm from psn+1 arming on unsignaled WQEs --

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] rdma/mana_ib: interrupt storm from psn+1 arming on unsignaled WQEs --

commit f323a2eb29a1338a158b3fde42ecaf1e6e1cc432
Author: Konstantin Taranov <[email protected]>
Subject: RDMA/mana_ib: Poll RC completions using PSN and FSN progress

This commit decodes RC receive completions, armed-PSN notifications, and MMQ completions. It retires requester and MMQ shadow entries using sequence numbers, honors cross-queue fence dependencies, and suppresses successful WCs for unsignaled work requests.

Quoted 4 lines
drivers/infiniband/hw/mana/cq.c +0 -0
index 843e2c64b32bc..0ceb67ed62afa 100644

[ ... ]

Quoted 1 line
drivers/infiniband/hw/mana/cq.c +0 -0
@@ -422,17 +422,104 @@ static void handle_rq_cqe(struct mana_ib_qp *qp, struct gdma_comp *cqe,

[ ... ]

Quoted 14 lines
drivers/infiniband/hw/mana/cq.c +14 -0
+static bool handle_psn_cqe(struct mana_ib_qp *qp, u32 psn,
+			   struct mana_cq_poll *poll)
+{
+	struct mana_ib_dev *mdev = container_of(qp->ibqp.device, struct mana_ib_dev, ib_dev);
+	struct shadow_wqe_header *shadow_wqe;
+	struct gdma_queue *recv_q;
+	bool consumed;
+	u32 arm_psn;
+
+	if (qp->ibqp.qp_type != IB_QPT_RC)
+		return true;
+
+	recv_q = qp->rc_qp.queues[MANA_RC_RECV_QUEUE_REQUESTER].kmem;
+

drivers/infiniband/hw/mana/cq.c static void handle_rq_cqe(struct mana_ib_qp *qp, struct gdma_comp *cqe, · line 490

Quoted 10 lines
drivers/infiniband/hw/mana/cq.c +10 -0
+	consumed = mana_advance_send(qp, false, psn, poll);
+	if (consumed) {
+		shadow_wqe = shadow_queue_get_next_to_signal(&qp->shadow_sq);
+		arm_psn = shadow_wqe ? shadow_wqe->wqe_offset_or_psn : psn + 1;
+		mana_gd_wq_ring_doorbell_ext(mdev_to_gc(mdev), recv_q, arm_psn,
+					     SET_ARM_BIT, MANA_PSN_CLIENT_OFFSET);
+	}
+
+	return consumed;
+}

[Severity: High]
Does unconditionally arming psn + 1 here when shadow_wqe is NULL trigger an interrupt storm?

When processing an ARMED_CMPL CQE here in handle_psn_cqe(), if there are no pending signaled WQEs (because shadow_queue_get_next_to_signal() skips over unsignaled ones and returns NULL), the driver unconditionally arms psn + 1.

The hardware then generates a CQE for the very next packet. handle_psn_cqe() will process this, find no signaled WQEs again, and arm psn + 2.

Does this cycle force the hardware to generate an interrupt and CQE for every single packet of every unsignaled WQE, defeating the unsignaled optimization?

Could arming logic be initiated during post_send() when a signaled WQE is posted, rather than relying on sequential fallback in the completion handler?

Quoted 4 lines
drivers/infiniband/hw/mana/shadow_queue.h +0 -0
index b5d6bb28bd452..9abf390cf3a30 100644

[ ... ]

Quoted 12 lines
drivers/infiniband/hw/mana/shadow_queue.h +8 -0
@@ -105,4 +108,23 @@ static inline void shadow_queue_advance_consumer(struct shadow_queue *queue)
 	smp_store_release(&queue->cons_idx, queue->cons_idx + 1);
 }
 
+static inline struct shadow_wqe_header *
+shadow_queue_get_next_to_signal(struct shadow_queue *queue)
+{
+	struct shadow_wqe_header *wqe = NULL;
+	/* Inspect only entries that have been fully published by posting. */
+	u64 prod_idx = smp_load_acquire(&queue->prod_idx);
+
+	queue->next_to_signal_idx = max(queue->next_to_signal_idx,

drivers/infiniband/hw/mana/shadow_queue.h static inline void shadow_queue_advance_consumer(struct shadow_queue *queue) · line 128

Quoted 10 lines
drivers/infiniband/hw/mana/shadow_queue.h +10 -0
+					queue->cons_idx);
+	while (queue->next_to_signal_idx < prod_idx) {
+		wqe = shadow_queue_get_element(queue, queue->next_to_signal_idx);
+		queue->next_to_signal_idx++;
+		if (!(wqe->flags & MANA_WQ_NO_SIGNAL_WC))
+			return wqe;
+	}
+
+	return NULL;
+}

[Severity: High]
Since this loop skips unsignaled WQEs and returns NULL when only unsignaled WQEs are pending, does it force the fallback psn + 1 arming behavior in handle_psn_cqe() mentioned above?

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=7
#15 lore raw

PATCH 8/10 RDMA/mana_ib: Flush and notify CQs when kernel QPs enter ERR

In reply to Konstantin Taranov

From: Konstantin Taranov <[email protected]>

Complete the RC software flush path by adding a kernel QP to the CQ error lists when it transitions to the ERR state.

Invoke the CQ handlers after the transition to flush pending work requests.

---
 drivers/infiniband/hw/mana/mana_ib.h |  1 +
 drivers/infiniband/hw/mana/qp.c      | 47 ++++++++++++++++++++++++++--
 2 files changed, 45 insertions(+), 3 deletions(-)
drivers/infiniband/hw/mana/mana_ib.h +1 -0
index 1fc816792fc6..755fb87bfda1 100644
@@ -269,6 +269,7 @@ struct mana_ib_qp {
 	bool pending_mmq_fence;
 	bool sq_sig_all;
 	enum ib_mtu mtu;
+	enum ib_qp_state state;
 
 	/* Serializes receive WR posting. */
 	spinlock_t rq_lock;
drivers/infiniband/hw/mana/qp.c +44 -3

94 lines not on this page · raw message

Signature · 1 line
2.43.0
#16 lore raw

Re: [PATCH rdma-next 08/10] RDMA/mana_ib: Flush and notify CQs when kernel QPs enter ERR

Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 9 lines

From: Konstantin Taranov <[email protected]>

Complete the RC software flush path by adding a kernel QP to the CQ error lists when it transitions to the ERR state.

Invoke the CQ handlers after the transition to flush pending work requests.

Sashiko has reviewed this patch and found no issues. It looks great!

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=8
#17 lore raw

PATCH 9/10 RDMA/mana_ib: Handle error CQEs for RC QPs

In reply to Konstantin Taranov

From: Konstantin Taranov <[email protected]>

Decode hardware error CQEs and process them on the corresponding work queue. Complete the failing shadow entry with the vendor status from the CQE.

Defer transitioning RC QPs to the ERR state so that the other work queues can be flushed.

---
 drivers/infiniband/hw/mana/cq.c      | 91 ++++++++++++++++++++++++++++
 drivers/infiniband/hw/mana/mana_ib.h | 41 ++++++++++++-
 drivers/infiniband/hw/mana/qp.c      | 17 ++++++
 3 files changed, 148 insertions(+), 1 deletion(-)
drivers/infiniband/hw/mana/cq.c +91 -0
index 29eb12c1848e..3157feaa3092 100644
@@ -18,6 +18,7 @@ static enum ib_wc_status vendor_error_to_wc_error(uint32_t vendor_error)
 		return IB_WC_SUCCESS;
 	case VENDOR_ERR_RX_PKT_LEN:
 	case VENDOR_ERR_RX_MSG_LEN_OVFL:
+	case VENDOR_ERR_RX_READRESP_LEN_MISMATCH:
 		return IB_WC_LOC_LEN_ERR;
 	case VENDOR_ERR_TX_GDMA_CORRUPTED_WQE:
 	case VENDOR_ERR_TX_PCIE_WQE:
@@ -37,12 +38,30 @@ static enum ib_wc_status vendor_error_to_wc_error(uint32_t vendor_error)
 	case VENDOR_ERR_TX_ATB_WQE_ACCESS_VIOLATION:
 	case VENDOR_ERR_TX_ATB_WQE_ADDR_RANGE:
 	case VENDOR_ERR_TX_ATB_WQE_CONFIG_ERR:
+	case VENDOR_ERR_TX_RDMA_ATB_CMD_MISS:
+	case VENDOR_ERR_TX_RDMA_ATB_CMD_IDX_ERROR:
+	case VENDOR_ERR_TX_RDMA_ATB_CMD_TAG_MISMATCH_ERROR:
+	case VENDOR_ERR_TX_RDMA_ATB_CMD_PDID_MISMATCH_ERROR:
+	case VENDOR_ERR_TX_RDMA_ATB_CMD_AR_ERROR:
+	case VENDOR_ERR_TX_RDMA_ATB_CMD_PT_OVF:
+	case VENDOR_ERR_TX_RDMA_ATB_CMD_PT_LENGHT_MISMATCH:
+	case VENDOR_ERR_TX_RDMA_ATB_CMD_ILLEGAL_CMD:
 	case VENDOR_ERR_RX_ATB_SGE_ADDR_RANGE:
 	case VENDOR_ERR_RX_ATB_SGE_MISSCONFIG:
 		return IB_WC_LOC_PROT_ERR;
 	case VENDOR_ERR_RX_ATB_SGE_ADDR_RIGHT:
 	case VENDOR_ERR_RX_GFID:
 		return IB_WC_LOC_ACCESS_ERR;
+	case VENDOR_ERR_RX_OP_REQ:
+		return IB_WC_REM_INV_REQ_ERR;
+	case VENDOR_ERR_RX_ATB_RKEY_MISCONFIG_ERR:
+	case VENDOR_ERR_RX_ATB_RKEY_ADDR_RIGHT:
+	case VENDOR_ERR_RX_ATB_RKEY_ADDR_RANGE:
+	case VENDOR_ERR_RX_REMOTE_ACCESS_NAK:
+		return IB_WC_REM_ACCESS_ERR;
+	case VENDOR_ERR_RX_INVALID_REQ_NAK:
+	case VENDOR_ERR_RX_REMOTE_OP_ERR_NAK:
+		return IB_WC_REM_OP_ERR;
 	case VENDOR_ERR_RX_MISBEHAVING_CLIENT:
 	case VENDOR_ERR_RX_CLIENT_ID:
 	case VENDOR_ERR_RX_PCIE:
@@ -58,6 +77,10 @@ static enum ib_wc_status vendor_error_to_wc_error(uint32_t vendor_error)
 	case VENDOR_ERR_RX_NOT_EMPTY_ON_DISABLE:
 	case VENDOR_ERR_SW_FLUSHED:
 		return IB_WC_WR_FLUSH_ERR;
+	case VENDOR_ERR_TX_RETRY_LIMIT_EXCEEDED:
+		return IB_WC_RETRY_EXC_ERR;
+	case VENDOR_ERR_RX_RNR_NAK:
+		return IB_WC_RNR_RETRY_EXC_ERR;
 	default:
 		return IB_WC_GENERAL_ERR;
 	}
@@ -489,6 +512,71 @@ static bool handle_psn_cqe(struct mana_ib_qp *qp, u32 psn,
 	return consumed;
 }
 
+static bool mana_cqe_is_send(struct mana_ib_qp *qp, struct gdma_comp *cqe)
+{
+	u32 qtype = cqe->wq_num & MANA_QID_SUBTYPE_MASK;
+
+	if (qp->ibqp.qp_type != IB_QPT_RC)
+		return cqe->is_sq;
+
+	return cqe->is_sq ? (qtype == MANA_QID_SUBTYPE_SREQ ||
+			    qtype == MANA_QID_SUBTYPE_MMQ) :
+			    qtype == MANA_QID_SUBTYPE_RREQ;
+}
+
+static void mana_ib_schedule_qp_error(struct mana_ib_qp *qp)
+{
+	/* CQ polling cannot sleep; teardown drains this reference before freeing. */
+	refcount_inc(&qp->refcount);
+	if (!schedule_work(&qp->error_work))
+		mana_put_qp_ref(qp);
+}
+
+static bool handle_error_cqe(struct mana_ib_cq *cq, struct mana_ib_qp *qp,
+			     struct gdma_comp *cqe, struct mana_cq_poll *poll)
+{
+	bool mmq = cqe->is_sq && (cqe->wq_num & MANA_QID_SUBTYPE_MASK) == MANA_QID_SUBTYPE_MMQ;
+	struct mana_rdma_cqe *rdma_cqe = (struct mana_rdma_cqe *)cqe->cqe_data;
+	bool send = mana_cqe_is_send(qp, cqe);
+	u32 vendor_err = rdma_cqe->error.vendor_error;
+	struct shadow_wqe_header *shadow_wqe;
+	struct shadow_queue *shadow_q;
+
+	if (send && qp->ibqp.qp_type == IB_QPT_RC) {
+		/* The error PSN/FSN belongs to the failing WQE, not a success. */
+		if (!mana_advance_send(qp, mmq, rdma_cqe->error.psn - 1, poll))
+			return false;
+		if (poll->produced == poll->budget)
+			return false;
+	}
+
+	if (!send)
+		shadow_q = &qp->shadow_rq;
+	else if (mmq)
+		shadow_q = &qp->shadow_mmq;
+	else
+		shadow_q = &qp->shadow_sq;
+
+	shadow_wqe = shadow_queue_get_next_to_consume(shadow_q);
+	if (shadow_wqe) {
+		mana_fill_wc(qp, poll, shadow_wqe,
+			     send ? shadow_wqe->send_opcode : IB_WC_RECV, vendor_err);
+		shadow_queue_advance_consumer(shadow_q);
+	}
+
+	if (send) {
+		if (list_empty(&qp->send_err_node))
+			list_add_tail(&qp->send_err_node, &cq->send_err_qp_list);
+	} else {
+		if (list_empty(&qp->recv_err_node))
+			list_add_tail(&qp->recv_err_node, &cq->recv_err_qp_list);
+	}
+	if (qp->ibqp.qp_type != IB_QPT_GSI)
+		mana_ib_schedule_qp_error(qp);
+
+	return true;
+}
+
 static bool next_shadow_wqe_is_sq(struct shadow_wqe_header *shadow_wqe_sq,
 				  struct shadow_wqe_header *shadow_wqe_mq)
 {
@@ -520,6 +608,9 @@ static bool mana_handle_cqe(struct mana_ib_cq *cq, struct mana_ib_dev *mdev,
 	case CQE_TYPE_LWR:
 		consumed = handle_mmq_cqe(qp, rdma_cqe->rc_mm.fsn, poll);
 		break;
+	case CQE_TYPE_ERROR:
+		consumed = handle_error_cqe(cq, qp, cqe, poll);
+		break;
 	case CQE_TYPE_UD_SEND:
 		if (cqe->is_sq) {
 			handle_ud_sq_cqe(qp, rdma_cqe, poll);
drivers/infiniband/hw/mana/mana_ib.h +40 -1

98 lines not on this page · raw message

drivers/infiniband/hw/mana/qp.c +17 -0
index c4992950fe17..ed6dbd66bd36 100644
@@ -5,6 +5,8 @@
 
 #include "mana_ib.h"
 
+static void mana_ib_qp_error_work(struct work_struct *work);
+
 static int mana_ib_cfg_vport_steering(struct mana_ib_dev *dev,
 				      struct net_device *ndev,
 				      mana_handle_t default_rxobj,
@@ -942,6 +944,7 @@ int mana_ib_create_qp(struct ib_qp *ibqp, struct ib_qp_init_attr *attr,
 	spin_lock_init(&qp->rq_lock);
 	spin_lock_init(&qp->sq_lock);
 	mutex_init(&qp->modify_lock);
+	INIT_WORK(&qp->error_work, mana_ib_qp_error_work);
 	INIT_LIST_HEAD(&qp->send_err_node);
 	INIT_LIST_HEAD(&qp->recv_err_node);
 
@@ -1135,6 +1138,20 @@ out_unlock:
 	return ret;
 }
 
+static void mana_ib_qp_error_work(struct work_struct *work)
+{
+	struct mana_ib_qp *qp = container_of(work, struct mana_ib_qp, error_work);
+	struct ib_qp_attr attr = { .qp_state = IB_QPS_ERR };
+	int ret;
+
+	ret = mana_ib_modify_qp(&qp->ibqp, &attr, IB_QP_STATE, NULL);
+	if (ret)
+		ibdev_err(qp->ibqp.device, "Failed to move QP %u to ERR: %d\n",
+			  qp->ibqp.qp_num, ret);
+
+	mana_put_qp_ref(qp);
+}
+
 static int mana_ib_destroy_qp_rss(struct mana_ib_qp *qp,
 				  struct ib_rwq_ind_table *ind_tbl,
 				  struct ib_udata *udata)
Signature · 1 line
2.43.0
#18 lore raw

Re: [PATCH rdma-next 09/10] RDMA/mana_ib: Handle error CQEs for RC QPs

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] rdma/mana_ib: module unload race due to missing workqueue teardown --

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] rdma/mana_ib: module unload race due to missing workqueue teardown --

commit a138de6b28986049d5aed34daaeffc27d848b82d
Author: Konstantin Taranov <[email protected]>

RDMA/mana_ib: Handle error CQEs for RC QPs

Decode hardware error CQEs and process them on the corresponding work queue. Complete the failing shadow entry with the vendor status from the CQE.

Defer transitioning RC QPs to the ERR state so that the other work queues can be flushed.

Quoted 4 lines
drivers/infiniband/hw/mana/qp.c +0 -0
index b3891956f0666..0f60909cbe848 100644

[ ... ]

Quoted 7 lines
drivers/infiniband/hw/mana/qp.c +3 -0
@@ -1136,6 +1139,20 @@ int mana_ib_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr,
 	return ret;
 }
 
+static void mana_ib_qp_error_work(struct work_struct *work)
+{
+	struct mana_ib_qp *qp = container_of(work, struct mana_ib_qp, error_work);

drivers/infiniband/hw/mana/qp.c int mana_ib_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, · line 1154

Quoted 10 lines
drivers/infiniband/hw/mana/qp.c +10 -0
+	struct ib_qp_attr attr = { .qp_state = IB_QPS_ERR };
+	int ret;
+
+	ret = mana_ib_modify_qp(&qp->ibqp, &attr, IB_QP_STATE, NULL);
+	if (ret)
+		ibdev_err(qp->ibqp.device, "Failed to move QP %u to ERR: %d\n",
+			  qp->ibqp.qp_num, ret);
+
+	mana_put_qp_ref(qp);
+}

[Severity: High]
Does this create a race condition with module unload?

If the QP destruction path waits on wait_for_completion(&qp->free) and proceeds immediately after mana_put_qp_ref() drops the refcount, the teardown can complete and the module can be unloaded before the worker thread finishes returning.

If the worker thread is preempted right after signaling the completion in mana_put_qp_ref(), it could later resume execution and attempt to return into potentially unmapped module memory, causing a kernel panic.

Should the teardown path explicitly flush or cancel the queued work instead of relying solely on refcounting for synchronization?

Quoted 4 lines
drivers/infiniband/hw/mana/qp.c +1 -0
+
 static int mana_ib_destroy_qp_rss(struct mana_ib_qp *qp,
 				  struct ib_rwq_ind_table *ind_tbl,
 				  struct ib_udata *udata)
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=9
#19 lore raw

PATCH 10/10 RDMA/mana_ib: Drain kernel receive and send queues

In reply to Konstantin Taranov

From: Konstantin Taranov <[email protected]>

Implement the core drain callbacks by moving the QP to ERR, posting a completion-bearing marker, and waiting until that marker is flushed.

---
 drivers/infiniband/hw/mana/device.c  |  2 +
 drivers/infiniband/hw/mana/mana_ib.h |  2 +
 drivers/infiniband/hw/mana/qp.c      | 91 ++++++++++++++++++++++++++++
 3 files changed, 95 insertions(+)
drivers/infiniband/hw/mana/device.c +2 -0
index cdaada7f6e8a..07c7fe678ff7 100644
@@ -38,6 +38,8 @@ static const struct ib_device_ops mana_ib_dev_ops = {
 	.destroy_rwq_ind_table = mana_ib_destroy_rwq_ind_table,
 	.destroy_wq = mana_ib_destroy_wq,
 	.disassociate_ucontext = mana_ib_disassociate_ucontext,
+	.drain_rq = mana_ib_drain_rq,
+	.drain_sq = mana_ib_drain_sq,
 	.get_dma_mr = mana_ib_get_dma_mr,
 	.get_link_layer = mana_ib_get_link_layer,
 	.get_port_immutable = mana_ib_get_port_immutable,
drivers/infiniband/hw/mana/mana_ib.h +2 -0
index 406b85abdef2..6eb06a1036ab 100644
@@ -1059,6 +1059,8 @@ int mana_ib_post_recv(struct ib_qp *ibqp, const struct ib_recv_wr *wr,
 int mana_ib_post_send(struct ib_qp *ibqp, const struct ib_send_wr *wr,
 		      const struct ib_send_wr **bad_wr);
 
+void mana_ib_drain_sq(struct ib_qp *ibqp);
+void mana_ib_drain_rq(struct ib_qp *ibqp);
 void mana_drain_gsi_sqs(struct mana_ib_dev *mdev);
 int mana_ib_poll_cq(struct ib_cq *ibcq, int num_entries, struct ib_wc *wc);
 int mana_ib_arm_cq(struct ib_cq *ibcq, enum ib_cq_notify_flags flags);
drivers/infiniband/hw/mana/qp.c +91 -0

94 lines not on this page · raw message

Signature · 1 line
2.43.0
#20 lore raw

Re: [PATCH rdma-next 10/10] RDMA/mana_ib: Drain kernel receive and send queues

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [High] rdma/mana_ib: null pointer dereference in mana_ib_drain_sq() on UD/GSI QPs - [Medium]…

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] rdma/mana_ib: null pointer dereference in mana_ib_drain_sq() on UD/GSI QPs
- [Medium] rdma/mana_ib: user-triggerable WARN_ONCE via manual comp_handler invocation for direct CQs --

commit cbadbb6a8acdc7477a3606271c2a70d35b56708b
Author: Konstantin Taranov <[email protected]>

RDMA/mana_ib: Drain kernel receive and send queues

This commit implements the core drain callbacks for the mana_ib driver by moving the QP to the ERR state, posting a completion-bearing marker, and waiting until that marker is flushed.

Quoted 4 lines
drivers/infiniband/hw/mana/qp.c +0 -0
index 0f60909cbe848..4e6f15fb4887b 100644

[ ... ]

drivers/infiniband/hw/mana/qp.c int mana_ib_destroy_qp(struct ib_qp *ibqp, struct ib_udata *udata) · line 1358

Quoted 11 lines
drivers/infiniband/hw/mana/qp.c +11 -0
+void mana_ib_drain_sq(struct ib_qp *ibqp)
+{
+	struct ib_qp_attr attr = { .qp_state = IB_QPS_ERR };
+	struct mana_ib_drain_cqe sdrain;
+	struct ib_rdma_wr swr = {
+		.wr = {
+			.next = NULL,
+			.wr_cqe = &sdrain.cqe,
+			.opcode = IB_WR_SEND,
+		},
+	};

[Severity: High]
Does this WR structure correctly handle IB_QPT_UD or IB_QPT_GSI queue types?

When mana_ib_drain_sq() is called on a UD QP, it submits this stack-allocated ib_rdma_wr structure to ib_post_send(). The mana driver's send path in mana_ib_post_send_ud_one() expects an ib_ud_wr structure and downcasts this generic WR into it.

Because the ib_rdma_wr structure is smaller than ib_ud_wr, the downcast results in an out-of-bounds stack read over the overlapping memory. This causes the evaluated ah pointer to be NULL.

When the hardware scatter-gather list is built later via gdma_sgl[0].address = ah->dma_handle, dereferencing the NULL ah pointer causes a kernel panic.

Since unprivileged userspace applications can create and drain UD QPs via uverbs, can this be exploited as a local denial-of-service vulnerability?

[ ... ]

drivers/infiniband/hw/mana/qp.c int mana_ib_destroy_qp(struct ib_qp *ibqp, struct ib_udata *udata) · line 1379

Quoted 8 lines
drivers/infiniband/hw/mana/qp.c +8 -0
+	ret = ib_post_send(ibqp, &swr.wr, &bad_swr);
+	if (ret) {
+		ibdev_err(ibqp->device, "Failed to post SQ drain WR: %d\n", ret);
+		return;
+	}
+
+	if (cq->comp_handler)
+		cq->comp_handler(cq, cq->cq_context);

[Severity: Medium]
Is it safe to execute the comp_handler here unconditionally before checking if the CQ uses direct polling?

For CQs instantiated with IB_POLL_DIRECT, the RDMA core strictly assigns ib_cq_completion_direct() to the comp_handler to emit a WARN_ONCE on any unsolicited execution.

Unprivileged users can trigger QP draining via uverbs on direct polling CQs, meaning they can trivially trip this warning.

If the system is configured with panic_on_warn, could this lead to an unexpected denial of service?

Quoted 2 lines
drivers/infiniband/hw/mana/qp.c +2 -0
+
+	if (cq->poll_ctx == IB_POLL_DIRECT) {

[ ... ]

Quoted 2 lines
drivers/infiniband/hw/mana/qp.c +2 -0
+void mana_ib_drain_rq(struct ib_qp *ibqp)
+{

[ ... ]

drivers/infiniband/hw/mana/qp.c int mana_ib_destroy_qp(struct ib_qp *ibqp, struct ib_udata *udata) · line 1416

Quoted 8 lines
drivers/infiniband/hw/mana/qp.c +8 -0
+	ret = ib_post_recv(ibqp, &rwr, NULL);
+	if (ret) {
+		ibdev_err(ibqp->device, "Failed to post RQ drain WR: %d\n", ret);
+		return;
+	}
+
+	if (cq->comp_handler)
+		cq->comp_handler(cq, cq->cq_context);

[Severity: Medium]
Similarly to the send queue drain, will this unconditional comp_handler execution trigger the core's unsolicited completion warning for direct CQs?

Quoted 2 lines
drivers/infiniband/hw/mana/qp.c +2 -0
+
+	if (cq->poll_ctx == IB_POLL_DIRECT) {

[ ... ]

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=10