Skip to contentSkip to conversation

RDMA/bnxt_re: Check cqe flags to know imm_data vs inv_irkey

PATCH Applied Selvin Xavier 2 participants 2 messages Oct 28, 2024 – Oct 30, 2024

Subsystems (2)
lore
Share & tools
lore

Selvin Xavier

author maintainer
#0 lore raw

PATCH 1/1 RDMA/bnxt_re: Check cqe flags to know imm_data vs inv_irkey

From: Kashyap Desai <[email protected]>

Invalidate rkey is cpu endian and immediate data is in big endian format. Both immediate data and invalidate the remote key returned by HW is in little endian format.

While handling the commit in fixes tag, the difference between immediate data and invalidate rkey endianness was not considered.

Without changes of this patch, Kernel ULP was failing while processing inv_rkey.

dmesg log snippet -
nvme nvme0: Bogus remote invalidation for rkey 0x2000019Fix in this patch

Do endianness conversion based on completion queue entry flag. Also, the HW completions are already converted to host endianness in bnxt_qplib_cq_process_res_rc and bnxt_qplib_cq_process_res_ud and there is no need to convert it again in bnxt_re_poll_cq. Modified the union to hold the correct data type.

Fixes 95b087f87b78 ("bnxt_re: Fix imm_data endianness")
---
 drivers/infiniband/hw/bnxt_re/ib_verbs.c | 7 +++++--
 drivers/infiniband/hw/bnxt_re/qplib_fp.h | 2 +-
 2 files changed, 6 insertions(+), 3 deletions(-)
drivers/infiniband/hw/bnxt_re/ib_verbs.c +5 -2
index e66ae9f..1600967 100644
@@ -3633,7 +3633,7 @@ static void bnxt_re_process_res_shadow_qp_wc(struct bnxt_re_qp *gsi_sqp,
 	wc->byte_len = orig_cqe->length;
 	wc->qp = &gsi_qp->ib_qp;
 
-	wc->ex.imm_data = cpu_to_be32(le32_to_cpu(orig_cqe->immdata));
+	wc->ex.imm_data = cpu_to_be32(orig_cqe->immdata);
 	wc->src_qp = orig_cqe->src_qp;
 	memcpy(wc->smac, orig_cqe->smac, ETH_ALEN);
 	if (bnxt_re_is_vlan_pkt(orig_cqe, &vlan_id, &sl)) {
@@ -3778,7 +3778,10 @@ int bnxt_re_poll_cq(struct ib_cq *ib_cq, int num_entries, struct ib_wc *wc)
 				 (unsigned long)(cqe->qp_handle),
 				 struct bnxt_re_qp, qplib_qp);
 			wc->qp = &qp->ib_qp;
-			wc->ex.imm_data = cpu_to_be32(le32_to_cpu(cqe->immdata));
+			if (cqe->flags & CQ_RES_RC_FLAGS_IMM)
+				wc->ex.imm_data = cpu_to_be32(cqe->immdata);
+			else
+				wc->ex.invalidate_rkey = cqe->invrkey;
 			wc->src_qp = cqe->src_qp;
 			memcpy(wc->smac, cqe->smac, ETH_ALEN);
 			wc->port_num = 1;
drivers/infiniband/hw/bnxt_re/qplib_fp.h +1 -1
index 820611a..f55958e 100644
@@ -391,7 +391,7 @@ struct bnxt_qplib_cqe {
 	u16				cfa_meta;
 	u64				wr_id;
 	union {
-		__le32			immdata;
+		u32			immdata;
 		u32			invrkey;
 	};
 	u64				qp_handle;
Signature · 1 line
2.5.5
#1 lore raw
Quoted 8 lines · Selvin Xavier

Invalidate rkey is cpu endian and immediate data is in big endian format. Both immediate data and invalidate the remote key returned by HW is in little endian format.

While handling the commit in fixes tag, the difference between immediate data and invalidate rkey endianness was not considered.

[...]

Applied, thanks!

[1/1] RDMA/bnxt_re: Check cqe flags to know imm_data vs inv_irkey

      https://git.kernel.org/rdma/rdma/c/808ca6de989c59

Best regards,

Signature · 1 line
Leon Romanovsky <[email protected]>