Skip to contentSkip to conversation

Retrieve HW GID context from ib_gid_attr

PATCH Applied Selvin Xavier 2 participants 4 messages Feb 19, 2020

Subsystems (2)
lore
Share & tools
lore

Selvin Xavier

author maintainer
#0 lore raw

v4 cover Retrieve HW GID context from ib_gid_attr

Provide an option for vendor drivers to get the HW GID context from the ib_gid_attr during modify_qp and create_ah. Required for drivers/HW that maintains HW gid index different than the host sgid_index.

Please review and merge

Thanks,
Selvin Xavier

v3 -> v4:
Addressed Jason's comments. Removed unnecessary validation and locking as the reference to the GID table entry should be taken before invoking the new symbol.

v2 -> v3:
Added a new symbol to retrieve the hw context.

v1 -> v2:
Addressed review comments from Parav

Selvin Xavier (2):
RDMA/core: Add helper function to retrieve driver gid context from gid
attr
RDMA/bnxt_re: Use rdma_read_gid_hw_context to retrieve HW gid index

 drivers/infiniband/core/cache.c          | 20 ++++++++++++++++++++
 drivers/infiniband/hw/bnxt_re/ib_verbs.c | 23 ++++++++++++-----------
 include/rdma/ib_cache.h                  |  1 +
 3 files changed, 33 insertions(+), 11 deletions(-)
Signature · 1 line
2.5.5
author maintainer
#1 lore raw

PATCH v4 1/2 RDMA/core: Add helper function to retrieve driver gid context from gid attr

Adding a helper function to retrieve the driver gid context from the gid attr.

---
 drivers/infiniband/core/cache.c | 20 ++++++++++++++++++++
 include/rdma/ib_cache.h         |  1 +
 2 files changed, 21 insertions(+)
drivers/infiniband/core/cache.c +20 -0
index 17bfedd..2ebf322 100644
@@ -973,6 +973,26 @@ int rdma_query_gid(struct ib_device *device, u8 port_num,
 EXPORT_SYMBOL(rdma_query_gid);
 
 /**
+ * rdma_read_gid_hw_context - Read the HW GID context from GID attribute
+ * @attr:		Potinter to the GID attribute
+ *
+ * rdma_read_gid_hw_context() reads the vendor drivers GID HW
+ * context corresponding to SGID attr. Callers are required to already
+ * be holding the reference to existing GID entry.
+ *
+ * Returns HW GID context
+ *
+ */
+void *rdma_read_gid_hw_context(const struct ib_gid_attr *attr)
+{
+	struct ib_gid_table_entry *entry =
+		container_of(attr, struct ib_gid_table_entry, attr);
+
+	return entry->context;
+}
+EXPORT_SYMBOL(rdma_read_gid_hw_context);
+
+/**
  * rdma_find_gid - Returns SGID attributes if the matching GID is found.
  * @device: The device to query.
  * @gid: The GID value to search for.
include/rdma/ib_cache.h +1 -0
index 870b5e6..e06d133 100644
@@ -39,6 +39,7 @@
 
 int rdma_query_gid(struct ib_device *device, u8 port_num, int index,
 		   union ib_gid *gid);
+void *rdma_read_gid_hw_context(const struct ib_gid_attr *attr);
 const struct ib_gid_attr *rdma_find_gid(struct ib_device *device,
 					const union ib_gid *gid,
 					enum ib_gid_type gid_type,
Signature · 1 line
2.5.5
author maintainer
#2 lore raw

PATCH v4 2/2 RDMA/bnxt_re: Use rdma_read_gid_hw_context to retrieve HW gid index

In reply to Selvin Xavier

bnxt_re HW maintains a GID table with only a single entry for the two duplicate GID entries (v1 and v2). Driver needs to map stack gid index to the HW table gid index. Use the new API rdma_read_gid_hw_context () to retrieve the HW GID context to get the HW table index.

---
 drivers/infiniband/hw/bnxt_re/ib_verbs.c | 23 ++++++++++++-----------
 1 file changed, 12 insertions(+), 11 deletions(-)
drivers/infiniband/hw/bnxt_re/ib_verbs.c +12 -11
index 52b6a4d..18579e8 100644
@@ -639,6 +639,7 @@ int bnxt_re_create_ah(struct ib_ah *ib_ah, struct rdma_ah_attr *ah_attr,
 	const struct ib_global_route *grh = rdma_ah_read_grh(ah_attr);
 	struct bnxt_re_dev *rdev = pd->rdev;
 	const struct ib_gid_attr *sgid_attr;
+	struct bnxt_re_gid_ctx *ctx;
 	struct bnxt_re_ah *ah = container_of(ib_ah, struct bnxt_re_ah, ib_ah);
 	u8 nw_type;
 	int rc;
@@ -654,19 +655,18 @@ int bnxt_re_create_ah(struct ib_ah *ib_ah, struct rdma_ah_attr *ah_attr,
 	/* Supply the configuration for the HW */
 	memcpy(ah->qplib_ah.dgid.data, grh->dgid.raw,
 	       sizeof(union ib_gid));
-	/*
-	 * If RoCE V2 is enabled, stack will have two entries for
-	 * each GID entry. Avoiding this duplicte entry in HW. Dividing
-	 * the GID index by 2 for RoCE V2
+	sgid_attr = grh->sgid_attr;
+	/* Get the HW context of the GID. The reference
+	 * of GID table entry is already taken by the caller.
 	 */
-	ah->qplib_ah.sgid_index = grh->sgid_index / 2;
+	ctx = rdma_read_gid_hw_context(sgid_attr);
+	ah->qplib_ah.sgid_index = ctx->idx;
 	ah->qplib_ah.host_sgid_index = grh->sgid_index;
 	ah->qplib_ah.traffic_class = grh->traffic_class;
 	ah->qplib_ah.flow_label = grh->flow_label;
 	ah->qplib_ah.hop_limit = grh->hop_limit;
 	ah->qplib_ah.sl = rdma_ah_get_sl(ah_attr);
 
-	sgid_attr = grh->sgid_attr;
 	/* Get network header type for this GID */
 	nw_type = rdma_gid_attr_network_type(sgid_attr);
 	ah->qplib_ah.nw_type = bnxt_re_stack_to_dev_nw_type(nw_type);
@@ -1593,6 +1593,7 @@ int bnxt_re_modify_qp(struct ib_qp *ib_qp, struct ib_qp_attr *qp_attr,
 		const struct ib_global_route *grh =
 			rdma_ah_read_grh(&qp_attr->ah_attr);
 		const struct ib_gid_attr *sgid_attr;
+		struct bnxt_re_gid_ctx *ctx;
 
 		qp->qplib_qp.modify_flags |= CMDQ_MODIFY_QP_MODIFY_MASK_DGID |
 				     CMDQ_MODIFY_QP_MODIFY_MASK_FLOW_LABEL |
@@ -1604,11 +1605,12 @@ int bnxt_re_modify_qp(struct ib_qp *ib_qp, struct ib_qp_attr *qp_attr,
 		memcpy(qp->qplib_qp.ah.dgid.data, grh->dgid.raw,
 		       sizeof(qp->qplib_qp.ah.dgid.data));
 		qp->qplib_qp.ah.flow_label = grh->flow_label;
-		/* If RoCE V2 is enabled, stack will have two entries for
-		 * each GID entry. Avoiding this duplicte entry in HW. Dividing
-		 * the GID index by 2 for RoCE V2
+		sgid_attr = grh->sgid_attr;
+		/* Get the HW context of the GID. The reference
+		 * of GID table entry is already taken by the caller.
 		 */
-		qp->qplib_qp.ah.sgid_index = grh->sgid_index / 2;
+		ctx = rdma_read_gid_hw_context(sgid_attr);
+		qp->qplib_qp.ah.sgid_index = ctx->idx;
 		qp->qplib_qp.ah.host_sgid_index = grh->sgid_index;
 		qp->qplib_qp.ah.hop_limit = grh->hop_limit;
 		qp->qplib_qp.ah.traffic_class = grh->traffic_class;
@@ -1616,7 +1618,6 @@ int bnxt_re_modify_qp(struct ib_qp *ib_qp, struct ib_qp_attr *qp_attr,
 		ether_addr_copy(qp->qplib_qp.ah.dmac,
 				qp_attr->ah_attr.roce.dmac);
 
-		sgid_attr = qp_attr->ah_attr.grh.sgid_attr;
 		rc = rdma_read_gid_l2_fields(sgid_attr, NULL,
 					     &qp->qplib_qp.smac[0]);
 		if (rc)
Signature · 1 line
2.5.5
#3 lore raw
In reply to Selvin Xavier
Quoted 18 lines · Selvin Xavier

Provide an option for vendor drivers to get the HW GID context from the ib_gid_attr during modify_qp and create_ah. Required for drivers/HW that maintains HW gid index different than the host sgid_index.

Please review and merge

Thanks,
Selvin Xavier

v3 -> v4:
Addressed Jason's comments. Removed unnecessary validation and locking as the reference to the GID table entry should be taken before invoking the new symbol.

v2 -> v3:
Added a new symbol to retrieve the hw context.

Quoted 8 lines

v1 -> v2:
Addressed review comments from Parav

Selvin Xavier (2):
RDMA/core: Add helper function to retrieve driver gid context from gid
attr
RDMA/bnxt_re: Use rdma_read_gid_hw_context to retrieve HW gid index

Applied to for-next, thanks

Jason