Skip to contentSkip to conversation

cxgb4 FR_NSMR_TPTE_WR support

PATCH Applied Steve Wise 5 participants 11 messages Sep 16, 2016 – Oct 3, 2016 4 messages not in the archive

Subsystems (5)
Networking Network drivers InfiniBand CXGB4 Ethernet driver (CXGB4) +1 more subsystems
lore
Share & tools
lore

Steve Wise

author
#0 lore raw

cover cxgb4 FR_NSMR_TPTE_WR support

This series enables a new work request to optimize small REG_MR operations. This is intended for 4.9. If everyone agrees, I suggest Doug take both the cxgb4 and iw_cxgb4 patches through his tree.

Thanks,

Steve.

---

Steve Wise (2):
cxgb4: advertise support for FR_NSMR_TPTE_WR
iw_cxgb4: add fast-path for small REG_MR operations

 drivers/infiniband/hw/cxgb4/cq.c                | 17 +++++++
 drivers/infiniband/hw/cxgb4/mem.c               |  2 +-
 drivers/infiniband/hw/cxgb4/qp.c                | 67 +++++++++++++++++++++----
 drivers/infiniband/hw/cxgb4/t4.h                |  4 +-
 drivers/infiniband/hw/cxgb4/t4fw_ri_api.h       | 12 +++++
 drivers/net/ethernet/chelsio/cxgb4/cxgb4.h      |  1 +
 drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c |  7 +++
 drivers/net/ethernet/chelsio/cxgb4/cxgb4_uld.h  |  1 +
 drivers/net/ethernet/chelsio/cxgb4/t4fw_api.h   |  2 +
 9 files changed, 102 insertions(+), 11 deletions(-)
Signature · 1 line
2.7.0
author
#1 lore raw

PATCH 1/2 cxgb4: advertise support for FR_NSMR_TPTE_WR

Query firmware for the FW_PARAMS_PARAM_DEV_RI_FR_NSMR_TPTE_WR parameter. If it exists and is 1, then advertise support for FR_NSMR_TPTE_WR to the ULDs.

Signed-off-by Steve Wise <[email protected]>
---
 drivers/net/ethernet/chelsio/cxgb4/cxgb4.h      | 1 +
 drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c | 7 +++++++
 drivers/net/ethernet/chelsio/cxgb4/cxgb4_uld.h  | 1 +
 drivers/net/ethernet/chelsio/cxgb4/t4fw_api.h   | 1 +
 4 files changed, 10 insertions(+)
drivers/net/ethernet/chelsio/cxgb4/cxgb4.h +1 -0
index 2e2aa9f..65207b3 100644
@@ -346,6 +346,7 @@ struct adapter_params {
 
 	unsigned int max_ordird_qp;       /* Max read depth per RDMA QP */
 	unsigned int max_ird_adapter;     /* Max read depth per adapter */
+	bool fr_nsmr_tpte_wr_support;	  /* FW support for FR_NSMR_TPTE_WR */
 };
 
 /* State needed to monitor the forward progress of SGE Ingress DMA activities
drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c +7 -0
index c762a8c..37e0c82 100644
@@ -2517,6 +2517,7 @@ static void uld_attach(struct adapter *adap, unsigned int uld)
 	lli.max_ird_adapter = adap->params.max_ird_adapter;
 	lli.ulptx_memwrite_dsgl = adap->params.ulptx_memwrite_dsgl;
 	lli.nodeid = dev_to_node(adap->pdev_dev);
+	lli.fr_nsmr_tpte_wr_support = adap->params.fr_nsmr_tpte_wr_support;
 
 	handle = ulds[uld].add(&lli);
 	if (IS_ERR(handle)) {

@@ -4016,6 +4017,12 @@ static int adap_init0(struct adapter *adap)
 		adap->params.ulptx_memwrite_dsgl = (ret == 0 && val[0] != 0);
 	}
 
+	/* See if FW supports FW_RI_FR_NSMR_TPTE_WR work request */
+	params[0] = FW_PARAM_DEV(RI_FR_NSMR_TPTE_WR);
+	ret = t4_query_params(adap, adap->mbox, adap->pf, 0,
+			      1, params, val);
+	adap->params.fr_nsmr_tpte_wr_support = (ret == 0 && val[0] != 0);
+
 	/*
 	 * Get device capabilities so we can determine what resources we need
 	 * to manage.
drivers/net/ethernet/chelsio/cxgb4/cxgb4_uld.h +1 -0
index f3c58aa..42e73f7 100644
@@ -280,6 +280,7 @@ struct cxgb4_lld_info {
 	unsigned int iscsi_llimit;	     /* chip's iscsi region llimit */
 	void **iscsi_ppm;		     /* iscsi page pod manager */
 	int nodeid;			     /* device numa node id */
+	bool fr_nsmr_tpte_wr_support;	     /* FW supports FR_NSMR_TPTE_WR */
 };
 
 struct cxgb4_uld_info {
drivers/net/ethernet/chelsio/cxgb4/t4fw_api.h +1 -0
index a89b307..9164d20 100644
@@ -1119,6 +1119,7 @@ enum fw_params_param_dev {
 	FW_PARAMS_PARAM_DEV_MAXIRD_ADAPTER = 0x14, /* max supported adap IRD */
 	FW_PARAMS_PARAM_DEV_ULPTX_MEMWRITE_DSGL = 0x17,
 	FW_PARAMS_PARAM_DEV_FWCACHE = 0x18,
+	FW_PARAMS_PARAM_DEV_RI_FR_NSMR_TPTE_WR	= 0x1C,
 };
 
 /*
Signature · 1 line
2.7.0

A message that is not in the archive (sent off-list or not yet received): cover.1474037695.git.swise-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org

#3 lore raw

PATCH 2/2 iw_cxgb4: add fast-path for small REG_MR operations

When processing a REG_MR work request, if fw supports the FW_RI_NSMR_TPTE_WR work request, and if the page list for this registration is <= 2 pages, and the current state of the mr is INVALID, then use FW_RI_NSMR_TPTE_WR to pass down a fully populated TPTE for FW to write. This avoids FW having to do an async read of the TPTE blocking the SQ until the read completes.

To know if the current MR state is INVALID or not, iw_cxgb4 must track the state of each fastreg MR. The c4iw_mr struct state is updated as REG_MR and LOCAL_INV WRs are posted and completed, when a reg_mr is destroyed, and when RECV completions are processed that include a local invalidation.

This optimization increases small IO IOPS for both iSER and NVMF.

Signed-off-by Steve Wise <[email protected]>
---
 drivers/infiniband/hw/cxgb4/cq.c              | 17 +++++++
 drivers/infiniband/hw/cxgb4/mem.c             |  2 +-
 drivers/infiniband/hw/cxgb4/qp.c              | 67 +++++++++++++++++++++++----
 drivers/infiniband/hw/cxgb4/t4.h              |  4 +-
 drivers/infiniband/hw/cxgb4/t4fw_ri_api.h     | 12 +++++
 drivers/net/ethernet/chelsio/cxgb4/t4fw_api.h |  1 +
 6 files changed, 92 insertions(+), 11 deletions(-)
drivers/infiniband/hw/cxgb4/cq.c +17 -0
index ac926c9..867b8cf 100644
@@ -666,6 +666,18 @@ skip_cqe:
 	return ret;
 }
 
+static void invalidate_mr(struct c4iw_dev *rhp, u32 rkey)
+{
+	struct c4iw_mr *mhp;
+	unsigned long flags;
+
+	spin_lock_irqsave(&rhp->lock, flags);
+	mhp = get_mhp(rhp, rkey >> 8);
+	if (mhp)
+		mhp->attr.state = 0;
+	spin_unlock_irqrestore(&rhp->lock, flags);
+}
+
 /*
  * Get one cq entry from c4iw and map it to openib.
  *

@@ -721,6 +733,7 @@ static int c4iw_poll_cq_one(struct c4iw_cq *chp, struct ib_wc *wc)
 		    CQE_OPCODE(&cqe) == FW_RI_SEND_WITH_SE_INV) {
 			wc->ex.invalidate_rkey = CQE_WRID_STAG(&cqe);
 			wc->wc_flags |= IB_WC_WITH_INVALIDATE;
+			invalidate_mr(qhp->rhp, wc->ex.invalidate_rkey);
 		}
 	} else {
 		switch (CQE_OPCODE(&cqe)) {

@@ -746,6 +759,10 @@ static int c4iw_poll_cq_one(struct c4iw_cq *chp, struct ib_wc *wc)
 			break;
 		case FW_RI_FAST_REGISTER:
 			wc->opcode = IB_WC_REG_MR;
+
+			/* Invalidate the MR if the fastreg failed */
+			if (CQE_STATUS(&cqe) != T4_ERR_SUCCESS)
+				invalidate_mr(qhp->rhp, CQE_WRID_FR_STAG(&cqe));
 			break;
 		default:
 			printk(KERN_ERR MOD "Unexpected opcode %d "
drivers/infiniband/hw/cxgb4/mem.c +1 -1
index 0b91b0f..80e2774 100644
@@ -695,7 +695,7 @@ struct ib_mr *c4iw_alloc_mr(struct ib_pd *pd,
 	mhp->attr.pdid = php->pdid;
 	mhp->attr.type = FW_RI_STAG_NSMR;
 	mhp->attr.stag = stag;
-	mhp->attr.state = 1;
+	mhp->attr.state = 0;
 	mmid = (stag) >> 8;
 	mhp->ibmr.rkey = mhp->ibmr.lkey = stag;
 	if (insert_handle(rhp, &rhp->mmidr, mhp, mmid)) {
drivers/infiniband/hw/cxgb4/qp.c +58 -9

95 lines not on this page · raw message

drivers/infiniband/hw/cxgb4/t4.h +3 -1
index 02173f4..862381a 100644
@@ -95,6 +95,7 @@ union t4_wr {
 	struct fw_ri_rdma_read_wr read;
 	struct fw_ri_bind_mw_wr bind;
 	struct fw_ri_fr_nsmr_wr fr;
+	struct fw_ri_fr_nsmr_tpte_wr fr_tpte;
 	struct fw_ri_inv_lstag_wr inv;
 	struct t4_status_page status;
 	__be64 flits[T4_EQ_ENTRY_SIZE / sizeof(__be64) * T4_SQ_NUM_SLOTS];

@@ -170,7 +171,7 @@ struct t4_cqe {
 			__be32 msn;
 		} rcqe;
 		struct {
-			u32 nada1;
+			u32 stag;
 			u16 nada2;
 			u16 cidx;
 		} scqe;

@@ -232,6 +233,7 @@ struct t4_cqe {
 
 /* used for SQ completion processing */
 #define CQE_WRID_SQ_IDX(x)	((x)->u.scqe.cidx)
+#define CQE_WRID_FR_STAG(x)     (be32_to_cpu((x)->u.scqe.stag))
 
 /* generic accessor macros */
 #define CQE_WRID_HI(x)		(be32_to_cpu((x)->u.gen.wrid_hi))
drivers/infiniband/hw/cxgb4/t4fw_ri_api.h +12 -0
index 1e26669..010c709 100644
@@ -669,6 +669,18 @@ struct fw_ri_fr_nsmr_wr {
 #define FW_RI_FR_NSMR_WR_DCACPU_G(x)	\
 	(((x) >> FW_RI_FR_NSMR_WR_DCACPU_S) & FW_RI_FR_NSMR_WR_DCACPU_M)
 
+struct fw_ri_fr_nsmr_tpte_wr {
+	__u8	opcode;
+	__u8   flags;
+	__u16  wrid;
+	__u8   r1[3];
+	__u8   len16;
+	__u32  r2;
+	__u32  stag;
+	struct fw_ri_tpte tpte;
+	__u64  pbl[2];
+};
+
 struct fw_ri_inv_lstag_wr {
 	__u8   opcode;
 	__u8   flags;
drivers/net/ethernet/chelsio/cxgb4/t4fw_api.h +1 -0
index 9164d20..3f46ca8 100644
@@ -100,6 +100,7 @@ enum fw_wr_opcodes {
 	FW_RI_RECV_WR                  = 0x17,
 	FW_RI_BIND_MW_WR               = 0x18,
 	FW_RI_FR_NSMR_WR               = 0x19,
+	FW_RI_FR_NSMR_TPTE_WR	       = 0x20,
 	FW_RI_INV_LSTAG_WR             = 0x1a,
 	FW_ISCSI_TX_DATA_WR	       = 0x45,
 	FW_LASTC2E_WR                  = 0x70
Signature · 6 lines
2.7.0

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#4 lore raw

Re: [PATCH 2/2] iw_cxgb4: add fast-path for small REG_MR operations

Quoted 16 lines · Steve Wise

When processing a REG_MR work request, if fw supports the FW_RI_NSMR_TPTE_WR work request, and if the page list for this registration is <= 2 pages, and the current state of the mr is INVALID, then use FW_RI_NSMR_TPTE_WR to pass down a fully populated TPTE for FW to write. This avoids FW having to do an async read of the TPTE blocking the SQ until the read completes.

To know if the current MR state is INVALID or not, iw_cxgb4 must track the state of each fastreg MR. The c4iw_mr struct state is updated as REG_MR and LOCAL_INV WRs are posted and completed, when a reg_mr is destroyed, and when RECV completions are processed that include a local invalidation.

This optimization increases small IO IOPS for both iSER and NVMF.

Signed-off-by Steve Wise <[email protected]>

---

<...>

drivers/infiniband/hw/cxgb4/qp.c static int build_rdma_recv(struct c4iw_qp *qhp, union t4_recv_wr *wqe, · line 618

Quoted 6 lines
drivers/infiniband/hw/cxgb4/qp.c +6 -0
+			      struct ib_reg_wr *wr, struct c4iw_mr *mhp,
+			      u8 *len16)
+{
+	__be64 *p = (__be64 *)fr->pbl;
+
+	fr->r2 = cpu_to_be32(0);

Is there any difference between the line above and "fr->r2 = 0"?

A message that is not in the archive (sent off-list or not yet received): 20160918142242.GJ2923-2ukJVAZIZ/[email protected]

#6 lore raw

RE: [PATCH 2/2] iw_cxgb4: add fast-path for small REG_MR operations

Quoted 7 lines · Steve Wise
Quoted 4 lines

When processing a REG_MR work request, if fw supports the FW_RI_NSMR_TPTE_WR work request, and if the page list for this registration is <= 2 pages, and the current state of the mr is INVALID, then use FW_RI_NSMR_TPTE_WR to pass down a fully populated TPTE for

FW

Quoted 1 line

to write. This avoids FW having to do an async read of the TPTE

blocking

Quoted 3 lines
Quoted 3 lines

the SQ until the read completes.

To know if the current MR state is INVALID or not, iw_cxgb4 must track

the

Quoted 4 lines
Quoted 2 lines

state of each fastreg MR. The c4iw_mr struct state is updated as REG_MR and LOCAL_INV WRs are posted and completed, when a reg_mr is

destroyed,

Quoted 1 line

and when RECV completions are processed that include a local

invalidation.

Quoted 8 lines
Quoted 5 lines

This optimization increases small IO IOPS for both iSER and NVMF.

Signed-off-by Steve Wise <[email protected]>

---

<...>

Quoted 8 lines

drivers/infiniband/hw/cxgb4/qp.c static int build_rdma_recv(struct c4iw_qp *qhp, union t4_recv_wr *wqe, · line 618

Quoted 6 lines
drivers/infiniband/hw/cxgb4/qp.c +6 -0
+			      struct ib_reg_wr *wr, struct c4iw_mr *mhp,
+			      u8 *len16)
+{
+	__be64 *p = (__be64 *)fr->pbl;
+
+	fr->r2 = cpu_to_be32(0);

Is there any difference between the line above and "fr->r2 = 0"?

It makes sparse happy, IIRC...

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#7 lore raw

Re: [PATCH 2/2] iw_cxgb4: add fast-path for small REG_MR operations

Quoted 34 lines · Steve Wise
Quoted 7 lines · Steve Wise
Quoted 4 lines

When processing a REG_MR work request, if fw supports the FW_RI_NSMR_TPTE_WR work request, and if the page list for this registration is <= 2 pages, and the current state of the mr is INVALID, then use FW_RI_NSMR_TPTE_WR to pass down a fully populated TPTE for

FW

Quoted 1 line

to write. This avoids FW having to do an async read of the TPTE

blocking

Quoted 3 lines
Quoted 3 lines

the SQ until the read completes.

To know if the current MR state is INVALID or not, iw_cxgb4 must track

the

Quoted 4 lines
Quoted 2 lines

state of each fastreg MR. The c4iw_mr struct state is updated as REG_MR and LOCAL_INV WRs are posted and completed, when a reg_mr is

destroyed,

Quoted 1 line

and when RECV completions are processed that include a local

invalidation.

Quoted 8 lines
Quoted 5 lines

This optimization increases small IO IOPS for both iSER and NVMF.

Signed-off-by Steve Wise <[email protected]>

---

<...>

Quoted 8 lines

drivers/infiniband/hw/cxgb4/qp.c static int build_rdma_recv(struct c4iw_qp *qhp, union t4_recv_wr *wqe, · line 618

Quoted 6 lines
drivers/infiniband/hw/cxgb4/qp.c +6 -0
+			      struct ib_reg_wr *wr, struct c4iw_mr *mhp,
+			      u8 *len16)
+{
+	__be64 *p = (__be64 *)fr->pbl;
+
+	fr->r2 = cpu_to_be32(0);

Is there any difference between the line above and "fr->r2 = 0"?

Quoted 1 line

It makes sparse happy, IIRC...

Strange, but ok :)

Quoted 3 lines
#8 lore raw
In reply to a message not in the archive

From: Steve Wise <[email protected]>
Date: Fri, 16 Sep 2016 07:54:55 -0700

Quoted 3 lines

This series enables a new work request to optimize small REG_MR operations. This is intended for 4.9. If everyone agrees, I suggest Doug take both the cxgb4 and iw_cxgb4 patches through his tree.

I'm assuming this mean that I do _not_ apply these to my tree. --
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): 20160919.011633.1357649774235825501.davem-fT/PcQaiUtIeIZ0/[email protected]

#10 lore raw
Quoted 10 lines

From: Steve Wise <[email protected]>
Date: Fri, 16 Sep 2016 07:54:55 -0700

Quoted 3 lines

This series enables a new work request to optimize small REG_MR operations. This is intended for 4.9. If everyone agrees, I suggest Doug take both the cxgb4 and iw_cxgb4 patches through his tree.

I'm assuming this mean that I do _not_ apply these to my tree.

Yes, if you're ok with that.

Steve.

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#11 lore raw

From: "Steve Wise" <[email protected]>
Date: Mon, 19 Sep 2016 09:17:01 -0500

Quoted 5 lines
Quoted 5 lines

From: Steve Wise <[email protected]>
Date: Fri, 16 Sep 2016 07:54:55 -0700

Quoted 7 lines
Quoted 5 lines
Quoted 3 lines

This series enables a new work request to optimize small REG_MR operations. This is intended for 4.9. If everyone agrees, I suggest Doug take both the cxgb4 and iw_cxgb4 patches through his tree.

I'm assuming this mean that I do _not_ apply these to my tree.

Yes, if you're ok with that.

I am.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): 20160919.102920.1126387243122900012.davem-fT/PcQaiUtIeIZ0/[email protected]

#13 lore raw
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines

From: Steve Wise <[email protected]>
Date: Fri, 16 Sep 2016 07:54:55 -0700

Quoted 3 lines

This series enables a new work request to optimize small REG_MR operations. This is intended for 4.9. If everyone agrees, I suggest Doug take both the cxgb4 and iw_cxgb4 patches through his tree.

Quoted 5 lines
Quoted 3 lines
Quoted 1 line

I'm assuming this mean that I do _not_ apply these to my tree.

Yes, if you're ok with that.

I am.

Doug, please include this for 4.9 if it looks good to you. Let me know.

Thanks,

Steve.

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

Applied
#14 lore raw
Quoted 14 lines · Steve Wise
Quoted 14 lines
Quoted 13 lines
Quoted 11 lines

From: Steve Wise <[email protected]>
Date: Fri, 16 Sep 2016 07:54:55 -0700

Quoted 3 lines

This series enables a new work request to optimize small REG_MR operations. This is intended for 4.9. If everyone agrees, I suggest Doug take both the cxgb4 and iw_cxgb4 patches through his tree.

I'm assuming this mean that I do _not_ apply these to my tree.

Yes, if you're ok with that.

Quoted 8 lines
Quoted 1 line

I am.

Doug, please include this for 4.9 if it looks good to you. Let me know.

Thanks,

Steve.

Applied, thanks.

Signature · 2 lines
Doug Ledford <[email protected]>
    GPG Key ID: 0E572FDD