Skip to contentSkip to conversation

IB/isert: allocate RW ctxs according to max IO size

PATCH Applied Max Gurtovoy 3 participants 3 messages Jul 8, 2020 – Jul 16, 2020

Subsystems (2)
InfiniBand iSCSI extensions for RDMA (iSER) target
lore
Share & tools
lore

Max Gurtovoy

author
#0 lore raw

PATCH 1/1 IB/isert: allocate RW ctxs according to max IO size

Current iSER target code allocates MR pool budget based on queue size. Since there is no handshake between iSER initiator and target on max IO size, we'll set the iSER target to support upto 16MiB IO operations and allocate the correct number of RDMA ctxs according to the factor of MR's per IO operation. This would guaranty sufficient size of the MR pool for the required IO queue depth and IO size.

Reported-by Krishnamraju Eraparaju <[email protected]>
Tested-by Krishnamraju Eraparaju <[email protected]>
Signed-off-by Max Gurtovoy <[email protected]>
---
 drivers/infiniband/ulp/isert/ib_isert.c | 6 ++++--
 drivers/infiniband/ulp/isert/ib_isert.h | 3 ++-
 2 files changed, 6 insertions(+), 3 deletions(-)
drivers/infiniband/ulp/isert/ib_isert.c +4 -2
index b7df38e..49f5f05 100644
@@ -119,7 +119,7 @@
 {
 	struct isert_device *device = isert_conn->device;
 	struct ib_qp_init_attr attr;
-	int ret;
+	int ret, factor;
 
 	memset(&attr, 0, sizeof(struct ib_qp_init_attr));
 	attr.event_handler = isert_qp_event_callback;

@@ -128,7 +128,9 @@
 	attr.recv_cq = comp->cq;
 	attr.cap.max_send_wr = ISERT_QP_MAX_REQ_DTOS + 1;
 	attr.cap.max_recv_wr = ISERT_QP_MAX_RECV_DTOS + 1;
-	attr.cap.max_rdma_ctxs = ISCSI_DEF_XMIT_CMDS_MAX;
+	factor = rdma_rw_mr_factor(device->ib_device, cma_id->port_num,
+				   ISCSI_ISER_MAX_SG_TABLESIZE);
+	attr.cap.max_rdma_ctxs = ISCSI_DEF_XMIT_CMDS_MAX * factor;
 	attr.cap.max_send_sge = device->ib_device->attrs.max_send_sge;
 	attr.cap.max_recv_sge = 1;
 	attr.sq_sig_type = IB_SIGNAL_REQ_WR;
drivers/infiniband/ulp/isert/ib_isert.h +2 -1
index 3b296ba..c9ccf1d 100644
@@ -63,7 +63,8 @@
 		(ISER_RX_PAYLOAD_SIZE + sizeof(u64) + sizeof(struct ib_sge) + \
 		 sizeof(struct ib_cqe) + sizeof(bool)))
 
-#define ISCSI_ISER_SG_TABLESIZE		256
+/* Maximum support is 16MB I/O size */
+#define ISCSI_ISER_MAX_SG_TABLESIZE	4096
 
 enum isert_desc_type {
 	ISCSI_TX_CONTROL,
Signature · 1 line
1.8.3.1
maintainer Applied
#2 lore raw
In reply to Max Gurtovoy
Quoted 7 lines · Max Gurtovoy

Current iSER target code allocates MR pool budget based on queue size. Since there is no handshake between iSER initiator and target on max IO size, we'll set the iSER target to support upto 16MiB IO operations and allocate the correct number of RDMA ctxs according to the factor of MR's per IO operation. This would guaranty sufficient size of the MR pool for the required IO queue depth and IO size.

Quoted 8 lines
Reported-by Krishnamraju Eraparaju <[email protected]>
Tested-by Krishnamraju Eraparaju <[email protected]>
Signed-off-by Max Gurtovoy <[email protected]>
Acked-by Sagi Grimberg <[email protected]>
---
 drivers/infiniband/ulp/isert/ib_isert.c | 6 ++++--
 drivers/infiniband/ulp/isert/ib_isert.h | 3 ++-
 2 files changed, 6 insertions(+), 3 deletions(-)

Applied to for-next, thanks

Jason