Skip to contentSkip to conversation

mlx5 Tx DevX/Verbs separation

PATCH In review Michael Baum 2 participants 17 messages Oct 1, 2020 – Oct 6, 2020 Network drivers

lore
Share & tools
lore
#0 lore raw

cover mlx5 Tx DevX/Verbs separation

The series is an arrangement to multi-OS support by net/mlx5 driver so it comes to ease the code management for OS which supports\doesn't support DevX\Verbs operations.

Michael Baum (15):
net/mlx5: fix send queue doorbell typo net/mlx5: fix unused variable in Txq creation net/mlx5: mitigate Tx queue reference counters net/mlx5: reorder Tx queue DevX object creation net/mlx5: reorder Tx queue Verbs object creation net/mlx5: reposition the event queue number field net/mlx5: separate Tx queue object creations net/mlx5: share Tx control code
net/mlx5: rearrange SQ and CQ creation in DevX module net/mlx5: rearrange QP creation in Verbs module net/mlx5: separate Tx queue object modification net/mlx5: share Tx queue object modification net/mlx5: remove Tx queue object type field net/mlx5: separate Rx queue state modification net/mlx5: remove Rx queue object type field

 drivers/net/mlx5/linux/mlx5_os.c    |  80 ++++
 drivers/net/mlx5/linux/mlx5_verbs.c | 296 ++++++++++++-
 drivers/net/mlx5/linux/mlx5_verbs.h |   3 +
 drivers/net/mlx5/mlx5.c             |  10 +
 drivers/net/mlx5/mlx5.h             |  61 ++-
 drivers/net/mlx5/mlx5_devx.c        | 593 +++++++++++++++++++++++--
 drivers/net/mlx5/mlx5_devx.h        |   3 +
 drivers/net/mlx5/mlx5_rxq.c         |   4 +-
 drivers/net/mlx5/mlx5_rxtx.c        | 105 +----
 drivers/net/mlx5/mlx5_rxtx.h        |  45 +-
 drivers/net/mlx5/mlx5_trigger.c     |  40 +-
 drivers/net/mlx5/mlx5_txpp.c        |  28 +-
 drivers/net/mlx5/mlx5_txq.c         | 850 ++----------------------------------
 drivers/net/mlx5/mlx5_vlan.c        |   5 +-
 14 files changed, 1087 insertions(+), 1036 deletions(-)
Signature · 1 line
1.8.3.1
#1 lore raw

PATCH 1/15 net/mlx5: fix send queue doorbell typo

As part of SQ creation for Tx queue objects, a HW doorbell memory should be allocated and mapped to the HW.

The SQ doorbell handler was wrongly saved on the CQ fields what caused wrong doorbell release in the Tx queue object destroy flow.

Save the SQ doorbell handler in the SQ fields.

Fixes 3a87b964edd3 ("net/mlx5: create Tx queues with DevX")
---
 drivers/net/mlx5/mlx5_txq.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
drivers/net/mlx5/mlx5_txq.c +4 -4
index 1bb667d..fc730fa 100644
@@ -1050,8 +1050,8 @@
 			dev->data->port_id, txq_data->idx);
 		goto error;
 	}
-	/* Allocate doorbell record for completion queue. */
-	txq_obj->cq_dbrec_offset = mlx5_get_dbr(sh->ctx,
+	/* Allocate doorbell record for send queue. */
+	txq_obj->sq_dbrec_offset = mlx5_get_dbr(sh->ctx,
 						&priv->dbrpgs,
 						&txq_obj->sq_dbrec_page);
 	if (txq_obj->sq_dbrec_offset < 0)
@@ -1076,9 +1076,9 @@
 	sq_attr.wq_attr.log_wq_stride = rte_log2_u32(MLX5_WQE_SIZE);
 	sq_attr.wq_attr.log_wq_sz = txq_data->wqe_n;
 	sq_attr.wq_attr.dbr_umem_valid = 1;
-	sq_attr.wq_attr.dbr_addr = txq_obj->cq_dbrec_offset;
+	sq_attr.wq_attr.dbr_addr = txq_obj->sq_dbrec_offset;
 	sq_attr.wq_attr.dbr_umem_id =
-			mlx5_os_get_umem_id(txq_obj->cq_dbrec_page->umem);
+			mlx5_os_get_umem_id(txq_obj->sq_dbrec_page->umem);
 	sq_attr.wq_attr.wq_umem_valid = 1;
 	sq_attr.wq_attr.wq_umem_id = mlx5_os_get_umem_id(txq_obj->sq_umem);
 	sq_attr.wq_attr.wq_umem_offset = (uintptr_t)txq_obj->sq_buf % page_size;
Signature · 1 line
1.8.3.1
#2 lore raw

PATCH 2/15 net/mlx5: fix unused variable in Txq creation

In reply to Michael Baum

When a CQ is not created by DevX, it be allocated by either DV function or by regular Verbs function.

The CQ DV attributes variable was wrongly defined and initialized in Tx queue creation while the CQ is created by the regular Verbs function what remained the attributes variable unused.

Remove the unused variable.

Fixes faf2667fe8d5 ("net/mlx5: separate DPDK from verbs Tx queue objects")
---
 drivers/net/mlx5/mlx5_txq.c | 4 ----
 1 file changed, 4 deletions(-)
drivers/net/mlx5/mlx5_txq.c +0 -4
index fc730fa..ef3137b 100644
@@ -1168,7 +1168,6 @@ struct mlx5_txq_obj *
 	struct mlx5_txq_obj *txq_obj = NULL;
 	union {
 		struct ibv_qp_init_attr_ex init;
-		struct ibv_cq_init_attr_ex cq;
 		struct ibv_qp_attr mod;
 	} attr;
 	unsigned int cqe_n;
@@ -1198,9 +1197,6 @@ struct mlx5_txq_obj *
 		return NULL;
 	}
 	memset(&tmpl, 0, sizeof(struct mlx5_txq_obj));
-	attr.cq = (struct ibv_cq_init_attr_ex){
-		.comp_mask = 0,
-	};
 	cqe_n = desc / MLX5_TX_COMP_THRESH +
 		1 + MLX5_TX_COMP_THRESH_INLINE_DIV;
 	tmpl.cq = mlx5_glue->create_cq(priv->sh->ctx, cqe_n, NULL, NULL, 0);
Signature · 1 line
1.8.3.1
#3 lore raw

PATCH 3/15 net/mlx5: mitigate Tx queue reference counters

In reply to Michael Baum

The Tx queue structures manage 2 different reference counter per queue:
txq_ctrl reference counter and txq_obj reference counter.

There is no real need to use two different counters, it just complicates the release functions.
Remove the txq_obj counter and use only the txq_ctrl counter.

---
 drivers/net/mlx5/mlx5_rxtx.h |  4 +-
 drivers/net/mlx5/mlx5_txq.c  | 98 ++++++++++++++------------------------------
 2 files changed, 32 insertions(+), 70 deletions(-)
drivers/net/mlx5/mlx5_rxtx.h +1 -3
index 9ffa028..d947e0e 100644
@@ -276,7 +276,6 @@ enum mlx5_txq_type {
 /* Verbs/DevX Tx queue elements. */
 struct mlx5_txq_obj {
 	LIST_ENTRY(mlx5_txq_obj) next; /* Pointer to the next element. */
-	rte_atomic32_t refcnt; /* Reference counter. */
 	struct mlx5_txq_ctrl *txq_ctrl; /* Pointer to the control queue. */
 	enum mlx5_txq_obj_type type; /* The txq object type. */
 	RTE_STD_C11
@@ -405,8 +404,7 @@ int mlx5_tx_queue_setup(struct rte_eth_dev *dev, uint16_t idx, uint16_t desc,
 void mlx5_tx_uar_uninit_secondary(struct rte_eth_dev *dev);
 struct mlx5_txq_obj *mlx5_txq_obj_new(struct rte_eth_dev *dev, uint16_t idx,
 				      enum mlx5_txq_obj_type type);
-struct mlx5_txq_obj *mlx5_txq_obj_get(struct rte_eth_dev *dev, uint16_t idx);
-int mlx5_txq_obj_release(struct mlx5_txq_obj *txq_ibv);
+void mlx5_txq_obj_release(struct mlx5_txq_obj *txq_obj);
 int mlx5_txq_obj_verify(struct rte_eth_dev *dev);
 struct mlx5_txq_ctrl *mlx5_txq_new(struct rte_eth_dev *dev, uint16_t idx,
 				   uint16_t desc, unsigned int socket,
drivers/net/mlx5/mlx5_txq.c +31 -67

152 lines not on this page · raw message

Signature · 1 line
1.8.3.1
#4 lore raw

PATCH 4/15 net/mlx5: reorder Tx queue DevX object creation

In reply to Michael Baum

Move the creation of the send queue and the completion queue resources from the mlx5_txq_obj_devx_new function into auxiliary functions.

---
 drivers/net/mlx5/mlx5_txq.c | 361 +++++++++++++++++++++++++++++---------------
 1 file changed, 239 insertions(+), 122 deletions(-)
drivers/net/mlx5/mlx5_txq.c +239 -122
index e8bf7d7..686b452 100644
@@ -839,140 +839,133 @@
 }
 
 /**
- * Destroy the Tx queue DevX object.
+ * Release DevX SQ resources.
  *
- * @param txq_obj
- *   Txq object to destroy
+ * @param txq_ctrl
+ *   DevX Tx queue object.
  */
 static void
-txq_release_sq_resources(struct mlx5_txq_obj *txq_obj)
+txq_release_devx_sq_resources(struct mlx5_txq_obj *txq_obj)
 {
-	MLX5_ASSERT(txq_obj->type == MLX5_TXQ_OBJ_TYPE_DEVX_SQ);
-
 	if (txq_obj->sq_devx)
 		claim_zero(mlx5_devx_cmd_destroy(txq_obj->sq_devx));
-	if (txq_obj->sq_dbrec_page)
-		claim_zero(mlx5_release_dbr
-				(&txq_obj->txq_ctrl->priv->dbrpgs,
-				mlx5_os_get_umem_id
-					(txq_obj->sq_dbrec_page->umem),
-				txq_obj->sq_dbrec_offset));
 	if (txq_obj->sq_umem)
 		claim_zero(mlx5_glue->devx_umem_dereg(txq_obj->sq_umem));
 	if (txq_obj->sq_buf)
 		mlx5_free(txq_obj->sq_buf);
+	if (txq_obj->sq_dbrec_page)
+		claim_zero(mlx5_release_dbr(&txq_obj->txq_ctrl->priv->dbrpgs,
+					    mlx5_os_get_umem_id
+						 (txq_obj->sq_dbrec_page->umem),
+					    txq_obj->sq_dbrec_offset));
+}
+
+/**
+ * Release DevX Tx CQ resources.
+ *
+ * @param txq_ctrl
+ *   DevX Tx queue object.
+ */
+static void
+txq_release_devx_cq_resources(struct mlx5_txq_obj *txq_obj)
+{
 	if (txq_obj->cq_devx)
 		claim_zero(mlx5_devx_cmd_destroy(txq_obj->cq_devx));
-	if (txq_obj->cq_dbrec_page)
-		claim_zero(mlx5_release_dbr
-				(&txq_obj->txq_ctrl->priv->dbrpgs,
-				mlx5_os_get_umem_id
-					(txq_obj->cq_dbrec_page->umem),
-				txq_obj->cq_dbrec_offset));
 	if (txq_obj->cq_umem)
 		claim_zero(mlx5_glue->devx_umem_dereg(txq_obj->cq_umem));
 	if (txq_obj->cq_buf)
 		mlx5_free(txq_obj->cq_buf);
+	if (txq_obj->cq_dbrec_page)
+		claim_zero(mlx5_release_dbr(&txq_obj->txq_ctrl->priv->dbrpgs,
+					    mlx5_os_get_umem_id
+						 (txq_obj->cq_dbrec_page->umem),
+					    txq_obj->cq_dbrec_offset));
 }
 
 /**
- * Create the Tx queue DevX object.
+ * Destroy the Tx queue DevX object.
+ *
+ * @param txq_obj
+ *   Txq object to destroy
+ */
+static void
+txq_release_devx_resources(struct mlx5_txq_obj *txq_obj)
+{
+	MLX5_ASSERT(txq_obj->type == MLX5_TXQ_OBJ_TYPE_DEVX_SQ);
+
+	txq_release_devx_sq_resources(txq_obj);
+	txq_release_devx_cq_resources(txq_obj);
+}
+
+#ifdef HAVE_MLX5DV_DEVX_UAR_OFFSET
+/**
+ * Create a DevX CQ object for a Tx queue.
  *
  * @param dev
  *   Pointer to Ethernet device.
+ * @param cqe_n
+ *   Number of entries in the CQ.
  * @param idx
- *   Queue index in DPDK Tx queue array
+ *   Queue index in DPDK Tx queue array.
+ * @param type
+ *   Type of the Tx queue object to create.
  *
  * @return
- *   The DevX object initialised, NULL otherwise and rte_errno is set.
+ *   The DevX CQ object initialized, NULL otherwise and rte_errno is set.
  */
-static struct mlx5_txq_obj *
-mlx5_txq_obj_devx_new(struct rte_eth_dev *dev, uint16_t idx)
+static struct mlx5_devx_obj *
+mlx5_devx_cq_new(struct rte_eth_dev *dev, uint32_t cqe_n, uint16_t idx,
+		 struct mlx5_txq_obj *txq_obj)
 {
-#ifndef HAVE_MLX5DV_DEVX_UAR_OFFSET
-	DRV_LOG(ERR, "port %u Tx queue %u cannot create with DevX, no UAR",
-		     dev->data->port_id, idx);
-	rte_errno = ENOMEM;
-	return NULL;
-#else
 	struct mlx5_priv *priv = dev->data->dev_private;
-	struct mlx5_dev_ctx_shared *sh = priv->sh;
 	struct mlx5_txq_data *txq_data = (*priv->txqs)[idx];
-	struct mlx5_txq_ctrl *txq_ctrl =
-		container_of(txq_data, struct mlx5_txq_ctrl, txq);
-	struct mlx5_devx_create_sq_attr sq_attr = { 0 };
-	struct mlx5_devx_modify_sq_attr msq_attr = { 0 };
+	struct mlx5_devx_obj *cq_obj = NULL;
 	struct mlx5_devx_cq_attr cq_attr = { 0 };
-	struct mlx5_txq_obj *txq_obj = NULL;
-	size_t page_size;
 	struct mlx5_cqe *cqe;
-	uint32_t i, nqe;
-	void *reg_addr;
-	size_t alignment = (size_t)-1;
-	int ret = 0;
+	size_t page_size;
+	size_t alignment;
+	uint32_t i;
+	int ret;
 
 	MLX5_ASSERT(txq_data);
-	MLX5_ASSERT(!txq_ctrl->obj);
+	MLX5_ASSERT(txq_obj);
 	page_size = rte_mem_page_size();
 	if (page_size == (size_t)-1) {
 		DRV_LOG(ERR, "Failed to get mem page size");
 		rte_errno = ENOMEM;
 		return NULL;
 	}
-	txq_obj = mlx5_malloc(MLX5_MEM_RTE | MLX5_MEM_ZERO,
-			      sizeof(struct mlx5_txq_obj), 0,
-			      txq_ctrl->socket);
-	if (!txq_obj) {
-		DRV_LOG(ERR,
-			"port %u Tx queue %u cannot allocate memory resources",
-			dev->data->port_id, txq_data->idx);
+	/* Allocate memory buffer for CQEs. */
+	alignment = MLX5_CQE_BUF_ALIGNMENT;
+	if (alignment == (size_t)-1) {
+		DRV_LOG(ERR, "Failed to get CQE buf alignment");
 		rte_errno = ENOMEM;
 		return NULL;
 	}
-	txq_obj->type = MLX5_TXQ_OBJ_TYPE_DEVX_SQ;
-	txq_obj->txq_ctrl = txq_ctrl;
-	txq_obj->dev = dev;
-	/* Create the Completion Queue. */
-	nqe = (1UL << txq_data->elts_n) / MLX5_TX_COMP_THRESH +
-	       1 + MLX5_TX_COMP_THRESH_INLINE_DIV;
-	nqe = 1UL << log2above(nqe);
-	if (nqe > UINT16_MAX) {
+	cqe_n = 1UL << log2above(cqe_n);
+	if (cqe_n > UINT16_MAX) {
 		DRV_LOG(ERR,
 			"port %u Tx queue %u requests to many CQEs %u",
-			dev->data->port_id, txq_data->idx, nqe);
+			dev->data->port_id, txq_data->idx, cqe_n);
 		rte_errno = EINVAL;
-		goto error;
-	}
-	/* Allocate memory buffer for CQEs. */
-	alignment = MLX5_CQE_BUF_ALIGNMENT;
-	if (alignment == (size_t)-1) {
-		DRV_LOG(ERR, "Failed to get mem page size");
-		rte_errno = ENOMEM;
-		goto error;
+		return NULL;
 	}
 	txq_obj->cq_buf = mlx5_malloc(MLX5_MEM_RTE | MLX5_MEM_ZERO,
-				      nqe * sizeof(struct mlx5_cqe),
+				      cqe_n * sizeof(struct mlx5_cqe),
 				      alignment,
-				      sh->numa_node);
+				      priv->sh->numa_node);
 	if (!txq_obj->cq_buf) {
 		DRV_LOG(ERR,
 			"port %u Tx queue %u cannot allocate memory (CQ)",
 			dev->data->port_id, txq_data->idx);
 		rte_errno = ENOMEM;
-		goto error;
+		return NULL;
 	}
-	txq_data->cqe_n = log2above(nqe);
-	txq_data->cqe_s = 1 << txq_data->cqe_n;
-	txq_data->cqe_m = txq_data->cqe_s - 1;
-	txq_data->cqes = (volatile struct mlx5_cqe *)txq_obj->cq_buf;
-	txq_data->cq_ci = 0;
-	txq_data->cq_pi = 0;
 	/* Register allocated buffer in user space with DevX. */
-	txq_obj->cq_umem = mlx5_glue->devx_umem_reg
-					(sh->ctx,
-					 (void *)txq_obj->cq_buf,
-					 nqe * sizeof(struct mlx5_cqe),
-					 IBV_ACCESS_LOCAL_WRITE);
+	txq_obj->cq_umem = mlx5_glue->devx_umem_reg(priv->sh->ctx,
+						(void *)txq_obj->cq_buf,
+						cqe_n * sizeof(struct mlx5_cqe),
+						IBV_ACCESS_LOCAL_WRITE);
 	if (!txq_obj->cq_umem) {
 		rte_errno = errno;
 		DRV_LOG(ERR,
@@ -981,46 +974,88 @@
 		goto error;
 	}
 	/* Allocate doorbell record for completion queue. */
-	txq_obj->cq_dbrec_offset = mlx5_get_dbr(sh->ctx,
+	txq_obj->cq_dbrec_offset = mlx5_get_dbr(priv->sh->ctx,
 						&priv->dbrpgs,
 						&txq_obj->cq_dbrec_page);
-	if (txq_obj->cq_dbrec_offset < 0)
+	if (txq_obj->cq_dbrec_offset < 0) {
+		rte_errno = errno;
+		DRV_LOG(ERR, "Failed to allocate CQ door-bell.");
 		goto error;
-	txq_data->cq_db = (volatile uint32_t *)(txq_obj->cq_dbrec_page->dbrs +
-						txq_obj->cq_dbrec_offset);
-	*txq_data->cq_db = 0;
-	/* Create completion queue object with DevX. */
+	}
 	cq_attr.cqe_size = (sizeof(struct mlx5_cqe) == 128) ?
 			    MLX5_CQE_SIZE_128B : MLX5_CQE_SIZE_64B;
-	cq_attr.uar_page_id = mlx5_os_get_devx_uar_page_id(sh->tx_uar);
-	cq_attr.eqn = sh->txpp.eqn;
+	cq_attr.uar_page_id = mlx5_os_get_devx_uar_page_id(priv->sh->tx_uar);
+	cq_attr.eqn = priv->sh->txpp.eqn;
 	cq_attr.q_umem_valid = 1;
 	cq_attr.q_umem_offset = (uintptr_t)txq_obj->cq_buf % page_size;
 	cq_attr.q_umem_id = mlx5_os_get_umem_id(txq_obj->cq_umem);
 	cq_attr.db_umem_valid = 1;
 	cq_attr.db_umem_offset = txq_obj->cq_dbrec_offset;
 	cq_attr.db_umem_id = mlx5_os_get_umem_id(txq_obj->cq_dbrec_page->umem);
-	cq_attr.log_cq_size = rte_log2_u32(nqe);
+	cq_attr.log_cq_size = rte_log2_u32(cqe_n);
 	cq_attr.log_page_size = rte_log2_u32(page_size);
-	txq_obj->cq_devx = mlx5_devx_cmd_create_cq(sh->ctx, &cq_attr);
-	if (!txq_obj->cq_devx) {
+	/* Create completion queue object with DevX. */
+	cq_obj = mlx5_devx_cmd_create_cq(priv->sh->ctx, &cq_attr);
+	if (!cq_obj) {
 		rte_errno = errno;
 		DRV_LOG(ERR, "port %u Tx queue %u CQ creation failure",
 			dev->data->port_id, idx);
 		goto error;
 	}
+	txq_data->cqe_n = log2above(cqe_n);
+	txq_data->cqe_s = 1 << txq_data->cqe_n;
 	/* Initial fill CQ buffer with invalid CQE opcode. */
 	cqe = (struct mlx5_cqe *)txq_obj->cq_buf;
 	for (i = 0; i < txq_data->cqe_s; i++) {
 		cqe->op_own = (MLX5_CQE_INVALID << 4) | MLX5_CQE_OWNER_MASK;
 		++cqe;
 	}
-	/* Create the Work Queue. */
-	nqe = RTE_MIN(1UL << txq_data->elts_n,
-		      (uint32_t)sh->device_attr.max_qp_wr);
+	return cq_obj;
+error:
+	ret = rte_errno;
+	txq_release_devx_cq_resources(txq_obj);
+	rte_errno = ret;
+	return NULL;
+}
+
+/**
+ * Create a SQ object using DevX.
+ *
+ * @param dev
+ *   Pointer to Ethernet device.
+ * @param idx
+ *   Queue index in DPDK Tx queue array.
+ * @param type
+ *   Type of the Tx queue object to create.
+ *
+ * @return
+ *   The DevX object initialized, NULL otherwise and rte_errno is set.
+ */
+static struct mlx5_devx_obj *
+mlx5_devx_sq_new(struct rte_eth_dev *dev, uint16_t idx,
+		 struct mlx5_txq_obj *txq_obj)
+{
+	struct mlx5_priv *priv = dev->data->dev_private;
+	struct mlx5_txq_data *txq_data = (*priv->txqs)[idx];
+	struct mlx5_devx_create_sq_attr sq_attr = { 0 };
+	struct mlx5_devx_obj *sq_obj = NULL;
+	size_t page_size;
+	uint32_t wqe_n;
+	int ret;
+
+	MLX5_ASSERT(txq_data);
+	MLX5_ASSERT(txq_obj);
+	page_size = rte_mem_page_size();
+	if (page_size == (size_t)-1) {
+		DRV_LOG(ERR, "Failed to get mem page size");
+		rte_errno = ENOMEM;
+		return NULL;
+	}
+	wqe_n = RTE_MIN(1UL << txq_data->elts_n,
+			(uint32_t)priv->sh->device_attr.max_qp_wr);
 	txq_obj->sq_buf = mlx5_malloc(MLX5_MEM_RTE | MLX5_MEM_ZERO,
-				      nqe * sizeof(struct mlx5_wqe),
-				      page_size, sh->numa_node);
+				      wqe_n * sizeof(struct mlx5_wqe),
+				      page_size, priv->sh->numa_node);
 	if (!txq_obj->sq_buf) {
 		DRV_LOG(ERR,
 			"port %u Tx queue %u cannot allocate memory (SQ)",
@@ -1028,20 +1063,11 @@
 		rte_errno = ENOMEM;
 		goto error;
 	}
-	txq_data->wqe_n = log2above(nqe);
-	txq_data->wqe_s = 1 << txq_data->wqe_n;
-	txq_data->wqe_m = txq_data->wqe_s - 1;
-	txq_data->wqes = (struct mlx5_wqe *)txq_obj->sq_buf;
-	txq_data->wqes_end = txq_data->wqes + txq_data->wqe_s;
-	txq_data->wqe_ci = 0;
-	txq_data->wqe_pi = 0;
-	txq_data->wqe_comp = 0;
-	txq_data->wqe_thres = txq_data->wqe_s / MLX5_TX_COMP_THRESH_INLINE_DIV;
 	/* Register allocated buffer in user space with DevX. */
 	txq_obj->sq_umem = mlx5_glue->devx_umem_reg
-					(sh->ctx,
+					(priv->sh->ctx,
 					 (void *)txq_obj->sq_buf,
-					 nqe * sizeof(struct mlx5_wqe),
+					 wqe_n * sizeof(struct mlx5_wqe),
 					 IBV_ACCESS_LOCAL_WRITE);
 	if (!txq_obj->sq_umem) {
 		rte_errno = errno;
@@ -1051,30 +1077,28 @@
 		goto error;
 	}
 	/* Allocate doorbell record for send queue. */
-	txq_obj->sq_dbrec_offset = mlx5_get_dbr(sh->ctx,
+	txq_obj->sq_dbrec_offset = mlx5_get_dbr(priv->sh->ctx,
 						&priv->dbrpgs,
 						&txq_obj->sq_dbrec_page);
-	if (txq_obj->sq_dbrec_offset < 0)
+	if (txq_obj->sq_dbrec_offset < 0) {
+		rte_errno = errno;
+		DRV_LOG(ERR, "Failed to allocate SQ door-bell.");
 		goto error;
-	txq_data->qp_db = (volatile uint32_t *)
-					(txq_obj->sq_dbrec_page->dbrs +
-					 txq_obj->sq_dbrec_offset +
-					 MLX5_SND_DBR * sizeof(uint32_t));
-	*txq_data->qp_db = 0;
-	/* Create Send Queue object with DevX. */
+	}
 	sq_attr.tis_lst_sz = 1;
-	sq_attr.tis_num = sh->tis->id;
+	sq_attr.tis_num = priv->sh->tis->id;
 	sq_attr.state = MLX5_SQC_STATE_RST;
 	sq_attr.cqn = txq_obj->cq_devx->id;
 	sq_attr.flush_in_error_en = 1;
 	sq_attr.allow_multi_pkt_send_wqe = !!priv->config.mps;
 	sq_attr.allow_swp = !!priv->config.swp;
 	sq_attr.min_wqe_inline_mode = priv->config.hca_attr.vport_inline_mode;
-	sq_attr.wq_attr.uar_page = mlx5_os_get_devx_uar_page_id(sh->tx_uar);
+	sq_attr.wq_attr.uar_page =
+				mlx5_os_get_devx_uar_page_id(priv->sh->tx_uar);
 	sq_attr.wq_attr.wq_type = MLX5_WQ_TYPE_CYCLIC;
-	sq_attr.wq_attr.pd = sh->pdn;
+	sq_attr.wq_attr.pd = priv->sh->pdn;
 	sq_attr.wq_attr.log_wq_stride = rte_log2_u32(MLX5_WQE_SIZE);
-	sq_attr.wq_attr.log_wq_sz = txq_data->wqe_n;
+	sq_attr.wq_attr.log_wq_sz = log2above(wqe_n);
 	sq_attr.wq_attr.dbr_umem_valid = 1;
 	sq_attr.wq_attr.dbr_addr = txq_obj->sq_dbrec_offset;
 	sq_attr.wq_attr.dbr_umem_id =
@@ -1082,13 +1106,106 @@
 	sq_attr.wq_attr.wq_umem_valid = 1;
 	sq_attr.wq_attr.wq_umem_id = mlx5_os_get_umem_id(txq_obj->sq_umem);
 	sq_attr.wq_attr.wq_umem_offset = (uintptr_t)txq_obj->sq_buf % page_size;
-	txq_obj->sq_devx = mlx5_devx_cmd_create_sq(sh->ctx, &sq_attr);
-	if (!txq_obj->sq_devx) {
+	/* Create Send Queue object with DevX. */
+	sq_obj = mlx5_devx_cmd_create_sq(priv->sh->ctx, &sq_attr);
+	if (!sq_obj) {
 		rte_errno = errno;
 		DRV_LOG(ERR, "port %u Tx queue %u SQ creation failure",
 			dev->data->port_id, idx);
 		goto error;
 	}
+	txq_data->wqe_n = log2above(wqe_n);
+	return sq_obj;
+error:
+	ret = rte_errno;
+	txq_release_devx_sq_resources(txq_obj);
+	rte_errno = ret;
+	return NULL;
+}
+#endif
+
+/**
+ * Create the Tx queue DevX object.
+ *
+ * @param dev
+ *   Pointer to Ethernet device.
+ * @param idx
+ *   Queue index in DPDK Tx queue array.
+ *
+ * @return
+ *   The DevX object initialised, NULL otherwise and rte_errno is set.
+ */
+static struct mlx5_txq_obj *
+mlx5_txq_obj_devx_new(struct rte_eth_dev *dev, uint16_t idx)
+{
+#ifndef HAVE_MLX5DV_DEVX_UAR_OFFSET
+	DRV_LOG(ERR, "port %u Tx queue %u cannot create with DevX, no UAR",
+		     dev->data->port_id, idx);
+	rte_errno = ENOMEM;
+	return NULL;
+#else
+	struct mlx5_priv *priv = dev->data->dev_private;
+	struct mlx5_dev_ctx_shared *sh = priv->sh;
+	struct mlx5_txq_data *txq_data = (*priv->txqs)[idx];
+	struct mlx5_txq_ctrl *txq_ctrl =
+		container_of(txq_data, struct mlx5_txq_ctrl, txq);
+	struct mlx5_devx_modify_sq_attr msq_attr = { 0 };
+	struct mlx5_txq_obj *txq_obj = NULL;
+	void *reg_addr;
+	uint32_t cqe_n;
+	int ret = 0;
+
+	MLX5_ASSERT(txq_data);
+	MLX5_ASSERT(!txq_ctrl->obj);
+	txq_obj = mlx5_malloc(MLX5_MEM_RTE | MLX5_MEM_ZERO,
+			      sizeof(struct mlx5_txq_obj), 0,
+			      txq_ctrl->socket);
+	if (!txq_obj) {
+		DRV_LOG(ERR,
+			"port %u Tx queue %u cannot allocate memory resources",
+			dev->data->port_id, txq_data->idx);
+		rte_errno = ENOMEM;
+		return NULL;
+	}
+	txq_obj->type = MLX5_TXQ_OBJ_TYPE_DEVX_SQ;
+	txq_obj->txq_ctrl = txq_ctrl;
+	txq_obj->dev = dev;
+	/* Create the Completion Queue. */
+	cqe_n = (1UL << txq_data->elts_n) / MLX5_TX_COMP_THRESH +
+		1 + MLX5_TX_COMP_THRESH_INLINE_DIV;
+	/* Create completion queue object with DevX. */
+	txq_obj->cq_devx = mlx5_devx_cq_new(dev, cqe_n, idx, txq_obj);
+	if (!txq_obj->cq_devx) {
+		rte_errno = errno;
+		goto error;
+	}
+	txq_data->cqe_m = txq_data->cqe_s - 1;
+	txq_data->cqes = (volatile struct mlx5_cqe *)txq_obj->cq_buf;
+	txq_data->cq_ci = 0;
+	txq_data->cq_pi = 0;
+	txq_data->cq_db = (volatile uint32_t *)(txq_obj->cq_dbrec_page->dbrs +
+						txq_obj->cq_dbrec_offset);
+	*txq_data->cq_db = 0;
+	/* Create Send Queue object with DevX. */
+	txq_obj->sq_devx = mlx5_devx_sq_new(dev, idx, txq_obj);
+	if (!txq_obj->sq_devx) {
+		rte_errno = errno;
+		goto error;
+	}
+	/* Create the Work Queue. */
+	txq_data->wqe_s = 1 << txq_data->wqe_n;
+	txq_data->wqe_m = txq_data->wqe_s - 1;
+	txq_data->wqes = (struct mlx5_wqe *)txq_obj->sq_buf;
+	txq_data->wqes_end = txq_data->wqes + txq_data->wqe_s;
+	txq_data->wqe_ci = 0;
+	txq_data->wqe_pi = 0;
+	txq_data->wqe_comp = 0;
+	txq_data->wqe_thres = txq_data->wqe_s / MLX5_TX_COMP_THRESH_INLINE_DIV;
+	txq_data->qp_db = (volatile uint32_t *)
+					(txq_obj->sq_dbrec_page->dbrs +
+					 txq_obj->sq_dbrec_offset +
+					 MLX5_SND_DBR * sizeof(uint32_t));
+	*txq_data->qp_db = 0;
 	txq_data->qp_num_8s = txq_obj->sq_devx->id << 8;
 	/* Change Send Queue state to Ready-to-Send. */
 	msq_attr.sq_state = MLX5_SQC_STATE_RST;
@@ -1131,7 +1248,7 @@
 	return txq_obj;
 error:
 	ret = rte_errno; /* Save rte_errno before cleanup. */
-	txq_release_sq_resources(txq_obj);
+	txq_release_devx_resources(txq_obj);
 	if (txq_data->fcqs) {
 		mlx5_free(txq_data->fcqs);
 		txq_data->fcqs = NULL;
@@ -1408,7 +1525,7 @@ struct mlx5_txq_obj *
 		if (txq_obj->tis)
 			claim_zero(mlx5_devx_cmd_destroy(txq_obj->tis));
 	} else if (txq_obj->type == MLX5_TXQ_OBJ_TYPE_DEVX_SQ) {
-		txq_release_sq_resources(txq_obj);
+		txq_release_devx_resources(txq_obj);
 	} else {
 		claim_zero(mlx5_glue->destroy_qp(txq_obj->qp));
 		claim_zero(mlx5_glue->destroy_cq(txq_obj->cq));
Signature · 1 line
1.8.3.1
#5 lore raw

PATCH 5/15 net/mlx5: reorder Tx queue Verbs object creation

In reply to Michael Baum

Move the creation of the completion queue from the mlx5_txq_obj_new function into an auxiliary function.

---
 drivers/net/mlx5/mlx5_txq.c | 117 +++++++++++++++++++++++++-------------------
 1 file changed, 68 insertions(+), 49 deletions(-)
drivers/net/mlx5/mlx5_txq.c +68 -49
index 686b452..f2ecfc4 100644
@@ -1260,6 +1260,66 @@
 }
 
 /**
+ * Create a QP Verbs object.
+ *
+ * @param dev
+ *   Pointer to Ethernet device.
+ * @param idx
+ *   Queue index in DPDK Tx queue array.
+ * @param rxq_obj
+ *   Pointer to Tx queue object data.
+ *
+ * @return
+ *   The QP Verbs object initialized, NULL otherwise and rte_errno is set.
+ */
+static struct ibv_qp *
+mlx5_ibv_qp_new(struct rte_eth_dev *dev, uint16_t idx,
+		struct mlx5_txq_obj *txq_obj)
+{
+	struct mlx5_priv *priv = dev->data->dev_private;
+	struct mlx5_txq_data *txq_data = (*priv->txqs)[idx];
+	struct mlx5_txq_ctrl *txq_ctrl =
+			container_of(txq_data, struct mlx5_txq_ctrl, txq);
+	struct ibv_qp *qp_obj = NULL;
+	struct ibv_qp_init_attr_ex qp_attr = { 0 };
+	const int desc = 1 << txq_data->elts_n;
+
+	MLX5_ASSERT(!txq_ctrl->obj);
+	/* CQ to be associated with the send queue. */
+	qp_attr.send_cq = txq_obj->cq;
+	/* CQ to be associated with the receive queue. */
+	qp_attr.recv_cq = txq_obj->cq;
+	/* Max number of outstanding WRs. */
+	qp_attr.cap.max_send_wr = ((priv->sh->device_attr.max_qp_wr < desc) ?
+				   priv->sh->device_attr.max_qp_wr : desc);
+	/*
+	 * Max number of scatter/gather elements in a WR, must be 1 to prevent
+	 * libmlx5 from trying to affect must be 1 to prevent libmlx5 from
+	 * trying to affect too much memory. TX gather is not impacted by the
+	 * device_attr.max_sge limit and will still work properly.
+	 */
+	qp_attr.cap.max_send_sge = 1;
+	qp_attr.qp_type = IBV_QPT_RAW_PACKET,
+	/* Do *NOT* enable this, completions events are managed per Tx burst. */
+	qp_attr.sq_sig_all = 0;
+	qp_attr.pd = priv->sh->pd;
+	qp_attr.comp_mask = IBV_QP_INIT_ATTR_PD;
+	if (txq_data->inlen_send)
+		qp_attr.cap.max_inline_data = txq_ctrl->max_inline_data;
+	if (txq_data->tso_en) {
+		qp_attr.max_tso_header = txq_ctrl->max_tso_header;
+		qp_attr.comp_mask |= IBV_QP_INIT_ATTR_MAX_TSO_HEADER;
+	}
+	qp_obj = mlx5_glue->create_qp_ex(priv->sh->ctx, &qp_attr);
+	if (qp_obj == NULL) {
+		DRV_LOG(ERR, "port %u Tx queue %u QP creation failure",
+			dev->data->port_id, idx);
+		rte_errno = errno;
+	}
+	return qp_obj;
+}
+
+/**
  * Create the Tx queue Verbs object.
  *
  * @param dev
@@ -1282,10 +1342,7 @@ struct mlx5_txq_obj *
 		container_of(txq_data, struct mlx5_txq_ctrl, txq);
 	struct mlx5_txq_obj tmpl;
 	struct mlx5_txq_obj *txq_obj = NULL;
-	union {
-		struct ibv_qp_init_attr_ex init;
-		struct ibv_qp_attr mod;
-	} attr;
+	struct ibv_qp_attr mod;
 	unsigned int cqe_n;
 	struct mlx5dv_qp qp = { .comp_mask = MLX5DV_QP_MASK_UAR_MMAP_OFFSET };
 	struct mlx5dv_cq cq_info;
@@ -1322,56 +1379,18 @@ struct mlx5_txq_obj *
 		rte_errno = errno;
 		goto error;
 	}
-	attr.init = (struct ibv_qp_init_attr_ex){
-		/* CQ to be associated with the send queue. */
-		.send_cq = tmpl.cq,
-		/* CQ to be associated with the receive queue. */
-		.recv_cq = tmpl.cq,
-		.cap = {
-			/* Max number of outstanding WRs. */
-			.max_send_wr =
-				((priv->sh->device_attr.max_qp_wr <
-				  desc) ?
-				 priv->sh->device_attr.max_qp_wr :
-				 desc),
-			/*
-			 * Max number of scatter/gather elements in a WR,
-			 * must be 1 to prevent libmlx5 from trying to affect
-			 * too much memory. TX gather is not impacted by the
-			 * device_attr.max_sge limit and will still work
-			 * properly.
-			 */
-			.max_send_sge = 1,
-		},
-		.qp_type = IBV_QPT_RAW_PACKET,
-		/*
-		 * Do *NOT* enable this, completions events are managed per
-		 * Tx burst.
-		 */
-		.sq_sig_all = 0,
-		.pd = priv->sh->pd,
-		.comp_mask = IBV_QP_INIT_ATTR_PD,
-	};
-	if (txq_data->inlen_send)
-		attr.init.cap.max_inline_data = txq_ctrl->max_inline_data;
-	if (txq_data->tso_en) {
-		attr.init.max_tso_header = txq_ctrl->max_tso_header;
-		attr.init.comp_mask |= IBV_QP_INIT_ATTR_MAX_TSO_HEADER;
-	}
-	tmpl.qp = mlx5_glue->create_qp_ex(priv->sh->ctx, &attr.init);
+	tmpl.qp = mlx5_ibv_qp_new(dev, idx, &tmpl);
 	if (tmpl.qp == NULL) {
-		DRV_LOG(ERR, "port %u Tx queue %u QP creation failure",
-			dev->data->port_id, idx);
 		rte_errno = errno;
 		goto error;
 	}
-	attr.mod = (struct ibv_qp_attr){
+	mod = (struct ibv_qp_attr){
 		/* Move the QP to this state. */
 		.qp_state = IBV_QPS_INIT,
 		/* IB device port number. */
 		.port_num = (uint8_t)priv->dev_port,
 	};
-	ret = mlx5_glue->modify_qp(tmpl.qp, &attr.mod,
+	ret = mlx5_glue->modify_qp(tmpl.qp, &mod,
 				   (IBV_QP_STATE | IBV_QP_PORT));
 	if (ret) {
 		DRV_LOG(ERR,
@@ -1380,10 +1399,10 @@ struct mlx5_txq_obj *
 		rte_errno = errno;
 		goto error;
 	}
-	attr.mod = (struct ibv_qp_attr){
+	mod = (struct ibv_qp_attr){
 		.qp_state = IBV_QPS_RTR
 	};
-	ret = mlx5_glue->modify_qp(tmpl.qp, &attr.mod, IBV_QP_STATE);
+	ret = mlx5_glue->modify_qp(tmpl.qp, &mod, IBV_QP_STATE);
 	if (ret) {
 		DRV_LOG(ERR,
 			"port %u Tx queue %u QP state to IBV_QPS_RTR failed",
@@ -1391,8 +1410,8 @@ struct mlx5_txq_obj *
 		rte_errno = errno;
 		goto error;
 	}
-	attr.mod.qp_state = IBV_QPS_RTS;
-	ret = mlx5_glue->modify_qp(tmpl.qp, &attr.mod, IBV_QP_STATE);
+	mod.qp_state = IBV_QPS_RTS;
+	ret = mlx5_glue->modify_qp(tmpl.qp, &mod, IBV_QP_STATE);
 	if (ret) {
 		DRV_LOG(ERR,
 			"port %u Tx queue %u QP state to IBV_QPS_RTS failed",
Signature · 1 line
1.8.3.1
#6 lore raw

PATCH 6/15 net/mlx5: reposition the event queue number field

In reply to Michael Baum

The eqn field has become a field of sh directly since it is also relevant for Tx and Rx.

---
 drivers/net/mlx5/mlx5.c      | 10 ++++++++++
 drivers/net/mlx5/mlx5.h      |  2 +-
 drivers/net/mlx5/mlx5_devx.c |  9 +--------
 drivers/net/mlx5/mlx5_txpp.c | 28 ++++++++--------------------
 drivers/net/mlx5/mlx5_txq.c  |  2 +-
 5 files changed, 21 insertions(+), 30 deletions(-)
drivers/net/mlx5/mlx5.c +10 -0
index 01ead6e..e5ca392 100644
@@ -925,6 +925,16 @@ struct mlx5_dev_ctx_shared *
 		goto error;
 	}
 	if (sh->devx) {
+		uint32_t lcore = (uint32_t)rte_lcore_to_cpu_id(-1);
+
+		/* Query the EQN for this core. */
+		err = mlx5_glue->devx_query_eqn(sh->ctx, lcore, &sh->eqn);
+		if (err) {
+			rte_errno = errno;
+			DRV_LOG(ERR, "Failed to query event queue number %d.",
+				rte_errno);
+			goto error;
+		}
 		err = mlx5_os_get_pdn(sh->pd, &sh->pdn);
 		if (err) {
 			DRV_LOG(ERR, "Fail to extract pdn from PD");
drivers/net/mlx5/mlx5.h +1 -1
index bd91e16..050d3a9 100644
@@ -561,7 +561,6 @@ struct mlx5_dev_txpp {
 	uint32_t tick; /* Completion tick duration in nanoseconds. */
 	uint32_t test; /* Packet pacing test mode. */
 	int32_t skew; /* Scheduling skew. */
-	uint32_t eqn; /* Event Queue number. */
 	struct rte_intr_handle intr_handle; /* Periodic interrupt. */
 	void *echan; /* Event Channel. */
 	struct mlx5_txpp_wq clock_queue; /* Clock Queue. */
@@ -603,6 +602,7 @@ struct mlx5_dev_ctx_shared {
 	LIST_ENTRY(mlx5_dev_ctx_shared) next;
 	uint32_t refcnt;
 	uint32_t devx:1; /* Opened with DV. */
+	uint32_t eqn; /* Event Queue number. */
 	uint32_t max_port; /* Maximal IB device port index. */
 	void *ctx; /* Verbs/DV/DevX context. */
 	void *pd; /* Protection Domain. */
drivers/net/mlx5/mlx5_devx.c +1 -8
index cb4a522..cddfe43 100644
@@ -350,11 +350,9 @@
 	struct mlx5_rxq_ctrl *rxq_ctrl =
 		container_of(rxq_data, struct mlx5_rxq_ctrl, rxq);
 	size_t page_size = rte_mem_page_size();
-	uint32_t lcore = (uint32_t)rte_lcore_to_cpu_id(-1);
 	unsigned int cqe_n = mlx5_rxq_cqe_num(rxq_data);
 	struct mlx5_devx_dbr_page *dbr_page;
 	int64_t dbr_offset;
-	uint32_t eqn = 0;
 	void *buf = NULL;
 	uint16_t event_nums[1] = {0};
 	uint32_t log_cqe_n;
@@ -392,12 +390,6 @@
 		cq_attr.cqe_size = MLX5_CQE_SIZE_128B;
 	log_cqe_n = log2above(cqe_n);
 	cq_size = sizeof(struct mlx5_cqe) * (1 << log_cqe_n);
-	/* Query the EQN for this core. */
-	if (mlx5_glue->devx_query_eqn(priv->sh->ctx, lcore, &eqn)) {
-		DRV_LOG(ERR, "Failed to query EQN for CQ.");
-		goto error;
-	}
-	cq_attr.eqn = eqn;
 	buf = rte_calloc_socket(__func__, 1, cq_size, page_size,
 				rxq_ctrl->socket);
 	if (!buf) {
@@ -425,6 +417,7 @@
 	rxq_data->cq_uar =
 			mlx5_os_get_devx_uar_base_addr(priv->sh->devx_rx_uar);
 	/* Create CQ using DevX API. */
+	cq_attr.eqn = priv->sh->eqn;
 	cq_attr.uar_page_id =
 			mlx5_os_get_devx_uar_page_id(priv->sh->devx_rx_uar);
 	cq_attr.q_umem_id = mlx5_os_get_umem_id(rxq_ctrl->cq_umem);
drivers/net/mlx5/mlx5_txpp.c +8 -20
index 011e479..37355fa 100644
@@ -31,36 +31,24 @@
 
 /* Destroy Event Queue Notification Channel. */
 static void
-mlx5_txpp_destroy_eqn(struct mlx5_dev_ctx_shared *sh)
+mlx5_txpp_destroy_event_channel(struct mlx5_dev_ctx_shared *sh)
 {
 	if (sh->txpp.echan) {
 		mlx5_glue->devx_destroy_event_channel(sh->txpp.echan);
 		sh->txpp.echan = NULL;
 	}
-	sh->txpp.eqn = 0;
 }
 
 /* Create Event Queue Notification Channel. */
 static int
-mlx5_txpp_create_eqn(struct mlx5_dev_ctx_shared *sh)
+mlx5_txpp_create_event_channel(struct mlx5_dev_ctx_shared *sh)
 {
-	uint32_t lcore;
-
 	MLX5_ASSERT(!sh->txpp.echan);
-	lcore = (uint32_t)rte_lcore_to_cpu_id(-1);
-	if (mlx5_glue->devx_query_eqn(sh->ctx, lcore, &sh->txpp.eqn)) {
-		rte_errno = errno;
-		DRV_LOG(ERR, "Failed to query EQ number %d.", rte_errno);
-		sh->txpp.eqn = 0;
-		return -rte_errno;
-	}
 	sh->txpp.echan = mlx5_glue->devx_create_event_channel(sh->ctx,
 			MLX5DV_DEVX_CREATE_EVENT_CHANNEL_FLAGS_OMIT_EV_DATA);
 	if (!sh->txpp.echan) {
-		sh->txpp.eqn = 0;
 		rte_errno = errno;
-		DRV_LOG(ERR, "Failed to create event channel %d.",
-			rte_errno);
+		DRV_LOG(ERR, "Failed to create event channel %d.", rte_errno);
 		return -rte_errno;
 	}
 	return 0;
@@ -285,7 +273,7 @@
 	cq_attr.cqe_size = (sizeof(struct mlx5_cqe) == 128) ?
 			    MLX5_CQE_SIZE_128B : MLX5_CQE_SIZE_64B;
 	cq_attr.uar_page_id = mlx5_os_get_devx_uar_page_id(sh->tx_uar);
-	cq_attr.eqn = sh->txpp.eqn;
+	cq_attr.eqn = sh->eqn;
 	cq_attr.q_umem_valid = 1;
 	cq_attr.q_umem_offset = 0;
 	cq_attr.q_umem_id = mlx5_os_get_umem_id(wq->cq_umem);
@@ -525,7 +513,7 @@
 	cq_attr.use_first_only = 1;
 	cq_attr.overrun_ignore = 1;
 	cq_attr.uar_page_id = mlx5_os_get_devx_uar_page_id(sh->tx_uar);
-	cq_attr.eqn = sh->txpp.eqn;
+	cq_attr.eqn = sh->eqn;
 	cq_attr.q_umem_valid = 1;
 	cq_attr.q_umem_offset = 0;
 	cq_attr.q_umem_id = mlx5_os_get_umem_id(wq->cq_umem);
@@ -951,7 +939,7 @@
 	sh->txpp.test = !!(tx_pp < 0);
 	sh->txpp.skew = priv->config.tx_skew;
 	sh->txpp.freq = priv->config.hca_attr.dev_freq_khz;
-	ret = mlx5_txpp_create_eqn(sh);
+	ret = mlx5_txpp_create_event_channel(sh);
 	if (ret)
 		goto exit;
 	ret = mlx5_txpp_alloc_pp_index(sh);
@@ -972,7 +960,7 @@
 		mlx5_txpp_destroy_rearm_queue(sh);
 		mlx5_txpp_destroy_clock_queue(sh);
 		mlx5_txpp_free_pp_index(sh);
-		mlx5_txpp_destroy_eqn(sh);
+		mlx5_txpp_destroy_event_channel(sh);
 		sh->txpp.tick = 0;
 		sh->txpp.test = 0;
 		sh->txpp.skew = 0;
@@ -994,7 +982,7 @@
 	mlx5_txpp_destroy_rearm_queue(sh);
 	mlx5_txpp_destroy_clock_queue(sh);
 	mlx5_txpp_free_pp_index(sh);
-	mlx5_txpp_destroy_eqn(sh);
+	mlx5_txpp_destroy_event_channel(sh);
 	sh->txpp.tick = 0;
 	sh->txpp.test = 0;
 	sh->txpp.skew = 0;
drivers/net/mlx5/mlx5_txq.c +1 -1
index f2ecfc4..c678971 100644
@@ -985,7 +985,7 @@
 	cq_attr.cqe_size = (sizeof(struct mlx5_cqe) == 128) ?
 			    MLX5_CQE_SIZE_128B : MLX5_CQE_SIZE_64B;
 	cq_attr.uar_page_id = mlx5_os_get_devx_uar_page_id(priv->sh->tx_uar);
-	cq_attr.eqn = priv->sh->txpp.eqn;
+	cq_attr.eqn = priv->sh->eqn;
 	cq_attr.q_umem_valid = 1;
 	cq_attr.q_umem_offset = (uintptr_t)txq_obj->cq_buf % page_size;
 	cq_attr.q_umem_id = mlx5_os_get_umem_id(txq_obj->cq_umem);
Signature · 1 line
1.8.3.1
#7 lore raw

PATCH 7/15 net/mlx5: separate Tx queue object creations

In reply to Michael Baum

As an arrangement to Windows OS support, the Verbs operations should be separated to another file.
By this way, the build can easily cut the unsupported Verbs APIs from the compilation process.

Define operation structure and DevX module in addition to the existing linux Verbs module.
Separate Tx object creation into the Verbs/DevX modules and update the operation structure according to the OS support and the user configuration.

---
 drivers/net/mlx5/linux/mlx5_os.c    |  73 ++++
 drivers/net/mlx5/linux/mlx5_verbs.c | 285 +++++++++++++
 drivers/net/mlx5/linux/mlx5_verbs.h |   4 +
 drivers/net/mlx5/mlx5.h             |  42 ++
 drivers/net/mlx5/mlx5_devx.c        | 531 +++++++++++++++++++++++-
 drivers/net/mlx5/mlx5_devx.h        |   4 +
 drivers/net/mlx5/mlx5_rxtx.h        |  43 +-
 drivers/net/mlx5/mlx5_trigger.c     |  11 +-
 drivers/net/mlx5/mlx5_txq.c         | 798 +-----------------------------------
 9 files changed, 942 insertions(+), 849 deletions(-)
drivers/net/mlx5/linux/mlx5_os.c +73 -0
index 188a6d4..c5332a0 100644
@@ -512,6 +512,70 @@
 }
 
 /**
+ * Create the Tx queue DevX/Verbs object.
+ *
+ * @param dev
+ *   Pointer to Ethernet device.
+ * @param idx
+ *   Queue index in DPDK Tx queue array.
+ *
+ * @return
+ *   The DevX/Verbs object initialized, NULL otherwise and rte_errno is set.
+ */
+static struct mlx5_txq_obj *
+mlx5_os_txq_obj_new(struct rte_eth_dev *dev, uint16_t idx)
+{
+	struct mlx5_priv *priv = dev->data->dev_private;
+	struct mlx5_dev_config *config = &priv->config;
+	struct mlx5_txq_data *txq_data = (*priv->txqs)[idx];
+	struct mlx5_txq_ctrl *txq_ctrl =
+			container_of(txq_data, struct mlx5_txq_ctrl, txq);
+
+	/*
+	 * When DevX is supported and DV flow is enable, and dest tir is enable,
+	 * hairpin functions use DevX API.
+	 * When, in addition, DV E-Switch is enable and DevX uar offset is
+	 * supported, all Tx functions also use DevX API.
+	 * Otherwise, all Tx functions use Verbs API.
+	 */
+	if (config->devx && config->dv_flow_en && config->dest_tir) {
+		if (txq_ctrl->type == MLX5_TXQ_TYPE_HAIRPIN)
+			return mlx5_txq_devx_obj_new(dev, idx);
+#ifdef HAVE_MLX5DV_DEVX_UAR_OFFSET
+		if (config->dv_esw_en)
+			return mlx5_txq_devx_obj_new(dev, idx);
+#endif
+	}
+	return mlx5_txq_ibv_obj_new(dev, idx);
+}
+
+/**
+ * Release an Tx DevX/verbs queue object.
+ *
+ * @param txq_obj
+ *   DevX/Verbs Tx queue object.
+ */
+static void
+mlx5_os_txq_obj_release(struct mlx5_txq_obj *txq_obj)
+{
+	struct mlx5_dev_config *config = &txq_obj->txq_ctrl->priv->config;
+
+	if (config->devx && config->dv_flow_en && config->dest_tir) {
+#ifdef HAVE_MLX5DV_DEVX_UAR_OFFSET
+		if (config->dv_esw_en) {
+			mlx5_txq_devx_obj_release(txq_obj);
+			return;
+		}
+#endif
+		if (txq_obj->txq_ctrl->type == MLX5_TXQ_TYPE_HAIRPIN) {
+			mlx5_txq_devx_obj_release(txq_obj);
+			return;
+		}
+	}
+	mlx5_txq_ibv_obj_release(txq_obj);
+}
+
+/**
  * Spawn an Ethernet device from Verbs information.
  *
  * @param dpdk_dev
@@ -1299,6 +1363,12 @@
 			goto error;
 		}
 	}
+	/*
+	 * Initialize the dev_ops structure with DevX/Verbs function pointers.
+	 * When DevX is supported and both DV flow and dest tir are enabled, all
+	 * Rx functions use DevX API (except for drop that has not yet been
+	 * implemented in DevX).
+	 */
 	if (config->devx && config->dv_flow_en && config->dest_tir) {
 		priv->obj_ops = devx_obj_ops;
 		priv->obj_ops.drop_action_create =
@@ -1308,6 +1378,9 @@
 	} else {
 		priv->obj_ops = ibv_obj_ops;
 	}
+	/* The Tx objects are managed by a specific linux wrapper functions. */
+	priv->obj_ops.txq_obj_new = mlx5_os_txq_obj_new;
+	priv->obj_ops.txq_obj_release = mlx5_os_txq_obj_release;
 	/* Supported Verbs flow priority number detection. */
 	err = mlx5_flow_discover_priorities(eth_dev);
 	if (err < 0) {
drivers/net/mlx5/linux/mlx5_verbs.c +285 -0

295 lines not on this page · raw message

drivers/net/mlx5/linux/mlx5_verbs.h +4 -0
index 2e69c0f..7f6bb99 100644
@@ -12,6 +12,10 @@ struct mlx5_verbs_ops {
 	mlx5_dereg_mr_t dereg_mr;
 };
 
+struct mlx5_txq_obj *mlx5_txq_ibv_obj_new(struct rte_eth_dev *dev,
+					  uint16_t idx);
+void mlx5_txq_ibv_obj_release(struct mlx5_txq_obj *txq_obj);
+
 /* Verbs ops struct */
 extern const struct mlx5_verbs_ops mlx5_verbs_ops;
 extern struct mlx5_obj_ops ibv_obj_ops;
drivers/net/mlx5/mlx5.h +42 -0

54 lines not on this page · raw message

drivers/net/mlx5/mlx5_devx.c +529 -2

550 lines not on this page · raw message

drivers/net/mlx5/mlx5_devx.h +4 -0
index 844985c..0bbbbc0 100644
@@ -7,6 +7,10 @@
 
 #include "mlx5.h"
 
+struct mlx5_txq_obj *mlx5_txq_devx_obj_new(struct rte_eth_dev *dev,
+					   uint16_t idx);
+void mlx5_txq_devx_obj_release(struct mlx5_txq_obj *txq_obj);
+
 extern struct mlx5_obj_ops devx_obj_ops;
 
 #endif /* RTE_PMD_MLX5_DEVX_H_ */
drivers/net/mlx5/mlx5_rxtx.h +1 -42

62 lines not on this page · raw message

drivers/net/mlx5/mlx5_trigger.c +2 -9
index 0f4d031..6763042 100644
@@ -55,16 +55,9 @@
 
 		if (!txq_ctrl)
 			continue;
-		if (txq_ctrl->type == MLX5_TXQ_TYPE_HAIRPIN) {
-			txq_ctrl->obj = mlx5_txq_obj_new
-				(dev, i, MLX5_TXQ_OBJ_TYPE_DEVX_HAIRPIN);
-		} else {
+		if (txq_ctrl->type == MLX5_TXQ_TYPE_STANDARD)
 			txq_alloc_elts(txq_ctrl);
-			txq_ctrl->obj = mlx5_txq_obj_new
-				(dev, i, priv->txpp_en ?
-				MLX5_TXQ_OBJ_TYPE_DEVX_SQ :
-				MLX5_TXQ_OBJ_TYPE_IBV);
-		}
+		txq_ctrl->obj = priv->obj_ops.txq_obj_new(dev, i);
 		if (!txq_ctrl->obj) {
 			rte_errno = ENOMEM;
 			goto error;
drivers/net/mlx5/mlx5_txq.c +2 -796

822 lines not on this page · raw message

Signature · 1 line
1.8.3.1
#8 lore raw

PATCH 8/15 net/mlx5: share Tx control code

In reply to Michael Baum

Move Tx object similar resources allocations and debug logs from DevX and Verbs modules to a shared location.

---
 drivers/net/mlx5/linux/mlx5_os.c    |  4 +-
 drivers/net/mlx5/linux/mlx5_verbs.c | 84 ++++++++++++-------------------------
 drivers/net/mlx5/linux/mlx5_verbs.h |  3 +-
 drivers/net/mlx5/mlx5.h             |  3 +-
 drivers/net/mlx5/mlx5_devx.c        | 75 +++++++--------------------------
 drivers/net/mlx5/mlx5_devx.h        |  3 +-
 drivers/net/mlx5/mlx5_trigger.c     | 31 +++++++++++++-
 drivers/net/mlx5/mlx5_txq.c         | 28 ++++++++-----
 8 files changed, 93 insertions(+), 138 deletions(-)
drivers/net/mlx5/linux/mlx5_os.c +2 -2
index c5332a0..0db2b5a 100644
@@ -520,9 +520,9 @@
  *   Queue index in DPDK Tx queue array.
  *
  * @return
- *   The DevX/Verbs object initialized, NULL otherwise and rte_errno is set.
+ *   0 on success, a negative errno value otherwise and rte_errno is set.
  */
-static struct mlx5_txq_obj *
+static int
 mlx5_os_txq_obj_new(struct rte_eth_dev *dev, uint16_t idx)
 {
 	struct mlx5_priv *priv = dev->data->dev_private;
drivers/net/mlx5/linux/mlx5_verbs.c +26 -58

195 lines not on this page · raw message

drivers/net/mlx5/linux/mlx5_verbs.h +1 -2
index 7f6bb99..0670f6c 100644
@@ -12,8 +12,7 @@ struct mlx5_verbs_ops {
 	mlx5_dereg_mr_t dereg_mr;
 };
 
-struct mlx5_txq_obj *mlx5_txq_ibv_obj_new(struct rte_eth_dev *dev,
-					  uint16_t idx);
+int mlx5_txq_ibv_obj_new(struct rte_eth_dev *dev, uint16_t idx);
 void mlx5_txq_ibv_obj_release(struct mlx5_txq_obj *txq_obj);
 
 /* Verbs ops struct */
drivers/net/mlx5/mlx5.h +1 -2
index 8679750..3093f6e 100644
@@ -789,8 +789,7 @@ struct mlx5_obj_ops {
 	void (*hrxq_destroy)(struct mlx5_hrxq *hrxq);
 	int (*drop_action_create)(struct rte_eth_dev *dev);
 	void (*drop_action_destroy)(struct rte_eth_dev *dev);
-	struct mlx5_txq_obj *(*txq_obj_new)(struct rte_eth_dev *dev,
-					    uint16_t idx);
+	int (*txq_obj_new)(struct rte_eth_dev *dev, uint16_t idx);
 	void (*txq_obj_release)(struct mlx5_txq_obj *txq_obj);
 };
 
drivers/net/mlx5/mlx5_devx.c +14 -61

149 lines not on this page · raw message

drivers/net/mlx5/mlx5_devx.h +1 -2
index 0bbbbc0..bc8a8d6 100644
@@ -7,8 +7,7 @@
 
 #include "mlx5.h"
 
-struct mlx5_txq_obj *mlx5_txq_devx_obj_new(struct rte_eth_dev *dev,
-					   uint16_t idx);
+int mlx5_txq_devx_obj_new(struct rte_eth_dev *dev, uint16_t idx);
 void mlx5_txq_devx_obj_release(struct mlx5_txq_obj *txq_obj);
 
 extern struct mlx5_obj_ops devx_obj_ops;
drivers/net/mlx5/mlx5_trigger.c +30 -1
index 6763042..e72e5fb 100644
@@ -52,16 +52,45 @@
 
 	for (i = 0; i != priv->txqs_n; ++i) {
 		struct mlx5_txq_ctrl *txq_ctrl = mlx5_txq_get(dev, i);
+		struct mlx5_txq_data *txq_data = &txq_ctrl->txq;
+		uint32_t flags = MLX5_MEM_RTE | MLX5_MEM_ZERO;
 
 		if (!txq_ctrl)
 			continue;
 		if (txq_ctrl->type == MLX5_TXQ_TYPE_STANDARD)
 			txq_alloc_elts(txq_ctrl);
-		txq_ctrl->obj = priv->obj_ops.txq_obj_new(dev, i);
+		MLX5_ASSERT(!txq_ctrl->obj);
+		txq_ctrl->obj = mlx5_malloc(flags, sizeof(struct mlx5_txq_obj),
+					    0, txq_ctrl->socket);
 		if (!txq_ctrl->obj) {
+			DRV_LOG(ERR, "Port %u Tx queue %u cannot allocate "
+				"memory resources.", dev->data->port_id,
+				txq_data->idx);
 			rte_errno = ENOMEM;
 			goto error;
 		}
+		ret = priv->obj_ops.txq_obj_new(dev, i);
+		if (ret < 0) {
+			mlx5_free(txq_ctrl->obj);
+			txq_ctrl->obj = NULL;
+			goto error;
+		}
+		if (txq_ctrl->type == MLX5_TXQ_TYPE_STANDARD) {
+			size_t size = txq_data->cqe_s * sizeof(*txq_data->fcqs);
+			txq_data->fcqs = mlx5_malloc(flags, size,
+						     RTE_CACHE_LINE_SIZE,
+						     txq_ctrl->socket);
+			if (!txq_data->fcqs) {
+				DRV_LOG(ERR, "Port %u Tx queue %u cannot "
+					"allocate memory (FCQ).",
+					dev->data->port_id, i);
+				rte_errno = ENOMEM;
+				goto error;
+			}
+		}
+		DRV_LOG(DEBUG, "Port %u txq %u updated with %p.",
+			dev->data->port_id, i, (void *)&txq_ctrl->obj);
+		LIST_INSERT_HEAD(&priv->txqsobj, txq_ctrl->obj, next);
 	}
 	return 0;
 error:
drivers/net/mlx5/mlx5_txq.c +18 -10
index c1d36c3..23213d9 100644
@@ -1298,21 +1298,29 @@ struct mlx5_txq_ctrl *
 mlx5_txq_release(struct rte_eth_dev *dev, uint16_t idx)
 {
 	struct mlx5_priv *priv = dev->data->dev_private;
-	struct mlx5_txq_ctrl *txq;
+	struct mlx5_txq_ctrl *txq_ctrl;
 
 	if (!(*priv->txqs)[idx])
 		return 0;
-	txq = container_of((*priv->txqs)[idx], struct mlx5_txq_ctrl, txq);
-	if (!rte_atomic32_dec_and_test(&txq->refcnt))
+	txq_ctrl = container_of((*priv->txqs)[idx], struct mlx5_txq_ctrl, txq);
+	if (!rte_atomic32_dec_and_test(&txq_ctrl->refcnt))
 		return 1;
-	if (txq->obj) {
-		priv->obj_ops.txq_obj_release(txq->obj);
-		txq->obj = NULL;
+	if (txq_ctrl->obj) {
+		priv->obj_ops.txq_obj_release(txq_ctrl->obj);
+		LIST_REMOVE(txq_ctrl->obj, next);
+		mlx5_free(txq_ctrl->obj);
+		txq_ctrl->obj = NULL;
+	}
+	if (txq_ctrl->type == MLX5_TXQ_TYPE_STANDARD) {
+		if (txq_ctrl->txq.fcqs) {
+			mlx5_free(txq_ctrl->txq.fcqs);
+			txq_ctrl->txq.fcqs = NULL;
+		}
+		txq_free_elts(txq_ctrl);
+		mlx5_mr_btree_free(&txq_ctrl->txq.mr_ctrl.cache_bh);
 	}
-	txq_free_elts(txq);
-	mlx5_mr_btree_free(&txq->txq.mr_ctrl.cache_bh);
-	LIST_REMOVE(txq, next);
-	mlx5_free(txq);
+	LIST_REMOVE(txq_ctrl, next);
+	mlx5_free(txq_ctrl);
 	(*priv->txqs)[idx] = NULL;
 	dev->data->tx_queue_state[idx] = RTE_ETH_QUEUE_STATE_STOPPED;
 	return 0;
Signature · 1 line
1.8.3.1
#9 lore raw

PATCH 9/15 net/mlx5: rearrange SQ and CQ creation in DevX module

In reply to Michael Baum

1. Rename functions to mention the internal resources.
2. Reduce the number of function arguments.

---
 drivers/net/mlx5/mlx5_devx.c | 127 +++++++++++++++++++++----------------------
 1 file changed, 62 insertions(+), 65 deletions(-)
drivers/net/mlx5/mlx5_devx.c +62 -65
index f3437a6..55fe946 100644
@@ -79,7 +79,7 @@
  *   DevX Rx queue object.
  */
 static void
-rxq_release_devx_rq_resources(struct mlx5_rxq_ctrl *rxq_ctrl)
+mlx5_rxq_release_devx_rq_resources(struct mlx5_rxq_ctrl *rxq_ctrl)
 {
 	struct mlx5_devx_dbr_page *dbr_page = rxq_ctrl->rq_dbrec_page;
 
@@ -106,7 +106,7 @@
  *   DevX Rx queue object.
  */
 static void
-rxq_release_devx_cq_resources(struct mlx5_rxq_ctrl *rxq_ctrl)
+mlx5_rxq_release_devx_cq_resources(struct mlx5_rxq_ctrl *rxq_ctrl)
 {
 	struct mlx5_devx_dbr_page *dbr_page = rxq_ctrl->cq_dbrec_page;
 
@@ -147,8 +147,8 @@
 		if (rxq_obj->devx_channel)
 			mlx5_glue->devx_destroy_event_channel
 							(rxq_obj->devx_channel);
-		rxq_release_devx_rq_resources(rxq_obj->rxq_ctrl);
-		rxq_release_devx_cq_resources(rxq_obj->rxq_ctrl);
+		mlx5_rxq_release_devx_rq_resources(rxq_obj->rxq_ctrl);
+		mlx5_rxq_release_devx_cq_resources(rxq_obj->rxq_ctrl);
 	}
 }
 
@@ -247,7 +247,7 @@
  *   The DevX RQ object initialized, NULL otherwise and rte_errno is set.
  */
 static struct mlx5_devx_obj *
-rxq_create_devx_rq_resources(struct rte_eth_dev *dev, uint16_t idx)
+mlx5_rxq_create_devx_rq_resources(struct rte_eth_dev *dev, uint16_t idx)
 {
 	struct mlx5_priv *priv = dev->data->dev_private;
 	struct mlx5_rxq_data *rxq_data = (*priv->rxqs)[idx];
@@ -325,7 +325,7 @@
 		goto error;
 	return rq;
 error:
-	rxq_release_devx_rq_resources(rxq_ctrl);
+	mlx5_rxq_release_devx_rq_resources(rxq_ctrl);
 	return NULL;
 }
 
@@ -341,7 +341,7 @@
  *   The DevX CQ object initialized, NULL otherwise and rte_errno is set.
  */
 static struct mlx5_devx_obj *
-rxq_create_devx_cq_resources(struct rte_eth_dev *dev, uint16_t idx)
+mlx5_rxq_create_devx_cq_resources(struct rte_eth_dev *dev, uint16_t idx)
 {
 	struct mlx5_devx_obj *cq_obj = 0;
 	struct mlx5_devx_cq_attr cq_attr = { 0 };
@@ -451,7 +451,7 @@
 error:
 	if (cq_obj)
 		mlx5_devx_cmd_destroy(cq_obj);
-	rxq_release_devx_cq_resources(rxq_ctrl);
+	mlx5_rxq_release_devx_cq_resources(rxq_ctrl);
 	return NULL;
 }
 
@@ -558,13 +558,13 @@
 		tmpl->fd = mlx5_os_get_devx_channel_fd(tmpl->devx_channel);
 	}
 	/* Create CQ using DevX API. */
-	tmpl->devx_cq = rxq_create_devx_cq_resources(dev, idx);
+	tmpl->devx_cq = mlx5_rxq_create_devx_cq_resources(dev, idx);
 	if (!tmpl->devx_cq) {
 		DRV_LOG(ERR, "Failed to create CQ.");
 		goto error;
 	}
 	/* Create RQ using DevX API. */
-	tmpl->rq = rxq_create_devx_rq_resources(dev, idx);
+	tmpl->rq = mlx5_rxq_create_devx_rq_resources(dev, idx);
 	if (!tmpl->rq) {
 		DRV_LOG(ERR, "Port %u Rx queue %u RQ creation failure.",
 			dev->data->port_id, idx);
@@ -589,8 +589,8 @@
 		claim_zero(mlx5_devx_cmd_destroy(tmpl->devx_cq));
 	if (tmpl->devx_channel)
 		mlx5_glue->devx_destroy_event_channel(tmpl->devx_channel);
-	rxq_release_devx_rq_resources(rxq_ctrl);
-	rxq_release_devx_cq_resources(rxq_ctrl);
+	mlx5_rxq_release_devx_rq_resources(rxq_ctrl);
+	mlx5_rxq_release_devx_cq_resources(rxq_ctrl);
 	rte_errno = ret; /* Restore rte_errno. */
 	return -rte_errno;
 }
@@ -878,7 +878,7 @@
  *   DevX Tx queue object.
  */
 static void
-txq_release_devx_sq_resources(struct mlx5_txq_obj *txq_obj)
+mlx5_txq_release_devx_sq_resources(struct mlx5_txq_obj *txq_obj)
 {
 	if (txq_obj->sq_devx)
 		claim_zero(mlx5_devx_cmd_destroy(txq_obj->sq_devx));
@@ -900,7 +900,7 @@
  *   DevX Tx queue object.
  */
 static void
-txq_release_devx_cq_resources(struct mlx5_txq_obj *txq_obj)
+mlx5_txq_release_devx_cq_resources(struct mlx5_txq_obj *txq_obj)
 {
 	if (txq_obj->cq_devx)
 		claim_zero(mlx5_devx_cmd_destroy(txq_obj->cq_devx));
@@ -922,40 +922,38 @@
  *   Txq object to destroy.
  */
 static void
-txq_release_devx_resources(struct mlx5_txq_obj *txq_obj)
+mlx5_txq_release_devx_resources(struct mlx5_txq_obj *txq_obj)
 {
 	MLX5_ASSERT(txq_obj->type == MLX5_TXQ_OBJ_TYPE_DEVX_SQ);
 
-	txq_release_devx_cq_resources(txq_obj);
-	txq_release_devx_sq_resources(txq_obj);
+	mlx5_txq_release_devx_cq_resources(txq_obj);
+	mlx5_txq_release_devx_sq_resources(txq_obj);
 }
 
 /**
- * Create a DevX CQ object for an Tx queue.
+ * Create a DevX CQ object and its resources for an Tx queue.
  *
  * @param dev
  *   Pointer to Ethernet device.
- * @param cqe_n
- *   Number of entries in the CQ.
  * @param idx
  *   Queue index in DPDK Tx queue array.
- * @param rxq_obj
- *   Pointer to Tx queue object data.
  *
  * @return
- *   The DevX CQ object initialized, NULL otherwise and rte_errno is set.
+ *   Number of CQEs in CQ, 0 otherwise and rte_errno is set.
  */
-static struct mlx5_devx_obj *
-mlx5_tx_devx_cq_new(struct rte_eth_dev *dev, uint32_t cqe_n, uint16_t idx,
-		    struct mlx5_txq_obj *txq_obj)
+static uint32_t
+mlx5_txq_create_devx_cq_resources(struct rte_eth_dev *dev, uint16_t idx)
 {
 	struct mlx5_priv *priv = dev->data->dev_private;
 	struct mlx5_txq_data *txq_data = (*priv->txqs)[idx];
-	struct mlx5_devx_obj *cq_obj = NULL;
+	struct mlx5_txq_ctrl *txq_ctrl =
+			container_of(txq_data, struct mlx5_txq_ctrl, txq);
+	struct mlx5_txq_obj *txq_obj = txq_ctrl->obj;
 	struct mlx5_devx_cq_attr cq_attr = { 0 };
 	struct mlx5_cqe *cqe;
 	size_t page_size;
 	size_t alignment;
+	uint32_t cqe_n;
 	uint32_t i;
 	int ret;
 
@@ -965,22 +963,25 @@
 	if (page_size == (size_t)-1) {
 		DRV_LOG(ERR, "Failed to get mem page size.");
 		rte_errno = ENOMEM;
-		return NULL;
+		return 0;
 	}
 	/* Allocate memory buffer for CQEs. */
 	alignment = MLX5_CQE_BUF_ALIGNMENT;
 	if (alignment == (size_t)-1) {
 		DRV_LOG(ERR, "Failed to get CQE buf alignment.");
 		rte_errno = ENOMEM;
-		return NULL;
+		return 0;
 	}
+	/* Create the Completion Queue. */
+	cqe_n = (1UL << txq_data->elts_n) / MLX5_TX_COMP_THRESH +
+		1 + MLX5_TX_COMP_THRESH_INLINE_DIV;
 	cqe_n = 1UL << log2above(cqe_n);
 	if (cqe_n > UINT16_MAX) {
 		DRV_LOG(ERR,
 			"Port %u Tx queue %u requests to many CQEs %u.",
 			dev->data->port_id, txq_data->idx, cqe_n);
 		rte_errno = EINVAL;
-		return NULL;
+		return 0;
 	}
 	txq_obj->cq_buf = mlx5_malloc(MLX5_MEM_RTE | MLX5_MEM_ZERO,
 				      cqe_n * sizeof(struct mlx5_cqe),
@@ -991,7 +992,7 @@
 			"Port %u Tx queue %u cannot allocate memory (CQ).",
 			dev->data->port_id, txq_data->idx);
 		rte_errno = ENOMEM;
-		return NULL;
+		return 0;
 	}
 	/* Register allocated buffer in user space with DevX. */
 	txq_obj->cq_umem = mlx5_glue->devx_umem_reg(priv->sh->ctx,
@@ -1027,50 +1028,47 @@
 	cq_attr.log_cq_size = rte_log2_u32(cqe_n);
 	cq_attr.log_page_size = rte_log2_u32(page_size);
 	/* Create completion queue object with DevX. */
-	cq_obj = mlx5_devx_cmd_create_cq(priv->sh->ctx, &cq_attr);
-	if (!cq_obj) {
+	txq_obj->cq_devx = mlx5_devx_cmd_create_cq(priv->sh->ctx, &cq_attr);
+	if (!txq_obj->cq_devx) {
 		rte_errno = errno;
 		DRV_LOG(ERR, "Port %u Tx queue %u CQ creation failure.",
 			dev->data->port_id, idx);
 		goto error;
 	}
-	txq_data->cqe_n = log2above(cqe_n);
-	txq_data->cqe_s = 1 << txq_data->cqe_n;
 	/* Initial fill CQ buffer with invalid CQE opcode. */
 	cqe = (struct mlx5_cqe *)txq_obj->cq_buf;
-	for (i = 0; i < txq_data->cqe_s; i++) {
+	for (i = 0; i < cqe_n; i++) {
 		cqe->op_own = (MLX5_CQE_INVALID << 4) | MLX5_CQE_OWNER_MASK;
 		++cqe;
 	}
-	return cq_obj;
+	return cqe_n;
 error:
 	ret = rte_errno;
-	txq_release_devx_cq_resources(txq_obj);
+	mlx5_txq_release_devx_cq_resources(txq_obj);
 	rte_errno = ret;
-	return NULL;
+	return 0;
 }
 
 /**
- * Create a SQ object using DevX.
+ * Create a SQ object and its resources using DevX.
  *
  * @param dev
  *   Pointer to Ethernet device.
  * @param idx
  *   Queue index in DPDK Tx queue array.
- * @param rxq_obj
- *   Pointer to Tx queue object data.
  *
  * @return
- *   The DevX SQ object initialized, NULL otherwise and rte_errno is set.
+ *   Number of WQEs in SQ, 0 otherwise and rte_errno is set.
  */
-static struct mlx5_devx_obj *
-mlx5_devx_sq_new(struct rte_eth_dev *dev, uint16_t idx,
-		 struct mlx5_txq_obj *txq_obj)
+static uint32_t
+mlx5_txq_create_devx_sq_resources(struct rte_eth_dev *dev, uint16_t idx)
 {
 	struct mlx5_priv *priv = dev->data->dev_private;
 	struct mlx5_txq_data *txq_data = (*priv->txqs)[idx];
+	struct mlx5_txq_ctrl *txq_ctrl =
+			container_of(txq_data, struct mlx5_txq_ctrl, txq);
+	struct mlx5_txq_obj *txq_obj = txq_ctrl->obj;
 	struct mlx5_devx_create_sq_attr sq_attr = { 0 };
-	struct mlx5_devx_obj *sq_obj = NULL;
 	size_t page_size;
 	uint32_t wqe_n;
 	int ret;
@@ -1081,7 +1079,7 @@
 	if (page_size == (size_t)-1) {
 		DRV_LOG(ERR, "Failed to get mem page size.");
 		rte_errno = ENOMEM;
-		return NULL;
+		return 0;
 	}
 	wqe_n = RTE_MIN(1UL << txq_data->elts_n,
 			(uint32_t)priv->sh->device_attr.max_qp_wr);
@@ -1117,7 +1115,6 @@
 		DRV_LOG(ERR, "Failed to allocate SQ door-bell.");
 		goto error;
 	}
-	txq_data->wqe_n = log2above(wqe_n);
 	sq_attr.tis_lst_sz = 1;
 	sq_attr.tis_num = priv->sh->tis->id;
 	sq_attr.state = MLX5_SQC_STATE_RST;
@@ -1131,7 +1128,7 @@
 	sq_attr.wq_attr.wq_type = MLX5_WQ_TYPE_CYCLIC;
 	sq_attr.wq_attr.pd = priv->sh->pdn;
 	sq_attr.wq_attr.log_wq_stride = rte_log2_u32(MLX5_WQE_SIZE);
-	sq_attr.wq_attr.log_wq_sz = txq_data->wqe_n;
+	sq_attr.wq_attr.log_wq_sz = log2above(wqe_n);
 	sq_attr.wq_attr.dbr_umem_valid = 1;
 	sq_attr.wq_attr.dbr_addr = txq_obj->sq_dbrec_offset;
 	sq_attr.wq_attr.dbr_umem_id =
@@ -1140,19 +1137,19 @@
 	sq_attr.wq_attr.wq_umem_id = mlx5_os_get_umem_id(txq_obj->sq_umem);
 	sq_attr.wq_attr.wq_umem_offset = (uintptr_t)txq_obj->sq_buf % page_size;
 	/* Create Send Queue object with DevX. */
-	sq_obj = mlx5_devx_cmd_create_sq(priv->sh->ctx, &sq_attr);
-	if (!sq_obj) {
+	txq_obj->sq_devx = mlx5_devx_cmd_create_sq(priv->sh->ctx, &sq_attr);
+	if (!txq_obj->sq_devx) {
 		rte_errno = errno;
 		DRV_LOG(ERR, "Port %u Tx queue %u SQ creation failure.",
 			dev->data->port_id, idx);
 		goto error;
 	}
-	return sq_obj;
+	return wqe_n;
 error:
 	ret = rte_errno;
-	txq_release_devx_sq_resources(txq_obj);
+	mlx5_txq_release_devx_sq_resources(txq_obj);
 	rte_errno = ret;
-	return NULL;
+	return 0;
 }
 #endif
 
@@ -1188,6 +1185,7 @@
 	struct mlx5_txq_obj *txq_obj = txq_ctrl->obj;
 	void *reg_addr;
 	uint32_t cqe_n;
+	uint32_t wqe_n;
 	int ret = 0;
 
 	MLX5_ASSERT(txq_data);
@@ -1195,15 +1193,13 @@
 	txq_obj->type = MLX5_TXQ_OBJ_TYPE_DEVX_SQ;
 	txq_obj->txq_ctrl = txq_ctrl;
 	txq_obj->dev = dev;
-	/* Create the Completion Queue. */
-	cqe_n = (1UL << txq_data->elts_n) / MLX5_TX_COMP_THRESH +
-		1 + MLX5_TX_COMP_THRESH_INLINE_DIV;
-	/* Create completion queue object with DevX. */
-	txq_obj->cq_devx = mlx5_tx_devx_cq_new(dev, cqe_n, idx, txq_obj);
-	if (!txq_obj->cq_devx) {
+	cqe_n = mlx5_txq_create_devx_cq_resources(dev, idx);
+	if (!cqe_n) {
 		rte_errno = errno;
 		goto error;
 	}
+	txq_data->cqe_n = log2above(cqe_n);
+	txq_data->cqe_s = 1 << txq_data->cqe_n;
 	txq_data->cqe_m = txq_data->cqe_s - 1;
 	txq_data->cqes = (volatile struct mlx5_cqe *)txq_obj->cq_buf;
 	txq_data->cq_ci = 0;
@@ -1212,12 +1208,13 @@
 						txq_obj->cq_dbrec_offset);
 	*txq_data->cq_db = 0;
 	/* Create Send Queue object with DevX. */
-	txq_obj->sq_devx = mlx5_devx_sq_new(dev, idx, txq_obj);
-	if (!txq_obj->sq_devx) {
+	wqe_n = mlx5_txq_create_devx_sq_resources(dev, idx);
+	if (!wqe_n) {
 		rte_errno = errno;
 		goto error;
 	}
 	/* Create the Work Queue. */
+	txq_data->wqe_n = log2above(wqe_n);
 	txq_data->wqe_s = 1 << txq_data->wqe_n;
 	txq_data->wqe_m = txq_data->wqe_s - 1;
 	txq_data->wqes = (struct mlx5_wqe *)txq_obj->sq_buf;
@@ -1262,7 +1259,7 @@
 	return 0;
 error:
 	ret = rte_errno; /* Save rte_errno before cleanup. */
-	txq_release_devx_resources(txq_obj);
+	mlx5_txq_release_devx_resources(txq_obj);
 	rte_errno = ret; /* Restore rte_errno. */
 	return -rte_errno;
 #endif
@@ -1283,7 +1280,7 @@
 			claim_zero(mlx5_devx_cmd_destroy(txq_obj->tis));
 #ifdef HAVE_MLX5DV_DEVX_UAR_OFFSET
 	} else {
-		txq_release_devx_resources(txq_obj);
+		mlx5_txq_release_devx_resources(txq_obj);
 #endif
 	}
 }
Signature · 1 line
1.8.3.1
#10 lore raw

PATCH 10/15 net/mlx5: rearrange QP creation in Verbs module

In reply to Michael Baum

1. Rename function to mention the internal resources.
2. Reduce the number of function arguments.

---
 drivers/net/mlx5/linux/mlx5_verbs.c | 15 ++++++---------
 1 file changed, 6 insertions(+), 9 deletions(-)
drivers/net/mlx5/linux/mlx5_verbs.c +6 -9
index 5568c75..0476d94 100644
@@ -789,15 +789,12 @@
  *   Pointer to Ethernet device.
  * @param idx
  *   Queue index in DPDK Tx queue array.
- * @param rxq_obj
- *   Pointer to Tx queue object data.
  *
  * @return
- *   The QP Verbs object initialized, NULL otherwise and rte_errno is set.
+ *   The QP Verbs object, NULL otherwise and rte_errno is set.
  */
 static struct ibv_qp *
-mlx5_ibv_qp_new(struct rte_eth_dev *dev, uint16_t idx,
-		struct mlx5_txq_obj *txq_obj)
+mlx5_txq_ibv_qp_create(struct rte_eth_dev *dev, uint16_t idx)
 {
 	struct mlx5_priv *priv = dev->data->dev_private;
 	struct mlx5_txq_data *txq_data = (*priv->txqs)[idx];
@@ -807,11 +804,11 @@
 	struct ibv_qp_init_attr_ex qp_attr = { 0 };
 	const int desc = 1 << txq_data->elts_n;
 
-	MLX5_ASSERT(txq_ctrl->obj);
+	MLX5_ASSERT(txq_ctrl->obj->cq);
 	/* CQ to be associated with the send queue. */
-	qp_attr.send_cq = txq_obj->cq;
+	qp_attr.send_cq = txq_ctrl->obj->cq;
 	/* CQ to be associated with the receive queue. */
-	qp_attr.recv_cq = txq_obj->cq;
+	qp_attr.recv_cq = txq_ctrl->obj->cq;
 	/* Max number of outstanding WRs. */
 	qp_attr.cap.max_send_wr = ((priv->sh->device_attr.max_qp_wr < desc) ?
 				   priv->sh->device_attr.max_qp_wr : desc);
@@ -890,7 +887,7 @@
 		rte_errno = errno;
 		goto error;
 	}
-	txq_obj->qp = mlx5_ibv_qp_new(dev, idx, txq_obj);
+	txq_obj->qp = mlx5_txq_ibv_qp_create(dev, idx);
 	if (txq_obj->qp == NULL) {
 		rte_errno = errno;
 		goto error;
Signature · 1 line
1.8.3.1
#11 lore raw

PATCH 11/15 net/mlx5: separate Tx queue object modification

In reply to Michael Baum

Separate Tx object modification to the Verbs and DevX modules.

---
 drivers/net/mlx5/linux/mlx5_os.c    |   7 +++
 drivers/net/mlx5/linux/mlx5_verbs.c |  65 ++++++++++++++++++++++
 drivers/net/mlx5/mlx5.h             |   9 ++++
 drivers/net/mlx5/mlx5_devx.c        |  57 ++++++++++++++++++++
 drivers/net/mlx5/mlx5_rxtx.c        |  78 ++-------------------------
 drivers/net/mlx5/mlx5_txq.c         | 104 ++++--------------------------------
 6 files changed, 152 insertions(+), 168 deletions(-)
drivers/net/mlx5/linux/mlx5_os.c +7 -0
index 0db2b5a..487714f 100644
@@ -1375,6 +1375,13 @@
 						ibv_obj_ops.drop_action_create;
 		priv->obj_ops.drop_action_destroy =
 						ibv_obj_ops.drop_action_destroy;
+#ifndef HAVE_MLX5DV_DEVX_UAR_OFFSET
+		priv->obj_ops.txq_obj_modify = ibv_obj_ops.txq_obj_modify;
+#else
+		if (!config->dv_esw_en)
+			priv->obj_ops.txq_obj_modify =
+						ibv_obj_ops.txq_obj_modify;
+#endif
 	} else {
 		priv->obj_ops = ibv_obj_ops;
 	}
drivers/net/mlx5/linux/mlx5_verbs.c +65 -0

76 lines not on this page · raw message

drivers/net/mlx5/mlx5.h +9 -0
index 3093f6e..7cbb09b 100644
@@ -774,6 +774,13 @@ struct mlx5_txq_obj {
 	};
 };
 
+enum mlx5_txq_modify_type {
+	MLX5_TXQ_MOD_RDY2RDY, /* modify state from ready to ready. */
+	MLX5_TXQ_MOD_RST2RDY, /* modify state from reset to ready. */
+	MLX5_TXQ_MOD_RDY2RST, /* modify state from ready to reset. */
+	MLX5_TXQ_MOD_ERR2RDY, /* modify state from error to ready. */
+};
+
 /* HW objects operations structure. */
 struct mlx5_obj_ops {
 	int (*rxq_obj_modify_vlan_strip)(struct mlx5_rxq_obj *rxq_obj, int on);
@@ -790,6 +797,8 @@ struct mlx5_obj_ops {
 	int (*drop_action_create)(struct rte_eth_dev *dev);
 	void (*drop_action_destroy)(struct rte_eth_dev *dev);
 	int (*txq_obj_new)(struct rte_eth_dev *dev, uint16_t idx);
+	int (*txq_obj_modify)(struct mlx5_txq_obj *obj,
+			      enum mlx5_txq_modify_type type, uint8_t dev_port);
 	void (*txq_obj_release)(struct mlx5_txq_obj *txq_obj);
 };
 
drivers/net/mlx5/mlx5_devx.c +57 -0

68 lines not on this page · raw message

drivers/net/mlx5/mlx5_rxtx.c +5 -73

84 lines not on this page · raw message

drivers/net/mlx5/mlx5_txq.c +9 -95

116 lines not on this page · raw message

Signature · 1 line
1.8.3.1
#12 lore raw

PATCH 12/15 net/mlx5: share Tx queue object modification

In reply to Michael Baum

Use new modify_qp functions for Tx object creation in DevX and Verbs modules.

---
 drivers/net/mlx5/linux/mlx5_verbs.c | 34 +++-------------------------------
 drivers/net/mlx5/mlx5_devx.c        |  7 ++-----
 drivers/net/mlx5/mlx5_txq.c         |  2 --
 3 files changed, 5 insertions(+), 38 deletions(-)
drivers/net/mlx5/linux/mlx5_verbs.c +3 -31
index 7d5ea37..ad6e3d7 100644
@@ -922,7 +922,6 @@
 	struct mlx5_txq_ctrl *txq_ctrl =
 		container_of(txq_data, struct mlx5_txq_ctrl, txq);
 	struct mlx5_txq_obj *txq_obj = txq_ctrl->obj;
-	struct ibv_qp_attr mod;
 	unsigned int cqe_n;
 	struct mlx5dv_qp qp;
 	struct mlx5dv_cq cq_info;
@@ -956,37 +955,10 @@
 		rte_errno = errno;
 		goto error;
 	}
-	mod = (struct ibv_qp_attr){
-		/* Move the QP to this state. */
-		.qp_state = IBV_QPS_INIT,
-		/* IB device port number. */
-		.port_num = (uint8_t)priv->dev_port,
-	};
-	ret = mlx5_glue->modify_qp(txq_obj->qp, &mod,
-				   (IBV_QP_STATE | IBV_QP_PORT));
-	if (ret) {
-		DRV_LOG(ERR,
-			"Port %u Tx queue %u QP state to IBV_QPS_INIT failed.",
-			dev->data->port_id, idx);
-		rte_errno = errno;
-		goto error;
-	}
-	mod = (struct ibv_qp_attr){
-		.qp_state = IBV_QPS_RTR
-	};
-	ret = mlx5_glue->modify_qp(txq_obj->qp, &mod, IBV_QP_STATE);
+	ret = mlx5_ibv_modify_qp(txq_obj, MLX5_TXQ_MOD_RST2RDY,
+				 (uint8_t)priv->dev_port);
 	if (ret) {
-		DRV_LOG(ERR,
-			"Port %u Tx queue %u QP state to IBV_QPS_RTR failed.",
-			dev->data->port_id, idx);
-		rte_errno = errno;
-		goto error;
-	}
-	mod.qp_state = IBV_QPS_RTS;
-	ret = mlx5_glue->modify_qp(txq_obj->qp, &mod, IBV_QP_STATE);
-	if (ret) {
-		DRV_LOG(ERR,
-			"Port %u Tx queue %u QP state to IBV_QPS_RTS failed.",
+		DRV_LOG(ERR, "Port %u Tx queue %u QP state modifying failed.",
 			dev->data->port_id, idx);
 		rte_errno = errno;
 		goto error;
drivers/net/mlx5/mlx5_devx.c +2 -5
index 7404a15..c876ae9 100644
@@ -1237,7 +1237,6 @@
 	return -rte_errno;
 #else
 	struct mlx5_dev_ctx_shared *sh = priv->sh;
-	struct mlx5_devx_modify_sq_attr msq_attr = { 0 };
 	struct mlx5_txq_obj *txq_obj = txq_ctrl->obj;
 	void *reg_addr;
 	uint32_t cqe_n;
@@ -1286,13 +1285,11 @@
 	*txq_data->qp_db = 0;
 	txq_data->qp_num_8s = txq_obj->sq_devx->id << 8;
 	/* Change Send Queue state to Ready-to-Send. */
-	msq_attr.sq_state = MLX5_SQC_STATE_RST;
-	msq_attr.state = MLX5_SQC_STATE_RDY;
-	ret = mlx5_devx_cmd_modify_sq(txq_obj->sq_devx, &msq_attr);
+	ret = mlx5_devx_modify_sq(txq_obj, MLX5_TXQ_MOD_RST2RDY, 0);
 	if (ret) {
 		rte_errno = errno;
 		DRV_LOG(ERR,
-			"Port %u Tx queue %u SP state to SQC_STATE_RDY failed.",
+			"Port %u Tx queue %u SQ state to SQC_STATE_RDY failed.",
 			dev->data->port_id, idx);
 		goto error;
 	}
drivers/net/mlx5/mlx5_txq.c +0 -2
index c31e446..af84f5f 100644
@@ -16,8 +16,6 @@
 #include <rte_common.h>
 #include <rte_eal_paging.h>
 
-#include <mlx5_glue.h>
-#include <mlx5_devx_cmds.h>
 #include <mlx5_common.h>
 #include <mlx5_common_mr.h>
 #include <mlx5_malloc.h>
Signature · 1 line
1.8.3.1
#13 lore raw

PATCH 13/15 net/mlx5: remove Tx queue object type field

In reply to Michael Baum

Once the separation between Verbs and DevX is done using function pointers, the type field of the Tx queue object structure becomes redundant and no more code is used.
Remove the unnecessary field from the structure.

---
 drivers/net/mlx5/linux/mlx5_verbs.c | 1 -
 drivers/net/mlx5/mlx5.h             | 8 --------
 drivers/net/mlx5/mlx5_devx.c        | 6 +-----
 3 files changed, 1 insertion(+), 14 deletions(-)
drivers/net/mlx5/linux/mlx5_verbs.c +0 -1
index ad6e3d7..6260b4e 100644
@@ -931,7 +931,6 @@
 
 	MLX5_ASSERT(txq_data);
 	MLX5_ASSERT(txq_obj);
-	txq_obj->type = MLX5_TXQ_OBJ_TYPE_IBV;
 	txq_obj->txq_ctrl = txq_ctrl;
 	priv->verbs_alloc_ctx.type = MLX5_VERBS_ALLOC_TYPE_TX_QUEUE;
 	priv->verbs_alloc_ctx.obj = txq_ctrl;
drivers/net/mlx5/mlx5.h +0 -8
index 7cbb09b..fd5dd87 100644
@@ -735,18 +735,10 @@ struct mlx5_hrxq {
 	uint8_t rss_key[]; /* Hash key. */
 };
 
-enum mlx5_txq_obj_type {
-	MLX5_TXQ_OBJ_TYPE_IBV,		/* mlx5_txq_obj with ibv_wq. */
-	MLX5_TXQ_OBJ_TYPE_DEVX_SQ,	/* mlx5_txq_obj with mlx5_devx_sq. */
-	MLX5_TXQ_OBJ_TYPE_DEVX_HAIRPIN,
-	/* mlx5_txq_obj with mlx5_devx_tq and hairpin support. */
-};
-
 /* Verbs/DevX Tx queue elements. */
 struct mlx5_txq_obj {
 	LIST_ENTRY(mlx5_txq_obj) next; /* Pointer to the next element. */
 	struct mlx5_txq_ctrl *txq_ctrl; /* Pointer to the control queue. */
-	enum mlx5_txq_obj_type type; /* The txq object type. */
 	RTE_STD_C11
 	union {
 		struct {
drivers/net/mlx5/mlx5_devx.c +1 -5
index c876ae9..430ad08 100644
@@ -890,7 +890,6 @@
 
 	MLX5_ASSERT(txq_data);
 	MLX5_ASSERT(tmpl);
-	tmpl->type = MLX5_TXQ_OBJ_TYPE_DEVX_HAIRPIN;
 	tmpl->txq_ctrl = txq_ctrl;
 	attr.hairpin = 1;
 	attr.tis_lst_sz = 1;
@@ -980,8 +979,6 @@
 static void
 mlx5_txq_release_devx_resources(struct mlx5_txq_obj *txq_obj)
 {
-	MLX5_ASSERT(txq_obj->type == MLX5_TXQ_OBJ_TYPE_DEVX_SQ);
-
 	mlx5_txq_release_devx_cq_resources(txq_obj);
 	mlx5_txq_release_devx_sq_resources(txq_obj);
 }
@@ -1245,7 +1242,6 @@
 
 	MLX5_ASSERT(txq_data);
 	MLX5_ASSERT(txq_obj);
-	txq_obj->type = MLX5_TXQ_OBJ_TYPE_DEVX_SQ;
 	txq_obj->txq_ctrl = txq_ctrl;
 	txq_obj->dev = dev;
 	cqe_n = mlx5_txq_create_devx_cq_resources(dev, idx);
@@ -1328,7 +1324,7 @@
 mlx5_txq_devx_obj_release(struct mlx5_txq_obj *txq_obj)
 {
 	MLX5_ASSERT(txq_obj);
-	if (txq_obj->type == MLX5_TXQ_OBJ_TYPE_DEVX_HAIRPIN) {
+	if (txq_obj->txq_ctrl->type == MLX5_TXQ_TYPE_HAIRPIN) {
 		if (txq_obj->tis)
 			claim_zero(mlx5_devx_cmd_destroy(txq_obj->tis));
 #ifdef HAVE_MLX5DV_DEVX_UAR_OFFSET
Signature · 1 line
1.8.3.1
#14 lore raw

PATCH 14/15 net/mlx5: separate Rx queue state modification

In reply to Michael Baum

Separate Rx state modification to the Verbs and DevX modules.

---
 drivers/net/mlx5/linux/mlx5_verbs.c |  9 +++++----
 drivers/net/mlx5/mlx5.h             |  9 ++++++++-
 drivers/net/mlx5/mlx5_devx.c        | 25 ++++++++++++++++++++-----
 drivers/net/mlx5/mlx5_rxq.c         |  4 ++--
 drivers/net/mlx5/mlx5_rxtx.c        | 27 +--------------------------
 5 files changed, 36 insertions(+), 38 deletions(-)
drivers/net/mlx5/linux/mlx5_verbs.c +5 -4
index 6260b4e..b4a6b5e 100644
@@ -4,7 +4,6 @@
 
 #include <stddef.h>
 #include <errno.h>
-#include <stdbool.h>
 #include <string.h>
 #include <stdint.h>
 #include <unistd.h>
@@ -97,16 +96,18 @@
  *
  * @param rxq_obj
  *   Verbs Rx queue object.
+ * @param type
+ *   Type of change queue state.
  *
  * @return
  *   0 on success, a negative errno value otherwise and rte_errno is set.
  */
 static int
-mlx5_ibv_modify_wq(struct mlx5_rxq_obj *rxq_obj, bool is_start)
+mlx5_ibv_modify_wq(struct mlx5_rxq_obj *rxq_obj, uint8_t type)
 {
 	struct ibv_wq_attr mod = {
 		.attr_mask = IBV_WQ_ATTR_STATE,
-		.wq_state = is_start ? IBV_WQS_RDY : IBV_WQS_RESET,
+		.wq_state = (enum ibv_wq_state)type,
 	};
 
 	return mlx5_glue->modify_wq(rxq_obj->wq, &mod);
@@ -418,7 +419,7 @@
 		goto error;
 	}
 	/* Change queue state to ready. */
-	ret = mlx5_ibv_modify_wq(tmpl, true);
+	ret = mlx5_ibv_modify_wq(tmpl, IBV_WQS_RDY);
 	if (ret) {
 		DRV_LOG(ERR,
 			"Port %u Rx queue %u WQ state to IBV_WQS_RDY failed.",
drivers/net/mlx5/mlx5.h +8 -1
index fd5dd87..f385b48 100644
@@ -766,6 +766,13 @@ struct mlx5_txq_obj {
 	};
 };
 
+enum mlx5_rxq_modify_type {
+	MLX5_RXQ_MOD_ERR2RST, /* modify state from error to reset. */
+	MLX5_RXQ_MOD_RST2RDY, /* modify state from reset to ready. */
+	MLX5_RXQ_MOD_RDY2ERR, /* modify state from ready to error. */
+	MLX5_RXQ_MOD_RDY2RST, /* modify state from ready to reset. */
+};
+
 enum mlx5_txq_modify_type {
 	MLX5_TXQ_MOD_RDY2RDY, /* modify state from ready to ready. */
 	MLX5_TXQ_MOD_RST2RDY, /* modify state from reset to ready. */
@@ -778,7 +785,7 @@ struct mlx5_obj_ops {
 	int (*rxq_obj_modify_vlan_strip)(struct mlx5_rxq_obj *rxq_obj, int on);
 	int (*rxq_obj_new)(struct rte_eth_dev *dev, uint16_t idx);
 	int (*rxq_event_get)(struct mlx5_rxq_obj *rxq_obj);
-	int (*rxq_obj_modify)(struct mlx5_rxq_obj *rxq_obj, bool is_start);
+	int (*rxq_obj_modify)(struct mlx5_rxq_obj *rxq_obj, uint8_t type);
 	void (*rxq_obj_release)(struct mlx5_rxq_obj *rxq_obj);
 	int (*ind_table_new)(struct rte_eth_dev *dev, const unsigned int log_n,
 			     struct mlx5_ind_table_obj *ind_tbl);
drivers/net/mlx5/mlx5_devx.c +20 -5
index 430ad08..a7c941c 100644
@@ -52,22 +52,37 @@
  *
  * @param rxq_obj
  *   DevX Rx queue object.
+ * @param type
+ *   Type of change queue state.
  *
  * @return
  *   0 on success, a negative errno value otherwise and rte_errno is set.
  */
 static int
-mlx5_devx_modify_rq(struct mlx5_rxq_obj *rxq_obj, bool is_start)
+mlx5_devx_modify_rq(struct mlx5_rxq_obj *rxq_obj, uint8_t type)
 {
 	struct mlx5_devx_modify_rq_attr rq_attr;
 
 	memset(&rq_attr, 0, sizeof(rq_attr));
-	if (is_start) {
+	switch (type) {
+	case MLX5_RXQ_MOD_ERR2RST:
+		rq_attr.rq_state = MLX5_RQC_STATE_ERR;
+		rq_attr.state = MLX5_RQC_STATE_RST;
+		break;
+	case MLX5_RXQ_MOD_RST2RDY:
 		rq_attr.rq_state = MLX5_RQC_STATE_RST;
 		rq_attr.state = MLX5_RQC_STATE_RDY;
-	} else {
+		break;
+	case MLX5_RXQ_MOD_RDY2ERR:
+		rq_attr.rq_state = MLX5_RQC_STATE_RDY;
+		rq_attr.state = MLX5_RQC_STATE_ERR;
+		break;
+	case MLX5_RXQ_MOD_RDY2RST:
 		rq_attr.rq_state = MLX5_RQC_STATE_RDY;
 		rq_attr.state = MLX5_RQC_STATE_RST;
+		break;
+	default:
+		break;
 	}
 	return mlx5_devx_cmd_modify_rq(rxq_obj->rq, &rq_attr);
 }
@@ -194,7 +209,7 @@
 	MLX5_ASSERT(rxq_obj);
 	MLX5_ASSERT(rxq_obj->rq);
 	if (rxq_obj->type == MLX5_RXQ_OBJ_TYPE_DEVX_HAIRPIN) {
-		mlx5_devx_modify_rq(rxq_obj, false);
+		mlx5_devx_modify_rq(rxq_obj, MLX5_RXQ_MOD_RDY2RST);
 		claim_zero(mlx5_devx_cmd_destroy(rxq_obj->rq));
 	} else {
 		MLX5_ASSERT(rxq_obj->devx_cq);
@@ -628,7 +643,7 @@
 		goto error;
 	}
 	/* Change queue state to ready. */
-	ret = mlx5_devx_modify_rq(tmpl, true);
+	ret = mlx5_devx_modify_rq(tmpl, MLX5_RXQ_MOD_RST2RDY);
 	if (ret)
 		goto error;
 	rxq_data->cq_arm_sn = 0;
drivers/net/mlx5/mlx5_rxq.c +2 -2
index c059e21..f1d8373 100644
@@ -513,7 +513,7 @@
 	int ret;
 
 	MLX5_ASSERT(rte_eal_process_type() == RTE_PROC_PRIMARY);
-	ret = priv->obj_ops.rxq_obj_modify(rxq_ctrl->obj, false);
+	ret = priv->obj_ops.rxq_obj_modify(rxq_ctrl->obj, MLX5_RXQ_MOD_RDY2RST);
 	if (ret) {
 		DRV_LOG(ERR, "Cannot change Rx WQ state to RESET:  %s",
 			strerror(errno));
@@ -612,7 +612,7 @@
 	/* Reset RQ consumer before moving queue ro READY state. */
 	*rxq->rq_db = rte_cpu_to_be_32(0);
 	rte_io_wmb();
-	ret = priv->obj_ops.rxq_obj_modify(rxq_ctrl->obj, true);
+	ret = priv->obj_ops.rxq_obj_modify(rxq_ctrl->obj, MLX5_RXQ_MOD_RST2RDY);
 	if (ret) {
 		DRV_LOG(ERR, "Cannot change Rx WQ state to READY:  %s",
 			strerror(errno));
drivers/net/mlx5/mlx5_rxtx.c +1 -26
index af4b4ba..dc22479 100644
@@ -16,8 +16,6 @@
 #include <rte_cycles.h>
 #include <rte_flow.h>
 
-#include <mlx5_glue.h>
-#include <mlx5_devx_cmds.h>
 #include <mlx5_prm.h>
 #include <mlx5_common.h>
 
@@ -901,30 +899,7 @@ enum mlx5_txcmp_code {
 		struct mlx5_rxq_ctrl *rxq_ctrl =
 			container_of(rxq, struct mlx5_rxq_ctrl, rxq);
 
-		if (rxq_ctrl->obj->type == MLX5_RXQ_OBJ_TYPE_IBV) {
-			struct ibv_wq_attr mod = {
-				.attr_mask = IBV_WQ_ATTR_STATE,
-				.wq_state = sm->state,
-			};
-
-			ret = mlx5_glue->modify_wq(rxq_ctrl->obj->wq, &mod);
-		} else { /* rxq_ctrl->obj->type == MLX5_RXQ_OBJ_TYPE_DEVX_RQ. */
-			struct mlx5_devx_modify_rq_attr rq_attr;
-
-			memset(&rq_attr, 0, sizeof(rq_attr));
-			if (sm->state == IBV_WQS_RESET) {
-				rq_attr.rq_state = MLX5_RQC_STATE_ERR;
-				rq_attr.state = MLX5_RQC_STATE_RST;
-			} else if (sm->state == IBV_WQS_RDY) {
-				rq_attr.rq_state = MLX5_RQC_STATE_RST;
-				rq_attr.state = MLX5_RQC_STATE_RDY;
-			} else if (sm->state == IBV_WQS_ERR) {
-				rq_attr.rq_state = MLX5_RQC_STATE_RDY;
-				rq_attr.state = MLX5_RQC_STATE_ERR;
-			}
-			ret = mlx5_devx_cmd_modify_rq(rxq_ctrl->obj->rq,
-						      &rq_attr);
-		}
+		ret = priv->obj_ops.rxq_obj_modify(rxq_ctrl->obj, sm->state);
 		if (ret) {
 			DRV_LOG(ERR, "Cannot change Rx WQ state to %u  - %s",
 					sm->state, strerror(errno));
Signature · 1 line
1.8.3.1
#15 lore raw

PATCH 15/15 net/mlx5: remove Rx queue object type field

In reply to Michael Baum

Once the separation between Verbs and DevX is done using function pointers, the type field of the Rx queue object structure becomes redundant and no more code is used.
Remove the unnecessary field from the structure.

---
 drivers/net/mlx5/linux/mlx5_verbs.c | 1 -
 drivers/net/mlx5/mlx5.h             | 8 --------
 drivers/net/mlx5/mlx5_devx.c        | 4 +---
 drivers/net/mlx5/mlx5_vlan.c        | 5 ++---
 4 files changed, 3 insertions(+), 15 deletions(-)
drivers/net/mlx5/linux/mlx5_verbs.c +0 -1
index b4a6b5e..494ddba 100644
@@ -368,7 +368,6 @@
 	MLX5_ASSERT(tmpl);
 	priv->verbs_alloc_ctx.type = MLX5_VERBS_ALLOC_TYPE_RX_QUEUE;
 	priv->verbs_alloc_ctx.obj = rxq_ctrl;
-	tmpl->type = MLX5_RXQ_OBJ_TYPE_IBV;
 	tmpl->rxq_ctrl = rxq_ctrl;
 	if (rxq_ctrl->irq) {
 		tmpl->ibv_channel =
drivers/net/mlx5/mlx5.h +0 -8
index f385b48..87d3c15 100644
@@ -676,18 +676,10 @@ struct mlx5_proc_priv {
 #define MLX5_PROC_PRIV(port_id) \
 	((struct mlx5_proc_priv *)rte_eth_devices[port_id].process_private)
 
-enum mlx5_rxq_obj_type {
-	MLX5_RXQ_OBJ_TYPE_IBV,          /* mlx5_rxq_obj with ibv_wq. */
-	MLX5_RXQ_OBJ_TYPE_DEVX_RQ,      /* mlx5_rxq_obj with mlx5_devx_rq. */
-	MLX5_RXQ_OBJ_TYPE_DEVX_HAIRPIN,
-	/* mlx5_rxq_obj with mlx5_devx_rq and hairpin support. */
-};
-
 /* Verbs/DevX Rx queue elements. */
 struct mlx5_rxq_obj {
 	LIST_ENTRY(mlx5_rxq_obj) next; /* Pointer to the next element. */
 	struct mlx5_rxq_ctrl *rxq_ctrl; /* Back pointer to parent. */
-	enum mlx5_rxq_obj_type type;
 	int fd; /* File descriptor for event channel */
 	RTE_STD_C11
 	union {
drivers/net/mlx5/mlx5_devx.c +1 -3
index a7c941c..11bda32 100644
@@ -208,7 +208,7 @@
 {
 	MLX5_ASSERT(rxq_obj);
 	MLX5_ASSERT(rxq_obj->rq);
-	if (rxq_obj->type == MLX5_RXQ_OBJ_TYPE_DEVX_HAIRPIN) {
+	if (rxq_obj->rxq_ctrl->type == MLX5_RXQ_TYPE_HAIRPIN) {
 		mlx5_devx_modify_rq(rxq_obj, MLX5_RXQ_MOD_RDY2RST);
 		claim_zero(mlx5_devx_cmd_destroy(rxq_obj->rq));
 	} else {
@@ -550,7 +550,6 @@
 
 	MLX5_ASSERT(rxq_data);
 	MLX5_ASSERT(tmpl);
-	tmpl->type = MLX5_RXQ_OBJ_TYPE_DEVX_HAIRPIN;
 	tmpl->rxq_ctrl = rxq_ctrl;
 	attr.hairpin = 1;
 	max_wq_data = priv->config.hca_attr.log_max_hairpin_wq_data_sz;
@@ -611,7 +610,6 @@
 	MLX5_ASSERT(tmpl);
 	if (rxq_ctrl->type == MLX5_RXQ_TYPE_HAIRPIN)
 		return mlx5_rxq_obj_hairpin_new(dev, idx);
-	tmpl->type = MLX5_RXQ_OBJ_TYPE_DEVX_RQ;
 	tmpl->rxq_ctrl = rxq_ctrl;
 	if (rxq_ctrl->irq) {
 		int devx_ev_flag =
drivers/net/mlx5/mlx5_vlan.c +2 -3
index 290503a..dbb9d36 100644
@@ -116,9 +116,8 @@
 	}
 	ret = priv->obj_ops.rxq_obj_modify_vlan_strip(rxq_ctrl->obj, on);
 	if (ret) {
-		DRV_LOG(ERR, "port %u failed to modify object %d stripping "
-			"mode: %s", dev->data->port_id,
-			rxq_ctrl->obj->type, strerror(rte_errno));
+		DRV_LOG(ERR, "Port %u failed to modify object stripping mode:"
+			" %s", dev->data->port_id, strerror(rte_errno));
 		return;
 	}
 	/* Update related bits in RX queue. */
Signature · 1 line
1.8.3.1
#16 lore raw
In reply to Michael Baum

Hi,

Quoted 45 lines

-----Original Message-----
From: Michael Baum <[email protected]>
Sent: Thursday, October 1, 2020 5:09 PM
To: [email protected]
Cc: Matan Azrad <[email protected]>; Raslan Darawsheh <[email protected]>; Slava Ovsiienko <[email protected]>
Subject: [PATCH v1 00/15] mlx5 Tx DevX/Verbs separation

The series is an arrangement to multi-OS support by net/mlx5 driver so it comes to ease the code management for OS which supports\doesn't support DevX\Verbs operations.

Michael Baum (15):
net/mlx5: fix send queue doorbell typo net/mlx5: fix unused variable in Txq creation net/mlx5: mitigate Tx queue reference counters net/mlx5: reorder Tx queue DevX object creation net/mlx5: reorder Tx queue Verbs object creation net/mlx5: reposition the event queue number field net/mlx5: separate Tx queue object creations net/mlx5: share Tx control code
net/mlx5: rearrange SQ and CQ creation in DevX module net/mlx5: rearrange QP creation in Verbs module net/mlx5: separate Tx queue object modification net/mlx5: share Tx queue object modification net/mlx5: remove Tx queue object type field net/mlx5: separate Rx queue state modification net/mlx5: remove Rx queue object type field

 drivers/net/mlx5/linux/mlx5_os.c    |  80 ++++
 drivers/net/mlx5/linux/mlx5_verbs.c | 296 ++++++++++++-
 drivers/net/mlx5/linux/mlx5_verbs.h |   3 +
 drivers/net/mlx5/mlx5.c             |  10 +
 drivers/net/mlx5/mlx5.h             |  61 ++-
 drivers/net/mlx5/mlx5_devx.c        | 593 +++++++++++++++++++++++--
 drivers/net/mlx5/mlx5_devx.h        |   3 +
 drivers/net/mlx5/mlx5_rxq.c         |   4 +-
 drivers/net/mlx5/mlx5_rxtx.c        | 105 +----
 drivers/net/mlx5/mlx5_rxtx.h        |  45 +-
 drivers/net/mlx5/mlx5_trigger.c     |  40 +-
 drivers/net/mlx5/mlx5_txpp.c        |  28 +-
 drivers/net/mlx5/mlx5_txq.c         | 850 ++----------------------------------
 drivers/net/mlx5/mlx5_vlan.c        |   5 +-
 14 files changed, 1087 insertions(+), 1036 deletions(-)
Quoted 2 lines

--
1.8.3.1

Series applied to next-net-mlx,

Kindest regards,
Raslan Darawsheh