Skip to contentSkip to conversation

ethdev: introduce hairpin memory capabilities

PATCH Superseded Dariusz Sosnowski 5 participants 30 messages Sep 19, 2022 – Nov 28, 2022 Network drivers

lore
Share & tools
lore
#0 lore raw

cover ethdev: introduce hairpin memory capabilities

This patch series introduces hairpin memory configuration options proposed in
http://patches.dpdk.org/project/dpdk/patch/[email protected]/
for Rx and Tx hairpin queues. It also implements handling of these options in mlx5 PMD and allows to use new hairpin options in testpmd (through --hairpin-mode option) and flow-perf (through --hairpin-conf option).

Dariusz Sosnowski (7):
ethdev: introduce hairpin memory capabilities common/mlx5: add hairpin SQ buffer type capabilities common/mlx5: add hairpin RQ buffer type capabilities net/mlx5: allow hairpin Tx queue in RTE memory net/mlx5: allow hairpin Rx queue in locked memory app/testpmd: add hairpin queues memory modes app/flow-perf: add hairpin queue memory config

 app/test-flow-perf/main.c             |  32 +++++
 app/test-pmd/parameters.c             |   2 +-
 app/test-pmd/testpmd.c                |  24 +++-
 app/test-pmd/testpmd.h                |   2 +-
 doc/guides/platform/mlx5.rst          |   5 +
 doc/guides/testpmd_app_ug/run_app.rst |  10 +-
 drivers/common/mlx5/mlx5_devx_cmds.c  |   8 ++
 drivers/common/mlx5/mlx5_devx_cmds.h  |   5 +
 drivers/common/mlx5/mlx5_prm.h        |  25 +++-
 drivers/net/mlx5/mlx5.h               |   2 +
 drivers/net/mlx5/mlx5_devx.c          | 170 +++++++++++++++++++++++---
 drivers/net/mlx5/mlx5_ethdev.c        |   6 +
 lib/ethdev/rte_ethdev.c               |  44 +++++++
 lib/ethdev/rte_ethdev.h               |  65 +++++++++-
 14 files changed, 373 insertions(+), 27 deletions(-)
Signature · 1 line
2.25.1
#1 lore raw

PATCH 1/7 ethdev: introduce hairpin memory capabilities

This patch introduces new hairpin queue configuration options through rte_eth_hairpin_conf struct, allowing to tune Rx and Tx hairpin queues memory configuration. Hairpin configuration is extended with the following fields:

- use_locked_device_memory - If set, PMD will use specialized on-device memory to store RX or TX hairpin queue data.
- use_rte_memory - If set, PMD will use DPDK-managed memory to store RX or TX hairpin queue data.
- force_memory - If set, PMD will be forced to use provided memory settings. If no appropriate resources are available, then device start will fail. If unset and no resources are available, PMD will fallback to using default type of resource for given queue.

Hairpin capabilities are also extended, to allow verification of support of given hairpin memory configurations. Struct rte_eth_hairpin_cap is extended with two additional fields of type rte_eth_hairpin_queue_cap:

- rx_cap - memory capabilities of hairpin RX queues.
- tx_cap - memory capabilities of hairpin TX queues.

Struct rte_eth_hairpin_queue_cap exposes whether given queue type supports use_locked_device_memory and use_rte_memory flags.

---
 lib/ethdev/rte_ethdev.c | 44 ++++++++++++++++++++++++++++
 lib/ethdev/rte_ethdev.h | 65 ++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 108 insertions(+), 1 deletion(-)
lib/ethdev/rte_ethdev.c +44 -0
index 1979dc0850..edcec08231 100644
@@ -1945,6 +1945,28 @@ rte_eth_rx_hairpin_queue_setup(uint16_t port_id, uint16_t rx_queue_id,
 			conf->peer_count, cap.max_rx_2_tx);
 		return -EINVAL;
 	}
+	if (conf->use_locked_device_memory && !cap.rx_cap.locked_device_memory) {
+		RTE_ETHDEV_LOG(ERR,
+			"Attempt to use locked device memory for Rx queue, which is not supported");
+		return -EINVAL;
+	}
+	if (conf->use_rte_memory && !cap.rx_cap.rte_memory) {
+		RTE_ETHDEV_LOG(ERR,
+			"Attempt to use DPDK memory for Rx queue, which is not supported");
+		return -EINVAL;
+	}
+	if (conf->use_locked_device_memory && conf->use_rte_memory) {
+		RTE_ETHDEV_LOG(ERR,
+			"Attempt to use mutually exclusive memory settings for Rx queue");
+		return -EINVAL;
+	}
+	if (conf->force_memory &&
+	    !conf->use_locked_device_memory &&
+	    !conf->use_rte_memory) {
+		RTE_ETHDEV_LOG(ERR,
+			"Attempt to force Rx queue memory settings, but none is set");
+		return -EINVAL;
+	}
 	if (conf->peer_count == 0) {
 		RTE_ETHDEV_LOG(ERR,
 			"Invalid value for number of peers for Rx queue(=%u), should be: > 0",
@@ -2111,6 +2133,28 @@ rte_eth_tx_hairpin_queue_setup(uint16_t port_id, uint16_t tx_queue_id,
 			conf->peer_count, cap.max_tx_2_rx);
 		return -EINVAL;
 	}
+	if (conf->use_locked_device_memory && !cap.tx_cap.locked_device_memory) {
+		RTE_ETHDEV_LOG(ERR,
+			"Attempt to use locked device memory for Tx queue, which is not supported");
+		return -EINVAL;
+	}
+	if (conf->use_rte_memory && !cap.tx_cap.rte_memory) {
+		RTE_ETHDEV_LOG(ERR,
+			"Attempt to use DPDK memory for Tx queue, which is not supported");
+		return -EINVAL;
+	}
+	if (conf->use_locked_device_memory && conf->use_rte_memory) {
+		RTE_ETHDEV_LOG(ERR,
+			"Attempt to use mutually exclusive memory settings for Tx queue");
+		return -EINVAL;
+	}
+	if (conf->force_memory &&
+	    !conf->use_locked_device_memory &&
+	    !conf->use_rte_memory) {
+		RTE_ETHDEV_LOG(ERR,
+			"Attempt to force Tx queue memory settings, but none is set");
+		return -EINVAL;
+	}
 	if (conf->peer_count == 0) {
 		RTE_ETHDEV_LOG(ERR,
 			"Invalid value for number of peers for Tx queue(=%u), should be: > 0",
lib/ethdev/rte_ethdev.h +64 -1

83 lines not on this page · raw message

Signature · 1 line
2.25.1
#2 lore raw

19/09/2022 18:37, Dariusz Sosnowski:

Quoted 4 lines

This patch introduces new hairpin queue configuration options through rte_eth_hairpin_conf struct, allowing to tune Rx and Tx hairpin queues memory configuration. Hairpin configuration is extended with the following fields:

What is the benefit?
How the user knows what to use?
Isn't it too much low level for a user?
Why it is not automatic in the driver?

[...]

lib/ethdev/rte_ethdev.h struct rte_eth_hairpin_conf { · line 1367

Quoted 6 lines
lib/ethdev/rte_ethdev.h +6 -0
+	/**
+	 * Use locked device memory as a backing storage.
+	 *
+	 * - When set, PMD will attempt to use on-device memory as a backing storage for descriptors
+	 *   and/or data in hairpin queue.
+	 * - When set, PMD will use detault memory type as a backing storage. Please refer to PMD

You probably mean "clear".
Please make lines shorter.
You should split lines logically, after a dot or at the end of a part.

Quoted 23 lines
lib/ethdev/rte_ethdev.h +23 -0
+	 *   documentation for details.
+	 *
+	 * API user should check if PMD supports this configuration flag using
+	 * @see rte_eth_dev_hairpin_capability_get.
+	 */
+	uint32_t use_locked_device_memory:1;
+
+	/**
+	 * Use DPDK memory as backing storage.
+	 *
+	 * - When set, PMD will attempt to use memory managed by DPDK as a backing storage
+	 *   for descriptors and/or data in hairpin queue.
+	 * - When clear, PMD will use default memory type as a backing storage. Please refer
+	 *   to PMD documentation for details.
+	 *
+	 * API user should check if PMD supports this configuration flag using
+	 * @see rte_eth_dev_hairpin_capability_get.
+	 */
+	uint32_t use_rte_memory:1;
+
+	/**
+	 * Force usage of hairpin memory configuration.
+	 *

lib/ethdev/rte_ethdev.h struct rte_eth_hairpin_conf { · line 1400

Quoted 10 lines
lib/ethdev/rte_ethdev.h +10 -0
+	 * - When set, PMD will attempt to use specified memory settings and
+	 *   if resource allocation fails, then hairpin queue setup will result in an
+	 *   error.
+	 * - When clear, PMD will attempt to use specified memory settings and
+	 *   if resource allocation fails, then PMD will retry allocation with default
+	 *   configuration.
+	 */
+	uint32_t force_memory:1;
+
+	uint32_t reserved:11; /**< Reserved bits. */

You can insert a blank line here.

Quoted 2 lines

struct rte_eth_hairpin_peer peers[RTE_ETH_MAX_HAIRPIN_PEERS];
};

#3 lore raw

Hi Thomas,

Quoted 1 line

What is the benefit?

Goal of this patchset is to present application developers with more options to fine tune hairpin configuration to their use case. I added more details regarding the possible benefits and motivation to the cover letter of v2.

Quoted 2 lines

How the user knows what to use? Why it is not automatic in the driver?

Basic assumption is that the default behavior of the PMD (mlx5 in that specific case) is a baseline for hairpin performance.
If that default performance is enough for a user, he will not have to change anything in the hairpin queue configuration.
If performance is not satisfying, user will have to experiment with different configurations e.g., if decreasing traffic latency is a priority for the user, user can check how his specific application works when use_locked_device_memory is used.

Specific performance gains of each of the hairpin options is not a given,
since hairpin performance will be a function of hairpin configuration and flow configuration (number of flows, types of flows, etc.).

Quoted 1 line

Isn't it too much low level for a user?

In my opinion, it's not too low level since purpose of these new options is fine tuning the hairpin performance to the specific use case of the application.

Quoted 6 lines
Quoted 2 lines

+ * - When set, PMD will use detault memory type as a backing
+ storage. Please refer to PMD

You probably mean "clear".
Please make lines shorter.
You should split lines logically, after a dot or at the end of a part.

Fixed in v2.

Quoted 7 lines

Review

Quoted 2 lines
(quoted diff) +2 -0
+
+     uint32_t reserved:11; /**< Reserved bits. */

You can insert a blank line here.

Quoted 2 lines
      struct rte_eth_hairpin_peer peers[RTE_ETH_MAX_HAIRPIN_PEERS];
};

Added in v2.

Best regards,
Dariusz Sosnowski

#4 lore raw

PATCH 2/7 common/mlx5: add hairpin SQ buffer type capabilities

In reply to Dariusz Sosnowski

This patch extends HCA_CAP and SQ Context structs available in PRM. This fields allow checking if NIC supports storing hairpin SQ's WQ buffer in host memory and configuring such memory placement.

HCA capabilities are extended with the following fields:

- hairpin_sq_wq_in_host_mem - If set, then NIC supports using host memory as a backing storage for hairpin SQ's WQ buffer.
- hairpin_sq_wqe_bb_size - Indicates the required size of SQ WQE basic block.

SQ Context is extended with hairpin_wq_buffer_type which informs NIC where SQ's WQ buffer will be stored. This field can take the following values:

- MLX5_SQC_HAIRPIN_WQ_BUFFER_TYPE_INTERNAL_BUFFER - WQ buffer will be stored in unlocked device memory.
- MLX5_SQC_HAIRPIN_WQ_BUFFER_TYPE_HOST_MEMORY - WQ buffer will be stored in host memory. Buffer is provided by PMD.

---
 drivers/common/mlx5/mlx5_devx_cmds.c |  5 +++++
 drivers/common/mlx5/mlx5_devx_cmds.h |  3 +++
 drivers/common/mlx5/mlx5_prm.h       | 15 +++++++++++++--
 3 files changed, 21 insertions(+), 2 deletions(-)
drivers/common/mlx5/mlx5_devx_cmds.c +5 -0
index 8880a9f3b5..2b12ce0d4c 100644
@@ -981,6 +981,10 @@ mlx5_devx_cmd_query_hca_attr(void *ctx,
 		}
 		attr->log_min_stride_wqe_sz = MLX5_GET(cmd_hca_cap_2, hcattr,
 						       log_min_stride_wqe_sz);
+		attr->hairpin_sq_wqe_bb_size = MLX5_GET(cmd_hca_cap_2, hcattr,
+							hairpin_sq_wqe_bb_size);
+		attr->hairpin_sq_wq_in_host_mem = MLX5_GET(cmd_hca_cap_2, hcattr,
+							   hairpin_sq_wq_in_host_mem);
 	}
 	if (attr->log_min_stride_wqe_sz == 0)
 		attr->log_min_stride_wqe_sz = MLX5_MPRQ_LOG_MIN_STRIDE_WQE_SIZE;
@@ -1698,6 +1702,7 @@ mlx5_devx_cmd_create_sq(void *ctx,
 	MLX5_SET(sqc, sq_ctx, hairpin, sq_attr->hairpin);
 	MLX5_SET(sqc, sq_ctx, non_wire, sq_attr->non_wire);
 	MLX5_SET(sqc, sq_ctx, static_sq_wq, sq_attr->static_sq_wq);
+	MLX5_SET(sqc, sq_ctx, hairpin_wq_buffer_type, sq_attr->hairpin_wq_buffer_type);
 	MLX5_SET(sqc, sq_ctx, user_index, sq_attr->user_index);
 	MLX5_SET(sqc, sq_ctx, cqn, sq_attr->cqn);
 	MLX5_SET(sqc, sq_ctx, packet_pacing_rate_limit_index,
drivers/common/mlx5/mlx5_devx_cmds.h +3 -0
index af6053a788..9ac2d75df4 100644
@@ -191,6 +191,8 @@ struct mlx5_hca_attr {
 	uint32_t log_max_hairpin_queues:5;
 	uint32_t log_max_hairpin_wq_data_sz:5;
 	uint32_t log_max_hairpin_num_packets:5;
+	uint32_t hairpin_sq_wqe_bb_size:4;
+	uint32_t hairpin_sq_wq_in_host_mem:1;
 	uint32_t vhca_id:16;
 	uint32_t relaxed_ordering_write:1;
 	uint32_t relaxed_ordering_read:1;
@@ -407,6 +409,7 @@ struct mlx5_devx_create_sq_attr {
 	uint32_t non_wire:1;
 	uint32_t static_sq_wq:1;
 	uint32_t ts_format:2;
+	uint32_t hairpin_wq_buffer_type:3;
 	uint32_t user_index:24;
 	uint32_t cqn:24;
 	uint32_t packet_pacing_rate_limit_index:16;
drivers/common/mlx5/mlx5_prm.h +13 -2
index 4346279c81..04d35ca845 100644
@@ -2020,7 +2020,11 @@ struct mlx5_ifc_cmd_hca_cap_2_bits {
 	u8 reserved_at_d8[0x3];
 	u8 log_max_conn_track_offload[0x5];
 	u8 reserved_at_e0[0x20]; /* End of DW7. */
-	u8 reserved_at_100[0x700];
+	u8 reserved_at_100[0x60];
+	u8 reserved_at_160[0x3];
+	u8 hairpin_sq_wqe_bb_size[0x5];
+	u8 hairpin_sq_wq_in_host_mem[0x1];
+	u8 reserved_at_169[0x697];
 };
 
 struct mlx5_ifc_esw_cap_bits {
@@ -2673,6 +2677,11 @@ enum {
 	MLX5_SQC_STATE_ERR  = 0x3,
 };
 
+enum {
+	MLX5_SQC_HAIRPIN_WQ_BUFFER_TYPE_INTERNAL_BUFFER = 0x0,
+	MLX5_SQC_HAIRPIN_WQ_BUFFER_TYPE_HOST_MEMORY = 0x1,
+};
+
 struct mlx5_ifc_sqc_bits {
 	u8 rlky[0x1];
 	u8 cd_master[0x1];
@@ -2686,7 +2695,9 @@ struct mlx5_ifc_sqc_bits {
 	u8 hairpin[0x1];
 	u8 non_wire[0x1];
 	u8 static_sq_wq[0x1];
-	u8 reserved_at_11[0x9];
+	u8 reserved_at_11[0x4];
+	u8 hairpin_wq_buffer_type[0x3];
+	u8 reserved_at_18[0x2];
 	u8 ts_format[0x02];
 	u8 reserved_at_1c[0x4];
 	u8 reserved_at_20[0x8];
Signature · 1 line
2.25.1
Acked-by on v1 2/7
#5 lore raw

RE: [PATCH 2/7] common/mlx5: add hairpin SQ buffer type capabilities

Quoted 21 lines

-----Original Message-----
From: Dariusz Sosnowski <[email protected]>
Sent: Monday, September 19, 2022 19:37
To: Matan Azrad <[email protected]>; Slava Ovsiienko <[email protected]>
Cc: [email protected]
Subject: [PATCH 2/7] common/mlx5: add hairpin SQ buffer type capabilities

This patch extends HCA_CAP and SQ Context structs available in PRM. This fields allow checking if NIC supports storing hairpin SQ's WQ buffer in host memory and configuring such memory placement.

HCA capabilities are extended with the following fields:

- hairpin_sq_wq_in_host_mem - If set, then NIC supports using host memory as a backing storage for hairpin SQ's WQ buffer.
- hairpin_sq_wqe_bb_size - Indicates the required size of SQ WQE basic block.

SQ Context is extended with hairpin_wq_buffer_type which informs NIC where SQ's WQ buffer will be stored. This field can take the following values:

Quoted 6 lines

- MLX5_SQC_HAIRPIN_WQ_BUFFER_TYPE_INTERNAL_BUFFER - WQ buffer will be stored in unlocked device memory.
- MLX5_SQC_HAIRPIN_WQ_BUFFER_TYPE_HOST_MEMORY - WQ buffer will be stored in host memory. Buffer is provided by PMD.

#6 lore raw

PATCH 3/7 common/mlx5: add hairpin RQ buffer type capabilities

In reply to Dariusz Sosnowski

This patch adds new HCA capability related to hairpin RQs. This new capability, hairpin_data_buffer_locked, indicates whether HCA supports locking data buffer of hairpin RQ in ICMC (Interconnect Context Memory Cache).

Struct used to define RQ configuration (RQ context) is extended with hairpin_data_buffer_type field, which configures data buffer for hairpin RQ. It can take the following values:

- MLX5_RQC_HAIRPIN_DATA_BUFFER_TYPE_UNLOCKED_INTERNAL_BUFFER - hairpin RQ's data buffer is stored in unlocked memory in ICMC.
- MLX5_RQC_HAIRPIN_DATA_BUFFER_TYPE_LOCKED_INTERNAL_BUFFER - hairpin RQ's data buffer is stored in locked memory in ICMC.

---
 drivers/common/mlx5/mlx5_devx_cmds.c |  3 +++
 drivers/common/mlx5/mlx5_devx_cmds.h |  2 ++
 drivers/common/mlx5/mlx5_prm.h       | 12 ++++++++++--
 3 files changed, 15 insertions(+), 2 deletions(-)
drivers/common/mlx5/mlx5_devx_cmds.c +3 -0
index 2b12ce0d4c..95b38783dc 100644
@@ -985,6 +985,8 @@ mlx5_devx_cmd_query_hca_attr(void *ctx,
 							hairpin_sq_wqe_bb_size);
 		attr->hairpin_sq_wq_in_host_mem = MLX5_GET(cmd_hca_cap_2, hcattr,
 							   hairpin_sq_wq_in_host_mem);
+		attr->hairpin_data_buffer_locked = MLX5_GET(cmd_hca_cap_2, hcattr,
+							    hairpin_data_buffer_locked);
 	}
 	if (attr->log_min_stride_wqe_sz == 0)
 		attr->log_min_stride_wqe_sz = MLX5_MPRQ_LOG_MIN_STRIDE_WQE_SIZE;
@@ -1285,6 +1287,7 @@ mlx5_devx_cmd_create_rq(void *ctx,
 	MLX5_SET(rqc, rq_ctx, state, rq_attr->state);
 	MLX5_SET(rqc, rq_ctx, flush_in_error_en, rq_attr->flush_in_error_en);
 	MLX5_SET(rqc, rq_ctx, hairpin, rq_attr->hairpin);
+	MLX5_SET(rqc, rq_ctx, hairpin_data_buffer_type, rq_attr->hairpin_data_buffer_type);
 	MLX5_SET(rqc, rq_ctx, user_index, rq_attr->user_index);
 	MLX5_SET(rqc, rq_ctx, cqn, rq_attr->cqn);
 	MLX5_SET(rqc, rq_ctx, counter_set_id, rq_attr->counter_set_id);
drivers/common/mlx5/mlx5_devx_cmds.h +2 -0
index 9ac2d75df4..cceaf3411d 100644
@@ -193,6 +193,7 @@ struct mlx5_hca_attr {
 	uint32_t log_max_hairpin_num_packets:5;
 	uint32_t hairpin_sq_wqe_bb_size:4;
 	uint32_t hairpin_sq_wq_in_host_mem:1;
+	uint32_t hairpin_data_buffer_locked:1;
 	uint32_t vhca_id:16;
 	uint32_t relaxed_ordering_write:1;
 	uint32_t relaxed_ordering_read:1;
@@ -313,6 +314,7 @@ struct mlx5_devx_create_rq_attr {
 	uint32_t state:4;
 	uint32_t flush_in_error_en:1;
 	uint32_t hairpin:1;
+	uint32_t hairpin_data_buffer_type:3;
 	uint32_t ts_format:2;
 	uint32_t user_index:24;
 	uint32_t cqn:24;
drivers/common/mlx5/mlx5_prm.h +10 -2
index 04d35ca845..9c1c93f916 100644
@@ -2024,7 +2024,8 @@ struct mlx5_ifc_cmd_hca_cap_2_bits {
 	u8 reserved_at_160[0x3];
 	u8 hairpin_sq_wqe_bb_size[0x5];
 	u8 hairpin_sq_wq_in_host_mem[0x1];
-	u8 reserved_at_169[0x697];
+	u8 hairpin_data_buffer_locked[0x1];
+	u8 reserved_at_16a[0x696];
 };
 
 struct mlx5_ifc_esw_cap_bits {
@@ -2304,7 +2305,9 @@ struct mlx5_ifc_rqc_bits {
 	u8 reserved_at_c[0x1];
 	u8 flush_in_error_en[0x1];
 	u8 hairpin[0x1];
-	u8 reserved_at_f[0xB];
+	u8 reserved_at_f[0x6];
+	u8 hairpin_data_buffer_type[0x3];
+	u8 reserved_at_a8[0x2];
 	u8 ts_format[0x02];
 	u8 reserved_at_1c[0x4];
 	u8 reserved_at_20[0x8];
@@ -2813,6 +2816,11 @@ enum {
 	MLX5_CQE_SIZE_128B = 0x1,
 };
 
+enum {
+	MLX5_RQC_HAIRPIN_DATA_BUFFER_TYPE_UNLOCKED_INTERNAL_BUFFER = 0x0,
+	MLX5_RQC_HAIRPIN_DATA_BUFFER_TYPE_LOCKED_INTERNAL_BUFFER = 0x1,
+};
+
 struct mlx5_ifc_cqc_bits {
 	u8 status[0x4];
 	u8 as_notify[0x1];
Signature · 1 line
2.25.1
Acked-by on v1 3/7
#7 lore raw

RE: [PATCH 3/7] common/mlx5: add hairpin RQ buffer type capabilities

Quoted 16 lines

-----Original Message-----
From: Dariusz Sosnowski <[email protected]>
Sent: Monday, September 19, 2022 19:37
To: Matan Azrad <[email protected]>; Slava Ovsiienko <[email protected]>
Cc: [email protected]
Subject: [PATCH 3/7] common/mlx5: add hairpin RQ buffer type capabilities

This patch adds new HCA capability related to hairpin RQs. This new capability, hairpin_data_buffer_locked, indicates whether HCA supports locking data buffer of hairpin RQ in ICMC (Interconnect Context Memory Cache).

Struct used to define RQ configuration (RQ context) is extended with hairpin_data_buffer_type field, which configures data buffer for hairpin RQ. It can take the following values:

Quoted 6 lines

- MLX5_RQC_HAIRPIN_DATA_BUFFER_TYPE_UNLOCKED_INTERNAL_BUFFER - hairpin RQ's data buffer is stored in unlocked memory in ICMC.
- MLX5_RQC_HAIRPIN_DATA_BUFFER_TYPE_LOCKED_INTERNAL_BUFFER - hairpin RQ's data buffer is stored in locked memory in ICMC.

#8 lore raw

PATCH 4/7 net/mlx5: allow hairpin Tx queue in RTE memory

In reply to Dariusz Sosnowski

This patch adds a capability to place hairpin Tx queue in host memory managed by DPDK. This capability is equivalent to storing hairpin SQ's WQ buffer in host memory.

Hairpin Tx queue creation is extended with allocating a memory buffer of proper size (calculated from required number of packets and WQE BB size advertised in HCA capabilities).

force_memory flag of hairpin queue configuration is also supported. If it is set and:

- allocation of memory buffer fails,
- or hairpin SQ creation fails,

then device start will fail. If it is unset, PMD will fallback to creating the hairpin SQ with WQ buffer located in unlocked device memory.

---
 drivers/net/mlx5/mlx5.h        |   2 +
 drivers/net/mlx5/mlx5_devx.c   | 119 ++++++++++++++++++++++++++++++---
 drivers/net/mlx5/mlx5_ethdev.c |   4 ++
 3 files changed, 116 insertions(+), 9 deletions(-)
drivers/net/mlx5/mlx5.h +2 -0
index 8af84aef50..f564d4b771 100644
@@ -1384,6 +1384,8 @@ struct mlx5_txq_obj {
 			struct mlx5_devx_obj *sq;
 			/* DevX object for Sx queue. */
 			struct mlx5_devx_obj *tis; /* The TIS object. */
+			void *umem_buf_wq_buffer;
+			struct mlx5dv_devx_umem *umem_obj_wq_buffer;
 		};
 		struct {
 			struct rte_eth_dev *dev;
drivers/net/mlx5/mlx5_devx.c +110 -9

155 lines not on this page · raw message

drivers/net/mlx5/mlx5_ethdev.c +4 -0
index 05c919ed39..7f5b01ac74 100644
@@ -729,6 +729,7 @@ int
 mlx5_hairpin_cap_get(struct rte_eth_dev *dev, struct rte_eth_hairpin_cap *cap)
 {
 	struct mlx5_priv *priv = dev->data->dev_private;
+	struct mlx5_hca_attr *hca_attr;
 
 	if (!mlx5_devx_obj_ops_en(priv->sh)) {
 		rte_errno = ENOTSUP;
@@ -738,5 +739,8 @@ mlx5_hairpin_cap_get(struct rte_eth_dev *dev, struct rte_eth_hairpin_cap *cap)
 	cap->max_rx_2_tx = 1;
 	cap->max_tx_2_rx = 1;
 	cap->max_nb_desc = 8192;
+	hca_attr = &priv->sh->cdev->config.hca_attr;
+	cap->tx_cap.locked_device_memory = 0;
+	cap->tx_cap.rte_memory = hca_attr->hairpin_sq_wq_in_host_mem;
 	return 0;
 }
Signature · 1 line
2.25.1
Acked-by on v1 4/7
#9 lore raw

RE: [PATCH 4/7] net/mlx5: allow hairpin Tx queue in RTE memory

Quoted 18 lines

-----Original Message-----
From: Dariusz Sosnowski <[email protected]>
Sent: Monday, September 19, 2022 19:37
To: Matan Azrad <[email protected]>; Slava Ovsiienko <[email protected]>
Cc: [email protected]
Subject: [PATCH 4/7] net/mlx5: allow hairpin Tx queue in RTE memory

This patch adds a capability to place hairpin Tx queue in host memory managed by DPDK. This capability is equivalent to storing hairpin SQ's WQ buffer in host memory.

Hairpin Tx queue creation is extended with allocating a memory buffer of proper size (calculated from required number of packets and WQE BB size advertised in HCA capabilities).

force_memory flag of hairpin queue configuration is also supported. If it is set and:

Quoted 7 lines

- allocation of memory buffer fails,
- or hairpin SQ creation fails,

then device start will fail. If it is unset, PMD will fallback to creating the hairpin SQ with WQ buffer located in unlocked device memory.

#10 lore raw

PATCH 5/7 net/mlx5: allow hairpin Rx queue in locked memory

In reply to Dariusz Sosnowski

This patch adds a capability to place hairpin Rx queue in locked device memory. This capability is equivalent to storing hairpin RQ's data buffers in locked internal device memory.

Hairpin Rx queue creation is extended with requesting that RQ is allocated in locked internal device memory. If allocation fails and force_memory hairpin configuration is set, then hairpin queue creation (and, as a result, device start) fails. If force_memory is unset, then PMD will fallback to allocating memory for hairpin RQ in unlocked internal device memory.

To allow such allocation, the user must set HAIRPIN_DATA_BUFFER_LOCK flag in FW using mlxconfig tool.

---
 doc/guides/platform/mlx5.rst   |  5 ++++
 drivers/net/mlx5/mlx5_devx.c   | 51 ++++++++++++++++++++++++++++------
 drivers/net/mlx5/mlx5_ethdev.c |  2 ++
 3 files changed, 49 insertions(+), 9 deletions(-)
doc/guides/platform/mlx5.rst +5 -0
index 38c1fdce4c..88a2961bb4 100644
@@ -548,6 +548,11 @@ Below are some firmware configurations listed.
 
    REAL_TIME_CLOCK_ENABLE=1
 
+- allow locking hairpin RQ data buffer in device memory::
+
+   HAIRPIN_DATA_BUFFER_LOCK=1
+   MEMIC_SIZE_LIMIT=0
+
 
 .. _mlx5_common_driver_options:
 
drivers/net/mlx5/mlx5_devx.c +42 -9

82 lines not on this page · raw message

drivers/net/mlx5/mlx5_ethdev.c +2 -0
index 7f5b01ac74..7f400da103 100644
@@ -740,6 +740,8 @@ mlx5_hairpin_cap_get(struct rte_eth_dev *dev, struct rte_eth_hairpin_cap *cap)
 	cap->max_tx_2_rx = 1;
 	cap->max_nb_desc = 8192;
 	hca_attr = &priv->sh->cdev->config.hca_attr;
+	cap->rx_cap.locked_device_memory = hca_attr->hairpin_data_buffer_locked;
+	cap->rx_cap.rte_memory = 0;
 	cap->tx_cap.locked_device_memory = 0;
 	cap->tx_cap.rte_memory = hca_attr->hairpin_sq_wq_in_host_mem;
 	return 0;
Signature · 1 line
2.25.1
Acked-by on v1 5/7
#11 lore raw

RE: [PATCH 5/7] net/mlx5: allow hairpin Rx queue in locked memory

Quoted 17 lines

-----Original Message-----
From: Dariusz Sosnowski <[email protected]>
Sent: Monday, September 19, 2022 19:37
To: Matan Azrad <[email protected]>; Slava Ovsiienko <[email protected]>
Cc: [email protected]
Subject: [PATCH 5/7] net/mlx5: allow hairpin Rx queue in locked memory

This patch adds a capability to place hairpin Rx queue in locked device memory. This capability is equivalent to storing hairpin RQ's data buffers in locked internal device memory.

Hairpin Rx queue creation is extended with requesting that RQ is allocated in locked internal device memory. If allocation fails and force_memory hairpin configuration is set, then hairpin queue creation (and, as a result, device start) fails. If force_memory is unset, then PMD will fallback to allocating memory for hairpin RQ in unlocked internal device memory.

Quoted 4 lines

To allow such allocation, the user must set HAIRPIN_DATA_BUFFER_LOCK flag in FW using mlxconfig tool.

#12 lore raw

Re: [PATCH 5/7] net/mlx5: allow hairpin Rx queue in locked memory

In reply to Dariusz Sosnowski

This series adds support for using device-managed MEMIC buffers on hairpin RQ instead of NIM. Was it ever considered as an alternative that the UMEM interface be extended to support MEMIC buffers instead?

I'm thinking that could simplify the hairpin-specific firmware bits being added in this series, e.g. no new HAIRPIN_DATA_BUFFER_LOCK TLV, and the MEMIC-backed UMEM can be passed to RQ using existing PRM bits.

I'm planning to file a feature request adding MEMIC support to UMEM, so I'd be interested in knowing if that's somehow not possible. My current use case is allocating 64 bytes of MEMIC for a collapsed CQE for something similar to the mlx5 packet send scheduling in DPDK.

Best regards,
Kenneth Jonassen

Quoted 23 lines · Dariusz Sosnowski

This patch adds a capability to place hairpin Rx queue in locked device memory. This capability is equivalent to storing hairpin RQ's data buffers in locked internal device memory.

Hairpin Rx queue creation is extended with requesting that RQ is allocated in locked internal device memory. If allocation fails and force_memory hairpin configuration is set, then hairpin queue creation (and, as a result, device start) fails. If force_memory is unset, then PMD will fallback to allocating memory for hairpin RQ in unlocked internal device memory.

To allow such allocation, the user must set HAIRPIN_DATA_BUFFER_LOCK flag in FW using mlxconfig tool.

---

doc/guides/platform/mlx5.rst   |  5 ++++
drivers/net/mlx5/mlx5_devx.c   | 51 ++++++++++++++++++++++++++++------
drivers/net/mlx5/mlx5_ethdev.c |  2 ++
3 files changed, 49 insertions(+), 9 deletions(-)
#13 lore raw

PATCH 6/7 app/testpmd: add hairpin queues memory modes

In reply to Dariusz Sosnowski

This patch extends hairpin-mode command line option of test-pmd application with an ability to configure whether Rx/Tx hairpin queue should use locked device memory or RTE memory.

For purposes of this configurations the following bits of 32 bit hairpin-mode are reserved:

- Bit 8 - If set, then force_memory flag will be set for hairpin RX queue.
- Bit 9 - If set, then force_memory flag will be set for hairpin TX queue.
- Bits 12-15 - Memory options for hairpin Rx queue:
- Bit 12 - If set, then use_locked_device_memory will be set.
- Bit 13 - If set, then use_rte_memory will be set.
- Bit 14 - Reserved for future use.
- Bit 15 - Reserved for future use.
- Bits 16-19 - Memory options for hairpin Tx queue:
- Bit 16 - If set, then use_locked_device_memory will be set.
- Bit 17 - If set, then use_rte_memory will be set.
- Bit 18 - Reserved for future use.
- Bit 19 - Reserved for future use.

---
 app/test-pmd/parameters.c             |  2 +-
 app/test-pmd/testpmd.c                | 24 +++++++++++++++++++++++-
 app/test-pmd/testpmd.h                |  2 +-
 doc/guides/testpmd_app_ug/run_app.rst | 10 ++++++++--
 4 files changed, 33 insertions(+), 5 deletions(-)
app/test-pmd/parameters.c +1 -1
index e3c9757f3f..662e6e4a36 100644
@@ -1162,7 +1162,7 @@ launch_args_parse(int argc, char** argv)
 				if (errno != 0 || end == optarg)
 					rte_exit(EXIT_FAILURE, "hairpin mode invalid\n");
 				else
-					hairpin_mode = (uint16_t)n;
+					hairpin_mode = (uint32_t)n;
 			}
 			if (!strcmp(lgopts[opt_idx].name, "burst")) {
 				n = atoi(optarg);
app/test-pmd/testpmd.c +23 -1
index addcbcac85..2fbd546073 100644
@@ -409,7 +409,7 @@ bool setup_on_probe_event = true;
 uint8_t clear_ptypes = true;
 
 /* Hairpin ports configuration mode. */
-uint16_t hairpin_mode;
+uint32_t hairpin_mode;
 
 /* Pretty printing of ethdev events */
 static const char * const eth_event_desc[] = {
@@ -2552,6 +2552,16 @@ port_is_started(portid_t port_id)
 	return 1;
 }
 
+#define HAIRPIN_MODE_RX_FORCE_MEMORY RTE_BIT32(8)
+#define HAIRPIN_MODE_TX_FORCE_MEMORY RTE_BIT32(9)
+
+#define HAIRPIN_MODE_RX_LOCKED_MEMORY RTE_BIT32(12)
+#define HAIRPIN_MODE_RX_RTE_MEMORY RTE_BIT32(13)
+
+#define HAIRPIN_MODE_TX_LOCKED_MEMORY RTE_BIT32(16)
+#define HAIRPIN_MODE_TX_RTE_MEMORY RTE_BIT32(17)
+
+
 /* Configure the Rx and Tx hairpin queues for the selected port. */
 static int
 setup_hairpin_queues(portid_t pi, portid_t p_pi, uint16_t cnt_pi)
@@ -2567,6 +2577,12 @@ setup_hairpin_queues(portid_t pi, portid_t p_pi, uint16_t cnt_pi)
 	uint16_t peer_tx_port = pi;
 	uint32_t manual = 1;
 	uint32_t tx_exp = hairpin_mode & 0x10;
+	uint32_t rx_force_memory = hairpin_mode & HAIRPIN_MODE_RX_FORCE_MEMORY;
+	uint32_t rx_locked_memory = hairpin_mode & HAIRPIN_MODE_RX_LOCKED_MEMORY;
+	uint32_t rx_rte_memory = hairpin_mode & HAIRPIN_MODE_RX_RTE_MEMORY;
+	uint32_t tx_force_memory = hairpin_mode & HAIRPIN_MODE_TX_FORCE_MEMORY;
+	uint32_t tx_locked_memory = hairpin_mode & HAIRPIN_MODE_TX_LOCKED_MEMORY;
+	uint32_t tx_rte_memory = hairpin_mode & HAIRPIN_MODE_TX_RTE_MEMORY;
 
 	if (!(hairpin_mode & 0xf)) {
 		peer_rx_port = pi;
@@ -2606,6 +2622,9 @@ setup_hairpin_queues(portid_t pi, portid_t p_pi, uint16_t cnt_pi)
 		hairpin_conf.peers[0].queue = i + nb_rxq;
 		hairpin_conf.manual_bind = !!manual;
 		hairpin_conf.tx_explicit = !!tx_exp;
+		hairpin_conf.force_memory = !!tx_force_memory;
+		hairpin_conf.use_locked_device_memory = !!tx_locked_memory;
+		hairpin_conf.use_rte_memory = !!tx_rte_memory;
 		diag = rte_eth_tx_hairpin_queue_setup
 			(pi, qi, nb_txd, &hairpin_conf);
 		i++;
@@ -2629,6 +2648,9 @@ setup_hairpin_queues(portid_t pi, portid_t p_pi, uint16_t cnt_pi)
 		hairpin_conf.peers[0].queue = i + nb_txq;
 		hairpin_conf.manual_bind = !!manual;
 		hairpin_conf.tx_explicit = !!tx_exp;
+		hairpin_conf.force_memory = !!rx_force_memory;
+		hairpin_conf.use_locked_device_memory = !!rx_locked_memory;
+		hairpin_conf.use_rte_memory = !!rx_rte_memory;
 		diag = rte_eth_rx_hairpin_queue_setup
 			(pi, qi, nb_rxd, &hairpin_conf);
 		i++;
app/test-pmd/testpmd.h +1 -1
index fb2f5195d3..bc4d9788fa 100644
@@ -542,7 +542,7 @@ extern uint16_t stats_period;
 extern struct rte_eth_xstat_name *xstats_display;
 extern unsigned int xstats_display_num;
 
-extern uint16_t hairpin_mode;
+extern uint32_t hairpin_mode;
 
 #ifdef RTE_LIB_LATENCYSTATS
 extern uint8_t latencystats_enabled;
doc/guides/testpmd_app_ug/run_app.rst +8 -2
index 30edef07ea..c91c231094 100644
@@ -556,10 +556,16 @@ The command line options are:
 
     Enable display of RX and TX burst stats.
 
-*   ``--hairpin-mode=0xXX``
+*   ``--hairpin-mode=0xXXXX``
 
-    Set the hairpin port mode with bitmask, only valid when hairpin queues number is set::
+    Set the hairpin port configuration with bitmask, only valid when hairpin queues number is set::
 
+	bit 18 - hairpin TX queues will use RTE memory
+	bit 16 - hairpin TX queues will use locked device memory
+	bit 13 - hairpin RX queues will use RTE memory
+	bit 12 - hairpin RX queues will use locked device memory
+	bit 9 - force memory settings of hairpin TX queue
+	bit 8 - force memory settings of hairpin RX queue
 	bit 4 - explicit Tx flow rule
 	bit 1 - two hairpin ports paired
 	bit 0 - two hairpin ports loop
Signature · 1 line
2.25.1
#14 lore raw

PATCH 7/7 app/flow-perf: add hairpin queue memory config

In reply to Dariusz Sosnowski

This patch adds the hairpin-conf command line parameter to flow-perf application. hairpin-conf parameter takes a hexadecimal bitmask with bits having the following meaning:

- Bit 0 - Force memory settings of hairpin RX queue.
- Bit 1 - Force memory settings of hairpin TX queue.
- Bit 4 - Use locked device memory for hairpin RX queue.
- Bit 5 - Use RTE memory for hairpin RX queue.
- Bit 8 - Use locked device memory for hairpin TX queue.
- Bit 9 - Use RTE memory for hairpin TX queue.

---
 app/test-flow-perf/main.c | 32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)
app/test-flow-perf/main.c +32 -0
index f375097028..4a9206803a 100644
@@ -46,6 +46,15 @@
 #define DEFAULT_RULES_BATCH     100000
 #define DEFAULT_GROUP                0
 
+#define HAIRPIN_RX_CONF_FORCE_MEMORY  (0x0001)
+#define HAIRPIN_TX_CONF_FORCE_MEMORY  (0x0002)
+
+#define HAIRPIN_RX_CONF_LOCKED_MEMORY (0x0010)
+#define HAIRPIN_RX_CONF_RTE_MEMORY    (0x0020)
+
+#define HAIRPIN_TX_CONF_LOCKED_MEMORY (0x0100)
+#define HAIRPIN_TX_CONF_RTE_MEMORY    (0x0200)
+
 struct rte_flow *flow;
 static uint8_t flow_group;
 
@@ -61,6 +70,7 @@ static uint32_t policy_id[MAX_PORTS];
 static uint8_t items_idx, actions_idx, attrs_idx;
 
 static uint64_t ports_mask;
+static uint64_t hairpin_conf_mask;
 static uint16_t dst_ports[RTE_MAX_ETHPORTS];
 static volatile bool force_quit;
 static bool dump_iterations;
@@ -482,6 +492,7 @@ usage(char *progname)
 	printf("  --enable-fwd: To enable packets forwarding"
 		" after insertion\n");
 	printf("  --portmask=N: hexadecimal bitmask of ports used\n");
+	printf("  --hairpin-conf=0xXXXX: hexadecimal bitmask of hairpin queue configuration\n");
 	printf("  --random-priority=N,S: use random priority levels "
 		"from 0 to (N - 1) for flows "
 		"and S as seed for pseudo-random number generator\n");
@@ -629,6 +640,7 @@ static void
 args_parse(int argc, char **argv)
 {
 	uint64_t pm, seed;
+	uint64_t hp_conf;
 	char **argvopt;
 	uint32_t prio;
 	char *token;
@@ -648,6 +660,7 @@ args_parse(int argc, char **argv)
 		{ "enable-fwd",                 0, 0, 0 },
 		{ "unique-data",                0, 0, 0 },
 		{ "portmask",                   1, 0, 0 },
+		{ "hairpin-conf",               1, 0, 0 },
 		{ "cores",                      1, 0, 0 },
 		{ "random-priority",            1, 0, 0 },
 		{ "meter-profile-alg",          1, 0, 0 },
@@ -880,6 +893,13 @@ args_parse(int argc, char **argv)
 					rte_exit(EXIT_FAILURE, "Invalid fwd port mask\n");
 				ports_mask = pm;
 			}
+			if (strcmp(lgopts[opt_idx].name, "hairpin-conf") == 0) {
+				end = NULL;
+				hp_conf = strtoull(optarg, &end, 16);
+				if ((optarg[0] == '\0') || (end == NULL) || (*end != '\0'))
+					rte_exit(EXIT_FAILURE, "Invalid hairpin config mask\n");
+				hairpin_conf_mask = hp_conf;
+			}
 			if (strcmp(lgopts[opt_idx].name,
 					"port-id") == 0) {
 				uint16_t port_idx = 0;
@@ -2035,6 +2055,12 @@ init_port(void)
 				hairpin_conf.peers[0].port = port_id;
 				hairpin_conf.peers[0].queue =
 					std_queue + tx_queues_count;
+				hairpin_conf.use_locked_device_memory =
+					!!(hairpin_conf_mask & HAIRPIN_RX_CONF_LOCKED_MEMORY);
+				hairpin_conf.use_rte_memory =
+					!!(hairpin_conf_mask & HAIRPIN_RX_CONF_RTE_MEMORY);
+				hairpin_conf.force_memory =
+					!!(hairpin_conf_mask & HAIRPIN_RX_CONF_FORCE_MEMORY);
 				ret = rte_eth_rx_hairpin_queue_setup(
 						port_id, hairpin_queue,
 						rxd_count, &hairpin_conf);
@@ -2050,6 +2076,12 @@ init_port(void)
 				hairpin_conf.peers[0].port = port_id;
 				hairpin_conf.peers[0].queue =
 					std_queue + rx_queues_count;
+				hairpin_conf.use_locked_device_memory =
+					!!(hairpin_conf_mask & HAIRPIN_TX_CONF_LOCKED_MEMORY);
+				hairpin_conf.use_rte_memory =
+					!!(hairpin_conf_mask & HAIRPIN_TX_CONF_RTE_MEMORY);
+				hairpin_conf.force_memory =
+					!!(hairpin_conf_mask & HAIRPIN_TX_CONF_FORCE_MEMORY);
 				ret = rte_eth_tx_hairpin_queue_setup(
 						port_id, hairpin_queue,
 						txd_count, &hairpin_conf);
Signature · 1 line
2.25.1
#15 lore raw

RE: [PATCH 7/7] app/flow-perf: add hairpin queue memory config

Hi Dariusz,

Quoted 18 lines

-----Original Message-----
From: Dariusz Sosnowski <[email protected]>
Sent: Monday, September 19, 2022 7:38 PM
To: Wisam Monther <[email protected]>
Cc: [email protected]
Subject: [PATCH 7/7] app/flow-perf: add hairpin queue memory config

This patch adds the hairpin-conf command line parameter to flow-perf application. hairpin-conf parameter takes a hexadecimal bitmask with bits having the following meaning:

- Bit 0 - Force memory settings of hairpin RX queue.
- Bit 1 - Force memory settings of hairpin TX queue.
- Bit 4 - Use locked device memory for hairpin RX queue.
- Bit 5 - Use RTE memory for hairpin RX queue.
- Bit 8 - Use locked device memory for hairpin TX queue.
- Bit 9 - Use RTE memory for hairpin TX queue.

Quoted 2 lines

---

You have some checks issues; can you please kindly check them?

BRs,
Wisam Jaddo

#16 lore raw

RE: [PATCH 7/7] app/flow-perf: add hairpin queue memory config

Hi,

Quoted 27 lines

-----Original Message-----
From: Wisam Monther <[email protected]>
Sent: Tuesday, October 4, 2022 14:25
To: Dariusz Sosnowski <[email protected]>
Cc: [email protected]
Subject: RE: [PATCH 7/7] app/flow-perf: add hairpin queue memory config

Hi Dariusz,

Quoted 18 lines

-----Original Message-----
From: Dariusz Sosnowski <[email protected]>
Sent: Monday, September 19, 2022 7:38 PM
To: Wisam Monther <[email protected]>
Cc: [email protected]
Subject: [PATCH 7/7] app/flow-perf: add hairpin queue memory config

This patch adds the hairpin-conf command line parameter to flow-perf application. hairpin-conf parameter takes a hexadecimal bitmask with bits having the following meaning:

- Bit 0 - Force memory settings of hairpin RX queue.
- Bit 1 - Force memory settings of hairpin TX queue.
- Bit 4 - Use locked device memory for hairpin RX queue.
- Bit 5 - Use RTE memory for hairpin RX queue.
- Bit 8 - Use locked device memory for hairpin TX queue.
- Bit 9 - Use RTE memory for hairpin TX queue.

Quoted 7 lines
Quoted 2 lines

---

You have some checks issues; can you please kindly check them?

BRs,
Wisam Jaddo

I sent the v2. The checks issue should be fixed in that version.

Thank you.

Best regards,
Dariusz Sosnowski

#17 lore raw
In reply to Dariusz Sosnowski

19/09/2022 18:37, Dariusz Sosnowski:

Quoted 5 lines

This patch series introduces hairpin memory configuration options proposed in
http://patches.dpdk.org/project/dpdk/patch/[email protected]/
for Rx and Tx hairpin queues. It also implements handling of these options in mlx5 PMD and allows to use new hairpin options in testpmd (through --hairpin-mode option) and flow-perf (through --hairpin-conf option).

2 things are missing in this series:

1/ motivation (why is this needed)
2/ compilation on Windows
looks like devx_umem_reg has 5 parameters in Windows glue!

#18 lore raw

Hi Thomas,

I sent the v2 of the patches.

Quoted 1 line

1/ motivation (why is this needed)

I added the feature motivation in the cover letter of v2.

Quoted 2 lines

2/ compilation on Windows
looks like devx_umem_reg has 5 parameters in Windows glue!

Fix for that is included in v2.

Best regards,
Dariusz Sosnowski

#19 lore raw

v2 cover ethdev: introduce hairpin memory capabilities

In reply to Dariusz Sosnowski

The hairpin queues are used to transmit packets received on the wire, back to the wire. How hairpin queues are implemented and configured is decided internally by the PMD and applications have no control over the configuration of Rx and Tx hairpin queues. This patchset addresses that by:

- Extending hairpin queue capabilities reported by PMDs.
- Exposing new configuration options for Rx and Tx hairpin queues.

Main goal of this patchset is to allow applications to provide configuration hints regarding memory placement of hairpin queues.
These hints specify whether buffers of hairpin queues should be placed in host memory or in dedicated device memory.

For example, in context of NVIDIA Connect-X and BlueField devices, this distinction is important for several reasons:

- By default, data buffers and packet descriptors are placed in device memory region which is shared with other resources (e.g. flow rules).
This results in memory contention on the device,
which may lead to degraded performance under heavy load.
- Placing hairpin queues in dedicated device memory can decrease latency of hairpinned traffic, since hairpin queue processing will not be memory starved by other operations.
Side effect of this memory configuration is that it leaves less memory for other resources, possibly causing memory contention in non-hairpin traffic.
- Placing hairpin queues in host memory can increase throughput of hairpinned
traffic at the cost of increasing latency.
Each packet processed by hairpin queues will incur additional PCI transactions (increase in latency), but memory contention on the device is avoided.

Depending on the workload and whether throughput or latency has a higher priority for developers, it would be beneficial if developers could choose the best hairpin configuration for their use case.

To address that, this patchset adds the following configuration options (in rte_eth_hairpin_conf struct):

- use_locked_device_memory - If set, PMD will allocate specialized on-device memory for the queue.
- use_rte_memory - If set, PMD will use DPDK-managed memory for the queue.
- force_memory - If set, PMD will be forced to use provided memory configuration. If no appropriate resources are available, the queue allocation will fail.
If unset and no appropriate resources are available, PMD will fallback to its default behavior.

Implementing support for these flags is optional and applications should be allowed to not set any of these new flags. This will result in default memory configuration provided by the PMD.
Application developers should consult the PMD documentation in that case.

These changes were originally proposed in http://patches.dpdk.org/project/dpdk/patch/[email protected]/.

Dariusz Sosnowski (8):
ethdev: introduce hairpin memory capabilities common/mlx5: add hairpin SQ buffer type capabilities common/mlx5: add hairpin RQ buffer type capabilities net/mlx5: allow hairpin Tx queue in RTE memory net/mlx5: allow hairpin Rx queue in locked memory
doc: add notes for hairpin to mlx5 documentation app/testpmd: add hairpin queues memory modes app/flow-perf: add hairpin queue memory config

 app/test-flow-perf/main.c              |  32 +++++
 app/test-pmd/parameters.c              |   2 +-
 app/test-pmd/testpmd.c                 |  24 +++-
 app/test-pmd/testpmd.h                 |   2 +-
 doc/guides/nics/mlx5.rst               |  37 ++++++
 doc/guides/platform/mlx5.rst           |   5 +
 doc/guides/rel_notes/release_22_11.rst |  10 ++
 doc/guides/testpmd_app_ug/run_app.rst  |  10 +-
 drivers/common/mlx5/mlx5_devx_cmds.c   |   8 ++
 drivers/common/mlx5/mlx5_devx_cmds.h   |   5 +
 drivers/common/mlx5/mlx5_prm.h         |  25 +++-
 drivers/net/mlx5/mlx5.h                |   2 +
 drivers/net/mlx5/mlx5_devx.c           | 170 ++++++++++++++++++++++---
 drivers/net/mlx5/mlx5_ethdev.c         |   6 +
 lib/ethdev/rte_ethdev.c                |  44 +++++++
 lib/ethdev/rte_ethdev.h                |  68 +++++++++-
 16 files changed, 422 insertions(+), 28 deletions(-)
Signature · 6 lines
v2:
* Fix Windows build by using mlx5_os_umem_dereg defined on both platforms to allocate memory for Tx hairpin queue.
* Added hairpin section to mlx5 PMD.
* Added info about new hairpin configuration options to DPDK release notes.

2.25.1
#20 lore raw

PATCH v2 1/8 ethdev: introduce hairpin memory capabilities

Before this patch, implementation details and configuration of hairpin queues were decided internally by the PMD. Applications had no control over the configuration of Rx and Tx hairpin queues, despite number of descriptors, explicit Tx flow mode and disabling automatic binding. This patch addresses that by adding:

- Hairpin queue capabilities reported by PMDs.
- New configuration options for Rx and Tx hairpin queues.

Main goal of this patch is to allow applications to provide configuration hints regarding placement of hairpin queues. These hints specify whether buffers of hairpin queues should be placed in host memory or in dedicated device memory. Different memory options may have different performance characteristics and hairpin configuration should be fine-tuned to the specific application and use case.

This patch introduces new hairpin queue configuration options through rte_eth_hairpin_conf struct, allowing to tune Rx and Tx hairpin queues memory configuration. Hairpin configuration is extended with the following fields:

- use_locked_device_memory - If set, PMD will use specialized on-device memory to store RX or TX hairpin queue data.
- use_rte_memory - If set, PMD will use DPDK-managed memory to store RX or TX hairpin queue data.
- force_memory - If set, PMD will be forced to use provided memory settings. If no appropriate resources are available, then device start will fail. If unset and no resources are available, PMD will fallback to using default type of resource for given queue.

If application chooses to use PMD default memory configuration, all of these flags should remain unset.

Hairpin capabilities are also extended, to allow verification of support of given hairpin memory configurations. Struct rte_eth_hairpin_cap is extended with two additional fields of type rte_eth_hairpin_queue_cap:

- rx_cap - memory capabilities of hairpin RX queues.
- tx_cap - memory capabilities of hairpin TX queues.

Struct rte_eth_hairpin_queue_cap exposes whether given queue type supports use_locked_device_memory and use_rte_memory flags.

---
 doc/guides/rel_notes/release_22_11.rst | 10 ++++
 lib/ethdev/rte_ethdev.c                | 44 +++++++++++++++++
 lib/ethdev/rte_ethdev.h                | 68 +++++++++++++++++++++++++-
 3 files changed, 120 insertions(+), 2 deletions(-)
doc/guides/rel_notes/release_22_11.rst +10 -0
index ac67e7e710..e5c48c6b18 100644
@@ -66,6 +66,16 @@ New Features
   Added new function ``rte_flow_async_action_handle_query()``,
   to query the action asynchronously.
 
+* **Added hairpin memory configurations options in ethdev API.**
+
+  Added new configuration flags for hairpin queues in ``rte_eth_hairpin_conf``:
+
+  * ``use_locked_device_memory``
+  * ``use_rte_memory``
+  * ``force_memory``
+
+  Each flag has a corresponding capability flag in ``rte_eth_hairpin_queue_cap`` struct.
+
 * **Updated Intel iavf driver.**
 
   * Added flow subscription support.
lib/ethdev/rte_ethdev.c +44 -0

56 lines not on this page · raw message

lib/ethdev/rte_ethdev.h +66 -2

89 lines not on this page · raw message

Signature · 1 line
2.25.1
#21 lore raw

PATCH v2 2/8 common/mlx5: add hairpin SQ buffer type capabilities

In reply to Dariusz Sosnowski

This patch extends HCA_CAP and SQ Context structs available in PRM. This fields allow checking if NIC supports storing hairpin SQ's WQ buffer in host memory and configuring such memory placement.

HCA capabilities are extended with the following fields:

- hairpin_sq_wq_in_host_mem - If set, then NIC supports using host memory as a backing storage for hairpin SQ's WQ buffer.
- hairpin_sq_wqe_bb_size - Indicates the required size of SQ WQE basic block.

SQ Context is extended with hairpin_wq_buffer_type which informs NIC where SQ's WQ buffer will be stored. This field can take the following values:

- MLX5_SQC_HAIRPIN_WQ_BUFFER_TYPE_INTERNAL_BUFFER - WQ buffer will be stored in unlocked device memory.
- MLX5_SQC_HAIRPIN_WQ_BUFFER_TYPE_HOST_MEMORY - WQ buffer will be stored in host memory. Buffer is provided by PMD.

---
 drivers/common/mlx5/mlx5_devx_cmds.c |  5 +++++
 drivers/common/mlx5/mlx5_devx_cmds.h |  3 +++
 drivers/common/mlx5/mlx5_prm.h       | 15 +++++++++++++--
 3 files changed, 21 insertions(+), 2 deletions(-)
drivers/common/mlx5/mlx5_devx_cmds.c +5 -0
index fb33023138..a1e8179568 100644
@@ -989,6 +989,10 @@ mlx5_devx_cmd_query_hca_attr(void *ctx,
 		}
 		attr->log_min_stride_wqe_sz = MLX5_GET(cmd_hca_cap_2, hcattr,
 						       log_min_stride_wqe_sz);
+		attr->hairpin_sq_wqe_bb_size = MLX5_GET(cmd_hca_cap_2, hcattr,
+							hairpin_sq_wqe_bb_size);
+		attr->hairpin_sq_wq_in_host_mem = MLX5_GET(cmd_hca_cap_2, hcattr,
+							   hairpin_sq_wq_in_host_mem);
 	}
 	if (attr->log_min_stride_wqe_sz == 0)
 		attr->log_min_stride_wqe_sz = MLX5_MPRQ_LOG_MIN_STRIDE_WQE_SIZE;
@@ -1706,6 +1710,7 @@ mlx5_devx_cmd_create_sq(void *ctx,
 	MLX5_SET(sqc, sq_ctx, hairpin, sq_attr->hairpin);
 	MLX5_SET(sqc, sq_ctx, non_wire, sq_attr->non_wire);
 	MLX5_SET(sqc, sq_ctx, static_sq_wq, sq_attr->static_sq_wq);
+	MLX5_SET(sqc, sq_ctx, hairpin_wq_buffer_type, sq_attr->hairpin_wq_buffer_type);
 	MLX5_SET(sqc, sq_ctx, user_index, sq_attr->user_index);
 	MLX5_SET(sqc, sq_ctx, cqn, sq_attr->cqn);
 	MLX5_SET(sqc, sq_ctx, packet_pacing_rate_limit_index,
drivers/common/mlx5/mlx5_devx_cmds.h +3 -0
index af6053a788..9ac2d75df4 100644
@@ -191,6 +191,8 @@ struct mlx5_hca_attr {
 	uint32_t log_max_hairpin_queues:5;
 	uint32_t log_max_hairpin_wq_data_sz:5;
 	uint32_t log_max_hairpin_num_packets:5;
+	uint32_t hairpin_sq_wqe_bb_size:4;
+	uint32_t hairpin_sq_wq_in_host_mem:1;
 	uint32_t vhca_id:16;
 	uint32_t relaxed_ordering_write:1;
 	uint32_t relaxed_ordering_read:1;
@@ -407,6 +409,7 @@ struct mlx5_devx_create_sq_attr {
 	uint32_t non_wire:1;
 	uint32_t static_sq_wq:1;
 	uint32_t ts_format:2;
+	uint32_t hairpin_wq_buffer_type:3;
 	uint32_t user_index:24;
 	uint32_t cqn:24;
 	uint32_t packet_pacing_rate_limit_index:16;
drivers/common/mlx5/mlx5_prm.h +13 -2
index 4346279c81..04d35ca845 100644
@@ -2020,7 +2020,11 @@ struct mlx5_ifc_cmd_hca_cap_2_bits {
 	u8 reserved_at_d8[0x3];
 	u8 log_max_conn_track_offload[0x5];
 	u8 reserved_at_e0[0x20]; /* End of DW7. */
-	u8 reserved_at_100[0x700];
+	u8 reserved_at_100[0x60];
+	u8 reserved_at_160[0x3];
+	u8 hairpin_sq_wqe_bb_size[0x5];
+	u8 hairpin_sq_wq_in_host_mem[0x1];
+	u8 reserved_at_169[0x697];
 };
 
 struct mlx5_ifc_esw_cap_bits {
@@ -2673,6 +2677,11 @@ enum {
 	MLX5_SQC_STATE_ERR  = 0x3,
 };
 
+enum {
+	MLX5_SQC_HAIRPIN_WQ_BUFFER_TYPE_INTERNAL_BUFFER = 0x0,
+	MLX5_SQC_HAIRPIN_WQ_BUFFER_TYPE_HOST_MEMORY = 0x1,
+};
+
 struct mlx5_ifc_sqc_bits {
 	u8 rlky[0x1];
 	u8 cd_master[0x1];
@@ -2686,7 +2695,9 @@ struct mlx5_ifc_sqc_bits {
 	u8 hairpin[0x1];
 	u8 non_wire[0x1];
 	u8 static_sq_wq[0x1];
-	u8 reserved_at_11[0x9];
+	u8 reserved_at_11[0x4];
+	u8 hairpin_wq_buffer_type[0x3];
+	u8 reserved_at_18[0x2];
 	u8 ts_format[0x02];
 	u8 reserved_at_1c[0x4];
 	u8 reserved_at_20[0x8];
Signature · 1 line
2.25.1
#22 lore raw

PATCH v2 3/8 common/mlx5: add hairpin RQ buffer type capabilities

In reply to Dariusz Sosnowski

This patch adds new HCA capability related to hairpin RQs. This new capability, hairpin_data_buffer_locked, indicates whether HCA supports locking data buffer of hairpin RQ in ICMC (Interconnect Context Memory Cache).

Struct used to define RQ configuration (RQ context) is extended with hairpin_data_buffer_type field, which configures data buffer for hairpin RQ. It can take the following values:

- MLX5_RQC_HAIRPIN_DATA_BUFFER_TYPE_UNLOCKED_INTERNAL_BUFFER - hairpin RQ's data buffer is stored in unlocked memory in ICMC.
- MLX5_RQC_HAIRPIN_DATA_BUFFER_TYPE_LOCKED_INTERNAL_BUFFER - hairpin RQ's data buffer is stored in locked memory in ICMC.

---
 drivers/common/mlx5/mlx5_devx_cmds.c |  3 +++
 drivers/common/mlx5/mlx5_devx_cmds.h |  2 ++
 drivers/common/mlx5/mlx5_prm.h       | 12 ++++++++++--
 3 files changed, 15 insertions(+), 2 deletions(-)
drivers/common/mlx5/mlx5_devx_cmds.c +3 -0
index a1e8179568..76f0b6724f 100644
@@ -993,6 +993,8 @@ mlx5_devx_cmd_query_hca_attr(void *ctx,
 							hairpin_sq_wqe_bb_size);
 		attr->hairpin_sq_wq_in_host_mem = MLX5_GET(cmd_hca_cap_2, hcattr,
 							   hairpin_sq_wq_in_host_mem);
+		attr->hairpin_data_buffer_locked = MLX5_GET(cmd_hca_cap_2, hcattr,
+							    hairpin_data_buffer_locked);
 	}
 	if (attr->log_min_stride_wqe_sz == 0)
 		attr->log_min_stride_wqe_sz = MLX5_MPRQ_LOG_MIN_STRIDE_WQE_SIZE;
@@ -1293,6 +1295,7 @@ mlx5_devx_cmd_create_rq(void *ctx,
 	MLX5_SET(rqc, rq_ctx, state, rq_attr->state);
 	MLX5_SET(rqc, rq_ctx, flush_in_error_en, rq_attr->flush_in_error_en);
 	MLX5_SET(rqc, rq_ctx, hairpin, rq_attr->hairpin);
+	MLX5_SET(rqc, rq_ctx, hairpin_data_buffer_type, rq_attr->hairpin_data_buffer_type);
 	MLX5_SET(rqc, rq_ctx, user_index, rq_attr->user_index);
 	MLX5_SET(rqc, rq_ctx, cqn, rq_attr->cqn);
 	MLX5_SET(rqc, rq_ctx, counter_set_id, rq_attr->counter_set_id);
drivers/common/mlx5/mlx5_devx_cmds.h +2 -0
index 9ac2d75df4..cceaf3411d 100644
@@ -193,6 +193,7 @@ struct mlx5_hca_attr {
 	uint32_t log_max_hairpin_num_packets:5;
 	uint32_t hairpin_sq_wqe_bb_size:4;
 	uint32_t hairpin_sq_wq_in_host_mem:1;
+	uint32_t hairpin_data_buffer_locked:1;
 	uint32_t vhca_id:16;
 	uint32_t relaxed_ordering_write:1;
 	uint32_t relaxed_ordering_read:1;
@@ -313,6 +314,7 @@ struct mlx5_devx_create_rq_attr {
 	uint32_t state:4;
 	uint32_t flush_in_error_en:1;
 	uint32_t hairpin:1;
+	uint32_t hairpin_data_buffer_type:3;
 	uint32_t ts_format:2;
 	uint32_t user_index:24;
 	uint32_t cqn:24;
drivers/common/mlx5/mlx5_prm.h +10 -2
index 04d35ca845..9c1c93f916 100644
@@ -2024,7 +2024,8 @@ struct mlx5_ifc_cmd_hca_cap_2_bits {
 	u8 reserved_at_160[0x3];
 	u8 hairpin_sq_wqe_bb_size[0x5];
 	u8 hairpin_sq_wq_in_host_mem[0x1];
-	u8 reserved_at_169[0x697];
+	u8 hairpin_data_buffer_locked[0x1];
+	u8 reserved_at_16a[0x696];
 };
 
 struct mlx5_ifc_esw_cap_bits {
@@ -2304,7 +2305,9 @@ struct mlx5_ifc_rqc_bits {
 	u8 reserved_at_c[0x1];
 	u8 flush_in_error_en[0x1];
 	u8 hairpin[0x1];
-	u8 reserved_at_f[0xB];
+	u8 reserved_at_f[0x6];
+	u8 hairpin_data_buffer_type[0x3];
+	u8 reserved_at_a8[0x2];
 	u8 ts_format[0x02];
 	u8 reserved_at_1c[0x4];
 	u8 reserved_at_20[0x8];
@@ -2813,6 +2816,11 @@ enum {
 	MLX5_CQE_SIZE_128B = 0x1,
 };
 
+enum {
+	MLX5_RQC_HAIRPIN_DATA_BUFFER_TYPE_UNLOCKED_INTERNAL_BUFFER = 0x0,
+	MLX5_RQC_HAIRPIN_DATA_BUFFER_TYPE_LOCKED_INTERNAL_BUFFER = 0x1,
+};
+
 struct mlx5_ifc_cqc_bits {
 	u8 status[0x4];
 	u8 as_notify[0x1];
Signature · 1 line
2.25.1
#23 lore raw

PATCH v2 4/8 net/mlx5: allow hairpin Tx queue in RTE memory

In reply to Dariusz Sosnowski

This patch adds a capability to place hairpin Tx queue in host memory managed by DPDK. This capability is equivalent to storing hairpin SQ's WQ buffer in host memory.

Hairpin Tx queue creation is extended with allocating a memory buffer of proper size (calculated from required number of packets and WQE BB size advertised in HCA capabilities).

force_memory flag of hairpin queue configuration is also supported. If it is set and:

- allocation of memory buffer fails,
- or hairpin SQ creation fails,

then device start will fail. If it is unset, PMD will fallback to creating the hairpin SQ with WQ buffer located in unlocked device memory.

---
 drivers/net/mlx5/mlx5.h        |   2 +
 drivers/net/mlx5/mlx5_devx.c   | 119 ++++++++++++++++++++++++++++++---
 drivers/net/mlx5/mlx5_ethdev.c |   4 ++
 3 files changed, 116 insertions(+), 9 deletions(-)
drivers/net/mlx5/mlx5.h +2 -0
index 95ecbea39e..3c9e6bad53 100644
@@ -1386,6 +1386,8 @@ struct mlx5_txq_obj {
 			struct mlx5_devx_obj *sq;
 			/* DevX object for Sx queue. */
 			struct mlx5_devx_obj *tis; /* The TIS object. */
+			void *umem_buf_wq_buffer;
+			void *umem_obj_wq_buffer;
 		};
 		struct {
 			struct rte_eth_dev *dev;
drivers/net/mlx5/mlx5_devx.c +110 -9

155 lines not on this page · raw message

drivers/net/mlx5/mlx5_ethdev.c +4 -0
index a5c7ca8c52..c59005ea2b 100644
@@ -729,6 +729,7 @@ int
 mlx5_hairpin_cap_get(struct rte_eth_dev *dev, struct rte_eth_hairpin_cap *cap)
 {
 	struct mlx5_priv *priv = dev->data->dev_private;
+	struct mlx5_hca_attr *hca_attr;
 
 	if (!mlx5_devx_obj_ops_en(priv->sh)) {
 		rte_errno = ENOTSUP;
@@ -738,5 +739,8 @@ mlx5_hairpin_cap_get(struct rte_eth_dev *dev, struct rte_eth_hairpin_cap *cap)
 	cap->max_rx_2_tx = 1;
 	cap->max_tx_2_rx = 1;
 	cap->max_nb_desc = 8192;
+	hca_attr = &priv->sh->cdev->config.hca_attr;
+	cap->tx_cap.locked_device_memory = 0;
+	cap->tx_cap.rte_memory = hca_attr->hairpin_sq_wq_in_host_mem;
 	return 0;
 }
Signature · 1 line
2.25.1
#24 lore raw

PATCH v2 5/8 net/mlx5: allow hairpin Rx queue in locked memory

In reply to Dariusz Sosnowski

This patch adds a capability to place hairpin Rx queue in locked device memory. This capability is equivalent to storing hairpin RQ's data buffers in locked internal device memory.

Hairpin Rx queue creation is extended with requesting that RQ is allocated in locked internal device memory. If allocation fails and force_memory hairpin configuration is set, then hairpin queue creation (and, as a result, device start) fails. If force_memory is unset, then PMD will fallback to allocating memory for hairpin RQ in unlocked internal device memory.

To allow such allocation, the user must set HAIRPIN_DATA_BUFFER_LOCK flag in FW using mlxconfig tool.

---
 doc/guides/platform/mlx5.rst   |  5 ++++
 drivers/net/mlx5/mlx5_devx.c   | 51 ++++++++++++++++++++++++++++------
 drivers/net/mlx5/mlx5_ethdev.c |  2 ++
 3 files changed, 49 insertions(+), 9 deletions(-)
doc/guides/platform/mlx5.rst +5 -0
index 46b394c4ee..3cc1dd29e2 100644
@@ -555,6 +555,11 @@ Below are some firmware configurations listed.
 
    REAL_TIME_CLOCK_ENABLE=1
 
+- allow locking hairpin RQ data buffer in device memory::
+
+   HAIRPIN_DATA_BUFFER_LOCK=1
+   MEMIC_SIZE_LIMIT=0
+
 
 .. _mlx5_common_driver_options:
 
drivers/net/mlx5/mlx5_devx.c +42 -9

82 lines not on this page · raw message

drivers/net/mlx5/mlx5_ethdev.c +2 -0
index c59005ea2b..4a85415ff3 100644
@@ -740,6 +740,8 @@ mlx5_hairpin_cap_get(struct rte_eth_dev *dev, struct rte_eth_hairpin_cap *cap)
 	cap->max_tx_2_rx = 1;
 	cap->max_nb_desc = 8192;
 	hca_attr = &priv->sh->cdev->config.hca_attr;
+	cap->rx_cap.locked_device_memory = hca_attr->hairpin_data_buffer_locked;
+	cap->rx_cap.rte_memory = 0;
 	cap->tx_cap.locked_device_memory = 0;
 	cap->tx_cap.rte_memory = hca_attr->hairpin_sq_wq_in_host_mem;
 	return 0;
Signature · 1 line
2.25.1
#25 lore raw

PATCH v2 6/8 doc: add notes for hairpin to mlx5 documentation

In reply to Dariusz Sosnowski

This patch extends mlx5 PMD documentation with more information regarding hairpin support.

The following is added to mlx5 PMD documentation:

- description of the default behavior of hairpin queues,
- description of use_locked_device_memory effect on hairpin queue configuration,
- description of use_rte_memory effect on hairpin queue configuration,
- DPDK and OFED requirements for new memory options for hairpin.

---
 doc/guides/nics/mlx5.rst | 37 +++++++++++++++++++++++++++++++++++++
 1 file changed, 37 insertions(+)
doc/guides/nics/mlx5.rst +37 -0
index 3d4ee31f8d..997cb19ba2 100644
@@ -1517,6 +1517,43 @@ behavior as librte_net_mlx4::
    > port config all rss all
    > port start all
 
+Notes for hairpin
+-----------------
+
+NVIDIA Connect-X and BlueField devices support specifying memory
+placement for hairpin Rx and Tx queues. This feature requires OFED 5.8.
+
+By default, data buffers and packet descriptors for hairpin queues are placed
+in device memory which is shared with other resources (e.g. flow rules).
+
+Starting with DPDK 22.11 and OFED 5.8 applications are allowed to:
+
+#. Place data buffers and Rx packet descriptors in dedicated device memory.
+   Application can request that configuration through ``use_locked_device_memory``
+   configuration option.
+
+   Placing data buffers and Rx packet descriptors in dedicated device memory
+   can decrease latency on hairpinned traffic, since traffic processing
+   for the hairpin queue will not be memory starved.
+
+   However, reserving device memory for hairpin Rx queues may decrease throughput
+   under heavy load, since less resources will be available on device.
+
+   This option is supported only for Rx hairpin queues.
+
+#. Place Tx packet descriptors in host memory.
+   Application can request that configuration through ``use_rte_memory``
+   configuration option.
+
+   Placing Tx packet descritors in host memory can increase traffic throughput.
+   This results in more resources available on the device for other purposes,
+   which reduces memory contention on device.
+   Side effect of this option is visible increase in latency, since each packet
+   incurs additional PCI transactions.
+
+   This option is supported only for Tx hairpin queues.
+
+
 Usage example
 -------------
 
Signature · 1 line
2.25.1
#26 lore raw

PATCH v2 7/8 app/testpmd: add hairpin queues memory modes

In reply to Dariusz Sosnowski

This patch extends hairpin-mode command line option of test-pmd application with an ability to configure whether Rx/Tx hairpin queue should use locked device memory or RTE memory.

For purposes of this configurations the following bits of 32 bit hairpin-mode are reserved:

- Bit 8 - If set, then force_memory flag will be set for hairpin RX queue.
- Bit 9 - If set, then force_memory flag will be set for hairpin TX queue.
- Bits 12-15 - Memory options for hairpin Rx queue:
- Bit 12 - If set, then use_locked_device_memory will be set.
- Bit 13 - If set, then use_rte_memory will be set.
- Bit 14 - Reserved for future use.
- Bit 15 - Reserved for future use.
- Bits 16-19 - Memory options for hairpin Tx queue:
- Bit 16 - If set, then use_locked_device_memory will be set.
- Bit 17 - If set, then use_rte_memory will be set.
- Bit 18 - Reserved for future use.
- Bit 19 - Reserved for future use.

---
 app/test-pmd/parameters.c             |  2 +-
 app/test-pmd/testpmd.c                | 24 +++++++++++++++++++++++-
 app/test-pmd/testpmd.h                |  2 +-
 doc/guides/testpmd_app_ug/run_app.rst | 10 ++++++++--
 4 files changed, 33 insertions(+), 5 deletions(-)
app/test-pmd/parameters.c +1 -1
index 1024b5419c..14752f9571 100644
@@ -1085,7 +1085,7 @@ launch_args_parse(int argc, char** argv)
 				if (errno != 0 || end == optarg)
 					rte_exit(EXIT_FAILURE, "hairpin mode invalid\n");
 				else
-					hairpin_mode = (uint16_t)n;
+					hairpin_mode = (uint32_t)n;
 			}
 			if (!strcmp(lgopts[opt_idx].name, "burst")) {
 				n = atoi(optarg);
app/test-pmd/testpmd.c +23 -1
index 39ee3d331d..bb1c901742 100644
@@ -409,7 +409,7 @@ bool setup_on_probe_event = true;
 uint8_t clear_ptypes = true;
 
 /* Hairpin ports configuration mode. */
-uint16_t hairpin_mode;
+uint32_t hairpin_mode;
 
 /* Pretty printing of ethdev events */
 static const char * const eth_event_desc[] = {
@@ -2519,6 +2519,16 @@ port_is_started(portid_t port_id)
 	return 1;
 }
 
+#define HAIRPIN_MODE_RX_FORCE_MEMORY RTE_BIT32(8)
+#define HAIRPIN_MODE_TX_FORCE_MEMORY RTE_BIT32(9)
+
+#define HAIRPIN_MODE_RX_LOCKED_MEMORY RTE_BIT32(12)
+#define HAIRPIN_MODE_RX_RTE_MEMORY RTE_BIT32(13)
+
+#define HAIRPIN_MODE_TX_LOCKED_MEMORY RTE_BIT32(16)
+#define HAIRPIN_MODE_TX_RTE_MEMORY RTE_BIT32(17)
+
+
 /* Configure the Rx and Tx hairpin queues for the selected port. */
 static int
 setup_hairpin_queues(portid_t pi, portid_t p_pi, uint16_t cnt_pi)
@@ -2534,6 +2544,12 @@ setup_hairpin_queues(portid_t pi, portid_t p_pi, uint16_t cnt_pi)
 	uint16_t peer_tx_port = pi;
 	uint32_t manual = 1;
 	uint32_t tx_exp = hairpin_mode & 0x10;
+	uint32_t rx_force_memory = hairpin_mode & HAIRPIN_MODE_RX_FORCE_MEMORY;
+	uint32_t rx_locked_memory = hairpin_mode & HAIRPIN_MODE_RX_LOCKED_MEMORY;
+	uint32_t rx_rte_memory = hairpin_mode & HAIRPIN_MODE_RX_RTE_MEMORY;
+	uint32_t tx_force_memory = hairpin_mode & HAIRPIN_MODE_TX_FORCE_MEMORY;
+	uint32_t tx_locked_memory = hairpin_mode & HAIRPIN_MODE_TX_LOCKED_MEMORY;
+	uint32_t tx_rte_memory = hairpin_mode & HAIRPIN_MODE_TX_RTE_MEMORY;
 
 	if (!(hairpin_mode & 0xf)) {
 		peer_rx_port = pi;
@@ -2573,6 +2589,9 @@ setup_hairpin_queues(portid_t pi, portid_t p_pi, uint16_t cnt_pi)
 		hairpin_conf.peers[0].queue = i + nb_rxq;
 		hairpin_conf.manual_bind = !!manual;
 		hairpin_conf.tx_explicit = !!tx_exp;
+		hairpin_conf.force_memory = !!tx_force_memory;
+		hairpin_conf.use_locked_device_memory = !!tx_locked_memory;
+		hairpin_conf.use_rte_memory = !!tx_rte_memory;
 		diag = rte_eth_tx_hairpin_queue_setup
 			(pi, qi, nb_txd, &hairpin_conf);
 		i++;
@@ -2596,6 +2615,9 @@ setup_hairpin_queues(portid_t pi, portid_t p_pi, uint16_t cnt_pi)
 		hairpin_conf.peers[0].queue = i + nb_txq;
 		hairpin_conf.manual_bind = !!manual;
 		hairpin_conf.tx_explicit = !!tx_exp;
+		hairpin_conf.force_memory = !!rx_force_memory;
+		hairpin_conf.use_locked_device_memory = !!rx_locked_memory;
+		hairpin_conf.use_rte_memory = !!rx_rte_memory;
 		diag = rte_eth_rx_hairpin_queue_setup
 			(pi, qi, nb_rxd, &hairpin_conf);
 		i++;
app/test-pmd/testpmd.h +1 -1
index 627a42ce3b..2244c25e97 100644
@@ -562,7 +562,7 @@ extern uint16_t stats_period;
 extern struct rte_eth_xstat_name *xstats_display;
 extern unsigned int xstats_display_num;
 
-extern uint16_t hairpin_mode;
+extern uint32_t hairpin_mode;
 
 #ifdef RTE_LIB_LATENCYSTATS
 extern uint8_t latencystats_enabled;
doc/guides/testpmd_app_ug/run_app.rst +8 -2
index 8b41b960c8..abc3ec10a0 100644
@@ -529,10 +529,16 @@ The command line options are:
 
     Enable display of RX and TX burst stats.
 
-*   ``--hairpin-mode=0xXX``
+*   ``--hairpin-mode=0xXXXX``
 
-    Set the hairpin port mode with bitmask, only valid when hairpin queues number is set::
+    Set the hairpin port configuration with bitmask, only valid when hairpin queues number is set::
 
+	bit 18 - hairpin TX queues will use RTE memory
+	bit 16 - hairpin TX queues will use locked device memory
+	bit 13 - hairpin RX queues will use RTE memory
+	bit 12 - hairpin RX queues will use locked device memory
+	bit 9 - force memory settings of hairpin TX queue
+	bit 8 - force memory settings of hairpin RX queue
 	bit 4 - explicit Tx flow rule
 	bit 1 - two hairpin ports paired
 	bit 0 - two hairpin ports loop
Signature · 1 line
2.25.1
#27 lore raw

PATCH v2 8/8 app/flow-perf: add hairpin queue memory config

In reply to Dariusz Sosnowski

This patch adds the hairpin-conf command line parameter to flow-perf application. hairpin-conf parameter takes a hexadecimal bitmask with bits having the following meaning:

- Bit 0 - Force memory settings of hairpin RX queue.
- Bit 1 - Force memory settings of hairpin TX queue.
- Bit 4 - Use locked device memory for hairpin RX queue.
- Bit 5 - Use RTE memory for hairpin RX queue.
- Bit 8 - Use locked device memory for hairpin TX queue.
- Bit 9 - Use RTE memory for hairpin TX queue.

---
 app/test-flow-perf/main.c | 32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)
app/test-flow-perf/main.c +32 -0
index f375097028..4a9206803a 100644
@@ -46,6 +46,15 @@
 #define DEFAULT_RULES_BATCH     100000
 #define DEFAULT_GROUP                0
 
+#define HAIRPIN_RX_CONF_FORCE_MEMORY  (0x0001)
+#define HAIRPIN_TX_CONF_FORCE_MEMORY  (0x0002)
+
+#define HAIRPIN_RX_CONF_LOCKED_MEMORY (0x0010)
+#define HAIRPIN_RX_CONF_RTE_MEMORY    (0x0020)
+
+#define HAIRPIN_TX_CONF_LOCKED_MEMORY (0x0100)
+#define HAIRPIN_TX_CONF_RTE_MEMORY    (0x0200)
+
 struct rte_flow *flow;
 static uint8_t flow_group;
 
@@ -61,6 +70,7 @@ static uint32_t policy_id[MAX_PORTS];
 static uint8_t items_idx, actions_idx, attrs_idx;
 
 static uint64_t ports_mask;
+static uint64_t hairpin_conf_mask;
 static uint16_t dst_ports[RTE_MAX_ETHPORTS];
 static volatile bool force_quit;
 static bool dump_iterations;
@@ -482,6 +492,7 @@ usage(char *progname)
 	printf("  --enable-fwd: To enable packets forwarding"
 		" after insertion\n");
 	printf("  --portmask=N: hexadecimal bitmask of ports used\n");
+	printf("  --hairpin-conf=0xXXXX: hexadecimal bitmask of hairpin queue configuration\n");
 	printf("  --random-priority=N,S: use random priority levels "
 		"from 0 to (N - 1) for flows "
 		"and S as seed for pseudo-random number generator\n");
@@ -629,6 +640,7 @@ static void
 args_parse(int argc, char **argv)
 {
 	uint64_t pm, seed;
+	uint64_t hp_conf;
 	char **argvopt;
 	uint32_t prio;
 	char *token;
@@ -648,6 +660,7 @@ args_parse(int argc, char **argv)
 		{ "enable-fwd",                 0, 0, 0 },
 		{ "unique-data",                0, 0, 0 },
 		{ "portmask",                   1, 0, 0 },
+		{ "hairpin-conf",               1, 0, 0 },
 		{ "cores",                      1, 0, 0 },
 		{ "random-priority",            1, 0, 0 },
 		{ "meter-profile-alg",          1, 0, 0 },
@@ -880,6 +893,13 @@ args_parse(int argc, char **argv)
 					rte_exit(EXIT_FAILURE, "Invalid fwd port mask\n");
 				ports_mask = pm;
 			}
+			if (strcmp(lgopts[opt_idx].name, "hairpin-conf") == 0) {
+				end = NULL;
+				hp_conf = strtoull(optarg, &end, 16);
+				if ((optarg[0] == '\0') || (end == NULL) || (*end != '\0'))
+					rte_exit(EXIT_FAILURE, "Invalid hairpin config mask\n");
+				hairpin_conf_mask = hp_conf;
+			}
 			if (strcmp(lgopts[opt_idx].name,
 					"port-id") == 0) {
 				uint16_t port_idx = 0;
@@ -2035,6 +2055,12 @@ init_port(void)
 				hairpin_conf.peers[0].port = port_id;
 				hairpin_conf.peers[0].queue =
 					std_queue + tx_queues_count;
+				hairpin_conf.use_locked_device_memory =
+					!!(hairpin_conf_mask & HAIRPIN_RX_CONF_LOCKED_MEMORY);
+				hairpin_conf.use_rte_memory =
+					!!(hairpin_conf_mask & HAIRPIN_RX_CONF_RTE_MEMORY);
+				hairpin_conf.force_memory =
+					!!(hairpin_conf_mask & HAIRPIN_RX_CONF_FORCE_MEMORY);
 				ret = rte_eth_rx_hairpin_queue_setup(
 						port_id, hairpin_queue,
 						rxd_count, &hairpin_conf);
@@ -2050,6 +2076,12 @@ init_port(void)
 				hairpin_conf.peers[0].port = port_id;
 				hairpin_conf.peers[0].queue =
 					std_queue + rx_queues_count;
+				hairpin_conf.use_locked_device_memory =
+					!!(hairpin_conf_mask & HAIRPIN_TX_CONF_LOCKED_MEMORY);
+				hairpin_conf.use_rte_memory =
+					!!(hairpin_conf_mask & HAIRPIN_TX_CONF_RTE_MEMORY);
+				hairpin_conf.force_memory =
+					!!(hairpin_conf_mask & HAIRPIN_TX_CONF_FORCE_MEMORY);
 				ret = rte_eth_tx_hairpin_queue_setup(
 						port_id, hairpin_queue,
 						txd_count, &hairpin_conf);
Signature · 1 line
2.25.1
Acked-by on v2 8/8
#28 lore raw

RE: [PATCH v2 8/8] app/flow-perf: add hairpin queue memory config

Hi,

Quoted 11 lines

-----Original Message-----
From: Dariusz Sosnowski <[email protected]>
Sent: Thursday, October 6, 2022 2:01 PM
To: Wisam Monther <[email protected]>
Cc: [email protected]
Subject: [PATCH v2 8/8] app/flow-perf: add hairpin queue memory config

This patch adds the hairpin-conf command line parameter to flow-perf application. hairpin-conf parameter takes a hexadecimal bitmask with bits having the following meaning:

Quoted 8 lines

- Bit 0 - Force memory settings of hairpin RX queue.
- Bit 1 - Force memory settings of hairpin TX queue.
- Bit 4 - Use locked device memory for hairpin RX queue.
- Bit 5 - Use RTE memory for hairpin RX queue.
- Bit 8 - Use locked device memory for hairpin TX queue.
- Bit 9 - Use RTE memory for hairpin TX queue.

BRs,
Wisam Jaddo

#29 lore raw
In reply to Dariusz Sosnowski

06/10/2022 13:00, Dariusz Sosnowski:

Quoted 46 lines

The hairpin queues are used to transmit packets received on the wire, back to the wire. How hairpin queues are implemented and configured is decided internally by the PMD and applications have no control over the configuration of Rx and Tx hairpin queues. This patchset addresses that by:

- Extending hairpin queue capabilities reported by PMDs.
- Exposing new configuration options for Rx and Tx hairpin queues.

Main goal of this patchset is to allow applications to provide configuration hints regarding memory placement of hairpin queues.
These hints specify whether buffers of hairpin queues should be placed in host memory or in dedicated device memory.

For example, in context of NVIDIA Connect-X and BlueField devices, this distinction is important for several reasons:

- By default, data buffers and packet descriptors are placed in device memory region which is shared with other resources (e.g. flow rules).
This results in memory contention on the device,
which may lead to degraded performance under heavy load.
- Placing hairpin queues in dedicated device memory can decrease latency of hairpinned traffic, since hairpin queue processing will not be memory starved by other operations.
Side effect of this memory configuration is that it leaves less memory for other resources, possibly causing memory contention in non-hairpin traffic.
- Placing hairpin queues in host memory can increase throughput of hairpinned
traffic at the cost of increasing latency.
Each packet processed by hairpin queues will incur additional PCI transactions (increase in latency), but memory contention on the device is avoided.

Depending on the workload and whether throughput or latency has a higher priority for developers, it would be beneficial if developers could choose the best hairpin configuration for their use case.

To address that, this patchset adds the following configuration options (in rte_eth_hairpin_conf struct):

- use_locked_device_memory - If set, PMD will allocate specialized on-device memory for the queue.
- use_rte_memory - If set, PMD will use DPDK-managed memory for the queue.
- force_memory - If set, PMD will be forced to use provided memory configuration. If no appropriate resources are available, the queue allocation will fail.
If unset and no appropriate resources are available, PMD will fallback to its default behavior.

Implementing support for these flags is optional and applications should be allowed to not set any of these new flags. This will result in default memory configuration provided by the PMD.
Application developers should consult the PMD documentation in that case.

These changes were originally proposed in http://patches.dpdk.org/project/dpdk/patch/[email protected]/.

Quoted 9 lines

Dariusz Sosnowski (8):
ethdev: introduce hairpin memory capabilities common/mlx5: add hairpin SQ buffer type capabilities common/mlx5: add hairpin RQ buffer type capabilities net/mlx5: allow hairpin Tx queue in RTE memory net/mlx5: allow hairpin Rx queue in locked memory
doc: add notes for hairpin to mlx5 documentation app/testpmd: add hairpin queues memory modes app/flow-perf: add hairpin queue memory config

Doc squashed in mlx5 commits.
Applied, thanks.