ethdev: introduce hairpin memory capabilities
cover ethdev: introduce hairpin memory capabilities
This patch series introduces hairpin memory configuration options proposed in
http://patches.dpdk.org/project/dpdk/patch/[email protected]/
for Rx and Tx hairpin queues. It also implements handling of these options in mlx5 PMD and allows to use new hairpin options in testpmd (through --hairpin-mode option) and flow-perf (through --hairpin-conf option).
Dariusz Sosnowski (7):
ethdev: introduce hairpin memory capabilities common/mlx5: add hairpin SQ buffer type capabilities common/mlx5: add hairpin RQ buffer type capabilities net/mlx5: allow hairpin Tx queue in RTE memory net/mlx5: allow hairpin Rx queue in locked memory app/testpmd: add hairpin queues memory modes app/flow-perf: add hairpin queue memory config
app/test-flow-perf/main.c | 32 +++++ app/test-pmd/parameters.c | 2 +- app/test-pmd/testpmd.c | 24 +++- app/test-pmd/testpmd.h | 2 +- doc/guides/platform/mlx5.rst | 5 + doc/guides/testpmd_app_ug/run_app.rst | 10 +- drivers/common/mlx5/mlx5_devx_cmds.c | 8 ++ drivers/common/mlx5/mlx5_devx_cmds.h | 5 + drivers/common/mlx5/mlx5_prm.h | 25 +++- drivers/net/mlx5/mlx5.h | 2 + drivers/net/mlx5/mlx5_devx.c | 170 +++++++++++++++++++++++--- drivers/net/mlx5/mlx5_ethdev.c | 6 + lib/ethdev/rte_ethdev.c | 44 +++++++ lib/ethdev/rte_ethdev.h | 65 +++++++++- 14 files changed, 373 insertions(+), 27 deletions(-)
Signature · 1 line
2.25.1
PATCH 1/7 ethdev: introduce hairpin memory capabilities
This patch introduces new hairpin queue configuration options through rte_eth_hairpin_conf struct, allowing to tune Rx and Tx hairpin queues memory configuration. Hairpin configuration is extended with the following fields:
- use_locked_device_memory - If set, PMD will use specialized on-device memory to store RX or TX hairpin queue data.
- use_rte_memory - If set, PMD will use DPDK-managed memory to store RX or TX hairpin queue data.
- force_memory - If set, PMD will be forced to use provided memory settings. If no appropriate resources are available, then device start will fail. If unset and no resources are available, PMD will fallback to using default type of resource for given queue.
Hairpin capabilities are also extended, to allow verification of support of given hairpin memory configurations. Struct rte_eth_hairpin_cap is extended with two additional fields of type rte_eth_hairpin_queue_cap:
- rx_cap - memory capabilities of hairpin RX queues.
- tx_cap - memory capabilities of hairpin TX queues.
Struct rte_eth_hairpin_queue_cap exposes whether given queue type supports use_locked_device_memory and use_rte_memory flags.
--- lib/ethdev/rte_ethdev.c | 44 ++++++++++++++++++++++++++++ lib/ethdev/rte_ethdev.h | 65 ++++++++++++++++++++++++++++++++++++++++- 2 files changed, 108 insertions(+), 1 deletion(-)
lib/ethdev/rte_ethdev.c +44 -0
@@ -1945,6 +1945,28 @@ rte_eth_rx_hairpin_queue_setup(uint16_t port_id, uint16_t rx_queue_id, conf->peer_count, cap.max_rx_2_tx);return -EINVAL ; } +if (conf->use_locked_device_memory && !cap.rx_cap.locked_device_memory) { +RTE_ETHDEV_LOG (ERR , +"Attempt to use locked device memory for Rx queue, which is not supported" ); +return -EINVAL ; + } +if (conf->use_rte_memory && !cap.rx_cap.rte_memory) { +RTE_ETHDEV_LOG (ERR , +"Attempt to use DPDK memory for Rx queue, which is not supported" ); +return -EINVAL ; + } +if (conf->use_locked_device_memory && conf->use_rte_memory) { +RTE_ETHDEV_LOG (ERR , +"Attempt to use mutually exclusive memory settings for Rx queue" ); +return -EINVAL ; + } +if (conf->force_memory && + !conf->use_locked_device_memory && + !conf->use_rte_memory) { +RTE_ETHDEV_LOG (ERR , +"Attempt to force Rx queue memory settings, but none is set" ); +return -EINVAL ; + }if (conf->peer_count ==0 ) { RTE_ETHDEV_LOG(ERR, "Invalid value for number of peers for Rx queue(=%u), should be: > 0",
@@ -2111,6 +2133,28 @@ rte_eth_tx_hairpin_queue_setup(uint16_t port_id, uint16_t tx_queue_id, conf->peer_count, cap.max_tx_2_rx);return -EINVAL ; } +if (conf->use_locked_device_memory && !cap.tx_cap.locked_device_memory) { +RTE_ETHDEV_LOG (ERR , +"Attempt to use locked device memory for Tx queue, which is not supported" ); +return -EINVAL ; + } +if (conf->use_rte_memory && !cap.tx_cap.rte_memory) { +RTE_ETHDEV_LOG (ERR , +"Attempt to use DPDK memory for Tx queue, which is not supported" ); +return -EINVAL ; + } +if (conf->use_locked_device_memory && conf->use_rte_memory) { +RTE_ETHDEV_LOG (ERR , +"Attempt to use mutually exclusive memory settings for Tx queue" ); +return -EINVAL ; + } +if (conf->force_memory && + !conf->use_locked_device_memory && + !conf->use_rte_memory) { +RTE_ETHDEV_LOG (ERR , +"Attempt to force Tx queue memory settings, but none is set" ); +return -EINVAL ; + }if (conf->peer_count ==0 ) { RTE_ETHDEV_LOG(ERR, "Invalid value for number of peers for Tx queue(=%u), should be: > 0",
lib/ethdev/rte_ethdev.h +64 -1
83 lines not on this page · raw message
Signature · 1 line
2.25.1
19/09/2022 18:37, Dariusz Sosnowski:
Quoted 4 lines
This patch introduces new hairpin queue configuration options through
rte_eth_hairpin_confstruct, allowing to tune Rx and Tx hairpin queues memory configuration. Hairpin configuration is extended with the following fields:
What is the benefit?
How the user knows what to use?
Isn't it too much low level for a user?
Why it is not automatic in the driver?
[...]
lib/ethdev/rte_ethdev.h struct rte_eth_hairpin_conf { · line 1367
Quoted 6 lines
lib/ethdev/rte_ethdev.h +6
-0+/** +* Use locked device memory as a backing storage. +* +* - When set, PMD will attempt to use on-device memory as a backing storage for descriptors +* and/or data in hairpin queue. +* - When set, PMD will use detault memory type as a backing storage. Please refer to PMD
You probably mean "clear".
Please make lines shorter.
You should split lines logically, after a dot or at the end of a part.
Quoted 23 lines
lib/ethdev/rte_ethdev.h +23
-0+* documentation for details. +* +* API user should check if PMD supports this configuration flag using +* @see rte_eth_dev_hairpin_capability_get. +*/ + uint32_t use_locked_device_memory:1; + +/** +* Use DPDK memory as backing storage. +* +* - When set, PMD will attempt to use memory managed by DPDK as a backing storage +* for descriptors and/or data in hairpin queue. +* - When clear, PMD will use default memory type as a backing storage. Please refer +* to PMD documentation for details. +* +* API user should check if PMD supports this configuration flag using +* @see rte_eth_dev_hairpin_capability_get. +*/ + uint32_t use_rte_memory:1; + +/** +* Force usage of hairpin memory configuration. +*
lib/ethdev/rte_ethdev.h struct rte_eth_hairpin_conf { · line 1400
Quoted 10 lines
lib/ethdev/rte_ethdev.h +10
-0+* - When set, PMD will attempt to use specified memory settings and +* if resource allocation fails, then hairpin queue setup will result in an +* error. +* - When clear, PMD will attempt to use specified memory settings and +* if resource allocation fails, then PMD will retry allocation with default +* configuration. +*/ + uint32_t force_memory:1; + + uint32_t reserved:11; /**< Reserved bits. */
You can insert a blank line here.
Quoted 2 lines
struct
rte_eth_hairpin_peerpeers[RTE_ETH_MAX_HAIRPIN_PEERS];
};
Hi Thomas,
Quoted 1 line
What is the benefit?
Goal of this patchset is to present application developers with more options to fine tune hairpin configuration to their use case. I added more details regarding the possible benefits and motivation to the cover letter of v2.
Quoted 2 lines
How the user knows what to use? Why it is not automatic in the driver?
Basic assumption is that the default behavior of the PMD (mlx5 in that specific case) is a baseline for hairpin performance.
If that default performance is enough for a user, he will not have to change anything in the hairpin queue configuration.
If performance is not satisfying, user will have to experiment with different configurations e.g., if decreasing traffic latency is a priority for the user, user can check how his specific application works when use_locked_device_memory is used.
Specific performance gains of each of the hairpin options is not a given,
since hairpin performance will be a function of hairpin configuration and flow configuration (number of flows, types of flows, etc.).
Quoted 1 line
Isn't it too much low level for a user?
In my opinion, it's not too low level since purpose of these new options is fine tuning the hairpin performance to the specific use case of the application.
Quoted 6 lines
Quoted 2 lines
+ * - When set, PMD will use detault memory type as a backing
+ storage. Please refer to PMDYou probably mean "clear".
Please make lines shorter.
You should split lines logically, after a dot or at the end of a part.
Fixed in v2.
Quoted 7 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+ + uint32_t reserved:11; /**< Reserved bits. */You can insert a blank line here.
Quoted 2 lines
struct rte_eth_hairpin_peer peers[RTE_ETH_MAX_HAIRPIN_PEERS]; };
Added in v2.
Best regards,
Dariusz Sosnowski
PATCH 2/7 common/mlx5: add hairpin SQ buffer type capabilities
In reply to Dariusz SosnowskiThis patch extends HCA_CAP and SQ Context structs available in PRM. This fields allow checking if NIC supports storing hairpin SQ's WQ buffer in host memory and configuring such memory placement.
HCA capabilities are extended with the following fields:
- hairpin_sq_wq_in_host_mem - If set, then NIC supports using host memory as a backing storage for hairpin SQ's WQ buffer.
- hairpin_sq_wqe_bb_size - Indicates the required size of SQ WQE basic block.
SQ Context is extended with hairpin_wq_buffer_type which informs NIC where SQ's WQ buffer will be stored. This field can take the following values:
- MLX5_SQC_HAIRPIN_WQ_BUFFER_TYPE_INTERNAL_BUFFER - WQ buffer will be stored in unlocked device memory.
- MLX5_SQC_HAIRPIN_WQ_BUFFER_TYPE_HOST_MEMORY - WQ buffer will be stored in host memory. Buffer is provided by PMD.
--- drivers/common/mlx5/mlx5_devx_cmds.c | 5 +++++ drivers/common/mlx5/mlx5_devx_cmds.h | 3 +++ drivers/common/mlx5/mlx5_prm.h | 15 +++++++++++++-- 3 files changed, 21 insertions(+), 2 deletions(-)
drivers/common/mlx5/mlx5_devx_cmds.c +5 -0
@@ -981,6 +981,10 @@ mlx5_devx_cmd_query_hca_attr(void *ctx, } attr->log_min_stride_wqe_sz =MLX5_GET (cmd_hca_cap_2, hcattr, log_min_stride_wqe_sz); + attr->hairpin_sq_wqe_bb_size =MLX5_GET (cmd_hca_cap_2, hcattr, + hairpin_sq_wqe_bb_size); + attr->hairpin_sq_wq_in_host_mem =MLX5_GET (cmd_hca_cap_2, hcattr, + hairpin_sq_wq_in_host_mem); }if (attr->log_min_stride_wqe_sz ==0 ) attr->log_min_stride_wqe_sz =MLX5_MPRQ_LOG_MIN_STRIDE_WQE_SIZE ;
@@ -1698,6 +1702,7 @@ mlx5_devx_cmd_create_sq(void *ctx,MLX5_SET (sqc, sq_ctx, hairpin, sq_attr->hairpin);MLX5_SET (sqc, sq_ctx, non_wire, sq_attr->non_wire);MLX5_SET (sqc, sq_ctx, static_sq_wq, sq_attr->static_sq_wq); +MLX5_SET (sqc, sq_ctx, hairpin_wq_buffer_type, sq_attr->hairpin_wq_buffer_type);MLX5_SET (sqc, sq_ctx, user_index, sq_attr->user_index);MLX5_SET (sqc, sq_ctx, cqn, sq_attr->cqn); MLX5_SET(sqc, sq_ctx, packet_pacing_rate_limit_index,
drivers/common/mlx5/mlx5_devx_cmds.h +3 -0
@@ -191,6 +191,8 @@ struct mlx5_hca_attr { uint32_t log_max_hairpin_queues:5; uint32_t log_max_hairpin_wq_data_sz:5; uint32_t log_max_hairpin_num_packets:5; + uint32_t hairpin_sq_wqe_bb_size:4; + uint32_t hairpin_sq_wq_in_host_mem:1; uint32_t vhca_id:16; uint32_t relaxed_ordering_write:1; uint32_t relaxed_ordering_read:1;
@@ -407,6 +409,7 @@ struct mlx5_devx_create_sq_attr { uint32_t non_wire:1; uint32_t static_sq_wq:1; uint32_t ts_format:2; + uint32_t hairpin_wq_buffer_type:3; uint32_t user_index:24; uint32_t cqn:24; uint32_t packet_pacing_rate_limit_index:16;
drivers/common/mlx5/mlx5_prm.h +13 -2
@@ -2020,7 +2020,11 @@ struct mlx5_ifc_cmd_hca_cap_2_bits {u8 reserved_at_d8[0x3 ];u8 log_max_conn_track_offload[0x5 ];u8 reserved_at_e0[0x20 ];/* End of DW7. */ -u8 reserved_at_100[0x700 ]; +u8 reserved_at_100[0x60 ]; +u8 reserved_at_160[0x3 ]; +u8 hairpin_sq_wqe_bb_size[0x5 ]; +u8 hairpin_sq_wq_in_host_mem[0x1 ]; +u8 reserved_at_169[0x697 ]; };struct mlx5_ifc_esw_cap_bits {
@@ -2673,6 +2677,11 @@ enum { MLX5_SQC_STATE_ERR = 0x3, }; +enum { +MLX5_SQC_HAIRPIN_WQ_BUFFER_TYPE_INTERNAL_BUFFER =0x0 , +MLX5_SQC_HAIRPIN_WQ_BUFFER_TYPE_HOST_MEMORY =0x1 , +}; +struct mlx5_ifc_sqc_bits {u8 rlky[0x1 ];u8 cd_master[0x1 ];
@@ -2686,7 +2695,9 @@ struct mlx5_ifc_sqc_bits {u8 hairpin[0x1 ];u8 non_wire[0x1 ];u8 static_sq_wq[0x1 ]; -u8 reserved_at_11[0x9 ]; +u8 reserved_at_11[0x4 ]; +u8 hairpin_wq_buffer_type[0x3 ]; +u8 reserved_at_18[0x2 ];u8 ts_format[0x02 ];u8 reserved_at_1c[0x4 ];u8 reserved_at_20[0x8 ];
Signature · 1 line
2.25.1
RE: [PATCH 2/7] common/mlx5: add hairpin SQ buffer type capabilities
Quoted 21 lines
-----Original Message-----
From: Dariusz Sosnowski <[email protected]>
Sent: Monday, September 19, 2022 19:37
To: Matan Azrad <[email protected]>; Slava Ovsiienko <[email protected]>
Cc: [email protected]
Subject: [PATCH 2/7] common/mlx5: add hairpin SQ buffer type capabilitiesThis patch extends
HCA_CAPand SQ Context structs available in PRM. This fields allow checking if NIC supports storing hairpin SQ's WQ buffer in host memory and configuring such memory placement.HCA capabilities are extended with the following fields:
-
hairpin_sq_wq_in_host_mem- If set, then NIC supports using host memory as a backing storage for hairpin SQ's WQ buffer.
-hairpin_sq_wqe_bb_size- Indicates the required size of SQ WQE basic block.SQ Context is extended with
hairpin_wq_buffer_typewhich informs NIC where SQ's WQ buffer will be stored. This field can take the following values:
Quoted 6 lines
-
MLX5_SQC_HAIRPIN_WQ_BUFFER_TYPE_INTERNAL_BUFFER- WQ buffer will be stored in unlocked device memory.
-MLX5_SQC_HAIRPIN_WQ_BUFFER_TYPE_HOST_MEMORY- WQ buffer will be stored in host memory. Buffer is provided by PMD.Signed-off-by Dariusz Sosnowski <[email protected]>
PATCH 3/7 common/mlx5: add hairpin RQ buffer type capabilities
In reply to Dariusz SosnowskiThis patch adds new HCA capability related to hairpin RQs. This new capability, hairpin_data_buffer_locked, indicates whether HCA supports locking data buffer of hairpin RQ in ICMC (Interconnect Context Memory Cache).
Struct used to define RQ configuration (RQ context) is extended with hairpin_data_buffer_type field, which configures data buffer for hairpin RQ. It can take the following values:
- MLX5_RQC_HAIRPIN_DATA_BUFFER_TYPE_UNLOCKED_INTERNAL_BUFFER - hairpin RQ's data buffer is stored in unlocked memory in ICMC.
- MLX5_RQC_HAIRPIN_DATA_BUFFER_TYPE_LOCKED_INTERNAL_BUFFER - hairpin RQ's data buffer is stored in locked memory in ICMC.
--- drivers/common/mlx5/mlx5_devx_cmds.c | 3 +++ drivers/common/mlx5/mlx5_devx_cmds.h | 2 ++ drivers/common/mlx5/mlx5_prm.h | 12 ++++++++++-- 3 files changed, 15 insertions(+), 2 deletions(-)
drivers/common/mlx5/mlx5_devx_cmds.c +3 -0
@@ -985,6 +985,8 @@ mlx5_devx_cmd_query_hca_attr(void *ctx, hairpin_sq_wqe_bb_size); attr->hairpin_sq_wq_in_host_mem =MLX5_GET (cmd_hca_cap_2, hcattr, hairpin_sq_wq_in_host_mem); + attr->hairpin_data_buffer_locked =MLX5_GET (cmd_hca_cap_2, hcattr, + hairpin_data_buffer_locked); }if (attr->log_min_stride_wqe_sz ==0 ) attr->log_min_stride_wqe_sz =MLX5_MPRQ_LOG_MIN_STRIDE_WQE_SIZE ;
@@ -1285,6 +1287,7 @@ mlx5_devx_cmd_create_rq(void *ctx,MLX5_SET (rqc, rq_ctx, state, rq_attr->state);MLX5_SET (rqc, rq_ctx, flush_in_error_en, rq_attr->flush_in_error_en);MLX5_SET (rqc, rq_ctx, hairpin, rq_attr->hairpin); +MLX5_SET (rqc, rq_ctx, hairpin_data_buffer_type, rq_attr->hairpin_data_buffer_type);MLX5_SET (rqc, rq_ctx, user_index, rq_attr->user_index);MLX5_SET (rqc, rq_ctx, cqn, rq_attr->cqn);MLX5_SET (rqc, rq_ctx, counter_set_id, rq_attr->counter_set_id);
drivers/common/mlx5/mlx5_devx_cmds.h +2 -0
@@ -193,6 +193,7 @@ struct mlx5_hca_attr { uint32_t log_max_hairpin_num_packets:5; uint32_t hairpin_sq_wqe_bb_size:4; uint32_t hairpin_sq_wq_in_host_mem:1; + uint32_t hairpin_data_buffer_locked:1; uint32_t vhca_id:16; uint32_t relaxed_ordering_write:1; uint32_t relaxed_ordering_read:1;
@@ -313,6 +314,7 @@ struct mlx5_devx_create_rq_attr { uint32_t state:4; uint32_t flush_in_error_en:1; uint32_t hairpin:1; + uint32_t hairpin_data_buffer_type:3; uint32_t ts_format:2; uint32_t user_index:24; uint32_t cqn:24;
drivers/common/mlx5/mlx5_prm.h +10 -2
@@ -2024,7 +2024,8 @@ struct mlx5_ifc_cmd_hca_cap_2_bits {u8 reserved_at_160[0x3 ];u8 hairpin_sq_wqe_bb_size[0x5 ];u8 hairpin_sq_wq_in_host_mem[0x1 ]; -u8 reserved_at_169[0x697 ]; +u8 hairpin_data_buffer_locked[0x1 ]; +u8 reserved_at_16a[0x696 ]; };struct mlx5_ifc_esw_cap_bits {
@@ -2304,7 +2305,9 @@ struct mlx5_ifc_rqc_bits {u8 reserved_at_c[0x1 ];u8 flush_in_error_en[0x1 ];u8 hairpin[0x1 ]; -u8 reserved_at_f[0xB ]; +u8 reserved_at_f[0x6 ]; +u8 hairpin_data_buffer_type[0x3 ]; +u8 reserved_at_a8[0x2 ];u8 ts_format[0x02 ];u8 reserved_at_1c[0x4 ];u8 reserved_at_20[0x8 ];
@@ -2813,6 +2816,11 @@ enum { MLX5_CQE_SIZE_128B = 0x1, }; +enum { +MLX5_RQC_HAIRPIN_DATA_BUFFER_TYPE_UNLOCKED_INTERNAL_BUFFER =0x0 , +MLX5_RQC_HAIRPIN_DATA_BUFFER_TYPE_LOCKED_INTERNAL_BUFFER =0x1 , +}; +struct mlx5_ifc_cqc_bits {u8 status[0x4 ];u8 as_notify[0x1 ];
Signature · 1 line
2.25.1
RE: [PATCH 3/7] common/mlx5: add hairpin RQ buffer type capabilities
Quoted 16 lines
-----Original Message-----
From: Dariusz Sosnowski <[email protected]>
Sent: Monday, September 19, 2022 19:37
To: Matan Azrad <[email protected]>; Slava Ovsiienko <[email protected]>
Cc: [email protected]
Subject: [PATCH 3/7] common/mlx5: add hairpin RQ buffer type capabilitiesThis patch adds new HCA capability related to hairpin RQs. This new capability,
hairpin_data_buffer_locked, indicates whether HCA supports locking data buffer of hairpin RQ in ICMC (Interconnect Context Memory Cache).Struct used to define RQ configuration (RQ context) is extended with
hairpin_data_buffer_typefield, which configures data buffer for hairpin RQ. It can take the following values:
Quoted 6 lines
-
MLX5_RQC_HAIRPIN_DATA_BUFFER_TYPE_UNLOCKED_INTERNAL_BUFFER- hairpin RQ's data buffer is stored in unlocked memory in ICMC.
-MLX5_RQC_HAIRPIN_DATA_BUFFER_TYPE_LOCKED_INTERNAL_BUFFER- hairpin RQ's data buffer is stored in locked memory in ICMC.Signed-off-by Dariusz Sosnowski <[email protected]>
PATCH 4/7 net/mlx5: allow hairpin Tx queue in RTE memory
In reply to Dariusz SosnowskiThis patch adds a capability to place hairpin Tx queue in host memory managed by DPDK. This capability is equivalent to storing hairpin SQ's WQ buffer in host memory.
Hairpin Tx queue creation is extended with allocating a memory buffer of proper size (calculated from required number of packets and WQE BB size advertised in HCA capabilities).
force_memory flag of hairpin queue configuration is also supported. If it is set and:
- allocation of memory buffer fails,
- or hairpin SQ creation fails,
then device start will fail. If it is unset, PMD will fallback to creating the hairpin SQ with WQ buffer located in unlocked device memory.
--- drivers/net/mlx5/mlx5.h | 2 + drivers/net/mlx5/mlx5_devx.c | 119 ++++++++++++++++++++++++++++++--- drivers/net/mlx5/mlx5_ethdev.c | 4 ++ 3 files changed, 116 insertions(+), 9 deletions(-)
drivers/net/mlx5/mlx5.h +2 -0
@@ -1384,6 +1384,8 @@ struct mlx5_txq_obj {struct mlx5_devx_obj *sq;/* DevX object for Sx queue. */ struct mlx5_devx_obj *tis;/* The TIS object. */ +void *umem_buf_wq_buffer; +struct mlx5dv_devx_umem *umem_obj_wq_buffer; };struct {struct rte_eth_dev *dev;
drivers/net/mlx5/mlx5_devx.c +110 -9
155 lines not on this page · raw message
drivers/net/mlx5/mlx5_ethdev.c +4 -0
@@ -729,6 +729,7 @@ int mlx5_hairpin_cap_get(struct rte_eth_dev *dev, struct rte_eth_hairpin_cap *cap) { struct mlx5_priv *priv = dev->data->dev_private; +struct mlx5_hca_attr *hca_attr;if (!mlx5_devx_obj_ops_en (priv->sh)) { rte_errno =ENOTSUP ;
@@ -738,5 +739,8 @@ mlx5_hairpin_cap_get(struct rte_eth_dev *dev, struct rte_eth_hairpin_cap *cap) cap->max_rx_2_tx =1 ; cap->max_tx_2_rx =1 ; cap->max_nb_desc =8192 ; + hca_attr = &priv->sh->cdev->config.hca_attr; + cap->tx_cap.locked_device_memory =0 ; + cap->tx_cap.rte_memory = hca_attr->hairpin_sq_wq_in_host_mem;return 0 ; }
Signature · 1 line
2.25.1
RE: [PATCH 4/7] net/mlx5: allow hairpin Tx queue in RTE memory
Quoted 18 lines
-----Original Message-----
From: Dariusz Sosnowski <[email protected]>
Sent: Monday, September 19, 2022 19:37
To: Matan Azrad <[email protected]>; Slava Ovsiienko <[email protected]>
Cc: [email protected]
Subject: [PATCH 4/7] net/mlx5: allow hairpin Tx queue in RTE memoryThis patch adds a capability to place hairpin Tx queue in host memory managed by DPDK. This capability is equivalent to storing hairpin SQ's WQ buffer in host memory.
Hairpin Tx queue creation is extended with allocating a memory buffer of proper size (calculated from required number of packets and WQE BB size advertised in HCA capabilities).
force_memoryflag of hairpin queue configuration is also supported. If it is set and:
Quoted 7 lines
- allocation of memory buffer fails,
- or hairpin SQ creation fails,then device start will fail. If it is unset, PMD will fallback to creating the hairpin SQ with WQ buffer located in unlocked device memory.
Signed-off-by Dariusz Sosnowski <[email protected]>
PATCH 5/7 net/mlx5: allow hairpin Rx queue in locked memory
In reply to Dariusz SosnowskiThis patch adds a capability to place hairpin Rx queue in locked device memory. This capability is equivalent to storing hairpin RQ's data buffers in locked internal device memory.
Hairpin Rx queue creation is extended with requesting that RQ is allocated in locked internal device memory. If allocation fails and force_memory hairpin configuration is set, then hairpin queue creation (and, as a result, device start) fails. If force_memory is unset, then PMD will fallback to allocating memory for hairpin RQ in unlocked internal device memory.
To allow such allocation, the user must set HAIRPIN_DATA_BUFFER_LOCK flag in FW using mlxconfig tool.
--- doc/guides/platform/mlx5.rst | 5 ++++ drivers/net/mlx5/mlx5_devx.c | 51 ++++++++++++++++++++++++++++------ drivers/net/mlx5/mlx5_ethdev.c | 2 ++ 3 files changed, 49 insertions(+), 9 deletions(-)
doc/guides/platform/mlx5.rst +5 -0
@@ -548,6 +548,11 @@ Below are some firmware configurations listed. REAL_TIME_CLOCK_ENABLE=1 +- allow locking hairpin RQ data buffer in device memory:: + + HAIRPIN_DATA_BUFFER_LOCK=1 + MEMIC_SIZE_LIMIT=0 + .. _mlx5_common_driver_options:
drivers/net/mlx5/mlx5_devx.c +42 -9
82 lines not on this page · raw message
drivers/net/mlx5/mlx5_ethdev.c +2 -0
@@ -740,6 +740,8 @@ mlx5_hairpin_cap_get(struct rte_eth_dev *dev, struct rte_eth_hairpin_cap *cap) cap->max_tx_2_rx =1 ; cap->max_nb_desc =8192 ; hca_attr = &priv->sh->cdev->config.hca_attr; + cap->rx_cap.locked_device_memory = hca_attr->hairpin_data_buffer_locked; + cap->rx_cap.rte_memory =0 ; cap->tx_cap.locked_device_memory =0 ; cap->tx_cap.rte_memory = hca_attr->hairpin_sq_wq_in_host_mem;return 0 ;
Signature · 1 line
2.25.1
RE: [PATCH 5/7] net/mlx5: allow hairpin Rx queue in locked memory
Quoted 17 lines
-----Original Message-----
From: Dariusz Sosnowski <[email protected]>
Sent: Monday, September 19, 2022 19:37
To: Matan Azrad <[email protected]>; Slava Ovsiienko <[email protected]>
Cc: [email protected]
Subject: [PATCH 5/7] net/mlx5: allow hairpin Rx queue in locked memoryThis patch adds a capability to place hairpin Rx queue in locked device memory. This capability is equivalent to storing hairpin RQ's data buffers in locked internal device memory.
Hairpin Rx queue creation is extended with requesting that RQ is allocated in locked internal device memory. If allocation fails and
force_memoryhairpin configuration is set, then hairpin queue creation (and, as a result, device start) fails. Ifforce_memoryis unset, then PMD will fallback to allocating memory for hairpin RQ in unlocked internal device memory.
Quoted 4 lines
To allow such allocation, the user must set
HAIRPIN_DATA_BUFFER_LOCKflag in FW using mlxconfig tool.Signed-off-by Dariusz Sosnowski <[email protected]>
Re: [PATCH 5/7] net/mlx5: allow hairpin Rx queue in locked memory
In reply to Dariusz SosnowskiThis series adds support for using device-managed MEMIC buffers on hairpin RQ instead of NIM. Was it ever considered as an alternative that the UMEM interface be extended to support MEMIC buffers instead?
I'm thinking that could simplify the hairpin-specific firmware bits being added in this series, e.g. no new HAIRPIN_DATA_BUFFER_LOCK TLV, and the MEMIC-backed UMEM can be passed to RQ using existing PRM bits.
I'm planning to file a feature request adding MEMIC support to UMEM, so I'd be interested in knowing if that's somehow not possible. My current use case is allocating 64 bytes of MEMIC for a collapsed CQE for something similar to the mlx5 packet send scheduling in DPDK.
Best regards,
Kenneth Jonassen
Quoted 23 lines · Dariusz Sosnowski
This patch adds a capability to place hairpin Rx queue in locked device memory. This capability is equivalent to storing hairpin RQ's data buffers in locked internal device memory.
Hairpin Rx queue creation is extended with requesting that RQ is allocated in locked internal device memory. If allocation fails and
force_memoryhairpin configuration is set, then hairpin queue creation (and, as a result, device start) fails. Ifforce_memoryis unset, then PMD will fallback to allocating memory for hairpin RQ in unlocked internal device memory.To allow such allocation, the user must set
HAIRPIN_DATA_BUFFER_LOCKflag in FW using mlxconfig tool.Signed-off-by Dariusz Sosnowski <[email protected]>---
doc/guides/platform/mlx5.rst | 5 ++++ drivers/net/mlx5/mlx5_devx.c | 51 ++++++++++++++++++++++++++++------ drivers/net/mlx5/mlx5_ethdev.c | 2 ++ 3 files changed, 49 insertions(+), 9 deletions(-)
PATCH 6/7 app/testpmd: add hairpin queues memory modes
In reply to Dariusz SosnowskiThis patch extends hairpin-mode command line option of test-pmd application with an ability to configure whether Rx/Tx hairpin queue should use locked device memory or RTE memory.
For purposes of this configurations the following bits of 32 bit hairpin-mode are reserved:
- Bit 8 - If set, then force_memory flag will be set for hairpin RX queue.
- Bit 9 - If set, then force_memory flag will be set for hairpin TX queue.
- Bits 12-15 - Memory options for hairpin Rx queue:
- Bit 12 - If set, then use_locked_device_memory will be set.
- Bit 13 - If set, then use_rte_memory will be set.
- Bit 14 - Reserved for future use.
- Bit 15 - Reserved for future use.
- Bits 16-19 - Memory options for hairpin Tx queue:
- Bit 16 - If set, then use_locked_device_memory will be set.
- Bit 17 - If set, then use_rte_memory will be set.
- Bit 18 - Reserved for future use.
- Bit 19 - Reserved for future use.
--- app/test-pmd/parameters.c | 2 +- app/test-pmd/testpmd.c | 24 +++++++++++++++++++++++- app/test-pmd/testpmd.h | 2 +- doc/guides/testpmd_app_ug/run_app.rst | 10 ++++++++-- 4 files changed, 33 insertions(+), 5 deletions(-)
app/test-pmd/parameters.c +1 -1
@@ -1162,7 +1162,7 @@ launch_args_parse(int argc, char** argv)if (errno !=0 || end == optarg)rte_exit (EXIT_FAILURE ,"hairpin mode invalid\n" );else - hairpin_mode = (uint16_t )n; + hairpin_mode = (uint32_t )n; }if (!strcmp (lgopts[opt_idx].name,"burst" )) { n =atoi (optarg);
app/test-pmd/testpmd.c +23 -1
@@ -409,7 +409,7 @@ bool setup_on_probe_event = true;uint8_t clear_ptypes = true;/* Hairpin ports configuration mode. */ -uint16_t hairpin_mode; +uint32_t hairpin_mode;/* Pretty printing of ethdev events */ static const char *const eth_event_desc[] = {
@@ -2552,6 +2552,16 @@ port_is_started(portid_t port_id)return 1 ; } +#define HAIRPIN_MODE_RX_FORCE_MEMORY RTE_BIT32(8) +#define HAIRPIN_MODE_TX_FORCE_MEMORY RTE_BIT32(9) + +#define HAIRPIN_MODE_RX_LOCKED_MEMORY RTE_BIT32(12) +#define HAIRPIN_MODE_RX_RTE_MEMORY RTE_BIT32(13) + +#define HAIRPIN_MODE_TX_LOCKED_MEMORY RTE_BIT32(16) +#define HAIRPIN_MODE_TX_RTE_MEMORY RTE_BIT32(17) + +/* Configure the Rx and Tx hairpin queues for the selected port. */ static int setup_hairpin_queues (portid_t pi,portid_t p_pi,uint16_t cnt_pi)
@@ -2567,6 +2577,12 @@ setup_hairpin_queues(portid_t pi, portid_t p_pi, uint16_t cnt_pi)uint16_t peer_tx_port = pi;uint32_t manual =1 ;uint32_t tx_exp = hairpin_mode &0x10 ; +uint32_t rx_force_memory = hairpin_mode &HAIRPIN_MODE_RX_FORCE_MEMORY ; +uint32_t rx_locked_memory = hairpin_mode &HAIRPIN_MODE_RX_LOCKED_MEMORY ; +uint32_t rx_rte_memory = hairpin_mode &HAIRPIN_MODE_RX_RTE_MEMORY ; +uint32_t tx_force_memory = hairpin_mode &HAIRPIN_MODE_TX_FORCE_MEMORY ; +uint32_t tx_locked_memory = hairpin_mode &HAIRPIN_MODE_TX_LOCKED_MEMORY ; +uint32_t tx_rte_memory = hairpin_mode &HAIRPIN_MODE_TX_RTE_MEMORY ;if (!(hairpin_mode &0xf )) { peer_rx_port = pi;
@@ -2606,6 +2622,9 @@ setup_hairpin_queues(portid_t pi, portid_t p_pi, uint16_t cnt_pi) hairpin_conf.peers[0 ].queue = i + nb_rxq; hairpin_conf.manual_bind = !!manual; hairpin_conf.tx_explicit = !!tx_exp; + hairpin_conf.force_memory = !!tx_force_memory; + hairpin_conf.use_locked_device_memory = !!tx_locked_memory; + hairpin_conf.use_rte_memory = !!tx_rte_memory; diag =rte_eth_tx_hairpin_queue_setup (pi, qi, nb_txd, &hairpin_conf); i++;
@@ -2629,6 +2648,9 @@ setup_hairpin_queues(portid_t pi, portid_t p_pi, uint16_t cnt_pi) hairpin_conf.peers[0 ].queue = i + nb_txq; hairpin_conf.manual_bind = !!manual; hairpin_conf.tx_explicit = !!tx_exp; + hairpin_conf.force_memory = !!rx_force_memory; + hairpin_conf.use_locked_device_memory = !!rx_locked_memory; + hairpin_conf.use_rte_memory = !!rx_rte_memory; diag =rte_eth_rx_hairpin_queue_setup (pi, qi, nb_rxd, &hairpin_conf); i++;
app/test-pmd/testpmd.h +1 -1
@@ -542,7 +542,7 @@ extern uint16_t stats_period;extern struct rte_eth_xstat_name *xstats_display;extern unsigned int xstats_display_num; -extern uint16_t hairpin_mode; +extern uint32_t hairpin_mode;#ifdef RTE_LIB_LATENCYSTATS extern uint8_t latencystats_enabled;
doc/guides/testpmd_app_ug/run_app.rst +8 -2
@@ -556,10 +556,16 @@ The command line options are: Enable display of RX and TX burst stats. -* ``--hairpin-mode=0xXX`` +* ``--hairpin-mode=0xXXXX`` - Set the hairpin port mode with bitmask, only valid when hairpin queues number is set:: + Set the hairpin port configuration with bitmask, only valid when hairpin queues number is set:: + bit 18 - hairpin TX queues will use RTE memory + bit 16 - hairpin TX queues will use locked device memory + bit 13 - hairpin RX queues will use RTE memory + bit 12 - hairpin RX queues will use locked device memory + bit 9 - force memory settings of hairpin TX queue + bit 8 - force memory settings of hairpin RX queue bit 4 - explicit Tx flow rule bit 1 - two hairpin ports paired bit 0 - two hairpin ports loop
Signature · 1 line
2.25.1
PATCH 7/7 app/flow-perf: add hairpin queue memory config
In reply to Dariusz SosnowskiThis patch adds the hairpin-conf command line parameter to flow-perf application. hairpin-conf parameter takes a hexadecimal bitmask with bits having the following meaning:
- Bit 0 - Force memory settings of hairpin RX queue.
- Bit 1 - Force memory settings of hairpin TX queue.
- Bit 4 - Use locked device memory for hairpin RX queue.
- Bit 5 - Use RTE memory for hairpin RX queue.
- Bit 8 - Use locked device memory for hairpin TX queue.
- Bit 9 - Use RTE memory for hairpin TX queue.
--- app/test-flow-perf/main.c | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+)
app/test-flow-perf/main.c +32 -0
@@ -46,6 +46,15 @@#define DEFAULT_RULES_BATCH 100000#define DEFAULT_GROUP 0 +#define HAIRPIN_RX_CONF_FORCE_MEMORY (0x0001) +#define HAIRPIN_TX_CONF_FORCE_MEMORY (0x0002) + +#define HAIRPIN_RX_CONF_LOCKED_MEMORY (0x0010) +#define HAIRPIN_RX_CONF_RTE_MEMORY (0x0020) + +#define HAIRPIN_TX_CONF_LOCKED_MEMORY (0x0100) +#define HAIRPIN_TX_CONF_RTE_MEMORY (0x0200) +struct rte_flow *flow;static uint8_t flow_group;
@@ -61,6 +70,7 @@ static uint32_t policy_id[MAX_PORTS];static uint8_t items_idx, actions_idx, attrs_idx;static uint64_t ports_mask; +static uint64_t hairpin_conf_mask;static uint16_t dst_ports[RTE_MAX_ETHPORTS ];static volatile bool force_quit;static bool dump_iterations;
@@ -482,6 +492,7 @@ usage(char *progname)printf (" --enable-fwd: To enable packets forwarding" " after insertion\n" );printf (" --portmask=N: hexadecimal bitmask of ports used\n" ); +printf (" --hairpin-conf=0xXXXX: hexadecimal bitmask of hairpin queue configuration\n" );printf (" --random-priority=N,S: use random priority levels " "from 0 to (N - 1) for flows " "and S as seed for pseudo-random number generator\n" );
@@ -629,6 +640,7 @@ static void args_parse(int argc, char **argv) { uint64_t pm, seed; +uint64_t hp_conf;char **argvopt;uint32_t prio;char *token;
@@ -648,6 +660,7 @@ args_parse(int argc, char **argv) { "enable-fwd", 0, 0, 0 }, { "unique-data", 0, 0, 0 }, { "portmask", 1, 0, 0 }, + { "hairpin-conf", 1, 0, 0 }, { "cores", 1, 0, 0 }, { "random-priority", 1, 0, 0 }, { "meter-profile-alg", 1, 0, 0 },
@@ -880,6 +893,13 @@ args_parse(int argc, char **argv)rte_exit (EXIT_FAILURE ,"Invalid fwd port mask\n" ); ports_mask = pm; } +if (strcmp (lgopts[opt_idx].name,"hairpin-conf" ) ==0 ) { + end =NULL ; + hp_conf =strtoull (optarg, &end,16 ); +if ((optarg[0 ] =='\0' ) || (end ==NULL ) || (*end !='\0' )) +rte_exit (EXIT_FAILURE ,"Invalid hairpin config mask\n" ); + hairpin_conf_mask = hp_conf; + }if (strcmp (lgopts[opt_idx].name,"port-id" ) ==0 ) {uint16_t port_idx =0 ;
@@ -2035,6 +2055,12 @@ init_port(void) hairpin_conf.peers[0 ].port = port_id; hairpin_conf.peers[0 ].queue = std_queue + tx_queues_count; + hairpin_conf.use_locked_device_memory = + !!(hairpin_conf_mask &HAIRPIN_RX_CONF_LOCKED_MEMORY ); + hairpin_conf.use_rte_memory = + !!(hairpin_conf_mask &HAIRPIN_RX_CONF_RTE_MEMORY ); + hairpin_conf.force_memory = + !!(hairpin_conf_mask &HAIRPIN_RX_CONF_FORCE_MEMORY ); ret =rte_eth_rx_hairpin_queue_setup ( port_id, hairpin_queue, rxd_count, &hairpin_conf);
@@ -2050,6 +2076,12 @@ init_port(void) hairpin_conf.peers[0 ].port = port_id; hairpin_conf.peers[0 ].queue = std_queue + rx_queues_count; + hairpin_conf.use_locked_device_memory = + !!(hairpin_conf_mask &HAIRPIN_TX_CONF_LOCKED_MEMORY ); + hairpin_conf.use_rte_memory = + !!(hairpin_conf_mask &HAIRPIN_TX_CONF_RTE_MEMORY ); + hairpin_conf.force_memory = + !!(hairpin_conf_mask &HAIRPIN_TX_CONF_FORCE_MEMORY ); ret =rte_eth_tx_hairpin_queue_setup ( port_id, hairpin_queue, txd_count, &hairpin_conf);
Signature · 1 line
2.25.1
RE: [PATCH 7/7] app/flow-perf: add hairpin queue memory config
Hi Dariusz,
Quoted 18 lines
-----Original Message-----
From: Dariusz Sosnowski <[email protected]>
Sent: Monday, September 19, 2022 7:38 PM
To: Wisam Monther <[email protected]>
Cc: [email protected]
Subject: [PATCH 7/7] app/flow-perf: add hairpin queue memory configThis patch adds the hairpin-conf command line parameter to flow-perf application. hairpin-conf parameter takes a hexadecimal bitmask with bits having the following meaning:
- Bit 0 - Force memory settings of hairpin RX queue.
- Bit 1 - Force memory settings of hairpin TX queue.
- Bit 4 - Use locked device memory for hairpin RX queue.
- Bit 5 - Use RTE memory for hairpin RX queue.
- Bit 8 - Use locked device memory for hairpin TX queue.
- Bit 9 - Use RTE memory for hairpin TX queue.
Quoted 2 lines
Signed-off-by Dariusz Sosnowski <[email protected]>---
You have some checks issues; can you please kindly check them?
BRs,
Wisam Jaddo
RE: [PATCH 7/7] app/flow-perf: add hairpin queue memory config
Hi,
Quoted 27 lines
-----Original Message-----
From: Wisam Monther <[email protected]>
Sent: Tuesday, October 4, 2022 14:25
To: Dariusz Sosnowski <[email protected]>
Cc: [email protected]
Subject: RE: [PATCH 7/7] app/flow-perf: add hairpin queue memory configHi Dariusz,
Quoted 18 lines
-----Original Message-----
From: Dariusz Sosnowski <[email protected]>
Sent: Monday, September 19, 2022 7:38 PM
To: Wisam Monther <[email protected]>
Cc: [email protected]
Subject: [PATCH 7/7] app/flow-perf: add hairpin queue memory configThis patch adds the hairpin-conf command line parameter to flow-perf application. hairpin-conf parameter takes a hexadecimal bitmask with bits having the following meaning:
- Bit 0 - Force memory settings of hairpin RX queue.
- Bit 1 - Force memory settings of hairpin TX queue.
- Bit 4 - Use locked device memory for hairpin RX queue.
- Bit 5 - Use RTE memory for hairpin RX queue.
- Bit 8 - Use locked device memory for hairpin TX queue.
- Bit 9 - Use RTE memory for hairpin TX queue.
Quoted 7 lines
Quoted 2 lines
Signed-off-by Dariusz Sosnowski <[email protected]>---
You have some checks issues; can you please kindly check them?
BRs,
Wisam Jaddo
I sent the v2. The checks issue should be fixed in that version.
Thank you.
Best regards,
Dariusz Sosnowski
19/09/2022 18:37, Dariusz Sosnowski:
Quoted 5 lines
This patch series introduces hairpin memory configuration options proposed in
http://patches.dpdk.org/project/dpdk/patch/[email protected]/
for Rx and Tx hairpin queues. It also implements handling of these options in mlx5 PMD and allows to use new hairpin options in testpmd (through--hairpin-modeoption) and flow-perf (through--hairpin-confoption).
2 things are missing in this series:
1/ motivation (why is this needed)
2/ compilation on Windows
looks like devx_umem_reg has 5 parameters in Windows glue!
Hi Thomas,
I sent the v2 of the patches.
Quoted 1 line
1/ motivation (why is this needed)
I added the feature motivation in the cover letter of v2.
Quoted 2 lines
2/ compilation on Windows
looks likedevx_umem_reghas 5 parameters in Windows glue!
Fix for that is included in v2.
Best regards,
Dariusz Sosnowski
v2 cover ethdev: introduce hairpin memory capabilities
In reply to Dariusz SosnowskiThe hairpin queues are used to transmit packets received on the wire, back to the wire. How hairpin queues are implemented and configured is decided internally by the PMD and applications have no control over the configuration of Rx and Tx hairpin queues. This patchset addresses that by:
- Extending hairpin queue capabilities reported by PMDs.
- Exposing new configuration options for Rx and Tx hairpin queues.
Main goal of this patchset is to allow applications to provide configuration hints regarding memory placement of hairpin queues.
These hints specify whether buffers of hairpin queues should be placed in host memory or in dedicated device memory.
For example, in context of NVIDIA Connect-X and BlueField devices, this distinction is important for several reasons:
- By default, data buffers and packet descriptors are placed in device memory region which is shared with other resources (e.g. flow rules).
This results in memory contention on the device,
which may lead to degraded performance under heavy load.
- Placing hairpin queues in dedicated device memory can decrease latency of hairpinned traffic, since hairpin queue processing will not be memory starved by other operations.
Side effect of this memory configuration is that it leaves less memory for other resources, possibly causing memory contention in non-hairpin traffic.
- Placing hairpin queues in host memory can increase throughput of hairpinned
traffic at the cost of increasing latency.
Each packet processed by hairpin queues will incur additional PCI transactions (increase in latency), but memory contention on the device is avoided.
Depending on the workload and whether throughput or latency has a higher priority for developers, it would be beneficial if developers could choose the best hairpin configuration for their use case.
To address that, this patchset adds the following configuration options (in rte_eth_hairpin_conf struct):
- use_locked_device_memory - If set, PMD will allocate specialized on-device memory for the queue.
- use_rte_memory - If set, PMD will use DPDK-managed memory for the queue.
- force_memory - If set, PMD will be forced to use provided memory configuration. If no appropriate resources are available, the queue allocation will fail.
If unset and no appropriate resources are available, PMD will fallback to its default behavior.
Implementing support for these flags is optional and applications should be allowed to not set any of these new flags. This will result in default memory configuration provided by the PMD.
Application developers should consult the PMD documentation in that case.
These changes were originally proposed in http://patches.dpdk.org/project/dpdk/patch/[email protected]/.
Dariusz Sosnowski (8):
ethdev: introduce hairpin memory capabilities common/mlx5: add hairpin SQ buffer type capabilities common/mlx5: add hairpin RQ buffer type capabilities net/mlx5: allow hairpin Tx queue in RTE memory net/mlx5: allow hairpin Rx queue in locked memory
doc: add notes for hairpin to mlx5 documentation app/testpmd: add hairpin queues memory modes app/flow-perf: add hairpin queue memory config
app/test-flow-perf/main.c | 32 +++++ app/test-pmd/parameters.c | 2 +- app/test-pmd/testpmd.c | 24 +++- app/test-pmd/testpmd.h | 2 +- doc/guides/nics/mlx5.rst | 37 ++++++ doc/guides/platform/mlx5.rst | 5 + doc/guides/rel_notes/release_22_11.rst | 10 ++ doc/guides/testpmd_app_ug/run_app.rst | 10 +- drivers/common/mlx5/mlx5_devx_cmds.c | 8 ++ drivers/common/mlx5/mlx5_devx_cmds.h | 5 + drivers/common/mlx5/mlx5_prm.h | 25 +++- drivers/net/mlx5/mlx5.h | 2 + drivers/net/mlx5/mlx5_devx.c | 170 ++++++++++++++++++++++--- drivers/net/mlx5/mlx5_ethdev.c | 6 + lib/ethdev/rte_ethdev.c | 44 +++++++ lib/ethdev/rte_ethdev.h | 68 +++++++++- 16 files changed, 422 insertions(+), 28 deletions(-)
Signature · 6 lines
v2: * Fix Windows build by using mlx5_os_umem_dereg defined on both platforms to allocate memory for Tx hairpin queue. * Added hairpin section to mlx5 PMD. * Added info about new hairpin configuration options to DPDK release notes. 2.25.1
PATCH v2 1/8 ethdev: introduce hairpin memory capabilities
Before this patch, implementation details and configuration of hairpin queues were decided internally by the PMD. Applications had no control over the configuration of Rx and Tx hairpin queues, despite number of descriptors, explicit Tx flow mode and disabling automatic binding. This patch addresses that by adding:
- Hairpin queue capabilities reported by PMDs.
- New configuration options for Rx and Tx hairpin queues.
Main goal of this patch is to allow applications to provide configuration hints regarding placement of hairpin queues. These hints specify whether buffers of hairpin queues should be placed in host memory or in dedicated device memory. Different memory options may have different performance characteristics and hairpin configuration should be fine-tuned to the specific application and use case.
This patch introduces new hairpin queue configuration options through rte_eth_hairpin_conf struct, allowing to tune Rx and Tx hairpin queues memory configuration. Hairpin configuration is extended with the following fields:
- use_locked_device_memory - If set, PMD will use specialized on-device memory to store RX or TX hairpin queue data.
- use_rte_memory - If set, PMD will use DPDK-managed memory to store RX or TX hairpin queue data.
- force_memory - If set, PMD will be forced to use provided memory settings. If no appropriate resources are available, then device start will fail. If unset and no resources are available, PMD will fallback to using default type of resource for given queue.
If application chooses to use PMD default memory configuration, all of these flags should remain unset.
Hairpin capabilities are also extended, to allow verification of support of given hairpin memory configurations. Struct rte_eth_hairpin_cap is extended with two additional fields of type rte_eth_hairpin_queue_cap:
- rx_cap - memory capabilities of hairpin RX queues.
- tx_cap - memory capabilities of hairpin TX queues.
Struct rte_eth_hairpin_queue_cap exposes whether given queue type supports use_locked_device_memory and use_rte_memory flags.
--- doc/guides/rel_notes/release_22_11.rst | 10 ++++ lib/ethdev/rte_ethdev.c | 44 +++++++++++++++++ lib/ethdev/rte_ethdev.h | 68 +++++++++++++++++++++++++- 3 files changed, 120 insertions(+), 2 deletions(-)
doc/guides/rel_notes/release_22_11.rst +10 -0
@@ -66,6 +66,16 @@ New Features Added new function ``rte_flow_async_action_handle_query()``, to query the action asynchronously. +* **Added hairpin memory configurations options in ethdev API.** + + Added new configuration flags for hairpin queues in ``rte_eth_hairpin_conf``: + + * ``use_locked_device_memory`` + * ``use_rte_memory`` + * ``force_memory`` + + Each flag has a corresponding capability flag in ``rte_eth_hairpin_queue_cap`` struct. + * **Updated Intel iavf driver.** * Added flow subscription support.
lib/ethdev/rte_ethdev.c +44 -0
56 lines not on this page · raw message
lib/ethdev/rte_ethdev.h +66 -2
89 lines not on this page · raw message
Signature · 1 line
2.25.1
PATCH v2 2/8 common/mlx5: add hairpin SQ buffer type capabilities
In reply to Dariusz SosnowskiThis patch extends HCA_CAP and SQ Context structs available in PRM. This fields allow checking if NIC supports storing hairpin SQ's WQ buffer in host memory and configuring such memory placement.
HCA capabilities are extended with the following fields:
- hairpin_sq_wq_in_host_mem - If set, then NIC supports using host memory as a backing storage for hairpin SQ's WQ buffer.
- hairpin_sq_wqe_bb_size - Indicates the required size of SQ WQE basic block.
SQ Context is extended with hairpin_wq_buffer_type which informs NIC where SQ's WQ buffer will be stored. This field can take the following values:
- MLX5_SQC_HAIRPIN_WQ_BUFFER_TYPE_INTERNAL_BUFFER - WQ buffer will be stored in unlocked device memory.
- MLX5_SQC_HAIRPIN_WQ_BUFFER_TYPE_HOST_MEMORY - WQ buffer will be stored in host memory. Buffer is provided by PMD.
--- drivers/common/mlx5/mlx5_devx_cmds.c | 5 +++++ drivers/common/mlx5/mlx5_devx_cmds.h | 3 +++ drivers/common/mlx5/mlx5_prm.h | 15 +++++++++++++-- 3 files changed, 21 insertions(+), 2 deletions(-)
drivers/common/mlx5/mlx5_devx_cmds.c +5 -0
@@ -989,6 +989,10 @@ mlx5_devx_cmd_query_hca_attr(void *ctx, } attr->log_min_stride_wqe_sz =MLX5_GET (cmd_hca_cap_2, hcattr, log_min_stride_wqe_sz); + attr->hairpin_sq_wqe_bb_size =MLX5_GET (cmd_hca_cap_2, hcattr, + hairpin_sq_wqe_bb_size); + attr->hairpin_sq_wq_in_host_mem =MLX5_GET (cmd_hca_cap_2, hcattr, + hairpin_sq_wq_in_host_mem); }if (attr->log_min_stride_wqe_sz ==0 ) attr->log_min_stride_wqe_sz =MLX5_MPRQ_LOG_MIN_STRIDE_WQE_SIZE ;
@@ -1706,6 +1710,7 @@ mlx5_devx_cmd_create_sq(void *ctx,MLX5_SET (sqc, sq_ctx, hairpin, sq_attr->hairpin);MLX5_SET (sqc, sq_ctx, non_wire, sq_attr->non_wire);MLX5_SET (sqc, sq_ctx, static_sq_wq, sq_attr->static_sq_wq); +MLX5_SET (sqc, sq_ctx, hairpin_wq_buffer_type, sq_attr->hairpin_wq_buffer_type);MLX5_SET (sqc, sq_ctx, user_index, sq_attr->user_index);MLX5_SET (sqc, sq_ctx, cqn, sq_attr->cqn); MLX5_SET(sqc, sq_ctx, packet_pacing_rate_limit_index,
drivers/common/mlx5/mlx5_devx_cmds.h +3 -0
@@ -191,6 +191,8 @@ struct mlx5_hca_attr { uint32_t log_max_hairpin_queues:5; uint32_t log_max_hairpin_wq_data_sz:5; uint32_t log_max_hairpin_num_packets:5; + uint32_t hairpin_sq_wqe_bb_size:4; + uint32_t hairpin_sq_wq_in_host_mem:1; uint32_t vhca_id:16; uint32_t relaxed_ordering_write:1; uint32_t relaxed_ordering_read:1;
@@ -407,6 +409,7 @@ struct mlx5_devx_create_sq_attr { uint32_t non_wire:1; uint32_t static_sq_wq:1; uint32_t ts_format:2; + uint32_t hairpin_wq_buffer_type:3; uint32_t user_index:24; uint32_t cqn:24; uint32_t packet_pacing_rate_limit_index:16;
drivers/common/mlx5/mlx5_prm.h +13 -2
@@ -2020,7 +2020,11 @@ struct mlx5_ifc_cmd_hca_cap_2_bits {u8 reserved_at_d8[0x3 ];u8 log_max_conn_track_offload[0x5 ];u8 reserved_at_e0[0x20 ];/* End of DW7. */ -u8 reserved_at_100[0x700 ]; +u8 reserved_at_100[0x60 ]; +u8 reserved_at_160[0x3 ]; +u8 hairpin_sq_wqe_bb_size[0x5 ]; +u8 hairpin_sq_wq_in_host_mem[0x1 ]; +u8 reserved_at_169[0x697 ]; };struct mlx5_ifc_esw_cap_bits {
@@ -2673,6 +2677,11 @@ enum { MLX5_SQC_STATE_ERR = 0x3, }; +enum { +MLX5_SQC_HAIRPIN_WQ_BUFFER_TYPE_INTERNAL_BUFFER =0x0 , +MLX5_SQC_HAIRPIN_WQ_BUFFER_TYPE_HOST_MEMORY =0x1 , +}; +struct mlx5_ifc_sqc_bits {u8 rlky[0x1 ];u8 cd_master[0x1 ];
@@ -2686,7 +2695,9 @@ struct mlx5_ifc_sqc_bits {u8 hairpin[0x1 ];u8 non_wire[0x1 ];u8 static_sq_wq[0x1 ]; -u8 reserved_at_11[0x9 ]; +u8 reserved_at_11[0x4 ]; +u8 hairpin_wq_buffer_type[0x3 ]; +u8 reserved_at_18[0x2 ];u8 ts_format[0x02 ];u8 reserved_at_1c[0x4 ];u8 reserved_at_20[0x8 ];
Signature · 1 line
2.25.1
PATCH v2 3/8 common/mlx5: add hairpin RQ buffer type capabilities
In reply to Dariusz SosnowskiThis patch adds new HCA capability related to hairpin RQs. This new capability, hairpin_data_buffer_locked, indicates whether HCA supports locking data buffer of hairpin RQ in ICMC (Interconnect Context Memory Cache).
Struct used to define RQ configuration (RQ context) is extended with hairpin_data_buffer_type field, which configures data buffer for hairpin RQ. It can take the following values:
- MLX5_RQC_HAIRPIN_DATA_BUFFER_TYPE_UNLOCKED_INTERNAL_BUFFER - hairpin RQ's data buffer is stored in unlocked memory in ICMC.
- MLX5_RQC_HAIRPIN_DATA_BUFFER_TYPE_LOCKED_INTERNAL_BUFFER - hairpin RQ's data buffer is stored in locked memory in ICMC.
--- drivers/common/mlx5/mlx5_devx_cmds.c | 3 +++ drivers/common/mlx5/mlx5_devx_cmds.h | 2 ++ drivers/common/mlx5/mlx5_prm.h | 12 ++++++++++-- 3 files changed, 15 insertions(+), 2 deletions(-)
drivers/common/mlx5/mlx5_devx_cmds.c +3 -0
@@ -993,6 +993,8 @@ mlx5_devx_cmd_query_hca_attr(void *ctx, hairpin_sq_wqe_bb_size); attr->hairpin_sq_wq_in_host_mem =MLX5_GET (cmd_hca_cap_2, hcattr, hairpin_sq_wq_in_host_mem); + attr->hairpin_data_buffer_locked =MLX5_GET (cmd_hca_cap_2, hcattr, + hairpin_data_buffer_locked); }if (attr->log_min_stride_wqe_sz ==0 ) attr->log_min_stride_wqe_sz =MLX5_MPRQ_LOG_MIN_STRIDE_WQE_SIZE ;
@@ -1293,6 +1295,7 @@ mlx5_devx_cmd_create_rq(void *ctx,MLX5_SET (rqc, rq_ctx, state, rq_attr->state);MLX5_SET (rqc, rq_ctx, flush_in_error_en, rq_attr->flush_in_error_en);MLX5_SET (rqc, rq_ctx, hairpin, rq_attr->hairpin); +MLX5_SET (rqc, rq_ctx, hairpin_data_buffer_type, rq_attr->hairpin_data_buffer_type);MLX5_SET (rqc, rq_ctx, user_index, rq_attr->user_index);MLX5_SET (rqc, rq_ctx, cqn, rq_attr->cqn);MLX5_SET (rqc, rq_ctx, counter_set_id, rq_attr->counter_set_id);
drivers/common/mlx5/mlx5_devx_cmds.h +2 -0
@@ -193,6 +193,7 @@ struct mlx5_hca_attr { uint32_t log_max_hairpin_num_packets:5; uint32_t hairpin_sq_wqe_bb_size:4; uint32_t hairpin_sq_wq_in_host_mem:1; + uint32_t hairpin_data_buffer_locked:1; uint32_t vhca_id:16; uint32_t relaxed_ordering_write:1; uint32_t relaxed_ordering_read:1;
@@ -313,6 +314,7 @@ struct mlx5_devx_create_rq_attr { uint32_t state:4; uint32_t flush_in_error_en:1; uint32_t hairpin:1; + uint32_t hairpin_data_buffer_type:3; uint32_t ts_format:2; uint32_t user_index:24; uint32_t cqn:24;
drivers/common/mlx5/mlx5_prm.h +10 -2
@@ -2024,7 +2024,8 @@ struct mlx5_ifc_cmd_hca_cap_2_bits {u8 reserved_at_160[0x3 ];u8 hairpin_sq_wqe_bb_size[0x5 ];u8 hairpin_sq_wq_in_host_mem[0x1 ]; -u8 reserved_at_169[0x697 ]; +u8 hairpin_data_buffer_locked[0x1 ]; +u8 reserved_at_16a[0x696 ]; };struct mlx5_ifc_esw_cap_bits {
@@ -2304,7 +2305,9 @@ struct mlx5_ifc_rqc_bits {u8 reserved_at_c[0x1 ];u8 flush_in_error_en[0x1 ];u8 hairpin[0x1 ]; -u8 reserved_at_f[0xB ]; +u8 reserved_at_f[0x6 ]; +u8 hairpin_data_buffer_type[0x3 ]; +u8 reserved_at_a8[0x2 ];u8 ts_format[0x02 ];u8 reserved_at_1c[0x4 ];u8 reserved_at_20[0x8 ];
@@ -2813,6 +2816,11 @@ enum { MLX5_CQE_SIZE_128B = 0x1, }; +enum { +MLX5_RQC_HAIRPIN_DATA_BUFFER_TYPE_UNLOCKED_INTERNAL_BUFFER =0x0 , +MLX5_RQC_HAIRPIN_DATA_BUFFER_TYPE_LOCKED_INTERNAL_BUFFER =0x1 , +}; +struct mlx5_ifc_cqc_bits {u8 status[0x4 ];u8 as_notify[0x1 ];
Signature · 1 line
2.25.1
PATCH v2 4/8 net/mlx5: allow hairpin Tx queue in RTE memory
In reply to Dariusz SosnowskiThis patch adds a capability to place hairpin Tx queue in host memory managed by DPDK. This capability is equivalent to storing hairpin SQ's WQ buffer in host memory.
Hairpin Tx queue creation is extended with allocating a memory buffer of proper size (calculated from required number of packets and WQE BB size advertised in HCA capabilities).
force_memory flag of hairpin queue configuration is also supported. If it is set and:
- allocation of memory buffer fails,
- or hairpin SQ creation fails,
then device start will fail. If it is unset, PMD will fallback to creating the hairpin SQ with WQ buffer located in unlocked device memory.
--- drivers/net/mlx5/mlx5.h | 2 + drivers/net/mlx5/mlx5_devx.c | 119 ++++++++++++++++++++++++++++++--- drivers/net/mlx5/mlx5_ethdev.c | 4 ++ 3 files changed, 116 insertions(+), 9 deletions(-)
drivers/net/mlx5/mlx5.h +2 -0
@@ -1386,6 +1386,8 @@ struct mlx5_txq_obj {struct mlx5_devx_obj *sq;/* DevX object for Sx queue. */ struct mlx5_devx_obj *tis;/* The TIS object. */ +void *umem_buf_wq_buffer; +void *umem_obj_wq_buffer; };struct {struct rte_eth_dev *dev;
drivers/net/mlx5/mlx5_devx.c +110 -9
155 lines not on this page · raw message
drivers/net/mlx5/mlx5_ethdev.c +4 -0
@@ -729,6 +729,7 @@ int mlx5_hairpin_cap_get(struct rte_eth_dev *dev, struct rte_eth_hairpin_cap *cap) { struct mlx5_priv *priv = dev->data->dev_private; +struct mlx5_hca_attr *hca_attr;if (!mlx5_devx_obj_ops_en (priv->sh)) { rte_errno =ENOTSUP ;
@@ -738,5 +739,8 @@ mlx5_hairpin_cap_get(struct rte_eth_dev *dev, struct rte_eth_hairpin_cap *cap) cap->max_rx_2_tx =1 ; cap->max_tx_2_rx =1 ; cap->max_nb_desc =8192 ; + hca_attr = &priv->sh->cdev->config.hca_attr; + cap->tx_cap.locked_device_memory =0 ; + cap->tx_cap.rte_memory = hca_attr->hairpin_sq_wq_in_host_mem;return 0 ; }
Signature · 1 line
2.25.1
PATCH v2 5/8 net/mlx5: allow hairpin Rx queue in locked memory
In reply to Dariusz SosnowskiThis patch adds a capability to place hairpin Rx queue in locked device memory. This capability is equivalent to storing hairpin RQ's data buffers in locked internal device memory.
Hairpin Rx queue creation is extended with requesting that RQ is allocated in locked internal device memory. If allocation fails and force_memory hairpin configuration is set, then hairpin queue creation (and, as a result, device start) fails. If force_memory is unset, then PMD will fallback to allocating memory for hairpin RQ in unlocked internal device memory.
To allow such allocation, the user must set HAIRPIN_DATA_BUFFER_LOCK flag in FW using mlxconfig tool.
--- doc/guides/platform/mlx5.rst | 5 ++++ drivers/net/mlx5/mlx5_devx.c | 51 ++++++++++++++++++++++++++++------ drivers/net/mlx5/mlx5_ethdev.c | 2 ++ 3 files changed, 49 insertions(+), 9 deletions(-)
doc/guides/platform/mlx5.rst +5 -0
@@ -555,6 +555,11 @@ Below are some firmware configurations listed. REAL_TIME_CLOCK_ENABLE=1 +- allow locking hairpin RQ data buffer in device memory:: + + HAIRPIN_DATA_BUFFER_LOCK=1 + MEMIC_SIZE_LIMIT=0 + .. _mlx5_common_driver_options:
drivers/net/mlx5/mlx5_devx.c +42 -9
82 lines not on this page · raw message
drivers/net/mlx5/mlx5_ethdev.c +2 -0
@@ -740,6 +740,8 @@ mlx5_hairpin_cap_get(struct rte_eth_dev *dev, struct rte_eth_hairpin_cap *cap) cap->max_tx_2_rx =1 ; cap->max_nb_desc =8192 ; hca_attr = &priv->sh->cdev->config.hca_attr; + cap->rx_cap.locked_device_memory = hca_attr->hairpin_data_buffer_locked; + cap->rx_cap.rte_memory =0 ; cap->tx_cap.locked_device_memory =0 ; cap->tx_cap.rte_memory = hca_attr->hairpin_sq_wq_in_host_mem;return 0 ;
Signature · 1 line
2.25.1
PATCH v2 6/8 doc: add notes for hairpin to mlx5 documentation
In reply to Dariusz SosnowskiThis patch extends mlx5 PMD documentation with more information regarding hairpin support.
The following is added to mlx5 PMD documentation:
- description of the default behavior of hairpin queues,
- description of use_locked_device_memory effect on hairpin queue configuration,
- description of use_rte_memory effect on hairpin queue configuration,
- DPDK and OFED requirements for new memory options for hairpin.
--- doc/guides/nics/mlx5.rst | 37 +++++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+)
doc/guides/nics/mlx5.rst +37 -0
@@ -1517,6 +1517,43 @@ behavior as librte_net_mlx4:: > port config all rss all > port start all +Notes for hairpin +----------------- + +NVIDIA Connect-X and BlueField devices support specifying memory +placement for hairpin Rx and Tx queues. This feature requires OFED 5.8. + +By default, data buffers and packet descriptors for hairpin queues are placed +in device memory which is shared with other resources (e.g. flow rules). + +Starting with DPDK 22.11 and OFED 5.8 applications are allowed to: + +#. Place data buffers and Rx packet descriptors in dedicated device memory. + Application can request that configuration through ``use_locked_device_memory`` + configuration option. + + Placing data buffers and Rx packet descriptors in dedicated device memory + can decrease latency on hairpinned traffic, since traffic processing + for the hairpin queue will not be memory starved. + + However, reserving device memory for hairpin Rx queues may decrease throughput + under heavy load, since less resources will be available on device. + + This option is supported only for Rx hairpin queues. + +#. Place Tx packet descriptors in host memory. + Application can request that configuration through ``use_rte_memory`` + configuration option. + + Placing Tx packet descritors in host memory can increase traffic throughput. + This results in more resources available on the device for other purposes, + which reduces memory contention on device. + Side effect of this option is visible increase in latency, since each packet + incurs additional PCI transactions. + + This option is supported only for Tx hairpin queues. + + Usage example -------------
Signature · 1 line
2.25.1
PATCH v2 7/8 app/testpmd: add hairpin queues memory modes
In reply to Dariusz SosnowskiThis patch extends hairpin-mode command line option of test-pmd application with an ability to configure whether Rx/Tx hairpin queue should use locked device memory or RTE memory.
For purposes of this configurations the following bits of 32 bit hairpin-mode are reserved:
- Bit 8 - If set, then force_memory flag will be set for hairpin RX queue.
- Bit 9 - If set, then force_memory flag will be set for hairpin TX queue.
- Bits 12-15 - Memory options for hairpin Rx queue:
- Bit 12 - If set, then use_locked_device_memory will be set.
- Bit 13 - If set, then use_rte_memory will be set.
- Bit 14 - Reserved for future use.
- Bit 15 - Reserved for future use.
- Bits 16-19 - Memory options for hairpin Tx queue:
- Bit 16 - If set, then use_locked_device_memory will be set.
- Bit 17 - If set, then use_rte_memory will be set.
- Bit 18 - Reserved for future use.
- Bit 19 - Reserved for future use.
--- app/test-pmd/parameters.c | 2 +- app/test-pmd/testpmd.c | 24 +++++++++++++++++++++++- app/test-pmd/testpmd.h | 2 +- doc/guides/testpmd_app_ug/run_app.rst | 10 ++++++++-- 4 files changed, 33 insertions(+), 5 deletions(-)
app/test-pmd/parameters.c +1 -1
@@ -1085,7 +1085,7 @@ launch_args_parse(int argc, char** argv)if (errno !=0 || end == optarg)rte_exit (EXIT_FAILURE ,"hairpin mode invalid\n" );else - hairpin_mode = (uint16_t )n; + hairpin_mode = (uint32_t )n; }if (!strcmp (lgopts[opt_idx].name,"burst" )) { n =atoi (optarg);
app/test-pmd/testpmd.c +23 -1
@@ -409,7 +409,7 @@ bool setup_on_probe_event = true;uint8_t clear_ptypes = true;/* Hairpin ports configuration mode. */ -uint16_t hairpin_mode; +uint32_t hairpin_mode;/* Pretty printing of ethdev events */ static const char *const eth_event_desc[] = {
@@ -2519,6 +2519,16 @@ port_is_started(portid_t port_id)return 1 ; } +#define HAIRPIN_MODE_RX_FORCE_MEMORY RTE_BIT32(8) +#define HAIRPIN_MODE_TX_FORCE_MEMORY RTE_BIT32(9) + +#define HAIRPIN_MODE_RX_LOCKED_MEMORY RTE_BIT32(12) +#define HAIRPIN_MODE_RX_RTE_MEMORY RTE_BIT32(13) + +#define HAIRPIN_MODE_TX_LOCKED_MEMORY RTE_BIT32(16) +#define HAIRPIN_MODE_TX_RTE_MEMORY RTE_BIT32(17) + +/* Configure the Rx and Tx hairpin queues for the selected port. */ static int setup_hairpin_queues (portid_t pi,portid_t p_pi,uint16_t cnt_pi)
@@ -2534,6 +2544,12 @@ setup_hairpin_queues(portid_t pi, portid_t p_pi, uint16_t cnt_pi)uint16_t peer_tx_port = pi;uint32_t manual =1 ;uint32_t tx_exp = hairpin_mode &0x10 ; +uint32_t rx_force_memory = hairpin_mode &HAIRPIN_MODE_RX_FORCE_MEMORY ; +uint32_t rx_locked_memory = hairpin_mode &HAIRPIN_MODE_RX_LOCKED_MEMORY ; +uint32_t rx_rte_memory = hairpin_mode &HAIRPIN_MODE_RX_RTE_MEMORY ; +uint32_t tx_force_memory = hairpin_mode &HAIRPIN_MODE_TX_FORCE_MEMORY ; +uint32_t tx_locked_memory = hairpin_mode &HAIRPIN_MODE_TX_LOCKED_MEMORY ; +uint32_t tx_rte_memory = hairpin_mode &HAIRPIN_MODE_TX_RTE_MEMORY ;if (!(hairpin_mode &0xf )) { peer_rx_port = pi;
@@ -2573,6 +2589,9 @@ setup_hairpin_queues(portid_t pi, portid_t p_pi, uint16_t cnt_pi) hairpin_conf.peers[0 ].queue = i + nb_rxq; hairpin_conf.manual_bind = !!manual; hairpin_conf.tx_explicit = !!tx_exp; + hairpin_conf.force_memory = !!tx_force_memory; + hairpin_conf.use_locked_device_memory = !!tx_locked_memory; + hairpin_conf.use_rte_memory = !!tx_rte_memory; diag =rte_eth_tx_hairpin_queue_setup (pi, qi, nb_txd, &hairpin_conf); i++;
@@ -2596,6 +2615,9 @@ setup_hairpin_queues(portid_t pi, portid_t p_pi, uint16_t cnt_pi) hairpin_conf.peers[0 ].queue = i + nb_txq; hairpin_conf.manual_bind = !!manual; hairpin_conf.tx_explicit = !!tx_exp; + hairpin_conf.force_memory = !!rx_force_memory; + hairpin_conf.use_locked_device_memory = !!rx_locked_memory; + hairpin_conf.use_rte_memory = !!rx_rte_memory; diag =rte_eth_rx_hairpin_queue_setup (pi, qi, nb_rxd, &hairpin_conf); i++;
app/test-pmd/testpmd.h +1 -1
@@ -562,7 +562,7 @@ extern uint16_t stats_period;extern struct rte_eth_xstat_name *xstats_display;extern unsigned int xstats_display_num; -extern uint16_t hairpin_mode; +extern uint32_t hairpin_mode;#ifdef RTE_LIB_LATENCYSTATS extern uint8_t latencystats_enabled;
doc/guides/testpmd_app_ug/run_app.rst +8 -2
@@ -529,10 +529,16 @@ The command line options are: Enable display of RX and TX burst stats. -* ``--hairpin-mode=0xXX`` +* ``--hairpin-mode=0xXXXX`` - Set the hairpin port mode with bitmask, only valid when hairpin queues number is set:: + Set the hairpin port configuration with bitmask, only valid when hairpin queues number is set:: + bit 18 - hairpin TX queues will use RTE memory + bit 16 - hairpin TX queues will use locked device memory + bit 13 - hairpin RX queues will use RTE memory + bit 12 - hairpin RX queues will use locked device memory + bit 9 - force memory settings of hairpin TX queue + bit 8 - force memory settings of hairpin RX queue bit 4 - explicit Tx flow rule bit 1 - two hairpin ports paired bit 0 - two hairpin ports loop
Signature · 1 line
2.25.1
PATCH v2 8/8 app/flow-perf: add hairpin queue memory config
In reply to Dariusz SosnowskiThis patch adds the hairpin-conf command line parameter to flow-perf application. hairpin-conf parameter takes a hexadecimal bitmask with bits having the following meaning:
- Bit 0 - Force memory settings of hairpin RX queue.
- Bit 1 - Force memory settings of hairpin TX queue.
- Bit 4 - Use locked device memory for hairpin RX queue.
- Bit 5 - Use RTE memory for hairpin RX queue.
- Bit 8 - Use locked device memory for hairpin TX queue.
- Bit 9 - Use RTE memory for hairpin TX queue.
--- app/test-flow-perf/main.c | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+)
app/test-flow-perf/main.c +32 -0
@@ -46,6 +46,15 @@#define DEFAULT_RULES_BATCH 100000#define DEFAULT_GROUP 0 +#define HAIRPIN_RX_CONF_FORCE_MEMORY (0x0001) +#define HAIRPIN_TX_CONF_FORCE_MEMORY (0x0002) + +#define HAIRPIN_RX_CONF_LOCKED_MEMORY (0x0010) +#define HAIRPIN_RX_CONF_RTE_MEMORY (0x0020) + +#define HAIRPIN_TX_CONF_LOCKED_MEMORY (0x0100) +#define HAIRPIN_TX_CONF_RTE_MEMORY (0x0200) +struct rte_flow *flow;static uint8_t flow_group;
@@ -61,6 +70,7 @@ static uint32_t policy_id[MAX_PORTS];static uint8_t items_idx, actions_idx, attrs_idx;static uint64_t ports_mask; +static uint64_t hairpin_conf_mask;static uint16_t dst_ports[RTE_MAX_ETHPORTS ];static volatile bool force_quit;static bool dump_iterations;
@@ -482,6 +492,7 @@ usage(char *progname)printf (" --enable-fwd: To enable packets forwarding" " after insertion\n" );printf (" --portmask=N: hexadecimal bitmask of ports used\n" ); +printf (" --hairpin-conf=0xXXXX: hexadecimal bitmask of hairpin queue configuration\n" );printf (" --random-priority=N,S: use random priority levels " "from 0 to (N - 1) for flows " "and S as seed for pseudo-random number generator\n" );
@@ -629,6 +640,7 @@ static void args_parse(int argc, char **argv) { uint64_t pm, seed; +uint64_t hp_conf;char **argvopt;uint32_t prio;char *token;
@@ -648,6 +660,7 @@ args_parse(int argc, char **argv) { "enable-fwd", 0, 0, 0 }, { "unique-data", 0, 0, 0 }, { "portmask", 1, 0, 0 }, + { "hairpin-conf", 1, 0, 0 }, { "cores", 1, 0, 0 }, { "random-priority", 1, 0, 0 }, { "meter-profile-alg", 1, 0, 0 },
@@ -880,6 +893,13 @@ args_parse(int argc, char **argv)rte_exit (EXIT_FAILURE ,"Invalid fwd port mask\n" ); ports_mask = pm; } +if (strcmp (lgopts[opt_idx].name,"hairpin-conf" ) ==0 ) { + end =NULL ; + hp_conf =strtoull (optarg, &end,16 ); +if ((optarg[0 ] =='\0' ) || (end ==NULL ) || (*end !='\0' )) +rte_exit (EXIT_FAILURE ,"Invalid hairpin config mask\n" ); + hairpin_conf_mask = hp_conf; + }if (strcmp (lgopts[opt_idx].name,"port-id" ) ==0 ) {uint16_t port_idx =0 ;
@@ -2035,6 +2055,12 @@ init_port(void) hairpin_conf.peers[0 ].port = port_id; hairpin_conf.peers[0 ].queue = std_queue + tx_queues_count; + hairpin_conf.use_locked_device_memory = + !!(hairpin_conf_mask &HAIRPIN_RX_CONF_LOCKED_MEMORY ); + hairpin_conf.use_rte_memory = + !!(hairpin_conf_mask &HAIRPIN_RX_CONF_RTE_MEMORY ); + hairpin_conf.force_memory = + !!(hairpin_conf_mask &HAIRPIN_RX_CONF_FORCE_MEMORY ); ret =rte_eth_rx_hairpin_queue_setup ( port_id, hairpin_queue, rxd_count, &hairpin_conf);
@@ -2050,6 +2076,12 @@ init_port(void) hairpin_conf.peers[0 ].port = port_id; hairpin_conf.peers[0 ].queue = std_queue + rx_queues_count; + hairpin_conf.use_locked_device_memory = + !!(hairpin_conf_mask &HAIRPIN_TX_CONF_LOCKED_MEMORY ); + hairpin_conf.use_rte_memory = + !!(hairpin_conf_mask &HAIRPIN_TX_CONF_RTE_MEMORY ); + hairpin_conf.force_memory = + !!(hairpin_conf_mask &HAIRPIN_TX_CONF_FORCE_MEMORY ); ret =rte_eth_tx_hairpin_queue_setup ( port_id, hairpin_queue, txd_count, &hairpin_conf);
Signature · 1 line
2.25.1
RE: [PATCH v2 8/8] app/flow-perf: add hairpin queue memory config
Hi,
Quoted 11 lines
-----Original Message-----
From: Dariusz Sosnowski <[email protected]>
Sent: Thursday, October 6, 2022 2:01 PM
To: Wisam Monther <[email protected]>
Cc: [email protected]
Subject: [PATCH v2 8/8] app/flow-perf: add hairpin queue memory configThis patch adds the hairpin-conf command line parameter to flow-perf application. hairpin-conf parameter takes a hexadecimal bitmask with bits having the following meaning:
Quoted 8 lines
- Bit 0 - Force memory settings of hairpin RX queue.
- Bit 1 - Force memory settings of hairpin TX queue.
- Bit 4 - Use locked device memory for hairpin RX queue.
- Bit 5 - Use RTE memory for hairpin RX queue.
- Bit 8 - Use locked device memory for hairpin TX queue.
- Bit 9 - Use RTE memory for hairpin TX queue.Signed-off-by Dariusz Sosnowski <[email protected]>
BRs,
Wisam Jaddo
06/10/2022 13:00, Dariusz Sosnowski:
Quoted 46 lines
The hairpin queues are used to transmit packets received on the wire, back to the wire. How hairpin queues are implemented and configured is decided internally by the PMD and applications have no control over the configuration of Rx and Tx hairpin queues. This patchset addresses that by:
- Extending hairpin queue capabilities reported by PMDs.
- Exposing new configuration options for Rx and Tx hairpin queues.Main goal of this patchset is to allow applications to provide configuration hints regarding memory placement of hairpin queues.
These hints specify whether buffers of hairpin queues should be placed in host memory or in dedicated device memory.For example, in context of NVIDIA Connect-X and BlueField devices, this distinction is important for several reasons:
- By default, data buffers and packet descriptors are placed in device memory region which is shared with other resources (e.g. flow rules).
This results in memory contention on the device,
which may lead to degraded performance under heavy load.
- Placing hairpin queues in dedicated device memory can decrease latency of hairpinned traffic, since hairpin queue processing will not be memory starved by other operations.
Side effect of this memory configuration is that it leaves less memory for other resources, possibly causing memory contention in non-hairpin traffic.
- Placing hairpin queues in host memory can increase throughput of hairpinned
traffic at the cost of increasing latency.
Each packet processed by hairpin queues will incur additional PCI transactions (increase in latency), but memory contention on the device is avoided.Depending on the workload and whether throughput or latency has a higher priority for developers, it would be beneficial if developers could choose the best hairpin configuration for their use case.
To address that, this patchset adds the following configuration options (in
rte_eth_hairpin_confstruct):-
use_locked_device_memory- If set, PMD will allocate specialized on-device memory for the queue.
-use_rte_memory- If set, PMD will use DPDK-managed memory for the queue.
-force_memory- If set, PMD will be forced to use provided memory configuration. If no appropriate resources are available, the queue allocation will fail.
If unset and no appropriate resources are available, PMD will fallback to its default behavior.Implementing support for these flags is optional and applications should be allowed to not set any of these new flags. This will result in default memory configuration provided by the PMD.
Application developers should consult the PMD documentation in that case.These changes were originally proposed in http://patches.dpdk.org/project/dpdk/patch/[email protected]/.
Quoted 9 lines
Dariusz Sosnowski (8):
ethdev: introduce hairpin memory capabilities common/mlx5: add hairpin SQ buffer type capabilities common/mlx5: add hairpin RQ buffer type capabilities net/mlx5: allow hairpin Tx queue in RTE memory net/mlx5: allow hairpin Rx queue in locked memory
doc: add notes for hairpin to mlx5 documentation app/testpmd: add hairpin queues memory modes app/flow-perf: add hairpin queue memory config
Doc squashed in mlx5 commits.
Applied, thanks.