Skip to contentSkip to conversation

[25.11] net/mlx5: add driver event and steering toggle APIs

PATCH New Maayan Kashani 1 participant 5 messages Aug 26, 2025

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover [25.11] net/mlx5: add driver event and steering toggle APIs

mlx5 PMD is a bifurcated driver, which means that instead of communicating with HW directly using UIO or VFIO, the driver uses existing kernel and userspace APIs for that purpose. mlx5 PMD requires that mlx5_core, mlx5_ib and ib_uverbs kernel modules are loaded, and that NIC is bound to mlx5_core driver.

Kernel bypass is achieved through rdma-core library
which exposes required functionalities e.g.,
mapping memory to the device, FW access for setting up Rx and Tx queues (through mlx5dv_devx_* interfaces) and flow rule offloading.

One benefit of such architecture is that it allows libraries and applications
to build additional features on top of DPDK, utilizing rdma-core directly.
For example it is possible to implement additional flow rule offloads.
This can be achieved by creating an IBV (IB Verbs) context and PD (protection domain) through rdma-core and sharing it with mlx5 PMD through cmd_fd and pd_handle device arguments provided during probing [1].
mlx5 PMD will then create all necessary resources for DPDK using imported context and PD.

What is missing for that approach to work, is that
external libraries and applications are not aware of HW identifiers of Rx and Tx queues (returned from mlx5dv_devx_* APIs) created by DPDK through imported context and PD. Knowledge of these identifiers is crucial for correct flow rule offload.

This patchset proposes the introduction of the following private PMD APIs to address the above use case:

1. mlx5 PMD driver events
2. mlx5 PMD steering toggle

mlx5 PMD driver events, exposed through rte_pmd_mlx5_driver_event_cb_register/unregister() functions, allow libraries and applications to register a custom callback.
This callback will be called whenever mlx5 PMD creates or destroys an Rx/Tx queue.
All necessary HW identifiers along with corresponding DPDK port and queue
indexes will be passed as parameters.

mlx5 PMD steering toggle, exposed through rte_pmd_mlx5_driver_disable/enable_steering() functions, allow libraries and applications to disable/enable flow rule offloading provided by mlx5 PMD. This is added to support use cases where library/application needs full control of flow rule offloading. Library/application will use data provided through driver events API to configure flow rules by itself. When flow rule offloading is disabled, DPDK flow API will return ENOSYS for mlx5 PMD.

Patches:

- Patch 1 - Add mlx5 PMD driver events private API
- Patches 2-3 - Minor clean ups in mlx5 PMD which make introduction of steering toggle private API easier.
- Patch 4 - Add mlx5 PMD steering toggle private API

[1]: https://doc.dpdk.org/guides/platform/mlx5.html#device-arguments

Dariusz Sosnowski (4):
net/mlx5: add driver event callbacks
net/mlx5: move eCPRI release function to flex
net/mlx5: rework Rx queue mark flag functions
net/mlx5: add steering toggle API

 drivers/net/mlx5/meson.build         |   1 +
 drivers/net/mlx5/mlx5.c              |  18 --
 drivers/net/mlx5/mlx5.h              |   1 +
 drivers/net/mlx5/mlx5_devx.c         |  17 ++
 drivers/net/mlx5/mlx5_driver_event.c | 300 +++++++++++++++++++++++++++
 drivers/net/mlx5/mlx5_driver_event.h |  18 ++
 drivers/net/mlx5/mlx5_flow.c         | 199 +++++++++++++++++-
 drivers/net/mlx5/mlx5_flow.h         |   7 +-
 drivers/net/mlx5/mlx5_flow_flex.c    |  18 ++
 drivers/net/mlx5/mlx5_trigger.c      |  30 +++
 drivers/net/mlx5/rte_pmd_mlx5.h      | 192 +++++++++++++++++
 11 files changed, 772 insertions(+), 29 deletions(-)
 create mode 100644 drivers/net/mlx5/mlx5_driver_event.c
 create mode 100644 drivers/net/mlx5/mlx5_driver_event.h
Signature · 1 line
2.21.0
#1 lore raw

PATCH 1/4 net/mlx5: add driver event callbacks

From: Dariusz Sosnowski <[email protected]>

mlx5 PMD is a bifurcated driver,
which means that instead of communicating with HW directly using UIO or VFIO, the driver uses existing kernel and userspace APIs for that purpose.

One specific area of this usage is creation and configuration of Rx and Tx queues. This is achieved through mlx5dv_devx_* family of APIs exposed by rdma-core.
They allow userspace processes to access NIC FW securely.

It is theoretically possible for other libraries or applications built on top of rdma-core to use Rx and Tx queues created by DPDK for example in HW offloading set up outside of DPDK. For example library/application can set up an offloaded flow rule which will direct the traffic to DPDK queue.

Such use case cannot be achieved right now, because neither DPDK nor mlx5 PMD expose the identifiers of Rx and Tx queues created through DevX.

This patch addresses such use case,
by adding new functions to mlx5 PMD private API:

- rte_pmd_mlx5_driver_event_cb_register()
- rte_pmd_mlx5_driver_event_cb_unregister()

These allow external users to register custom callbacks, which will be called whenever mlx5 PMD performs some operation (driver event) on a managed HW object, which was allocated through DevX.
At the moment the following driver events are supported:

- Rx queue creation,
- Tx queue destruction,
- Rx queue creation,
- Tx queue destruction.

---
 drivers/net/mlx5/meson.build         |   1 +
 drivers/net/mlx5/mlx5_devx.c         |  17 ++
 drivers/net/mlx5/mlx5_driver_event.c | 300 +++++++++++++++++++++++++++
 drivers/net/mlx5/mlx5_driver_event.h |  18 ++
 drivers/net/mlx5/rte_pmd_mlx5.h      | 136 ++++++++++++
 5 files changed, 472 insertions(+)
 create mode 100644 drivers/net/mlx5/mlx5_driver_event.c
 create mode 100644 drivers/net/mlx5/mlx5_driver_event.h
drivers/net/mlx5/meson.build +1 -0
index f16fe181930..d67fb0969f8 100644
@@ -18,6 +18,7 @@ headers = files('rte_pmd_mlx5.h')
 sources = files(
         'mlx5.c',
         'mlx5_devx.c',
+        'mlx5_driver_event.c',
         'mlx5_ethdev.c',
         'mlx5_flow.c',
         'mlx5_flow_aso.c',
drivers/net/mlx5/mlx5_devx.c +17 -0
index 10bd93c29a4..673c9f3902f 100644
@@ -20,6 +20,7 @@
 
 #include "mlx5.h"
 #include "mlx5_common_os.h"
+#include "mlx5_driver_event.h"
 #include "mlx5_tx.h"
 #include "mlx5_rx.h"
 #include "mlx5_utils.h"
@@ -239,6 +240,12 @@ mlx5_rxq_devx_obj_release(struct mlx5_rxq_priv *rxq)
 
 	if (rxq_obj == NULL)
 		return;
+	/*
+	 * Notify external users that Rx queue will be destroyed.
+	 * Skip notification for not started queues and internal drop queue.
+	 */
+	if (rxq->ctrl->started && rxq != rxq->priv->drop_queue.rxq)
+		mlx5_driver_event_notify_rxq_destroy(rxq);
 	if (rxq_obj->rxq_ctrl->is_hairpin) {
 		if (rxq_obj->rq == NULL)
 			return;
@@ -612,6 +619,8 @@ mlx5_rxq_obj_hairpin_new(struct mlx5_rxq_priv *rxq)
 		return -rte_errno;
 	}
 create_rq_set_state:
+	/* Notify external users that Rx queue was created. */
+	mlx5_driver_event_notify_rxq_create(rxq);
 	priv->dev_data->rx_queue_state[idx] = RTE_ETH_QUEUE_STATE_HAIRPIN;
 	return 0;
 }
@@ -691,6 +700,8 @@ mlx5_rxq_devx_obj_new(struct mlx5_rxq_priv *rxq)
 		}
 		rxq_ctrl->wqn = rxq->devx_rq.rq->id;
 	}
+	/* Notify external users that Rx queue was created. */
+	mlx5_driver_event_notify_rxq_create(rxq);
 	priv->dev_data->rx_queue_state[rxq->idx] = RTE_ETH_QUEUE_STATE_STARTED;
 	return 0;
 error:
@@ -1440,6 +1451,8 @@ mlx5_txq_obj_hairpin_new(struct rte_eth_dev *dev, uint16_t idx)
 		rte_errno = errno;
 		return -rte_errno;
 	}
+	/* Notify external users that Tx queue was created. */
+	mlx5_driver_event_notify_txq_create(txq_ctrl);
 	return 0;
 }
 
@@ -1670,6 +1683,8 @@ mlx5_txq_devx_obj_new(struct rte_eth_dev *dev, uint16_t idx)
 		priv->consec_tx_mem.cq_cur_off += txq_data->cq_mem_len;
 	ppriv->uar_table[txq_data->idx] = sh->tx_uar.bf_db;
 	dev->data->tx_queue_state[idx] = RTE_ETH_QUEUE_STATE_STARTED;
+	/* Notify external users that Tx queue was created. */
+	mlx5_driver_event_notify_txq_create(txq_ctrl);
 	return 0;
 error:
 	ret = rte_errno; /* Save rte_errno before cleanup. */
@@ -1689,6 +1704,8 @@ void
 mlx5_txq_devx_obj_release(struct mlx5_txq_obj *txq_obj)
 {
 	MLX5_ASSERT(txq_obj);
+	/* Notify external users that Tx queue will be destroyed. */
+	mlx5_driver_event_notify_txq_destroy(txq_obj->txq_ctrl);
 	if (txq_obj->txq_ctrl->is_hairpin) {
 		if (txq_obj->sq) {
 			claim_zero(mlx5_devx_cmd_destroy(txq_obj->sq));
drivers/net/mlx5/mlx5_driver_event.c new file +300 -0

300 lines not on this page · raw message

drivers/net/mlx5/mlx5_driver_event.h new file +18 -0
new file mode 100644
index 00000000000..ff4ce6e186f
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2025 NVIDIA Corporation & Affiliates
+ */
+
+#ifndef RTE_PMD_MLX5_DRIVER_EVENT_H_
+#define RTE_PMD_MLX5_DRIVER_EVENT_H_
+
+/* Forward declarations. */
+struct mlx5_rxq_priv;
+struct mlx5_txq_ctrl;
+
+void mlx5_driver_event_notify_rxq_create(struct mlx5_rxq_priv *rxq);
+void mlx5_driver_event_notify_rxq_destroy(struct mlx5_rxq_priv *rxq);
+
+void mlx5_driver_event_notify_txq_create(struct mlx5_txq_ctrl *txq_ctrl);
+void mlx5_driver_event_notify_txq_destroy(struct mlx5_txq_ctrl *txq_ctrl);
+
+#endif /* RTE_PMD_MLX5_DRIVER_EVENT_H_ */
drivers/net/mlx5/rte_pmd_mlx5.h +136 -0

149 lines not on this page · raw message

Signature · 1 line
2.21.0
#2 lore raw

PATCH 2/4 net/mlx5: move eCPRI release function to flex

In reply to Maayan Kashani

From: Dariusz Sosnowski <[email protected]>

Move mlx5_flex_parser_ecpri_release() from main source file to the file dedicated for flex parser logic.
Also, make the function non-static so the function can be used in flow steering clean up introduced in follow up patches.

---
 drivers/net/mlx5/mlx5.c           | 18 ------------------
 drivers/net/mlx5/mlx5.h           |  1 +
 drivers/net/mlx5/mlx5_flow_flex.c | 18 ++++++++++++++++++
 3 files changed, 19 insertions(+), 18 deletions(-)
drivers/net/mlx5/mlx5.c +0 -18
index 1321be779bb..ece29fb2164 100644
@@ -1044,24 +1044,6 @@ mlx5_flex_parser_ecpri_alloc(struct rte_eth_dev *dev)
 	return (rte_errno == 0) ? -ENODEV : -rte_errno;
 }
 
-/*
- * Destroy the flex parser node, including the parser itself, input / output
- * arcs and DW samples. Resources could be reused then.
- *
- * @param dev
- *   Pointer to Ethernet device structure.
- */
-static void
-mlx5_flex_parser_ecpri_release(struct rte_eth_dev *dev)
-{
-	struct mlx5_priv *priv = dev->data->dev_private;
-	struct mlx5_ecpri_parser_profile *prf = &priv->sh->ecpri_parser;
-
-	if (prf->obj)
-		mlx5_devx_cmd_destroy(prf->obj);
-	prf->obj = NULL;
-}
-
 /*
  * Allocation of a flex parser for srh. Once refcnt is zero, the resources held
  * by this parser will be freed.
drivers/net/mlx5/mlx5.h +1 -0
index 53f0a27445a..32f98f969ce 100644
@@ -2735,6 +2735,7 @@ int mlx5_flex_acquire_index(struct rte_eth_dev *dev,
 			    struct rte_flow_item_flex_handle *handle,
 			    bool acquire);
 int mlx5_flex_release_index(struct rte_eth_dev *dev, int index);
+void mlx5_flex_parser_ecpri_release(struct rte_eth_dev *dev);
 
 /* Flex parser list callbacks. */
 struct mlx5_list_entry *mlx5_flex_parser_create_cb(void *list_ctx, void *ctx);
drivers/net/mlx5/mlx5_flow_flex.c +18 -0
index afed16985ac..b1174fe0eda 100644
@@ -1557,3 +1557,21 @@ mlx5_flex_parser_clone_free_cb(void *list_ctx, struct mlx5_list_entry *entry)
 	RTE_SET_USED(list_ctx);
 	mlx5_free(fp);
 }
+
+/*
+ * Destroy the flex parser node, including the parser itself, input / output
+ * arcs and DW samples. Resources could be reused then.
+ *
+ * @param dev
+ *   Pointer to Ethernet device structure.
+ */
+void
+mlx5_flex_parser_ecpri_release(struct rte_eth_dev *dev)
+{
+	struct mlx5_priv *priv = dev->data->dev_private;
+	struct mlx5_ecpri_parser_profile *prf = &priv->sh->ecpri_parser;
+
+	if (prf->obj)
+		mlx5_devx_cmd_destroy(prf->obj);
+	prf->obj = NULL;
+}
Signature · 1 line
2.21.0
#3 lore raw

PATCH 3/4 net/mlx5: rework Rx queue mark flag functions

In reply to Maayan Kashani

From: Dariusz Sosnowski <[email protected]>

This patch:

- Renames flow_rxq_mark_flag_set() to mlx5_flow_rxq_mark_flag_set() so that it has a proper prefix and makes it non-static.
- Renames flow_rxq_flags_clear() to mlx5_flow_rxq_flags_clear() so that is has a proper prefix and makes it non-static.

Both of these functions will be used in a follow up patch to implement flow steering clean up when disabling steering.

---
 drivers/net/mlx5/mlx5_flow.c | 12 ++++++------
 drivers/net/mlx5/mlx5_flow.h |  4 +++-
 2 files changed, 9 insertions(+), 7 deletions(-)
drivers/net/mlx5/mlx5_flow.c +6 -6
index cc9ec73dfee..e6a057160cb 100644
@@ -1630,8 +1630,8 @@ flow_drv_rxq_flags_set(struct rte_eth_dev *dev,
 	}
 }
 
-static void
-flow_rxq_mark_flag_set(struct rte_eth_dev *dev)
+void
+mlx5_flow_rxq_mark_flag_set(struct rte_eth_dev *dev)
 {
 	struct mlx5_priv *priv = dev->data->dev_private;
 	struct mlx5_rxq_ctrl *rxq_ctrl;
@@ -1687,7 +1687,7 @@ flow_rxq_flags_set(struct rte_eth_dev *dev, struct rte_flow *flow)
 
 	MLX5_ASSERT(wks);
 	if (wks->mark)
-		flow_rxq_mark_flag_set(dev);
+		mlx5_flow_rxq_mark_flag_set(dev);
 	SILIST_FOREACH(priv->sh->ipool[MLX5_IPOOL_MLX5_FLOW], flow->dev_handles,
 		       handle_idx, dev_handle, next)
 		flow_drv_rxq_flags_set(dev, dev_handle);
@@ -1784,8 +1784,8 @@ flow_rxq_flags_trim(struct rte_eth_dev *dev, struct rte_flow *flow)
  * @param dev
  *   Pointer to Ethernet device.
  */
-static void
-flow_rxq_flags_clear(struct rte_eth_dev *dev)
+void
+mlx5_flow_rxq_flags_clear(struct rte_eth_dev *dev)
 {
 	struct mlx5_priv *priv = dev->data->dev_private;
 	unsigned int i;
@@ -8177,7 +8177,7 @@ mlx5_flow_stop_default(struct rte_eth_dev *dev)
 	}
 #endif
 	flow_mreg_del_default_copy_action(dev);
-	flow_rxq_flags_clear(dev);
+	mlx5_flow_rxq_flags_clear(dev);
 }
 
 /**
drivers/net/mlx5/mlx5_flow.h +3 -1
index e890e732c34..36be7660012 100644
@@ -3592,6 +3592,8 @@ int mlx5_flow_item_field_width(struct rte_eth_dev *dev,
 			   enum rte_flow_field_id field, int inherit,
 			   const struct rte_flow_attr *attr,
 			   struct rte_flow_error *error);
+void mlx5_flow_rxq_mark_flag_set(struct rte_eth_dev *dev);
+void mlx5_flow_rxq_flags_clear(struct rte_eth_dev *dev);
 uintptr_t flow_legacy_list_create(struct rte_eth_dev *dev, enum mlx5_flow_type type,
 				const struct rte_flow_attr *attr,
 				const struct rte_flow_item items[],
@@ -3768,5 +3770,5 @@ mlx5_flow_hw_action_flags_get(const struct rte_flow_action actions[],
 
 #include "mlx5_nta_sample.h"
 
-#endif
+#endif /* HAVE_MLX5_HWS_SUPPORT */
 #endif /* RTE_PMD_MLX5_FLOW_H_ */
Signature · 1 line
2.21.0
#4 lore raw

PATCH 4/4 net/mlx5: add steering toggle API

In reply to Maayan Kashani

From: Dariusz Sosnowski <[email protected]>

This patch adds:

- rte_pmd_mlx5_driver_disable_steering()
- rte_pmd_mlx5_driver_enable_steering()

private mlx5 PMD APIs, which allow applications to enable/disable flow rule handling in mlx5 PMD (both internally and externally managed). It allows applications (along with driver event callback API) to use external libraries to configure flow rules which would forward traffic to Rx and Tx queues managed by DPDK.

---
 drivers/net/mlx5/mlx5_flow.c    | 187 +++++++++++++++++++++++++++++++-
 drivers/net/mlx5/mlx5_flow.h    |   3 +
 drivers/net/mlx5/mlx5_trigger.c |  30 +++++
 drivers/net/mlx5/rte_pmd_mlx5.h |  56 ++++++++++
 4 files changed, 272 insertions(+), 4 deletions(-)
drivers/net/mlx5/mlx5_flow.c +183 -4
index e6a057160cb..1de398982a9 100644
@@ -8165,9 +8165,12 @@ mlx5_flow_list_flush(struct rte_eth_dev *dev, enum mlx5_flow_type type,
 void
 mlx5_flow_stop_default(struct rte_eth_dev *dev)
 {
-#ifdef HAVE_MLX5_HWS_SUPPORT
 	struct mlx5_priv *priv = dev->data->dev_private;
 
+	if (mlx5_flow_is_steering_disabled())
+		return;
+
+#ifdef HAVE_MLX5_HWS_SUPPORT
 	if (priv->sh->config.dv_flow_en == 2) {
 		mlx5_flow_nta_del_default_copy_action(dev);
 		if (!rte_atomic_load_explicit(&priv->hws_mark_refcnt,
@@ -8175,6 +8178,8 @@ mlx5_flow_stop_default(struct rte_eth_dev *dev)
 			flow_hw_rxq_flag_set(dev, false);
 		return;
 	}
+#else
+	RTE_SET_USED(priv);
 #endif
 	flow_mreg_del_default_copy_action(dev);
 	mlx5_flow_rxq_flags_clear(dev);
@@ -8220,10 +8225,12 @@ int
 mlx5_flow_start_default(struct rte_eth_dev *dev)
 {
 	struct rte_flow_error error;
-#ifdef HAVE_MLX5_HWS_SUPPORT
-	struct mlx5_priv *priv = dev->data->dev_private;
 
-	if (priv->sh->config.dv_flow_en == 2) {
+	if (mlx5_flow_is_steering_disabled())
+		return 0;
+
+#ifdef HAVE_MLX5_HWS_SUPPORT
+	if (MLX5_SH(dev)->config.dv_flow_en == 2) {
 		/*
 		 * Ignore this failure, if the proxy port is not started, other
 		 * default jump actions are not created and this rule will not
@@ -8879,6 +8886,13 @@ int
 mlx5_flow_ops_get(struct rte_eth_dev *dev __rte_unused,
 		  const struct rte_flow_ops **ops)
 {
+	if (mlx5_flow_is_steering_disabled()) {
+		DRV_LOG(WARNING, "port %u flow API is not supported since steering was disabled",
+			dev->data->port_id);
+		*ops = NULL;
+		return 0;
+	}
+
 	*ops = &mlx5_flow_ops;
 	return 0;
 }
@@ -12347,3 +12361,168 @@ mlx5_ctrl_flow_uc_dmac_vlan_exists(struct rte_eth_dev *dev,
 	}
 	return exists;
 }
+
+static bool mlx5_steering_disabled;
+
+bool
+mlx5_flow_is_steering_disabled(void)
+{
+	return mlx5_steering_disabled;
+}
+
+static void
+flow_disable_steering_flush(struct rte_eth_dev *dev)
+{
+	/*
+	 * This repeats the steps done in mlx5_dev_stop(), with a small difference:
+	 * - mlx5_flow_hw_cleanup_ctrl_rx_templates() and mlx5_action_handle_detach()
+	 * They are rearranged to make it work with different dev->data->dev_started.
+	 * Please see a TODO note in mlx5_dev_stop().
+	 */
+
+	mlx5_flow_stop_default(dev);
+	mlx5_traffic_disable(dev);
+	mlx5_flow_list_flush(dev, MLX5_FLOW_TYPE_GEN, true);
+	mlx5_flow_meter_rxq_flush(dev);
+#ifdef HAVE_MLX5_HWS_SUPPORT
+	mlx5_flow_hw_cleanup_ctrl_rx_templates(dev);
+#endif
+	mlx5_action_handle_detach(dev);
+}
+
+static void
+flow_disable_steering_cleanup(struct rte_eth_dev *dev)
+{
+	/*
+	 * See mlx5_dev_close(). Only steps not done on mlx5_dev_stop() are executed here.
+	 * Necessary steps are copied as is because steering resource cleanup in mlx5_dev_close()
+	 * is interleaved with other steps.
+	 * TODO: Rework steering resource cleanup in mlx5_dev_close() to allow code reuse.
+	 */
+
+	struct mlx5_priv *priv = dev->data->dev_private;
+
+	mlx5_action_handle_flush(dev);
+	mlx5_flow_meter_flush(dev, NULL);
+	mlx5_flex_parser_ecpri_release(dev);
+	mlx5_flex_item_port_cleanup(dev);
+	mlx5_indirect_list_handles_release(dev);
+#ifdef HAVE_MLX5_HWS_SUPPORT
+	flow_hw_destroy_vport_action(dev);
+	flow_hw_resource_release(dev);
+	flow_hw_clear_port_info(dev);
+	if (priv->tlv_options != NULL) {
+		/* Free the GENEVE TLV parser resource. */
+		claim_zero(mlx5_geneve_tlv_options_destroy(priv->tlv_options, priv->sh->phdev));
+		priv->tlv_options = NULL;
+	}
+	if (priv->ptype_rss_groups) {
+		mlx5_ipool_destroy(priv->ptype_rss_groups);
+		priv->ptype_rss_groups = NULL;
+	}
+	if (priv->dr_ctx) {
+		claim_zero(mlx5dr_context_close(priv->dr_ctx));
+		priv->dr_ctx = NULL;
+	}
+#else
+	RTE_SET_USED(priv);
+#endif
+}
+
+typedef void (*run_on_related_cb_t)(struct rte_eth_dev *dev);
+
+static void
+flow_disable_steering_run_on_related(struct rte_eth_dev *dev,
+				     run_on_related_cb_t cb)
+{
+	struct mlx5_priv *priv = dev->data->dev_private;
+	uint16_t other_port_id;
+	uint16_t proxy_port_id;
+	uint16_t port_id;
+	int ret __rte_unused;
+
+	if (priv->sh->config.dv_esw_en) {
+		ret = mlx5_flow_pick_transfer_proxy(dev, &proxy_port_id, NULL);
+		if (ret != 0) {
+			/*
+			 * This case should not happen because E-Switch is enabled.
+			 * However, in any case, release resources on the given port
+			 * and log the misconfigured port.
+			 */
+			DRV_LOG(ERR, "port %u unable to find transfer proxy port ret=%d",
+				priv->dev_data->port_id, ret);
+			cb(dev);
+			return;
+		}
+
+		/* Run callback on representors. */
+		MLX5_ETH_FOREACH_DEV(other_port_id, dev->device) {
+			struct rte_eth_dev *other_dev = &rte_eth_devices[other_port_id];
+
+			if (other_port_id != proxy_port_id)
+				cb(other_dev);
+		}
+
+		/* Run callback on proxy port. */
+		cb(&rte_eth_devices[proxy_port_id]);
+	} else if (rte_atomic_load_explicit(&priv->shared_refcnt, rte_memory_order_relaxed) > 0) {
+		/* Run callback on guest ports. */
+		MLX5_ETH_FOREACH_DEV(port_id, NULL) {
+			struct rte_eth_dev *other_dev = &rte_eth_devices[port_id];
+			struct mlx5_priv *other_priv = other_dev->data->dev_private;
+
+			if (other_priv->shared_host == dev)
+				cb(other_dev);
+		}
+
+		/* Run callback on host port. */
+		cb(dev);
+	} else {
+		cb(dev);
+	}
+}
+
+RTE_EXPORT_EXPERIMENTAL_SYMBOL(rte_pmd_mlx5_disable_steering, 25.11)
+void
+rte_pmd_mlx5_disable_steering(void)
+{
+	uint16_t port_id;
+
+	if (mlx5_steering_disabled)
+		return;
+
+	MLX5_ETH_FOREACH_DEV(port_id, NULL) {
+		struct rte_eth_dev *dev = &rte_eth_devices[port_id];
+
+		if (mlx5_hws_active(dev)) {
+			flow_disable_steering_run_on_related(dev, flow_disable_steering_flush);
+			flow_disable_steering_run_on_related(dev, flow_disable_steering_cleanup);
+		} else {
+			flow_disable_steering_flush(dev);
+			flow_disable_steering_cleanup(dev);
+		}
+
+		mlx5_flow_rxq_mark_flag_set(dev);
+	}
+
+	mlx5_steering_disabled = true;
+}
+
+RTE_EXPORT_EXPERIMENTAL_SYMBOL(rte_pmd_mlx5_enable_steering, 25.11)
+int
+rte_pmd_mlx5_enable_steering(void)
+{
+	uint16_t port_id;
+
+	if (!mlx5_steering_disabled)
+		return 0;
+
+	/* If any mlx5 port is probed, disallow enabling steering. */
+	port_id = mlx5_eth_find_next(0, NULL);
+	if (port_id != RTE_MAX_ETHPORTS)
+		return -EBUSY;
+
+	mlx5_steering_disabled = false;
+
+	return 0;
+}
drivers/net/mlx5/mlx5_flow.h +3 -0
index 36be7660012..8201b7aa4e3 100644
@@ -3670,6 +3670,9 @@ flow_hw_get_ipv6_route_ext_mod_id_from_ctx(void *dr_ctx, uint8_t idx)
 }
 void
 mlx5_indirect_list_handles_release(struct rte_eth_dev *dev);
+
+bool mlx5_flow_is_steering_disabled(void);
+
 #ifdef HAVE_MLX5_HWS_SUPPORT
 
 #define MLX5_REPR_STC_MEMORY_LOG 11
drivers/net/mlx5/mlx5_trigger.c +30 -0
index 6c6f228afd1..b104ca9f520 100644
@@ -1253,6 +1253,14 @@ mlx5_dev_start(struct rte_eth_dev *dev)
 	if (priv->sh->config.dv_flow_en == 2) {
 		struct rte_flow_error error = { 0, };
 
+		/*
+		 * If steering is disabled, then:
+		 * - There are no limitations regarding port start ordering,
+		 *   since no flow rules need to be created as part of port start.
+		 * - Non template API initialization will be skipped.
+		 */
+		if (mlx5_flow_is_steering_disabled())
+			goto continue_dev_start;
 		/*If previous configuration does not exist. */
 		if (!(priv->dr_ctx)) {
 			ret = flow_hw_init(dev, &error);
@@ -1420,6 +1428,8 @@ mlx5_dev_start(struct rte_eth_dev *dev)
 			dev->data->port_id, rte_strerror(rte_errno));
 		goto error;
 	}
+	if (mlx5_flow_is_steering_disabled())
+		mlx5_flow_rxq_mark_flag_set(dev);
 	rte_wmb();
 	dev->tx_pkt_burst = mlx5_select_tx_function(dev);
 	dev->rx_pkt_burst = mlx5_select_rx_function(dev);
@@ -1530,6 +1540,13 @@ mlx5_dev_stop(struct rte_eth_dev *dev)
 
 #ifdef HAVE_MLX5_HWS_SUPPORT
 	if (priv->sh->config.dv_flow_en == 2) {
+		/*
+		 * If steering is disabled,
+		 * then there are no limitations regarding port stop ordering,
+		 * since no flow rules need to be destroyed as part of port stop.
+		 */
+		if (mlx5_flow_is_steering_disabled())
+			goto continue_dev_stop;
 		/* If there is no E-Switch, then there are no start/stop order limitations. */
 		if (!priv->sh->config.dv_esw_en)
 			goto continue_dev_stop;
@@ -1552,6 +1569,8 @@ mlx5_dev_stop(struct rte_eth_dev *dev)
 	mlx5_mp_os_req_stop_rxtx(dev);
 	rte_delay_us_sleep(1000 * priv->rxqs_n);
 	DRV_LOG(DEBUG, "port %u stopping device", dev->data->port_id);
+	if (mlx5_flow_is_steering_disabled())
+		mlx5_flow_rxq_flags_clear(dev);
 	mlx5_flow_stop_default(dev);
 	/* Control flows for default traffic can be removed firstly. */
 	mlx5_traffic_disable(dev);
@@ -1692,6 +1711,9 @@ mlx5_traffic_enable(struct rte_eth_dev *dev)
 	unsigned int j;
 	int ret;
 
+	if (mlx5_flow_is_steering_disabled())
+		return 0;
+
 #ifdef HAVE_MLX5_HWS_SUPPORT
 	if (priv->sh->config.dv_flow_en == 2)
 		return mlx5_traffic_enable_hws(dev);
@@ -1878,6 +1900,9 @@ mlx5_traffic_disable_legacy(struct rte_eth_dev *dev)
 void
 mlx5_traffic_disable(struct rte_eth_dev *dev)
 {
+	if (mlx5_flow_is_steering_disabled())
+		return;
+
 #ifdef HAVE_MLX5_HWS_SUPPORT
 	struct mlx5_priv *priv = dev->data->dev_private;
 
@@ -1900,6 +1925,9 @@ mlx5_traffic_disable(struct rte_eth_dev *dev)
 int
 mlx5_traffic_restart(struct rte_eth_dev *dev)
 {
+	if (mlx5_flow_is_steering_disabled())
+		return 0;
+
 	if (dev->data->dev_started) {
 		mlx5_traffic_disable(dev);
 #ifdef HAVE_MLX5_HWS_SUPPORT
@@ -1915,6 +1943,8 @@ mac_flows_update_needed(struct rte_eth_dev *dev)
 {
 	struct mlx5_priv *priv = dev->data->dev_private;
 
+	if (mlx5_flow_is_steering_disabled())
+		return false;
 	if (!dev->data->dev_started)
 		return false;
 	if (dev->data->promiscuous)
drivers/net/mlx5/rte_pmd_mlx5.h +56 -0

62 lines not on this page · raw message

Signature · 1 line
2.21.0