Skip to contentSkip to conversation

mlx5: Support setting a parent for a devlink rate node

PATCH Applied Tariq Toukan 4 participants 8 messages Mar 10, 2025 – Mar 18, 2025

Subsystems (4)
lore
Share & tools
lore
author maintainer
#2 lore raw

PATCH 2/4 net/mlx5: Introduce hierarchy level tracking on scheduling nodes

In reply to Tariq Toukan

From: Carolina Jubran <[email protected]>

Add a level field to mlx5_esw_sched_node to track the hierarchy depth of each scheduling node. This allows enforcement of the scheduling depth constraints based on log_esw_max_sched_depth.

Modify esw_qos_node_set_parent() and __esw_qos_alloc_node() to correctly assign hierarchy levels. Ensure that nodes inherit their parent’s level incrementally.

---
 .../net/ethernet/mellanox/mlx5/core/esw/qos.c | 19 ++++++++++++++++---
 1 file changed, 16 insertions(+), 3 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/esw/qos.c +16 -3
index c56027838a57..959e4446327d 100644
@@ -90,8 +90,22 @@ struct mlx5_esw_sched_node {
 	struct list_head children;
 	/* Valid only if this node is associated with a vport. */
 	struct mlx5_vport *vport;
+	/* Level in the hierarchy. The root node level is 1. */
+	u8 level;
 };
 
+static void esw_qos_node_attach_to_parent(struct mlx5_esw_sched_node *node)
+{
+	if (!node->parent) {
+		/* Root children are assigned a depth level of 2. */
+		node->level = 2;
+		list_add_tail(&node->entry, &node->esw->qos.domain->nodes);
+	} else {
+		node->level = node->parent->level + 1;
+		list_add_tail(&node->entry, &node->parent->children);
+	}
+}
+
 static void
 esw_qos_node_set_parent(struct mlx5_esw_sched_node *node, struct mlx5_esw_sched_node *parent)
 {
@@ -99,6 +113,7 @@ esw_qos_node_set_parent(struct mlx5_esw_sched_node *node, struct mlx5_esw_sched_
 	node->parent = parent;
 	list_add_tail(&node->entry, &parent->children);
 	node->esw = parent->esw;
+	node->level = parent->level + 1;
 }
 
 void mlx5_esw_qos_vport_qos_free(struct mlx5_vport *vport)
@@ -358,7 +373,6 @@ static struct mlx5_esw_sched_node *
 __esw_qos_alloc_node(struct mlx5_eswitch *esw, u32 tsar_ix, enum sched_node_type type,
 		     struct mlx5_esw_sched_node *parent)
 {
-	struct list_head *parent_children;
 	struct mlx5_esw_sched_node *node;
 
 	node = kzalloc(sizeof(*node), GFP_KERNEL);
@@ -370,8 +384,7 @@ __esw_qos_alloc_node(struct mlx5_eswitch *esw, u32 tsar_ix, enum sched_node_type
 	node->type = type;
 	node->parent = parent;
 	INIT_LIST_HEAD(&node->children);
-	parent_children = parent ? &parent->children : &esw->qos.domain->nodes;
-	list_add_tail(&node->entry, parent_children);
+	esw_qos_node_attach_to_parent(node);
 
 	return node;
 }
Signature · 1 line
2.31.1

Tariq Toukan

author maintainer
#0 lore raw

cover mlx5: Support setting a parent for a devlink rate node

Hi,

This series by Carolina adds mlx5 support for the setting of a parent to devlink rate nodes.

By introducing a hierarchical level to scheduling nodes, these changes allow for more granular control over bandwidth allocation and isolation of Virtual Functions.

Function renaming for parent setting on leafs:
- net/mlx5: Rename devlink rate parent set function for leaf nodes

Add support for hierarchy level tracking:
- net/mlx5: Introduce hierarchy level tracking on scheduling nodes
- net/mlx5: Preserve rate settings when creating a rate node

Support setting parent for rate nodes:
- net/mlx5: Add support for setting parent of nodes

Regards,
Tariq

Carolina Jubran (4):
net/mlx5: Rename devlink rate parent set function for leaf nodes net/mlx5: Introduce hierarchy level tracking on scheduling nodes net/mlx5: Preserve rate settings when creating a rate node net/mlx5: Add support for setting parent of nodes

 .../net/ethernet/mellanox/mlx5/core/devlink.c |   3 +-
 .../net/ethernet/mellanox/mlx5/core/esw/qos.c | 146 ++++++++++++++++--
 .../net/ethernet/mellanox/mlx5/core/esw/qos.h |  12 +-
 3 files changed, 143 insertions(+), 18 deletions(-)

base-commit: 8ef890df4031121a94407c84659125cbccd3fdbe

Signature · 1 line
2.31.1
author maintainer
#1 lore raw

PATCH 1/4 net/mlx5: Rename devlink rate parent set function for leaf nodes

From: Carolina Jubran <[email protected]>

Rename mlx5_esw_devlink_rate_parent_set() to mlx5_esw_devlink_rate_leaf_parent_set() to distinguish setting a parent for leafs from nodes, which is not yet supported.

---
 drivers/net/ethernet/mellanox/mlx5/core/devlink.c | 2 +-
 drivers/net/ethernet/mellanox/mlx5/core/esw/qos.c | 8 ++++----
 drivers/net/ethernet/mellanox/mlx5/core/esw/qos.h | 8 ++++----
 3 files changed, 9 insertions(+), 9 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/devlink.c +1 -1
index 98d4306929f3..39202540a142 100644
@@ -324,7 +324,7 @@ static const struct devlink_ops mlx5_devlink_ops = {
 	.rate_node_tx_max_set = mlx5_esw_devlink_rate_node_tx_max_set,
 	.rate_node_new = mlx5_esw_devlink_rate_node_new,
 	.rate_node_del = mlx5_esw_devlink_rate_node_del,
-	.rate_leaf_parent_set = mlx5_esw_devlink_rate_parent_set,
+	.rate_leaf_parent_set = mlx5_esw_devlink_rate_leaf_parent_set,
 #endif
 #ifdef CONFIG_MLX5_SF_MANAGER
 	.port_new = mlx5_devlink_sf_port_new,
drivers/net/ethernet/mellanox/mlx5/core/esw/qos.c +4 -4
index 823c1ba456cd..c56027838a57 100644
@@ -986,10 +986,10 @@ int mlx5_esw_qos_vport_update_parent(struct mlx5_vport *vport, struct mlx5_esw_s
 	return err;
 }
 
-int mlx5_esw_devlink_rate_parent_set(struct devlink_rate *devlink_rate,
-				     struct devlink_rate *parent,
-				     void *priv, void *parent_priv,
-				     struct netlink_ext_ack *extack)
+int mlx5_esw_devlink_rate_leaf_parent_set(struct devlink_rate *devlink_rate,
+					  struct devlink_rate *parent,
+					  void *priv, void *parent_priv,
+					  struct netlink_ext_ack *extack)
 {
 	struct mlx5_esw_sched_node *node;
 	struct mlx5_vport *vport = priv;
drivers/net/ethernet/mellanox/mlx5/core/esw/qos.h +4 -4
index 6eb8f6a648c8..43a40bda7d19 100644
@@ -29,10 +29,10 @@ int mlx5_esw_devlink_rate_node_new(struct devlink_rate *rate_node, void **priv,
 				   struct netlink_ext_ack *extack);
 int mlx5_esw_devlink_rate_node_del(struct devlink_rate *rate_node, void *priv,
 				   struct netlink_ext_ack *extack);
-int mlx5_esw_devlink_rate_parent_set(struct devlink_rate *devlink_rate,
-				     struct devlink_rate *parent,
-				     void *priv, void *parent_priv,
-				     struct netlink_ext_ack *extack);
+int mlx5_esw_devlink_rate_leaf_parent_set(struct devlink_rate *devlink_rate,
+					  struct devlink_rate *parent,
+					  void *priv, void *parent_priv,
+					  struct netlink_ext_ack *extack);
 #endif
 
 #endif
Signature · 1 line
2.31.1
author maintainer
#3 lore raw

PATCH 3/4 net/mlx5: Preserve rate settings when creating a rate node

In reply to Tariq Toukan

From: Carolina Jubran <[email protected]>

Modify esw_qos_create_node_sched_elem() to receive max_rate and bw_share values while maintaining the previous configuration.

This change is essential for the upcoming patch that will modify rate nodes and requires the existing settings to be preserved unless explicitly changed.

---
 drivers/net/ethernet/mellanox/mlx5/core/esw/qos.c | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/esw/qos.c +9 -4
index 959e4446327d..3c850efb4ca3 100644
@@ -320,8 +320,9 @@ static int esw_qos_set_node_min_rate(struct mlx5_esw_sched_node *node,
 	return 0;
 }
 
-static int esw_qos_create_node_sched_elem(struct mlx5_core_dev *dev, u32 parent_element_id,
-					  u32 *tsar_ix)
+static int
+esw_qos_create_node_sched_elem(struct mlx5_core_dev *dev, u32 parent_element_id,
+			       u32 max_rate, u32 bw_share, u32 *tsar_ix)
 {
 	u32 tsar_ctx[MLX5_ST_SZ_DW(scheduling_context)] = {};
 	void *attr;
@@ -338,6 +339,8 @@ static int esw_qos_create_node_sched_elem(struct mlx5_core_dev *dev, u32 parent_
 		 SCHEDULING_CONTEXT_ELEMENT_TYPE_TSAR);
 	MLX5_SET(scheduling_context, tsar_ctx, parent_element_id,
 		 parent_element_id);
+	MLX5_SET(scheduling_context, tsar_ctx, max_average_bw, max_rate);
+	MLX5_SET(scheduling_context, tsar_ctx, bw_share, bw_share);
 	attr = MLX5_ADDR_OF(scheduling_context, tsar_ctx, element_attributes);
 	MLX5_SET(tsar_element, attr, tsar_type, TSAR_ELEMENT_TSAR_TYPE_DWRR);
 
@@ -409,7 +412,8 @@ __esw_qos_create_vports_sched_node(struct mlx5_eswitch *esw, struct mlx5_esw_sch
 	u32 tsar_ix;
 	int err;
 
-	err = esw_qos_create_node_sched_elem(esw->dev, esw->qos.root_tsar_ix, &tsar_ix);
+	err = esw_qos_create_node_sched_elem(esw->dev, esw->qos.root_tsar_ix, 0,
+					     0, &tsar_ix);
 	if (err) {
 		NL_SET_ERR_MSG_MOD(extack, "E-Switch create TSAR for node failed");
 		return ERR_PTR(err);
@@ -476,7 +480,8 @@ static int esw_qos_create(struct mlx5_eswitch *esw, struct netlink_ext_ack *exta
 	if (!MLX5_CAP_GEN(dev, qos) || !MLX5_CAP_QOS(dev, esw_scheduling))
 		return -EOPNOTSUPP;
 
-	err = esw_qos_create_node_sched_elem(esw->dev, 0, &esw->qos.root_tsar_ix);
+	err = esw_qos_create_node_sched_elem(esw->dev, 0, 0, 0,
+					     &esw->qos.root_tsar_ix);
 	if (err) {
 		esw_warn(dev, "E-Switch create root TSAR failed (%d)\n", err);
 		return err;
Signature · 1 line
2.31.1
author maintainer
#4 lore raw

PATCH 4/4 net/mlx5: Add support for setting parent of nodes

In reply to Tariq Toukan

From: Carolina Jubran <[email protected]>

Introduce mlx5_esw_devlink_rate_node_parent_set() to allow assigning a parent to scheduling nodes.
Implement mlx5_esw_qos_node_update_parent() and mlx5_esw_qos_node_validate_set_parent() to enforce constraints on node reassignment.

Don't allow reassignment of nodes with active rate objects.

Update esw_qos_node_set_parent() to handle cases where
the parent is NULL. A NULL parent indicates that the scheduling element is attached to the root scheduling element, and since only rate nodes can be connected to the root, this update is now necessary.

---
 .../net/ethernet/mellanox/mlx5/core/devlink.c |   1 +
 .../net/ethernet/mellanox/mlx5/core/esw/qos.c | 108 +++++++++++++++++-
 .../net/ethernet/mellanox/mlx5/core/esw/qos.h |   4 +
 3 files changed, 110 insertions(+), 3 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/devlink.c +1 -0
index 39202540a142..df5a2e717ddd 100644
@@ -325,6 +325,7 @@ static const struct devlink_ops mlx5_devlink_ops = {
 	.rate_node_new = mlx5_esw_devlink_rate_node_new,
 	.rate_node_del = mlx5_esw_devlink_rate_node_del,
 	.rate_leaf_parent_set = mlx5_esw_devlink_rate_leaf_parent_set,
+	.rate_node_parent_set = mlx5_esw_devlink_rate_node_parent_set,
 #endif
 #ifdef CONFIG_MLX5_SF_MANAGER
 	.port_new = mlx5_devlink_sf_port_new,
drivers/net/ethernet/mellanox/mlx5/core/esw/qos.c +105 -3

117 lines not on this page · raw message

drivers/net/ethernet/mellanox/mlx5/core/esw/qos.h +4 -0
index 43a40bda7d19..ed40ec8f027e 100644
@@ -33,6 +33,10 @@ int mlx5_esw_devlink_rate_leaf_parent_set(struct devlink_rate *devlink_rate,
 					  struct devlink_rate *parent,
 					  void *priv, void *parent_priv,
 					  struct netlink_ext_ack *extack);
+int mlx5_esw_devlink_rate_node_parent_set(struct devlink_rate *devlink_rate,
+					  struct devlink_rate *parent,
+					  void *priv, void *parent_priv,
+					  struct netlink_ext_ack *extack);
 #endif
 
 #endif
Signature · 1 line
2.31.1
Reviewed-by on the v1 series
#6 lore raw
In reply to Tariq Toukan
Quoted 19 lines · Tariq Toukan

Hi,

This series by Carolina adds mlx5 support for the setting of a parent to devlink rate nodes.

By introducing a hierarchical level to scheduling nodes, these changes allow for more granular control over bandwidth allocation and isolation of Virtual Functions.

Function renaming for parent setting on leafs:
- net/mlx5: Rename devlink rate parent set function for leaf nodes

Add support for hierarchy level tracking:
- net/mlx5: Introduce hierarchy level tracking on scheduling nodes
- net/mlx5: Preserve rate settings when creating a rate node

Support setting parent for rate nodes:
- net/mlx5: Add support for setting parent of nodes

Quoted 9 lines

Regards,
Tariq

Carolina Jubran (4):
net/mlx5: Rename devlink rate parent set function for leaf nodes net/mlx5: Introduce hierarchy level tracking on scheduling nodes net/mlx5: Preserve rate settings when creating a rate node net/mlx5: Add support for setting parent of nodes

Nice work. The whole series was a pleasant read.

Quoted 7 lines
 .../net/ethernet/mellanox/mlx5/core/devlink.c |   3 +-
 .../net/ethernet/mellanox/mlx5/core/esw/qos.c | 146 ++++++++++++++++--
 .../net/ethernet/mellanox/mlx5/core/esw/qos.h |  12 +-
 3 files changed, 143 insertions(+), 18 deletions(-)

base-commit: 8ef890df4031121a94407c84659125cbccd3fdbe

maintainer
#5 lore raw

Re: [PATCH net-next 4/4] net/mlx5: Add support for setting parent of nodes

In reply to Tariq Toukan
Quoted 19 lines · Tariq Toukan
drivers/net/ethernet/mellanox/mlx5/core/esw/qos.c +15 -0
@@ -1018,3 +1018,105 @@ int mlx5_esw_devlink_rate_leaf_parent_set(struct devlink_rate *devlink_rate,
 	node = parent_priv;
 	return mlx5_esw_qos_vport_update_parent(vport, node, extack);
 }
+
+static int
+mlx5_esw_qos_node_validate_set_parent(struct mlx5_esw_sched_node *node,
+				      struct mlx5_esw_sched_node *parent,
+				      struct netlink_ext_ack *extack)
+{
+	u8 new_level, max_level;
+
+	if (parent && parent->esw != node->esw) {
+		NL_SET_ERR_MSG_MOD(extack,
+				   "Cannot assign node to another E-Switch");
+		return -EOPNOTSUPP;
+	}
+
+	if (!list_empty(&node->children)) {

drivers/net/ethernet/mellanox/mlx5/core/esw/qos.c int mlx5_esw_devlink_rate_leaf_parent_set(struct devlink_rate *devlink_rate, · line 1045

Quoted 10 lines
drivers/net/ethernet/mellanox/mlx5/core/esw/qos.c +10 -0
+		NL_SET_ERR_MSG_MOD(extack,
+				   "Cannot reassign a node that contains rate objects");
+		return -EOPNOTSUPP;
+	}
+
+	new_level = parent ? parent->level + 1 : 2;
+	max_level = 1 << MLX5_CAP_QOS(node->esw->dev, log_esw_max_sched_depth);
+	if (new_level > max_level) {
+		NL_SET_ERR_MSG_MOD(extack,
+				   "Node hierarchy depth exceeds the maximum supported level");

Minor nit for a possible small follow-up: it could be possibly useful to add to the extact the current and max level.

/P

#7 lore raw
In reply to Tariq Toukan
This series was applied to netdev/net-next.git (main) by Paolo Abeni <[email protected]>:

Hello:

This series was applied to netdev/net-next.git (main) by Paolo Abeni <[email protected]>:

Quoted 10 lines · On Mon, 10 Mar 2025 23:26:52 +0200 you wrote:

Hi,

This series by Carolina adds mlx5 support for the setting of a parent to devlink rate nodes.

By introducing a hierarchical level to scheduling nodes, these changes allow for more granular control over bandwidth allocation and isolation of Virtual Functions.

[...]

Here is the summary with links:
- [net-next,1/4] net/mlx5: Rename devlink rate parent set function for leaf nodes

    https://git.kernel.org/netdev/net-next/c/b407b4b804cd
  - [net-next,2/4] net/mlx5: Introduce hierarchy level tracking on scheduling nodes
    https://git.kernel.org/netdev/net-next/c/498bd79cb92b
  - [net-next,3/4] net/mlx5: Preserve rate settings when creating a rate node
    https://git.kernel.org/netdev/net-next/c/f88c349c75e3
  - [net-next,4/4] net/mlx5: Add support for setting parent of nodes
    https://git.kernel.org/netdev/net-next/c/9c7bbf4c3304

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html