Skip to contentSkip to conversation

Add multiple priorities support to mlx5 RDMA TRANSPORT tables

PATCH In review Leon Romanovsky 3 participants 7 messages Jun 17, 2025 – Jun 26, 2025

Subsystems (5)
Networking Network drivers InfiniBand Mellanox mlx5 core +1 more subsystems
lore
Share & tools
lore

Leon Romanovsky

author
#0 lore raw

cover Add multiple priorities support to mlx5 RDMA TRANSPORT tables

Hi,

This short series from Patrisious extends mlx5 flow steering logic to allow creation rule creation with priorities in RDMA TRANSPORT tables.

Thanks

Patrisious Haddad (2):
net/mlx5: fs, add multiple prios to RDMA TRANSPORT steering domain RDMA/mlx5: Add multiple priorities support to RDMA TRANSPORT userspace
tables

 drivers/infiniband/hw/mlx5/fs.c               | 40 ++++++++++++-------
 drivers/infiniband/hw/mlx5/fs.h               |  8 +++-
 drivers/infiniband/hw/mlx5/mlx5_ib.h          |  4 +-
 .../net/ethernet/mellanox/mlx5/core/fs_core.c | 30 ++++++++++----
 include/linux/mlx5/fs.h                       |  2 +-
 5 files changed, 56 insertions(+), 28 deletions(-)
Signature · 1 line
2.49.0
author
#1 lore raw

PATCH 1/2 net/mlx5: fs, add multiple prios to RDMA TRANSPORT steering domain

From: Patrisious Haddad <[email protected]>

RDMA TRANSPORT domains were initially limited to a single priority. This change allows the domains to have multiple priorities, making it possible to add several rules and control the order in which they're evaluated.

Signed-off-by Patrisious Haddad <[email protected]>
Reviewed-by Mark Bloch <[email protected]>
Signed-off-by Leon Romanovsky <[email protected]>
---
 .../net/ethernet/mellanox/mlx5/core/fs_core.c | 30 ++++++++++++++-----
 include/linux/mlx5/fs.h                       |  2 +-
 2 files changed, 23 insertions(+), 9 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/fs_core.c +22 -8
index 23a7e8e7adfa..63b2aa44084b 100644
@@ -3242,34 +3242,48 @@ static int
 init_rdma_transport_rx_root_ns_one(struct mlx5_flow_steering *steering,
 				   int vport_idx)
 {
+	struct mlx5_flow_root_namespace *root_ns;
 	struct fs_prio *prio;
+	int i;
 
 	steering->rdma_transport_rx_root_ns[vport_idx] =
 		create_root_ns(steering, FS_FT_RDMA_TRANSPORT_RX);
 	if (!steering->rdma_transport_rx_root_ns[vport_idx])
 		return -ENOMEM;
 
-	/* create 1 prio*/
-	prio = fs_create_prio(&steering->rdma_transport_rx_root_ns[vport_idx]->ns,
-			      MLX5_RDMA_TRANSPORT_BYPASS_PRIO, 1);
-	return PTR_ERR_OR_ZERO(prio);
+	root_ns = steering->rdma_transport_rx_root_ns[vport_idx];
+
+	for (i = 0; i < MLX5_RDMA_TRANSPORT_BYPASS_PRIO; i++) {
+		prio = fs_create_prio(&root_ns->ns, i, 1);
+		if (IS_ERR(prio))
+			return PTR_ERR(prio);
+	}
+	set_prio_attrs(root_ns);
+	return 0;
 }
 
 static int
 init_rdma_transport_tx_root_ns_one(struct mlx5_flow_steering *steering,
 				   int vport_idx)
 {
+	struct mlx5_flow_root_namespace *root_ns;
 	struct fs_prio *prio;
+	int i;
 
 	steering->rdma_transport_tx_root_ns[vport_idx] =
 		create_root_ns(steering, FS_FT_RDMA_TRANSPORT_TX);
 	if (!steering->rdma_transport_tx_root_ns[vport_idx])
 		return -ENOMEM;
 
-	/* create 1 prio*/
-	prio = fs_create_prio(&steering->rdma_transport_tx_root_ns[vport_idx]->ns,
-			      MLX5_RDMA_TRANSPORT_BYPASS_PRIO, 1);
-	return PTR_ERR_OR_ZERO(prio);
+	root_ns = steering->rdma_transport_tx_root_ns[vport_idx];
+
+	for (i = 0; i < MLX5_RDMA_TRANSPORT_BYPASS_PRIO; i++) {
+		prio = fs_create_prio(&root_ns->ns, i, 1);
+		if (IS_ERR(prio))
+			return PTR_ERR(prio);
+	}
+	set_prio_attrs(root_ns);
+	return 0;
 }
 
 static int init_rdma_transport_rx_root_ns(struct mlx5_flow_steering *steering)
include/linux/mlx5/fs.h +1 -1
index 939e58c2f386..86055d55836d 100644
@@ -40,7 +40,7 @@
 
 #define MLX5_SET_CFG(p, f, v) MLX5_SET(create_flow_group_in, p, f, v)
 
-#define MLX5_RDMA_TRANSPORT_BYPASS_PRIO 0
+#define MLX5_RDMA_TRANSPORT_BYPASS_PRIO 16
 #define MLX5_FS_MAX_POOL_SIZE BIT(30)
 
 enum mlx5_flow_destination_type {
Signature · 1 line
2.49.0
maintainer
#2 lore raw

Re: [PATCH mlx5-next 1/2] net/mlx5: fs, add multiple prios to RDMA TRANSPORT steering domain

drivers/net/ethernet/mellanox/mlx5/core/fs_core.c static int · line 3261

Quoted 6 lines · On Tue, 17 Jun 2025 11:19:15 +0300 Leon Romanovsky wrote:
drivers/net/ethernet/mellanox/mlx5/core/fs_core.c +6 -0
+	for (i = 0; i < MLX5_RDMA_TRANSPORT_BYPASS_PRIO; i++) {
+		prio = fs_create_prio(&root_ns->ns, i, 1);
+		if (IS_ERR(prio))
+			return PTR_ERR(prio);
+	}
+	set_prio_attrs(root_ns);

Looking at the PR now -- y'all sure this doesn't need any extra cleanup if creation of non-first prio fails?

author
#3 lore raw

Re: [PATCH mlx5-next 1/2] net/mlx5: fs, add multiple prios to RDMA TRANSPORT steering domain

Quoted 10 lines · Jakub Kicinski

drivers/net/ethernet/mellanox/mlx5/core/fs_core.c static int · line 3261

Quoted 6 lines · On Tue, 17 Jun 2025 11:19:15 +0300 Leon Romanovsky wrote:
drivers/net/ethernet/mellanox/mlx5/core/fs_core.c +6 -0
+	for (i = 0; i < MLX5_RDMA_TRANSPORT_BYPASS_PRIO; i++) {
+		prio = fs_create_prio(&root_ns->ns, i, 1);
+		if (IS_ERR(prio))
+			return PTR_ERR(prio);
+	}
+	set_prio_attrs(root_ns);

Looking at the PR now -- y'all sure this doesn't need any extra cleanup if creation of non-first prio fails?

Yes, it was missed.

Thanks

author
#4 lore raw

PATCH 2/2 RDMA/mlx5: Add multiple priorities support to RDMA TRANSPORT userspace tables

In reply to Leon Romanovsky

From: Patrisious Haddad <[email protected]>

Support the creation of RDMA TRANSPORT tables over multiple priorities via matcher creation.

Signed-off-by Patrisious Haddad <[email protected]>
Reviewed-by Mark Bloch <[email protected]>
Signed-off-by Leon Romanovsky <[email protected]>
---
 drivers/infiniband/hw/mlx5/fs.c      | 40 +++++++++++++++++-----------
 drivers/infiniband/hw/mlx5/fs.h      |  8 ++++--
 drivers/infiniband/hw/mlx5/mlx5_ib.h |  4 +--
 3 files changed, 33 insertions(+), 19 deletions(-)
drivers/infiniband/hw/mlx5/fs.c +25 -15
index 680627f1de33..ebcc05f766e1 100644
@@ -1966,7 +1966,8 @@ _get_flow_table(struct mlx5_ib_dev *dev, u16 user_priority,
 		break;
 	case MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX:
 	case MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX:
-		if (ib_port == 0 || user_priority > MLX5_RDMA_TRANSPORT_BYPASS_PRIO)
+		if (ib_port == 0 ||
+		    user_priority >= MLX5_RDMA_TRANSPORT_BYPASS_PRIO)
 			return ERR_PTR(-EINVAL);
 		ret = mlx5_ib_fill_transport_ns_info(dev, ns_type, &flags,
 						     &vport_idx, &vport,

@@ -2016,10 +2017,10 @@ _get_flow_table(struct mlx5_ib_dev *dev, u16 user_priority,
 		prio = &dev->flow_db->rdma_tx[priority];
 		break;
 	case MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX:
-		prio = &dev->flow_db->rdma_transport_rx[ib_port - 1];
+		prio = &dev->flow_db->rdma_transport_rx[priority][ib_port - 1];
 		break;
 	case MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX:
-		prio = &dev->flow_db->rdma_transport_tx[ib_port - 1];
+		prio = &dev->flow_db->rdma_transport_tx[priority][ib_port - 1];
 		break;
 	default: return ERR_PTR(-EINVAL);
 	}

@@ -3466,31 +3467,40 @@ static const struct ib_device_ops flow_ops = {
 
 int mlx5_ib_fs_init(struct mlx5_ib_dev *dev)
 {
+	int i, j;
+
 	dev->flow_db = kzalloc(sizeof(*dev->flow_db), GFP_KERNEL);
 
 	if (!dev->flow_db)
 		return -ENOMEM;
 
-	dev->flow_db->rdma_transport_rx = kcalloc(dev->num_ports,
-					sizeof(struct mlx5_ib_flow_prio),
-					GFP_KERNEL);
-	if (!dev->flow_db->rdma_transport_rx)
-		goto free_flow_db;
+	for (i = 0; i < MLX5_RDMA_TRANSPORT_BYPASS_PRIO; i++) {
+		dev->flow_db->rdma_transport_rx[i] =
+			kcalloc(dev->num_ports,
+				sizeof(struct mlx5_ib_flow_prio), GFP_KERNEL);
+		if (!dev->flow_db->rdma_transport_rx[i])
+			goto free_rdma_transport_rx;
+	}
 
-	dev->flow_db->rdma_transport_tx = kcalloc(dev->num_ports,
-					sizeof(struct mlx5_ib_flow_prio),
-					GFP_KERNEL);
-	if (!dev->flow_db->rdma_transport_tx)
-		goto free_rdma_transport_rx;
+	for (j = 0; j < MLX5_RDMA_TRANSPORT_BYPASS_PRIO; j++) {
+		dev->flow_db->rdma_transport_tx[j] =
+			kcalloc(dev->num_ports,
+				sizeof(struct mlx5_ib_flow_prio), GFP_KERNEL);
+		if (!dev->flow_db->rdma_transport_tx[j])
+			goto free_rdma_transport_tx;
+	}
 
 	mutex_init(&dev->flow_db->lock);
 
 	ib_set_device_ops(&dev->ib_dev, &flow_ops);
 	return 0;
 
+free_rdma_transport_tx:
+	while (j--)
+		kfree(dev->flow_db->rdma_transport_tx[j]);
 free_rdma_transport_rx:
-	kfree(dev->flow_db->rdma_transport_rx);
-free_flow_db:
+	while (i--)
+		kfree(dev->flow_db->rdma_transport_rx[i]);
 	kfree(dev->flow_db);
 	return -ENOMEM;
 }
drivers/infiniband/hw/mlx5/fs.h +6 -2
index 2ebe86e5be10..7abba0e2837c 100644
@@ -13,6 +13,8 @@ void mlx5_ib_fs_cleanup_anchor(struct mlx5_ib_dev *dev);
 
 static inline void mlx5_ib_fs_cleanup(struct mlx5_ib_dev *dev)
 {
+	int i;
+
 	/* When a steering anchor is created, a special flow table is also
 	 * created for the user to reference. Since the user can reference it,
 	 * the kernel cannot trust that when the user destroys the steering

@@ -25,8 +27,10 @@ static inline void mlx5_ib_fs_cleanup(struct mlx5_ib_dev *dev)
 	 * is a safe assumption that all references are gone.
 	 */
 	mlx5_ib_fs_cleanup_anchor(dev);
-	kfree(dev->flow_db->rdma_transport_tx);
-	kfree(dev->flow_db->rdma_transport_rx);
+	for (i = 0; i < MLX5_RDMA_TRANSPORT_BYPASS_PRIO; i++)
+		kfree(dev->flow_db->rdma_transport_tx[i]);
+	for (i = 0; i < MLX5_RDMA_TRANSPORT_BYPASS_PRIO; i++)
+		kfree(dev->flow_db->rdma_transport_rx[i]);
 	kfree(dev->flow_db);
 }
 #endif /* _MLX5_IB_FS_H */
drivers/infiniband/hw/mlx5/mlx5_ib.h +2 -2
index 9c6ec5a968b1..ef627e2f8e6d 100644
@@ -320,8 +320,8 @@ struct mlx5_ib_flow_db {
 	struct mlx5_ib_flow_prio	rdma_tx[MLX5_IB_NUM_FLOW_FT];
 	struct mlx5_ib_flow_prio	opfcs[MLX5_IB_OPCOUNTER_MAX];
 	struct mlx5_flow_table		*lag_demux_ft;
-	struct mlx5_ib_flow_prio        *rdma_transport_rx;
-	struct mlx5_ib_flow_prio        *rdma_transport_tx;
+	struct mlx5_ib_flow_prio        *rdma_transport_rx[MLX5_RDMA_TRANSPORT_BYPASS_PRIO];
+	struct mlx5_ib_flow_prio        *rdma_transport_tx[MLX5_RDMA_TRANSPORT_BYPASS_PRIO];
 	/* Protect flow steering bypass flow tables
 	 * when add/del flow rules.
 	 * only single add/removal of flow steering rule could be done
Signature · 1 line
2.49.0
#5 lore raw
In reply to Leon Romanovsky
Quoted 4 lines · Leon Romanovsky

Hi,

This short series from Patrisious extends mlx5 flow steering logic to allow creation rule creation with priorities in RDMA TRANSPORT tables.

        ^^^^^^^^

This "creation" should be removed?

Yanjun.Zhu

Quoted 15 lines

Thanks

Patrisious Haddad (2):
net/mlx5: fs, add multiple prios to RDMA TRANSPORT steering domain RDMA/mlx5: Add multiple priorities support to RDMA TRANSPORT userspace
tables

  drivers/infiniband/hw/mlx5/fs.c               | 40 ++++++++++++-------
  drivers/infiniband/hw/mlx5/fs.h               |  8 +++-
  drivers/infiniband/hw/mlx5/mlx5_ib.h          |  4 +-
  .../net/ethernet/mellanox/mlx5/core/fs_core.c | 30 ++++++++++----
  include/linux/mlx5/fs.h                       |  2 +-
  5 files changed, 56 insertions(+), 28 deletions(-)
author
#6 lore raw
Quoted 7 lines · Zhu Yanjun
Quoted 4 lines · Leon Romanovsky

Hi,

This short series from Patrisious extends mlx5 flow steering logic to allow creation rule creation with priorities in RDMA TRANSPORT tables.

        ^^^^^^^^

This "creation" should be removed?

This is typo on cover letter which is going to be disposed.

Thanks

Quoted 20 lines

Yanjun.Zhu

Quoted 15 lines

Thanks

Patrisious Haddad (2):
net/mlx5: fs, add multiple prios to RDMA TRANSPORT steering domain RDMA/mlx5: Add multiple priorities support to RDMA TRANSPORT userspace
tables

  drivers/infiniband/hw/mlx5/fs.c               | 40 ++++++++++++-------
  drivers/infiniband/hw/mlx5/fs.h               |  8 +++-
  drivers/infiniband/hw/mlx5/mlx5_ib.h          |  4 +-
  .../net/ethernet/mellanox/mlx5/core/fs_core.c | 30 ++++++++++----
  include/linux/mlx5/fs.h                       |  2 +-
  5 files changed, 56 insertions(+), 28 deletions(-)