Skip to contentSkip to conversation

Extend mlx5_ib software steering interface

PATCH Applied Leon Romanovsky 2 participants 9 messages Sep 3, 2020 – Sep 18, 2020

Subsystems (5)
lore
Share & tools
lore
#0 lore raw

cover Extend mlx5_ib software steering interface

From: Leon Romanovsky <[email protected]>

This series from Alex extends software steering interface to support devices with extra capability "sw_owner_2" which will replace existing "sw_owner".

Thanks

Alex Vesker (3):
RDMA/mlx5: Add sw_owner_v2 bit capability
RDMA/mlx5: Allow DM allocation for sw_owner_v2 enabled devices RDMA/mlx5: Expose TIR and QP ICM address for sw_owner_v2 devices

 drivers/infiniband/hw/mlx5/main.c | 4 +++-
 drivers/infiniband/hw/mlx5/qp.c   | 6 ++++--
 include/linux/mlx5/mlx5_ifc.h     | 3 ++-
 3 files changed, 9 insertions(+), 4 deletions(-)

--
2.26.2

#1 lore raw

PATCH 1/3 RDMA/mlx5: Add sw_owner_v2 bit capability

From: Alex Vesker <[email protected]>

Added sw_owner_v2 which will be enabled for future devices, replacing sw_owner bit.

---
 include/linux/mlx5/mlx5_ifc.h | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
include/linux/mlx5/mlx5_ifc.h +2 -1
index aee25e4fb2cc..651591a2965d 100644
@@ -420,7 +420,8 @@ struct mlx5_ifc_flow_table_prop_layout_bits {
 	u8         reserved_at_1a[0x2];
 	u8         ipsec_encrypt[0x1];
 	u8         ipsec_decrypt[0x1];
-	u8         reserved_at_1e[0x2];
+	u8         sw_owner_v2[0x1];
+	u8         reserved_at_1f[0x1];
 
 	u8         termination_table_raw_traffic[0x1];
 	u8         reserved_at_21[0x1];

--
2.26.2

#2 lore raw

PATCH 2/3 RDMA/mlx5: Allow DM allocation for sw_owner_v2 enabled devices

In reply to Leon Romanovsky

From: Alex Vesker <[email protected]>

sw_owner_v2 will replace sw_owner for future devices, this means that if sw_owner_v2 is set sw_owner should be ignored and DM allocation is required for sw_owner_v2 devices to function.

---
 drivers/infiniband/hw/mlx5/main.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
drivers/infiniband/hw/mlx5/main.c +3 -1
index 99dbef0bccbc..8963b806ad19 100644
@@ -2343,7 +2343,9 @@ static inline int check_dm_type_support(struct mlx5_ib_dev *dev,
 			return -EPERM;
 
 		if (!(MLX5_CAP_FLOWTABLE_NIC_RX(dev->mdev, sw_owner) ||
-		      MLX5_CAP_FLOWTABLE_NIC_TX(dev->mdev, sw_owner)))
+		      MLX5_CAP_FLOWTABLE_NIC_TX(dev->mdev, sw_owner) ||
+		      MLX5_CAP_FLOWTABLE_NIC_RX(dev->mdev, sw_owner_v2) ||
+		      MLX5_CAP_FLOWTABLE_NIC_TX(dev->mdev, sw_owner_v2)))
 			return -EOPNOTSUPP;
 		break;
 	}
Signature · 1 line
2.26.2
maintainer
#3 lore raw

Re: [PATCH rdma-next 2/3] RDMA/mlx5: Allow DM allocation for sw_owner_v2 enabled devices

Quoted 15 lines · Leon Romanovsky

From: Alex Vesker <[email protected]>

sw_owner_v2 will replace sw_owner for future devices, this means that if sw_owner_v2 is set sw_owner should be ignored and DM allocation is required for sw_owner_v2 devices to function.

 drivers/infiniband/hw/mlx5/main.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
drivers/infiniband/hw/mlx5/main.c +1 -0
index 99dbef0bccbc..8963b806ad19 100644
+++ b/drivers/infiniband/hw/mlx5/main.c

drivers/infiniband/hw/mlx5/main.c static inline int check_dm_type_support(struct mlx5_ib_dev *dev, · line 2349

Quoted 8 lines
drivers/infiniband/hw/mlx5/main.c +3 -1
@@ -2343,7 +2343,9 @@ static inline int check_dm_type_support(struct mlx5_ib_dev *dev,
 			return -EPERM;
 
 		if (!(MLX5_CAP_FLOWTABLE_NIC_RX(dev->mdev, sw_owner) ||
-		      MLX5_CAP_FLOWTABLE_NIC_TX(dev->mdev, sw_owner)))
+		      MLX5_CAP_FLOWTABLE_NIC_TX(dev->mdev, sw_owner) ||
+		      MLX5_CAP_FLOWTABLE_NIC_RX(dev->mdev, sw_owner_v2) ||
+		      MLX5_CAP_FLOWTABLE_NIC_TX(dev->mdev, sw_owner_v2)))
 			return -EOPNOTSUPP;

Shouldn't user space ask for MLX5_IB_UAPI_DM_TYPE_STEERING_SW_V2_ICM types too?

What happens if old user space runs on a new device?

Jason

#4 lore raw

Re: [PATCH rdma-next 2/3] RDMA/mlx5: Allow DM allocation for sw_owner_v2 enabled devices

Quoted 18 lines · Jason Gunthorpe
Quoted 17 lines · Leon Romanovsky

From: Alex Vesker <[email protected]>

sw_owner_v2 will replace sw_owner for future devices, this means that if sw_owner_v2 is set sw_owner should be ignored and DM allocation is required for sw_owner_v2 devices to function.

 drivers/infiniband/hw/mlx5/main.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
drivers/infiniband/hw/mlx5/main.c +1 -0
index 99dbef0bccbc..8963b806ad19 100644
+++ b/drivers/infiniband/hw/mlx5/main.c
@@ -2343,7 +2343,9 @@ static inline int check_dm_type_support(struct mlx5_ib_dev *dev,
 			return -EPERM;
 
Quoted 9 lines

drivers/infiniband/hw/mlx5/main.c static inline int check_dm_type_support(struct mlx5_ib_dev *dev, · line 2349

Quoted 6 lines
drivers/infiniband/hw/mlx5/main.c +3 -1
 		if (!(MLX5_CAP_FLOWTABLE_NIC_RX(dev->mdev, sw_owner) ||
-		      MLX5_CAP_FLOWTABLE_NIC_TX(dev->mdev, sw_owner)))
+		      MLX5_CAP_FLOWTABLE_NIC_TX(dev->mdev, sw_owner) ||
+		      MLX5_CAP_FLOWTABLE_NIC_RX(dev->mdev, sw_owner_v2) ||
+		      MLX5_CAP_FLOWTABLE_NIC_TX(dev->mdev, sw_owner_v2)))
 			return -EOPNOTSUPP;

Shouldn't user space ask for MLX5_IB_UAPI_DM_TYPE_STEERING_SW_V2_ICM types too?

For sw_owner_v2 devices, we will return exactly same type of DM, so there is no need for a new one. The change is to allow creation of such DM in addition to existing sw_owner devices.

Quoted 2 lines

What happens if old user space runs on a new device?

Old userspace will have sw_owner_v2 cap enabled and will get DM as before. The patch diff is hard to read but the end result will be.

	if (!(MLX5_CAP_FLOWTABLE_NIC_RX(dev->mdev, sw_owner) ||
	      MLX5_CAP_FLOWTABLE_NIC_TX(dev->mdev, sw_owner) ||
	      MLX5_CAP_FLOWTABLE_NIC_RX(dev->mdev, sw_owner_v2) ||
	      MLX5_CAP_FLOWTABLE_NIC_TX(dev->mdev, sw_owner_v2)))
		return -EOPNOTSUPP;

DevX users are expected to check capabilities and be ready to work with missing ones.

Thanks

Quoted 2 lines

Jason

#5 lore raw

PATCH 3/3 RDMA/mlx5: Expose TIR and QP ICM address for sw_owner_v2 devices

In reply to Leon Romanovsky

From: Alex Vesker <[email protected]>

Expose the ICM address to access TIR and QP, this will allow sw_owned_v2 devices to steer traffic to TIRs and QPs same as done with sw_owner capability.

---
 drivers/infiniband/hw/mlx5/qp.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
drivers/infiniband/hw/mlx5/qp.c +4 -2
index 22280ea92006..ada852fcc427 100644
@@ -1477,7 +1477,8 @@ static int create_raw_packet_qp(struct mlx5_ib_dev *dev, struct mlx5_ib_qp *qp,
 			resp->comp_mask |= MLX5_IB_CREATE_QP_RESP_MASK_RQN;
 			resp->tirn = rq->tirn;
 			resp->comp_mask |= MLX5_IB_CREATE_QP_RESP_MASK_TIRN;
-			if (MLX5_CAP_FLOWTABLE_NIC_RX(dev->mdev, sw_owner)) {
+			if (MLX5_CAP_FLOWTABLE_NIC_RX(dev->mdev, sw_owner) ||
+			    MLX5_CAP_FLOWTABLE_NIC_RX(dev->mdev, sw_owner_v2)) {
 				resp->tir_icm_addr = MLX5_GET(
 					create_tir_out, out, icm_address_31_0);
 				resp->tir_icm_addr |=
@@ -1739,7 +1740,8 @@ static int create_rss_raw_qp_tir(struct mlx5_ib_dev *dev, struct ib_pd *pd,
 	if (mucontext->devx_uid) {
 		params->resp.comp_mask |= MLX5_IB_CREATE_QP_RESP_MASK_TIRN;
 		params->resp.tirn = qp->rss_qp.tirn;
-		if (MLX5_CAP_FLOWTABLE_NIC_RX(dev->mdev, sw_owner)) {
+		if (MLX5_CAP_FLOWTABLE_NIC_RX(dev->mdev, sw_owner) ||
+		    MLX5_CAP_FLOWTABLE_NIC_RX(dev->mdev, sw_owner_v2)) {
 			params->resp.tir_icm_addr =
 				MLX5_GET(create_tir_out, out, icm_address_31_0);
 			params->resp.tir_icm_addr |=
Signature · 1 line
2.26.2
maintainer
#6 lore raw
In reply to Leon Romanovsky
Quoted 6 lines · Leon Romanovsky

From: Leon Romanovsky <[email protected]>

This series from Alex extends software steering interface to support devices with extra capability "sw_owner_2" which will replace existing "sw_owner".

Quoted 6 lines

Thanks

Alex Vesker (3):
RDMA/mlx5: Add sw_owner_v2 bit capability
RDMA/mlx5: Allow DM allocation for sw_owner_v2 enabled devices RDMA/mlx5: Expose TIR and QP ICM address for sw_owner_v2 devices

Ok, can you update the shared branch with the first patch? Thanks

Jason

#7 lore raw
Quoted 7 lines · Jason Gunthorpe
Quoted 6 lines · Leon Romanovsky

From: Leon Romanovsky <[email protected]>

This series from Alex extends software steering interface to support devices with extra capability "sw_owner_2" which will replace existing "sw_owner".

Quoted 8 lines
Quoted 6 lines

Thanks

Alex Vesker (3):
RDMA/mlx5: Add sw_owner_v2 bit capability
RDMA/mlx5: Allow DM allocation for sw_owner_v2 enabled devices RDMA/mlx5: Expose TIR and QP ICM address for sw_owner_v2 devices

Ok, can you update the shared branch with the first patch? Thanks

Done, thanks
9d8feb460adb RDMA/mlx5: Add sw_owner_v2 bit capability

Quoted 2 lines

Jason

maintainer Applied
#8 lore raw
Quoted 15 lines · Leon Romanovsky
Quoted 14 lines · Jason Gunthorpe
Quoted 12 lines · Leon Romanovsky

From: Leon Romanovsky <[email protected]>

This series from Alex extends software steering interface to support devices with extra capability "sw_owner_2" which will replace existing "sw_owner".

Thanks

Alex Vesker (3):
RDMA/mlx5: Add sw_owner_v2 bit capability
RDMA/mlx5: Allow DM allocation for sw_owner_v2 enabled devices RDMA/mlx5: Expose TIR and QP ICM address for sw_owner_v2 devices

Quoted 4 lines
Quoted 1 line

Ok, can you update the shared branch with the first patch? Thanks

Done, thanks
9d8feb460adb RDMA/mlx5: Add sw_owner_v2 bit capability

Applied to for-next, thanks

Jason