Extend UAR to have DevX UID

PATCH In review Leon Romanovsky 2 participants 5 messages Sep 14, 2021 – Sep 20, 2021

Subsystems (5)

Networking Network drivers InfiniBand Mellanox mlx5 core +1 more subsystems

lore

Share & tools

lore

Leon Romanovsky

author

2021-09-14 23:11 UTC #0 lore raw

cover Extend UAR to have DevX UID

From: Leon Romanovsky <[email protected]>

Hi,

This is short series from Meir that adds DevX UID to the UAR.

Thanks

Meir Lichtinger (2):
net/mlx5: Add uid field to UAR allocation structures IB/mlx5: Enable UAR to have DevX UID

 drivers/infiniband/hw/mlx5/cmd.c  | 24 ++++++++++++++
 drivers/infiniband/hw/mlx5/cmd.h  |  2 ++
 drivers/infiniband/hw/mlx5/main.c | 55 +++++++++++++++++--------------
 include/linux/mlx5/mlx5_ifc.h     |  4 +--
 4 files changed, 59 insertions(+), 26 deletions(-)

Signature · 1 line

2.31.1

Leon Romanovsky

author

2021-09-14 23:11 UTC #1 lore raw

PATCH 1/2 net/mlx5: Add uid field to UAR allocation structures

From: Meir Lichtinger <[email protected]>

Add uid field to mlx5_ifc_alloc_uar_in_bits and
mlx5_ifc_dealloc_uar_out_bits structs.

This field will be used by FW to manage UAR according to uid.

Signed-off-by Meir Lichtinger <[email protected]>

Reviewed-by Yishai Hadas <[email protected]>

Signed-off-by Leon Romanovsky <[email protected]>

---
 include/linux/mlx5/mlx5_ifc.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

include/linux/mlx5/mlx5_ifc.h +2 -2

index 52e5baad3b93..4a7e6914ed9b 100644
@@ -7573,7 +7573,7 @@ struct mlx5_ifc_dealloc_uar_out_bits {
 
 struct mlx5_ifc_dealloc_uar_in_bits {
 	u8         opcode[0x10];
-	u8         reserved_at_10[0x10];
+	u8         uid[0x10];
 
 	u8         reserved_at_20[0x10];
 	u8         op_mod[0x10];

@@ -8420,7 +8420,7 @@ struct mlx5_ifc_alloc_uar_out_bits {
 
 struct mlx5_ifc_alloc_uar_in_bits {
 	u8         opcode[0x10];
-	u8         reserved_at_10[0x10];
+	u8         uid[0x10];
 
 	u8         reserved_at_20[0x10];
 	u8         op_mod[0x10];

Signature · 1 line

2.31.1

Leon Romanovsky

author

2021-09-14 23:11 UTC #2 lore raw

PATCH 2/2 IB/mlx5: Enable UAR to have DevX UID

In reply to Leon Romanovsky

From: Meir Lichtinger <[email protected]>

UID field was added to alloc_uar and dealloc_uar PRM command, to specify DevX UID for UAR. This change enables firmware validating user access to its own UAR resources.

For the kernel allocated UARs the UID will stay 0 as of today.

Signed-off-by Meir Lichtinger <[email protected]>

Reviewed-by Yishai Hadas <[email protected]>

Signed-off-by Leon Romanovsky <[email protected]>

---
 drivers/infiniband/hw/mlx5/cmd.c  | 24 ++++++++++++++
 drivers/infiniband/hw/mlx5/cmd.h  |  2 ++
 drivers/infiniband/hw/mlx5/main.c | 55 +++++++++++++++++--------------
 3 files changed, 57 insertions(+), 24 deletions(-)

drivers/infiniband/hw/mlx5/cmd.c +24 -0

index a8db8a051170..0fe3c4ceec43 100644
@@ -206,3 +206,27 @@ int mlx5_cmd_mad_ifc(struct mlx5_core_dev *dev, const void *inb, void *outb,
 	kfree(in);
 	return err;
 }
+
+int mlx5_ib_cmd_uar_alloc(struct mlx5_core_dev *dev, u32 *uarn, u16 uid)
+{
+	u32 out[MLX5_ST_SZ_DW(alloc_uar_out)] = {};
+	u32 in[MLX5_ST_SZ_DW(alloc_uar_in)] = {};
+	int err;
+
+	MLX5_SET(alloc_uar_in, in, opcode, MLX5_CMD_OP_ALLOC_UAR);
+	MLX5_SET(alloc_uar_in, in, uid, uid);
+	err = mlx5_cmd_exec_inout(dev, alloc_uar, in, out);
+	if (!err)
+		*uarn = MLX5_GET(alloc_uar_out, out, uar);
+	return err;
+}
+
+int mlx5_ib_cmd_uar_dealloc(struct mlx5_core_dev *dev, u32 uarn, u16 uid)
+{
+	u32 in[MLX5_ST_SZ_DW(dealloc_uar_in)] = {};
+
+	MLX5_SET(dealloc_uar_in, in, opcode, MLX5_CMD_OP_DEALLOC_UAR);
+	MLX5_SET(dealloc_uar_in, in, uar, uarn);
+	MLX5_SET(dealloc_uar_in, in, uid, uid);
+	return mlx5_cmd_exec_in(dev, dealloc_uar, in);
+}

drivers/infiniband/hw/mlx5/cmd.h +2 -0

index 66c96292ed43..a008938e52f4 100644
@@ -57,4 +57,6 @@ int mlx5_cmd_xrcd_alloc(struct mlx5_core_dev *dev, u32 *xrcdn, u16 uid);
 int mlx5_cmd_xrcd_dealloc(struct mlx5_core_dev *dev, u32 xrcdn, u16 uid);
 int mlx5_cmd_mad_ifc(struct mlx5_core_dev *dev, const void *inb, void *outb,
 		     u16 opmod, u8 port);
+int mlx5_ib_cmd_uar_alloc(struct mlx5_core_dev *dev, u32 *uarn, u16 uid);
+int mlx5_ib_cmd_uar_dealloc(struct mlx5_core_dev *dev, u32 uarn, u16 uid);
 #endif /* MLX5_IB_CMD_H */

drivers/infiniband/hw/mlx5/main.c +31 ~~-24~~

157 lines not on this page · raw message

Signature · 1 line

2.31.1

Jason Gunthorpe

maintainer

2021-09-15 13:47 UTC #3 lore raw

Re: [PATCH rdma-next 2/2] IB/mlx5: Enable UAR to have DevX UID

Quoted 25 lines · Leon Romanovsky

From: Meir Lichtinger <[email protected]>
UID field was added to alloc_uar and dealloc_uar PRM command, to specify DevX UID for UAR. This change enables firmware validating user access to its own UAR resources.
For the kernel allocated UARs the UID will stay 0 as of today.
Signed-off-by Meir Lichtinger <[email protected]>
Reviewed-by Yishai Hadas <[email protected]>
Signed-off-by Leon Romanovsky <[email protected]>
 drivers/infiniband/hw/mlx5/cmd.c  | 24 ++++++++++++++
 drivers/infiniband/hw/mlx5/cmd.h  |  2 ++
 drivers/infiniband/hw/mlx5/main.c | 55 +++++++++++++++++--------------
 3 files changed, 57 insertions(+), 24 deletions(-)
drivers/infiniband/hw/mlx5/cmd.c +3 -0
index a8db8a051170..0fe3c4ceec43 100644
+++ b/drivers/infiniband/hw/mlx5/cmd.c
@@ -206,3 +206,27 @@ int mlx5_cmd_mad_ifc(struct mlx5_core_dev *dev, const void *inb, void *outb,
 	kfree(in);
 	return err;
 }
+
+int mlx5_ib_cmd_uar_alloc(struct mlx5_core_dev *dev, u32 *uarn, u16 uid)

drivers/infiniband/hw/mlx5/cmd.c int mlx5_cmd_mad_ifc(struct mlx5_core_dev *dev, const void *inb, void *outb, · line 220

Quoted 10 lines

drivers/infiniband/hw/mlx5/cmd.c +10 -0

+{
+	u32 out[MLX5_ST_SZ_DW(alloc_uar_out)] = {};
+	u32 in[MLX5_ST_SZ_DW(alloc_uar_in)] = {};
+	int err;
+
+	MLX5_SET(alloc_uar_in, in, opcode, MLX5_CMD_OP_ALLOC_UAR);
+	MLX5_SET(alloc_uar_in, in, uid, uid);
+	err = mlx5_cmd_exec_inout(dev, alloc_uar, in, out);
+	if (!err)
+		*uarn = MLX5_GET(alloc_uar_out, out, uar);

Success oriented flow:

if (err)

     return err;

*uarn = MLX5_GET(alloc_uar_out, out, uar);
return 0;

And why did we add entirely new functions instead of just adding a uid argument to the core ones? Or, why doesn't this delete the old core functions that look unused outside of IB anyhow?

Jason

Leon Romanovsky

author

2021-09-20 13:21 UTC #4 lore raw

Re: [PATCH rdma-next 2/2] IB/mlx5: Enable UAR to have DevX UID

Quoted 39 lines · Jason Gunthorpe

Quoted 25 lines · Leon Romanovsky
From: Meir Lichtinger <[email protected]>
UID field was added to alloc_uar and dealloc_uar PRM command, to specify DevX UID for UAR. This change enables firmware validating user access to its own UAR resources.
For the kernel allocated UARs the UID will stay 0 as of today.
Signed-off-by Meir Lichtinger <[email protected]>
Reviewed-by Yishai Hadas <[email protected]>
Signed-off-by Leon Romanovsky <[email protected]>
 drivers/infiniband/hw/mlx5/cmd.c  | 24 ++++++++++++++
 drivers/infiniband/hw/mlx5/cmd.h  |  2 ++
 drivers/infiniband/hw/mlx5/main.c | 55 +++++++++++++++++--------------
 3 files changed, 57 insertions(+), 24 deletions(-)
drivers/infiniband/hw/mlx5/cmd.c +3 -0
index a8db8a051170..0fe3c4ceec43 100644
+++ b/drivers/infiniband/hw/mlx5/cmd.c
@@ -206,3 +206,27 @@ int mlx5_cmd_mad_ifc(struct mlx5_core_dev *dev, const void *inb, void *outb,
 	kfree(in);
 	return err;
 }
+
+int mlx5_ib_cmd_uar_alloc(struct mlx5_core_dev *dev, u32 *uarn, u16 uid)
drivers/infiniband/hw/mlx5/cmd.c int mlx5_cmd_mad_ifc(struct mlx5_core_dev *dev, const void *inb, void *outb, · line 220
Quoted 10 lines
drivers/infiniband/hw/mlx5/cmd.c +10 -0
+{
+	u32 out[MLX5_ST_SZ_DW(alloc_uar_out)] = {};
+	u32 in[MLX5_ST_SZ_DW(alloc_uar_in)] = {};
+	int err;
+
+	MLX5_SET(alloc_uar_in, in, opcode, MLX5_CMD_OP_ALLOC_UAR);
+	MLX5_SET(alloc_uar_in, in, uid, uid);
+	err = mlx5_cmd_exec_inout(dev, alloc_uar, in, out);
+	if (!err)
+		*uarn = MLX5_GET(alloc_uar_out, out, uar);
Success oriented flow:

Quoted 8 lines

if (err)
     return err;
*uarn = MLX5_GET(alloc_uar_out, out, uar);
return 0;
And why did we add entirely new functions instead of just adding a uid argument to the core ones? Or, why doesn't this delete the old core functions that look unused outside of IB anyhow?

We didn't want to add not-needed for mlx5_core uid field, the rest comments are valid and I'm sorry that I missed them.

Thanks

Quoted 2 lines

Jason