Skip to contentSkip to conversation

Introduce mlx5 user space driver over VFIO

PATCH In review Yishai Hadas 3 participants 36 messages Jul 20, 2021 – Aug 1, 2021

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover Introduce mlx5 user space driver over VFIO

This series introduces mlx5 user space driver over VFIO.

This enables an application to take full ownership on the opened device and run any firmware command (e.g. port up/down) without any concern to hurt someone else.

The application look and feel is like regular RDMA application over DEVX, it uses verbs API to open/close a device and then mostly uses DEVX APIs to interact with the device.

To achieve it, few mlx5 DV APIs were introduced, it includes:
- An API to get ibv_device for a given mlx5 PCI name.
- APIs to manage device specific events.

Detailed man pages were added to describe the expected usage of those APIs.

The mlx5 VFIO driver implemented the basic verbs APIs as of managing MR/PD and the DEVX APIs which are required to write an RDMA application.

The VFIO uAPIs were used to setup mlx5 vfio context by reading the device initialization segment, setup DMA and enables the device command interface.

In addition, the series includes pyverbs stuff which runs DEVX like testing over RDMA and VFIO mlx5 devices.

Some extra documentation of the benefits and the motivation to use VFIO can be found here [1].

PR was sent [2].

[1] https://www.kernel.org/doc/html/latest/driver-api/vfio.html
[2] https://github.com/linux-rdma/rdma-core/pull/1034

Yishai

Edward Srouji (10):
pyverbs: Support DevX UMEM registration pyverbs/mlx5: Support EQN querying pyverbs/mlx5: Support more DevX objects
pyverbs: Add auxiliary memory functions pyverbs/mlx5: Add support to extract mlx5dv objects pyverbs/mlx5: Wrap mlx5_cqe64 struct and add enums
tests: Add MAC address to the tests' args
tests: Add mlx5 DevX data path test pyverbs/mlx5: Support mlx5 devices over VFIO
tests: Add a test for mlx5 over VFIO

Maor Gottlieb (1):
mlx5: Enable debug functionality for vfio

Mark Zhang (5):
util: Add interval_set support
mlx5: Support fast teardown over vfio
mlx5: VFIO poll_health support
mlx5: Set DV context ops
mlx5: Implement mlx5dv devx_obj APIs over vfio

Yishai Hadas (11):
Update kernel headers
mlx5: Introduce mlx5dv_get_vfio_device_list()
verbs: Enable verbs_open_device() to work over non sysfs devices
mlx5: Setup mlx5 vfio context
mlx5: Add mlx5_vfio_cmd_exec() support
mlx5: vfio setup function support
mlx5: vfio setup basic caps
mlx5: Enable interrupt command mode over vfio
mlx5: Introduce vfio APIs to process events
mlx5: Implement basic verbs operation for PD and MR over vfio
mlx5: Support initial DEVX/DV APIs over vfio

 Documentation/pyverbs.md                           |   34 +
 debian/ibverbs-providers.symbols                   |    4 +
 kernel-headers/CMakeLists.txt                      |    4 +
 kernel-headers/linux/vfio.h                        | 1374 ++++++++
 libibverbs/device.c                                |   39 +-
 libibverbs/sysfs.c                                 |    5 +
 providers/mlx5/CMakeLists.txt                      |    3 +-
 providers/mlx5/dr_rule.c                           |   10 +-
 providers/mlx5/libmlx5.map                         |    7 +
 providers/mlx5/man/CMakeLists.txt                  |    3 +
 .../mlx5/man/mlx5dv_get_vfio_device_list.3.md      |   64 +
 providers/mlx5/man/mlx5dv_vfio_get_events_fd.3.md  |   41 +
 providers/mlx5/man/mlx5dv_vfio_process_events.3.md |   43 +
 providers/mlx5/mlx5.c                              |  376 ++-
 providers/mlx5/mlx5.h                              |  187 +-
 providers/mlx5/mlx5_ifc.h                          | 1206 ++++++-
 providers/mlx5/mlx5_vfio.c                         | 3379 ++++++++++++++++++++
 providers/mlx5/mlx5_vfio.h                         |  329 ++
 providers/mlx5/mlx5dv.h                            |   25 +
 providers/mlx5/verbs.c                             |  966 +++++-
 pyverbs/CMakeLists.txt                             |    7 +
 pyverbs/dma_util.pyx                               |   25 +
 pyverbs/mem_alloc.pyx                              |   46 +-
 pyverbs/pd.pyx                                     |    4 +
 pyverbs/providers/mlx5/CMakeLists.txt              |    4 +-
 pyverbs/providers/mlx5/libmlx5.pxd                 |  103 +-
 pyverbs/providers/mlx5/mlx5_vfio.pxd               |   15 +
 pyverbs/providers/mlx5/mlx5_vfio.pyx               |  116 +
 pyverbs/providers/mlx5/mlx5dv.pxd                  |   20 +
 pyverbs/providers/mlx5/mlx5dv.pyx                  |  277 +-
 pyverbs/providers/mlx5/mlx5dv_enums.pxd            |   42 +
 pyverbs/providers/mlx5/mlx5dv_objects.pxd          |   28 +
 pyverbs/providers/mlx5/mlx5dv_objects.pyx          |  214 ++
 tests/CMakeLists.txt                               |    3 +
 tests/args_parser.py                               |    5 +
 tests/base.py                                      |   14 +-
 tests/mlx5_base.py                                 |  460 ++-
 tests/mlx5_prm_structs.py                          | 1046 ++++++
 tests/test_mlx5_devx.py                            |   23 +
 tests/test_mlx5_vfio.py                            |  104 +
 util/CMakeLists.txt                                |    2 +
 util/interval_set.c                                |  208 ++
 util/interval_set.h                                |   77 +
 util/util.h                                        |    5 +
 44 files changed, 10650 insertions(+), 297 deletions(-)
 create mode 100644 kernel-headers/linux/vfio.h
 create mode 100644 providers/mlx5/man/mlx5dv_get_vfio_device_list.3.md
 create mode 100644 providers/mlx5/man/mlx5dv_vfio_get_events_fd.3.md
 create mode 100644 providers/mlx5/man/mlx5dv_vfio_process_events.3.md
 create mode 100644 providers/mlx5/mlx5_vfio.c
 create mode 100644 providers/mlx5/mlx5_vfio.h
 create mode 100644 pyverbs/dma_util.pyx
 create mode 100644 pyverbs/providers/mlx5/mlx5_vfio.pxd
 create mode 100644 pyverbs/providers/mlx5/mlx5_vfio.pyx
 create mode 100644 pyverbs/providers/mlx5/mlx5dv_objects.pxd
 create mode 100644 pyverbs/providers/mlx5/mlx5dv_objects.pyx
 create mode 100644 tests/mlx5_prm_structs.py
 create mode 100644 tests/test_mlx5_devx.py
 create mode 100644 tests/test_mlx5_vfio.py
 create mode 100644 util/interval_set.c
 create mode 100644 util/interval_set.h
Signature · 1 line
1.8.3.1
#1 lore raw

PATCH 1/27 Update kernel headers

To commit e73f0f0ee754 ("Linux 5.14-rc1");

---
 kernel-headers/CMakeLists.txt |    4 +
 kernel-headers/linux/vfio.h   | 1374 +++++++++++++++++++++++++++++++++++++++++
 2 files changed, 1378 insertions(+)
 create mode 100644 kernel-headers/linux/vfio.h
kernel-headers/CMakeLists.txt +4 -0
index b961892..d9621ee 100644
@@ -26,6 +26,10 @@ publish_internal_headers(rdma
   rdma/vmw_pvrdma-abi.h
   )
 
+publish_internal_headers(linux
+  linux/vfio.h
+  )
+
 publish_internal_headers(rdma/hfi
   rdma/hfi/hfi1_ioctl.h
   rdma/hfi/hfi1_user.h
kernel-headers/linux/vfio.h new file +1374 -0

1,374 lines not on this page · raw message

Signature · 1 line
1.8.3.1
#2 lore raw

PATCH 2/27 mlx5: Introduce mlx5dv_get_vfio_device_list()

In reply to Yishai Hadas

Introduce mlx5dv_get_vfio_device_list() API for getting list of mlx5 devices which can be used over VFIO.

A man page with the expected usage was added.

---
 debian/ibverbs-providers.symbols                   |   2 +
 providers/mlx5/CMakeLists.txt                      |   3 +-
 providers/mlx5/libmlx5.map                         |   5 +
 providers/mlx5/man/CMakeLists.txt                  |   1 +
 .../mlx5/man/mlx5dv_get_vfio_device_list.3.md      |  64 +++++++
 providers/mlx5/mlx5.c                              |   4 +-
 providers/mlx5/mlx5.h                              |   1 +
 providers/mlx5/mlx5_vfio.c                         | 190 +++++++++++++++++++++
 providers/mlx5/mlx5_vfio.h                         |  27 +++
 providers/mlx5/mlx5dv.h                            |  13 ++
 10 files changed, 307 insertions(+), 3 deletions(-)
 create mode 100644 providers/mlx5/man/mlx5dv_get_vfio_device_list.3.md
 create mode 100644 providers/mlx5/mlx5_vfio.c
 create mode 100644 providers/mlx5/mlx5_vfio.h
debian/ibverbs-providers.symbols +2 -0
index 294832b..64e29b1 100644
@@ -28,6 +28,7 @@ libmlx5.so.1 ibverbs-providers #MINVER#
  MLX5_1.18@MLX5_1.18 34
  MLX5_1.19@MLX5_1.19 35
  MLX5_1.20@MLX5_1.20 36
+ MLX5_1.21@MLX5_1.21 37
  mlx5dv_init_obj@MLX5_1.0 13
  mlx5dv_init_obj@MLX5_1.2 15
  mlx5dv_query_device@MLX5_1.0 13
@@ -133,6 +134,7 @@ libmlx5.so.1 ibverbs-providers #MINVER#
  mlx5dv_map_ah_to_qp@MLX5_1.20 36
  mlx5dv_qp_cancel_posted_send_wrs@MLX5_1.20 36
  _mlx5dv_mkey_check@MLX5_1.20 36
+ mlx5dv_get_vfio_device_list@MLX5_1.21 37
 libefa.so.1 ibverbs-providers #MINVER#
 * Build-Depends-Package: libibverbs-dev
  EFA_1.0@EFA_1.0 24
providers/mlx5/CMakeLists.txt +2 -1
index 69abdd1..45e397e 100644
@@ -11,7 +11,7 @@ if (MLX5_MW_DEBUG)
 endif()
 
 rdma_shared_provider(mlx5 libmlx5.map
-  1 1.20.${PACKAGE_VERSION}
+  1 1.21.${PACKAGE_VERSION}
   buf.c
   cq.c
   dbrec.c
@@ -30,6 +30,7 @@ rdma_shared_provider(mlx5 libmlx5.map
   dr_table.c
   dr_send.c
   mlx5.c
+  mlx5_vfio.c
   qp.c
   srq.c
   verbs.c
providers/mlx5/libmlx5.map +5 -0
index af7541d..3e8a4d8 100644
@@ -189,3 +189,8 @@ MLX5_1.20 {
 		mlx5dv_qp_cancel_posted_send_wrs;
 		_mlx5dv_mkey_check;
 } MLX5_1.19;
+
+MLX5_1.21 {
+        global:
+		mlx5dv_get_vfio_device_list;
+} MLX5_1.20;
providers/mlx5/man/CMakeLists.txt +1 -0
index bb6499d..91aebed 100644
@@ -22,6 +22,7 @@ rdma_man_pages(
   mlx5dv_dump.3.md
   mlx5dv_flow_action_esp.3.md
   mlx5dv_get_clock_info.3
+  mlx5dv_get_vfio_device_list.3.md
   mlx5dv_init_obj.3
   mlx5dv_is_supported.3.md
   mlx5dv_map_ah_to_qp.3.md
providers/mlx5/man/mlx5dv_get_vfio_device_list.3.md new file +64 -0

64 lines not on this page · raw message

providers/mlx5/mlx5.c +2 -2
index e172b9d..46d7748 100644
@@ -62,7 +62,7 @@ static void mlx5_free_context(struct ibv_context *ibctx);
 #endif
 
 #define HCA(v, d) VERBS_PCI_MATCH(PCI_VENDOR_ID_##v, d, NULL)
-static const struct verbs_match_ent hca_table[] = {
+const struct verbs_match_ent mlx5_hca_table[] = {
 	VERBS_DRIVER_ID(RDMA_DRIVER_MLX5),
 	HCA(MELLANOX, 0x1011),	/* MT4113 Connect-IB */
 	HCA(MELLANOX, 0x1012),	/* Connect-IB Virtual Function */
@@ -2410,7 +2410,7 @@ static const struct verbs_device_ops mlx5_dev_ops = {
 	.name = "mlx5",
 	.match_min_abi_version = MLX5_UVERBS_MIN_ABI_VERSION,
 	.match_max_abi_version = MLX5_UVERBS_MAX_ABI_VERSION,
-	.match_table = hca_table,
+	.match_table = mlx5_hca_table,
 	.alloc_device = mlx5_device_alloc,
 	.uninit_device = mlx5_uninit_device,
 	.alloc_context = mlx5_alloc_context,
providers/mlx5/mlx5.h +1 -0
index ac2f88c..3862007 100644
@@ -94,6 +94,7 @@ enum {
 
 extern uint32_t mlx5_debug_mask;
 extern int mlx5_freeze_on_error_cqe;
+extern const struct verbs_match_ent mlx5_hca_table[];
 
 #ifdef MLX5_DEBUG
 #define mlx5_dbg(fp, mask, format, arg...)				\
providers/mlx5/mlx5_vfio.c new file +190 -0

190 lines not on this page · raw message

providers/mlx5/mlx5_vfio.h new file +27 -0
new file mode 100644
index 0000000..6ba4254
@@ -0,0 +1,27 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2021, NVIDIA CORPORATION & AFFILIATES. All rights reserved
+ */
+
+#ifndef MLX5_VFIO_H
+#define MLX5_VFIO_H
+
+#include <stddef.h>
+#include <stdio.h>
+
+#include <infiniband/driver.h>
+
+struct mlx5_vfio_device {
+	struct verbs_device vdev;
+	char *pci_name;
+	char vfio_path[IBV_SYSFS_PATH_MAX];
+	int page_size;
+	uint32_t flags;
+};
+
+static inline struct mlx5_vfio_device *to_mvfio_dev(struct ibv_device *ibdev)
+{
+	return container_of(ibdev, struct mlx5_vfio_device, vdev.device);
+}
+
+#endif
providers/mlx5/mlx5dv.h +13 -0
index 2eba232..e657527 100644
@@ -1474,6 +1474,19 @@ struct mlx5dv_context_attr {
 
 bool mlx5dv_is_supported(struct ibv_device *device);
 
+enum mlx5dv_vfio_context_attr_flags {
+	MLX5DV_VFIO_CTX_FLAGS_INIT_LINK_DOWN = 1 << 0,
+};
+
+struct mlx5dv_vfio_context_attr {
+	const char *pci_name;
+	uint32_t flags; /* Use enum mlx5dv_vfio_context_attr_flags */
+	uint64_t comp_mask;
+};
+
+struct ibv_device **
+mlx5dv_get_vfio_device_list(struct mlx5dv_vfio_context_attr *attr);
+
 struct ibv_context *
 mlx5dv_open_device(struct ibv_device *device, struct mlx5dv_context_attr *attr);
 
Signature · 1 line
1.8.3.1
#3 lore raw

PATCH 3/27 mlx5: Enable debug functionality for vfio

In reply to Yishai Hadas

From: Maor Gottlieb <[email protected]>

Usage will be in next patches.

---
 providers/mlx5/mlx5.c | 28 ++++++++++++++--------------
 providers/mlx5/mlx5.h |  4 ++++
 2 files changed, 18 insertions(+), 14 deletions(-)
providers/mlx5/mlx5.c +14 -14
index 46d7748..1abaa8c 100644
@@ -583,7 +583,7 @@ static int get_total_uuars(int page_size)
 	return size;
 }
 
-static void open_debug_file(struct mlx5_context *ctx)
+void mlx5_open_debug_file(FILE **dbg_fp)
 {
 	char *env;
 	FILE *default_dbg_fp = NULL;
@@ -594,25 +594,25 @@ static void open_debug_file(struct mlx5_context *ctx)
 
 	env = getenv("MLX5_DEBUG_FILE");
 	if (!env) {
-		ctx->dbg_fp = default_dbg_fp;
+		*dbg_fp = default_dbg_fp;
 		return;
 	}
 
-	ctx->dbg_fp = fopen(env, "aw+");
-	if (!ctx->dbg_fp) {
-		ctx->dbg_fp = default_dbg_fp;
-		mlx5_err(ctx->dbg_fp, "Failed opening debug file %s\n", env);
+	*dbg_fp = fopen(env, "aw+");
+	if (!*dbg_fp) {
+		*dbg_fp = default_dbg_fp;
+		mlx5_err(*dbg_fp, "Failed opening debug file %s\n", env);
 		return;
 	}
 }
 
-static void close_debug_file(struct mlx5_context *ctx)
+void mlx5_close_debug_file(FILE *dbg_fp)
 {
-	if (ctx->dbg_fp && ctx->dbg_fp != stderr)
-		fclose(ctx->dbg_fp);
+	if (dbg_fp && dbg_fp != stderr)
+		fclose(dbg_fp);
 }
 
-static void set_debug_mask(void)
+void mlx5_set_debug_mask(void)
 {
 	char *env;
 
@@ -2036,7 +2036,7 @@ static int get_uar_info(struct mlx5_device *mdev,
 
 static void mlx5_uninit_context(struct mlx5_context *context)
 {
-	close_debug_file(context);
+	mlx5_close_debug_file(context->dbg_fp);
 
 	verbs_uninit_context(&context->ibv_ctx);
 	free(context);
@@ -2056,8 +2056,8 @@ static struct mlx5_context *mlx5_init_context(struct ibv_device *ibdev,
 	if (!context)
 		return NULL;
 
-	open_debug_file(context);
-	set_debug_mask();
+	mlx5_open_debug_file(&context->dbg_fp);
+	mlx5_set_debug_mask();
 	set_freeze_on_error();
 	if (gethostname(context->hostname, sizeof(context->hostname)))
 		strcpy(context->hostname, "host_unknown");
@@ -2377,7 +2377,7 @@ static void mlx5_free_context(struct ibv_context *ibctx)
 		       page_size);
 	if (context->clock_info_page)
 		munmap((void *)context->clock_info_page, page_size);
-	close_debug_file(context);
+	mlx5_close_debug_file(context->dbg_fp);
 	clean_dyn_uars(ibctx);
 	reserved_qpn_blks_free(context);
 
providers/mlx5/mlx5.h +4 -0
index 3862007..7436bc8 100644
@@ -992,6 +992,10 @@ static inline struct mlx5_flow *to_mflow(struct ibv_flow *flow_id)
 
 bool is_mlx5_dev(struct ibv_device *device);
 
+void mlx5_open_debug_file(FILE **dbg_fp);
+void mlx5_close_debug_file(FILE *dbg_fp);
+void mlx5_set_debug_mask(void);
+
 int mlx5_alloc_buf(struct mlx5_buf *buf, size_t size, int page_size);
 void mlx5_free_buf(struct mlx5_buf *buf);
 int mlx5_alloc_buf_contig(struct mlx5_context *mctx, struct mlx5_buf *buf,
Signature · 1 line
1.8.3.1
#4 lore raw

Re: [PATCH rdma-core 03/27] mlx5: Enable debug functionality for vfio

Quoted 10 lines · Yishai Hadas

From: Maor Gottlieb <[email protected]>

Usage will be in next patches.

---
 providers/mlx5/mlx5.c | 28 ++++++++++++++--------------
 providers/mlx5/mlx5.h |  4 ++++
 2 files changed, 18 insertions(+), 14 deletions(-)

Probably, this patch will be needed to be changed after "Verbs logging API" PR https://github.com/linux-rdma/rdma-core/pull/1030

Thanks

#5 lore raw

Re: [PATCH rdma-core 03/27] mlx5: Enable debug functionality for vfio

Quoted 11 lines · Leon Romanovsky
Quoted 10 lines · Yishai Hadas

From: Maor Gottlieb <[email protected]>

Usage will be in next patches.

---
  providers/mlx5/mlx5.c | 28 ++++++++++++++--------------
  providers/mlx5/mlx5.h |  4 ++++
  2 files changed, 18 insertions(+), 14 deletions(-)
Quoted 4 lines

Probably, this patch will be needed to be changed after "Verbs logging API" PR https://github.com/linux-rdma/rdma-core/pull/1030

Thanks

Well, not really, this patch just reorganizes things inside mlx5 for code sharing.

Yishai

#6 lore raw

Re: [PATCH rdma-core 03/27] mlx5: Enable debug functionality for vfio

Quoted 12 lines · Yishai Hadas
Quoted 11 lines · Leon Romanovsky
Quoted 10 lines · Yishai Hadas

From: Maor Gottlieb <[email protected]>

Usage will be in next patches.

---
  providers/mlx5/mlx5.c | 28 ++++++++++++++--------------
  providers/mlx5/mlx5.h |  4 ++++
  2 files changed, 18 insertions(+), 14 deletions(-)
Quoted 7 lines
Quoted 4 lines

Probably, this patch will be needed to be changed after "Verbs logging API" PR https://github.com/linux-rdma/rdma-core/pull/1030

Thanks

Well, not really, this patch just reorganizes things inside mlx5 for code sharing.

After Gal's PR, I expect to see all mlx5 file/debug logic gone except some minimal conversion logic to support old legacy variables.

All that get_env_... code will go.

Thanks

Quoted 3 lines

Yishai

#7 lore raw

Re: [PATCH rdma-core 03/27] mlx5: Enable debug functionality for vfio

Quoted 17 lines · Leon Romanovsky
Quoted 16 lines · Yishai Hadas
Quoted 15 lines · Leon Romanovsky
Quoted 10 lines · Yishai Hadas

From: Maor Gottlieb <[email protected]>

Usage will be in next patches.

---
   providers/mlx5/mlx5.c | 28 ++++++++++++++--------------
   providers/mlx5/mlx5.h |  4 ++++
   2 files changed, 18 insertions(+), 14 deletions(-)

Probably, this patch will be needed to be changed after "Verbs logging API" PR https://github.com/linux-rdma/rdma-core/pull/1030

Thanks

Quoted 9 lines
Quoted 2 lines

Well, not really, this patch just reorganizes things inside mlx5 for code sharing.

After Gal's PR, I expect to see all mlx5 file/debug logic gone except some minimal conversion logic to support old legacy variables.

All that get_env_... code will go.

Thanks

Looking on current VERBs logging PR, it doesn't give a clean path conversion from mlx5.

For example it missed a debug granularity per object (e.g. QP, CQ, etc.) , in addition it doesn't define a driver specific options as we have today in mlx5 (e.g. MLX5_DBG_DR).

I believe that this should be added before going forward with the logging PR to enable a clean transition later on.

The transition of mlx5 should preserve current API/semantics (ENV, etc.) and is an orthogonal task.

Yishai

#8 lore raw

Re: [PATCH rdma-core 03/27] mlx5: Enable debug functionality for vfio

Quoted 34 lines · Yishai Hadas
Quoted 17 lines · Leon Romanovsky
Quoted 16 lines · Yishai Hadas
Quoted 15 lines · Leon Romanovsky
Quoted 10 lines · Yishai Hadas

From: Maor Gottlieb <[email protected]>

Usage will be in next patches.

---
   providers/mlx5/mlx5.c | 28 ++++++++++++++-------------- providers/mlx5/mlx5.h |  4 ++++

   2 files changed, 18 insertions(+), 14 deletions(-)

Probably, this patch will be needed to be changed after "Verbs logging API" PR https://github.com/linux-rdma/rdma-core/pull/1030

Thanks

Quoted 9 lines
Quoted 2 lines

Well, not really, this patch just reorganizes things inside mlx5 for code sharing.

After Gal's PR, I expect to see all mlx5 file/debug logic gone except some minimal conversion logic to support old legacy variables.

All that get_env_... code will go.

Thanks

Looking on current VERBs logging PR, it doesn't give a clean path conversion from mlx5.

For example it missed a debug granularity per object (e.g. QP, CQ, etc.) , in addition it doesn't define a driver specific options as we have today in mlx5 (e.g. MLX5_DBG_DR).

Quoted 5 lines

I believe that this should be added before going forward with the logging PR to enable a clean transition later on.

The transition of mlx5 should preserve current API/semantics (ENV, etc.) and is an orthogonal task.

Yishai, if you have any more concerns please share them in the PR.. The discussion there is going on for a while and you've been quiet so I assumed you're fine with it.

I disagree about needing to support everything that exists in mlx5 today, the purpose of the generic mechanism is to unify the environment variables, driver specific options do the opposite. IMO masking out a few prints isn't worth the divergence.

If the mlx5 provider has backwards compatibility issues it doesn't necessarily have to use this API, but we can at least covert most existing providers and all future providers that wish to support such functionality in a unified way.

BTW, why even insist on having backwards compatibility here? Do you actually have useful code that relies on debug environment variables?

#9 lore raw

Re: [PATCH rdma-core 03/27] mlx5: Enable debug functionality for vfio

Quoted 40 lines · Gal Pressman
Quoted 34 lines · Yishai Hadas
Quoted 17 lines · Leon Romanovsky
Quoted 16 lines · Yishai Hadas
Quoted 15 lines · Leon Romanovsky
Quoted 10 lines · Yishai Hadas

From: Maor Gottlieb <[email protected]>

Usage will be in next patches.

---
    providers/mlx5/mlx5.c | 28 ++++++++++++++--------------
    providers/mlx5/mlx5.h |  4 ++++
    2 files changed, 18 insertions(+), 14 deletions(-)

Probably, this patch will be needed to be changed after "Verbs logging API" PR https://github.com/linux-rdma/rdma-core/pull/1030

Thanks

Quoted 9 lines
Quoted 2 lines

Well, not really, this patch just reorganizes things inside mlx5 for code sharing.

After Gal's PR, I expect to see all mlx5 file/debug logic gone except some minimal conversion logic to support old legacy variables.

All that get_env_... code will go.

Thanks

Looking on current VERBs logging PR, it doesn't give a clean path conversion from mlx5.

For example it missed a debug granularity per object (e.g. QP, CQ, etc.) , in addition it doesn't define a driver specific options as we have today in mlx5 (e.g. MLX5_DBG_DR).

Quoted 5 lines

I believe that this should be added before going forward with the logging PR to enable a clean transition later on.

The transition of mlx5 should preserve current API/semantics (ENV, etc.) and is an orthogonal task.

Quoted 8 lines

Yishai, if you have any more concerns please share them in the PR.. The discussion there is going on for a while and you've been quiet so I assumed you're fine with it.

I disagree about needing to support everything that exists in mlx5 today, the purpose of the generic mechanism is to unify the environment variables, driver specific options do the opposite. IMO masking out a few prints isn't worth the divergence.

The options in mlx5 gives more granularity and supports vendor specific options, this may be needed down the road by other vendors as well.

If we come with a new API need to consider such options in the general case.

NP, I'll comment in the logging PR as well.

Quoted 4 lines

If the mlx5 provider has backwards compatibility issues it doesn't necessarily have to use this API, but we can at least covert most existing providers and all future providers that wish to support such functionality in a unified way.

The point was that current suggestion doesn't allow a clean transition for mlx5, so we won't use it unless the API will give a matching alternative.

Quoted 2 lines

BTW, why even insist on having backwards compatibility here? Do you actually have useful code that relies on debug environment variables?

Logging options (env, mask, etc.) are kind of API which need to be preserved, it's used in the field for debug purposes, no reason to loose granularity and trace.

Yishai

#10 lore raw

Re: [PATCH rdma-core 03/27] mlx5: Enable debug functionality for vfio

Quoted 67 lines · Yishai Hadas
Quoted 40 lines · Gal Pressman
Quoted 34 lines · Yishai Hadas
Quoted 17 lines · Leon Romanovsky
Quoted 16 lines · Yishai Hadas
Quoted 15 lines · Leon Romanovsky
Quoted 10 lines · Yishai Hadas

From: Maor Gottlieb <[email protected]>

Usage will be in next patches.

---
    providers/mlx5/mlx5.c | 28 ++++++++++++++-------------- providers/mlx5/mlx5.h |  4 ++++

    2 files changed, 18 insertions(+), 14 deletions(-)

Probably, this patch will be needed to be changed after "Verbs logging API" PR https://github.com/linux-rdma/rdma-core/pull/1030

Thanks

Quoted 9 lines
Quoted 2 lines

Well, not really, this patch just reorganizes things inside mlx5 for code sharing.

After Gal's PR, I expect to see all mlx5 file/debug logic gone except some minimal conversion logic to support old legacy variables.

All that get_env_... code will go.

Thanks

Looking on current VERBs logging PR, it doesn't give a clean path conversion from mlx5.

For example it missed a debug granularity per object (e.g. QP, CQ, etc.) , in addition it doesn't define a driver specific options as we have today in mlx5 (e.g. MLX5_DBG_DR).

Quoted 5 lines

I believe that this should be added before going forward with the logging PR to enable a clean transition later on.

The transition of mlx5 should preserve current API/semantics (ENV, etc.) and is an orthogonal task.

Quoted 8 lines

Yishai, if you have any more concerns please share them in the PR.. The discussion there is going on for a while and you've been quiet so I assumed you're fine with it.

I disagree about needing to support everything that exists in mlx5 today, the purpose of the generic mechanism is to unify the environment variables, driver specific options do the opposite. IMO masking out a few prints isn't worth the divergence.

The options in mlx5 gives more granularity and supports vendor specific options, this may be needed down the road by other vendors as well.

If we come with a new API need to consider such options in the general case.

NP, I'll comment in the logging PR as well.

Quoted 4 lines

If the mlx5 provider has backwards compatibility issues it doesn't necessarily have to use this API, but we can at least covert most existing providers and all future providers that wish to support such functionality in a unified way.

The point was that current suggestion doesn't allow a clean transition for mlx5, so we won't use it unless the API will give a matching alternative.

Quoted 6 lines
Quoted 2 lines

BTW, why even insist on having backwards compatibility here? Do you actually have useful code that relies on debug environment variables?

Logging options (env, mask, etc.) are kind of API which need to be preserved, it's used in the field for debug purposes, no reason to loose granularity and trace.

It's used in the field by *people*, which unlike legacy code can learn to use the new debug env variables, they don't need backwards compatibility (the same as users of debugfs).
I addressed the granularity issue in the PR.

#11 lore raw

PATCH 4/27 util: Add interval_set support

In reply to Yishai Hadas

From: Mark Zhang <[email protected]>

Add interval_set functionality to support range management.

This functionality enables to insert/get valid ranges of values and maintains the contiguity of them internally.

This will be used in down stream patches from this series to set/get valid iova ranges from this data structure.

---
 util/CMakeLists.txt |   2 +
 util/interval_set.c | 208 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 util/interval_set.h |  77 +++++++++++++++++++
 3 files changed, 287 insertions(+)
 create mode 100644 util/interval_set.c
 create mode 100644 util/interval_set.h
util/CMakeLists.txt +2 -0
index e8646bf..d8a66be 100644
@@ -1,6 +1,7 @@
 publish_internal_headers(util
   cl_qmap.h
   compiler.h
+  interval_set.h
   node_name_map.h
   rdma_nl.h
   symver.h
@@ -9,6 +10,7 @@ publish_internal_headers(util
 
 set(C_FILES
   cl_map.c
+  interval_set.c
   node_name_map.c
   open_cdev.c
   rdma_nl.c
util/interval_set.c new file +208 -0

208 lines not on this page · raw message

util/interval_set.h new file +77 -0

77 lines not on this page · raw message

Signature · 1 line
1.8.3.1
#12 lore raw

PATCH 5/27 verbs: Enable verbs_open_device() to work over non sysfs devices

In reply to Yishai Hadas

Enable verbs_open_device() to work over non sysfs devices as of mlx5 over VFIO.

Any other API over verbs_sysfs_dev should fail cleanly.

---
 libibverbs/device.c | 39 ++++++++++++++++++++++-----------------
 libibverbs/sysfs.c  |  5 +++++
 2 files changed, 27 insertions(+), 17 deletions(-)
libibverbs/device.c +22 -17
index 2f0b3b7..bd6a019 100644
@@ -124,7 +124,7 @@ LATEST_SYMVER_FUNC(ibv_get_device_guid, 1_1, "IBVERBS_1.1",
 	int i;
 
 	pthread_mutex_lock(&dev_list_lock);
-	if (sysfs_dev->flags & VSYSFS_READ_NODE_GUID) {
+	if (sysfs_dev && sysfs_dev->flags & VSYSFS_READ_NODE_GUID) {
 		guid = sysfs_dev->node_guid;
 		pthread_mutex_unlock(&dev_list_lock);
 		return htobe64(guid);
@@ -154,7 +154,7 @@ int ibv_get_device_index(struct ibv_device *device)
 {
 	struct verbs_sysfs_dev *sysfs_dev = verbs_get_device(device)->sysfs;
 
-	return sysfs_dev->ibdev_idx;
+	return sysfs_dev ? sysfs_dev->ibdev_idx : -1;
 }
 
 void verbs_init_cq(struct ibv_cq *cq, struct ibv_context *context,
@@ -323,18 +323,20 @@ static void set_lib_ops(struct verbs_context *vctx)
 struct ibv_context *verbs_open_device(struct ibv_device *device, void *private_data)
 {
 	struct verbs_device *verbs_device = verbs_get_device(device);
-	int cmd_fd;
+	int cmd_fd = -1;
 	struct verbs_context *context_ex;
 	int ret;
 
-	/*
-	 * We'll only be doing writes, but we need O_RDWR in case the
-	 * provider needs to mmap() the file.
-	 */
-	cmd_fd = open_cdev(verbs_device->sysfs->sysfs_name,
-			   verbs_device->sysfs->sysfs_cdev);
-	if (cmd_fd < 0)
-		return NULL;
+	if (verbs_device->sysfs) {
+		/*
+		 * We'll only be doing writes, but we need O_RDWR in case the
+		 * provider needs to mmap() the file.
+		 */
+		cmd_fd = open_cdev(verbs_device->sysfs->sysfs_name,
+				   verbs_device->sysfs->sysfs_cdev);
+		if (cmd_fd < 0)
+			return NULL;
+	}
 
 	/*
 	 * cmd_fd ownership is transferred into alloc_context, if it fails
@@ -345,11 +347,13 @@ struct ibv_context *verbs_open_device(struct ibv_device *device, void *private_d
 		return NULL;
 
 	set_lib_ops(context_ex);
-	if (context_ex->context.async_fd == -1) {
-		ret = ibv_cmd_alloc_async_fd(&context_ex->context);
-		if (ret) {
-			ibv_close_device(&context_ex->context);
-			return NULL;
+	if (verbs_device->sysfs) {
+		if (context_ex->context.async_fd == -1) {
+			ret = ibv_cmd_alloc_async_fd(&context_ex->context);
+			if (ret) {
+				ibv_close_device(&context_ex->context);
+				return NULL;
+			}
 		}
 	}
 
@@ -428,7 +432,8 @@ out:
 void verbs_uninit_context(struct verbs_context *context_ex)
 {
 	free(context_ex->priv);
-	close(context_ex->context.cmd_fd);
+	if (context_ex->context.cmd_fd != -1)
+		close(context_ex->context.cmd_fd);
 	if (context_ex->context.async_fd != -1)
 		close(context_ex->context.async_fd);
 	ibverbs_device_put(context_ex->context.device);
libibverbs/sysfs.c +5 -0
index 8ba4472..d898432 100644
@@ -127,6 +127,11 @@ int ibv_read_ibdev_sysfs_file(char *buf, size_t size,
 	va_list va;
 	int res;
 
+	if (!sysfs_dev) {
+		errno = EINVAL;
+		return -1;
+	}
+
 	va_start(va, fnfmt);
 	if (vasprintf(&path, fnfmt, va) < 0) {
 		va_end(va);
Signature · 1 line
1.8.3.1
#13 lore raw

PATCH 6/27 mlx5: Setup mlx5 vfio context

In reply to Yishai Hadas

Setup mlx5 vfio context by using the ioctl API over vfio to read the device initialization segment and other required stuff.

As part of that the applicable IOVA ranges are set to be ready for DMA usage and the command interface is initialized.

---
 providers/mlx5/mlx5.h      |   2 +
 providers/mlx5/mlx5_vfio.c | 740 +++++++++++++++++++++++++++++++++++++++++++++
 providers/mlx5/mlx5_vfio.h | 134 ++++++++
 3 files changed, 876 insertions(+)
providers/mlx5/mlx5.h +2 -0
index 7436bc8..7e7d70d 100644
@@ -72,6 +72,7 @@ enum {
 
 enum {
 	MLX5_ADAPTER_PAGE_SIZE		= 4096,
+	MLX5_ADAPTER_PAGE_SHIFT		= 12,
 };
 
 #define MLX5_CQ_PREFIX "MLX_CQ"
@@ -90,6 +91,7 @@ enum {
 	MLX5_DBG_CQ_CQE		= 1 << 4,
 	MLX5_DBG_CONTIG		= 1 << 5,
 	MLX5_DBG_DR		= 1 << 6,
+	MLX5_DBG_VFIO		= 1 << 7,
 };
 
 extern uint32_t mlx5_debug_mask;
providers/mlx5/mlx5_vfio.c +740 -0

755 lines not on this page · raw message

providers/mlx5/mlx5_vfio.h +134 -0

151 lines not on this page · raw message

Signature · 1 line
1.8.3.1
#14 lore raw

PATCH 7/27 mlx5: Add mlx5_vfio_cmd_exec() support

In reply to Yishai Hadas

Add mlx5_vfio_cmd_exec() support to enable running commands.

This includes the required functionality to prepare both the in/out command layouts and handle the response upon issuing the command.

---
 providers/mlx5/mlx5_ifc.h  |  38 ++++++
 providers/mlx5/mlx5_vfio.c | 287 +++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 325 insertions(+)
providers/mlx5/mlx5_ifc.h +38 -0
index 31d0bdc..aef6196 100644
@@ -73,6 +73,25 @@ enum {
 	MLX5_CMD_OP_SYNC_STEERING = 0xb00,
 };
 
+enum {
+	MLX5_CMD_STAT_OK = 0x0,
+	MLX5_CMD_STAT_INT_ERR = 0x1,
+	MLX5_CMD_STAT_BAD_OP_ERR = 0x2,
+	MLX5_CMD_STAT_BAD_PARAM_ERR = 0x3,
+	MLX5_CMD_STAT_BAD_SYS_STATE_ERR = 0x4,
+	MLX5_CMD_STAT_BAD_RES_ERR = 0x5,
+	MLX5_CMD_STAT_RES_BUSY = 0x6,
+	MLX5_CMD_STAT_LIM_ERR = 0x8,
+	MLX5_CMD_STAT_BAD_RES_STATE_ERR = 0x9,
+	MLX5_CMD_STAT_IX_ERR = 0xa,
+	MLX5_CMD_STAT_NO_RES_ERR = 0xf,
+	MLX5_CMD_STAT_BAD_INP_LEN_ERR = 0x50,
+	MLX5_CMD_STAT_BAD_OUTP_LEN_ERR = 0x51,
+	MLX5_CMD_STAT_BAD_QP_STATE_ERR = 0x10,
+	MLX5_CMD_STAT_BAD_PKT_ERR = 0x30,
+	MLX5_CMD_STAT_BAD_SIZE_OUTS_CQES_ERR = 0x40,
+};
+
 struct mlx5_ifc_atomic_caps_bits {
 	u8         reserved_at_0[0x40];
 
@@ -4093,4 +4112,23 @@ struct mlx5_ifc_create_psv_in_bits {
 	u8         reserved_at_60[0x20];
 };
 
+struct mlx5_ifc_mbox_out_bits {
+	u8	status[0x8];
+	u8	reserved_at_8[0x18];
+
+	u8	syndrome[0x20];
+
+	u8	reserved_at_40[0x40];
+};
+
+struct mlx5_ifc_mbox_in_bits {
+	u8	opcode[0x10];
+	u8	uid[0x10];
+
+	u8	reserved_at_20[0x10];
+	u8	op_mod[0x10];
+
+	u8	reserved_at_40[0x40];
+};
+
 #endif /* MLX5_IFC_H */
providers/mlx5/mlx5_vfio.c +287 -0

309 lines not on this page · raw message

Signature · 1 line
1.8.3.1
#15 lore raw

PATCH 8/27 mlx5: vfio setup function support

In reply to Yishai Hadas

Setup device function support by following the required command sequence and steps based on the device specification.

---
 providers/mlx5/mlx5_ifc.h  | 215 +++++++++++++++++++++++++++++++++++++++
 providers/mlx5/mlx5_vfio.c | 246 +++++++++++++++++++++++++++++++++++++++++++++
 providers/mlx5/mlx5_vfio.h |  16 +++
 providers/mlx5/mlx5dv.h    |   4 +
 4 files changed, 481 insertions(+)
providers/mlx5/mlx5_ifc.h +215 -0
index aef6196..ac741cd 100644
@@ -41,6 +41,13 @@ enum mlx5_cap_mode {
 
 enum {
 	MLX5_CMD_OP_QUERY_HCA_CAP = 0x100,
+	MLX5_CMD_OP_INIT_HCA = 0x102,
+	MLX5_CMD_OP_TEARDOWN_HCA = 0x103,
+	MLX5_CMD_OP_ENABLE_HCA = 0x104,
+	MLX5_CMD_OP_QUERY_PAGES = 0x107,
+	MLX5_CMD_OP_MANAGE_PAGES = 0x108,
+	MLX5_CMD_OP_QUERY_ISSI = 0x10a,
+	MLX5_CMD_OP_SET_ISSI = 0x10b,
 	MLX5_CMD_OP_CREATE_MKEY = 0x200,
 	MLX5_CMD_OP_CREATE_QP = 0x500,
 	MLX5_CMD_OP_RST2INIT_QP = 0x502,
@@ -55,6 +62,7 @@ enum {
 	MLX5_CMD_OP_QUERY_ESW_VPORT_CONTEXT = 0x752,
 	MLX5_CMD_OP_QUERY_NIC_VPORT_CONTEXT = 0x754,
 	MLX5_CMD_OP_QUERY_ROCE_ADDRESS = 0x760,
+	MLX5_CMD_OP_ACCESS_REG = 0x805,
 	MLX5_CMD_OP_QUERY_LAG = 0x842,
 	MLX5_CMD_OP_CREATE_TIR = 0x900,
 	MLX5_CMD_OP_MODIFY_SQ = 0x905,
@@ -92,6 +100,16 @@ enum {
 	MLX5_CMD_STAT_BAD_SIZE_OUTS_CQES_ERR = 0x40,
 };
 
+enum {
+	MLX5_PAGES_CANT_GIVE = 0,
+	MLX5_PAGES_GIVE = 1,
+	MLX5_PAGES_TAKE = 2,
+};
+
+enum {
+	MLX5_REG_HOST_ENDIANNESS = 0x7004,
+};
+
 struct mlx5_ifc_atomic_caps_bits {
 	u8         reserved_at_0[0x40];
 
@@ -4131,4 +4149,201 @@ struct mlx5_ifc_mbox_in_bits {
 	u8	reserved_at_40[0x40];
 };
 
+struct mlx5_ifc_enable_hca_in_bits {
+	u8         opcode[0x10];
+	u8         reserved_at_10[0x10];
+
+	u8         reserved_at_20[0x10];
+	u8         op_mod[0x10];
+
+	u8         reserved_at_40[0x10];
+	u8         function_id[0x10];
+
+	u8         reserved_at_60[0x20];
+};
+
+struct mlx5_ifc_enable_hca_out_bits {
+	u8         status[0x8];
+	u8         reserved_at_8[0x18];
+
+	u8         syndrome[0x20];
+
+	u8         reserved_at_40[0x20];
+};
+
+struct mlx5_ifc_query_issi_out_bits {
+	u8         status[0x8];
+	u8         reserved_at_8[0x18];
+
+	u8         syndrome[0x20];
+
+	u8         reserved_at_40[0x10];
+	u8         current_issi[0x10];
+
+	u8         reserved_at_60[0xa0];
+
+	u8         reserved_at_100[76][0x8];
+	u8         supported_issi_dw0[0x20];
+};
+
+struct mlx5_ifc_query_issi_in_bits {
+	u8         opcode[0x10];
+	u8         reserved_at_10[0x10];
+
+	u8         reserved_at_20[0x10];
+	u8         op_mod[0x10];
+
+	u8         reserved_at_40[0x40];
+};
+
+struct mlx5_ifc_set_issi_out_bits {
+	u8         status[0x8];
+	u8         reserved_at_8[0x18];
+
+	u8         syndrome[0x20];
+
+	u8         reserved_at_40[0x40];
+};
+
+struct mlx5_ifc_set_issi_in_bits {
+	u8         opcode[0x10];
+	u8         reserved_at_10[0x10];
+
+	u8         reserved_at_20[0x10];
+	u8         op_mod[0x10];
+
+	u8         reserved_at_40[0x10];
+	u8         current_issi[0x10];
+
+	u8         reserved_at_60[0x20];
+};
+
+struct mlx5_ifc_query_pages_out_bits {
+	u8         status[0x8];
+	u8         reserved_at_8[0x18];
+
+	u8         syndrome[0x20];
+
+	u8	   embedded_cpu_function[0x01];
+	u8	   reserved_bits[0x0f];
+	u8	   function_id[0x10];
+
+	u8	   num_pages[0x20];
+};
+
+struct mlx5_ifc_query_pages_in_bits {
+	u8	opcode[0x10];
+	u8	reserved_at_10[0x10];
+
+	u8	reserved_at_20[0x10];
+	u8	op_mod[0x10];
+
+	u8	reserved_at_40[0x10];
+	u8	function_id[0x10];
+
+	u8	reserved_at_60[0x20];
+};
+
+struct mlx5_ifc_manage_pages_out_bits {
+	u8         status[0x8];
+	u8         reserved_at_8[0x18];
+
+	u8         syndrome[0x20];
+
+	u8         output_num_entries[0x20];
+
+	u8         reserved_at_60[0x20];
+
+	u8         pas[][0x40];
+};
+
+struct mlx5_ifc_manage_pages_in_bits {
+	u8         opcode[0x10];
+	u8         reserved_at_10[0x10];
+
+	u8         reserved_at_20[0x10];
+	u8         op_mod[0x10];
+
+	u8         embedded_cpu_function[0x1];
+	u8         reserved_at_41[0xf];
+	u8         function_id[0x10];
+
+	u8         input_num_entries[0x20];
+
+	u8         pas[][0x40];
+};
+
+struct mlx5_ifc_teardown_hca_out_bits {
+	u8         status[0x8];
+	u8         reserved_at_8[0x18];
+
+	u8         syndrome[0x20];
+
+	u8         reserved_at_40[0x3f];
+
+	u8         state[0x1];
+};
+
+enum {
+	MLX5_TEARDOWN_HCA_IN_PROFILE_GRACEFUL_CLOSE = 0x0,
+};
+
+struct mlx5_ifc_teardown_hca_in_bits {
+	u8         opcode[0x10];
+	u8         reserved_at_10[0x10];
+
+	u8         reserved_at_20[0x10];
+	u8         op_mod[0x10];
+
+	u8         reserved_at_40[0x10];
+	u8         profile[0x10];
+
+	u8         reserved_at_60[0x20];
+};
+
+struct mlx5_ifc_init_hca_out_bits {
+	u8         status[0x8];
+	u8         reserved_at_8[0x18];
+
+	u8         syndrome[0x20];
+
+	u8         reserved_at_40[0x40];
+};
+
+struct mlx5_ifc_init_hca_in_bits {
+	u8         opcode[0x10];
+	u8         reserved_at_10[0x10];
+
+	u8         reserved_at_20[0x10];
+	u8         op_mod[0x10];
+
+	u8         reserved_at_40[0x40];
+};
+
+struct mlx5_ifc_access_register_out_bits {
+	u8         status[0x8];
+	u8         reserved_at_8[0x18];
+
+	u8         syndrome[0x20];
+
+	u8         reserved_at_40[0x40];
+
+	u8         register_data[][0x20];
+};
+
+struct mlx5_ifc_access_register_in_bits {
+	u8         opcode[0x10];
+	u8         reserved_at_10[0x10];
+
+	u8         reserved_at_20[0x10];
+	u8         op_mod[0x10];
+
+	u8         reserved_at_40[0x10];
+	u8         register_id[0x10];
+
+	u8         argument[0x20];
+
+	u8         register_data[][0x20];
+};
+
 #endif /* MLX5_IFC_H */
providers/mlx5/mlx5_vfio.c +246 -0

267 lines not on this page · raw message

providers/mlx5/mlx5_vfio.h +16 -0
index 392ddcb..36b1f40 100644
@@ -13,6 +13,9 @@
 #include <infiniband/driver.h>
 #include <util/interval_set.h>
 
+#define FW_INIT_WAIT_MS 2
+#define FW_PRE_INIT_TIMEOUT_MILI 120000
+
 enum {
 	MLX5_MAX_COMMANDS = 32,
 	MLX5_CMD_DATA_BLOCK_SIZE = 512,
@@ -32,6 +35,19 @@ struct mlx5_vfio_device {
 	uint32_t flags;
 };
 
+#if __BYTE_ORDER == __LITTLE_ENDIAN
+#define MLX5_SET_HOST_ENDIANNESS 0
+#elif __BYTE_ORDER == __BIG_ENDIAN
+#define MLX5_SET_HOST_ENDIANNESS 0x80
+#else
+#error Host endianness not defined
+#endif
+
+struct mlx5_reg_host_endianness {
+	uint8_t he;
+	uint8_t rsvd[15];
+};
+
 struct health_buffer {
 	__be32		assert_var[5];
 	__be32		rsvd0[3];
providers/mlx5/mlx5dv.h +4 -0
index e657527..6aaea37 100644
@@ -1687,6 +1687,10 @@ static inline uint64_t _devx_get64(const void *p, size_t bit_off)
 
 #define DEVX_GET64(typ, p, fld) _devx_get64(p, __devx_bit_off(typ, fld))
 
+#define DEVX_ARRAY_SET64(typ, p, fld, idx, v) do { \
+	DEVX_SET64(typ, p, fld[idx], v); \
+} while (0)
+
 struct mlx5dv_dr_domain;
 struct mlx5dv_dr_table;
 struct mlx5dv_dr_matcher;
Signature · 1 line
1.8.3.1
#16 lore raw

PATCH 9/27 mlx5: vfio setup basic caps

In reply to Yishai Hadas

Set basic caps that are required to initialize the device properly.

---
 providers/mlx5/mlx5_ifc.h  |  87 ++++++++++++++++++++-
 providers/mlx5/mlx5_vfio.c | 185 ++++++++++++++++++++++++++++++++++++++++++++-
 providers/mlx5/mlx5_vfio.h |  21 +++++
 3 files changed, 290 insertions(+), 3 deletions(-)
providers/mlx5/mlx5_ifc.h +85 -2
index ac741cd..082ac1f 100644
@@ -36,6 +36,7 @@
 #define u8 uint8_t
 
 enum mlx5_cap_mode {
+	HCA_CAP_OPMOD_GET_MAX = 0,
 	HCA_CAP_OPMOD_GET_CUR	= 1,
 };
 
@@ -46,6 +47,7 @@ enum {
 	MLX5_CMD_OP_ENABLE_HCA = 0x104,
 	MLX5_CMD_OP_QUERY_PAGES = 0x107,
 	MLX5_CMD_OP_MANAGE_PAGES = 0x108,
+	MLX5_CMD_OP_SET_HCA_CAP = 0x109,
 	MLX5_CMD_OP_QUERY_ISSI = 0x10a,
 	MLX5_CMD_OP_SET_ISSI = 0x10b,
 	MLX5_CMD_OP_CREATE_MKEY = 0x200,
@@ -61,6 +63,7 @@ enum {
 	MLX5_CMD_OP_QUERY_DCT = 0x713,
 	MLX5_CMD_OP_QUERY_ESW_VPORT_CONTEXT = 0x752,
 	MLX5_CMD_OP_QUERY_NIC_VPORT_CONTEXT = 0x754,
+	MLX5_CMD_OP_MODIFY_NIC_VPORT_CONTEXT = 0x755,
 	MLX5_CMD_OP_QUERY_ROCE_ADDRESS = 0x760,
 	MLX5_CMD_OP_ACCESS_REG = 0x805,
 	MLX5_CMD_OP_QUERY_LAG = 0x842,
@@ -110,6 +113,11 @@ enum {
 	MLX5_REG_HOST_ENDIANNESS = 0x7004,
 };
 
+enum {
+	MLX5_CAP_PORT_TYPE_IB  = 0x0,
+	MLX5_CAP_PORT_TYPE_ETH = 0x1,
+};
+
 struct mlx5_ifc_atomic_caps_bits {
 	u8         reserved_at_0[0x40];
 
@@ -140,7 +148,8 @@ struct mlx5_ifc_atomic_caps_bits {
 };
 
 struct mlx5_ifc_roce_cap_bits {
-	u8         reserved_0[0x5];
+	u8         reserved_0[0x4];
+	u8         sw_r_roce_src_udp_port[0x1];
 	u8         fl_rc_qp_when_roce_disabled[0x1];
 	u8         fl_rc_qp_when_roce_enabled[0x1];
 	u8         reserved_at_7[0x17];
@@ -912,7 +921,8 @@ struct mlx5_ifc_cmd_hca_cap_bits {
 	u8         uar_4k[0x1];
 	u8         reserved_at_241[0x9];
 	u8         uar_sz[0x6];
-	u8         reserved_at_250[0x3];
+	u8         reserved_at_250[0x2];
+	u8         umem_uid_0[0x1];
 	u8         log_max_dc_cnak_qps[0x5];
 	u8         log_pg_sz[0x8];
 
@@ -1339,8 +1349,11 @@ struct mlx5_ifc_query_hca_cap_in_bits {
 };
 
 enum mlx5_cap_type {
+	MLX5_CAP_GENERAL = 0,
 	MLX5_CAP_ODP = 2,
 	MLX5_CAP_ATOMIC = 3,
+	MLX5_CAP_ROCE,
+	MLX5_CAP_NUM,
 };
 
 enum {
@@ -4346,4 +4359,74 @@ struct mlx5_ifc_access_register_in_bits {
 	u8         register_data[][0x20];
 };
 
+struct mlx5_ifc_modify_nic_vport_context_out_bits {
+	u8         status[0x8];
+	u8         reserved_at_8[0x18];
+
+	u8         syndrome[0x20];
+
+	u8         reserved_at_40[0x40];
+};
+
+struct mlx5_ifc_modify_nic_vport_field_select_bits {
+	u8         reserved_at_0[0x12];
+	u8         affiliation[0x1];
+	u8         reserved_at_13[0x1];
+	u8         disable_uc_local_lb[0x1];
+	u8         disable_mc_local_lb[0x1];
+	u8         node_guid[0x1];
+	u8         port_guid[0x1];
+	u8         min_inline[0x1];
+	u8         mtu[0x1];
+	u8         change_event[0x1];
+	u8         promisc[0x1];
+	u8         permanent_address[0x1];
+	u8         addresses_list[0x1];
+	u8         roce_en[0x1];
+	u8         reserved_at_1f[0x1];
+};
+
+struct mlx5_ifc_modify_nic_vport_context_in_bits {
+	u8         opcode[0x10];
+	u8         reserved_at_10[0x10];
+
+	u8         reserved_at_20[0x10];
+	u8         op_mod[0x10];
+
+	u8         other_vport[0x1];
+	u8         reserved_at_41[0xf];
+	u8         vport_number[0x10];
+
+	struct mlx5_ifc_modify_nic_vport_field_select_bits field_select;
+
+	u8         reserved_at_80[0x780];
+
+	struct mlx5_ifc_nic_vport_context_bits nic_vport_context;
+};
+
+struct mlx5_ifc_set_hca_cap_out_bits {
+	u8         status[0x8];
+	u8         reserved_at_8[0x18];
+
+	u8         syndrome[0x20];
+
+	u8         reserved_at_40[0x40];
+};
+
+struct mlx5_ifc_set_hca_cap_in_bits {
+	u8         opcode[0x10];
+	u8         reserved_at_10[0x10];
+
+	u8         reserved_at_20[0x10];
+	u8         op_mod[0x10];
+
+	u8         other_function[0x1];
+	u8         reserved_at_41[0xf];
+	u8         function_id[0x10];
+
+	u8         reserved_at_60[0x20];
+
+	union mlx5_ifc_hca_cap_union_bits capability;
+};
+
 #endif /* MLX5_IFC_H */
providers/mlx5/mlx5_vfio.c +184 -1

203 lines not on this page · raw message

providers/mlx5/mlx5_vfio.h +21 -0
index 36b1f40..225c1b9 100644
@@ -12,6 +12,7 @@
 
 #include <infiniband/driver.h>
 #include <util/interval_set.h>
+#include "mlx5_ifc.h"
 
 #define FW_INIT_WAIT_MS 2
 #define FW_PRE_INIT_TIMEOUT_MILI 120000
@@ -43,6 +44,22 @@ struct mlx5_vfio_device {
 #error Host endianness not defined
 #endif
 
+/* GET Dev Caps macros */
+#define MLX5_VFIO_CAP_GEN(ctx, cap) \
+	DEVX_GET(cmd_hca_cap, ctx->caps.hca_cur[MLX5_CAP_GENERAL], cap)
+
+#define MLX5_VFIO_CAP_GEN_64(mdev, cap) \
+	DEVX_GET64(cmd_hca_cap, mdev->caps.hca_cur[MLX5_CAP_GENERAL], cap)
+
+#define MLX5_VFIO_CAP_GEN_MAX(ctx, cap) \
+	DEVX_GET(cmd_hca_cap, ctx->caps.hca_max[MLX5_CAP_GENERAL], cap)
+
+#define MLX5_VFIO_CAP_ROCE(ctx, cap) \
+	DEVX_GET(roce_cap, ctx->caps.hca_cur[MLX5_CAP_ROCE], cap)
+
+#define MLX5_VFIO_CAP_ROCE_MAX(ctx, cap) \
+	DEVX_GET(roce_cap, ctx->caps.hca_max[MLX5_CAP_ROCE], cap)
+
 struct mlx5_reg_host_endianness {
 	uint8_t he;
 	uint8_t rsvd[15];
@@ -162,6 +179,10 @@ struct mlx5_vfio_context {
 	size_t bar_map_size;
 	struct mlx5_vfio_cmd cmd;
 	bool have_eq;
+	struct {
+		uint32_t hca_cur[MLX5_CAP_NUM][DEVX_UN_SZ_DW(hca_cap_union)];
+		uint32_t hca_max[MLX5_CAP_NUM][DEVX_UN_SZ_DW(hca_cap_union)];
+	} caps;
 };
 
 static inline struct mlx5_vfio_device *to_mvfio_dev(struct ibv_device *ibdev)
Signature · 1 line
1.8.3.1
#17 lore raw

PATCH 10/27 mlx5: Support fast teardown over vfio

In reply to Yishai Hadas

From: Mark Zhang <[email protected]>

Add vfio fast teardown support; If it fails then do regular teardown.

---
 providers/mlx5/mlx5_ifc.h  |  5 +++
 providers/mlx5/mlx5_vfio.c | 76 +++++++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 80 insertions(+), 1 deletion(-)
providers/mlx5/mlx5_ifc.h +5 -0
index 082ac1f..4b7a4c2 100644
@@ -4286,6 +4286,10 @@ struct mlx5_ifc_manage_pages_in_bits {
 	u8         pas[][0x40];
 };
 
+enum {
+	MLX5_TEARDOWN_HCA_OUT_FORCE_STATE_FAIL = 0x1,
+};
+
 struct mlx5_ifc_teardown_hca_out_bits {
 	u8         status[0x8];
 	u8         reserved_at_8[0x18];
@@ -4299,6 +4303,7 @@ struct mlx5_ifc_teardown_hca_out_bits {
 
 enum {
 	MLX5_TEARDOWN_HCA_IN_PROFILE_GRACEFUL_CLOSE = 0x0,
+	MLX5_TEARDOWN_HCA_IN_PROFILE_PREPARE_FAST_TEARDOWN = 0x2,
 };
 
 struct mlx5_ifc_teardown_hca_in_bits {
providers/mlx5/mlx5_vfio.c +75 -1

88 lines not on this page · raw message

Signature · 1 line
1.8.3.1
#18 lore raw

PATCH 11/27 mlx5: Enable interrupt command mode over vfio

In reply to Yishai Hadas

Enable interrupt command mode over vfio by using EQ and its related device stuff.

---
 providers/mlx5/mlx5_ifc.h  | 150 ++++++++++++++++++
 providers/mlx5/mlx5_vfio.c | 373 ++++++++++++++++++++++++++++++++++++++++++++-
 providers/mlx5/mlx5_vfio.h |  65 ++++++++
 3 files changed, 582 insertions(+), 6 deletions(-)
providers/mlx5/mlx5_ifc.h +150 -0
index 4b7a4c2..2129779 100644
@@ -51,6 +51,8 @@ enum {
 	MLX5_CMD_OP_QUERY_ISSI = 0x10a,
 	MLX5_CMD_OP_SET_ISSI = 0x10b,
 	MLX5_CMD_OP_CREATE_MKEY = 0x200,
+	MLX5_CMD_OP_CREATE_EQ = 0x301,
+	MLX5_CMD_OP_DESTROY_EQ = 0x302,
 	MLX5_CMD_OP_CREATE_QP = 0x500,
 	MLX5_CMD_OP_RST2INIT_QP = 0x502,
 	MLX5_CMD_OP_INIT2RTR_QP = 0x503,
@@ -65,6 +67,8 @@ enum {
 	MLX5_CMD_OP_QUERY_NIC_VPORT_CONTEXT = 0x754,
 	MLX5_CMD_OP_MODIFY_NIC_VPORT_CONTEXT = 0x755,
 	MLX5_CMD_OP_QUERY_ROCE_ADDRESS = 0x760,
+	MLX5_CMD_OP_ALLOC_UAR = 0x802,
+	MLX5_CMD_OP_DEALLOC_UAR = 0x803,
 	MLX5_CMD_OP_ACCESS_REG = 0x805,
 	MLX5_CMD_OP_QUERY_LAG = 0x842,
 	MLX5_CMD_OP_CREATE_TIR = 0x900,
@@ -118,6 +122,15 @@ enum {
 	MLX5_CAP_PORT_TYPE_ETH = 0x1,
 };
 
+enum mlx5_event {
+	MLX5_EVENT_TYPE_CMD = 0x0a,
+	MLX5_EVENT_TYPE_PAGE_REQUEST = 0xb,
+};
+
+enum {
+	MLX5_EQ_DOORBEL_OFFSET = 0x40,
+};
+
 struct mlx5_ifc_atomic_caps_bits {
 	u8         reserved_at_0[0x40];
 
@@ -4434,4 +4447,141 @@ struct mlx5_ifc_set_hca_cap_in_bits {
 	union mlx5_ifc_hca_cap_union_bits capability;
 };
 
+struct mlx5_ifc_alloc_uar_out_bits {
+	u8         status[0x8];
+	u8         reserved_at_8[0x18];
+
+	u8         syndrome[0x20];
+
+	u8         reserved_at_40[0x8];
+	u8         uar[0x18];
+
+	u8         reserved_at_60[0x20];
+};
+
+struct mlx5_ifc_alloc_uar_in_bits {
+	u8         opcode[0x10];
+	u8         reserved_at_10[0x10];
+
+	u8         reserved_at_20[0x10];
+	u8         op_mod[0x10];
+
+	u8         reserved_at_40[0x40];
+};
+
+struct mlx5_ifc_dealloc_uar_out_bits {
+	u8         status[0x8];
+	u8         reserved_at_8[0x18];
+
+	u8         syndrome[0x20];
+
+	u8         reserved_at_40[0x40];
+};
+
+struct mlx5_ifc_dealloc_uar_in_bits {
+	u8         opcode[0x10];
+	u8         reserved_at_10[0x10];
+
+	u8         reserved_at_20[0x10];
+	u8         op_mod[0x10];
+
+	u8         reserved_at_40[0x8];
+	u8         uar[0x18];
+
+	u8         reserved_at_60[0x20];
+};
+
+struct mlx5_ifc_eqc_bits {
+	u8         status[0x4];
+	u8         reserved_at_4[0x9];
+	u8         ec[0x1];
+	u8         oi[0x1];
+	u8         reserved_at_f[0x5];
+	u8         st[0x4];
+	u8         reserved_at_18[0x8];
+
+	u8         reserved_at_20[0x20];
+
+	u8         reserved_at_40[0x14];
+	u8         page_offset[0x6];
+	u8         reserved_at_5a[0x6];
+
+	u8         reserved_at_60[0x3];
+	u8         log_eq_size[0x5];
+	u8         uar_page[0x18];
+
+	u8         reserved_at_80[0x20];
+
+	u8         reserved_at_a0[0x18];
+	u8         intr[0x8];
+
+	u8         reserved_at_c0[0x3];
+	u8         log_page_size[0x5];
+	u8         reserved_at_c8[0x18];
+
+	u8         reserved_at_e0[0x60];
+
+	u8         reserved_at_140[0x8];
+	u8         consumer_counter[0x18];
+
+	u8         reserved_at_160[0x8];
+	u8         producer_counter[0x18];
+
+	u8         reserved_at_180[0x80];
+};
+
+struct mlx5_ifc_create_eq_out_bits {
+	u8         status[0x8];
+	u8         reserved_at_8[0x18];
+
+	u8         syndrome[0x20];
+
+	u8         reserved_at_40[0x18];
+	u8         eq_number[0x8];
+
+	u8         reserved_at_60[0x20];
+};
+
+struct mlx5_ifc_create_eq_in_bits {
+	u8         opcode[0x10];
+	u8         uid[0x10];
+
+	u8         reserved_at_20[0x10];
+	u8         op_mod[0x10];
+
+	u8         reserved_at_40[0x40];
+
+	struct mlx5_ifc_eqc_bits eq_context_entry;
+
+	u8         reserved_at_280[0x40];
+
+	u8         event_bitmask[4][0x40];
+
+	u8         reserved_at_3c0[0x4c0];
+
+	u8         pas[][0x40];
+};
+
+struct mlx5_ifc_destroy_eq_out_bits {
+	u8         status[0x8];
+	u8         reserved_at_8[0x18];
+
+	u8         syndrome[0x20];
+
+	u8         reserved_at_40[0x40];
+};
+
+struct mlx5_ifc_destroy_eq_in_bits {
+	u8         opcode[0x10];
+	u8         reserved_at_10[0x10];
+
+	u8         reserved_at_20[0x10];
+	u8         op_mod[0x10];
+
+	u8         reserved_at_40[0x18];
+	u8         eq_number[0x8];
+
+	u8         reserved_at_60[0x20];
+};
+
 #endif /* MLX5_IFC_H */
providers/mlx5/mlx5_vfio.c +367 -6

448 lines not on this page · raw message

providers/mlx5/mlx5_vfio.h +65 -0

95 lines not on this page · raw message

Signature · 1 line
1.8.3.1
#19 lore raw

PATCH 12/27 mlx5: Introduce vfio APIs to process events

In reply to Yishai Hadas

Introduce vfio APIs to process events, it includes:
- mlx5dv_vfio_get_events_fd()
- mlx5dv_vfio_process_events()

The first API returns an FD number that should be used to detect whether events exist.

The second API should be called to process the existing events.

At that step the PAGE request event support was added in addition to support async command mode to let the mlx5dv_vfio_process_events() return to the caller without blocking.

Detailed man pages were added to describe the expected usage of the new APIs.

---
 debian/ibverbs-providers.symbols                   |   2 +
 providers/mlx5/libmlx5.map                         |   2 +
 providers/mlx5/man/CMakeLists.txt                  |   2 +
 providers/mlx5/man/mlx5dv_vfio_get_events_fd.3.md  |  41 ++++
 providers/mlx5/man/mlx5dv_vfio_process_events.3.md |  43 ++++
 providers/mlx5/mlx5_vfio.c                         | 231 ++++++++++++++++++++-
 providers/mlx5/mlx5_vfio.h                         |  14 ++
 providers/mlx5/mlx5dv.h                            |   8 +
 8 files changed, 332 insertions(+), 11 deletions(-)
 create mode 100644 providers/mlx5/man/mlx5dv_vfio_get_events_fd.3.md
 create mode 100644 providers/mlx5/man/mlx5dv_vfio_process_events.3.md
debian/ibverbs-providers.symbols +2 -0
index 64e29b1..3e36592 100644
@@ -135,6 +135,8 @@ libmlx5.so.1 ibverbs-providers #MINVER#
  mlx5dv_qp_cancel_posted_send_wrs@MLX5_1.20 36
  _mlx5dv_mkey_check@MLX5_1.20 36
  mlx5dv_get_vfio_device_list@MLX5_1.21 37
+ mlx5dv_vfio_get_events_fd@MLX5_1.21 37
+ mlx5dv_vfio_process_events@MLX5_1.21 37
 libefa.so.1 ibverbs-providers #MINVER#
 * Build-Depends-Package: libibverbs-dev
  EFA_1.0@EFA_1.0 24
providers/mlx5/libmlx5.map +2 -0
index 3e8a4d8..d6294d8 100644
@@ -193,4 +193,6 @@ MLX5_1.20 {
 MLX5_1.21 {
         global:
 		mlx5dv_get_vfio_device_list;
+		mlx5dv_vfio_get_events_fd;
+		mlx5dv_vfio_process_events;
 } MLX5_1.20;
providers/mlx5/man/CMakeLists.txt +2 -0
index 91aebed..cb3525c 100644
@@ -40,6 +40,8 @@ rdma_man_pages(
   mlx5dv_sched_node_create.3.md
   mlx5dv_ts_to_ns.3
   mlx5dv_wr_mkey_configure.3.md
+  mlx5dv_vfio_get_events_fd.3.md
+  mlx5dv_vfio_process_events.3.md
   mlx5dv_wr_post.3.md
   mlx5dv_wr_set_mkey_sig_block.3.md
   mlx5dv.7
providers/mlx5/man/mlx5dv_vfio_get_events_fd.3.md new file +41 -0

41 lines not on this page · raw message

providers/mlx5/man/mlx5dv_vfio_process_events.3.md new file +43 -0

43 lines not on this page · raw message

providers/mlx5/mlx5_vfio.c +220 -11

338 lines not on this page · raw message

providers/mlx5/mlx5_vfio.h +14 -0
index 449a5c5..8e240c8 100644
@@ -151,9 +151,17 @@ struct mlx5_cmd_msg {
 	struct mlx5_cmd_mailbox *next;
 };
 
+
 typedef int (*vfio_cmd_slot_comp)(struct mlx5_vfio_context *ctx,
 				  unsigned long slot);
 
+struct cmd_async_data {
+	void *buff_in;
+	int ilen;
+	void *buff_out;
+	int olen;
+};
+
 struct mlx5_vfio_cmd_slot {
 	struct mlx5_cmd_layout *lay;
 	struct mlx5_cmd_msg in;
@@ -161,6 +169,11 @@ struct mlx5_vfio_cmd_slot {
 	pthread_mutex_t lock;
 	int completion_event_fd;
 	vfio_cmd_slot_comp comp_func;
+	/* async cmd caller data */
+	bool in_use;
+	struct cmd_async_data curr;
+	bool is_pending;
+	struct cmd_async_data pending;
 };
 
 struct mlx5_vfio_cmd {
@@ -245,6 +258,7 @@ struct mlx5_vfio_context {
 		uint32_t hca_cur[MLX5_CAP_NUM][DEVX_UN_SZ_DW(hca_cap_union)];
 		uint32_t hca_max[MLX5_CAP_NUM][DEVX_UN_SZ_DW(hca_cap_union)];
 	} caps;
+
 	struct mlx5_eq async_eq;
 	struct mlx5_vfio_eqs_uar eqs_uar;
 	pthread_mutex_t eq_lock;
providers/mlx5/mlx5dv.h +8 -0
index 6aaea37..c9699ec 100644
@@ -1487,6 +1487,14 @@ struct mlx5dv_vfio_context_attr {
 struct ibv_device **
 mlx5dv_get_vfio_device_list(struct mlx5dv_vfio_context_attr *attr);
 
+int mlx5dv_vfio_get_events_fd(struct ibv_context *ibctx);
+
+/* This API should run from application thread and maintain device events.
+ * The application is responsible to get the events FD by calling mlx5dv_vfio_get_events_fd
+ * and once the FD is pollable call the API to let driver process the ready events.
+ */
+int mlx5dv_vfio_process_events(struct ibv_context *context);
+
 struct ibv_context *
 mlx5dv_open_device(struct ibv_device *device, struct mlx5dv_context_attr *attr);
 
Signature · 1 line
1.8.3.1
#20 lore raw

PATCH 13/27 mlx5: VFIO poll_health support

In reply to Yishai Hadas

From: Mark Zhang <[email protected]>

Add firmware health polling support in vfio driver.

Such a case is not expected and we refer it as some fatal error in the firmware that should be avoided/fixed.

The health buffer check is triggered by the application upon its call to mlx5dv_vfio_process_events().

---
 providers/mlx5/mlx5_vfio.c | 168 +++++++++++++++++++++++++++++++++++++++++++++
 providers/mlx5/mlx5_vfio.h |  10 ++-
 2 files changed, 177 insertions(+), 1 deletion(-)
providers/mlx5/mlx5_vfio.c +168 -0
index 85ee25b..c37358c 100644
@@ -22,6 +22,8 @@
 #include <poll.h>
 #include <util/mmio.h>
 
+#include <ccan/array_size.h>
+
 #include "mlx5dv.h"
 #include "mlx5_vfio.h"
 #include "mlx5.h"
@@ -1910,6 +1912,7 @@ enum mlx5_cmd_addr_l_sz_offset {
 
 enum {
 	MLX5_NIC_IFC_DISABLED = 1,
+	MLX5_NIC_IFC_SW_RESET = 7,
 };
 
 static uint8_t mlx5_vfio_get_nic_state(struct mlx5_vfio_context *ctx)
@@ -1978,6 +1981,169 @@ static int mlx5_vfio_teardown_hca(struct mlx5_vfio_context *ctx)
 	return mlx5_vfio_teardown_hca_regular(ctx);
 }
 
+static bool sensor_pci_not_working(struct mlx5_init_seg *init_seg)
+{
+	/* Offline PCI reads return 0xffffffff */
+	return (be32toh(mmio_read32_be(&init_seg->health.fw_ver)) == 0xffffffff);
+}
+
+enum mlx5_fatal_assert_bit_offsets {
+	MLX5_RFR_OFFSET = 31,
+};
+
+static bool sensor_fw_synd_rfr(struct mlx5_init_seg *init_seg)
+{
+	uint32_t rfr = be32toh(mmio_read32_be(&init_seg->health.rfr)) >> MLX5_RFR_OFFSET;
+	uint8_t synd = mmio_read8(&init_seg->health.synd);
+
+	return (rfr && synd);
+}
+
+enum  {
+	MLX5_SENSOR_NO_ERR = 0,
+	MLX5_SENSOR_PCI_COMM_ERR = 1,
+	MLX5_SENSOR_NIC_DISABLED = 3,
+	MLX5_SENSOR_NIC_SW_RESET = 4,
+	MLX5_SENSOR_FW_SYND_RFR = 5,
+};
+
+static uint32_t mlx5_health_check_fatal_sensors(struct mlx5_vfio_context *ctx)
+{
+	if (sensor_pci_not_working(ctx->bar_map))
+		return MLX5_SENSOR_PCI_COMM_ERR;
+
+	if (mlx5_vfio_get_nic_state(ctx) == MLX5_NIC_IFC_DISABLED)
+		return MLX5_SENSOR_NIC_DISABLED;
+
+	if (mlx5_vfio_get_nic_state(ctx) == MLX5_NIC_IFC_SW_RESET)
+		return MLX5_SENSOR_NIC_SW_RESET;
+
+	if (sensor_fw_synd_rfr(ctx->bar_map))
+		return MLX5_SENSOR_FW_SYND_RFR;
+
+	return MLX5_SENSOR_NO_ERR;
+}
+
+enum {
+	MLX5_HEALTH_SYNDR_FW_ERR = 0x1,
+	MLX5_HEALTH_SYNDR_IRISC_ERR = 0x7,
+	MLX5_HEALTH_SYNDR_HW_UNRECOVERABLE_ERR = 0x8,
+	MLX5_HEALTH_SYNDR_CRC_ERR = 0x9,
+	MLX5_HEALTH_SYNDR_FETCH_PCI_ERR = 0xa,
+	MLX5_HEALTH_SYNDR_HW_FTL_ERR = 0xb,
+	MLX5_HEALTH_SYNDR_ASYNC_EQ_OVERRUN_ERR = 0xc,
+	MLX5_HEALTH_SYNDR_EQ_ERR = 0xd,
+	MLX5_HEALTH_SYNDR_EQ_INV = 0xe,
+	MLX5_HEALTH_SYNDR_FFSER_ERR = 0xf,
+	MLX5_HEALTH_SYNDR_HIGH_TEMP = 0x10,
+};
+
+static const char *hsynd_str(u8 synd)
+{
+	switch (synd) {
+	case MLX5_HEALTH_SYNDR_FW_ERR:
+		return "firmware internal error";
+	case MLX5_HEALTH_SYNDR_IRISC_ERR:
+		return "irisc not responding";
+	case MLX5_HEALTH_SYNDR_HW_UNRECOVERABLE_ERR:
+		return "unrecoverable hardware error";
+	case MLX5_HEALTH_SYNDR_CRC_ERR:
+		return "firmware CRC error";
+	case MLX5_HEALTH_SYNDR_FETCH_PCI_ERR:
+		return "ICM fetch PCI error";
+	case MLX5_HEALTH_SYNDR_HW_FTL_ERR:
+		return "HW fatal error\n";
+	case MLX5_HEALTH_SYNDR_ASYNC_EQ_OVERRUN_ERR:
+		return "async EQ buffer overrun";
+	case MLX5_HEALTH_SYNDR_EQ_ERR:
+		return "EQ error";
+	case MLX5_HEALTH_SYNDR_EQ_INV:
+		return "Invalid EQ referenced";
+	case MLX5_HEALTH_SYNDR_FFSER_ERR:
+		return "FFSER error";
+	case MLX5_HEALTH_SYNDR_HIGH_TEMP:
+		return "High temperature";
+	default:
+		return "unrecognized error";
+	}
+}
+
+static void print_health_info(struct mlx5_vfio_context *ctx)
+{
+	struct mlx5_init_seg *iseg = ctx->bar_map;
+	struct health_buffer *h = &iseg->health;
+	char fw_str[18] = {};
+	int i;
+
+	/* If the syndrome is 0, the device is OK and no need to print buffer */
+	if (!mmio_read8(&h->synd))
+		return;
+
+	for (i = 0; i < ARRAY_SIZE(h->assert_var); i++)
+		mlx5_err(ctx->dbg_fp, "assert_var[%d] 0x%08x\n",
+			 i, be32toh(mmio_read32_be(h->assert_var + i)));
+
+	mlx5_err(ctx->dbg_fp, "assert_exit_ptr 0x%08x\n",
+		 be32toh(mmio_read32_be(&h->assert_exit_ptr)));
+	mlx5_err(ctx->dbg_fp, "assert_callra 0x%08x\n",
+		 be32toh(mmio_read32_be(&h->assert_callra)));
+	sprintf(fw_str, "%d.%d.%d",
+		be32toh(mmio_read32_be(&iseg->fw_rev)) & 0xffff,
+		be32toh(mmio_read32_be(&iseg->fw_rev)) >> 16,
+		be32toh(mmio_read32_be(&iseg->cmdif_rev_fw_sub)) & 0xffff);
+	mlx5_err(ctx->dbg_fp, "fw_ver %s\n", fw_str);
+	mlx5_err(ctx->dbg_fp, "hw_id 0x%08x\n", be32toh(mmio_read32_be(&h->hw_id)));
+	mlx5_err(ctx->dbg_fp, "irisc_index %d\n", mmio_read8(&h->irisc_index));
+	mlx5_err(ctx->dbg_fp, "synd 0x%x: %s\n", mmio_read8(&h->synd),
+		 hsynd_str(mmio_read8(&h->synd)));
+	mlx5_err(ctx->dbg_fp, "ext_synd 0x%04x\n",
+		 be16toh(mmio_read16_be(&h->ext_synd)));
+	mlx5_err(ctx->dbg_fp, "raw fw_ver 0x%08x\n",
+		 be32toh(mmio_read32_be(&iseg->fw_rev)));
+}
+
+static void mlx5_vfio_poll_health(struct mlx5_vfio_context *ctx)
+{
+	struct mlx5_vfio_health_state *hstate = &ctx->health_state;
+	uint32_t fatal_error, count;
+	struct timeval tv;
+	uint64_t time;
+	int ret;
+
+	ret = gettimeofday(&tv, NULL);
+	if (ret)
+		return;
+
+	time = (uint64_t)tv.tv_sec * 1000 + tv.tv_usec / 1000;
+	if (time - hstate->prev_time < POLL_HEALTH_INTERVAL)
+		return;
+
+	fatal_error = mlx5_health_check_fatal_sensors(ctx);
+	if (fatal_error) {
+		mlx5_err(ctx->dbg_fp, "%s: Fatal error %u detected\n",
+			 __func__, fatal_error);
+		goto err;
+	}
+	count = be32toh(mmio_read32_be(&ctx->bar_map->health_counter)) & 0xffffff;
+	if (count == hstate->prev_count)
+		++hstate->miss_counter;
+	else
+		hstate->miss_counter = 0;
+
+	hstate->prev_time = time;
+	hstate->prev_count = count;
+	if (hstate->miss_counter == MAX_MISSES) {
+		mlx5_err(ctx->dbg_fp,
+			 "device's health compromised - reached miss count\n");
+		goto err;
+	}
+
+	return;
+err:
+	print_health_info(ctx);
+	abort();
+}
+
 static int mlx5_vfio_setup_function(struct mlx5_vfio_context *ctx)
 {
 	int err;
@@ -2232,6 +2398,8 @@ int mlx5dv_vfio_process_events(struct ibv_context *ibctx)
 	uint64_t u;
 	ssize_t s;
 
+	mlx5_vfio_poll_health(ctx);
+
 	/* read to re-arm the FD and process all existing events */
 	s = read(ctx->cmd_comp_fd, &u, sizeof(uint64_t));
 	if (s < 0 && errno != EAGAIN) {
providers/mlx5/mlx5_vfio.h +9 -1
index 8e240c8..296d6d1 100644
@@ -240,6 +240,14 @@ struct mlx5_vfio_eqs_uar {
 	uint64_t iova;
 };
 
+#define POLL_HEALTH_INTERVAL 1000 /* ms */
+#define MAX_MISSES 3
+struct mlx5_vfio_health_state {
+	uint64_t prev_time; /* ms */
+	uint32_t prev_count;
+	uint32_t miss_counter;
+};
+
 struct mlx5_vfio_context {
 	struct verbs_context vctx;
 	int container_fd;
@@ -258,7 +266,7 @@ struct mlx5_vfio_context {
 		uint32_t hca_cur[MLX5_CAP_NUM][DEVX_UN_SZ_DW(hca_cap_union)];
 		uint32_t hca_max[MLX5_CAP_NUM][DEVX_UN_SZ_DW(hca_cap_union)];
 	} caps;
-
+	struct mlx5_vfio_health_state health_state;
 	struct mlx5_eq async_eq;
 	struct mlx5_vfio_eqs_uar eqs_uar;
 	pthread_mutex_t eq_lock;
Signature · 1 line
1.8.3.1
#21 lore raw

PATCH 14/27 mlx5: Implement basic verbs operation for PD and MR over vfio

In reply to Yishai Hadas

Implement basic verbs operation for PD and MR over vfio, this includes:
- PD alloc/dealloc
- MR reg/dereg.

---
 providers/mlx5/mlx5_ifc.h  |  76 ++++++++++++-
 providers/mlx5/mlx5_vfio.c | 273 +++++++++++++++++++++++++++++++++++++++++++++
 providers/mlx5/mlx5_vfio.h |  25 +++++
 util/util.h                |   5 +
 4 files changed, 377 insertions(+), 2 deletions(-)
providers/mlx5/mlx5_ifc.h +74 -2
index 2129779..1cbe846 100644
@@ -51,6 +51,7 @@ enum {
 	MLX5_CMD_OP_QUERY_ISSI = 0x10a,
 	MLX5_CMD_OP_SET_ISSI = 0x10b,
 	MLX5_CMD_OP_CREATE_MKEY = 0x200,
+	MLX5_CMD_OP_DESTROY_MKEY = 0x202,
 	MLX5_CMD_OP_CREATE_EQ = 0x301,
 	MLX5_CMD_OP_DESTROY_EQ = 0x302,
 	MLX5_CMD_OP_CREATE_QP = 0x500,
@@ -67,6 +68,8 @@ enum {
 	MLX5_CMD_OP_QUERY_NIC_VPORT_CONTEXT = 0x754,
 	MLX5_CMD_OP_MODIFY_NIC_VPORT_CONTEXT = 0x755,
 	MLX5_CMD_OP_QUERY_ROCE_ADDRESS = 0x760,
+	MLX5_CMD_OP_ALLOC_PD = 0x800,
+	MLX5_CMD_OP_DEALLOC_PD = 0x801,
 	MLX5_CMD_OP_ALLOC_UAR = 0x802,
 	MLX5_CMD_OP_DEALLOC_UAR = 0x803,
 	MLX5_CMD_OP_ACCESS_REG = 0x805,
@@ -1380,7 +1383,8 @@ enum {
 };
 
 enum {
-	MLX5_MKC_ACCESS_MODE_KLMS  = 0x2,
+	MLX5_MKC_ACCESS_MODE_MTT = 0x1,
+	MLX5_MKC_ACCESS_MODE_KLMS = 0x2,
 };
 
 struct mlx5_ifc_mkc_bits {
@@ -1425,7 +1429,9 @@ struct mlx5_ifc_mkc_bits {
 
 	u8         translations_octword_size[0x20];
 
-	u8         reserved_at_1c0[0x1b];
+	u8         reserved_at_1c0[0x19];
+	u8         relaxed_ordering_read[0x1];
+	u8         reserved_at_1d9[0x1];
 	u8         log_page_size[0x5];
 
 	u8         reserved_at_1e0[0x20];
@@ -1467,6 +1473,28 @@ struct mlx5_ifc_create_mkey_in_bits {
 	u8         klm_pas_mtt[0][0x20];
 };
 
+struct mlx5_ifc_destroy_mkey_out_bits {
+	u8         status[0x8];
+	u8         reserved_at_8[0x18];
+
+	u8         syndrome[0x20];
+
+	u8         reserved_at_40[0x40];
+};
+
+struct mlx5_ifc_destroy_mkey_in_bits {
+	u8         opcode[0x10];
+	u8         uid[0x10];
+
+	u8         reserved_at_20[0x10];
+	u8         op_mod[0x10];
+
+	u8         reserved_at_40[0x8];
+	u8         mkey_index[0x18];
+
+	u8         reserved_at_60[0x20];
+};
+
 struct mlx5_ifc_l2_hdr_bits {
 	u8         dmac_47_16[0x20];
 	u8         dmac_15_0[0x10];
@@ -4584,4 +4612,48 @@ struct mlx5_ifc_destroy_eq_in_bits {
 	u8         reserved_at_60[0x20];
 };
 
+struct mlx5_ifc_alloc_pd_out_bits {
+	u8         status[0x8];
+	u8         reserved_at_8[0x18];
+
+	u8         syndrome[0x20];
+
+	u8         reserved_at_40[0x8];
+	u8         pd[0x18];
+
+	u8         reserved_at_60[0x20];
+};
+
+struct mlx5_ifc_alloc_pd_in_bits {
+	u8         opcode[0x10];
+	u8         uid[0x10];
+
+	u8         reserved_at_20[0x10];
+	u8         op_mod[0x10];
+
+	u8         reserved_at_40[0x40];
+};
+
+struct mlx5_ifc_dealloc_pd_out_bits {
+	u8         status[0x8];
+	u8         reserved_at_8[0x18];
+
+	u8         syndrome[0x20];
+
+	u8         reserved_at_40[0x40];
+};
+
+struct mlx5_ifc_dealloc_pd_in_bits {
+	u8         opcode[0x10];
+	u8         uid[0x10];
+
+	u8         reserved_at_20[0x10];
+	u8         op_mod[0x10];
+
+	u8         reserved_at_40[0x8];
+	u8         pd[0x18];
+
+	u8         reserved_at_60[0x20];
+};
+
 #endif /* MLX5_IFC_H */
providers/mlx5/mlx5_vfio.c +273 -0

297 lines not on this page · raw message

providers/mlx5/mlx5_vfio.h +25 -0
index 296d6d1..5311c6f 100644
@@ -23,17 +23,37 @@ enum {
 	MLX5_PCI_CMD_XPORT = 7,
 };
 
+enum mlx5_ib_mtt_access_flags {
+	MLX5_MTT_READ  = (1 << 0),
+	MLX5_MTT_WRITE = (1 << 1),
+};
+
+enum {
+	MLX5_MAX_PAGE_SHIFT = 31,
+};
+
+#define MLX5_MTT_PRESENT (MLX5_MTT_READ | MLX5_MTT_WRITE)
+
 enum {
 	MLX5_VFIO_BLOCK_SIZE = 2 * 1024 * 1024,
 	MLX5_VFIO_BLOCK_NUM_PAGES = MLX5_VFIO_BLOCK_SIZE / MLX5_ADAPTER_PAGE_SIZE,
 };
 
+struct mlx5_vfio_mr {
+	struct verbs_mr vmr;
+	uint64_t iova;
+	uint64_t iova_page_size;
+	uint64_t iova_aligned_offset;
+	uint64_t iova_reg_size;
+};
+
 struct mlx5_vfio_device {
 	struct verbs_device vdev;
 	char *pci_name;
 	char vfio_path[IBV_SYSFS_PATH_MAX];
 	int page_size;
 	uint32_t flags;
+	atomic_int mkey_var;
 };
 
 #if __BYTE_ORDER == __LITTLE_ENDIAN
@@ -282,4 +302,9 @@ static inline struct mlx5_vfio_context *to_mvfio_ctx(struct ibv_context *ibctx)
 	return container_of(ibctx, struct mlx5_vfio_context, vctx.context);
 }
 
+static inline struct mlx5_vfio_mr *to_mvfio_mr(struct ibv_mr *ibmr)
+{
+	return container_of(ibmr, struct mlx5_vfio_mr, vmr.ibv_mr);
+}
+
 #endif
util/util.h +5 -0
index 2c05631..45f5065 100644
@@ -70,6 +70,11 @@ static inline unsigned long align(unsigned long val, unsigned long align)
 	return (val + align - 1) & ~(align - 1);
 }
 
+static inline unsigned long align_down(unsigned long val, unsigned long _align)
+{
+	return align(val - (_align - 1), _align);
+}
+
 static inline uint64_t roundup_pow_of_two(uint64_t n)
 {
 	return n == 1 ? 1 : 1ULL << ilog64(n - 1);
Signature · 1 line
1.8.3.1
#22 lore raw

PATCH 15/27 mlx5: Set DV context ops

In reply to Yishai Hadas

From: Mark Zhang <[email protected]>

Wrap DV APIs and call the matching function pointer if supported. This comes to handle both rdma and vfio flows.

---
 providers/mlx5/dr_rule.c   |  10 +-
 providers/mlx5/mlx5.c      | 344 ++++++++++++----
 providers/mlx5/mlx5.h      | 180 ++++++++-
 providers/mlx5/mlx5_vfio.c |  27 ++
 providers/mlx5/mlx5_vfio.h |   1 +
 providers/mlx5/verbs.c     | 966 +++++++++++++++++++++++++++++++++++++--------
 6 files changed, 1287 insertions(+), 241 deletions(-)
providers/mlx5/dr_rule.c +5 -5
index f763399..6291685 100644
@@ -1341,11 +1341,11 @@ dr_rule_create_rule_root(struct mlx5dv_dr_matcher *matcher,
 	if (ret)
 		goto free_attr_aux;
 
-	rule->flow = __mlx5dv_create_flow(matcher->dv_matcher,
-					  value,
-					  num_actions,
-					  attr,
-					  attr_aux);
+	rule->flow = _mlx5dv_create_flow(matcher->dv_matcher,
+					 value,
+					 num_actions,
+					 attr,
+					 attr_aux);
 	if (!rule->flow)
 		goto remove_action_members;
 
providers/mlx5/mlx5.c +276 -68

589 lines not on this page · raw message

providers/mlx5/mlx5.h +174 -6

208 lines not on this page · raw message

providers/mlx5/mlx5_vfio.c +27 -0
index e85a8cc..23c6eeb 100644
@@ -2459,6 +2459,25 @@ end:
 	return NULL;
 }
 
+static struct mlx5dv_devx_obj *
+vfio_devx_obj_create(struct ibv_context *context, const void *in,
+		     size_t inlen, void *out, size_t outlen)
+{
+	errno = EOPNOTSUPP;
+	return NULL;
+}
+
+static int vfio_devx_obj_query(struct mlx5dv_devx_obj *obj, const void *in,
+				size_t inlen, void *out, size_t outlen)
+{
+	return EOPNOTSUPP;
+}
+
+static struct mlx5_dv_context_ops mlx5_vfio_dv_ctx_ops = {
+	.devx_obj_create = vfio_devx_obj_create,
+	.devx_obj_query = vfio_devx_obj_query,
+};
+
 static void mlx5_vfio_uninit_context(struct mlx5_vfio_context *ctx)
 {
 	mlx5_close_debug_file(ctx->dbg_fp);
@@ -2524,6 +2543,7 @@ mlx5_vfio_alloc_context(struct ibv_device *ibdev,
 		goto func_teardown;
 
 	verbs_set_ops(&mctx->vctx, &mlx5_vfio_common_ops);
+	mctx->dv_ctx_ops = &mlx5_vfio_dv_ctx_ops;
 	return &mctx->vctx;
 
 func_teardown:
@@ -2729,3 +2749,10 @@ end:
 	free(list);
 	return NULL;
 }
+
+bool is_mlx5_vfio_dev(struct ibv_device *device)
+{
+	struct verbs_device *verbs_device = verbs_get_device(device);
+
+	return verbs_device->ops == &mlx5_vfio_dev_ops;
+}
providers/mlx5/mlx5_vfio.h +1 -0
index 5311c6f..79b8033 100644
@@ -290,6 +290,7 @@ struct mlx5_vfio_context {
 	struct mlx5_eq async_eq;
 	struct mlx5_vfio_eqs_uar eqs_uar;
 	pthread_mutex_t eq_lock;
+	struct mlx5_dv_context_ops *dv_ctx_ops;
 };
 
 static inline struct mlx5_vfio_device *to_mvfio_dev(struct ibv_device *ibdev)
providers/mlx5/verbs.c +804 -162

1,398 lines not on this page · raw message

Signature · 1 line
1.8.3.1
#23 lore raw

PATCH 16/27 mlx5: Support initial DEVX/DV APIs over vfio

In reply to Yishai Hadas

Support initial DEVX/DV APIs over vfio for UMEM/UAR/EQN usage.

---
 providers/mlx5/mlx5_ifc.h  |  70 ++++++++++++++
 providers/mlx5/mlx5_vfio.c | 228 ++++++++++++++++++++++++++++++++++++++++++++-
 providers/mlx5/mlx5_vfio.h |  10 ++
 3 files changed, 307 insertions(+), 1 deletion(-)
providers/mlx5/mlx5_ifc.h +70 -0
index 1cbe846..1bd7466 100644
@@ -88,6 +88,8 @@ enum {
 	MLX5_CMD_OP_CREATE_GENERAL_OBJECT = 0xa00,
 	MLX5_CMD_OP_MODIFY_GENERAL_OBJECT = 0xa01,
 	MLX5_CMD_OP_QUERY_GENERAL_OBJECT = 0xa02,
+	MLX5_CMD_OP_CREATE_UMEM = 0xa08,
+	MLX5_CMD_OP_DESTROY_UMEM = 0xa0a,
 	MLX5_CMD_OP_SYNC_STEERING = 0xb00,
 };
 
@@ -4656,4 +4658,72 @@ struct mlx5_ifc_dealloc_pd_in_bits {
 	u8         reserved_at_60[0x20];
 };
 
+struct mlx5_ifc_mtt_bits {
+	u8         ptag_63_32[0x20];
+
+	u8         ptag_31_8[0x18];
+	u8         reserved_at_38[0x6];
+	u8         wr_en[0x1];
+	u8         rd_en[0x1];
+};
+
+struct mlx5_ifc_umem_bits {
+	u8         reserved_at_0[0x80];
+
+	u8         reserved_at_80[0x1b];
+	u8         log_page_size[0x5];
+
+	u8         page_offset[0x20];
+
+	u8         num_of_mtt[0x40];
+
+	struct mlx5_ifc_mtt_bits  mtt[];
+};
+
+struct mlx5_ifc_create_umem_in_bits {
+	u8         opcode[0x10];
+	u8         uid[0x10];
+
+	u8         reserved_at_20[0x10];
+	u8         op_mod[0x10];
+
+	u8         reserved_at_40[0x40];
+
+	struct mlx5_ifc_umem_bits  umem;
+};
+
+struct mlx5_ifc_create_umem_out_bits {
+	u8         status[0x8];
+	u8         reserved_at_8[0x18];
+
+	u8         syndrome[0x20];
+
+	u8         reserved_at_40[0x8];
+	u8         umem_id[0x18];
+
+	u8         reserved_at_60[0x20];
+};
+
+struct mlx5_ifc_destroy_umem_in_bits {
+	u8        opcode[0x10];
+	u8        uid[0x10];
+
+	u8        reserved_at_20[0x10];
+	u8        op_mod[0x10];
+
+	u8        reserved_at_40[0x8];
+	u8        umem_id[0x18];
+
+	u8        reserved_at_60[0x20];
+};
+
+struct mlx5_ifc_destroy_umem_out_bits {
+	u8        status[0x8];
+	u8        reserved_at_8[0x18];
+
+	u8        syndrome[0x20];
+
+	u8        reserved_at_40[0x40];
+};
+
 #endif /* MLX5_IFC_H */
providers/mlx5/mlx5_vfio.c +227 -1

258 lines not on this page · raw message

providers/mlx5/mlx5_vfio.h +10 -0
index 79b8033..766c48c 100644
@@ -47,6 +47,16 @@ struct mlx5_vfio_mr {
 	uint64_t iova_reg_size;
 };
 
+struct mlx5_vfio_devx_umem {
+	struct mlx5dv_devx_umem dv_devx_umem;
+	struct ibv_context *context;
+	void *addr;
+	size_t size;
+	uint64_t iova;
+	uint64_t iova_size;
+	uint64_t iova_reg_size;
+};
+
 struct mlx5_vfio_device {
 	struct verbs_device vdev;
 	char *pci_name;
Signature · 1 line
1.8.3.1
#24 lore raw

PATCH 17/27 mlx5: Implement mlx5dv devx_obj APIs over vfio

In reply to Yishai Hadas

From: Mark Zhang <[email protected]>

Implement mlx5dv vfio APIs: devx_obj_create/query/modify/destroy, as well as the devx_general_cmd

---
 providers/mlx5/mlx5_ifc.h  | 565 ++++++++++++++++++++++++++++++++++++++++++++-
 providers/mlx5/mlx5_vfio.c | 413 ++++++++++++++++++++++++++++++++-
 providers/mlx5/mlx5_vfio.h |   8 +
 3 files changed, 975 insertions(+), 11 deletions(-)
providers/mlx5/mlx5_ifc.h +563 -2
index 1bd7466..175fe4a 100644
@@ -54,7 +54,10 @@ enum {
 	MLX5_CMD_OP_DESTROY_MKEY = 0x202,
 	MLX5_CMD_OP_CREATE_EQ = 0x301,
 	MLX5_CMD_OP_DESTROY_EQ = 0x302,
+	MLX5_CMD_OP_CREATE_CQ = 0x400,
+	MLX5_CMD_OP_DESTROY_CQ = 0x401,
 	MLX5_CMD_OP_CREATE_QP = 0x500,
+	MLX5_CMD_OP_DESTROY_QP = 0x501,
 	MLX5_CMD_OP_RST2INIT_QP = 0x502,
 	MLX5_CMD_OP_INIT2RTR_QP = 0x503,
 	MLX5_CMD_OP_RTR2RTS_QP = 0x504,
@@ -63,31 +66,71 @@ enum {
 	MLX5_CMD_OP_INIT2INIT_QP = 0x50e,
 	MLX5_CMD_OP_CREATE_PSV = 0x600,
 	MLX5_CMD_OP_DESTROY_PSV = 0x601,
+	MLX5_CMD_OP_CREATE_SRQ = 0x700,
+	MLX5_CMD_OP_DESTROY_SRQ = 0x701,
+	MLX5_CMD_OP_CREATE_XRC_SRQ = 0x705,
+	MLX5_CMD_OP_DESTROY_XRC_SRQ = 0x706,
+	MLX5_CMD_OP_CREATE_DCT = 0x710,
+	MLX5_CMD_OP_DESTROY_DCT = 0x711,
 	MLX5_CMD_OP_QUERY_DCT = 0x713,
+	MLX5_CMD_OP_CREATE_XRQ = 0x717,
+	MLX5_CMD_OP_DESTROY_XRQ = 0x718,
 	MLX5_CMD_OP_QUERY_ESW_VPORT_CONTEXT = 0x752,
 	MLX5_CMD_OP_QUERY_NIC_VPORT_CONTEXT = 0x754,
 	MLX5_CMD_OP_MODIFY_NIC_VPORT_CONTEXT = 0x755,
 	MLX5_CMD_OP_QUERY_ROCE_ADDRESS = 0x760,
+	MLX5_CMD_OP_ALLOC_Q_COUNTER = 0x771,
+	MLX5_CMD_OP_DEALLOC_Q_COUNTER = 0x772,
+	MLX5_CMD_OP_CREATE_SCHEDULING_ELEMENT = 0x782,
+	MLX5_CMD_OP_DESTROY_SCHEDULING_ELEMENT = 0x783,
 	MLX5_CMD_OP_ALLOC_PD = 0x800,
 	MLX5_CMD_OP_DEALLOC_PD = 0x801,
 	MLX5_CMD_OP_ALLOC_UAR = 0x802,
 	MLX5_CMD_OP_DEALLOC_UAR = 0x803,
 	MLX5_CMD_OP_ACCESS_REG = 0x805,
+	MLX5_CMD_OP_ATTACH_TO_MCG = 0x806,
+	MLX5_CMD_OP_DETACH_FROM_MCG = 0x807,
+	MLX5_CMD_OP_ALLOC_XRCD = 0x80e,
+	MLX5_CMD_OP_DEALLOC_XRCD = 0x80f,
+	MLX5_CMD_OP_ALLOC_TRANSPORT_DOMAIN = 0x816,
+	MLX5_CMD_OP_DEALLOC_TRANSPORT_DOMAIN = 0x817,
+	MLX5_CMD_OP_ADD_VXLAN_UDP_DPORT = 0x827,
+	MLX5_CMD_OP_DELETE_VXLAN_UDP_DPORT = 0x828,
+	MLX5_CMD_OP_SET_L2_TABLE_ENTRY = 0x829,
+	MLX5_CMD_OP_DELETE_L2_TABLE_ENTRY = 0x82b,
 	MLX5_CMD_OP_QUERY_LAG = 0x842,
 	MLX5_CMD_OP_CREATE_TIR = 0x900,
+	MLX5_CMD_OP_DESTROY_TIR = 0x902,
+	MLX5_CMD_OP_CREATE_SQ = 0x904,
 	MLX5_CMD_OP_MODIFY_SQ = 0x905,
+	MLX5_CMD_OP_DESTROY_SQ = 0x906,
+	MLX5_CMD_OP_CREATE_RQ = 0x908,
+	MLX5_CMD_OP_DESTROY_RQ = 0x90a,
+	MLX5_CMD_OP_CREATE_RMP = 0x90c,
+	MLX5_CMD_OP_DESTROY_RMP = 0x90e,
+	MLX5_CMD_OP_CREATE_TIS = 0x912,
 	MLX5_CMD_OP_MODIFY_TIS = 0x913,
+	MLX5_CMD_OP_DESTROY_TIS = 0x914,
 	MLX5_CMD_OP_QUERY_TIS = 0x915,
+	MLX5_CMD_OP_CREATE_RQT = 0x916,
+	MLX5_CMD_OP_DESTROY_RQT = 0x918,
 	MLX5_CMD_OP_CREATE_FLOW_TABLE = 0x930,
+	MLX5_CMD_OP_DESTROY_FLOW_TABLE = 0x931,
 	MLX5_CMD_OP_QUERY_FLOW_TABLE = 0x932,
 	MLX5_CMD_OP_CREATE_FLOW_GROUP = 0x933,
+	MLX5_CMD_OP_DESTROY_FLOW_GROUP = 0x934,
 	MLX5_CMD_OP_SET_FLOW_TABLE_ENTRY = 0x936,
+	MLX5_CMD_OP_DELETE_FLOW_TABLE_ENTRY = 0x938,
 	MLX5_CMD_OP_CREATE_FLOW_COUNTER = 0x939,
+	MLX5_CMD_OP_DEALLOC_FLOW_COUNTER = 0x93a,
 	MLX5_CMD_OP_ALLOC_PACKET_REFORMAT_CONTEXT = 0x93d,
 	MLX5_CMD_OP_DEALLOC_PACKET_REFORMAT_CONTEXT = 0x93e,
+	MLX5_CMD_OP_ALLOC_MODIFY_HEADER_CONTEXT = 0x940,
+	MLX5_CMD_OP_DEALLOC_MODIFY_HEADER_CONTEXT = 0x941,
 	MLX5_CMD_OP_CREATE_GENERAL_OBJECT = 0xa00,
 	MLX5_CMD_OP_MODIFY_GENERAL_OBJECT = 0xa01,
 	MLX5_CMD_OP_QUERY_GENERAL_OBJECT = 0xa02,
+	MLX5_CMD_OP_DESTROY_GENERAL_OBJECT = 0xa03,
 	MLX5_CMD_OP_CREATE_UMEM = 0xa08,
 	MLX5_CMD_OP_DESTROY_UMEM = 0xa0a,
 	MLX5_CMD_OP_SYNC_STEERING = 0xb00,
@@ -236,6 +279,27 @@ struct mlx5_ifc_create_flow_table_out_bits {
 	u8         icm_address_31_0[0x20];
 };
 
+struct mlx5_ifc_destroy_flow_table_in_bits {
+	u8         opcode[0x10];
+	u8         reserved_at_10[0x10];
+
+	u8         reserved_at_20[0x20];
+
+	u8         other_vport[0x1];
+	u8         reserved_at_41[0xf];
+	u8         vport_number[0x10];
+
+	u8         reserved_at_60[0x20];
+
+	u8         table_type[0x8];
+	u8         reserved_at_88[0x18];
+
+	u8         reserved_at_a0[0x8];
+	u8         table_id[0x18];
+
+	u8         reserved_at_c0[0x140];
+};
+
 struct mlx5_ifc_query_flow_table_in_bits {
 	u8         opcode[0x10];
 	u8         reserved_at_10[0x10];
@@ -2991,6 +3055,17 @@ struct mlx5_ifc_alloc_flow_counter_out_bits {
 	u8	reserved_at_60[0x20];
 };
 
+struct mlx5_ifc_dealloc_flow_counter_in_bits {
+	u8         opcode[0x10];
+	u8         reserved_at_10[0x10];
+
+	u8         reserved_at_20[0x20];
+
+	u8         flow_counter_id[0x20];
+
+	u8         reserved_at_60[0x20];
+};
+
 enum {
 	MLX5_OBJ_TYPE_FLOW_METER = 0x000a,
 	MLX5_OBJ_TYPE_MATCH_DEFINER = 0x0018,
@@ -3422,6 +3497,18 @@ struct mlx5_ifc_create_tir_out_bits {
 	u8         icm_address_31_0[0x20];
 };
 
+struct mlx5_ifc_destroy_tir_in_bits {
+	u8         opcode[0x10];
+	u8         uid[0x10];
+
+	u8         reserved_at_20[0x20];
+
+	u8         reserved_at_40[0x8];
+	u8         tirn[0x18];
+
+	u8         reserved_at_60[0x20];
+};
+
 struct mlx5_ifc_create_qp_out_bits {
 	u8         status[0x8];
 	u8         reserved_at_8[0x18];
@@ -3459,6 +3546,18 @@ struct mlx5_ifc_create_qp_in_bits {
 	u8         pas[0][0x40];
 };
 
+struct mlx5_ifc_destroy_qp_in_bits {
+	u8         opcode[0x10];
+	u8         uid[0x10];
+
+	u8         reserved_at_20[0x20];
+
+	u8         reserved_at_40[0x8];
+	u8         qpn[0x18];
+
+	u8         reserved_at_60[0x20];
+};
+
 enum mlx5_qpc_opt_mask_32 {
 	MLX5_QPC_OPT_MASK_32_QOS_QUEUE_GROUP_ID = 1 << 1,
 	MLX5_QPC_OPT_MASK_32_UDP_SPORT = 1 << 2,
@@ -3898,7 +3997,13 @@ struct mlx5_ifc_create_flow_group_in_bits {
 	u8         opcode[0x10];
 	u8         reserved_at_10[0x10];
 
-	u8         reserved_at_20[0x60];
+	u8         reserved_at_20[0x20];
+
+	u8         other_vport[0x1];
+	u8         reserved_at_41[0xf];
+	u8         vport_number[0x10];
+
+	u8         reserved_at_60[0x20];
 
 	u8         table_type[0x8];
 	u8         reserved_at_88[0x18];
@@ -3921,6 +4026,29 @@ struct mlx5_ifc_create_flow_group_out_bits {
 	u8         reserved_at_60[0x20];
 };
 
+struct mlx5_ifc_destroy_flow_group_in_bits {
+	u8         opcode[0x10];
+	u8         reserved_at_10[0x10];
+
+	u8         reserved_at_20[0x20];
+
+	u8         other_vport[0x1];
+	u8         reserved_at_41[0xf];
+	u8         vport_number[0x10];
+
+	u8         reserved_at_60[0x20];
+
+	u8         table_type[0x8];
+	u8         reserved_at_88[0x18];
+
+	u8         reserved_at_a0[0x8];
+	u8         table_id[0x18];
+
+	u8         group_id[0x20];
+
+	u8         reserved_at_e0[0x120];
+};
+
 struct mlx5_ifc_dest_format_bits {
 	u8         destination_type[0x8];
 	u8         destination_id[0x18];
@@ -3977,7 +4105,14 @@ struct mlx5_ifc_set_fte_in_bits {
 	u8         opcode[0x10];
 	u8         reserved_at_10[0x10];
 
-	u8         reserved_at_20[0x60];
+	u8         reserved_at_20[0x10];
+	u8         op_mod[0x10];
+
+	u8         other_vport[0x1];
+	u8         reserved_at_41[0xf];
+	u8         vport_number[0x10];
+
+	u8         reserved_at_60[0x20];
 
 	u8         table_type[0x8];
 	u8         reserved_at_88[0x18];
@@ -4186,6 +4321,18 @@ struct mlx5_ifc_create_psv_in_bits {
 	u8         reserved_at_60[0x20];
 };
 
+struct mlx5_ifc_destroy_psv_in_bits {
+	u8         opcode[0x10];
+	u8         reserved_at_10[0x10];
+
+	u8         reserved_at_20[0x20];
+
+	u8         reserved_at_40[0x8];
+	u8         psvn[0x18];
+
+	u8         reserved_at_60[0x20];
+};
+
 struct mlx5_ifc_mbox_out_bits {
 	u8	status[0x8];
 	u8	reserved_at_8[0x18];
@@ -4726,4 +4873,418 @@ struct mlx5_ifc_destroy_umem_out_bits {
 	u8        reserved_at_40[0x40];
 };
 
+struct mlx5_ifc_delete_fte_in_bits {
+	u8         opcode[0x10];
+	u8         reserved_at_10[0x10];
+
+	u8         reserved_at_20[0x20];
+
+	u8         other_vport[0x1];
+	u8         reserved_at_41[0xf];
+	u8         vport_number[0x10];
+
+	u8         reserved_at_60[0x20];
+
+	u8         table_type[0x8];
+	u8         reserved_at_88[0x18];
+
+	u8         reserved_at_a0[0x8];
+	u8         table_id[0x18];
+
+	u8         reserved_at_c0[0x40];
+
+	u8         flow_index[0x20];
+
+	u8         reserved_at_120[0xe0];
+};
+
+struct mlx5_ifc_create_cq_out_bits {
+	u8         reserved_at_0[0x40];
+
+	u8         reserved_at_40[0x8];
+	u8         cqn[0x18];
+
+	u8         reserved_at_60[0x20];
+};
+
+struct mlx5_ifc_destroy_cq_in_bits {
+	u8         opcode[0x10];
+	u8         uid[0x10];
+
+	u8         reserved_at_20[0x20];
+
+	u8         reserved_at_40[0x8];
+	u8         cqn[0x18];
+
+	u8         reserved_at_60[0x20];
+};
+
+struct mlx5_ifc_alloc_transport_domain_out_bits {
+	u8         reserved_at_0[0x40];
+
+	u8         reserved_at_40[0x8];
+	u8         transport_domain[0x18];
+
+	u8         reserved_at_60[0x20];
+};
+
+struct mlx5_ifc_dealloc_transport_domain_in_bits {
+	u8         opcode[0x10];
+	u8         uid[0x10];
+
+	u8         reserved_at_20[0x20];
+
+	u8         reserved_at_40[0x8];
+	u8         transport_domain[0x18];
+
+	u8         reserved_at_60[0x20];
+};
+
+struct mlx5_ifc_create_rmp_out_bits {
+	u8         reserved_at_0[0x40];
+
+	u8         reserved_at_40[0x8];
+	u8         rmpn[0x18];
+
+	u8         reserved_at_60[0x20];
+};
+
+struct mlx5_ifc_destroy_rmp_in_bits {
+	u8         opcode[0x10];
+	u8         uid[0x10];
+
+	u8         reserved_at_20[0x20];
+
+	u8         reserved_at_40[0x8];
+	u8         rmpn[0x18];
+
+	u8         reserved_at_60[0x20];
+};
+
+struct mlx5_ifc_create_sq_out_bits {
+	u8         reserved_at_0[0x40];
+
+	u8         reserved_at_40[0x8];
+	u8         sqn[0x18];
+
+	u8         reserved_at_60[0x20];
+};
+
+struct mlx5_ifc_destroy_sq_in_bits {
+	u8         opcode[0x10];
+	u8         uid[0x10];
+
+	u8         reserved_at_20[0x20];
+
+	u8         reserved_at_40[0x8];
+	u8         sqn[0x18];
+
+	u8         reserved_at_60[0x20];
+};
+
+struct mlx5_ifc_create_rq_out_bits {
+	u8         reserved_at_0[0x40];
+
+	u8         reserved_at_40[0x8];
+	u8         rqn[0x18];
+
+	u8         reserved_at_60[0x20];
+};
+
+struct mlx5_ifc_destroy_rq_in_bits {
+	u8         opcode[0x10];
+	u8         uid[0x10];
+
+	u8         reserved_at_20[0x20];
+
+	u8         reserved_at_40[0x8];
+	u8         rqn[0x18];
+
+	u8         reserved_at_60[0x20];
+};
+
+struct mlx5_ifc_create_rqt_out_bits {
+	u8         reserved_at_0[0x40];
+
+	u8         reserved_at_40[0x8];
+	u8         rqtn[0x18];
+
+	u8         reserved_at_60[0x20];
+};
+
+struct mlx5_ifc_destroy_rqt_in_bits {
+	u8         opcode[0x10];
+	u8         uid[0x10];
+
+	u8         reserved_at_20[0x20];
+
+	u8         reserved_at_40[0x8];
+	u8         rqtn[0x18];
+
+	u8         reserved_at_60[0x20];
+};
+
+struct mlx5_ifc_create_tis_out_bits {
+	u8         reserved_at_0[0x40];
+
+	u8         reserved_at_40[0x8];
+	u8         tisn[0x18];
+
+	u8         reserved_at_60[0x20];
+};
+
+struct mlx5_ifc_destroy_tis_in_bits {
+	u8         opcode[0x10];
+	u8         uid[0x10];
+
+	u8         reserved_at_20[0x20];
+
+	u8         reserved_at_40[0x8];
+	u8         tisn[0x18];
+
+	u8         reserved_at_60[0x20];
+};
+
+struct mlx5_ifc_alloc_q_counter_out_bits {
+	u8         reserved_at_0[0x40];
+
+	u8         reserved_at_40[0x18];
+	u8         counter_set_id[0x8];
+
+	u8         reserved_at_60[0x20];
+};
+
+struct mlx5_ifc_dealloc_q_counter_in_bits {
+	u8         opcode[0x10];
+	u8         reserved_at_10[0x10];
+
+	u8         reserved_at_20[0x20];
+
+	u8         reserved_at_40[0x18];
+	u8         counter_set_id[0x8];
+
+	u8         reserved_at_60[0x20];
+};
+
+struct mlx5_ifc_alloc_modify_header_context_out_bits {
+	u8         reserved_at_0[0x40];
+
+	u8         modify_header_id[0x20];
+
+	u8         reserved_at_60[0x20];
+};
+
+struct mlx5_ifc_dealloc_modify_header_context_in_bits {
+	u8         opcode[0x10];
+	u8         reserved_at_10[0x10];
+
+	u8         reserved_at_20[0x20];
+
+	u8         modify_header_id[0x20];
+
+	u8         reserved_at_60[0x20];
+};
+
+struct mlx5_ifc_create_scheduling_element_out_bits {
+	u8         reserved_at_0[0x80];
+
+	u8         scheduling_element_id[0x20];
+
+	u8         reserved_at_a0[0x160];
+};
+
+struct mlx5_ifc_create_scheduling_element_in_bits {
+	u8         opcode[0x10];
+	u8         reserved_at_10[0x10];
+
+	u8         reserved_at_20[0x20];
+
+	u8         scheduling_hierarchy[0x8];
+	u8         reserved_at_48[0x18];
+
+	u8         reserved_at_60[0x3a0];
+};
+
+struct mlx5_ifc_destroy_scheduling_element_in_bits {
+	u8         opcode[0x10];
+	u8         reserved_at_10[0x10];
+
+	u8         reserved_at_20[0x20];
+
+	u8         scheduling_hierarchy[0x8];
+	u8         reserved_at_48[0x18];
+
+	u8         scheduling_element_id[0x20];
+
+	u8         reserved_at_80[0x180];
+};
+
+struct mlx5_ifc_add_vxlan_udp_dport_in_bits {
+	u8         reserved_at_0[0x60];
+
+	u8         reserved_at_60[0x10];
+	u8         vxlan_udp_port[0x10];
+};
+
+struct mlx5_ifc_delete_vxlan_udp_dport_in_bits {
+	u8         opcode[0x10];
+	u8         reserved_at_10[0x10];
+
+	u8         reserved_at_20[0x40];
+
+	u8         reserved_at_60[0x10];
+	u8         vxlan_udp_port[0x10];
+};
+
+struct mlx5_ifc_set_l2_table_entry_in_bits {
+	u8         reserved_at_0[0xa0];
+
+	u8         reserved_at_a0[0x8];
+	u8         table_index[0x18];
+
+	u8         reserved_at_c0[0x140];
+
+};
+
+struct mlx5_ifc_delete_l2_table_entry_in_bits {
+	u8         opcode[0x10];
+	u8         reserved_at_10[0x10];
+
+	u8         reserved_at_20[0x80];
+
+	u8         reserved_at_a0[0x8];
+	u8         table_index[0x18];
+
+	u8         reserved_at_c0[0x140];
+};
+
+struct mlx5_ifc_create_srq_out_bits {
+	u8         reserved_at_0[0x40];
+
+	u8         reserved_at_40[0x8];
+	u8         srqn[0x18];
+
+	u8         reserved_at_60[0x20];
+};
+
+struct mlx5_ifc_destroy_srq_in_bits {
+	u8         opcode[0x10];
+	u8         uid[0x10];
+
+	u8         reserved_at_20[0x20];
+
+	u8         reserved_at_40[0x8];
+	u8         srqn[0x18];
+
+	u8         reserved_at_60[0x20];
+};
+
+struct mlx5_ifc_create_xrc_srq_out_bits {
+	u8         reserved_at_0[0x40];
+
+	u8         reserved_at_40[0x8];
+	u8         xrc_srqn[0x18];
+
+	u8         reserved_at_60[0x20];
+};
+
+struct mlx5_ifc_destroy_xrc_srq_in_bits {
+	u8         opcode[0x10];
+	u8         uid[0x10];
+
+	u8         reserved_at_20[0x20];
+
+	u8         reserved_at_40[0x8];
+	u8         xrc_srqn[0x18];
+
+	u8         reserved_at_60[0x20];
+};
+
+struct mlx5_ifc_create_dct_out_bits {
+	u8         reserved_at_0[0x40];
+
+	u8         reserved_at_40[0x8];
+	u8         dctn[0x18];
+
+	u8         ece[0x20];
+};
+
+struct mlx5_ifc_destroy_dct_in_bits {
+	u8         opcode[0x10];
+	u8         uid[0x10];
+
+	u8         reserved_at_20[0x20];
+
+	u8         reserved_at_40[0x8];
+	u8         dctn[0x18];
+
+	u8         reserved_at_60[0x20];
+};
+
+struct mlx5_ifc_create_xrq_out_bits {
+	u8         reserved_at_0[0x40];
+
+	u8         reserved_at_40[0x8];
+	u8         xrqn[0x18];
+
+	u8         reserved_at_60[0x20];
+};
+
+struct mlx5_ifc_destroy_xrq_in_bits {
+	u8         opcode[0x10];
+	u8         uid[0x10];
+
+	u8         reserved_at_20[0x20];
+
+	u8         reserved_at_40[0x8];
+	u8         xrqn[0x18];
+
+	u8         reserved_at_60[0x20];
+};
+
+struct mlx5_ifc_attach_to_mcg_in_bits {
+	u8         reserved_at_0[0x40];
+
+	u8         reserved_at_40[0x8];
+	u8         qpn[0x18];
+
+	u8         reserved_at_60[0x20];
+
+	u8         multicast_gid[16][0x8];
+};
+
+struct mlx5_ifc_detach_from_mcg_in_bits {
+	u8         opcode[0x10];
+	u8         uid[0x10];
+
+	u8         reserved_at_20[0x20];
+
+	u8         reserved_at_40[0x8];
+	u8         qpn[0x18];
+
+	u8         reserved_at_60[0x20];
+
+	u8         multicast_gid[16][0x8];
+};
+
+struct mlx5_ifc_alloc_xrcd_out_bits {
+	u8         reserved_at_0[0x40];
+
+	u8         reserved_at_40[0x8];
+	u8         xrcd[0x18];
+
+	u8         reserved_at_60[0x20];
+};
+
+struct mlx5_ifc_dealloc_xrcd_in_bits {
+	u8         opcode[0x10];
+	u8         uid[0x10];
+
+	u8         reserved_at_20[0x20];
+
+	u8         reserved_at_40[0x8];
+	u8         xrcd[0x18];
+
+	u8         reserved_at_60[0x20];
+};
 #endif /* MLX5_IFC_H */
providers/mlx5/mlx5_vfio.c +404 -9

433 lines not on this page · raw message

providers/mlx5/mlx5_vfio.h +8 -0
index 766c48c..2165a22 100644
@@ -9,6 +9,7 @@
 #include <stddef.h>
 #include <stdio.h>
 #include "mlx5.h"
+#include "mlx5_ifc.h"
 
 #include <infiniband/driver.h>
 #include <util/interval_set.h>
@@ -303,6 +304,13 @@ struct mlx5_vfio_context {
 	struct mlx5_dv_context_ops *dv_ctx_ops;
 };
 
+#define MLX5_MAX_DESTROY_INBOX_SIZE_DW	DEVX_ST_SZ_DW(delete_fte_in)
+struct mlx5_devx_obj {
+	struct mlx5dv_devx_obj dv_obj;
+	uint32_t dinbox[MLX5_MAX_DESTROY_INBOX_SIZE_DW];
+	uint32_t dinlen;
+};
+
 static inline struct mlx5_vfio_device *to_mvfio_dev(struct ibv_device *ibdev)
 {
 	return container_of(ibdev, struct mlx5_vfio_device, vdev.device);
Signature · 1 line
1.8.3.1
#25 lore raw

PATCH 18/27 pyverbs: Support DevX UMEM registration

In reply to Yishai Hadas

From: Edward Srouji <[email protected]>

Add support to register a DevX UMEM and UMEM extended, a user memory to be used by the DevX interface for DMA.

---
 pyverbs/providers/mlx5/libmlx5.pxd | 21 +++++++--
 pyverbs/providers/mlx5/mlx5dv.pxd  |  8 ++++
 pyverbs/providers/mlx5/mlx5dv.pyx  | 93 ++++++++++++++++++++++++++++++++++++--
 3 files changed, 116 insertions(+), 6 deletions(-)
pyverbs/providers/mlx5/libmlx5.pxd +18 -3
index 8dd0141..66b8705 100644
@@ -212,6 +212,17 @@ cdef extern from 'infiniband/mlx5dv.h':
         uint8_t     fm_ce_se
         uint32_t    imm
 
+    cdef struct mlx5dv_devx_umem:
+        uint32_t umem_id;
+
+    cdef struct mlx5dv_devx_umem_in:
+        void        *addr
+        size_t      size
+        uint32_t    access
+        uint64_t    pgsz_bitmap
+        uint64_t    comp_mask
+
+
     void mlx5dv_set_ctrl_seg(mlx5_wqe_ctrl_seg *seg, uint16_t pi, uint8_t opcode,
                              uint8_t opmod, uint32_t qp_num, uint8_t fm_ce_se,
                              uint8_t ds, uint8_t signature, uint32_t imm)
@@ -298,11 +309,15 @@ cdef extern from 'infiniband/mlx5dv.h':
     int mlx5dv_map_ah_to_qp(v.ibv_ah *ah, uint32_t qp_num)
 
     # DevX APIs
-    mlx5dv_devx_uar *mlx5dv_devx_alloc_uar(v.ibv_context *context,
-                                           uint32_t flags)
+    mlx5dv_devx_uar *mlx5dv_devx_alloc_uar(v.ibv_context *context, uint32_t flags)
     void mlx5dv_devx_free_uar(mlx5dv_devx_uar *devx_uar)
     int mlx5dv_devx_general_cmd(v.ibv_context *context, const void *in_,
-                                size_t inlen, void *out, size_t outlen);
+                                size_t inlen, void *out, size_t outlen)
+    mlx5dv_devx_umem *mlx5dv_devx_umem_reg(v.ibv_context *ctx, void *addr,
+                                           size_t size, unsigned long access)
+    mlx5dv_devx_umem *mlx5dv_devx_umem_reg_ex(v.ibv_context *ctx,
+                                              mlx5dv_devx_umem_in *umem_in)
+    int mlx5dv_devx_umem_dereg(mlx5dv_devx_umem *umem)
 
     # Mkey setters
     void mlx5dv_wr_mkey_configure(mlx5dv_qp_ex *mqp, mlx5dv_mkey *mkey,
pyverbs/providers/mlx5/mlx5dv.pxd +8 -0
index 18b208e..154a117 100644
@@ -11,6 +11,8 @@ from pyverbs.cq cimport CQEX
 
 
 cdef class Mlx5Context(Context):
+    cdef object devx_umems
+    cdef add_ref(self, obj)
     cpdef close(self)
 
 cdef class Mlx5DVContextAttr(PyverbsObject):
@@ -69,3 +71,9 @@ cdef class Wqe(PyverbsCM):
     cdef void *addr
     cdef int is_user_addr
     cdef object segments
+
+cdef class Mlx5UMEM(PyverbsCM):
+    cdef dv.mlx5dv_devx_umem *umem
+    cdef Context context
+    cdef void *addr
+    cdef object is_user_addr
pyverbs/providers/mlx5/mlx5dv.pyx +90 -3

127 lines not on this page · raw message

Signature · 1 line
1.8.3.1
#26 lore raw

PATCH 19/27 pyverbs/mlx5: Support EQN querying

In reply to Yishai Hadas

From: Edward Srouji <[email protected]>

Add the ability to retrieve a device's EQ number which relates to a given input vector.
The query is done over DevX.

---
 pyverbs/providers/mlx5/libmlx5.pxd |  1 +
 pyverbs/providers/mlx5/mlx5dv.pyx  | 12 ++++++++++++
 2 files changed, 13 insertions(+)
pyverbs/providers/mlx5/libmlx5.pxd +1 -0
index 66b8705..ba2c6ec 100644
@@ -318,6 +318,7 @@ cdef extern from 'infiniband/mlx5dv.h':
     mlx5dv_devx_umem *mlx5dv_devx_umem_reg_ex(v.ibv_context *ctx,
                                               mlx5dv_devx_umem_in *umem_in)
     int mlx5dv_devx_umem_dereg(mlx5dv_devx_umem *umem)
+    int mlx5dv_devx_query_eqn(v.ibv_context *context, uint32_t vector, uint32_t *eqn)
 
     # Mkey setters
     void mlx5dv_wr_mkey_configure(mlx5dv_qp_ex *mqp, mlx5dv_mkey *mkey,
pyverbs/providers/mlx5/mlx5dv.pyx +12 -0
index d16aed1..2c47cb6 100644
@@ -262,6 +262,18 @@ cdef class Mlx5Context(Context):
         free(clock_info)
         return ns_time
 
+    def devx_query_eqn(self, vector):
+        """
+        Query EQN for a given vector id.
+        :param vector: Completion vector number
+        :return: The device EQ number which relates to the given input vector
+        """
+        cdef uint32_t eqn
+        rc = dv.mlx5dv_devx_query_eqn(self.context, vector, &eqn)
+        if rc:
+            raise PyverbsRDMAError('Failed to query EQN', rc)
+        return eqn
+
     cdef add_ref(self, obj):
         try:
             Context.add_ref(self, obj)
Signature · 1 line
1.8.3.1
#27 lore raw

PATCH 20/27 pyverbs/mlx5: Support more DevX objects

In reply to Yishai Hadas

From: Edward Srouji <[email protected]>

A DevX object represents some underlay firmware object, the input command to create it is some raw data given by the user application which should match the device specification.

A new class Mlx5DevxObj was added that would present any DevX object that could be creating using the create_obj DevX API.
The command's output is stored in the object's instance, and the object can be modified/queried and destroyed.

---
 Documentation/pyverbs.md           |  34 ++++++++++++
 pyverbs/providers/mlx5/libmlx5.pxd |   7 +++
 pyverbs/providers/mlx5/mlx5dv.pxd  |   6 +++
 pyverbs/providers/mlx5/mlx5dv.pyx  | 103 ++++++++++++++++++++++++++++++++++++-
 4 files changed, 149 insertions(+), 1 deletion(-)
Documentation/pyverbs.md +34 -0
index c7fa761..bcbde05 100644
@@ -745,3 +745,37 @@ Below is the output when printing the spec.
     Dst mac         : de:de:de:00:de:de    mask: ff:ff:ff:ff:ff:ff
     Ether type      : 8451                 mask: 65535
     Vlan tag        : 8961                 mask: 65535
+
+
+##### MLX5 DevX Objects
+A DevX object represents some underlay firmware object, the input command to
+create it is some raw data given by the user application which should match the
+device specification.
+Upon successful creation, the output buffer includes the raw data from the device
+according to its specification and is stored in the Mlx5DevxObj instance. This
+data can be used as part of related firmware commands to this object.
+In addition to creation, the user can query/modify and destroy the object.
+
+Although weakrefs and DevX objects closure are added and handled by
+Pyverbs, the users must manually close these objects when finished, and
+should not let them be handled by the GC, or by closing the Mlx5Context directly,
+since there's no guarantee that the DevX objects are closed in the correct order,
+because Mlx5DevxObj is a general class that can be any of the device's available
+objects.
+But Pyverbs does guarantee to close DevX UARs and UMEMs in order, and after
+closing the other DevX objects.
+
+The following code snippet shows how to allocate and destroy a PD object over DevX.
+```python
+from pyverbs.providers.mlx5.mlx5dv import Mlx5Context, Mlx5DVContextAttr, Mlx5DevxObj
+import pyverbs.providers.mlx5.mlx5_enums as dve
+import struct
+
+attr = Mlx5DVContextAttr(dve.MLX5DV_CONTEXT_FLAGS_DEVX)
+ctx = Mlx5Context(attr, 'rocep8s0f0')
+MLX5_CMD_OP_ALLOC_PD = 0x800
+MLX5_CMD_OP_ALLOC_PD_OUTLEN = 0x10
+cmd_in = struct.pack('!H14s', MLX5_CMD_OP_ALLOC_PD, bytes(0))
+pd = Mlx5DevxObj(ctx, cmd_in, MLX5_CMD_OP_ALLOC_PD_OUTLEN)
+pd.close()
+```
pyverbs/providers/mlx5/libmlx5.pxd +7 -0
index ba2c6ec..34691a9 100644
@@ -319,6 +319,13 @@ cdef extern from 'infiniband/mlx5dv.h':
                                               mlx5dv_devx_umem_in *umem_in)
     int mlx5dv_devx_umem_dereg(mlx5dv_devx_umem *umem)
     int mlx5dv_devx_query_eqn(v.ibv_context *context, uint32_t vector, uint32_t *eqn)
+    mlx5dv_devx_obj *mlx5dv_devx_obj_create(v.ibv_context *context, const void *_in,
+                                            size_t inlen, void *out, size_t outlen)
+    int mlx5dv_devx_obj_query(mlx5dv_devx_obj *obj, const void *in_,
+                              size_t inlen, void *out, size_t outlen)
+    int mlx5dv_devx_obj_modify(mlx5dv_devx_obj *obj, const void *in_,
+                               size_t inlen, void *out, size_t outlen)
+    int mlx5dv_devx_obj_destroy(mlx5dv_devx_obj *obj)
 
     # Mkey setters
     void mlx5dv_wr_mkey_configure(mlx5dv_qp_ex *mqp, mlx5dv_mkey *mkey,
pyverbs/providers/mlx5/mlx5dv.pxd +6 -0
index 154a117..2b758fe 100644
@@ -12,6 +12,7 @@ from pyverbs.cq cimport CQEX
 
 cdef class Mlx5Context(Context):
     cdef object devx_umems
+    cdef object devx_objs
     cdef add_ref(self, obj)
     cpdef close(self)
 
@@ -77,3 +78,8 @@ cdef class Mlx5UMEM(PyverbsCM):
     cdef Context context
     cdef void *addr
     cdef object is_user_addr
+
+cdef class Mlx5DevxObj(PyverbsCM):
+    cdef dv.mlx5dv_devx_obj *obj
+    cdef Context context
+    cdef object out_view
pyverbs/providers/mlx5/mlx5dv.pyx +102 -1

127 lines not on this page · raw message

Signature · 1 line
1.8.3.1
#28 lore raw

PATCH 21/27 pyverbs: Add auxiliary memory functions

In reply to Yishai Hadas

From: Edward Srouji <[email protected]>

Add some auxiliary functions to mem_alloc module to expose to Python users the ability to read/write from/to memory, with chuncks of 4/8 bytes.
In addition, expose udma memory barriers and the ability to write on MMIO.

---
 pyverbs/CMakeLists.txt |  7 +++++++
 pyverbs/dma_util.pyx   | 25 +++++++++++++++++++++++++
 pyverbs/mem_alloc.pyx  | 46 +++++++++++++++++++++++++++++++++++++++++++++-
 3 files changed, 77 insertions(+), 1 deletion(-)
 create mode 100644 pyverbs/dma_util.pyx
pyverbs/CMakeLists.txt +7 -0
index c532b4c..f403719 100644
@@ -12,6 +12,12 @@ else()
   set(DMABUF_ALLOC dmabuf_alloc_stub.c)
 endif()
 
+if (HAVE_COHERENT_DMA)
+  set(DMA_UTIL dma_util.pyx)
+else()
+  set(DMA_UTIL "")
+endif()
+
 rdma_cython_module(pyverbs ""
   addr.pyx
   base.pyx
@@ -19,6 +25,7 @@ rdma_cython_module(pyverbs ""
   cmid.pyx
   cq.pyx
   device.pyx
+  ${DMA_UTIL}
   dmabuf.pyx
   ${DMABUF_ALLOC}
   enums.pyx
pyverbs/dma_util.pyx new file +25 -0
new file mode 100644
index 0000000..36d5f9b
@@ -0,0 +1,25 @@
+# SPDX-License-Identifier: (GPL-2.0 OR Linux-OpenIB)
+# Copyright (c) 2021 Nvidia, Inc. All rights reserved. See COPYING file
+
+#cython: language_level=3
+
+from libc.stdint cimport uintptr_t, uint64_t
+
+cdef extern from 'util/udma_barrier.h':
+    cdef void udma_to_device_barrier()
+    cdef void udma_from_device_barrier()
+
+cdef extern from 'util/mmio.h':
+   cdef void mmio_write64_be(void *addr, uint64_t val)
+
+
+def udma_to_dev_barrier():
+    udma_to_device_barrier()
+
+
+def udma_from_dev_barrier():
+    udma_from_device_barrier()
+
+
+def mmio_write64_as_be(addr, val):
+    mmio_write64_be(<void*><uintptr_t> addr, val)
pyverbs/mem_alloc.pyx +45 -1

64 lines not on this page · raw message

Signature · 1 line
1.8.3.1
#29 lore raw

PATCH 22/27 pyverbs/mlx5: Add support to extract mlx5dv objects

In reply to Yishai Hadas

From: Edward Srouji <[email protected]>

Support extraction of mlx5dv objects from ibv objects. This allows the users to access the object numbers and much more attributes needed for some usage such as DevX.
Currently there's a support for the following objects: PD, CQ, QP and SRQ.

---
 pyverbs/pd.pyx                            |   4 +
 pyverbs/providers/mlx5/CMakeLists.txt     |   1 +
 pyverbs/providers/mlx5/libmlx5.pxd        |  52 ++++++++
 pyverbs/providers/mlx5/mlx5dv_enums.pxd   |  17 +++
 pyverbs/providers/mlx5/mlx5dv_objects.pxd |  28 ++++
 pyverbs/providers/mlx5/mlx5dv_objects.pyx | 214 ++++++++++++++++++++++++++++++
 6 files changed, 316 insertions(+)
 create mode 100644 pyverbs/providers/mlx5/mlx5dv_objects.pxd
 create mode 100644 pyverbs/providers/mlx5/mlx5dv_objects.pyx
pyverbs/pd.pyx +4 -0
index 2c0c424..e8d3e1d 100644
@@ -139,6 +139,10 @@ cdef class PD(PyverbsCM):
     def handle(self):
         return self.pd.handle
 
+    @property
+    def pd(self):
+        return <object>self.pd
+
 
 cdef void *pd_alloc(v.ibv_pd *pd, void *pd_context, size_t size,
                   size_t alignment, v.uint64_t resource_type):
pyverbs/providers/mlx5/CMakeLists.txt +1 -0
index cdb6be2..4763c61 100644
@@ -11,5 +11,6 @@ rdma_cython_module(pyverbs/providers/mlx5 mlx5
   mlx5_enums.pyx
   mlx5dv_flow.pyx
   mlx5dv_mkey.pyx
+  mlx5dv_objects.pyx
   mlx5dv_sched.pyx
 )
pyverbs/providers/mlx5/libmlx5.pxd +52 -0

70 lines not on this page · raw message

pyverbs/providers/mlx5/mlx5dv_enums.pxd +17 -0
index d0a29bc..9f8d1a1 100644
@@ -176,6 +176,23 @@ cdef extern from 'infiniband/mlx5dv.h':
         MLX5DV_DR_DOMAIN_TYPE_NIC_TX
         MLX5DV_DR_DOMAIN_TYPE_FDB
 
+    cpdef enum mlx5dv_qp_comp_mask:
+        MLX5DV_QP_MASK_UAR_MMAP_OFFSET
+        MLX5DV_QP_MASK_RAW_QP_HANDLES
+        MLX5DV_QP_MASK_RAW_QP_TIR_ADDR
+
+    cpdef enum mlx5dv_srq_comp_mask:
+        MLX5DV_SRQ_MASK_SRQN
+
+    cpdef enum mlx5dv_obj_type:
+        MLX5DV_OBJ_QP
+        MLX5DV_OBJ_CQ
+        MLX5DV_OBJ_SRQ
+        MLX5DV_OBJ_RWQ
+        MLX5DV_OBJ_DM
+        MLX5DV_OBJ_AH
+        MLX5DV_OBJ_PD
+
     cpdef unsigned long long MLX5DV_RES_TYPE_QP
     cpdef unsigned long long MLX5DV_RES_TYPE_RWQ
     cpdef unsigned long long MLX5DV_RES_TYPE_DBR
pyverbs/providers/mlx5/mlx5dv_objects.pxd new file +28 -0
new file mode 100644
index 0000000..e8eab84
@@ -0,0 +1,28 @@
+# SPDX-License-Identifier: (GPL-2.0 OR Linux-OpenIB)
+# Copyright (c) 2021 Nvidia, Inc. All rights reserved. See COPYING file
+
+#cython: language_level=3
+
+cimport pyverbs.providers.mlx5.libmlx5 as dv
+from pyverbs.base cimport PyverbsObject
+
+
+cdef class Mlx5DvPD(PyverbsObject):
+    cdef dv.mlx5dv_pd dv_pd
+
+cdef class Mlx5DvCQ(PyverbsObject):
+    cdef dv.mlx5dv_cq dv_cq
+
+cdef class Mlx5DvQP(PyverbsObject):
+    cdef dv.mlx5dv_qp dv_qp
+
+cdef class Mlx5DvSRQ(PyverbsObject):
+    cdef dv.mlx5dv_srq dv_srq
+
+cdef class Mlx5DvObj(PyverbsObject):
+    cdef dv.mlx5dv_obj obj
+    cdef Mlx5DvCQ dv_cq
+    cdef Mlx5DvQP dv_qp
+    cdef Mlx5DvPD dv_pd
+    cdef Mlx5DvSRQ dv_srq
+
pyverbs/providers/mlx5/mlx5dv_objects.pyx new file +214 -0

214 lines not on this page · raw message

Signature · 1 line
1.8.3.1
#30 lore raw

PATCH 23/27 pyverbs/mlx5: Wrap mlx5_cqe64 struct and add enums

In reply to Yishai Hadas

From: Edward Srouji <[email protected]>

Add a Mlx5Cqe64 class that wraps mlx5_cqe64 C struct, and provide an easy way to users to get/set its fields.
In addition expose related enums.
Both are relevant and necessary for DevX data path.

---
 pyverbs/providers/mlx5/libmlx5.pxd      | 14 +++++++
 pyverbs/providers/mlx5/mlx5dv.pxd       |  3 ++
 pyverbs/providers/mlx5/mlx5dv.pyx       | 71 +++++++++++++++++++++++++++++++++
 pyverbs/providers/mlx5/mlx5dv_enums.pxd | 22 ++++++++++
 4 files changed, 110 insertions(+)
pyverbs/providers/mlx5/libmlx5.pxd +14 -0
index de4008d..af034ad 100644
@@ -273,12 +273,26 @@ cdef extern from 'infiniband/mlx5dv.h':
         qp  qp
         srq srq
 
+    cdef struct mlx5_cqe64:
+        uint16_t    wqe_id
+        uint32_t    imm_inval_pkey
+        uint32_t    byte_cnt
+        uint64_t    timestamp
+        uint16_t    wqe_counter
+        uint8_t     signature
+        uint8_t     op_own
+
 
     void mlx5dv_set_ctrl_seg(mlx5_wqe_ctrl_seg *seg, uint16_t pi, uint8_t opcode,
                              uint8_t opmod, uint32_t qp_num, uint8_t fm_ce_se,
                              uint8_t ds, uint8_t signature, uint32_t imm)
     void mlx5dv_set_data_seg(mlx5_wqe_data_seg *seg, uint32_t length,
                              uint32_t lkey, uintptr_t address)
+    uint8_t mlx5dv_get_cqe_owner(mlx5_cqe64 *cqe)
+    void mlx5dv_set_cqe_owner(mlx5_cqe64 *cqe, uint8_t val)
+    uint8_t mlx5dv_get_cqe_se(mlx5_cqe64 *cqe)
+    uint8_t mlx5dv_get_cqe_format(mlx5_cqe64 *cqe)
+    uint8_t mlx5dv_get_cqe_opcode(mlx5_cqe64 *cqe)
     bool mlx5dv_is_supported(v.ibv_device *device)
     v.ibv_context* mlx5dv_open_device(v.ibv_device *device,
                                       mlx5dv_context_attr *attr)
pyverbs/providers/mlx5/mlx5dv.pxd +3 -0
index 2b758fe..968cbdb 100644
@@ -83,3 +83,6 @@ cdef class Mlx5DevxObj(PyverbsCM):
     cdef dv.mlx5dv_devx_obj *obj
     cdef Context context
     cdef object out_view
+
+cdef class Mlx5Cqe64(PyverbsObject):
+    cdef dv.mlx5_cqe64 *cqe
pyverbs/providers/mlx5/mlx5dv.pyx +71 -0

74 lines not on this page · raw message

pyverbs/providers/mlx5/mlx5dv_enums.pxd +22 -0
index 9f8d1a1..60713e8 100644
@@ -193,6 +193,28 @@ cdef extern from 'infiniband/mlx5dv.h':
         MLX5DV_OBJ_AH
         MLX5DV_OBJ_PD
 
+    cpdef enum:
+        MLX5_RCV_DBR
+        MLX5_SND_DBR
+
+    cpdef enum:
+        MLX5_CQE_OWNER_MASK
+        MLX5_CQE_REQ
+        MLX5_CQE_RESP_WR_IMM
+        MLX5_CQE_RESP_SEND
+        MLX5_CQE_RESP_SEND_IMM
+        MLX5_CQE_RESP_SEND_INV
+        MLX5_CQE_RESIZE_CQ
+        MLX5_CQE_NO_PACKET
+        MLX5_CQE_SIG_ERR
+        MLX5_CQE_REQ_ERR
+        MLX5_CQE_RESP_ERR
+        MLX5_CQE_INVALID
+
+    cpdef enum:
+        MLX5_SEND_WQE_BB
+        MLX5_SEND_WQE_SHIFT
+
     cpdef unsigned long long MLX5DV_RES_TYPE_QP
     cpdef unsigned long long MLX5DV_RES_TYPE_RWQ
     cpdef unsigned long long MLX5DV_RES_TYPE_DBR
Signature · 1 line
1.8.3.1
#31 lore raw

PATCH 24/27 tests: Add MAC address to the tests' args

In reply to Yishai Hadas

From: Edward Srouji <[email protected]>

Append the MAC address of the relevant interface to the tests' arguments.

---
 tests/base.py | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)
tests/base.py +8 -5
index dcebdc7..f5518d1 100644
@@ -131,6 +131,7 @@ class RDMATestCase(unittest.TestCase):
         self.pkey_index = pkey_index
         self.gid_type = gid_type if gid_index is None else None
         self.ip_addr = None
+        self.mac_addr = None
         self.pre_environment = {}
         self.server = None
         self.client = None
@@ -168,13 +169,14 @@ class RDMATestCase(unittest.TestCase):
         return out.decode().split('\n')[0]
 
     @staticmethod
-    def get_ip_address(ifname):
+    def get_ip_mac_address(ifname):
         out = subprocess.check_output(['ip', '-j', 'addr', 'show', ifname])
         loaded_json = json.loads(out.decode())
         interface = loaded_json[0]['addr_info'][0]['local']
+        mac = loaded_json[0]['address']
         if 'fe80::' in interface:
             interface = interface + '%' + ifname
-        return interface
+        return interface, mac
 
     def setUp(self):
         """
@@ -242,11 +244,11 @@ class RDMATestCase(unittest.TestCase):
                 continue
             net_name = self.get_net_name(dev)
             try:
-                ip_addr = self.get_ip_address(net_name)
+                ip_addr, mac_addr = self.get_ip_mac_address(net_name)
             except (KeyError, IndexError):
-                self.args.append([dev, port, idx, None])
+                self.args.append([dev, port, idx, None, None])
             else:
-                self.args.append([dev, port, idx, ip_addr])
+                self.args.append([dev, port, idx, ip_addr, mac_addr])
 
     def _add_gids_per_device(self, ctx, dev):
         self._add_gids_per_port(ctx, dev, self.ib_port)
@@ -264,6 +266,7 @@ class RDMATestCase(unittest.TestCase):
         self.ib_port = args[1]
         self.gid_index = args[2]
         self.ip_addr = args[3]
+        self.mac_addr = args[4]
 
     def set_env_variable(self, var, value):
         """
Signature · 1 line
1.8.3.1
#32 lore raw

PATCH 25/27 tests: Add mlx5 DevX data path test

In reply to Yishai Hadas

From: Edward Srouji <[email protected]>

Extend the mlx5 tests' infrastructure to support mlx5 DevX objects creation.
Add a data path test that creates all the needed resources to be able to create an RC QP, modify it to RTS state and does SEMD_IMM traffic over DevX objects.
The tests does validate the data including the received "immediate" value, and checks the validity of the received CQEs.

---
 tests/CMakeLists.txt      |    2 +
 tests/base.py             |    1 +
 tests/mlx5_base.py        |  460 +++++++++++++++++++-
 tests/mlx5_prm_structs.py | 1046 +++++++++++++++++++++++++++++++++++++++++++++
 tests/test_mlx5_devx.py   |   23 +
 5 files changed, 1529 insertions(+), 3 deletions(-)
 create mode 100644 tests/mlx5_prm_structs.py
 create mode 100644 tests/test_mlx5_devx.py
tests/CMakeLists.txt +2 -0
index 7333f25..7b079d8 100644
@@ -8,6 +8,7 @@ rdma_python_test(tests
   base_rdmacm.py
   efa_base.py
   mlx5_base.py
+  mlx5_prm_structs.py
   rdmacm_utils.py
   test_addr.py
   test_cq.py
@@ -20,6 +21,7 @@ rdma_python_test(tests
   test_fork.py
   test_mlx5_cq.py
   test_mlx5_dc.py
+  test_mlx5_devx.py
   test_mlx5_dm_ops.py
   test_mlx5_dr.py
   test_mlx5_flow.py
tests/base.py +1 -0
index f5518d1..e9f36ab 100644
@@ -30,6 +30,7 @@ PATH_MTU = e.IBV_MTU_1024
 MAX_DEST_RD_ATOMIC = 1
 NUM_OF_PROCESSES = 2
 MC_IP_PREFIX = '230'
+MAX_RDMA_ATOMIC = 20
 MAX_RD_ATOMIC = 1
 MIN_RNR_TIMER =12
 RETRY_CNT = 7
tests/mlx5_base.py +457 -3

480 lines not on this page · raw message

tests/mlx5_prm_structs.py new file +1046 -0

1,046 lines not on this page · raw message

tests/test_mlx5_devx.py new file +23 -0
new file mode 100644
index 0000000..c43dcd5
@@ -0,0 +1,23 @@
+# SPDX-License-Identifier: (GPL-2.0 OR Linux-OpenIB)
+# Copyright (c) 2021 Nvidia Inc. All rights reserved. See COPYING file
+
+"""
+Test module for mlx5 DevX.
+"""
+
+from tests.mlx5_base import Mlx5DevxRcResources, Mlx5DevxTrafficBase
+
+
+class Mlx5DevxRcTrafficTest(Mlx5DevxTrafficBase):
+    """
+    Test various functionality of mlx5 DevX objects
+    """
+
+    def test_devx_rc_qp_send_imm_traffic(self):
+        """
+        Creates two DevX RC QPs and modifies them to RTS state.
+        Then does SEND_IMM traffic.
+        """
+        self.create_players(Mlx5DevxRcResources)
+        # Send traffic
+        self.send_imm_traffic()
Signature · 1 line
1.8.3.1
#33 lore raw

PATCH 26/27 pyverbs/mlx5: Support mlx5 devices over VFIO

In reply to Yishai Hadas

From: Edward Srouji <[email protected]>

Support opening an mlx5 device and creating a context over VFIO by adding Mlx5VfioContext class based on Mlx5Context to allow using DevX APIs.

---
 pyverbs/providers/mlx5/CMakeLists.txt   |   3 +-
 pyverbs/providers/mlx5/libmlx5.pxd      |   8 +++
 pyverbs/providers/mlx5/mlx5_vfio.pxd    |  15 +++++
 pyverbs/providers/mlx5/mlx5_vfio.pyx    | 116 ++++++++++++++++++++++++++++++++
 pyverbs/providers/mlx5/mlx5dv.pxd       |   3 +
 pyverbs/providers/mlx5/mlx5dv_enums.pxd |   3 +
 6 files changed, 147 insertions(+), 1 deletion(-)
 create mode 100644 pyverbs/providers/mlx5/mlx5_vfio.pxd
 create mode 100644 pyverbs/providers/mlx5/mlx5_vfio.pyx
pyverbs/providers/mlx5/CMakeLists.txt +2 -1
index 4763c61..c0b5869 100644
@@ -7,8 +7,9 @@ rdma_cython_module(pyverbs/providers/mlx5 mlx5
   dr_matcher.pyx
   dr_rule.pyx
   dr_table.pyx
-  mlx5dv.pyx
   mlx5_enums.pyx
+  mlx5_vfio.pyx
+  mlx5dv.pyx
   mlx5dv_flow.pyx
   mlx5dv_mkey.pyx
   mlx5dv_objects.pyx
pyverbs/providers/mlx5/libmlx5.pxd +8 -0
index af034ad..e0904f5 100644
@@ -223,6 +223,11 @@ cdef extern from 'infiniband/mlx5dv.h':
         uint64_t    pgsz_bitmap
         uint64_t    comp_mask
 
+    cdef struct mlx5dv_vfio_context_attr:
+        const char  *pci_name
+        uint32_t    flags
+        uint64_t    comp_mask
+
     cdef struct mlx5dv_pd:
         uint32_t    pdn
         uint64_t    comp_mask
@@ -372,6 +377,9 @@ cdef extern from 'infiniband/mlx5dv.h':
                              uint64_t device_timestamp)
     int mlx5dv_get_clock_info(v.ibv_context *ctx_in, mlx5dv_clock_info *clock_info)
     int mlx5dv_map_ah_to_qp(v.ibv_ah *ah, uint32_t qp_num)
+    v.ibv_device **mlx5dv_get_vfio_device_list(mlx5dv_vfio_context_attr *attr)
+    int mlx5dv_vfio_get_events_fd(v.ibv_context *ibctx)
+    int mlx5dv_vfio_process_events(v.ibv_context *context)
 
     # DevX APIs
     mlx5dv_devx_uar *mlx5dv_devx_alloc_uar(v.ibv_context *context, uint32_t flags)
pyverbs/providers/mlx5/mlx5_vfio.pxd new file +15 -0
new file mode 100644
index 0000000..0e9facd
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: (GPL-2.0 OR Linux-OpenIB)
+# Copyright (c) 2021 Nvidia, Inc. All rights reserved. See COPYING file
+
+#cython: language_level=3
+
+from pyverbs.providers.mlx5.mlx5dv cimport Mlx5Context
+cimport pyverbs.providers.mlx5.libmlx5 as dv
+from pyverbs.base cimport PyverbsObject
+
+
+cdef class Mlx5VfioContext(Mlx5Context):
+    pass
+
+cdef class Mlx5VfioAttr(PyverbsObject):
+    cdef dv.mlx5dv_vfio_context_attr attr
pyverbs/providers/mlx5/mlx5_vfio.pyx new file +116 -0

116 lines not on this page · raw message

pyverbs/providers/mlx5/mlx5dv.pxd +3 -0
index 968cbdb..490c697 100644
@@ -86,3 +86,6 @@ cdef class Mlx5DevxObj(PyverbsCM):
 
 cdef class Mlx5Cqe64(PyverbsObject):
     cdef dv.mlx5_cqe64 *cqe
+
+cdef class Mlx5VfioAttr(PyverbsObject):
+    cdef dv.mlx5dv_vfio_context_attr attr
pyverbs/providers/mlx5/mlx5dv_enums.pxd +3 -0
index 60713e8..94599ab 100644
@@ -215,6 +215,9 @@ cdef extern from 'infiniband/mlx5dv.h':
         MLX5_SEND_WQE_BB
         MLX5_SEND_WQE_SHIFT
 
+    cpdef enum mlx5dv_vfio_context_attr_flags:
+        MLX5DV_VFIO_CTX_FLAGS_INIT_LINK_DOWN
+
     cpdef unsigned long long MLX5DV_RES_TYPE_QP
     cpdef unsigned long long MLX5DV_RES_TYPE_RWQ
     cpdef unsigned long long MLX5DV_RES_TYPE_DBR
Signature · 1 line
1.8.3.1
#34 lore raw

PATCH 27/27 tests: Add a test for mlx5 over VFIO

In reply to Yishai Hadas

From: Edward Srouji <[email protected]>

Add a test that opens an mlx5 vfio context, creates two DevX QPs on it, and modifies them to RTS state. Then performs traffic with SEND_IMM. An additional cmd line argument was added for the tests, to allow users to pass a PCI device instead of an RDMA device for this test.

---
 tests/CMakeLists.txt    |   1 +
 tests/args_parser.py    |   5 +++
 tests/test_mlx5_vfio.py | 104 ++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 110 insertions(+)
 create mode 100644 tests/test_mlx5_vfio.py
tests/CMakeLists.txt +1 -0
index 7b079d8..cabeba0 100644
@@ -36,6 +36,7 @@ rdma_python_test(tests
   test_mlx5_uar.py
   test_mlx5_udp_sport.py
   test_mlx5_var.py
+  test_mlx5_vfio.py
   test_mr.py
   test_odp.py
   test_pd.py
tests/args_parser.py +5 -0
index 5bc53b0..4312121 100644
@@ -1,5 +1,6 @@
 # SPDX-License-Identifier: (GPL-2.0 OR Linux-OpenIB)
 # Copyright (c) 2020 Kamal Heib <[email protected]>, All rights reserved.  See COPYING file
+# Copyright (c) 2021 Nvidia, Inc. All rights reserved. See COPYING file
 
 import argparse
 import sys
@@ -16,6 +17,10 @@ class ArgsParser(object):
         parser = argparse.ArgumentParser()
         parser.add_argument('--dev',
                             help='RDMA device to run the tests on')
+        parser.add_argument('--pci-dev',
+                            help='PCI device to run the tests on, which is '
+                                 'needed by some tests where the RDMA device is '
+                                 'not available (e.g. VFIO)')
         parser.add_argument('--port',
                             help='Use port <port> of RDMA device', type=int,
                             default=1)
tests/test_mlx5_vfio.py new file +104 -0

104 lines not on this page · raw message

Signature · 1 line
1.8.3.1
#35 lore raw
In reply to Yishai Hadas
Quoted 115 lines · Yishai Hadas

This series introduces mlx5 user space driver over VFIO.

This enables an application to take full ownership on the opened device and run any firmware command (e.g. port up/down) without any concern to hurt someone else.

The application look and feel is like regular RDMA application over DEVX, it uses verbs API to open/close a device and then mostly uses DEVX APIs to interact with the device.

To achieve it, few mlx5 DV APIs were introduced, it includes:
- An API to get ibv_device for a given mlx5 PCI name.
- APIs to manage device specific events.

Detailed man pages were added to describe the expected usage of those APIs.

The mlx5 VFIO driver implemented the basic verbs APIs as of managing MR/PD and the DEVX APIs which are required to write an RDMA application.

The VFIO uAPIs were used to setup mlx5 vfio context by reading the device initialization segment, setup DMA and enables the device command interface.

In addition, the series includes pyverbs stuff which runs DEVX like testing over RDMA and VFIO mlx5 devices.

Some extra documentation of the benefits and the motivation to use VFIO can be found here [1].

PR was sent [2].

[1] https://www.kernel.org/doc/html/latest/driver-api/vfio.html
[2] https://github.com/linux-rdma/rdma-core/pull/1034

Yishai

Edward Srouji (10):
pyverbs: Support DevX UMEM registration pyverbs/mlx5: Support EQN querying pyverbs/mlx5: Support more DevX objects
pyverbs: Add auxiliary memory functions pyverbs/mlx5: Add support to extract mlx5dv objects pyverbs/mlx5: Wrap mlx5_cqe64 struct and add enums
tests: Add MAC address to the tests' args
tests: Add mlx5 DevX data path test pyverbs/mlx5: Support mlx5 devices over VFIO
tests: Add a test for mlx5 over VFIO

Maor Gottlieb (1):
mlx5: Enable debug functionality for vfio

Mark Zhang (5):
util: Add interval_set support
mlx5: Support fast teardown over vfio
mlx5: VFIO poll_health support
mlx5: Set DV context ops
mlx5: Implement mlx5dv devx_obj APIs over vfio

Yishai Hadas (11):
Update kernel headers
mlx5: Introduce mlx5dv_get_vfio_device_list()
verbs: Enable verbs_open_device() to work over non sysfs devices
mlx5: Setup mlx5 vfio context
mlx5: Add mlx5_vfio_cmd_exec() support
mlx5: vfio setup function support
mlx5: vfio setup basic caps
mlx5: Enable interrupt command mode over vfio
mlx5: Introduce vfio APIs to process events
mlx5: Implement basic verbs operation for PD and MR over vfio
mlx5: Support initial DEVX/DV APIs over vfio

  Documentation/pyverbs.md                           |   34 +
  debian/ibverbs-providers.symbols                   |    4 +
  kernel-headers/CMakeLists.txt                      |    4 +
  kernel-headers/linux/vfio.h                        | 1374 ++++++++
  libibverbs/device.c                                |   39 +-
  libibverbs/sysfs.c                                 |    5 +
  providers/mlx5/CMakeLists.txt                      |    3 +-
  providers/mlx5/dr_rule.c                           |   10 +-
  providers/mlx5/libmlx5.map                         |    7 +
  providers/mlx5/man/CMakeLists.txt                  |    3 +
  .../mlx5/man/mlx5dv_get_vfio_device_list.3.md      |   64 +
  providers/mlx5/man/mlx5dv_vfio_get_events_fd.3.md  |   41 +
  providers/mlx5/man/mlx5dv_vfio_process_events.3.md |   43 +
  providers/mlx5/mlx5.c                              |  376 ++-
  providers/mlx5/mlx5.h                              |  187 +-
  providers/mlx5/mlx5_ifc.h                          | 1206 ++++++-
  providers/mlx5/mlx5_vfio.c                         | 3379 ++++++++++++++++++++
  providers/mlx5/mlx5_vfio.h                         |  329 ++
  providers/mlx5/mlx5dv.h                            |   25 +
  providers/mlx5/verbs.c                             |  966 +++++-
  pyverbs/CMakeLists.txt                             |    7 +
  pyverbs/dma_util.pyx                               |   25 +
  pyverbs/mem_alloc.pyx                              |   46 +-
  pyverbs/pd.pyx                                     |    4 +
  pyverbs/providers/mlx5/CMakeLists.txt              |    4 +-
  pyverbs/providers/mlx5/libmlx5.pxd                 |  103 +-
  pyverbs/providers/mlx5/mlx5_vfio.pxd               |   15 +
  pyverbs/providers/mlx5/mlx5_vfio.pyx               |  116 +
  pyverbs/providers/mlx5/mlx5dv.pxd                  |   20 +
  pyverbs/providers/mlx5/mlx5dv.pyx                  |  277 +-
  pyverbs/providers/mlx5/mlx5dv_enums.pxd            |   42 +
  pyverbs/providers/mlx5/mlx5dv_objects.pxd          |   28 +
  pyverbs/providers/mlx5/mlx5dv_objects.pyx          |  214 ++
  tests/CMakeLists.txt                               |    3 +
  tests/args_parser.py                               |    5 +
  tests/base.py                                      |   14 +-
  tests/mlx5_base.py                                 |  460 ++-
  tests/mlx5_prm_structs.py                          | 1046 ++++++
  tests/test_mlx5_devx.py                            |   23 +
  tests/test_mlx5_vfio.py                            |  104 +
  util/CMakeLists.txt                                |    2 +
  util/interval_set.c                                |  208 ++
  util/interval_set.h                                |   77 +
  util/util.h                                        |    5 +
  44 files changed, 10650 insertions(+), 297 deletions(-)
Quoted 17 lines

create mode 100644 kernel-headers/linux/vfio.h
create mode 100644 providers/mlx5/man/mlx5dv_get_vfio_device_list.3.md create mode 100644 providers/mlx5/man/mlx5dv_vfio_get_events_fd.3.md create mode 100644 providers/mlx5/man/mlx5dv_vfio_process_events.3.md create mode 100644 providers/mlx5/mlx5_vfio.c
create mode 100644 providers/mlx5/mlx5_vfio.h
create mode 100644 pyverbs/dma_util.pyx
create mode 100644 pyverbs/providers/mlx5/mlx5_vfio.pxd create mode 100644 pyverbs/providers/mlx5/mlx5_vfio.pyx create mode 100644 pyverbs/providers/mlx5/mlx5dv_objects.pxd create mode 100644 pyverbs/providers/mlx5/mlx5dv_objects.pyx create mode 100644 tests/mlx5_prm_structs.py
create mode 100644 tests/test_mlx5_devx.py
create mode 100644 tests/test_mlx5_vfio.py
create mode 100644 util/interval_set.c
create mode 100644 util/interval_set.h

The PR was merged.

Yishai