Introduce mlx5 user space driver over VFIO
Subsystems (2)
cover Introduce mlx5 user space driver over VFIO
This series introduces mlx5 user space driver over VFIO.
This enables an application to take full ownership on the opened device and run any firmware command (e.g. port up/down) without any concern to hurt someone else.
The application look and feel is like regular RDMA application over DEVX, it uses verbs API to open/close a device and then mostly uses DEVX APIs to interact with the device.
To achieve it, few mlx5 DV APIs were introduced, it includes:
- An API to get ibv_device for a given mlx5 PCI name.
- APIs to manage device specific events.
Detailed man pages were added to describe the expected usage of those APIs.
The mlx5 VFIO driver implemented the basic verbs APIs as of managing MR/PD and the DEVX APIs which are required to write an RDMA application.
The VFIO uAPIs were used to setup mlx5 vfio context by reading the device initialization segment, setup DMA and enables the device command interface.
In addition, the series includes pyverbs stuff which runs DEVX like testing over RDMA and VFIO mlx5 devices.
Some extra documentation of the benefits and the motivation to use VFIO can be found here [1].
PR was sent [2].
[1] https://www.kernel.org/doc/html/latest/driver-api/vfio.html
[2] https://github.com/linux-rdma/rdma-core/pull/1034
Yishai
Edward Srouji (10):
pyverbs: Support DevX UMEM registration pyverbs/mlx5: Support EQN querying pyverbs/mlx5: Support more DevX objects
pyverbs: Add auxiliary memory functions pyverbs/mlx5: Add support to extract mlx5dv objects pyverbs/mlx5: Wrap mlx5_cqe64 struct and add enums
tests: Add MAC address to the tests' args
tests: Add mlx5 DevX data path test pyverbs/mlx5: Support mlx5 devices over VFIO
tests: Add a test for mlx5 over VFIO
Maor Gottlieb (1):
mlx5: Enable debug functionality for vfio
Mark Zhang (5):
util: Add interval_set support
mlx5: Support fast teardown over vfio
mlx5: VFIO poll_health support
mlx5: Set DV context ops
mlx5: Implement mlx5dv devx_obj APIs over vfio
Yishai Hadas (11):
Update kernel headers
mlx5: Introduce mlx5dv_get_vfio_device_list()
verbs: Enable verbs_open_device() to work over non sysfs devices
mlx5: Setup mlx5 vfio context
mlx5: Add mlx5_vfio_cmd_exec() support
mlx5: vfio setup function support
mlx5: vfio setup basic caps
mlx5: Enable interrupt command mode over vfio
mlx5: Introduce vfio APIs to process events
mlx5: Implement basic verbs operation for PD and MR over vfio
mlx5: Support initial DEVX/DV APIs over vfio
Documentation/pyverbs.md | 34 + debian/ibverbs-providers.symbols | 4 + kernel-headers/CMakeLists.txt | 4 + kernel-headers/linux/vfio.h | 1374 ++++++++ libibverbs/device.c | 39 +- libibverbs/sysfs.c | 5 + providers/mlx5/CMakeLists.txt | 3 +- providers/mlx5/dr_rule.c | 10 +- providers/mlx5/libmlx5.map | 7 + providers/mlx5/man/CMakeLists.txt | 3 + .../mlx5/man/mlx5dv_get_vfio_device_list.3.md | 64 + providers/mlx5/man/mlx5dv_vfio_get_events_fd.3.md | 41 + providers/mlx5/man/mlx5dv_vfio_process_events.3.md | 43 + providers/mlx5/mlx5.c | 376 ++- providers/mlx5/mlx5.h | 187 +- providers/mlx5/mlx5_ifc.h | 1206 ++++++- providers/mlx5/mlx5_vfio.c | 3379 ++++++++++++++++++++ providers/mlx5/mlx5_vfio.h | 329 ++ providers/mlx5/mlx5dv.h | 25 + providers/mlx5/verbs.c | 966 +++++- pyverbs/CMakeLists.txt | 7 + pyverbs/dma_util.pyx | 25 + pyverbs/mem_alloc.pyx | 46 +- pyverbs/pd.pyx | 4 + pyverbs/providers/mlx5/CMakeLists.txt | 4 +- pyverbs/providers/mlx5/libmlx5.pxd | 103 +- pyverbs/providers/mlx5/mlx5_vfio.pxd | 15 + pyverbs/providers/mlx5/mlx5_vfio.pyx | 116 + pyverbs/providers/mlx5/mlx5dv.pxd | 20 + pyverbs/providers/mlx5/mlx5dv.pyx | 277 +- pyverbs/providers/mlx5/mlx5dv_enums.pxd | 42 + pyverbs/providers/mlx5/mlx5dv_objects.pxd | 28 + pyverbs/providers/mlx5/mlx5dv_objects.pyx | 214 ++ tests/CMakeLists.txt | 3 + tests/args_parser.py | 5 + tests/base.py | 14 +- tests/mlx5_base.py | 460 ++- tests/mlx5_prm_structs.py | 1046 ++++++ tests/test_mlx5_devx.py | 23 + tests/test_mlx5_vfio.py | 104 + util/CMakeLists.txt | 2 + util/interval_set.c | 208 ++ util/interval_set.h | 77 + util/util.h | 5 + 44 files changed, 10650 insertions(+), 297 deletions(-) create mode 100644 kernel-headers/linux/vfio.h create mode 100644 providers/mlx5/man/mlx5dv_get_vfio_device_list.3.md create mode 100644 providers/mlx5/man/mlx5dv_vfio_get_events_fd.3.md create mode 100644 providers/mlx5/man/mlx5dv_vfio_process_events.3.md create mode 100644 providers/mlx5/mlx5_vfio.c create mode 100644 providers/mlx5/mlx5_vfio.h create mode 100644 pyverbs/dma_util.pyx create mode 100644 pyverbs/providers/mlx5/mlx5_vfio.pxd create mode 100644 pyverbs/providers/mlx5/mlx5_vfio.pyx create mode 100644 pyverbs/providers/mlx5/mlx5dv_objects.pxd create mode 100644 pyverbs/providers/mlx5/mlx5dv_objects.pyx create mode 100644 tests/mlx5_prm_structs.py create mode 100644 tests/test_mlx5_devx.py create mode 100644 tests/test_mlx5_vfio.py create mode 100644 util/interval_set.c create mode 100644 util/interval_set.h
Signature · 1 line
1.8.3.1
PATCH 1/27 Update kernel headers
To commit e73f0f0ee754 ("Linux 5.14-rc1");
--- kernel-headers/CMakeLists.txt | 4 + kernel-headers/linux/vfio.h | 1374 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 1378 insertions(+) create mode 100644 kernel-headers/linux/vfio.h
kernel-headers/CMakeLists.txt +4 -0
@@ -26,6 +26,10 @@ publish_internal_headers(rdma rdma/vmw_pvrdma-abi.h ) +publish_internal_headers(linux + linux/vfio.h + ) + publish_internal_headers(rdma/hfi rdma/hfi/hfi1_ioctl.h rdma/hfi/hfi1_user.h
kernel-headers/linux/vfio.h new file +1374 -0
1,374 lines not on this page · raw message
Signature · 1 line
1.8.3.1
PATCH 2/27 mlx5: Introduce mlx5dv_get_vfio_device_list()
In reply to Yishai HadasIntroduce mlx5dv_get_vfio_device_list() API for getting list of mlx5 devices which can be used over VFIO.
A man page with the expected usage was added.
--- debian/ibverbs-providers.symbols | 2 + providers/mlx5/CMakeLists.txt | 3 +- providers/mlx5/libmlx5.map | 5 + providers/mlx5/man/CMakeLists.txt | 1 + .../mlx5/man/mlx5dv_get_vfio_device_list.3.md | 64 +++++++ providers/mlx5/mlx5.c | 4 +- providers/mlx5/mlx5.h | 1 + providers/mlx5/mlx5_vfio.c | 190 +++++++++++++++++++++ providers/mlx5/mlx5_vfio.h | 27 +++ providers/mlx5/mlx5dv.h | 13 ++ 10 files changed, 307 insertions(+), 3 deletions(-) create mode 100644 providers/mlx5/man/mlx5dv_get_vfio_device_list.3.md create mode 100644 providers/mlx5/mlx5_vfio.c create mode 100644 providers/mlx5/mlx5_vfio.h
debian/ibverbs-providers.symbols +2 -0
@@ -28,6 +28,7 @@ libmlx5.so.1 ibverbs-providers #MINVER# MLX5_1.18@MLX5_1.18 34 MLX5_1.19@MLX5_1.19 35 MLX5_1.20@MLX5_1.20 36 + MLX5_1.21@MLX5_1.21 37 mlx5dv_init_obj@MLX5_1.0 13 mlx5dv_init_obj@MLX5_1.2 15 mlx5dv_query_device@MLX5_1.0 13
@@ -133,6 +134,7 @@ libmlx5.so.1 ibverbs-providers #MINVER# mlx5dv_map_ah_to_qp@MLX5_1.20 36 mlx5dv_qp_cancel_posted_send_wrs@MLX5_1.20 36 _mlx5dv_mkey_check@MLX5_1.20 36 + mlx5dv_get_vfio_device_list@MLX5_1.21 37 libefa.so.1 ibverbs-providers #MINVER# * Build-Depends-Package: libibverbs-dev EFA_1.0@EFA_1.0 24
providers/mlx5/CMakeLists.txt +2 -1
@@ -11,7 +11,7 @@ if (MLX5_MW_DEBUG) endif() rdma_shared_provider(mlx5 libmlx5.map - 1 1.20.${PACKAGE_VERSION} + 1 1.21.${PACKAGE_VERSION} buf.c cq.c dbrec.c
@@ -30,6 +30,7 @@ rdma_shared_provider(mlx5 libmlx5.map dr_table.c dr_send.c mlx5.c + mlx5_vfio.c qp.c srq.c verbs.c
providers/mlx5/libmlx5.map +5 -0
@@ -189,3 +189,8 @@ MLX5_1.20 { mlx5dv_qp_cancel_posted_send_wrs; _mlx5dv_mkey_check; } MLX5_1.19; + +MLX5_1.21 { + global: + mlx5dv_get_vfio_device_list; +} MLX5_1.20;
providers/mlx5/man/CMakeLists.txt +1 -0
@@ -22,6 +22,7 @@ rdma_man_pages( mlx5dv_dump.3.md mlx5dv_flow_action_esp.3.md mlx5dv_get_clock_info.3 + mlx5dv_get_vfio_device_list.3.md mlx5dv_init_obj.3 mlx5dv_is_supported.3.md mlx5dv_map_ah_to_qp.3.md
providers/mlx5/man/mlx5dv_get_vfio_device_list.3.md new file +64 -0
64 lines not on this page · raw message
providers/mlx5/mlx5.c +2 -2
@@ -62,7 +62,7 @@ static void mlx5_free_context(struct ibv_context *ibctx);#endif #define HCA (v, d) VERBS_PCI_MATCH(PCI_VENDOR_ID_##v, d, NULL) -static const struct verbs_match_ent hca_table[] = { +const struct verbs_match_ent mlx5_hca_table[] = {VERBS_DRIVER_ID (RDMA_DRIVER_MLX5 ),HCA (MELLANOX ,0x1011 ),/* MT4113 Connect-IB */ HCA (MELLANOX ,0x1012 ),/* Connect-IB Virtual Function */
@@ -2410,7 +2410,7 @@ static const struct verbs_device_ops mlx5_dev_ops = { .name = "mlx5", .match_min_abi_version = MLX5_UVERBS_MIN_ABI_VERSION, .match_max_abi_version = MLX5_UVERBS_MAX_ABI_VERSION, - .match_table = hca_table, + .match_table = mlx5_hca_table, .alloc_device = mlx5_device_alloc, .uninit_device = mlx5_uninit_device, .alloc_context = mlx5_alloc_context,
providers/mlx5/mlx5.h +1 -0
@@ -94,6 +94,7 @@ enum {extern uint32_t mlx5_debug_mask;extern int mlx5_freeze_on_error_cqe; +extern const struct verbs_match_ent mlx5_hca_table[];#ifdef MLX5_DEBUG #define mlx5_dbg(fp, mask, format, arg...) \
providers/mlx5/mlx5_vfio.c new file +190 -0
190 lines not on this page · raw message
providers/mlx5/mlx5_vfio.h new file +27 -0
@@ -0,0 +1,27 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* Copyright (c) 2021, NVIDIA CORPORATION & AFFILIATES. All rights reserved +*/ + +#ifndef MLX5_VFIO_H +#define MLX5_VFIO_H + +#include <stddef.h> +#include <stdio.h> + +#include <infiniband/driver.h> + +struct mlx5_vfio_device { +struct verbs_device vdev; +char *pci_name; +char vfio_path[IBV_SYSFS_PATH_MAX ]; +int page_size; +uint32_t flags; +}; + +static inline struct mlx5_vfio_device *to_mvfio_dev (struct ibv_device *ibdev) +{ + return container_of(ibdev, struct mlx5_vfio_device, vdev.device); +} + +#endif
providers/mlx5/mlx5dv.h +13 -0
@@ -1474,6 +1474,19 @@ struct mlx5dv_context_attr {bool mlx5dv_is_supported (struct ibv_device *device); +enum mlx5dv_vfio_context_attr_flags { +MLX5DV_VFIO_CTX_FLAGS_INIT_LINK_DOWN =1 <<0 , +}; + +struct mlx5dv_vfio_context_attr { +const char *pci_name; +uint32_t flags;/* Use enum mlx5dv_vfio_context_attr_flags */ +uint64_t comp_mask; +}; + +struct ibv_device ** +mlx5dv_get_vfio_device_list (struct mlx5dv_vfio_context_attr *attr); +struct ibv_context *mlx5dv_open_device (struct ibv_device *device,struct mlx5dv_context_attr *attr);
Signature · 1 line
1.8.3.1
PATCH 3/27 mlx5: Enable debug functionality for vfio
In reply to Yishai HadasFrom: Maor Gottlieb <[email protected]>
Usage will be in next patches.
--- providers/mlx5/mlx5.c | 28 ++++++++++++++-------------- providers/mlx5/mlx5.h | 4 ++++ 2 files changed, 18 insertions(+), 14 deletions(-)
providers/mlx5/mlx5.c +14 -14
@@ -583,7 +583,7 @@ static int get_total_uuars(int page_size)return size; } -static void open_debug_file (struct mlx5_context *ctx) +void mlx5_open_debug_file (FILE **dbg_fp) {char *env;FILE *default_dbg_fp =NULL ;
@@ -594,25 +594,25 @@ static void open_debug_file(struct mlx5_context *ctx) env =getenv ("MLX5_DEBUG_FILE" );if (!env) { - ctx->dbg_fp = default_dbg_fp; + *dbg_fp = default_dbg_fp;return ; } - ctx->dbg_fp =fopen (env,"aw+" ); -if (!ctx->dbg_fp) { - ctx->dbg_fp = default_dbg_fp; -mlx5_err (ctx->dbg_fp,"Failed opening debug file %s\n" , env); + *dbg_fp =fopen (env,"aw+" ); +if (!*dbg_fp) { + *dbg_fp = default_dbg_fp; +mlx5_err (*dbg_fp,"Failed opening debug file %s\n" , env);return ; } } -static void close_debug_file (struct mlx5_context *ctx) +void mlx5_close_debug_file (FILE *dbg_fp) { -if (ctx->dbg_fp && ctx->dbg_fp != stderr) -fclose (ctx->dbg_fp); +if (dbg_fp && dbg_fp != stderr) +fclose (dbg_fp); } -static void set_debug_mask (void ) +void mlx5_set_debug_mask (void ) {char *env;
@@ -2036,7 +2036,7 @@ static int get_uar_info(struct mlx5_device *mdev,static void mlx5_uninit_context (struct mlx5_context *context) { -close_debug_file (context); +mlx5_close_debug_file (context->dbg_fp);verbs_uninit_context (&context->ibv_ctx);free (context);
@@ -2056,8 +2056,8 @@ static struct mlx5_context *mlx5_init_context(struct ibv_device *ibdev,if (!context)return NULL ; -open_debug_file (context); -set_debug_mask (); +mlx5_open_debug_file (&context->dbg_fp); +mlx5_set_debug_mask ();set_freeze_on_error ();if (gethostname (context->hostname,sizeof (context->hostname)))strcpy (context->hostname,"host_unknown" );
@@ -2377,7 +2377,7 @@ static void mlx5_free_context(struct ibv_context *ibctx) page_size);if (context->clock_info_page)munmap ((void *)context->clock_info_page, page_size); -close_debug_file (context); +mlx5_close_debug_file (context->dbg_fp);clean_dyn_uars (ibctx);reserved_qpn_blks_free (context);
providers/mlx5/mlx5.h +4 -0
@@ -992,6 +992,10 @@ static inline struct mlx5_flow *to_mflow(struct ibv_flow *flow_id)bool is_mlx5_dev (struct ibv_device *device); +void mlx5_open_debug_file (FILE **dbg_fp); +void mlx5_close_debug_file (FILE *dbg_fp); +void mlx5_set_debug_mask (void ); +int mlx5_alloc_buf (struct mlx5_buf *buf,size_t size,int page_size);void mlx5_free_buf (struct mlx5_buf *buf); int mlx5_alloc_buf_contig(struct mlx5_context *mctx, struct mlx5_buf *buf,
Signature · 1 line
1.8.3.1
Re: [PATCH rdma-core 03/27] mlx5: Enable debug functionality for vfio
Quoted 10 lines · Yishai Hadas
From: Maor Gottlieb <[email protected]>
Usage will be in next patches.
Signed-off-by Maor Gottlieb <[email protected]>Signed-off-by Yishai Hadas <[email protected]>--- providers/mlx5/mlx5.c | 28 ++++++++++++++-------------- providers/mlx5/mlx5.h | 4 ++++ 2 files changed, 18 insertions(+), 14 deletions(-)
Probably, this patch will be needed to be changed after "Verbs logging API" PR https://github.com/linux-rdma/rdma-core/pull/1030
Thanks
Re: [PATCH rdma-core 03/27] mlx5: Enable debug functionality for vfio
Quoted 11 lines · Leon Romanovsky
Quoted 10 lines · Yishai Hadas
From: Maor Gottlieb <[email protected]>
Usage will be in next patches.
Signed-off-by Maor Gottlieb <[email protected]>Signed-off-by Yishai Hadas <[email protected]>--- providers/mlx5/mlx5.c | 28 ++++++++++++++-------------- providers/mlx5/mlx5.h | 4 ++++ 2 files changed, 18 insertions(+), 14 deletions(-)
Quoted 4 lines
Probably, this patch will be needed to be changed after "Verbs logging API" PR https://github.com/linux-rdma/rdma-core/pull/1030
Thanks
Well, not really, this patch just reorganizes things inside mlx5 for code sharing.
Yishai
Re: [PATCH rdma-core 03/27] mlx5: Enable debug functionality for vfio
Quoted 12 lines · Yishai Hadas
Quoted 11 lines · Leon Romanovsky
Quoted 10 lines · Yishai Hadas
From: Maor Gottlieb <[email protected]>
Usage will be in next patches.
Signed-off-by Maor Gottlieb <[email protected]>Signed-off-by Yishai Hadas <[email protected]>--- providers/mlx5/mlx5.c | 28 ++++++++++++++-------------- providers/mlx5/mlx5.h | 4 ++++ 2 files changed, 18 insertions(+), 14 deletions(-)
Quoted 7 lines
Quoted 4 lines
Probably, this patch will be needed to be changed after "Verbs logging API" PR https://github.com/linux-rdma/rdma-core/pull/1030
Thanks
Well, not really, this patch just reorganizes things inside mlx5 for code sharing.
After Gal's PR, I expect to see all mlx5 file/debug logic gone except some minimal conversion logic to support old legacy variables.
All that get_env_... code will go.
Thanks
Quoted 3 lines
Yishai
Re: [PATCH rdma-core 03/27] mlx5: Enable debug functionality for vfio
Quoted 17 lines · Leon Romanovsky
Quoted 16 lines · Yishai Hadas
Quoted 15 lines · Leon Romanovsky
Quoted 10 lines · Yishai Hadas
From: Maor Gottlieb <[email protected]>
Usage will be in next patches.
Signed-off-by Maor Gottlieb <[email protected]>Signed-off-by Yishai Hadas <[email protected]>--- providers/mlx5/mlx5.c | 28 ++++++++++++++-------------- providers/mlx5/mlx5.h | 4 ++++ 2 files changed, 18 insertions(+), 14 deletions(-)Probably, this patch will be needed to be changed after "Verbs logging API" PR https://github.com/linux-rdma/rdma-core/pull/1030
Thanks
Quoted 9 lines
Quoted 2 lines
Well, not really, this patch just reorganizes things inside mlx5 for code sharing.
After Gal's PR, I expect to see all mlx5 file/debug logic gone except some minimal conversion logic to support old legacy variables.
All that
get_env_... code will go.Thanks
Looking on current VERBs logging PR, it doesn't give a clean path conversion from mlx5.
For example it missed a debug granularity per object (e.g. QP, CQ, etc.) , in addition it doesn't define a driver specific options as we have today in mlx5 (e.g. MLX5_DBG_DR).
I believe that this should be added before going forward with the logging PR to enable a clean transition later on.
The transition of mlx5 should preserve current API/semantics (ENV, etc.) and is an orthogonal task.
Yishai
Re: [PATCH rdma-core 03/27] mlx5: Enable debug functionality for vfio
Quoted 34 lines · Yishai Hadas
Quoted 17 lines · Leon Romanovsky
Quoted 16 lines · Yishai Hadas
Quoted 15 lines · Leon Romanovsky
Quoted 10 lines · Yishai Hadas
From: Maor Gottlieb <[email protected]>
Usage will be in next patches.
Signed-off-by Maor Gottlieb <[email protected]>Signed-off-by Yishai Hadas <[email protected]>---
providers/mlx5/mlx5.c| 28 ++++++++++++++--------------providers/mlx5/mlx5.h| 4 ++++2 files changed, 18 insertions(+), 14 deletions(-)Probably, this patch will be needed to be changed after "Verbs logging API" PR https://github.com/linux-rdma/rdma-core/pull/1030
Thanks
Quoted 9 lines
Quoted 2 lines
Well, not really, this patch just reorganizes things inside mlx5 for code sharing.
After Gal's PR, I expect to see all mlx5 file/debug logic gone except some minimal conversion logic to support old legacy variables.
All that
get_env_... code will go.Thanks
Looking on current VERBs logging PR, it doesn't give a clean path conversion from mlx5.
For example it missed a debug granularity per object (e.g. QP, CQ, etc.) , in addition it doesn't define a driver specific options as we have today in mlx5 (e.g.
MLX5_DBG_DR).
Quoted 5 lines
I believe that this should be added before going forward with the logging PR to enable a clean transition later on.
The transition of mlx5 should preserve current API/semantics (ENV, etc.) and is an orthogonal task.
Yishai, if you have any more concerns please share them in the PR.. The discussion there is going on for a while and you've been quiet so I assumed you're fine with it.
I disagree about needing to support everything that exists in mlx5 today, the purpose of the generic mechanism is to unify the environment variables, driver specific options do the opposite. IMO masking out a few prints isn't worth the divergence.
If the mlx5 provider has backwards compatibility issues it doesn't necessarily have to use this API, but we can at least covert most existing providers and all future providers that wish to support such functionality in a unified way.
BTW, why even insist on having backwards compatibility here? Do you actually have useful code that relies on debug environment variables?
Re: [PATCH rdma-core 03/27] mlx5: Enable debug functionality for vfio
Quoted 40 lines · Gal Pressman
Quoted 34 lines · Yishai Hadas
Quoted 17 lines · Leon Romanovsky
Quoted 16 lines · Yishai Hadas
Quoted 15 lines · Leon Romanovsky
Quoted 10 lines · Yishai Hadas
From: Maor Gottlieb <[email protected]>
Usage will be in next patches.
Signed-off-by Maor Gottlieb <[email protected]>Signed-off-by Yishai Hadas <[email protected]>--- providers/mlx5/mlx5.c | 28 ++++++++++++++-------------- providers/mlx5/mlx5.h | 4 ++++ 2 files changed, 18 insertions(+), 14 deletions(-)Probably, this patch will be needed to be changed after "Verbs logging API" PR https://github.com/linux-rdma/rdma-core/pull/1030
Thanks
Quoted 9 lines
Quoted 2 lines
Well, not really, this patch just reorganizes things inside mlx5 for code sharing.
After Gal's PR, I expect to see all mlx5 file/debug logic gone except some minimal conversion logic to support old legacy variables.
All that
get_env_... code will go.Thanks
Looking on current VERBs logging PR, it doesn't give a clean path conversion from mlx5.
For example it missed a debug granularity per object (e.g. QP, CQ, etc.) , in addition it doesn't define a driver specific options as we have today in mlx5 (e.g.
MLX5_DBG_DR).Quoted 5 lines
I believe that this should be added before going forward with the logging PR to enable a clean transition later on.
The transition of mlx5 should preserve current API/semantics (ENV, etc.) and is an orthogonal task.
Quoted 8 lines
Yishai, if you have any more concerns please share them in the PR.. The discussion there is going on for a while and you've been quiet so I assumed you're fine with it.
I disagree about needing to support everything that exists in mlx5 today, the purpose of the generic mechanism is to unify the environment variables, driver specific options do the opposite. IMO masking out a few prints isn't worth the divergence.
The options in mlx5 gives more granularity and supports vendor specific options, this may be needed down the road by other vendors as well.
If we come with a new API need to consider such options in the general case.
NP, I'll comment in the logging PR as well.
Quoted 4 lines
If the mlx5 provider has backwards compatibility issues it doesn't necessarily have to use this API, but we can at least covert most existing providers and all future providers that wish to support such functionality in a unified way.
The point was that current suggestion doesn't allow a clean transition for mlx5, so we won't use it unless the API will give a matching alternative.
Quoted 2 lines
BTW, why even insist on having backwards compatibility here? Do you actually have useful code that relies on debug environment variables?
Logging options (env, mask, etc.) are kind of API which need to be preserved, it's used in the field for debug purposes, no reason to loose granularity and trace.
Yishai
Re: [PATCH rdma-core 03/27] mlx5: Enable debug functionality for vfio
Quoted 67 lines · Yishai Hadas
Quoted 40 lines · Gal Pressman
Quoted 34 lines · Yishai Hadas
Quoted 17 lines · Leon Romanovsky
Quoted 16 lines · Yishai Hadas
Quoted 15 lines · Leon Romanovsky
Quoted 10 lines · Yishai Hadas
From: Maor Gottlieb <[email protected]>
Usage will be in next patches.
Signed-off-by Maor Gottlieb <[email protected]>Signed-off-by Yishai Hadas <[email protected]>---
providers/mlx5/mlx5.c| 28 ++++++++++++++--------------providers/mlx5/mlx5.h| 4 ++++2 files changed, 18 insertions(+), 14 deletions(-)Probably, this patch will be needed to be changed after "Verbs logging API" PR https://github.com/linux-rdma/rdma-core/pull/1030
Thanks
Quoted 9 lines
Quoted 2 lines
Well, not really, this patch just reorganizes things inside mlx5 for code sharing.
After Gal's PR, I expect to see all mlx5 file/debug logic gone except some minimal conversion logic to support old legacy variables.
All that
get_env_... code will go.Thanks
Looking on current VERBs logging PR, it doesn't give a clean path conversion from mlx5.
For example it missed a debug granularity per object (e.g. QP, CQ, etc.) , in addition it doesn't define a driver specific options as we have today in mlx5 (e.g.
MLX5_DBG_DR).Quoted 5 lines
I believe that this should be added before going forward with the logging PR to enable a clean transition later on.
The transition of mlx5 should preserve current API/semantics (ENV, etc.) and is an orthogonal task.
Quoted 8 lines
Yishai, if you have any more concerns please share them in the PR.. The discussion there is going on for a while and you've been quiet so I assumed you're fine with it.
I disagree about needing to support everything that exists in mlx5 today, the purpose of the generic mechanism is to unify the environment variables, driver specific options do the opposite. IMO masking out a few prints isn't worth the divergence.
The options in mlx5 gives more granularity and supports vendor specific options, this may be needed down the road by other vendors as well.
If we come with a new API need to consider such options in the general case.
NP, I'll comment in the logging PR as well.
Quoted 4 lines
If the mlx5 provider has backwards compatibility issues it doesn't necessarily have to use this API, but we can at least covert most existing providers and all future providers that wish to support such functionality in a unified way.
The point was that current suggestion doesn't allow a clean transition for mlx5, so we won't use it unless the API will give a matching alternative.
Quoted 6 lines
Quoted 2 lines
BTW, why even insist on having backwards compatibility here? Do you actually have useful code that relies on debug environment variables?
Logging options (env, mask, etc.) are kind of API which need to be preserved, it's used in the field for debug purposes, no reason to loose granularity and trace.
It's used in the field by *people*, which unlike legacy code can learn to use the new debug env variables, they don't need backwards compatibility (the same as users of debugfs).
I addressed the granularity issue in the PR.
PATCH 4/27 util: Add interval_set support
In reply to Yishai HadasFrom: Mark Zhang <[email protected]>
Add interval_set functionality to support range management.
This functionality enables to insert/get valid ranges of values and maintains the contiguity of them internally.
This will be used in down stream patches from this series to set/get valid iova ranges from this data structure.
--- util/CMakeLists.txt | 2 + util/interval_set.c | 208 ++++++++++++++++++++++++++++++++++++++++++++++++++++ util/interval_set.h | 77 +++++++++++++++++++ 3 files changed, 287 insertions(+) create mode 100644 util/interval_set.c create mode 100644 util/interval_set.h
util/CMakeLists.txt +2 -0
@@ -1,6 +1,7 @@ publish_internal_headers(util cl_qmap.h compiler.h + interval_set.h node_name_map.h rdma_nl.h symver.h
@@ -9,6 +10,7 @@ publish_internal_headers(util set(C_FILES cl_map.c + interval_set.c node_name_map.c open_cdev.c rdma_nl.c
util/interval_set.c new file +208 -0
208 lines not on this page · raw message
util/interval_set.h new file +77 -0
77 lines not on this page · raw message
Signature · 1 line
1.8.3.1
PATCH 5/27 verbs: Enable verbs_open_device() to work over non sysfs devices
In reply to Yishai HadasEnable verbs_open_device() to work over non sysfs devices as of mlx5 over VFIO.
Any other API over verbs_sysfs_dev should fail cleanly.
--- libibverbs/device.c | 39 ++++++++++++++++++++++----------------- libibverbs/sysfs.c | 5 +++++ 2 files changed, 27 insertions(+), 17 deletions(-)
libibverbs/device.c +22 -17
@@ -124,7 +124,7 @@ LATEST_SYMVER_FUNC(ibv_get_device_guid, 1_1, "IBVERBS_1.1",int i;pthread_mutex_lock (&dev_list_lock); -if (sysfs_dev->flags &VSYSFS_READ_NODE_GUID ) { +if (sysfs_dev && sysfs_dev->flags &VSYSFS_READ_NODE_GUID ) { guid = sysfs_dev->node_guid;pthread_mutex_unlock (&dev_list_lock);return htobe64 (guid);
@@ -154,7 +154,7 @@ int ibv_get_device_index(struct ibv_device *device) {struct verbs_sysfs_dev *sysfs_dev =verbs_get_device (device)->sysfs; -return sysfs_dev->ibdev_idx; +return sysfs_dev ? sysfs_dev->ibdev_idx :-1 ; } void verbs_init_cq(struct ibv_cq *cq, struct ibv_context *context,
@@ -323,18 +323,20 @@ static void set_lib_ops(struct verbs_context *vctx)struct ibv_context *verbs_open_device (struct ibv_device *device,void *private_data) {struct verbs_device *verbs_device =verbs_get_device (device); -int cmd_fd; +int cmd_fd =-1 ;struct verbs_context *context_ex;int ret; -/* -* We'll only be doing writes, but we need O_RDWR in case the -* provider needs to mmap() the file. -*/ - cmd_fd =open_cdev (verbs_device->sysfs->sysfs_name, - verbs_device->sysfs->sysfs_cdev); -if (cmd_fd <0 ) -return NULL ; +if (verbs_device->sysfs) { +/* +* We'll only be doing writes, but we need O_RDWR in case the +* provider needs to mmap() the file. +*/ + cmd_fd =open_cdev (verbs_device->sysfs->sysfs_name, + verbs_device->sysfs->sysfs_cdev); +if (cmd_fd <0 ) +return NULL ; + }/* * cmd_fd ownership is transferred into alloc_context, if it fails
@@ -345,11 +347,13 @@ struct ibv_context *verbs_open_device(struct ibv_device *device, void *private_dreturn NULL ;set_lib_ops (context_ex); -if (context_ex->context.async_fd ==-1 ) { - ret =ibv_cmd_alloc_async_fd (&context_ex->context); -if (ret) { -ibv_close_device (&context_ex->context); -return NULL ; +if (verbs_device->sysfs) { +if (context_ex->context.async_fd ==-1 ) { + ret =ibv_cmd_alloc_async_fd (&context_ex->context); +if (ret) { +ibv_close_device (&context_ex->context); +return NULL ; + } } }
@@ -428,7 +432,8 @@ out:void verbs_uninit_context (struct verbs_context *context_ex) {free (context_ex->priv); -close (context_ex->context.cmd_fd); +if (context_ex->context.cmd_fd !=-1 ) +close (context_ex->context.cmd_fd);if (context_ex->context.async_fd !=-1 )close (context_ex->context.async_fd);ibverbs_device_put (context_ex->context.device);
libibverbs/sysfs.c +5 -0
@@ -127,6 +127,11 @@ int ibv_read_ibdev_sysfs_file(char *buf, size_t size,va_list va;int res; +if (!sysfs_dev) { + errno =EINVAL ; +return -1 ; + } +va_start (va, fnfmt);if (vasprintf (&path, fnfmt, va) <0 ) {va_end (va);
Signature · 1 line
1.8.3.1
PATCH 6/27 mlx5: Setup mlx5 vfio context
In reply to Yishai HadasSetup mlx5 vfio context by using the ioctl API over vfio to read the device initialization segment and other required stuff.
As part of that the applicable IOVA ranges are set to be ready for DMA usage and the command interface is initialized.
--- providers/mlx5/mlx5.h | 2 + providers/mlx5/mlx5_vfio.c | 740 +++++++++++++++++++++++++++++++++++++++++++++ providers/mlx5/mlx5_vfio.h | 134 ++++++++ 3 files changed, 876 insertions(+)
providers/mlx5/mlx5.h +2 -0
@@ -72,6 +72,7 @@ enum {enum {MLX5_ADAPTER_PAGE_SIZE =4096 , +MLX5_ADAPTER_PAGE_SHIFT =12 , };#define MLX5_CQ_PREFIX "MLX_CQ"
@@ -90,6 +91,7 @@ enum {MLX5_DBG_CQ_CQE =1 <<4 ,MLX5_DBG_CONTIG =1 <<5 ,MLX5_DBG_DR =1 <<6 , + MLX5_DBG_VFIO = 1 << 7, };extern uint32_t mlx5_debug_mask;
providers/mlx5/mlx5_vfio.c +740 -0
755 lines not on this page · raw message
providers/mlx5/mlx5_vfio.h +134 -0
151 lines not on this page · raw message
Signature · 1 line
1.8.3.1
PATCH 7/27 mlx5: Add mlx5_vfio_cmd_exec() support
In reply to Yishai HadasAdd mlx5_vfio_cmd_exec() support to enable running commands.
This includes the required functionality to prepare both the in/out command layouts and handle the response upon issuing the command.
--- providers/mlx5/mlx5_ifc.h | 38 ++++++ providers/mlx5/mlx5_vfio.c | 287 +++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 325 insertions(+)
providers/mlx5/mlx5_ifc.h +38 -0
@@ -73,6 +73,25 @@ enum { MLX5_CMD_OP_SYNC_STEERING = 0xb00, }; +enum { +MLX5_CMD_STAT_OK =0x0 , +MLX5_CMD_STAT_INT_ERR =0x1 , +MLX5_CMD_STAT_BAD_OP_ERR =0x2 , +MLX5_CMD_STAT_BAD_PARAM_ERR =0x3 , +MLX5_CMD_STAT_BAD_SYS_STATE_ERR =0x4 , +MLX5_CMD_STAT_BAD_RES_ERR =0x5 , +MLX5_CMD_STAT_RES_BUSY =0x6 , +MLX5_CMD_STAT_LIM_ERR =0x8 , +MLX5_CMD_STAT_BAD_RES_STATE_ERR =0x9 , +MLX5_CMD_STAT_IX_ERR =0xa , +MLX5_CMD_STAT_NO_RES_ERR =0xf , +MLX5_CMD_STAT_BAD_INP_LEN_ERR =0x50 , +MLX5_CMD_STAT_BAD_OUTP_LEN_ERR =0x51 , +MLX5_CMD_STAT_BAD_QP_STATE_ERR =0x10 , +MLX5_CMD_STAT_BAD_PKT_ERR =0x30 , +MLX5_CMD_STAT_BAD_SIZE_OUTS_CQES_ERR =0x40 , +}; +struct mlx5_ifc_atomic_caps_bits {u8 reserved_at_0[0x40 ];
@@ -4093,4 +4112,23 @@ struct mlx5_ifc_create_psv_in_bits { u8 reserved_at_60[0x20]; }; +struct mlx5_ifc_mbox_out_bits { +u8 status[0x8 ]; +u8 reserved_at_8[0x18 ]; + +u8 syndrome[0x20 ]; + +u8 reserved_at_40[0x40 ]; +}; + +struct mlx5_ifc_mbox_in_bits { +u8 opcode[0x10 ]; +u8 uid[0x10 ]; + +u8 reserved_at_20[0x10 ]; +u8 op_mod[0x10 ]; + +u8 reserved_at_40[0x40 ]; +}; +#endif /* MLX5_IFC_H */
providers/mlx5/mlx5_vfio.c +287 -0
309 lines not on this page · raw message
Signature · 1 line
1.8.3.1
PATCH 8/27 mlx5: vfio setup function support
In reply to Yishai HadasSetup device function support by following the required command sequence and steps based on the device specification.
--- providers/mlx5/mlx5_ifc.h | 215 +++++++++++++++++++++++++++++++++++++++ providers/mlx5/mlx5_vfio.c | 246 +++++++++++++++++++++++++++++++++++++++++++++ providers/mlx5/mlx5_vfio.h | 16 +++ providers/mlx5/mlx5dv.h | 4 + 4 files changed, 481 insertions(+)
providers/mlx5/mlx5_ifc.h +215 -0
@@ -41,6 +41,13 @@ enum mlx5_cap_mode {enum {MLX5_CMD_OP_QUERY_HCA_CAP =0x100 , +MLX5_CMD_OP_INIT_HCA =0x102 , +MLX5_CMD_OP_TEARDOWN_HCA =0x103 , +MLX5_CMD_OP_ENABLE_HCA =0x104 , +MLX5_CMD_OP_QUERY_PAGES =0x107 , +MLX5_CMD_OP_MANAGE_PAGES =0x108 , +MLX5_CMD_OP_QUERY_ISSI =0x10a , +MLX5_CMD_OP_SET_ISSI =0x10b ,MLX5_CMD_OP_CREATE_MKEY =0x200 ,MLX5_CMD_OP_CREATE_QP =0x500 ,MLX5_CMD_OP_RST2INIT_QP =0x502 ,
@@ -55,6 +62,7 @@ enum {MLX5_CMD_OP_QUERY_ESW_VPORT_CONTEXT =0x752 ,MLX5_CMD_OP_QUERY_NIC_VPORT_CONTEXT =0x754 ,MLX5_CMD_OP_QUERY_ROCE_ADDRESS =0x760 , +MLX5_CMD_OP_ACCESS_REG =0x805 ,MLX5_CMD_OP_QUERY_LAG =0x842 ,MLX5_CMD_OP_CREATE_TIR =0x900 , MLX5_CMD_OP_MODIFY_SQ = 0x905,
@@ -92,6 +100,16 @@ enum { MLX5_CMD_STAT_BAD_SIZE_OUTS_CQES_ERR = 0x40, }; +enum { +MLX5_PAGES_CANT_GIVE =0 , +MLX5_PAGES_GIVE =1 , +MLX5_PAGES_TAKE =2 , +}; + +enum { +MLX5_REG_HOST_ENDIANNESS =0x7004 , +}; +struct mlx5_ifc_atomic_caps_bits {u8 reserved_at_0[0x40 ];
@@ -4131,4 +4149,201 @@ struct mlx5_ifc_mbox_in_bits {u8 reserved_at_40[0x40 ]; }; +struct mlx5_ifc_enable_hca_in_bits { +u8 opcode[0x10 ]; +u8 reserved_at_10[0x10 ]; + +u8 reserved_at_20[0x10 ]; +u8 op_mod[0x10 ]; + +u8 reserved_at_40[0x10 ]; +u8 function_id[0x10 ]; + +u8 reserved_at_60[0x20 ]; +}; + +struct mlx5_ifc_enable_hca_out_bits { +u8 status[0x8 ]; +u8 reserved_at_8[0x18 ]; + +u8 syndrome[0x20 ]; + +u8 reserved_at_40[0x20 ]; +}; + +struct mlx5_ifc_query_issi_out_bits { +u8 status[0x8 ]; +u8 reserved_at_8[0x18 ]; + +u8 syndrome[0x20 ]; + +u8 reserved_at_40[0x10 ]; +u8 current_issi[0x10 ]; + +u8 reserved_at_60[0xa0 ]; + +u8 reserved_at_100[76 ][0x8 ]; +u8 supported_issi_dw0[0x20 ]; +}; + +struct mlx5_ifc_query_issi_in_bits { +u8 opcode[0x10 ]; +u8 reserved_at_10[0x10 ]; + +u8 reserved_at_20[0x10 ]; +u8 op_mod[0x10 ]; + +u8 reserved_at_40[0x40 ]; +}; + +struct mlx5_ifc_set_issi_out_bits { +u8 status[0x8 ]; +u8 reserved_at_8[0x18 ]; + +u8 syndrome[0x20 ]; + +u8 reserved_at_40[0x40 ]; +}; + +struct mlx5_ifc_set_issi_in_bits { +u8 opcode[0x10 ]; +u8 reserved_at_10[0x10 ]; + +u8 reserved_at_20[0x10 ]; +u8 op_mod[0x10 ]; + +u8 reserved_at_40[0x10 ]; +u8 current_issi[0x10 ]; + +u8 reserved_at_60[0x20 ]; +}; + +struct mlx5_ifc_query_pages_out_bits { +u8 status[0x8 ]; +u8 reserved_at_8[0x18 ]; + +u8 syndrome[0x20 ]; + +u8 embedded_cpu_function[0x01 ]; +u8 reserved_bits[0x0f ]; +u8 function_id[0x10 ]; + +u8 num_pages[0x20 ]; +}; + +struct mlx5_ifc_query_pages_in_bits { +u8 opcode[0x10 ]; +u8 reserved_at_10[0x10 ]; + +u8 reserved_at_20[0x10 ]; +u8 op_mod[0x10 ]; + +u8 reserved_at_40[0x10 ]; +u8 function_id[0x10 ]; + +u8 reserved_at_60[0x20 ]; +}; + +struct mlx5_ifc_manage_pages_out_bits { +u8 status[0x8 ]; +u8 reserved_at_8[0x18 ]; + +u8 syndrome[0x20 ]; + +u8 output_num_entries[0x20 ]; + +u8 reserved_at_60[0x20 ]; + +u8 pas[][0x40 ]; +}; + +struct mlx5_ifc_manage_pages_in_bits { +u8 opcode[0x10 ]; +u8 reserved_at_10[0x10 ]; + +u8 reserved_at_20[0x10 ]; +u8 op_mod[0x10 ]; + +u8 embedded_cpu_function[0x1 ]; +u8 reserved_at_41[0xf ]; +u8 function_id[0x10 ]; + +u8 input_num_entries[0x20 ]; + +u8 pas[][0x40 ]; +}; + +struct mlx5_ifc_teardown_hca_out_bits { +u8 status[0x8 ]; +u8 reserved_at_8[0x18 ]; + +u8 syndrome[0x20 ]; + +u8 reserved_at_40[0x3f ]; + +u8 state[0x1 ]; +}; + +enum { +MLX5_TEARDOWN_HCA_IN_PROFILE_GRACEFUL_CLOSE =0x0 , +}; + +struct mlx5_ifc_teardown_hca_in_bits { +u8 opcode[0x10 ]; +u8 reserved_at_10[0x10 ]; + +u8 reserved_at_20[0x10 ]; +u8 op_mod[0x10 ]; + +u8 reserved_at_40[0x10 ]; +u8 profile[0x10 ]; + +u8 reserved_at_60[0x20 ]; +}; + +struct mlx5_ifc_init_hca_out_bits { +u8 status[0x8 ]; +u8 reserved_at_8[0x18 ]; + +u8 syndrome[0x20 ]; + +u8 reserved_at_40[0x40 ]; +}; + +struct mlx5_ifc_init_hca_in_bits { +u8 opcode[0x10 ]; +u8 reserved_at_10[0x10 ]; + +u8 reserved_at_20[0x10 ]; +u8 op_mod[0x10 ]; + +u8 reserved_at_40[0x40 ]; +}; + +struct mlx5_ifc_access_register_out_bits { +u8 status[0x8 ]; +u8 reserved_at_8[0x18 ]; + +u8 syndrome[0x20 ]; + +u8 reserved_at_40[0x40 ]; + +u8 register_data[][0x20 ]; +}; + +struct mlx5_ifc_access_register_in_bits { +u8 opcode[0x10 ]; +u8 reserved_at_10[0x10 ]; + +u8 reserved_at_20[0x10 ]; +u8 op_mod[0x10 ]; + +u8 reserved_at_40[0x10 ]; +u8 register_id[0x10 ]; + +u8 argument[0x20 ]; + +u8 register_data[][0x20 ]; +}; +#endif /* MLX5_IFC_H */
providers/mlx5/mlx5_vfio.c +246 -0
267 lines not on this page · raw message
providers/mlx5/mlx5_vfio.h +16 -0
@@ -13,6 +13,9 @@#include <infiniband/driver.h> #include <util/interval_set.h> +#define FW_INIT_WAIT_MS 2 +#define FW_PRE_INIT_TIMEOUT_MILI 120000 +enum {MLX5_MAX_COMMANDS =32 ,MLX5_CMD_DATA_BLOCK_SIZE =512 ,
@@ -32,6 +35,19 @@ struct mlx5_vfio_device {uint32_t flags; }; +#if __BYTE_ORDER == __LITTLE_ENDIAN +#define MLX5_SET_HOST_ENDIANNESS 0 +#elif __BYTE_ORDER == __BIG_ENDIAN +#define MLX5_SET_HOST_ENDIANNESS 0x80 +#else +#error Host endianness not defined +#endif + +struct mlx5_reg_host_endianness { +uint8_t he; +uint8_t rsvd[15 ]; +}; +struct health_buffer {__be32 assert_var[5 ];__be32 rsvd0[3 ];
providers/mlx5/mlx5dv.h +4 -0
@@ -1687,6 +1687,10 @@ static inline uint64_t _devx_get64(const void *p, size_t bit_off)#define DEVX_GET64 (typ, p, fld) _devx_get64(p, __devx_bit_off(typ, fld)) +#define DEVX_ARRAY_SET64 (typ, p, fld, idx, v) do { \ + DEVX_SET64(typ, p, fld[idx], v); \ +} while (0) +struct mlx5dv_dr_domain ;struct mlx5dv_dr_table ;struct mlx5dv_dr_matcher ;
Signature · 1 line
1.8.3.1
PATCH 9/27 mlx5: vfio setup basic caps
In reply to Yishai HadasSet basic caps that are required to initialize the device properly.
--- providers/mlx5/mlx5_ifc.h | 87 ++++++++++++++++++++- providers/mlx5/mlx5_vfio.c | 185 ++++++++++++++++++++++++++++++++++++++++++++- providers/mlx5/mlx5_vfio.h | 21 +++++ 3 files changed, 290 insertions(+), 3 deletions(-)
providers/mlx5/mlx5_ifc.h +85 -2
@@ -36,6 +36,7 @@#define u8 uint8_tenum mlx5_cap_mode { +HCA_CAP_OPMOD_GET_MAX =0 ,HCA_CAP_OPMOD_GET_CUR =1 , };
@@ -46,6 +47,7 @@ enum {MLX5_CMD_OP_ENABLE_HCA =0x104 ,MLX5_CMD_OP_QUERY_PAGES =0x107 ,MLX5_CMD_OP_MANAGE_PAGES =0x108 , +MLX5_CMD_OP_SET_HCA_CAP =0x109 ,MLX5_CMD_OP_QUERY_ISSI =0x10a ,MLX5_CMD_OP_SET_ISSI =0x10b , MLX5_CMD_OP_CREATE_MKEY = 0x200,
@@ -61,6 +63,7 @@ enum {MLX5_CMD_OP_QUERY_DCT =0x713 ,MLX5_CMD_OP_QUERY_ESW_VPORT_CONTEXT =0x752 ,MLX5_CMD_OP_QUERY_NIC_VPORT_CONTEXT =0x754 , +MLX5_CMD_OP_MODIFY_NIC_VPORT_CONTEXT =0x755 ,MLX5_CMD_OP_QUERY_ROCE_ADDRESS =0x760 ,MLX5_CMD_OP_ACCESS_REG =0x805 , MLX5_CMD_OP_QUERY_LAG = 0x842,
@@ -110,6 +113,11 @@ enum { MLX5_REG_HOST_ENDIANNESS = 0x7004, }; +enum { +MLX5_CAP_PORT_TYPE_IB =0x0 , +MLX5_CAP_PORT_TYPE_ETH =0x1 , +}; +struct mlx5_ifc_atomic_caps_bits {u8 reserved_at_0[0x40 ];
@@ -140,7 +148,8 @@ struct mlx5_ifc_atomic_caps_bits { };struct mlx5_ifc_roce_cap_bits { -u8 reserved_0[0x5 ]; +u8 reserved_0[0x4 ]; +u8 sw_r_roce_src_udp_port[0x1 ];u8 fl_rc_qp_when_roce_disabled[0x1 ];u8 fl_rc_qp_when_roce_enabled[0x1 ];u8 reserved_at_7[0x17 ];
@@ -912,7 +921,8 @@ struct mlx5_ifc_cmd_hca_cap_bits {u8 uar_4k[0x1 ];u8 reserved_at_241[0x9 ];u8 uar_sz[0x6 ]; -u8 reserved_at_250[0x3 ]; +u8 reserved_at_250[0x2 ]; +u8 umem_uid_0[0x1 ];u8 log_max_dc_cnak_qps[0x5 ];u8 log_pg_sz[0x8 ];
@@ -1339,8 +1349,11 @@ struct mlx5_ifc_query_hca_cap_in_bits { };enum mlx5_cap_type { +MLX5_CAP_GENERAL =0 ,MLX5_CAP_ODP =2 ,MLX5_CAP_ATOMIC =3 , +MLX5_CAP_ROCE , +MLX5_CAP_NUM , };enum {
@@ -4346,4 +4359,74 @@ struct mlx5_ifc_access_register_in_bits {u8 register_data[][0x20 ]; }; +struct mlx5_ifc_modify_nic_vport_context_out_bits { +u8 status[0x8 ]; +u8 reserved_at_8[0x18 ]; + +u8 syndrome[0x20 ]; + +u8 reserved_at_40[0x40 ]; +}; + +struct mlx5_ifc_modify_nic_vport_field_select_bits { +u8 reserved_at_0[0x12 ]; +u8 affiliation[0x1 ]; +u8 reserved_at_13[0x1 ]; +u8 disable_uc_local_lb[0x1 ]; +u8 disable_mc_local_lb[0x1 ]; +u8 node_guid[0x1 ]; +u8 port_guid[0x1 ]; +u8 min_inline[0x1 ]; +u8 mtu[0x1 ]; +u8 change_event[0x1 ]; +u8 promisc[0x1 ]; +u8 permanent_address[0x1 ]; +u8 addresses_list[0x1 ]; +u8 roce_en[0x1 ]; +u8 reserved_at_1f[0x1 ]; +}; + +struct mlx5_ifc_modify_nic_vport_context_in_bits { +u8 opcode[0x10 ]; +u8 reserved_at_10[0x10 ]; + +u8 reserved_at_20[0x10 ]; +u8 op_mod[0x10 ]; + +u8 other_vport[0x1 ]; +u8 reserved_at_41[0xf ]; +u8 vport_number[0x10 ]; + +struct mlx5_ifc_modify_nic_vport_field_select_bits field_select; + +u8 reserved_at_80[0x780 ]; + +struct mlx5_ifc_nic_vport_context_bits nic_vport_context; +}; + +struct mlx5_ifc_set_hca_cap_out_bits { +u8 status[0x8 ]; +u8 reserved_at_8[0x18 ]; + +u8 syndrome[0x20 ]; + +u8 reserved_at_40[0x40 ]; +}; + +struct mlx5_ifc_set_hca_cap_in_bits { +u8 opcode[0x10 ]; +u8 reserved_at_10[0x10 ]; + +u8 reserved_at_20[0x10 ]; +u8 op_mod[0x10 ]; + +u8 other_function[0x1 ]; +u8 reserved_at_41[0xf ]; +u8 function_id[0x10 ]; + +u8 reserved_at_60[0x20 ]; + +union mlx5_ifc_hca_cap_union_bits capability; +}; +#endif /* MLX5_IFC_H */
providers/mlx5/mlx5_vfio.c +184 -1
203 lines not on this page · raw message
providers/mlx5/mlx5_vfio.h +21 -0
@@ -12,6 +12,7 @@#include <infiniband/driver.h> #include <util/interval_set.h> +#include "mlx5_ifc.h" #define FW_INIT_WAIT_MS 2#define FW_PRE_INIT_TIMEOUT_MILI 120000
@@ -43,6 +44,22 @@ struct mlx5_vfio_device {#error Host endianness not defined#endif +/* GET Dev Caps macros */ +#define MLX5_VFIO_CAP_GEN (ctx, cap) \ + DEVX_GET(cmd_hca_cap, ctx->caps.hca_cur[MLX5_CAP_GENERAL], cap) + +#define MLX5_VFIO_CAP_GEN_64 (mdev, cap) \ + DEVX_GET64(cmd_hca_cap, mdev->caps.hca_cur[MLX5_CAP_GENERAL], cap) + +#define MLX5_VFIO_CAP_GEN_MAX (ctx, cap) \ + DEVX_GET(cmd_hca_cap, ctx->caps.hca_max[MLX5_CAP_GENERAL], cap) + +#define MLX5_VFIO_CAP_ROCE (ctx, cap) \ + DEVX_GET(roce_cap, ctx->caps.hca_cur[MLX5_CAP_ROCE], cap) + +#define MLX5_VFIO_CAP_ROCE_MAX (ctx, cap) \ + DEVX_GET(roce_cap, ctx->caps.hca_max[MLX5_CAP_ROCE], cap) +struct mlx5_reg_host_endianness {uint8_t he;uint8_t rsvd[15 ];
@@ -162,6 +179,10 @@ struct mlx5_vfio_context {size_t bar_map_size;struct mlx5_vfio_cmd cmd;bool have_eq; +struct { +uint32_t hca_cur[MLX5_CAP_NUM ][DEVX_UN_SZ_DW (hca_cap_union)]; +uint32_t hca_max[MLX5_CAP_NUM ][DEVX_UN_SZ_DW (hca_cap_union)]; + } caps; };static inline struct mlx5_vfio_device *to_mvfio_dev (struct ibv_device *ibdev)
Signature · 1 line
1.8.3.1
PATCH 10/27 mlx5: Support fast teardown over vfio
In reply to Yishai HadasFrom: Mark Zhang <[email protected]>
Add vfio fast teardown support; If it fails then do regular teardown.
--- providers/mlx5/mlx5_ifc.h | 5 +++ providers/mlx5/mlx5_vfio.c | 76 +++++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 80 insertions(+), 1 deletion(-)
providers/mlx5/mlx5_ifc.h +5 -0
@@ -4286,6 +4286,10 @@ struct mlx5_ifc_manage_pages_in_bits {u8 pas[][0x40 ]; }; +enum { +MLX5_TEARDOWN_HCA_OUT_FORCE_STATE_FAIL =0x1 , +}; +struct mlx5_ifc_teardown_hca_out_bits {u8 status[0x8 ];u8 reserved_at_8[0x18 ];
@@ -4299,6 +4303,7 @@ struct mlx5_ifc_teardown_hca_out_bits {enum {MLX5_TEARDOWN_HCA_IN_PROFILE_GRACEFUL_CLOSE =0x0 , +MLX5_TEARDOWN_HCA_IN_PROFILE_PREPARE_FAST_TEARDOWN =0x2 , };struct mlx5_ifc_teardown_hca_in_bits {
providers/mlx5/mlx5_vfio.c +75 -1
88 lines not on this page · raw message
Signature · 1 line
1.8.3.1
PATCH 11/27 mlx5: Enable interrupt command mode over vfio
In reply to Yishai HadasEnable interrupt command mode over vfio by using EQ and its related device stuff.
--- providers/mlx5/mlx5_ifc.h | 150 ++++++++++++++++++ providers/mlx5/mlx5_vfio.c | 373 ++++++++++++++++++++++++++++++++++++++++++++- providers/mlx5/mlx5_vfio.h | 65 ++++++++ 3 files changed, 582 insertions(+), 6 deletions(-)
providers/mlx5/mlx5_ifc.h +150 -0
@@ -51,6 +51,8 @@ enum {MLX5_CMD_OP_QUERY_ISSI =0x10a ,MLX5_CMD_OP_SET_ISSI =0x10b ,MLX5_CMD_OP_CREATE_MKEY =0x200 , +MLX5_CMD_OP_CREATE_EQ =0x301 , +MLX5_CMD_OP_DESTROY_EQ =0x302 ,MLX5_CMD_OP_CREATE_QP =0x500 ,MLX5_CMD_OP_RST2INIT_QP =0x502 , MLX5_CMD_OP_INIT2RTR_QP = 0x503,
@@ -65,6 +67,8 @@ enum {MLX5_CMD_OP_QUERY_NIC_VPORT_CONTEXT =0x754 ,MLX5_CMD_OP_MODIFY_NIC_VPORT_CONTEXT =0x755 ,MLX5_CMD_OP_QUERY_ROCE_ADDRESS =0x760 , +MLX5_CMD_OP_ALLOC_UAR =0x802 , +MLX5_CMD_OP_DEALLOC_UAR =0x803 ,MLX5_CMD_OP_ACCESS_REG =0x805 ,MLX5_CMD_OP_QUERY_LAG =0x842 , MLX5_CMD_OP_CREATE_TIR = 0x900,
@@ -118,6 +122,15 @@ enum { MLX5_CAP_PORT_TYPE_ETH = 0x1, }; +enum mlx5_event { +MLX5_EVENT_TYPE_CMD =0x0a , +MLX5_EVENT_TYPE_PAGE_REQUEST =0xb , +}; + +enum { +MLX5_EQ_DOORBEL_OFFSET =0x40 , +}; +struct mlx5_ifc_atomic_caps_bits {u8 reserved_at_0[0x40 ];
@@ -4434,4 +4447,141 @@ struct mlx5_ifc_set_hca_cap_in_bits {union mlx5_ifc_hca_cap_union_bits capability; }; +struct mlx5_ifc_alloc_uar_out_bits { +u8 status[0x8 ]; +u8 reserved_at_8[0x18 ]; + +u8 syndrome[0x20 ]; + +u8 reserved_at_40[0x8 ]; +u8 uar[0x18 ]; + +u8 reserved_at_60[0x20 ]; +}; + +struct mlx5_ifc_alloc_uar_in_bits { +u8 opcode[0x10 ]; +u8 reserved_at_10[0x10 ]; + +u8 reserved_at_20[0x10 ]; +u8 op_mod[0x10 ]; + +u8 reserved_at_40[0x40 ]; +}; + +struct mlx5_ifc_dealloc_uar_out_bits { +u8 status[0x8 ]; +u8 reserved_at_8[0x18 ]; + +u8 syndrome[0x20 ]; + +u8 reserved_at_40[0x40 ]; +}; + +struct mlx5_ifc_dealloc_uar_in_bits { +u8 opcode[0x10 ]; +u8 reserved_at_10[0x10 ]; + +u8 reserved_at_20[0x10 ]; +u8 op_mod[0x10 ]; + +u8 reserved_at_40[0x8 ]; +u8 uar[0x18 ]; + +u8 reserved_at_60[0x20 ]; +}; + +struct mlx5_ifc_eqc_bits { +u8 status[0x4 ]; +u8 reserved_at_4[0x9 ]; +u8 ec[0x1 ]; +u8 oi[0x1 ]; +u8 reserved_at_f[0x5 ]; +u8 st[0x4 ]; +u8 reserved_at_18[0x8 ]; + +u8 reserved_at_20[0x20 ]; + +u8 reserved_at_40[0x14 ]; +u8 page_offset[0x6 ]; +u8 reserved_at_5a[0x6 ]; + +u8 reserved_at_60[0x3 ]; +u8 log_eq_size[0x5 ]; +u8 uar_page[0x18 ]; + +u8 reserved_at_80[0x20 ]; + +u8 reserved_at_a0[0x18 ]; +u8 intr[0x8 ]; + +u8 reserved_at_c0[0x3 ]; +u8 log_page_size[0x5 ]; +u8 reserved_at_c8[0x18 ]; + +u8 reserved_at_e0[0x60 ]; + +u8 reserved_at_140[0x8 ]; +u8 consumer_counter[0x18 ]; + +u8 reserved_at_160[0x8 ]; +u8 producer_counter[0x18 ]; + +u8 reserved_at_180[0x80 ]; +}; + +struct mlx5_ifc_create_eq_out_bits { +u8 status[0x8 ]; +u8 reserved_at_8[0x18 ]; + +u8 syndrome[0x20 ]; + +u8 reserved_at_40[0x18 ]; +u8 eq_number[0x8 ]; + +u8 reserved_at_60[0x20 ]; +}; + +struct mlx5_ifc_create_eq_in_bits { +u8 opcode[0x10 ]; +u8 uid[0x10 ]; + +u8 reserved_at_20[0x10 ]; +u8 op_mod[0x10 ]; + +u8 reserved_at_40[0x40 ]; + +struct mlx5_ifc_eqc_bits eq_context_entry; + +u8 reserved_at_280[0x40 ]; + +u8 event_bitmask[4 ][0x40 ]; + +u8 reserved_at_3c0[0x4c0 ]; + +u8 pas[][0x40 ]; +}; + +struct mlx5_ifc_destroy_eq_out_bits { +u8 status[0x8 ]; +u8 reserved_at_8[0x18 ]; + +u8 syndrome[0x20 ]; + +u8 reserved_at_40[0x40 ]; +}; + +struct mlx5_ifc_destroy_eq_in_bits { +u8 opcode[0x10 ]; +u8 reserved_at_10[0x10 ]; + +u8 reserved_at_20[0x10 ]; +u8 op_mod[0x10 ]; + +u8 reserved_at_40[0x18 ]; +u8 eq_number[0x8 ]; + +u8 reserved_at_60[0x20 ]; +}; +#endif /* MLX5_IFC_H */
providers/mlx5/mlx5_vfio.c +367 -6
448 lines not on this page · raw message
providers/mlx5/mlx5_vfio.h +65 -0
95 lines not on this page · raw message
Signature · 1 line
1.8.3.1
PATCH 12/27 mlx5: Introduce vfio APIs to process events
In reply to Yishai HadasIntroduce vfio APIs to process events, it includes:
- mlx5dv_vfio_get_events_fd()
- mlx5dv_vfio_process_events()
The first API returns an FD number that should be used to detect whether events exist.
The second API should be called to process the existing events.
At that step the PAGE request event support was added in addition to support async command mode to let the mlx5dv_vfio_process_events() return to the caller without blocking.
Detailed man pages were added to describe the expected usage of the new APIs.
--- debian/ibverbs-providers.symbols | 2 + providers/mlx5/libmlx5.map | 2 + providers/mlx5/man/CMakeLists.txt | 2 + providers/mlx5/man/mlx5dv_vfio_get_events_fd.3.md | 41 ++++ providers/mlx5/man/mlx5dv_vfio_process_events.3.md | 43 ++++ providers/mlx5/mlx5_vfio.c | 231 ++++++++++++++++++++- providers/mlx5/mlx5_vfio.h | 14 ++ providers/mlx5/mlx5dv.h | 8 + 8 files changed, 332 insertions(+), 11 deletions(-) create mode 100644 providers/mlx5/man/mlx5dv_vfio_get_events_fd.3.md create mode 100644 providers/mlx5/man/mlx5dv_vfio_process_events.3.md
debian/ibverbs-providers.symbols +2 -0
@@ -135,6 +135,8 @@ libmlx5.so.1 ibverbs-providers #MINVER# mlx5dv_qp_cancel_posted_send_wrs@MLX5_1.20 36 _mlx5dv_mkey_check@MLX5_1.20 36 mlx5dv_get_vfio_device_list@MLX5_1.21 37 + mlx5dv_vfio_get_events_fd@MLX5_1.21 37 + mlx5dv_vfio_process_events@MLX5_1.21 37 libefa.so.1 ibverbs-providers #MINVER# * Build-Depends-Package: libibverbs-dev EFA_1.0@EFA_1.0 24
providers/mlx5/libmlx5.map +2 -0
@@ -193,4 +193,6 @@ MLX5_1.20 { MLX5_1.21 { global: mlx5dv_get_vfio_device_list; + mlx5dv_vfio_get_events_fd; + mlx5dv_vfio_process_events; } MLX5_1.20;
providers/mlx5/man/CMakeLists.txt +2 -0
@@ -40,6 +40,8 @@ rdma_man_pages( mlx5dv_sched_node_create.3.md mlx5dv_ts_to_ns.3 mlx5dv_wr_mkey_configure.3.md + mlx5dv_vfio_get_events_fd.3.md + mlx5dv_vfio_process_events.3.md mlx5dv_wr_post.3.md mlx5dv_wr_set_mkey_sig_block.3.md mlx5dv.7
providers/mlx5/man/mlx5dv_vfio_get_events_fd.3.md new file +41 -0
41 lines not on this page · raw message
providers/mlx5/man/mlx5dv_vfio_process_events.3.md new file +43 -0
43 lines not on this page · raw message
providers/mlx5/mlx5_vfio.c +220 -11
338 lines not on this page · raw message
providers/mlx5/mlx5_vfio.h +14 -0
@@ -151,9 +151,17 @@ struct mlx5_cmd_msg {struct mlx5_cmd_mailbox *next; }; +typedef int (*vfio_cmd_slot_comp )(struct mlx5_vfio_context *ctx,unsigned long slot); +struct cmd_async_data { +void *buff_in; +int ilen; +void *buff_out; +int olen; +}; +struct mlx5_vfio_cmd_slot {struct mlx5_cmd_layout *lay;struct mlx5_cmd_msg in;
@@ -161,6 +169,11 @@ struct mlx5_vfio_cmd_slot {pthread_mutex_t lock;int completion_event_fd;vfio_cmd_slot_comp comp_func; +/* async cmd caller data */ +bool in_use; +struct cmd_async_data curr; +bool is_pending; +struct cmd_async_data pending; };struct mlx5_vfio_cmd {
@@ -245,6 +258,7 @@ struct mlx5_vfio_context {uint32_t hca_cur[MLX5_CAP_NUM ][DEVX_UN_SZ_DW (hca_cap_union)];uint32_t hca_max[MLX5_CAP_NUM ][DEVX_UN_SZ_DW (hca_cap_union)]; } caps; +struct mlx5_eq async_eq;struct mlx5_vfio_eqs_uar eqs_uar;pthread_mutex_t eq_lock;
providers/mlx5/mlx5dv.h +8 -0
@@ -1487,6 +1487,14 @@ struct mlx5dv_vfio_context_attr {struct ibv_device **mlx5dv_get_vfio_device_list (struct mlx5dv_vfio_context_attr *attr); +int mlx5dv_vfio_get_events_fd (struct ibv_context *ibctx); + +/* This API should run from application thread and maintain device events. +* The application is responsible to get the events FD by calling mlx5dv_vfio_get_events_fd +* and once the FD is pollable call the API to let driver process the ready events. +*/ +int mlx5dv_vfio_process_events (struct ibv_context *context); +struct ibv_context *mlx5dv_open_device (struct ibv_device *device,struct mlx5dv_context_attr *attr);
Signature · 1 line
1.8.3.1
PATCH 13/27 mlx5: VFIO poll_health support
In reply to Yishai HadasFrom: Mark Zhang <[email protected]>
Add firmware health polling support in vfio driver.
Such a case is not expected and we refer it as some fatal error in the firmware that should be avoided/fixed.
The health buffer check is triggered by the application upon its call to mlx5dv_vfio_process_events().
--- providers/mlx5/mlx5_vfio.c | 168 +++++++++++++++++++++++++++++++++++++++++++++ providers/mlx5/mlx5_vfio.h | 10 ++- 2 files changed, 177 insertions(+), 1 deletion(-)
providers/mlx5/mlx5_vfio.c +168 -0
@@ -22,6 +22,8 @@#include <poll.h> #include <util/mmio.h> +#include <ccan/array_size.h> +#include "mlx5dv.h" #include "mlx5_vfio.h" #include "mlx5.h"
@@ -1910,6 +1912,7 @@ enum mlx5_cmd_addr_l_sz_offset {enum {MLX5_NIC_IFC_DISABLED =1 , +MLX5_NIC_IFC_SW_RESET =7 , };static uint8_t mlx5_vfio_get_nic_state (struct mlx5_vfio_context *ctx)
@@ -1978,6 +1981,169 @@ static int mlx5_vfio_teardown_hca(struct mlx5_vfio_context *ctx)return mlx5_vfio_teardown_hca_regular (ctx); } +static bool sensor_pci_not_working (struct mlx5_init_seg *init_seg) +{ +/* Offline PCI reads return 0xffffffff */ +return (be32toh (mmio_read32_be (&init_seg->health.fw_ver)) ==0xffffffff ); +} + +enum mlx5_fatal_assert_bit_offsets { +MLX5_RFR_OFFSET =31 , +}; + +static bool sensor_fw_synd_rfr (struct mlx5_init_seg *init_seg) +{ +uint32_t rfr =be32toh (mmio_read32_be (&init_seg->health.rfr)) >>MLX5_RFR_OFFSET ; +uint8_t synd =mmio_read8 (&init_seg->health.synd); + +return (rfr && synd); +} + +enum { +MLX5_SENSOR_NO_ERR =0 , +MLX5_SENSOR_PCI_COMM_ERR =1 , +MLX5_SENSOR_NIC_DISABLED =3 , +MLX5_SENSOR_NIC_SW_RESET =4 , +MLX5_SENSOR_FW_SYND_RFR =5 , +}; + +static uint32_t mlx5_health_check_fatal_sensors (struct mlx5_vfio_context *ctx) +{ +if (sensor_pci_not_working (ctx->bar_map)) +return MLX5_SENSOR_PCI_COMM_ERR ; + +if (mlx5_vfio_get_nic_state (ctx) ==MLX5_NIC_IFC_DISABLED ) +return MLX5_SENSOR_NIC_DISABLED ; + +if (mlx5_vfio_get_nic_state (ctx) ==MLX5_NIC_IFC_SW_RESET ) +return MLX5_SENSOR_NIC_SW_RESET ; + +if (sensor_fw_synd_rfr (ctx->bar_map)) +return MLX5_SENSOR_FW_SYND_RFR ; + +return MLX5_SENSOR_NO_ERR ; +} + +enum { +MLX5_HEALTH_SYNDR_FW_ERR =0x1 , +MLX5_HEALTH_SYNDR_IRISC_ERR =0x7 , +MLX5_HEALTH_SYNDR_HW_UNRECOVERABLE_ERR =0x8 , +MLX5_HEALTH_SYNDR_CRC_ERR =0x9 , +MLX5_HEALTH_SYNDR_FETCH_PCI_ERR =0xa , +MLX5_HEALTH_SYNDR_HW_FTL_ERR =0xb , +MLX5_HEALTH_SYNDR_ASYNC_EQ_OVERRUN_ERR =0xc , +MLX5_HEALTH_SYNDR_EQ_ERR =0xd , +MLX5_HEALTH_SYNDR_EQ_INV =0xe , +MLX5_HEALTH_SYNDR_FFSER_ERR =0xf , +MLX5_HEALTH_SYNDR_HIGH_TEMP =0x10 , +}; + +static const char *hsynd_str (u8 synd) +{ +switch (synd) { +case MLX5_HEALTH_SYNDR_FW_ERR : +return "firmware internal error" ; +case MLX5_HEALTH_SYNDR_IRISC_ERR : +return "irisc not responding" ; +case MLX5_HEALTH_SYNDR_HW_UNRECOVERABLE_ERR : +return "unrecoverable hardware error" ; +case MLX5_HEALTH_SYNDR_CRC_ERR : +return "firmware CRC error" ; +case MLX5_HEALTH_SYNDR_FETCH_PCI_ERR : +return "ICM fetch PCI error" ; +case MLX5_HEALTH_SYNDR_HW_FTL_ERR : +return "HW fatal error\n" ; +case MLX5_HEALTH_SYNDR_ASYNC_EQ_OVERRUN_ERR : +return "async EQ buffer overrun" ; +case MLX5_HEALTH_SYNDR_EQ_ERR : +return "EQ error" ; +case MLX5_HEALTH_SYNDR_EQ_INV : +return "Invalid EQ referenced" ; +case MLX5_HEALTH_SYNDR_FFSER_ERR : +return "FFSER error" ; +case MLX5_HEALTH_SYNDR_HIGH_TEMP : +return "High temperature" ; +default : +return "unrecognized error" ; + } +} + +static void print_health_info (struct mlx5_vfio_context *ctx) +{ +struct mlx5_init_seg *iseg = ctx->bar_map; +struct health_buffer *h = &iseg->health; +char fw_str[18 ] = {}; +int i; + +/* If the syndrome is 0, the device is OK and no need to print buffer */ +if (!mmio_read8 (&h->synd)) +return ; + +for (i =0 ; i <ARRAY_SIZE (h->assert_var); i++) +mlx5_err (ctx->dbg_fp,"assert_var[%d] 0x%08x\n" , + i,be32toh (mmio_read32_be (h->assert_var + i))); + +mlx5_err (ctx->dbg_fp,"assert_exit_ptr 0x%08x\n" , +be32toh (mmio_read32_be (&h->assert_exit_ptr))); +mlx5_err (ctx->dbg_fp,"assert_callra 0x%08x\n" , +be32toh (mmio_read32_be (&h->assert_callra))); +sprintf (fw_str,"%d.%d.%d" , +be32toh (mmio_read32_be (&iseg->fw_rev)) &0xffff , +be32toh (mmio_read32_be (&iseg->fw_rev)) >>16 , +be32toh (mmio_read32_be (&iseg->cmdif_rev_fw_sub)) &0xffff ); +mlx5_err (ctx->dbg_fp,"fw_ver %s\n" , fw_str); +mlx5_err (ctx->dbg_fp,"hw_id 0x%08x\n" ,be32toh (mmio_read32_be (&h->hw_id))); +mlx5_err (ctx->dbg_fp,"irisc_index %d\n" ,mmio_read8 (&h->irisc_index)); +mlx5_err (ctx->dbg_fp,"synd 0x%x: %s\n" ,mmio_read8 (&h->synd), +hsynd_str (mmio_read8 (&h->synd))); +mlx5_err (ctx->dbg_fp,"ext_synd 0x%04x\n" , +be16toh (mmio_read16_be (&h->ext_synd))); +mlx5_err (ctx->dbg_fp,"raw fw_ver 0x%08x\n" , +be32toh (mmio_read32_be (&iseg->fw_rev))); +} + +static void mlx5_vfio_poll_health (struct mlx5_vfio_context *ctx) +{ +struct mlx5_vfio_health_state *hstate = &ctx->health_state; +uint32_t fatal_error, count; +struct timeval tv; +uint64_t time; +int ret; + + ret =gettimeofday (&tv,NULL ); +if (ret) +return ; + + time = (uint64_t )tv.tv_sec *1000 + tv.tv_usec /1000 ; +if (time - hstate->prev_time <POLL_HEALTH_INTERVAL ) +return ; + + fatal_error =mlx5_health_check_fatal_sensors (ctx); +if (fatal_error) { +mlx5_err (ctx->dbg_fp,"%s: Fatal error %u detected\n" , + __func__, fatal_error); + goto err; + } + count =be32toh (mmio_read32_be (&ctx->bar_map->health_counter)) &0xffffff ; +if (count == hstate->prev_count) + ++hstate->miss_counter; +else + hstate->miss_counter =0 ; + + hstate->prev_time = time; + hstate->prev_count = count; +if (hstate->miss_counter ==MAX_MISSES ) { +mlx5_err (ctx->dbg_fp, +"device's health compromised - reached miss count\n" ); + goto err; + } + +return ; +err: +print_health_info (ctx); +abort (); +} +static int mlx5_vfio_setup_function (struct mlx5_vfio_context *ctx) {int err;
@@ -2232,6 +2398,8 @@ int mlx5dv_vfio_process_events(struct ibv_context *ibctx)uint64_t u;ssize_t s; +mlx5_vfio_poll_health (ctx); +/* read to re-arm the FD and process all existing events */ s =read (ctx->cmd_comp_fd, &u,sizeof (uint64_t ));if (s <0 && errno !=EAGAIN ) {
providers/mlx5/mlx5_vfio.h +9 -1
@@ -240,6 +240,14 @@ struct mlx5_vfio_eqs_uar {uint64_t iova; }; +#define POLL_HEALTH_INTERVAL 1000/* ms */ +#define MAX_MISSES 3 +struct mlx5_vfio_health_state { +uint64_t prev_time;/* ms */ +uint32_t prev_count; +uint32_t miss_counter; +}; +struct mlx5_vfio_context {struct verbs_context vctx;int container_fd;
@@ -258,7 +266,7 @@ struct mlx5_vfio_context {uint32_t hca_cur[MLX5_CAP_NUM ][DEVX_UN_SZ_DW (hca_cap_union)];uint32_t hca_max[MLX5_CAP_NUM ][DEVX_UN_SZ_DW (hca_cap_union)]; } caps; - +struct mlx5_vfio_health_state health_state;struct mlx5_eq async_eq;struct mlx5_vfio_eqs_uar eqs_uar;pthread_mutex_t eq_lock;
Signature · 1 line
1.8.3.1
PATCH 14/27 mlx5: Implement basic verbs operation for PD and MR over vfio
In reply to Yishai HadasImplement basic verbs operation for PD and MR over vfio, this includes:
- PD alloc/dealloc
- MR reg/dereg.
--- providers/mlx5/mlx5_ifc.h | 76 ++++++++++++- providers/mlx5/mlx5_vfio.c | 273 +++++++++++++++++++++++++++++++++++++++++++++ providers/mlx5/mlx5_vfio.h | 25 +++++ util/util.h | 5 + 4 files changed, 377 insertions(+), 2 deletions(-)
providers/mlx5/mlx5_ifc.h +74 -2
@@ -51,6 +51,7 @@ enum {MLX5_CMD_OP_QUERY_ISSI =0x10a ,MLX5_CMD_OP_SET_ISSI =0x10b ,MLX5_CMD_OP_CREATE_MKEY =0x200 , +MLX5_CMD_OP_DESTROY_MKEY =0x202 ,MLX5_CMD_OP_CREATE_EQ =0x301 ,MLX5_CMD_OP_DESTROY_EQ =0x302 , MLX5_CMD_OP_CREATE_QP = 0x500,
@@ -67,6 +68,8 @@ enum {MLX5_CMD_OP_QUERY_NIC_VPORT_CONTEXT =0x754 ,MLX5_CMD_OP_MODIFY_NIC_VPORT_CONTEXT =0x755 ,MLX5_CMD_OP_QUERY_ROCE_ADDRESS =0x760 , +MLX5_CMD_OP_ALLOC_PD =0x800 , +MLX5_CMD_OP_DEALLOC_PD =0x801 ,MLX5_CMD_OP_ALLOC_UAR =0x802 ,MLX5_CMD_OP_DEALLOC_UAR =0x803 , MLX5_CMD_OP_ACCESS_REG = 0x805,
@@ -1380,7 +1383,8 @@ enum { };enum { -MLX5_MKC_ACCESS_MODE_KLMS =0x2 , +MLX5_MKC_ACCESS_MODE_MTT =0x1 , +MLX5_MKC_ACCESS_MODE_KLMS =0x2 , };struct mlx5_ifc_mkc_bits {
@@ -1425,7 +1429,9 @@ struct mlx5_ifc_mkc_bits {u8 translations_octword_size[0x20 ]; -u8 reserved_at_1c0[0x1b ]; +u8 reserved_at_1c0[0x19 ]; +u8 relaxed_ordering_read[0x1 ]; +u8 reserved_at_1d9[0x1 ];u8 log_page_size[0x5 ];u8 reserved_at_1e0[0x20 ];
@@ -1467,6 +1473,28 @@ struct mlx5_ifc_create_mkey_in_bits {u8 klm_pas_mtt[0 ][0x20 ]; }; +struct mlx5_ifc_destroy_mkey_out_bits { +u8 status[0x8 ]; +u8 reserved_at_8[0x18 ]; + +u8 syndrome[0x20 ]; + +u8 reserved_at_40[0x40 ]; +}; + +struct mlx5_ifc_destroy_mkey_in_bits { +u8 opcode[0x10 ]; +u8 uid[0x10 ]; + +u8 reserved_at_20[0x10 ]; +u8 op_mod[0x10 ]; + +u8 reserved_at_40[0x8 ]; +u8 mkey_index[0x18 ]; + +u8 reserved_at_60[0x20 ]; +}; +struct mlx5_ifc_l2_hdr_bits {u8 dmac_47_16[0x20 ];u8 dmac_15_0[0x10 ];
@@ -4584,4 +4612,48 @@ struct mlx5_ifc_destroy_eq_in_bits {u8 reserved_at_60[0x20 ]; }; +struct mlx5_ifc_alloc_pd_out_bits { +u8 status[0x8 ]; +u8 reserved_at_8[0x18 ]; + +u8 syndrome[0x20 ]; + +u8 reserved_at_40[0x8 ]; +u8 pd[0x18 ]; + +u8 reserved_at_60[0x20 ]; +}; + +struct mlx5_ifc_alloc_pd_in_bits { +u8 opcode[0x10 ]; +u8 uid[0x10 ]; + +u8 reserved_at_20[0x10 ]; +u8 op_mod[0x10 ]; + +u8 reserved_at_40[0x40 ]; +}; + +struct mlx5_ifc_dealloc_pd_out_bits { +u8 status[0x8 ]; +u8 reserved_at_8[0x18 ]; + +u8 syndrome[0x20 ]; + +u8 reserved_at_40[0x40 ]; +}; + +struct mlx5_ifc_dealloc_pd_in_bits { +u8 opcode[0x10 ]; +u8 uid[0x10 ]; + +u8 reserved_at_20[0x10 ]; +u8 op_mod[0x10 ]; + +u8 reserved_at_40[0x8 ]; +u8 pd[0x18 ]; + +u8 reserved_at_60[0x20 ]; +}; +#endif /* MLX5_IFC_H */
providers/mlx5/mlx5_vfio.c +273 -0
297 lines not on this page · raw message
providers/mlx5/mlx5_vfio.h +25 -0
@@ -23,17 +23,37 @@ enum { MLX5_PCI_CMD_XPORT = 7, }; +enum mlx5_ib_mtt_access_flags { +MLX5_MTT_READ = (1 <<0 ), +MLX5_MTT_WRITE = (1 <<1 ), +}; + +enum { +MLX5_MAX_PAGE_SHIFT =31 , +}; + +#define MLX5_MTT_PRESENT (MLX5_MTT_READ | MLX5_MTT_WRITE) +enum {MLX5_VFIO_BLOCK_SIZE =2 *1024 *1024 ,MLX5_VFIO_BLOCK_NUM_PAGES =MLX5_VFIO_BLOCK_SIZE /MLX5_ADAPTER_PAGE_SIZE , }; +struct mlx5_vfio_mr { +struct verbs_mr vmr; +uint64_t iova; +uint64_t iova_page_size; +uint64_t iova_aligned_offset; +uint64_t iova_reg_size; +}; +struct mlx5_vfio_device {struct verbs_device vdev;char *pci_name;char vfio_path[IBV_SYSFS_PATH_MAX ];int page_size;uint32_t flags; +atomic_int mkey_var; };#if __BYTE_ORDER == __LITTLE_ENDIAN
@@ -282,4 +302,9 @@ static inline struct mlx5_vfio_context *to_mvfio_ctx(struct ibv_context *ibctx) return container_of(ibctx, struct mlx5_vfio_context, vctx.context); } +static inline struct mlx5_vfio_mr *to_mvfio_mr (struct ibv_mr *ibmr) +{ + return container_of(ibmr, struct mlx5_vfio_mr, vmr.ibv_mr); +} +#endif
util/util.h +5 -0
@@ -70,6 +70,11 @@ static inline unsigned long align(unsigned long val, unsigned long align)return (val + align -1 ) & ~(align -1 ); } +static inline unsigned long align_down (unsigned long val,unsigned long _align) +{ +return align (val - (_align -1 ), _align); +} +static inline uint64_t roundup_pow_of_two (uint64_t n) {return n ==1 ?1 :1ULL <<ilog64 (n -1 );
Signature · 1 line
1.8.3.1
PATCH 15/27 mlx5: Set DV context ops
In reply to Yishai HadasFrom: Mark Zhang <[email protected]>
Wrap DV APIs and call the matching function pointer if supported. This comes to handle both rdma and vfio flows.
--- providers/mlx5/dr_rule.c | 10 +- providers/mlx5/mlx5.c | 344 ++++++++++++---- providers/mlx5/mlx5.h | 180 ++++++++- providers/mlx5/mlx5_vfio.c | 27 ++ providers/mlx5/mlx5_vfio.h | 1 + providers/mlx5/verbs.c | 966 +++++++++++++++++++++++++++++++++++++-------- 6 files changed, 1287 insertions(+), 241 deletions(-)
providers/mlx5/dr_rule.c +5 -5
@@ -1341,11 +1341,11 @@ dr_rule_create_rule_root(struct mlx5dv_dr_matcher *matcher,if (ret) goto free_attr_aux; - rule->flow =__mlx5dv_create_flow (matcher->dv_matcher, - value, - num_actions, - attr, - attr_aux); + rule->flow =_mlx5dv_create_flow (matcher->dv_matcher, + value, + num_actions, + attr, + attr_aux);if (!rule->flow) goto remove_action_members;
providers/mlx5/mlx5.c +276 -68
589 lines not on this page · raw message
providers/mlx5/mlx5.h +174 -6
208 lines not on this page · raw message
providers/mlx5/mlx5_vfio.c +27 -0
@@ -2459,6 +2459,25 @@ end:return NULL ; } +static struct mlx5dv_devx_obj * +vfio_devx_obj_create (struct ibv_context *context,const void *in, +size_t inlen,void *out,size_t outlen) +{ + errno =EOPNOTSUPP ; +return NULL ; +} + +static int vfio_devx_obj_query (struct mlx5dv_devx_obj *obj,const void *in, +size_t inlen,void *out,size_t outlen) +{ +return EOPNOTSUPP ; +} + +static struct mlx5_dv_context_ops mlx5_vfio_dv_ctx_ops = { + .devx_obj_create = vfio_devx_obj_create, + .devx_obj_query = vfio_devx_obj_query, +}; +static void mlx5_vfio_uninit_context (struct mlx5_vfio_context *ctx) {mlx5_close_debug_file (ctx->dbg_fp);
@@ -2524,6 +2543,7 @@ mlx5_vfio_alloc_context(struct ibv_device *ibdev, goto func_teardown;verbs_set_ops (&mctx->vctx, &mlx5_vfio_common_ops); + mctx->dv_ctx_ops = &mlx5_vfio_dv_ctx_ops;return &mctx->vctx; func_teardown:
@@ -2729,3 +2749,10 @@ end:free (list);return NULL ; } + +bool is_mlx5_vfio_dev (struct ibv_device *device) +{ +struct verbs_device *verbs_device =verbs_get_device (device); + +return verbs_device->ops == &mlx5_vfio_dev_ops; +}
providers/mlx5/mlx5_vfio.h +1 -0
@@ -290,6 +290,7 @@ struct mlx5_vfio_context {struct mlx5_eq async_eq;struct mlx5_vfio_eqs_uar eqs_uar;pthread_mutex_t eq_lock; +struct mlx5_dv_context_ops *dv_ctx_ops; };static inline struct mlx5_vfio_device *to_mvfio_dev (struct ibv_device *ibdev)
providers/mlx5/verbs.c +804 -162
1,398 lines not on this page · raw message
Signature · 1 line
1.8.3.1
PATCH 16/27 mlx5: Support initial DEVX/DV APIs over vfio
In reply to Yishai HadasSupport initial DEVX/DV APIs over vfio for UMEM/UAR/EQN usage.
--- providers/mlx5/mlx5_ifc.h | 70 ++++++++++++++ providers/mlx5/mlx5_vfio.c | 228 ++++++++++++++++++++++++++++++++++++++++++++- providers/mlx5/mlx5_vfio.h | 10 ++ 3 files changed, 307 insertions(+), 1 deletion(-)
providers/mlx5/mlx5_ifc.h +70 -0
@@ -88,6 +88,8 @@ enum {MLX5_CMD_OP_CREATE_GENERAL_OBJECT =0xa00 ,MLX5_CMD_OP_MODIFY_GENERAL_OBJECT =0xa01 ,MLX5_CMD_OP_QUERY_GENERAL_OBJECT =0xa02 , +MLX5_CMD_OP_CREATE_UMEM =0xa08 , +MLX5_CMD_OP_DESTROY_UMEM =0xa0a , MLX5_CMD_OP_SYNC_STEERING = 0xb00, };
@@ -4656,4 +4658,72 @@ struct mlx5_ifc_dealloc_pd_in_bits {u8 reserved_at_60[0x20 ]; }; +struct mlx5_ifc_mtt_bits { +u8 ptag_63_32[0x20 ]; + +u8 ptag_31_8[0x18 ]; +u8 reserved_at_38[0x6 ]; +u8 wr_en[0x1 ]; +u8 rd_en[0x1 ]; +}; + +struct mlx5_ifc_umem_bits { +u8 reserved_at_0[0x80 ]; + +u8 reserved_at_80[0x1b ]; +u8 log_page_size[0x5 ]; + +u8 page_offset[0x20 ]; + +u8 num_of_mtt[0x40 ]; + +struct mlx5_ifc_mtt_bits mtt[]; +}; + +struct mlx5_ifc_create_umem_in_bits { +u8 opcode[0x10 ]; +u8 uid[0x10 ]; + +u8 reserved_at_20[0x10 ]; +u8 op_mod[0x10 ]; + +u8 reserved_at_40[0x40 ]; + +struct mlx5_ifc_umem_bits umem; +}; + +struct mlx5_ifc_create_umem_out_bits { +u8 status[0x8 ]; +u8 reserved_at_8[0x18 ]; + +u8 syndrome[0x20 ]; + +u8 reserved_at_40[0x8 ]; +u8 umem_id[0x18 ]; + +u8 reserved_at_60[0x20 ]; +}; + +struct mlx5_ifc_destroy_umem_in_bits { +u8 opcode[0x10 ]; +u8 uid[0x10 ]; + +u8 reserved_at_20[0x10 ]; +u8 op_mod[0x10 ]; + +u8 reserved_at_40[0x8 ]; +u8 umem_id[0x18 ]; + +u8 reserved_at_60[0x20 ]; +}; + +struct mlx5_ifc_destroy_umem_out_bits { +u8 status[0x8 ]; +u8 reserved_at_8[0x18 ]; + +u8 syndrome[0x20 ]; + +u8 reserved_at_40[0x40 ]; +}; +#endif /* MLX5_IFC_H */
providers/mlx5/mlx5_vfio.c +227 -1
258 lines not on this page · raw message
providers/mlx5/mlx5_vfio.h +10 -0
@@ -47,6 +47,16 @@ struct mlx5_vfio_mr {uint64_t iova_reg_size; }; +struct mlx5_vfio_devx_umem { +struct mlx5dv_devx_umem dv_devx_umem; +struct ibv_context *context; +void *addr; +size_t size; +uint64_t iova; +uint64_t iova_size; +uint64_t iova_reg_size; +}; +struct mlx5_vfio_device {struct verbs_device vdev;char *pci_name;
Signature · 1 line
1.8.3.1
PATCH 17/27 mlx5: Implement mlx5dv devx_obj APIs over vfio
In reply to Yishai HadasFrom: Mark Zhang <[email protected]>
Implement mlx5dv vfio APIs: devx_obj_create/query/modify/destroy, as well as the devx_general_cmd
--- providers/mlx5/mlx5_ifc.h | 565 ++++++++++++++++++++++++++++++++++++++++++++- providers/mlx5/mlx5_vfio.c | 413 ++++++++++++++++++++++++++++++++- providers/mlx5/mlx5_vfio.h | 8 + 3 files changed, 975 insertions(+), 11 deletions(-)
providers/mlx5/mlx5_ifc.h +563 -2
@@ -54,7 +54,10 @@ enum {MLX5_CMD_OP_DESTROY_MKEY =0x202 ,MLX5_CMD_OP_CREATE_EQ =0x301 ,MLX5_CMD_OP_DESTROY_EQ =0x302 , +MLX5_CMD_OP_CREATE_CQ =0x400 , +MLX5_CMD_OP_DESTROY_CQ =0x401 ,MLX5_CMD_OP_CREATE_QP =0x500 , +MLX5_CMD_OP_DESTROY_QP =0x501 ,MLX5_CMD_OP_RST2INIT_QP =0x502 ,MLX5_CMD_OP_INIT2RTR_QP =0x503 , MLX5_CMD_OP_RTR2RTS_QP = 0x504,
@@ -63,31 +66,71 @@ enum {MLX5_CMD_OP_INIT2INIT_QP =0x50e ,MLX5_CMD_OP_CREATE_PSV =0x600 ,MLX5_CMD_OP_DESTROY_PSV =0x601 , +MLX5_CMD_OP_CREATE_SRQ =0x700 , +MLX5_CMD_OP_DESTROY_SRQ =0x701 , +MLX5_CMD_OP_CREATE_XRC_SRQ =0x705 , +MLX5_CMD_OP_DESTROY_XRC_SRQ =0x706 , +MLX5_CMD_OP_CREATE_DCT =0x710 , +MLX5_CMD_OP_DESTROY_DCT =0x711 ,MLX5_CMD_OP_QUERY_DCT =0x713 , +MLX5_CMD_OP_CREATE_XRQ =0x717 , +MLX5_CMD_OP_DESTROY_XRQ =0x718 ,MLX5_CMD_OP_QUERY_ESW_VPORT_CONTEXT =0x752 ,MLX5_CMD_OP_QUERY_NIC_VPORT_CONTEXT =0x754 ,MLX5_CMD_OP_MODIFY_NIC_VPORT_CONTEXT =0x755 ,MLX5_CMD_OP_QUERY_ROCE_ADDRESS =0x760 , +MLX5_CMD_OP_ALLOC_Q_COUNTER =0x771 , +MLX5_CMD_OP_DEALLOC_Q_COUNTER =0x772 , +MLX5_CMD_OP_CREATE_SCHEDULING_ELEMENT =0x782 , +MLX5_CMD_OP_DESTROY_SCHEDULING_ELEMENT =0x783 ,MLX5_CMD_OP_ALLOC_PD =0x800 ,MLX5_CMD_OP_DEALLOC_PD =0x801 ,MLX5_CMD_OP_ALLOC_UAR =0x802 ,MLX5_CMD_OP_DEALLOC_UAR =0x803 ,MLX5_CMD_OP_ACCESS_REG =0x805 , +MLX5_CMD_OP_ATTACH_TO_MCG =0x806 , +MLX5_CMD_OP_DETACH_FROM_MCG =0x807 , +MLX5_CMD_OP_ALLOC_XRCD =0x80e , +MLX5_CMD_OP_DEALLOC_XRCD =0x80f , +MLX5_CMD_OP_ALLOC_TRANSPORT_DOMAIN =0x816 , +MLX5_CMD_OP_DEALLOC_TRANSPORT_DOMAIN =0x817 , +MLX5_CMD_OP_ADD_VXLAN_UDP_DPORT =0x827 , +MLX5_CMD_OP_DELETE_VXLAN_UDP_DPORT =0x828 , +MLX5_CMD_OP_SET_L2_TABLE_ENTRY =0x829 , +MLX5_CMD_OP_DELETE_L2_TABLE_ENTRY =0x82b ,MLX5_CMD_OP_QUERY_LAG =0x842 ,MLX5_CMD_OP_CREATE_TIR =0x900 , +MLX5_CMD_OP_DESTROY_TIR =0x902 , +MLX5_CMD_OP_CREATE_SQ =0x904 ,MLX5_CMD_OP_MODIFY_SQ =0x905 , +MLX5_CMD_OP_DESTROY_SQ =0x906 , +MLX5_CMD_OP_CREATE_RQ =0x908 , +MLX5_CMD_OP_DESTROY_RQ =0x90a , +MLX5_CMD_OP_CREATE_RMP =0x90c , +MLX5_CMD_OP_DESTROY_RMP =0x90e , +MLX5_CMD_OP_CREATE_TIS =0x912 ,MLX5_CMD_OP_MODIFY_TIS =0x913 , +MLX5_CMD_OP_DESTROY_TIS =0x914 ,MLX5_CMD_OP_QUERY_TIS =0x915 , +MLX5_CMD_OP_CREATE_RQT =0x916 , +MLX5_CMD_OP_DESTROY_RQT =0x918 ,MLX5_CMD_OP_CREATE_FLOW_TABLE =0x930 , +MLX5_CMD_OP_DESTROY_FLOW_TABLE =0x931 ,MLX5_CMD_OP_QUERY_FLOW_TABLE =0x932 ,MLX5_CMD_OP_CREATE_FLOW_GROUP =0x933 , +MLX5_CMD_OP_DESTROY_FLOW_GROUP =0x934 ,MLX5_CMD_OP_SET_FLOW_TABLE_ENTRY =0x936 , +MLX5_CMD_OP_DELETE_FLOW_TABLE_ENTRY =0x938 ,MLX5_CMD_OP_CREATE_FLOW_COUNTER =0x939 , +MLX5_CMD_OP_DEALLOC_FLOW_COUNTER =0x93a ,MLX5_CMD_OP_ALLOC_PACKET_REFORMAT_CONTEXT =0x93d ,MLX5_CMD_OP_DEALLOC_PACKET_REFORMAT_CONTEXT =0x93e , +MLX5_CMD_OP_ALLOC_MODIFY_HEADER_CONTEXT =0x940 , +MLX5_CMD_OP_DEALLOC_MODIFY_HEADER_CONTEXT =0x941 ,MLX5_CMD_OP_CREATE_GENERAL_OBJECT =0xa00 ,MLX5_CMD_OP_MODIFY_GENERAL_OBJECT =0xa01 ,MLX5_CMD_OP_QUERY_GENERAL_OBJECT =0xa02 , +MLX5_CMD_OP_DESTROY_GENERAL_OBJECT =0xa03 ,MLX5_CMD_OP_CREATE_UMEM =0xa08 ,MLX5_CMD_OP_DESTROY_UMEM =0xa0a , MLX5_CMD_OP_SYNC_STEERING = 0xb00,
@@ -236,6 +279,27 @@ struct mlx5_ifc_create_flow_table_out_bits {u8 icm_address_31_0[0x20 ]; }; +struct mlx5_ifc_destroy_flow_table_in_bits { +u8 opcode[0x10 ]; +u8 reserved_at_10[0x10 ]; + +u8 reserved_at_20[0x20 ]; + +u8 other_vport[0x1 ]; +u8 reserved_at_41[0xf ]; +u8 vport_number[0x10 ]; + +u8 reserved_at_60[0x20 ]; + +u8 table_type[0x8 ]; +u8 reserved_at_88[0x18 ]; + +u8 reserved_at_a0[0x8 ]; +u8 table_id[0x18 ]; + +u8 reserved_at_c0[0x140 ]; +}; +struct mlx5_ifc_query_flow_table_in_bits {u8 opcode[0x10 ];u8 reserved_at_10[0x10 ];
@@ -2991,6 +3055,17 @@ struct mlx5_ifc_alloc_flow_counter_out_bits {u8 reserved_at_60[0x20 ]; }; +struct mlx5_ifc_dealloc_flow_counter_in_bits { +u8 opcode[0x10 ]; +u8 reserved_at_10[0x10 ]; + +u8 reserved_at_20[0x20 ]; + +u8 flow_counter_id[0x20 ]; + +u8 reserved_at_60[0x20 ]; +}; +enum {MLX5_OBJ_TYPE_FLOW_METER =0x000a ,MLX5_OBJ_TYPE_MATCH_DEFINER =0x0018 ,
@@ -3422,6 +3497,18 @@ struct mlx5_ifc_create_tir_out_bits {u8 icm_address_31_0[0x20 ]; }; +struct mlx5_ifc_destroy_tir_in_bits { +u8 opcode[0x10 ]; +u8 uid[0x10 ]; + +u8 reserved_at_20[0x20 ]; + +u8 reserved_at_40[0x8 ]; +u8 tirn[0x18 ]; + +u8 reserved_at_60[0x20 ]; +}; +struct mlx5_ifc_create_qp_out_bits {u8 status[0x8 ];u8 reserved_at_8[0x18 ];
@@ -3459,6 +3546,18 @@ struct mlx5_ifc_create_qp_in_bits {u8 pas[0 ][0x40 ]; }; +struct mlx5_ifc_destroy_qp_in_bits { +u8 opcode[0x10 ]; +u8 uid[0x10 ]; + +u8 reserved_at_20[0x20 ]; + +u8 reserved_at_40[0x8 ]; +u8 qpn[0x18 ]; + +u8 reserved_at_60[0x20 ]; +}; +enum mlx5_qpc_opt_mask_32 {MLX5_QPC_OPT_MASK_32_QOS_QUEUE_GROUP_ID =1 <<1 ,MLX5_QPC_OPT_MASK_32_UDP_SPORT =1 <<2 ,
@@ -3898,7 +3997,13 @@ struct mlx5_ifc_create_flow_group_in_bits {u8 opcode[0x10 ];u8 reserved_at_10[0x10 ]; -u8 reserved_at_20[0x60 ]; +u8 reserved_at_20[0x20 ]; + +u8 other_vport[0x1 ]; +u8 reserved_at_41[0xf ]; +u8 vport_number[0x10 ]; + +u8 reserved_at_60[0x20 ];u8 table_type[0x8 ];u8 reserved_at_88[0x18 ];
@@ -3921,6 +4026,29 @@ struct mlx5_ifc_create_flow_group_out_bits {u8 reserved_at_60[0x20 ]; }; +struct mlx5_ifc_destroy_flow_group_in_bits { +u8 opcode[0x10 ]; +u8 reserved_at_10[0x10 ]; + +u8 reserved_at_20[0x20 ]; + +u8 other_vport[0x1 ]; +u8 reserved_at_41[0xf ]; +u8 vport_number[0x10 ]; + +u8 reserved_at_60[0x20 ]; + +u8 table_type[0x8 ]; +u8 reserved_at_88[0x18 ]; + +u8 reserved_at_a0[0x8 ]; +u8 table_id[0x18 ]; + +u8 group_id[0x20 ]; + +u8 reserved_at_e0[0x120 ]; +}; +struct mlx5_ifc_dest_format_bits {u8 destination_type[0x8 ];u8 destination_id[0x18 ];
@@ -3977,7 +4105,14 @@ struct mlx5_ifc_set_fte_in_bits {u8 opcode[0x10 ];u8 reserved_at_10[0x10 ]; -u8 reserved_at_20[0x60 ]; +u8 reserved_at_20[0x10 ]; +u8 op_mod[0x10 ]; + +u8 other_vport[0x1 ]; +u8 reserved_at_41[0xf ]; +u8 vport_number[0x10 ]; + +u8 reserved_at_60[0x20 ];u8 table_type[0x8 ];u8 reserved_at_88[0x18 ];
@@ -4186,6 +4321,18 @@ struct mlx5_ifc_create_psv_in_bits {u8 reserved_at_60[0x20 ]; }; +struct mlx5_ifc_destroy_psv_in_bits { +u8 opcode[0x10 ]; +u8 reserved_at_10[0x10 ]; + +u8 reserved_at_20[0x20 ]; + +u8 reserved_at_40[0x8 ]; +u8 psvn[0x18 ]; + +u8 reserved_at_60[0x20 ]; +}; +struct mlx5_ifc_mbox_out_bits {u8 status[0x8 ];u8 reserved_at_8[0x18 ];
@@ -4726,4 +4873,418 @@ struct mlx5_ifc_destroy_umem_out_bits {u8 reserved_at_40[0x40 ]; }; +struct mlx5_ifc_delete_fte_in_bits { +u8 opcode[0x10 ]; +u8 reserved_at_10[0x10 ]; + +u8 reserved_at_20[0x20 ]; + +u8 other_vport[0x1 ]; +u8 reserved_at_41[0xf ]; +u8 vport_number[0x10 ]; + +u8 reserved_at_60[0x20 ]; + +u8 table_type[0x8 ]; +u8 reserved_at_88[0x18 ]; + +u8 reserved_at_a0[0x8 ]; +u8 table_id[0x18 ]; + +u8 reserved_at_c0[0x40 ]; + +u8 flow_index[0x20 ]; + +u8 reserved_at_120[0xe0 ]; +}; + +struct mlx5_ifc_create_cq_out_bits { +u8 reserved_at_0[0x40 ]; + +u8 reserved_at_40[0x8 ]; +u8 cqn[0x18 ]; + +u8 reserved_at_60[0x20 ]; +}; + +struct mlx5_ifc_destroy_cq_in_bits { +u8 opcode[0x10 ]; +u8 uid[0x10 ]; + +u8 reserved_at_20[0x20 ]; + +u8 reserved_at_40[0x8 ]; +u8 cqn[0x18 ]; + +u8 reserved_at_60[0x20 ]; +}; + +struct mlx5_ifc_alloc_transport_domain_out_bits { +u8 reserved_at_0[0x40 ]; + +u8 reserved_at_40[0x8 ]; +u8 transport_domain[0x18 ]; + +u8 reserved_at_60[0x20 ]; +}; + +struct mlx5_ifc_dealloc_transport_domain_in_bits { +u8 opcode[0x10 ]; +u8 uid[0x10 ]; + +u8 reserved_at_20[0x20 ]; + +u8 reserved_at_40[0x8 ]; +u8 transport_domain[0x18 ]; + +u8 reserved_at_60[0x20 ]; +}; + +struct mlx5_ifc_create_rmp_out_bits { +u8 reserved_at_0[0x40 ]; + +u8 reserved_at_40[0x8 ]; +u8 rmpn[0x18 ]; + +u8 reserved_at_60[0x20 ]; +}; + +struct mlx5_ifc_destroy_rmp_in_bits { +u8 opcode[0x10 ]; +u8 uid[0x10 ]; + +u8 reserved_at_20[0x20 ]; + +u8 reserved_at_40[0x8 ]; +u8 rmpn[0x18 ]; + +u8 reserved_at_60[0x20 ]; +}; + +struct mlx5_ifc_create_sq_out_bits { +u8 reserved_at_0[0x40 ]; + +u8 reserved_at_40[0x8 ]; +u8 sqn[0x18 ]; + +u8 reserved_at_60[0x20 ]; +}; + +struct mlx5_ifc_destroy_sq_in_bits { +u8 opcode[0x10 ]; +u8 uid[0x10 ]; + +u8 reserved_at_20[0x20 ]; + +u8 reserved_at_40[0x8 ]; +u8 sqn[0x18 ]; + +u8 reserved_at_60[0x20 ]; +}; + +struct mlx5_ifc_create_rq_out_bits { +u8 reserved_at_0[0x40 ]; + +u8 reserved_at_40[0x8 ]; +u8 rqn[0x18 ]; + +u8 reserved_at_60[0x20 ]; +}; + +struct mlx5_ifc_destroy_rq_in_bits { +u8 opcode[0x10 ]; +u8 uid[0x10 ]; + +u8 reserved_at_20[0x20 ]; + +u8 reserved_at_40[0x8 ]; +u8 rqn[0x18 ]; + +u8 reserved_at_60[0x20 ]; +}; + +struct mlx5_ifc_create_rqt_out_bits { +u8 reserved_at_0[0x40 ]; + +u8 reserved_at_40[0x8 ]; +u8 rqtn[0x18 ]; + +u8 reserved_at_60[0x20 ]; +}; + +struct mlx5_ifc_destroy_rqt_in_bits { +u8 opcode[0x10 ]; +u8 uid[0x10 ]; + +u8 reserved_at_20[0x20 ]; + +u8 reserved_at_40[0x8 ]; +u8 rqtn[0x18 ]; + +u8 reserved_at_60[0x20 ]; +}; + +struct mlx5_ifc_create_tis_out_bits { +u8 reserved_at_0[0x40 ]; + +u8 reserved_at_40[0x8 ]; +u8 tisn[0x18 ]; + +u8 reserved_at_60[0x20 ]; +}; + +struct mlx5_ifc_destroy_tis_in_bits { +u8 opcode[0x10 ]; +u8 uid[0x10 ]; + +u8 reserved_at_20[0x20 ]; + +u8 reserved_at_40[0x8 ]; +u8 tisn[0x18 ]; + +u8 reserved_at_60[0x20 ]; +}; + +struct mlx5_ifc_alloc_q_counter_out_bits { +u8 reserved_at_0[0x40 ]; + +u8 reserved_at_40[0x18 ]; +u8 counter_set_id[0x8 ]; + +u8 reserved_at_60[0x20 ]; +}; + +struct mlx5_ifc_dealloc_q_counter_in_bits { +u8 opcode[0x10 ]; +u8 reserved_at_10[0x10 ]; + +u8 reserved_at_20[0x20 ]; + +u8 reserved_at_40[0x18 ]; +u8 counter_set_id[0x8 ]; + +u8 reserved_at_60[0x20 ]; +}; + +struct mlx5_ifc_alloc_modify_header_context_out_bits { +u8 reserved_at_0[0x40 ]; + +u8 modify_header_id[0x20 ]; + +u8 reserved_at_60[0x20 ]; +}; + +struct mlx5_ifc_dealloc_modify_header_context_in_bits { +u8 opcode[0x10 ]; +u8 reserved_at_10[0x10 ]; + +u8 reserved_at_20[0x20 ]; + +u8 modify_header_id[0x20 ]; + +u8 reserved_at_60[0x20 ]; +}; + +struct mlx5_ifc_create_scheduling_element_out_bits { +u8 reserved_at_0[0x80 ]; + +u8 scheduling_element_id[0x20 ]; + +u8 reserved_at_a0[0x160 ]; +}; + +struct mlx5_ifc_create_scheduling_element_in_bits { +u8 opcode[0x10 ]; +u8 reserved_at_10[0x10 ]; + +u8 reserved_at_20[0x20 ]; + +u8 scheduling_hierarchy[0x8 ]; +u8 reserved_at_48[0x18 ]; + +u8 reserved_at_60[0x3a0 ]; +}; + +struct mlx5_ifc_destroy_scheduling_element_in_bits { +u8 opcode[0x10 ]; +u8 reserved_at_10[0x10 ]; + +u8 reserved_at_20[0x20 ]; + +u8 scheduling_hierarchy[0x8 ]; +u8 reserved_at_48[0x18 ]; + +u8 scheduling_element_id[0x20 ]; + +u8 reserved_at_80[0x180 ]; +}; + +struct mlx5_ifc_add_vxlan_udp_dport_in_bits { +u8 reserved_at_0[0x60 ]; + +u8 reserved_at_60[0x10 ]; +u8 vxlan_udp_port[0x10 ]; +}; + +struct mlx5_ifc_delete_vxlan_udp_dport_in_bits { +u8 opcode[0x10 ]; +u8 reserved_at_10[0x10 ]; + +u8 reserved_at_20[0x40 ]; + +u8 reserved_at_60[0x10 ]; +u8 vxlan_udp_port[0x10 ]; +}; + +struct mlx5_ifc_set_l2_table_entry_in_bits { +u8 reserved_at_0[0xa0 ]; + +u8 reserved_at_a0[0x8 ]; +u8 table_index[0x18 ]; + +u8 reserved_at_c0[0x140 ]; + +}; + +struct mlx5_ifc_delete_l2_table_entry_in_bits { +u8 opcode[0x10 ]; +u8 reserved_at_10[0x10 ]; + +u8 reserved_at_20[0x80 ]; + +u8 reserved_at_a0[0x8 ]; +u8 table_index[0x18 ]; + +u8 reserved_at_c0[0x140 ]; +}; + +struct mlx5_ifc_create_srq_out_bits { +u8 reserved_at_0[0x40 ]; + +u8 reserved_at_40[0x8 ]; +u8 srqn[0x18 ]; + +u8 reserved_at_60[0x20 ]; +}; + +struct mlx5_ifc_destroy_srq_in_bits { +u8 opcode[0x10 ]; +u8 uid[0x10 ]; + +u8 reserved_at_20[0x20 ]; + +u8 reserved_at_40[0x8 ]; +u8 srqn[0x18 ]; + +u8 reserved_at_60[0x20 ]; +}; + +struct mlx5_ifc_create_xrc_srq_out_bits { +u8 reserved_at_0[0x40 ]; + +u8 reserved_at_40[0x8 ]; +u8 xrc_srqn[0x18 ]; + +u8 reserved_at_60[0x20 ]; +}; + +struct mlx5_ifc_destroy_xrc_srq_in_bits { +u8 opcode[0x10 ]; +u8 uid[0x10 ]; + +u8 reserved_at_20[0x20 ]; + +u8 reserved_at_40[0x8 ]; +u8 xrc_srqn[0x18 ]; + +u8 reserved_at_60[0x20 ]; +}; + +struct mlx5_ifc_create_dct_out_bits { +u8 reserved_at_0[0x40 ]; + +u8 reserved_at_40[0x8 ]; +u8 dctn[0x18 ]; + +u8 ece[0x20 ]; +}; + +struct mlx5_ifc_destroy_dct_in_bits { +u8 opcode[0x10 ]; +u8 uid[0x10 ]; + +u8 reserved_at_20[0x20 ]; + +u8 reserved_at_40[0x8 ]; +u8 dctn[0x18 ]; + +u8 reserved_at_60[0x20 ]; +}; + +struct mlx5_ifc_create_xrq_out_bits { +u8 reserved_at_0[0x40 ]; + +u8 reserved_at_40[0x8 ]; +u8 xrqn[0x18 ]; + +u8 reserved_at_60[0x20 ]; +}; + +struct mlx5_ifc_destroy_xrq_in_bits { +u8 opcode[0x10 ]; +u8 uid[0x10 ]; + +u8 reserved_at_20[0x20 ]; + +u8 reserved_at_40[0x8 ]; +u8 xrqn[0x18 ]; + +u8 reserved_at_60[0x20 ]; +}; + +struct mlx5_ifc_attach_to_mcg_in_bits { +u8 reserved_at_0[0x40 ]; + +u8 reserved_at_40[0x8 ]; +u8 qpn[0x18 ]; + +u8 reserved_at_60[0x20 ]; + +u8 multicast_gid[16 ][0x8 ]; +}; + +struct mlx5_ifc_detach_from_mcg_in_bits { +u8 opcode[0x10 ]; +u8 uid[0x10 ]; + +u8 reserved_at_20[0x20 ]; + +u8 reserved_at_40[0x8 ]; +u8 qpn[0x18 ]; + +u8 reserved_at_60[0x20 ]; + +u8 multicast_gid[16 ][0x8 ]; +}; + +struct mlx5_ifc_alloc_xrcd_out_bits { +u8 reserved_at_0[0x40 ]; + +u8 reserved_at_40[0x8 ]; +u8 xrcd[0x18 ]; + +u8 reserved_at_60[0x20 ]; +}; + +struct mlx5_ifc_dealloc_xrcd_in_bits { +u8 opcode[0x10 ]; +u8 uid[0x10 ]; + +u8 reserved_at_20[0x20 ]; + +u8 reserved_at_40[0x8 ]; +u8 xrcd[0x18 ]; + +u8 reserved_at_60[0x20 ]; +};#endif /* MLX5_IFC_H */
providers/mlx5/mlx5_vfio.c +404 -9
433 lines not on this page · raw message
providers/mlx5/mlx5_vfio.h +8 -0
@@ -9,6 +9,7 @@#include <stddef.h> #include <stdio.h> #include "mlx5.h" +#include "mlx5_ifc.h" #include <infiniband/driver.h> #include <util/interval_set.h>
@@ -303,6 +304,13 @@ struct mlx5_vfio_context {struct mlx5_dv_context_ops *dv_ctx_ops; }; +#define MLX5_MAX_DESTROY_INBOX_SIZE_DW DEVX_ST_SZ_DW(delete_fte_in) +struct mlx5_devx_obj { +struct mlx5dv_devx_obj dv_obj; +uint32_t dinbox[MLX5_MAX_DESTROY_INBOX_SIZE_DW ]; +uint32_t dinlen; +}; +static inline struct mlx5_vfio_device *to_mvfio_dev (struct ibv_device *ibdev) { return container_of(ibdev, struct mlx5_vfio_device, vdev.device);
Signature · 1 line
1.8.3.1
PATCH 18/27 pyverbs: Support DevX UMEM registration
In reply to Yishai HadasFrom: Edward Srouji <[email protected]>
Add support to register a DevX UMEM and UMEM extended, a user memory to be used by the DevX interface for DMA.
--- pyverbs/providers/mlx5/libmlx5.pxd | 21 +++++++-- pyverbs/providers/mlx5/mlx5dv.pxd | 8 ++++ pyverbs/providers/mlx5/mlx5dv.pyx | 93 ++++++++++++++++++++++++++++++++++++-- 3 files changed, 116 insertions(+), 6 deletions(-)
pyverbs/providers/mlx5/libmlx5.pxd +18 -3
@@ -212,6 +212,17 @@ cdef extern from 'infiniband/mlx5dv.h': uint8_t fm_ce_se uint32_t imm + cdef struct mlx5dv_devx_umem: + uint32_t umem_id; + + cdef struct mlx5dv_devx_umem_in: + void *addr + size_t size + uint32_t access + uint64_t pgsz_bitmap + uint64_t comp_mask + + void mlx5dv_set_ctrl_seg(mlx5_wqe_ctrl_seg *seg, uint16_t pi, uint8_t opcode, uint8_t opmod, uint32_t qp_num, uint8_t fm_ce_se, uint8_t ds, uint8_t signature, uint32_t imm)
@@ -298,11 +309,15 @@ cdef extern from 'infiniband/mlx5dv.h': int mlx5dv_map_ah_to_qp(v.ibv_ah *ah, uint32_t qp_num) # DevX APIs - mlx5dv_devx_uar *mlx5dv_devx_alloc_uar(v.ibv_context *context, - uint32_t flags) + mlx5dv_devx_uar *mlx5dv_devx_alloc_uar(v.ibv_context *context, uint32_t flags) void mlx5dv_devx_free_uar(mlx5dv_devx_uar *devx_uar) int mlx5dv_devx_general_cmd(v.ibv_context *context, const void *in_, - size_t inlen, void *out, size_t outlen); + size_t inlen, void *out, size_t outlen) + mlx5dv_devx_umem *mlx5dv_devx_umem_reg(v.ibv_context *ctx, void *addr, + size_t size, unsigned long access) + mlx5dv_devx_umem *mlx5dv_devx_umem_reg_ex(v.ibv_context *ctx, + mlx5dv_devx_umem_in *umem_in) + int mlx5dv_devx_umem_dereg(mlx5dv_devx_umem *umem) # Mkey setters void mlx5dv_wr_mkey_configure(mlx5dv_qp_ex *mqp, mlx5dv_mkey *mkey,
pyverbs/providers/mlx5/mlx5dv.pxd +8 -0
@@ -11,6 +11,8 @@ from pyverbs.cq cimport CQEX cdef class Mlx5Context(Context): + cdef object devx_umems + cdef add_ref(self, obj) cpdef close(self) cdef class Mlx5DVContextAttr(PyverbsObject):
@@ -69,3 +71,9 @@ cdef class Wqe(PyverbsCM): cdef void *addr cdef int is_user_addr cdef object segments + +cdef class Mlx5UMEM(PyverbsCM): + cdef dv.mlx5dv_devx_umem *umem + cdef Context context + cdef void *addr + cdef object is_user_addr
pyverbs/providers/mlx5/mlx5dv.pyx +90 -3
127 lines not on this page · raw message
Signature · 1 line
1.8.3.1
PATCH 19/27 pyverbs/mlx5: Support EQN querying
In reply to Yishai HadasFrom: Edward Srouji <[email protected]>
Add the ability to retrieve a device's EQ number which relates to a given input vector.
The query is done over DevX.
--- pyverbs/providers/mlx5/libmlx5.pxd | 1 + pyverbs/providers/mlx5/mlx5dv.pyx | 12 ++++++++++++ 2 files changed, 13 insertions(+)
pyverbs/providers/mlx5/libmlx5.pxd +1 -0
@@ -318,6 +318,7 @@ cdef extern from 'infiniband/mlx5dv.h': mlx5dv_devx_umem *mlx5dv_devx_umem_reg_ex(v.ibv_context *ctx, mlx5dv_devx_umem_in *umem_in) int mlx5dv_devx_umem_dereg(mlx5dv_devx_umem *umem) + int mlx5dv_devx_query_eqn(v.ibv_context *context, uint32_t vector, uint32_t *eqn) # Mkey setters void mlx5dv_wr_mkey_configure(mlx5dv_qp_ex *mqp, mlx5dv_mkey *mkey,
pyverbs/providers/mlx5/mlx5dv.pyx +12 -0
@@ -262,6 +262,18 @@ cdef class Mlx5Context(Context): free(clock_info) return ns_time + def devx_query_eqn(self, vector): + """ + Query EQN for a given vector id. + :param vector: Completion vector number + :return: The device EQ number which relates to the given input vector + """ + cdef uint32_t eqn + rc = dv.mlx5dv_devx_query_eqn(self.context, vector, &eqn) + if rc: + raise PyverbsRDMAError('Failed to query EQN', rc) + return eqn + cdef add_ref(self, obj): try: Context.add_ref(self, obj)
Signature · 1 line
1.8.3.1
PATCH 20/27 pyverbs/mlx5: Support more DevX objects
In reply to Yishai HadasFrom: Edward Srouji <[email protected]>
A DevX object represents some underlay firmware object, the input command to create it is some raw data given by the user application which should match the device specification.
A new class Mlx5DevxObj was added that would present any DevX object that could be creating using the create_obj DevX API.
The command's output is stored in the object's instance, and the object can be modified/queried and destroyed.
--- Documentation/pyverbs.md | 34 ++++++++++++ pyverbs/providers/mlx5/libmlx5.pxd | 7 +++ pyverbs/providers/mlx5/mlx5dv.pxd | 6 +++ pyverbs/providers/mlx5/mlx5dv.pyx | 103 ++++++++++++++++++++++++++++++++++++- 4 files changed, 149 insertions(+), 1 deletion(-)
Documentation/pyverbs.md +34 -0
@@ -745,3 +745,37 @@ Below is the output when printing the spec. Dst mac : de:de:de:00:de:de mask: ff:ff:ff:ff:ff:ff Ether type : 8451 mask: 65535 Vlan tag : 8961 mask: 65535 + + +##### MLX5 DevX Objects +A DevX object represents some underlay firmware object, the input command to +create it is some raw data given by the user application which should match the +device specification. +Upon successful creation, the output buffer includes the raw data from the device +according to its specification and is stored in the Mlx5DevxObj instance. This +data can be used as part of related firmware commands to this object. +In addition to creation, the user can query/modify and destroy the object. + +Although weakrefs and DevX objects closure are added and handled by +Pyverbs, the users must manually close these objects when finished, and +should not let them be handled by the GC, or by closing the Mlx5Context directly, +since there's no guarantee that the DevX objects are closed in the correct order, +because Mlx5DevxObj is a general class that can be any of the device's available +objects. +But Pyverbs does guarantee to close DevX UARs and UMEMs in order, and after +closing the other DevX objects. + +The following code snippet shows how to allocate and destroy a PD object over DevX. +```python +from pyverbs.providers.mlx5.mlx5dv import Mlx5Context, Mlx5DVContextAttr, Mlx5DevxObj +import pyverbs.providers.mlx5.mlx5_enums as dve +import struct + +attr = Mlx5DVContextAttr(dve.MLX5DV_CONTEXT_FLAGS_DEVX) +ctx = Mlx5Context(attr, 'rocep8s0f0') +MLX5_CMD_OP_ALLOC_PD = 0x800 +MLX5_CMD_OP_ALLOC_PD_OUTLEN = 0x10 +cmd_in = struct.pack('!H14s', MLX5_CMD_OP_ALLOC_PD, bytes(0)) +pd = Mlx5DevxObj(ctx, cmd_in, MLX5_CMD_OP_ALLOC_PD_OUTLEN) +pd.close() +```
pyverbs/providers/mlx5/libmlx5.pxd +7 -0
@@ -319,6 +319,13 @@ cdef extern from 'infiniband/mlx5dv.h': mlx5dv_devx_umem_in *umem_in) int mlx5dv_devx_umem_dereg(mlx5dv_devx_umem *umem) int mlx5dv_devx_query_eqn(v.ibv_context *context, uint32_t vector, uint32_t *eqn) + mlx5dv_devx_obj *mlx5dv_devx_obj_create(v.ibv_context *context, const void *_in, + size_t inlen, void *out, size_t outlen) + int mlx5dv_devx_obj_query(mlx5dv_devx_obj *obj, const void *in_, + size_t inlen, void *out, size_t outlen) + int mlx5dv_devx_obj_modify(mlx5dv_devx_obj *obj, const void *in_, + size_t inlen, void *out, size_t outlen) + int mlx5dv_devx_obj_destroy(mlx5dv_devx_obj *obj) # Mkey setters void mlx5dv_wr_mkey_configure(mlx5dv_qp_ex *mqp, mlx5dv_mkey *mkey,
pyverbs/providers/mlx5/mlx5dv.pxd +6 -0
@@ -12,6 +12,7 @@ from pyverbs.cq cimport CQEX cdef class Mlx5Context(Context): cdef object devx_umems + cdef object devx_objs cdef add_ref(self, obj) cpdef close(self)
@@ -77,3 +78,8 @@ cdef class Mlx5UMEM(PyverbsCM): cdef Context context cdef void *addr cdef object is_user_addr + +cdef class Mlx5DevxObj(PyverbsCM): + cdef dv.mlx5dv_devx_obj *obj + cdef Context context + cdef object out_view
pyverbs/providers/mlx5/mlx5dv.pyx +102 -1
127 lines not on this page · raw message
Signature · 1 line
1.8.3.1
PATCH 21/27 pyverbs: Add auxiliary memory functions
In reply to Yishai HadasFrom: Edward Srouji <[email protected]>
Add some auxiliary functions to mem_alloc module to expose to Python users the ability to read/write from/to memory, with chuncks of 4/8 bytes.
In addition, expose udma memory barriers and the ability to write on MMIO.
--- pyverbs/CMakeLists.txt | 7 +++++++ pyverbs/dma_util.pyx | 25 +++++++++++++++++++++++++ pyverbs/mem_alloc.pyx | 46 +++++++++++++++++++++++++++++++++++++++++++++- 3 files changed, 77 insertions(+), 1 deletion(-) create mode 100644 pyverbs/dma_util.pyx
pyverbs/CMakeLists.txt +7 -0
@@ -12,6 +12,12 @@ else() set(DMABUF_ALLOC dmabuf_alloc_stub.c) endif() +if (HAVE_COHERENT_DMA) + set(DMA_UTIL dma_util.pyx) +else() + set(DMA_UTIL "") +endif() + rdma_cython_module(pyverbs "" addr.pyx base.pyx
@@ -19,6 +25,7 @@ rdma_cython_module(pyverbs "" cmid.pyx cq.pyx device.pyx + ${DMA_UTIL} dmabuf.pyx ${DMABUF_ALLOC} enums.pyx
pyverbs/dma_util.pyx new file +25 -0
@@ -0,0 +1,25 @@ +# SPDX-License-Identifier: (GPL-2.0 OR Linux-OpenIB) +# Copyright (c) 2021 Nvidia, Inc. All rights reserved. See COPYING file + +#cython: language_level=3 + +from libc.stdint cimport uintptr_t, uint64_t + +cdef extern from 'util/udma_barrier.h': + cdef void udma_to_device_barrier() + cdef void udma_from_device_barrier() + +cdef extern from 'util/mmio.h': + cdef void mmio_write64_be(void *addr, uint64_t val) + + +def udma_to_dev_barrier(): + udma_to_device_barrier() + + +def udma_from_dev_barrier(): + udma_from_device_barrier() + + +def mmio_write64_as_be(addr, val): + mmio_write64_be(<void*><uintptr_t> addr, val)
pyverbs/mem_alloc.pyx +45 -1
64 lines not on this page · raw message
Signature · 1 line
1.8.3.1
PATCH 22/27 pyverbs/mlx5: Add support to extract mlx5dv objects
In reply to Yishai HadasFrom: Edward Srouji <[email protected]>
Support extraction of mlx5dv objects from ibv objects. This allows the users to access the object numbers and much more attributes needed for some usage such as DevX.
Currently there's a support for the following objects: PD, CQ, QP and SRQ.
--- pyverbs/pd.pyx | 4 + pyverbs/providers/mlx5/CMakeLists.txt | 1 + pyverbs/providers/mlx5/libmlx5.pxd | 52 ++++++++ pyverbs/providers/mlx5/mlx5dv_enums.pxd | 17 +++ pyverbs/providers/mlx5/mlx5dv_objects.pxd | 28 ++++ pyverbs/providers/mlx5/mlx5dv_objects.pyx | 214 ++++++++++++++++++++++++++++++ 6 files changed, 316 insertions(+) create mode 100644 pyverbs/providers/mlx5/mlx5dv_objects.pxd create mode 100644 pyverbs/providers/mlx5/mlx5dv_objects.pyx
pyverbs/pd.pyx +4 -0
@@ -139,6 +139,10 @@ cdef class PD(PyverbsCM): def handle(self): return self.pd.handle + @property + def pd(self): + return <object>self.pd + cdef void *pd_alloc(v.ibv_pd *pd, void *pd_context, size_t size, size_t alignment, v.uint64_t resource_type):
pyverbs/providers/mlx5/CMakeLists.txt +1 -0
@@ -11,5 +11,6 @@ rdma_cython_module(pyverbs/providers/mlx5 mlx5 mlx5_enums.pyx mlx5dv_flow.pyx mlx5dv_mkey.pyx + mlx5dv_objects.pyx mlx5dv_sched.pyx )
pyverbs/providers/mlx5/libmlx5.pxd +52 -0
70 lines not on this page · raw message
pyverbs/providers/mlx5/mlx5dv_enums.pxd +17 -0
@@ -176,6 +176,23 @@ cdef extern from 'infiniband/mlx5dv.h': MLX5DV_DR_DOMAIN_TYPE_NIC_TX MLX5DV_DR_DOMAIN_TYPE_FDB + cpdef enum mlx5dv_qp_comp_mask: + MLX5DV_QP_MASK_UAR_MMAP_OFFSET + MLX5DV_QP_MASK_RAW_QP_HANDLES + MLX5DV_QP_MASK_RAW_QP_TIR_ADDR + + cpdef enum mlx5dv_srq_comp_mask: + MLX5DV_SRQ_MASK_SRQN + + cpdef enum mlx5dv_obj_type: + MLX5DV_OBJ_QP + MLX5DV_OBJ_CQ + MLX5DV_OBJ_SRQ + MLX5DV_OBJ_RWQ + MLX5DV_OBJ_DM + MLX5DV_OBJ_AH + MLX5DV_OBJ_PD + cpdef unsigned long long MLX5DV_RES_TYPE_QP cpdef unsigned long long MLX5DV_RES_TYPE_RWQ cpdef unsigned long long MLX5DV_RES_TYPE_DBR
pyverbs/providers/mlx5/mlx5dv_objects.pxd new file +28 -0
@@ -0,0 +1,28 @@ +# SPDX-License-Identifier: (GPL-2.0 OR Linux-OpenIB) +# Copyright (c) 2021 Nvidia, Inc. All rights reserved. See COPYING file + +#cython: language_level=3 + +cimport pyverbs.providers.mlx5.libmlx5 as dv +from pyverbs.base cimport PyverbsObject + + +cdef class Mlx5DvPD(PyverbsObject): + cdef dv.mlx5dv_pd dv_pd + +cdef class Mlx5DvCQ(PyverbsObject): + cdef dv.mlx5dv_cq dv_cq + +cdef class Mlx5DvQP(PyverbsObject): + cdef dv.mlx5dv_qp dv_qp + +cdef class Mlx5DvSRQ(PyverbsObject): + cdef dv.mlx5dv_srq dv_srq + +cdef class Mlx5DvObj(PyverbsObject): + cdef dv.mlx5dv_obj obj + cdef Mlx5DvCQ dv_cq + cdef Mlx5DvQP dv_qp + cdef Mlx5DvPD dv_pd + cdef Mlx5DvSRQ dv_srq +
pyverbs/providers/mlx5/mlx5dv_objects.pyx new file +214 -0
214 lines not on this page · raw message
Signature · 1 line
1.8.3.1
PATCH 23/27 pyverbs/mlx5: Wrap mlx5_cqe64 struct and add enums
In reply to Yishai HadasFrom: Edward Srouji <[email protected]>
Add a Mlx5Cqe64 class that wraps mlx5_cqe64 C struct, and provide an easy way to users to get/set its fields.
In addition expose related enums.
Both are relevant and necessary for DevX data path.
--- pyverbs/providers/mlx5/libmlx5.pxd | 14 +++++++ pyverbs/providers/mlx5/mlx5dv.pxd | 3 ++ pyverbs/providers/mlx5/mlx5dv.pyx | 71 +++++++++++++++++++++++++++++++++ pyverbs/providers/mlx5/mlx5dv_enums.pxd | 22 ++++++++++ 4 files changed, 110 insertions(+)
pyverbs/providers/mlx5/libmlx5.pxd +14 -0
@@ -273,12 +273,26 @@ cdef extern from 'infiniband/mlx5dv.h': qp qp srq srq + cdef struct mlx5_cqe64: + uint16_t wqe_id + uint32_t imm_inval_pkey + uint32_t byte_cnt + uint64_t timestamp + uint16_t wqe_counter + uint8_t signature + uint8_t op_own + void mlx5dv_set_ctrl_seg(mlx5_wqe_ctrl_seg *seg, uint16_t pi, uint8_t opcode, uint8_t opmod, uint32_t qp_num, uint8_t fm_ce_se, uint8_t ds, uint8_t signature, uint32_t imm) void mlx5dv_set_data_seg(mlx5_wqe_data_seg *seg, uint32_t length, uint32_t lkey, uintptr_t address) + uint8_t mlx5dv_get_cqe_owner(mlx5_cqe64 *cqe) + void mlx5dv_set_cqe_owner(mlx5_cqe64 *cqe, uint8_t val) + uint8_t mlx5dv_get_cqe_se(mlx5_cqe64 *cqe) + uint8_t mlx5dv_get_cqe_format(mlx5_cqe64 *cqe) + uint8_t mlx5dv_get_cqe_opcode(mlx5_cqe64 *cqe) bool mlx5dv_is_supported(v.ibv_device *device) v.ibv_context* mlx5dv_open_device(v.ibv_device *device, mlx5dv_context_attr *attr)
pyverbs/providers/mlx5/mlx5dv.pxd +3 -0
@@ -83,3 +83,6 @@ cdef class Mlx5DevxObj(PyverbsCM): cdef dv.mlx5dv_devx_obj *obj cdef Context context cdef object out_view + +cdef class Mlx5Cqe64(PyverbsObject): + cdef dv.mlx5_cqe64 *cqe
pyverbs/providers/mlx5/mlx5dv.pyx +71 -0
74 lines not on this page · raw message
pyverbs/providers/mlx5/mlx5dv_enums.pxd +22 -0
@@ -193,6 +193,28 @@ cdef extern from 'infiniband/mlx5dv.h': MLX5DV_OBJ_AH MLX5DV_OBJ_PD + cpdef enum: + MLX5_RCV_DBR + MLX5_SND_DBR + + cpdef enum: + MLX5_CQE_OWNER_MASK + MLX5_CQE_REQ + MLX5_CQE_RESP_WR_IMM + MLX5_CQE_RESP_SEND + MLX5_CQE_RESP_SEND_IMM + MLX5_CQE_RESP_SEND_INV + MLX5_CQE_RESIZE_CQ + MLX5_CQE_NO_PACKET + MLX5_CQE_SIG_ERR + MLX5_CQE_REQ_ERR + MLX5_CQE_RESP_ERR + MLX5_CQE_INVALID + + cpdef enum: + MLX5_SEND_WQE_BB + MLX5_SEND_WQE_SHIFT + cpdef unsigned long long MLX5DV_RES_TYPE_QP cpdef unsigned long long MLX5DV_RES_TYPE_RWQ cpdef unsigned long long MLX5DV_RES_TYPE_DBR
Signature · 1 line
1.8.3.1
PATCH 24/27 tests: Add MAC address to the tests' args
In reply to Yishai HadasFrom: Edward Srouji <[email protected]>
Append the MAC address of the relevant interface to the tests' arguments.
--- tests/base.py | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-)
tests/base.py +8 -5
@@ -131,6 +131,7 @@ class RDMATestCase(unittest.TestCase): self.pkey_index = pkey_index self.gid_type = gid_type if gid_index is None else None self.ip_addr = None + self.mac_addr = None self.pre_environment = {} self.server = None self.client = None
@@ -168,13 +169,14 @@ class RDMATestCase(unittest.TestCase): return out.decode().split('\n')[0] @staticmethod - def get_ip_address(ifname): + def get_ip_mac_address(ifname): out = subprocess.check_output(['ip', '-j', 'addr', 'show', ifname]) loaded_json = json.loads(out.decode()) interface = loaded_json[0]['addr_info'][0]['local'] + mac = loaded_json[0]['address'] if 'fe80::' in interface: interface = interface + '%' + ifname - return interface + return interface, mac def setUp(self): """
@@ -242,11 +244,11 @@ class RDMATestCase(unittest.TestCase): continue net_name = self.get_net_name(dev) try: - ip_addr = self.get_ip_address(net_name) + ip_addr, mac_addr = self.get_ip_mac_address(net_name) except (KeyError, IndexError): - self.args.append([dev, port, idx, None]) + self.args.append([dev, port, idx, None, None]) else: - self.args.append([dev, port, idx, ip_addr]) + self.args.append([dev, port, idx, ip_addr, mac_addr]) def _add_gids_per_device(self, ctx, dev): self._add_gids_per_port(ctx, dev, self.ib_port)
@@ -264,6 +266,7 @@ class RDMATestCase(unittest.TestCase): self.ib_port = args[1] self.gid_index = args[2] self.ip_addr = args[3] + self.mac_addr = args[4] def set_env_variable(self, var, value): """
Signature · 1 line
1.8.3.1
PATCH 25/27 tests: Add mlx5 DevX data path test
In reply to Yishai HadasFrom: Edward Srouji <[email protected]>
Extend the mlx5 tests' infrastructure to support mlx5 DevX objects creation.
Add a data path test that creates all the needed resources to be able to create an RC QP, modify it to RTS state and does SEMD_IMM traffic over DevX objects.
The tests does validate the data including the received "immediate" value, and checks the validity of the received CQEs.
--- tests/CMakeLists.txt | 2 + tests/base.py | 1 + tests/mlx5_base.py | 460 +++++++++++++++++++- tests/mlx5_prm_structs.py | 1046 +++++++++++++++++++++++++++++++++++++++++++++ tests/test_mlx5_devx.py | 23 + 5 files changed, 1529 insertions(+), 3 deletions(-) create mode 100644 tests/mlx5_prm_structs.py create mode 100644 tests/test_mlx5_devx.py
tests/CMakeLists.txt +2 -0
@@ -8,6 +8,7 @@ rdma_python_test(tests base_rdmacm.py efa_base.py mlx5_base.py + mlx5_prm_structs.py rdmacm_utils.py test_addr.py test_cq.py
@@ -20,6 +21,7 @@ rdma_python_test(tests test_fork.py test_mlx5_cq.py test_mlx5_dc.py + test_mlx5_devx.py test_mlx5_dm_ops.py test_mlx5_dr.py test_mlx5_flow.py
tests/base.py +1 -0
@@ -30,6 +30,7 @@ PATH_MTU = e.IBV_MTU_1024 MAX_DEST_RD_ATOMIC = 1 NUM_OF_PROCESSES = 2 MC_IP_PREFIX = '230' +MAX_RDMA_ATOMIC = 20 MAX_RD_ATOMIC = 1 MIN_RNR_TIMER =12 RETRY_CNT = 7
tests/mlx5_base.py +457 -3
480 lines not on this page · raw message
tests/mlx5_prm_structs.py new file +1046 -0
1,046 lines not on this page · raw message
tests/test_mlx5_devx.py new file +23 -0
@@ -0,0 +1,23 @@ +# SPDX-License-Identifier: (GPL-2.0 OR Linux-OpenIB) +# Copyright (c) 2021 Nvidia Inc. All rights reserved. See COPYING file + +""" +Test module for mlx5 DevX. +""" + +from tests.mlx5_base import Mlx5DevxRcResources, Mlx5DevxTrafficBase + + +class Mlx5DevxRcTrafficTest(Mlx5DevxTrafficBase): + """ + Test various functionality of mlx5 DevX objects + """ + + def test_devx_rc_qp_send_imm_traffic(self): + """ + Creates two DevX RC QPs and modifies them to RTS state. + Then does SEND_IMM traffic. + """ + self.create_players(Mlx5DevxRcResources) + # Send traffic + self.send_imm_traffic()
Signature · 1 line
1.8.3.1
PATCH 26/27 pyverbs/mlx5: Support mlx5 devices over VFIO
In reply to Yishai HadasFrom: Edward Srouji <[email protected]>
Support opening an mlx5 device and creating a context over VFIO by adding Mlx5VfioContext class based on Mlx5Context to allow using DevX APIs.
--- pyverbs/providers/mlx5/CMakeLists.txt | 3 +- pyverbs/providers/mlx5/libmlx5.pxd | 8 +++ pyverbs/providers/mlx5/mlx5_vfio.pxd | 15 +++++ pyverbs/providers/mlx5/mlx5_vfio.pyx | 116 ++++++++++++++++++++++++++++++++ pyverbs/providers/mlx5/mlx5dv.pxd | 3 + pyverbs/providers/mlx5/mlx5dv_enums.pxd | 3 + 6 files changed, 147 insertions(+), 1 deletion(-) create mode 100644 pyverbs/providers/mlx5/mlx5_vfio.pxd create mode 100644 pyverbs/providers/mlx5/mlx5_vfio.pyx
pyverbs/providers/mlx5/CMakeLists.txt +2 -1
@@ -7,8 +7,9 @@ rdma_cython_module(pyverbs/providers/mlx5 mlx5 dr_matcher.pyx dr_rule.pyx dr_table.pyx - mlx5dv.pyx mlx5_enums.pyx + mlx5_vfio.pyx + mlx5dv.pyx mlx5dv_flow.pyx mlx5dv_mkey.pyx mlx5dv_objects.pyx
pyverbs/providers/mlx5/libmlx5.pxd +8 -0
@@ -223,6 +223,11 @@ cdef extern from 'infiniband/mlx5dv.h': uint64_t pgsz_bitmap uint64_t comp_mask + cdef struct mlx5dv_vfio_context_attr: + const char *pci_name + uint32_t flags + uint64_t comp_mask + cdef struct mlx5dv_pd: uint32_t pdn uint64_t comp_mask
@@ -372,6 +377,9 @@ cdef extern from 'infiniband/mlx5dv.h': uint64_t device_timestamp) int mlx5dv_get_clock_info(v.ibv_context *ctx_in, mlx5dv_clock_info *clock_info) int mlx5dv_map_ah_to_qp(v.ibv_ah *ah, uint32_t qp_num) + v.ibv_device **mlx5dv_get_vfio_device_list(mlx5dv_vfio_context_attr *attr) + int mlx5dv_vfio_get_events_fd(v.ibv_context *ibctx) + int mlx5dv_vfio_process_events(v.ibv_context *context) # DevX APIs mlx5dv_devx_uar *mlx5dv_devx_alloc_uar(v.ibv_context *context, uint32_t flags)
pyverbs/providers/mlx5/mlx5_vfio.pxd new file +15 -0
@@ -0,0 +1,15 @@ +# SPDX-License-Identifier: (GPL-2.0 OR Linux-OpenIB) +# Copyright (c) 2021 Nvidia, Inc. All rights reserved. See COPYING file + +#cython: language_level=3 + +from pyverbs.providers.mlx5.mlx5dv cimport Mlx5Context +cimport pyverbs.providers.mlx5.libmlx5 as dv +from pyverbs.base cimport PyverbsObject + + +cdef class Mlx5VfioContext(Mlx5Context): + pass + +cdef class Mlx5VfioAttr(PyverbsObject): + cdef dv.mlx5dv_vfio_context_attr attr
pyverbs/providers/mlx5/mlx5_vfio.pyx new file +116 -0
116 lines not on this page · raw message
pyverbs/providers/mlx5/mlx5dv.pxd +3 -0
@@ -86,3 +86,6 @@ cdef class Mlx5DevxObj(PyverbsCM): cdef class Mlx5Cqe64(PyverbsObject): cdef dv.mlx5_cqe64 *cqe + +cdef class Mlx5VfioAttr(PyverbsObject): + cdef dv.mlx5dv_vfio_context_attr attr
pyverbs/providers/mlx5/mlx5dv_enums.pxd +3 -0
@@ -215,6 +215,9 @@ cdef extern from 'infiniband/mlx5dv.h': MLX5_SEND_WQE_BB MLX5_SEND_WQE_SHIFT + cpdef enum mlx5dv_vfio_context_attr_flags: + MLX5DV_VFIO_CTX_FLAGS_INIT_LINK_DOWN + cpdef unsigned long long MLX5DV_RES_TYPE_QP cpdef unsigned long long MLX5DV_RES_TYPE_RWQ cpdef unsigned long long MLX5DV_RES_TYPE_DBR
Signature · 1 line
1.8.3.1
PATCH 27/27 tests: Add a test for mlx5 over VFIO
In reply to Yishai HadasFrom: Edward Srouji <[email protected]>
Add a test that opens an mlx5 vfio context, creates two DevX QPs on it, and modifies them to RTS state. Then performs traffic with SEND_IMM. An additional cmd line argument was added for the tests, to allow users to pass a PCI device instead of an RDMA device for this test.
--- tests/CMakeLists.txt | 1 + tests/args_parser.py | 5 +++ tests/test_mlx5_vfio.py | 104 ++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 110 insertions(+) create mode 100644 tests/test_mlx5_vfio.py
tests/CMakeLists.txt +1 -0
@@ -36,6 +36,7 @@ rdma_python_test(tests test_mlx5_uar.py test_mlx5_udp_sport.py test_mlx5_var.py + test_mlx5_vfio.py test_mr.py test_odp.py test_pd.py
tests/args_parser.py +5 -0
@@ -1,5 +1,6 @@ # SPDX-License-Identifier: (GPL-2.0 OR Linux-OpenIB) # Copyright (c) 2020 Kamal Heib <[email protected]>, All rights reserved. See COPYING file +# Copyright (c) 2021 Nvidia, Inc. All rights reserved. See COPYING file import argparse import sys
@@ -16,6 +17,10 @@ class ArgsParser(object): parser = argparse.ArgumentParser() parser.add_argument('--dev', help='RDMA device to run the tests on') + parser.add_argument('--pci-dev', + help='PCI device to run the tests on, which is ' + 'needed by some tests where the RDMA device is ' + 'not available (e.g. VFIO)') parser.add_argument('--port', help='Use port <port> of RDMA device', type=int, default=1)
tests/test_mlx5_vfio.py new file +104 -0
104 lines not on this page · raw message
Signature · 1 line
1.8.3.1
Quoted 115 lines · Yishai Hadas
This series introduces mlx5 user space driver over VFIO.
This enables an application to take full ownership on the opened device and run any firmware command (e.g. port up/down) without any concern to hurt someone else.
The application look and feel is like regular RDMA application over DEVX, it uses verbs API to open/close a device and then mostly uses DEVX APIs to interact with the device.
To achieve it, few mlx5 DV APIs were introduced, it includes:
- An API to getibv_devicefor a given mlx5 PCI name.
- APIs to manage device specific events.Detailed man pages were added to describe the expected usage of those APIs.
The mlx5 VFIO driver implemented the basic verbs APIs as of managing MR/PD and the DEVX APIs which are required to write an RDMA application.
The VFIO uAPIs were used to setup mlx5 vfio context by reading the device initialization segment, setup DMA and enables the device command interface.
In addition, the series includes pyverbs stuff which runs DEVX like testing over RDMA and VFIO mlx5 devices.
Some extra documentation of the benefits and the motivation to use VFIO can be found here [1].
PR was sent [2].
[1] https://www.kernel.org/doc/html/latest/driver-api/vfio.html
[2] https://github.com/linux-rdma/rdma-core/pull/1034Yishai
Edward Srouji (10):
pyverbs: Support DevX UMEM registration pyverbs/mlx5: Support EQN querying pyverbs/mlx5: Support more DevX objects
pyverbs: Add auxiliary memory functions pyverbs/mlx5: Add support to extract mlx5dv objects pyverbs/mlx5: Wrapmlx5_cqe64struct and add enums
tests: Add MAC address to the tests' args
tests: Add mlx5 DevX data path test pyverbs/mlx5: Support mlx5 devices over VFIO
tests: Add a test for mlx5 over VFIOMaor Gottlieb (1):
mlx5: Enable debug functionality for vfioMark Zhang (5):
util: Addinterval_setsupport
mlx5: Support fast teardown over vfio
mlx5: VFIOpoll_healthsupport
mlx5: Set DV context ops
mlx5: Implement mlx5dvdevx_objAPIs over vfioYishai Hadas (11):
Update kernel headers
mlx5: Introducemlx5dv_get_vfio_device_list()
verbs: Enableverbs_open_device()to work over non sysfs devices
mlx5: Setup mlx5 vfio context
mlx5: Addmlx5_vfio_cmd_exec()support
mlx5: vfio setup function support
mlx5: vfio setup basic caps
mlx5: Enable interrupt command mode over vfio
mlx5: Introduce vfio APIs to process events
mlx5: Implement basic verbs operation for PD and MR over vfio
mlx5: Support initial DEVX/DV APIs over vfioDocumentation/pyverbs.md | 34 + debian/ibverbs-providers.symbols | 4 + kernel-headers/CMakeLists.txt | 4 + kernel-headers/linux/vfio.h | 1374 ++++++++ libibverbs/device.c | 39 +- libibverbs/sysfs.c | 5 + providers/mlx5/CMakeLists.txt | 3 +- providers/mlx5/dr_rule.c | 10 +- providers/mlx5/libmlx5.map | 7 + providers/mlx5/man/CMakeLists.txt | 3 + .../mlx5/man/mlx5dv_get_vfio_device_list.3.md | 64 + providers/mlx5/man/mlx5dv_vfio_get_events_fd.3.md | 41 + providers/mlx5/man/mlx5dv_vfio_process_events.3.md | 43 + providers/mlx5/mlx5.c | 376 ++- providers/mlx5/mlx5.h | 187 +- providers/mlx5/mlx5_ifc.h | 1206 ++++++- providers/mlx5/mlx5_vfio.c | 3379 ++++++++++++++++++++ providers/mlx5/mlx5_vfio.h | 329 ++ providers/mlx5/mlx5dv.h | 25 + providers/mlx5/verbs.c | 966 +++++- pyverbs/CMakeLists.txt | 7 + pyverbs/dma_util.pyx | 25 + pyverbs/mem_alloc.pyx | 46 +- pyverbs/pd.pyx | 4 + pyverbs/providers/mlx5/CMakeLists.txt | 4 +- pyverbs/providers/mlx5/libmlx5.pxd | 103 +- pyverbs/providers/mlx5/mlx5_vfio.pxd | 15 + pyverbs/providers/mlx5/mlx5_vfio.pyx | 116 + pyverbs/providers/mlx5/mlx5dv.pxd | 20 + pyverbs/providers/mlx5/mlx5dv.pyx | 277 +- pyverbs/providers/mlx5/mlx5dv_enums.pxd | 42 + pyverbs/providers/mlx5/mlx5dv_objects.pxd | 28 + pyverbs/providers/mlx5/mlx5dv_objects.pyx | 214 ++ tests/CMakeLists.txt | 3 + tests/args_parser.py | 5 + tests/base.py | 14 +- tests/mlx5_base.py | 460 ++- tests/mlx5_prm_structs.py | 1046 ++++++ tests/test_mlx5_devx.py | 23 + tests/test_mlx5_vfio.py | 104 + util/CMakeLists.txt | 2 + util/interval_set.c | 208 ++ util/interval_set.h | 77 + util/util.h | 5 + 44 files changed, 10650 insertions(+), 297 deletions(-)
Quoted 17 lines
create mode 100644
kernel-headers/linux/vfio.h
create mode 100644providers/mlx5/man/mlx5dv_get_vfio_device_list.3.mdcreate mode 100644providers/mlx5/man/mlx5dv_vfio_get_events_fd.3.mdcreate mode 100644providers/mlx5/man/mlx5dv_vfio_process_events.3.mdcreate mode 100644providers/mlx5/mlx5_vfio.c
create mode 100644providers/mlx5/mlx5_vfio.h
create mode 100644 pyverbs/dma_util.pyx
create mode 100644 pyverbs/providers/mlx5/mlx5_vfio.pxd create mode 100644 pyverbs/providers/mlx5/mlx5_vfio.pyx create mode 100644 pyverbs/providers/mlx5/mlx5dv_objects.pxd create mode 100644 pyverbs/providers/mlx5/mlx5dv_objects.pyx create mode 100644tests/mlx5_prm_structs.py
create mode 100644tests/test_mlx5_devx.py
create mode 100644tests/test_mlx5_vfio.py
create mode 100644util/interval_set.c
create mode 100644util/interval_set.h
The PR was merged.
Yishai