vdpa/mlx5: Optimze MKEY operations
Get series with b4
b4 am [email protected]Versions newest first · linked by author and title
- v2 Latest7 patches 3 repliesIn review
- v17 patches 11 repliesSuperseded
Changes in v2 from the cover letter
- Swapped flex array usage for plain zero length array in first patch.
- Updated code to use Scope-Based Cleanup Helpers where appropriate (only second patch).
- Added macro define for MTT alignment in first patch.
- Improved commit messages/comments based on review comments.
- Removed extra newlines.
---
Patches in v2 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/7 | vdpa/mlx5: Create direct MKEYs in paralleldrivers/ | +98 | 0 replies | |
| 2/7 | vdpa/mlx5: Delete direct MKEYs in paralleldrivers/ | +64 | 0 replies | |
| 3/7 | vdpa/mlx5: Rename functiondrivers/ drivers/ drivers/ | +6 | 0 replies | |
| 4/7 | vdpa/mlx5: Extract mr members in own resource structdrivers/ drivers/ drivers/ +1 more | +44 | 0 replies | |
| 5/7 | vdpa/mlx5: Rename mr_mtx -> lockdrivers/ drivers/ drivers/ +1 more | +16 | 0 replies | |
| 6/7 | vdpa/mlx5: Introduce init/destroy for MR resourcesdrivers/ drivers/ drivers/ +1 more | +26 | 0 replies | |
| 7/7 | vdpa/mlx5: Postpone MR deletiondrivers/ drivers/ drivers/ | +64 | 0 replies |
Reviews trailers per patch
All 7 patches have a Reviewed-by or Acked-by.
| Patch | Cosmin Ratiu | Eugenio Pérez |
|---|---|---|
| 1/7 | R-b | |
| 2/7 | R-b | A-b |
| 3/7 | R-b | |
| 4/7 | R-b | A-b |
| 5/7 | R-b | A-b |
| 6/7 | R-b | |
| 7/7 | R-b |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.
Cover letter [PATCH vhost v2 0/7] vdpa/mlx5: Optimze MKEY operations
This series improves the time of .set_map() operations by parallelizing the MKEY creation and deletion for direct MKEYs. Looking at the top level MKEY creation/deletion functions, the following improvement can be seen:
|-------------------+-------------| | operation | improvement | |-------------------+-------------| | create_user_mr() | 3-5x | | destroy_user_mr() | 8x | |-------------------+-------------|
The last part of the series introduces lazy MKEY deletion which postpones the MKEY deletion to a later point in a workqueue.
As this series and the previous ones were targeting live migration, we can also observe improvements on this front:
|-------------------+------------------+------------------| | Stage | Downtime #1 (ms) | Downtime #2 (ms) | |-------------------+------------------+------------------| | Baseline | 3140 | 3630 | | Parallel MKEY ops | 1200 | 2000 | | Deferred deletion | 1014 | 1253 | |-------------------+------------------+------------------|
Test configuration: 256 GB VM, 32 CPUs x 2 threads per core, 4 x mlx5 vDPA devices x 32 VQs (16 VQPs)
This series must be applied on top of the parallel VQ suspend/resume series [0].
[0] https://lore.kernel.org/all/[email protected]/
---
v2:
- Swapped flex array usage for plain zero length array in first patch.
- Updated code to use Scope-Based Cleanup Helpers where appropriate (only second patch).
- Added macro define for MTT alignment in first patch.
- Improved commit messages/comments based on review comments.
- Removed extra newlines.
---
Dragos Tatulea (7):
vdpa/mlx5: Create direct MKEYs in parallel vdpa/mlx5: Delete direct MKEYs in parallel vdpa/mlx5: Rename function
vdpa/mlx5: Extract mr members in own resource struct vdpa/mlx5: Rename mr_mtx -> lock
vdpa/mlx5: Introduce init/destroy for MR resources vdpa/mlx5: Postpone MR deletion
drivers/vdpa/mlx5/core/mlx5_vdpa.h | 25 ++- drivers/vdpa/mlx5/core/mr.c | 288 +++++++++++++++++++++++++---- drivers/vdpa/mlx5/core/resources.c | 3 - drivers/vdpa/mlx5/net/mlx5_vnet.c | 53 +++--- 4 files changed, 296 insertions(+), 73 deletions(-)
Signature · 1 line
2.45.1