Skip to contentSkip to conversation

RDMA/mlx5: Implement mkeys management via LIFO queue

PATCH Applied Leon Romanovsky 1 participant 2 messages Sep 21, 2023 – Sep 26, 2023

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 RDMA/mlx5: Implement mkeys management via LIFO queue

From: Shay Drory <[email protected]>

Currently, mkeys are managed via xarray. This implementation leads to a degradation in cases many MRs are unregistered in parallel, due to xarray internal implementation, for example: deregistration 1M MRs via 64 threads is taking ~15% more time[1].

Hence, implement mkeys management via LIFO queue, which solved the degradation.

[1]
2.8us in kernel v5.19 compare to 3.2us in kernel v6.4

---
v1:
* Changed size of struct mlx5_mkeys_page to be aligned to PAGE_SIZE
v0: https://lore.kernel.org/all/96049c4bd3346a98240483ed2d22c5b1c7155c8a.1695203535.git.leon@kernel.org

---
 drivers/infiniband/hw/mlx5/mlx5_ib.h |  21 +-
 drivers/infiniband/hw/mlx5/mr.c      | 324 ++++++++++++---------------
 drivers/infiniband/hw/mlx5/umr.c     |   4 +-
 3 files changed, 169 insertions(+), 180 deletions(-)
drivers/infiniband/hw/mlx5/mlx5_ib.h +18 -3
index 16713baf0d06..bbe79b86c717 100644
@@ -753,10 +753,25 @@ struct umr_common {
 	unsigned int state;
 };
 
+#define NUM_MKEYS_PER_PAGE \
+	((PAGE_SIZE - sizeof(struct list_head)) / sizeof(u32))
+
+struct mlx5_mkeys_page {
+	u32 mkeys[NUM_MKEYS_PER_PAGE];
+	struct list_head list;
+};
+static_assert(sizeof(struct mlx5_mkeys_page) == PAGE_SIZE);
+
+struct mlx5_mkeys_queue {
+	struct list_head pages_list;
+	u32 num_pages;
+	unsigned long ci;
+	spinlock_t lock; /* sync list ops */
+};
+
 struct mlx5_cache_ent {
-	struct xarray		mkeys;
-	unsigned long		stored;
-	unsigned long		reserved;
+	struct mlx5_mkeys_queue	mkeys_queue;
+	u32			pending;
 
 	char                    name[4];
 
drivers/infiniband/hw/mlx5/mr.c +149 -175

589 lines not on this page · raw message

drivers/infiniband/hw/mlx5/umr.c +2 -2
index 234bf30db731..e76142f6fa88 100644
@@ -332,8 +332,8 @@ static int mlx5r_umr_post_send_wait(struct mlx5_ib_dev *dev, u32 mkey,
 
 		WARN_ON_ONCE(1);
 		mlx5_ib_warn(dev,
-			"reg umr failed (%u). Trying to recover and resubmit the flushed WQEs\n",
-			umr_context.status);
+			"reg umr failed (%u). Trying to recover and resubmit the flushed WQEs, mkey = %u\n",
+			umr_context.status, mkey);
 		mutex_lock(&umrc->lock);
 		err = mlx5r_umr_recover(dev);
 		mutex_unlock(&umrc->lock);
Signature · 1 line
2.41.0
author Applied
#1 lore raw
Quoted 9 lines · Leon Romanovsky

Currently, mkeys are managed via xarray. This implementation leads to a degradation in cases many MRs are unregistered in parallel, due to xarray internal implementation, for example: deregistration 1M MRs via 64 threads is taking ~15% more time[1].

Hence, implement mkeys management via LIFO queue, which solved the degradation.

[...]

Applied, thanks!

[1/1] RDMA/mlx5: Implement mkeys management via LIFO queue

      https://git.kernel.org/rdma/rdma/c/57e7071683ef61

Best regards,

Signature · 1 line
Leon Romanovsky <[email protected]>