Skip to contentSkip to conversation

Two small fixes to MR cache series

PATCH Applied Leon Romanovsky 2 participants 8 messages Feb 2, 2023 – Feb 6, 2023

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover Two small fixes to MR cache series

From: Leon Romanovsky <[email protected]>

Hi,

There are two small patches with fix and cleanup to previously accepted MR cache series from Michael.

Thanks

Leon Romanovsky (2):
RDMA/mlx5: Fix MR cache debugfs in switchdev mode RDMA/mlx5: Remove impossible check of mkey cache cleanup failure

 drivers/infiniband/hw/mlx5/main.c    |  7 +------
 drivers/infiniband/hw/mlx5/mlx5_ib.h |  2 +-
 drivers/infiniband/hw/mlx5/mr.c      | 10 ++++++----
 3 files changed, 8 insertions(+), 11 deletions(-)
Signature · 1 line
2.39.1
#1 lore raw

PATCH 1/2 RDMA/mlx5: Fix MR cache debugfs in switchdev mode

From: Leon Romanovsky <[email protected]>

Block MR cache debugfs creation while in switchdev mode and add missing debugfs cleanup in error path.

This change fixes the following debugfs errors:

bond0: (slave enp8s0f1): Enslaving as a backup interface with an up link mlx5_core 0000:08:00.0: lag map: port 1:1 port 2:1
mlx5_core 0000:08:00.0: shared_fdb:1 mode:queue_affinity mlx5_core 0000:08:00.0: Operation mode is single FDB
debugfs: Directory '2' with parent '/' already present!
...
debugfs: Directory '22' with parent '/' already present!

Fixes 73d09b2fe833 ("RDMA/mlx5: Introduce mlx5r_cache_rb_key")
---
 drivers/infiniband/hw/mlx5/mr.c | 4 ++++
 1 file changed, 4 insertions(+)
drivers/infiniband/hw/mlx5/mr.c +4 -0
index 3a9a787184fc..3f410eef58e4 100644
@@ -848,6 +848,9 @@ static void mlx5_mkey_cache_debugfs_add_ent(struct mlx5_ib_dev *dev,
 	int order = order_base_2(ent->rb_key.ndescs);
 	struct dentry *dir;
 
+	if (!mlx5_debugfs_root || dev->is_rep)
+		return;
+
 	if (ent->rb_key.access_mode == MLX5_MKC_ACCESS_MODE_KSM)
 		order = MLX5_IMR_KSM_CACHE_ENTRY + 2;
 
@@ -1006,6 +1009,7 @@ int mlx5_mkey_cache_init(struct mlx5_ib_dev *dev)
 
 err:
 	mutex_unlock(&cache->rb_lock);
+	mlx5_mkey_cache_debugfs_cleanup(dev);
 	mlx5_ib_warn(dev, "failed to create mkey cache entry\n");
 	return ret;
 }
Signature · 1 line
2.39.1
#6 lore raw

PATCH 2/2 RDMA/mlx5: Remove impossible check of mkey cache cleanup failure

In reply to Leon Romanovsky

From: Leon Romanovsky <[email protected]>

mlx5_mkey_cache_cleanup() can't fail and can be changed to be void.

---
 drivers/infiniband/hw/mlx5/main.c    | 7 +------
 drivers/infiniband/hw/mlx5/mlx5_ib.h | 2 +-
 drivers/infiniband/hw/mlx5/mr.c      | 6 ++----
 3 files changed, 4 insertions(+), 11 deletions(-)
drivers/infiniband/hw/mlx5/main.c +1 -6
index 10f12e9a4dc3..e252ba138690 100644
@@ -4005,12 +4005,7 @@ static int mlx5_ib_stage_ib_reg_init(struct mlx5_ib_dev *dev)
 
 static void mlx5_ib_stage_pre_ib_reg_umr_cleanup(struct mlx5_ib_dev *dev)
 {
-	int err;
-
-	err = mlx5_mkey_cache_cleanup(dev);
-	if (err)
-		mlx5_ib_warn(dev, "mr cache cleanup failed\n");
-
+	mlx5_mkey_cache_cleanup(dev);
 	mlx5r_umr_resource_cleanup(dev);
 }
 
drivers/infiniband/hw/mlx5/mlx5_ib.h +1 -1
index 58d02e778c4b..93420b73bf20 100644
@@ -1337,7 +1337,7 @@ void mlx5_ib_populate_pas(struct ib_umem *umem, size_t page_size, __be64 *pas,
 void mlx5_ib_copy_pas(u64 *old, u64 *new, int step, int num);
 int mlx5_ib_get_cqe_size(struct ib_cq *ibcq);
 int mlx5_mkey_cache_init(struct mlx5_ib_dev *dev);
-int mlx5_mkey_cache_cleanup(struct mlx5_ib_dev *dev);
+void mlx5_mkey_cache_cleanup(struct mlx5_ib_dev *dev);
 struct mlx5_cache_ent *
 mlx5r_cache_create_ent_locked(struct mlx5_ib_dev *dev,
 			      struct mlx5r_cache_rb_key rb_key,
drivers/infiniband/hw/mlx5/mr.c +2 -4
index 3f410eef58e4..68102297b9cb 100644
@@ -1014,14 +1014,14 @@ int mlx5_mkey_cache_init(struct mlx5_ib_dev *dev)
 	return ret;
 }
 
-int mlx5_mkey_cache_cleanup(struct mlx5_ib_dev *dev)
+void mlx5_mkey_cache_cleanup(struct mlx5_ib_dev *dev)
 {
 	struct rb_root *root = &dev->cache.rb_root;
 	struct mlx5_cache_ent *ent;
 	struct rb_node *node;
 
 	if (!dev->cache.wq)
-		return 0;
+		return;
 
 	cancel_delayed_work_sync(&dev->cache.remove_ent_dwork);
 	mutex_lock(&dev->cache.rb_lock);
@@ -1048,8 +1048,6 @@ int mlx5_mkey_cache_cleanup(struct mlx5_ib_dev *dev)
 
 	destroy_workqueue(dev->cache.wq);
 	del_timer_sync(&dev->delay_timer);
-
-	return 0;
 }
 
 struct ib_mr *mlx5_ib_get_dma_mr(struct ib_pd *pd, int acc)
Signature · 1 line
2.39.1
maintainer
#2 lore raw

Re: [PATCH rdma-next 1/2] RDMA/mlx5: Fix MR cache debugfs in switchdev mode

In reply to Leon Romanovsky
Quoted 4 lines · Leon Romanovsky

From: Leon Romanovsky <[email protected]>

Block MR cache debugfs creation while in switchdev mode and add missing debugfs cleanup in error path.

Why does switchdev have anything to do with this?

Jason

#3 lore raw

Re: [PATCH rdma-next 1/2] RDMA/mlx5: Fix MR cache debugfs in switchdev mode

Quoted 8 lines · Jason Gunthorpe
Quoted 4 lines · Leon Romanovsky

From: Leon Romanovsky <[email protected]>

Block MR cache debugfs creation while in switchdev mode and add missing debugfs cleanup in error path.

Why does switchdev have anything to do with this?

Because we always had the following code in cleanup:
697 static void mlx5_mkey_cache_debugfs_cleanup(struct mlx5_ib_dev *dev) 698 {
699 if (!mlx5_debugfs_root || dev->is_rep)
700 return;
701

MR cache shouldn't be used at all for IB representors, and more comprehensive patch will take more work than this simple solution.

Thanks

Quoted 1 line

Jason

maintainer
#4 lore raw

Re: [PATCH rdma-next 1/2] RDMA/mlx5: Fix MR cache debugfs in switchdev mode

Quoted 11 lines · Leon Romanovsky
Quoted 8 lines · Jason Gunthorpe
Quoted 4 lines · Leon Romanovsky

From: Leon Romanovsky <[email protected]>

Block MR cache debugfs creation while in switchdev mode and add missing debugfs cleanup in error path.

Why does switchdev have anything to do with this?

Because we always had the following code in cleanup:

Quoted 8 lines
   697 static void mlx5_mkey_cache_debugfs_cleanup(struct mlx5_ib_dev *dev)
   698 {
   699         if (!mlx5_debugfs_root || dev->is_rep)
   700                 return;
   701

MR cache shouldn't be used at all for IB representors, and more comprehensive patch will take more work than this simple solution.

That make sense, the commit message should explain it..

Jason

#5 lore raw

Re: [PATCH rdma-next 1/2] RDMA/mlx5: Fix MR cache debugfs in switchdev mode

Quoted 18 lines · Jason Gunthorpe
Quoted 17 lines · Leon Romanovsky
Quoted 8 lines · Jason Gunthorpe
Quoted 4 lines · Leon Romanovsky

From: Leon Romanovsky <[email protected]>

Block MR cache debugfs creation while in switchdev mode and add missing debugfs cleanup in error path.

Why does switchdev have anything to do with this?

Because we always had the following code in cleanup:

   697 static void mlx5_mkey_cache_debugfs_cleanup(struct mlx5_ib_dev *dev)
   698 {
   699         if (!mlx5_debugfs_root || dev->is_rep)
   700                 return;
   701
Quoted 4 lines
Quoted 2 lines

MR cache shouldn't be used at all for IB representors, and more comprehensive patch will take more work than this simple solution.

That make sense, the commit message should explain it..

OK, I added the following to the commit message.

RDMA/mlx5: Fix MR cache debugfs error in IB representors mode

Block MR cache debugfs creation for IB representor flow as MR cache shouldn't be used at all in that mode. As part of this change, add missing debugfs cleanup in error path too.

RDMA/mlx5: Fix MR cache debugfs error in IB representors mode

Block MR cache debugfs creation for IB representor flow as MR cache shouldn't be used at all in that mode. As part of this change, add missing debugfs cleanup in error path too.

Thanks

Quoted 2 lines

Jason

author Applied
#7 lore raw
In reply to Leon Romanovsky
Quoted 8 lines · Leon Romanovsky

From: Leon Romanovsky <[email protected]>

Hi,

There are two small patches with fix and cleanup to previously accepted MR cache series from Michael.

[...]

Applied, thanks!

[1/2] RDMA/mlx5: Fix MR cache debugfs in switchdev mode

      https://git.kernel.org/rdma/rdma/c/828cf5936bea24
[2/2] RDMA/mlx5: Remove impossible check of mkey cache cleanup failure
      https://git.kernel.org/rdma/rdma/c/85f9e38a5ac7d3

Best regards,

Signature · 1 line
Leon Romanovsky <[email protected]>