Skip to contentSkip to conversation

IB/mlx5: Report correctly tag matching rendezvous capability

PATCH In review Leon Romanovsky 3 participants 5 messages Jul 5, 2019 – Jul 8, 2019

Subsystems (2)
InfiniBand Mellanox MLX5 IB driver
lore
Share & tools
lore

Leon Romanovsky

author
#0 lore raw

PATCH 1/1 IB/mlx5: Report correctly tag matching rendezvous capability

From: Danit Goldberg <[email protected]>

Tag matching with rendezvous offload for RC transport is controlled by FW and before this change, it was advertised to user as supported without any relation to FW.

Separate tag matching for rendezvous and eager protocols, so users will see real capabilities.

Cc <[email protected]> # 4.13
Fixes eb761894351d ("IB/mlx5: Fill XRQ capabilities")
Signed-off-by Danit Goldberg <[email protected]>
Reviewed-by Yishai Hadas <[email protected]>
Reviewed-by Artemy Kovalyov <[email protected]>
Signed-off-by Leon Romanovsky <[email protected]>
---
 drivers/infiniband/hw/mlx5/main.c | 8 ++++++--
 include/rdma/ib_verbs.h           | 4 ++--
 2 files changed, 8 insertions(+), 4 deletions(-)
drivers/infiniband/hw/mlx5/main.c +6 -2
index 07a05b0b9e42..c2a5780cb394 100644
@@ -1046,15 +1046,19 @@ static int mlx5_ib_query_device(struct ib_device *ibdev,
 	}
 
 	if (MLX5_CAP_GEN(mdev, tag_matching)) {
-		props->tm_caps.max_rndv_hdr_size = MLX5_TM_MAX_RNDV_MSG_SIZE;
 		props->tm_caps.max_num_tags =
 			(1 << MLX5_CAP_GEN(mdev, log_tag_matching_list_sz)) - 1;
-		props->tm_caps.flags = IB_TM_CAP_RC;
 		props->tm_caps.max_ops =
 			1 << MLX5_CAP_GEN(mdev, log_max_qp_sz);
 		props->tm_caps.max_sge = MLX5_TM_MAX_SGE;
 	}
 
+	if (MLX5_CAP_GEN(mdev, tag_matching) &&
+	    MLX5_CAP_GEN(mdev, rndv_offload_rc)) {
+		props->tm_caps.flags = IB_TM_CAP_RNDV_RC;
+		props->tm_caps.max_rndv_hdr_size = MLX5_TM_MAX_RNDV_MSG_SIZE;
+	}
+
 	if (MLX5_CAP_GEN(dev->mdev, cq_moderation)) {
 		props->cq_caps.max_cq_moderation_count =
 						MLX5_MAX_CQ_COUNT;
include/rdma/ib_verbs.h +2 -2
index 30eb68f36109..c5f8a9f17063 100644
@@ -308,8 +308,8 @@ struct ib_rss_caps {
 };
 
 enum ib_tm_cap_flags {
-	/*  Support tag matching on RC transport */
-	IB_TM_CAP_RC		    = 1 << 0,
+	/*  Support tag matching with rendezvous offload for RC transport */
+	IB_TM_CAP_RNDV_RC = 1 << 0,
 };
 
 struct ib_tm_caps {
Signature · 1 line
2.20.1
#1 lore raw
Quoted 48 lines · Leon Romanovsky

From: Danit Goldberg <[email protected]>

Tag matching with rendezvous offload for RC transport is controlled by FW and before this change, it was advertised to user as supported without any relation to FW.

Separate tag matching for rendezvous and eager protocols, so users will see real capabilities.

Cc <[email protected]> # 4.13
Fixes eb761894351d ("IB/mlx5: Fill XRQ capabilities")
Signed-off-by Danit Goldberg <[email protected]>
Reviewed-by Yishai Hadas <[email protected]>
Reviewed-by Artemy Kovalyov <[email protected]>
Signed-off-by Leon Romanovsky <[email protected]>
 drivers/infiniband/hw/mlx5/main.c | 8 ++++++--
 include/rdma/ib_verbs.h           | 4 ++--
 2 files changed, 8 insertions(+), 4 deletions(-)
drivers/infiniband/hw/mlx5/main.c +7 -2
index 07a05b0b9e42..c2a5780cb394 100644
+++ b/drivers/infiniband/hw/mlx5/main.c

@@ -1046,15 +1046,19 @@ static int mlx5_ib_query_device(struct ib_device *ibdev,
 	}
 
 	if (MLX5_CAP_GEN(mdev, tag_matching)) {
-		props->tm_caps.max_rndv_hdr_size = MLX5_TM_MAX_RNDV_MSG_SIZE;
 		props->tm_caps.max_num_tags =
 			(1 << MLX5_CAP_GEN(mdev, log_tag_matching_list_sz)) - 1;
-		props->tm_caps.flags = IB_TM_CAP_RC;
 		props->tm_caps.max_ops =
 			1 << MLX5_CAP_GEN(mdev, log_max_qp_sz);
 		props->tm_caps.max_sge = MLX5_TM_MAX_SGE;
 	}
 
+	if (MLX5_CAP_GEN(mdev, tag_matching) &&
+	    MLX5_CAP_GEN(mdev, rndv_offload_rc)) {
+		props->tm_caps.flags = IB_TM_CAP_RNDV_RC;
+		props->tm_caps.max_rndv_hdr_size = MLX5_TM_MAX_RNDV_MSG_SIZE;
+	}
+
 	if (MLX5_CAP_GEN(dev->mdev, cq_moderation)) {
 		props->cq_caps.max_cq_moderation_count =
 						MLX5_MAX_CQ_COUNT;
include/rdma/ib_verbs.h +1 -0
index 30eb68f36109..c5f8a9f17063 100644
+++ b/include/rdma/ib_verbs.h

include/rdma/ib_verbs.h struct ib_rss_caps { · line 313

Quoted 8 lines
include/rdma/ib_verbs.h +2 -2
@@ -308,8 +308,8 @@ struct ib_rss_caps {
 };
 
 enum ib_tm_cap_flags {
-	/*  Support tag matching on RC transport */
-	IB_TM_CAP_RC		    = 1 << 0,
+	/*  Support tag matching with rendezvous offload for RC transport */
+	IB_TM_CAP_RNDV_RC = 1 << 0,
 };

This is in the wrong header, right?

Jason

author
#2 lore raw
Quoted 51 lines · Jason Gunthorpe
Quoted 50 lines · Leon Romanovsky

From: Danit Goldberg <[email protected]>

Tag matching with rendezvous offload for RC transport is controlled by FW and before this change, it was advertised to user as supported without any relation to FW.

Separate tag matching for rendezvous and eager protocols, so users will see real capabilities.

Cc <[email protected]> # 4.13
Fixes eb761894351d ("IB/mlx5: Fill XRQ capabilities")
Signed-off-by Danit Goldberg <[email protected]>
Reviewed-by Yishai Hadas <[email protected]>
Reviewed-by Artemy Kovalyov <[email protected]>
Signed-off-by Leon Romanovsky <[email protected]>
 drivers/infiniband/hw/mlx5/main.c | 8 ++++++--
 include/rdma/ib_verbs.h           | 4 ++--
 2 files changed, 8 insertions(+), 4 deletions(-)
drivers/infiniband/hw/mlx5/main.c +7 -2
index 07a05b0b9e42..c2a5780cb394 100644
+++ b/drivers/infiniband/hw/mlx5/main.c

@@ -1046,15 +1046,19 @@ static int mlx5_ib_query_device(struct ib_device *ibdev,
 	}
 
 	if (MLX5_CAP_GEN(mdev, tag_matching)) {
-		props->tm_caps.max_rndv_hdr_size = MLX5_TM_MAX_RNDV_MSG_SIZE;
 		props->tm_caps.max_num_tags =
 			(1 << MLX5_CAP_GEN(mdev, log_tag_matching_list_sz)) - 1;
-		props->tm_caps.flags = IB_TM_CAP_RC;
 		props->tm_caps.max_ops =
 			1 << MLX5_CAP_GEN(mdev, log_max_qp_sz);
 		props->tm_caps.max_sge = MLX5_TM_MAX_SGE;
 	}
 
+	if (MLX5_CAP_GEN(mdev, tag_matching) &&
+	    MLX5_CAP_GEN(mdev, rndv_offload_rc)) {
+		props->tm_caps.flags = IB_TM_CAP_RNDV_RC;
+		props->tm_caps.max_rndv_hdr_size = MLX5_TM_MAX_RNDV_MSG_SIZE;
+	}
+
 	if (MLX5_CAP_GEN(dev->mdev, cq_moderation)) {
 		props->cq_caps.max_cq_moderation_count =
 						MLX5_MAX_CQ_COUNT;
include/rdma/ib_verbs.h +1 -0
index 30eb68f36109..c5f8a9f17063 100644
+++ b/include/rdma/ib_verbs.h

@@ -308,8 +308,8 @@ struct ib_rss_caps {
 };
Quoted 8 lines

include/rdma/ib_verbs.h struct ib_rss_caps { · line 313

Quoted 6 lines
include/rdma/ib_verbs.h +2 -2
 enum ib_tm_cap_flags {
-	/*  Support tag matching on RC transport */
-	IB_TM_CAP_RC		    = 1 << 0,
+	/*  Support tag matching with rendezvous offload for RC transport */
+	IB_TM_CAP_RNDV_RC = 1 << 0,
 };

This is in the wrong header, right?

It predates our all-to-uapi headers approach and moving to UAPI this struct is definitely too much for a fix which should go to stable@.

Thanks

Quoted 2 lines

Jason

#3 lore raw
Quoted 15 lines · Leon Romanovsky
Quoted 6 lines
Quoted 6 lines
include/rdma/ib_verbs.h +1 -0
index 30eb68f36109..c5f8a9f17063 100644
+++ b/include/rdma/ib_verbs.h

@@ -308,8 +308,8 @@ struct ib_rss_caps {
 };
Quoted 8 lines

include/rdma/ib_verbs.h struct ib_rss_caps { · line 313

Quoted 6 lines
include/rdma/ib_verbs.h +2 -2
 enum ib_tm_cap_flags {
-	/*  Support tag matching on RC transport */
-	IB_TM_CAP_RC		    = 1 << 0,
+	/*  Support tag matching with rendezvous offload for RC transport */
+	IB_TM_CAP_RNDV_RC = 1 << 0,
 };

This is in the wrong header, right?

Quoted 2 lines

It predates our all-to-uapi headers approach and moving to UAPI this struct is definitely too much for a fix which should go to stable@.

Well, lets send a patch to fix it please

Jason

#4 lore raw
In reply to Leon Romanovsky
Quoted 15 lines · Leon Romanovsky

From: Danit Goldberg <[email protected]>

Tag matching with rendezvous offload for RC transport is controlled by FW and before this change, it was advertised to user as supported without any relation to FW.

Separate tag matching for rendezvous and eager protocols, so users will see real capabilities.

Cc <[email protected]> # 4.13
Fixes eb761894351d ("IB/mlx5: Fill XRQ capabilities")
Signed-off-by Danit Goldberg <[email protected]>
Reviewed-by Yishai Hadas <[email protected]>
Reviewed-by Artemy Kovalyov <[email protected]>
Signed-off-by Leon Romanovsky <[email protected]>
Quoted 4 lines
---
 drivers/infiniband/hw/mlx5/main.c | 8 ++++++--
 include/rdma/ib_verbs.h           | 4 ++--
 2 files changed, 8 insertions(+), 4 deletions(-)

This is a bit late, but ince this is for stable, applied to for-next, thanks

Jason