Skip to contentSkip to conversation

Extend rdmatool to print driver QPs too

PATCH Applied Leon Romanovsky 3 participants 5 messages Apr 30, 2024 – May 3, 2024

Subsystems (3)
Networking Network drivers InfiniBand
lore
Share & tools
lore

Leon Romanovsky

author
#0 lore raw

cover Extend rdmatool to print driver QPs too

From: Leon Romanovsky <[email protected]>

This series is complimentary to the kernel patches that exposed to the userspace the driver-specific QPs.

https://lore.kernel.org/all/2607bb3ddec3cae3443c2ea19e9f700825d20a98.1713268997.git.leon@kernel.org

Thanks

Chiara Meiohas (2):
rdma: update uapi header
rdma: Add an option to display driver-specific QPs in the rdma tool

 rdma/include/uapi/rdma/rdma_netlink.h |  6 ++++++
 rdma/res-qp.c                         | 15 +++++++++++++++
 rdma/res.c                            |  5 +++++
 rdma/utils.c                          |  1 +
 4 files changed, 27 insertions(+)
Signature · 1 line
2.44.0
author
#1 lore raw

PATCH 1/2 rdma: update uapi header

From: Chiara Meiohas <[email protected]>

Update rdma_netlink.h file up to kernel commit e18fa0bbcedf ("RDMA/core: Add an option to display driver-specific QPs in the rdmatool")

Signed-off-by Chiara Meiohas <[email protected]>
Signed-off-by Leon Romanovsky <[email protected]>
---
 rdma/include/uapi/rdma/rdma_netlink.h | 6 ++++++
 1 file changed, 6 insertions(+)
rdma/include/uapi/rdma/rdma_netlink.h +6 -0
index e8861b5e..a6c8a52d 100644
@@ -558,6 +558,12 @@ enum rdma_nldev_attr {
 
 	RDMA_NLDEV_SYS_ATTR_PRIVILEGED_QKEY_MODE, /* u8 */
 
+	RDMA_NLDEV_ATTR_DRIVER_DETAILS,		/* u8 */
+	/*
+	 * QP subtype string, used for driver QPs
+	 */
+	RDMA_NLDEV_ATTR_RES_SUBTYPE,		/* string */
+
 	/*
 	 * Always the end
 	 */
Signature · 1 line
2.44.0
Reviewed-by on v1 1/2
#2 lore raw

RE: [EXTERNAL] [PATCH iproute2-next 1/2] rdma: update uapi header

Quoted 11 lines

-----Original Message-----
From: Leon Romanovsky <[email protected]>
Sent: Tuesday, April 30, 2024 3:48 PM
To: David Ahern <[email protected]>
Cc: Chiara Meiohas <[email protected]>; Jason Gunthorpe <[email protected]>; linux-netdev <[email protected]>; RDMA mailing list <[email protected]>
Subject: [EXTERNAL] [PATCH iproute2-next 1/2] rdma: update uapi header

Quoted 7 lines

From: Chiara Meiohas <[email protected]>

Update rdma_netlink.h file up to kernel commit e18fa0bbcedf ("RDMA/core: Add an option to display driver-specific QPs in the rdmatool")

Signed-off-by Chiara Meiohas <[email protected]>
Signed-off-by Leon Romanovsky <[email protected]>
Reviewed-by Subbaraya Sundeep <[email protected]>

Thanks,
Sundeep

Quoted 25 lines

---
rdma/include/uapi/rdma/rdma_netlink.h | 6 ++++++

1 file changed, 6 insertions(+)
rdma/include/uapi/rdma/rdma_netlink.h +6 -1
index e8861b5e..a6c8a52d 100644
@@ -558,6 +558,12 @@ enum rdma_nldev_attr {
 
 	RDMA_NLDEV_SYS_ATTR_PRIVILEGED_QKEY_MODE, /* u8 */
 
+	RDMA_NLDEV_ATTR_DRIVER_DETAILS,		/* u8 */
+	/*
+	 * QP subtype string, used for driver QPs
+	 */
+	RDMA_NLDEV_ATTR_RES_SUBTYPE,		/* string */
+
 	/*
 	 * Always the end
 	 */
--

2.44.0

author
#3 lore raw

PATCH 2/2 rdma: Add an option to display driver-specific QPs in the rdma tool

In reply to Leon Romanovsky

From: Chiara Meiohas <[email protected]>

Utilize the -dd flag (driver-specific details) in the rdmatool to view driver-specific QPs which are not exposed yet.

The following examples show mlx5 UMR QP which is visible now:

$ rdma resource show qp link ibp8s0f1
link ibp8s0f1/1 lqpn 360 type UD state RTS sq-psn 0 comm [mlx5_ib]
link ibp8s0f1/1 lqpn 0 type SMI state RTS sq-psn 0 comm [ib_core]
link ibp8s0f1/1 lqpn 1 type GSI state RTS sq-psn 0 comm [ib_core]
$ rdma resource show qp link ibp8s0f1 -dd
link ibp8s0f1/1 lqpn 360 type UD state RTS sq-psn 0 comm [mlx5_ib]
link ibp8s0f1/1 lqpn 465 type DRIVER subtype REG_UMR state RTS sq-psn 0 comm [mlx5_ib]
link ibp8s0f1/1 lqpn 0 type SMI state RTS sq-psn 0 comm [ib_core]
link ibp8s0f1/1 lqpn 1 type GSI state RTS sq-psn 0 comm [ib_core]
$ rdma resource show
0: ibp8s0f0: pd 3 cq 4 qp 3 cm_id 0 mr 0 ctx 0 srq 2
1: ibp8s0f1: pd 3 cq 4 qp 3 cm_id 0 mr 0 ctx 0 srq 2
$ rdma resource show -dd
0: ibp8s0f0: pd 3 cq 4 qp 4 cm_id 0 mr 0 ctx 0 srq 2
1: ibp8s0f1: pd 3 cq 4 qp 4 cm_id 0 mr 0 ctx 0 srq 2
Signed-off-by Chiara Meiohas <[email protected]>
Signed-off-by Leon Romanovsky <[email protected]>
---
 rdma/res-qp.c | 15 +++++++++++++++
 rdma/res.c    |  5 +++++
 rdma/utils.c  |  1 +
 3 files changed, 21 insertions(+)
rdma/res-qp.c +15 -0
index 65ff54ab..a47225df 100644
@@ -40,6 +40,15 @@ static void print_type(uint32_t val)
 	print_string(PRINT_ANY, "type", "type %s ", qp_types_to_str(val));
 }
 
+/*
+ * print the subtype only if the QPT is IB_QPT_DRIVER
+ */
+static void print_subtype(uint8_t type, const char *sub_type)
+{
+	if (type == 0xFF && sub_type)
+		print_string(PRINT_ANY, "subtype", "subtype %s ", sub_type);
+}
+
 static void print_state(uint32_t val)
 {
 	print_string(PRINT_ANY, "state", "state %s ", qp_states_to_str(val));

@@ -81,6 +90,7 @@ static int res_qp_line(struct rd *rd, const char *name, int idx,
 {
 	uint32_t lqpn, rqpn = 0, rq_psn = 0, sq_psn;
 	uint8_t type, state, path_mig_state = 0;
+	const char* sub_type = NULL;
 	uint32_t port = 0, pid = 0;
 	uint32_t pdn = 0;
 	char *comm = NULL;

@@ -134,6 +144,10 @@ static int res_qp_line(struct rd *rd, const char *name, int idx,
 		    nla_line[RDMA_NLDEV_ATTR_RES_PATH_MIG_STATE]))
 		goto out;
 
+	if (nla_line[RDMA_NLDEV_ATTR_RES_SUBTYPE])
+		sub_type =
+			mnl_attr_get_str(nla_line[RDMA_NLDEV_ATTR_RES_SUBTYPE]);
+
 	type = mnl_attr_get_u8(nla_line[RDMA_NLDEV_ATTR_RES_TYPE]);
 	if (rd_is_string_filtered_attr(rd, "type", qp_types_to_str(type),
 				       nla_line[RDMA_NLDEV_ATTR_RES_TYPE]))

@@ -164,6 +178,7 @@ static int res_qp_line(struct rd *rd, const char *name, int idx,
 	print_rqpn(rqpn, nla_line);
 
 	print_type(type);
+	print_subtype(type, sub_type);
 	print_state(state);
 
 	print_rqpsn(rq_psn, nla_line);
rdma/res.c +5 -0
index 3e024134..c311513a 100644
@@ -99,6 +99,8 @@ int _res_send_idx_msg(struct rd *rd, uint32_t command, mnl_cb_t callback,
 				 RDMA_NLDEV_ATTR_PORT_INDEX, rd->port_idx);
 
 	mnl_attr_put_u32(rd->nlh, id, idx);
+	mnl_attr_put_u8(rd->nlh, RDMA_NLDEV_ATTR_DRIVER_DETAILS,
+			rd->show_driver_details);
 
 	if (command == RDMA_NLDEV_CMD_STAT_GET)
 		mnl_attr_put_u32(rd->nlh, RDMA_NLDEV_ATTR_STAT_RES,

@@ -121,6 +123,9 @@ int _res_send_msg(struct rd *rd, uint32_t command, mnl_cb_t callback)
 		flags |= NLM_F_DUMP;
 
 	rd_prepare_msg(rd, command, &seq, flags);
+
+	mnl_attr_put_u8(rd->nlh, RDMA_NLDEV_ATTR_DRIVER_DETAILS,
+			rd->show_driver_details);
 	mnl_attr_put_u32(rd->nlh, RDMA_NLDEV_ATTR_DEV_INDEX, rd->dev_idx);
 	if (rd->port_idx)
 		mnl_attr_put_u32(rd->nlh,
rdma/utils.c +1 -0
index 27595a38..0f41013a 100644
@@ -423,6 +423,7 @@ static const enum mnl_attr_data_type nldev_policy[RDMA_NLDEV_ATTR_MAX] = {
 	[RDMA_NLDEV_ATTR_RES_SQ_PSN]		= MNL_TYPE_U32,
 	[RDMA_NLDEV_ATTR_RES_PATH_MIG_STATE]	= MNL_TYPE_U8,
 	[RDMA_NLDEV_ATTR_RES_TYPE]		= MNL_TYPE_U8,
+	[RDMA_NLDEV_ATTR_RES_SUBTYPE]           = MNL_TYPE_NUL_STRING,
 	[RDMA_NLDEV_ATTR_RES_STATE]		= MNL_TYPE_U8,
 	[RDMA_NLDEV_ATTR_RES_PID]		= MNL_TYPE_U32,
 	[RDMA_NLDEV_ATTR_RES_KERN_NAME]	= MNL_TYPE_NUL_STRING,
Signature · 1 line
2.44.0
CI Applied
#4 lore raw
In reply to Leon Romanovsky
This series was applied to iproute2/iproute2-next.git (main) by David Ahern <[email protected]>:

Hello:

This series was applied to iproute2/iproute2-next.git (main) by David Ahern <[email protected]>:

Quoted 8 lines · On Tue, 30 Apr 2024 13:18:18 +0300 you wrote:

From: Leon Romanovsky <[email protected]>

This series is complimentary to the kernel patches that exposed to the userspace the driver-specific QPs.

https://lore.kernel.org/all/2607bb3ddec3cae3443c2ea19e9f700825d20a98.1713268997.git.leon@kernel.org

[...]

Here is the summary with links:
- [iproute2-next,1/2] rdma: update uapi header

    https://git.kernel.org/pub/scm/network/iproute2/iproute2-next.git/commit/?id=e459ea4392c4
  - [iproute2-next,2/2] rdma: Add an option to display driver-specific QPs in the rdma tool
    https://git.kernel.org/pub/scm/network/iproute2/iproute2-next.git/commit/?id=57d7a8fd904c

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html