Skip to contentSkip to conversation

Supports to add/delete IB devices with type SMI

RFC In review Mark Zhang 2 participants 4 messages Jul 1, 2024 – Jul 4, 2024 InfiniBand

lore
Share & tools
lore

Mark Zhang

author
#0 lore raw

cover Supports to add/delete IB devices with type SMI

This series supports to add/delete an IB device with type SMI. This is complimentary to the kernel patches that support to IB sub device and mlx5 implementation.

https://lore.kernel.org/all/[email protected]/

There's no kernel commit ID for the rdma_netlink.h change as the kernel part isn't accepted yet.

Thanks

Mark Zhang (2):
Update rdma_netlink.h file upto kernel commit
rdma: Supports to add/delete a device with type SMI

 man/man8/rdma-dev.8                   |  40 +++++++++
 rdma/dev.c                            | 120 ++++++++++++++++++++++++++
 rdma/include/uapi/rdma/rdma_netlink.h |  13 +++
 rdma/rdma.h                           |   2 +
 rdma/utils.c                          |   2 +
 5 files changed, 177 insertions(+)
Signature · 1 line
2.26.3
author
#1 lore raw

RFC 1/2 Update rdma_netlink.h file upto kernel commit

---
 rdma/include/uapi/rdma/rdma_netlink.h | 13 +++++++++++++
 1 file changed, 13 insertions(+)
rdma/include/uapi/rdma/rdma_netlink.h +13 -0
index a6c8a52d..aea1d5ef 100644
@@ -301,6 +301,10 @@ enum rdma_nldev_command {
 
 	RDMA_NLDEV_CMD_RES_SRQ_GET_RAW,
 
+	RDMA_NLDEV_CMD_NEWDEV,
+
+	RDMA_NLDEV_CMD_DELDEV,
+
 	RDMA_NLDEV_NUM_OPS
 };
 
@@ -564,6 +568,10 @@ enum rdma_nldev_attr {
 	 */
 	RDMA_NLDEV_ATTR_RES_SUBTYPE,		/* string */
 
+	RDMA_NLDEV_ATTR_DEV_TYPE,		/* u8 */
+
+	RDMA_NLDEV_ATTR_PARENT_NAME,		/* string */
+
 	/*
 	 * Always the end
 	 */
@@ -602,4 +610,9 @@ enum rdma_nl_counter_mask {
 	RDMA_COUNTER_MASK_QP_TYPE = 1,
 	RDMA_COUNTER_MASK_PID = 1 << 1,
 };
+
+/* Supported rdma device types. */
+enum rdma_nl_dev_type {
+	RDMA_DEVICE_TYPE_SMI = 1,
+};
 #endif /* _RDMA_NETLINK_H */
Signature · 1 line
2.26.3
author
#2 lore raw

RFC 2/2 rdma: Supports to add/delete a device with type SMI

In reply to Mark Zhang

This patch adds a new device attribute "type", as well as supports to add and delete a rdma device with a specific type. This new device provides a subset of functionalists defined in IBTA spec.

Currently only type "SMI" is supported: A SMI device provides SMI (QP0) interface; This device and it's parent associates with the same HCA port and shares the physical link, so when the parent doesn't support SMI, It allows the subnet manager to configure the link.

This patch also supports to print device type and parent if any.

Examples:

$ rdma dev add smi1 type SMI parent ibp8s0f1
$ rdma dev show smi1
2: smi1: node_type ca fw 20.38.1002 node_guid 9803:9b03:009f:d5ef sys_image_guid 9803:9b03:009f:d5ee type smi parent ibp8s0f1
$ rdma dev del smi1
---
 man/man8/rdma-dev.8 |  40 +++++++++++++++
 rdma/dev.c          | 120 ++++++++++++++++++++++++++++++++++++++++++++
 rdma/rdma.h         |   2 +
 rdma/utils.c        |   2 +
 4 files changed, 164 insertions(+)
man/man8/rdma-dev.8 +40 -0
index 368cdc7c..e26f738c 100644
@@ -40,6 +40,18 @@ rdma-dev \- RDMA device configuration
 .BR adaptive-moderation
 .BR [on/off]
 
+.ti -8
+.B rdma dev add
+.RI "[ " NAME " ]"
+.B type
+.RI "[ " TYPE " ]"
+.B parent
+.RI "[ " PARENT_DEV " ]"
+
+.ti -8
+.B rdma dev delete
+.RI "[ " NAME " ]"
+
 .ti -8
 .B rdma dev help
 
@@ -53,6 +65,22 @@ rdma-dev \- RDMA device configuration
 - specifies the RDMA device to show.
 If this argument is omitted all devices are listed.
 
+.SS rdma dev add - Add a RDMA device with a specific type and parent IB device; This new device provides subset of functionalities defined in IBTA spec.
+.SS rdma dev delete - Delete a RDMA device which is created with the "add" command.
+.I NAME
+- The name of the device being added/deleted.
+
+.I TYPE
+- The type of the device being added; It specifies which functionalities it provides. Supported device type:
+.sp
+.in +8
+.B SMI
+- A device with this type provides SMI (QP0) interface. This device and it's parent associates with the same HCA port and shares the physical link, so when the parent doesn't support SMI, then this type of device can be created to allow the subnet manager to configure the link.
+.in -8
+
+.I PARENT_DEV
+- The name of its parent IB device.
+
 .SH "EXAMPLES"
 .PP
 rdma dev
@@ -84,6 +112,16 @@ Sets the state of adaptive interrupt moderation for the RDMA device.
 This is a global setting for the RDMA device but the value is printed for each CQ individually because the state is constant from CQ allocation.
 .RE
 .PP
+rdma dev add smi1 type SMI parent ibp8s0f1
+.RS 4
+Add a new IB device with name "smi1", type "SMI", and "ibp8s0f1" as its parent device.
+.RE
+.PP
+rdma dev delete smi1
+.RS 4
+Delete the IB device "smi1".
+.RE
+.PP
 
 .SH SEE ALSO
 .BR ip (8),
@@ -96,3 +134,5 @@ This is a global setting for the RDMA device but the value is printed for each C
 
 .SH AUTHOR
 Leon Romanovsky <[email protected]>
+.br
+Mark Zhang <[email protected]>
rdma/dev.c +120 -0

148 lines not on this page · raw message

rdma/rdma.h +2 -0
index df1852db..d224ec57 100644
@@ -72,6 +72,8 @@ struct rd {
 	struct list_head filter_list;
 	char *link_name;
 	char *link_type;
+	char *dev_name;
+	int dev_type;
 };
 
 struct rd_cmd {
rdma/utils.c +2 -0
index 0f41013a..4d3803b5 100644
@@ -475,6 +475,8 @@ static const enum mnl_attr_data_type nldev_policy[RDMA_NLDEV_ATTR_MAX] = {
 	[RDMA_NLDEV_ATTR_DEV_DIM] = MNL_TYPE_U8,
 	[RDMA_NLDEV_ATTR_RES_RAW] = MNL_TYPE_BINARY,
 	[RDMA_NLDEV_SYS_ATTR_PRIVILEGED_QKEY_MODE] = MNL_TYPE_U8,
+	[RDMA_NLDEV_ATTR_DEV_TYPE] = MNL_TYPE_U8,
+	[RDMA_NLDEV_ATTR_PARENT_NAME] = MNL_TYPE_STRING,
 };
 
 static int rd_attr_check(const struct nlattr *attr, int *typep)
Signature · 1 line
2.26.3
#3 lore raw
In reply to Mark Zhang
Quoted 6 lines · Mark Zhang

This series supports to add/delete an IB device with type SMI. This is complimentary to the kernel patches that support to IB sub device and mlx5 implementation.

https://lore.kernel.org/all/[email protected]/

Quoted 7 lines

There's no kernel commit ID for the rdma_netlink.h change as the kernel part isn't accepted yet.

Thanks

Mark Zhang (2):
Update rdma_netlink.h file upto kernel commit

Kernel part was accepted, please resend this series with the updated first commit.

Thanks

Quoted 13 lines

rdma: Supports to add/delete a device with type SMI

 man/man8/rdma-dev.8                   |  40 +++++++++
 rdma/dev.c                            | 120 ++++++++++++++++++++++++++
 rdma/include/uapi/rdma/rdma_netlink.h |  13 +++
 rdma/rdma.h                           |   2 +
 rdma/utils.c                          |   2 +
 5 files changed, 177 insertions(+)
Signature · 1 line
2.26.3