Another round of exported properties for RDMAtool
cover Another round of exported properties for RDMAtool
From: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/[email protected]>
hi Doug,
This patch set is based on previously sent RDMA netlink work.
!!! Important to note, that LIDs and SM_LIDS are exported as u32, to support OmniPath natively.
Thanks
Leon Romanovsky (4):
RDMA/netlink: Export guid and sys_image_guid
RDMA/netlink: Advertise IB subnet prefix
RDMA/netink: Export lids and sm_lids
RDMA/netlink: Export LID mask counter (LMC)
drivers/infiniband/core/nldev.c | 21 ++++++++++++++++++++- include/uapi/rdma/rdma_netlink.h | 31 +++++++++++++++++++++++++++++++ 2 files changed, 51 insertions(+), 1 deletion(-)
--
2.13.1
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 20170628133445.16550-1-leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org
PATCH 1/4 RDMA/netlink: Export guid and sys_image_guid
From: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/[email protected]>
Add GUID and system image GUID to the device properties exported by RDMA netlink, to be used by RDMAtool.
--- drivers/infiniband/core/nldev.c | 7 +++++++ include/uapi/rdma/rdma_netlink.h | 13 +++++++++++++ 2 files changed, 20 insertions(+)
drivers/infiniband/core/nldev.c +7 -0
@@ -42,6 +42,8 @@ static const struct nla_policy nldev_policy[RDMA_NLDEV_ATTR_MAX] = { [RDMA_NLDEV_ATTR_PORT_INDEX] = { .type = NLA_U32 }, [RDMA_NLDEV_ATTR_FW_VERSION ] = { .type =NLA_NUL_STRING , .len = IB_FW_VERSION_NAME_MAX - 1}, + [RDMA_NLDEV_ATTR_GUID] = { .type = NLA_U64 }, + [RDMA_NLDEV_ATTR_SYS_IMAGE_GUID] = { .type = NLA_U64 }, };static int fill_dev_info (struct sk_buff *msg,struct ib_device *device)
@@ -64,6 +66,11 @@ static int fill_dev_info(struct sk_buff *msg, struct ib_device *device)if (nla_put_string (msg,RDMA_NLDEV_ATTR_FW_VERSION , fw))return -EMSGSIZE ; +if (nla_put_u64_64bit (msg,RDMA_NLDEV_ATTR_GUID , device->node_guid,0 )) +return -EMSGSIZE ; +if (nla_put_u64_64bit (msg,RDMA_NLDEV_ATTR_SYS_IMAGE_GUID , + device->attrs.sys_image_guid,0 )) +return -EMSGSIZE ;return 0 ; }
include/uapi/rdma/rdma_netlink.h +13 -0
@@ -266,6 +266,19 @@ enum rdma_nldev_attr {* Devices without FW will return empty string terminated by "\0" */ RDMA_NLDEV_ATTR_FW_VERSION ,/* string */ + +/* +* GUID (in network byte order) associated with the RDMA device. +*/ +RDMA_NLDEV_ATTR_GUID ,/* u64 */ + +/* +* System image GUID (in network byte order) associated with +* this RDMA device and other devices which are part of a +* single system. +*/ +RDMA_NLDEV_ATTR_SYS_IMAGE_GUID ,/* u64 */ +RDMA_NLDEV_ATTR_MAX };#endif /* _UAPI_RDMA_NETLINK_H */
Signature · 6 lines
2.13.1 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 20170628133445.16550-2-leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org
Re: [PATCH rdma-next 1/4] RDMA/netlink: Export guid and sys_image_guid
drivers/infiniband/core/nldev.c static const struct nla_policy nldev_policy[RDMA_NLDEV_ATTR_MAX] = { · line 45
Quoted 1 line · Leon Romanovsky
drivers/infiniband/core/nldev.c +1
-0+ [RDMA_NLDEV_ATTR_GUID] = { .type = NLA_U64 },
This should identify which guid it is. Port? Node? Device? Something else?
If it is the port guid should it be able to carry the entire table, not just the 0th entry?
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 20170628160025.GA14375-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org
Re: [PATCH rdma-next 1/4] RDMA/netlink: Export guid and sys_image_guid
Quoted 5 lines · Jason Gunthorpe
drivers/infiniband/core/nldev.c static const struct nla_policy nldev_policy[RDMA_NLDEV_ATTR_MAX] = { · line 45
Quoted 1 line · Leon Romanovsky
drivers/infiniband/core/nldev.c +1
-0+ [RDMA_NLDEV_ATTR_GUID] = { .type = NLA_U64 },This should identify which guid it is. Port? Node? Device? Something else?
It is my mistake, I was under impression of sysfs. It is node_guid.
Quoted 5 lines
If it is the port guid should it be able to carry the entire table, not just the 0th entry?
Jason
PATCH 2/4 RDMA/netlink: Advertise IB subnet prefix
In reply to a message not in the archiveFrom: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/[email protected]>
Add IB subnet prefix to the port properties exported by RDMA netlink.
--- drivers/infiniband/core/nldev.c | 4 ++++ include/uapi/rdma/rdma_netlink.h | 5 +++++ 2 files changed, 9 insertions(+)
drivers/infiniband/core/nldev.c +4 -0
@@ -44,6 +44,7 @@ static const struct nla_policy nldev_policy[RDMA_NLDEV_ATTR_MAX] = { .len = IB_FW_VERSION_NAME_MAX - 1}, [RDMA_NLDEV_ATTR_GUID] = { .type = NLA_U64 }, [RDMA_NLDEV_ATTR_SYS_IMAGE_GUID] = { .type = NLA_U64 }, + [RDMA_NLDEV_ATTR_SUBNET_PREFIX] = { .type = NLA_U64 }, };static int fill_dev_info (struct sk_buff *msg,struct ib_device *device)
@@ -95,6 +96,9 @@ static int fill_port_info(struct sk_buff *msg,if (nla_put_u64_64bit (msg,RDMA_NLDEV_ATTR_CAP_FLAGS , (u64 )attr.port_cap_flags,0 ))return -EMSGSIZE ; +if (nla_put_u64_64bit (msg,RDMA_NLDEV_ATTR_SUBNET_PREFIX , + attr.subnet_prefix,0 )) +return -EMSGSIZE ;return 0 ; }
include/uapi/rdma/rdma_netlink.h +5 -0
@@ -279,6 +279,11 @@ enum rdma_nldev_attr {*/ RDMA_NLDEV_ATTR_SYS_IMAGE_GUID ,/* u64 */ +/* +* Subnet prefix (in host byte order) +*/ +RDMA_NLDEV_ATTR_SUBNET_PREFIX ,/* u64 */ +RDMA_NLDEV_ATTR_MAX };#endif /* _UAPI_RDMA_NETLINK_H */
Signature · 6 lines
2.13.1 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 20170628133445.16550-3-leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org
Re: [PATCH rdma-next 2/4] RDMA/netlink: Advertise IB subnet prefix
include/uapi/rdma/rdma_netlink.h enum rdma_nldev_attr { · line 285
Quoted 4 lines · Leon Romanovsky
include/uapi/rdma/rdma_netlink.h +4
-0+/* +* Subnet prefix (in host byte order) +*/ + RDMA_NLDEV_ATTR_SUBNET_PREFIX, /* u64 */
I've always hated how random the be / host choices are in our API.
Since the GUID is in host order I recommend making the subnet_prefix in host order too.
At the very least, the comments beside anything not in host order should be 'be64' not u64
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 20170628160251.GB14375-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org
Re: [PATCH rdma-next 2/4] RDMA/netlink: Advertise IB subnet prefix
Quoted 8 lines · Jason Gunthorpe
include/uapi/rdma/rdma_netlink.h enum rdma_nldev_attr { · line 285
Quoted 4 lines · Leon Romanovsky
include/uapi/rdma/rdma_netlink.h +4
-0+/* +* Subnet prefix (in host byte order) +*/ + RDMA_NLDEV_ATTR_SUBNET_PREFIX, /* u64 */I've always hated how random the be / host choices are in our API.
Agree
Quoted 6 lines
Since the GUID is in host order I recommend making the
subnet_prefixin host order too.At the very least, the comments beside anything not in host order should be 'be64' not u64
Those comments represent netlink types. It doesn't have be64.
Quoted 2 lines
Jason
A message that is not in the archive (sent off-list or not yet received): 20170628161020.GI1248-U/[email protected]
Re: [PATCH rdma-next 2/4] RDMA/netlink: Advertise IB subnet prefix
Quoted 7 lines · Leon Romanovsky
Quoted 5 lines
Since the GUID is in host order I recommend making the
subnet_prefixin host order too.At the very least, the comments beside anything not in host order should be 'be64' not u64
Those comments represent netlink types. It doesn't have be64.
Then don't send b64 down netlink as a netlink u64 type?
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 20170628161236.GC28839-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org
Re: [PATCH rdma-next 2/4] RDMA/netlink: Advertise IB subnet prefix
Quoted 10 lines · Jason Gunthorpe
Quoted 7 lines · Leon Romanovsky
Quoted 5 lines
Since the GUID is in host order I recommend making the
subnet_prefixin host order too.At the very least, the comments beside anything not in host order should be 'be64' not u64
Those comments represent netlink types. It doesn't have be64.
Then don't send b64 down netlink as a netlink u64 type?
Agree, conversion to u64 is the right way to do.
Quoted 2 lines
Jason
PATCH 3/4 RDMA/netink: Export lids and sm_lids
In reply to a message not in the archiveFrom: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/[email protected]>
According to the IB specification, the LID and SM_LID are 16-bit wide, but to support OmniPath users, export it as 32-bit value from the beginning.
--- drivers/infiniband/core/nldev.c | 7 ++++++- include/uapi/rdma/rdma_netlink.h | 8 ++++++++ 2 files changed, 14 insertions(+), 1 deletion(-)
drivers/infiniband/core/nldev.c +6 -1
@@ -45,6 +45,8 @@ static const struct nla_policy nldev_policy[RDMA_NLDEV_ATTR_MAX] = { [RDMA_NLDEV_ATTR_GUID] = { .type = NLA_U64 }, [RDMA_NLDEV_ATTR_SYS_IMAGE_GUID] = { .type = NLA_U64 }, [RDMA_NLDEV_ATTR_SUBNET_PREFIX] = { .type = NLA_U64 }, + [RDMA_NLDEV_ATTR_LID] = { .type = NLA_U32 }, + [RDMA_NLDEV_ATTR_SM_LID] = { .type = NLA_U32 }, };static int fill_dev_info (struct sk_buff *msg,struct ib_device *device)
@@ -99,7 +101,10 @@ static int fill_port_info(struct sk_buff *msg,if (nla_put_u64_64bit (msg,RDMA_NLDEV_ATTR_SUBNET_PREFIX , attr.subnet_prefix,0 ))return -EMSGSIZE ; - +if (nla_put_u32 (msg,RDMA_NLDEV_ATTR_LID , attr.lid)) +return -EMSGSIZE ; +if (nla_put_u32 (msg,RDMA_NLDEV_ATTR_SM_LID , attr.sm_lid)) +return -EMSGSIZE ;return 0 ; }
include/uapi/rdma/rdma_netlink.h +8 -0
@@ -284,6 +284,14 @@ enum rdma_nldev_attr {*/ RDMA_NLDEV_ATTR_SUBNET_PREFIX ,/* u64 */ +/* +* Local Identifier (LID), +* According to IB specification, It is 16-bit address assigned +* by the Subnet Manager. Extended to be 32-bit for OmniPath users. +*/ +RDMA_NLDEV_ATTR_LID ,/* u32 */ +RDMA_NLDEV_ATTR_SM_LID ,/* u32 */ +RDMA_NLDEV_ATTR_MAX };#endif /* _UAPI_RDMA_NETLINK_H */
--
2.13.1
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 20170628133445.16550-4-leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org
Re: [PATCH rdma-next 3/4] RDMA/netink: Export lids and sm_lids
drivers/infiniband/core/nldev.c static int fill_port_info(struct sk_buff *msg, · line 107
Quoted 4 lines · Leon Romanovsky
drivers/infiniband/core/nldev.c +4
-0+if (nla_put_u32 (msg,RDMA_NLDEV_ATTR_LID , attr.lid)) +return -EMSGSIZE ; +if (nla_put_u32 (msg,RDMA_NLDEV_ATTR_SM_LID , attr.sm_lid)) +return -EMSGSIZE ;
Is this careful to only stuff attributes if they actuall exists? Eg these should not be present for iwarp/roce/etc
Same comment for all attributes. Only present if they are valid.
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 20170628160513.GC14375-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org
Re: [PATCH rdma-next 3/4] RDMA/netink: Export lids and sm_lids
Quoted 10 lines · Jason Gunthorpe
drivers/infiniband/core/nldev.c static int fill_port_info(struct sk_buff *msg, · line 107
Quoted 4 lines · Leon Romanovsky
drivers/infiniband/core/nldev.c +4
-0+if (nla_put_u32 (msg,RDMA_NLDEV_ATTR_LID , attr.lid)) +return -EMSGSIZE ; +if (nla_put_u32 (msg,RDMA_NLDEV_ATTR_SM_LID , attr.sm_lid)) +return -EMSGSIZE ;Is this careful to only stuff attributes if they actuall exists? Eg these should not be present for iwarp/roce/etc
Same comment for all attributes. Only present if they are valid.
It will be zero if it is not exist, but you are right it is better to avoid to fill unneeded fields.
Quoted 2 lines
Jason
A message that is not in the archive (sent off-list or not yet received): 20170628161616.GK1248-U/[email protected]
Re: [PATCH rdma-next 3/4] RDMA/netink: Export lids and sm_lids
Quoted 10 lines · Leon Romanovsky
Quoted 9 lines · Jason Gunthorpe
drivers/infiniband/core/nldev.c static int fill_port_info(struct sk_buff *msg, · line 107
Quoted 4 lines · Leon Romanovsky
drivers/infiniband/core/nldev.c +4
-0+if (nla_put_u32 (msg,RDMA_NLDEV_ATTR_LID , attr.lid)) +return -EMSGSIZE ; +if (nla_put_u32 (msg,RDMA_NLDEV_ATTR_SM_LID , attr.sm_lid)) +return -EMSGSIZE ;Is this careful to only stuff attributes if they actuall exists? Eg these should not be present for iwarp/roce/etc
Quoted 4 lines
Quoted 1 line
Same comment for all attributes. Only present if they are valid.
It will be zero if it is not exist, but you are right it is better to avoid to fill unneeded fields.
Not existing is the netlink way. Same comment for everything attributes. Many of the guids do not apply to iwarp, IIRC..
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 20170628161903.GA28996-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org
Re: [PATCH rdma-next 3/4] RDMA/netink: Export lids and sm_lids
Quoted 11 lines · Jason Gunthorpe
Quoted 10 lines · Leon Romanovsky
Quoted 9 lines · Jason Gunthorpe
Review
Quoted 4 lines · Leon Romanovsky
(quoted diff) +4
-0+ if (nla_put_u32(msg, RDMA_NLDEV_ATTR_LID, attr.lid)) + return -EMSGSIZE; + if (nla_put_u32(msg, RDMA_NLDEV_ATTR_SM_LID, attr.sm_lid)) + return -EMSGSIZE;Is this careful to only stuff attributes if they actuall exists? Eg these should not be present for iwarp/roce/etc
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
Same comment for all attributes. Only present if they are valid.
It will be zero if it is not exist, but you are right it is better to avoid to fill unneeded fields.
Not existing is the netlink way. Same comment for everything attributes. Many of the guids do not apply to iwarp, IIRC..
I got it, this is why I prefer to have basic RDMAtool with minimal prints (device name + caps) be accepted and only after that start work on every field in serial manner.
Thanks
Quoted 2 lines
Jason
PATCH 4/4 RDMA/netlink: Export LID mask counter (LMC)
In reply to a message not in the archiveFrom: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/[email protected]>
--- drivers/infiniband/core/nldev.c | 3 +++ include/uapi/rdma/rdma_netlink.h | 5 +++++ 2 files changed, 8 insertions(+)
drivers/infiniband/core/nldev.c +3 -0
@@ -47,6 +47,7 @@ static const struct nla_policy nldev_policy[RDMA_NLDEV_ATTR_MAX] = { [RDMA_NLDEV_ATTR_SUBNET_PREFIX] = { .type = NLA_U64 }, [RDMA_NLDEV_ATTR_LID] = { .type = NLA_U32 }, [RDMA_NLDEV_ATTR_SM_LID] = { .type = NLA_U32 }, + [RDMA_NLDEV_ATTR_LMC] = { .type = NLA_U8 }, };static int fill_dev_info (struct sk_buff *msg,struct ib_device *device)
@@ -105,6 +106,8 @@ static int fill_port_info(struct sk_buff *msg,return -EMSGSIZE ;if (nla_put_u32 (msg,RDMA_NLDEV_ATTR_SM_LID , attr.sm_lid))return -EMSGSIZE ; +if (nla_put_u8 (msg,RDMA_NLDEV_ATTR_LMC , attr.lmc)) +return -EMSGSIZE ;return 0 ; }
include/uapi/rdma/rdma_netlink.h +5 -0
@@ -292,6 +292,11 @@ enum rdma_nldev_attr {RDMA_NLDEV_ATTR_LID ,/* u32 */ RDMA_NLDEV_ATTR_SM_LID ,/* u32 */ +/* +* LID mask count (LMC) +*/ +RDMA_NLDEV_ATTR_LMC ,/* u8 */ +RDMA_NLDEV_ATTR_MAX };#endif /* _UAPI_RDMA_NETLINK_H */
Signature · 6 lines
2.13.1 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html