RAW format dumps through RDMAtool
Subsystems (3)
cover RAW format dumps through RDMAtool
From: Leon Romanovsky <[email protected]>
Changelog:
v1:
* Kernel part was accepted, so this series has correct SHA for the kernel header update patch.
* Aligned implementation to the final kernel solution of query interface.
v0:
https://lore.kernel.org/linux-rdma/[email protected]
-----------------------------------------------------------------------------
Hi,
The following series adds support to get the RDMA resource data in RAW format. The main motivation for doing this is to enable vendors to return the entire QP/CQ/MR data without a need from the vendor to set each field separately.
Thanks
Maor Gottlieb (4):
rdma: update uapi headers
rdma: Add support to get QP in raw format
rdma: Add support to get CQ in raw format
rdma: Add support to get MR in raw format
man/man8/rdma-resource.8 | 5 +++++ man/man8/rdma.8 | 4 ++++ rdma/include/uapi/rdma/rdma_netlink.h | 8 +++++++ rdma/rdma.c | 10 +++++++-- rdma/rdma.h | 7 +++++-- rdma/res-cq.c | 20 ++++++++++++++++-- rdma/res-mr.c | 21 +++++++++++++++++-- rdma/res-qp.c | 20 ++++++++++++++++-- rdma/res.h | 30 ++++++++++++++++++++++----- rdma/utils.c | 20 ++++++++++++++++++ 10 files changed, 130 insertions(+), 15 deletions(-)
--
2.26.2
PATCH 1/4 rdma: update uapi headers
From: Maor Gottlieb <[email protected]>
Update rdma_netlink.h file upto kernel commit ba1f4991cc55 ("RDMA: Add support to dump resource tracker in RAW format")
--- rdma/include/uapi/rdma/rdma_netlink.h | 8 ++++++++ 1 file changed, 8 insertions(+)
rdma/include/uapi/rdma/rdma_netlink.h +8 -0
@@ -287,6 +287,12 @@ enum rdma_nldev_command {RDMA_NLDEV_CMD_STAT_DEL , +RDMA_NLDEV_CMD_RES_QP_GET_RAW ,/* can dump */ + +RDMA_NLDEV_CMD_RES_CQ_GET_RAW ,/* can dump */ + +RDMA_NLDEV_CMD_RES_MR_GET_RAW ,/* can dump */ + RDMA_NLDEV_NUM_OPS };
@@ -525,6 +531,8 @@ enum rdma_nldev_attr {*/ RDMA_NLDEV_ATTR_DEV_DIM ,/* u8 */ + RDMA_NLDEV_ATTR_RES_RAW, /* binary */ +/* * Always the end */
Signature · 1 line
2.26.2
Re: [PATCH iproute2-next v1 1/4] rdma: update uapi headers
Quoted 4 lines · Leon Romanovsky
From: Maor Gottlieb <[email protected]>
Update
rdma_netlink.hfile upto kernel commitba1f4991cc55("RDMA: Add support to dump resource tracker in RAW format")
David,
The SHA was changed because of the rebase on top of our testing branch. 65959522f806 RDMA: Add support to dump resource tracker in RAW format
Do you want me to resend the series?
Thanks
Re: [PATCH iproute2-next v1 1/4] rdma: update uapi headers
Quoted 6 lines · Leon Romanovsky
Quoted 4 lines · Leon Romanovsky
From: Maor Gottlieb <[email protected]>
Update
rdma_netlink.hfile upto kernel commitba1f4991cc55("RDMA: Add support to dump resource tracker in RAW format")
Quoted 7 lines
David,
The SHA was changed because of the rebase on top of our testing branch.
65959522f806RDMA: Add support to dump resource tracker in RAW formatDo you want me to resend the series?
no need just for that; I can fix before applying.
Re: [PATCH iproute2-next v1 1/4] rdma: update uapi headers
In reply to Leon RomanovskyQuoted 6 lines · Leon Romanovsky
rdma/include/uapi/rdma/rdma_netlink.h +0
-0@@ -287,6 +287,12 @@ enum rdma_nldev_command {
rdma/include/uapi/rdma/rdma_netlink.h enum rdma_nldev_command { · line 297
Quoted 10 lines
rdma/include/uapi/rdma/rdma_netlink.h +6
-0RDMA_NLDEV_CMD_STAT_DEL , +RDMA_NLDEV_CMD_RES_QP_GET_RAW ,/* can dump */ + +RDMA_NLDEV_CMD_RES_CQ_GET_RAW ,/* can dump */ + +RDMA_NLDEV_CMD_RES_MR_GET_RAW ,/* can dump */ + RDMA_NLDEV_NUM_OPS };
you are inserting new commands in the middle which breaks existing users of this API.
Re: [PATCH iproute2-next v1 1/4] rdma: update uapi headers
Quoted 9 lines · David Ahern
Quoted 8 lines · Leon Romanovsky
rdma/include/uapi/rdma/rdma_netlink.h +0
-0@@ -287,6 +287,12 @@ enum rdma_nldev_command { RDMA_NLDEV_CMD_STAT_DEL,
Quoted 11 lines
rdma/include/uapi/rdma/rdma_netlink.h enum rdma_nldev_command { · line 297
Quoted 8 lines
rdma/include/uapi/rdma/rdma_netlink.h +6
-0+RDMA_NLDEV_CMD_RES_QP_GET_RAW ,/* can dump */ + +RDMA_NLDEV_CMD_RES_CQ_GET_RAW ,/* can dump */ + +RDMA_NLDEV_CMD_RES_MR_GET_RAW ,/* can dump */ + RDMA_NLDEV_NUM_OPS };you are inserting new commands in the middle which breaks existing users of this API.
RDMA_NLDEV_NUM_OPS is not a command, but enum item to help calculate array size, exactly like devlink_command in include/uapi/linux/devlink.h.
Thanks
Quoted 1 line
Re: [PATCH iproute2-next v1 1/4] rdma: update uapi headers
Quoted 2 lines · Leon Romanovsky
RDMA_NLDEV_NUM_OPSis not a command, but enum item to help calculate array size, exactly likedevlink_commandininclude/uapi/linux/devlink.h.
ok. usually the last field is __FOO_MAX not FOO_NUM.
Re: [PATCH iproute2-next v1 1/4] rdma: update uapi headers
Quoted 5 lines · David Ahern
Quoted 2 lines · Leon Romanovsky
RDMA_NLDEV_NUM_OPSis not a command, but enum item to help calculate array size, exactly likedevlink_commandininclude/uapi/linux/devlink.h.ok. usually the last field is
__FOO_MAXnotFOO_NUM.
I used same naming style as we had for other enums in rdma_netlink.h.
Thanks
PATCH 2/4 rdma: Add support to get QP in raw format
In reply to Leon RomanovskyFrom: Maor Gottlieb <[email protected]>
Add 'raw' argument to get the resource in raw format. When RDMA_NLDEV_ATTR_RES_RAW is set in the netlink message, then the resource fields are in raw format, print it as byte array.
Example:
$rdma res show qp link rocep0s12f0/1 lqpn 1137 -j -r [{"ifindex":7,"ifname":"mlx5_1","port":1, "data":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,...]}]
--- man/man8/rdma-resource.8 | 5 +++++ man/man8/rdma.8 | 4 ++++ rdma/rdma.c | 10 ++++++++-- rdma/rdma.h | 7 +++++-- rdma/res-qp.c | 20 ++++++++++++++++++-- rdma/res.h | 26 +++++++++++++++++++++----- rdma/utils.c | 20 ++++++++++++++++++++ 7 files changed, 81 insertions(+), 11 deletions(-)
man/man8/rdma-resource.8 +5 -0
@@ -83,6 +83,11 @@ rdma res show qp link mlx5_4/1 lqpn 0-6 Limit to specific Local QPNs. .RE .PP +rdma res show qp link mlx5_4/1 lqpn 6 -r +.RS 4 +Driver specific details in raw format. +.RE +.PP rdma resource show cm_id dst-port 7174 .RS 4 Show CM_IDs with destination ip port of 7174.
man/man8/rdma.8 +4 -0
@@ -51,6 +51,10 @@ If there were any errors during execution of the commands, the application retur .BR "\-d" , " --details" Output detailed information. Adding a second \-d includes driver-specific details. +.TP +.BR "\-r" , " --raw" +Output includes driver-specific details in raw format. + .TP .BR "\-p" , " --pretty" When combined with -j generate a pretty JSON output.
rdma/rdma.c +8 -2
@@ -13,7 +13,7 @@ static void help(char *name)pr_out ("Usage: %s [ OPTIONS ] OBJECT { COMMAND | help }\n" " %s [ -f[orce] ] -b[atch] filename\n" "where OBJECT := { dev | link | resource | system | statistic | help }\n" -" OPTIONS := { -V[ersion] | -d[etails] | -j[son] | -p[retty]}\n" , name, name); +" OPTIONS := { -V[ersion] | -d[etails] | -j[son] | -p[retty] -r[aw]}\n" , name, name); }static int cmd_help (struct rd *rd)
@@ -112,6 +112,7 @@ int main(int argc, char **argv) { "json", no_argument, NULL, 'j' }, { "pretty", no_argument, NULL, 'p' }, { "details", no_argument, NULL, 'd' }, + { "raw", no_argument, NULL, 'r' }, { "force", no_argument, NULL, 'f' }, { "batch", required_argument, NULL, 'b' }, { NULL, 0, NULL, 0 }
@@ -120,6 +121,7 @@ int main(int argc, char **argv)const char *batch_file =NULL ;bool show_details = false;bool json_output = false; +bool show_raw = false;bool force = false;struct rd rd = {};char *filename;
@@ -127,7 +129,7 @@ int main(int argc, char **argv)int err; filename =basename (argv[0 ]); -while ((opt =getopt_long (argc, argv,":Vhdpjfb:" , +while ((opt =getopt_long (argc, argv,":Vhdrpjfb:" , long_options,NULL )) >=0 ) {switch (opt) {case 'V' :
@@ -143,6 +145,9 @@ int main(int argc, char **argv)else show_details = true;break ; +case 'r' : + show_raw = true; +break ;case 'j' : json_output =1 ;break ;
@@ -172,6 +177,7 @@ int main(int argc, char **argv) rd.show_driver_details = show_driver_details; rd.json_output = json_output; rd.pretty_output = pretty; + rd.show_raw = show_raw; err =rd_init (&rd, filename);if (err)
rdma/rdma.h +5 -2
@@ -57,8 +57,9 @@ struct rd {int argc;char **argv;char *filename; -bool show_details; -bool show_driver_details; + uint8_t show_details:1; + uint8_t show_driver_details:1; + uint8_t show_raw:1;struct list_head dev_map_list;uint32_t dev_idx;uint32_t port_idx;
@@ -134,9 +135,11 @@ int rd_attr_check(const struct nlattr *attr, int *typep);* Print helpers */ void print_driver_table (struct rd *rd,struct nlattr *tb); +void print_raw_data (struct rd *rd,struct nlattr **nla_line);void newline (struct rd *rd);void newline_indent (struct rd *rd);void print_on_off (struct rd *rd,const char *key_str,bool on); +void print_raw_data (struct rd *rd,struct nlattr **nla_line);#define MAX_LINE_LENGTH 80#endif /* _RDMA_TOOL_H_ */
rdma/res-qp.c +18 -2
@@ -64,6 +64,20 @@ static void print_pathmig(struct rd *rd, uint32_t val, struct nlattr **nla_line) "path-mig-state %s ", path_mig_to_str(val)); } +static int res_qp_line_raw (struct rd *rd,const char *name,int idx, +struct nlattr **nla_line) +{ +if (!nla_line[RDMA_NLDEV_ATTR_RES_RAW ]) +return MNL_CB_ERROR ; + +open_json_object (NULL ); +print_link (rd, idx, name, rd->port_idx, nla_line); +print_raw_data (rd, nla_line); +newline (rd); + +return MNL_CB_OK ; +} +static int res_qp_line (struct rd *rd,const char *name,int idx,struct nlattr **nla_line) {
@@ -184,7 +198,8 @@ int res_qp_idx_parse_cb(const struct nlmsghdr *nlh, void *data) name =mnl_attr_get_str (tb[RDMA_NLDEV_ATTR_DEV_NAME ]); idx =mnl_attr_get_u32 (tb[RDMA_NLDEV_ATTR_DEV_INDEX ]); -return res_qp_line (rd, name, idx, tb); +return (rd->show_raw) ?res_qp_line_raw (rd, name, idx, tb) : +res_qp_line (rd, name, idx, tb); }int res_qp_parse_cb (const struct nlmsghdr *nlh,void *data)
@@ -212,7 +227,8 @@ int res_qp_parse_cb(const struct nlmsghdr *nlh, void *data)if (ret !=MNL_CB_OK )break ; - ret =res_qp_line (rd, name, idx, nla_line); + ret = (rd->show_raw) ?res_qp_line_raw (rd, name, idx, nla_line) : +res_qp_line (rd, name, idx, nla_line);if (ret !=MNL_CB_OK )break ; }
rdma/res.h +21 -5
@@ -23,24 +23,40 @@ int res_cm_id_idx_parse_cb(const struct nlmsghdr *nlh, void *data);int res_qp_parse_cb (const struct nlmsghdr *nlh,void *data);int res_qp_idx_parse_cb (const struct nlmsghdr *nlh,void *data); +static inline uint32_t res_get_command (uint32_t command,struct rd *rd) +{ +if (!rd->show_raw) +return command; + +switch (command) { +case RDMA_NLDEV_CMD_RES_QP_GET : +return RDMA_NLDEV_CMD_RES_QP_GET_RAW ; +default : +return command; + } +} + #define RES_FUNC(name, command, valid_filters, strict_port, id) \ static inline int _##name(struct rd *rd) \ { \ - uint32_t idx; \ + uint32_t idx, _command; \ int ret; \ + _command = res_get_command(command, rd); \ if (id) { \ ret = rd_doit_index(rd, &idx); \ if (ret) { \ rd->suppress_errors = true; \ - ret = _res_send_idx_msg(rd, command, \ + ret = _res_send_idx_msg(rd, _command, \ name##_idx_parse_cb, \ idx, id); \ - if (!ret) \ + if (!ret || rd->show_raw) \ return ret; \ - /* Fallback for old systems without .doit callbacks */ \ + /* Fallback for old systems without .doit callbacks. \ + * Kernel that supports raw, for sure supports doit. \ + */ \ } \ } \ - return _res_send_msg(rd, command, name##_parse_cb); \ + return _res_send_msg(rd, _command, name##_parse_cb); \ } \ static inline int name(struct rd *rd) \ { \
rdma/utils.c +20 -0
@@ -450,6 +450,7 @@ static const enum mnl_attr_data_type nldev_policy[RDMA_NLDEV_ATTR_MAX] = { [RDMA_NLDEV_ATTR_STAT_RES] = MNL_TYPE_U32, [RDMA_NLDEV_ATTR_STAT_AUTO_MODE_MASK] = MNL_TYPE_U32, [RDMA_NLDEV_ATTR_DEV_DIM] = MNL_TYPE_U8, + [RDMA_NLDEV_ATTR_RES_RAW] = MNL_TYPE_BINARY, };int rd_attr_check (const struct nlattr *attr,int *typep)
@@ -890,6 +891,25 @@ static int print_driver_entry(struct rd *rd, struct nlattr *key_attr,return ret; } +void print_raw_data (struct rd *rd,struct nlattr **nla_line) +{ +uint8_t *data; +uint32_t len; +int i =0 ; + +if (!rd->show_raw) +return ; + + len =mnl_attr_get_payload_len (nla_line[RDMA_NLDEV_ATTR_RES_RAW ]); + data =mnl_attr_get_payload (nla_line[RDMA_NLDEV_ATTR_RES_RAW ]); +open_json_array (PRINT_JSON ,"data" ); +while (i < len) { +print_color_uint (PRINT_ANY ,COLOR_NONE ,NULL ,"%d" , data[i]); + i++; + } +close_json_array (PRINT_ANY ,">" ); +} +void print_driver_table (struct rd *rd,struct nlattr *tb) {int print_type =RDMA_NLDEV_PRINT_TYPE_UNSPEC ;
Signature · 1 line
2.26.2
PATCH 3/4 rdma: Add support to get CQ in raw format
In reply to Leon RomanovskyFrom: Maor Gottlieb <[email protected]>
Add the required support to print CQ data in raw format. Example:
$rdma res show cq dev mlx5_2 cqn 1 -r -j
[{"ifindex":8,"ifname":"mlx5_2",
"data":[0,4,255,254,0,0,0,0,0,0,0,0,16,28,...]}]
--- rdma/res-cq.c | 20 ++++++++++++++++++-- rdma/res.h | 2 ++ 2 files changed, 20 insertions(+), 2 deletions(-)
rdma/res-cq.c +18 -2
@@ -39,6 +39,20 @@ static void print_cq_dim_setting(struct rd *rd, struct nlattr *attr)print_on_off (rd,"adaptive-moderation" , dim_setting); } +static int res_cq_line_raw (struct rd *rd,const char *name,int idx, +struct nlattr **nla_line) +{ +if (!nla_line[RDMA_NLDEV_ATTR_RES_RAW ]) +return MNL_CB_ERROR ; + +open_json_object (NULL ); +print_dev (rd, idx, name); +print_raw_data (rd, nla_line); +newline (rd); + +return MNL_CB_OK ; +} +static int res_cq_line (struct rd *rd,const char *name,int idx,struct nlattr **nla_line) {
@@ -128,7 +142,8 @@ int res_cq_idx_parse_cb(const struct nlmsghdr *nlh, void *data) name =mnl_attr_get_str (tb[RDMA_NLDEV_ATTR_DEV_NAME ]); idx =mnl_attr_get_u32 (tb[RDMA_NLDEV_ATTR_DEV_INDEX ]); -return res_cq_line (rd, name, idx, tb); +return (rd->show_raw) ?res_cq_line_raw (rd, name, idx, tb) : +res_cq_line (rd, name, idx, tb); }int res_cq_parse_cb (const struct nlmsghdr *nlh,void *data)
@@ -156,7 +171,8 @@ int res_cq_parse_cb(const struct nlmsghdr *nlh, void *data)if (ret !=MNL_CB_OK )break ; - ret =res_cq_line (rd, name, idx, nla_line); + ret = (rd->show_raw) ?res_cq_line_raw (rd, name, idx, nla_line) : +res_cq_line (rd, name, idx, nla_line);if (ret !=MNL_CB_OK )break ;
rdma/res.h +2 -0
@@ -31,6 +31,8 @@ static inline uint32_t res_get_command(uint32_t command, struct rd *rd)switch (command) {case RDMA_NLDEV_CMD_RES_QP_GET :return RDMA_NLDEV_CMD_RES_QP_GET_RAW ; +case RDMA_NLDEV_CMD_RES_CQ_GET : +return RDMA_NLDEV_CMD_RES_CQ_GET_RAW ;default :return command; }
Signature · 1 line
2.26.2
PATCH 4/4 rdma: Add support to get MR in raw format
In reply to Leon RomanovskyFrom: Maor Gottlieb <[email protected]>
Add the required support to print MR data in raw format. Example:
$rdma res show mr dev mlx5_1 mrn 2 -r -j [{"ifindex":7,"ifname":"mlx5_1", "data":[0,4,255,254,0,0,0,0,0,0,0,0,16,28,0,216,...]}]
--- rdma/res-mr.c | 21 +++++++++++++++++++-- rdma/res.h | 2 ++ 2 files changed, 21 insertions(+), 2 deletions(-)
rdma/res-mr.c +19 -2
@@ -7,6 +7,20 @@#include "res.h" #include <inttypes.h> +static int res_mr_line_raw (struct rd *rd,const char *name,int idx, +struct nlattr **nla_line) +{ +if (!nla_line[RDMA_NLDEV_ATTR_RES_RAW ]) +return MNL_CB_ERROR ; + +open_json_object (NULL ); +print_dev (rd, idx, name); +print_raw_data (rd, nla_line); +newline (rd); + +return MNL_CB_OK ; +} +static int res_mr_line (struct rd *rd,const char *name,int idx,struct nlattr **nla_line) {
@@ -69,6 +83,7 @@ static int res_mr_line(struct rd *rd, const char *name, int idx,print_comm (rd, comm, nla_line);print_driver_table (rd, nla_line[RDMA_NLDEV_ATTR_DRIVER ]); +print_raw_data (rd, nla_line);newline (rd); out:
@@ -91,7 +106,8 @@ int res_mr_idx_parse_cb(const struct nlmsghdr *nlh, void *data) name =mnl_attr_get_str (tb[RDMA_NLDEV_ATTR_DEV_NAME ]); idx =mnl_attr_get_u32 (tb[RDMA_NLDEV_ATTR_DEV_INDEX ]); -return res_mr_line (rd, name, idx, tb); +return (rd->show_raw) ?res_mr_line_raw (rd, name, idx, tb) : +res_mr_line (rd, name, idx, tb); }int res_mr_parse_cb (const struct nlmsghdr *nlh,void *data)
@@ -119,7 +135,8 @@ int res_mr_parse_cb(const struct nlmsghdr *nlh, void *data)if (ret !=MNL_CB_OK )break ; - ret =res_mr_line (rd, name, idx, nla_line); + ret = (rd->show_raw) ?res_mr_line_raw (rd, name, idx, nla_line) : +res_mr_line (rd, name, idx, nla_line);if (ret !=MNL_CB_OK )break ; }
rdma/res.h +2 -0
@@ -33,6 +33,8 @@ static inline uint32_t res_get_command(uint32_t command, struct rd *rd)return RDMA_NLDEV_CMD_RES_QP_GET_RAW ;case RDMA_NLDEV_CMD_RES_CQ_GET :return RDMA_NLDEV_CMD_RES_CQ_GET_RAW ; +case RDMA_NLDEV_CMD_RES_MR_GET : +return RDMA_NLDEV_CMD_RES_MR_GET_RAW ;default :return command; }
Signature · 1 line
2.26.2
David Ahern
AppliedQuoted 11 lines · Leon Romanovsky
From: Leon Romanovsky <[email protected]>
Changelog:
v1:
* Kernel part was accepted, so this series has correct SHA for the kernel header update patch.
* Aligned implementation to the final kernel solution of query interface.
v0:
https://lore.kernel.org/linux-rdma/[email protected]
Quoted 9 lines
-----------------------------------------------------------------------------
Hi,
The following series adds support to get the RDMA resource data in RAW format. The main motivation for doing this is to enable vendors to return the entire QP/CQ/MR data without a need from the vendor to set each field separately.
applied to iproute2-next. Thanks
Quoted 16 lines · David Ahern
Quoted 15 lines · Leon Romanovsky
From: Leon Romanovsky <[email protected]>
Changelog:
v1:
* Kernel part was accepted, so this series has correct SHA for the kernel header update patch.
* Aligned implementation to the final kernel solution of query interface.
v0:
https://lore.kernel.org/linux-rdma/[email protected]-----------------------------------------------------------------------------
Hi,
Quoted 7 lines
Quoted 5 lines
The following series adds support to get the RDMA resource data in RAW format. The main motivation for doing this is to enable vendors to return the entire QP/CQ/MR data without a need from the vendor to set each field separately.
applied to iproute2-next. Thanks
Thanks