Dynamic rdma link creation
Subsystems (3)
v3 cover Dynamic rdma link creation
This series adds rdmatool support for creating/deleting rdma links. This will be used, mainly, by soft rdma drivers to allow adding/deleting rdma links over netdev interfaces. It provides the user side for the following kernel changes merged in linux-5.1.
Changes since v2:
- move checks for required parameters in the parameter handlers
- move final 'link add' processing to link_add_netdev()
- added reviewed-by tags
Changes since v1:
- move error receive checking from rd_sendrecv_msg() to rd_recv_msg().
- Add rd->suppress_errors to allow control over whether errors when reading a response should be ignored. Namely: resource queries can get errors like "none found" when querying for resources, and this error should not be displayed. So on a rd object basis, error suppression can be controlled.
- Rebased on rdma/for-next UABI (no need to sync rdma_netlink.h now)
- use chains of struct rd_cmd and rd_exec_cmd vs open coding the parsing for the 'link add' command.
- minor nit resolution
- added .mailmap file. If this is not desired for iproute2, then please drop the patch.
Changes since RFC:
- add rd_sendrecv_msg() and make use of it in dev_set as well as the new link commands.
- fixed problems with the man pages
- changed the command line to use "netdev" as the keyword for the network device, do avoid confused with the ib_device name.
- got rid of the "type" parameter for link delete. Also pass down the device index instead of the name, using the common rd services for validating the device name and fetching the index.
Thanks,
Steve.
Steve Wise (4):
Add .mailmap file
rdma: add helper rd_sendrecv_msg()
rdma: add 'link add/delete' commands
rdma: man page update for link add/delete
.mailmap | 8 +++++ man/man8/rdma-link.8 | 47 ++++++++++++++++++++++++++ rdma/dev.c | 2 +- rdma/link.c | 78 ++++++++++++++++++++++++++++++++++++++++++++ rdma/rdma.h | 4 +++ rdma/res.h | 1 + rdma/utils.c | 18 ++++++++++ 7 files changed, 157 insertions(+), 1 deletion(-) create mode 100644 .mailmap
Signature · 1 line
2.20.1
PATCH v3 1/4 Add .mailmap file
.mailmap allows tracking multiple email addresses to the proper user name.
--- .mailmap | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 .mailmap
.mailmap new file +8 -0
@@ -0,0 +1,8 @@ +# +# This list is used by git-shortlog to fix a few botched name translations +# in the git archive, either because the author's full name was messed up +# and/or not always written the same way, making contributions from the +# same person appearing not to be so or badly displayed. +# +Steve Wise <[email protected]> <[email protected]> +Steve Wise <[email protected]> <[email protected]>
Signature · 1 line
2.20.1
PATCH v3 2/4 rdma: add helper rd_sendrecv_msg()
In reply to Steve WiseThis function sends the constructed netlink message and then receives the response.
Change rd_recv_msg() to display any error messages.
Change 'rdma dev set' to use rd_sendrecv_msg().
--- rdma/dev.c | 2 +- rdma/rdma.h | 2 ++ rdma/res.h | 1 + rdma/utils.c | 18 ++++++++++++++++++ 4 files changed, 22 insertions(+), 1 deletion(-)
rdma/dev.c +1 -1
@@ -268,7 +268,7 @@ static int dev_set_name(struct rd *rd)mnl_attr_put_u32 (rd->nlh,RDMA_NLDEV_ATTR_DEV_INDEX , rd->dev_idx);mnl_attr_put_strz (rd->nlh,RDMA_NLDEV_ATTR_DEV_NAME ,rd_argv (rd)); -return rd_send_msg (rd); +return rd_sendrecv_msg (rd, seq); }static int dev_one_set (struct rd *rd)
rdma/rdma.h +2 -0
@@ -68,6 +68,7 @@ struct rd {json_writer_t *jw;bool json_output;bool pretty_output; +bool suppress_errors;struct list_head filter_list; };
@@ -119,6 +120,7 @@ bool rd_is_string_filtered_attr(struct rd *rd, const char *key, const char *val,*/ int rd_send_msg (struct rd *rd);int rd_recv_msg (struct rd *rd,mnl_cb_t callback,void *data,uint32_t seq); +int rd_sendrecv_msg (struct rd *rd,unsigned int seq);void rd_prepare_msg (struct rd *rd,uint32_t cmd,uint32_t *seq,uint16_t flags);int rd_dev_init_cb (const struct nlmsghdr *nlh,void *data);int rd_attr_cb (const struct nlattr *attr,void *data);
rdma/res.h +1 -0
@@ -31,6 +31,7 @@ int res_qp_idx_parse_cb(const struct nlmsghdr *nlh, void *data);if (id) { \ ret =rd_doit_index (rd, &idx); \if (ret) { \ + rd->suppress_errors = true; \ ret =_res_send_idx_msg (rd, command, \ name##_idx_parse_cb, \ idx, id); \
rdma/utils.c +18 -0
@@ -693,10 +693,28 @@ int rd_recv_msg(struct rd *rd, mnl_cb_t callback, void *data, unsigned int seq) ret =mnl_cb_run (buf, ret, seq, portid, callback, data); }while (ret >0 ); +if (ret <0 && !rd->suppress_errors) +perror ("error" ); +mnl_socket_close (rd->nl);return ret; } +static int null_cb (const struct nlmsghdr *nlh,void *data) +{ +return MNL_CB_OK ; +} + +int rd_sendrecv_msg (struct rd *rd,unsigned int seq) +{ +int ret; + + ret =rd_send_msg (rd); +if (!ret) + ret =rd_recv_msg (rd, null_cb, rd, seq); +return ret; +} +static struct dev_map *_dev_map_lookup (struct rd *rd,const char *dev_name) {struct dev_map *dev_map;
Signature · 1 line
2.20.1
PATCH v3 3/4 rdma: add 'link add/delete' commands
In reply to Steve WiseAdd new 'link' subcommand 'add' and 'delete' to allow binding a soft-rdma device to a netdev interface.
EG:
rdma link add rxe_eth0 type rxe netdev eth0
rdma link delete rxe_eth0
--- rdma/link.c | 78 +++++++++++++++++++++++++++++++++++++++++++++++++++++ rdma/rdma.h | 2 ++ 2 files changed, 80 insertions(+)
rdma/link.c +78 -0
@@ -9,6 +9,9 @@static int link_help (struct rd *rd) {pr_out ("Usage: %s link show [DEV/PORT_INDEX]\n" , rd->filename); +pr_out ("Usage: %s link add NAME type TYPE netdev NETDEV\n" , + rd->filename); +pr_out ("Usage: %s link delete NAME\n" , rd->filename);return 0 ; }
@@ -336,10 +339,85 @@ static int link_show(struct rd *rd)return rd_exec_link (rd, link_one_show, true); } +static int link_add_netdev (struct rd *rd) +{ +char *link_netdev; +uint32_t seq; + +if (rd_no_arg (rd)) { +pr_err ("Please provide a net device name.\n" ); +return -EINVAL ; + } + + link_netdev =rd_argv (rd); +rd_prepare_msg (rd,RDMA_NLDEV_CMD_NEWLINK , &seq, + (NLM_F_REQUEST |NLM_F_ACK )); +mnl_attr_put_strz (rd->nlh,RDMA_NLDEV_ATTR_DEV_NAME , rd->link_name); +mnl_attr_put_strz (rd->nlh,RDMA_NLDEV_ATTR_LINK_TYPE , rd->link_type); +mnl_attr_put_strz (rd->nlh,RDMA_NLDEV_ATTR_NDEV_NAME , link_netdev); +return rd_sendrecv_msg (rd, seq); +} + +static int link_add_type (struct rd *rd) +{ +const struct rd_cmd cmds[] = { + {NULL , link_help}, + {"netdev" , link_add_netdev}, + {0 } + }; + +if (rd_no_arg (rd)) { +pr_err ("Please provide a link type name.\n" ); +return -EINVAL ; + } + rd->link_type =rd_argv (rd); +rd_arg_inc (rd); +return rd_exec_cmd (rd, cmds,"parameter" ); +} + +static int link_add (struct rd *rd) +{ +const struct rd_cmd cmds[] = { + {NULL , link_help}, + {"type" , link_add_type}, + {0 } + }; + +if (rd_no_arg (rd)) { +pr_err ("Please provide a link name to add.\n" ); +return -EINVAL ; + } + rd->link_name =rd_argv (rd); +rd_arg_inc (rd); + +return rd_exec_cmd (rd, cmds,"parameter" ); +} + +static int _link_del (struct rd *rd) +{ +uint32_t seq; + +if (!rd_no_arg (rd)) { +pr_err ("Unknown parameter %s\n" ,rd_argv (rd)); +return -EINVAL ; + } +rd_prepare_msg (rd,RDMA_NLDEV_CMD_DELLINK , &seq, + (NLM_F_REQUEST |NLM_F_ACK )); +mnl_attr_put_u32 (rd->nlh,RDMA_NLDEV_ATTR_DEV_INDEX , rd->dev_idx); +return rd_sendrecv_msg (rd, seq); +} + +static int link_del (struct rd *rd) +{ +return rd_exec_require_dev (rd, _link_del); +} +int cmd_link (struct rd *rd) {const struct rd_cmd cmds[] = { {NULL , link_show }, + {"add" , link_add }, + {"delete" , link_del }, {"show" , link_show }, {"list" , link_show }, {"help" , link_help },
rdma/rdma.h +2 -0
@@ -70,6 +70,8 @@ struct rd {bool pretty_output;bool suppress_errors;struct list_head filter_list; +char *link_name; +char *link_type; };struct rd_cmd {
Signature · 1 line
2.20.1
PATCH v3 4/4 rdma: man page update for link add/delete
In reply to Steve WiseUpdate the 'rdma link' man page with 'link add/delete' info.
--- man/man8/rdma-link.8 | 47 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 47 insertions(+)
man/man8/rdma-link.8 +47 -0
@@ -22,6 +22,18 @@ rdma-link \- rdma link configuration .B rdma link show .RI "[ " DEV/PORT_INDEX " ]" +.ti -8 +.B rdma link add +.BR NAME +.BR type +.BR TYPE +.BR netdev +.BR NETDEV + +.ti -8 +.B rdma link delete +.RI NAME + .ti -8 .B rdma link help
@@ -33,6 +45,31 @@ rdma-link \- rdma link configuration - specifies the RDMA link to show. If this argument is omitted all links are listed. +.SS rdma link add NAME type TYPE netdev NETDEV - add an rdma link for the specified type to the network device +.sp +.BR NAME +- specifies the new name of the rdma link to add + +.BR TYPE +- specifies which rdma type to use. Link types: +.sp +.in +8 +.B rxe +- Soft RoCE driver +.sp +.B siw +- Soft iWARP driver +.in -8 + +.BR NETDEV +- specifies the network device to which the link is bound + +.SS rdma link delete NAME - delete an rdma link +.PP +.BR NAME +- specifies the name of the rdma link to delete +.PP + .SH "EXAMPLES" .PP rdma link show
@@ -45,6 +82,16 @@ rdma link show mlx5_2/1 Shows the state of specified rdma link. .RE .PP +rdma link add rxe_eth0 type rxe netdev eth0 +.RS 4 +Adds a RXE link named rxe_eth0 to network device eth0 +.RE +.PP +rdma link del rxe_eth0 +.RS 4 +Removes RXE link rxe_eth0 +.RE +.PP .SH SEE ALSO .BR rdma (8),
Signature · 1 line
2.20.1
Re: [PATCH v3 iproute2-next 4/4] rdma: man page update for link add/delete
Quoted 7 lines · Steve Wise
Update the 'rdma link' man page with 'link add/delete' info.
Signed-off-by Steve Wise <[email protected]>--- man/man8/rdma-link.8 | 47 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 47 insertions(+)
Thanks,
Quoted 5 lines · Steve Wise
This series adds rdmatool support for creating/deleting rdma links. This will be used, mainly, by soft rdma drivers to allow adding/deleting rdma links over netdev interfaces. It provides the user side for the following kernel changes merged in linux-5.1.
Thanks a lot Steve,
Looks very good.
David Ahern
AppliedQuoted 5 lines · Steve Wise
This series adds rdmatool support for creating/deleting rdma links. This will be used, mainly, by soft rdma drivers to allow adding/deleting rdma links over netdev interfaces. It provides the user side for the following kernel changes merged in linux-5.1.
applied to iproute2-next. Thanks