Skip to contentSkip to conversation

RDMA tool

RFC In review Leon Romanovsky 15 participants 33 messages May 4, 2017 – May 9, 2017 10 messages not in the archive

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

cover RDMA tool

From: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/[email protected]>

----
This is initial phase to understand if user experience for this tool fits RDMA and netdev communities exepectations. Also I would like to get feedback if it is really worth to provide legacy sysfs for old kernels, or maybe I should implement netlink from the beginning and abandon sysfs completely.
-----

Hi,

Please find below, the patch set with initial implementation of configuration tool for RDMA subsystem, which will be supplementary tool to already existed tools in netdev community (ip, devlink, ethtool, ..).

In opposite to netdev community, where standard tools exist to configure and present different devices abilities, RDMA subsystem historically lacked it.

Following our discussion both in mailing list [1] and at the LPC 2016 [2], we would like to propose this RDMA tool to be part of iproute2 package and finally improve this situation.

The development of tool was influenced by ip and devlink tools. This implies to the object->command interface and naming convention.

In order to close object model, ensure reuse of existing code and make this tool usable from day one, we decided to implement wrappers over legacy sysfs prior to implementing netlink functionality. As a nice bonus, it will allow to use this tool with old kernels too.

It is important to mention that any future extension will be required to be done with netlink, so for already existing objects small conversion to netlink will be unavoidable.

  # rdma -h
  Usage: rdma [ OPTIONS ] OBJECT { COMMAND | help }
  where  OBJECT := { dev | link | ipoib | memory | stats | protocols | providers | monitor }
  OPTIONS := { -V[ersion] }

* DEV object equals to CA in IBTA specification and will provide a way to configure/present settings relevant to specific struct ib_device.

* LINK object represents port in IBTA specification and will give access to struct ib_port_immutable. From the day one, It prints netdev name of the corresponding IB port that makes ibdev2netdev script redundant.

* IPoIB object is supposed to be specific for IP-over-Infiniband upper layer protocol [3]. This ULP was mainly configured by combination of various sysfs knobs together with ethtool. Such situation adds challenges to add new and expose old configuration settings due to the mix between different subsystems.

* MEMORY object will be used to configure memory related settings, e.g. on-demand-paging (ODP), force-mr (force usage of MRs for RDMA READ/WRITE operation).

* STATS object is needed for everything related to statistics (per-PID, per-QP, per-device etc.). Despite the fact that RDMA devices provide extensive set of counters, the decision was to implement it in netlink directly, because there is a need to add filter mechanism to them, which doesn't exist now.

* PROTOCOLS object is going to be used for device special treatment of global to protocol settings (e.g. set device in RoCEv2 mode as a default, instead of RoCEv1, instead of configfs).

* PROVIDERS objects gives ability to get specific to the device information, like supported kABI objects [4].

* MONITOR object is needed to debug netlink communication and will follow standard functionality, which exists in ip and devlink tools.

There are number of ULPs which are not covered by this tool yet:
* HFI-VNIC - I have no access to the HW and believe that Intel will add native object support for it.
* Other storage related ULPs (iSER and SRP) were not introduced too, because they have special tools (scci-target-utils) to configure them. However it will be pretty straightforward to introduce new object, if there is demand for it.

At the initial stage, we implemented infrastructure to read legacy sysfs entries (Patch #1), initial man pages (Patch #7) and provided future object examples (Patch #2-6) to allow parallel development.

Following patches will focus on cleaning user interface, parsing other relevant entries in similar fashion to the link capability mask (Patch #8) and providing netlink interface.

These patches were tested with two following setups:
* Setup A:
- Two Mellanox ConnectX-4 devices (one port)
- One Mellanox Connect-IB device (two ports)
* Setup B:
- One Mellanox ConnectX-4 device (one port)
- One Mellanox ConnectX-3 Pro device (two ports)

Please consider the inclusion of the RDMA tool into iproute2 package, so other participants will be able to speed up development.

[1] https://www.mail-archive.com/[email protected]/msg148523.html
[2] http://www.medkio.com/talks/lpc_debug.pdf
[3] https://tools.ietf.org/html/rfc4392
[4] http://marc.info/?l=linux-rdma&m=149261526916544&w=2

TODO: Add json output

Cc Stephen Hemminger <[email protected]>
Cc Doug Ledford <[email protected]>
Cc Jiri Pirko <jiri-VPRAkNaXOzVWk0Htik3J/[email protected]>
Cc Ariel Almog <ariela-VPRAkNaXOzVWk0Htik3J/[email protected]>
Cc Dennis Dalessandro <[email protected]>
Cc Ram Amrani <ram.amrani-YGCgFSpz5w/[email protected]>
Cc Bart Van Assche <[email protected]>
Cc Sagi Grimberg <[email protected]>
Cc Jason Gunthorpe <jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
Cc Christoph Hellwig <[email protected]>
Cc Or Gerlitz <ogerlitz-VPRAkNaXOzVWk0Htik3J/[email protected]>
Cc Linux RDMA <[email protected]>
Cc Linux Netdev <[email protected]>

Leon Romanovsky (8):
rdma: Add basic infrastructure for RDMA tool
rdma: Add dev object
rdma: Add link object
rdma: Add IPoIB object
rdma: Add memory object
rdma: add stubs for future objects
man: rdma.8: Document objects and commands
rdma: Add link capability parsing

 Makefile          |   2 +-
 man/man8/Makefile |   3 +-
 man/man8/rdma.8   | 109 +++++++++++++++++++
 rdma/.gitignore   |   1 +
 rdma/Makefile     |  15 +++
 rdma/dev.c        | 101 ++++++++++++++++++
 rdma/ipoib.c      |  54 ++++++++++
 rdma/link.c       | 160 ++++++++++++++++++++++++++++
 rdma/memory.c     |  30 ++++++
 rdma/monitor.c    |  22 ++++
 rdma/protocols.c  |  22 ++++
 rdma/providers.c  |  28 +++++
 rdma/rdma.c       | 104 ++++++++++++++++++
 rdma/rdma.h       |  93 ++++++++++++++++
 rdma/stats.c      |  22 ++++
 rdma/utils.c      | 313 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 16 files changed, 1077 insertions(+), 2 deletions(-)
 create mode 100644 man/man8/rdma.8
 create mode 100644 rdma/.gitignore
 create mode 100644 rdma/Makefile
 create mode 100644 rdma/dev.c
 create mode 100644 rdma/ipoib.c
 create mode 100644 rdma/link.c
 create mode 100644 rdma/memory.c
 create mode 100644 rdma/monitor.c
 create mode 100644 rdma/protocols.c
 create mode 100644 rdma/providers.c
 create mode 100644 rdma/rdma.c
 create mode 100644 rdma/rdma.h
 create mode 100644 rdma/stats.c
 create mode 100644 rdma/utils.c

--
2.12.2

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): 20170504180216.7665-1-leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org

#2 lore raw

RFC 1/8 rdma: Add basic infrastructure for RDMA tool

From: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/[email protected]>

RDMA devices are cross-functional devices from one side, but very tailored for the specific markets from another.

Such diversity caused to spread of RDMA related configuration across various tools, e.g. devlink, ip, ethtool, ib specific and vendor specific solutions.

This patch adds ability to fill device and port information by reading sysfs entries (legacy).

All future configuration settings will be implemented in netlink format, to be aligned with iproute2 package.

---
 Makefile        |   2 +-
 rdma/.gitignore |   1 +
 rdma/Makefile   |  15 +++
 rdma/rdma.c     |  96 +++++++++++++++++
 rdma/rdma.h     |  77 ++++++++++++++
 rdma/utils.c    | 313 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 6 files changed, 503 insertions(+), 1 deletion(-)
 create mode 100644 rdma/.gitignore
 create mode 100644 rdma/Makefile
 create mode 100644 rdma/rdma.c
 create mode 100644 rdma/rdma.h
 create mode 100644 rdma/utils.c
Makefile +1 -1
index 18de7dcb..c255063b 100644
@@ -52,7 +52,7 @@ WFLAGS += -Wmissing-declarations -Wold-style-definition -Wformat=2
 CFLAGS := $(WFLAGS) $(CCOPTS) -I../include $(DEFINES) $(CFLAGS)
 YACCFLAGS = -d -t -v
 
-SUBDIRS=lib ip tc bridge misc netem genl tipc devlink man
+SUBDIRS=lib ip tc bridge misc netem genl tipc devlink rdma man
 
 LIBNETLINK=../lib/libnetlink.a ../lib/libutil.a
 LDLIBS += $(LIBNETLINK)
rdma/.gitignore new file +1 -0
new file mode 100644
index 00000000..51fb172b
@@ -0,0 +1 @@
+rdma
rdma/Makefile new file +15 -0
new file mode 100644
index 00000000..65248b31
@@ -0,0 +1,15 @@
+include ../Config
+
+RDMA_OBJ = rdma.o utils.o
+TARGETS=rdma
+
+all:	$(TARGETS) $(LIBS)
+
+rdma:	$(RDMA_OBJ)
+	$(QUIET_LINK)$(CC) $^ -o $@
+
+install: all
+	install -m 0755 $(TARGETS) $(DESTDIR)$(SBINDIR)
+
+clean:
+	rm -f $(RDMA_OBJ) $(TARGETS)
rdma/rdma.c new file +96 -0

96 lines not on this page · raw message

rdma/rdma.h new file +77 -0

77 lines not on this page · raw message

rdma/utils.c new file +313 -0

313 lines not on this page · raw message

--
2.12.2

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#3 lore raw

RFC 4/8 rdma: Add IPoIB object

In reply to a message not in the archive

From: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/[email protected]>

IPoIB object allows configuration and presentation of information for IP-over-Infiniband user level protocol.

Supported commands are show, set and help.

---
 rdma/Makefile |  2 +-
 rdma/ipoib.c  | 54 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 rdma/rdma.c   |  3 ++-
 rdma/rdma.h   |  1 +
 4 files changed, 58 insertions(+), 2 deletions(-)
 create mode 100644 rdma/ipoib.c
rdma/Makefile +1 -1
index cf54ed36..dd702b9f 100644
@@ -1,6 +1,6 @@
 include ../Config
 
-RDMA_OBJ = rdma.o utils.o dev.o link.o
+RDMA_OBJ = rdma.o utils.o dev.o link.o ipoib.o
 TARGETS=rdma
 
 all:	$(TARGETS) $(LIBS)
rdma/ipoib.c new file +54 -0

54 lines not on this page · raw message

rdma/rdma.c +2 -1
index 55cbf0e3..ffd70899 100644
@@ -17,7 +17,7 @@
 static void help(char *name)
 {
 	pr_out("Usage: %s [ OPTIONS ] OBJECT { COMMAND | help }\n"
-	       "where  OBJECT := { dev | link }\n"
+	       "where  OBJECT := { dev | link | ipoib }\n"
 	       "       OPTIONS := { -V[ersion] }\n", name);
 }
 
@@ -33,6 +33,7 @@ static int rd_cmd(struct rdma *rd)
 		{ NULL,		obj_help },
 		{ "dev",	obj_dev },
 		{ "link",	obj_link },
+		{ "ipoib",	obj_ipoib },
 		{ "help",	obj_help },
 		{ 0 }
 	};
rdma/rdma.h +1 -0
index bdb77b5e..1fef4eb8 100644
@@ -64,6 +64,7 @@ struct rdma_obj {
  */
 int obj_dev(struct rdma *rd);
 int obj_link(struct rdma *rd);
+int obj_ipoib(struct rdma *rd);
 
 /*
  * Parser interface

--
2.12.2

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#4 lore raw

RFC 7/8 man: rdma.8: Document objects and commands

In reply to a message not in the archive

From: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/[email protected]>

---
 man/man8/Makefile |   3 +-
 man/man8/rdma.8   | 109 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 111 insertions(+), 1 deletion(-)
 create mode 100644 man/man8/rdma.8
man/man8/Makefile +2 -1
index f3318644..81979a07 100644
@@ -19,7 +19,8 @@ MAN8PAGES = $(TARGETS) ip.8 arpd.8 lnstat.8 routel.8 rtacct.8 rtmon.8 rtpr.8 ss.
 	tc-simple.8 tc-skbedit.8 tc-vlan.8 tc-xt.8 tc-skbmod.8 tc-ife.8 \
 	tc-tunnel_key.8 tc-sample.8 \
 	devlink.8 devlink-dev.8 devlink-monitor.8 devlink-port.8 devlink-sb.8 \
-	ifstat.8
+	ifstat.8 \
+	rdma.8
 
 all: $(TARGETS)
 
man/man8/rdma.8 new file +109 -0

109 lines not on this page · raw message

--
2.12.2

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#5 lore raw

RFC 8/8 rdma: Add link capability parsing

In reply to a message not in the archive

From: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/[email protected]>

Add parsing interface for the cap_mask

$./rdma/rdma link show mlx5_2/2 cap_mask
3/2: mlx5_2/2: sm off notice off trap on opt_ipd off auto_migr off sl_map on mkey_nvram off
pkey_nvram off led_info off sm_disabled off sys_image_guid on pkey_sw_ext_port_trap off extended_speeds on cm on snmp_tunnel off reinit off device_mgmt off vendor_class on dr_notice off cap_mask_notice on boot_mgmt off link_latency off client_reg on ip_based_gids on

---
 rdma/link.c  | 74 +++++++++++++++++++++++++++++++++++++++++++++++++-----------
 rdma/rdma.h  |  4 ++++
 rdma/utils.c |  4 ++--
 3 files changed, 67 insertions(+), 15 deletions(-)
rdma/link.c +61 -13
index e86ff399..e9880914 100644
@@ -14,13 +14,48 @@
 static int link_help(struct rdma *rd)
 {
 	pr_out("Usage: %s link show [ DEV | DEV/PORT ]\n", rd->filename);
+	pr_out("       %s link show [ DEV | DEV/PORT ] cap_mask\n", rd->filename);
 	pr_out("       %s link set DEV/PORT { type { eth | ib | auto } |\n", rd->filename);
 	pr_out("                                lb_unicast { on | off } |\n");
 	pr_out("                                lb_multicast { on | off } }\n");
 	return 0;
 }
 
-static void dev_one_show(const struct dev_map *dev_map, uint32_t port_idx_first, uint32_t port_idx_last)
+static void print_cap_mask(uint32_t cap_mask)
+{
+#define PRINT_PORT_CAP(name, val, offset)		(printf(" %s %s", name, (((val) >> (offset))&0x1)?"on":"off"))
+
+	/* Naive copy/paste from include/rdma/ib_verbs.h */
+ 	PRINT_PORT_CAP("sm", cap_mask, 1);
+ 	PRINT_PORT_CAP("notice", cap_mask, 2);
+ 	PRINT_PORT_CAP("trap", cap_mask, 3);
+ 	PRINT_PORT_CAP("opt_ipd", cap_mask, 4);
+ 	PRINT_PORT_CAP("auto_migr", cap_mask, 5);
+	PRINT_PORT_CAP("sl_map", cap_mask, 6);
+ 	PRINT_PORT_CAP("mkey_nvram", cap_mask, 7);
+	printf("\n\t");
+ 	PRINT_PORT_CAP("pkey_nvram", cap_mask, 8);
+ 	PRINT_PORT_CAP("led_info", cap_mask, 9);
+ 	PRINT_PORT_CAP("sm_disabled", cap_mask, 10);
+ 	PRINT_PORT_CAP("sys_image_guid", cap_mask, 11);
+	PRINT_PORT_CAP("pkey_sw_ext_port_trap", cap_mask, 12);
+	printf("\n\t");
+	PRINT_PORT_CAP("extended_speeds", cap_mask, 14);
+ 	PRINT_PORT_CAP("cm", cap_mask, 16);
+ 	PRINT_PORT_CAP("snmp_tunnel", cap_mask, 17);
+ 	PRINT_PORT_CAP("reinit", cap_mask, 18);
+ 	PRINT_PORT_CAP("device_mgmt", cap_mask, 19);
+ 	PRINT_PORT_CAP("vendor_class", cap_mask, 20);
+ 	PRINT_PORT_CAP("dr_notice", cap_mask, 21);
+	printf("\n\t");
+	PRINT_PORT_CAP("cap_mask_notice", cap_mask, 22);
+ 	PRINT_PORT_CAP("boot_mgmt", cap_mask, 23);
+ 	PRINT_PORT_CAP("link_latency", cap_mask, 24);
+ 	PRINT_PORT_CAP("client_reg", cap_mask, 25);
+ 	PRINT_PORT_CAP("ip_based_gids", cap_mask, 26);
+}
+static void dev_one_show(struct rdma *rd, const struct dev_map *dev_map,
+			 uint32_t port_idx_first, uint32_t port_idx_last)
 {
 	char *nodes[] = { "cap_mask",
 			  "lid",
@@ -35,23 +70,36 @@ static void dev_one_show(const struct dev_map *dev_map, uint32_t port_idx_first,
 
 	struct port_map *port_map;
 	char data[4096];
+	uint32_t cap_mask;
+	bool cap_mask_r = false;
 	int i, j;
 
+	rd_arg_inc(rd);
+	if (rd_argv_match(rd, "cap_mask"))
+		cap_mask_r = true;
+
 	for(j = port_idx_first ; j <= port_idx_last; j++) {
 		pr_out("%u/%u: %s/%u:", dev_map->idx, j, dev_map->dev_name, j);
-		list_for_each_entry(port_map, &dev_map->port_map_list, list)
-			if (j == port_map->idx)
-			       printf(" ifname %s", (port_map->ifname)?:"NONE");
+		if (cap_mask_r) {
+			rdma_sysfs_read_ib(dev_map->dev_name, 1, nodes[0], data);
+			cap_mask = strtoul(data, NULL, 16);
+			print_cap_mask(cap_mask);
+		}
+		else {
+			list_for_each_entry(port_map, &dev_map->port_map_list, list)
+				if (j == port_map->idx)
+				       printf(" ifname %s", (port_map->ifname)?:"NONE");
 
-		for (i = 0 ; nodes[i] ; i++) {
-			if (rdma_sysfs_read_ib(dev_map->dev_name, j, nodes[i], data))
-				continue;
+			for (i = 0 ; nodes[i] ; i++) {
+				if (rdma_sysfs_read_ib(dev_map->dev_name, j, nodes[i], data))
+					continue;
 
-			/* Split line before "phys_state" */
-			if (!strcmp(nodes[i], "phys_state"))
-				printf("\n\t");
+				/* Split line before "phys_state" */
+				if (!strcmp(nodes[i], "phys_state"))
+					printf("\n\t");
 
-			pr_out(" %s %s", nodes[i], data);
+				pr_out(" %s %s", nodes[i], data);
+			}
 		}
 		pr_out("\n");
 	}
@@ -63,7 +111,7 @@ static int link_show(struct rdma *rd)
 
 	if (rd_no_arg(rd)) {
 		list_for_each_entry(dev_map, &rd->dev_map_list, list)
-			dev_one_show(dev_map, 1, dev_map->num_ports);
+			dev_one_show(rd, dev_map, 1, dev_map->num_ports);
 	}
 	else {
 		uint32_t port_idx;
@@ -81,7 +129,7 @@ static int link_show(struct rdma *rd)
 			num_ports = dev_map->num_ports;
 		}
 
-		dev_one_show(dev_map, port_idx, num_ports);
+		dev_one_show(rd, dev_map, port_idx, num_ports);
 	}
 	return 0;
 }
rdma/rdma.h +4 -0
index 11d940d7..12c87048 100644
@@ -76,6 +76,10 @@ int obj_monitor(struct rdma *rd);
  */
 bool rd_no_arg(struct rdma *rd);
 uint32_t get_port_from_argv(struct rdma *rd);
+bool rd_no_arg(struct rdma *rd);
+bool rd_argv_match(struct rdma *rd, const char *pattern);
+void rd_arg_inc(struct rdma *rd);
+uint32_t get_port_from_argv(struct rdma *rd);
 
 int rdma_exec_cmd(struct rdma *rd, const struct rdma_obj *o, const char *str);
 int rdma_sysfs_read_ib(const char *name, int port, const char *field, char *res);
rdma/utils.c +2 -2
index 568d7c0a..fd5fe77f 100644
@@ -40,14 +40,14 @@ static int strcmpx(const char *str1, const char *str2)
 	return strncmp(str1, str2, strlen(str1));
 }
 
-static bool rd_argv_match(struct rdma *rd, const char *pattern)
+bool rd_argv_match(struct rdma *rd, const char *pattern)
 {
 	if (!rd_argc(rd))
 		return false;
 	return strcmpx(rd_argv(rd), pattern) == 0;
 }
 
-static void rd_arg_inc(struct rdma *rd)
+void rd_arg_inc(struct rdma *rd)
 {
 	if (!rd_argc(rd))
 		return;

--
2.12.2

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#6 lore raw
In reply to a message not in the archive
Quoted 4 lines · Leon Romanovsky

In order to close object model, ensure reuse of existing code and make this tool usable from day one, we decided to implement wrappers over legacy sysfs prior to implementing netlink functionality. As a nice bonus, it will allow to use this tool with old kernels too.

This sounds wrong. We don't support legacy ioctl interface for the 'ip' command, either. I think rdma should be converted to netlink first and the new tool should only use netlink.

Jiri
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#7 lore raw
Quoted 9 lines · Jiri Benc
Quoted 4 lines · Leon Romanovsky

In order to close object model, ensure reuse of existing code and make this tool usable from day one, we decided to implement wrappers over legacy sysfs prior to implementing netlink functionality. As a nice bonus, it will allow to use this tool with old kernels too.

This sounds wrong. We don't support legacy ioctl interface for the 'ip' command, either. I think rdma should be converted to netlink first and the new tool should only use netlink.

RDMA in slightly different situation than "ip" tool was. "ip" was implemented when tools like ifconfig existed. It allowed to old and new systems to be configured to some degree. In RDMA community, there are no similar tools like "ifconfig". Implementation in netlink-only interface will leave old systems without common tool at all.

As an upstream-oriented person, I personally fine with that, but anyway would like to get wider agreement/disagreement on that, before removing sysfs parsing logic from the rdmatool.

Thanks

Quoted 2 lines

Jiri

A message that is not in the archive (sent off-list or not yet received): 20170505131754.GH22833-U/[email protected]

#9 lore raw
Quoted 17 lines · Fri, May 05, 2017 at 03:17:54PM CEST, [email protected].…
Quoted 9 lines · Jiri Benc
Quoted 4 lines · Leon Romanovsky

In order to close object model, ensure reuse of existing code and make this tool usable from day one, we decided to implement wrappers over legacy sysfs prior to implementing netlink functionality. As a nice bonus, it will allow to use this tool with old kernels too.

This sounds wrong. We don't support legacy ioctl interface for the 'ip' command, either. I think rdma should be converted to netlink first and the new tool should only use netlink.

RDMA in slightly different situation than "ip" tool was. "ip" was implemented when tools like ifconfig existed. It allowed to old and new systems to be configured to some degree. In RDMA community, there are no similar tools like "ifconfig". Implementation in netlink-only interface will leave old systems without common tool at all.

Quoted 3 lines

As an upstream-oriented person, I personally fine with that, but anyway would like to get wider agreement/disagreement on that, before removing sysfs parsing logic from the rdmatool.

I tend to agree with Jiri Benc. I fear that supporting sysfs + netlink api later on for the same things will make the code unnecessary complex. Also, the legacy sysfs will most likely stay there forever so there will be no actual motivation to port the existing things to the new netlink api.

For the prototyping purposes, I belive that what you did makes perfect sense. But for the actual mergable version, my feeling is that we need to strictly stick with new netlink rdma interface and just forget about the old sysfs one. Distros would have to backport the new kernel rdma netlink api.

Yes, this will be little bit more painful at the beginning, but in the long run, I believe it will save some severe headaches.

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#10 lore raw
Quoted 28 lines · Jiri Pirko
Quoted 17 lines · Fri, May 05, 2017 at 03:17:54PM CEST, [email protected].…
Quoted 9 lines · Jiri Benc
Quoted 4 lines · Leon Romanovsky

In order to close object model, ensure reuse of existing code and make this tool usable from day one, we decided to implement wrappers over legacy sysfs prior to implementing netlink functionality. As a nice bonus, it will allow to use this tool with old kernels too.

This sounds wrong. We don't support legacy ioctl interface for the 'ip' command, either. I think rdma should be converted to netlink first and the new tool should only use netlink.

RDMA in slightly different situation than "ip" tool was. "ip" was implemented when tools like ifconfig existed. It allowed to old and new systems to be configured to some degree. In RDMA community, there are no similar tools like "ifconfig". Implementation in netlink-only interface will leave old systems without common tool at all.

Quoted 3 lines

As an upstream-oriented person, I personally fine with that, but anyway would like to get wider agreement/disagreement on that, before removing sysfs parsing logic from the rdmatool.

I tend to agree with Jiri Benc. I fear that supporting sysfs + netlink api later on for the same things will make the code unnecessary complex. Also, the legacy sysfs will most likely stay there forever so there will be no actual motivation to port the existing things to the new netlink api.

Quoted 5 lines

For the prototyping purposes, I belive that what you did makes perfect sense. But for the actual mergable version, my feeling is that we need to strictly stick with new netlink rdma interface and just forget about the old sysfs one. Distros would have to backport the new kernel rdma netlink api.

Thanks,
It looks like that most of the comments are in favor of netlink-only solution.

Quoted 4 lines

Yes, this will be little bit more painful at the beginning, but in the long run, I believe it will save some severe headaches.

#11 lore raw
Quoted 35 lines · On Sun, 7 May 2017 09:33:29 +0300 Leon Romanovsky <[email protected]> wrote:
Quoted 28 lines · Jiri Pirko
Quoted 17 lines · Fri, May 05, 2017 at 03:17:54PM CEST, [email protected] wrote:
Quoted 9 lines · Jiri Benc
Quoted 4 lines · Leon Romanovsky

In order to close object model, ensure reuse of existing code and make this tool usable from day one, we decided to implement wrappers over legacy sysfs prior to implementing netlink functionality. As a nice bonus, it will allow to use this tool with old kernels too.

This sounds wrong. We don't support legacy ioctl interface for the 'ip' command, either. I think rdma should be converted to netlink first and the new tool should only use netlink.

RDMA in slightly different situation than "ip" tool was. "ip" was implemented when tools like ifconfig existed. It allowed to old and new systems to be configured to some degree. In RDMA community, there are no similar tools like "ifconfig". Implementation in netlink-only interface will leave old systems without common tool at all.

Quoted 3 lines

As an upstream-oriented person, I personally fine with that, but anyway would like to get wider agreement/disagreement on that, before removing sysfs parsing logic from the rdmatool.

I tend to agree with Jiri Benc. I fear that supporting sysfs + netlink api later on for the same things will make the code unnecessary complex. Also, the legacy sysfs will most likely stay there forever so there will be no actual motivation to port the existing things to the new netlink api.

Quoted 5 lines

For the prototyping purposes, I belive that what you did makes perfect sense. But for the actual mergable version, my feeling is that we need to strictly stick with new netlink rdma interface and just forget about the old sysfs one. Distros would have to backport the new kernel rdma netlink api.

Quoted 3 lines

Thanks,
It looks like that most of the comments are in favor of netlink-only solution.

If current (like 4.10 or later) kernel support netlink only solution, that makes sense. When I created bridge command; it also abandoned the old ioctl interface.

#12 lore raw
In reply to Leon Romanovsky
Quoted 35 lines · Leon Romanovsky
Quoted 28 lines · Jiri Pirko
Quoted 17 lines · Fri, May 05, 2017 at 03:17:54PM CEST, [email protected] wrote:
Quoted 9 lines · Jiri Benc
Quoted 4 lines · Leon Romanovsky

In order to close object model, ensure reuse of existing code and make this tool usable from day one, we decided to implement wrappers over legacy sysfs prior to implementing netlink functionality. As a nice bonus, it will allow to use this tool with old kernels too.

This sounds wrong. We don't support legacy ioctl interface for the 'ip' command, either. I think rdma should be converted to netlink first and the new tool should only use netlink.

RDMA in slightly different situation than "ip" tool was. "ip" was implemented when tools like ifconfig existed. It allowed to old and new systems to be configured to some degree. In RDMA community, there are no similar tools like "ifconfig". Implementation in netlink-only interface will leave old systems without common tool at all.

Quoted 3 lines

As an upstream-oriented person, I personally fine with that, but anyway would like to get wider agreement/disagreement on that, before removing sysfs parsing logic from the rdmatool.

I tend to agree with Jiri Benc. I fear that supporting sysfs + netlink api later on for the same things will make the code unnecessary complex. Also, the legacy sysfs will most likely stay there forever so there will be no actual motivation to port the existing things to the new netlink api.

Quoted 5 lines

For the prototyping purposes, I belive that what you did makes perfect sense. But for the actual mergable version, my feeling is that we need to strictly stick with new netlink rdma interface and just forget about the old sysfs one. Distros would have to backport the new kernel rdma netlink api.

Quoted 3 lines

Thanks,
It looks like that most of the comments are in favor of netlink-only solution.

Leon, I like the thought bw comp support. After all this is a user level tool so it should be possible to make a clean implementation that makes the old stuff easy to remove at some point. It will also attract users much sooner than if they have to have their own if-then-else logic around everything to be able to support old and new.

Quoted 3 lines
Quoted 3 lines

Yes, this will be little bit more painful at the beginning, but in the long run, I believe it will save some severe headaches.

IMHO, some headache will be there anyway, just a matter of how how far out it gets.

Knut

#13 lore raw

RFC 2/8 rdma: Add dev object

In reply to Leon Romanovsky

From: Leon Romanovsky <[email protected]>

Device (dev) object represents struct ib_device to user space.

The supported commands are show, set and help.

Print all devices:

 # rdma dev
1: mlx5_0: board_id MT_2190110032 fw_pages 261002 fw_ver 12.17.2046 hca_type MT4115 hw_rev 0
	node_desc hpchead HCA-1 node_guid e41d:2d03:0066:dee6 node_type 1: CA reg_pages 0
	sys_image_guid e41d:2d03:0066:dee6
2: mlx5_1: board_id MT_2190110032 fw_pages 250793 fw_ver 12.17.2046 hca_type MT4115 hw_rev 0
	node_desc hpchead HCA-2 node_guid e41d:2d03:0066:dee7 node_type 1: CA reg_pages 0
	sys_image_guid e41d:2d03:0066:dee6
3: mlx5_2: board_id MT_1210110019 fw_pages 68067 fw_ver 10.16.1020 hca_type MT4113 hw_rev 0
	node_desc hpchead HCA-3 node_guid 0002:c903:0016:75b0 node_type 1: CA reg_pages 0
	sys_image_guid 0002:c903:0016:75b0

Print specific device:

 # rdma dev show mlx5_1
2: mlx5_1: board_id MT_2190110032 fw_pages 250793 fw_ver 12.17.2046 hca_type MT4115 hw_rev 0
	node_desc hpchead HCA-2 node_guid e41d:2d03:0066:dee7 node_type 1: CA reg_pages 0
	sys_image_guid e41d:2d03:0066:dee6
---
 rdma/Makefile |   2 +-
 rdma/dev.c    | 101 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 rdma/rdma.c   |   3 +-
 rdma/rdma.h   |   5 +++
 4 files changed, 109 insertions(+), 2 deletions(-)
 create mode 100644 rdma/dev.c
rdma/Makefile +1 -1
index 65248b31..67e349b0 100644
@@ -1,6 +1,6 @@
 include ../Config
 
-RDMA_OBJ = rdma.o utils.o
+RDMA_OBJ = rdma.o utils.o dev.o
 TARGETS=rdma
 
 all:	$(TARGETS) $(LIBS)
rdma/dev.c new file +101 -0

101 lines not on this page · raw message

rdma/rdma.c +2 -1
index bc7d1483..7c537c5e 100644
@@ -17,7 +17,7 @@
 static void help(char *name)
 {
 	pr_out("Usage: %s [ OPTIONS ] OBJECT { COMMAND | help }\n"
-	       "where  OBJECT := { }\n"
+	       "where  OBJECT := { dev }\n"
 	       "       OPTIONS := { -V[ersion] }\n", name);
 }
 
@@ -31,6 +31,7 @@ static int rd_cmd(struct rdma *rd)
 {
 	const struct rdma_obj objs[] = {
 		{ NULL,		obj_help },
+		{ "dev",	obj_dev },
 		{ "help",	obj_help },
 		{ 0 }
 	};
rdma/rdma.h +5 -0
index 156bb74c..2d81cd92 100644
@@ -60,6 +60,11 @@ struct rdma_obj {
 };
 
 /*
+ * Command interfaces
+ */
+int obj_dev(struct rdma *rd);
+
+/*
  * Parser interface
  */
 bool rd_no_arg(struct rdma *rd);
#14 lore raw

RFC 3/8 rdma: Add link object

In reply to Leon Romanovsky

From: Leon Romanovsky <[email protected]>

Link object represents port of struct ib_device.

Supported commands are show, set and help.

Print all links for all devices:

 # rdma link
1/1: mlx5_0/1: ifname ib0 cap_mask 0x2651e848 lid 0x13 lid_mask_count 0 link_layer InfiniBand
	phys_state 5: LinkUp rate 100 Gb/sec (4X EDR) sm_lid 0x2 sm_sl 0 state 4: ACTIVE
2/1: mlx5_1/1: ifname ib1 cap_mask 0x2651e848 lid 0xffff lid_mask_count 0 link_layer InfiniBand
	phys_state 3: Disabled rate 10 Gb/sec (4X) sm_lid 0x0 sm_sl 0 state 1: DOWN
3/1: mlx5_2/1: ifname ib2 cap_mask 0x26516848 lid 0x1a lid_mask_count 0 link_layer InfiniBand
	phys_state 5: LinkUp rate 56 Gb/sec (4X FDR) sm_lid 0x2 sm_sl 0 state 4: ACTIVE
3/2: mlx5_2/2: ifname ib3 cap_mask 0x26516848 lid 0xffff lid_mask_count 0 link_layer InfiniBand
	phys_state 3: Disabled rate 10 Gb/sec (4X) sm_lid 0x0 sm_sl 0 state 1: DOWN

Print all links for specific device:

 # rdma link show mlx5_2
3/1: mlx5_2/1: ifname ib2 cap_mask 0x26516848 lid 0x1a lid_mask_count 0 link_layer InfiniBand
	phys_state 5: LinkUp rate 56 Gb/sec (4X FDR) sm_lid 0x2 sm_sl 0 state 4: ACTIVE
3/2: mlx5_2/2: ifname ib3 cap_mask 0x26516848 lid 0xffff lid_mask_count 0 link_layer InfiniBand
	phys_state 3: Disabled rate 10 Gb/sec (4X) sm_lid 0x0 sm_sl 0 state 1: DOWN

Print specific link:

 # rdma link show mlx5_2/2
3/2: mlx5_2/2: ifname ib3 cap_mask 0x26516848 lid 0xffff lid_mask_count 0 link_layer InfiniBand
	phys_state 3: Disabled rate 10 Gb/sec (4X) sm_lid 0x0 sm_sl 0 state 1: DOWN

Set parameter;

 # rdma link set mlx5_2/2 type auto lb_unicast off
---
 rdma/Makefile |   2 +-
 rdma/link.c   | 112 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 rdma/rdma.c   |   3 +-
 rdma/rdma.h   |   1 +
 4 files changed, 116 insertions(+), 2 deletions(-)
 create mode 100644 rdma/link.c
rdma/Makefile +1 -1
index 67e349b0..cf54ed36 100644
@@ -1,6 +1,6 @@
 include ../Config
 
-RDMA_OBJ = rdma.o utils.o dev.o
+RDMA_OBJ = rdma.o utils.o dev.o link.o
 TARGETS=rdma
 
 all:	$(TARGETS) $(LIBS)
rdma/link.c new file +112 -0

112 lines not on this page · raw message

rdma/rdma.c +2 -1
index 7c537c5e..55cbf0e3 100644
@@ -17,7 +17,7 @@
 static void help(char *name)
 {
 	pr_out("Usage: %s [ OPTIONS ] OBJECT { COMMAND | help }\n"
-	       "where  OBJECT := { dev }\n"
+	       "where  OBJECT := { dev | link }\n"
 	       "       OPTIONS := { -V[ersion] }\n", name);
 }
 
@@ -32,6 +32,7 @@ static int rd_cmd(struct rdma *rd)
 	const struct rdma_obj objs[] = {
 		{ NULL,		obj_help },
 		{ "dev",	obj_dev },
+		{ "link",	obj_link },
 		{ "help",	obj_help },
 		{ 0 }
 	};
rdma/rdma.h +1 -0
index 2d81cd92..bdb77b5e 100644
@@ -63,6 +63,7 @@ struct rdma_obj {
  * Command interfaces
  */
 int obj_dev(struct rdma *rd);
+int obj_link(struct rdma *rd);
 
 /*
  * Parser interface
#15 lore raw

RFC 5/8 rdma: Add memory object

In reply to Leon Romanovsky

From: Leon Romanovsky <[email protected]>

Memory object gives to the user ability to manipulate over general properties of memory for the specific devices. The memory properties have broader usage than dev object can provide.

For example, on-demand-paging (ODP) configurations are mostly software related.

---
 rdma/Makefile |  2 +-
 rdma/memory.c | 30 ++++++++++++++++++++++++++++++
 rdma/rdma.c   |  3 ++-
 rdma/rdma.h   |  1 +
 4 files changed, 34 insertions(+), 2 deletions(-)
 create mode 100644 rdma/memory.c
rdma/Makefile +1 -1
index dd702b9f..5cf0d29f 100644
@@ -1,6 +1,6 @@
 include ../Config
 
-RDMA_OBJ = rdma.o utils.o dev.o link.o ipoib.o
+RDMA_OBJ = rdma.o utils.o dev.o link.o ipoib.o memory.o
 TARGETS=rdma
 
 all:	$(TARGETS) $(LIBS)
rdma/memory.c new file +30 -0
new file mode 100644
index 00000000..68fd5dd3
@@ -0,0 +1,30 @@
+/*
+ * memory.c	RDMA tool
+ *
+ *              This program is free software; you can redistribute it and/or
+ *              modify it under the terms of the GNU General Public License
+ *              as published by the Free Software Foundation; either version
+ *              2 of the License, or (at your option) any later version.
+ *
+ * Authors:     Leon Romanovsky <[email protected]>
+ */
+
+#include "rdma.h"
+
+static void memory_help(char *filename)
+{
+	pr_out("Usage: %s memory show [ DEV ]\n", filename);
+	pr_out("       %s memory set DEV { odp { off | on } |\n", filename);
+	pr_out("       %s                  memic SIZE }\n", filename);
+}
+
+int obj_memory(struct rdma *rd)
+{
+	if (dev_map_init(rd)) {
+		pr_err("There are no RDMA devices\n");
+		return -ENOENT;
+	}
+
+	memory_help(rd->filename);
+	return 0;
+}
rdma/rdma.c +2 -1
index ffd70899..094d490d 100644
@@ -17,7 +17,7 @@
 static void help(char *name)
 {
 	pr_out("Usage: %s [ OPTIONS ] OBJECT { COMMAND | help }\n"
-	       "where  OBJECT := { dev | link | ipoib }\n"
+	       "where  OBJECT := { dev | link | ipoib | memory }\n"
 	       "       OPTIONS := { -V[ersion] }\n", name);
 }
 
@@ -34,6 +34,7 @@ static int rd_cmd(struct rdma *rd)
 		{ "dev",	obj_dev },
 		{ "link",	obj_link },
 		{ "ipoib",	obj_ipoib },
+		{ "memory",	obj_memory },
 		{ "help",	obj_help },
 		{ 0 }
 	};
rdma/rdma.h +1 -0
index 1fef4eb8..dcff066f 100644
@@ -65,6 +65,7 @@ struct rdma_obj {
 int obj_dev(struct rdma *rd);
 int obj_link(struct rdma *rd);
 int obj_ipoib(struct rdma *rd);
+int obj_memory(struct rdma *rd);
 
 /*
  * Parser interface
#16 lore raw

RFC 6/8 rdma: add stubs for future objects

In reply to Leon Romanovsky

From: Leon Romanovsky <[email protected]>

The following objects (monitor, providers, stats and protocols) are not implemented yet, however it is worth to place their stubs in the code.

This will serve as an initial starting point for other developers to extend RDMA tool.

---
 rdma/Makefile    |  2 +-
 rdma/monitor.c   | 22 ++++++++++++++++++++++
 rdma/protocols.c | 22 ++++++++++++++++++++++
 rdma/providers.c | 28 ++++++++++++++++++++++++++++
 rdma/rdma.c      |  6 +++++-
 rdma/rdma.h      |  4 ++++
 rdma/stats.c     | 22 ++++++++++++++++++++++
 7 files changed, 104 insertions(+), 2 deletions(-)
 create mode 100644 rdma/monitor.c
 create mode 100644 rdma/protocols.c
 create mode 100644 rdma/providers.c
 create mode 100644 rdma/stats.c
rdma/Makefile +1 -1
index 5cf0d29f..eb71da68 100644
@@ -1,6 +1,6 @@
 include ../Config
 
-RDMA_OBJ = rdma.o utils.o dev.o link.o ipoib.o memory.o
+RDMA_OBJ = rdma.o utils.o dev.o link.o ipoib.o memory.o stats.o protocols.o providers.o monitor.o
 TARGETS=rdma
 
 all:	$(TARGETS) $(LIBS)
rdma/monitor.c new file +22 -0
new file mode 100644
index 00000000..99d4b042
@@ -0,0 +1,22 @@
+/*
+ * monitor.c	RDMA tool
+ *
+ *              This program is free software; you can redistribute it and/or
+ *              modify it under the terms of the GNU General Public License
+ *              as published by the Free Software Foundation; either version
+ *              2 of the License, or (at your option) any later version.
+ *
+ * Authors:     Leon Romanovsky <[email protected]>
+ */
+
+#include "rdma.h"
+
+int obj_monitor(struct rdma *rd)
+{
+	if (dev_map_init(rd)) {
+		pr_err("There are no RDMA devices\n");
+		return -ENOENT;
+	}
+
+	return 0;
+}
rdma/protocols.c new file +22 -0
new file mode 100644
index 00000000..26de7d2b
@@ -0,0 +1,22 @@
+/*
+ * protocols.c	RDMA tool
+ *
+ *              This program is free software; you can redistribute it and/or
+ *              modify it under the terms of the GNU General Public License
+ *              as published by the Free Software Foundation; either version
+ *              2 of the License, or (at your option) any later version.
+ *
+ * Authors:     Leon Romanovsky <[email protected]>
+ */
+
+#include "rdma.h"
+
+int obj_protocols(struct rdma *rd)
+{
+	if (dev_map_init(rd)) {
+		pr_err("There are no RDMA devices\n");
+		return -ENOENT;
+	}
+
+	return 0;
+}
rdma/providers.c new file +28 -0
new file mode 100644
index 00000000..8d516cca
@@ -0,0 +1,28 @@
+/*
+ * providers.c	RDMA tool
+ *
+ *              This program is free software; you can redistribute it and/or
+ *              modify it under the terms of the GNU General Public License
+ *              as published by the Free Software Foundation; either version
+ *              2 of the License, or (at your option) any later version.
+ *
+ * Authors:     Leon Romanovsky <[email protected]>
+ */
+
+#include "rdma.h"
+
+static void providers_help(char *filename)
+{
+	pr_out("Usage: %s providers show [ DEV ]\n", filename);
+}
+
+int obj_providers(struct rdma *rd)
+{
+	if (dev_map_init(rd)) {
+		pr_err("There are no RDMA devices\n");
+		return -ENOENT;
+	}
+
+	providers_help(rd->filename);
+	return 0;
+}
rdma/rdma.c +5 -1
index 094d490d..a0a3ec81 100644
@@ -17,7 +17,7 @@
 static void help(char *name)
 {
 	pr_out("Usage: %s [ OPTIONS ] OBJECT { COMMAND | help }\n"
-	       "where  OBJECT := { dev | link | ipoib | memory }\n"
+	       "where  OBJECT := { dev | link | ipoib | memory | stats | protocols | providers | monitor }\n"
 	       "       OPTIONS := { -V[ersion] }\n", name);
 }
 
@@ -35,6 +35,10 @@ static int rd_cmd(struct rdma *rd)
 		{ "link",	obj_link },
 		{ "ipoib",	obj_ipoib },
 		{ "memory",	obj_memory },
+		{ "stats",	obj_stats },
+		{ "providers",	obj_providers },
+		{ "protocols",	obj_protocols },
+		{ "monitor",	obj_monitor },
 		{ "help",	obj_help },
 		{ 0 }
 	};
rdma/rdma.h +4 -0
index dcff066f..11d940d7 100644
@@ -66,6 +66,10 @@ int obj_dev(struct rdma *rd);
 int obj_link(struct rdma *rd);
 int obj_ipoib(struct rdma *rd);
 int obj_memory(struct rdma *rd);
+int obj_protocols(struct rdma *rd);
+int obj_stats(struct rdma *rd);
+int obj_providers(struct rdma *rd);
+int obj_monitor(struct rdma *rd);
 
 /*
  * Parser interface
rdma/stats.c new file +22 -0
new file mode 100644
index 00000000..a557e59b
@@ -0,0 +1,22 @@
+/*
+ * stats.c	RDMA tool
+ *
+ *              This program is free software; you can redistribute it and/or
+ *              modify it under the terms of the GNU General Public License
+ *              as published by the Free Software Foundation; either version
+ *              2 of the License, or (at your option) any later version.
+ *
+ * Authors:     Leon Romanovsky <[email protected]>
+ */
+
+#include "rdma.h"
+
+int obj_stats(struct rdma *rd)
+{
+	if (dev_map_init(rd)) {
+		pr_err("There are no RDMA devices\n");
+		return -ENOENT;
+	}
+
+	return 0;
+}
#17 lore raw
In reply to Leon Romanovsky
Quoted 3 lines · Leon Romanovsky

Following our discussion both in mailing list [1] and at the LPC 2016 [2], we would like to propose this RDMA tool to be part of iproute2 package and finally improve this situation.

Hello Leon,

Although I really appreciate your work: can you clarify why you would like to add *RDMA* functionality to an *IP routing* tool? I haven't found any motivation for adding RDMA functionality to iproute2 in [1].

Thanks,

Bart.

#18 lore raw
Quoted 10 lines · Bart Van Assche
Quoted 3 lines · Leon Romanovsky

Following our discussion both in mailing list [1] and at the LPC 2016 [2], we would like to propose this RDMA tool to be part of iproute2 package and finally improve this situation.

Hello Leon,

Although I really appreciate your work: can you clarify why you would like to add *RDMA* functionality to an *IP routing* tool? I haven't found any motivation for adding RDMA functionality to iproute2 in [1].

We are planning to reuse the same infrastructure provided by iproute2, like netlink parsing, access to distributions, same CLI and same standards.

Right now, RDMA is already tightened to netdev: iWARP, RoCE, IPoIB, HFI-VNIC. Many drivers (mlx, qed, i40, cxgb) are sharing code between net and RDMA.

I do expect that iproute2 will be installed on every machine with any type of connection, including IB and OPA.

So I think that it is enough to be part of that suite and don't invent our own for one specific tool.

Thanks

Quoted 7 lines

Thanks,

Bart.--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): 20170504182542.GD22833-U/[email protected]

#20 lore raw
Quoted 19 lines · Leon Romanovsky
Quoted 10 lines · Bart Van Assche
Quoted 3 lines · Leon Romanovsky

Following our discussion both in mailing list [1] and at the LPC 2016 [2], we would like to propose this RDMA tool to be part of iproute2 package and finally improve this situation.

Hello Leon,

Although I really appreciate your work: can you clarify why you would like to add *RDMA* functionality to an *IP routing* tool? I haven't found any motivation for adding RDMA functionality to iproute2 in [1].

We are planning to reuse the same infrastructure provided by iproute2, like netlink parsing, access to distributions, same CLI and same standards.

Right now, RDMA is already tightened to netdev: iWARP, RoCE, IPoIB, HFI-VNIC. Many drivers (mlx, qed, i40, cxgb) are sharing code between net and RDMA.

Quoted 5 lines

I do expect that iproute2 will be installed on every machine with any type of connection, including IB and OPA.

So I think that it is enough to be part of that suite and don't invent our own for one specific tool.

Hello Leon,

Sorry but to me that sounds like a weak argument for including RDMA functionality in iproute2. There is already a library for communication over netlink sockets, namely libnl. Is there functionality that is in iproute2 but not in libnl and that is needed for the new tool? If so, have you considered to create a new library for that functionality?

Thanks,

Bart.--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): [email protected]

#22 lore raw
Quoted 26 lines · Bart Van Assche
Quoted 19 lines · Leon Romanovsky
Quoted 10 lines · Bart Van Assche
Quoted 3 lines · Leon Romanovsky

Following our discussion both in mailing list [1] and at the LPC 2016 [2], we would like to propose this RDMA tool to be part of iproute2 package and finally improve this situation.

Hello Leon,

Although I really appreciate your work: can you clarify why you would like to add *RDMA* functionality to an *IP routing* tool? I haven't found any motivation for adding RDMA functionality to iproute2 in [1].

We are planning to reuse the same infrastructure provided by iproute2, like netlink parsing, access to distributions, same CLI and same standards.

Right now, RDMA is already tightened to netdev: iWARP, RoCE, IPoIB, HFI-VNIC. Many drivers (mlx, qed, i40, cxgb) are sharing code between net and RDMA.

Quoted 5 lines

I do expect that iproute2 will be installed on every machine with any type of connection, including IB and OPA.

So I think that it is enough to be part of that suite and don't invent our own for one specific tool.

Quoted 7 lines

Hello Leon,

Sorry but to me that sounds like a weak argument for including RDMA functionality in iproute2. There is already a library for communication over netlink sockets, namely libnl. Is there functionality that is in iproute2 but not in libnl and that is needed for the new tool? If so, have you considered to create a new library for that functionality?

It is not hard to create new tool, the hardest part is to ensure that it is part of the distributions. Did you count how many months we are trying to add rdma-core to debian?

I have enough headache with that and don't want another one.

Do you have situation in mind where you will have RDMA device without iproute2 installed?

Thanks

Quoted 4 lines

Thanks,

Bart.

#23 lore raw
Quoted 35 lines · Leon Romanovsky
Quoted 26 lines · Bart Van Assche
Quoted 19 lines · Leon Romanovsky
Quoted 10 lines · Bart Van Assche
Quoted 3 lines · Leon Romanovsky

Following our discussion both in mailing list [1] and at the LPC 2016 [2], we would like to propose this RDMA tool to be part of iproute2 package and finally improve this situation.

Hello Leon,

Although I really appreciate your work: can you clarify why you would like to add *RDMA* functionality to an *IP routing* tool? I haven't found any motivation for adding RDMA functionality to iproute2 in [1].

We are planning to reuse the same infrastructure provided by iproute2, like netlink parsing, access to distributions, same CLI and same standards.

Right now, RDMA is already tightened to netdev: iWARP, RoCE, IPoIB, HFI-VNIC. Many drivers (mlx, qed, i40, cxgb) are sharing code between net and RDMA.

Quoted 5 lines

I do expect that iproute2 will be installed on every machine with any type of connection, including IB and OPA.

So I think that it is enough to be part of that suite and don't invent our own for one specific tool.

Quoted 7 lines

Hello Leon,

Sorry but to me that sounds like a weak argument for including RDMA functionality in iproute2. There is already a library for communication over netlink sockets, namely libnl. Is there functionality that is in iproute2 but not in libnl and that is needed for the new tool? If so, have you considered to create a new library for that functionality?

Quoted 3 lines

It is not hard to create new tool, the hardest part is to ensure that it is part of the distributions. Did you count how many months we are trying to add rdma-core to debian?

I do agree that it is a strange pairing and am not really a fan. However at the end of the day it's just a name for a repo/package. If the iproute folks are fine to include rdma in their repo/package, great we can leverage their code for CLI and other common stuff.

Now if the interface was something like "ip -FlagForRdma ..." I would object to that, but the interface is "rdma ... " so from users perspective it's different tools. They don't need to care that it was sourced from a common git repo.

Just as an aside this already works a bit with OPA:

 $ ./rdma link
1/1: hfi1_0/1: ifname NONE cap_mask 0x00410022 lid 0x1 lid_mask_count 0 link_layer InfiniBand
         phys_state 5: LinkUp rate 100 Gb/sec (4X EDR) sm_lid 0x1 sm_sl 
0 state 4: ACTIVE

Leon I'll get you more feedback and testing, I've just been really bogged down this week, sorry.

-Denny

A message that is not in the archive (sent off-list or not yet received): 2dee6cde-0406-b101-0fe6-c1f6de7c1b1a-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org

#25 lore raw
Quoted 52 lines · Dennis Dalessandro
Quoted 35 lines · Leon Romanovsky
Quoted 26 lines · Bart Van Assche
Quoted 19 lines · Leon Romanovsky
Quoted 10 lines · Bart Van Assche
Quoted 3 lines · Leon Romanovsky

Following our discussion both in mailing list [1] and at the LPC 2016 [2], we would like to propose this RDMA tool to be part of iproute2 package and finally improve this situation.

Hello Leon,

Although I really appreciate your work: can you clarify why you would like to add *RDMA* functionality to an *IP routing* tool? I haven't found any motivation for adding RDMA functionality to iproute2 in [1].

We are planning to reuse the same infrastructure provided by iproute2, like netlink parsing, access to distributions, same CLI and same standards.

Right now, RDMA is already tightened to netdev: iWARP, RoCE, IPoIB, HFI-VNIC. Many drivers (mlx, qed, i40, cxgb) are sharing code between net and RDMA.

Quoted 5 lines

I do expect that iproute2 will be installed on every machine with any type of connection, including IB and OPA.

So I think that it is enough to be part of that suite and don't invent our own for one specific tool.

Quoted 7 lines

Hello Leon,

Sorry but to me that sounds like a weak argument for including RDMA functionality in iproute2. There is already a library for communication over netlink sockets, namely libnl. Is there functionality that is in iproute2 but not in libnl and that is needed for the new tool? If so, have you considered to create a new library for that functionality?

Quoted 3 lines

It is not hard to create new tool, the hardest part is to ensure that it is part of the distributions. Did you count how many months we are trying to add rdma-core to debian?

I do agree that it is a strange pairing and am not really a fan. However at the end of the day it's just a name for a repo/package. If the iproute folks are fine to include rdma in their repo/package, great we can leverage their code for CLI and other common stuff.

Now if the interface was something like "ip -FlagForRdma ..." I would object to that, but the interface is "rdma ... " so from users perspective it's different tools. They don't need to care that it was sourced from a common git repo.

Just as an aside this already works a bit with OPA:

Quoted 8 lines
 $ ./rdma link
1/1: hfi1_0/1: ifname NONE cap_mask 0x00410022 lid 0x1 lid_mask_count 0
link_layer InfiniBand
         phys_state 5: LinkUp rate 100 Gb/sec (4X EDR) sm_lid 0x1 sm_sl 0
state 4: ACTIVE

Leon I'll get you more feedback and testing, I've just been really bogged down this week, sorry.

Thanks Denny,

Before you are starting to test it, can you please provide your feedback on my initial questions? Usability and need of sysfs.

----
This is initial phase to understand if user experience for this tool fits RDMA and netdev communities exepectations. Also I would like to get feedback if it is really worth to provide legacy sysfs for old kernels, or maybe I should implement netlink from the beginning and abandon sysfs completely.
-----

P.S. I believe this will give you wrong output, because it parses IB port cap_mask. $./rdma link show hfi1_0/1 cap_mask

Thanks

Quoted 5 lines

-Denny

A message that is not in the archive (sent off-list or not yet received): 20170504194242.GF22833-U/[email protected]

#27 lore raw
Quoted 64 lines · Leon Romanovsky
Quoted 52 lines · Dennis Dalessandro
Quoted 35 lines · Leon Romanovsky
Quoted 26 lines · Bart Van Assche
Quoted 19 lines · Leon Romanovsky
Quoted 10 lines · Bart Van Assche
Quoted 3 lines · Leon Romanovsky

Following our discussion both in mailing list [1] and at the LPC 2016 [2], we would like to propose this RDMA tool to be part of iproute2 package and finally improve this situation.

Hello Leon,

Although I really appreciate your work: can you clarify why you would like to add *RDMA* functionality to an *IP routing* tool? I haven't found any motivation for adding RDMA functionality to iproute2 in [1].

We are planning to reuse the same infrastructure provided by iproute2, like netlink parsing, access to distributions, same CLI and same standards.

Right now, RDMA is already tightened to netdev: iWARP, RoCE, IPoIB, HFI-VNIC. Many drivers (mlx, qed, i40, cxgb) are sharing code between net and RDMA.

Quoted 5 lines

I do expect that iproute2 will be installed on every machine with any type of connection, including IB and OPA.

So I think that it is enough to be part of that suite and don't invent our own for one specific tool.

Quoted 7 lines

Hello Leon,

Sorry but to me that sounds like a weak argument for including RDMA functionality in iproute2. There is already a library for communication over netlink sockets, namely libnl. Is there functionality that is in iproute2 but not in libnl and that is needed for the new tool? If so, have you considered to create a new library for that functionality?

Quoted 3 lines

It is not hard to create new tool, the hardest part is to ensure that it is part of the distributions. Did you count how many months we are trying to add rdma-core to debian?

I do agree that it is a strange pairing and am not really a fan. However at the end of the day it's just a name for a repo/package. If the iproute folks are fine to include rdma in their repo/package, great we can leverage their code for CLI and other common stuff.

Now if the interface was something like "ip -FlagForRdma ..." I would object to that, but the interface is "rdma ... " so from users perspective it's different tools. They don't need to care that it was sourced from a common git repo.

Just as an aside this already works a bit with OPA:

Quoted 8 lines
 $ ./rdma link
1/1: hfi1_0/1: ifname NONE cap_mask 0x00410022 lid 0x1 lid_mask_count 0
link_layer InfiniBand
         phys_state 5: LinkUp rate 100 Gb/sec (4X EDR) sm_lid 0x1 sm_sl 0
state 4: ACTIVE

Leon I'll get you more feedback and testing, I've just been really bogged down this week, sorry.

Thanks Denny,

Quoted 8 lines

Before you are starting to test it, can you please provide your feedback on my initial questions? Usability and need of sysfs.
----
This is initial phase to understand if user experience for this tool fits RDMA and netdev communities exepectations. Also I would like to get feedback if it is really worth to provide legacy sysfs for old kernels, or maybe I should implement netlink from the beginning and abandon sysfs completely.
-----

For the initial phase I think you did the right thing by reading sysfs. I like the ability for the tool to be compatible with legacy kernels, but at the same time I don't know if it's worth the hassle. I won't fight too hard either way.

Perhaps we should take a stab and seeing what a dual sysfs/netlink interface would look like, and see just how hard and complicated it really is. Then we can make that call. You already have the sysfs version, and have to do netink anyway, so let's not rip out what's there just yet, this is an RFC after all, not like you are asking for this exact version to be merged yet.

As far as usability, I think what's here is a great start and we can continue to refine. I'm particularly interested in the stats capabilities. In particular being able to filter what stats are shown and watch as they change over time. RDMA devices have lots of counters and stats. A common tool for users to be able to get that data across HW types would be a really good thing.

-Denny

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): b82f5d3d-f198-3410-af85-85befc14a2ec-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org

#29 lore raw
Quoted 79 lines · Dennis Dalessandro
Quoted 64 lines · Leon Romanovsky
Quoted 52 lines · Dennis Dalessandro
Quoted 35 lines · Leon Romanovsky
Quoted 26 lines · Bart Van Assche
Quoted 19 lines · Leon Romanovsky
Quoted 10 lines · Bart Van Assche
Quoted 3 lines · Leon Romanovsky

Following our discussion both in mailing list [1] and at the LPC 2016 [2], we would like to propose this RDMA tool to be part of iproute2 package and finally improve this situation.

Hello Leon,

Although I really appreciate your work: can you clarify why you would like to add *RDMA* functionality to an *IP routing* tool? I haven't found any motivation for adding RDMA functionality to iproute2 in [1].

We are planning to reuse the same infrastructure provided by iproute2, like netlink parsing, access to distributions, same CLI and same standards.

Right now, RDMA is already tightened to netdev: iWARP, RoCE, IPoIB, HFI-VNIC. Many drivers (mlx, qed, i40, cxgb) are sharing code between net and RDMA.

Quoted 5 lines

I do expect that iproute2 will be installed on every machine with any type of connection, including IB and OPA.

So I think that it is enough to be part of that suite and don't invent our own for one specific tool.

Quoted 7 lines

Hello Leon,

Sorry but to me that sounds like a weak argument for including RDMA functionality in iproute2. There is already a library for communication over netlink sockets, namely libnl. Is there functionality that is in iproute2 but not in libnl and that is needed for the new tool? If so, have you considered to create a new library for that functionality?

Quoted 3 lines

It is not hard to create new tool, the hardest part is to ensure that it is part of the distributions. Did you count how many months we are trying to add rdma-core to debian?

I do agree that it is a strange pairing and am not really a fan. However at the end of the day it's just a name for a repo/package. If the iproute folks are fine to include rdma in their repo/package, great we can leverage their code for CLI and other common stuff.

Now if the interface was something like "ip -FlagForRdma ..." I would object to that, but the interface is "rdma ... " so from users perspective it's different tools. They don't need to care that it was sourced from a common git repo.

Just as an aside this already works a bit with OPA:

Quoted 8 lines
 $ ./rdma link
1/1: hfi1_0/1: ifname NONE cap_mask 0x00410022 lid 0x1 lid_mask_count 0
link_layer InfiniBand
         phys_state 5: LinkUp rate 100 Gb/sec (4X EDR) sm_lid 0x1 sm_sl 0
state 4: ACTIVE

Leon I'll get you more feedback and testing, I've just been really bogged down this week, sorry.

Thanks Denny,

Quoted 8 lines

Before you are starting to test it, can you please provide your feedback on my initial questions? Usability and need of sysfs.
----
This is initial phase to understand if user experience for this tool fits RDMA and netdev communities exepectations. Also I would like to get feedback if it is really worth to provide legacy sysfs for old kernels, or maybe I should implement netlink from the beginning and abandon sysfs completely.
-----

For the initial phase I think you did the right thing by reading sysfs. I like the ability for the tool to be compatible with legacy kernels, but at the same time I don't know if it's worth the hassle. I won't fight too hard either way.

Quoted 5 lines

Perhaps we should take a stab and seeing what a dual sysfs/netlink interface would look like, and see just how hard and complicated it really is. Then we can make that call. You already have the sysfs version, and have to do netink anyway, so let's not rip out what's there just yet, this is an RFC after all, not like you are asking for this exact version to be merged yet.

Not at all, it is too early to merge it, at the end it is POC.

Quoted 7 lines

As far as usability, I think what's here is a great start and we can continue to refine. I'm particularly interested in the stats capabilities. In particular being able to filter what stats are shown and watch as they change over time. RDMA devices have lots of counters and stats. A common tool for users to be able to get that data across HW types would be a really good thing.

Yeah, the statistics will be tough nut to crack.

Quoted 19 lines

-Denny

#30 lore raw
In reply to a message not in the archive
Quoted 62 lines · Dennis Dalessandro
Quoted 55 lines · Leon Romanovsky
Quoted 43 lines · Bart Van Assche
Quoted 31 lines · Leon Romanovsky
Quoted 9 lines · Bart Van Assche
Quoted 6 lines · Leon Romanovsky

Following our discussion both in mailing list [1] and at the LPC 2016 [2],
we would like to propose this RDMA tool to be part of iproute2 package
and finally improve this situation.

Quoted 7 lines

Hello Leon,

Although I really appreciate your work: can you clarify why you would like to
add *RDMA* functionality to an *IP routing* tool? I haven't found any motivation
for adding RDMA functionality to iproute2 in [1].

We are planning to reuse the same infrastructure provided by iproute2,
like netlink parsing, access to distributions, same CLI and same standards.

Right now, RDMA is already tightened to netdev: iWARP, RoCE, IPoIB, HFI-VNIC.
Many drivers (mlx, qed, i40, cxgb) are sharing code between net and
RDMA.

Quoted 7 lines

I do expect that iproute2 will be installed on every machine with any
type of connection, including IB and OPA.

So I think that it is enough to be part of that suite and don't invent
our own for one specific tool.

Hello Leon,

Quoted 9 lines

Sorry but to me that sounds like a weak argument for including RDMA functionality
in iproute2. There is already a library for communication over netlink sockets,
namely libnl. Is there functionality that is in iproute2 but not in libnl and
that is needed for the new tool? If so, have you considered to create a new
library for that functionality?

Quoted 5 lines

It is not hard to create new tool, the hardest part is to ensure that it is
part of the distributions. Did you count how many months we are trying to
add rdma-core to debian?

Quoted 6 lines

I do agree that it is a strange pairing and am not really a fan. However
at the end of the day it's just a name for a repo/package. If the iproute folks are fine to include rdma in their repo/package, great we
can leverage their code for CLI and other common stuff.

If you look into the iproute2 package, it becomes clear that the name iproute2 is historical and not really accurate any more.  It contains things like the bridge control software, tc for controlling send queues, and many things network related but not routing related.  The rdma tool is a perfectly fine fit in the sense that it is an additional network management tool IMO.

For reference, here's the list of stuff already in iproute on my Fedora 24 box:

/usr/sbin/arpd
/usr/sbin/bridge
/usr/sbin/cbq
/usr/sbin/ctstat
/usr/sbin/genl
/usr/sbin/ifcfg
/usr/sbin/ifstat
/usr/sbin/ip
/usr/sbin/lnstat
/usr/sbin/nstat
/usr/sbin/routef
/usr/sbin/routel
/usr/sbin/rtacct
/usr/sbin/rtmon
/usr/sbin/rtpr
/usr/sbin/rtstat
/usr/sbin/ss
/usr/sbin/tc
/usr/sbin/tipc

And in fact, if you check, tipc is almost similar to RDMA ;-)  So, I suggest people not get hung up on the name iproute2, the fit is fine when you look deeper into the nature of the package.

Signature · 9 lines
Doug Ledford <[email protected]>
    GPG KeyID: B826A3330E572FDD
   
Key fingerprint = AE6B 1BDA 122B 23B4 265B  1274 B826 A333 0E57 2FDD

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): 1493930758.3041.231.camel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org

#32 lore raw
Quoted 100 lines · On Thu, 04 May 2017 16:45:58 -0400 Doug Ledford…
Quoted 62 lines · Dennis Dalessandro
Quoted 55 lines · Leon Romanovsky
Quoted 43 lines · Bart Van Assche
Quoted 31 lines · Leon Romanovsky
Quoted 9 lines · Bart Van Assche
Quoted 6 lines · Leon Romanovsky

Following our discussion both in mailing list [1] and at the LPC 2016 [2],
we would like to propose this RDMA tool to be part of iproute2 package
and finally improve this situation.

Quoted 7 lines

Hello Leon,

Although I really appreciate your work: can you clarify why you would like to
add *RDMA* functionality to an *IP routing* tool? I haven't found any motivation
for adding RDMA functionality to iproute2 in [1].

We are planning to reuse the same infrastructure provided by iproute2,
like netlink parsing, access to distributions, same CLI and same standards.

Right now, RDMA is already tightened to netdev: iWARP, RoCE, IPoIB, HFI-VNIC.
Many drivers (mlx, qed, i40, cxgb) are sharing code between net and
RDMA.

Quoted 7 lines

I do expect that iproute2 will be installed on every machine with any
type of connection, including IB and OPA.

So I think that it is enough to be part of that suite and don't invent
our own for one specific tool.

Hello Leon,

Quoted 9 lines

Sorry but to me that sounds like a weak argument for including RDMA functionality
in iproute2. There is already a library for communication over netlink sockets,
namely libnl. Is there functionality that is in iproute2 but not in libnl and
that is needed for the new tool? If so, have you considered to create a new
library for that functionality?

Quoted 5 lines

It is not hard to create new tool, the hardest part is to ensure that it is
part of the distributions. Did you count how many months we are trying to
add rdma-core to debian?

Quoted 6 lines

I do agree that it is a strange pairing and am not really a fan. However
at the end of the day it's just a name for a repo/package. If the iproute folks are fine to include rdma in their repo/package, great we
can leverage their code for CLI and other common stuff.

If you look into the iproute2 package, it becomes clear that the name iproute2 is historical and not really accurate any more.  It contains things like the bridge control software, tc for controlling send queues, and many things network related but not routing related.  The rdma tool is a perfectly fine fit in the sense that it is an additional network management tool IMO.

For reference, here's the list of stuff already in iproute on my Fedora 24 box:

/usr/sbin/arpd
/usr/sbin/bridge
/usr/sbin/cbq
/usr/sbin/ctstat
/usr/sbin/genl
/usr/sbin/ifcfg
/usr/sbin/ifstat
/usr/sbin/ip
/usr/sbin/lnstat
/usr/sbin/nstat
/usr/sbin/routef
/usr/sbin/routel
/usr/sbin/rtacct
/usr/sbin/rtmon
/usr/sbin/rtpr
/usr/sbin/rtstat
/usr/sbin/ss
/usr/sbin/tc
/usr/sbin/tipc

Quoted 4 lines

And in fact, if you check, tipc is almost similar to RDMA ;-)  So, I suggest people not get hung up on the name iproute2, the fit is fine when you look deeper into the nature of the package.

Iproute2 is a collection like busybox. It has bridging, devlink and tipc already. --
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#33 lore raw
In reply to Leon Romanovsky
Quoted 3 lines · Leon Romanovsky

It is not hard to create new tool, the hardest part is to ensure that it is part of the distributions. Did you count how many months we are trying to add rdma-core to debian?

Hello Leon,

Sorry but I was not aware that the effort to add rdma-core to Debian was taking that long. Please let me know if I can help with that effort.

Bart.

#34 lore raw
In reply to Bart Van Assche
Quoted 10 lines · Thu, May 04, 2017 at 08:10:54PM CEST, [email protected] wrote:
Quoted 3 lines · Leon Romanovsky

Following our discussion both in mailing list [1] and at the LPC 2016 [2], we would like to propose this RDMA tool to be part of iproute2 package and finally improve this situation.

Hello Leon,

Although I really appreciate your work: can you clarify why you would like to add *RDMA* functionality to an *IP routing* tool? I haven't found any motivation for adding RDMA functionality to iproute2 in [1].

Bart, please realize that iproute2 is much more than "*IP routing* tool". I understand you got confused by the name. Please see sources. Your comment is totally pointless...

#35 lore raw

T24gU2F0LCAyMDE3LTA1LTA2IGF0IDEyOjQwICswMjAwLCBKaXJpIFBpcmtvIHdyb3RlOg0KPiBU aHUsIE1heSAwNCwgMjAxNyBhdCAwODoxMDo1NFBNIENFU1QsIEJhcnQuVmFuQXNzY2hlQHNhbmRp c2suY29tIHdyb3RlOg0KPiA+IE9uIFRodSwgMjAxNy0wNS0wNCBhdCAyMTowMiArMDMwMCwgTGVv biBSb21hbm92c2t5IHdyb3RlOg0KPiA+ID4gRm9sbG93aW5nIG91ciBkaXNjdXNzaW9uIGJvdGgg aW4gbWFpbGluZyBsaXN0IFsxXSBhbmQgYXQgdGhlIExQQyAyMDE2IFsyXSwNCj4gPiA+IHdlIHdv dWxkIGxpa2UgdG8gcHJvcG9zZSB0aGlzIFJETUEgdG9vbCB0byBiZSBwYXJ0IG9mIGlwcm91dGUy IHBhY2thZ2UNCj4gPiA+IGFuZCBmaW5hbGx5IGltcHJvdmUgdGhpcyBzaXR1YXRpb24uDQo+ID4g DQo+ID4gQWx0aG91Z2ggSSByZWFsbHkgYXBwcmVjaWF0ZSB5b3VyIHdvcms6IGNhbiB5b3UgY2xh cmlmeSB3aHkgeW91IHdvdWxkIGxpa2UgdG8NCj4gPiBhZGQgKlJETUEqIGZ1bmN0aW9uYWxpdHkg dG8gYW4gKklQIHJvdXRpbmcqIHRvb2w/IEkgaGF2ZW4ndCBmb3VuZCBhbnkgbW90aXZhdGlvbg0K PiA+IGZvciBhZGRpbmcgUkRNQSBmdW5jdGlvbmFsaXR5IHRvIGlwcm91dGUyIGluIFsxXS4NCj4g DQo+IEJhcnQsIHBsZWFzZSByZWFsaXplIHRoYXQgaXByb3V0ZTIgaXMgbXVjaCBtb3JlIHRoYW4g IipJUCByb3V0aW5nKiB0b29sIi4NCj4gSSB1bmRlcnN0YW5kIHlvdSBnb3QgY29uZnVzZWQgYnkg dGhlIG5hbWUuIFBsZWFzZSBzZWUgc291cmNlcy4gWW91ciBjb21tZW50DQo+IGlzIHRvdGFsbHkg cG9pbnRsZXNzLi4uDQoNCkkgYXNrZWQgZm9yIGEgY2xhcmlmaWNhdGlvbiB0aGF0IHNob3VsZCBo YXZlIGJlZW4gaW4gdGhlIGNvdmVyIGxldHRlciBidXQgdGhhdA0Kd2FzIG1pc3NpbmcgZnJvbSB0 aGF0IGNvdmVyIGxldHRlci4gU28gSSB0aGluayB0aGF0IHdhcyB0aGUgcmlnaHQgdGhpbmcgdG8g ZG8NCmluc3RlYWQgb2YgcG9pbnRsZXNzLiBCVFcsIGNhbiB5b3UgZXhwbGFpbiB3aHkgeW91IGFy ZSB1c2luZyBhbiBlLW1haWwgYWRkcmVzcw0KdGhhdCBpcyBoaWRpbmcgdGhhdCB5b3UgYXJlIGEg TWVsbGFub3ggZW1wbG95ZWU/DQoNCkJhcnQu
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#36 lore raw
Quoted 11 lines · Bart Van Assche
Quoted 10 lines · Jiri Pirko
Quoted 8 lines · Thu, May 04, 2017 at 08:10:54PM CEST, [email protected] wrote:
Quoted 3 lines · Leon Romanovsky

Following our discussion both in mailing list [1] and at the LPC 2016 [2], we would like to propose this RDMA tool to be part of iproute2 package and finally improve this situation.

Although I really appreciate your work: can you clarify why you would like to add *RDMA* functionality to an *IP routing* tool? I haven't found any motivation for adding RDMA functionality to iproute2 in [1].

Quoted 8 lines
Quoted 3 lines

Bart, please realize that iproute2 is much more than "*IP routing* tool". I understand you got confused by the name. Please see sources. Your comment is totally pointless...

I asked for a clarification that should have been in the cover letter but that was missing from that cover letter. So I think that was the right thing to do instead of pointless. BTW, can you explain why you are using an e-mail address that is hiding that you are a Mellanox employee?

For the same reason as I do. It is much easier to use outside servers than Mellanox's IT infrastructure.

Right now, I'm speaking for myself, but for me, to properly answer with @mellanox.com, I need to use very specific mail setup, while for any other addresses I can and use any sane setup, including mobile application.

Quoted 2 lines

Bart.

#37 lore raw
In reply to Bart Van Assche
Quoted 11 lines · Sat, May 06, 2017 at 04:40:24PM CEST, [email protected] wrote:
Quoted 10 lines · Jiri Pirko
Quoted 8 lines · Thu, May 04, 2017 at 08:10:54PM CEST, [email protected] wrote:
Quoted 3 lines · Leon Romanovsky

Following our discussion both in mailing list [1] and at the LPC 2016 [2], we would like to propose this RDMA tool to be part of iproute2 package and finally improve this situation.

Although I really appreciate your work: can you clarify why you would like to add *RDMA* functionality to an *IP routing* tool? I haven't found any motivation for adding RDMA functionality to iproute2 in [1].

Quoted 6 lines
Quoted 3 lines

Bart, please realize that iproute2 is much more than "*IP routing* tool". I understand you got confused by the name. Please see sources. Your comment is totally pointless...

I asked for a clarification that should have been in the cover letter but that was missing from that cover letter. So I think that was the right thing to do

I think that was just complete misunderstanding about what iproute2 is.

Quoted 2 lines

instead of pointless. BTW, can you explain why you are using an e-mail address that is hiding that you are a Mellanox employee?

Who sais I have to do that? This is funny...

A message that is not in the archive (sent off-list or not yet received): [email protected]

#39 lore raw
Quoted 12 lines · Jiri Pirko
Quoted 11 lines · Sat, May 06, 2017 at 04:40:24PM CEST, Bart.VanAssche-XdAiOPVOjttBDgjK7y7TUQ@pub…
Quoted 10 lines · Jiri Pirko
Quoted 8 lines · Thu, May 04, 2017 at 08:10:54PM CEST, Bart.VanAssche-XdAiOPVOjttBDgjK7y7TUQ@pub…
Quoted 3 lines · Leon Romanovsky

Following our discussion both in mailing list [1] and at the LPC 2016 [2], we would like to propose this RDMA tool to be part of iproute2 package and finally improve this situation.

Although I really appreciate your work: can you clarify why you would like to add *RDMA* functionality to an *IP routing* tool? I haven't found any motivation for adding RDMA functionality to iproute2 in [1].

Quoted 8 lines
Quoted 6 lines
Quoted 3 lines

Bart, please realize that iproute2 is much more than "*IP routing* tool". I understand you got confused by the name. Please see sources. Your comment is totally pointless...

I asked for a clarification that should have been in the cover letter but that was missing from that cover letter. So I think that was the right thing to do

I think that was just complete misunderstanding about what iproute2 is.

Hello Jiri,

I do not agree with your reply. The abbreviation "IP" occurs in the package name and that is a reference to the "Internet Protocol". As far as I know originally the iproute2 package contained only tools related to the Internet Protocol. Other tools, e.g. the tipc tool, were added later on. What I'm wondering about is whether it really is a good idea to add tools like tipc and rdma to the iproute2 package. The iproute2 package is so essential that it gets installed on every Linux system, including embedded systems and smartphones based on Linux. Several companies maintain embedded Linux distributions and tools to build software images. These tools provide a user interface that allows to select what packages go into such an image. Adding tools like tipc and rdma to the iproute2 package makes it harder than necessary for those who build software images for embedded devices to minimize the size of such an image. As you probably know even today the size of a software image still matters for embedded devices. Something else I have been wondering about is whether bundling the tipc and rdma tools in the iproute2 package will make the job harder of people who build Android ROMs? The ip tool is present in every Android ROM, and the size of these ROMs matters because the larger these ROMs are the less space remains for apps.

Bart.--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): [email protected]

#41 lore raw
Quoted 43 lines · On Mon, 8 May 2017 15:19:28 +0000 Bart Van Assche…
Quoted 12 lines · Jiri Pirko
Quoted 11 lines · Sat, May 06, 2017 at 04:40:24PM CEST, Bart.VanAssche-XdAiOPVOjttBDgjK7y7TUQ@pub…
Quoted 10 lines · Jiri Pirko
Quoted 8 lines · Thu, May 04, 2017 at 08:10:54PM CEST, Bart.VanAssche-XdAiOPVOjttBDgjK7y7TUQ@pub…
Quoted 3 lines · Leon Romanovsky

Following our discussion both in mailing list [1] and at the LPC 2016 [2], we would like to propose this RDMA tool to be part of iproute2 package and finally improve this situation.

Although I really appreciate your work: can you clarify why you would like to add *RDMA* functionality to an *IP routing* tool? I haven't found any motivation for adding RDMA functionality to iproute2 in [1].

Quoted 8 lines
Quoted 6 lines
Quoted 3 lines

Bart, please realize that iproute2 is much more than "*IP routing* tool". I understand you got confused by the name. Please see sources. Your comment is totally pointless...

I asked for a clarification that should have been in the cover letter but that was missing from that cover letter. So I think that was the right thing to do

I think that was just complete misunderstanding about what iproute2 is.

Hello Jiri,

I do not agree with your reply. The abbreviation "IP" occurs in the package name and that is a reference to the "Internet Protocol". As far as I know originally the iproute2 package contained only tools related to the Internet Protocol. Other tools, e.g. the tipc tool, were added later on. What I'm wondering about is whether it really is a good idea to add tools like tipc and rdma to the iproute2 package. The iproute2 package is so essential that it gets installed on every Linux system, including embedded systems and smartphones based on Linux. Several companies maintain embedded Linux distributions and tools to build software images. These tools provide a user interface that allows to select what packages go into such an image. Adding tools like tipc and rdma to the iproute2 package makes it harder than necessary for those who build software images for embedded devices to minimize the size of such an image. As you probably know even today the size of a software image still matters for embedded devices. Something else I have been wondering about is whether bundling the tipc and rdma tools in the iproute2 package will make the job harder of people who build Android ROMs? The ip tool is present in every Android ROM, and the size of these ROMs matters because the larger these ROMs are the less space remains for apps.

Quoted 1 line

Bart.

I would assume embedded world does not use the standard distro model of "got to have them all". The sane way to do builds for embedded is build everything in the source, but selectively install components based on a Bill Of Materials file.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#42 lore raw
In reply to Bart Van Assche
Quoted 3 lines

Several companies maintain embedded Linux
distributions and tools to build software images. These tools provide a user interface that allows to select what packages go into such an image.

The tools allow you to select what binary packages are placed into the image. You can build multiple binary packages from one source package. Desktop distributions are not likely to do this for something as small as iproute2. But embedded distributions can easily break up iproute2 into a number of smaller packages, as you suggested, tipc, devlink, tc, bridge, ss, etc.

Openwrt does exactly this:

https://github.com/openwrt-mirror/openwrt/blob/master/package/network/utils/iproute2/Makefile

Andrew