Skip to contentSkip to conversation

thunderbolt: Add DMA traffic test driver

PATCH In review Mika Westerberg 2 participants 13 messages Nov 10, 2020 – Nov 11, 2020

Subsystems (5)
lore
Share & tools
lore
#0 lore raw

v2 cover thunderbolt: Add DMA traffic test driver

Hi all,

This series adds a new Thunderbolt service driver that can be used on manufacturing floor to test that each Thunderbolt/USB4 port is functional. It can be done either using a special loopback dongle that has RX and TX lanes crossed, or by connecting a cable back to the host (for those who don't have these dongles).

This takes advantage of the existing XDomain protocol and creates XDomain devices for the loops back to the host where the DMA traffic test driver can bind to.

The DMA traffic test driver creates a tunnel through the fabric and then sends and receives data frames over the tunnel checking for different errors.

The previous version can be found here:

https://lore.kernel.org/linux-usb/[email protected]/

Changes from the previous version:

* Fix resource leak in tb_xdp_handle_request() (patch 2/10)
* Use debugfs_remove_recursive() in tb_service_debugfs_remove() (patch 6/10)
* Add tags from Yehezkel

Isaac Hazan (4):
thunderbolt: Add link_speed and link_width to XDomain
thunderbolt: Add functions for enabling and disabling lane bonding on XDomain
thunderbolt: Add DMA traffic test driver
MAINTAINERS: Add Isaac as maintainer of Thunderbolt DMA traffic test driver

Mika Westerberg (6):
thunderbolt: Do not clear USB4 router protocol adapter IFC and ISE bits
thunderbolt: Find XDomain by route instead of UUID
thunderbolt: Create XDomain devices for loops back to the host
thunderbolt: Create debugfs directory automatically for services
thunderbolt: Make it possible to allocate one directional DMA tunnel
thunderbolt: Add support for end-to-end flow control

 .../ABI/testing/sysfs-bus-thunderbolt         |  28 +
 MAINTAINERS                                   |   6 +
 drivers/net/thunderbolt.c                     |   2 +-
 drivers/thunderbolt/Kconfig                   |  13 +
 drivers/thunderbolt/Makefile                  |   3 +
 drivers/thunderbolt/ctl.c                     |   4 +-
 drivers/thunderbolt/debugfs.c                 |  24 +
 drivers/thunderbolt/dma_test.c                | 736 ++++++++++++++++++
 drivers/thunderbolt/nhi.c                     |  36 +-
 drivers/thunderbolt/path.c                    |  13 +-
 drivers/thunderbolt/switch.c                  |  33 +-
 drivers/thunderbolt/tb.h                      |   8 +
 drivers/thunderbolt/tunnel.c                  |  50 +-
 drivers/thunderbolt/xdomain.c                 | 148 +++-
 include/linux/thunderbolt.h                   |  18 +-
 15 files changed, 1080 insertions(+), 42 deletions(-)
 create mode 100644 drivers/thunderbolt/dma_test.c
Signature · 1 line
2.28.0
#1 lore raw

PATCH v2 1/10 thunderbolt: Do not clear USB4 router protocol adapter IFC and ISE bits

These fields are marked as vendor defined in the USB4 spec and should not be modified by the software, so only clear them when we are dealing with pre-USB4 hardware.

---
 drivers/thunderbolt/path.c | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)
drivers/thunderbolt/path.c +10 -3
index 03e7b714deab..7c2c45d9ba4a 100644
@@ -406,10 +406,17 @@ static int __tb_path_deactivate_hop(struct tb_port *port, int hop_index,
 
 		if (!hop.pending) {
 			if (clear_fc) {
-				/* Clear flow control */
-				hop.ingress_fc = 0;
+				/*
+				 * Clear flow control. Protocol adapters
+				 * IFC and ISE bits are vendor defined
+				 * in the USB4 spec so we clear them
+				 * only for pre-USB4 adapters.
+				 */
+				if (!tb_switch_is_usb4(port->sw)) {
+					hop.ingress_fc = 0;
+					hop.ingress_shared_buffer = 0;
+				}
 				hop.egress_fc = 0;
-				hop.ingress_shared_buffer = 0;
 				hop.egress_shared_buffer = 0;
 
 				return tb_port_write(port, &hop, TB_CFG_HOPS,
Signature · 1 line
2.28.0
#2 lore raw

PATCH v2 2/10 thunderbolt: Find XDomain by route instead of UUID

In reply to Mika Westerberg

We are going to represent loops back to the host also as XDomains and they all have the same (host) UUID, so finding them needs to use route string instead. This also requires that we check if the XDomain device is added to the bus before its properties can be updated. Otherwise the remote UUID might not be populated yet.

---
 drivers/thunderbolt/xdomain.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
drivers/thunderbolt/xdomain.c +5 -5
index 48907853732a..e436e9efa7e7 100644
@@ -587,8 +587,6 @@ static void tb_xdp_handle_request(struct work_struct *work)
 		break;
 
 	case PROPERTIES_CHANGED_REQUEST: {
-		const struct tb_xdp_properties_changed *xchg =
-			(const struct tb_xdp_properties_changed *)pkg;
 		struct tb_xdomain *xd;
 
 		ret = tb_xdp_properties_changed_response(ctl, route, sequence);
@@ -598,10 +596,12 @@ static void tb_xdp_handle_request(struct work_struct *work)
 		 * the xdomain related to this connection as well in
 		 * case there is a change in services it offers.
 		 */
-		xd = tb_xdomain_find_by_uuid_locked(tb, &xchg->src_uuid);
+		xd = tb_xdomain_find_by_route_locked(tb, route);
 		if (xd) {
-			queue_delayed_work(tb->wq, &xd->get_properties_work,
-					   msecs_to_jiffies(50));
+			if (device_is_registered(&xd->dev)) {
+				queue_delayed_work(tb->wq, &xd->get_properties_work,
+						   msecs_to_jiffies(50));
+			}
 			tb_xdomain_put(xd);
 		}
 
Signature · 1 line
2.28.0
#3 lore raw

PATCH v2 3/10 thunderbolt: Create XDomain devices for loops back to the host

In reply to Mika Westerberg

It is perfectly possible to have loops back from the routers to the host, or even from one host port to another. Instead of ignoring these, we create XDomain devices for each. This allows creating services such as DMA traffic test that is used in manufacturing for example.

---
 drivers/thunderbolt/xdomain.c | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)
drivers/thunderbolt/xdomain.c +1 -3
index e436e9efa7e7..da229ac4e471 100644
@@ -961,10 +961,8 @@ static void tb_xdomain_get_uuid(struct work_struct *work)
 		return;
 	}
 
-	if (uuid_equal(&uuid, xd->local_uuid)) {
+	if (uuid_equal(&uuid, xd->local_uuid))
 		dev_dbg(&xd->dev, "intra-domain loop detected\n");
-		return;
-	}
 
 	/*
 	 * If the UUID is different, there is another domain connected
Signature · 1 line
2.28.0
#4 lore raw

PATCH v2 4/10 thunderbolt: Add link_speed and link_width to XDomain

In reply to Mika Westerberg

From: Isaac Hazan <[email protected]>

Link speed and link width are needed for checking expected values in case of using a loopback service.

---
 .../ABI/testing/sysfs-bus-thunderbolt         | 28 ++++++++
 drivers/thunderbolt/switch.c                  |  9 ++-
 drivers/thunderbolt/tb.h                      |  1 +
 drivers/thunderbolt/xdomain.c                 | 65 +++++++++++++++++++
 include/linux/thunderbolt.h                   |  4 ++
 5 files changed, 106 insertions(+), 1 deletion(-)
Documentation/ABI/testing/sysfs-bus-thunderbolt +28 -0
index 0b4ab9e4b8f4..a91b4b24496e 100644
@@ -1,3 +1,31 @@
+What:		/sys/bus/thunderbolt/devices/<xdomain>/rx_speed
+Date:		Feb 2021
+KernelVersion:	5.11
+Contact:	Isaac Hazan <[email protected]>
+Description:	This attribute reports the XDomain RX speed per lane.
+		All RX lanes run at the same speed.
+
+What:		/sys/bus/thunderbolt/devices/<xdomain>/rx_lanes
+Date:		Feb 2021
+KernelVersion:	5.11
+Contact:	Isaac Hazan <[email protected]>
+Description:	This attribute reports the number of RX lanes the XDomain
+		is using simultaneously through its upstream port.
+
+What:		/sys/bus/thunderbolt/devices/<xdomain>/tx_speed
+Date:		Feb 2021
+KernelVersion:	5.11
+Contact:	Isaac Hazan <[email protected]>
+Description:	This attribute reports the XDomain TX speed per lane.
+		All TX lanes run at the same speed.
+
+What:		/sys/bus/thunderbolt/devices/<xdomain>/tx_lanes
+Date:		Feb 2021
+KernelVersion:	5.11
+Contact:	Isaac Hazan <[email protected]>
+Description:	This attribute reports number of TX lanes the XDomain
+		is using simultaneously through its upstream port.
+
 What: /sys/bus/thunderbolt/devices/.../domainX/boot_acl
 Date:		Jun 2018
 KernelVersion:	4.17
drivers/thunderbolt/switch.c +8 -1
index c73bbfe69ba1..05a360901790 100644
@@ -932,7 +932,14 @@ int tb_port_get_link_speed(struct tb_port *port)
 	return speed == LANE_ADP_CS_1_CURRENT_SPEED_GEN3 ? 20 : 10;
 }
 
-static int tb_port_get_link_width(struct tb_port *port)
+/**
+ * tb_port_get_link_width() - Get current link width
+ * @port: Port to check (USB4 or CIO)
+ *
+ * Returns link width. Return values can be 1 (Single-Lane), 2 (Dual-Lane)
+ * or negative errno in case of failure.
+ */
+int tb_port_get_link_width(struct tb_port *port)
 {
 	u32 val;
 	int ret;
drivers/thunderbolt/tb.h +1 -0
index a9995e21b916..3a826315049e 100644
@@ -862,6 +862,7 @@ struct tb_port *tb_next_port_on_path(struct tb_port *start, struct tb_port *end,
 	     (p) = tb_next_port_on_path((src), (dst), (p)))
 
 int tb_port_get_link_speed(struct tb_port *port);
+int tb_port_get_link_width(struct tb_port *port);
 
 int tb_switch_find_vse_cap(struct tb_switch *sw, enum tb_switch_vse_cap vsec);
 int tb_switch_find_cap(struct tb_switch *sw, enum tb_switch_cap cap);
drivers/thunderbolt/xdomain.c +65 -0

86 lines not on this page · raw message

include/linux/thunderbolt.h +4 -0
index 5db2b11ab085..e441af88ed77 100644
@@ -179,6 +179,8 @@ void tb_unregister_property_dir(const char *key, struct tb_property_dir *dir);
  * @lock: Lock to serialize access to the following fields of this structure
  * @vendor_name: Name of the vendor (or %NULL if not known)
  * @device_name: Name of the device (or %NULL if not known)
+ * @link_speed: Speed of the link in Gb/s
+ * @link_width: Width of the link (1 or 2)
  * @is_unplugged: The XDomain is unplugged
  * @resume: The XDomain is being resumed
  * @needs_uuid: If the XDomain does not have @remote_uuid it will be
@@ -223,6 +225,8 @@ struct tb_xdomain {
 	struct mutex lock;
 	const char *vendor_name;
 	const char *device_name;
+	unsigned int link_speed;
+	unsigned int link_width;
 	bool is_unplugged;
 	bool resume;
 	bool needs_uuid;
Signature · 1 line
2.28.0
#5 lore raw

PATCH v2 5/10 thunderbolt: Add functions for enabling and disabling lane bonding on XDomain

In reply to Mika Westerberg

From: Isaac Hazan <[email protected]>

These can be used by service drivers to enable and disable lane bonding as needed.

---
 drivers/thunderbolt/switch.c  | 24 +++++++++++--
 drivers/thunderbolt/tb.h      |  3 ++
 drivers/thunderbolt/xdomain.c | 66 +++++++++++++++++++++++++++++++++++
 include/linux/thunderbolt.h   |  2 ++
 4 files changed, 92 insertions(+), 3 deletions(-)
drivers/thunderbolt/switch.c +21 -3
index 05a360901790..cdfd8cccfe19 100644
@@ -503,12 +503,13 @@ static void tb_dump_port(struct tb *tb, struct tb_regs_port_header *port)
 
 /**
  * tb_port_state() - get connectedness state of a port
+ * @port: the port to check
  *
  * The port must have a TB_CAP_PHY (i.e. it should be a real port).
  *
  * Return: Returns an enum tb_port_state on success or an error code on failure.
  */
-static int tb_port_state(struct tb_port *port)
+int tb_port_state(struct tb_port *port)
 {
 	struct tb_cap_phy phy;
 	int res;
@@ -1008,7 +1009,16 @@ static int tb_port_set_link_width(struct tb_port *port, unsigned int width)
 			     port->cap_phy + LANE_ADP_CS_1, 1);
 }
 
-static int tb_port_lane_bonding_enable(struct tb_port *port)
+/**
+ * tb_port_lane_bonding_enable() - Enable bonding on port
+ * @port: port to enable
+ *
+ * Enable bonding by setting the link width of the port and the
+ * other port in case of dual link port.
+ *
+ * Return: %0 in case of success and negative errno in case of error
+ */
+int tb_port_lane_bonding_enable(struct tb_port *port)
 {
 	int ret;
 
@@ -1038,7 +1048,15 @@ static int tb_port_lane_bonding_enable(struct tb_port *port)
 	return 0;
 }
 
-static void tb_port_lane_bonding_disable(struct tb_port *port)
+/**
+ * tb_port_lane_bonding_disable() - Disable bonding on port
+ * @port: port to disable
+ *
+ * Disable bonding by setting the link width of the port and the
+ * other port in case of dual link port.
+ *
+ */
+void tb_port_lane_bonding_disable(struct tb_port *port)
 {
 	port->dual_link_port->bonded = false;
 	port->bonded = false;
drivers/thunderbolt/tb.h +3 -0
index 3a826315049e..e98d3561648d 100644
@@ -863,6 +863,9 @@ struct tb_port *tb_next_port_on_path(struct tb_port *start, struct tb_port *end,
 
 int tb_port_get_link_speed(struct tb_port *port);
 int tb_port_get_link_width(struct tb_port *port);
+int tb_port_state(struct tb_port *port);
+int tb_port_lane_bonding_enable(struct tb_port *port);
+void tb_port_lane_bonding_disable(struct tb_port *port);
 
 int tb_switch_find_vse_cap(struct tb_switch *sw, enum tb_switch_vse_cap vsec);
 int tb_switch_find_cap(struct tb_switch *sw, enum tb_switch_cap cap);
drivers/thunderbolt/xdomain.c +66 -0

84 lines not on this page · raw message

include/linux/thunderbolt.h +2 -0
index e441af88ed77..0a747f92847e 100644
@@ -247,6 +247,8 @@ struct tb_xdomain {
 	u8 depth;
 };
 
+int tb_xdomain_lane_bonding_enable(struct tb_xdomain *xd);
+void tb_xdomain_lane_bonding_disable(struct tb_xdomain *xd);
 int tb_xdomain_enable_paths(struct tb_xdomain *xd, u16 transmit_path,
 			    u16 transmit_ring, u16 receive_path,
 			    u16 receive_ring);
Signature · 1 line
2.28.0
#6 lore raw

PATCH v2 6/10 thunderbolt: Create debugfs directory automatically for services

In reply to Mika Westerberg

This allows service drivers to use it as parent directory if they need to add their own debugfs entries.

---
 drivers/thunderbolt/debugfs.c | 24 ++++++++++++++++++++++++
 drivers/thunderbolt/tb.h      |  4 ++++
 drivers/thunderbolt/xdomain.c |  3 +++
 include/linux/thunderbolt.h   |  4 ++++
 4 files changed, 35 insertions(+)
drivers/thunderbolt/debugfs.c +24 -0
index 3680b2784ea1..e53ca8270acd 100644
@@ -690,6 +690,30 @@ void tb_switch_debugfs_remove(struct tb_switch *sw)
 	debugfs_remove_recursive(sw->debugfs_dir);
 }
 
+/**
+ * tb_service_debugfs_init() - Add debugfs directory for service
+ * @svc: Thunderbolt service pointer
+ *
+ * Adds debugfs directory for service.
+ */
+void tb_service_debugfs_init(struct tb_service *svc)
+{
+	svc->debugfs_dir = debugfs_create_dir(dev_name(&svc->dev),
+					      tb_debugfs_root);
+}
+
+/**
+ * tb_service_debugfs_remove() - Remove service debugfs directory
+ * @svc: Thunderbolt service pointer
+ *
+ * Removes the previously created debugfs directory for @svc.
+ */
+void tb_service_debugfs_remove(struct tb_service *svc)
+{
+	debugfs_remove_recursive(svc->debugfs_dir);
+	svc->debugfs_dir = NULL;
+}
+
 void tb_debugfs_init(void)
 {
 	tb_debugfs_root = debugfs_create_dir("thunderbolt", NULL);
drivers/thunderbolt/tb.h +4 -0
index e98d3561648d..a21000649009 100644
@@ -1027,11 +1027,15 @@ void tb_debugfs_init(void);
 void tb_debugfs_exit(void);
 void tb_switch_debugfs_init(struct tb_switch *sw);
 void tb_switch_debugfs_remove(struct tb_switch *sw);
+void tb_service_debugfs_init(struct tb_service *svc);
+void tb_service_debugfs_remove(struct tb_service *svc);
 #else
 static inline void tb_debugfs_init(void) { }
 static inline void tb_debugfs_exit(void) { }
 static inline void tb_switch_debugfs_init(struct tb_switch *sw) { }
 static inline void tb_switch_debugfs_remove(struct tb_switch *sw) { }
+static inline void tb_service_debugfs_init(struct tb_service *svc) { }
+static inline void tb_service_debugfs_remove(struct tb_service *svc) { }
 #endif
 
 #ifdef CONFIG_USB4_KUNIT_TEST
drivers/thunderbolt/xdomain.c +3 -0
index 65108216bfe3..1a0491b461fd 100644
@@ -779,6 +779,7 @@ static void tb_service_release(struct device *dev)
 	struct tb_service *svc = container_of(dev, struct tb_service, dev);
 	struct tb_xdomain *xd = tb_service_parent(svc);
 
+	tb_service_debugfs_remove(svc);
 	ida_simple_remove(&xd->service_ids, svc->id);
 	kfree(svc->key);
 	kfree(svc);
@@ -892,6 +893,8 @@ static void enumerate_services(struct tb_xdomain *xd)
 		svc->dev.parent = &xd->dev;
 		dev_set_name(&svc->dev, "%s.%d", dev_name(&xd->dev), svc->id);
 
+		tb_service_debugfs_init(svc);
+
 		if (device_register(&svc->dev)) {
 			put_device(&svc->dev);
 			break;
include/linux/thunderbolt.h +4 -0
index 0a747f92847e..a844fd5d96ab 100644
@@ -350,6 +350,9 @@ void tb_unregister_protocol_handler(struct tb_protocol_handler *handler);
  * @prtcvers: Protocol version from the properties directory
  * @prtcrevs: Protocol software revision from the properties directory
  * @prtcstns: Protocol settings mask from the properties directory
+ * @debugfs_dir: Pointer to the service debugfs directory. Always created
+ *		 when debugfs is enabled. Can be used by service drivers to
+ *		 add their own entries under the service.
  *
  * Each domain exposes set of services it supports as collection of
  * properties. For each service there will be one corresponding
@@ -363,6 +366,7 @@ struct tb_service {
 	u32 prtcvers;
 	u32 prtcrevs;
 	u32 prtcstns;
+	struct dentry *debugfs_dir;
 };
 
 static inline struct tb_service *tb_service_get(struct tb_service *svc)
Signature · 1 line
2.28.0
#7 lore raw

PATCH v2 7/10 thunderbolt: Make it possible to allocate one directional DMA tunnel

In reply to Mika Westerberg

With DMA tunnels it is possible that the service using it does not require bi-directional paths so make RX and TX optional (but of course one of them needs to be set).

---
 drivers/thunderbolt/tunnel.c | 50 ++++++++++++++++++++++--------------
 1 file changed, 31 insertions(+), 19 deletions(-)
drivers/thunderbolt/tunnel.c +31 -19
index 829b6ccdd5d4..dcdf9c7a9cae 100644
@@ -34,9 +34,6 @@
 #define TB_DP_AUX_PATH_OUT		1
 #define TB_DP_AUX_PATH_IN		2
 
-#define TB_DMA_PATH_OUT			0
-#define TB_DMA_PATH_IN			1
-
 static const char * const tb_tunnel_names[] = { "PCI", "DP", "DMA", "USB3" };
 
 #define __TB_TUNNEL_PRINT(level, tunnel, fmt, arg...)                   \
@@ -829,10 +826,10 @@ static void tb_dma_init_path(struct tb_path *path, unsigned int isb,
  * @nhi: Host controller port
  * @dst: Destination null port which the other domain is connected to
  * @transmit_ring: NHI ring number used to send packets towards the
- *		   other domain
+ *		   other domain. Set to %0 if TX path is not needed.
  * @transmit_path: HopID used for transmitting packets
  * @receive_ring: NHI ring number used to receive packets from the
- *		  other domain
+ *		  other domain. Set to %0 if RX path is not needed.
  * @reveive_path: HopID used for receiving packets
  *
  * Return: Returns a tb_tunnel on success or NULL on failure.
@@ -843,10 +840,19 @@ struct tb_tunnel *tb_tunnel_alloc_dma(struct tb *tb, struct tb_port *nhi,
 				      int receive_path)
 {
 	struct tb_tunnel *tunnel;
+	size_t npaths = 0, i = 0;
 	struct tb_path *path;
 	u32 credits;
 
-	tunnel = tb_tunnel_alloc(tb, 2, TB_TUNNEL_DMA);
+	if (receive_ring)
+		npaths++;
+	if (transmit_ring)
+		npaths++;
+
+	if (WARN_ON(!npaths))
+		return NULL;
+
+	tunnel = tb_tunnel_alloc(tb, npaths, TB_TUNNEL_DMA);
 	if (!tunnel)
 		return NULL;
 
@@ -856,22 +862,28 @@ struct tb_tunnel *tb_tunnel_alloc_dma(struct tb *tb, struct tb_port *nhi,
 
 	credits = tb_dma_credits(nhi);
 
-	path = tb_path_alloc(tb, dst, receive_path, nhi, receive_ring, 0, "DMA RX");
-	if (!path) {
-		tb_tunnel_free(tunnel);
-		return NULL;
+	if (receive_ring) {
+		path = tb_path_alloc(tb, dst, receive_path, nhi, receive_ring, 0,
+				     "DMA RX");
+		if (!path) {
+			tb_tunnel_free(tunnel);
+			return NULL;
+		}
+		tb_dma_init_path(path, TB_PATH_NONE, TB_PATH_SOURCE | TB_PATH_INTERNAL,
+				 credits);
+		tunnel->paths[i++] = path;
 	}
-	tb_dma_init_path(path, TB_PATH_NONE, TB_PATH_SOURCE | TB_PATH_INTERNAL,
-			 credits);
-	tunnel->paths[TB_DMA_PATH_IN] = path;
 
-	path = tb_path_alloc(tb, nhi, transmit_ring, dst, transmit_path, 0, "DMA TX");
-	if (!path) {
-		tb_tunnel_free(tunnel);
-		return NULL;
+	if (transmit_ring) {
+		path = tb_path_alloc(tb, nhi, transmit_ring, dst, transmit_path, 0,
+				     "DMA TX");
+		if (!path) {
+			tb_tunnel_free(tunnel);
+			return NULL;
+		}
+		tb_dma_init_path(path, TB_PATH_SOURCE, TB_PATH_ALL, credits);
+		tunnel->paths[i++] = path;
 	}
-	tb_dma_init_path(path, TB_PATH_SOURCE, TB_PATH_ALL, credits);
-	tunnel->paths[TB_DMA_PATH_OUT] = path;
 
 	return tunnel;
 }
Signature · 1 line
2.28.0
#8 lore raw

PATCH v2 8/10 thunderbolt: Add support for end-to-end flow control

In reply to Mika Westerberg

USB4 spec defines end-to-end (E2E) flow control that can be used between hosts to prevent overflow of a RX ring. We previously had this partially implemented but that code was removed with commit 53f13319d131 ("thunderbolt: Get rid of E2E workaround") with the idea that we add it back properly if there ever is need. Now that we are going to add DMA traffic test driver (in subsequent patches) this can be useful.

For this reason we modify tb_ring_alloc_rx/tx() so that they accept RING_FLAG_E2E and configure the hardware ring accordingly. The RX side also requires passing TX HopID (e2e_tx_hop) used in the credit grant packets.

---
 drivers/net/thunderbolt.c   |  2 +-
 drivers/thunderbolt/ctl.c   |  4 ++--
 drivers/thunderbolt/nhi.c   | 36 ++++++++++++++++++++++++++++++++----
 include/linux/thunderbolt.h |  8 +++++++-
 4 files changed, 42 insertions(+), 8 deletions(-)
drivers/net/thunderbolt.c +1 -1
index 3160443ef3b9..d7b5f87eaa15 100644
@@ -866,7 +866,7 @@ static int tbnet_open(struct net_device *dev)
 	eof_mask = BIT(TBIP_PDF_FRAME_END);
 
 	ring = tb_ring_alloc_rx(xd->tb->nhi, -1, TBNET_RING_SIZE,
-				RING_FLAG_FRAME, sof_mask, eof_mask,
+				RING_FLAG_FRAME, 0, sof_mask, eof_mask,
 				tbnet_start_poll, net);
 	if (!ring) {
 		netdev_err(dev, "failed to allocate Rx ring\n");
drivers/thunderbolt/ctl.c +2 -2
index 9894b8f63064..1d86e27a0ef3 100644
@@ -628,8 +628,8 @@ struct tb_ctl *tb_ctl_alloc(struct tb_nhi *nhi, event_cb cb, void *cb_data)
 	if (!ctl->tx)
 		goto err;
 
-	ctl->rx = tb_ring_alloc_rx(nhi, 0, 10, RING_FLAG_NO_SUSPEND, 0xffff,
-				0xffff, NULL, NULL);
+	ctl->rx = tb_ring_alloc_rx(nhi, 0, 10, RING_FLAG_NO_SUSPEND, 0, 0xffff,
+				   0xffff, NULL, NULL);
 	if (!ctl->rx)
 		goto err;
 
drivers/thunderbolt/nhi.c +32 -4
index 3f79baa54829..a69bc6b49405 100644
@@ -483,7 +483,7 @@ static int nhi_alloc_hop(struct tb_nhi *nhi, struct tb_ring *ring)
 
 static struct tb_ring *tb_ring_alloc(struct tb_nhi *nhi, u32 hop, int size,
 				     bool transmit, unsigned int flags,
-				     u16 sof_mask, u16 eof_mask,
+				     int e2e_tx_hop, u16 sof_mask, u16 eof_mask,
 				     void (*start_poll)(void *),
 				     void *poll_data)
 {
@@ -506,6 +506,7 @@ static struct tb_ring *tb_ring_alloc(struct tb_nhi *nhi, u32 hop, int size,
 	ring->is_tx = transmit;
 	ring->size = size;
 	ring->flags = flags;
+	ring->e2e_tx_hop = e2e_tx_hop;
 	ring->sof_mask = sof_mask;
 	ring->eof_mask = eof_mask;
 	ring->head = 0;
@@ -550,7 +551,7 @@ static struct tb_ring *tb_ring_alloc(struct tb_nhi *nhi, u32 hop, int size,
 struct tb_ring *tb_ring_alloc_tx(struct tb_nhi *nhi, int hop, int size,
 				 unsigned int flags)
 {
-	return tb_ring_alloc(nhi, hop, size, true, flags, 0, 0, NULL, NULL);
+	return tb_ring_alloc(nhi, hop, size, true, flags, 0, 0, 0, NULL, NULL);
 }
 EXPORT_SYMBOL_GPL(tb_ring_alloc_tx);
 
@@ -560,6 +561,7 @@ EXPORT_SYMBOL_GPL(tb_ring_alloc_tx);
  * @hop: HopID (ring) to allocate. Pass %-1 for automatic allocation.
  * @size: Number of entries in the ring
  * @flags: Flags for the ring
+ * @e2e_tx_hop: Transmit HopID when E2E is enabled in @flags
  * @sof_mask: Mask of PDF values that start a frame
  * @eof_mask: Mask of PDF values that end a frame
  * @start_poll: If not %NULL the ring will call this function when an
@@ -568,10 +570,11 @@ EXPORT_SYMBOL_GPL(tb_ring_alloc_tx);
  * @poll_data: Optional data passed to @start_poll
  */
 struct tb_ring *tb_ring_alloc_rx(struct tb_nhi *nhi, int hop, int size,
-				 unsigned int flags, u16 sof_mask, u16 eof_mask,
+				 unsigned int flags, int e2e_tx_hop,
+				 u16 sof_mask, u16 eof_mask,
 				 void (*start_poll)(void *), void *poll_data)
 {
-	return tb_ring_alloc(nhi, hop, size, false, flags, sof_mask, eof_mask,
+	return tb_ring_alloc(nhi, hop, size, false, flags, e2e_tx_hop, sof_mask, eof_mask,
 			     start_poll, poll_data);
 }
 EXPORT_SYMBOL_GPL(tb_ring_alloc_rx);
@@ -618,6 +621,31 @@ void tb_ring_start(struct tb_ring *ring)
 		ring_iowrite32options(ring, sof_eof_mask, 4);
 		ring_iowrite32options(ring, flags, 0);
 	}
+
+	/*
+	 * Now that the ring valid bit is set we can configure E2E if
+	 * enabled for the ring.
+	 */
+	if (ring->flags & RING_FLAG_E2E) {
+		if (!ring->is_tx) {
+			u32 hop;
+
+			hop = ring->e2e_tx_hop << REG_RX_OPTIONS_E2E_HOP_SHIFT;
+			hop &= REG_RX_OPTIONS_E2E_HOP_MASK;
+			flags |= hop;
+
+			dev_dbg(&ring->nhi->pdev->dev,
+				"enabling E2E for %s %d with TX HopID %d\n",
+				RING_TYPE(ring), ring->hop, ring->e2e_tx_hop);
+		} else {
+			dev_dbg(&ring->nhi->pdev->dev, "enabling E2E for %s %d\n",
+				RING_TYPE(ring), ring->hop);
+		}
+
+		flags |= RING_FLAG_E2E_FLOW_CONTROL;
+		ring_iowrite32options(ring, flags, 0);
+	}
+
 	ring_interrupt_active(ring, true);
 	ring->running = true;
 err:
include/linux/thunderbolt.h +7 -1
index a844fd5d96ab..034dccf93955 100644
@@ -481,6 +481,8 @@ struct tb_nhi {
  * @irq: MSI-X irq number if the ring uses MSI-X. %0 otherwise.
  * @vector: MSI-X vector number the ring uses (only set if @irq is > 0)
  * @flags: Ring specific flags
+ * @e2e_tx_hop: Transmit HopID when E2E is enabled. Only applicable to
+ *		RX ring. For TX ring this should be set to %0.
  * @sof_mask: Bit mask used to detect start of frame PDF
  * @eof_mask: Bit mask used to detect end of frame PDF
  * @start_poll: Called when ring interrupt is triggered to start
@@ -504,6 +506,7 @@ struct tb_ring {
 	int irq;
 	u8 vector;
 	unsigned int flags;
+	int e2e_tx_hop;
 	u16 sof_mask;
 	u16 eof_mask;
 	void (*start_poll)(void *data);
@@ -514,6 +517,8 @@ struct tb_ring {
 #define RING_FLAG_NO_SUSPEND	BIT(0)
 /* Configure the ring to be in frame mode */
 #define RING_FLAG_FRAME		BIT(1)
+/* Enable end-to-end flow control */
+#define RING_FLAG_E2E		BIT(2)
 
 struct ring_frame;
 typedef void (*ring_cb)(struct tb_ring *, struct ring_frame *, bool canceled);
@@ -562,7 +567,8 @@ struct ring_frame {
 struct tb_ring *tb_ring_alloc_tx(struct tb_nhi *nhi, int hop, int size,
 				 unsigned int flags);
 struct tb_ring *tb_ring_alloc_rx(struct tb_nhi *nhi, int hop, int size,
-				 unsigned int flags, u16 sof_mask, u16 eof_mask,
+				 unsigned int flags, int e2e_tx_hop,
+				 u16 sof_mask, u16 eof_mask,
 				 void (*start_poll)(void *), void *poll_data);
 void tb_ring_start(struct tb_ring *ring);
 void tb_ring_stop(struct tb_ring *ring);
Signature · 1 line
2.28.0
#9 lore raw

PATCH v2 9/10 thunderbolt: Add DMA traffic test driver

In reply to Mika Westerberg

From: Isaac Hazan <[email protected]>

This driver allows sending DMA traffic over XDomain connection. Specifically over a loopback connection using either a Thunderbolt/USB4 cable that is connected back to the host router port, or a special loopback dongle that has RX and TX lines crossed. This can be useful at manufacturing floor to check whether Thunderbolt/USB4 ports are functional.

The driver exposes debugfs directory under the XDomain service that can be used to configure the driver, start the test and check the results.

If a loopback dongle is used the steps to send and receive 1000 packets can be done like:

  # modprobe thunderbolt_dma_test
  # echo 1000 > /sys/kernel/debug/thunderbolt/<service_id>/dma_test/packets_to_receive
  # echo 1000 > /sys/kernel/debug/thunderbolt/<service_id>/dma_test/packets_to_send
  # echo 1 > /sys/kernel/debug/thunderbolt/<service_id>/dma_test/test
  # cat /sys/kernel/debug/thunderbolt/<service_id>/dma_test/status

When a cable is connected back to host then there are two Thunderbolt services, one is configured for receiving (does not matter which one):

  # modprobe thunderbolt_dma_test
  # echo 1000 > /sys/kernel/debug/thunderbolt/<service_a>/dma_test/packets_to_receive
  # echo 1 > /sys/kernel/debug/thunderbolt/<service_a>/dma_test/test

The other one for sending:

  # echo 1000 > /sys/kernel/debug/thunderbolt/<service_b>/dma_test/packets_to_send
  # echo 1 > /sys/kernel/debug/thunderbolt/<service_b>/dma_test/test

Results can be read from both services status attributes.

---
 drivers/thunderbolt/Kconfig    |  13 +
 drivers/thunderbolt/Makefile   |   3 +
 drivers/thunderbolt/dma_test.c | 736 +++++++++++++++++++++++++++++++++
 3 files changed, 752 insertions(+)
 create mode 100644 drivers/thunderbolt/dma_test.c
drivers/thunderbolt/Kconfig +13 -0
index 7fc058f81d00..4bfec8a28064 100644
@@ -31,4 +31,17 @@ config USB4_KUNIT_TEST
 	bool "KUnit tests"
 	depends on KUNIT=y
 
+config USB4_DMA_TEST
+	tristate "DMA traffic test driver"
+	depends on DEBUG_FS
+	help
+	  This allows sending and receiving DMA traffic through loopback
+	  connection. Loopback connection can be done by either special
+	  dongle that has TX/RX lines crossed, or by simply connecting a
+	  cable back to the host. Only enable this if you know what you
+	  are doing. Normal users and distro kernels should say N here.
+
+	  To compile this driver a module, choose M here. The module will be
+	  called thunderbolt_dma_test.
+
 endif # USB4
drivers/thunderbolt/Makefile +3 -0
index 571537371072..7aa48f6c41d9 100644
@@ -7,3 +7,6 @@ thunderbolt-objs += nvm.o retimer.o quirks.o
 thunderbolt-${CONFIG_ACPI} += acpi.o
 thunderbolt-$(CONFIG_DEBUG_FS) += debugfs.o
 thunderbolt-${CONFIG_USB4_KUNIT_TEST} += test.o
+
+thunderbolt_dma_test-${CONFIG_USB4_DMA_TEST} += dma_test.o
+obj-$(CONFIG_USB4_DMA_TEST) += thunderbolt_dma_test.o
drivers/thunderbolt/dma_test.c new file +736 -0

736 lines not on this page · raw message

Signature · 1 line
2.28.0
#10 lore raw

PATCH v2 10/10 MAINTAINERS: Add Isaac as maintainer of Thunderbolt DMA traffic test driver

In reply to Mika Westerberg

From: Isaac Hazan <[email protected]>

I will be maintaining the Thunderbolt DMA traffic test driver.

---
 MAINTAINERS | 6 ++++++
 1 file changed, 6 insertions(+)
MAINTAINERS +6 -0
index 3da6d8c154e4..83c4c66f8188 100644
@@ -17378,6 +17378,12 @@ W:	http://thinkwiki.org/wiki/Ibm-acpi
 T:	git git://repo.or.cz/linux-2.6/linux-acpi-2.6/ibm-acpi-2.6.git
 F:	drivers/platform/x86/thinkpad_acpi.c
 
+THUNDERBOLT DMA TRAFFIC TEST DRIVER
+M:	Isaac Hazan <[email protected]>
+L:	[email protected]
+S:	Maintained
+F:	drivers/thunderbolt/dma_test.c
+
 THUNDERBOLT DRIVER
 M:	Andreas Noever <[email protected]>
 M:	Michael Jamet <[email protected]>
Signature · 1 line
2.28.0
maintainer Reviewed-by on the v2 series
#11 lore raw
In reply to Mika Westerberg
Quoted 18 lines · Mika Westerberg

Hi all,

This series adds a new Thunderbolt service driver that can be used on manufacturing floor to test that each Thunderbolt/USB4 port is functional. It can be done either using a special loopback dongle that has RX and TX lanes crossed, or by connecting a cable back to the host (for those who don't have these dongles).

This takes advantage of the existing XDomain protocol and creates XDomain devices for the loops back to the host where the DMA traffic test driver can bind to.

The DMA traffic test driver creates a tunnel through the fabric and then sends and receives data frames over the tunnel checking for different errors.

The previous version can be found here:

Quoted 7 lines

https://lore.kernel.org/linux-usb/[email protected]/

Changes from the previous version:

* Fix resource leak in tb_xdp_handle_request() (patch 2/10)
* Use debugfs_remove_recursive() in tb_service_debugfs_remove() (patch 6/10)
* Add tags from Yehezkel

author Applied
#12 lore raw
Quoted 21 lines · Greg Kroah-Hartman
Quoted 20 lines · Mika Westerberg

Hi all,

This series adds a new Thunderbolt service driver that can be used on manufacturing floor to test that each Thunderbolt/USB4 port is functional. It can be done either using a special loopback dongle that has RX and TX lanes crossed, or by connecting a cable back to the host (for those who don't have these dongles).

This takes advantage of the existing XDomain protocol and creates XDomain devices for the loops back to the host where the DMA traffic test driver can bind to.

The DMA traffic test driver creates a tunnel through the fabric and then sends and receives data frames over the tunnel checking for different errors.

The previous version can be found here:

https://lore.kernel.org/linux-usb/[email protected]/

Quoted 7 lines
Quoted 5 lines

Changes from the previous version:

* Fix resource leak in tb_xdp_handle_request() (patch 2/10)
* Use debugfs_remove_recursive() in tb_service_debugfs_remove() (patch 6/10)
* Add tags from Yehezkel

Thanks!

Applied the series to thunderbolt.git/next.