Skip to contentSkip to conversation

thunderbolt: USB4STREAM improvements

PATCH In review Mika Westerberg 2 participants 9 messages Jul 28 – Aug 5

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover thunderbolt: USB4STREAM improvements

Hi all,

This series has a couple of non-critical fixes for USB4STREAM driver and then makes it work better with io_uring. In addition to that we add a busy polling mode that can be used by applications that need lower latency than what is possible with the interrupt mode and worker scheduling.

Mika Westerberg (5):
thunderbolt: stream: Restore consumer if copying from iter fails
thunderbolt: stream: Fix possible short reads/writes
thunderbolt: stream: Support IOCB_NOWAIT in non-blocking I/O as well
thunderbolt: Make interrupt optional for rings
thunderbolt: stream: Add support for busy polling

 .../ABI/testing/configfs-thunderbolt_stream   |  15 +
 drivers/thunderbolt/nhi.c                     |  27 +-
 drivers/thunderbolt/stream.c                  | 256 ++++++++++++++----
 include/linux/thunderbolt.h                   |   2 +
 4 files changed, 243 insertions(+), 57 deletions(-)
Signature · 1 line
2.50.1
#1 lore raw

PATCH 1/5 thunderbolt: stream: Restore consumer if copying from iter fails

In tbstream_dev_alloc_tx() if copying data from iterator fails we leave the consumer pointer as is wasting one entry in the ring. Fix this by restoring the consumer back in case of failure.

Fixes 6db21d817b43 ("thunderbolt: Add support for USB4STREAM")
---
 drivers/thunderbolt/stream.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
drivers/thunderbolt/stream.c +3 -1
index c1f5c55583d0..279a0cfc2ac8 100644
@@ -512,8 +512,10 @@ tbstream_dev_alloc_tx(struct tbstream_dev *sdev, enum tbstream_frame_pdf pdf,
 	dma_sync_single_for_cpu(dma_dev, sf->frame.buffer_phy, size,
 				DMA_TO_DEVICE);
 	if (pdf == TBSTREAM_DATA) {
-		if (copy_page_from_iter(sf->page, 0, size, from) != size)
+		if (copy_page_from_iter(sf->page, 0, size, from) != size) {
+			sdev->tx_ring.cons--;
 			return ERR_PTR(-EFAULT);
+		}
 	} else {
 		memset(page_address(sf->page), 0, size);
 	}
Signature · 1 line
2.50.1
#2 lore raw

PATCH 2/5 thunderbolt: stream: Fix possible short reads/writes

In reply to Mika Westerberg

Since copy_page_{to|from}_iter() advances the iterator and makes iov_iter_count() reflect the remaining bytes, subtracting nbytes from it makes it count it twice resulting in possible short reads/writes on a read/write spanning multiple frames.

Fix this by using iov_iter_count() directly.

Fixes 6db21d817b43 ("thunderbolt: Add support for USB4STREAM")
---
 drivers/thunderbolt/stream.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
drivers/thunderbolt/stream.c +4 -4
index 279a0cfc2ac8..20e6ad586879 100644
@@ -673,7 +673,7 @@ tbstream_dev_fops_read_iter(struct kiocb *kiocb, struct iov_iter *to)
 	}
 
 	nbytes = 0;
-	while (nbytes < iov_iter_count(to)) {
+	while (iov_iter_count(to)) {
 		struct tbstream_frame *sf;
 		size_t size, sf_size;
 
@@ -695,7 +695,7 @@ tbstream_dev_fops_read_iter(struct kiocb *kiocb, struct iov_iter *to)
 		}
 
 		sf_size = tb_ring_frame_size(&sf->frame);
-		size = min(iov_iter_count(to) - nbytes, sf_size);
+		size = min(iov_iter_count(to), sf_size);
 
 		if (copy_page_to_iter(sf->page, sf->offset, size, to) != size) {
 			ret = -EFAULT;
@@ -765,10 +765,10 @@ tbstream_dev_fops_write_iter(struct kiocb *kiocb, struct iov_iter *from)
 	}
 
 	nbytes = 0;
-	while (nbytes < iov_iter_count(from)) {
+	while (iov_iter_count(from)) {
 		size_t size;
 
-		size = min(iov_iter_count(from) - nbytes, TB_MAX_FRAME_SIZE);
+		size = min(iov_iter_count(from), TB_MAX_FRAME_SIZE);
 		ret = tbstream_dev_send_data(sdev, from, size);
 		if (ret) {
 			/*
Signature · 1 line
2.50.1
#3 lore raw

PATCH 3/5 thunderbolt: stream: Support IOCB_NOWAIT in non-blocking I/O as well

In reply to Mika Westerberg

For read_iter/write_iter() it is also possible to pass IOCB_NOWAIT with the kiocb to indicate non-blocking read/write. For instance io_uring does this. So take this into account on read and write paths.

---
 drivers/thunderbolt/stream.c | 40 +++++++++++++++++++++++++++---------
 1 file changed, 30 insertions(+), 10 deletions(-)
drivers/thunderbolt/stream.c +30 -10
index 20e6ad586879..06c8620708b1 100644
@@ -633,10 +633,23 @@ static void tbstream_dev_stop(struct tbstream_dev *sdev)
 	sdev->tx_ring.ring = NULL;
 }
 
+static int tbstream_dev_lock(struct tbstream_dev *sdev, bool nowait)
+{
+	if (nowait) {
+		if (!mutex_trylock(&sdev->lock))
+			return -EAGAIN;
+	} else {
+		if (mutex_lock_interruptible(&sdev->lock))
+			return -ERESTARTSYS;
+	}
+	return 0;
+}
+
 static ssize_t
 tbstream_dev_fops_read_iter(struct kiocb *kiocb, struct iov_iter *to)
 {
 	struct file *file = kiocb->ki_filp;
+	bool nowait = file->f_flags & O_NONBLOCK || kiocb->ki_flags & IOCB_NOWAIT;
 	struct tbstream_dev *sdev = to_tbstream_dev(file->private_data);
 	size_t nbytes;
 	int ret;
@@ -645,14 +658,16 @@ tbstream_dev_fops_read_iter(struct kiocb *kiocb, struct iov_iter *to)
 	if (ret)
 		return ret;
 
-	if (mutex_lock_interruptible(&sdev->lock))
-		return -ERESTARTSYS;
+	ret = tbstream_dev_lock(sdev, nowait);
+	if (ret)
+		return ret;
 
 	while (!tbstream_ring_available(&sdev->rx_ring)) {
 		mutex_unlock(&sdev->lock);
 
-		if (file->f_flags & O_NONBLOCK)
+		if (nowait)
 			return -EAGAIN;
+
 		ret = wait_event_interruptible(sdev->wait,
 				tbstream_ring_available(&sdev->rx_ring) ||
 				tbstream_dev_valid(sdev) != 0 ||
@@ -668,8 +683,9 @@ tbstream_dev_fops_read_iter(struct kiocb *kiocb, struct iov_iter *to)
 		if (tbstream_dev_closed(sdev) || tbstream_dev_removed(sdev))
 			return 0;
 
-		if (mutex_lock_interruptible(&sdev->lock))
-			return -ERESTARTSYS;
+		ret = tbstream_dev_lock(sdev, nowait);
+		if (ret)
+			return ret;
 	}
 
 	nbytes = 0;
@@ -729,6 +745,7 @@ static ssize_t
 tbstream_dev_fops_write_iter(struct kiocb *kiocb, struct iov_iter *from)
 {
 	struct file *file = kiocb->ki_filp;
+	bool nowait = file->f_flags & O_NONBLOCK || kiocb->ki_flags & IOCB_NOWAIT;
 	struct tbstream_dev *sdev = to_tbstream_dev(file->private_data);
 	size_t nbytes;
 	int ret;
@@ -737,14 +754,16 @@ tbstream_dev_fops_write_iter(struct kiocb *kiocb, struct iov_iter *from)
 	if (ret)
 		return ret;
 
-	if (mutex_lock_interruptible(&sdev->lock))
-		return -ERESTARTSYS;
+	ret = tbstream_dev_lock(sdev, nowait);
+	if (ret)
+		return ret;
 
 	while (!tbstream_ring_available(&sdev->tx_ring)) {
 		mutex_unlock(&sdev->lock);
 
-		if (file->f_flags & O_NONBLOCK)
+		if (nowait)
 			return -EAGAIN;
+
 		ret = wait_event_interruptible(sdev->wait,
 				tbstream_ring_available(&sdev->tx_ring) ||
 				tbstream_dev_valid(sdev) != 0 ||
@@ -760,8 +779,9 @@ tbstream_dev_fops_write_iter(struct kiocb *kiocb, struct iov_iter *from)
 		if (tbstream_dev_closed(sdev) || tbstream_dev_removed(sdev))
 			return -ENXIO;
 
-		if (mutex_lock_interruptible(&sdev->lock))
-			return -ERESTARTSYS;
+		ret = tbstream_dev_lock(sdev, nowait);
+		if (ret)
+			return ret;
 	}
 
 	nbytes = 0;
Signature · 1 line
2.50.1
#4 lore raw

PATCH 4/5 thunderbolt: Make interrupt optional for rings

In reply to Mika Westerberg

For some use-cases it does make sense to poll the rings directly instead of relying on the interrupt. For this reason add a new flag RING_FLAG_NO_INTERRUPT that can be used to allocate ring in polled mode.

Assisted-by Claude:claude-opus-4-8
---
 drivers/thunderbolt/nhi.c   | 27 +++++++++++++++++++++------
 include/linux/thunderbolt.h |  2 ++
 2 files changed, 23 insertions(+), 6 deletions(-)
drivers/thunderbolt/nhi.c +21 -6
index 35e3c119d5ee..a4816db5cacd 100644
@@ -235,6 +235,12 @@ static void ring_write_descriptors(struct tb_ring *ring)
 {
 	struct ring_frame *frame, *n;
 	struct ring_desc *descriptor;
+	u32 flags;
+
+	flags = RING_DESC_POSTED;
+	if (!(ring->flags & RING_FLAG_NO_INTERRUPT))
+		flags |= RING_DESC_INTERRUPT;
+
 	list_for_each_entry_safe(frame, n, &ring->queue, list) {
 		if (ring_full(ring))
 			break;
@@ -242,7 +248,7 @@ static void ring_write_descriptors(struct tb_ring *ring)
 		descriptor = &ring->descriptors[ring->head];
 		descriptor->phys = frame->buffer_phy;
 		descriptor->time = 0;
-		descriptor->flags = RING_DESC_POSTED | RING_DESC_INTERRUPT;
+		descriptor->flags = flags;
 		if (ring->is_tx) {
 			descriptor->length = frame->size;
 			descriptor->eof = frame->eof;
@@ -339,8 +345,9 @@ EXPORT_SYMBOL_GPL(__tb_ring_enqueue);
  * @ring: Ring to poll
  *
  * This function can be called when @start_poll callback of the @ring
- * has been called. It will read one completed frame from the ring and
- * return it to the caller.
+ * has been called or the ring is created with %RING_FLAG_NO_INTERRUPT.
+ * It will read one completed frame from the ring and return it to the
+ * caller.
  *
  * Return: Pointer to &struct ring_frame, %NULL if there is no more
  * completed frames.
@@ -538,6 +545,12 @@ static struct tb_ring *tb_ring_alloc(struct tb_nhi *nhi, u32 hop, int size,
 	dev_dbg(nhi->dev, "allocating %s ring %d of size %d\n",
 		transmit ? "TX" : "RX", hop, size);
 
+	if ((flags & RING_FLAG_NO_INTERRUPT) && start_poll) {
+		dev_WARN(nhi->dev,
+			 "start_poll() and NO_INTERRUPT cannot be used at the same time\n");
+		return NULL;
+	}
+
 	ring = kzalloc_obj(*ring);
 	if (!ring)
 		return NULL;
@@ -568,7 +581,7 @@ static struct tb_ring *tb_ring_alloc(struct tb_nhi *nhi, u32 hop, int size,
 	if (!ring->descriptors)
 		goto err_free_ring;
 
-	if (nhi->ops->request_ring_irq) {
+	if (!(flags & RING_FLAG_NO_INTERRUPT) && nhi->ops->request_ring_irq) {
 		if (nhi->ops->request_ring_irq(ring, flags & RING_FLAG_NO_SUSPEND))
 			goto err_free_descs;
 	}
@@ -701,7 +714,8 @@ void tb_ring_start(struct tb_ring *ring)
 		ring_iowrite32options(ring, flags, 0);
 	}
 
-	ring_interrupt_active(ring, true);
+	if (!(ring->flags & RING_FLAG_NO_INTERRUPT))
+		ring_interrupt_active(ring, true);
 	ring->running = true;
 err:
 	spin_unlock(&ring->lock);
@@ -761,7 +775,8 @@ void tb_ring_stop(struct tb_ring *ring)
 			 RING_TYPE(ring), ring->hop);
 		goto err;
 	}
-	ring_interrupt_active(ring, false);
+	if (!(ring->flags & RING_FLAG_NO_INTERRUPT))
+		ring_interrupt_active(ring, false);
 
 	ring_iowrite32options(ring, 0, 0);
 	ring_iowrite64desc(ring, 0, 0);
include/linux/thunderbolt.h +2 -0
index feb1af175cfd..42bbfc007324 100644
@@ -592,6 +592,8 @@ struct tb_ring {
 #define RING_FLAG_FRAME		BIT(1)
 /* Enable end-to-end flow control */
 #define RING_FLAG_E2E		BIT(2)
+/* Do not enable interrupt for the ring */
+#define RING_FLAG_NO_INTERRUPT	BIT(3)
 
 struct ring_frame;
 typedef void (*ring_cb)(struct tb_ring *, struct ring_frame *, bool canceled);
Signature · 1 line
2.50.1
#5 lore raw

PATCH 5/5 thunderbolt: stream: Add support for busy polling

In reply to Mika Westerberg

Using interrupts and scheduling workers increase latency so latency critical applications may want to avoid that. Make this possible in USB4STREAM by adding a new ConfigFS attribute: busy_poll that, when activated switches the rings to polling mode. The cost for lower latency is that this burns more CPU cycles and things like poll(2) cannot be used.

Assisted-by Claude:claude-opus-4-8
---
 .../ABI/testing/configfs-thunderbolt_stream   |  15 ++
 drivers/thunderbolt/stream.c                  | 212 ++++++++++++++----
 2 files changed, 187 insertions(+), 40 deletions(-)
Documentation/ABI/testing/configfs-thunderbolt_stream +15 -0
index 7abc6b73a1e4..cbecb3d8db50 100644
@@ -27,6 +27,21 @@ Description:
 		default values. If there is an advertised remote stream
 		with the same name, uses its values as the default.
 
+What:		/sys/kernel/config/thunderbolt/stream/<xdomain>.<service>/$name/busy_poll
+Date:		Nov 2026
+KernelVersion:	v7.3
+Contact:	Mika Westerberg <[email protected]>
+Description:
+		Instead of using interrupts for completing the frames in
+		the TX/RX rings, busy poll them directly from the
+		read(2) and write(2) calls. This burns more CPU cycles
+		but provides lower latency for applications that need it.
+
+		This also makes poll(2) return EPOLLERR because
+		interrupts do not provide wakeup anymore. Likewise a
+		blocking read(2) without available data busy-spins until
+		data arrives or a signal is received.
+
 What:		/sys/kernel/config/thunderbolt/stream/<xdomain>.<service>/$name/index
 Date:		Sep 2026
 KernelVersion:	v7.2
drivers/thunderbolt/stream.c +172 -40

316 lines not on this page · raw message

Signature · 1 line
2.50.1
#6 lore raw

Re: [PATCH 5/5] thunderbolt: stream: Add support for busy polling

Quoted 6 lines · Mika Westerberg

Using interrupts and scheduling workers increase latency so latency critical applications may want to avoid that. Make this possible in USB4STREAM by adding a new ConfigFS attribute: busy_poll that, when activated switches the rings to polling mode. The cost for lower latency is that this burns more CPU cycles and things like poll(2) cannot be used.

Hm, would it be possible to auto-detect whether polling is more performant for the current workload and automatically enable and disable it? E.g. by counting the number of interrupts in a given interval and enabling/disabling polling based on a high water mark / low water mark? Those marks could then be finetuned from user space, but moving in and out of polling automatically would be more desirable I think. Forcing polling could be achieved by setting the water marks to a special value, e.g. 0 or UINT_MAX.

Thanks,

Lukas

#7 lore raw

Re: [PATCH 5/5] thunderbolt: stream: Add support for busy polling

Hi,

Quoted 8 lines · Lukas Wunner
Quoted 6 lines · Mika Westerberg

Using interrupts and scheduling workers increase latency so latency critical applications may want to avoid that. Make this possible in USB4STREAM by adding a new ConfigFS attribute: busy_poll that, when activated switches the rings to polling mode. The cost for lower latency is that this burns more CPU cycles and things like poll(2) cannot be used.

Quoted 8 lines

Hm, would it be possible to auto-detect whether polling is more performant for the current workload and automatically enable and disable it? E.g. by counting the number of interrupts in a given interval and enabling/disabling polling based on a high water mark / low water mark? Those marks could then be finetuned from user space, but moving in and out of polling automatically would be more desirable I think. Forcing polling could be achieved by setting the water marks to a special value, e.g. 0 or UINT_MAX.

I think the applications that want the minimal latency and end up setting this will put it to the "minimal" anyway (because they don't tolerate any additional latencies caused by interrupts and the worker scheduling) and for the rest they can already tune this by setting non-default values to "throttling" attribute (eg. 0 to avoid it completely, makes latency lower while still keeping it from hogging the CPU fully).

We can introduce auto-tuning mode afterwards as you suggest if it turns out to be wanted feature.

#8 lore raw
In reply to Mika Westerberg
Quoted 5 lines · Mika Westerberg

thunderbolt: stream: Restore consumer if copying from iter fails
thunderbolt: stream: Fix possible short reads/writes
thunderbolt: stream: Support IOCB_NOWAIT in non-blocking I/O as well
thunderbolt: Make interrupt optional for rings
thunderbolt: stream: Add support for busy polling

All applied to thunderbolt.git/next.