Skip to contentSkip to conversation

spi: dw: use threaded interrupt

PATCH Superseded Jisheng Zhang 2 participants 6 messages Aug 3 – Aug 7 SPI

v2 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

v2 cover spi: dw: use threaded interrupt

To avoid blocking for an excessive amount of time, eventually impacting on system responsiveness, hard interrupt handlers should finish executing in as little time as possible.

Use threaded interrupt and move the SPI transfer handling to an interrupt thread.

After that, since the dw_reader() and dw_writer() are called in threaded ISR now, so we can delay the unmasking interrupts until no rx and tx action is taken, thus reduce the interrupt numbers further.

patch1 and patch2 are small clean ups
patch3 does the conversion and optimization.

Here are the performance numbers:

Tested with below two cmds
./spidev_test -D /dev/spidev1.3 -s 30000000 -S 327680 -I 1

./spidev_test -D /dev/spidev1.3 -s 30000000 -S 327680 -I 1000 ./rtla timerlat top -q -k -P f:95

The first cmd is to check the interrupt numbers optmizaion result, the 2nd cmd group is to check the threaded interrupt improvement.

Before the patch:
each 320KB spi spidev_test transfer triggers 33118 interrupts

spidev_test reports ~22090 kbps
and rtla reports:
Timer Latency

  0 00:00:37   |          IRQ Timer Latency (us)        |         Thread Timer Latency (us)
CPU COUNT      |      cur       min       avg       max |      cur       min       avg       max
  0 #9958      |        1         0        67    103394 |        6         4      2198    105031
  1 #36902     |        1         0         1        18 |        5         4         5        29

After the patch:
each 320KB spi spidev_test transfer only triggers 1 interrupts spidev_test reports ~23520 kbps
and now rtla reports:
Timer Latency

  0 00:00:58   |          IRQ Timer Latency (us)        |         Thread Timer Latency (us)
CPU COUNT      |      cur       min       avg       max |      cur       min       avg       max
  0 #58362     |        1         0         0        29 |        6         3         4        56
  1 #58363     |        1         0         1        23 |        6         4         5        68

In summary:
before the patch after the patch
33118 interrutps 1 interrupts reduced by 33117 times!
103394 us max latency 29 us max latency reduced by 3564 times!
22090 kbps rate 23520 kbps rate improved by 6.5%

Since v1:
- rebase against latest version
- drop two patches which have been merged
- correct some performance numbers
- don't move request irq code block so no changes for err handling code path
- move spi_finalize_current_transfer to the end of threaded irq fn
- don't rely on irq status in threaded fn, but try rx and tx as much as possible in the loop

Jisheng Zhang (3):
spi: dw: use DW_SPI_ISR directly
spi: dw: use DW_SPI_INT_MASK instead of hardcoded 0xff
spi: dw: use threaded interrupt and optimize the threaded ISR

 drivers/spi/spi-dw-core.c | 96 ++++++++++++++++++++++++---------------
 1 file changed, 60 insertions(+), 36 deletions(-)
Signature · 1 line
2.53.0
#1 lore raw

PATCH v2 1/3 spi: dw: use DW_SPI_ISR directly

The DW_SPI_ISR register reports the masked interrupts, no need to mask again.

---
 drivers/spi/spi-dw-core.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/spi/spi-dw-core.c +1 -1
index ad8aad958c76..f0d5528a0e8a 100644
@@ -252,7 +252,7 @@ static irqreturn_t dw_spi_irq(int irq, void *dev_id)
 {
 	struct spi_controller *ctlr = dev_id;
 	struct dw_spi *dws = spi_controller_get_devdata(ctlr);
-	u16 irq_status = dw_readl(dws, DW_SPI_ISR) & DW_SPI_INT_MASK;
+	u16 irq_status = dw_readl(dws, DW_SPI_ISR);
 
 	if (!irq_status)
 		return IRQ_NONE;
Signature · 1 line
2.53.0
#2 lore raw

PATCH v2 2/3 spi: dw: use DW_SPI_INT_MASK instead of hardcoded 0xff

In reply to Jisheng Zhang

The Interrupt Mask Register valid bits is bit[5:0] which is well defined with DW_SPI_INT_MASK, use it instead of the incorrect(but no harm) and hardcoded 0xff.

---
 drivers/spi/spi-dw-core.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
drivers/spi/spi-dw-core.c +4 -4
index f0d5528a0e8a..460e77b34ed9 100644
@@ -228,7 +228,7 @@ static irqreturn_t dw_spi_transfer_handler(struct dw_spi *dws)
 	 */
 	dw_reader(dws);
 	if (!dws->rx_len) {
-		dw_spi_mask_intr(dws, 0xff);
+		dw_spi_mask_intr(dws, DW_SPI_INT_MASK);
 		spi_finalize_current_transfer(dws->ctlr);
 	} else if (dws->rx_len <= dw_readl(dws, DW_SPI_RXFTLR)) {
 		dw_writel(dws, DW_SPI_RXFTLR, dws->rx_len - 1);
@@ -258,7 +258,7 @@ static irqreturn_t dw_spi_irq(int irq, void *dev_id)
 		return IRQ_NONE;
 
 	if (!ctlr->cur_msg) {
-		dw_spi_mask_intr(dws, 0xff);
+		dw_spi_mask_intr(dws, DW_SPI_INT_MASK);
 		return IRQ_HANDLED;
 	}
 
@@ -445,7 +445,7 @@ static int dw_spi_transfer_one(struct spi_controller *ctlr,
 	dws->dma_mapped = spi_xfer_is_dma_mapped(ctlr, spi, transfer);
 
 	/* For poll mode just disable all interrupts */
-	dw_spi_mask_intr(dws, 0xff);
+	dw_spi_mask_intr(dws, DW_SPI_INT_MASK);
 
 	if (dws->dma_mapped) {
 		ret = dws->dma_ops->dma_setup(dws, transfer);
@@ -706,7 +706,7 @@ static int dw_spi_exec_mem_op(struct spi_mem *mem, const struct spi_mem_op *op)
 
 	dw_spi_update_config(dws, mem->spi, &cfg);
 
-	dw_spi_mask_intr(dws, 0xff);
+	dw_spi_mask_intr(dws, DW_SPI_INT_MASK);
 
 	dw_spi_enable_chip(dws, 1);
 
Signature · 1 line
2.53.0
#3 lore raw

PATCH v2 3/3 spi: dw: use threaded interrupt and optimize the threaded ISR

In reply to Jisheng Zhang

To avoid blocking for an excessive amount of time, eventually impacting on system responsiveness, hard interrupt handlers should finish executing in as little time as possible.

Use threaded interrupt and move the SPI transfer handling to an interrupt thread.

After that, since the dw_reader() and dw_writer() are called in threaded ISR now, so we can delay the unmasking interrupts until no rx and tx action is taken, thus reduce the interrupt numbers further.

Tested with below two cmds
./spidev_test -D /dev/spidev1.3 -s 30000000 -S 327680 -I 1

./spidev_test -D /dev/spidev1.3 -s 30000000 -S 327680 -I 1000 ./rtla timerlat top -q -k -P f:95

The first cmd is to check the interrupt numbers optmizaion result, the 2nd cmd group is to check the threaded interrupt improvement.

Before the patch:
each 320KB spi spidev_test transfer triggers 33118 interrupts

spidev_test reports ~22090 kbps
and rtla reports:
Timer Latency

  0 00:00:37   |          IRQ Timer Latency (us)        |         Thread Timer Latency (us)
CPU COUNT      |      cur       min       avg       max |      cur       min       avg       max
  0 #9958      |        1         0        67    103394 |        6         4      2198    105031
  1 #36902     |        1         0         1        18 |        5         4         5        29

After the patch:
each 320KB spi spidev_test transfer only triggers 1 interrupts spidev_test reports ~23520 kbps
and now rtla reports:
Timer Latency

  0 00:00:58   |          IRQ Timer Latency (us)        |         Thread Timer Latency (us)
CPU COUNT      |      cur       min       avg       max |      cur       min       avg       max
  0 #58362     |        1         0         0        29 |        6         3         4        56
  1 #58363     |        1         0         1        23 |        6         4         5        68

In summary:
before the patch after the patch
33118 interrutps 1 interrupts reduced by 33117 times!
103394 us max latency 29 us max latency reduced by 3564 times!
22090 kbps rate 23520 kbps rate improved by 6.5%

---
 drivers/spi/spi-dw-core.c | 88 +++++++++++++++++++++++++--------------
 1 file changed, 56 insertions(+), 32 deletions(-)
drivers/spi/spi-dw-core.c +56 -32
index 460e77b34ed9..a5ca893bd118 100644
@@ -132,10 +132,11 @@ static inline u32 dw_spi_rx_max(struct dw_spi *dws)
 	return min_t(u32, dws->rx_len, dw_readl(dws, DW_SPI_RXFLR));
 }
 
-static void dw_writer(struct dw_spi *dws)
+static u32 dw_writer(struct dw_spi *dws)
 {
 	u32 max = dw_spi_tx_max(dws);
 	u32 txw = 0;
+	u32 tx = 0;
 
 	while (max--) {
 		if (dws->tx) {
@@ -150,13 +151,16 @@ static void dw_writer(struct dw_spi *dws)
 		}
 		dw_write_io_reg(dws, DW_SPI_DR, txw);
 		--dws->tx_len;
+		++tx;
 	}
+	return tx;
 }
 
-static void dw_reader(struct dw_spi *dws)
+static u32 dw_reader(struct dw_spi *dws)
 {
 	u32 max = dw_spi_rx_max(dws);
 	u32 rxw;
+	u32 rx = 0;
 
 	while (max--) {
 		rxw = dw_read_io_reg(dws, DW_SPI_DR);
@@ -171,7 +175,9 @@ static void dw_reader(struct dw_spi *dws)
 			dws->rx += dws->n_bytes;
 		}
 		--dws->rx_len;
+		++rx;
 	}
+	return rx;
 }
 
 int dw_spi_check_status(struct dw_spi *dws, bool raw)
@@ -210,42 +216,60 @@ int dw_spi_check_status(struct dw_spi *dws, bool raw)
 }
 EXPORT_SYMBOL_NS_GPL(dw_spi_check_status, "SPI_DW_CORE");
 
-static irqreturn_t dw_spi_transfer_handler(struct dw_spi *dws)
+static irqreturn_t dw_spi_irq_thread_fn(int irq, void *dev_id)
 {
-	u16 irq_status = dw_readl(dws, DW_SPI_ISR);
+	struct spi_controller *ctlr = dev_id;
+	struct dw_spi *dws = spi_controller_get_devdata(ctlr);
+	u32 rx, tx, imask, mask = 0;
+	bool finalize = false;
 
-	if (dw_spi_check_status(dws, false)) {
+	do {
+		/*
+		 * Read data from the Rx FIFO every time we've got a chance executing
+		 * this method. If there is nothing left to receive, terminate the
+		 * procedure. Otherwise adjust the Rx FIFO Threshold level if it's a
+		 * final stage of the transfer. By doing so we'll get the next IRQ
+		 * right when the leftover incoming data is received.
+		 */
+		rx = dw_reader(dws);
+		if (!dws->rx_len) {
+			mask |= DW_SPI_INT_MASK;
+			finalize = true;
+		} else if (dws->rx_len <= dw_readl(dws, DW_SPI_RXFTLR)) {
+			dw_writel(dws, DW_SPI_RXFTLR, dws->rx_len - 1);
+		}
+
+		/*
+		 * Send data out as much as possible. The Tx FIFO Empty IRQ will be
+		 * disabled after the data transmission is finished so not to
+		 * have the TXE IRQ flood at the final stage of the transfer.
+		 */
+		tx = dw_writer(dws);
+		if (!dws->tx_len)
+			mask |= DW_SPI_INT_TXEI;
+	} while (rx != 0 || tx != 0);
+
+	imask = DW_SPI_INT_TXEI | DW_SPI_INT_TXOI |
+		DW_SPI_INT_RXUI | DW_SPI_INT_RXOI | DW_SPI_INT_RXFI;
+	imask &= ~mask;
+	dw_spi_umask_intr(dws, imask);
+
+	if (finalize)
 		spi_finalize_current_transfer(dws->ctlr);
-		return IRQ_HANDLED;
-	}
 
-	/*
-	 * Read data from the Rx FIFO every time we've got a chance executing
-	 * this method. If there is nothing left to receive, terminate the
-	 * procedure. Otherwise adjust the Rx FIFO Threshold level if it's a
-	 * final stage of the transfer. By doing so we'll get the next IRQ
-	 * right when the leftover incoming data is received.
-	 */
-	dw_reader(dws);
-	if (!dws->rx_len) {
-		dw_spi_mask_intr(dws, DW_SPI_INT_MASK);
+	return IRQ_HANDLED;
+}
+
+static irqreturn_t dw_spi_transfer_handler(struct dw_spi *dws)
+{
+	if (dw_spi_check_status(dws, false)) {
 		spi_finalize_current_transfer(dws->ctlr);
-	} else if (dws->rx_len <= dw_readl(dws, DW_SPI_RXFTLR)) {
-		dw_writel(dws, DW_SPI_RXFTLR, dws->rx_len - 1);
+		return IRQ_HANDLED;
 	}
 
-	/*
-	 * Send data out if Tx FIFO Empty IRQ is received. The IRQ will be
-	 * disabled after the data transmission is finished so not to
-	 * have the TXE IRQ flood at the final stage of the transfer.
-	 */
-	if (irq_status & DW_SPI_INT_TXEI) {
-		dw_writer(dws);
-		if (!dws->tx_len)
-			dw_spi_mask_intr(dws, DW_SPI_INT_TXEI);
-	}
+	dw_spi_mask_intr(dws, DW_SPI_INT_MASK);
 
-	return IRQ_HANDLED;
+	return IRQ_WAKE_THREAD;
 }
 
 static irqreturn_t dw_spi_irq(int irq, void *dev_id)
@@ -946,8 +970,8 @@ int dw_spi_add_controller(struct device *dev, struct dw_spi *dws)
 	/* Basic HW init */
 	dw_spi_hw_init(dev, dws);
 
-	ret = request_irq(dws->irq, dw_spi_irq, IRQF_SHARED, dev_name(dev),
-			  ctlr);
+	ret = request_threaded_irq(dws->irq, dw_spi_irq, dw_spi_irq_thread_fn,
+				   IRQF_SHARED, dev_name(dev), ctlr);
 	if (ret < 0 && ret != -ENOTCONN) {
 		dev_err(dev, "can not request IRQ\n");
 		goto err_free_ctlr;
Signature · 1 line
2.53.0
maintainer
#4 lore raw

Re: [PATCH v2 3/3] spi: dw: use threaded interrupt and optimize the threaded ISR

Quoted 6 lines · Jisheng Zhang

To avoid blocking for an excessive amount of time, eventually impacting on system responsiveness, hard interrupt handlers should finish executing in as little time as possible.

Use threaded interrupt and move the SPI transfer handling to an interrupt thread.

It might be safer to make this happen only in host mode, not in target mode - while the performance improvement will apply for both target mode has no control over the clocking of data by the host so is much more vulnerable to dropping data in a threaded interrupt if the system is loaded. In host mode this isn't an issue since we'll simply stop clocking data while waiting for the scheduler to get round to the interrupt thread.

maintainer
#5 lore raw
In reply to Jisheng Zhang
Quoted 3 lines · Jisheng Zhang

To avoid blocking for an excessive amount of time, eventually impacting on system responsiveness, hard interrupt handlers should finish executing in as little time as possible.

This is broadly good but needs a rebase against current code, I do have a couple of smaller comments.