Skip to contentSkip to conversation

serial: stm32: Move hard IRQ handling to threaded interrupt context

PATCH In review Marek Vasut 3 participants 6 messages Dec 7, 2022 – Dec 16, 2022

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 serial: stm32: Move hard IRQ handling to threaded interrupt context

Avoid locking in hard interrupt context, move the entirety of hard IRQ context code into the threaded IRQ handler. This fixes the following splat with preempt-rt enabled:

 BUG: scheduling while atomic: (mount)/1289/0x00010001
 Modules linked in:
 Preemption disabled at:
 [<c0119127>] irq_enter_rcu+0xb/0x42
 CPU: 0 PID: 1289 Comm: (mount) Not tainted 6.1.0-rc7-rt5-stable-standard-00006-gd70aeccb9f0f #17
 Hardware name: STM32 (Device Tree Support)
  unwind_backtrace from show_stack+0xb/0xc
  show_stack from dump_stack_lvl+0x2b/0x34
  dump_stack_lvl from __schedule_bug+0x53/0x80
  __schedule_bug from __schedule+0x47/0x404
  __schedule from schedule_rtlock+0x15/0x34
  schedule_rtlock from rtlock_slowlock_locked+0x1d7/0x57e
  rtlock_slowlock_locked from rt_spin_lock+0x29/0x3c
  rt_spin_lock from stm32_usart_interrupt+0xa9/0x110
  stm32_usart_interrupt from __handle_irq_event_percpu+0x73/0x14e
  __handle_irq_event_percpu from handle_irq_event_percpu+0x9/0x22
  handle_irq_event_percpu from handle_irq_event+0x53/0x76
  handle_irq_event from handle_fasteoi_irq+0x65/0xa8
  handle_fasteoi_irq from handle_irq_desc+0xf/0x18
  handle_irq_desc from gic_handle_irq+0x45/0x54
  gic_handle_irq from generic_handle_arch_irq+0x19/0x2c
  generic_handle_arch_irq from call_with_stack+0xd/0x10

---
Cc: Alexandre Torgue <[email protected]>
Cc: Erwan Le Ray <[email protected]>
Cc: Greg Kroah-Hartman <[email protected]>
Cc: Jean Philippe Romain <[email protected]>
Cc: Jiri Slaby <[email protected]>
Cc: Maxime Coquelin <[email protected]>
Cc: Sebastian Andrzej Siewior <[email protected]>
Cc: Thomas Gleixner <[email protected]>
Cc: Valentin Caron <[email protected]>
Cc: [email protected]
Cc: [email protected]
To: [email protected]

---
 drivers/tty/serial/stm32-usart.c | 27 +++++++--------------------
 1 file changed, 7 insertions(+), 20 deletions(-)
drivers/tty/serial/stm32-usart.c +7 -20
index a1490033aa164..f5bce0be02676 100644
@@ -745,14 +745,15 @@ static void stm32_usart_transmit_chars(struct uart_port *port)
 	}
 }
 
-static irqreturn_t stm32_usart_interrupt(int irq, void *ptr)
+static irqreturn_t stm32_usart_threaded_interrupt(int irq, void *ptr)
 {
 	struct uart_port *port = ptr;
 	struct tty_port *tport = &port->state->port;
 	struct stm32_port *stm32_port = to_stm32_port(port);
 	const struct stm32_usart_offsets *ofs = &stm32_port->info->ofs;
-	u32 sr;
+	unsigned long flags;
 	unsigned int size;
+	u32 sr;
 
 	sr = readl_relaxed(port->membase + ofs->isr);
 
@@ -792,27 +793,13 @@ static irqreturn_t stm32_usart_interrupt(int irq, void *ptr)
 	}
 
 	if ((sr & USART_SR_TXE) && !(stm32_port->tx_ch)) {
-		spin_lock(&port->lock);
+		spin_lock_irqsave(&port->lock, flags);
 		stm32_usart_transmit_chars(port);
-		spin_unlock(&port->lock);
+		spin_unlock_irqrestore(&port->lock, flags);
 	}
 
-	if (stm32_usart_rx_dma_enabled(port))
-		return IRQ_WAKE_THREAD;
-	else
-		return IRQ_HANDLED;
-}
-
-static irqreturn_t stm32_usart_threaded_interrupt(int irq, void *ptr)
-{
-	struct uart_port *port = ptr;
-	struct tty_port *tport = &port->state->port;
-	struct stm32_port *stm32_port = to_stm32_port(port);
-	unsigned int size;
-	unsigned long flags;
-
 	/* Receiver timeout irq for DMA RX */
-	if (!stm32_port->throttled) {
+	if (stm32_usart_rx_dma_enabled(port) && !stm32_port->throttled) {
 		spin_lock_irqsave(&port->lock, flags);
 		size = stm32_usart_receive_chars(port, false);
 		uart_unlock_and_check_sysrq_irqrestore(port, flags);
@@ -1015,7 +1002,7 @@ static int stm32_usart_startup(struct uart_port *port)
 	u32 val;
 	int ret;
 
-	ret = request_threaded_irq(port->irq, stm32_usart_interrupt,
+	ret = request_threaded_irq(port->irq, NULL,
 				   stm32_usart_threaded_interrupt,
 				   IRQF_ONESHOT | IRQF_NO_SUSPEND,
 				   name, port);
Signature · 1 line
2.35.1
#1 lore raw

- Jean Philippe Romain <[email protected]>

Could you please drop that email address in future post as I did? It comes back here with:
| Remote-MTA: dns; mxb-00178001.gslb.pphosted.com
| Diagnostic-Code: smtp; 550 5.1.1 User Unknown

drivers/tty/serial/stm32-usart.c static int stm32_usart_startup(struct uart_port *port) · line 1008

Quoted 13 lines · Marek Vasut
drivers/tty/serial/stm32-usart.c +1 -1
index a1490033aa164..f5bce0be02676 100644
@@ -1015,7 +1002,7 @@ static int stm32_usart_startup(struct uart_port *port)
 	u32 val;
 	int ret;
 
-	ret = request_threaded_irq(port->irq, stm32_usart_interrupt,
+	ret = request_threaded_irq(port->irq, NULL,
 				   stm32_usart_threaded_interrupt,
 				   IRQF_ONESHOT | IRQF_NO_SUSPEND,
 				   name, port);

So why don't you just

	request_irq(port->irq, stm32_usart_interrupt, IRQF_NO_SUSPEND,
		    name, port)

? If there is a requirement to always run in threaded mode please document it by at least making it part of the commit description.

Sebastian

#2 lore raw
In reply to Marek Vasut

Hi Marek,

I've got a patch in the same spirit in downstream. The test campaign reveals performance issues with this patch.

In fact, hard IRQ have been introduced in stm32-usart driver to solve performance issues due to short FIFO size (16 bytes).

We are currently testing another patch, similar as your RFC proposition, for RT context. But results are not ready yet. We can wait them before merging this big change into driver ?

Thanks,
Valentin

Quoted 107 lines · Marek Vasut

Avoid locking in hard interrupt context, move the entirety of hard IRQ context code into the threaded IRQ handler. This fixes the following splat with preempt-rt enabled:

  BUG: scheduling while atomic: (mount)/1289/0x00010001
  Modules linked in:
  Preemption disabled at:
  [<c0119127>] irq_enter_rcu+0xb/0x42
  CPU: 0 PID: 1289 Comm: (mount) Not tainted 6.1.0-rc7-rt5-stable-standard-00006-gd70aeccb9f0f #17
  Hardware name: STM32 (Device Tree Support)
   unwind_backtrace from show_stack+0xb/0xc
   show_stack from dump_stack_lvl+0x2b/0x34
   dump_stack_lvl from __schedule_bug+0x53/0x80
   __schedule_bug from __schedule+0x47/0x404
   __schedule from schedule_rtlock+0x15/0x34
   schedule_rtlock from rtlock_slowlock_locked+0x1d7/0x57e
   rtlock_slowlock_locked from rt_spin_lock+0x29/0x3c
   rt_spin_lock from stm32_usart_interrupt+0xa9/0x110
   stm32_usart_interrupt from __handle_irq_event_percpu+0x73/0x14e
   __handle_irq_event_percpu from handle_irq_event_percpu+0x9/0x22
   handle_irq_event_percpu from handle_irq_event+0x53/0x76
   handle_irq_event from handle_fasteoi_irq+0x65/0xa8
   handle_fasteoi_irq from handle_irq_desc+0xf/0x18
   handle_irq_desc from gic_handle_irq+0x45/0x54
   gic_handle_irq from generic_handle_arch_irq+0x19/0x2c
   generic_handle_arch_irq from call_with_stack+0xd/0x10

---
Cc: Alexandre Torgue <[email protected]>
Cc: Erwan Le Ray <[email protected]>
Cc: Greg Kroah-Hartman <[email protected]>
Cc: Jean Philippe Romain <[email protected]>
Cc: Jiri Slaby <[email protected]>
Cc: Maxime Coquelin <[email protected]>
Cc: Sebastian Andrzej Siewior <[email protected]>
Cc: Thomas Gleixner <[email protected]>
Cc: Valentin Caron <[email protected]>
Cc: [email protected]
Cc: [email protected]
To: [email protected]

---
  drivers/tty/serial/stm32-usart.c | 27 +++++++--------------------
  1 file changed, 7 insertions(+), 20 deletions(-)
drivers/tty/serial/stm32-usart.c +7 -20
index a1490033aa164..f5bce0be02676 100644
@@ -745,14 +745,15 @@ static void stm32_usart_transmit_chars(struct uart_port *port)
  	}
  }
 
-static irqreturn_t stm32_usart_interrupt(int irq, void *ptr)
+static irqreturn_t stm32_usart_threaded_interrupt(int irq, void *ptr)
  {
  	struct uart_port *port = ptr;
  	struct tty_port *tport = &port->state->port;
  	struct stm32_port *stm32_port = to_stm32_port(port);
  	const struct stm32_usart_offsets *ofs = &stm32_port->info->ofs;
-	u32 sr;
+	unsigned long flags;
  	unsigned int size;
+	u32 sr;
 
  	sr = readl_relaxed(port->membase + ofs->isr);
 
@@ -792,27 +793,13 @@ static irqreturn_t stm32_usart_interrupt(int irq, void *ptr)
  	}
 
  	if ((sr & USART_SR_TXE) && !(stm32_port->tx_ch)) {
-		spin_lock(&port->lock);
+		spin_lock_irqsave(&port->lock, flags);
  		stm32_usart_transmit_chars(port);
-		spin_unlock(&port->lock);
+		spin_unlock_irqrestore(&port->lock, flags);
  	}
 
-	if (stm32_usart_rx_dma_enabled(port))
-		return IRQ_WAKE_THREAD;
-	else
-		return IRQ_HANDLED;
-}
-
-static irqreturn_t stm32_usart_threaded_interrupt(int irq, void *ptr)
-{
-	struct uart_port *port = ptr;
-	struct tty_port *tport = &port->state->port;
-	struct stm32_port *stm32_port = to_stm32_port(port);
-	unsigned int size;
-	unsigned long flags;
-
  	/* Receiver timeout irq for DMA RX */
-	if (!stm32_port->throttled) {
+	if (stm32_usart_rx_dma_enabled(port) && !stm32_port->throttled) {
  		spin_lock_irqsave(&port->lock, flags);
  		size = stm32_usart_receive_chars(port, false);
  		uart_unlock_and_check_sysrq_irqrestore(port, flags);
@@ -1015,7 +1002,7 @@ static int stm32_usart_startup(struct uart_port *port)
  	u32 val;
  	int ret;
 
-	ret = request_threaded_irq(port->irq, stm32_usart_interrupt,
+	ret = request_threaded_irq(port->irq, NULL,
  				   stm32_usart_threaded_interrupt,
  				   IRQF_ONESHOT | IRQF_NO_SUSPEND,
  				   name, port);
#3 lore raw
Quoted 1 line · Valentin CARON

Hi Marek,

Hi,

Quoted 7 lines

I've got a patch in the same spirit in downstream. The test campaign reveals performance issues with this patch.

In fact, hard IRQ have been introduced in stm32-usart driver to solve performance issues due to short FIFO size (16 bytes).

We are currently testing another patch, similar as your RFC proposition, for RT context. But results are not ready yet. We can wait them before merging this big change into driver ?

Can you post your patch as an RFC so others can test as well ?

#4 lore raw

Hi Marek,

I test your patch with a loop-back test between usart3 and uart7 on STM32MP157C-DK2. It shows a speed transfer diminution (about -1,5%), some locks during transfer, RT throttling etc... (I use lszrz tool).

But finally, I think your patch is the best solution that we have. Other solutions always throws an RT exception on a spin_lock.

Thanks,
Valentin

Quoted 14 lines · Marek Vasut
Quoted 1 line · Valentin CARON

Hi Marek,

Hi,

Quoted 7 lines

I've got a patch in the same spirit in downstream. The test campaign reveals performance issues with this patch.

In fact, hard IRQ have been introduced in stm32-usart driver to solve performance issues due to short FIFO size (16 bytes).

We are currently testing another patch, similar as your RFC proposition, for RT context. But results are not ready yet. We can wait them before merging this big change into driver ?

Can you post your patch as an RFC so others can test as well ?

#5 lore raw
Quoted 1 line · Valentin CARON

Hi Marek,

Hi,

Quoted 6 lines

I test your patch with a loop-back test between usart3 and uart7 on STM32MP157C-DK2. It shows a speed transfer diminution (about -1,5%), some locks during transfer, RT throttling etc... (I use lszrz tool).

But finally, I think your patch is the best solution that we have. Other solutions always throws an RT exception on a spin_lock.

I just sent a V2, so let's see how that works.

Thanks for testing!