Skip to contentSkip to conversation

riscv: Fix local irq restore when flags indicates irq disabled

PATCH Superseded Xu Lu 4 participants 10 messages Jul 25, 2023 – Jun 19, 2024 RISC-V

lore
Share & tools
lore

Xu Lu

author
#0 lore raw

PATCH 1/1 riscv: Fix local irq restore when flags indicates irq disabled

When arch_local_irq_restore() is called with flags indicating irqs disabled, we need to clear SR_IE bit in CSR_STATUS, whereas current implementation based on csr_set() function only sets SR_IE bit of CSR_STATUS when SR_IE bit of flags is high and does nothing when SR_IE bit of flags is low.

This commit supplies csr clear operation when calling irq restore function with flags indicating irq disabled.

Fixes 6d60b6ee0c97 ("RISC-V: Device, timer, IRQs, and the SBI")
---
 arch/riscv/include/asm/irqflags.h | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
arch/riscv/include/asm/irqflags.h +4 -1
index 08d4d6a5b7e9..7c31fc3c3559 100644
@@ -49,7 +49,10 @@ static inline int arch_irqs_disabled(void)
 /* set interrupt enabled status */
 static inline void arch_local_irq_restore(unsigned long flags)
 {
-	csr_set(CSR_STATUS, flags & SR_IE);
+	if (flags & SR_IE)
+		csr_set(CSR_STATUS, SR_IE);
+	else
+		csr_clear(CSR_STATUS, SR_IE);
 }
 
 #endif /* _ASM_RISCV_IRQFLAGS_H */
Signature · 7 lines
2.39.2 (Apple Git-143)


_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
maintainer
#1 lore raw
Quoted 20 lines · On Tue, 25 Jul 2023 00:05:49 PDT (-0700), [email protected] wrote:

When arch_local_irq_restore() is called with flags indicating irqs disabled, we need to clear SR_IE bit in CSR_STATUS, whereas current implementation based on csr_set() function only sets SR_IE bit of CSR_STATUS when SR_IE bit of flags is high and does nothing when SR_IE bit of flags is low.

This commit supplies csr clear operation when calling irq restore function with flags indicating irq disabled.

Fixes 6d60b6ee0c97 ("RISC-V: Device, timer, IRQs, and the SBI")
---
 arch/riscv/include/asm/irqflags.h | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
arch/riscv/include/asm/irqflags.h +0 -0
index 08d4d6a5b7e9..7c31fc3c3559 100644

arch/riscv/include/asm/irqflags.h static inline int arch_irqs_disabled(void) · line 55

Quoted 8 lines
arch/riscv/include/asm/irqflags.h +4 -1
@@ -49,7 +49,10 @@ static inline int arch_irqs_disabled(void)
 /* set interrupt enabled status */
 static inline void arch_local_irq_restore(unsigned long flags)
 {
-	csr_set(CSR_STATUS, flags & SR_IE);
+	if (flags & SR_IE)
+		csr_set(CSR_STATUS, SR_IE);
+	else
+		csr_clear(CSR_STATUS, SR_IE);

Unless I'm missing something, the original version is correct: local_irq_restore() must be paired with a local_irq_save(), so we can only get here with interrupts disabled.

Quoted 3 lines
arch/riscv/include/asm/irqflags.h +0 -0
 }
 
 #endif /* _ASM_RISCV_IRQFLAGS_H */
author
#2 lore raw
Quoted 26 lines · Palmer Dabbelt
Quoted 24 lines · On Tue, 25 Jul 2023 00:05:49 PDT (-0700), [email protected] wrote:

When arch_local_irq_restore() is called with flags indicating irqs disabled, we need to clear SR_IE bit in CSR_STATUS, whereas current implementation based on csr_set() function only sets SR_IE bit of CSR_STATUS when SR_IE bit of flags is high and does nothing when SR_IE bit of flags is low.

This commit supplies csr clear operation when calling irq restore function with flags indicating irq disabled.

Fixes 6d60b6ee0c97 ("RISC-V: Device, timer, IRQs, and the SBI")
---
 arch/riscv/include/asm/irqflags.h | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
arch/riscv/include/asm/irqflags.h +0 -1
index 08d4d6a5b7e9..7c31fc3c3559 100644
@@ -49,7 +49,10 @@ static inline int arch_irqs_disabled(void)
 /* set interrupt enabled status */
 static inline void arch_local_irq_restore(unsigned long flags)
 {
-     csr_set(CSR_STATUS, flags & SR_IE);
Quoted 8 lines

arch/riscv/include/asm/irqflags.h static inline int arch_irqs_disabled(void) · line 55

Quoted 4 lines
arch/riscv/include/asm/irqflags.h +4 -0
+     if (flags & SR_IE)
+             csr_set(CSR_STATUS, SR_IE);
+     else
+             csr_clear(CSR_STATUS, SR_IE);

Unless I'm missing something, the original version is correct:
local_irq_restore() must be paired with a local_irq_save(), so we can only get here with interrupts disabled.

Yes, it is correct if local_irq_save() is called when irq is enabled before. The flags returned will be SR_IE. And it is safe to call local_irq_restore() with flag SR_IE in any case.

However, if local_irq_save() is called when irq is disabled. The SR_IE bit of flag returned is clear. If some code between local_irq_save() and local_irq_restore() reenables irq, causing the SR_IE bit of CSR_STATUS back to 1, then local_irq_restore() can not restore irq status back to disabled.

Here is an example in existing driver (may not belong to riscv arch) in drivers/net/wireless/realtek/rtlwifi/rtl8723ae/hw.c. The pseudo code behaves like:

int rtl8723e_hw_init(struct ieee80211_hw *hw)

{
     int err;
     unsigned long flags;
     ...
     local_irq_save_flags(flags);
     local_irq_enable();
     ...
     local_irq_restore(flags);
     ...
     return err;
}
Quoted 4 lines

arch/riscv/include/asm/irqflags.h static inline int arch_irqs_disabled(void) · line 58

Quoted 4 lines
arch/riscv/include/asm/irqflags.h +0 -0
 }
 
 #endif /* _ASM_RISCV_IRQFLAGS_H */

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

author
#3 lore raw
Quoted 51 lines · 旭路
Quoted 26 lines · Palmer Dabbelt
Quoted 24 lines · On Tue, 25 Jul 2023 00:05:49 PDT (-0700), [email protected] wrote:

When arch_local_irq_restore() is called with flags indicating irqs disabled, we need to clear SR_IE bit in CSR_STATUS, whereas current implementation based on csr_set() function only sets SR_IE bit of CSR_STATUS when SR_IE bit of flags is high and does nothing when SR_IE bit of flags is low.

This commit supplies csr clear operation when calling irq restore function with flags indicating irq disabled.

Fixes 6d60b6ee0c97 ("RISC-V: Device, timer, IRQs, and the SBI")
---
 arch/riscv/include/asm/irqflags.h | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
arch/riscv/include/asm/irqflags.h +0 -1
index 08d4d6a5b7e9..7c31fc3c3559 100644
@@ -49,7 +49,10 @@ static inline int arch_irqs_disabled(void)
 /* set interrupt enabled status */
 static inline void arch_local_irq_restore(unsigned long flags)
 {
-     csr_set(CSR_STATUS, flags & SR_IE);
Quoted 8 lines

arch/riscv/include/asm/irqflags.h static inline int arch_irqs_disabled(void) · line 55

Quoted 4 lines
arch/riscv/include/asm/irqflags.h +4 -0
+     if (flags & SR_IE)
+             csr_set(CSR_STATUS, SR_IE);
+     else
+             csr_clear(CSR_STATUS, SR_IE);

Unless I'm missing something, the original version is correct:
local_irq_restore() must be paired with a local_irq_save(), so we can only get here with interrupts disabled.

Yes, it is correct if local_irq_save() is called when irq is enabled before. The flags returned will be SR_IE. And it is safe to call local_irq_restore() with flag SR_IE in any case.

However, if local_irq_save() is called when irq is disabled. The SR_IE bit of flag returned is clear. If some code between local_irq_save() and local_irq_restore() reenables irq, causing the SR_IE bit of CSR_STATUS back to 1, then local_irq_restore() can not restore irq status back to disabled.

Here is an example in existing driver (may not belong to riscv arch) in drivers/net/wireless/realtek/rtlwifi/rtl8723ae/hw.c. The pseudo code behaves like:

int rtl8723e_hw_init(struct ieee80211_hw *hw)

Quoted 17 lines
{
     int err;
     unsigned long flags;
     ...
     local_irq_save_flags(flags);
     local_irq_enable();
     ...
     local_irq_restore(flags);
     ...
     return err;
}
Quoted 4 lines
Quoted 4 lines
arch/riscv/include/asm/irqflags.h +0 -0
 }
 
 #endif /* _ASM_RISCV_IRQFLAGS_H */

A gentle ping.

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

#4 lore raw

Hi Xu Lu,

Quoted 61 lines · Xu Lu
Quoted 60 lines · 旭路
Quoted 16 lines · Palmer Dabbelt
Quoted 15 lines · On Tue, 25 Jul 2023 00:05:49 PDT (-0700), [email protected] wrote:

When arch_local_irq_restore() is called with flags indicating irqs disabled, we need to clear SR_IE bit in CSR_STATUS, whereas current implementation based on csr_set() function only sets SR_IE bit of CSR_STATUS when SR_IE bit of flags is high and does nothing when SR_IE bit of flags is low.

This commit supplies csr clear operation when calling irq restore function with flags indicating irq disabled.

Fixes 6d60b6ee0c97 ("RISC-V: Device, timer, IRQs, and the SBI")
---
  arch/riscv/include/asm/irqflags.h | 5 ++++-
  1 file changed, 4 insertions(+), 1 deletion(-)
Quoted 16 lines

arch/riscv/include/asm/irqflags.h static inline int arch_irqs_disabled(void) · line 55

Quoted 13 lines
arch/riscv/include/asm/irqflags.h +4 -1
index 08d4d6a5b7e9..7c31fc3c3559 100644
@@ -49,7 +49,10 @@ static inline int arch_irqs_disabled(void)
  /* set interrupt enabled status */
  static inline void arch_local_irq_restore(unsigned long flags)
  {
-     csr_set(CSR_STATUS, flags & SR_IE);
+     if (flags & SR_IE)
+             csr_set(CSR_STATUS, SR_IE);
+     else
+             csr_clear(CSR_STATUS, SR_IE);

Unless I'm missing something, the original version is correct:
local_irq_restore() must be paired with a local_irq_save(), so we can only get here with interrupts disabled.

Yes, it is correct if local_irq_save() is called when irq is enabled before. The flags returned will be SR_IE. And it is safe to call local_irq_restore() with flag SR_IE in any case.

However, if local_irq_save() is called when irq is disabled. The SR_IE bit of flag returned is clear. If some code between local_irq_save() and local_irq_restore() reenables irq, causing the SR_IE bit of CSR_STATUS back to 1, then local_irq_restore() can not restore irq status back to disabled.

Here is an example in existing driver (may not belong to riscv arch) in drivers/net/wireless/realtek/rtlwifi/rtl8723ae/hw.c. The pseudo code behaves like:

int rtl8723e_hw_init(struct ieee80211_hw *hw)

{
      int err;
      unsigned long flags;
      ...
      local_irq_save_flags(flags);
      local_irq_enable();
      ...
      local_irq_restore(flags);
      ...
      return err;
}
Quoted 4 lines
Quoted 3 lines
Quoted 3 lines

arch/riscv/include/asm/irqflags.h static inline int arch_irqs_disabled(void) · line 58

Quoted 3 lines
arch/riscv/include/asm/irqflags.h +0 -0
  }
 
  #endif /* _ASM_RISCV_IRQFLAGS_H */

A gentle ping.

This got lost but this is still correct and needed.

Do you think you can respin a new version? Otherwise, I'll do it and keep you SoB.

Thanks,

Alex

Quoted 5 lines

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

author
#5 lore raw

Hi Alex,

I will send a new version later today.

Thanks,

Xu Lu

Quoted 86 lines · Alexandre Ghiti

Hi Xu Lu,

Quoted 61 lines · Xu Lu
Quoted 60 lines · 旭路
Quoted 16 lines · Palmer Dabbelt
Quoted 15 lines · On Tue, 25 Jul 2023 00:05:49 PDT (-0700), [email protected] wrote:

When arch_local_irq_restore() is called with flags indicating irqs disabled, we need to clear SR_IE bit in CSR_STATUS, whereas current implementation based on csr_set() function only sets SR_IE bit of CSR_STATUS when SR_IE bit of flags is high and does nothing when SR_IE bit of flags is low.

This commit supplies csr clear operation when calling irq restore function with flags indicating irq disabled.

Fixes 6d60b6ee0c97 ("RISC-V: Device, timer, IRQs, and the SBI")
---
  arch/riscv/include/asm/irqflags.h | 5 ++++-
  1 file changed, 4 insertions(+), 1 deletion(-)
Quoted 16 lines

arch/riscv/include/asm/irqflags.h static inline int arch_irqs_disabled(void) · line 55

Quoted 13 lines
arch/riscv/include/asm/irqflags.h +4 -1
index 08d4d6a5b7e9..7c31fc3c3559 100644
@@ -49,7 +49,10 @@ static inline int arch_irqs_disabled(void)
  /* set interrupt enabled status */
  static inline void arch_local_irq_restore(unsigned long flags)
  {
-     csr_set(CSR_STATUS, flags & SR_IE);
+     if (flags & SR_IE)
+             csr_set(CSR_STATUS, SR_IE);
+     else
+             csr_clear(CSR_STATUS, SR_IE);

Unless I'm missing something, the original version is correct:
local_irq_restore() must be paired with a local_irq_save(), so we can only get here with interrupts disabled.

Yes, it is correct if local_irq_save() is called when irq is enabled before. The flags returned will be SR_IE. And it is safe to call local_irq_restore() with flag SR_IE in any case.

However, if local_irq_save() is called when irq is disabled. The SR_IE bit of flag returned is clear. If some code between local_irq_save() and local_irq_restore() reenables irq, causing the SR_IE bit of CSR_STATUS back to 1, then local_irq_restore() can not restore irq status back to disabled.

Here is an example in existing driver (may not belong to riscv arch) in drivers/net/wireless/realtek/rtlwifi/rtl8723ae/hw.c. The pseudo code behaves like:

int rtl8723e_hw_init(struct ieee80211_hw *hw)

{
      int err;
      unsigned long flags;
      ...
      local_irq_save_flags(flags);
      local_irq_enable();
      ...
      local_irq_restore(flags);
      ...
      return err;
}
Quoted 4 lines
Quoted 3 lines
Quoted 3 lines

arch/riscv/include/asm/irqflags.h static inline int arch_irqs_disabled(void) · line 58

Quoted 3 lines
arch/riscv/include/asm/irqflags.h +0 -0
  }
 
  #endif /* _ASM_RISCV_IRQFLAGS_H */

A gentle ping.

This got lost but this is still correct and needed.

Do you think you can respin a new version? Otherwise, I'll do it and keep you SoB.

Thanks,

Alex

Quoted 5 lines

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

#6 lore raw
Quoted 3 lines · Xu Lu

Hi Alex,

I will send a new version later today.

Great, thanks, I'll make sure it lands in -fixes.

Sorry it did not get merged sooner.

Thanks,

Alex

Quoted 93 lines

Thanks,

Xu Lu

Quoted 69 lines · Alexandre Ghiti

Hi Xu Lu,

Quoted 61 lines · Xu Lu
Quoted 60 lines · 旭路
Quoted 16 lines · Palmer Dabbelt
Quoted 15 lines · On Tue, 25 Jul 2023 00:05:49 PDT (-0700), [email protected] wrote:

When arch_local_irq_restore() is called with flags indicating irqs disabled, we need to clear SR_IE bit in CSR_STATUS, whereas current implementation based on csr_set() function only sets SR_IE bit of CSR_STATUS when SR_IE bit of flags is high and does nothing when SR_IE bit of flags is low.

This commit supplies csr clear operation when calling irq restore function with flags indicating irq disabled.

Fixes 6d60b6ee0c97 ("RISC-V: Device, timer, IRQs, and the SBI")
---
   arch/riscv/include/asm/irqflags.h | 5 ++++-
   1 file changed, 4 insertions(+), 1 deletion(-)
Quoted 16 lines

arch/riscv/include/asm/irqflags.h static inline int arch_irqs_disabled(void) · line 55

Quoted 13 lines
arch/riscv/include/asm/irqflags.h +4 -1
index 08d4d6a5b7e9..7c31fc3c3559 100644
@@ -49,7 +49,10 @@ static inline int arch_irqs_disabled(void)
   /* set interrupt enabled status */
   static inline void arch_local_irq_restore(unsigned long flags)
   {
-     csr_set(CSR_STATUS, flags & SR_IE);
+     if (flags & SR_IE)
+             csr_set(CSR_STATUS, SR_IE);
+     else
+             csr_clear(CSR_STATUS, SR_IE);

Unless I'm missing something, the original version is correct:
local_irq_restore() must be paired with a local_irq_save(), so we can only get here with interrupts disabled.

Yes, it is correct if local_irq_save() is called when irq is enabled before. The flags returned will be SR_IE. And it is safe to call local_irq_restore() with flag SR_IE in any case.

However, if local_irq_save() is called when irq is disabled. The SR_IE bit of flag returned is clear. If some code between local_irq_save() and local_irq_restore() reenables irq, causing the SR_IE bit of CSR_STATUS back to 1, then local_irq_restore() can not restore irq status back to disabled.

Here is an example in existing driver (may not belong to riscv arch) in drivers/net/wireless/realtek/rtlwifi/rtl8723ae/hw.c. The pseudo code behaves like:

int rtl8723e_hw_init(struct ieee80211_hw *hw)

{
       int err;
       unsigned long flags;
       ...
       local_irq_save_flags(flags);
       local_irq_enable();
       ...
       local_irq_restore(flags);
       ...
       return err;
}
Quoted 4 lines
Quoted 3 lines
Quoted 3 lines

arch/riscv/include/asm/irqflags.h static inline int arch_irqs_disabled(void) · line 58

Quoted 3 lines
arch/riscv/include/asm/irqflags.h +0 -0
   }
 
   #endif /* _ASM_RISCV_IRQFLAGS_H */

A gentle ping.

Quoted 14 lines

This got lost but this is still correct and needed.

Do you think you can respin a new version? Otherwise, I'll do it and keep you SoB.

Thanks,

Alex

Quoted 4 lines

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

#7 lore raw
In reply to Alexandre Ghiti

(merging replies)

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

However, if local_irq_save() is called when irq is disabled. The SR_IE bit of flag returned is clear. If some code between local_irq_save() and local_irq_restore() reenables irq, causing the SR_IE bit of CSR_STATUS back to 1, then local_irq_restore() can not restore irq status back to disabled.

But doesn't that represent some bogus manipulation of the irq flags? cf.

config DEBUG_IRQFLAGS
bool "Debug IRQ flag manipulation"
help
Enables checks for potentially unsafe enabling or disabling of interrupts, such as calling raw_local_irq_restore() when interrupts are enabled.

in particular, raw_check_bogus_irq_restore() in raw_local_irq_restore().

Quoted 1 line

This got lost but this is still correct and needed.

You mean because of the mentioned rtl8723e example? are there other such instances? IOW, why do you say that the changes in question are needed?

Andrea

#8 lore raw

Hi Andrea,

Quoted 15 lines · Andrea Parri

(merging replies)

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

However, if local_irq_save() is called when irq is disabled. The SR_IE bit of flag returned is clear. If some code between local_irq_save() and local_irq_restore() reenables irq, causing the SR_IE bit of CSR_STATUS back to 1, then local_irq_restore() can not restore irq status back to disabled.

But doesn't that represent some bogus manipulation of the irq flags? cf.

config DEBUG_IRQFLAGS
bool "Debug IRQ flag manipulation"
help
Enables checks for potentially unsafe enabling or disabling of interrupts, such as calling raw_local_irq_restore() when interrupts are enabled.

Quoted 6 lines

in particular, raw_check_bogus_irq_restore() in raw_local_irq_restore().

Quoted 1 line

This got lost but this is still correct and needed.

You mean because of the mentioned rtl8723e example? are there other such instances? IOW, why do you say that the changes in question are needed?

Simply because the scenario in this driver and I looked at the arm64 implementation which restores flags unconditionally.

But if that's considered bogus, let's drop this. Sorry Xu for the noise, and thanks Andrea for pointing this.

Alex

Quoted 2 lines

Andrea

author
#9 lore raw

Hi Alex,

Quoted 22 lines · Alexandre Ghiti

Hi Andrea,

Quoted 18 lines · Andrea Parri

(merging replies)

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

However, if local_irq_save() is called when irq is disabled. The SR_IE bit of flag returned is clear. If some code between local_irq_save() and local_irq_restore() reenables irq, causing the SR_IE bit of CSR_STATUS back to 1, then local_irq_restore() can not restore irq status back to disabled.

But doesn't that represent some bogus manipulation of the irq flags? cf.

config DEBUG_IRQFLAGS
bool "Debug IRQ flag manipulation"
help
Enables checks for potentially unsafe enabling or disabling of interrupts, such as calling raw_local_irq_restore() when interrupts are enabled.

in particular, raw_check_bogus_irq_restore() in raw_local_irq_restore().

Quoted 7 lines
Quoted 3 lines
Quoted 1 line

This got lost but this is still correct and needed.

You mean because of the mentioned rtl8723e example? are there other such instances? IOW, why do you say that the changes in question are needed?

Simply because the scenario in this driver and I looked at the arm64 implementation which restores flags unconditionally.

Indeed, ARM64 implementation restores flags unconditionally.

Actually in the beginning I found several drivers use local_irq_restore after local_irq_enable.
Such drivers include 'scsi/arm/acornscsi.c', 'net/wireless/realtek/rtlwifi/rtlxxx/hw.c', etc.
I wonder whether the semantics of local_irq_restore should be restoring irq flags unconditionally and thus I submitted this patch.

Quoted 3 lines

But if that's considered bogus, let's drop this. Sorry Xu for the noise, and thanks Andrea for pointing this.

It's OK if we think this is not a problem. Just ignore this patch.

Thanks!

Xu Lu

Quoted 6 lines

Alex

Quoted 2 lines

Andrea