Skip to contentSkip to conversation

powerpc/entry: Use hard_irq_disable() in arch_interrupt_exit_prepare()

PATCH In review Mukesh Kumar Chaurasiya (IBM) 4 participants 11 messages Aug 12 – Aug 20 PowerPC

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 powerpc/entry: Use hard_irq_disable() in arch_interrupt_exit_prepare()

commit 334f3f6d7a16 ("powerpc/entry: Disable interrupts before irqentry_exit") fixed a BUG in preempt_schedule_irq() by calling local_irq_disable() unconditionally in arch_interrupt_exit_prepare() before irqentry_exit(). The fix is correct in intent but uses the wrong primitive for PPC64.

On PPC64, local_irq_disable() only sets the soft-mask (irq_soft_mask = IRQS_DISABLED). It does not set PACA_IRQ_HARD_DIS in irq_happened and does not clear MSR[EE].

This causes a continuous WARN_ON boot hang on Power11 pSeries LPARs using the dedicated-cede cpuidle path. When the CPU wakes from H_CEDE, interrupt_exit_kernel_prepare() calls replay_soft_interrupts(), which dispatches pending async handlers (timer_interrupt, do_IRQ) using the DEFINE_INTERRUPT_HANDLER_ASYNC macro. That macro calls arch_interrupt_async_exit_prepare() -> arch_interrupt_exit_prepare() before irqentry_exit(). With local_irq_disable(), PACA_IRQ_HARD_DIS is not set, but next_interrupt() unconditionally asserts it:

  WARN_ON(!(local_paca->irq_happened & PACA_IRQ_HARD_DIS));

This fires on every replayed interrupt, looping indefinitely and preventing boot completion.

Fix this by replacing local_irq_disable() with hard_irq_disable(). On PPC64, hard_irq_disable() sets irq_soft_mask to IRQS_ALL_DISABLED, sets PACA_IRQ_HARD_DIS in irq_happened, and clears MSR[EE] — satisfying all of:

- lockdep_assert_irqs_disabled() in irqentry_exit_to_kernel_mode()
- next_interrupt()'s WARN_ON(!(irq_happened & PACA_IRQ_HARD_DIS))
- preempt_schedule_irq()'s BUG_ON(!irqs_disabled())

On PPC32/non-64, hard_irq_disable() is equivalent to local_irq_disable(), so there is no regression on those platforms.

Fixes 334f3f6d7a16 ("powerpc/entry: Disable interrupts before irqentry_exit")
---
 arch/powerpc/include/asm/entry-common.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/powerpc/include/asm/entry-common.h +1 -1
index c5adb5006361..de64389b0815 100644
@@ -270,7 +270,7 @@ static inline void arch_interrupt_exit_prepare(struct pt_regs *regs)
 	}
 
 	/* irqentry_exit expects to be called with interrupts disabled */
-	local_irq_disable();
+	hard_irq_disable();
 }
 
 static inline void arch_interrupt_async_enter_prepare(struct pt_regs *regs)
Signature · 1 line
2.55.0
Tested-by on v1 1/1
#1 lore raw
Quoted 32 lines · Mukesh Kumar Chaurasiya (IBM)

commit 334f3f6d7a16 ("powerpc/entry: Disable interrupts before irqentry_exit") fixed a BUG in preempt_schedule_irq() by calling local_irq_disable() unconditionally in arch_interrupt_exit_prepare() before irqentry_exit(). The fix is correct in intent but uses the wrong primitive for PPC64.

On PPC64, local_irq_disable() only sets the soft-mask (irq_soft_mask = IRQS_DISABLED). It does not set PACA_IRQ_HARD_DIS in irq_happened and does not clear MSR[EE].

This causes a continuous WARN_ON boot hang on Power11 pSeries LPARs using the dedicated-cede cpuidle path. When the CPU wakes from H_CEDE, interrupt_exit_kernel_prepare() calls replay_soft_interrupts(), which dispatches pending async handlers (timer_interrupt, do_IRQ) using the DEFINE_INTERRUPT_HANDLER_ASYNC macro. That macro calls arch_interrupt_async_exit_prepare() -> arch_interrupt_exit_prepare() before irqentry_exit(). With local_irq_disable(), PACA_IRQ_HARD_DIS is not set, but next_interrupt() unconditionally asserts it:

   WARN_ON(!(local_paca->irq_happened & PACA_IRQ_HARD_DIS));

This fires on every replayed interrupt, looping indefinitely and preventing boot completion.

Fix this by replacing local_irq_disable() with hard_irq_disable(). On PPC64, hard_irq_disable() sets irq_soft_mask to IRQS_ALL_DISABLED, sets PACA_IRQ_HARD_DIS in irq_happened, and clears MSR[EE] — satisfying all of:

- lockdep_assert_irqs_disabled() in irqentry_exit_to_kernel_mode()
- next_interrupt()'s WARN_ON(!(irq_happened & PACA_IRQ_HARD_DIS))
- preempt_schedule_irq()'s BUG_ON(!irqs_disabled())

Quoted 8 lines

On PPC32/non-64, hard_irq_disable() is equivalent to local_irq_disable(), so there is no regression on those platforms.

Fixes 334f3f6d7a16 ("powerpc/entry: Disable interrupts before irqentry_exit")

---

Tested this, and it fixes the reported issue.

Regards,

Venkat.

Quoted 16 lines
  arch/powerpc/include/asm/entry-common.h | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
arch/powerpc/include/asm/entry-common.h +1 -1
index c5adb5006361..de64389b0815 100644
@@ -270,7 +270,7 @@ static inline void arch_interrupt_exit_prepare(struct pt_regs *regs)
  	}
 
  	/* irqentry_exit expects to be called with interrupts disabled */
-	local_irq_disable();
+	hard_irq_disable();
  }
 
  static inline void arch_interrupt_async_enter_prepare(struct pt_regs *regs)
#2 lore raw

Le 13/08/2026 à 08:36, Venkat Rao Bagalkote a écrit :

Quoted 44 lines · Mukesh Kumar Chaurasiya (IBM)
Quoted 32 lines

commit 334f3f6d7a16 ("powerpc/entry: Disable interrupts before irqentry_exit") fixed a BUG in preempt_schedule_irq() by calling local_irq_disable() unconditionally in arch_interrupt_exit_prepare() before irqentry_exit(). The fix is correct in intent but uses the wrong primitive for PPC64.

On PPC64, local_irq_disable() only sets the soft-mask (irq_soft_mask = IRQS_DISABLED). It does not set PACA_IRQ_HARD_DIS in irq_happened and does not clear MSR[EE].

This causes a continuous WARN_ON boot hang on Power11 pSeries LPARs using the dedicated-cede cpuidle path. When the CPU wakes from H_CEDE, interrupt_exit_kernel_prepare() calls replay_soft_interrupts(), which dispatches pending async handlers (timer_interrupt, do_IRQ) using the DEFINE_INTERRUPT_HANDLER_ASYNC macro. That macro calls arch_interrupt_async_exit_prepare() -> arch_interrupt_exit_prepare() before irqentry_exit(). With local_irq_disable(), PACA_IRQ_HARD_DIS is not set, but next_interrupt() unconditionally asserts it:

   WARN_ON(!(local_paca->irq_happened & PACA_IRQ_HARD_DIS));

This fires on every replayed interrupt, looping indefinitely and preventing boot completion.

Fix this by replacing local_irq_disable() with hard_irq_disable(). On PPC64, hard_irq_disable() sets irq_soft_mask to IRQS_ALL_DISABLED, sets PACA_IRQ_HARD_DIS in irq_happened, and clears MSR[EE] — satisfying all of:

   - lockdep_assert_irqs_disabled() in irqentry_exit_to_kernel_mode()
   - next_interrupt()'s WARN_ON(!(irq_happened & PACA_IRQ_HARD_DIS))
   - preempt_schedule_irq()'s BUG_ON(!irqs_disabled())

Quoted 8 lines

On PPC32/non-64, hard_irq_disable() is equivalent to local_irq_disable(), so there is no regression on those platforms.

Fixes 334f3f6d7a16 ("powerpc/entry: Disable interrupts before irqentry_exit")
Closes https://eur01.safelinks.protection.outlook.com/? url=https%3A%2F%2Flore.kernel.org%2Fall%2F6f9bfb0f-b14c-468e-bb9f- c157d120d0dc%40linux.ibm.com%2F&data=05%7C02%7Cchristophe.leroy%40csgroup.eu%7Cf780f6a2abcf425c875808def90533e4%7C8b87af7d86474dc78df45f69a2011bb5%7C0%7C0%7C639221997929524168%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=kOer%2B610LkQs0ZKBn%2FutIB6j1bMoFasK4DmX4AEWOXA%3D&reserved=0

---

Quoted 3 lines

Tested this, and it fixes the reported issue.

Didn't you say yesterday [1] that the change doesn't fix the issue ? Did I miss something ?

[1] https://lore.kernel.org/all/[email protected]/

Quoted 22 lines

Regards,

Venkat.

Quoted 15 lines

  arch/powerpc/include/asm/entry-common.h | 2 +-

  1 file changed, 1 insertion(+), 1 deletion(-)
arch/powerpc/include/asm/entry-common.h +1 -1
index c5adb5006361..de64389b0815 100644
@@ -270,7 +270,7 @@ static inline void 
 arch_interrupt_exit_prepare(struct pt_regs *regs)
       }
       /* irqentry_exit expects to be called with interrupts disabled */
-    local_irq_disable();
+    hard_irq_disable();
   }
   static inline void arch_interrupt_async_enter_prepare(struct pt_regs *regs)
#3 lore raw
Quoted 48 lines · Christophe Leroy (CS GROUP)

Le 13/08/2026 à 08:36, Venkat Rao Bagalkote a écrit :

Quoted 45 lines · Mukesh Kumar Chaurasiya (IBM)
Quoted 41 lines

commit 334f3f6d7a16 ("powerpc/entry: Disable interrupts before irqentry_exit") fixed a BUG in preempt_schedule_irq() by calling local_irq_disable() unconditionally in arch_interrupt_exit_prepare() before irqentry_exit(). The fix is correct in intent but uses the wrong primitive for PPC64.

On PPC64, local_irq_disable() only sets the soft-mask (irq_soft_mask = IRQS_DISABLED). It does not set PACA_IRQ_HARD_DIS in irq_happened and does not clear MSR[EE].

This causes a continuous WARN_ON boot hang on Power11 pSeries LPARs using the dedicated-cede cpuidle path. When the CPU wakes from H_CEDE, interrupt_exit_kernel_prepare() calls replay_soft_interrupts(), which dispatches pending async handlers (timer_interrupt, do_IRQ) using the DEFINE_INTERRUPT_HANDLER_ASYNC macro. That macro calls arch_interrupt_async_exit_prepare() -> arch_interrupt_exit_prepare() before irqentry_exit(). With local_irq_disable(), PACA_IRQ_HARD_DIS is not set, but next_interrupt() unconditionally asserts it:

   WARN_ON(!(local_paca->irq_happened & PACA_IRQ_HARD_DIS));

This fires on every replayed interrupt, looping indefinitely and preventing boot completion.

Fix this by replacing local_irq_disable() with hard_irq_disable(). On PPC64, hard_irq_disable() sets irq_soft_mask to IRQS_ALL_DISABLED, sets PACA_IRQ_HARD_DIS in irq_happened, and clears MSR[EE] — satisfying all of:

   - lockdep_assert_irqs_disabled() in irqentry_exit_to_kernel_mode()
   - next_interrupt()'s WARN_ON(!(irq_happened & PACA_IRQ_HARD_DIS))
   - preempt_schedule_irq()'s BUG_ON(!irqs_disabled())

On PPC32/non-64, hard_irq_disable() is equivalent to local_irq_disable(), so there is no regression on those platforms.

Fixes 334f3f6d7a16 ("powerpc/entry: Disable interrupts before irqentry_exit")
Closes https://eur01.safelinks.protection.outlook.com/? url=https%3A%2F%2Flore.kernel.org%2Fall%2F6f9bfb0f-b14c-468e-bb9f- c157d120d0dc%40linux.ibm.com%2F&data=05%7C02%7Cchristophe.leroy%40csgroup.eu%7Cf780f6a2abcf425c875808def90533e4%7C8b87af7d86474dc78df45f69a2011bb5%7C0%7C0%7C639221997929524168%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=kOer%2B610LkQs0ZKBn%2FutIB6j1bMoFasK4DmX4AEWOXA%3D&reserved=0

---

Quoted 10 lines
Quoted 3 lines

Tested this, and it fixes the reported issue.

Didn't you say yesterday [1] that the change doesn't fix the issue ? Did I miss something ?

[1] https://lore.kernel.org/all/[email protected]/

Hi Christophe,

I think there are two separate issues being mixed together here, both of which were reported by me.

The first issue is the early boot hang. I tested the patch in that context and reported that the boot hang still persisted.

The second issue is the WARN in __replay_soft_interrupts(). For that issue, the same patch fixes the problem, which is why I provided the Tested-by tag.

I believe the confusion comes from the fact that the same patch ended up being discussed in relation to both reports. My earlier comment that the patch did not fix the issue was referring to the boot hang, while my Tested-by was for the WARN in __replay_soft_interrupts().

So, to clarify:

Boot hang issue: not fixed by the patch.
WARN in __replay_soft_interrupts(): fixed by the patch.

Regards,
Venkat

Quoted 23 lines
Quoted 22 lines

Regards,

Venkat.

Quoted 15 lines

arch/powerpc/include/asm/entry-common.h | 2 +-

  1 file changed, 1 insertion(+), 1 deletion(-)
arch/powerpc/include/asm/entry-common.h +1 -1
index c5adb5006361..de64389b0815 100644
@@ -270,7 +270,7 @@ static inline void 
 arch_interrupt_exit_prepare(struct pt_regs *regs)
       }
       /* irqentry_exit expects to be called with interrupts disabled */
-    local_irq_disable();
+    hard_irq_disable();
   }
   static inline void arch_interrupt_async_enter_prepare(struct pt_regs *regs)
#4 lore raw
In reply to Mukesh Kumar Chaurasiya (IBM)
Quoted 28 lines · Mukesh Kumar Chaurasiya (IBM)

commit 334f3f6d7a16 ("powerpc/entry: Disable interrupts before irqentry_exit") fixed a BUG in preempt_schedule_irq() by calling local_irq_disable() unconditionally in arch_interrupt_exit_prepare() before irqentry_exit(). The fix is correct in intent but uses the wrong primitive for PPC64.

On PPC64, local_irq_disable() only sets the soft-mask (irq_soft_mask = IRQS_DISABLED). It does not set PACA_IRQ_HARD_DIS in irq_happened and does not clear MSR[EE].

This causes a continuous WARN_ON boot hang on Power11 pSeries LPARs using the dedicated-cede cpuidle path. When the CPU wakes from H_CEDE, interrupt_exit_kernel_prepare() calls replay_soft_interrupts(), which dispatches pending async handlers (timer_interrupt, do_IRQ) using the DEFINE_INTERRUPT_HANDLER_ASYNC macro. That macro calls arch_interrupt_async_exit_prepare() -> arch_interrupt_exit_prepare() before irqentry_exit(). With local_irq_disable(), PACA_IRQ_HARD_DIS is not set, but next_interrupt() unconditionally asserts it:

   WARN_ON(!(local_paca->irq_happened & PACA_IRQ_HARD_DIS));

This fires on every replayed interrupt, looping indefinitely and preventing boot completion.

Fix this by replacing local_irq_disable() with hard_irq_disable(). On PPC64, hard_irq_disable() sets irq_soft_mask to IRQS_ALL_DISABLED, sets PACA_IRQ_HARD_DIS in irq_happened, and clears MSR[EE] — satisfying all of:

Quoted 7 lines

- lockdep_assert_irqs_disabled() in irqentry_exit_to_kernel_mode()
- next_interrupt()'s WARN_ON(!(irq_happened & PACA_IRQ_HARD_DIS))
- preempt_schedule_irq()'s BUG_ON(!irqs_disabled())

On PPC32/non-64, hard_irq_disable() is equivalent to local_irq_disable(), so there is no regression on those platforms.

This is good explanation, but what i not understanding is,

why pattern of setting PACA_IRQ_HARD_DIS changed?
Previous code at interrupt_exit_kernel_prepare which did local_irq_disable too.

Please check where was PACA_IRQ_HARD_DIS set without GENERIC_ENTRY which was preventing this from happening?

Quoted 21 lines
Fixes 334f3f6d7a16 ("powerpc/entry: Disable interrupts before irqentry_exit")
---
  arch/powerpc/include/asm/entry-common.h | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
arch/powerpc/include/asm/entry-common.h +1 -1
index c5adb5006361..de64389b0815 100644
@@ -270,7 +270,7 @@ static inline void arch_interrupt_exit_prepare(struct pt_regs *regs)
  	}
 
  	/* irqentry_exit expects to be called with interrupts disabled */
-	local_irq_disable();
+	hard_irq_disable();
  }
 
  static inline void arch_interrupt_async_enter_prepare(struct pt_regs *regs)
#5 lore raw

On Fri, Aug 14, 2026 at 11:06:43AM +0530, Shrikanth Hegde wrote:
[...]

Quoted 7 lines

This is good explanation, but what i not understanding is,

why pattern of setting PACA_IRQ_HARD_DIS changed?
Previous code at interrupt_exit_kernel_prepare which did local_irq_disable too.

Please check where was PACA_IRQ_HARD_DIS set without GENERIC_ENTRY which was preventing this from happening?

That's an excellent point. I think we are looking at this incorrectly, If this is related to the nested soft replay interrupt than this whole dynamics changes, local_irq_disable will work for the actual interrupt but when we go nested something is telling that the nested interrupt that interrupts are enabled. Let me look at this more deeply.

Regards,
Mukesh

#6 lore raw
Quoted 10 lines · Mukesh Kumar Chaurasiya

On Fri, Aug 14, 2026 at 11:06:43AM +0530, Shrikanth Hegde wrote:
[...]

Quoted 7 lines

This is good explanation, but what i not understanding is,

why pattern of setting PACA_IRQ_HARD_DIS changed?
Previous code at interrupt_exit_kernel_prepare which did local_irq_disable too.

Please check where was PACA_IRQ_HARD_DIS set without GENERIC_ENTRY which was preventing this from happening?

Quoted 8 lines

That's an excellent point. I think we are looking at this incorrectly, If this is related to the nested soft replay interrupt than this whole dynamics changes, local_irq_disable will work for the actual interrupt but when we go nested something is telling that the nested interrupt that interrupts are enabled. Let me look at this more deeply.

Regards,
Mukesh

Hey Venkat,

Can you try this diff,

arch/powerpc/kernel/irq_64.c +1 -1
index d5c48d1b0a31..2fbfebef74fe 100644
@@ -117,7 +117,7 @@ static __no_kcsan void __replay_soft_interrupts(void)
 	local_paca->irq_happened |= PACA_IRQ_REPLAYING;
 
 	ppc_save_regs(&regs);
-	regs.softe = IRQS_ENABLED;
+	regs.softe = IRQS_DISABLED;
 	regs.msr |= MSR_EE;
 
 	/*
#7 lore raw
Quoted 28 lines · Mukesh Kumar Chaurasiya
Quoted 9 lines · Mukesh Kumar Chaurasiya

On Fri, Aug 14, 2026 at 11:06:43AM +0530, Shrikanth Hegde wrote:
[...]

Quoted 7 lines

This is good explanation, but what i not understanding is,

why pattern of setting PACA_IRQ_HARD_DIS changed?
Previous code at interrupt_exit_kernel_prepare which did local_irq_disable too.

Please check where was PACA_IRQ_HARD_DIS set without GENERIC_ENTRY which was preventing this from happening?

Quoted 8 lines

That's an excellent point. I think we are looking at this incorrectly, If this is related to the nested soft replay interrupt than this whole dynamics changes, local_irq_disable will work for the actual interrupt but when we go nested something is telling that the nested interrupt that interrupts are enabled. Let me look at this more deeply.

Regards,
Mukesh

Hey Venkat,

Can you try this diff,

arch/powerpc/kernel/irq_64.c +0 -0
index d5c48d1b0a31..2fbfebef74fe 100644

arch/powerpc/kernel/irq_64.c static __no_kcsan void __replay_soft_interrupts(void) · line 123

Quoted 8 lines
arch/powerpc/kernel/irq_64.c +1 -1
@@ -117,7 +117,7 @@ static __no_kcsan void __replay_soft_interrupts(void)
  	local_paca->irq_happened |= PACA_IRQ_REPLAYING;
 
  	ppc_save_regs(&regs);
-	regs.softe = IRQS_ENABLED;
+	regs.softe = IRQS_DISABLED;
  	regs.msr |= MSR_EE;
 
  	/*

Hello Mukesh,

With the proposed change, I still the issue.

[    0.742633] ------------[ cut here ]------------
[    0.742633] WARNING: arch/powerpc/kernel/irq_64.c:75 at __replay_soft_interrupts+0x104/0x220, CPU#6: swapper/0/1 [    0.742637] Modules linked in:
[    0.742638] CPU: 6 UID: 0 PID: 1 Comm: swapper/0 Tainted: G   W           7.2.0-rc7-dirty #25 PREEMPT
[    0.742639] Tainted: [W]=WARN
[    0.742640] Hardware name: IBM,9080-HEX Power11 (architected) 0x820200 0xf000007 of:IBM,FW1110.01 (NH1110_069) hv:phyp pSeries [    0.742641] NIP:  c00000000003a384 LR: c00000000003a360 CTR: c000000000029f00
[    0.742642] REGS: c000000009757410 TRAP: 0700   Tainted: G W            (7.2.0-rc7-dirty)
[    0.742643] MSR:  8000000002021033 <SF,VEC,ME,IR,DR,RI,LE> CR: 88024824  XER: 00000003
[    0.742646] CFAR: c00000000002a708 IRQMASK: 1
[    0.742646] GPR00: c00000000003a360 c0000000097576b0 c000000001b28100 c0000000097576d8
[    0.742646] GPR04: 0000000000000000 0000000000000001 c00000000968d200 c0000000096eb200
[    0.742646] GPR08: 0000000000000001 0000000000000041 0000000000000001 000000002c1b8100
[    0.742646] GPR12: c000000000029f00 c0000005ffff8f00 0000000000000000 c00000000968d200
[    0.742646] GPR16: c000000002f3aaf8 0000000000000000 c00000000284e138 0000000000000000
[    0.742646] GPR20: c00000000303aaf8 c00000000303ade8 0000000000000001 0000000000000000
[    0.742646] GPR24: c00000000303ae08 0000000000000000 0000000000000000 0000000000000002
[    0.742646] GPR28: 0000000000000003 fcffffffffffffff fcffffffffffffff c00000000968d200
[    0.742662] NIP [c00000000003a384] __replay_soft_interrupts+0x104/0x220
[    0.742665] LR [c00000000003a360] __replay_soft_interrupts+0xe0/0x220
[    0.742668] Call Trace:
[    0.742668] [c0000000097576b0] [c00000000003a360] __replay_soft_interrupts+0xe0/0x220 (unreliable)
[    0.742672] [c000000009757860] [c00000000003a694] arch_local_irq_restore+0x1f4/0x2d8
[    0.742675] [c0000000097578b0] [c0000000002face0] console_flush_all+0x3ec/0x558
[    0.742676] [c000000009757a00] [c0000000002faee4] console_unlock+0x98/0x1d8
[    0.742678] [c000000009757a80] [c0000000002fc698] vprintk_emit+0x2c0/0x3f4
[    0.742680] [c000000009757af0] [c0000000002fdc70] vprintk+0x30/0x7c
[    0.742682] [c000000009757b10] [c0000000002f6c78] _printk+0x3c/0x50
[    0.742683] [c000000009757b30] [c00000000205d814] load_system_certificate_list+0x24/0x5c
[    0.742685] [c000000009757b90] [c00000000001111c] do_one_initcall+0x5c/0x3f0
[    0.742687] [c000000009757c70] [c00000000200684c] do_initcalls+0x11c/0x270
[    0.742689] [c000000009757d30] [c000000002006c78] kernel_init_freeable+0x228/0x3c0
[    0.742690] [c000000009757de0] [c000000000011600] kernel_init+0x30/0x260

[    0.742692] [c000000009757e50] [c00000000000df7c] ret_from_kernel_user_thread+0x14/0x1c

Regards,

Venkat.

#8 lore raw
Quoted 1 line · Venkat Rao Bagalkote

[...]

Quoted 77 lines
Quoted 9 lines

Hey Venkat,

Can you try this diff,

arch/powerpc/kernel/irq_64.c +0 -0
index d5c48d1b0a31..2fbfebef74fe 100644

arch/powerpc/kernel/irq_64.c static __no_kcsan void __replay_soft_interrupts(void) · line 122

Quoted 7 lines
arch/powerpc/kernel/irq_64.c +1 -1
@@ -117,7 +117,7 @@ static __no_kcsan void __replay_soft_interrupts(void)
  	local_paca->irq_happened |= PACA_IRQ_REPLAYING;
  	ppc_save_regs(&regs);
-	regs.softe = IRQS_ENABLED;
+	regs.softe = IRQS_DISABLED;
  	regs.msr |= MSR_EE;
  	/*
 

Hello Mukesh,

With the proposed change, I still the issue.

[    0.742633] ------------[ cut here ]------------
[    0.742633] WARNING: arch/powerpc/kernel/irq_64.c:75 at __replay_soft_interrupts+0x104/0x220, CPU#6: swapper/0/1
[    0.742637] Modules linked in:
[    0.742638] CPU: 6 UID: 0 PID: 1 Comm: swapper/0 Tainted: G   W 7.2.0-rc7-dirty #25 PREEMPT
[    0.742639] Tainted: [W]=WARN
[    0.742640] Hardware name: IBM,9080-HEX Power11 (architected) 0x820200 0xf000007 of:IBM,FW1110.01 (NH1110_069) hv:phyp pSeries
[    0.742641] NIP:  c00000000003a384 LR: c00000000003a360 CTR:
c000000000029f00
[    0.742642] REGS: c000000009757410 TRAP: 0700   Tainted: G W (7.2.0-rc7-dirty)
[    0.742643] MSR:  8000000002021033 <SF,VEC,ME,IR,DR,RI,LE> CR: 88024824
XER: 00000003
[    0.742646] CFAR: c00000000002a708 IRQMASK: 1
[    0.742646] GPR00: c00000000003a360 c0000000097576b0 c000000001b28100 c0000000097576d8
[    0.742646] GPR04: 0000000000000000 0000000000000001 c00000000968d200 c0000000096eb200
[    0.742646] GPR08: 0000000000000001 0000000000000041 0000000000000001 000000002c1b8100
[    0.742646] GPR12: c000000000029f00 c0000005ffff8f00 0000000000000000 c00000000968d200
[    0.742646] GPR16: c000000002f3aaf8 0000000000000000 c00000000284e138 0000000000000000
[    0.742646] GPR20: c00000000303aaf8 c00000000303ade8 0000000000000001 0000000000000000
[    0.742646] GPR24: c00000000303ae08 0000000000000000 0000000000000000 0000000000000002
[    0.742646] GPR28: 0000000000000003 fcffffffffffffff fcffffffffffffff c00000000968d200
[    0.742662] NIP [c00000000003a384] __replay_soft_interrupts+0x104/0x220 [    0.742665] LR [c00000000003a360] __replay_soft_interrupts+0xe0/0x220 [    0.742668] Call Trace:
[    0.742668] [c0000000097576b0] [c00000000003a360] __replay_soft_interrupts+0xe0/0x220 (unreliable)
[    0.742672] [c000000009757860] [c00000000003a694] arch_local_irq_restore+0x1f4/0x2d8
[    0.742675] [c0000000097578b0] [c0000000002face0] console_flush_all+0x3ec/0x558
[    0.742676] [c000000009757a00] [c0000000002faee4] console_unlock+0x98/0x1d8
[    0.742678] [c000000009757a80] [c0000000002fc698]

vprintk_emit+0x2c0/0x3f4
[    0.742680] [c000000009757af0] [c0000000002fdc70] vprintk+0x30/0x7c
[    0.742682] [c000000009757b10] [c0000000002f6c78] _printk+0x3c/0x50

[    0.742683] [c000000009757b30] [c00000000205d814] load_system_certificate_list+0x24/0x5c
[    0.742685] [c000000009757b90] [c00000000001111c] do_one_initcall+0x5c/0x3f0
[    0.742687] [c000000009757c70] [c00000000200684c] do_initcalls+0x11c/0x270
[    0.742689] [c000000009757d30] [c000000002006c78]

kernel_init_freeable+0x228/0x3c0
[    0.742690] [c000000009757de0] [c000000000011600] kernel_init+0x30/0x260
Quoted 8 lines

[    0.742692] [c000000009757e50] [c00000000000df7c] ret_from_kernel_user_thread+0x14/0x1c

Regards,

Venkat.

Hey Venkat,
Thanks for verifying.

The regs.softe = IRQS_DISABLED change only prevents irqentry_exit_cond_resched() preemption path from running — it does nothing about PACA_IRQ_HARD_DIS being cleared by the handler and not restored. hard_irq_disable() fixes both: it atomically sets PACA_IRQ_HARD_DIS, sets irq_soft_mask = IRQS_ALL_DISABLED, and clears MSR[EE], leaving a fully consistent disabled state that next_interrupt() expects, regardless of which entry path (idle wakeup or irqrestore) triggered the replay.

To answer's shrikanth question
it worked before because there was literally no interrupt-disabling code on the handler exit path during replay. The new GENERIC_ENTRY infrastructure brought a real exit sequence with it, and that sequence needs hard_irq_disable() — not local_irq_disable() — to keep PPC64's two-level interrupt state (irq_soft_mask + PACA_IRQ_HARD_DIS) consistent.

Hey Madhavan,
I think we are ok with this patch if there are no more objections from anyone.

Regards,
Mukesh

#9 lore raw
Quoted 107 lines · Mukesh Kumar Chaurasiya
Quoted 1 line · Venkat Rao Bagalkote

[...]

Quoted 77 lines
Quoted 9 lines

Hey Venkat,

Can you try this diff,

arch/powerpc/kernel/irq_64.c +0 -0
index d5c48d1b0a31..2fbfebef74fe 100644

arch/powerpc/kernel/irq_64.c static __no_kcsan void __replay_soft_interrupts(void) · line 122

Quoted 7 lines
arch/powerpc/kernel/irq_64.c +1 -1
@@ -117,7 +117,7 @@ static __no_kcsan void __replay_soft_interrupts(void)
   	local_paca->irq_happened |= PACA_IRQ_REPLAYING;
   	ppc_save_regs(&regs);
-	regs.softe = IRQS_ENABLED;
+	regs.softe = IRQS_DISABLED;
   	regs.msr |= MSR_EE;
   	/*
 

Hello Mukesh,

With the proposed change, I still the issue.

[    0.742633] ------------[ cut here ]------------
[    0.742633] WARNING: arch/powerpc/kernel/irq_64.c:75 at __replay_soft_interrupts+0x104/0x220, CPU#6: swapper/0/1
[    0.742637] Modules linked in:
[    0.742638] CPU: 6 UID: 0 PID: 1 Comm: swapper/0 Tainted: G   W
 7.2.0-rc7-dirty #25 PREEMPT
[    0.742639] Tainted: [W]=WARN
[    0.742640] Hardware name: IBM,9080-HEX Power11 (architected) 0x820200 0xf000007 of:IBM,FW1110.01 (NH1110_069) hv:phyp pSeries
[    0.742641] NIP:  c00000000003a384 LR: c00000000003a360 CTR:
c000000000029f00
[    0.742642] REGS: c000000009757410 TRAP: 0700   Tainted: G W (7.2.0-rc7-dirty)
[    0.742643] MSR:  8000000002021033 <SF,VEC,ME,IR,DR,RI,LE> CR: 88024824
XER: 00000003
[    0.742646] CFAR: c00000000002a708 IRQMASK: 1
[    0.742646] GPR00: c00000000003a360 c0000000097576b0 c000000001b28100 c0000000097576d8
[    0.742646] GPR04: 0000000000000000 0000000000000001 c00000000968d200 c0000000096eb200
[    0.742646] GPR08: 0000000000000001 0000000000000041 0000000000000001 000000002c1b8100
[    0.742646] GPR12: c000000000029f00 c0000005ffff8f00 0000000000000000 c00000000968d200
[    0.742646] GPR16: c000000002f3aaf8 0000000000000000 c00000000284e138 0000000000000000
[    0.742646] GPR20: c00000000303aaf8 c00000000303ade8 0000000000000001 0000000000000000
[    0.742646] GPR24: c00000000303ae08 0000000000000000 0000000000000000 0000000000000002
[    0.742646] GPR28: 0000000000000003 fcffffffffffffff fcffffffffffffff c00000000968d200
[    0.742662] NIP [c00000000003a384] __replay_soft_interrupts+0x104/0x220 [    0.742665] LR [c00000000003a360] __replay_soft_interrupts+0xe0/0x220 [    0.742668] Call Trace:
[    0.742668] [c0000000097576b0] [c00000000003a360] __replay_soft_interrupts+0xe0/0x220 (unreliable)
[    0.742672] [c000000009757860] [c00000000003a694] arch_local_irq_restore+0x1f4/0x2d8
[    0.742675] [c0000000097578b0] [c0000000002face0] console_flush_all+0x3ec/0x558
[    0.742676] [c000000009757a00] [c0000000002faee4] console_unlock+0x98/0x1d8
[    0.742678] [c000000009757a80] [c0000000002fc698]

vprintk_emit+0x2c0/0x3f4
[    0.742680] [c000000009757af0] [c0000000002fdc70] vprintk+0x30/0x7c
[    0.742682] [c000000009757b10] [c0000000002f6c78] _printk+0x3c/0x50

[    0.742683] [c000000009757b30] [c00000000205d814] load_system_certificate_list+0x24/0x5c
[    0.742685] [c000000009757b90] [c00000000001111c] do_one_initcall+0x5c/0x3f0
[    0.742687] [c000000009757c70] [c00000000200684c] do_initcalls+0x11c/0x270
[    0.742689] [c000000009757d30] [c000000002006c78]

kernel_init_freeable+0x228/0x3c0
[    0.742690] [c000000009757de0] [c000000000011600] kernel_init+0x30/0x260
Quoted 8 lines

[    0.742692] [c000000009757e50] [c00000000000df7c] ret_from_kernel_user_thread+0x14/0x1c

Regards,

Venkat.

Hey Venkat,
Thanks for verifying.

The regs.softe = IRQS_DISABLED change only prevents irqentry_exit_cond_resched() preemption path from running — it does nothing about PACA_IRQ_HARD_DIS being cleared by the handler and not restored. hard_irq_disable() fixes both: it atomically sets PACA_IRQ_HARD_DIS, sets irq_soft_mask = IRQS_ALL_DISABLED, and clears MSR[EE], leaving a fully consistent disabled state that next_interrupt() expects, regardless of which entry path (idle wakeup or irqrestore) triggered the replay.

To answer's shrikanth question
it worked before because there was literally no interrupt-disabling code on the handler exit path during replay. The new GENERIC_ENTRY infrastructure brought a real exit sequence with it, and that sequence needs hard_irq_disable() — not local_irq_disable() — to keep PPC64's two-level interrupt state (irq_soft_mask + PACA_IRQ_HARD_DIS) consistent.

Quoted 4 lines

Hey Madhavan,
I think we are ok with this patch if there are no more objections from anyone.

Venkat,

Can you collect vmcore with panic on warning enabled?

It maybe nested replay of soft interrupts which maybe causing problems here. We should see the regs state.

Quoted 2 lines

Regards,
Mukesh

#10 lore raw
Quoted 9 lines · Shrikanth Hegde

Can you collect vmcore with panic on warning enabled?

It maybe nested replay of soft interrupts which maybe causing problems here. We should see the regs state.

Quoted 2 lines

Regards,
Mukesh

Hey Venkat,

I have identified the root cause for this and discussed it with Shrikanth also. We have agreed that the fix we have now is a proper fix and why we got this issue now and not earlier. I'll send out a new revision for this.

Regards,
Mukesh