Skip to contentSkip to conversation

Generic IRQ entry/exit support for powerpc

PATCH Superseded Mukesh Kumar Chaurasiya 5 participants 17 messages Dec 29, 2025 – Jan 8

Subsystems (2)

v3 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

v3 cover Generic IRQ entry/exit support for powerpc

Adding support for the generic irq entry/exit handling for PowerPC. The goal is to bring PowerPC in line with other architectures that already use the common irq entry infrastructure, reducing duplicated code and making it easier to share future changes in entry/exit paths.

This is slightly tested of ppc64le and ppc32.

The performance benchmarks are below:

perf bench syscall usec/op

| Syscall | Base       | New        | change % |
| ------- | ---------- | ---------- | -------- |
| basic   | 0.173212   | 0.133294   | -23.05   |
| execve  | 363.176190 | 357.148150 | -1.66    |
| fork    | 853.657880 | 840.268800 | -1.57    |
| getpgid | 0.174832   | 0.135372   | -22.57   |

perf bench syscall ops/sec

| Syscall | Base    | New     | change % |
| ------- | ------- | ------- | -------- |
| basic   | 6006021 | 7502236 | +24.91   |
| execve  | 2753    | 2800    | +1.71    |
| fork    | 1171    | 1190    | +1.62    |
| getpgid | 5942117 | 7387040 | +24.32   |

IPI latency benchmark

| Metric         | Base (ns)     | Test (ns)     | change % |
| -------------- | ------------- | ------------- | -------- |
| Dry-run        | 206652.45     | 209317.37     | +1.29    |
| Self-IPI       | 3567895.23    | 3590444.77    | +0.63    |
| Normal IPI     | 148570416.17  | 148564173.40  | -0.00    |
| Broadcast IPI  | 4033489673.38 | 4007319512.62 | -0.65    |
| Broadcast lock | 4011023005.48 | 4010267885.93 | -0.02    |

Thats very close to performance earlier with arch specific handling.

Tests done:
- Build and boot on ppc64le pseries.
- Build and boot on ppc64le powernv8 powernv9 powernv10.
- Build and boot on ppc32.
- Performance benchmark done with perf syscall basic on pseries.

Changelog:

V2 -> V3
- #ifdef CONFIG_GENERIC_IRQ_ENTRY removed from unnecessary places
- Some functions made __always_inline
- pt_regs padding changed to match 16byte interrupt stack alignment
- And some cosmetic changes from reviews from earlier patch
V2: https://lore.kernel.org/all/[email protected]/

V1 -> V2
- Fix an issue where context tracking was showing warnings for incorrect context
V1: https://lore.kernel.org/all/[email protected]/

RFC -> PATCH V1
- Fix for ppc32 spitting out kuap lock warnings.
- ppc64le powernv8 crash fix.
- Review comments incorporated from previous RFC.
RFC https://lore.kernel.org/all/[email protected]/

Mukesh Kumar Chaurasiya (8):
powerpc: rename arch_irq_disabled_regs
powerpc: Prepare to build with generic entry/exit framework
powerpc: introduce arch_enter_from_user_mode
powerpc: Introduce syscall exit arch functions
powerpc: add exit_flags field in pt_regs
powerpc: Prepare for IRQ entry exit
powerpc: Enable IRQ generic entry/exit path.
powerpc: Enable Generic Entry/Exit for syscalls.

 arch/powerpc/Kconfig                    |   2 +
 arch/powerpc/include/asm/entry-common.h | 533 ++++++++++++++++++++++++
 arch/powerpc/include/asm/hw_irq.h       |   4 +-
 arch/powerpc/include/asm/interrupt.h    | 386 +++--------------
 arch/powerpc/include/asm/ptrace.h       |   6 +-
 arch/powerpc/include/asm/signal.h       |   1 -
 arch/powerpc/include/asm/stacktrace.h   |   6 +
 arch/powerpc/include/asm/syscall.h      |   5 +
 arch/powerpc/include/asm/thread_info.h  |   1 +
 arch/powerpc/include/uapi/asm/ptrace.h  |  14 +-
 arch/powerpc/kernel/interrupt.c         | 255 ++----------
 arch/powerpc/kernel/ptrace/ptrace.c     | 142 +------
 arch/powerpc/kernel/signal.c            |  25 +-
 arch/powerpc/kernel/syscall.c           | 119 +-----
 arch/powerpc/kernel/traps.c             |   2 +-
 arch/powerpc/kernel/watchdog.c          |   2 +-
 arch/powerpc/perf/core-book3s.c         |   2 +-
 17 files changed, 681 insertions(+), 824 deletions(-)
 create mode 100644 arch/powerpc/include/asm/entry-common.h
Signature · 1 line
2.52.0
#1 lore raw

PATCH v3 1/8 powerpc: rename arch_irq_disabled_regs

From: Mukesh Kumar Chaurasiya <[email protected]>

Rename arch_irq_disabled_regs() to regs_irqs_disabled() to align with the naming used in the generic irqentry framework. This makes the function available for use both in the PowerPC architecture code and in the common entry/exit paths shared with other architectures.

This is a preparatory change for enabling the generic irqentry framework on PowerPC.

---
 arch/powerpc/include/asm/hw_irq.h    |  4 ++--
 arch/powerpc/include/asm/interrupt.h | 16 ++++++++--------
 arch/powerpc/kernel/interrupt.c      |  4 ++--
 arch/powerpc/kernel/syscall.c        |  2 +-
 arch/powerpc/kernel/traps.c          |  2 +-
 arch/powerpc/kernel/watchdog.c       |  2 +-
 arch/powerpc/perf/core-book3s.c      |  2 +-
 7 files changed, 16 insertions(+), 16 deletions(-)
arch/powerpc/include/asm/hw_irq.h +2 -2
index 1078ba88efaf..8dfe36b442a5 100644
@@ -393,7 +393,7 @@ static inline void do_hard_irq_enable(void)
 	__hard_irq_enable();
 }
 
-static inline bool arch_irq_disabled_regs(struct pt_regs *regs)
+static inline bool regs_irqs_disabled(struct pt_regs *regs)
 {
 	return (regs->softe & IRQS_DISABLED);
 }
@@ -466,7 +466,7 @@ static inline bool arch_irqs_disabled(void)
 
 #define hard_irq_disable()		arch_local_irq_disable()
 
-static inline bool arch_irq_disabled_regs(struct pt_regs *regs)
+static inline bool regs_irqs_disabled(struct pt_regs *regs)
 {
 	return !(regs->msr & MSR_EE);
 }
arch/powerpc/include/asm/interrupt.h +8 -8
index eb0e4a20b818..0e2cddf8bd21 100644
@@ -172,7 +172,7 @@ static inline void interrupt_enter_prepare(struct pt_regs *regs)
 	/* Enable MSR[RI] early, to support kernel SLB and hash faults */
 #endif
 
-	if (!arch_irq_disabled_regs(regs))
+	if (!regs_irqs_disabled(regs))
 		trace_hardirqs_off();
 
 	if (user_mode(regs)) {
@@ -192,11 +192,11 @@ static inline void interrupt_enter_prepare(struct pt_regs *regs)
 			CT_WARN_ON(ct_state() != CT_STATE_KERNEL &&
 				   ct_state() != CT_STATE_IDLE);
 		INT_SOFT_MASK_BUG_ON(regs, is_implicit_soft_masked(regs));
-		INT_SOFT_MASK_BUG_ON(regs, arch_irq_disabled_regs(regs) &&
-					   search_kernel_restart_table(regs->nip));
+		INT_SOFT_MASK_BUG_ON(regs, regs_irqs_disabled(regs) &&
+				     search_kernel_restart_table(regs->nip));
 	}
-	INT_SOFT_MASK_BUG_ON(regs, !arch_irq_disabled_regs(regs) &&
-				   !(regs->msr & MSR_EE));
+	INT_SOFT_MASK_BUG_ON(regs, !regs_irqs_disabled(regs) &&
+			     !(regs->msr & MSR_EE));
 
 	booke_restore_dbcr0();
 }
@@ -298,7 +298,7 @@ static inline void interrupt_nmi_enter_prepare(struct pt_regs *regs, struct inte
 		 * Adjust regs->softe to be soft-masked if it had not been
 		 * reconcied (e.g., interrupt entry with MSR[EE]=0 but softe
 		 * not yet set disabled), or if it was in an implicit soft
-		 * masked state. This makes arch_irq_disabled_regs(regs)
+		 * masked state. This makes regs_irqs_disabled(regs)
 		 * behave as expected.
 		 */
 		regs->softe = IRQS_ALL_DISABLED;
@@ -372,7 +372,7 @@ static inline void interrupt_nmi_exit_prepare(struct pt_regs *regs, struct inter
 
 #ifdef CONFIG_PPC64
 #ifdef CONFIG_PPC_BOOK3S
-	if (arch_irq_disabled_regs(regs)) {
+	if (regs_irqs_disabled(regs)) {
 		unsigned long rst = search_kernel_restart_table(regs->nip);
 		if (rst)
 			regs_set_return_ip(regs, rst);
@@ -661,7 +661,7 @@ void replay_soft_interrupts(void);
 
 static inline void interrupt_cond_local_irq_enable(struct pt_regs *regs)
 {
-	if (!arch_irq_disabled_regs(regs))
+	if (!regs_irqs_disabled(regs))
 		local_irq_enable();
 }
 
arch/powerpc/kernel/interrupt.c +2 -2
index aea6f7e8e9c6..9ce2013e70ae 100644
@@ -347,7 +347,7 @@ notrace unsigned long interrupt_exit_user_prepare(struct pt_regs *regs)
 	unsigned long ret;
 
 	BUG_ON(regs_is_unrecoverable(regs));
-	BUG_ON(arch_irq_disabled_regs(regs));
+	BUG_ON(regs_irqs_disabled(regs));
 	CT_WARN_ON(ct_state() == CT_STATE_USER);
 
 	/*
@@ -396,7 +396,7 @@ notrace unsigned long interrupt_exit_kernel_prepare(struct pt_regs *regs)
 
 	local_irq_disable();
 
-	if (!arch_irq_disabled_regs(regs)) {
+	if (!regs_irqs_disabled(regs)) {
 		/* Returning to a kernel context with local irqs enabled. */
 		WARN_ON_ONCE(!(regs->msr & MSR_EE));
 again:
arch/powerpc/kernel/syscall.c +1 -1
index be159ad4b77b..9f03a6263fb4 100644
@@ -32,7 +32,7 @@ notrace long system_call_exception(struct pt_regs *regs, unsigned long r0)
 
 	BUG_ON(regs_is_unrecoverable(regs));
 	BUG_ON(!user_mode(regs));
-	BUG_ON(arch_irq_disabled_regs(regs));
+	BUG_ON(regs_irqs_disabled(regs));
 
 #ifdef CONFIG_PPC_PKEY
 	if (mmu_has_feature(MMU_FTR_PKEY)) {
arch/powerpc/kernel/traps.c +1 -1
index cb8e9357383e..629f2a2d4780 100644
@@ -1956,7 +1956,7 @@ DEFINE_INTERRUPT_HANDLER_RAW(performance_monitor_exception)
 	 * prevent hash faults on user addresses when reading callchains (and
 	 * looks better from an irq tracing perspective).
 	 */
-	if (IS_ENABLED(CONFIG_PPC64) && unlikely(arch_irq_disabled_regs(regs)))
+	if (IS_ENABLED(CONFIG_PPC64) && unlikely(regs_irqs_disabled(regs)))
 		performance_monitor_exception_nmi(regs);
 	else
 		performance_monitor_exception_async(regs);
arch/powerpc/kernel/watchdog.c +1 -1
index 2429cb1c7baa..6111cbbde069 100644
@@ -373,7 +373,7 @@ DEFINE_INTERRUPT_HANDLER_NMI(soft_nmi_interrupt)
 	u64 tb;
 
 	/* should only arrive from kernel, with irqs disabled */
-	WARN_ON_ONCE(!arch_irq_disabled_regs(regs));
+	WARN_ON_ONCE(!regs_irqs_disabled(regs));
 
 	if (!cpumask_test_cpu(cpu, &wd_cpus_enabled))
 		return 0;
arch/powerpc/perf/core-book3s.c +1 -1
index 8b0081441f85..f7518b7e3055 100644
@@ -2482,7 +2482,7 @@ static void __perf_event_interrupt(struct pt_regs *regs)
 	 * will trigger a PMI after waking up from idle. Since counter values are _not_
 	 * saved/restored in idle path, can lead to below "Can't find PMC" message.
 	 */
-	if (unlikely(!found) && !arch_irq_disabled_regs(regs))
+	if (unlikely(!found) && !regs_irqs_disabled(regs))
 		printk_ratelimited(KERN_WARNING "Can't find PMC that caused IRQ\n");
 
 	/*
Signature · 1 line
2.52.0
Reviewed-by on v3 1/8
#2 lore raw

Re: [PATCH v3 1/8] powerpc: rename arch_irq_disabled_regs

Quoted 12 lines · Mukesh Kumar Chaurasiya

From: Mukesh Kumar Chaurasiya <[email protected]>

Rename arch_irq_disabled_regs() to regs_irqs_disabled() to align with the naming used in the generic irqentry framework. This makes the function available for use both in the PowerPC architecture code and in the common entry/exit paths shared with other architectures.

This is a preparatory change for enabling the generic irqentry framework on PowerPC.

Quoted 9 lines
---
 arch/powerpc/include/asm/hw_irq.h    |  4 ++--
 arch/powerpc/include/asm/interrupt.h | 16 ++++++++--------
 arch/powerpc/kernel/interrupt.c      |  4 ++--
 arch/powerpc/kernel/syscall.c        |  2 +-
 arch/powerpc/kernel/traps.c          |  2 +-
 arch/powerpc/kernel/watchdog.c       |  2 +-
 arch/powerpc/perf/core-book3s.c      |  2 +-
 7 files changed, 16 insertions(+), 16 deletions(-)
Quoted 153 lines
arch/powerpc/include/asm/hw_irq.h +2 -2
index 1078ba88efaf..8dfe36b442a5 100644
@@ -393,7 +393,7 @@ static inline void do_hard_irq_enable(void)
 	__hard_irq_enable();
 }
 
-static inline bool arch_irq_disabled_regs(struct pt_regs *regs)
+static inline bool regs_irqs_disabled(struct pt_regs *regs)
 {
 	return (regs->softe & IRQS_DISABLED);
 }
@@ -466,7 +466,7 @@ static inline bool arch_irqs_disabled(void)
 
 #define hard_irq_disable()		arch_local_irq_disable()
 
-static inline bool arch_irq_disabled_regs(struct pt_regs *regs)
+static inline bool regs_irqs_disabled(struct pt_regs *regs)
 {
 	return !(regs->msr & MSR_EE);
 }
arch/powerpc/include/asm/interrupt.h +8 -8
index eb0e4a20b818..0e2cddf8bd21 100644
@@ -172,7 +172,7 @@ static inline void interrupt_enter_prepare(struct pt_regs *regs)
 	/* Enable MSR[RI] early, to support kernel SLB and hash faults */
 #endif
 
-	if (!arch_irq_disabled_regs(regs))
+	if (!regs_irqs_disabled(regs))
 		trace_hardirqs_off();
 
 	if (user_mode(regs)) {
@@ -192,11 +192,11 @@ static inline void interrupt_enter_prepare(struct pt_regs *regs)
 			CT_WARN_ON(ct_state() != CT_STATE_KERNEL &&
 				   ct_state() != CT_STATE_IDLE);
 		INT_SOFT_MASK_BUG_ON(regs, is_implicit_soft_masked(regs));
-		INT_SOFT_MASK_BUG_ON(regs, arch_irq_disabled_regs(regs) &&
-					   search_kernel_restart_table(regs->nip));
+		INT_SOFT_MASK_BUG_ON(regs, regs_irqs_disabled(regs) &&
+				     search_kernel_restart_table(regs->nip));
 	}
-	INT_SOFT_MASK_BUG_ON(regs, !arch_irq_disabled_regs(regs) &&
-				   !(regs->msr & MSR_EE));
+	INT_SOFT_MASK_BUG_ON(regs, !regs_irqs_disabled(regs) &&
+			     !(regs->msr & MSR_EE));
 
 	booke_restore_dbcr0();
 }
@@ -298,7 +298,7 @@ static inline void interrupt_nmi_enter_prepare(struct pt_regs *regs, struct inte
 		 * Adjust regs->softe to be soft-masked if it had not been
 		 * reconcied (e.g., interrupt entry with MSR[EE]=0 but softe
 		 * not yet set disabled), or if it was in an implicit soft
-		 * masked state. This makes arch_irq_disabled_regs(regs)
+		 * masked state. This makes regs_irqs_disabled(regs)
 		 * behave as expected.
 		 */
 		regs->softe = IRQS_ALL_DISABLED;
@@ -372,7 +372,7 @@ static inline void interrupt_nmi_exit_prepare(struct pt_regs *regs, struct inter
 
 #ifdef CONFIG_PPC64
 #ifdef CONFIG_PPC_BOOK3S
-	if (arch_irq_disabled_regs(regs)) {
+	if (regs_irqs_disabled(regs)) {
 		unsigned long rst = search_kernel_restart_table(regs->nip);
 		if (rst)
 			regs_set_return_ip(regs, rst);
@@ -661,7 +661,7 @@ void replay_soft_interrupts(void);
 
 static inline void interrupt_cond_local_irq_enable(struct pt_regs *regs)
 {
-	if (!arch_irq_disabled_regs(regs))
+	if (!regs_irqs_disabled(regs))
 		local_irq_enable();
 }
 
arch/powerpc/kernel/interrupt.c +2 -2
index aea6f7e8e9c6..9ce2013e70ae 100644
@@ -347,7 +347,7 @@ notrace unsigned long interrupt_exit_user_prepare(struct pt_regs *regs)
 	unsigned long ret;
 
 	BUG_ON(regs_is_unrecoverable(regs));
-	BUG_ON(arch_irq_disabled_regs(regs));
+	BUG_ON(regs_irqs_disabled(regs));
 	CT_WARN_ON(ct_state() == CT_STATE_USER);
 
 	/*
@@ -396,7 +396,7 @@ notrace unsigned long interrupt_exit_kernel_prepare(struct pt_regs *regs)
 
 	local_irq_disable();
 
-	if (!arch_irq_disabled_regs(regs)) {
+	if (!regs_irqs_disabled(regs)) {
 		/* Returning to a kernel context with local irqs enabled. */
 		WARN_ON_ONCE(!(regs->msr & MSR_EE));
 again:
arch/powerpc/kernel/syscall.c +1 -1
index be159ad4b77b..9f03a6263fb4 100644
@@ -32,7 +32,7 @@ notrace long system_call_exception(struct pt_regs *regs, unsigned long r0)
 
 	BUG_ON(regs_is_unrecoverable(regs));
 	BUG_ON(!user_mode(regs));
-	BUG_ON(arch_irq_disabled_regs(regs));
+	BUG_ON(regs_irqs_disabled(regs));
 
 #ifdef CONFIG_PPC_PKEY
 	if (mmu_has_feature(MMU_FTR_PKEY)) {
arch/powerpc/kernel/traps.c +1 -1
index cb8e9357383e..629f2a2d4780 100644
@@ -1956,7 +1956,7 @@ DEFINE_INTERRUPT_HANDLER_RAW(performance_monitor_exception)
 	 * prevent hash faults on user addresses when reading callchains (and
 	 * looks better from an irq tracing perspective).
 	 */
-	if (IS_ENABLED(CONFIG_PPC64) && unlikely(arch_irq_disabled_regs(regs)))
+	if (IS_ENABLED(CONFIG_PPC64) && unlikely(regs_irqs_disabled(regs)))
 		performance_monitor_exception_nmi(regs);
 	else
 		performance_monitor_exception_async(regs);
arch/powerpc/kernel/watchdog.c +1 -1
index 2429cb1c7baa..6111cbbde069 100644
@@ -373,7 +373,7 @@ DEFINE_INTERRUPT_HANDLER_NMI(soft_nmi_interrupt)
 	u64 tb;
 
 	/* should only arrive from kernel, with irqs disabled */
-	WARN_ON_ONCE(!arch_irq_disabled_regs(regs));
+	WARN_ON_ONCE(!regs_irqs_disabled(regs));
 
 	if (!cpumask_test_cpu(cpu, &wd_cpus_enabled))
 		return 0;
arch/powerpc/perf/core-book3s.c +1 -1
index 8b0081441f85..f7518b7e3055 100644
@@ -2482,7 +2482,7 @@ static void __perf_event_interrupt(struct pt_regs *regs)
 	 * will trigger a PMI after waking up from idle. Since counter values are _not_
 	 * saved/restored in idle path, can lead to below "Can't find PMC" message.
 	 */
-	if (unlikely(!found) && !arch_irq_disabled_regs(regs))
+	if (unlikely(!found) && !regs_irqs_disabled(regs))
 		printk_ratelimited(KERN_WARNING "Can't find PMC that caused IRQ\n");
 
 	/*
#3 lore raw

PATCH v3 2/8 powerpc: Prepare to build with generic entry/exit framework

In reply to Mukesh Kumar Chaurasiya

From: Mukesh Kumar Chaurasiya <[email protected]>

This patch introduces preparatory changes needed to support building PowerPC with the generic entry/exit (irqentry) framework.

The following infrastructure updates are added:
- Add a syscall_work field to struct thread_info to hold SYSCALL_WORK_* flags.
- Provide a stub implementation of arch_syscall_is_vdso_sigreturn(), returning false for now.
- Introduce on_thread_stack() helper to detect if the current stack pointer lies within the task’s kernel stack.

These additions enable later integration with the generic entry/exit infrastructure while keeping existing PowerPC behavior unchanged.

No functional change is intended in this patch.

---
 arch/powerpc/include/asm/entry-common.h | 8 ++++++++
 arch/powerpc/include/asm/stacktrace.h   | 6 ++++++
 arch/powerpc/include/asm/syscall.h      | 5 +++++
 arch/powerpc/include/asm/thread_info.h  | 1 +
 4 files changed, 20 insertions(+)
 create mode 100644 arch/powerpc/include/asm/entry-common.h
arch/powerpc/include/asm/entry-common.h new file +8 -0
new file mode 100644
index 000000000000..05ce0583b600
@@ -0,0 +1,8 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+#ifndef _ASM_PPC_ENTRY_COMMON_H
+#define _ASM_PPC_ENTRY_COMMON_H
+
+#include <asm/stacktrace.h>
+
+#endif /* _ASM_PPC_ENTRY_COMMON_H */
arch/powerpc/include/asm/stacktrace.h +6 -0
index 6149b53b3bc8..987f2e996262 100644
@@ -10,4 +10,10 @@
 
 void show_user_instructions(struct pt_regs *regs);
 
+static __always_inline bool on_thread_stack(void)
+{
+	return !(((unsigned long)(current->stack) ^ current_stack_pointer)
+			& ~(THREAD_SIZE - 1));
+}
+
 #endif /* _ASM_POWERPC_STACKTRACE_H */
arch/powerpc/include/asm/syscall.h +5 -0
index 4b3c52ed6e9d..834fcc4f7b54 100644
@@ -139,4 +139,9 @@ static inline int syscall_get_arch(struct task_struct *task)
 	else
 		return AUDIT_ARCH_PPC64;
 }
+
+static inline bool arch_syscall_is_vdso_sigreturn(struct pt_regs *regs)
+{
+	return false;
+}
 #endif	/* _ASM_SYSCALL_H */
arch/powerpc/include/asm/thread_info.h +1 -0
index b0f200aba2b3..9c8270354f0b 100644
@@ -57,6 +57,7 @@ struct thread_info {
 #ifdef CONFIG_SMP
 	unsigned int	cpu;
 #endif
+	unsigned long	syscall_work;		/* SYSCALL_WORK_ flags */
 	unsigned long	local_flags;		/* private flags for thread */
 #ifdef CONFIG_LIVEPATCH_64
 	unsigned long *livepatch_sp;
Signature · 1 line
2.52.0
#4 lore raw

PATCH v3 3/8 powerpc: introduce arch_enter_from_user_mode

In reply to Mukesh Kumar Chaurasiya

From: Mukesh Kumar Chaurasiya <[email protected]>

Implement the arch_enter_from_user_mode() hook required by the generic entry/exit framework. This helper prepares the CPU state when entering the kernel from userspace, ensuring correct handling of KUAP/KUEP, transactional memory, and debug register state.

As part of this change, move booke_load_dbcr0() from interrupt.c to interrupt.h so it can be used by the new helper without introducing cross-file dependencies.

This patch contains no functional changes, it is purely preparatory for enabling the generic syscall and interrupt entry path on PowerPC.

---
 arch/powerpc/include/asm/entry-common.h | 96 +++++++++++++++++++++++++
 arch/powerpc/include/asm/interrupt.h    | 22 ++++++
 arch/powerpc/kernel/interrupt.c         | 22 ------
 3 files changed, 118 insertions(+), 22 deletions(-)
arch/powerpc/include/asm/entry-common.h +96 -0
index 05ce0583b600..4bfbc806f5ad 100644
@@ -3,6 +3,102 @@
 #ifndef _ASM_PPC_ENTRY_COMMON_H
 #define _ASM_PPC_ENTRY_COMMON_H
 
+#include <asm/cputime.h>
+#include <asm/interrupt.h>
 #include <asm/stacktrace.h>
+#include <asm/tm.h>
+
+static __always_inline void arch_enter_from_user_mode(struct pt_regs *regs)
+{
+	kuap_lock();
+
+	if (IS_ENABLED(CONFIG_PPC_IRQ_SOFT_MASK_DEBUG))
+		BUG_ON(irq_soft_mask_return() != IRQS_ALL_DISABLED);
+
+	BUG_ON(regs_is_unrecoverable(regs));
+	BUG_ON(!user_mode(regs));
+	BUG_ON(regs_irqs_disabled(regs));
+
+#ifdef CONFIG_PPC_PKEY
+	if (mmu_has_feature(MMU_FTR_PKEY) && trap_is_syscall(regs)) {
+		unsigned long amr, iamr;
+		bool flush_needed = false;
+		/*
+		 * When entering from userspace we mostly have the AMR/IAMR
+		 * different from kernel default values. Hence don't compare.
+		 */
+		amr = mfspr(SPRN_AMR);
+		iamr = mfspr(SPRN_IAMR);
+		regs->amr  = amr;
+		regs->iamr = iamr;
+		if (mmu_has_feature(MMU_FTR_KUAP)) {
+			mtspr(SPRN_AMR, AMR_KUAP_BLOCKED);
+			flush_needed = true;
+		}
+		if (mmu_has_feature(MMU_FTR_BOOK3S_KUEP)) {
+			mtspr(SPRN_IAMR, AMR_KUEP_BLOCKED);
+			flush_needed = true;
+		}
+		if (flush_needed)
+			isync();
+	}
+#endif
+	kuap_assert_locked();
+	booke_restore_dbcr0();
+	account_cpu_user_entry();
+	account_stolen_time();
+
+	/*
+	 * This is not required for the syscall exit path, but makes the
+	 * stack frame look nicer. If this was initialised in the first stack
+	 * frame, or if the unwinder was taught the first stack frame always
+	 * returns to user with IRQS_ENABLED, this store could be avoided!
+	 */
+	irq_soft_mask_regs_set_state(regs, IRQS_ENABLED);
+
+	/*
+	 * If system call is called with TM active, set _TIF_RESTOREALL to
+	 * prevent RFSCV being used to return to userspace, because POWER9
+	 * TM implementation has problems with this instruction returning to
+	 * transactional state. Final register values are not relevant because
+	 * the transaction will be aborted upon return anyway. Or in the case
+	 * of unsupported_scv SIGILL fault, the return state does not much
+	 * matter because it's an edge case.
+	 */
+	if (IS_ENABLED(CONFIG_PPC_TRANSACTIONAL_MEM) &&
+	    unlikely(MSR_TM_TRANSACTIONAL(regs->msr)))
+		set_bits(_TIF_RESTOREALL, &current_thread_info()->flags);
+
+	/*
+	 * If the system call was made with a transaction active, doom it and
+	 * return without performing the system call. Unless it was an
+	 * unsupported scv vector, in which case it's treated like an illegal
+	 * instruction.
+	 */
+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
+	if (unlikely(MSR_TM_TRANSACTIONAL(regs->msr)) &&
+	    !trap_is_unsupported_scv(regs)) {
+		/* Enable TM in the kernel, and disable EE (for scv) */
+		hard_irq_disable();
+		mtmsr(mfmsr() | MSR_TM);
+
+		/* tabort, this dooms the transaction, nothing else */
+		asm volatile(".long 0x7c00071d | ((%0) << 16)"
+			     :: "r"(TM_CAUSE_SYSCALL | TM_CAUSE_PERSISTENT));
+
+		/*
+		 * Userspace will never see the return value. Execution will
+		 * resume after the tbegin. of the aborted transaction with the
+		 * checkpointed register state. A context switch could occur
+		 * or signal delivered to the process before resuming the
+		 * doomed transaction context, but that should all be handled
+		 * as expected.
+		 */
+		return;
+	}
+#endif /* CONFIG_PPC_TRANSACTIONAL_MEM */
+}
+
+#define arch_enter_from_user_mode arch_enter_from_user_mode
 
 #endif /* _ASM_PPC_ENTRY_COMMON_H */
arch/powerpc/include/asm/interrupt.h +22 -0
index 0e2cddf8bd21..5a1c83bc4dae 100644
@@ -138,6 +138,28 @@ static inline void nap_adjust_return(struct pt_regs *regs)
 #endif
 }
 
+static __always_inline void booke_load_dbcr0(void)
+{
+#ifdef CONFIG_PPC_ADV_DEBUG_REGS
+	unsigned long dbcr0 = current->thread.debug.dbcr0;
+
+	if (likely(!(dbcr0 & DBCR0_IDM)))
+		return;
+
+	/*
+	 * Check to see if the dbcr0 register is set up to debug.
+	 * Use the internal debug mode bit to do this.
+	 */
+	mtmsr(mfmsr() & ~MSR_DE);
+	if (IS_ENABLED(CONFIG_PPC32)) {
+		isync();
+		global_dbcr0[smp_processor_id()] = mfspr(SPRN_DBCR0);
+	}
+	mtspr(SPRN_DBCR0, dbcr0);
+	mtspr(SPRN_DBSR, -1);
+#endif
+}
+
 static inline void booke_restore_dbcr0(void)
 {
 #ifdef CONFIG_PPC_ADV_DEBUG_REGS
arch/powerpc/kernel/interrupt.c +0 -22
index 9ce2013e70ae..7265c4635c75 100644
@@ -78,28 +78,6 @@ static notrace __always_inline bool prep_irq_for_enabled_exit(bool restartable)
 	return true;
 }
 
-static notrace void booke_load_dbcr0(void)
-{
-#ifdef CONFIG_PPC_ADV_DEBUG_REGS
-	unsigned long dbcr0 = current->thread.debug.dbcr0;
-
-	if (likely(!(dbcr0 & DBCR0_IDM)))
-		return;
-
-	/*
-	 * Check to see if the dbcr0 register is set up to debug.
-	 * Use the internal debug mode bit to do this.
-	 */
-	mtmsr(mfmsr() & ~MSR_DE);
-	if (IS_ENABLED(CONFIG_PPC32)) {
-		isync();
-		global_dbcr0[smp_processor_id()] = mfspr(SPRN_DBCR0);
-	}
-	mtspr(SPRN_DBCR0, dbcr0);
-	mtspr(SPRN_DBSR, -1);
-#endif
-}
-
 static notrace void check_return_regs_valid(struct pt_regs *regs)
 {
 #ifdef CONFIG_PPC_BOOK3S_64
Signature · 1 line
2.52.0
#5 lore raw

PATCH v3 4/8 powerpc: Introduce syscall exit arch functions

In reply to Mukesh Kumar Chaurasiya

From: Mukesh Kumar Chaurasiya <[email protected]>

Add PowerPC-specific implementations of the generic syscall exit hooks used by the generic entry/exit framework:

- arch_exit_to_user_mode_work_prepare()
- arch_exit_to_user_mode_work()

These helpers handle user state restoration when returning from the kernel to userspace, including FPU/VMX/VSX state, transactional memory, KUAP restore, and per-CPU accounting.

Additionally, move check_return_regs_valid() from interrupt.c to interrupt.h so it can be shared by the new entry/exit logic.

No functional change is intended with this patch.

---
 arch/powerpc/include/asm/entry-common.h | 49 +++++++++++++++
 arch/powerpc/include/asm/interrupt.h    | 82 +++++++++++++++++++++++++
 arch/powerpc/kernel/interrupt.c         | 81 ------------------------
 3 files changed, 131 insertions(+), 81 deletions(-)
arch/powerpc/include/asm/entry-common.h +49 -0
index 4bfbc806f5ad..2684bc91e79b 100644
@@ -6,6 +6,7 @@
 #include <asm/cputime.h>
 #include <asm/interrupt.h>
 #include <asm/stacktrace.h>
+#include <asm/switch_to.h>
 #include <asm/tm.h>
 
 static __always_inline void arch_enter_from_user_mode(struct pt_regs *regs)
@@ -101,4 +102,52 @@ static __always_inline void arch_enter_from_user_mode(struct pt_regs *regs)
 
 #define arch_enter_from_user_mode arch_enter_from_user_mode
 
+static inline void arch_exit_to_user_mode_prepare(struct pt_regs *regs,
+						  unsigned long ti_work)
+{
+	unsigned long mathflags;
+
+	if (IS_ENABLED(CONFIG_PPC_BOOK3S_64) && IS_ENABLED(CONFIG_PPC_FPU)) {
+		if (IS_ENABLED(CONFIG_PPC_TRANSACTIONAL_MEM) &&
+		    unlikely((ti_work & _TIF_RESTORE_TM))) {
+			restore_tm_state(regs);
+		} else {
+			mathflags = MSR_FP;
+
+			if (cpu_has_feature(CPU_FTR_VSX))
+				mathflags |= MSR_VEC | MSR_VSX;
+			else if (cpu_has_feature(CPU_FTR_ALTIVEC))
+				mathflags |= MSR_VEC;
+
+			/*
+			 * If userspace MSR has all available FP bits set,
+			 * then they are live and no need to restore. If not,
+			 * it means the regs were given up and restore_math
+			 * may decide to restore them (to avoid taking an FP
+			 * fault).
+			 */
+			if ((regs->msr & mathflags) != mathflags)
+				restore_math(regs);
+		}
+	}
+
+	check_return_regs_valid(regs);
+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
+	local_paca->tm_scratch = regs->msr;
+#endif
+	/* Restore user access locks last */
+	kuap_user_restore(regs);
+}
+
+#define arch_exit_to_user_mode_prepare arch_exit_to_user_mode_prepare
+
+static __always_inline void arch_exit_to_user_mode(void)
+{
+	booke_load_dbcr0();
+
+	account_cpu_user_exit();
+}
+
+#define arch_exit_to_user_mode arch_exit_to_user_mode
+
 #endif /* _ASM_PPC_ENTRY_COMMON_H */
arch/powerpc/include/asm/interrupt.h +82 -0

94 lines not on this page · raw message

arch/powerpc/kernel/interrupt.c +0 -81

93 lines not on this page · raw message

Signature · 1 line
2.52.0
#6 lore raw

PATCH v3 5/8 powerpc: add exit_flags field in pt_regs

In reply to Mukesh Kumar Chaurasiya

From: Mukesh Kumar Chaurasiya <[email protected]>

Add a new field exit_flags in the pt_regs structure. This field will hold the flags set during interrupt or syscall execution that are required during exit to user mode.

Specifically, the TIF_RESTOREALL flag, stored in this field, helps the exit routine determine if any NVGPRs were modified and need to be restored before returning to userspace.

This addition ensures a clean and architecture-specific mechanism to track per-syscall or per-interrupt state transitions related to register restore.

Changes:
- Add exit_flags and __pt_regs_pad to maintain 16-byte stack alignment
- Update asm-offsets.c and ptrace.c for offset and validation
- Update PT_* constants in uapi header to reflect the new layout

---
 arch/powerpc/include/asm/ptrace.h      |  3 +++
 arch/powerpc/include/uapi/asm/ptrace.h | 14 +++++++++-----
 arch/powerpc/kernel/ptrace/ptrace.c    |  1 +
 3 files changed, 13 insertions(+), 5 deletions(-)
arch/powerpc/include/asm/ptrace.h +3 -0
index 94aa1de2b06e..2e741ea57b80 100644
@@ -53,6 +53,9 @@ struct pt_regs
 				unsigned long esr;
 			};
 			unsigned long result;
+			unsigned long exit_flags;
+			/* Maintain 16 byte interrupt stack alignment */
+			unsigned long __pt_regs_pad[3];
 		};
 	};
 #if defined(CONFIG_PPC64) || defined(CONFIG_PPC_KUAP)
arch/powerpc/include/uapi/asm/ptrace.h +9 -5
index 01e630149d48..a393b7f2760a 100644
@@ -55,6 +55,8 @@ struct pt_regs
 	unsigned long dar;		/* Fault registers */
 	unsigned long dsisr;		/* on 4xx/Book-E used for ESR */
 	unsigned long result;		/* Result of a system call */
+	unsigned long exit_flags;	/* System call exit flags */
+	unsigned long __pt_regs_pad[3];	/* Maintain 16 byte interrupt stack alignment */
 };
 
 #endif /* __ASSEMBLER__ */
@@ -114,10 +116,12 @@ struct pt_regs
 #define PT_DAR	41
 #define PT_DSISR 42
 #define PT_RESULT 43
-#define PT_DSCR 44
-#define PT_REGS_COUNT 44
+#define PT_EXIT_FLAGS 44
+#define PT_PAD 47 /* 3 times */
+#define PT_DSCR 48
+#define PT_REGS_COUNT 48
 
-#define PT_FPR0	48	/* each FP reg occupies 2 slots in this space */
+#define PT_FPR0	(PT_REGS_COUNT + 4)	/* each FP reg occupies 2 slots in this space */
 
 #ifndef __powerpc64__
 
@@ -129,7 +133,7 @@ struct pt_regs
 #define PT_FPSCR (PT_FPR0 + 32)	/* each FP reg occupies 1 slot in 64-bit space */
 
 
-#define PT_VR0 82	/* each Vector reg occupies 2 slots in 64-bit */
+#define PT_VR0	(PT_FPSCR + 2)	/* <82> each Vector reg occupies 2 slots in 64-bit */
 #define PT_VSCR (PT_VR0 + 32*2 + 1)
 #define PT_VRSAVE (PT_VR0 + 33*2)
 
@@ -137,7 +141,7 @@ struct pt_regs
 /*
  * Only store first 32 VSRs here. The second 32 VSRs in VR0-31
  */
-#define PT_VSR0 150	/* each VSR reg occupies 2 slots in 64-bit */
+#define PT_VSR0	(PT_VRSAVE + 2)	/* each VSR reg occupies 2 slots in 64-bit */
 #define PT_VSR31 (PT_VSR0 + 2*31)
 #endif /* __powerpc64__ */
 
arch/powerpc/kernel/ptrace/ptrace.c +1 -0
index c6997df63287..2134b6d155ff 100644
@@ -432,6 +432,7 @@ void __init pt_regs_check(void)
 	CHECK_REG(PT_DAR, dar);
 	CHECK_REG(PT_DSISR, dsisr);
 	CHECK_REG(PT_RESULT, result);
+	CHECK_REG(PT_EXIT_FLAGS, exit_flags);
 	#undef CHECK_REG
 
 	BUILD_BUG_ON(PT_REGS_COUNT != sizeof(struct user_pt_regs) / sizeof(unsigned long));
Signature · 1 line
2.52.0
#7 lore raw

PATCH v3 6/8 powerpc: Prepare for IRQ entry exit

In reply to Mukesh Kumar Chaurasiya

From: Mukesh Kumar Chaurasiya <[email protected]>

Move interrupt entry and exit helper routines from interrupt.h into the PowerPC-specific entry-common.h header as a preparatory step for enabling the generic entry/exit framework.

This consolidation places all PowerPC interrupt entry/exit handling in a single common header, aligning with the generic entry infrastructure. The helpers provide architecture-specific handling for interrupt and NMI entry/exit sequences, including:

- arch_interrupt_enter/exit_prepare()
- arch_interrupt_async_enter/exit_prepare()
- arch_interrupt_nmi_enter/exit_prepare()
- Supporting helpers such as nap_adjust_return(), check_return_regs_valid(), debug register maintenance, and soft mask handling.

The functions are copied verbatim from interrupt.h.Subsequent patches will integrate these routines into the generic entry/exit flow.

No functional change intended.

---
 arch/powerpc/include/asm/entry-common.h | 419 ++++++++++++++++++++++++
 1 file changed, 419 insertions(+)
arch/powerpc/include/asm/entry-common.h +419 -0
index 2684bc91e79b..295e1db56d2e 100644
@@ -5,10 +5,429 @@
 
 #include <asm/cputime.h>
 #include <asm/interrupt.h>
+#include <asm/runlatch.h>
 #include <asm/stacktrace.h>
 #include <asm/switch_to.h>
 #include <asm/tm.h>
 
+#ifdef CONFIG_PPC_IRQ_SOFT_MASK_DEBUG
+/*
+ * WARN/BUG is handled with a program interrupt so minimise checks here to
+ * avoid recursion and maximise the chance of getting the first oops handled.
+ */
+#define INT_SOFT_MASK_BUG_ON(regs, cond)				\
+do {									\
+	if ((user_mode(regs) || (TRAP(regs) != INTERRUPT_PROGRAM)))	\
+		BUG_ON(cond);						\
+} while (0)
+#else
+#define INT_SOFT_MASK_BUG_ON(regs, cond)
+#endif
+
+#ifdef CONFIG_PPC_BOOK3S_64
+extern char __end_soft_masked[];
+bool search_kernel_soft_mask_table(unsigned long addr);
+unsigned long search_kernel_restart_table(unsigned long addr);
+
+DECLARE_STATIC_KEY_FALSE(interrupt_exit_not_reentrant);
+
+static inline bool is_implicit_soft_masked(struct pt_regs *regs)
+{
+	if (user_mode(regs))
+		return false;
+
+	if (regs->nip >= (unsigned long)__end_soft_masked)
+		return false;
+
+	return search_kernel_soft_mask_table(regs->nip);
+}
+
+static inline void srr_regs_clobbered(void)
+{
+	local_paca->srr_valid = 0;
+	local_paca->hsrr_valid = 0;
+}
+#else
+static inline unsigned long search_kernel_restart_table(unsigned long addr)
+{
+	return 0;
+}
+
+static inline bool is_implicit_soft_masked(struct pt_regs *regs)
+{
+	return false;
+}
+
+static inline void srr_regs_clobbered(void)
+{
+}
+#endif
+
+static inline void nap_adjust_return(struct pt_regs *regs)
+{
+#ifdef CONFIG_PPC_970_NAP
+	if (unlikely(test_thread_local_flags(_TLF_NAPPING))) {
+		/* Can avoid a test-and-clear because NMIs do not call this */
+		clear_thread_local_flags(_TLF_NAPPING);
+		regs_set_return_ip(regs, (unsigned long)power4_idle_nap_return);
+	}
+#endif
+}
+
+static inline void booke_load_dbcr0(void)
+{
+#ifdef CONFIG_PPC_ADV_DEBUG_REGS
+	unsigned long dbcr0 = current->thread.debug.dbcr0;
+
+	if (likely(!(dbcr0 & DBCR0_IDM)))
+		return;
+
+	/*
+	 * Check to see if the dbcr0 register is set up to debug.
+	 * Use the internal debug mode bit to do this.
+	 */
+	mtmsr(mfmsr() & ~MSR_DE);
+	if (IS_ENABLED(CONFIG_PPC32)) {
+		isync();
+		global_dbcr0[smp_processor_id()] = mfspr(SPRN_DBCR0);
+	}
+	mtspr(SPRN_DBCR0, dbcr0);
+	mtspr(SPRN_DBSR, -1);
+#endif
+}
+
+static inline void booke_restore_dbcr0(void)
+{
+#ifdef CONFIG_PPC_ADV_DEBUG_REGS
+	unsigned long dbcr0 = current->thread.debug.dbcr0;
+
+	if (IS_ENABLED(CONFIG_PPC32) && unlikely(dbcr0 & DBCR0_IDM)) {
+		mtspr(SPRN_DBSR, -1);
+		mtspr(SPRN_DBCR0, global_dbcr0[smp_processor_id()]);
+	}
+#endif
+}
+
+static inline void check_return_regs_valid(struct pt_regs *regs)
+{
+#ifdef CONFIG_PPC_BOOK3S_64
+	unsigned long trap, srr0, srr1;
+	static bool warned;
+	u8 *validp;
+	char *h;
+
+	if (trap_is_scv(regs))
+		return;
+
+	trap = TRAP(regs);
+	// EE in HV mode sets HSRRs like 0xea0
+	if (cpu_has_feature(CPU_FTR_HVMODE) && trap == INTERRUPT_EXTERNAL)
+		trap = 0xea0;
+
+	switch (trap) {
+	case 0x980:
+	case INTERRUPT_H_DATA_STORAGE:
+	case 0xe20:
+	case 0xe40:
+	case INTERRUPT_HMI:
+	case 0xe80:
+	case 0xea0:
+	case INTERRUPT_H_FAC_UNAVAIL:
+	case 0x1200:
+	case 0x1500:
+	case 0x1600:
+	case 0x1800:
+		validp = &local_paca->hsrr_valid;
+		if (!READ_ONCE(*validp))
+			return;
+
+		srr0 = mfspr(SPRN_HSRR0);
+		srr1 = mfspr(SPRN_HSRR1);
+		h = "H";
+
+		break;
+	default:
+		validp = &local_paca->srr_valid;
+		if (!READ_ONCE(*validp))
+			return;
+
+		srr0 = mfspr(SPRN_SRR0);
+		srr1 = mfspr(SPRN_SRR1);
+		h = "";
+		break;
+	}
+
+	if (srr0 == regs->nip && srr1 == regs->msr)
+		return;
+
+	/*
+	 * A NMI / soft-NMI interrupt may have come in after we found
+	 * srr_valid and before the SRRs are loaded. The interrupt then
+	 * comes in and clobbers SRRs and clears srr_valid. Then we load
+	 * the SRRs here and test them above and find they don't match.
+	 *
+	 * Test validity again after that, to catch such false positives.
+	 *
+	 * This test in general will have some window for false negatives
+	 * and may not catch and fix all such cases if an NMI comes in
+	 * later and clobbers SRRs without clearing srr_valid, but hopefully
+	 * such things will get caught most of the time, statistically
+	 * enough to be able to get a warning out.
+	 */
+	if (!READ_ONCE(*validp))
+		return;
+
+	if (!data_race(warned)) {
+		data_race(warned = true);
+		pr_warn("%sSRR0 was: %lx should be: %lx\n", h, srr0, regs->nip);
+		pr_warn("%sSRR1 was: %lx should be: %lx\n", h, srr1, regs->msr);
+		show_regs(regs);
+	}
+
+	WRITE_ONCE(*validp, 0); /* fixup */
+#endif
+}
+
+static inline void arch_interrupt_enter_prepare(struct pt_regs *regs)
+{
+#ifdef CONFIG_PPC64
+	irq_soft_mask_set(IRQS_ALL_DISABLED);
+
+	/*
+	 * If the interrupt was taken with HARD_DIS clear, then enable MSR[EE].
+	 * Asynchronous interrupts get here with HARD_DIS set (see below), so
+	 * this enables MSR[EE] for synchronous interrupts. IRQs remain
+	 * soft-masked. The interrupt handler may later call
+	 * interrupt_cond_local_irq_enable() to achieve a regular process
+	 * context.
+	 */
+	if (!(local_paca->irq_happened & PACA_IRQ_HARD_DIS)) {
+		INT_SOFT_MASK_BUG_ON(regs, !(regs->msr & MSR_EE));
+		__hard_irq_enable();
+	} else {
+		__hard_RI_enable();
+	}
+	/* Enable MSR[RI] early, to support kernel SLB and hash faults */
+#endif
+
+	if (!regs_irqs_disabled(regs))
+		trace_hardirqs_off();
+
+	if (user_mode(regs)) {
+		kuap_lock();
+		account_cpu_user_entry();
+		account_stolen_time();
+	} else {
+		kuap_save_and_lock(regs);
+		/*
+		 * CT_WARN_ON comes here via program_check_exception,
+		 * so avoid recursion.
+		 */
+		if (TRAP(regs) != INTERRUPT_PROGRAM)
+			CT_WARN_ON(ct_state() != CT_STATE_KERNEL &&
+				   ct_state() != CT_STATE_IDLE);
+		INT_SOFT_MASK_BUG_ON(regs, is_implicit_soft_masked(regs));
+		INT_SOFT_MASK_BUG_ON(regs, regs_irqs_disabled(regs) &&
+				     search_kernel_restart_table(regs->nip));
+	}
+	INT_SOFT_MASK_BUG_ON(regs, !regs_irqs_disabled(regs) &&
+			     !(regs->msr & MSR_EE));
+
+	booke_restore_dbcr0();
+}
+
+/*
+ * Care should be taken to note that arch_interrupt_exit_prepare and
+ * arch_interrupt_async_exit_prepare do not necessarily return immediately to
+ * regs context (e.g., if regs is usermode, we don't necessarily return to
+ * user mode). Other interrupts might be taken between here and return,
+ * context switch / preemption may occur in the exit path after this, or a
+ * signal may be delivered, etc.
+ *
+ * The real interrupt exit code is platform specific, e.g.,
+ * interrupt_exit_user_prepare / interrupt_exit_kernel_prepare for 64s.
+ *
+ * However arch_interrupt_nmi_exit_prepare does return directly to regs, because
+ * NMIs do not do "exit work" or replay soft-masked interrupts.
+ */
+static inline void arch_interrupt_exit_prepare(struct pt_regs *regs)
+{
+}
+
+static inline void arch_interrupt_async_enter_prepare(struct pt_regs *regs)
+{
+#ifdef CONFIG_PPC64
+	/* Ensure arch_interrupt_enter_prepare does not enable MSR[EE] */
+	local_paca->irq_happened |= PACA_IRQ_HARD_DIS;
+#endif
+	arch_interrupt_enter_prepare(regs);
+#ifdef CONFIG_PPC_BOOK3S_64
+	/*
+	 * RI=1 is set by arch_interrupt_enter_prepare, so this thread flags access
+	 * has to come afterward (it can cause SLB faults).
+	 */
+	if (cpu_has_feature(CPU_FTR_CTRL) &&
+	    !test_thread_local_flags(_TLF_RUNLATCH))
+		__ppc64_runlatch_on();
+#endif
+	irq_enter();
+}
+
+static inline void arch_interrupt_async_exit_prepare(struct pt_regs *regs)
+{
+	/*
+	 * Adjust at exit so the main handler sees the true NIA. This must
+	 * come before irq_exit() because irq_exit can enable interrupts, and
+	 * if another interrupt is taken before nap_adjust_return has run
+	 * here, then that interrupt would return directly to idle nap return.
+	 */
+	nap_adjust_return(regs);
+
+	irq_exit();
+	arch_interrupt_exit_prepare(regs);
+}
+
+struct interrupt_nmi_state {
+#ifdef CONFIG_PPC64
+	u8 irq_soft_mask;
+	u8 irq_happened;
+	u8 ftrace_enabled;
+	u64 softe;
+#endif
+};
+
+static inline bool nmi_disables_ftrace(struct pt_regs *regs)
+{
+	/* Allow DEC and PMI to be traced when they are soft-NMI */
+	if (IS_ENABLED(CONFIG_PPC_BOOK3S_64)) {
+		if (TRAP(regs) == INTERRUPT_DECREMENTER)
+			return false;
+		if (TRAP(regs) == INTERRUPT_PERFMON)
+			return false;
+	}
+	if (IS_ENABLED(CONFIG_PPC_BOOK3E_64)) {
+		if (TRAP(regs) == INTERRUPT_PERFMON)
+			return false;
+	}
+
+	return true;
+}
+
+static inline void arch_interrupt_nmi_enter_prepare(struct pt_regs *regs,
+						    struct interrupt_nmi_state *state)
+{
+#ifdef CONFIG_PPC64
+	state->irq_soft_mask = local_paca->irq_soft_mask;
+	state->irq_happened = local_paca->irq_happened;
+	state->softe = regs->softe;
+
+	/*
+	 * Set IRQS_ALL_DISABLED unconditionally so irqs_disabled() does
+	 * the right thing, and set IRQ_HARD_DIS. We do not want to reconcile
+	 * because that goes through irq tracing which we don't want in NMI.
+	 */
+	local_paca->irq_soft_mask = IRQS_ALL_DISABLED;
+	local_paca->irq_happened |= PACA_IRQ_HARD_DIS;
+
+	if (!(regs->msr & MSR_EE) || is_implicit_soft_masked(regs)) {
+		/*
+		 * Adjust regs->softe to be soft-masked if it had not been
+		 * reconcied (e.g., interrupt entry with MSR[EE]=0 but softe
+		 * not yet set disabled), or if it was in an implicit soft
+		 * masked state. This makes regs_irqs_disabled(regs)
+		 * behave as expected.
+		 */
+		regs->softe = IRQS_ALL_DISABLED;
+	}
+
+	__hard_RI_enable();
+
+	/* Don't do any per-CPU operations until interrupt state is fixed */
+
+	if (nmi_disables_ftrace(regs)) {
+		state->ftrace_enabled = this_cpu_get_ftrace_enabled();
+		this_cpu_set_ftrace_enabled(0);
+	}
+#endif
+
+	/* If data relocations are enabled, it's safe to use nmi_enter() */
+	if (mfmsr() & MSR_DR) {
+		nmi_enter();
+		return;
+	}
+
+	/*
+	 * But do not use nmi_enter() for pseries hash guest taking a real-mode
+	 * NMI because not everything it touches is within the RMA limit.
+	 */
+	if (IS_ENABLED(CONFIG_PPC_BOOK3S_64) &&
+	    firmware_has_feature(FW_FEATURE_LPAR) &&
+	    !radix_enabled())
+		return;
+
+	/*
+	 * Likewise, don't use it if we have some form of instrumentation (like
+	 * KASAN shadow) that is not safe to access in real mode (even on radix)
+	 */
+	if (IS_ENABLED(CONFIG_KASAN))
+		return;
+
+	/*
+	 * Likewise, do not use it in real mode if percpu first chunk is not
+	 * embedded. With CONFIG_NEED_PER_CPU_PAGE_FIRST_CHUNK enabled there
+	 * are chances where percpu allocation can come from vmalloc area.
+	 */
+	if (percpu_first_chunk_is_paged)
+		return;
+
+	/* Otherwise, it should be safe to call it */
+	nmi_enter();
+}
+
+static inline void arch_interrupt_nmi_exit_prepare(struct pt_regs *regs,
+						   struct interrupt_nmi_state *state)
+{
+	if (mfmsr() & MSR_DR) {
+		// nmi_exit if relocations are on
+		nmi_exit();
+	} else if (IS_ENABLED(CONFIG_PPC_BOOK3S_64) &&
+		   firmware_has_feature(FW_FEATURE_LPAR) &&
+		   !radix_enabled()) {
+		// no nmi_exit for a pseries hash guest taking a real mode exception
+	} else if (IS_ENABLED(CONFIG_KASAN)) {
+		// no nmi_exit for KASAN in real mode
+	} else if (percpu_first_chunk_is_paged) {
+		// no nmi_exit if percpu first chunk is not embedded
+	} else {
+		nmi_exit();
+	}
+
+	/*
+	 * nmi does not call nap_adjust_return because nmi should not create
+	 * new work to do (must use irq_work for that).
+	 */
+
+#ifdef CONFIG_PPC64
+#ifdef CONFIG_PPC_BOOK3S
+	if (regs_irqs_disabled(regs)) {
+		unsigned long rst = search_kernel_restart_table(regs->nip);
+
+		if (rst)
+			regs_set_return_ip(regs, rst);
+	}
+#endif
+
+	if (nmi_disables_ftrace(regs))
+		this_cpu_set_ftrace_enabled(state->ftrace_enabled);
+
+	/* Check we didn't change the pending interrupt mask. */
+	WARN_ON_ONCE((state->irq_happened | PACA_IRQ_HARD_DIS) != local_paca->irq_happened);
+	regs->softe = state->softe;
+	local_paca->irq_happened = state->irq_happened;
+	local_paca->irq_soft_mask = state->irq_soft_mask;
+#endif
+}
+
 static __always_inline void arch_enter_from_user_mode(struct pt_regs *regs)
 {
 	kuap_lock();
Signature · 1 line
2.52.0
#8 lore raw

PATCH v3 7/8 powerpc: Enable IRQ generic entry/exit path.

In reply to Mukesh Kumar Chaurasiya

From: Mukesh Kumar Chaurasiya <[email protected]>

Enable the generic IRQ entry/exit infrastructure on PowerPC by selecting GENERIC_IRQ_ENTRY and integrating the architecture-specific interrupt handlers with the generic entry/exit APIs.

This change replaces PowerPC’s local interrupt entry/exit handling with calls to the generic irqentry_* helpers, aligning the architecture with the common kernel entry model. The macros that define interrupt, async, and NMI handlers are updated to use irqentry_enter()/irqentry_exit() and irqentry_nmi_enter()/irqentry_nmi_exit() where applicable.

Key updates include:
- Select GENERIC_IRQ_ENTRY in Kconfig.
- Replace interrupt_enter/exit_prepare() with arch_interrupt_* helpers.
- Integrate irqentry_enter()/exit() in standard and async interrupt paths.
- Integrate irqentry_nmi_enter()/exit() in NMI handlers.
- Remove redundant irq_enter()/irq_exit() calls now handled generically.
- Use irqentry_exit_cond_resched() for preemption checks.
- Remove unused code.

This change establishes the necessary wiring for PowerPC to use the generic IRQ entry/exit framework while maintaining existing semantics.

---
 arch/powerpc/Kconfig                    |   1 +
 arch/powerpc/include/asm/entry-common.h |  61 +--
 arch/powerpc/include/asm/interrupt.h    | 484 +++---------------------
 arch/powerpc/kernel/interrupt.c         |  15 +-
 4 files changed, 72 insertions(+), 489 deletions(-)
arch/powerpc/Kconfig +1 -0
index 9537a61ebae0..398cef868c14 100644
@@ -207,6 +207,7 @@ config PPC
 	select GENERIC_GETTIMEOFDAY
 	select GENERIC_IDLE_POLL_SETUP
 	select GENERIC_IOREMAP
+	select GENERIC_IRQ_ENTRY
 	select GENERIC_IRQ_SHOW
 	select GENERIC_IRQ_SHOW_LEVEL
 	select GENERIC_PCI_IOMAP		if PCI
arch/powerpc/include/asm/entry-common.h +11 -50

90 lines not on this page · raw message

arch/powerpc/include/asm/interrupt.h +58 -426

529 lines not on this page · raw message

arch/powerpc/kernel/interrupt.c +2 -13
index a09226019466..d202405e4e8e 100644
@@ -24,10 +24,6 @@
 unsigned long global_dbcr0[NR_CPUS];
 #endif
 
-#if defined(CONFIG_PREEMPT_DYNAMIC)
-DEFINE_STATIC_KEY_TRUE(sk_dynamic_irqentry_exit_cond_resched);
-#endif
-
 #ifdef CONFIG_PPC_BOOK3S_64
 DEFINE_STATIC_KEY_FALSE(interrupt_exit_not_reentrant);
 static inline bool exit_must_hard_disable(void)
@@ -245,7 +241,6 @@ notrace unsigned long interrupt_exit_user_prepare(struct pt_regs *regs)
 
 	BUG_ON(regs_is_unrecoverable(regs));
 	BUG_ON(regs_irqs_disabled(regs));
-	CT_WARN_ON(ct_state() == CT_STATE_USER);
 
 	/*
 	 * We don't need to restore AMR on the way back to userspace for KUAP.
@@ -297,13 +292,8 @@ notrace unsigned long interrupt_exit_kernel_prepare(struct pt_regs *regs)
 		/* Returning to a kernel context with local irqs enabled. */
 		WARN_ON_ONCE(!(regs->msr & MSR_EE));
 again:
-		if (need_irq_preemption()) {
-			/* Return to preemptible kernel context */
-			if (unlikely(read_thread_flags() & _TIF_NEED_RESCHED)) {
-				if (preempt_count() == 0)
-					preempt_schedule_irq();
-			}
-		}
+		if (need_irq_preemption())
+			irqentry_exit_cond_resched();
 
 		check_return_regs_valid(regs);
 
@@ -373,7 +363,6 @@ notrace unsigned long interrupt_exit_user_restart(struct pt_regs *regs)
 #endif
 
 	trace_hardirqs_off();
-	user_exit_irqoff();
 	account_cpu_user_entry();
 
 	BUG_ON(!user_mode(regs));
Signature · 1 line
2.52.0
#9 lore raw

Re: [PATCH v3 7/8] powerpc: Enable IRQ generic entry/exit path.

Quoted 37 lines · Mukesh Kumar Chaurasiya

From: Mukesh Kumar Chaurasiya <[email protected]>

Enable the generic IRQ entry/exit infrastructure on PowerPC by selecting GENERIC_IRQ_ENTRY and integrating the architecture-specific interrupt handlers with the generic entry/exit APIs.

This change replaces PowerPC’s local interrupt entry/exit handling with calls to the generic irqentry_* helpers, aligning the architecture with the common kernel entry model. The macros that define interrupt, async, and NMI handlers are updated to use irqentry_enter()/irqentry_exit() and irqentry_nmi_enter()/irqentry_nmi_exit() where applicable.

Key updates include:
- Select GENERIC_IRQ_ENTRY in Kconfig.
- Replace interrupt_enter/exit_prepare() with arch_interrupt_* helpers.
- Integrate irqentry_enter()/exit() in standard and async interrupt paths.
- Integrate irqentry_nmi_enter()/exit() in NMI handlers.
- Remove redundant irq_enter()/irq_exit() calls now handled generically.
- Use irqentry_exit_cond_resched() for preemption checks.
- Remove unused code.

This change establishes the necessary wiring for PowerPC to use the generic IRQ entry/exit framework while maintaining existing semantics.

---
  arch/powerpc/Kconfig                    |   1 +
  arch/powerpc/include/asm/entry-common.h |  61 +--
  arch/powerpc/include/asm/interrupt.h    | 484 +++---------------------
  arch/powerpc/kernel/interrupt.c         |  15 +-
  4 files changed, 72 insertions(+), 489 deletions(-)
arch/powerpc/Kconfig +0 -0
index 9537a61ebae0..398cef868c14 100644

arch/powerpc/Kconfig config PPC · line 213

Quoted 7 lines
arch/powerpc/Kconfig +1 -0
@@ -207,6 +207,7 @@ config PPC
  	select GENERIC_GETTIMEOFDAY
  	select GENERIC_IDLE_POLL_SETUP
  	select GENERIC_IOREMAP
+	select GENERIC_IRQ_ENTRY
  	select GENERIC_IRQ_SHOW
  	select GENERIC_IRQ_SHOW_LEVEL
  	select GENERIC_PCI_IOMAP		if PCI

There is no need for GENERIC_IRQ_ENTRY. I don't see a reason why we need to have both.

Enabling GENERIC_ENTRY which is done in next patch will enable this. It does the same thing and one less kconfig to worry about.

This would allow some easier code refactoring IMO.

- You can keep adding code to asm/entry-common.h. This will help avoid moving to interrupt.h in between.

- You can add another patch as cleanup post GENERIC_ENTRY for easier review for seccomp/ptrace.

...

- Some more ct_warning in arch/powerpc are probably not necessary (can be done post the series too)

- Some more cleanups can be done w.r.t to lockdep and tracing hardirq etc. For example in next_interrupt. I think replay soft interrupts will end up making irqentry_enter/exit calls which does same or similar thing. (can be done post the series too)

- can we move syscall_exit_prepare/ into kernel/syscall.c? (can be done post the series too)

#10 lore raw

Re: [PATCH v3 7/8] powerpc: Enable IRQ generic entry/exit path.

Quoted 51 lines · Shrikanth Hegde
Quoted 37 lines · Mukesh Kumar Chaurasiya

From: Mukesh Kumar Chaurasiya <[email protected]>

Enable the generic IRQ entry/exit infrastructure on PowerPC by selecting GENERIC_IRQ_ENTRY and integrating the architecture-specific interrupt handlers with the generic entry/exit APIs.

This change replaces PowerPC’s local interrupt entry/exit handling with calls to the generic irqentry_* helpers, aligning the architecture with the common kernel entry model. The macros that define interrupt, async, and NMI handlers are updated to use irqentry_enter()/irqentry_exit() and irqentry_nmi_enter()/irqentry_nmi_exit() where applicable.

Key updates include:
- Select GENERIC_IRQ_ENTRY in Kconfig.
- Replace interrupt_enter/exit_prepare() with arch_interrupt_* helpers.
- Integrate irqentry_enter()/exit() in standard and async interrupt paths.
- Integrate irqentry_nmi_enter()/exit() in NMI handlers.
- Remove redundant irq_enter()/irq_exit() calls now handled generically.
- Use irqentry_exit_cond_resched() for preemption checks.
- Remove unused code.

This change establishes the necessary wiring for PowerPC to use the generic IRQ entry/exit framework while maintaining existing semantics.

---
  arch/powerpc/Kconfig                    |   1 +
  arch/powerpc/include/asm/entry-common.h |  61 +--
  arch/powerpc/include/asm/interrupt.h    | 484 +++---------------------
  arch/powerpc/kernel/interrupt.c         |  15 +-
  4 files changed, 72 insertions(+), 489 deletions(-)
arch/powerpc/Kconfig +0 -0
index 9537a61ebae0..398cef868c14 100644

arch/powerpc/Kconfig config PPC · line 213

Quoted 7 lines
arch/powerpc/Kconfig +1 -0
@@ -207,6 +207,7 @@ config PPC
  	select GENERIC_GETTIMEOFDAY
  	select GENERIC_IDLE_POLL_SETUP
  	select GENERIC_IOREMAP
+	select GENERIC_IRQ_ENTRY
  	select GENERIC_IRQ_SHOW
  	select GENERIC_IRQ_SHOW_LEVEL
  	select GENERIC_PCI_IOMAP		if PCI

There is no need for GENERIC_IRQ_ENTRY. I don't see a reason why we need to have both.

Quoted 8 lines

Enabling GENERIC_ENTRY which is done in next patch will enable this. It does the same thing and one less kconfig to worry about.

This would allow some easier code refactoring IMO.

- You can keep adding code to asm/entry-common.h. This will help avoid moving to interrupt.h in between.

Yeah this would be good, will send out in next revision.

Quoted 3 lines

- You can add another patch as cleanup post GENERIC_ENTRY for easier review for seccomp/ptrace.

Sure.

Quoted 5 lines

...

- Some more ct_warning in arch/powerpc are probably not necessary (can be done post the series too)

Quoted 7 lines

- Some more cleanups can be done w.r.t to lockdep and tracing hardirq etc. For example in next_interrupt. I think replay soft interrupts will end up making irqentry_enter/exit calls which does same or similar thing. (can be done post the series too)

- can we move syscall_exit_prepare/ into kernel/syscall.c? (can be done post the series too)

Sure will do all of above in a separate series.

Regards,
Mukesh

#11 lore raw

PATCH v3 8/8 powerpc: Enable Generic Entry/Exit for syscalls.

In reply to Mukesh Kumar Chaurasiya

From: Mukesh Kumar Chaurasiya <[email protected]>

Convert the PowerPC syscall entry and exit paths to use the generic entry/exit framework by selecting GENERIC_ENTRY and integrating with the common syscall handling routines.

This change transitions PowerPC away from its custom syscall entry and exit code to use the generic helpers such as:
- syscall_enter_from_user_mode()
- syscall_exit_to_user_mode()

As part of this migration:
- The architecture now selects GENERIC_ENTRY in Kconfig.
- Old tracing, seccomp, and audit handling in ptrace.c is removed in favor of generic entry infrastructure.
- interrupt.c and syscall.c are simplified to delegate context management and user exit handling to the generic entry path.
- The new pt_regs field exit_flags introduced earlier is now used to carry per-syscall exit state flags (e.g. _TIF_RESTOREALL).

This aligns PowerPC with the common entry code used by other architectures and reduces duplicated logic around syscall tracing, context tracking, and signal handling.

The performance benchmarks from perf bench basic syscall are below:

perf bench syscall usec/op

| Syscall | Base       | New        | change % |
| ------- | ---------- | ---------- | -------- |
| basic   | 0.173212   | 0.133294   | -23.05   |
| execve  | 363.176190 | 357.148150 | -1.66    |
| fork    | 853.657880 | 840.268800 | -1.57    |
| getpgid | 0.174832   | 0.135372   | -22.57   |

perf bench syscall ops/sec

| Syscall | Base    | New     | change % |
| ------- | ------- | ------- | -------- |
| basic   | 6006021 | 7502236 | +24.91   |
| execve  | 2753    | 2800    | +1.71    |
| fork    | 1171    | 1190    | +1.62    |
| getpgid | 5942117 | 7387040 | +24.32   |

IPI latency benchmark

| Metric         | Base (ns)     | Test (ns)     | change % |
| -------------- | ------------- | ------------- | -------- |
| Dry-run        | 206652.45     | 209317.37     | +1.29    |
| Self-IPI       | 3567895.23    | 3590444.77    | +0.63    |
| Normal IPI     | 148570416.17  | 148564173.40  | -0.00    |
| Broadcast IPI  | 4033489673.38 | 4007319512.62 | -0.65    |
| Broadcast lock | 4011023005.48 | 4010267885.93 | -0.02    |

Thats very close to performance earlier with arch specific handling.

---
 arch/powerpc/Kconfig                |   1 +
 arch/powerpc/include/asm/ptrace.h   |   3 -
 arch/powerpc/include/asm/signal.h   |   1 -
 arch/powerpc/kernel/interrupt.c     | 137 +++++++++------------------
 arch/powerpc/kernel/ptrace/ptrace.c | 141 ----------------------------
 arch/powerpc/kernel/signal.c        |  25 ++---
 arch/powerpc/kernel/syscall.c       | 119 +----------------------
 7 files changed, 54 insertions(+), 373 deletions(-)
arch/powerpc/Kconfig +1 -0
index 398cef868c14..c73aa9c73692 100644
@@ -204,6 +204,7 @@ config PPC
 	select GENERIC_CPU_AUTOPROBE
 	select GENERIC_CPU_VULNERABILITIES	if PPC_BARRIER_NOSPEC
 	select GENERIC_EARLY_IOREMAP
+	select GENERIC_ENTRY
 	select GENERIC_GETTIMEOFDAY
 	select GENERIC_IDLE_POLL_SETUP
 	select GENERIC_IOREMAP
arch/powerpc/include/asm/ptrace.h +0 -3
index 2e741ea57b80..fdeb97421785 100644
@@ -177,9 +177,6 @@ extern unsigned long profile_pc(struct pt_regs *regs);
 #define profile_pc(regs) instruction_pointer(regs)
 #endif
 
-long do_syscall_trace_enter(struct pt_regs *regs);
-void do_syscall_trace_leave(struct pt_regs *regs);
-
 static inline void set_return_regs_changed(void)
 {
 #ifdef CONFIG_PPC_BOOK3S_64
arch/powerpc/include/asm/signal.h +0 -1
index 922d43700fb4..21af92cdb237 100644
@@ -7,7 +7,6 @@
 #include <uapi/asm/ptrace.h>
 
 struct pt_regs;
-void do_notify_resume(struct pt_regs *regs, unsigned long thread_info_flags);
 
 unsigned long get_min_sigframe_size_32(void);
 unsigned long get_min_sigframe_size_64(void);
arch/powerpc/kernel/interrupt.c +42 -95

198 lines not on this page · raw message

arch/powerpc/kernel/ptrace/ptrace.c +0 -141

153 lines not on this page · raw message

arch/powerpc/kernel/signal.c +8 -17
index aa17e62f3754..bb42a8b6c642 100644
@@ -6,6 +6,7 @@
  *    Extracted from signal_32.c and signal_64.c
  */
 
+#include <linux/entry-common.h>
 #include <linux/resume_user_mode.h>
 #include <linux/signal.h>
 #include <linux/uprobes.h>
@@ -292,23 +293,6 @@ static void do_signal(struct task_struct *tsk)
 	signal_setup_done(ret, &ksig, test_thread_flag(TIF_SINGLESTEP));
 }
 
-void do_notify_resume(struct pt_regs *regs, unsigned long thread_info_flags)
-{
-	if (thread_info_flags & _TIF_UPROBE)
-		uprobe_notify_resume(regs);
-
-	if (thread_info_flags & _TIF_PATCH_PENDING)
-		klp_update_patch_state(current);
-
-	if (thread_info_flags & (_TIF_SIGPENDING | _TIF_NOTIFY_SIGNAL)) {
-		BUG_ON(regs != current->thread.regs);
-		do_signal(current);
-	}
-
-	if (thread_info_flags & _TIF_NOTIFY_RESUME)
-		resume_user_mode_work(regs);
-}
-
 static unsigned long get_tm_stackpointer(struct task_struct *tsk)
 {
 	/* When in an active transaction that takes a signal, we need to be
@@ -368,3 +352,10 @@ void signal_fault(struct task_struct *tsk, struct pt_regs *regs,
 		printk_ratelimited(regs->msr & MSR_64BIT ? fm64 : fm32, tsk->comm,
 				   task_pid_nr(tsk), where, ptr, regs->nip, regs->link);
 }
+
+void arch_do_signal_or_restart(struct pt_regs *regs)
+{
+	BUG_ON(regs != current->thread.regs);
+	regs->exit_flags |= _TIF_RESTOREALL;
+	do_signal(current);
+}
arch/powerpc/kernel/syscall.c +3 -116

133 lines not on this page · raw message

Signature · 1 line
2.52.0
#12 lore raw
In reply to Mukesh Kumar Chaurasiya
Quoted 9 lines · Mukesh Kumar Chaurasiya

Adding support for the generic irq entry/exit handling for PowerPC. The goal is to bring PowerPC in line with other architectures that already use the common irq entry infrastructure, reducing duplicated code and making it easier to share future changes in entry/exit paths.

This is slightly tested of ppc64le and ppc32.

The performance benchmarks are below:

Quoted 9 lines

perf bench syscall usec/op

| Syscall | Base       | New        | change % |
| ------- | ---------- | ---------- | -------- |
| basic   | 0.173212   | 0.133294   | -23.05   |
| execve  | 363.176190 | 357.148150 | -1.66    |
| fork    | 853.657880 | 840.268800 | -1.57    |
| getpgid | 0.174832   | 0.135372   | -22.57   |

Could you please add a description for whether its an improvement or regression.

Looking at it, it seems time taken for operation to complete. So it is an improvement. Seeing negative numbers one might think its a regression.

Also, are these average numbers?
Could you get the numbers with preempt=lazy and with context tracking on?

Quoted 20 lines

perf bench syscall ops/sec

| Syscall | Base    | New     | change % |
| ------- | ------- | ------- | -------- |
| basic   | 6006021 | 7502236 | +24.91   |
| execve  | 2753    | 2800    | +1.71    |
| fork    | 1171    | 1190    | +1.62    |
| getpgid | 5942117 | 7387040 | +24.32   |

IPI latency benchmark

| Metric         | Base (ns)     | Test (ns)     | change % |
| -------------- | ------------- | ------------- | -------- |
| Dry-run        | 206652.45     | 209317.37     | +1.29    |
| Self-IPI       | 3567895.23    | 3590444.77    | +0.63    |
| Normal IPI     | 148570416.17  | 148564173.40  | -0.00    |
| Broadcast IPI  | 4033489673.38 | 4007319512.62 | -0.65    |
| Broadcast lock | 4011023005.48 | 4010267885.93 | -0.02    |
#13 lore raw
Quoted 22 lines · Shrikanth Hegde
Quoted 9 lines · Mukesh Kumar Chaurasiya

Adding support for the generic irq entry/exit handling for PowerPC. The goal is to bring PowerPC in line with other architectures that already use the common irq entry infrastructure, reducing duplicated code and making it easier to share future changes in entry/exit paths.

This is slightly tested of ppc64le and ppc32.

The performance benchmarks are below:

Quoted 9 lines

perf bench syscall usec/op

| Syscall | Base       | New        | change % |
| ------- | ---------- | ---------- | -------- |
| basic   | 0.173212   | 0.133294   | -23.05   |
| execve  | 363.176190 | 357.148150 | -1.66    |
| fork    | 853.657880 | 840.268800 | -1.57    |
| getpgid | 0.174832   | 0.135372   | -22.57   |
Quoted 6 lines

Could you please add a description for whether its an improvement or regression.

Looking at it, it seems time taken for operation to complete. So it is an improvement. Seeing negative numbers one might think its a regression.

Sure will add an explanation.

Quoted 1 line

Also, are these average numbers?

Yes, as there was very high run to run variation i took an avg of 100 runs.

Quoted 1 line

Could you get the numbers with preempt=lazy and with context tracking on?

Sure will do those too.

Regards,
Mukesh

Quoted 22 lines
Quoted 21 lines

perf bench syscall ops/sec

| Syscall | Base    | New     | change % |
| ------- | ------- | ------- | -------- |
| basic   | 6006021 | 7502236 | +24.91   |
| execve  | 2753    | 2800    | +1.71    |
| fork    | 1171    | 1190    | +1.62    |
| getpgid | 5942117 | 7387040 | +24.32   |

IPI latency benchmark

| Metric         | Base (ns)     | Test (ns)     | change % |
| -------------- | ------------- | ------------- | -------- |
| Dry-run        | 206652.45     | 209317.37     | +1.29    |
| Self-IPI       | 3567895.23    | 3590444.77    | +0.63    |
| Normal IPI     | 148570416.17  | 148564173.40  | -0.00    |
| Broadcast IPI  | 4033489673.38 | 4007319512.62 | -0.65    |
| Broadcast lock | 4011023005.48 | 4010267885.93 | -0.02    |
#14 lore raw
In reply to Mukesh Kumar Chaurasiya

With this patch set, build fails.

Failure logs:

ld: warning: discarding dynamic section .glink
ld: warning: discarding dynamic section .plt
ld: linkage table error against `memset'
ld: stubs don't match calculated size
ld: can not build stubs: bad value
ld: init/main.o: in function `do_one_initcall':
/root/linux/init/main.c:1384: undefined reference to `memcpy'
ld: arch/powerpc/kernel/process.o: in function `arch_dup_task_struct':
/root/linux/arch/powerpc/kernel/process.c:1724: undefined reference to `memcpy'
ld: arch/powerpc/kernel/process.o: in function `copy_thread':
/root/linux/arch/powerpc/kernel/process.c:1812: undefined reference to `memcpy'
ld: arch/powerpc/kernel/signal.o: in function `do_signal':
/root/linux/arch/powerpc/kernel/signal.c:247: undefined reference to `memset'
ld: arch/powerpc/kernel/sysfs.o: in function `show_smt_snooze_delay':
/root/linux/arch/powerpc/kernel/sysfs.c:66: undefined reference to `memcpy'
ld: /root/linux/arch/powerpc/kernel/sysfs.c:66: undefined reference to `memcpy'
ld: arch/powerpc/kernel/time.o: in function `register_decrementer_clockevent':
/root/linux/arch/powerpc/kernel/time.c:834: undefined reference to `memcpy'
ld: arch/powerpc/kernel/time.o: in function `platform_device_register_resndata':
/root/linux/./include/linux/platform_device.h:160: undefined reference to `memset'
ld: arch/powerpc/kernel/ptrace/ptrace-noadv.o: in function `ptrace_set_debugreg':
/root/linux/arch/powerpc/kernel/ptrace/ptrace-noadv.c:130: undefined reference to `memcpy'
ld: arch/powerpc/kernel/paca.o: in function `init_lppaca':
/root/linux/arch/powerpc/kernel/paca.c:112: undefined reference to `memset'
ld: arch/powerpc/kernel/hw_breakpoint.o: in function `ptrace_triggered':
/root/linux/arch/powerpc/kernel/hw_breakpoint.c:607: undefined reference to `memcpy'
ld: arch/powerpc/kernel/mce.o: in function `get_mce_event':
/root/linux/arch/powerpc/kernel/mce.c:206: undefined reference to `memcpy'
ld: /root/linux/arch/powerpc/kernel/mce.c:206: undefined reference to `memcpy'
ld: arch/powerpc/kernel/mce_power.o: in function `mce_find_instr_ea_and_phys':
/root/linux/arch/powerpc/kernel/mce_power.c:461: undefined reference to `memcpy'
ld: arch/powerpc/kernel/security.o: in function `cpu_show_meltdown':
/root/linux/arch/powerpc/kernel/security.c:172: undefined reference to `memcpy'
ld: arch/powerpc/kernel/security.o:/root/linux/arch/powerpc/kernel/security.c:166: more undefined references to `memcpy' follow
ld: arch/powerpc/mm/mem.o: in function `execmem_arch_setup':
/root/linux/arch/powerpc/mm/mem.c:423: undefined reference to `memset'
ld: arch/powerpc/mm/book3s64/subpage_prot.o: in function `subpage_mark_vma_nohuge':
/root/linux/arch/powerpc/mm/book3s64/subpage_prot.c:155: undefined reference to `memset'
ld: arch/powerpc/mm/numa.o: in function `vphn_get_nid':
/root/linux/arch/powerpc/mm/numa.c:676: undefined reference to `memset'
ld: arch/powerpc/mm/numa.o: in function `find_and_update_cpu_nid':
/root/linux/arch/powerpc/mm/numa.c:1416: undefined reference to `memset'
ld: arch/powerpc/mm/numa.o: in function `cpu_to_coregroup_id':
/root/linux/arch/powerpc/mm/numa.c:1438: undefined reference to `memset'
ld: arch/powerpc/mm/ptdump/ptdump.o:/root/linux/arch/powerpc/mm/ptdump/ptdump.c:370: more undefined references to `memset' follow
ld: arch/powerpc/lib/mem_64.o: in function `__memmove':
/root/linux/arch/powerpc/lib/mem_64.S:103: undefined reference to memcpy' /root/linux/arch/powerpc/lib/mem_64.S:103:(.text+0x10c): call to memcpy' lacks nop, can't restore toc; (plt call stub)
ld: final link failed: bad value
make[2]: *** [scripts/Makefile.vmlinux:72: vmlinux.unstripped] Error 1
make[1]: *** [/root/linux/Makefile:1276: vmlinux] Error 2
make[1]: *** Waiting for unfinished jobs....
make: *** [Makefile:248: __sub-make] Error 2

Regards,
Venkat.

Quoted 101 lines · Mukesh Kumar Chaurasiya

Adding support for the generic irq entry/exit handling for PowerPC. The goal is to bring PowerPC in line with other architectures that already use the common irq entry infrastructure, reducing duplicated code and making it easier to share future changes in entry/exit paths.

This is slightly tested of ppc64le and ppc32.

The performance benchmarks are below:

perf bench syscall usec/op

| Syscall | Base       | New        | change % |
| ------- | ---------- | ---------- | -------- |
| basic   | 0.173212   | 0.133294   | -23.05   |
| execve  | 363.176190 | 357.148150 | -1.66    |
| fork    | 853.657880 | 840.268800 | -1.57    |
| getpgid | 0.174832   | 0.135372   | -22.57   |

perf bench syscall ops/sec

| Syscall | Base    | New     | change % |
| ------- | ------- | ------- | -------- |
| basic   | 6006021 | 7502236 | +24.91   |
| execve  | 2753    | 2800    | +1.71    |
| fork    | 1171    | 1190    | +1.62    |
| getpgid | 5942117 | 7387040 | +24.32   |

IPI latency benchmark

| Metric         | Base (ns)     | Test (ns)     | change % |
| -------------- | ------------- | ------------- | -------- |
| Dry-run        | 206652.45     | 209317.37     | +1.29    |
| Self-IPI       | 3567895.23    | 3590444.77    | +0.63    |
| Normal IPI     | 148570416.17  | 148564173.40  | -0.00    |
| Broadcast IPI  | 4033489673.38 | 4007319512.62 | -0.65    |
| Broadcast lock | 4011023005.48 | 4010267885.93 | -0.02    |

Thats very close to performance earlier with arch specific handling.

Tests done:
- Build and boot on ppc64le pseries.
- Build and boot on ppc64le powernv8 powernv9 powernv10.
- Build and boot on ppc32.
- Performance benchmark done with perf syscall basic on pseries.

Changelog:

V2 -> V3
- #ifdef CONFIG_GENERIC_IRQ_ENTRY removed from unnecessary places
- Some functions made __always_inline
- pt_regs padding changed to match 16byte interrupt stack alignment
- And some cosmetic changes from reviews from earlier patch
V2: https://lore.kernel.org/all/[email protected]/

V1 -> V2
- Fix an issue where context tracking was showing warnings for incorrect context
V1: https://lore.kernel.org/all/[email protected]/

RFC -> PATCH V1
- Fix for ppc32 spitting out kuap lock warnings.
- ppc64le powernv8 crash fix.
- Review comments incorporated from previous RFC.
RFC https://lore.kernel.org/all/[email protected]/

Mukesh Kumar Chaurasiya (8):
powerpc: rename arch_irq_disabled_regs
powerpc: Prepare to build with generic entry/exit framework
powerpc: introduce arch_enter_from_user_mode
powerpc: Introduce syscall exit arch functions
powerpc: add exit_flags field in pt_regs
powerpc: Prepare for IRQ entry exit
powerpc: Enable IRQ generic entry/exit path.
powerpc: Enable Generic Entry/Exit for syscalls.

arch/powerpc/Kconfig                    |   2 +
arch/powerpc/include/asm/entry-common.h | 533 ++++++++++++++++++++++++
arch/powerpc/include/asm/hw_irq.h       |   4 +-
arch/powerpc/include/asm/interrupt.h    | 386 +++--------------
arch/powerpc/include/asm/ptrace.h       |   6 +-
arch/powerpc/include/asm/signal.h       |   1 -
arch/powerpc/include/asm/stacktrace.h   |   6 +
arch/powerpc/include/asm/syscall.h      |   5 +
arch/powerpc/include/asm/thread_info.h  |   1 +
arch/powerpc/include/uapi/asm/ptrace.h  |  14 +-
arch/powerpc/kernel/interrupt.c         | 255 ++----------
arch/powerpc/kernel/ptrace/ptrace.c     | 142 +------
arch/powerpc/kernel/signal.c            |  25 +-
arch/powerpc/kernel/syscall.c           | 119 +-----
arch/powerpc/kernel/traps.c             |   2 +-
arch/powerpc/kernel/watchdog.c          |   2 +-
arch/powerpc/perf/core-book3s.c         |   2 +-
17 files changed, 681 insertions(+), 824 deletions(-)

create mode 100644 arch/powerpc/include/asm/entry-common.h

Signature · 1 line
2.52.0
#15 lore raw

Hi,

Le 08/01/2026 à 06:16, Venkat a écrit :

Quoted 6 lines

With this patch set, build fails.

Failure logs:

ld: init/main.o: in function `do_one_initcall':
/root/linux/init/main.c:1384: undefined reference to `memcpy'

I guess you have KASAN enabled ?

See https://lore.kernel.org/all/[email protected]/

This is because of commit 69d4c0d32186 ("entry, kasan, x86: Disallow overriding mem*() functions") which changed the core behaviour in 2023 without taking care of other architectures than x86 who was the only one with CONFIG_GENERIC_ENTRY ?

For powerpc the expected behaviour introduced in 2019 by commit 26deb04342e3 ("powerpc: prepare string/mem functions for KASAN") needs to be modified the same way to fit with the core change.

Christophe

#16 lore raw
Quoted 20 lines · Christophe Leroy (CS GROUP)

Hi,

Le 08/01/2026 à 06:16, Venkat a écrit :

Quoted 6 lines

With this patch set, build fails.

Failure logs:

ld: init/main.o: in function `do_one_initcall':
/root/linux/init/main.c:1384: undefined reference to `memcpy'

I guess you have KASAN enabled ?

See
https://lore.kernel.org/all/[email protected]/

This is because of commit 69d4c0d32186 ("entry, kasan, x86: Disallow overriding mem*() functions") which changed the core behaviour in 2023 without taking care of other architectures than x86 who was the only one with CONFIG_GENERIC_ENTRY ?

Quoted 5 lines

For powerpc the expected behaviour introduced in 2019 by commit 26deb04342e3 ("powerpc: prepare string/mem functions for KASAN") needs to be modified the same way to fit with the core change.

Christophe

Hey Christophe, Venkat

I am working on this. I'll send out a new revision soon.

Regards,
Mukesh