Generic IRQ entry/exit support for powerpc
Subsystems (5)
v5 cover Generic IRQ entry/exit support for powerpc
Adding support for the generic irq entry/exit handling for PowerPC. The goal is to bring PowerPC in line with other architectures that already use the common irq entry infrastructure, reducing duplicated code and making it easier to share future changes in entry/exit paths.
This is slightly tested of ppc64le and ppc32.
The performance benchmarks are below:
perf bench syscall usec/op (-ve is improvement)
| Syscall | Base | test | change % | | ------- | ----------- | ----------- | -------- | | basic | 0.093543 | 0.093023 | -0.56 | | execve | 446.557781 | 450.107172 | +0.79 | | fork | 1142.204391 | 1156.377214 | +1.24 | | getpgid | 0.097666 | 0.092677 | -5.11 |
perf bench syscall ops/sec (+ve is improvement)
| Syscall | Base | New | change % | | ------- | -------- | -------- | -------- | | basic | 10690548 | 10750140 | +0.56 | | execve | 2239 | 2221 | -0.80 | | fork | 875 | 864 | -1.26 | | getpgid | 10239026 | 10790324 | +5.38 |
IPI latency benchmark (-ve is improvement)
| Metric | Base (ns) | New (ns) | % Change | | -------------- | ------------- | ------------- | -------- | | Dry run | 583136.56 | 584136.35 | 0.17% | | Self IPI | 4167393.42 | 4149093.90 | -0.44% | | Normal IPI | 61769347.82 | 61753728.39 | -0.03% | | Broadcast IPI | 2235584825.02 | 2227521401.45 | -0.36% | | Broadcast lock | 2164964433.31 | 2125658641.76 | -1.82% |
Thats very close to performance earlier with arch specific handling.
Tests done:
- Build and boot on ppc64le pseries.
- Build and boot on ppc64le powernv8 powernv9 powernv10.
- Build and boot on ppc32.
- Performance benchmark done with perf syscall basic on pseries.
Changelog:
V4 -> V5:
- Rebased on latest mainline
V4: https://lore.kernel.org/all/[email protected]/
V3 -> V4:
- Fixed the issue in older gcc version where linker couldn't find mem functions
- Merged IRQ enable and syscall enable into a single patch
- Cleanup for unused functions done in separate patch.
- Some other cosmetic changes
V3: https://lore.kernel.org/all/[email protected]/
V2 -> V3:
- #ifdef CONFIG_GENERIC_IRQ_ENTRY removed from unnecessary places
- Some functions made __always_inline
- pt_regs padding changed to match 16byte interrupt stack alignment
- And some cosmetic changes from reviews from earlier patch
V2: https://lore.kernel.org/all/[email protected]/
V1 -> V2:
- Fix an issue where context tracking was showing warnings for incorrect context
V1: https://lore.kernel.org/all/[email protected]/
RFC -> PATCH V1:
- Fix for ppc32 spitting out kuap lock warnings.
- ppc64le powernv8 crash fix.
- Review comments incorporated from previous RFC.
RFC https://lore.kernel.org/all/[email protected]/
Mukesh Kumar Chaurasiya (8):
powerpc: rename arch_irq_disabled_regs
powerpc: Prepare to build with generic entry/exit framework
powerpc: introduce arch_enter_from_user_mode
powerpc: Introduce syscall exit arch functions
powerpc: add exit_flags field in pt_regs
powerpc: Prepare for IRQ entry exit
powerpc: Enable GENERIC_ENTRY feature
powerpc: Remove unused functions
arch/powerpc/Kconfig | 1 + arch/powerpc/include/asm/entry-common.h | 533 ++++++++++++++++++++++++ arch/powerpc/include/asm/hw_irq.h | 4 +- arch/powerpc/include/asm/interrupt.h | 386 +++-------------- arch/powerpc/include/asm/kasan.h | 15 +- arch/powerpc/include/asm/ptrace.h | 6 +- arch/powerpc/include/asm/signal.h | 1 - arch/powerpc/include/asm/stacktrace.h | 6 + arch/powerpc/include/asm/syscall.h | 5 + arch/powerpc/include/asm/thread_info.h | 1 + arch/powerpc/include/uapi/asm/ptrace.h | 14 +- arch/powerpc/kernel/interrupt.c | 254 ++--------- arch/powerpc/kernel/ptrace/ptrace.c | 142 +------ arch/powerpc/kernel/signal.c | 25 +- arch/powerpc/kernel/syscall.c | 119 +----- arch/powerpc/kernel/traps.c | 2 +- arch/powerpc/kernel/watchdog.c | 2 +- arch/powerpc/perf/core-book3s.c | 2 +- 18 files changed, 690 insertions(+), 828 deletions(-) create mode 100644 arch/powerpc/include/asm/entry-common.h
Signature · 1 line
2.53.0
PATCH v5 1/8 powerpc: rename arch_irq_disabled_regs
From: Mukesh Kumar Chaurasiya <[email protected]>
Rename arch_irq_disabled_regs() to regs_irqs_disabled() to align with the naming used in the generic irqentry framework. This makes the function available for use both in the PowerPC architecture code and in the common entry/exit paths shared with other architectures.
This is a preparatory change for enabling the generic irqentry framework on PowerPC.
--- arch/powerpc/include/asm/hw_irq.h | 4 ++-- arch/powerpc/include/asm/interrupt.h | 16 ++++++++-------- arch/powerpc/kernel/interrupt.c | 4 ++-- arch/powerpc/kernel/syscall.c | 2 +- arch/powerpc/kernel/traps.c | 2 +- arch/powerpc/kernel/watchdog.c | 2 +- arch/powerpc/perf/core-book3s.c | 2 +- 7 files changed, 16 insertions(+), 16 deletions(-)
arch/powerpc/include/asm/hw_irq.h +2 -2
@@ -393,7 +393,7 @@ static inline void do_hard_irq_enable(void)__hard_irq_enable (); } -static inline bool arch_irq_disabled_regs (struct pt_regs *regs) +static inline bool regs_irqs_disabled (struct pt_regs *regs) {return (regs->softe &IRQS_DISABLED ); }
@@ -466,7 +466,7 @@ static inline bool arch_irqs_disabled(void)#define hard_irq_disable () arch_local_irq_disable() -static inline bool arch_irq_disabled_regs (struct pt_regs *regs) +static inline bool regs_irqs_disabled (struct pt_regs *regs) {return !(regs->msr &MSR_EE ); }
arch/powerpc/include/asm/interrupt.h +8 -8
@@ -172,7 +172,7 @@ static inline void interrupt_enter_prepare(struct pt_regs *regs)/* Enable MSR[RI] early, to support kernel SLB and hash faults */ #endif -if (!arch_irq_disabled_regs (regs)) +if (!regs_irqs_disabled (regs))trace_hardirqs_off ();if (user_mode (regs)) {
@@ -192,11 +192,11 @@ static inline void interrupt_enter_prepare(struct pt_regs *regs)CT_WARN_ON (ct_state () !=CT_STATE_KERNEL &&ct_state () !=CT_STATE_IDLE );INT_SOFT_MASK_BUG_ON (regs,is_implicit_soft_masked (regs)); -INT_SOFT_MASK_BUG_ON (regs,arch_irq_disabled_regs (regs) && -search_kernel_restart_table (regs->nip)); +INT_SOFT_MASK_BUG_ON (regs,regs_irqs_disabled (regs) && +search_kernel_restart_table (regs->nip)); } -INT_SOFT_MASK_BUG_ON (regs, !arch_irq_disabled_regs (regs) && - !(regs->msr &MSR_EE )); +INT_SOFT_MASK_BUG_ON (regs, !regs_irqs_disabled (regs) && + !(regs->msr &MSR_EE ));booke_restore_dbcr0 (); }
@@ -298,7 +298,7 @@ static inline void interrupt_nmi_enter_prepare(struct pt_regs *regs, struct inte* Adjust regs->softe to be soft-masked if it had not been * reconcied (e.g., interrupt entry with MSR[EE]=0 but softe * not yet set disabled), or if it was in an implicit soft -* masked state. This makes arch_irq_disabled_regs(regs) +* masked state. This makes regs_irqs_disabled(regs) * behave as expected. */ regs->softe =IRQS_ALL_DISABLED ;
@@ -372,7 +372,7 @@ static inline void interrupt_nmi_exit_prepare(struct pt_regs *regs, struct inter #ifdef CONFIG_PPC64 #ifdef CONFIG_PPC_BOOK3S - if (arch_irq_disabled_regs(regs)) { + if (regs_irqs_disabled(regs)) { unsigned long rst = search_kernel_restart_table(regs->nip); if (rst) regs_set_return_ip(regs, rst);
@@ -661,7 +661,7 @@ void replay_soft_interrupts(void);static inline void interrupt_cond_local_irq_enable (struct pt_regs *regs) { -if (!arch_irq_disabled_regs (regs)) +if (!regs_irqs_disabled (regs))local_irq_enable (); }
arch/powerpc/kernel/interrupt.c +2 -2
@@ -347,7 +347,7 @@ notrace unsigned long interrupt_exit_user_prepare(struct pt_regs *regs)unsigned long ret;BUG_ON (regs_is_unrecoverable (regs)); -BUG_ON (arch_irq_disabled_regs (regs)); +BUG_ON (regs_irqs_disabled (regs));CT_WARN_ON (ct_state () ==CT_STATE_USER );/*
@@ -396,7 +396,7 @@ notrace unsigned long interrupt_exit_kernel_prepare(struct pt_regs *regs)local_irq_disable (); -if (!arch_irq_disabled_regs (regs)) { +if (!regs_irqs_disabled (regs)) {/* Returning to a kernel context with local irqs enabled. */ WARN_ON_ONCE (!(regs->msr &MSR_EE )); again:
arch/powerpc/kernel/syscall.c +1 -1
@@ -32,7 +32,7 @@ notrace long system_call_exception(struct pt_regs *regs, unsigned long r0)BUG_ON (regs_is_unrecoverable (regs));BUG_ON (!user_mode (regs)); -BUG_ON (arch_irq_disabled_regs (regs)); +BUG_ON (regs_irqs_disabled (regs));#ifdef CONFIG_PPC_PKEY if (mmu_has_feature (MMU_FTR_PKEY )) {
arch/powerpc/kernel/traps.c +1 -1
@@ -1956,7 +1956,7 @@ DEFINE_INTERRUPT_HANDLER_RAW(performance_monitor_exception)* prevent hash faults on user addresses when reading callchains (and * looks better from an irq tracing perspective). */ -if (IS_ENABLED (CONFIG_PPC64 ) &&unlikely (arch_irq_disabled_regs (regs))) +if (IS_ENABLED (CONFIG_PPC64 ) &&unlikely (regs_irqs_disabled (regs)))performance_monitor_exception_nmi (regs);else performance_monitor_exception_async (regs);
arch/powerpc/kernel/watchdog.c +1 -1
@@ -376,7 +376,7 @@ DEFINE_INTERRUPT_HANDLER_NMI(soft_nmi_interrupt)u64 tb;/* should only arrive from kernel, with irqs disabled */ -WARN_ON_ONCE (!arch_irq_disabled_regs (regs)); +WARN_ON_ONCE (!regs_irqs_disabled (regs));if (!cpumask_test_cpu (cpu, &wd_cpus_enabled))return 0 ;
arch/powerpc/perf/core-book3s.c +1 -1
@@ -2482,7 +2482,7 @@ static void __perf_event_interrupt(struct pt_regs *regs)* will trigger a PMI after waking up from idle. Since counter values are _not_ * saved/restored in idle path, can lead to below "Can't find PMC" message. */ -if (unlikely (!found) && !arch_irq_disabled_regs (regs)) +if (unlikely (!found) && !regs_irqs_disabled (regs))printk_ratelimited (KERN_WARNING "Can't find PMC that caused IRQ\n" );/*
Signature · 1 line
2.53.0
PATCH v5 2/8 powerpc: Prepare to build with generic entry/exit framework
In reply to Mukesh Kumar Chaurasiya (IBM)From: Mukesh Kumar Chaurasiya <[email protected]>
This patch introduces preparatory changes needed to support building PowerPC with the generic entry/exit (irqentry) framework.
The following infrastructure updates are added:
- Add a syscall_work field to struct thread_info to hold SYSCALL_WORK_* flags.
- Provide a stub implementation of arch_syscall_is_vdso_sigreturn(), returning false for now.
- Introduce on_thread_stack() helper to detect if the current stack pointer lies within the task’s kernel stack.
These additions enable later integration with the generic entry/exit infrastructure while keeping existing PowerPC behavior unchanged.
No functional change is intended in this patch.
--- arch/powerpc/include/asm/entry-common.h | 8 ++++++++ arch/powerpc/include/asm/stacktrace.h | 6 ++++++ arch/powerpc/include/asm/syscall.h | 5 +++++ arch/powerpc/include/asm/thread_info.h | 1 + 4 files changed, 20 insertions(+) create mode 100644 arch/powerpc/include/asm/entry-common.h
arch/powerpc/include/asm/entry-common.h new file +8 -0
@@ -0,0 +1,8 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef _ASM_PPC_ENTRY_COMMON_H +#define _ASM_PPC_ENTRY_COMMON_H + +#include <asm/stacktrace.h> + +#endif /* _ASM_PPC_ENTRY_COMMON_H */
arch/powerpc/include/asm/stacktrace.h +6 -0
@@ -10,4 +10,10 @@void show_user_instructions (struct pt_regs *regs); +static __always_inline bool on_thread_stack(void) +{ +return !(((unsigned long )(current->stack) ^ current_stack_pointer) + & ~(THREAD_SIZE -1 )); +} +#endif /* _ASM_POWERPC_STACKTRACE_H */
arch/powerpc/include/asm/syscall.h +5 -0
@@ -139,4 +139,9 @@ static inline int syscall_get_arch(struct task_struct *task) else return AUDIT_ARCH_PPC64; } + +static inline bool arch_syscall_is_vdso_sigreturn (struct pt_regs *regs) +{ +return false; +}#endif /* _ASM_SYSCALL_H */
arch/powerpc/include/asm/thread_info.h +1 -0
@@ -57,6 +57,7 @@ struct thread_info {#ifdef CONFIG_SMP unsigned int cpu;#endif +unsigned long syscall_work;/* SYSCALL_WORK_ flags */ unsigned long local_flags;/* private flags for thread */ #ifdef CONFIG_LIVEPATCH_64 unsigned long *livepatch_sp;
Signature · 1 line
2.53.0
PATCH v5 3/8 powerpc: introduce arch_enter_from_user_mode
In reply to Mukesh Kumar Chaurasiya (IBM)From: Mukesh Kumar Chaurasiya <[email protected]>
Implement the arch_enter_from_user_mode() hook required by the generic entry/exit framework. This helper prepares the CPU state when entering the kernel from userspace, ensuring correct handling of KUAP/KUEP, transactional memory, and debug register state.
This patch contains no functional changes, it is purely preparatory for enabling the generic syscall and interrupt entry path on PowerPC.
--- arch/powerpc/include/asm/entry-common.h | 118 ++++++++++++++++++++++++ 1 file changed, 118 insertions(+)
arch/powerpc/include/asm/entry-common.h +118 -0
@@ -3,6 +3,124 @@#ifndef _ASM_PPC_ENTRY_COMMON_H#define _ASM_PPC_ENTRY_COMMON_H +#include <asm/cputime.h> +#include <asm/interrupt.h> #include <asm/stacktrace.h> +#include <asm/tm.h> + +static __always_inline void booke_load_dbcr0(void) +{ +#ifdef CONFIG_PPC_ADV_DEBUG_REGS +unsigned long dbcr0 = current->thread.debug.dbcr0; + +if (likely (!(dbcr0 &DBCR0_IDM ))) +return ; + +/* +* Check to see if the dbcr0 register is set up to debug. +* Use the internal debug mode bit to do this. +*/ +mtmsr (mfmsr () & ~MSR_DE ); +if (IS_ENABLED (CONFIG_PPC32 )) { +isync (); + global_dbcr0[smp_processor_id ()] =mfspr (SPRN_DBCR0 ); + } +mtspr (SPRN_DBCR0 , dbcr0); +mtspr (SPRN_DBSR ,-1 ); +#endif +} + +static __always_inline void arch_enter_from_user_mode(struct pt_regs *regs) +{ +kuap_lock (); + +if (IS_ENABLED (CONFIG_PPC_IRQ_SOFT_MASK_DEBUG )) +BUG_ON (irq_soft_mask_return () !=IRQS_ALL_DISABLED ); + +BUG_ON (regs_is_unrecoverable (regs)); +BUG_ON (!user_mode (regs)); +BUG_ON (regs_irqs_disabled (regs)); + +#ifdef CONFIG_PPC_PKEY +if (mmu_has_feature (MMU_FTR_PKEY ) &&trap_is_syscall (regs)) { +unsigned long amr, iamr; +bool flush_needed = false; +/* +* When entering from userspace we mostly have the AMR/IAMR +* different from kernel default values. Hence don't compare. +*/ + amr =mfspr (SPRN_AMR ); + iamr =mfspr (SPRN_IAMR ); + regs->amr = amr; + regs->iamr = iamr; +if (mmu_has_feature (MMU_FTR_KUAP )) { +mtspr (SPRN_AMR ,AMR_KUAP_BLOCKED ); + flush_needed = true; + } +if (mmu_has_feature (MMU_FTR_BOOK3S_KUEP )) { +mtspr (SPRN_IAMR ,AMR_KUEP_BLOCKED ); + flush_needed = true; + } +if (flush_needed) +isync (); + } +#endif +kuap_assert_locked (); +booke_restore_dbcr0 (); +account_cpu_user_entry (); +account_stolen_time (); + +/* +* This is not required for the syscall exit path, but makes the +* stack frame look nicer. If this was initialised in the first stack +* frame, or if the unwinder was taught the first stack frame always +* returns to user with IRQS_ENABLED, this store could be avoided! +*/ +irq_soft_mask_regs_set_state (regs,IRQS_ENABLED ); + +/* +* If system call is called with TM active, set _TIF_RESTOREALL to +* prevent RFSCV being used to return to userspace, because POWER9 +* TM implementation has problems with this instruction returning to +* transactional state. Final register values are not relevant because +* the transaction will be aborted upon return anyway. Or in the case +* of unsupported_scv SIGILL fault, the return state does not much +* matter because it's an edge case. +*/ +if (IS_ENABLED (CONFIG_PPC_TRANSACTIONAL_MEM ) && +unlikely (MSR_TM_TRANSACTIONAL (regs->msr))) +set_bits (_TIF_RESTOREALL, ¤t_thread_info ()->flags); + +/* +* If the system call was made with a transaction active, doom it and +* return without performing the system call. Unless it was an +* unsupported scv vector, in which case it's treated like an illegal +* instruction. +*/ +#ifdef CONFIG_PPC_TRANSACTIONAL_MEM +if (unlikely (MSR_TM_TRANSACTIONAL (regs->msr)) && + !trap_is_unsupported_scv (regs)) { +/* Enable TM in the kernel, and disable EE (for scv) */ +hard_irq_disable (); +mtmsr (mfmsr () |MSR_TM ); + +/* tabort, this dooms the transaction, nothing else */ + asmvolatile (".long 0x7c00071d | ((%0) << 16)" + ::"r" (TM_CAUSE_SYSCALL |TM_CAUSE_PERSISTENT )); + +/* +* Userspace will never see the return value. Execution will +* resume after the tbegin. of the aborted transaction with the +* checkpointed register state. A context switch could occur +* or signal delivered to the process before resuming the +* doomed transaction context, but that should all be handled +* as expected. +*/ +return ; + } +#endif /* CONFIG_PPC_TRANSACTIONAL_MEM */ +} + +#define arch_enter_from_user_mode arch_enter_from_user_mode#endif /* _ASM_PPC_ENTRY_COMMON_H */
Signature · 1 line
2.53.0
PATCH v5 4/8 powerpc: Introduce syscall exit arch functions
In reply to Mukesh Kumar Chaurasiya (IBM)From: Mukesh Kumar Chaurasiya <[email protected]>
Add PowerPC-specific implementations of the generic syscall exit hooks used by the generic entry/exit framework:
- arch_exit_to_user_mode_work_prepare()
- arch_exit_to_user_mode_work()
These helpers handle user state restoration when returning from the kernel to userspace, including FPU/VMX/VSX state, transactional memory, KUAP restore, and per-CPU accounting.
Additionally, move check_return_regs_valid() from interrupt.c to interrupt.h so it can be shared by the new entry/exit logic.
No functional change is intended with this patch.
--- arch/powerpc/include/asm/entry-common.h | 49 +++++++++++++++++++++++++ 1 file changed, 49 insertions(+)
arch/powerpc/include/asm/entry-common.h +49 -0
@@ -6,6 +6,7 @@#include <asm/cputime.h> #include <asm/interrupt.h> #include <asm/stacktrace.h> +#include <asm/switch_to.h> #include <asm/tm.h> static __always_inline void booke_load_dbcr0(void)
@@ -123,4 +124,52 @@ static __always_inline void arch_enter_from_user_mode(struct pt_regs *regs)#define arch_enter_from_user_mode arch_enter_from_user_mode +static inline void arch_exit_to_user_mode_prepare (struct pt_regs *regs, +unsigned long ti_work) +{ +unsigned long mathflags; + +if (IS_ENABLED (CONFIG_PPC_BOOK3S_64 ) &&IS_ENABLED (CONFIG_PPC_FPU )) { +if (IS_ENABLED (CONFIG_PPC_TRANSACTIONAL_MEM ) && +unlikely ((ti_work & _TIF_RESTORE_TM))) { +restore_tm_state (regs); + }else { + mathflags =MSR_FP ; + +if (cpu_has_feature (CPU_FTR_VSX )) + mathflags |=MSR_VEC |MSR_VSX ; +else if (cpu_has_feature (CPU_FTR_ALTIVEC )) + mathflags |=MSR_VEC ; + +/* +* If userspace MSR has all available FP bits set, +* then they are live and no need to restore. If not, +* it means the regs were given up and restore_math +* may decide to restore them (to avoid taking an FP +* fault). +*/ +if ((regs->msr & mathflags) != mathflags) +restore_math (regs); + } + } + +check_return_regs_valid (regs); +#ifdef CONFIG_PPC_TRANSACTIONAL_MEM + local_paca->tm_scratch = regs->msr; +#endif +/* Restore user access locks last */ +kuap_user_restore (regs); +} + +#define arch_exit_to_user_mode_prepare arch_exit_to_user_mode_prepare + +static __always_inline void arch_exit_to_user_mode(void) +{ +booke_load_dbcr0 (); + +account_cpu_user_exit (); +} + +#define arch_exit_to_user_mode arch_exit_to_user_mode +#endif /* _ASM_PPC_ENTRY_COMMON_H */
Signature · 1 line
2.53.0
PATCH v5 5/8 powerpc: add exit_flags field in pt_regs
In reply to Mukesh Kumar Chaurasiya (IBM)From: Mukesh Kumar Chaurasiya <[email protected]>
Add a new field exit_flags in the pt_regs structure. This field will hold the flags set during interrupt or syscall execution that are required during exit to user mode.
Specifically, the TIF_RESTOREALL flag, stored in this field, helps the exit routine determine if any NVGPRs were modified and need to be restored before returning to userspace.
This addition ensures a clean and architecture-specific mechanism to track per-syscall or per-interrupt state transitions related to register restore.
Changes:
- Add exit_flags and __pt_regs_pad to maintain 16-byte stack alignment
- Update asm-offsets.c and ptrace.c for offset and validation
- Update PT_* constants in uapi header to reflect the new layout
--- arch/powerpc/include/asm/ptrace.h | 3 +++ arch/powerpc/include/uapi/asm/ptrace.h | 14 +++++++++----- arch/powerpc/kernel/ptrace/ptrace.c | 1 + 3 files changed, 13 insertions(+), 5 deletions(-)
arch/powerpc/include/asm/ptrace.h +3 -0
@@ -53,6 +53,9 @@ struct pt_regsunsigned long esr; };unsigned long result; +unsigned long exit_flags; +/* Maintain 16 byte interrupt stack alignment */ +unsigned long __pt_regs_pad[3 ]; }; };#if defined(CONFIG_PPC64 ) || defined(CONFIG_PPC_KUAP )
arch/powerpc/include/uapi/asm/ptrace.h +9 -5
@@ -55,6 +55,8 @@ struct pt_regsunsigned long dar;/* Fault registers */ unsigned long dsisr;/* on 4xx/Book-E used for ESR */ unsigned long result;/* Result of a system call */ +unsigned long exit_flags;/* System call exit flags */ + unsigned long __pt_regs_pad[3]; /* Maintain 16 byte interrupt stack alignment */ };#endif /* __ASSEMBLER__ */
@@ -114,10 +116,12 @@ struct pt_regs#define PT_DAR 41#define PT_DSISR 42#define PT_RESULT 43 -#define PT_DSCR 44 -#define PT_REGS_COUNT 44 +#define PT_EXIT_FLAGS 44 +#define PT_PAD 47/* 3 times */ +#define PT_DSCR 48 +#define PT_REGS_COUNT 48 -#define PT_FPR0 48/* each FP reg occupies 2 slots in this space */ +#define PT_FPR0 (PT_REGS_COUNT + 4)/* each FP reg occupies 2 slots in this space */ #ifndef __powerpc64__
@@ -129,7 +133,7 @@ struct pt_regs#define PT_FPSCR (PT_FPR0 + 32)/* each FP reg occupies 1 slot in 64-bit space */ -#define PT_VR0 82/* each Vector reg occupies 2 slots in 64-bit */ +#define PT_VR0 (PT_FPSCR + 2)/* <82> each Vector reg occupies 2 slots in 64-bit */ #define PT_VSCR (PT_VR0 + 32*2 + 1)#define PT_VRSAVE (PT_VR0 + 33*2)
@@ -137,7 +141,7 @@ struct pt_regs/* * Only store first 32 VSRs here. The second 32 VSRs in VR0-31 */ -#define PT_VSR0 150/* each VSR reg occupies 2 slots in 64-bit */ +#define PT_VSR0 (PT_VRSAVE + 2)/* each VSR reg occupies 2 slots in 64-bit */ #define PT_VSR31 (PT_VSR0 + 2*31)#endif /* __powerpc64__ */
arch/powerpc/kernel/ptrace/ptrace.c +1 -0
@@ -432,6 +432,7 @@ void __init pt_regs_check(void)CHECK_REG (PT_DAR , dar);CHECK_REG (PT_DSISR , dsisr);CHECK_REG (PT_RESULT , result); +CHECK_REG (PT_EXIT_FLAGS , exit_flags);#undef CHECK_REGBUILD_BUG_ON (PT_REGS_COUNT !=sizeof (struct user_pt_regs ) /sizeof (unsigned long ));
Signature · 1 line
2.53.0
PATCH v5 6/8 powerpc: Prepare for IRQ entry exit
In reply to Mukesh Kumar Chaurasiya (IBM)From: Mukesh Kumar Chaurasiya <[email protected]>
Move interrupt entry and exit helper routines from interrupt.h into the PowerPC-specific entry-common.h header as a preparatory step for enabling the generic entry/exit framework.
This consolidation places all PowerPC interrupt entry/exit handling in a single common header, aligning with the generic entry infrastructure. The helpers provide architecture-specific handling for interrupt and NMI entry/exit sequences, including:
- arch_interrupt_enter/exit_prepare()
- arch_interrupt_async_enter/exit_prepare()
- arch_interrupt_nmi_enter/exit_prepare()
- Supporting helpers such as nap_adjust_return(), check_return_regs_valid(), debug register maintenance, and soft mask handling.
The functions are copied verbatim from interrupt.h.Subsequent patches will integrate these routines into the generic entry/exit flow.
No functional change intended.
--- arch/powerpc/include/asm/entry-common.h | 358 ++++++++++++++++++++++++ 1 file changed, 358 insertions(+)
arch/powerpc/include/asm/entry-common.h +358 -0
@@ -5,10 +5,75 @@#include <asm/cputime.h> #include <asm/interrupt.h> +#include <asm/runlatch.h> #include <asm/stacktrace.h> #include <asm/switch_to.h> #include <asm/tm.h> +#ifdef CONFIG_PPC_IRQ_SOFT_MASK_DEBUG +/* +* WARN/BUG is handled with a program interrupt so minimise checks here to +* avoid recursion and maximise the chance of getting the first oops handled. +*/ +#define INT_SOFT_MASK_BUG_ON (regs, cond) \ +do { \ + if ((user_mode(regs) || (TRAP(regs) != INTERRUPT_PROGRAM))) \ + BUG_ON(cond); \ +} while (0) +#else +#define INT_SOFT_MASK_BUG_ON (regs, cond) +#endif + +#ifdef CONFIG_PPC_BOOK3S_64 +extern char __end_soft_masked[]; +bool search_kernel_soft_mask_table (unsigned long addr); +unsigned long search_kernel_restart_table (unsigned long addr); + +DECLARE_STATIC_KEY_FALSE (interrupt_exit_not_reentrant); + +static inline bool is_implicit_soft_masked (struct pt_regs *regs) +{ +if (user_mode (regs)) +return false; + +if (regs->nip >= (unsigned long )__end_soft_masked) +return false; + +return search_kernel_soft_mask_table (regs->nip); +} + +static inline void srr_regs_clobbered (void ) +{ + local_paca->srr_valid =0 ; + local_paca->hsrr_valid =0 ; +} +#else +static inline unsigned long search_kernel_restart_table (unsigned long addr) +{ +return 0 ; +} + +static inline bool is_implicit_soft_masked (struct pt_regs *regs) +{ +return false; +} + +static inline void srr_regs_clobbered (void ) +{ +} +#endif + +static inline void nap_adjust_return (struct pt_regs *regs) +{ +#ifdef CONFIG_PPC_970_NAP +if (unlikely (test_thread_local_flags (_TLF_NAPPING))) { +/* Can avoid a test-and-clear because NMIs do not call this */ +clear_thread_local_flags (_TLF_NAPPING); +regs_set_return_ip (regs, (unsigned long )power4_idle_nap_return); + } +#endif +} + static __always_inline void booke_load_dbcr0(void) {#ifdef CONFIG_PPC_ADV_DEBUG_REGS
@@ -31,6 +96,299 @@ static __always_inline void booke_load_dbcr0(void)#endif } +static inline void booke_restore_dbcr0 (void ) +{ +#ifdef CONFIG_PPC_ADV_DEBUG_REGS +unsigned long dbcr0 = current->thread.debug.dbcr0; + +if (IS_ENABLED (CONFIG_PPC32 ) &&unlikely (dbcr0 &DBCR0_IDM )) { +mtspr (SPRN_DBSR ,-1 ); +mtspr (SPRN_DBCR0 , global_dbcr0[smp_processor_id ()]); + } +#endif +} + +static inline void check_return_regs_valid (struct pt_regs *regs) +{ +#ifdef CONFIG_PPC_BOOK3S_64 +unsigned long trap, srr0, srr1; +static bool warned; +u8 *validp; +char *h; + +if (trap_is_scv (regs)) +return ; + + trap =TRAP (regs); +// EE in HV mode sets HSRRs like 0xea0 +if (cpu_has_feature (CPU_FTR_HVMODE ) && trap ==INTERRUPT_EXTERNAL ) + trap =0xea0 ; + +switch (trap) { +case 0x980 : +case INTERRUPT_H_DATA_STORAGE : +case 0xe20 : +case 0xe40 : +case INTERRUPT_HMI : +case 0xe80 : +case 0xea0 : +case INTERRUPT_H_FAC_UNAVAIL : +case 0x1200 : +case 0x1500 : +case 0x1600 : +case 0x1800 : + validp = &local_paca->hsrr_valid; +if (!READ_ONCE (*validp)) +return ; + + srr0 =mfspr (SPRN_HSRR0 ); + srr1 =mfspr (SPRN_HSRR1 ); + h ="H" ; + +break ; +default : + validp = &local_paca->srr_valid; +if (!READ_ONCE (*validp)) +return ; + + srr0 =mfspr (SPRN_SRR0 ); + srr1 =mfspr (SPRN_SRR1 ); + h ="" ; +break ; + } + +if (srr0 == regs->nip && srr1 == regs->msr) +return ; + +/* +* A NMI / soft-NMI interrupt may have come in after we found +* srr_valid and before the SRRs are loaded. The interrupt then +* comes in and clobbers SRRs and clears srr_valid. Then we load +* the SRRs here and test them above and find they don't match. +* +* Test validity again after that, to catch such false positives. +* +* This test in general will have some window for false negatives +* and may not catch and fix all such cases if an NMI comes in +* later and clobbers SRRs without clearing srr_valid, but hopefully +* such things will get caught most of the time, statistically +* enough to be able to get a warning out. +*/ +if (!READ_ONCE (*validp)) +return ; + +if (!data_race (warned)) { +data_race (warned = true); +pr_warn ("%sSRR0 was: %lx should be: %lx\n" , h, srr0, regs->nip); +pr_warn ("%sSRR1 was: %lx should be: %lx\n" , h, srr1, regs->msr); +show_regs (regs); + } + +WRITE_ONCE (*validp,0 );/* fixup */ +#endif +} + +static inline void arch_interrupt_enter_prepare (struct pt_regs *regs) +{ +#ifdef CONFIG_PPC64 +irq_soft_mask_set (IRQS_ALL_DISABLED ); + +/* +* If the interrupt was taken with HARD_DIS clear, then enable MSR[EE]. +* Asynchronous interrupts get here with HARD_DIS set (see below), so +* this enables MSR[EE] for synchronous interrupts. IRQs remain +* soft-masked. The interrupt handler may later call +* interrupt_cond_local_irq_enable() to achieve a regular process +* context. +*/ +if (!(local_paca->irq_happened &PACA_IRQ_HARD_DIS )) { +INT_SOFT_MASK_BUG_ON (regs, !(regs->msr &MSR_EE )); +__hard_irq_enable (); + }else { +__hard_RI_enable (); + } +/* Enable MSR[RI] early, to support kernel SLB and hash faults */ +#endif + +if (!regs_irqs_disabled (regs)) +trace_hardirqs_off (); + +if (user_mode (regs)) { +kuap_lock (); +account_cpu_user_entry (); +account_stolen_time (); + }else { +kuap_save_and_lock (regs); +/* +* CT_WARN_ON comes here via program_check_exception, +* so avoid recursion. +*/ +if (TRAP (regs) !=INTERRUPT_PROGRAM ) +CT_WARN_ON (ct_state () !=CT_STATE_KERNEL && +ct_state () !=CT_STATE_IDLE ); +INT_SOFT_MASK_BUG_ON (regs,is_implicit_soft_masked (regs)); +INT_SOFT_MASK_BUG_ON (regs,regs_irqs_disabled (regs) && +search_kernel_restart_table (regs->nip)); + } +INT_SOFT_MASK_BUG_ON (regs, !regs_irqs_disabled (regs) && + !(regs->msr &MSR_EE )); + +booke_restore_dbcr0 (); +} + +/* +* Care should be taken to note that arch_interrupt_exit_prepare and +* arch_interrupt_async_exit_prepare do not necessarily return immediately to +* regs context (e.g., if regs is usermode, we don't necessarily return to +* user mode). Other interrupts might be taken between here and return, +* context switch / preemption may occur in the exit path after this, or a +* signal may be delivered, etc. +* +* The real interrupt exit code is platform specific, e.g., +* interrupt_exit_user_prepare / interrupt_exit_kernel_prepare for 64s. +* +* However arch_interrupt_nmi_exit_prepare does return directly to regs, because +* NMIs do not do "exit work" or replay soft-masked interrupts. +*/ +static inline void arch_interrupt_exit_prepare (struct pt_regs *regs) +{ +if (user_mode (regs)) { +BUG_ON (regs_is_unrecoverable (regs)); +BUG_ON (regs_irqs_disabled (regs)); +/* +* We don't need to restore AMR on the way back to userspace for KUAP. +* AMR can only have been unlocked if we interrupted the kernel. +*/ +kuap_assert_locked (); + +local_irq_disable (); + } +} + +static inline void arch_interrupt_async_enter_prepare (struct pt_regs *regs) +{ +#ifdef CONFIG_PPC64 +/* Ensure arch_interrupt_enter_prepare does not enable MSR[EE] */ + local_paca->irq_happened |=PACA_IRQ_HARD_DIS ; +#endif +arch_interrupt_enter_prepare (regs); +#ifdef CONFIG_PPC_BOOK3S_64 +/* +* RI=1 is set by arch_interrupt_enter_prepare, so this thread flags access +* has to come afterward (it can cause SLB faults). +*/ +if (cpu_has_feature (CPU_FTR_CTRL ) && + !test_thread_local_flags (_TLF_RUNLATCH)) +__ppc64_runlatch_on (); +#endif +} + +static inline void arch_interrupt_async_exit_prepare (struct pt_regs *regs) +{ +/* +* Adjust at exit so the main handler sees the true NIA. This must +* come before irq_exit() because irq_exit can enable interrupts, and +* if another interrupt is taken before nap_adjust_return has run +* here, then that interrupt would return directly to idle nap return. +*/ +nap_adjust_return (regs); + +arch_interrupt_exit_prepare (regs); +} + +struct interrupt_nmi_state { +#ifdef CONFIG_PPC64 +u8 irq_soft_mask; +u8 irq_happened; +u8 ftrace_enabled; +u64 softe; +#endif +}; + +static inline bool nmi_disables_ftrace (struct pt_regs *regs) +{ +/* Allow DEC and PMI to be traced when they are soft-NMI */ +if (IS_ENABLED (CONFIG_PPC_BOOK3S_64 )) { +if (TRAP (regs) ==INTERRUPT_DECREMENTER ) +return false; +if (TRAP (regs) ==INTERRUPT_PERFMON ) +return false; + } +if (IS_ENABLED (CONFIG_PPC_BOOK3E_64 )) { +if (TRAP (regs) ==INTERRUPT_PERFMON ) +return false; + } + +return true; +} + +static inline void arch_interrupt_nmi_enter_prepare (struct pt_regs *regs, +struct interrupt_nmi_state *state) +{ +#ifdef CONFIG_PPC64 + state->irq_soft_mask = local_paca->irq_soft_mask; + state->irq_happened = local_paca->irq_happened; + state->softe = regs->softe; + +/* +* Set IRQS_ALL_DISABLED unconditionally so irqs_disabled() does +* the right thing, and set IRQ_HARD_DIS. We do not want to reconcile +* because that goes through irq tracing which we don't want in NMI. +*/ + local_paca->irq_soft_mask =IRQS_ALL_DISABLED ; + local_paca->irq_happened |=PACA_IRQ_HARD_DIS ; + +if (!(regs->msr &MSR_EE ) ||is_implicit_soft_masked (regs)) { +/* +* Adjust regs->softe to be soft-masked if it had not been +* reconcied (e.g., interrupt entry with MSR[EE]=0 but softe +* not yet set disabled), or if it was in an implicit soft +* masked state. This makes regs_irqs_disabled(regs) +* behave as expected. +*/ + regs->softe =IRQS_ALL_DISABLED ; + } + +__hard_RI_enable (); + +/* Don't do any per-CPU operations until interrupt state is fixed */ + +if (nmi_disables_ftrace (regs)) { + state->ftrace_enabled =this_cpu_get_ftrace_enabled (); +this_cpu_set_ftrace_enabled (0 ); + } +#endif +} + +static inline void arch_interrupt_nmi_exit_prepare (struct pt_regs *regs, +struct interrupt_nmi_state *state) +{ +/* +* nmi does not call nap_adjust_return because nmi should not create +* new work to do (must use irq_work for that). +*/ + +#ifdef CONFIG_PPC64 +#ifdef CONFIG_PPC_BOOK3S +if (regs_irqs_disabled (regs)) { +unsigned long rst =search_kernel_restart_table (regs->nip); + +if (rst) +regs_set_return_ip (regs, rst); + } +#endif + +if (nmi_disables_ftrace (regs)) +this_cpu_set_ftrace_enabled (state->ftrace_enabled); + +/* Check we didn't change the pending interrupt mask. */ +WARN_ON_ONCE ((state->irq_happened |PACA_IRQ_HARD_DIS ) != local_paca->irq_happened); + regs->softe = state->softe; + local_paca->irq_happened = state->irq_happened; + local_paca->irq_soft_mask = state->irq_soft_mask; +#endif +} + static __always_inline void arch_enter_from_user_mode(struct pt_regs *regs) {kuap_lock ();
Signature · 1 line
2.53.0
PATCH v5 7/8 powerpc: Enable GENERIC_ENTRY feature
In reply to Mukesh Kumar Chaurasiya (IBM)From: Mukesh Kumar Chaurasiya <[email protected]>
Enable the generic IRQ entry/exit infrastructure on PowerPC by selecting GENERIC_ENTRY and integrating the architecture-specific interrupt and syscall handlers with the generic entry/exit APIs.
This change replaces PowerPC’s local interrupt entry/exit handling with calls to the generic irqentry_* helpers, aligning the architecture with the common kernel entry model. The macros that define interrupt, async, and NMI handlers are updated to use irqentry_enter()/irqentry_exit() and irqentry_nmi_enter()/irqentry_nmi_exit() where applicable also convert the PowerPC syscall entry and exit paths to use the generic entry/exit framework and integrating with the common syscall handling routines.
Key updates include:
- The architecture now selects GENERIC_ENTRY in Kconfig.
- Replace interrupt_enter/exit_prepare() with arch_interrupt_* helpers.
- Integrate irqentry_enter()/exit() in standard and async interrupt paths.
- Integrate irqentry_nmi_enter()/exit() in NMI handlers.
- Remove redundant irq_enter()/irq_exit() calls now handled generically.
- Use irqentry_exit_cond_resched() for preemption checks.
- interrupt.c and syscall.c are simplified to delegate context management and user exit handling to the generic entry path.
- The new pt_regs field exit_flags introduced earlier is now used to carry per-syscall exit state flags (e.g. _TIF_RESTOREALL).
- Remove unused code.
This change establishes the necessary wiring for PowerPC to use the generic IRQ entry/exit framework while maintaining existing semantics. This aligns PowerPC with the common entry code used by other architectures and reduces duplicated logic around syscall tracing, context tracking, and signal handling.
The performance benchmarks from perf bench basic syscall are below:
perf bench syscall usec/op (-ve is improvement)
| Syscall | Base | test | change % | | ------- | ----------- | ----------- | -------- | | basic | 0.093543 | 0.093023 | -0.56 | | execve | 446.557781 | 450.107172 | +0.79 | | fork | 1142.204391 | 1156.377214 | +1.24 | | getpgid | 0.097666 | 0.092677 | -5.11 |
perf bench syscall ops/sec (+ve is improvement)
| Syscall | Base | New | change % | | ------- | -------- | -------- | -------- | | basic | 10690548 | 10750140 | +0.56 | | execve | 2239 | 2221 | -0.80 | | fork | 875 | 864 | -1.26 | | getpgid | 10239026 | 10790324 | +5.38 |
IPI latency benchmark (-ve is improvement)
| Metric | Base (ns) | New (ns) | % Change | | -------------- | ------------- | ------------- | -------- | | Dry run | 583136.56 | 584136.35 | 0.17% | | Self IPI | 4167393.42 | 4149093.90 | -0.44% | | Normal IPI | 61769347.82 | 61753728.39 | -0.03% | | Broadcast IPI | 2235584825.02 | 2227521401.45 | -0.36% | | Broadcast lock | 2164964433.31 | 2125658641.76 | -1.82% |
Thats very close to performance earlier with arch specific handling.
--- arch/powerpc/Kconfig | 1 + arch/powerpc/include/asm/interrupt.h | 384 +++++---------------------- arch/powerpc/include/asm/kasan.h | 15 +- arch/powerpc/kernel/interrupt.c | 250 +++-------------- arch/powerpc/kernel/ptrace/ptrace.c | 3 - arch/powerpc/kernel/signal.c | 8 + arch/powerpc/kernel/syscall.c | 119 +-------- 7 files changed, 124 insertions(+), 656 deletions(-)
arch/powerpc/Kconfig +1 -0
@@ -206,6 +206,7 @@ config PPC select GENERIC_CPU_AUTOPROBE select GENERIC_CPU_VULNERABILITIES if PPC_BARRIER_NOSPEC select GENERIC_EARLY_IOREMAP + select GENERIC_ENTRY select GENERIC_GETTIMEOFDAY select GENERIC_IDLE_POLL_SETUP select GENERIC_IOREMAP
arch/powerpc/include/asm/interrupt.h +60 -324
427 lines not on this page · raw message
arch/powerpc/include/asm/kasan.h +10 -5
@@ -3,14 +3,19 @@#define __ASM_KASAN_H#if defined(CONFIG_KASAN ) && !defined(CONFIG_CC_HAS_KASAN_MEMINTRINSIC_PREFIX ) -#define _GLOBAL_KASAN (fn) _GLOBAL(__##fn) -#define _GLOBAL_TOC_KASAN (fn) _GLOBAL_TOC(__##fn) -#define EXPORT_SYMBOL_KASAN (fn) EXPORT_SYMBOL(__##fn) -#else +#define _GLOBAL_KASAN (fn) \ + _GLOBAL(fn); \ + _GLOBAL(__##fn) +#define _GLOBAL_TOC_KASAN (fn) \ + _GLOBAL_TOC(fn); \ + _GLOBAL_TOC(__##fn) +#define EXPORT_SYMBOL_KASAN (fn) \ + EXPORT_SYMBOL(__##fn) +#else /* CONFIG_KASAN && !CONFIG_CC_HAS_KASAN_MEMINTRINSIC_PREFIX */ #define _GLOBAL_KASAN (fn) _GLOBAL(fn)#define _GLOBAL_TOC_KASAN (fn) _GLOBAL_TOC(fn)#define EXPORT_SYMBOL_KASAN (fn) -#endif +#endif /* CONFIG_KASAN && !CONFIG_CC_HAS_KASAN_MEMINTRINSIC_PREFIX */ #ifndef __ASSEMBLER__
arch/powerpc/kernel/interrupt.c +42 -208
329 lines not on this page · raw message
arch/powerpc/kernel/ptrace/ptrace.c +0 -3
@@ -21,9 +21,6 @@#include <asm/switch_to.h> #include <asm/debug.h> -#define CREATE_TRACE_POINTS -#include <trace/events/syscalls.h> -#include "ptrace-decl.h" /*
arch/powerpc/kernel/signal.c +8 -0
@@ -6,6 +6,7 @@* Extracted from signal_32.c and signal_64.c */ +#include <linux/entry-common.h> #include <linux/resume_user_mode.h> #include <linux/signal.h> #include <linux/uprobes.h>
@@ -368,3 +369,10 @@ void signal_fault(struct task_struct *tsk, struct pt_regs *regs,printk_ratelimited (regs->msr &MSR_64BIT ? fm64 : fm32, tsk->comm,task_pid_nr (tsk), where, ptr, regs->nip, regs->link); } + +void arch_do_signal_or_restart (struct pt_regs *regs) +{ +BUG_ON (regs != current->thread.regs); + regs->exit_flags |= _TIF_RESTOREALL; +do_signal (current); +}
arch/powerpc/kernel/syscall.c +3 -116
133 lines not on this page · raw message
Signature · 1 line
2.53.0
PATCH v5 8/8 powerpc: Remove unused functions
In reply to Mukesh Kumar Chaurasiya (IBM)From: Mukesh Kumar Chaurasiya <[email protected]>
After enabling GENERIC_ENTRY some functions are left unused. Cleanup all those functions which includes:
- do_syscall_trace_enter
- do_syscall_trace_leave
- do_notify_resume
- do_seccomp
--- arch/powerpc/include/asm/ptrace.h | 3 - arch/powerpc/include/asm/signal.h | 1 - arch/powerpc/kernel/ptrace/ptrace.c | 138 ---------------------------- arch/powerpc/kernel/signal.c | 17 ---- 4 files changed, 159 deletions(-)
arch/powerpc/include/asm/ptrace.h +0 -3
@@ -177,9 +177,6 @@ extern unsigned long profile_pc(struct pt_regs *regs);#define profile_pc (regs) instruction_pointer(regs)#endif -long do_syscall_trace_enter (struct pt_regs *regs); -void do_syscall_trace_leave (struct pt_regs *regs); -static inline void set_return_regs_changed (void ) {#ifdef CONFIG_PPC_BOOK3S_64
arch/powerpc/include/asm/signal.h +0 -1
@@ -7,7 +7,6 @@#include <uapi/asm/ptrace.h> struct pt_regs ; -void do_notify_resume (struct pt_regs *regs,unsigned long thread_info_flags);unsigned long get_min_sigframe_size_32 (void );unsigned long get_min_sigframe_size_64 (void );
arch/powerpc/kernel/ptrace/ptrace.c +0 -138
144 lines not on this page · raw message
arch/powerpc/kernel/signal.c +0 -17
@@ -293,23 +293,6 @@ static void do_signal(struct task_struct *tsk)signal_setup_done (ret, &ksig,test_thread_flag (TIF_SINGLESTEP )); } -void do_notify_resume (struct pt_regs *regs,unsigned long thread_info_flags) -{ -if (thread_info_flags & _TIF_UPROBE) -uprobe_notify_resume (regs); - -if (thread_info_flags & _TIF_PATCH_PENDING) -klp_update_patch_state (current); - -if (thread_info_flags & (_TIF_SIGPENDING | _TIF_NOTIFY_SIGNAL)) { -BUG_ON (regs != current->thread.regs); -do_signal (current); - } - -if (thread_info_flags & _TIF_NOTIFY_RESUME) -resume_user_mode_work (regs); -} -static unsigned long get_tm_stackpointer (struct task_struct *tsk) {/* When in an active transaction that takes a signal, we need to be
Signature · 1 line
2.53.0
Quoted 8 lines · Mukesh Kumar Chaurasiya (IBM)
Adding support for the generic irq entry/exit handling for PowerPC. The goal is to bring PowerPC in line with other architectures that already use the common irq entry infrastructure, reducing duplicated code and making it easier to share future changes in entry/exit paths.
This is slightly tested of ppc64le and ppc32.
[...]
Applied to powerpc/next.
[1/8] powerpc: rename arch_irq_disabled_regs
https://git.kernel.org/powerpc/c/baa590018fbfc3e7384b604c7a7951eecf10a523 [2/8] powerpc: Prepare to build with generic entry/exit framework https://git.kernel.org/powerpc/c/dec63ea6fafffb91811381d9a4686d4bc39f01ae [3/8] powerpc: introduce arch_enter_from_user_mode https://git.kernel.org/powerpc/c/09a9d3a8499db606563eb5e75a993419f4d1901e [4/8] powerpc: Introduce syscall exit arch functions https://git.kernel.org/powerpc/c/02565a782c1ee7e8ada38ad24a698e01d6ea9ff8 [5/8] powerpc: add exit_flags field in pt_regs https://git.kernel.org/powerpc/c/d7a6797e0bc1ee7bddb8a298f2d1acddb300efb3 [6/8] powerpc: Prepare for IRQ entry exit https://git.kernel.org/powerpc/c/893082ac769ba92a1338e7552d97de769f352a3e [7/8] powerpc: Enable GENERIC_ENTRY feature https://git.kernel.org/powerpc/c/bee25f97ad24641df55cb3887eb5bfcb2b9d8fbc [8/8] powerpc: Remove unused functions https://git.kernel.org/powerpc/c/6ed60999d33d49251b42976a5511f1bb089797ed
cheers
Quoted 5 lines · Mukesh Kumar Chaurasiya (IBM)
Adding support for the generic irq entry/exit handling for PowerPC. The goal is to bring PowerPC in line with other architectures that already use the common irq entry infrastructure, reducing duplicated code and making it easier to share future changes in entry/exit paths.
Hello,
applying these patches on top of Linux 6.12 causes seccomp to return different error (ENNOSYS instead of EPERM) causing some seccomp tests to fail.
A minimal reporoducer is
podman run --rm --security-opt seccomp=<(echo '{"defaultAction":"SCMP_ACT_ALLOW","syscalls":[{"names":["mkdir"],"action":"SCMP_ACT_ERRNO"}]}') docker.io/library/alpine mkdir /tmp/test
This should produce 'Operation not permitted' but produces 'Function not implemented'.
Can you have a look?
Thanks
Michal
Quoted 20 lines · Michal Suchánek
Quoted 5 lines · Mukesh Kumar Chaurasiya (IBM)
Adding support for the generic irq entry/exit handling for PowerPC. The goal is to bring PowerPC in line with other architectures that already use the common irq entry infrastructure, reducing duplicated code and making it easier to share future changes in entry/exit paths.
Hello,
applying these patches on top of Linux 6.12 causes seccomp to return different error (ENNOSYS instead of EPERM) causing some seccomp tests to fail.
A minimal reporoducer is
podman run --rm --security-opt seccomp=<(echo '{"defaultAction":"SCMP_ACT_ALLOW","syscalls":[{"names":["mkdir"],"action":"SCMP_ACT_ERRNO"}]}') docker.io/library/alpine mkdir
/tmp/testThis should produce 'Operation not permitted' but produces 'Function not implemented'.
Quoted 8 lines
Can you have a look?
Thanks
Michal
Hey Michal,
Thanks for reporting this. I'll take a look at this.
Regards,
Mukesh
This breaks booting on the PowerMac G5 (PowerMac7,3), there is a panic somwhere in an irq.
Signature · 3 lines
Andreas Schwab, [email protected] GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 "And now for something completely different."
Christophe Leroy (CS GROUP)
reviewerLe 04/07/2026 à 17:49, Andreas Schwab a écrit :
Quoted 3 lines
This breaks booting on the PowerMac G5 (PowerMac7,3), there is a panic somwhere in an irq.
Do you have more details ? Like at least the dump of the panic Oops ?
Can it be reproduced on QEMU ?
Christophe
Quoted 6 lines · Christophe Leroy (CS GROUP)
Le 04/07/2026 à 17:49, Andreas Schwab a écrit :
Quoted 3 lines
This breaks booting on the PowerMac G5 (PowerMac7,3), there is a panic somwhere in an irq.
Do you have more details ? Like at least the dump of the panic Oops ?
NIP .power4_idle_nap+0x30/0x34
LR .power4_idle+0x64/0x70
--- interrupt: 900
.arch_spu_idle+0x88/0x100 .default_idle_call+0x3c/0x5c .do_idle+0x194/0x2b4 .cpu_startup_entry+0x3c/0x40 .start_secondary+0x6b0/0x820 start_secondary_prolog+0x10/0x14
Quoted 1 line
Can it be reproduced on QEMU ?
No, qemu does not support SMP.
Signature · 3 lines
Andreas Schwab, [email protected] GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 "And now for something completely different."
Shrikanth Hegde
reviewerQuoted 20 lines · Andreas Schwab
Quoted 6 lines · Christophe Leroy (CS GROUP)
Le 04/07/2026 à 17:49, Andreas Schwab a écrit :
Quoted 3 lines
This breaks booting on the PowerMac G5 (PowerMac7,3), there is a panic somwhere in an irq.
Do you have more details ? Like at least the dump of the panic Oops ?
NIP .power4_idle_nap+0x30/0x34
LR .power4_idle+0x64/0x70
--- interrupt: 900.arch_spu_idle+0x88/0x100 .default_idle_call+0x3c/0x5c .do_idle+0x194/0x2b4 .cpu_startup_entry+0x3c/0x40 .start_secondary+0x6b0/0x820 start_secondary_prolog+0x10/0x14
Quoted 4 lines
Quoted 1 line
Can it be reproduced on QEMU ?
No, qemu does not support SMP.
Can you try "powersave=off" commandline to disable idlestate and see if it boots?
Other call stacks are very likely not possible such as sched_balance_find_src_group
Quoted 2 lines · Shrikanth Hegde
Can you try "powersave=off" commandline to disable idlestate and see if it boots?
Yes, that allows it to boot normally.
Signature · 3 lines
Andreas Schwab, [email protected] GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 "And now for something completely different."
BUG: Unable to handle kernel data access at 0x30af548c0000028
Faulting instruction address: 0xc0000000000af47c
c0000000000aee6c <.sched_balance_find_src_group>:
c0000000000af474: e9 3f 00 18 ld r9,24(r31) c0000000000af478: e9 29 0d 48 ld r9,3400(r9) c0000000000af47c: 89 49 00 28 lbz r10,40(r9) c0000000000af480: 7c 0a e0 40 cmplw r10,r28
Signature · 3 lines
Andreas Schwab, [email protected] GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 "And now for something completely different."
Shrikanth Hegde
reviewerHi Andreas.
Quoted 10 lines · Andreas Schwab
BUG: Unable to handle kernel data access at 0x30af548c0000028Faulting instruction address:
0xc0000000000af47c
c0000000000aee6c<.sched_balance_find_src_group>:c0000000000af474: e9 3f 00 18 ld r9,24(r31) c0000000000af478: e9 29 0d 48 ld r9,3400(r9) c0000000000af47c: 89 49 00 28 lbz r10,40(r9) c0000000000af480: 7c 0a e0 40 cmplw r10,r28
Does the stack-trace change every time?
There were couple more fixes on top of it. Is it included in your kernel already?
https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git/commit/?h=next&id=334f3f6d7a1660fd65597bad9960bfa33a716d35
https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git/commit/?h=next&id=4e1187e12de40b5301977b2476d21b569358dafb
Quoted 12 lines · Shrikanth Hegde
Hi Andreas.
Quoted 8 lines · Andreas Schwab
BUG: Unable to handle kernel data access at 0x30af548c0000028 Faulting instruction address: 0xc0000000000af47c c0000000000aee6c <.sched_balance_find_src_group>: c0000000000af474: e9 3f 00 18 ld r9,24(r31) c0000000000af478: e9 29 0d 48 ld r9,3400(r9) c0000000000af47c: 89 49 00 28 lbz r10,40(r9) c0000000000af480: 7c 0a e0 40 cmplw r10,r28
Quoted 1 line
Does the stack-trace change every time?
Most of the time I get nothing, and when I get something it is mostly scrolled out already (there is only so much you can put in 80x25). I had to hack the kernel to reduce the output to the bare minimum.
Quoted 3 lines
There were couple more fixes on top of it. Is it included in your kernel already?
https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git/commit/?h=next&id=334f3f6d7a1660fd65597bad9960bfa33a716d35
https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git/commit/?h=next&id=4e1187e12de40b5301977b2476d21b569358dafb
They are all in 7.2-rc1 already.
Signature · 3 lines
Andreas Schwab, [email protected] GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 "And now for something completely different."
Oops: Exception in kernel mode, sig: 4 [#1] Oops: Kernel access of bad area, sig: 11 [#1]
kernel tried to execute exec-protected page (c0000001bfa8c380) - exploit attempt? (uid: 0) Faulting instruction address: 0xc0000001bfa8c380
BUG: Unable to handle kernel instruction fetch Oops: Kernel Access of bad area, sig: 11 [#1]
Signature · 3 lines
Andreas Schwab, [email protected] GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 "And now for something completely different."
Most of the time it prints nothing, and when it prints something it is mostly scrolled out.
.handle_softirqs+0xcc/0x264 .do_softirq_own_stack+0x30/0x38 .__irq_exit_rcu++0x7c/0x120
.timer_interrupt+90x1ß9/0x2c4
decrementer_common_virt+9x214/0x22ß
Signature · 3 lines
Andreas Schwab, [email protected] GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 "And now for something completely different."
Quoted 4 lines · Andreas Schwab
Most of the time it prints nothing, and when it prints something it is mostly scrolled out.
Quoted 10 lines
.handle_softirqs+0xcc/0x264 .do_softirq_own_stack+0x30/0x38 .__irq_exit_rcu++0x7c/0x120.timer_interrupt+90x1ß9/0x2c4
decrementer_common_virt+9x214/0x22ßSignature · 3 lines
Andreas Schwab, [email protected] GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 "And now for something completely different."
Hey,
Thanks for reporting this. I don't have a powermacG5 to test this. Can you try out the below diff and check whether this fixes the issue? Please note this is not a proper fix, I just want to understand the issue so that i can make a proper fix.
Regards,
Mukesh
arch/powerpc/include/asm/entry-common.h +0 -8
@@ -286,14 +286,6 @@ static inline void arch_interrupt_async_enter_prepare(struct pt_regs *regs)static inline void arch_interrupt_async_exit_prepare (struct pt_regs *regs) { -/* -* Adjust at exit so the main handler sees the true NIA. This must -* come before irq_exit() because irq_exit can enable interrupts, and -* if another interrupt is taken before nap_adjust_return has run -* here, then that interrupt would return directly to idle nap return. -*/ -nap_adjust_return (regs); -arch_interrupt_exit_prepare (regs); }
arch/powerpc/include/asm/interrupt.h +1 -0
@@ -246,6 +246,7 @@ interrupt_handler void func(struct pt_regs *regs) \instrumentation_begin (); \irq_enter_rcu (); \ ____##func (regs); \ + nap_adjust_return(regs); \ irq_exit_rcu(); \ instrumentation_end(); \ arch_interrupt_async_exit_prepare(regs); \
Quoted 24 lines · Mukesh Kumar Chaurasiya
arch/powerpc/include/asm/entry-common.h +0
-8@@ -286,14 +286,6 @@ static inline void arch_interrupt_async_enter_prepare(struct pt_regs *regs)static inline void arch_interrupt_async_exit_prepare (struct pt_regs *regs) { -/* -* Adjust at exit so the main handler sees the true NIA. This must -* come before irq_exit() because irq_exit can enable interrupts, and -* if another interrupt is taken before nap_adjust_return has run -* here, then that interrupt would return directly to idle nap return. -*/ -nap_adjust_return (regs); -arch_interrupt_exit_prepare (regs); }arch/powerpc/include/asm/interrupt.h +0
-0
arch/powerpc/include/asm/interrupt.h interrupt_handler void func(struct pt_regs *regs) \ · line 252
Quoted 7 lines
arch/powerpc/include/asm/interrupt.h +1
-0@@ -246,6 +246,7 @@ interrupt_handler void func(struct pt_regs *regs) \instrumentation_begin (); \irq_enter_rcu (); \ ____##func (regs); \ + nap_adjust_return(regs); \ irq_exit_rcu(); \ instrumentation_end(); \ arch_interrupt_async_exit_prepare(regs); \
This allows the system to boot normally.
Signature · 3 lines
Andreas Schwab, [email protected] GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 "And now for something completely different."
Quoted 21 lines · Andreas Schwab
Quoted 19 lines · Mukesh Kumar Chaurasiya
arch/powerpc/include/asm/entry-common.h +0
-8@@ -286,14 +286,6 @@ static inline void arch_interrupt_async_enter_prepare(struct pt_regs *regs)static inline void arch_interrupt_async_exit_prepare (struct pt_regs *regs) { -/* -* Adjust at exit so the main handler sees the true NIA. This must -* come before irq_exit() because irq_exit can enable interrupts, and -* if another interrupt is taken before nap_adjust_return has run -* here, then that interrupt would return directly to idle nap return. -*/ -nap_adjust_return (regs); -arch_interrupt_exit_prepare (regs); }
Quoted 15 lines
arch/powerpc/include/asm/interrupt.h interrupt_handler void func(struct pt_regs *regs) \ · line 252
Quoted 12 lines
arch/powerpc/include/asm/interrupt.h +1
-0@@ -246,6 +246,7 @@ interrupt_handler void func(struct pt_regs *regs) \instrumentation_begin (); \irq_enter_rcu (); \ ____##func (regs); \ + nap_adjust_return(regs); \ irq_exit_rcu(); \ instrumentation_end(); \ arch_interrupt_async_exit_prepare(regs); \This allows the system to boot normally.
Thanks for testing this. I am assuming you tried this without "powersave=off" command line option.
I'll send out a fix for this.
Regards,
Mukesh
Quoted 4 lines
Signature · 3 lines
Andreas Schwab, [email protected] GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 "And now for something completely different."
Hi Mukesh,
Quoted 37 lines · Mukesh Kumar Chaurasiya
Quoted 21 lines · Andreas Schwab
Quoted 19 lines · Mukesh Kumar Chaurasiya
arch/powerpc/include/asm/entry-common.h +0
-8@@ -286,14 +286,6 @@ static inline void arch_interrupt_async_enter_prepare(struct pt_regs *regs)static inline void arch_interrupt_async_exit_prepare (struct pt_regs *regs) { -/* -* Adjust at exit so the main handler sees the true NIA. This must -* come before irq_exit() because irq_exit can enable interrupts, and -* if another interrupt is taken before nap_adjust_return has run -* here, then that interrupt would return directly to idle nap return. -*/ -nap_adjust_return (regs); -arch_interrupt_exit_prepare (regs); }Quoted 15 lines
arch/powerpc/include/asm/interrupt.h interrupt_handler void func(struct pt_regs *regs) \ · line 252
Quoted 12 lines
arch/powerpc/include/asm/interrupt.h +1
-0@@ -246,6 +246,7 @@ interrupt_handler void func(struct pt_regs *regs) \instrumentation_begin (); \irq_enter_rcu (); \ ____##func (regs); \ + nap_adjust_return(regs); \ irq_exit_rcu(); \ instrumentation_end(); \ arch_interrupt_async_exit_prepare(regs); \This allows the system to boot normally.
Quoted 2 lines
Thanks for testing this. I am assuming you tried this without "powersave=off" command line option.
I am here after bisecting the same problem. (I also use a G5.)
I can also confirm that the above hack on top of v7.2-rc3 will boot (without specifying powersave=off).
John Ogness
Hi Mukesh, Shrikanth, Madhavan,
I'm seeing a ~19% throughput regression on ppc64le (POWER10) in syscall-heavy workloads between kernel 7.1 and 7.2-rc4, which I've narrowed down to the GENERIC_ENTRY enablement for powerpc (commit bee25f97ad24 "powerpc: Enable GENERIC_ENTRY feature").
The regression manifests specifically when SELinux is enabled. Regression disappears with selinux=0. With SELinux disabled, 7.2 is actually slightly *faster* than 7.1.
Test: stress-ng --kill 1 -t 23 (single-thread kill() syscall loop)
Machine: IBM,9080-HEX POWER10, 8 vCPUs (1 core SMT-8), 64 GiB
Results (bogo-ops/sec, higher is better):
Kernel Enforcing Permissive Disabled SELinux tax ------ --------- ---------- -------- ----------- 7.1 69,107 70,248 93,566 -26.1% 7.2-rc4 56,247 59,075 99,240 -43.3% Delta -18.6% -15.9% +6.1%
Key observations:
- SELinux permissive ~ enforcing on both kernels (overhead is in the code path, not policy evaluation)
- SELinux disabled: 7.2 is 6% faster than 7.1 -- the generic entry path itself is not slower for the base syscall
- The SELinux "tax" nearly doubles from 26% to 43% between 7.1->7.2
- No changes to security/selinux/avc.c between v7.1 and v7.2-rc4
The kill() hot path (sys_kill -> check_kill_permission -> security_task_kill -> selinux_task_kill -> avc_has_perm -> avc_lookup) wraps every AVC lookup in rcu_read_lock()/rcu_read_unlock(). On ppc64le with PREEMPT_RCU, each pair requires lwsync/isync barriers. I suspect the generic entry path changes something in how context tracking, tracing, or RCU interacts with the syscall fast path that amplifies the per-call RCU cost.
Note: this is on top of a separate ~36% regression from 6.12->7.1 caused by HAVE_PREEMPT_DYNAMIC_KEY enabling CONFIG_PREEMPT_RCU on ppc64le (commit 6ad7751537e8), which I reported separately on the v4 thread.
Thank you
Jirka
Quoted 114 lines · Mukesh Kumar Chaurasiya (IBM)
Adding support for the generic irq entry/exit handling for PowerPC. The goal is to bring PowerPC in line with other architectures that already use the common irq entry infrastructure, reducing duplicated code and making it easier to share future changes in entry/exit paths.
This is slightly tested of ppc64le and ppc32.
The performance benchmarks are below:
perf bench syscall usec/op (-ve is improvement)
| Syscall | Base | test | change % | | ------- | ----------- | ----------- | -------- | | basic | 0.093543 | 0.093023 | -0.56 | | execve | 446.557781 | 450.107172 | +0.79 | | fork | 1142.204391 | 1156.377214 | +1.24 | | getpgid | 0.097666 | 0.092677 | -5.11 |perf bench syscall ops/sec (+ve is improvement)
| Syscall | Base | New | change % | | ------- | -------- | -------- | -------- | | basic | 10690548 | 10750140 | +0.56 | | execve | 2239 | 2221 | -0.80 | | fork | 875 | 864 | -1.26 | | getpgid | 10239026 | 10790324 | +5.38 |IPI latency benchmark (-ve is improvement)
| Metric | Base (ns) | New (ns) | % Change | | -------------- | ------------- | ------------- | -------- | | Dry run | 583136.56 | 584136.35 | 0.17% | | Self IPI | 4167393.42 | 4149093.90 | -0.44% | | Normal IPI | 61769347.82 | 61753728.39 | -0.03% | | Broadcast IPI | 2235584825.02 | 2227521401.45 | -0.36% | | Broadcast lock | 2164964433.31 | 2125658641.76 | -1.82% |Thats very close to performance earlier with arch specific handling.
Tests done:
- Build and boot on ppc64le pseries.
- Build and boot on ppc64le powernv8 powernv9 powernv10.
- Build and boot on ppc32.
- Performance benchmark done with perf syscall basic on pseries.Changelog:
V4 -> V5:
- Rebased on latest mainline
V4: https://lore.kernel.org/all/[email protected]/V3 -> V4:
- Fixed the issue in older gcc version where linker couldn't find mem functions
- Merged IRQ enable and syscall enable into a single patch
- Cleanup for unused functions done in separate patch.
- Some other cosmetic changes
V3: https://lore.kernel.org/all/[email protected]/V2 -> V3:
- #ifdefCONFIG_GENERIC_IRQ_ENTRYremoved from unnecessary places
- Some functions made__always_inline
-pt_regspadding changed to match 16byte interrupt stack alignment
- And some cosmetic changes from reviews from earlier patch
V2: https://lore.kernel.org/all/[email protected]/V1 -> V2:
- Fix an issue where context tracking was showing warnings for incorrect context
V1: https://lore.kernel.org/all/[email protected]/RFC -> PATCH V1:
- Fix for ppc32 spitting out kuap lock warnings.
- ppc64le powernv8 crash fix.
- Review comments incorporated from previous RFC.
RFC https://lore.kernel.org/all/[email protected]/Mukesh Kumar Chaurasiya (8):
powerpc: renamearch_irq_disabled_regs
powerpc: Prepare to build with generic entry/exit framework
powerpc: introducearch_enter_from_user_mode
powerpc: Introduce syscall exit arch functions
powerpc: addexit_flagsfield inpt_regs
powerpc: Prepare for IRQ entry exit
powerpc: EnableGENERIC_ENTRYfeature
powerpc: Remove unused functionsarch/powerpc/Kconfig | 1 + arch/powerpc/include/asm/entry-common.h | 533 ++++++++++++++++++++++++ arch/powerpc/include/asm/hw_irq.h | 4 +- arch/powerpc/include/asm/interrupt.h | 386 +++-------------- arch/powerpc/include/asm/kasan.h | 15 +- arch/powerpc/include/asm/ptrace.h | 6 +- arch/powerpc/include/asm/signal.h | 1 - arch/powerpc/include/asm/stacktrace.h | 6 + arch/powerpc/include/asm/syscall.h | 5 + arch/powerpc/include/asm/thread_info.h | 1 + arch/powerpc/include/uapi/asm/ptrace.h | 14 +- arch/powerpc/kernel/interrupt.c | 254 ++--------- arch/powerpc/kernel/ptrace/ptrace.c | 142 +------ arch/powerpc/kernel/signal.c | 25 +- arch/powerpc/kernel/syscall.c | 119 +----- arch/powerpc/kernel/traps.c | 2 +- arch/powerpc/kernel/watchdog.c | 2 +- arch/powerpc/perf/core-book3s.c | 2 +- 18 files changed, 690 insertions(+), 828 deletions(-) create mode 100644 arch/powerpc/include/asm/entry-common.h--
2.53.0
Signature · 1 line
-Jirka
Quoted 37 lines · Jirka Hladky
Hi Mukesh, Shrikanth, Madhavan,
I'm seeing a ~19% throughput regression on ppc64le (POWER10) in syscall-heavy workloads between kernel 7.1 and 7.2-rc4, which I've narrowed down to the
GENERIC_ENTRYenablement for powerpc (commitbee25f97ad24"powerpc: EnableGENERIC_ENTRYfeature").The regression manifests specifically when SELinux is enabled. Regression disappears with selinux=0. With SELinux disabled, 7.2 is actually slightly *faster* than 7.1.
Test: stress-ng --kill 1 -t 23 (single-thread
kill()syscall loop)
Machine: IBM,9080-HEX POWER10, 8 vCPUs (1 core SMT-8), 64 GiBResults (bogo-ops/sec, higher is better):
Kernel Enforcing Permissive Disabled SELinux tax ------ --------- ---------- -------- ----------- 7.1 69,107 70,248 93,566 -26.1% 7.2-rc4 56,247 59,075 99,240 -43.3% Delta -18.6% -15.9% +6.1%Key observations:
- SELinux permissive ~ enforcing on both kernels (overhead is in the code path, not policy evaluation)
- SELinux disabled: 7.2 is 6% faster than 7.1 -- the generic entry path itself is not slower for the base syscall
- The SELinux "tax" nearly doubles from 26% to 43% between 7.1->7.2
- No changes tosecurity/selinux/avc.cbetween v7.1 and v7.2-rc4The
kill()hot path (sys_kill->check_kill_permission->security_task_kill->selinux_task_kill->avc_has_perm->avc_lookup) wraps every AVC lookup in rcu_read_lock()/rcu_read_unlock(). On ppc64le withPREEMPT_RCU, each pair requires lwsync/isync barriers. I suspect the generic entry path changes something in how context tracking, tracing, or RCU interacts with the syscall fast path that amplifies the per-call RCU cost.
Quoted 6 lines
Note: this is on top of a separate ~36% regression from 6.12->7.1 caused by
HAVE_PREEMPT_DYNAMIC_KEYenablingCONFIG_PREEMPT_RCUon ppc64le (commit6ad7751537e8), which I reported separately on the v4 thread.Thank you
Jirka
[...]
Hey Jirka,
Thanks for reporting this.
I'll take a look.
Thanks,
Mukesh
Quoted 45 lines · Mukesh Kumar Chaurasiya
Quoted 37 lines · Jirka Hladky
Hi Mukesh, Shrikanth, Madhavan,
I'm seeing a ~19% throughput regression on ppc64le (POWER10) in syscall-heavy workloads between kernel 7.1 and 7.2-rc4, which I've narrowed down to the
GENERIC_ENTRYenablement for powerpc (commitbee25f97ad24"powerpc: EnableGENERIC_ENTRYfeature").The regression manifests specifically when SELinux is enabled. Regression disappears with selinux=0. With SELinux disabled, 7.2 is actually slightly *faster* than 7.1.
Test: stress-ng --kill 1 -t 23 (single-thread
kill()syscall loop)
Machine: IBM,9080-HEX POWER10, 8 vCPUs (1 core SMT-8), 64 GiBResults (bogo-ops/sec, higher is better):
Kernel Enforcing Permissive Disabled SELinux tax ------ --------- ---------- -------- ----------- 7.1 69,107 70,248 93,566 -26.1% 7.2-rc4 56,247 59,075 99,240 -43.3% Delta -18.6% -15.9% +6.1%Key observations:
- SELinux permissive ~ enforcing on both kernels (overhead is in the code path, not policy evaluation)
- SELinux disabled: 7.2 is 6% faster than 7.1 -- the generic entry path itself is not slower for the base syscall
- The SELinux "tax" nearly doubles from 26% to 43% between 7.1->7.2
- No changes tosecurity/selinux/avc.cbetween v7.1 and v7.2-rc4The
kill()hot path (sys_kill->check_kill_permission->security_task_kill->selinux_task_kill->avc_has_perm->avc_lookup) wraps every AVC lookup in rcu_read_lock()/rcu_read_unlock(). On ppc64le withPREEMPT_RCU, each pair requires lwsync/isync barriers. I suspect the generic entry path changes something in how context tracking, tracing, or RCU interacts with the syscall fast path that amplifies the per-call RCU cost.Quoted 6 lines
Note: this is on top of a separate ~36% regression from 6.12->7.1 caused by
HAVE_PREEMPT_DYNAMIC_KEYenablingCONFIG_PREEMPT_RCUon ppc64le (commit6ad7751537e8), which I reported separately on the v4 thread.Thank you
Jirka
Quoted 9 lines
[...]
Hey Jirka,Thanks for reporting this.
I'll take a look.Thanks,
Mukesh
Thank you, Mukesh! I'm happy to run additional tests on the POWER10 machine if you have patches to try or specific experiments you'd like me to do.
Jirka
Quoted 53 lines · Jirka Hladky
Quoted 51 lines · Mukesh Kumar Chaurasiya
Quoted 37 lines · Jirka Hladky
Hi Mukesh, Shrikanth, Madhavan,
I'm seeing a ~19% throughput regression on ppc64le (POWER10) in syscall-heavy workloads between kernel 7.1 and 7.2-rc4, which I've narrowed down to the
GENERIC_ENTRYenablement for powerpc (commitbee25f97ad24"powerpc: EnableGENERIC_ENTRYfeature").The regression manifests specifically when SELinux is enabled. Regression disappears with selinux=0. With SELinux disabled, 7.2 is actually slightly *faster* than 7.1.
Test: stress-ng --kill 1 -t 23 (single-thread
kill()syscall loop)
Machine: IBM,9080-HEX POWER10, 8 vCPUs (1 core SMT-8), 64 GiBResults (bogo-ops/sec, higher is better):
Kernel Enforcing Permissive Disabled SELinux tax ------ --------- ---------- -------- ----------- 7.1 69,107 70,248 93,566 -26.1% 7.2-rc4 56,247 59,075 99,240 -43.3% Delta -18.6% -15.9% +6.1%Key observations:
- SELinux permissive ~ enforcing on both kernels (overhead is in the code path, not policy evaluation)
- SELinux disabled: 7.2 is 6% faster than 7.1 -- the generic entry path itself is not slower for the base syscall
- The SELinux "tax" nearly doubles from 26% to 43% between 7.1->7.2
- No changes tosecurity/selinux/avc.cbetween v7.1 and v7.2-rc4The
kill()hot path (sys_kill->check_kill_permission->security_task_kill->selinux_task_kill->avc_has_perm->avc_lookup) wraps every AVC lookup in rcu_read_lock()/rcu_read_unlock(). On ppc64le withPREEMPT_RCU, each pair requires lwsync/isync barriers. I suspect the generic entry path changes something in how context tracking, tracing, or RCU interacts with the syscall fast path that amplifies the per-call RCU cost.Quoted 6 lines
Note: this is on top of a separate ~36% regression from 6.12->7.1 caused by
HAVE_PREEMPT_DYNAMIC_KEYenablingCONFIG_PREEMPT_RCUon ppc64le (commit6ad7751537e8), which I reported separately on the v4 thread.Thank you
Jirka[...]
Hey Jirka,Thanks for reporting this.
I'll take a look.
Quoted 9 lines
Quoted 3 lines
Thanks,
MukeshThank you, Mukesh! I'm happy to run additional tests on the POWER10 machine if you have patches to try or specific experiments you'd like me to do.
Jirka
Hey Jirka,
Sry about the delayed response, i got occupied with functional issues. Anyways, I don't seem to reproduce it with default fedora policy. Can you share the details of policy which you are using?
Regards,
Mukesh
Quoted 53 lines · Jirka Hladky
Quoted 51 lines · Mukesh Kumar Chaurasiya
Quoted 37 lines · Jirka Hladky
Hi Mukesh, Shrikanth, Madhavan,
I'm seeing a ~19% throughput regression on ppc64le (POWER10) in syscall-heavy workloads between kernel 7.1 and 7.2-rc4, which I've narrowed down to the
GENERIC_ENTRYenablement for powerpc (commitbee25f97ad24"powerpc: EnableGENERIC_ENTRYfeature").The regression manifests specifically when SELinux is enabled. Regression disappears with selinux=0. With SELinux disabled, 7.2 is actually slightly *faster* than 7.1.
Test: stress-ng --kill 1 -t 23 (single-thread
kill()syscall loop)
Machine: IBM,9080-HEX POWER10, 8 vCPUs (1 core SMT-8), 64 GiBResults (bogo-ops/sec, higher is better):
Kernel Enforcing Permissive Disabled SELinux tax ------ --------- ---------- -------- ----------- 7.1 69,107 70,248 93,566 -26.1% 7.2-rc4 56,247 59,075 99,240 -43.3% Delta -18.6% -15.9% +6.1%Key observations:
- SELinux permissive ~ enforcing on both kernels (overhead is in the code path, not policy evaluation)
- SELinux disabled: 7.2 is 6% faster than 7.1 -- the generic entry path itself is not slower for the base syscall
- The SELinux "tax" nearly doubles from 26% to 43% between 7.1->7.2
- No changes tosecurity/selinux/avc.cbetween v7.1 and v7.2-rc4The
kill()hot path (sys_kill->check_kill_permission->security_task_kill->selinux_task_kill->avc_has_perm->avc_lookup) wraps every AVC lookup in rcu_read_lock()/rcu_read_unlock(). On ppc64le withPREEMPT_RCU, each pair requires lwsync/isync barriers. I suspect the generic entry path changes something in how context tracking, tracing, or RCU interacts with the syscall fast path that amplifies the per-call RCU cost.Quoted 6 lines
Note: this is on top of a separate ~36% regression from 6.12->7.1 caused by
HAVE_PREEMPT_DYNAMIC_KEYenablingCONFIG_PREEMPT_RCUon ppc64le (commit6ad7751537e8), which I reported separately on the v4 thread.Thank you
Jirka[...]
Hey Jirka,Thanks for reporting this.
I'll take a look.
Quoted 9 lines
Quoted 3 lines
Thanks,
MukeshThank you, Mukesh! I'm happy to run additional tests on the POWER10 machine if you have patches to try or specific experiments you'd like me to do.
Jirka
can you try this with the below diff?
include/linux/rcutiny.h +1 -0
@@ -120,6 +120,7 @@ static inline bool rcu_preempt_need_deferred_qs(struct task_struct *t)return false; }static inline void rcu_preempt_deferred_qs (struct task_struct *t) { } +static inline void rcu_ct_kernel_exit_qs (void ) { }void rcu_scheduler_starting (void );static inline void rcu_end_inkernel_boot (void ) { }static inline bool rcu_inkernel_boot_has_ended (void ) {return true; }
include/linux/rcutree.h +1 -0
@@ -87,6 +87,7 @@ static inline void rcu_irq_exit_check_preempt(void) { }struct task_struct ;void rcu_preempt_deferred_qs (struct task_struct *t); +void rcu_ct_kernel_exit_qs (void );void exit_rcu (void );
kernel/context_tracking.c +1 -1
@@ -118,7 +118,7 @@ static void noinstr ct_kernel_exit(bool user, int offset)lockdep_assert_irqs_disabled ();trace_rcu_watching (TPS ("End" ),ct_nesting (),0 ,ct_rcu_watching ());WARN_ON_ONCE (IS_ENABLED (CONFIG_RCU_EQS_DEBUG ) && !user && !is_idle_task (current)); -rcu_preempt_deferred_qs (current); +rcu_ct_kernel_exit_qs ();// instrumentation for the noinstr ct_kernel_exit_state() instrument_atomic_write (&ct->state,sizeof (ct->state));
kernel/rcu/tree.c +19 -0
@@ -368,6 +368,25 @@ notrace void rcu_momentary_eqs(void) }EXPORT_SYMBOL_GPL (rcu_momentary_eqs); +/** +* rcu_ct_kernel_exit_qs - report deferred QS on syscall/exception exit if needed +* +* Called from ct_kernel_exit() on every return to userspace. Guards the +* rcu_preempt_deferred_qs() call with rcu_preempt_need_deferred_qs() so that +* on the common fast path -- where nothing is deferred -- we avoid the +* cache-line traffic on current->rcu_read_unlock_special that the unconditional +* call causes. This is particularly significant on weakly-ordered architectures +* (e.g. ppc64le) where rcu_read_lock/unlock issue lwsync/isync barriers and +* already touch that cache line in the syscall body. +* +* Follows the same pattern used by rcu_flavor_sched_clock_irq(). +*/ +notrace void rcu_ct_kernel_exit_qs(void) +{ +if (rcu_preempt_need_deferred_qs (current)) +rcu_preempt_deferred_qs (current); +} +/** * rcu_is_cpu_rrupt_from_idle - see if 'interrupted' from idle *
Hi Mukesh,
Re: policy details - I'm running an ELN kernel on RHEL-10.2, with SELinux in enforcing mode using the default (targeted) policy, nothing custom or hardened on top of it. Let me know if you need anything more specific (policy version, booleans, etc.) and I'll pull it.
Re: the rcu_ct_kernel_exit_qs() diff — thanks, that guard against the unconditional rcu_preempt_deferred_qs() call lines up well with what I was seeing on the RCU-barrier side. I'll apply it on top of 7.2 on POWER10 box and re-run the stress-ng --kill numbers (enforcing/permissive/disabled) to see how much of the SELinux tax it recovers. No need to prepare anything separately.
I'm OOO tomorrow, but I'll get to this by the end of the week and report back with results.
Thanks
Jirka
Quoted 136 lines · Mukesh Kumar Chaurasiya
Quoted 53 lines · Jirka Hladky
Quoted 51 lines · Mukesh Kumar Chaurasiya
Quoted 37 lines · Jirka Hladky
Hi Mukesh, Shrikanth, Madhavan,
I'm seeing a ~19% throughput regression on ppc64le (POWER10) in syscall-heavy workloads between kernel 7.1 and 7.2-rc4, which I've narrowed down to the
GENERIC_ENTRYenablement for powerpc (commitbee25f97ad24"powerpc: EnableGENERIC_ENTRYfeature").The regression manifests specifically when SELinux is enabled. Regression disappears with selinux=0. With SELinux disabled, 7.2 is actually slightly *faster* than 7.1.
Test: stress-ng --kill 1 -t 23 (single-thread
kill()syscall loop)
Machine: IBM,9080-HEX POWER10, 8 vCPUs (1 core SMT-8), 64 GiBResults (bogo-ops/sec, higher is better):
Kernel Enforcing Permissive Disabled SELinux tax ------ --------- ---------- -------- ----------- 7.1 69,107 70,248 93,566 -26.1% 7.2-rc4 56,247 59,075 99,240 -43.3% Delta -18.6% -15.9% +6.1%Key observations:
- SELinux permissive ~ enforcing on both kernels (overhead is in the code path, not policy evaluation)
- SELinux disabled: 7.2 is 6% faster than 7.1 -- the generic entry path itself is not slower for the base syscall
- The SELinux "tax" nearly doubles from 26% to 43% between 7.1->7.2
- No changes tosecurity/selinux/avc.cbetween v7.1 and v7.2-rc4The
kill()hot path (sys_kill->check_kill_permission->security_task_kill->selinux_task_kill->avc_has_perm->avc_lookup) wraps every AVC lookup in rcu_read_lock()/rcu_read_unlock(). On ppc64le withPREEMPT_RCU, each pair requires lwsync/isync barriers. I suspect the generic entry path changes something in how context tracking, tracing, or RCU interacts with the syscall fast path that amplifies the per-call RCU cost.Quoted 6 lines
Note: this is on top of a separate ~36% regression from 6.12->7.1 caused by
HAVE_PREEMPT_DYNAMIC_KEYenablingCONFIG_PREEMPT_RCUon ppc64le (commit6ad7751537e8), which I reported separately on the v4 thread.Thank you
Jirka[...]
Hey Jirka,Thanks for reporting this.
I'll take a look.Quoted 9 lines
Quoted 3 lines
Thanks,
MukeshThank you, Mukesh! I'm happy to run additional tests on the POWER10 machine if you have patches to try or specific experiments you'd like me to do.
Jirka
can you try this with the below diff?
include/linux/rcutiny.h +1
-0@@ -120,6 +120,7 @@ static inline bool rcu_preempt_need_deferred_qs(struct task_struct *t)return false; }static inline void rcu_preempt_deferred_qs (struct task_struct *t) { } +static inline void rcu_ct_kernel_exit_qs (void ) { }void rcu_scheduler_starting (void );static inline void rcu_end_inkernel_boot (void ) { }static inline bool rcu_inkernel_boot_has_ended (void ) {return true; }include/linux/rcutree.h +1
-0@@ -87,6 +87,7 @@ static inline void rcu_irq_exit_check_preempt(void) { }struct task_struct ;void rcu_preempt_deferred_qs (struct task_struct *t); +void rcu_ct_kernel_exit_qs (void );void exit_rcu (void );kernel/context_tracking.c +1
-1@@ -118,7 +118,7 @@ static void noinstr ct_kernel_exit(bool user, int offset)lockdep_assert_irqs_disabled ();trace_rcu_watching (TPS ("End" ),ct_nesting (),0 ,ct_rcu_watching ());WARN_ON_ONCE (IS_ENABLED (CONFIG_RCU_EQS_DEBUG ) && !user && !is_idle_task (current)); -rcu_preempt_deferred_qs (current); +rcu_ct_kernel_exit_qs ();// instrumentation for the noinstr ct_kernel_exit_state() instrument_atomic_write (&ct->state,sizeof (ct->state));kernel/rcu/tree.c +19
-0@@ -368,6 +368,25 @@ notrace void rcu_momentary_eqs(void) }EXPORT_SYMBOL_GPL (rcu_momentary_eqs); +/** +* rcu_ct_kernel_exit_qs - report deferred QS on syscall/exception exit if needed +* +* Called from ct_kernel_exit() on every return to userspace. Guards the +* rcu_preempt_deferred_qs() call with rcu_preempt_need_deferred_qs() so that +* on the common fast path -- where nothing is deferred -- we avoid the +* cache-line traffic on current->rcu_read_unlock_special that the unconditional +* call causes. This is particularly significant on weakly-ordered architectures +* (e.g. ppc64le) where rcu_read_lock/unlock issue lwsync/isync barriers and +* already touch that cache line in the syscall body. +* +* Follows the same pattern used by rcu_flavor_sched_clock_irq(). +*/ +notrace void rcu_ct_kernel_exit_qs(void) +{ +if (rcu_preempt_need_deferred_qs (current)) +rcu_preempt_deferred_qs (current); +} +/** * rcu_is_cpu_rrupt_from_idle - see if 'interrupted' from idle *
Signature · 1 line
-Jirka
Hi Mukesh,
Following up on the ppc64le SELinux/kill() regression - it doesn't reproduce with proper multi-iteration sampling. stress-ng --kill has ~30% run-to-run variance
on our LPARs. Part of the variance comes from LPARs being much less isolated than I'd assumed - we saw contention bleed across LPARs on the same physical server, even between "Dedicated" partitions. With 15 iterations per config, 7.1 and 7.2-rc4 are statistically even (within ~3%) across enforcing/ permissive/disabled, confirmed on a second, physically separate LPAR (7.1 vs 7.2-rc6). This matches what you said - you couldn't reproduce it either.
I also built 7.2-rc4 from source with your proposed rcu_ct_kernel_exit_qs() patch and re-ran the same sweep - it shows no consistent effect vs. stock 7.2-rc4 (differences within the same noise band), which is consistent with there being no real regression here for it to address.
Sorry for the false alarm, and thank you for the patch and for taking the time to look into this - I don't think it's needed for this issue, though it may still be a reasonable efficiency improvement on its own merits (it does avoid the unconditional rcu_preempt_deferred_qs() call on the common fast path). I'll update the internal RHEL tracking ticket (RHEL-216740) to reflect that this doesn't reproduce.
Thank you!
Jirka
Quoted 164 lines · Jirka Hladky
Hi Mukesh,
Re: policy details - I'm running an ELN kernel on RHEL-10.2, with SELinux in enforcing mode using the default (targeted) policy, nothing custom or hardened on top of it. Let me know if you need anything more specific (policy version, booleans, etc.) and I'll pull it.
Re: the
rcu_ct_kernel_exit_qs()diff — thanks, that guard against the unconditionalrcu_preempt_deferred_qs()call lines up well with what I was seeing on the RCU-barrier side. I'll apply it on top of 7.2 on POWER10 box and re-run the stress-ng --kill numbers (enforcing/permissive/disabled) to see how much of the SELinux tax it recovers. No need to prepare anything separately.I'm OOO tomorrow, but I'll get to this by the end of the week and report back with results.
Thanks
JirkaQuoted 126 lines · Mukesh Kumar Chaurasiya
Quoted 53 lines · Jirka Hladky
Quoted 51 lines · Mukesh Kumar Chaurasiya
Quoted 37 lines · Jirka Hladky
Hi Mukesh, Shrikanth, Madhavan,
I'm seeing a ~19% throughput regression on ppc64le (POWER10) in syscall-heavy workloads between kernel 7.1 and 7.2-rc4, which I've narrowed down to the
GENERIC_ENTRYenablement for powerpc (commitbee25f97ad24"powerpc: EnableGENERIC_ENTRYfeature").The regression manifests specifically when SELinux is enabled. Regression disappears with selinux=0. With SELinux disabled, 7.2 is actually slightly *faster* than 7.1.
Test: stress-ng --kill 1 -t 23 (single-thread
kill()syscall loop)
Machine: IBM,9080-HEX POWER10, 8 vCPUs (1 core SMT-8), 64 GiBResults (bogo-ops/sec, higher is better):
Kernel Enforcing Permissive Disabled SELinux tax ------ --------- ---------- -------- ----------- 7.1 69,107 70,248 93,566 -26.1% 7.2-rc4 56,247 59,075 99,240 -43.3% Delta -18.6% -15.9% +6.1%Key observations:
- SELinux permissive ~ enforcing on both kernels (overhead is in the code path, not policy evaluation)
- SELinux disabled: 7.2 is 6% faster than 7.1 -- the generic entry path itself is not slower for the base syscall
- The SELinux "tax" nearly doubles from 26% to 43% between 7.1->7.2
- No changes tosecurity/selinux/avc.cbetween v7.1 and v7.2-rc4The
kill()hot path (sys_kill->check_kill_permission->security_task_kill->selinux_task_kill->avc_has_perm->avc_lookup) wraps every AVC lookup in rcu_read_lock()/rcu_read_unlock(). On ppc64le withPREEMPT_RCU, each pair requires lwsync/isync barriers. I suspect the generic entry path changes something in how context tracking, tracing, or RCU interacts with the syscall fast path that amplifies the per-call RCU cost.Quoted 6 lines
Note: this is on top of a separate ~36% regression from 6.12->7.1 caused by
HAVE_PREEMPT_DYNAMIC_KEYenablingCONFIG_PREEMPT_RCUon ppc64le (commit6ad7751537e8), which I reported separately on the v4 thread.Thank you
Jirka[...]
Hey Jirka,Thanks for reporting this.
I'll take a look.Quoted 9 lines
Quoted 3 lines
Thanks,
MukeshThank you, Mukesh! I'm happy to run additional tests on the POWER10 machine if you have patches to try or specific experiments you'd like me to do.
Jirka
can you try this with the below diff?
include/linux/rcutiny.h +1
-0@@ -120,6 +120,7 @@ static inline bool rcu_preempt_need_deferred_qs(struct task_struct *t)return false; }static inline void rcu_preempt_deferred_qs (struct task_struct *t) { } +static inline void rcu_ct_kernel_exit_qs (void ) { }void rcu_scheduler_starting (void );static inline void rcu_end_inkernel_boot (void ) { }static inline bool rcu_inkernel_boot_has_ended (void ) {return true; }include/linux/rcutree.h +1
-0@@ -87,6 +87,7 @@ static inline void rcu_irq_exit_check_preempt(void) { }struct task_struct ;void rcu_preempt_deferred_qs (struct task_struct *t); +void rcu_ct_kernel_exit_qs (void );void exit_rcu (void );kernel/context_tracking.c +1
-1@@ -118,7 +118,7 @@ static void noinstr ct_kernel_exit(bool user, int offset)lockdep_assert_irqs_disabled ();trace_rcu_watching (TPS ("End" ),ct_nesting (),0 ,ct_rcu_watching ());WARN_ON_ONCE (IS_ENABLED (CONFIG_RCU_EQS_DEBUG ) && !user && !is_idle_task (current)); -rcu_preempt_deferred_qs (current); +rcu_ct_kernel_exit_qs ();// instrumentation for the noinstr ct_kernel_exit_state() instrument_atomic_write (&ct->state,sizeof (ct->state));kernel/rcu/tree.c +12
-0@@ -368,6 +368,25 @@ notrace void rcu_momentary_eqs(void) }EXPORT_SYMBOL_GPL (rcu_momentary_eqs); +/** +* rcu_ct_kernel_exit_qs - report deferred QS on syscall/exception exit if needed +* +* Called from ct_kernel_exit() on every return to userspace. Guards the +* rcu_preempt_deferred_qs() call with rcu_preempt_need_deferred_qs() so that +* on the common fast path -- where nothing is deferred -- we avoid the +* cache-line traffic on current->rcu_read_unlock_special that the unconditional +* call causes. This is particularly significant on weakly-ordered architectures +* (e.g. ppc64le) where rcu_read_lock/unlock issue lwsync/isync barriers and +* already touch that cache line in the syscall body. +* +* Follows the same pattern used by rcu_flavor_sched_clock_irq(). kernel/rcu/tree.c notrace void rcu_momentary_eqs(void) · line 392
Quoted 10 lines
kernel/rcu/tree.c +7
-0+*/ +notrace void rcu_ct_kernel_exit_qs(void) +{ +if (rcu_preempt_need_deferred_qs (current)) +rcu_preempt_deferred_qs (current); +} +/** * rcu_is_cpu_rrupt_from_idle - see if 'interrupted' from idle * --
-Jirka
Signature · 1 line
-Jirka
Oh, I forgot to add the actual results from the LPAR:
Model name: POWER10 (architected), altivec supported Model: 2.0 (pvr 0080 0200) Type : Dedicated Mode : Capped Entitled Capacity : 1.00 Memory Mode : Dedicated
Results (median bogo-ops/sec, N=15 per cell):
Mode 7.1 7.2-rc4 Delta Enforcing 68,275 70,154 +2.7% Permissive 68,082 67,166 -1.3% Disabled 99,654 100,510 +0.9%
Quoted 200 lines · Jirka Hladky
Hi Mukesh,
Following up on the ppc64le SELinux/kill() regression - it doesn't reproduce with proper multi-iteration sampling. stress-ng --kill has ~30% run-to-run variance
on our LPARs. Part of the variance comes from LPARs being much less isolated than I'd assumed - we saw contention bleed across LPARs on the same physical server, even between "Dedicated" partitions. With 15 iterations per config, 7.1 and 7.2-rc4 are statistically even (within ~3%) across enforcing/ permissive/disabled, confirmed on a second, physically separate LPAR (7.1 vs 7.2-rc6). This matches what you said - you couldn't reproduce it either.I also built 7.2-rc4 from source with your proposed
rcu_ct_kernel_exit_qs()patch and re-ran the same sweep - it shows no consistent effect vs. stock 7.2-rc4 (differences within the same noise band), which is consistent with there being no real regression here for it to address.Sorry for the false alarm, and thank you for the patch and for taking the time to look into this - I don't think it's needed for this issue, though it may still be a reasonable efficiency improvement on its own merits (it does avoid the unconditional
rcu_preempt_deferred_qs()call on the common fast path). I'll update the internal RHEL tracking ticket (RHEL-216740) to reflect that this doesn't reproduce.Thank you!
JirkaQuoted 156 lines · Jirka Hladky
Hi Mukesh,
Re: policy details - I'm running an ELN kernel on RHEL-10.2, with SELinux in enforcing mode using the default (targeted) policy, nothing custom or hardened on top of it. Let me know if you need anything more specific (policy version, booleans, etc.) and I'll pull it.
Re: the
rcu_ct_kernel_exit_qs()diff — thanks, that guard against the unconditionalrcu_preempt_deferred_qs()call lines up well with what I was seeing on the RCU-barrier side. I'll apply it on top of 7.2 on POWER10 box and re-run the stress-ng --kill numbers (enforcing/permissive/disabled) to see how much of the SELinux tax it recovers. No need to prepare anything separately.I'm OOO tomorrow, but I'll get to this by the end of the week and report back with results.
Thanks
JirkaQuoted 132 lines · Mukesh Kumar Chaurasiya
Quoted 53 lines · Jirka Hladky
Quoted 51 lines · Mukesh Kumar Chaurasiya
Quoted 37 lines · Jirka Hladky
Hi Mukesh, Shrikanth, Madhavan,
I'm seeing a ~19% throughput regression on ppc64le (POWER10) in syscall-heavy workloads between kernel 7.1 and 7.2-rc4, which I've narrowed down to the
GENERIC_ENTRYenablement for powerpc (commitbee25f97ad24"powerpc: EnableGENERIC_ENTRYfeature").The regression manifests specifically when SELinux is enabled. Regression disappears with selinux=0. With SELinux disabled, 7.2 is actually slightly *faster* than 7.1.
Test: stress-ng --kill 1 -t 23 (single-thread
kill()syscall loop)
Machine: IBM,9080-HEX POWER10, 8 vCPUs (1 core SMT-8), 64 GiBResults (bogo-ops/sec, higher is better):
Kernel Enforcing Permissive Disabled SELinux tax ------ --------- ---------- -------- ----------- 7.1 69,107 70,248 93,566 -26.1% 7.2-rc4 56,247 59,075 99,240 -43.3% Delta -18.6% -15.9% +6.1%Key observations:
- SELinux permissive ~ enforcing on both kernels (overhead is in the code path, not policy evaluation)
- SELinux disabled: 7.2 is 6% faster than 7.1 -- the generic entry path itself is not slower for the base syscall
- The SELinux "tax" nearly doubles from 26% to 43% between 7.1->7.2
- No changes tosecurity/selinux/avc.cbetween v7.1 and v7.2-rc4The
kill()hot path (sys_kill->check_kill_permission->security_task_kill->selinux_task_kill->avc_has_perm->avc_lookup) wraps every AVC lookup in rcu_read_lock()/rcu_read_unlock(). On ppc64le withPREEMPT_RCU, each pair requires lwsync/isync barriers. I suspect the generic entry path changes something in how context tracking, tracing, or RCU interacts with the syscall fast path that amplifies the per-call RCU cost.Quoted 6 lines
Note: this is on top of a separate ~36% regression from 6.12->7.1 caused by
HAVE_PREEMPT_DYNAMIC_KEYenablingCONFIG_PREEMPT_RCUon ppc64le (commit6ad7751537e8), which I reported separately on the v4 thread.Thank you
Jirka[...]
Hey Jirka,Thanks for reporting this.
I'll take a look.Quoted 9 lines
Quoted 3 lines
Thanks,
MukeshThank you, Mukesh! I'm happy to run additional tests on the POWER10 machine if you have patches to try or specific experiments you'd like me to do.
Jirka
can you try this with the below diff?
include/linux/rcutiny.h +1
-0@@ -120,6 +120,7 @@ static inline bool rcu_preempt_need_deferred_qs(struct task_struct *t)return false; }static inline void rcu_preempt_deferred_qs (struct task_struct *t) { } +static inline void rcu_ct_kernel_exit_qs (void ) { }void rcu_scheduler_starting (void );static inline void rcu_end_inkernel_boot (void ) { }static inline bool rcu_inkernel_boot_has_ended (void ) {return true; }include/linux/rcutree.h +1
-0@@ -87,6 +87,7 @@ static inline void rcu_irq_exit_check_preempt(void) { }struct task_struct ;void rcu_preempt_deferred_qs (struct task_struct *t); +void rcu_ct_kernel_exit_qs (void );void exit_rcu (void );kernel/context_tracking.c +1
-1@@ -118,7 +118,7 @@ static void noinstr ct_kernel_exit(bool user, int offset)lockdep_assert_irqs_disabled ();trace_rcu_watching (TPS ("End" ),ct_nesting (),0 ,ct_rcu_watching ());WARN_ON_ONCE (IS_ENABLED (CONFIG_RCU_EQS_DEBUG ) && !user && !is_idle_task (current)); -rcu_preempt_deferred_qs (current); +rcu_ct_kernel_exit_qs ();// instrumentation for the noinstr ct_kernel_exit_state() instrument_atomic_write (&ct->state,sizeof (ct->state));kernel/rcu/tree.c +19
-0@@ -368,6 +368,25 @@ notrace void rcu_momentary_eqs(void) }EXPORT_SYMBOL_GPL (rcu_momentary_eqs); +/** +* rcu_ct_kernel_exit_qs - report deferred QS on syscall/exception exit if needed +* +* Called from ct_kernel_exit() on every return to userspace. Guards the +* rcu_preempt_deferred_qs() call with rcu_preempt_need_deferred_qs() so that +* on the common fast path -- where nothing is deferred -- we avoid the +* cache-line traffic on current->rcu_read_unlock_special that the unconditional +* call causes. This is particularly significant on weakly-ordered architectures +* (e.g. ppc64le) where rcu_read_lock/unlock issue lwsync/isync barriers and +* already touch that cache line in the syscall body. +* +* Follows the same pattern used by rcu_flavor_sched_clock_irq(). +*/ +notrace void rcu_ct_kernel_exit_qs(void) +{ +if (rcu_preempt_need_deferred_qs (current)) +rcu_preempt_deferred_qs (current); +} +/** Quoted 8 lines
kernel/rcu/tree.c notrace void rcu_momentary_eqs(void) · line 392
Quoted 4 lines
kernel/rcu/tree.c +0
-0* rcu_is_cpu_rrupt_from_idle - see if 'interrupted' from idle * --
-Jirka--
-Jirka
Signature · 1 line
-Jirka
Quoted 9 lines · Jirka Hladky
Oh, I forgot to add the actual results from the LPAR:
Model name: POWER10 (architected), altivec supported Model: 2.0 (pvr 0080 0200) Type : Dedicated Mode : Capped Entitled Capacity : 1.00 Memory Mode : Dedicated
Quoted 6 lines
Results (median bogo-ops/sec, N=15 per cell):
Mode 7.1 7.2-rc4 Delta Enforcing 68,275 70,154 +2.7% Permissive 68,082 67,166 -1.3% Disabled 99,654 100,510 +0.9%
[...]
Hey Jirka,
Thanks for confirming. Also with the diff I shared, did you see any difference in the performance.
Regards,
Mukesh
Hi Mukesh,
I built 7.2-rc4 from source with your proposed rcu_ct_kernel_exit_qs() patch and re-ran the same sweep - it shows minor improvements vs. stock 7.2-rc4
Full results, median bogo-ops/sec over 15 runs each:
Mode 7.1 7.2-rc4 7.2-rc4+patch Enforcing 68,275 70,154 71,585 Permissive 68,082 67,166 68,705 Disabled 99,654 100,510 101,016
The patch vs. stock 7.2-rc4 differences (+4.8%/+0.9%/+1.3%) are hard to distinquish
from run-to-run noise we've been seeing on this hardware, but they are consistent.
I think it's a useful efficiency improvement on its own merits, as it does avoid the unconditional rcu_preempt_deferred_qs() call on the common fast path. Even with only small performance gains, the change looks correct and low-risk to me, and I'd like to see it land in mainline. Let me know if there's
anything else you'd like me to test before you post it, or if it's already queued up
separately from this thread.
Thank you!
Jirka
Quoted 26 lines · Mukesh Kumar Chaurasiya
Quoted 9 lines · Jirka Hladky
Oh, I forgot to add the actual results from the LPAR:
Model name: POWER10 (architected), altivec supported Model: 2.0 (pvr 0080 0200) Type : Dedicated Mode : Capped Entitled Capacity : 1.00 Memory Mode : DedicatedQuoted 6 lines
Results (median bogo-ops/sec, N=15 per cell):
Mode 7.1 7.2-rc4 Delta Enforcing 68,275 70,154 +2.7% Permissive 68,082 67,166 -1.3% Disabled 99,654 100,510 +0.9%[...]
Hey Jirka,Thanks for confirming. Also with the diff I shared, did you see any difference in the performance.
Regards,
Mukesh
Signature · 1 line
-Jirka
Hi Mukesh,
I built 7.2-rc4 from source with your proposed rcu_ct_kernel_exit_qs() patch and re-ran the same sweep - it shows minor improvements vs. stock 7.2-rc4
Full results, median bogo-ops/sec over 15 runs each:
Mode 7.1 7.2-rc4 7.2-rc4+patch Enforcing 68,275 70,154 71,585 Permissive 68,082 67,166 68,705 Disabled 99,654 100,510 101,016
The patch vs. stock 7.2-rc4 differences (+4.8%/+0.9%/+1.3%) are hard to distinquish
from run-to-run noise we've been seeing on this hardware, but they are consistent.
I think it's a useful efficiency improvement on its own merits, as it does avoid the unconditional rcu_preempt_deferred_qs() call on the common fast path. Even with only small performance gains, the change looks correct and low-risk to me, and I'd like to see it land in mainline. Let me know if there's
anything else you'd like me to test before you post it, or if it's already queued up
separately from this thread.
Thank you!
Jirka
Quoted 58 lines · Jirka Hladky
Hi Mukesh,
I built 7.2-rc4 from source with your proposed
rcu_ct_kernel_exit_qs()patch and re-ran the same sweep - it shows minor improvements vs. stock 7.2-rc4Full results, median bogo-ops/sec over 15 runs each:
Mode 7.1 7.2-rc4 7.2-rc4+patch Enforcing 68,275 70,154 71,585 Permissive 68,082 67,166 68,705 Disabled 99,654 100,510 101,016The patch vs. stock 7.2-rc4 differences (+4.8%/+0.9%/+1.3%) are hard to distinquish from run-to-run noise we've been seeing on this hardware, but they are consistent.
I think it's a useful efficiency improvement on its own merits, as it does avoid the unconditional
rcu_preempt_deferred_qs()call on the common fast path. Even with only small performance gains, the change looks correct and low-risk to me, and I'd like to see it land in mainline. Let me know if there's anything else you'd like me to test before you post it, or if it's already queued up separately from this thread.Thank you!
JirkaQuoted 17 lines · Mukesh Kumar Chaurasiya
Quoted 9 lines · Jirka Hladky
Oh, I forgot to add the actual results from the LPAR:
Model name: POWER10 (architected), altivec supported Model: 2.0 (pvr 0080 0200) Type : Dedicated Mode : Capped Entitled Capacity : 1.00 Memory Mode : DedicatedQuoted 6 lines
Results (median bogo-ops/sec, N=15 per cell):
Mode 7.1 7.2-rc4 Delta Enforcing 68,275 70,154 +2.7% Permissive 68,082 67,166 -1.3% Disabled 99,654 100,510 +0.9%Quoted 9 lines
[...]
Hey Jirka,Thanks for confirming. Also with the diff I shared, did you see any difference in the performance.
Regards,
Mukesh--
-Jirka
Signature · 1 line
-Jirka
Quoted 27 lines · Jirka Hladky
Hi Mukesh,
I built 7.2-rc4 from source with your proposed
rcu_ct_kernel_exit_qs()patch and re-ran the same sweep - it shows minor improvements vs. stock 7.2-rc4Full results, median bogo-ops/sec over 15 runs each:
Mode 7.1 7.2-rc4 7.2-rc4+patch Enforcing 68,275 70,154 71,585 Permissive 68,082 67,166 68,705 Disabled 99,654 100,510 101,016The patch vs. stock 7.2-rc4 differences (+4.8%/+0.9%/+1.3%) are hard to distinquish
from run-to-run noise we've been seeing on this hardware, but they are consistent.I think it's a useful efficiency improvement on its own merits, as it does avoid the unconditional
rcu_preempt_deferred_qs()call on the common fast path. Even with only small performance gains, the change looks correct and low-risk to me, and I'd like to see it land in mainline. Let me know if there's
anything else you'd like me to test before you post it, or if it's already queued up
separately from this thread.
Quoted 3 lines
Thank you!
Jirka
Hey Jirka,
Thanks for the verification. I think that's all i wanted. I would send out a patch for this soon. Thanks for all the support.
Regards,
Mukesh
[...]
Quoted 10 lines · Mukesh Kumar Chaurasiya
Hey Jirka,
Thanks for the verification. I think that's all i wanted. I would send out a patch for this soon. Thanks for all the support.
Regards,
Mukesh
[...]
Thank you, Mukesh!
Signature · 1 line
-Jirka