powerpc: Remove STACK_FRAME_OVERHEAD
Subsystems (4)
Nicholas Piggin
reviewercover powerpc: Remove STACK_FRAME_OVERHEAD
Since RFC:
- Fix a compile bug.
- Fix BookE KVM properly. Hopefully -- I don't have a BookE KVM environment to test. Can QEMU do it? Is it still tested?
- Drop the last two patches that changed the stack layout, they can be done later.
- Drop the load/store-multiple change to 32-bit.
Thanks,
Nick
Nicholas Piggin (17):
KVM: PPC: Book3E: Fix CONFIG_TRACE_IRQFLAGS support powerpc/64: Remove asm interrupt tracing call helpers powerpc/perf: callchain validate kernel stack pointer bounds
powerpc: Rearrange copy_thread child stack creation powerpc/pseries: hvcall stack frame overhead
powerpc: simplify ppc_save_regs
powerpc: add definition for pt_regs offset within an interrupt frame
powerpc: add a definition for the marker offset within the interrupt
frame
powerpc: Rename STACK_FRAME_MARKER and derive it from frame offset
powerpc: add a define for the user interrupt frame size
powerpc: add a define for the switch frame size and regs offset
powerpc: copy_thread fill in interrupt frame marker and back chain
powerpc: copy_thread add a back chain to the switch stack frame
powerpc: split validate_sp into two functions
powerpc: allow minimum sized kernel stack frames powerpc/64: ELFv2 use minimal stack frames in int and switch frame
sizes
powerpc: remove STACK_FRAME_OVERHEAD
arch/powerpc/include/asm/irqflags.h | 58 ------------- arch/powerpc/include/asm/kvm_ppc.h | 12 +++ arch/powerpc/include/asm/processor.h | 15 +++- arch/powerpc/include/asm/ptrace.h | 37 ++++++--- arch/powerpc/kernel/asm-offsets.c | 9 +- arch/powerpc/kernel/entry_32.S | 14 ++-- arch/powerpc/kernel/exceptions-64e.S | 44 +++++----- arch/powerpc/kernel/exceptions-64s.S | 82 +++++++++---------- arch/powerpc/kernel/head_32.h | 4 +- arch/powerpc/kernel/head_40x.S | 2 +- arch/powerpc/kernel/head_44x.S | 6 +- arch/powerpc/kernel/head_64.S | 6 +- arch/powerpc/kernel/head_85xx.S | 8 +- arch/powerpc/kernel/head_8xx.S | 2 +- arch/powerpc/kernel/head_book3s_32.S | 4 +- arch/powerpc/kernel/head_booke.h | 4 +- arch/powerpc/kernel/interrupt_64.S | 32 ++++---- arch/powerpc/kernel/irq.c | 4 +- arch/powerpc/kernel/kgdb.c | 2 +- arch/powerpc/kernel/misc_32.S | 2 +- arch/powerpc/kernel/misc_64.S | 4 +- arch/powerpc/kernel/optprobes_head.S | 4 +- arch/powerpc/kernel/ppc_save_regs.S | 57 ++++--------- arch/powerpc/kernel/process.c | 54 +++++++----- arch/powerpc/kernel/smp.c | 2 +- arch/powerpc/kernel/stacktrace.c | 10 +-- arch/powerpc/kernel/tm.S | 8 +- arch/powerpc/kernel/trace/ftrace_mprofile.S | 2 +- arch/powerpc/kvm/book3s_hv_rmhandlers.S | 2 +- arch/powerpc/kvm/booke.c | 3 + arch/powerpc/kvm/bookehv_interrupts.S | 9 -- .../lib/test_emulate_step_exec_instr.S | 2 +- arch/powerpc/perf/callchain.c | 9 +- arch/powerpc/platforms/pseries/hvCall.S | 38 +++++---- arch/powerpc/xmon/xmon.c | 10 +-- 35 files changed, 259 insertions(+), 302 deletions(-)
Signature · 1 line
2.37.2
Nicholas Piggin
reviewerPATCH 1/17 KVM: PPC: Book3E: Fix CONFIG_TRACE_IRQFLAGS support
32-bit does not trace_irqs_off() to match the trace_irqs_on() call in kvmppc_fix_ee_before_entry(). This can lead to irqs being enabled twice in the trace, and the irqs-off region between guest exit and the host enabling local irqs again is not properly traced.
64-bit code does call this, but from asm code where volatiles are live and so incorrectly get clobbered.
Move the irq reconcile into C to fix both problems.
--- arch/powerpc/include/asm/kvm_ppc.h | 12 ++++++++++++ arch/powerpc/kvm/booke.c | 3 +++ arch/powerpc/kvm/bookehv_interrupts.S | 9 --------- 3 files changed, 15 insertions(+), 9 deletions(-)
arch/powerpc/include/asm/kvm_ppc.h +12 -0
@@ -1014,6 +1014,18 @@ static inline void kvmppc_fix_ee_before_entry(void)#endif } +static inline void kvmppc_fix_ee_after_exit (void ) +{ +#ifdef CONFIG_PPC64 +/* Only need to enable IRQs by hard enabling them after this */ + local_paca->irq_happened =PACA_IRQ_HARD_DIS ; +irq_soft_mask_set (IRQS_ALL_DISABLED ); +#endif + +trace_hardirqs_off (); +} + +static inline ulong kvmppc_get_ea_indexed (struct kvm_vcpu *vcpu,int ra,int rb) {ulong ea;
arch/powerpc/kvm/booke.c +3 -0
@@ -1015,6 +1015,9 @@ int kvmppc_handle_exit(struct kvm_vcpu *vcpu, unsigned int exit_nr)u32 last_inst =KVM_INST_FETCH_FAILED ;enum emulation_result emulated =EMULATE_DONE ; +/* Fix irq state (pairs with kvmppc_fix_ee_before_entry()) */ +kvmppc_fix_ee_after_exit (); +/* update before a new last_exit_type is rewritten */ kvmppc_update_timing_stats (vcpu);
arch/powerpc/kvm/bookehv_interrupts.S +0 -9
@@ -424,15 +424,6 @@ _GLOBAL(kvmppc_resume_host) mtspr SPRN_EPCR, r3 isync -#ifdef CONFIG_64BIT - /* - * We enter with interrupts disabled in hardware, but - * we need to call RECONCILE_IRQ_STATE to ensure - * that the software state is kept in sync. - */ - RECONCILE_IRQ_STATE(r3,r5) -#endif - /* Switch to kernel stack and jump to handler. */ mr r3, r4 mr r5, r14 /* intno */
Signature · 1 line
2.37.2
Nicholas Piggin
reviewerPATCH 2/17 powerpc/64: Remove asm interrupt tracing call helpers
In reply to Nicholas PigginThese are now unused. Remove.
--- arch/powerpc/include/asm/irqflags.h | 58 ----------------------------- 1 file changed, 58 deletions(-)
arch/powerpc/include/asm/irqflags.h +0 -58
@@ -11,64 +11,6 @@*/ #include <asm/hw_irq.h> -#else -#ifdef CONFIG_TRACE_IRQFLAGS -#ifdef CONFIG_IRQSOFF_TRACER -/* -* Since the ftrace irqsoff latency trace checks CALLER_ADDR1, -* which is the stack frame here, we need to force a stack frame -* in case we came from user space. -*/ -#define TRACE_WITH_FRAME_BUFFER (func) \ - mflr r0; \ - stdu r1, -STACK_FRAME_OVERHEAD(r1); \ - std r0, 16(r1); \ - stdu r1, -STACK_FRAME_OVERHEAD(r1); \ - bl func; \ - ld r1, 0(r1); \ - ld r1, 0(r1); -#else -#define TRACE_WITH_FRAME_BUFFER (func) \ - bl func; -#endif - -/* -* These are calls to C code, so the caller must be prepared for volatiles to -* be clobbered. -*/ -#define TRACE_ENABLE_INTS TRACE_WITH_FRAME_BUFFER(trace_hardirqs_on) -#define TRACE_DISABLE_INTS TRACE_WITH_FRAME_BUFFER(trace_hardirqs_off) - -/* -* This is used by assembly code to soft-disable interrupts first and -* reconcile irq state. -* -* NB: This may call C code, so the caller must be prepared for volatiles to -* be clobbered. -*/ -#define RECONCILE_IRQ_STATE (__rA, __rB) \ - lbz __rA,PACAIRQSOFTMASK(r13); \ - lbz __rB,PACAIRQHAPPENED(r13); \ - andi. __rA,__rA,IRQS_DISABLED; \ - li __rA,IRQS_DISABLED; \ - ori __rB,__rB,PACA_IRQ_HARD_DIS; \ - stb __rB,PACAIRQHAPPENED(r13); \ - bne 44f; \ - stb __rA,PACAIRQSOFTMASK(r13); \ - TRACE_DISABLE_INTS; \ -44: - -#else -#define TRACE_ENABLE_INTS -#define TRACE_DISABLE_INTS - -#define RECONCILE_IRQ_STATE (__rA, __rB) \ - lbz __rA,PACAIRQHAPPENED(r13); \ - li __rB,IRQS_DISABLED; \ - ori __rA,__rA,PACA_IRQ_HARD_DIS; \ - stb __rB,PACAIRQSOFTMASK(r13); \ - stb __rA,PACAIRQHAPPENED(r13) -#endif #endif #endif
Signature · 1 line
2.37.2
Nicholas Piggin
reviewerPATCH 3/17 powerpc/perf: callchain validate kernel stack pointer bounds
In reply to Nicholas PigginThe interrupt frame detection and loads from the hypothetical pt_regs are not bounds-checked. The next-frame validation only bounds-checks STACK_FRAME_OVERHEAD, which does not include the pt_regs. Add another test for this.
The user could set r1 to be equal to the address matching the first interrupt frame - STACK_INT_FRAME_SIZE, which is in the previous page due to the kernel redzone, and induce the kernel to load the marker from there. Possibly this could cause a crash at least. If the user could induce the previous page to contain a valid marker, then it might be able to direct perf to read specific memory addresses in a way that could be transmitted back to the user in the perf data.
---
Not sure if my attack scenario is actually valid, but I think there is some concern here...
Thanks,
Nick
arch/powerpc/perf/callchain.c | 1 + 1 file changed, 1 insertion(+)
arch/powerpc/perf/callchain.c +1 -0
@@ -61,6 +61,7 @@ perf_callchain_kernel(struct perf_callchain_entry_ctx *entry, struct pt_regs *re next_sp = fp[0 ];if (next_sp == sp +STACK_INT_FRAME_SIZE && +validate_sp (sp, current,STACK_INT_FRAME_SIZE ) && fp[STACK_FRAME_MARKER ] ==STACK_FRAME_REGS_MARKER ) {/* * This looks like an interrupt frame for an
Signature · 1 line
2.37.2
Nicholas Piggin
reviewerPATCH 4/17 powerpc: Rearrange copy_thread child stack creation
In reply to Nicholas PigginThis makes it a bit clearer where the stack frame is created, and will allow easier use of some of the stack offset constants in a later change.
--- arch/powerpc/kernel/process.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-)
arch/powerpc/kernel/process.c +6 -5
@@ -1726,13 +1726,16 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args)klp_init_thread_info (p); +/* Create initial stack frame. */ + sp -= (sizeof (struct pt_regs ) +STACK_FRAME_OVERHEAD ); + ((unsigned long *)sp)[0 ] =0 ; +/* Copy registers */ - sp -=sizeof (struct pt_regs ); - childregs = (struct pt_regs *) sp; + childregs = (struct pt_regs *)(sp +STACK_FRAME_OVERHEAD );if (unlikely (args->fn)) {/* kernel thread */ memset (childregs,0 ,sizeof (struct pt_regs )); - childregs->gpr[1 ] = sp +sizeof (struct pt_regs ); + childregs->gpr[1 ] = sp + (sizeof (struct pt_regs ) +STACK_FRAME_OVERHEAD );/* function */ if (args->fn) childregs->gpr[14 ] =ppc_function_entry ((void *)args->fn);
@@ -1767,7 +1770,6 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args) f = ret_from_fork; } childregs->msr &= ~(MSR_FP |MSR_VEC |MSR_VSX ); - sp -=STACK_FRAME_OVERHEAD ;/* * The way this works is that at some point in the future
@@ -1777,7 +1779,6 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args)* do some house keeping and then return from the fork or clone * system call, using the stack frame created above. */ - ((unsigned long *)sp)[0 ] =0 ; sp -=sizeof (struct pt_regs ); kregs = (struct pt_regs *) sp; sp -=STACK_FRAME_OVERHEAD ;
Signature · 1 line
2.37.2
Nicholas Piggin
reviewerPATCH 5/17 powerpc/pseries: hvcall stack frame overhead
In reply to Nicholas PigginThis call may use the min size stack frame. The scratch space used is in the caller's parameter area frame, not this function's frame.
--- arch/powerpc/platforms/pseries/hvCall.S | 38 +++++++++++++------------ 1 file changed, 20 insertions(+), 18 deletions(-)
arch/powerpc/platforms/pseries/hvCall.S +20 -18
@@ -27,7 +27,9 @@ hcall_tracepoint_refcount: /* * precall must preserve all registers. use unused STK_PARAM() - * areas to save snapshots and opcode. + * areas to save snapshots and opcode. STK_PARAM() in the caller's + * frame will be available even on ELFv2 because these are all + * variadic functions. */ #define HCALL_INST_PRECALL(FIRST_REG) \ mflr r0; \
@@ -41,29 +43,29 @@ hcall_tracepoint_refcount: std r10,STK_PARAM(R10)(r1); \ std r0,16(r1); \ addi r4,r1,STK_PARAM(FIRST_REG); \ - stdu r1,-STACK_FRAME_OVERHEAD(r1); \ + stdu r1,-STACK_FRAME_MIN_SIZE(r1); \ bl __trace_hcall_entry; \ - ld r3,STACK_FRAME_OVERHEAD+STK_PARAM(R3)(r1); \ - ld r4,STACK_FRAME_OVERHEAD+STK_PARAM(R4)(r1); \ - ld r5,STACK_FRAME_OVERHEAD+STK_PARAM(R5)(r1); \ - ld r6,STACK_FRAME_OVERHEAD+STK_PARAM(R6)(r1); \ - ld r7,STACK_FRAME_OVERHEAD+STK_PARAM(R7)(r1); \ - ld r8,STACK_FRAME_OVERHEAD+STK_PARAM(R8)(r1); \ - ld r9,STACK_FRAME_OVERHEAD+STK_PARAM(R9)(r1); \ - ld r10,STACK_FRAME_OVERHEAD+STK_PARAM(R10)(r1) + ld r3,STACK_FRAME_MIN_SIZE+STK_PARAM(R3)(r1); \ + ld r4,STACK_FRAME_MIN_SIZE+STK_PARAM(R4)(r1); \ + ld r5,STACK_FRAME_MIN_SIZE+STK_PARAM(R5)(r1); \ + ld r6,STACK_FRAME_MIN_SIZE+STK_PARAM(R6)(r1); \ + ld r7,STACK_FRAME_MIN_SIZE+STK_PARAM(R7)(r1); \ + ld r8,STACK_FRAME_MIN_SIZE+STK_PARAM(R8)(r1); \ + ld r9,STACK_FRAME_MIN_SIZE+STK_PARAM(R9)(r1); \ + ld r10,STACK_FRAME_MIN_SIZE+STK_PARAM(R10)(r1) /* * postcall is performed immediately before function return which * allows liberal use of volatile registers. */ #define __HCALL_INST_POSTCALL \ - ld r0,STACK_FRAME_OVERHEAD+STK_PARAM(R3)(r1); \ - std r3,STACK_FRAME_OVERHEAD+STK_PARAM(R3)(r1); \ + ld r0,STACK_FRAME_MIN_SIZE+STK_PARAM(R3)(r1); \ + std r3,STACK_FRAME_MIN_SIZE+STK_PARAM(R3)(r1); \ mr r4,r3; \ mr r3,r0; \ bl __trace_hcall_exit; \ - ld r0,STACK_FRAME_OVERHEAD+16(r1); \ - addi r1,r1,STACK_FRAME_OVERHEAD; \ + ld r0,STACK_FRAME_MIN_SIZE+16(r1); \ + addi r1,r1,STACK_FRAME_MIN_SIZE; \ ld r3,STK_PARAM(R3)(r1); \ mtlr r0
@@ -303,14 +305,14 @@ plpar_hcall9_trace: mr r7,r8 mr r8,r9 mr r9,r10 - ld r10,STACK_FRAME_OVERHEAD+STK_PARAM(R11)(r1) - ld r11,STACK_FRAME_OVERHEAD+STK_PARAM(R12)(r1) - ld r12,STACK_FRAME_OVERHEAD+STK_PARAM(R13)(r1) + ld r10,STACK_FRAME_MIN_SIZE+STK_PARAM(R11)(r1) + ld r11,STACK_FRAME_MIN_SIZE+STK_PARAM(R12)(r1) + ld r12,STACK_FRAME_MIN_SIZE+STK_PARAM(R13)(r1) HVSC mr r0,r12 - ld r12,STACK_FRAME_OVERHEAD+STK_PARAM(R4)(r1) + ld r12,STACK_FRAME_MIN_SIZE+STK_PARAM(R4)(r1) std r4,0(r12) std r5,8(r12) std r6,16(r12)
Signature · 1 line
2.37.2
Nicholas Piggin
reviewerPATCH 6/17 powerpc: simplify ppc_save_regs
In reply to Nicholas PigginAdjust the pt_regs pointer so the interrupt frame offsets can be used to save registers.
--- arch/powerpc/kernel/ppc_save_regs.S | 57 ++++++++--------------------- 1 file changed, 15 insertions(+), 42 deletions(-)
arch/powerpc/kernel/ppc_save_regs.S +15 -42
@@ -21,60 +21,33 @@ * different ABIs, though). */ _GLOBAL(ppc_save_regs) - PPC_STL r0,0*SZL(r3) + /* This allows stack frame accessor macros and offsets to be used */ + subi r3,r3,STACK_FRAME_OVERHEAD + PPC_STL r0,GPR0(r3) #ifdef CONFIG_PPC32 - stmw r2, 2*SZL(r3) + stmw r2,GPR2(r3) #else - PPC_STL r2,2*SZL(r3) - PPC_STL r3,3*SZL(r3) - PPC_STL r4,4*SZL(r3) - PPC_STL r5,5*SZL(r3) - PPC_STL r6,6*SZL(r3) - PPC_STL r7,7*SZL(r3) - PPC_STL r8,8*SZL(r3) - PPC_STL r9,9*SZL(r3) - PPC_STL r10,10*SZL(r3) - PPC_STL r11,11*SZL(r3) - PPC_STL r12,12*SZL(r3) - PPC_STL r13,13*SZL(r3) - PPC_STL r14,14*SZL(r3) - PPC_STL r15,15*SZL(r3) - PPC_STL r16,16*SZL(r3) - PPC_STL r17,17*SZL(r3) - PPC_STL r18,18*SZL(r3) - PPC_STL r19,19*SZL(r3) - PPC_STL r20,20*SZL(r3) - PPC_STL r21,21*SZL(r3) - PPC_STL r22,22*SZL(r3) - PPC_STL r23,23*SZL(r3) - PPC_STL r24,24*SZL(r3) - PPC_STL r25,25*SZL(r3) - PPC_STL r26,26*SZL(r3) - PPC_STL r27,27*SZL(r3) - PPC_STL r28,28*SZL(r3) - PPC_STL r29,29*SZL(r3) - PPC_STL r30,30*SZL(r3) - PPC_STL r31,31*SZL(r3) + SAVE_GPRS(2, 31, r3) lbz r0,PACAIRQSOFTMASK(r13) - PPC_STL r0,SOFTE-STACK_FRAME_OVERHEAD(r3) + PPC_STL r0,SOFTE(r3) #endif /* go up one stack frame for SP */ PPC_LL r4,0(r1) - PPC_STL r4,1*SZL(r3) + PPC_STL r4,GPR1(r3) /* get caller's LR */ PPC_LL r0,LRSAVE(r4) - PPC_STL r0,_LINK-STACK_FRAME_OVERHEAD(r3) + PPC_STL r0,_LINK(r3) mflr r0 - PPC_STL r0,_NIP-STACK_FRAME_OVERHEAD(r3) + PPC_STL r0,_NIP(r3) mfmsr r0 - PPC_STL r0,_MSR-STACK_FRAME_OVERHEAD(r3) + PPC_STL r0,_MSR(r3) mfctr r0 - PPC_STL r0,_CTR-STACK_FRAME_OVERHEAD(r3) + PPC_STL r0,_CTR(r3) mfxer r0 - PPC_STL r0,_XER-STACK_FRAME_OVERHEAD(r3) + PPC_STL r0,_XER(r3) mfcr r0 - PPC_STL r0,_CCR-STACK_FRAME_OVERHEAD(r3) + PPC_STL r0,_CCR(r3) li r0,0 - PPC_STL r0,_TRAP-STACK_FRAME_OVERHEAD(r3) - PPC_STL r0,ORIG_GPR3-STACK_FRAME_OVERHEAD(r3) + PPC_STL r0,_TRAP(r3) + PPC_STL r0,ORIG_GPR3(r3) blr
Signature · 1 line
2.37.2
Nicholas Piggin
reviewerPATCH 7/17 powerpc: add definition for pt_regs offset within an interrupt frame
In reply to Nicholas PigginThis is a common offset that currently uses the overloaded STACK_FRAME_OVERHEAD constant. It's easier to read and more flexible to use a specific regs offset for this.
--- arch/powerpc/include/asm/ptrace.h | 2 + arch/powerpc/kernel/asm-offsets.c | 7 +- arch/powerpc/kernel/entry_32.S | 6 +- arch/powerpc/kernel/exceptions-64e.S | 42 +++++----- arch/powerpc/kernel/exceptions-64s.S | 80 +++++++++---------- arch/powerpc/kernel/head_32.h | 2 +- arch/powerpc/kernel/head_85xx.S | 4 +- arch/powerpc/kernel/head_booke.h | 2 +- arch/powerpc/kernel/interrupt_64.S | 22 ++--- arch/powerpc/kernel/kgdb.c | 2 +- arch/powerpc/kernel/optprobes_head.S | 4 +- arch/powerpc/kernel/ppc_save_regs.S | 2 +- arch/powerpc/kernel/process.c | 4 +- arch/powerpc/kernel/tm.S | 8 +- arch/powerpc/kernel/trace/ftrace_mprofile.S | 2 +- .../lib/test_emulate_step_exec_instr.S | 2 +- arch/powerpc/perf/callchain.c | 2 +- arch/powerpc/xmon/xmon.c | 7 +- 18 files changed, 100 insertions(+), 100 deletions(-)
arch/powerpc/include/asm/ptrace.h +2 -0
@@ -124,6 +124,7 @@ struct pt_regs#define STACK_FRAME_LR_SAVE 2/* Location of LR in stack frame */ #define STACK_INT_FRAME_SIZE (sizeof(struct pt_regs) + \ STACK_FRAME_OVERHEAD + KERNEL_REDZONE_SIZE) +#define STACK_INT_FRAME_REGS STACK_FRAME_OVERHEAD#define STACK_FRAME_MARKER 12#ifdef CONFIG_PPC64_ELF_ABI_V2
@@ -143,6 +144,7 @@ struct pt_regs#define STACK_FRAME_OVERHEAD 16/* size of minimum stack frame */ #define STACK_FRAME_LR_SAVE 1/* Location of LR in stack frame */ #define STACK_INT_FRAME_SIZE (sizeof(struct pt_regs) + STACK_FRAME_OVERHEAD) +#define STACK_INT_FRAME_REGS STACK_FRAME_OVERHEAD#define STACK_FRAME_MARKER 2#define STACK_FRAME_MIN_SIZE STACK_FRAME_OVERHEAD
arch/powerpc/kernel/asm-offsets.c +3 -4
@@ -72,7 +72,7 @@#endif #define STACK_PT_REGS_OFFSET (sym, val) \ - DEFINE(sym, STACK_FRAME_OVERHEAD + offsetof(struct pt_regs, val)) + DEFINE(sym, STACK_INT_FRAME_REGS + offsetof(struct pt_regs, val))int main (void ) {
@@ -167,9 +167,8 @@ int main(void)OFFSET (THREAD_CKVRSTATE , thread_struct, ckvr_state.vr);OFFSET (THREAD_CKVRSAVE , thread_struct, ckvrsave);OFFSET (THREAD_CKFPSTATE , thread_struct, ckfp_state.fpr); -/* Local pt_regs on stack for Transactional Memory funcs. */ -DEFINE (TM_FRAME_SIZE ,STACK_FRAME_OVERHEAD + -sizeof (struct pt_regs ) +16 ); +/* Local pt_regs on stack in int frame form, plus 16 bytes for TM */ +DEFINE (TM_FRAME_SIZE ,STACK_INT_FRAME_SIZE +16 );#endif /* CONFIG_PPC_TRANSACTIONAL_MEM */ OFFSET (TI_LOCAL_FLAGS , thread_info, local_flags);
arch/powerpc/kernel/entry_32.S +3 -3
@@ -123,12 +123,12 @@ transfer_to_syscall: kuep_lock /* Calling convention has r3 = regs, r4 = orig r0 */ - addi r3,r1,STACK_FRAME_OVERHEAD + addi r3,r1,STACK_INT_FRAME_REGS mr r4,r0 bl system_call_exception ret_from_syscall: - addi r4,r1,STACK_FRAME_OVERHEAD + addi r4,r1,STACK_INT_FRAME_REGS li r5,0 bl syscall_exit_prepare #ifdef CONFIG_PPC_47x
@@ -293,7 +293,7 @@ _ASM_NOKPROBE_SYMBOL(fast_exception_return) .globl interrupt_return interrupt_return: lwz r4,_MSR(r1) - addi r3,r1,STACK_FRAME_OVERHEAD + addi r3,r1,STACK_INT_FRAME_REGS andi. r0,r4,MSR_PR beq .Lkernel_interrupt_return bl interrupt_exit_user_prepare
arch/powerpc/kernel/exceptions-64e.S +21 -21
168 lines not on this page · raw message
arch/powerpc/kernel/exceptions-64s.S +40 -40
320 lines not on this page · raw message
arch/powerpc/kernel/head_32.h +1 -1
@@ -127,7 +127,7 @@ _ASM_NOKPROBE_SYMBOL(\name\()_virt)mfspr r10,SPRN_XER addi r2, r2, -THREAD stw r10,_XER(r1) - addi r3,r1,STACK_FRAME_OVERHEAD + addi r3,r1,STACK_INT_FRAME_REGS .endm .macro prepare_transfer_to_handler
arch/powerpc/kernel/head_85xx.S +2 -2
@@ -972,10 +972,10 @@ _GLOBAL(__giveup_spe) li r4,THREAD_ACC evstddx evr6, r4, r3 /* save off accumulator */ beq 1f - lwz r4,_MSR-STACK_FRAME_OVERHEAD(r5) + lwz r4,_MSR-STACK_INT_FRAME_REGS(r5) lis r3,MSR_SPE@h andc r4,r4,r3 /* disable SPE for previous task */ - stw r4,_MSR-STACK_FRAME_OVERHEAD(r5) + stw r4,_MSR-STACK_INT_FRAME_REGS(r5) 1: blr #endif /* CONFIG_SPE */
arch/powerpc/kernel/head_booke.h +1 -1
@@ -99,7 +99,7 @@ END_BTB_FLUSH_SECTIONmfspr r10,SPRN_XER addi r2, r2, -THREAD stw r10,_XER(r1) - addi r3,r1,STACK_FRAME_OVERHEAD + addi r3,r1,STACK_INT_FRAME_REGS .endm .macro prepare_transfer_to_handler
arch/powerpc/kernel/interrupt_64.S +11 -11
@@ -78,7 +78,7 @@ _ASM_NOKPROBE_SYMBOL(system_call_vectored_\name) std r12,_CCR(r1) std r3,ORIG_GPR3(r1) /* Calling convention has r3 = regs, r4 = orig r0 */ - addi r3,r1,STACK_FRAME_OVERHEAD + addi r3,r1,STACK_INT_FRAME_REGS mr r4,r0 LOAD_REG_IMMEDIATE(r11, STACK_FRAME_REGS_MARKER) std r11,-16(r3) /* "regshere" marker */
@@ -99,7 +99,7 @@ END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR) bl system_call_exception .Lsyscall_vectored_\name\()_exit: - addi r4,r1,STACK_FRAME_OVERHEAD + addi r4,r1,STACK_INT_FRAME_REGS li r5,1 /* scv */ bl syscall_exit_prepare std r1,PACA_EXIT_SAVE_R1(r13) /* save r1 for restart */
@@ -176,7 +176,7 @@ _ASM_NOKPROBE_SYMBOL(syscall_vectored_\name\()_restart) ld r1,PACA_EXIT_SAVE_R1(r13) LOAD_PACA_TOC() ld r3,RESULT(r1) - addi r4,r1,STACK_FRAME_OVERHEAD + addi r4,r1,STACK_INT_FRAME_REGS li r11,IRQS_ALL_DISABLED stb r11,PACAIRQSOFTMASK(r13) bl syscall_exit_restart
@@ -251,7 +251,7 @@ END_BTB_FLUSH_SECTION std r12,_CCR(r1) std r3,ORIG_GPR3(r1) /* Calling convention has r3 = regs, r4 = orig r0 */ - addi r3,r1,STACK_FRAME_OVERHEAD + addi r3,r1,STACK_INT_FRAME_REGS mr r4,r0 LOAD_REG_IMMEDIATE(r11, STACK_FRAME_REGS_MARKER) std r11,-16(r3) /* "regshere" marker */
@@ -278,7 +278,7 @@ END_BTB_FLUSH_SECTION bl system_call_exception .Lsyscall_exit: - addi r4,r1,STACK_FRAME_OVERHEAD + addi r4,r1,STACK_INT_FRAME_REGS li r5,0 /* !scv */ bl syscall_exit_prepare std r1,PACA_EXIT_SAVE_R1(r13) /* save r1 for restart */
@@ -357,7 +357,7 @@ _ASM_NOKPROBE_SYMBOL(syscall_restart) ld r1,PACA_EXIT_SAVE_R1(r13) LOAD_PACA_TOC() ld r3,RESULT(r1) - addi r4,r1,STACK_FRAME_OVERHEAD + addi r4,r1,STACK_INT_FRAME_REGS li r11,IRQS_ALL_DISABLED stb r11,PACAIRQSOFTMASK(r13) bl syscall_exit_restart
@@ -388,7 +388,7 @@ _ASM_NOKPROBE_SYMBOL(fast_interrupt_return_srr) andi. r0,r5,MSR_RI li r3,0 /* 0 return value, no EMULATE_STACK_STORE */ bne+ .Lfast_kernel_interrupt_return_srr - addi r3,r1,STACK_FRAME_OVERHEAD + addi r3,r1,STACK_INT_FRAME_REGS bl unrecoverable_exception b . /* should not get here */ #else
@@ -406,7 +406,7 @@ _ASM_NOKPROBE_SYMBOL(interrupt_return_\srr\()) beq interrupt_return_\srr\()_kernel interrupt_return_\srr\()_user: /* make backtraces match the _kernel variant */ _ASM_NOKPROBE_SYMBOL(interrupt_return_\srr\()_user) - addi r3,r1,STACK_FRAME_OVERHEAD + addi r3,r1,STACK_INT_FRAME_REGS bl interrupt_exit_user_prepare cmpdi r3,0 bne- .Lrestore_nvgprs_\srr
@@ -503,7 +503,7 @@ _ASM_NOKPROBE_SYMBOL(interrupt_return_\srr\()_user_restart) GET_PACA(r13) ld r1,PACA_EXIT_SAVE_R1(r13) LOAD_PACA_TOC() - addi r3,r1,STACK_FRAME_OVERHEAD + addi r3,r1,STACK_INT_FRAME_REGS li r11,IRQS_ALL_DISABLED stb r11,PACAIRQSOFTMASK(r13) bl interrupt_exit_user_restart
@@ -518,7 +518,7 @@ RESTART_TABLE(.Linterrupt_return_\srr\()_user_rst_start, .Linterrupt_return_\srr .balign IFETCH_ALIGN_BYTES interrupt_return_\srr\()_kernel: _ASM_NOKPROBE_SYMBOL(interrupt_return_\srr\()_kernel) - addi r3,r1,STACK_FRAME_OVERHEAD + addi r3,r1,STACK_INT_FRAME_REGS bl interrupt_exit_kernel_prepare std r1,PACA_EXIT_SAVE_R1(r13) /* save r1 for restart */
@@ -684,7 +684,7 @@ _ASM_NOKPROBE_SYMBOL(interrupt_return_\srr\()_kernel_restart) GET_PACA(r13) ld r1,PACA_EXIT_SAVE_R1(r13) LOAD_PACA_TOC() - addi r3,r1,STACK_FRAME_OVERHEAD + addi r3,r1,STACK_INT_FRAME_REGS li r11,IRQS_ALL_DISABLED stb r11,PACAIRQSOFTMASK(r13) bl interrupt_exit_kernel_restart
arch/powerpc/kernel/kgdb.c +1 -1
@@ -191,7 +191,7 @@ static int kgdb_break_match(struct pt_regs *regs)void sleeping_thread_to_gdb_regs (unsigned long *gdb_regs,struct task_struct *p) {struct pt_regs *regs = (struct pt_regs *)(p->thread.ksp + -STACK_FRAME_OVERHEAD ); +STACK_INT_FRAME_REGS );unsigned long *ptr = gdb_regs;int reg;
arch/powerpc/kernel/optprobes_head.S +2 -2
@@ -85,7 +85,7 @@ optprobe_template_op_address: TEMPLATE_FOR_IMM_LOAD_INSNS /* 2. pt_regs pointer in r4 */ - addi r4,r1,STACK_FRAME_OVERHEAD + addi r4,r1,STACK_INT_FRAME_REGS .global optprobe_template_call_handler optprobe_template_call_handler:
@@ -96,7 +96,7 @@ optprobe_template_call_handler: * Parameters for instruction emulation: * 1. Pass SP in register r3. */ - addi r3,r1,STACK_FRAME_OVERHEAD + addi r3,r1,STACK_INT_FRAME_REGS .global optprobe_template_insn optprobe_template_insn:
arch/powerpc/kernel/ppc_save_regs.S +1 -1
@@ -22,7 +22,7 @@ */ _GLOBAL(ppc_save_regs) /* This allows stack frame accessor macros and offsets to be used */ - subi r3,r3,STACK_FRAME_OVERHEAD + subi r3,r3,STACK_INT_FRAME_REGS PPC_STL r0,GPR0(r3) #ifdef CONFIG_PPC32 stmw r2,GPR2(r3)
arch/powerpc/kernel/process.c +2 -2
@@ -2231,12 +2231,12 @@ void __no_sanitize_address show_stack(struct task_struct *tsk,/* * See if this is an exception frame. -* We look for the "regshere" marker in the current frame. +* We look for the "regs" marker in the current frame. */ if (validate_sp (sp, tsk,STACK_FRAME_WITH_PT_REGS ) && stack[STACK_FRAME_MARKER ] ==STACK_FRAME_REGS_MARKER ) {struct pt_regs *regs = (struct pt_regs *) - (sp +STACK_FRAME_OVERHEAD ); + (sp +STACK_INT_FRAME_REGS ); lr = regs->link; printk("%s--- interrupt: %lx at %pS\n",
arch/powerpc/kernel/tm.S +4 -4
@@ -117,7 +117,7 @@ _GLOBAL(tm_reclaim) std r2, STK_GOT(r1) stdu r1, -TM_FRAME_SIZE(r1) - /* We've a struct pt_regs at [r1+STACK_FRAME_OVERHEAD]. */ + /* We've a struct pt_regs at [r1+STACK_INT_FRAME_REGS]. */ std r3, STK_PARAM(R3)(r1) SAVE_NVGPRS(r1)
@@ -222,7 +222,7 @@ _GLOBAL(tm_reclaim) * Make r7 look like an exception frame so that we can use the neat * GPRx(n) macros. r7 is NOT a pt_regs ptr! */ - subi r7, r7, STACK_FRAME_OVERHEAD + subi r7, r7, STACK_INT_FRAME_REGS /* Sync the userland GPRs 2-12, 14-31 to thread->regs: */ SAVE_GPR(0, r7) /* user r0 */
@@ -359,7 +359,7 @@ _GLOBAL(__tm_recheckpoint) stdu r1, -TM_FRAME_SIZE(r1) /* - * We've a struct pt_regs at [r1+STACK_FRAME_OVERHEAD]. + * We've a struct pt_regs at [r1+STACK_INT_FRAME_REGS]. * This is used for backing up the NVGPRs: */ SAVE_NVGPRS(r1)
@@ -379,7 +379,7 @@ _GLOBAL(__tm_recheckpoint) * Make r7 look like an exception frame so that we can use the neat * GPRx(n) macros. r7 is now NOT a pt_regs ptr! */ - subi r7, r7, STACK_FRAME_OVERHEAD + subi r7, r7, STACK_INT_FRAME_REGS /* We need to setup MSR for FP/VMX/VSX register save instructions. */ mfmsr r6
arch/powerpc/kernel/trace/ftrace_mprofile.S +1 -1
@@ -110,7 +110,7 @@ .endif /* Load &pt_regs in r6 for call below */ - addi r6, r1, STACK_FRAME_OVERHEAD + addi r6, r1, STACK_INT_FRAME_REGS .endm .macro ftrace_regs_exit allregs
arch/powerpc/lib/test_emulate_step_exec_instr.S +1 -1
@@ -16,7 +16,7 @@ _GLOBAL(exec_instr) /* * Stack frame layout (INT_FRAME_SIZE bytes) - * In-memory pt_regs (SP + STACK_FRAME_OVERHEAD) + * In-memory pt_regs (SP + STACK_INT_FRAME_REGS) * Scratch space (SP + 8) * Back chain (SP + 0) */
arch/powerpc/perf/callchain.c +1 -1
@@ -67,7 +67,7 @@ perf_callchain_kernel(struct perf_callchain_entry_ctx *entry, struct pt_regs *re* This looks like an interrupt frame for an * interrupt that occurred in the kernel */ - regs = (struct pt_regs *)(sp +STACK_FRAME_OVERHEAD ); + regs = (struct pt_regs *)(sp +STACK_INT_FRAME_REGS ); next_ip = regs->nip; lr = regs->link; level =0 ;
arch/powerpc/xmon/xmon.c +3 -4
@@ -1781,14 +1781,13 @@ static void xmon_show_stack(unsigned long sp, unsigned long lr,xmon_print_symbol (ip," " ,"\n" ); } -/* Look for "regshere" marker to see if this is +/* Look for "regs" marker to see if this is an exception frame. */ if (mread (sp +MARKER_OFFSET , &marker,sizeof (unsigned long )) && marker ==STACK_FRAME_REGS_MARKER ) { -if (mread (sp +STACK_FRAME_OVERHEAD , ®s,sizeof (regs)) - !=sizeof (regs)) { +if (mread (sp +STACK_INT_FRAME_REGS , ®s,sizeof (regs)) !=sizeof (regs)) {printf ("Couldn't read registers at %lx\n" , - sp +STACK_FRAME_OVERHEAD ); + sp +STACK_INT_FRAME_REGS );break ; } printf("--- Exception: %lx %s at ", regs.trap,
Signature · 1 line
2.37.2
Nicholas Piggin
reviewerPATCH 8/17 powerpc: add a definition for the marker offset within the interrupt frame
In reply to Nicholas PigginDefine a constant rather than open-code the offset for the "regs" marker.
--- arch/powerpc/include/asm/ptrace.h | 2 ++ arch/powerpc/kernel/entry_32.S | 2 +- arch/powerpc/kernel/exceptions-64e.S | 2 +- arch/powerpc/kernel/exceptions-64s.S | 2 +- arch/powerpc/kernel/head_32.h | 2 +- arch/powerpc/kernel/head_booke.h | 2 +- arch/powerpc/kernel/interrupt_64.S | 10 +++++----- arch/powerpc/kvm/book3s_hv_rmhandlers.S | 2 +- 8 files changed, 13 insertions(+), 11 deletions(-)
arch/powerpc/include/asm/ptrace.h +2 -0
@@ -125,6 +125,7 @@ struct pt_regs#define STACK_INT_FRAME_SIZE (sizeof(struct pt_regs) + \ STACK_FRAME_OVERHEAD + KERNEL_REDZONE_SIZE)#define STACK_INT_FRAME_REGS STACK_FRAME_OVERHEAD +#define STACK_INT_FRAME_MARKER (STACK_FRAME_OVERHEAD - 16)#define STACK_FRAME_MARKER 12#ifdef CONFIG_PPC64_ELF_ABI_V2
@@ -145,6 +146,7 @@ struct pt_regs#define STACK_FRAME_LR_SAVE 1/* Location of LR in stack frame */ #define STACK_INT_FRAME_SIZE (sizeof(struct pt_regs) + STACK_FRAME_OVERHEAD)#define STACK_INT_FRAME_REGS STACK_FRAME_OVERHEAD +#define STACK_INT_FRAME_MARKER (STACK_FRAME_OVERHEAD - 8)#define STACK_FRAME_MARKER 2#define STACK_FRAME_MIN_SIZE STACK_FRAME_OVERHEAD
arch/powerpc/kernel/entry_32.S +1 -1
@@ -114,7 +114,7 @@ transfer_to_syscall: addi r12,r12,STACK_FRAME_REGS_MARKER@l stw r9,_MSR(r1) li r2, INTERRUPT_SYSCALL - stw r12,8(r1) + stw r12,STACK_INT_FRAME_MARKER(r1) stw r2,_TRAP(r1) SAVE_GPR(0, r1) SAVE_GPRS(3, 8, r1)
arch/powerpc/kernel/exceptions-64e.S +1 -1
@@ -391,7 +391,7 @@ exc_##n##_common: \ std r10,_CCR(r1); /* store orig CR in stackframe */ \ std r9,GPR1(r1); /* store stack frame back link */ \ std r11,SOFTE(r1); /* and save it to stackframe */ \ - std r12,STACK_FRAME_OVERHEAD-16(r1); /* mark the frame */ \ + std r12,STACK_INT_FRAME_MARKER(r1); /* mark the frame */ \ std r3,_TRAP(r1); /* set trap number */ \ std r0,RESULT(r1); /* clear regs->result */ \ SAVE_NVGPRS(r1);
arch/powerpc/kernel/exceptions-64s.S +1 -1
@@ -591,7 +591,7 @@ END_FTR_SECTION_IFSET(CPU_FTR_CFAR) li r10,0 LOAD_REG_IMMEDIATE(r11, STACK_FRAME_REGS_MARKER) std r10,RESULT(r1) /* clear regs->result */ - std r11,STACK_FRAME_OVERHEAD-16(r1) /* mark the frame */ + std r11,STACK_INT_FRAME_MARKER(r1) /* mark the frame */ .endm /*
arch/powerpc/kernel/head_32.h +1 -1
@@ -112,7 +112,7 @@ _ASM_NOKPROBE_SYMBOL(\name\()_virt) stw r0,GPR0(r1) lis r10,STACK_FRAME_REGS_MARKER@ha /* exception frame marker */ addi r10,r10,STACK_FRAME_REGS_MARKER@l - stw r10,8(r1) + stw r10,STACK_INT_FRAME_MARKER(r1) li r10, \trapno stw r10,_TRAP(r1) SAVE_GPRS(3, 8, r1)
arch/powerpc/kernel/head_booke.h +1 -1
@@ -84,7 +84,7 @@ END_BTB_FLUSH_SECTION stw r0,GPR0(r1) lis r10, STACK_FRAME_REGS_MARKER@ha /* exception frame marker */ addi r10, r10, STACK_FRAME_REGS_MARKER@l - stw r10, 8(r1) + stw r10, STACK_INT_FRAME_MARKER(r1) li r10, \trapno stw r10,_TRAP(r1) SAVE_GPRS(3, 8, r1)
arch/powerpc/kernel/interrupt_64.S +5 -5
@@ -77,11 +77,11 @@ _ASM_NOKPROBE_SYMBOL(system_call_vectored_\name) std r11,_TRAP(r1) std r12,_CCR(r1) std r3,ORIG_GPR3(r1) + LOAD_REG_IMMEDIATE(r11, STACK_FRAME_REGS_MARKER) + std r11,STACK_INT_FRAME_MARKER(r1) /* "regs" marker */ /* Calling convention has r3 = regs, r4 = orig r0 */ addi r3,r1,STACK_INT_FRAME_REGS mr r4,r0 - LOAD_REG_IMMEDIATE(r11, STACK_FRAME_REGS_MARKER) - std r11,-16(r3) /* "regshere" marker */ BEGIN_FTR_SECTION HMT_MEDIUM
@@ -250,11 +250,11 @@ END_BTB_FLUSH_SECTION std r11,_TRAP(r1) std r12,_CCR(r1) std r3,ORIG_GPR3(r1) + LOAD_REG_IMMEDIATE(r11, STACK_FRAME_REGS_MARKER) + std r11,STACK_INT_FRAME_MARKER(r1) /* "regs" marker */ /* Calling convention has r3 = regs, r4 = orig r0 */ addi r3,r1,STACK_INT_FRAME_REGS mr r4,r0 - LOAD_REG_IMMEDIATE(r11, STACK_FRAME_REGS_MARKER) - std r11,-16(r3) /* "regshere" marker */ #ifdef CONFIG_PPC_BOOK3S li r11,1
@@ -637,7 +637,7 @@ ALT_FTR_SECTION_END_IFCLR(CPU_FTR_STCX_CHECKS_ADDRESS) * Leaving a stale STACK_FRAME_REGS_MARKER on the stack can confuse * the reliable stack unwinder later on. Clear it. */ - std r0,STACK_FRAME_OVERHEAD-16(r1) + std r0,STACK_INT_FRAME_MARKER(r1) REST_GPRS(2, 5, r1)
arch/powerpc/kvm/book3s_hv_rmhandlers.S +1 -1
@@ -2729,7 +2729,7 @@ kvmppc_bad_host_intr: std r6, SOFTE(r1) LOAD_PACA_TOC() LOAD_REG_IMMEDIATE(3, STACK_FRAME_REGS_MARKER) - std r3, STACK_FRAME_OVERHEAD-16(r1) + std r3, STACK_INT_FRAME_MARKER(r1) /* * XXX On POWER7 and POWER8, we just spin here since we don't
Signature · 1 line
2.37.2
Nicholas Piggin
reviewerPATCH 9/17 powerpc: Rename STACK_FRAME_MARKER and derive it from frame offset
In reply to Nicholas PigginThis is a count of longs from the stack pointer to the regs marker. Rename it to make it more distinct from the other byte offsets. It can be derived from the byte offset definitions just added.
--- arch/powerpc/include/asm/ptrace.h | 4 ++-- arch/powerpc/kernel/process.c | 2 +- arch/powerpc/kernel/stacktrace.c | 2 +- arch/powerpc/perf/callchain.c | 2 +- arch/powerpc/xmon/xmon.c | 3 +-- 5 files changed, 6 insertions(+), 7 deletions(-)
arch/powerpc/include/asm/ptrace.h +2 -2
@@ -126,7 +126,6 @@ struct pt_regs STACK_FRAME_OVERHEAD + KERNEL_REDZONE_SIZE)#define STACK_INT_FRAME_REGS STACK_FRAME_OVERHEAD#define STACK_INT_FRAME_MARKER (STACK_FRAME_OVERHEAD - 16) -#define STACK_FRAME_MARKER 12#ifdef CONFIG_PPC64_ELF_ABI_V2 #define STACK_FRAME_MIN_SIZE 32
@@ -147,7 +146,6 @@ struct pt_regs#define STACK_INT_FRAME_SIZE (sizeof(struct pt_regs) + STACK_FRAME_OVERHEAD)#define STACK_INT_FRAME_REGS STACK_FRAME_OVERHEAD#define STACK_INT_FRAME_MARKER (STACK_FRAME_OVERHEAD - 8) -#define STACK_FRAME_MARKER 2#define STACK_FRAME_MIN_SIZE STACK_FRAME_OVERHEAD/* Size of stack frame allocated when calling signal handler. */
@@ -155,6 +153,8 @@ struct pt_regs#endif /* __powerpc64__ */ +#define STACK_INT_FRAME_MARKER_LONGS (STACK_INT_FRAME_MARKER/sizeof(long)) +#ifndef __ASSEMBLY__#include <asm/paca.h>
arch/powerpc/kernel/process.c +1 -1
@@ -2234,7 +2234,7 @@ void __no_sanitize_address show_stack(struct task_struct *tsk,* We look for the "regs" marker in the current frame. */ if (validate_sp (sp, tsk,STACK_FRAME_WITH_PT_REGS ) - && stack[STACK_FRAME_MARKER ] ==STACK_FRAME_REGS_MARKER ) { + && stack[STACK_INT_FRAME_MARKER_LONGS ] ==STACK_FRAME_REGS_MARKER ) {struct pt_regs *regs = (struct pt_regs *) (sp +STACK_INT_FRAME_REGS );
arch/powerpc/kernel/stacktrace.c +1 -1
@@ -136,7 +136,7 @@ int __no_sanitize_address arch_stack_walk_reliable(stack_trace_consume_fn consum/* Mark stacktraces with exception frames as unreliable. */ if (sp <= stack_end -STACK_INT_FRAME_SIZE && - stack[STACK_FRAME_MARKER ] ==STACK_FRAME_REGS_MARKER ) { + stack[STACK_INT_FRAME_MARKER_LONGS ] ==STACK_FRAME_REGS_MARKER ) {return -EINVAL ; }
arch/powerpc/perf/callchain.c +1 -1
@@ -62,7 +62,7 @@ perf_callchain_kernel(struct perf_callchain_entry_ctx *entry, struct pt_regs *reif (next_sp == sp +STACK_INT_FRAME_SIZE &&validate_sp (sp, current,STACK_INT_FRAME_SIZE ) && - fp[STACK_FRAME_MARKER ] ==STACK_FRAME_REGS_MARKER ) { + fp[STACK_INT_FRAME_MARKER_LONGS ] ==STACK_FRAME_REGS_MARKER ) {/* * This looks like an interrupt frame for an * interrupt that occurred in the kernel
arch/powerpc/xmon/xmon.c +1 -2
@@ -1720,7 +1720,6 @@ static void get_function_bounds(unsigned long pc, unsigned long *startp, }#define LRSAVE_OFFSET (STACK_FRAME_LR_SAVE * sizeof(unsigned long)) -#define MARKER_OFFSET (STACK_FRAME_MARKER * sizeof(unsigned long))static void xmon_show_stack (unsigned long sp,unsigned long lr,unsigned long pc)
@@ -1783,7 +1782,7 @@ static void xmon_show_stack(unsigned long sp, unsigned long lr,/* Look for "regs" marker to see if this is an exception frame. */ -if (mread (sp +MARKER_OFFSET , &marker,sizeof (unsigned long )) +if (mread (sp +STACK_INT_FRAME_MARKER , &marker,sizeof (unsigned long )) && marker ==STACK_FRAME_REGS_MARKER ) {if (mread (sp +STACK_INT_FRAME_REGS , ®s,sizeof (regs)) !=sizeof (regs)) { printf("Couldn't read registers at %lx\n",
Signature · 1 line
2.37.2
Nicholas Piggin
reviewerPATCH 10/17 powerpc: add a define for the user interrupt frame size
In reply to Nicholas PigginThe user interrupt frame is a different size from the kernel frame, so give it its own name.
--- arch/powerpc/include/asm/ptrace.h | 6 +++--- arch/powerpc/kernel/process.c | 6 +++--- arch/powerpc/kernel/stacktrace.c | 4 ++-- 3 files changed, 8 insertions(+), 8 deletions(-)
arch/powerpc/include/asm/ptrace.h +3 -3
@@ -122,8 +122,7 @@ struct pt_regs#define STACK_FRAME_OVERHEAD 112/* size of minimum stack frame */ #define STACK_FRAME_LR_SAVE 2/* Location of LR in stack frame */ -#define STACK_INT_FRAME_SIZE (sizeof(struct pt_regs) + \ - STACK_FRAME_OVERHEAD + KERNEL_REDZONE_SIZE) +#define STACK_USER_INT_FRAME_SIZE (sizeof(struct pt_regs) + STACK_FRAME_OVERHEAD)#define STACK_INT_FRAME_REGS STACK_FRAME_OVERHEAD#define STACK_INT_FRAME_MARKER (STACK_FRAME_OVERHEAD - 16)
@@ -143,7 +142,7 @@ struct pt_regs#define KERNEL_REDZONE_SIZE 0#define STACK_FRAME_OVERHEAD 16/* size of minimum stack frame */ #define STACK_FRAME_LR_SAVE 1/* Location of LR in stack frame */ -#define STACK_INT_FRAME_SIZE (sizeof(struct pt_regs) + STACK_FRAME_OVERHEAD) +#define STACK_USER_INT_FRAME_SIZE (sizeof(struct pt_regs) + STACK_FRAME_OVERHEAD)#define STACK_INT_FRAME_REGS STACK_FRAME_OVERHEAD#define STACK_INT_FRAME_MARKER (STACK_FRAME_OVERHEAD - 8)#define STACK_FRAME_MIN_SIZE STACK_FRAME_OVERHEAD
@@ -153,6 +152,7 @@ struct pt_regs#endif /* __powerpc64__ */ +#define STACK_INT_FRAME_SIZE (KERNEL_REDZONE_SIZE + STACK_USER_INT_FRAME_SIZE)#define STACK_INT_FRAME_MARKER_LONGS (STACK_INT_FRAME_MARKER/sizeof(long))#ifndef __ASSEMBLY__
arch/powerpc/kernel/process.c +3 -3
@@ -1727,15 +1727,15 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args)klp_init_thread_info (p);/* Create initial stack frame. */ - sp -= (sizeof (struct pt_regs ) +STACK_FRAME_OVERHEAD ); + sp -=STACK_USER_INT_FRAME_SIZE ; ((unsigned long *)sp)[0 ] =0 ;/* Copy registers */ - childregs = (struct pt_regs *)(sp +STACK_FRAME_OVERHEAD ); + childregs = (struct pt_regs *)(sp +STACK_INT_FRAME_REGS );if (unlikely (args->fn)) {/* kernel thread */ memset (childregs,0 ,sizeof (struct pt_regs )); - childregs->gpr[1 ] = sp + (sizeof (struct pt_regs ) +STACK_FRAME_OVERHEAD ); + childregs->gpr[1 ] = sp +STACK_USER_INT_FRAME_SIZE ;/* function */ if (args->fn) childregs->gpr[14 ] =ppc_function_entry ((void *)args->fn);
arch/powerpc/kernel/stacktrace.c +2 -2
@@ -77,7 +77,7 @@ int __no_sanitize_address arch_stack_walk_reliable(stack_trace_consume_fn consum/* * For user tasks, this is the SP value loaded on * kernel entry, see "PACAKSAVE(r13)" in _switch() and -* system_call_common()/EXCEPTION_PROLOG_COMMON(). +* system_call_common(). * * Likewise for non-swapper kernel threads, * this also happens to be the top of the stack
@@ -88,7 +88,7 @@ int __no_sanitize_address arch_stack_walk_reliable(stack_trace_consume_fn consum* an unreliable stack trace until it's been * _switch()'ed to for the first time. */ - stack_end -=STACK_FRAME_OVERHEAD +sizeof (struct pt_regs ); + stack_end -=STACK_USER_INT_FRAME_SIZE ; }else {/* * idle tasks have a custom stack layout,
Signature · 1 line
2.37.2
Nicholas Piggin
reviewerPATCH 11/17 powerpc: add a define for the switch frame size and regs offset
In reply to Nicholas PigginThis is open-coded in process.c, ppc32 uses a different define with the same value, and the C definition is name differently which makes it an extra indirection to grep for.
--- arch/powerpc/include/asm/ptrace.h | 6 ++++-- arch/powerpc/kernel/asm-offsets.c | 2 +- arch/powerpc/kernel/entry_32.S | 6 +++--- arch/powerpc/kernel/process.c | 12 ++++++++---- 4 files changed, 16 insertions(+), 10 deletions(-)
arch/powerpc/include/asm/ptrace.h +4 -2
@@ -97,8 +97,6 @@ struct pt_regs#endif -#define STACK_FRAME_WITH_PT_REGS (STACK_FRAME_OVERHEAD + sizeof(struct pt_regs)) -// Always displays as "REGS" in memory dumps #ifdef CONFIG_CPU_BIG_ENDIAN #define STACK_FRAME_REGS_MARKER ASM_CONST(0x52454753)
@@ -125,6 +123,8 @@ struct pt_regs#define STACK_USER_INT_FRAME_SIZE (sizeof(struct pt_regs) + STACK_FRAME_OVERHEAD)#define STACK_INT_FRAME_REGS STACK_FRAME_OVERHEAD#define STACK_INT_FRAME_MARKER (STACK_FRAME_OVERHEAD - 16) +#define STACK_SWITCH_FRAME_SIZE (sizeof(struct pt_regs) + STACK_FRAME_OVERHEAD) +#define STACK_SWITCH_FRAME_REGS STACK_FRAME_OVERHEAD#ifdef CONFIG_PPC64_ELF_ABI_V2 #define STACK_FRAME_MIN_SIZE 32
@@ -146,6 +146,8 @@ struct pt_regs#define STACK_INT_FRAME_REGS STACK_FRAME_OVERHEAD#define STACK_INT_FRAME_MARKER (STACK_FRAME_OVERHEAD - 8)#define STACK_FRAME_MIN_SIZE STACK_FRAME_OVERHEAD +#define STACK_SWITCH_FRAME_SIZE (sizeof(struct pt_regs) + STACK_FRAME_OVERHEAD) +#define STACK_SWITCH_FRAME_REGS STACK_FRAME_OVERHEAD/* Size of stack frame allocated when calling signal handler. */ #define __SIGNAL_FRAMESIZE 64
arch/powerpc/kernel/asm-offsets.c +1 -1
@@ -260,7 +260,7 @@ int main(void)/* Interrupt register frame */ DEFINE (INT_FRAME_SIZE ,STACK_INT_FRAME_SIZE ); -DEFINE (SWITCH_FRAME_SIZE ,STACK_FRAME_WITH_PT_REGS ); +DEFINE (SWITCH_FRAME_SIZE ,STACK_SWITCH_FRAME_SIZE );STACK_PT_REGS_OFFSET (GPR0 , gpr[0 ]);STACK_PT_REGS_OFFSET (GPR1 , gpr[1 ]);STACK_PT_REGS_OFFSET (GPR2 , gpr[2 ]);
arch/powerpc/kernel/entry_32.S +3 -3
@@ -215,9 +215,9 @@ ret_from_kernel_thread: * in arch/ppc/kernel/process.c */ _GLOBAL(_switch) - stwu r1,-INT_FRAME_SIZE(r1) + stwu r1,-SWITCH_FRAME_SIZE(r1) mflr r0 - stw r0,INT_FRAME_SIZE+4(r1) + stw r0,SWITCH_FRAME_SIZE+4(r1) /* r3-r12 are caller saved -- Cort */ SAVE_NVGPRS(r1) stw r0,_NIP(r1) /* Return to switch caller */
@@ -248,7 +248,7 @@ _GLOBAL(_switch) lwz r4,_NIP(r1) /* Return to _switch caller in new task */ mtlr r4 - addi r1,r1,INT_FRAME_SIZE + addi r1,r1,SWITCH_FRAME_SIZE blr .globl fast_exception_return
arch/powerpc/kernel/process.c +8 -4
@@ -1779,10 +1779,10 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args)* do some house keeping and then return from the fork or clone * system call, using the stack frame created above. */ - sp -=sizeof (struct pt_regs ); - kregs = (struct pt_regs *) sp; - sp -=STACK_FRAME_OVERHEAD ; + sp -=STACK_SWITCH_FRAME_SIZE ; + kregs = (struct pt_regs *)(sp +STACK_SWITCH_FRAME_REGS ); p->thread.ksp = sp; +#ifdef CONFIG_HAVE_HW_BREAKPOINT for (i =0 ; i <nr_wp_slots (); i++) p->thread.ptrace_bps[i] =NULL ;
@@ -2232,8 +2232,12 @@ void __no_sanitize_address show_stack(struct task_struct *tsk,/* * See if this is an exception frame. * We look for the "regs" marker in the current frame. +* +* STACK_SWITCH_FRAME_SIZE being the smallest frame that +* could hold a pt_regs, if that does not fit then it can't +* have regs. */ -if (validate_sp (sp, tsk,STACK_FRAME_WITH_PT_REGS ) +if (validate_sp (sp, tsk,STACK_SWITCH_FRAME_SIZE ) && stack[STACK_INT_FRAME_MARKER_LONGS ] ==STACK_FRAME_REGS_MARKER ) {struct pt_regs *regs = (struct pt_regs *) (sp +STACK_INT_FRAME_REGS );
Signature · 1 line
2.37.2
Nicholas Piggin
reviewerPATCH 12/17 powerpc: copy_thread fill in interrupt frame marker and back chain
In reply to Nicholas PigginBacktraces will not recognise the fork system call interrupt without the regs marker. And regular interrupt entry from userspace creates the back chain to the user stack, so do this for the initial fork frame too, to be consistent.
--- arch/powerpc/kernel/process.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
arch/powerpc/kernel/process.c +3 -1
@@ -1728,12 +1728,13 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args)/* Create initial stack frame. */ sp -=STACK_USER_INT_FRAME_SIZE ; - ((unsigned long *)sp)[0 ] =0 ; + *(unsigned long *)(sp +STACK_INT_FRAME_MARKER ) =STACK_FRAME_REGS_MARKER ;/* Copy registers */ childregs = (struct pt_regs *)(sp +STACK_INT_FRAME_REGS );if (unlikely (args->fn)) {/* kernel thread */ + ((unsigned long *)sp)[0 ] =0 ;memset (childregs,0 ,sizeof (struct pt_regs )); childregs->gpr[1 ] = sp +STACK_USER_INT_FRAME_SIZE ;/* function */
@@ -1753,6 +1754,7 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args) *childregs = *regs;if (usp) childregs->gpr[1 ] = usp; + ((unsigned long *)sp)[0 ] = childregs->gpr[1 ]; p->thread.regs = childregs;/* 64s sets this in ret_from_fork */ if (!IS_ENABLED (CONFIG_PPC_BOOK3S_64 ))
Signature · 1 line
2.37.2
Nicholas Piggin
reviewerPATCH 13/17 powerpc: copy_thread add a back chain to the switch stack frame
In reply to Nicholas PigginStack unwinders need LR and the back chain as a minimum. The switch stack uses regs->nip for its return pointer rather than lrsave, so that was not set in the fork frame, and neither was the back chain. This change sets those fields in the stack.
With this and the previous change, a stack trace in the switch or interrupt stack goes from looking like this:
Oops: Exception in kernel mode, sig: 5 [#1] LE PAGE_SIZE=64K MMU=Radix SMP NR_CPUS=2048 NUMA pSeries Modules linked in: CPU: 3 PID: 90 Comm: systemd Not tainted NIP: c000000000011060 LR: c000000000010f68 CTR: 0000000000007fff [ ... regs ... ] NIP [c000000000011060] _switch+0x160/0x17c LR [c000000000010f68] _switch+0x68/0x17c Call Trace:
To this:
Oops: Exception in kernel mode, sig: 5 [#1] LE PAGE_SIZE=64K MMU=Radix SMP NR_CPUS=2048 NUMA pSeries CPU: 0 PID: 93 Comm: systemd Not tainted NIP: c000000000011060 LR: c000000000010f68 CTR: 0000000000007fff [ ... regs ... ] NIP [c000000000011060] _switch+0x160/0x17c LR [c000000000010f68] _switch+0x68/0x17c Call Trace: [c000000005a93e10] [c00000000000cdbc] ret_from_fork_scv+0x0/0x54 --- interrupt: 3000 at 0x7fffa72f56d8 NIP: 00007fffa72f56d8 LR: 0000000000000000 CTR: 0000000000000000 [ ... regs ... ] NIP [00007fffa72f56d8] 0x7fffa72f56d8 LR [0000000000000000] 0x0 --- interrupt: 3000
--- arch/powerpc/kernel/process.c | 2 ++ 1 file changed, 2 insertions(+)
arch/powerpc/kernel/process.c +2 -0
@@ -1781,7 +1781,9 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args)* do some house keeping and then return from the fork or clone * system call, using the stack frame created above. */ + ((unsigned long *)sp)[STACK_FRAME_LR_SAVE ] = (unsigned long )f; sp -=STACK_SWITCH_FRAME_SIZE ; + ((unsigned long *)sp)[0 ] = sp +STACK_SWITCH_FRAME_SIZE ; kregs = (struct pt_regs *)(sp +STACK_SWITCH_FRAME_REGS ); p->thread.ksp = sp;
Signature · 1 line
2.37.2
Nicholas Piggin
reviewerPATCH 14/17 powerpc: split validate_sp into two functions
In reply to Nicholas PigginMost callers just want to validate an arbitrary kernel stack pointer, some need a particular size. Make the size case the exceptional one with an extra function.
--- arch/powerpc/include/asm/processor.h | 15 ++++++++++++--- arch/powerpc/kernel/process.c | 23 ++++++++++++++--------- arch/powerpc/kernel/stacktrace.c | 2 +- arch/powerpc/perf/callchain.c | 6 +++--- 4 files changed, 30 insertions(+), 16 deletions(-)
arch/powerpc/include/asm/processor.h +12 -3
@@ -374,9 +374,18 @@ static inline unsigned long __pack_fe01(unsigned int fpmode)#endif -/* Check that a certain kernel stack pointer is valid in task_struct p */ -int validate_sp (unsigned long sp,struct task_struct *p, -unsigned long nbytes); +/* +* Check that a certain kernel stack pointer is a valid (minimum sized) +* stack frame in task_struct p. +*/ +int validate_sp (unsigned long sp,struct task_struct *p); + +/* +* validate the stack frame of a particular minimum size, used for when we are +* looking at a certain object in the stack beyond the minimum. +*/ +int validate_sp_size (unsigned long sp,struct task_struct *p, +unsigned long nbytes);/* * Prefetch macros.
arch/powerpc/kernel/process.c +14 -9
@@ -2128,9 +2128,12 @@ static inline int valid_emergency_stack(unsigned long sp, struct task_struct *p,return 0 ; } - -int validate_sp (unsigned long sp,struct task_struct *p, -unsigned long nbytes) +/* +* validate the stack frame of a particular minimum size, used for when we are +* looking at a certain object in the stack beyond the minimum. +*/ +int validate_sp_size (unsigned long sp,struct task_struct *p, +unsigned long nbytes) {unsigned long stack_page = (unsigned long )task_stack_page (p);
@@ -2146,7 +2149,10 @@ int validate_sp(unsigned long sp, struct task_struct *p,return valid_emergency_stack (sp, p, nbytes); } -EXPORT_SYMBOL (validate_sp); +int validate_sp (unsigned long sp,struct task_struct *p) +{ +return validate_sp_size (sp, p,STACK_FRAME_OVERHEAD ); +}static unsigned long ___get_wchan (struct task_struct *p) {
@@ -2154,13 +2160,12 @@ static unsigned long ___get_wchan(struct task_struct *p)int count =0 ; sp = p->thread.ksp; -if (!validate_sp (sp, p,STACK_FRAME_OVERHEAD )) +if (!validate_sp (sp, p))return 0 ; do { sp = READ_ONCE_NOCHECK(*(unsigned long *)sp); - if (!validate_sp(sp, p, STACK_FRAME_OVERHEAD) || - task_is_running(p)) + if (!validate_sp(sp, p) || task_is_running(p)) return 0; if (count > 0) { ip = READ_ONCE_NOCHECK(((unsigned long *)sp)[STACK_FRAME_LR_SAVE]);
@@ -2214,7 +2219,7 @@ void __no_sanitize_address show_stack(struct task_struct *tsk, lr =0 ;printk ("%sCall Trace:\n" , loglvl); do { - if (!validate_sp(sp, tsk, STACK_FRAME_OVERHEAD)) + if (!validate_sp(sp, tsk)) break; stack = (unsigned long *) sp;
@@ -2241,7 +2246,7 @@ void __no_sanitize_address show_stack(struct task_struct *tsk,* could hold a pt_regs, if that does not fit then it can't * have regs. */ -if (validate_sp (sp, tsk,STACK_SWITCH_FRAME_SIZE ) +if (validate_sp_size (sp, tsk,STACK_SWITCH_FRAME_SIZE ) && stack[STACK_INT_FRAME_MARKER_LONGS ] ==STACK_FRAME_REGS_MARKER ) {struct pt_regs *regs = (struct pt_regs *) (sp +STACK_INT_FRAME_REGS );
arch/powerpc/kernel/stacktrace.c +1 -1
@@ -43,7 +43,7 @@ void __no_sanitize_address arch_stack_walk(stack_trace_consume_fn consume_entry,unsigned long *stack = (unsigned long *) sp;unsigned long newsp, ip; -if (!validate_sp (sp, task,STACK_FRAME_OVERHEAD )) +if (!validate_sp (sp, task))return ; newsp = stack[0 ];
arch/powerpc/perf/callchain.c +3 -3
@@ -27,7 +27,7 @@ static int valid_next_sp(unsigned long sp, unsigned long prev_sp) {if (sp &0xf )return 0 ;/* must be 16-byte aligned */ -if (!validate_sp (sp, current,STACK_FRAME_OVERHEAD )) +if (!validate_sp (sp, current))return 0 ;if (sp >= prev_sp +STACK_FRAME_MIN_SIZE )return 1 ;
@@ -53,7 +53,7 @@ perf_callchain_kernel(struct perf_callchain_entry_ctx *entry, struct pt_regs *re sp = regs->gpr[1 ];perf_callchain_store (entry,perf_instruction_pointer (regs)); -if (!validate_sp (sp, current,STACK_FRAME_OVERHEAD )) +if (!validate_sp (sp, current))return ;for (;;) {
@@ -61,7 +61,7 @@ perf_callchain_kernel(struct perf_callchain_entry_ctx *entry, struct pt_regs *re next_sp = fp[0 ];if (next_sp == sp +STACK_INT_FRAME_SIZE && -validate_sp (sp, current,STACK_INT_FRAME_SIZE ) && +validate_sp_size (sp, current,STACK_INT_FRAME_SIZE ) && fp[STACK_INT_FRAME_MARKER_LONGS ] ==STACK_FRAME_REGS_MARKER ) {/* * This looks like an interrupt frame for an
Signature · 1 line
2.37.2
Nicholas Piggin
reviewerPATCH 15/17 powerpc: allow minimum sized kernel stack frames
In reply to Nicholas PigginThis affects only 64-bit ELFv2 kernels, and reduces the minimum asm-created stack frame size from 112 to 32 byte on those kernels.
--- arch/powerpc/kernel/head_40x.S | 2 +- arch/powerpc/kernel/head_44x.S | 6 +++--- arch/powerpc/kernel/head_64.S | 6 +++--- arch/powerpc/kernel/head_85xx.S | 4 ++-- arch/powerpc/kernel/head_8xx.S | 2 +- arch/powerpc/kernel/head_book3s_32.S | 4 ++-- arch/powerpc/kernel/irq.c | 4 ++-- arch/powerpc/kernel/misc_32.S | 2 +- arch/powerpc/kernel/misc_64.S | 4 ++-- arch/powerpc/kernel/process.c | 2 +- arch/powerpc/kernel/smp.c | 2 +- arch/powerpc/kernel/stacktrace.c | 2 +- 12 files changed, 20 insertions(+), 20 deletions(-)
arch/powerpc/kernel/head_40x.S +1 -1
@@ -602,7 +602,7 @@ start_here: lis r1,init_thread_union@ha addi r1,r1,init_thread_union@l li r0,0 - stwu r0,THREAD_SIZE-STACK_FRAME_OVERHEAD(r1) + stwu r0,THREAD_SIZE-STACK_FRAME_MIN_SIZE(r1) bl early_init /* We have to do this with MMU on */
arch/powerpc/kernel/head_44x.S +3 -3
@@ -109,7 +109,7 @@ _GLOBAL(_start); lis r1,init_thread_union@h ori r1,r1,init_thread_union@l li r0,0 - stwu r0,THREAD_SIZE-STACK_FRAME_OVERHEAD(r1) + stwu r0,THREAD_SIZE-STACK_FRAME_MIN_SIZE(r1) bl early_init
@@ -1012,7 +1012,7 @@ _GLOBAL(start_secondary_47x) */ lis r1,temp_boot_stack@h ori r1,r1,temp_boot_stack@l - addi r1,r1,1024-STACK_FRAME_OVERHEAD + addi r1,r1,1024-STACK_FRAME_MIN_SIZE li r0,0 stw r0,0(r1) bl mmu_init_secondary
@@ -1025,7 +1025,7 @@ _GLOBAL(start_secondary_47x) lwz r1,TASK_STACK(r2) /* Current stack pointer */ - addi r1,r1,THREAD_SIZE-STACK_FRAME_OVERHEAD + addi r1,r1,THREAD_SIZE-STACK_FRAME_MIN_SIZE li r0,0 stw r0,0(r1)
arch/powerpc/kernel/head_64.S +3 -3
@@ -424,7 +424,7 @@ generic_secondary_common_init: /* Create a temp kernel stack for use before relocation is on. */ ld r1,PACAEMERGSP(r13) - subi r1,r1,STACK_FRAME_OVERHEAD + subi r1,r1,STACK_FRAME_MIN_SIZE /* See if we need to call a cpu state restore handler */ LOAD_REG_ADDR(r23, cur_cpu_spec)
@@ -780,7 +780,7 @@ _GLOBAL(pmac_secondary_start) /* Create a temp kernel stack for use before relocation is on. */ ld r1,PACAEMERGSP(r13) - subi r1,r1,STACK_FRAME_OVERHEAD + subi r1,r1,STACK_FRAME_MIN_SIZE b __secondary_start
@@ -958,7 +958,7 @@ start_here_multiplatform: LOAD_REG_IMMEDIATE(r1,THREAD_SIZE) add r1,r3,r1 li r0,0 - stdu r0,-STACK_FRAME_OVERHEAD(r1) + stdu r0,-STACK_FRAME_MIN_SIZE(r1) /* * Do very early kernel initializations, including initial hash table
arch/powerpc/kernel/head_85xx.S +2 -2
@@ -229,7 +229,7 @@ set_ivor: lis r1,init_thread_union@h ori r1,r1,init_thread_union@l li r0,0 - stwu r0,THREAD_SIZE-STACK_FRAME_OVERHEAD(r1) + stwu r0,THREAD_SIZE-STACK_FRAME_MIN_SIZE(r1) #ifdef CONFIG_SMP stw r24, TASK_CPU(r2)
@@ -1044,7 +1044,7 @@ __secondary_start: lwz r1,TASK_STACK(r2) /* stack */ - addi r1,r1,THREAD_SIZE-STACK_FRAME_OVERHEAD + addi r1,r1,THREAD_SIZE-STACK_FRAME_MIN_SIZE li r0,0 stw r0,0(r1)
arch/powerpc/kernel/head_8xx.S +1 -1
@@ -537,7 +537,7 @@ start_here: ori r0, r0, STACK_END_MAGIC@l stw r0, 0(r1) li r0,0 - stwu r0,THREAD_SIZE-STACK_FRAME_OVERHEAD(r1) + stwu r0,THREAD_SIZE-STACK_FRAME_MIN_SIZE(r1) lis r6, swapper_pg_dir@ha tophys(r6,r6)
arch/powerpc/kernel/head_book3s_32.S +2 -2
@@ -840,7 +840,7 @@ __secondary_start: lwz r1,TASK_STACK(r1) /* stack */ - addi r1,r1,THREAD_SIZE-STACK_FRAME_OVERHEAD + addi r1,r1,THREAD_SIZE-STACK_FRAME_MIN_SIZE li r0,0 tophys(r3,r1) stw r0,0(r3)
@@ -966,7 +966,7 @@ END_MMU_FTR_SECTION_IFCLR(MMU_FTR_HPTE_TABLE) lis r1,init_thread_union@ha addi r1,r1,init_thread_union@l li r0,0 - stwu r0,THREAD_SIZE-STACK_FRAME_OVERHEAD(r1) + stwu r0,THREAD_SIZE-STACK_FRAME_MIN_SIZE(r1) /* * Do early platform-specific initialization, * and set up the MMU.
arch/powerpc/kernel/irq.c +2 -2
@@ -210,7 +210,7 @@ static __always_inline void call_do_softirq(const void *sp)PPC_LL " %%r1, 0(%%r1) ;" : // Outputs : // Inputs - [sp] "b" (sp), [offset] "i" (THREAD_SIZE - STACK_FRAME_OVERHEAD), + [sp] "b" (sp), [offset] "i" (THREAD_SIZE - STACK_FRAME_MIN_SIZE), [callee] "i" (__do_softirq) : // Clobbers "lr", "xer", "ctr", "memory", "cr0", "cr1", "cr5", "cr6",
@@ -264,7 +264,7 @@ static __always_inline void call_do_irq(struct pt_regs *regs, void *sp) : // Outputs "+r" (r3) : // Inputs - [sp] "b" (sp), [offset] "i" (THREAD_SIZE - STACK_FRAME_OVERHEAD), + [sp] "b" (sp), [offset] "i" (THREAD_SIZE - STACK_FRAME_MIN_SIZE), [callee] "i" (__do_irq) : // Clobbers "lr", "xer", "ctr", "memory", "cr0", "cr1", "cr5", "cr6",
arch/powerpc/kernel/misc_32.S +1 -1
@@ -382,7 +382,7 @@ EXPORT_SYMBOL(__bswapdi2) _GLOBAL(start_secondary_resume) /* Reset stack */ rlwinm r1, r1, 0, 0, 31 - THREAD_SHIFT - addi r1,r1,THREAD_SIZE-STACK_FRAME_OVERHEAD + addi r1,r1,THREAD_SIZE-STACK_FRAME_MIN_SIZE li r3,0 stw r3,0(r1) /* Zero the stack frame pointer */ bl start_secondary
arch/powerpc/kernel/misc_64.S +2 -2
@@ -384,7 +384,7 @@ _GLOBAL(kexec_sequence) std r0,16(r1) /* switch stacks to newstack -- &kexec_stack.stack */ - stdu r1,THREAD_SIZE-STACK_FRAME_OVERHEAD(r3) + stdu r1,THREAD_SIZE-STACK_FRAME_MIN_SIZE(r3) mr r1,r3 li r0,0
@@ -401,7 +401,7 @@ _GLOBAL(kexec_sequence) std r26,-48(r1) std r25,-56(r1) - stdu r1,-STACK_FRAME_OVERHEAD-64(r1) + stdu r1,-STACK_FRAME_MIN_SIZE-64(r1) /* save args into preserved regs */ mr r31,r3 /* newstack (both) */
arch/powerpc/kernel/process.c +1 -1
@@ -2151,7 +2151,7 @@ int validate_sp_size(unsigned long sp, struct task_struct *p,int validate_sp (unsigned long sp,struct task_struct *p) { -return validate_sp_size (sp, p,STACK_FRAME_OVERHEAD ); +return validate_sp_size (sp, p,STACK_FRAME_MIN_SIZE ); }static unsigned long ___get_wchan (struct task_struct *p)
arch/powerpc/kernel/smp.c +1 -1
@@ -1249,7 +1249,7 @@ static void cpu_idle_thread_init(unsigned int cpu, struct task_struct *idle)#ifdef CONFIG_PPC64 paca_ptrs[cpu]->__current = idle; paca_ptrs[cpu]->kstack = (unsigned long )task_stack_page (idle) + -THREAD_SIZE -STACK_FRAME_OVERHEAD ; +THREAD_SIZE -STACK_FRAME_MIN_SIZE ;#endif task_thread_info (idle)->cpu = cpu; secondary_current = current_set[cpu] = idle;
arch/powerpc/kernel/stacktrace.c +1 -1
@@ -94,7 +94,7 @@ int __no_sanitize_address arch_stack_walk_reliable(stack_trace_consume_fn consum* idle tasks have a custom stack layout, * c.f. cpu_idle_thread_init(). */ - stack_end -=STACK_FRAME_OVERHEAD ; + stack_end -=STACK_FRAME_MIN_SIZE ; }if (task == current)
Signature · 1 line
2.37.2
Nicholas Piggin
reviewerPATCH 16/17 powerpc/64: ELFv2 use minimal stack frames in int and switch frame sizes
In reply to Nicholas PigginAdjust the ELFv2 interrupt and switch frames to the minimum C ABI size, plus pt_regs, plus 16 bytes for the aligned regs marker for the int frame (and the switch frame needs to match that because it uses the same regs offset as the int frame).
This saves 80 bytes of kernel stack per interrupt. It's the principle of getting our accounting right that's more important than the practical saving.
--- arch/powerpc/include/asm/ptrace.h | 21 +++++++++++++++------ 1 file changed, 15 insertions(+), 6 deletions(-)
arch/powerpc/include/asm/ptrace.h +15 -5
@@ -120,16 +119,26 @@ struct pt_regs#define STACK_FRAME_OVERHEAD 112/* size of minimum stack frame */ #define STACK_FRAME_LR_SAVE 2/* Location of LR in stack frame */ + +#ifdef CONFIG_PPC64_ELF_ABI_V2 +#define STACK_FRAME_MIN_SIZE 32 +#define STACK_USER_INT_FRAME_SIZE (sizeof(struct pt_regs) + STACK_FRAME_MIN_SIZE + 16) +#define STACK_INT_FRAME_REGS (STACK_FRAME_MIN_SIZE + 16) +#define STACK_INT_FRAME_MARKER STACK_FRAME_MIN_SIZE +#define STACK_SWITCH_FRAME_SIZE (sizeof(struct pt_regs) + STACK_FRAME_MIN_SIZE + 16) +#define STACK_SWITCH_FRAME_REGS (STACK_FRAME_MIN_SIZE + 16) +#else +/* +* The ELFv1 ABI specifies 48 bytes plus a minimum 64 byte parameter save +* area. This parameter area is not used by calls to C from interrupt entry, +* so the second from last one of those is used for the frame marker. +*/ +#define STACK_FRAME_MIN_SIZE 112#define STACK_USER_INT_FRAME_SIZE (sizeof(struct pt_regs) + STACK_FRAME_OVERHEAD)#define STACK_INT_FRAME_REGS STACK_FRAME_OVERHEAD#define STACK_INT_FRAME_MARKER (STACK_FRAME_OVERHEAD - 16)#define STACK_SWITCH_FRAME_SIZE (sizeof(struct pt_regs) + STACK_FRAME_OVERHEAD)#define STACK_SWITCH_FRAME_REGS STACK_FRAME_OVERHEAD - -#ifdef CONFIG_PPC64_ELF_ABI_V2 -#define STACK_FRAME_MIN_SIZE 32 -#else -#define STACK_FRAME_MIN_SIZE STACK_FRAME_OVERHEAD#endif /* Size of dummy stack frame allocated when calling signal handler. */
Signature · 1 line
2.37.2
Nicholas Piggin
reviewerPATCH 17/17 powerpc: remove STACK_FRAME_OVERHEAD
In reply to Nicholas PigginThis is equal to STACK_FRAME_MIN_SIZE on 32-bit and 64-bit ELFv1, and no longer used in 64-bit ELFv2, so replace STACK_FRAME_OVERHEAD occurrences with STACK_FRAME_MIN_SIZE.
--- arch/powerpc/include/asm/ptrace.h | 24 +++++++++++------------- 1 file changed, 11 insertions(+), 13 deletions(-)
arch/powerpc/include/asm/ptrace.h +11 -13
@@ -117,7 +117,6 @@ struct pt_regs#define USER_REDZONE_SIZE 512#define KERNEL_REDZONE_SIZE 288 -#define STACK_FRAME_OVERHEAD 112/* size of minimum stack frame */ #define STACK_FRAME_LR_SAVE 2/* Location of LR in stack frame */ #ifdef CONFIG_PPC64_ELF_ABI_V2
@@ -134,11 +133,11 @@ struct pt_regs* so the second from last one of those is used for the frame marker. */ #define STACK_FRAME_MIN_SIZE 112 -#define STACK_USER_INT_FRAME_SIZE (sizeof(struct pt_regs) + STACK_FRAME_OVERHEAD) -#define STACK_INT_FRAME_REGS STACK_FRAME_OVERHEAD -#define STACK_INT_FRAME_MARKER (STACK_FRAME_OVERHEAD - 16) -#define STACK_SWITCH_FRAME_SIZE (sizeof(struct pt_regs) + STACK_FRAME_OVERHEAD) -#define STACK_SWITCH_FRAME_REGS STACK_FRAME_OVERHEAD +#define STACK_USER_INT_FRAME_SIZE (sizeof(struct pt_regs) + STACK_FRAME_MIN_SIZE) +#define STACK_INT_FRAME_REGS STACK_FRAME_MIN_SIZE +#define STACK_INT_FRAME_MARKER (STACK_FRAME_MIN_SIZE - 16) +#define STACK_SWITCH_FRAME_SIZE (sizeof(struct pt_regs) + STACK_FRAME_MIN_SIZE) +#define STACK_SWITCH_FRAME_REGS STACK_FRAME_MIN_SIZE#endif /* Size of dummy stack frame allocated when calling signal handler. */
@@ -149,14 +148,13 @@ struct pt_regs#define USER_REDZONE_SIZE 0#define KERNEL_REDZONE_SIZE 0 -#define STACK_FRAME_OVERHEAD 16/* size of minimum stack frame */ +#define STACK_FRAME_MIN_SIZE 16#define STACK_FRAME_LR_SAVE 1/* Location of LR in stack frame */ -#define STACK_USER_INT_FRAME_SIZE (sizeof(struct pt_regs) + STACK_FRAME_OVERHEAD) -#define STACK_INT_FRAME_REGS STACK_FRAME_OVERHEAD -#define STACK_INT_FRAME_MARKER (STACK_FRAME_OVERHEAD - 8) -#define STACK_FRAME_MIN_SIZE STACK_FRAME_OVERHEAD -#define STACK_SWITCH_FRAME_SIZE (sizeof(struct pt_regs) + STACK_FRAME_OVERHEAD) -#define STACK_SWITCH_FRAME_REGS STACK_FRAME_OVERHEAD +#define STACK_USER_INT_FRAME_SIZE (sizeof(struct pt_regs) + STACK_FRAME_MIN_SIZE) +#define STACK_INT_FRAME_REGS STACK_FRAME_MIN_SIZE +#define STACK_INT_FRAME_MARKER (STACK_FRAME_MIN_SIZE - 8) +#define STACK_SWITCH_FRAME_SIZE (sizeof(struct pt_regs) + STACK_FRAME_MIN_SIZE) +#define STACK_SWITCH_FRAME_REGS STACK_FRAME_MIN_SIZE/* Size of stack frame allocated when calling signal handler. */ #define __SIGNAL_FRAMESIZE 64
Signature · 1 line
2.37.2
Michael Ellerman
AppliedQuoted 9 lines · Nicholas Piggin
Since RFC:
- Fix a compile bug.
- Fix BookE KVM properly. Hopefully -- I don't have a BookE KVM environment to test. Can QEMU do it? Is it still tested?
- Drop the last two patches that changed the stack layout, they can be done later.
- Drop the load/store-multiple change to 32-bit.[...]
Applied to powerpc/next.
[01/17] KVM: PPC: Book3E: Fix CONFIG_TRACE_IRQFLAGS support
https://git.kernel.org/powerpc/c/6c645b01e536757a9e1a9f72c13767f9b3f8559f [02/17] powerpc/64: Remove asm interrupt tracing call helpers https://git.kernel.org/powerpc/c/d6aee468e4ecbfec46a3eafae4d31d6efc0d4da4 [03/17] powerpc/perf: callchain validate kernel stack pointer bounds https://git.kernel.org/powerpc/c/32c5209214bd8d4f8c4e9d9b630ef4c671f58e79 [04/17] powerpc: Rearrange copy_thread child stack creation https://git.kernel.org/powerpc/c/bc0677363d0ffaec0c56685291e97b080116976c [05/17] powerpc/pseries: hvcall stack frame overhead https://git.kernel.org/powerpc/c/baa49d81a94bb4170e7f2f4d97016772117d0f60 [06/17] powerpc: simplify ppc_save_regs https://git.kernel.org/powerpc/c/37195b820d32c23bdefce3f460ed7de48a57e5e4 [07/17] powerpc: add definition for pt_regs offset within an interrupt frame https://git.kernel.org/powerpc/c/c03be0a3f3cc656eab5c427b78959b8f1b169a11 [08/17] powerpc: add a definition for the marker offset within the interrupt frame https://git.kernel.org/powerpc/c/d2e8ff9f1492f44c5a6d93f759eea27574d753de [09/17] powerpc: Rename STACK_FRAME_MARKER and derive it from frame offset https://git.kernel.org/powerpc/c/e856e336924b0ecd0b7058e65e6b3e7266ee0b95 [10/17] powerpc: add a define for the user interrupt frame size https://git.kernel.org/powerpc/c/1223e5a20f7fb3c31c91a328d1a04ed26d5e889b [11/17] powerpc: add a define for the switch frame size and regs offset https://git.kernel.org/powerpc/c/6f291a03819e4051ebc870471d26915ef2e6ba31 [12/17] powerpc: copy_thread fill in interrupt frame marker and back chain https://git.kernel.org/powerpc/c/6895dfc0474170c492191c126fcfc420f7771a09 [13/17] powerpc: copy_thread add a back chain to the switch stack frame https://git.kernel.org/powerpc/c/edbd0387f3249cc7e102f86d4852a9a9f3bb1305 [14/17] powerpc: split validate_sp into two functions https://git.kernel.org/powerpc/c/4cefb0f6c555971b3e6544a9b15470f9d1f12089 [15/17] powerpc: allow minimum sized kernel stack frames https://git.kernel.org/powerpc/c/90f1b43196c5e79f6c986a359011a19857984c27 [16/17] powerpc/64: ELFv2 use minimal stack frames in int and switch frame sizes https://git.kernel.org/powerpc/c/cd52414d5a6ccea6ce956ef05161fe824522a107 [17/17] powerpc: remove STACK_FRAME_OVERHEAD https://git.kernel.org/powerpc/c/dfecd06bc5524517ed7737c30eaaf747338b280a
cheers