powerpc/64: syscalls in C
Subsystems (2)
Nicholas Piggin
reviewerv2 cover powerpc/64: syscalls in C
Accounted for some feedback.
Nicholas Piggin (4):
powerpc: convert to copy_thread_tls
powerpc/64: remove support for kernel-mode syscalls powerpc/64: system call remove non-volatile GPR save optimisation powerpc/64: system call implement the bulk of the logic in C
arch/powerpc/Kconfig | 1 + arch/powerpc/include/asm/asm-prototypes.h | 11 - .../powerpc/include/asm/book3s/64/kup-radix.h | 12 +- arch/powerpc/include/asm/cputime.h | 22 + arch/powerpc/include/asm/ptrace.h | 3 + arch/powerpc/include/asm/signal.h | 2 + arch/powerpc/include/asm/switch_to.h | 5 + arch/powerpc/include/asm/time.h | 3 + arch/powerpc/kernel/Makefile | 3 +- arch/powerpc/kernel/entry_64.S | 421 +++--------------- arch/powerpc/kernel/exceptions-64s.S | 2 - arch/powerpc/kernel/process.c | 9 +- arch/powerpc/kernel/signal.h | 2 - arch/powerpc/kernel/syscall_64.c | 177 ++++++++ arch/powerpc/kernel/syscalls/syscall.tbl | 22 +- 15 files changed, 307 insertions(+), 388 deletions(-) create mode 100644 arch/powerpc/kernel/syscall_64.c
Signature · 1 line
2.22.0
Nicholas Piggin
reviewerPATCH v2 1/4 powerpc: convert to copy_thread_tls
Commit 3033f14ab78c3 ("clone: support passing tls argument via C rather than pt_regs magic") introduced the HAVE_COPY_THREAD_TLS option. Use it to avoid a subtle assumption about the argument ordering of clone type syscalls.
---
No change since v1.
arch/powerpc/Kconfig | 1 + arch/powerpc/kernel/process.c | 9 +++++---- 2 files changed, 6 insertions(+), 4 deletions(-)
arch/powerpc/Kconfig +1 -0
@@ -182,6 +182,7 @@ config PPC select HAVE_STACKPROTECTOR if PPC64 && $(cc-option,-mstack-protector-guard=tls -mstack-protector-guard-reg=r13) select HAVE_STACKPROTECTOR if PPC32 && $(cc-option,-mstack-protector-guard=tls -mstack-protector-guard-reg=r2) select HAVE_CONTEXT_TRACKING if PPC64 + select HAVE_COPY_THREAD_TLS select HAVE_DEBUG_KMEMLEAK select HAVE_DEBUG_STACKOVERFLOW select HAVE_DYNAMIC_FTRACE
arch/powerpc/kernel/process.c +5 -4
@@ -1600,8 +1600,9 @@ static void setup_ksp_vsid(struct task_struct *p, unsigned long sp)/* * Copy architecture-specific thread state */ -int copy_thread (unsigned long clone_flags,unsigned long usp, -unsigned long kthread_arg,struct task_struct *p) +int copy_thread_tls (unsigned long clone_flags,unsigned long usp, +unsigned long kthread_arg,struct task_struct *p, +unsigned long tls) {struct pt_regs *childregs, *kregs;extern void ret_from_fork (void );
@@ -1642,10 +1643,10 @@ int copy_thread(unsigned long clone_flags, unsigned long usp,if (clone_flags &CLONE_SETTLS ) {#ifdef CONFIG_PPC64 if (!is_32bit_task ()) - childregs->gpr[13 ] = childregs->gpr[6 ]; + childregs->gpr[13 ] = tls; else#endif - childregs->gpr[2 ] = childregs->gpr[6 ]; + childregs->gpr[2 ] = tls; } f = ret_from_fork;
Signature · 1 line
2.22.0
Nicholas Piggin
reviewerPATCH v2 2/4 powerpc/64: remove support for kernel-mode syscalls
In reply to Nicholas PigginThere is support for the kernel to execute the 'sc 0' instruction and make a system call to itself. This is a relic that is unused in the tree, therefore untested. It's also highly questionable for modules to be doing this.
---
No change since v1.
arch/powerpc/kernel/entry_64.S | 21 ++++++--------------- arch/powerpc/kernel/exceptions-64s.S | 2 -- 2 files changed, 6 insertions(+), 17 deletions(-)
arch/powerpc/kernel/entry_64.S +6 -15
@@ -69,24 +69,20 @@ BEGIN_FTR_SECTION bne .Ltabort_syscall END_FTR_SECTION_IFSET(CPU_FTR_TM) #endif - andi. r10,r12,MSR_PR mr r10,r1 - addi r1,r1,-INT_FRAME_SIZE - beq- 1f ld r1,PACAKSAVE(r13) -1: std r10,0(r1) + std r10,0(r1) std r11,_NIP(r1) std r12,_MSR(r1) std r0,GPR0(r1) std r10,GPR1(r1) - beq 2f /* if from kernel mode */ #ifdef CONFIG_PPC_FSL_BOOK3E START_BTB_FLUSH_SECTION BTB_FLUSH(r10) END_BTB_FLUSH_SECTION #endif ACCOUNT_CPU_USER_ENTRY(r13, r10, r11) -2: std r2,GPR2(r1) + std r2,GPR2(r1) std r3,GPR3(r1) mfcr r2 std r4,GPR4(r1)
@@ -122,14 +118,13 @@ END_BTB_FLUSH_SECTION #if defined(CONFIG_VIRT_CPU_ACCOUNTING_NATIVE) && defined(CONFIG_PPC_SPLPAR) BEGIN_FW_FTR_SECTION - beq 33f - /* if from user, see if there are any DTL entries to process */ + /* see if there are any DTL entries to process */ ld r10,PACALPPACAPTR(r13) /* get ptr to VPA */ ld r11,PACA_DTL_RIDX(r13) /* get log read index */ addi r10,r10,LPPACA_DTLIDX LDX_BE r10,0,r10 /* get log write index */ - cmpd cr1,r11,r10 - beq+ cr1,33f + cmpd r11,r10 + beq+ 33f bl accumulate_stolen_time REST_GPR(0,r1) REST_4GPRS(3,r1)
@@ -203,6 +198,7 @@ system_call: /* label this so stack traces look sane */ mtctr r12 bctrl /* Call handler */ + /* syscall_exit can exit to kernel mode, via ret_from_kernel_thread */ .Lsyscall_exit: std r3,RESULT(r1)
@@ -216,11 +212,6 @@ system_call: /* label this so stack traces look sane */ ld r12, PACA_THREAD_INFO(r13) ld r8,_MSR(r1) -#ifdef CONFIG_PPC_BOOK3S - /* No MSR:RI on BookE */ - andi. r10,r8,MSR_RI - beq- .Lunrecov_restore -#endif /* * This is a few instructions into the actual syscall exit path (which actually
arch/powerpc/kernel/exceptions-64s.S +0 -2
@@ -1521,8 +1521,6 @@ EXC_COMMON(trap_0b_common, 0xb00, unknown_exception) * system call / hypercall (0xc00, 0x4c00) * * The system call exception is invoked with "sc 0" and does not alter HV bit. - * There is support for kernel code to invoke system calls but there are no - * in-tree users. * * The hypercall is invoked with "sc 1" and sets HV=1. *
Signature · 1 line
2.22.0
Nicholas Piggin
reviewerPATCH v2 3/4 powerpc/64: system call remove non-volatile GPR save optimisation
In reply to Nicholas Pigginpowerpc has an optimisation where interrupts avoid saving the non-volatile (or callee saved) registers to the interrupt stack frame if they are not required.
Two problems with this are that an interrupt does not always know whether it will need non-volatiles; and if it does need them, they can only be saved from the entry-scoped asm code (because we don't control what the C compiler does with these registers).
system calls are the most difficult: some system calls always require all registers (e.g., fork, to copy regs into the child). Sometimes registers are only required under certain conditions (e.g., tracing, signal delivery). These cases require ugly logic in the call chains (e.g., ppc_fork), and require a lot of logic to be implemented in asm.
So remove the optimisation for system calls, and always save NVGPRs on entry. Modern high performance CPUs are not so sensitive, because the stores are dense in cache and can be hidden by other expensive work in the syscall path -- the null syscall selftests benchmark on POWER9 is not slowed (124.40ns before and 123.64ns after, i.e., within the noise).
Other interrupts retain the NVGPR optimisation for now.
---
Changes since v1:
- Improve changelog
- Fix clone3 spu table entry (Segher)
arch/powerpc/kernel/entry_64.S | 72 +++++------------------- arch/powerpc/kernel/syscalls/syscall.tbl | 22 +++++--- 2 files changed, 28 insertions(+), 66 deletions(-)
arch/powerpc/kernel/entry_64.S +13 -59
@@ -98,13 +98,14 @@ END_BTB_FLUSH_SECTION std r11,_XER(r1) std r11,_CTR(r1) std r9,GPR13(r1) + SAVE_NVGPRS(r1) mflr r10 /* * This clears CR0.SO (bit 28), which is the error indication on * return from this system call. */ rldimi r2,r11,28,(63-28) - li r11,0xc01 + li r11,0xc00 std r10,_LINK(r1) std r11,_TRAP(r1) std r3,ORIG_GPR3(r1)
@@ -323,7 +324,6 @@ END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR) /* Traced system call support */ .Lsyscall_dotrace: - bl save_nvgprs addi r3,r1,STACK_FRAME_OVERHEAD bl do_syscall_trace_enter
@@ -408,7 +408,6 @@ END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR) mtmsrd r10,1 #endif /* CONFIG_PPC_BOOK3E */ - bl save_nvgprs addi r3,r1,STACK_FRAME_OVERHEAD bl do_syscall_trace_leave b ret_from_except
@@ -442,62 +441,6 @@ END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR) _ASM_NOKPROBE_SYMBOL(system_call_common); _ASM_NOKPROBE_SYMBOL(system_call_exit); -/* Save non-volatile GPRs, if not already saved. */ -_GLOBAL(save_nvgprs) - ld r11,_TRAP(r1) - andi. r0,r11,1 - beqlr- - SAVE_NVGPRS(r1) - clrrdi r0,r11,1 - std r0,_TRAP(r1) - blr -_ASM_NOKPROBE_SYMBOL(save_nvgprs); - - -/* - * The sigsuspend and rt_sigsuspend system calls can call do_signal - * and thus put the process into the stopped state where we might - * want to examine its user state with ptrace. Therefore we need - * to save all the nonvolatile registers (r14 - r31) before calling - * the C code. Similarly, fork, vfork and clone need the full - * register state on the stack so that it can be copied to the child. - */ - -_GLOBAL(ppc_fork) - bl save_nvgprs - bl sys_fork - b .Lsyscall_exit - -_GLOBAL(ppc_vfork) - bl save_nvgprs - bl sys_vfork - b .Lsyscall_exit - -_GLOBAL(ppc_clone) - bl save_nvgprs - bl sys_clone - b .Lsyscall_exit - -_GLOBAL(ppc_clone3) - bl save_nvgprs - bl sys_clone3 - b .Lsyscall_exit - -_GLOBAL(ppc32_swapcontext) - bl save_nvgprs - bl compat_sys_swapcontext - b .Lsyscall_exit - -_GLOBAL(ppc64_swapcontext) - bl save_nvgprs - bl sys_swapcontext - b .Lsyscall_exit - -_GLOBAL(ppc_switch_endian) - bl save_nvgprs - bl sys_switch_endian - b .Lsyscall_exit - _GLOBAL(ret_from_fork) bl schedule_tail REST_NVGPRS(r1)
@@ -516,6 +459,17 @@ _GLOBAL(ret_from_kernel_thread) li r3,0 b .Lsyscall_exit +/* Save non-volatile GPRs, if not already saved. */ +_GLOBAL(save_nvgprs) + ld r11,_TRAP(r1) + andi. r0,r11,1 + beqlr- + SAVE_NVGPRS(r1) + clrrdi r0,r11,1 + std r0,_TRAP(r1) + blr +_ASM_NOKPROBE_SYMBOL(save_nvgprs); + #ifdef CONFIG_PPC_BOOK3S_64 #define FLUSH_COUNT_CACHE \
arch/powerpc/kernel/syscalls/syscall.tbl +15 -7
@@ -9,7 +9,9 @@ # 0 nospu restart_syscall sys_restart_syscall 1 nospu exit sys_exit -2 nospu fork ppc_fork +2 32 fork ppc_fork sys_fork +2 64 fork sys_fork +2 spu fork sys_ni_syscall 3 common read sys_read 4 common write sys_write 5 common open sys_open compat_sys_open
@@ -158,7 +160,9 @@ 119 32 sigreturn sys_sigreturn compat_sys_sigreturn 119 64 sigreturn sys_ni_syscall 119 spu sigreturn sys_ni_syscall -120 nospu clone ppc_clone +120 32 clone ppc_clone sys_clone +120 64 clone sys_clone +120 spu clone sys_ni_syscall 121 common setdomainname sys_setdomainname 122 common uname sys_newuname 123 common modify_ldt sys_ni_syscall
@@ -240,7 +244,9 @@ 186 spu sendfile sys_sendfile64 187 common getpmsg sys_ni_syscall 188 common putpmsg sys_ni_syscall -189 nospu vfork ppc_vfork +189 32 vfork ppc_vfork sys_vfork +189 64 vfork sys_vfork +189 spu vfork sys_ni_syscall 190 common ugetrlimit sys_getrlimit compat_sys_getrlimit 191 common readahead sys_readahead compat_sys_readahead 192 32 mmap2 sys_mmap2 compat_sys_mmap2
@@ -316,8 +322,8 @@ 248 32 clock_nanosleep sys_clock_nanosleep_time32 248 64 clock_nanosleep sys_clock_nanosleep 248 spu clock_nanosleep sys_clock_nanosleep -249 32 swapcontext ppc_swapcontext ppc32_swapcontext -249 64 swapcontext ppc64_swapcontext +249 32 swapcontext ppc_swapcontext compat_sys_swapcontext +249 64 swapcontext sys_swapcontext 249 spu swapcontext sys_ni_syscall 250 common tgkill sys_tgkill 251 32 utimes sys_utimes_time32
@@ -456,7 +462,7 @@ 361 common bpf sys_bpf 362 nospu execveat sys_execveat compat_sys_execveat 363 32 switch_endian sys_ni_syscall -363 64 switch_endian ppc_switch_endian +363 64 switch_endian sys_switch_endian 363 spu switch_endian sys_ni_syscall 364 common userfaultfd sys_userfaultfd 365 common membarrier sys_membarrier
@@ -516,4 +522,6 @@ 432 common fsmount sys_fsmount 433 common fspick sys_fspick 434 common pidfd_open sys_pidfd_open -435 nospu clone3 ppc_clone3 +435 32 clone3 ppc_clone3 sys_clone3 +435 64 clone3 sys_clone3 +435 spu clone3 sys_ni_syscall
Signature · 1 line
2.22.0
Re: [PATCH v2 3/4] powerpc/64: system call remove non-volatile GPR save optimisation
Le 27/08/2019 à 15:55, Nicholas Piggin a écrit :
Quoted 9 lines
powerpc has an optimisation where interrupts avoid saving the non-volatile (or callee saved) registers to the interrupt stack frame if they are not required.
Two problems with this are that an interrupt does not always know whether it will need non-volatiles; and if it does need them, they can only be saved from the entry-scoped asm code (because we don't control what the C compiler does with these registers).
Quoted 5 lines
system calls are the most difficult: some system calls always require all registers (e.g., fork, to copy regs into the child). Sometimes registers are only required under certain conditions (e.g., tracing, signal delivery). These cases require ugly logic in the call chains (e.g.,
ppc_fork), and require a lot of logic to be implemented in asm.
Do you really find it ugly to just call function nvgprs() before calling sys_fork() ? I guess there are things a lot uglier.
Quoted 6 lines
So remove the optimisation for system calls, and always save NVGPRs on entry. Modern high performance CPUs are not so sensitive, because the stores are dense in cache and can be hidden by other expensive work in the syscall path -- the null syscall selftests benchmark on POWER9 is not slowed (124.40ns before and 123.64ns after, i.e., within the noise).
I did the test on PPC32:
On an 885, null_syscall reports 2227ns (132MHz) If saving non-volatile regs, it goes to 2419, ie +8.6%
On an 8321, null_syscall reports 1021ns (333MHz) If saving non-volatile regs, it goes to 1100, ie +7.7%
So unless going to C compensates this degradation, I guess it is not worth it on PPC32.
Quoted 53 lines
Other interrupts retain the NVGPR optimisation for now.
Signed-off-by Nicholas Piggin <[email protected]>---
Changes since v1:
- Improve changelog
- Fix clone3 spu table entry (Segher)arch/powerpc/kernel/entry_64.S | 72 +++++------------------- arch/powerpc/kernel/syscalls/syscall.tbl | 22 +++++--- 2 files changed, 28 insertions(+), 66 deletions(-)arch/powerpc/kernel/entry_64.S +2
-3@@ -98,13 +98,14 @@ END_BTB_FLUSH_SECTION std r11,_XER(r1) std r11,_CTR(r1) std r9,GPR13(r1) + SAVE_NVGPRS(r1) mflr r10 /* * This clears CR0.SO (bit 28), which is the error indication on * return from this system call. */ rldimi r2,r11,28,(63-28) - li r11,0xc01 + li r11,0xc00 std r10,_LINK(r1) std r11,_TRAP(r1) std r3,ORIG_GPR3(r1)@@ -323,7 +324,6 @@ END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR) /* Traced system call support */ .Lsyscall_dotrace: - bl save_nvgprs addi r3,r1,STACK_FRAME_OVERHEAD bl do_syscall_trace_enter@@ -408,7 +408,6 @@ END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR) mtmsrd r10,1 #endif /* CONFIG_PPC_BOOK3E */ - bl save_nvgprs addi r3,r1,STACK_FRAME_OVERHEAD bl do_syscall_trace_leave b ret_from_except@@ -442,62 +441,6 @@ END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR) _ASM_NOKPROBE_SYMBOL(system_call_common); _ASM_NOKPROBE_SYMBOL(system_call_exit);
arch/powerpc/kernel/entry_64.S END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR) · line 454
Quoted 10 lines
arch/powerpc/kernel/entry_64.S +0
-10-/* Save non-volatile GPRs, if not already saved. */ -_GLOBAL(save_nvgprs) - ld r11,_TRAP(r1) - andi. r0,r11,1 - beqlr- - SAVE_NVGPRS(r1) - clrrdi r0,r11,1 - std r0,_TRAP(r1) - blr -_ASM_NOKPROBE_SYMBOL(save_nvgprs);
I see it is added back somewhere below. Why don't you leave it where it is ?
Quoted 54 lines
arch/powerpc/kernel/entry_64.S +1
-46- - -/* - * The sigsuspend and rt_sigsuspend system calls can call do_signal - * and thus put the process into the stopped state where we might - * want to examine its user state with ptrace. Therefore we need - * to save all the nonvolatile registers (r14 - r31) before calling - * the C code. Similarly, fork, vfork and clone need the full - * register state on the stack so that it can be copied to the child. - */ - -_GLOBAL(ppc_fork) - bl save_nvgprs - bl sys_fork - b .Lsyscall_exit - -_GLOBAL(ppc_vfork) - bl save_nvgprs - bl sys_vfork - b .Lsyscall_exit - -_GLOBAL(ppc_clone) - bl save_nvgprs - bl sys_clone - b .Lsyscall_exit - -_GLOBAL(ppc_clone3) - bl save_nvgprs - bl sys_clone3 - b .Lsyscall_exit - -_GLOBAL(ppc32_swapcontext) - bl save_nvgprs - bl compat_sys_swapcontext - b .Lsyscall_exit - -_GLOBAL(ppc64_swapcontext) - bl save_nvgprs - bl sys_swapcontext - b .Lsyscall_exit - -_GLOBAL(ppc_switch_endian) - bl save_nvgprs - bl sys_switch_endian - b .Lsyscall_exit - _GLOBAL(ret_from_fork) bl schedule_tail REST_NVGPRS(r1)@@ -516,6 +459,17 @@ _GLOBAL(ret_from_kernel_thread) li r3,0 b .Lsyscall_exit +/* Save non-volatile GPRs, if not already saved. */
arch/powerpc/kernel/entry_64.S _GLOBAL(ret_from_kernel_thread) · line 472
Quoted 10 lines
arch/powerpc/kernel/entry_64.S +10
-0+_GLOBAL(save_nvgprs) + ld r11,_TRAP(r1) + andi. r0,r11,1 + beqlr- + SAVE_NVGPRS(r1) + clrrdi r0,r11,1 + std r0,_TRAP(r1) + blr +_ASM_NOKPROBE_SYMBOL(save_nvgprs); +
Moved here.
arch/powerpc/kernel/entry_64.S _GLOBAL(ret_from_kernel_thread) · line 475
Quoted 3 lines
arch/powerpc/kernel/entry_64.S +0
-0#ifdef CONFIG_PPC_BOOK3S_64 #define FLUSH_COUNT_CACHE \
Christophe
Nicholas Piggin
reviewerRe: [PATCH v2 3/4] powerpc/64: system call remove non-volatile GPR save optimisation
Christophe Leroy's on August 28, 2019 7:02 pm:
Quoted 12 lines
Le 27/08/2019 à 15:55, Nicholas Piggin a écrit :
Quoted 9 lines
powerpc has an optimisation where interrupts avoid saving the non-volatile (or callee saved) registers to the interrupt stack frame if they are not required.
Two problems with this are that an interrupt does not always know whether it will need non-volatiles; and if it does need them, they can only be saved from the entry-scoped asm code (because we don't control what the C compiler does with these registers).
Quoted 8 lines
Quoted 5 lines
system calls are the most difficult: some system calls always require all registers (e.g., fork, to copy regs into the child). Sometimes registers are only required under certain conditions (e.g., tracing, signal delivery). These cases require ugly logic in the call chains (e.g.,
ppc_fork), and require a lot of logic to be implemented in asm.Do you really find it ugly to just call function
nvgprs()before callingsys_fork()? I guess there are things a lot uglier.
That's not the ugly part, the ugly part is trashing the link register and then branching directly to where it was supposed to return, which is bad for any CPU which has a return predictor so we try to eliminate it from the ppc64 kernel.
Quoted 8 lines
Quoted 5 lines
So remove the optimisation for system calls, and always save NVGPRs on entry. Modern high performance CPUs are not so sensitive, because the stores are dense in cache and can be hidden by other expensive work in the syscall path -- the null syscall selftests benchmark on POWER9 is not slowed (124.40ns before and 123.64ns after, i.e., within the noise).
I did the test on PPC32:
Quoted 8 lines
On an 885,
null_syscallreports 2227ns (132MHz) If saving non-volatile regs, it goes to 2419, ie +8.6%On an 8321,
null_syscallreports 1021ns (333MHz) If saving non-volatile regs, it goes to 1100, ie +7.7%So unless going to C compensates this degradation, I guess it is not worth it on PPC32.
Yeah that's unfortunate. It is a good optimization for small cores.
I doubt going to C would help for PPC32, probably be even slower.
Quoted 13 lines
arch/powerpc/kernel/entry_64.S END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR) · line 454
Quoted 11 lines
arch/powerpc/kernel/entry_64.S +0
-10-/* Save non-volatile GPRs, if not already saved. */ -_GLOBAL(save_nvgprs) - ld r11,_TRAP(r1) - andi. r0,r11,1 - beqlr- - SAVE_NVGPRS(r1) - clrrdi r0,r11,1 - std r0,_TRAP(r1) - blr -_ASM_NOKPROBE_SYMBOL(save_nvgprs);I see it is added back somewhere below. Why don't you leave it where it is ?
No longer used by syscalls so I it out from between other syscall related code to improve icache.
Thanks,
Nick
Nicholas Piggin
reviewerPATCH v2 4/4 powerpc/64: system call implement the bulk of the logic in C
In reply to Nicholas PigginSystem call entry and particularly exit code is beyond the limit of what is reasonable to implement in asm.
This conversion moves all conditional branches out of the asm code, except for the case that all GPRs should be restored at exit.
Null syscall test is about 5% faster after this patch, because the exit work is handled under local_irq_disable, and the hard mask and pending interrupt replay is handled after that, which avoids games with MSR.
---
Changes since v1:
- Improve changelog
- Lot of code cleanups, moving helpers out to proper header locations, etc (Christophe).
- Split unnecessary change that affected ppc32 out. I will submit it independently (Christophe).
arch/powerpc/include/asm/asm-prototypes.h | 11 - .../powerpc/include/asm/book3s/64/kup-radix.h | 12 +- arch/powerpc/include/asm/cputime.h | 22 ++ arch/powerpc/include/asm/ptrace.h | 3 + arch/powerpc/include/asm/signal.h | 2 + arch/powerpc/include/asm/switch_to.h | 5 + arch/powerpc/include/asm/time.h | 3 + arch/powerpc/kernel/Makefile | 3 +- arch/powerpc/kernel/entry_64.S | 340 +++--------------- arch/powerpc/kernel/signal.h | 2 - arch/powerpc/kernel/syscall_64.c | 177 +++++++++ 11 files changed, 273 insertions(+), 307 deletions(-) create mode 100644 arch/powerpc/kernel/syscall_64.c
arch/powerpc/include/asm/asm-prototypes.h +0 -11
@@ -92,14 +92,6 @@ long sys_switch_endian(void); notrace unsigned int __check_irq_replay(void); void notrace restore_interrupts(void); -/* ptrace */ -long do_syscall_trace_enter (struct pt_regs *regs); -void do_syscall_trace_leave (struct pt_regs *regs); - -/* process */ -void restore_math (struct pt_regs *regs); -void restore_tm_state (struct pt_regs *regs); -/* prom_init (OpenFirmware) */ unsigned long __init prom_init(unsigned long r3, unsigned long r4, unsigned long pp,
@@ -110,9 +102,6 @@ unsigned long __init prom_init(unsigned long r3, unsigned long r4,void __init early_setup (unsigned long dt_ptr);void early_setup_secondary (void ); -/* time */ -void accumulate_stolen_time (void ); -/* misc runtime */ extern u64 __bswapdi2 (u64 );extern s64 __lshrdi3 (s64 ,int );
arch/powerpc/include/asm/book3s/64/kup-radix.h +11 -1
@@ -3,6 +3,7 @@#define _ASM_POWERPC_BOOK3S_64_KUP_RADIX_H#include <linux/const.h> +#include <asm/reg.h> #define AMR_KUAP_BLOCK_READ UL(0x4000000000000000)#define AMR_KUAP_BLOCK_WRITE UL(0x8000000000000000)
@@ -56,7 +57,16 @@#ifdef CONFIG_PPC_KUAP -#include <asm/reg.h> +#include <asm/mmu.h> +#include <asm/ptrace.h> + +static inline void kuap_check_amr (void ) +{ +#ifdef CONFIG_PPC_KUAP_DEBUG +if (mmu_has_feature (MMU_FTR_RADIX_KUAP )) +WARN_ON_ONCE (mfspr (SPRN_AMR ) !=AMR_KUAP_BLOCKED ); +#endif +}/* * We support individually allowing read or write, but we don't support nesting
arch/powerpc/include/asm/cputime.h +22 -0
@@ -60,6 +60,28 @@ static inline void arch_vtime_task_switch(struct task_struct *prev) }#endif +static inline void account_cpu_user_entry (void ) +{ +unsigned long tb =mftb (); + +get_accounting (current)->utime += (tb -get_accounting (current)->starttime_user); +get_accounting (current)->starttime = tb; +} +static inline void account_cpu_user_exit (void ) +{ +unsigned long tb =mftb (); + +get_accounting (current)->stime += (tb -get_accounting (current)->starttime); +get_accounting (current)->starttime_user = tb; +} +#endif /* __KERNEL__ */ +#else /* CONFIG_VIRT_CPU_ACCOUNTING_NATIVE */ +static inline void account_cpu_user_entry (void ) +{ +} +static inline void account_cpu_user_exit (void ) +{ +}#endif /* CONFIG_VIRT_CPU_ACCOUNTING_NATIVE */ #endif /* __POWERPC_CPUTIME_H */
arch/powerpc/include/asm/ptrace.h +3 -0
@@ -138,6 +138,9 @@ extern unsigned long profile_pc(struct pt_regs *regs);#define profile_pc (regs) instruction_pointer(regs)#endif +long do_syscall_trace_enter (struct pt_regs *regs); +void do_syscall_trace_leave (struct pt_regs *regs); +#define kernel_stack_pointer (regs) ((regs)->gpr[1])static inline int is_syscall_success (struct pt_regs *regs) {
arch/powerpc/include/asm/signal.h +2 -0
@@ -6,4 +6,6 @@#include <uapi/asm/signal.h> #include <uapi/asm/ptrace.h> +void do_notify_resume (struct pt_regs *regs,unsigned long thread_info_flags); +#endif /* _ASM_POWERPC_SIGNAL_H */
arch/powerpc/include/asm/switch_to.h +5 -0
@@ -5,6 +5,7 @@#ifndef _ASM_POWERPC_SWITCH_TO_H#define _ASM_POWERPC_SWITCH_TO_H +#include <linux/sched.h> #include <asm/reg.h> struct thread_struct ;
@@ -22,6 +23,10 @@ extern void switch_booke_debug_regs(struct debug_reg *new_debug);extern int emulate_altivec (struct pt_regs *); +void restore_math (struct pt_regs *regs); + +void restore_tm_state (struct pt_regs *regs); +extern void flush_all_to_thread (struct task_struct *);extern void giveup_all (struct task_struct *);
arch/powerpc/include/asm/time.h +3 -0
@@ -199,5 +199,8 @@ DECLARE_PER_CPU(u64, decrementers_next_tb);/* Convert timebase ticks to nanoseconds */ unsigned long long tb_to_ns (unsigned long long tb_ticks); +/* SPLPAR */ +void accumulate_stolen_time (void ); +#endif /* __KERNEL__ */ #endif /* __POWERPC_TIME_H */
arch/powerpc/kernel/Makefile +2 -1
@@ -52,7 +52,8 @@ obj-y := cputable.o ptrace.o syscalls.o \ of_platform.o prom_parse.o obj-$(CONFIG_PPC64) += setup_64.o sys_ppc32.o \ signal_64.o ptrace32.o \ - paca.o nvram_64.o firmware.o + paca.o nvram_64.o firmware.o \ + syscall_64.o obj-$(CONFIG_VDSO32) += vdso32/ obj-$(CONFIG_PPC_WATCHDOG) += watchdog.o obj-$(CONFIG_HAVE_HW_BREAKPOINT) += hw_breakpoint.o
arch/powerpc/kernel/entry_64.S +48 -292
419 lines not on this page · raw message
arch/powerpc/kernel/signal.h +0 -2
@@ -10,8 +10,6 @@#ifndef _POWERPC_ARCH_SIGNAL_H#define _POWERPC_ARCH_SIGNAL_H -extern void do_notify_resume (struct pt_regs *regs,unsigned long thread_info_flags); -extern void __user *get_sigframe (struct ksignal *ksig,unsigned long sp,size_t frame_size,int is_32);
arch/powerpc/kernel/syscall_64.c new file +177 -0
177 lines not on this page · raw message
Signature · 1 line
2.22.0
Re: [PATCH v2 4/4] powerpc/64: system call implement the bulk of the logic in C
Le 27/08/2019 à 15:55, Nicholas Piggin a écrit :
Quoted 76 lines
System call entry and particularly exit code is beyond the limit of what is reasonable to implement in asm.
This conversion moves all conditional branches out of the asm code, except for the case that all GPRs should be restored at exit.
Null syscall test is about 5% faster after this patch, because the exit work is handled under
local_irq_disable, and the hard mask and pending interrupt replay is handled after that, which avoids games with MSR.Signed-off-by Nicholas Piggin <[email protected]>---
Changes since v1:
- Improve changelog
- Lot of code cleanups, moving helpers out to proper header locations, etc (Christophe).
- Split unnecessary change that affected ppc32 out. I will submit it independently (Christophe).arch/powerpc/include/asm/asm-prototypes.h | 11 - .../powerpc/include/asm/book3s/64/kup-radix.h | 12 +- arch/powerpc/include/asm/cputime.h | 22 ++ arch/powerpc/include/asm/ptrace.h | 3 + arch/powerpc/include/asm/signal.h | 2 + arch/powerpc/include/asm/switch_to.h | 5 + arch/powerpc/include/asm/time.h | 3 + arch/powerpc/kernel/Makefile | 3 +- arch/powerpc/kernel/entry_64.S | 340 +++--------------- arch/powerpc/kernel/signal.h | 2 - arch/powerpc/kernel/syscall_64.c | 177 +++++++++ 11 files changed, 273 insertions(+), 307 deletions(-)create mode 100644
arch/powerpc/kernel/syscall_64.carch/powerpc/include/asm/asm-prototypes.h +0
-11@@ -92,14 +92,6 @@ long sys_switch_endian(void); notrace unsigned int __check_irq_replay(void); void notrace restore_interrupts(void); -/* ptrace */ -long do_syscall_trace_enter (struct pt_regs *regs); -void do_syscall_trace_leave (struct pt_regs *regs); - -/* process */ -void restore_math (struct pt_regs *regs); -void restore_tm_state (struct pt_regs *regs); -/* prom_init (OpenFirmware) */ unsigned long __init prom_init(unsigned long r3, unsigned long r4, unsigned long pp,@@ -110,9 +102,6 @@ unsigned long __init prom_init(unsigned long r3, unsigned long r4,void __init early_setup (unsigned long dt_ptr);void early_setup_secondary (void ); -/* time */ -void accumulate_stolen_time (void ); -/* misc runtime */ extern u64 __bswapdi2 (u64 );extern s64 __lshrdi3 (s64 ,int );arch/powerpc/include/asm/book3s/64/kup-radix.h +1
-0@@ -3,6 +3,7 @@#define _ASM_POWERPC_BOOK3S_64_KUP_RADIX_H#include <linux/const.h> +#include <asm/reg.h> #define AMR_KUAP_BLOCK_READ UL(0x4000000000000000)#define AMR_KUAP_BLOCK_WRITE UL(0x8000000000000000)@@ -56,7 +57,16 @@
arch/powerpc/include/asm/book3s/64/kup-radix.h line 66
Quoted 10 lines
arch/powerpc/include/asm/book3s/64/kup-radix.h +7
-1#ifdef CONFIG_PPC_KUAP -#include <asm/reg.h> +#include <asm/mmu.h> +#include <asm/ptrace.h> + +static inline void kuap_check_amr (void ) +{ +#ifdef CONFIG_PPC_KUAP_DEBUG +if (mmu_has_feature (MMU_FTR_RADIX_KUAP ))
Better:
if (IS_ENABLED(CONFIG_PPC_KUAP_DEBUG) &&
mmu_has_feature(MMU_FTR_RADIX_KUAP))
Quoted 11 lines
arch/powerpc/include/asm/book3s/64/kup-radix.h +3
-0+WARN_ON_ONCE (mfspr (SPRN_AMR ) !=AMR_KUAP_BLOCKED ); +#endif +}/* * We support individually allowing read or write, but we don't support nesting arch/powerpc/include/asm/cputime.h +0
-0
arch/powerpc/include/asm/cputime.h static inline void arch_vtime_task_switch(struct task_struct *prev) · line 69
Quoted 10 lines
arch/powerpc/include/asm/cputime.h +7
-0@@ -60,6 +60,28 @@ static inline void arch_vtime_task_switch(struct task_struct *prev) }#endif +static inline void account_cpu_user_entry (void ) +{ +unsigned long tb =mftb (); + +get_accounting (current)->utime += (tb -get_accounting (current)->starttime_user); +get_accounting (current)->starttime = tb; +}
Can you check the generated assembly ? I remember having bad result with get_accouting() being used several times in a arch_vtime_task_switch() before commit 60f1d2893ee6 ("powerpc/time: inline arch_vtime_task_switch()")
Regardless, I think it would look better as:
static inline void account_cpu_user_entry(void)
{
unsigned long tb = mftb();
struct cpu_accounting_data *acct = get_accounting(current);
acct->utime += (tb - acct->starttime_user);
acct->starttime = tb;
}arch/powerpc/include/asm/cputime.h static inline void arch_vtime_task_switch(struct task_struct *prev) · line 76
Quoted 7 lines
arch/powerpc/include/asm/cputime.h +7
-0+static inline void account_cpu_user_exit (void ) +{ +unsigned long tb =mftb (); + +get_accounting (current)->stime += (tb -get_accounting (current)->starttime); +get_accounting (current)->starttime_user = tb; +}
Same here.
Christophe
Nicholas Piggin
reviewerRe: [PATCH v2 4/4] powerpc/64: system call implement the bulk of the logic in C
Christophe Leroy's on August 28, 2019 4:51 pm:
Quoted 14 lines
Le 27/08/2019 à 15:55, Nicholas Piggin a écrit :
arch/powerpc/include/asm/book3s/64/kup-radix.h line 66
Quoted 8 lines
arch/powerpc/include/asm/book3s/64/kup-radix.h +7
-1-#include <asm/reg.h> +#include <asm/mmu.h> +#include <asm/ptrace.h> + +static inline void kuap_check_amr (void ) +{ +#ifdef CONFIG_PPC_KUAP_DEBUG +if (mmu_has_feature (MMU_FTR_RADIX_KUAP ))Better:
Quoted 2 lines
if (IS_ENABLED(CONFIG_PPC_KUAP_DEBUG) &&
mmu_has_feature(MMU_FTR_RADIX_KUAP))
That is better.
Quoted 14 lines
Quoted 14 lines
arch/powerpc/include/asm/book3s/64/kup-radix.h +3
-0+WARN_ON_ONCE (mfspr (SPRN_AMR ) !=AMR_KUAP_BLOCKED ); +#endif +}/* * We support individually allowing read or write, but we don't support nesting arch/powerpc/include/asm/cputime.h +0
-0@@ -60,6 +60,28 @@ static inline void arch_vtime_task_switch(struct task_struct *prev) }#endif
Quoted 12 lines
arch/powerpc/include/asm/cputime.h static inline void arch_vtime_task_switch(struct task_struct *prev) · line 69
Quoted 7 lines
arch/powerpc/include/asm/cputime.h +7
-0+static inline void account_cpu_user_entry (void ) +{ +unsigned long tb =mftb (); + +get_accounting (current)->utime += (tb -get_accounting (current)->starttime_user); +get_accounting (current)->starttime = tb; +}Can you check the generated assembly ? I remember having bad result with
get_accouting()being used several times in aarch_vtime_task_switch()
before commit60f1d2893ee6("powerpc/time: inlinearch_vtime_task_switch()")
It's fine on 64s but it's accounting is a constant offset from r13 so simple load/store can be done.
Quoted 10 lines
Regardless, I think it would look better as:
static inline void
account_cpu_user_entry(void){ unsigned long tb = mftb(); struct cpu_accounting_data *acct = get_accounting(current); acct->utime += (tb - acct->starttime_user); acct->starttime = tb; }
Yeah that's nicer.
Quoted 10 lines
arch/powerpc/include/asm/cputime.h static inline void arch_vtime_task_switch(struct task_struct *prev) · line 76
Quoted 8 lines
arch/powerpc/include/asm/cputime.h +7
-0+static inline void account_cpu_user_exit (void ) +{ +unsigned long tb =mftb (); + +get_accounting (current)->stime += (tb -get_accounting (current)->starttime); +get_accounting (current)->starttime_user = tb; +}Same here.
Will do.
Thanks,
Nick
Re: [PATCH v2 4/4] powerpc/64: system call implement the bulk of the logic in C
In reply to Nicholas PigginQuoted 144 lines · On Tue, 27 Aug 2019 23:55:48 +1000 Nicholas Piggin <[email protected]> wrote:
System call entry and particularly exit code is beyond the limit of what is reasonable to implement in asm.
This conversion moves all conditional branches out of the asm code, except for the case that all GPRs should be restored at exit.
Null syscall test is about 5% faster after this patch, because the exit work is handled under
local_irq_disable, and the hard mask and pending interrupt replay is handled after that, which avoids games with MSR.Signed-off-by Nicholas Piggin <[email protected]>---
Changes since v1:
- Improve changelog
- Lot of code cleanups, moving helpers out to proper header locations, etc (Christophe).
- Split unnecessary change that affected ppc32 out. I will submit it independently (Christophe).arch/powerpc/include/asm/asm-prototypes.h | 11 - .../powerpc/include/asm/book3s/64/kup-radix.h | 12 +- arch/powerpc/include/asm/cputime.h | 22 ++ arch/powerpc/include/asm/ptrace.h | 3 + arch/powerpc/include/asm/signal.h | 2 + arch/powerpc/include/asm/switch_to.h | 5 + arch/powerpc/include/asm/time.h | 3 + arch/powerpc/kernel/Makefile | 3 +- arch/powerpc/kernel/entry_64.S | 340 +++--------------- arch/powerpc/kernel/signal.h | 2 - arch/powerpc/kernel/syscall_64.c | 177 +++++++++ 11 files changed, 273 insertions(+), 307 deletions(-) create mode 100644 arch/powerpc/kernel/syscall_64.carch/powerpc/include/asm/asm-prototypes.h +0
-11@@ -92,14 +92,6 @@ long sys_switch_endian(void); notrace unsigned int __check_irq_replay(void); void notrace restore_interrupts(void); -/* ptrace */ -long do_syscall_trace_enter (struct pt_regs *regs); -void do_syscall_trace_leave (struct pt_regs *regs); - -/* process */ -void restore_math (struct pt_regs *regs); -void restore_tm_state (struct pt_regs *regs); -/* prom_init (OpenFirmware) */ unsigned long __init prom_init(unsigned long r3, unsigned long r4, unsigned long pp,@@ -110,9 +102,6 @@ unsigned long __init prom_init(unsigned long r3, unsigned long r4,void __init early_setup (unsigned long dt_ptr);void early_setup_secondary (void ); -/* time */ -void accumulate_stolen_time (void ); -/* misc runtime */ extern u64 __bswapdi2 (u64 );extern s64 __lshrdi3 (s64 ,int );arch/powerpc/include/asm/book3s/64/kup-radix.h +11
-1@@ -3,6 +3,7 @@#define _ASM_POWERPC_BOOK3S_64_KUP_RADIX_H#include <linux/const.h> +#include <asm/reg.h> #define AMR_KUAP_BLOCK_READ UL(0x4000000000000000)#define AMR_KUAP_BLOCK_WRITE UL(0x8000000000000000)@@ -56,7 +57,16 @@#ifdef CONFIG_PPC_KUAP -#include <asm/reg.h> +#include <asm/mmu.h> +#include <asm/ptrace.h> + +static inline void kuap_check_amr (void ) +{ +#ifdef CONFIG_PPC_KUAP_DEBUG +if (mmu_has_feature (MMU_FTR_RADIX_KUAP )) +WARN_ON_ONCE (mfspr (SPRN_AMR ) !=AMR_KUAP_BLOCKED ); +#endif +}/* * We support individually allowing read or write, but we don't support nesting arch/powerpc/include/asm/cputime.h +22
-0@@ -60,6 +60,28 @@ static inline void arch_vtime_task_switch(struct task_struct *prev) }#endif +static inline void account_cpu_user_entry (void ) +{ +unsigned long tb =mftb (); + +get_accounting (current)->utime += (tb -get_accounting (current)->starttime_user); +get_accounting (current)->starttime = tb; +} +static inline void account_cpu_user_exit (void ) +{ +unsigned long tb =mftb (); + +get_accounting (current)->stime += (tb -get_accounting (current)->starttime); +get_accounting (current)->starttime_user = tb; +} +#endif /* __KERNEL__ */ +#else /* CONFIG_VIRT_CPU_ACCOUNTING_NATIVE */ +static inline void account_cpu_user_entry (void ) +{ +} +static inline void account_cpu_user_exit (void ) +{ +}#endif /* CONFIG_VIRT_CPU_ACCOUNTING_NATIVE */ #endif /* __POWERPC_CPUTIME_H */ arch/powerpc/include/asm/ptrace.h +3
-0@@ -138,6 +138,9 @@ extern unsigned long profile_pc(struct pt_regs *regs);#define profile_pc (regs) instruction_pointer(regs)#endif +long do_syscall_trace_enter (struct pt_regs *regs); +void do_syscall_trace_leave (struct pt_regs *regs); +#define kernel_stack_pointer (regs) ((regs)->gpr[1])static inline int is_syscall_success (struct pt_regs *regs) {arch/powerpc/include/asm/signal.h +0
-0
arch/powerpc/include/asm/signal.h line 10
Quoted 5 lines
arch/powerpc/include/asm/signal.h +2
-0@@ -6,4 +6,6 @@#include <uapi/asm/signal.h> #include <uapi/asm/ptrace.h> +void do_notify_resume (struct pt_regs *regs,unsigned long thread_info_flags); +
/srv/kernel/arch/powerpc/include/asm/signal.h:9:30: warning: ‘struct pt_regs’ declared inside parameter list will not be visible outside of this definition or declaration
void do_notify_resume(struct pt_regs *regs, unsigned long thread_info_flags);
uapi/asm/ptrace.h defines user_pt_regs and asm/ptrace.h pt_regs.
I am not really sure which you wanted.
Thanks
Michal
Nicholas Piggin
reviewerRe: [PATCH v2 4/4] powerpc/64: system call implement the bulk of the logic in C
Michal Suchánek's on August 29, 2019 1:30 am:
Quoted 150 lines
Quoted 147 lines · On Tue, 27 Aug 2019 23:55:48 +1000 Nicholas Piggin <[email protected]> wrote:
System call entry and particularly exit code is beyond the limit of what is reasonable to implement in asm.
This conversion moves all conditional branches out of the asm code, except for the case that all GPRs should be restored at exit.
Null syscall test is about 5% faster after this patch, because the exit work is handled under
local_irq_disable, and the hard mask and pending interrupt replay is handled after that, which avoids games with MSR.Signed-off-by Nicholas Piggin <[email protected]>---
Changes since v1:
- Improve changelog
- Lot of code cleanups, moving helpers out to proper header locations, etc (Christophe).
- Split unnecessary change that affected ppc32 out. I will submit it independently (Christophe).arch/powerpc/include/asm/asm-prototypes.h | 11 - .../powerpc/include/asm/book3s/64/kup-radix.h | 12 +- arch/powerpc/include/asm/cputime.h | 22 ++ arch/powerpc/include/asm/ptrace.h | 3 + arch/powerpc/include/asm/signal.h | 2 + arch/powerpc/include/asm/switch_to.h | 5 + arch/powerpc/include/asm/time.h | 3 + arch/powerpc/kernel/Makefile | 3 +- arch/powerpc/kernel/entry_64.S | 340 +++--------------- arch/powerpc/kernel/signal.h | 2 - arch/powerpc/kernel/syscall_64.c | 177 +++++++++ 11 files changed, 273 insertions(+), 307 deletions(-) create mode 100644 arch/powerpc/kernel/syscall_64.carch/powerpc/include/asm/asm-prototypes.h +0
-11@@ -92,14 +92,6 @@ long sys_switch_endian(void); notrace unsigned int __check_irq_replay(void); void notrace restore_interrupts(void); -/* ptrace */ -long do_syscall_trace_enter (struct pt_regs *regs); -void do_syscall_trace_leave (struct pt_regs *regs); - -/* process */ -void restore_math (struct pt_regs *regs); -void restore_tm_state (struct pt_regs *regs); -/* prom_init (OpenFirmware) */ unsigned long __init prom_init(unsigned long r3, unsigned long r4, unsigned long pp,@@ -110,9 +102,6 @@ unsigned long __init prom_init(unsigned long r3, unsigned long r4,void __init early_setup (unsigned long dt_ptr);void early_setup_secondary (void ); -/* time */ -void accumulate_stolen_time (void ); -/* misc runtime */ extern u64 __bswapdi2 (u64 );extern s64 __lshrdi3 (s64 ,int );arch/powerpc/include/asm/book3s/64/kup-radix.h +11
-1@@ -3,6 +3,7 @@#define _ASM_POWERPC_BOOK3S_64_KUP_RADIX_H#include <linux/const.h> +#include <asm/reg.h> #define AMR_KUAP_BLOCK_READ UL(0x4000000000000000)#define AMR_KUAP_BLOCK_WRITE UL(0x8000000000000000)@@ -56,7 +57,16 @@#ifdef CONFIG_PPC_KUAP -#include <asm/reg.h> +#include <asm/mmu.h> +#include <asm/ptrace.h> + +static inline void kuap_check_amr (void ) +{ +#ifdef CONFIG_PPC_KUAP_DEBUG +if (mmu_has_feature (MMU_FTR_RADIX_KUAP )) +WARN_ON_ONCE (mfspr (SPRN_AMR ) !=AMR_KUAP_BLOCKED ); +#endif +}/* * We support individually allowing read or write, but we don't support nesting arch/powerpc/include/asm/cputime.h +22
-0@@ -60,6 +60,28 @@ static inline void arch_vtime_task_switch(struct task_struct *prev) }#endif +static inline void account_cpu_user_entry (void ) +{ +unsigned long tb =mftb (); + +get_accounting (current)->utime += (tb -get_accounting (current)->starttime_user); +get_accounting (current)->starttime = tb; +} +static inline void account_cpu_user_exit (void ) +{ +unsigned long tb =mftb (); + +get_accounting (current)->stime += (tb -get_accounting (current)->starttime); +get_accounting (current)->starttime_user = tb; +} +#endif /* __KERNEL__ */ +#else /* CONFIG_VIRT_CPU_ACCOUNTING_NATIVE */ +static inline void account_cpu_user_entry (void ) +{ +} +static inline void account_cpu_user_exit (void ) +{ +}#endif /* CONFIG_VIRT_CPU_ACCOUNTING_NATIVE */ #endif /* __POWERPC_CPUTIME_H */ arch/powerpc/include/asm/ptrace.h +3
-0@@ -138,6 +138,9 @@ extern unsigned long profile_pc(struct pt_regs *regs);#define profile_pc (regs) instruction_pointer(regs)#endif +long do_syscall_trace_enter (struct pt_regs *regs); +void do_syscall_trace_leave (struct pt_regs *regs); +#define kernel_stack_pointer (regs) ((regs)->gpr[1])static inline int is_syscall_success (struct pt_regs *regs) {arch/powerpc/include/asm/signal.h +0
-0@@ -6,4 +6,6 @@#include <uapi/asm/signal.h> #include <uapi/asm/ptrace.h>
Quoted 4 lines
arch/powerpc/include/asm/signal.h line 10
Quoted 2 lines
arch/powerpc/include/asm/signal.h +2
-0+void do_notify_resume (struct pt_regs *regs,unsigned long thread_info_flags); +
/srv/kernel/arch/powerpc/include/asm/signal.h:9:30: warning: ‘struct pt_regs’ declared inside parameter list will not be visible outside of this definition or declaration
voiddo_notify_resume(struct pt_regs *regs, unsigned long thread_info_flags);
Thanks I didn't see that.
Quoted 3 lines
uapi/asm/ptrace.hdefinesuser_pt_regsandasm/ptrace.hpt_regs.I am not really sure which you wanted.
pt_regs. That is the struct that gets saved on the kernel stack by interrupts. user_pt_regs is part of the ptrace user ABI.
Thanks,
Nick
kbuild test robot
build botRe: [PATCH v2 4/4] powerpc/64: system call implement the bulk of the logic in C
In reply to Nicholas PigginI love your patch! Yet something to improve:
Hi Nicholas,
I love your patch! Yet something to improve:
[auto build test ERROR on linus/master]
[cannot apply to v5.3-rc6 next-20190830]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
url: https://github.com/0day-ci/linux/commits/Nicholas-Piggin/powerpc-64-syscalls-in-C/20190828-064221
config: powerpc64-defconfig (attached as .config)
compiler: powerpc64-linux-gcc (GCC) 7.4.0
reproduce:
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
GCC_VERSION=7.4.0 make.cross ARCH=powerpc64
If you fix the issue, kindly add following tag
All errors (new ones prefixed by >>):
powerpc64-linux-ld: warning: orphan section .gnu.hash' from linker stubs' being placed in section .gnu.hash'. arch/powerpc/kernel/syscall_64.o: In function .system_call_exception':
>> syscall_64.c:(.text+0x180): undefined reference to `.tabort_syscall'
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
Re: [PATCH v2 4/4] powerpc/64: system call implement the bulk of the logic in C
Quoted 17 lines · On Sat, 31 Aug 2019 02:48:26 +0800 kbuild test robot <[email protected]> wrote:
Hi Nicholas,
I love your patch! Yet something to improve:
[auto build test ERROR on linus/master]
[cannot apply to v5.3-rc6 next-20190830]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]url: https://github.com/0day-ci/linux/commits/Nicholas-Piggin/powerpc-64-syscalls-in-C/20190828-064221
config: powerpc64-defconfig (attached as .config)
compiler: powerpc64-linux-gcc (GCC) 7.4.0
reproduce:
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
GCC_VERSION=7.4.0 make.cross ARCH=powerpc64
Quoted 8 lines
If you fix the issue, kindly add following tag
Reported-by kbuild test robot <[email protected]>All errors (new ones prefixed by >>):
powerpc64-linux-ld: warning: orphan section
.gnu.hash' fromlinker stubs' being placed in section.gnu.hash'. arch/powerpc/kernel/syscall_64.o: In function.system_call_exception':Quoted 1 line
Quoted 1 line
syscall_64.c:(.text+0x180): undefined reference to `.tabort_syscall'
Interestingly it builds and boots for me. Is this something about dotted vs dotless symbols depending on some config options?
I see there is _GLOBAL(ret_from_fork) just below so maybe we need _GLOBAL(tabort_syscall) instead of .globl tabort_syscall as well.
Thanks
Michal
Michael Ellerman
reviewerRe: [PATCH v2 4/4] powerpc/64: system call implement the bulk of the logic in C
Quoted 23 lines · Michal Suchánek
Quoted 20 lines · On Sat, 31 Aug 2019 02:48:26 +0800 kbuild test robot <[email protected]> wrote:
Hi Nicholas,
I love your patch! Yet something to improve:
[auto build test ERROR on linus/master]
[cannot apply to v5.3-rc6 next-20190830]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]url: https://github.com/0day-ci/linux/commits/Nicholas-Piggin/powerpc-64-syscalls-in-C/20190828-064221
config: powerpc64-defconfig (attached as .config)
compiler: powerpc64-linux-gcc (GCC) 7.4.0
reproduce:
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
GCC_VERSION=7.4.0 make.cross ARCH=powerpc64If you fix the issue, kindly add following tag
Reported-by kbuild test robot <[email protected]>
Quoted 8 lines
Quoted 5 lines
All errors (new ones prefixed by >>):
powerpc64-linux-ld: warning: orphan section
.gnu.hash' fromlinker stubs' being placed in section.gnu.hash'. arch/powerpc/kernel/syscall_64.o: In function.system_call_exception':Quoted 1 line
Quoted 1 line
syscall_64.c:(.text+0x180): undefined reference to `.tabort_syscall'
Interestingly it builds and boots for me. Is this something about dotted vs dotless symbols depending on some config options?
It's the big endian build that fails, which is ELFv1, and the linker is saying it can't find a function called .tabort_syscall - which is understandable because there isn't one. There's a text address called tabort_syscall but it has no function descriptor so you can't call it normally from C.
Quoted 2 lines
I see there is
_GLOBAL(ret_from_fork)just below so maybe we need_GLOBAL(tabort_syscall)instead of .globltabort_syscallas well.
Yes, on ELFv1 the _GLOBAL macros creates a function descriptor for you.
This fixes it for me:
arch/powerpc/kernel/entry_64.S +1 -2
@@ -170,8 +170,7 @@ _ASM_NOKPROBE_SYMBOL(system_call_common); b .Lsyscall_restore_regs_cont #ifdef CONFIG_PPC_TRANSACTIONAL_MEM - .globl tabort_syscall -tabort_syscall: +_GLOBAL(tabort_syscall) /* Firstly we need to enable TM in the kernel */ mfmsr r10 li r9, 1
cheers
Le 27/08/2019 à 15:55, Nicholas Piggin a écrit :
Quoted 7 lines
Accounted for some feedback.
Nicholas Piggin (4):
powerpc: convert tocopy_thread_tls
powerpc/64: remove support for kernel-mode syscalls powerpc/64: system call remove non-volatile GPR save optimisation powerpc/64: system call implement the bulk of the logic in C
Would it be possible to split in the following parts:
1/ Implement in C whatever can be implemented without removing non-volatile GPR save optimisation 2/ Remove non-volatile GPR save optimisation
3/ Implement in C everything else
Christophe
Quoted 19 lines
arch/powerpc/Kconfig | 1 + arch/powerpc/include/asm/asm-prototypes.h | 11 - .../powerpc/include/asm/book3s/64/kup-radix.h | 12 +- arch/powerpc/include/asm/cputime.h | 22 + arch/powerpc/include/asm/ptrace.h | 3 + arch/powerpc/include/asm/signal.h | 2 + arch/powerpc/include/asm/switch_to.h | 5 + arch/powerpc/include/asm/time.h | 3 + arch/powerpc/kernel/Makefile | 3 +- arch/powerpc/kernel/entry_64.S | 421 +++--------------- arch/powerpc/kernel/exceptions-64s.S | 2 - arch/powerpc/kernel/process.c | 9 +- arch/powerpc/kernel/signal.h | 2 - arch/powerpc/kernel/syscall_64.c | 177 ++++++++ arch/powerpc/kernel/syscalls/syscall.tbl | 22 +- 15 files changed, 307 insertions(+), 388 deletions(-)create mode 100644
arch/powerpc/kernel/syscall_64.c
Nicholas Piggin
reviewerChristophe Leroy's on August 28, 2019 7:06 pm:
Quoted 11 lines
Le 27/08/2019 à 15:55, Nicholas Piggin a écrit :
Quoted 7 lines
Accounted for some feedback.
Nicholas Piggin (4):
powerpc: convert tocopy_thread_tls
powerpc/64: remove support for kernel-mode syscalls powerpc/64: system call remove non-volatile GPR save optimisation powerpc/64: system call implement the bulk of the logic in C
Quoted 6 lines
Would it be possible to split in the following parts:
1/ Implement in C whatever can be implemented without removing non-volatile GPR save optimisation
2/ Remove non-volatile GPR save optimisation
3/ Implement in C everything else
Hmm. I'll have a look but I would rather not go back and add the intermediate state I was hoping to avoid. I'll think about it and if it's not too difficult I will try to add something. I have an idea.
With your nvregs performance test on ppc32, are you doing the nvgpr restore? The fast path should be able to avoid that.
Thanks,
Nick
Le 28/08/2019 à 11:49, Nicholas Piggin a écrit :
Quoted 19 lines
Christophe Leroy's on August 28, 2019 7:06 pm:
Quoted 11 lines
Le 27/08/2019 à 15:55, Nicholas Piggin a écrit :
Quoted 7 lines
Accounted for some feedback.
Nicholas Piggin (4):
powerpc: convert tocopy_thread_tls
powerpc/64: remove support for kernel-mode syscalls powerpc/64: system call remove non-volatile GPR save optimisation powerpc/64: system call implement the bulk of the logic in CQuoted 6 lines
Would it be possible to split in the following parts:
1/ Implement in C whatever can be implemented without removing non-volatile GPR save optimisation
2/ Remove non-volatile GPR save optimisation
3/ Implement in C everything else
Quoted 7 lines
Hmm. I'll have a look but I would rather not go back and add the intermediate state I was hoping to avoid. I'll think about it and if it's not too difficult I will try to add something. I have an idea.
With your nvregs performance test on ppc32, are you doing the nvgpr restore? The fast path should be able to avoid that.
I only added the SAVE_NVGPRS call in the syscall entry macro just after the saving of volatile regs, and changed the trap from \trapno+1 to \trapno
Christophe
Nicholas Piggin
reviewerChristophe Leroy's on August 28, 2019 7:55 pm:
Quoted 27 lines
Le 28/08/2019 à 11:49, Nicholas Piggin a écrit :
Quoted 24 lines
Christophe Leroy's on August 28, 2019 7:06 pm:
Quoted 11 lines
Le 27/08/2019 à 15:55, Nicholas Piggin a écrit :
Quoted 7 lines
Accounted for some feedback.
Nicholas Piggin (4):
powerpc: convert tocopy_thread_tls
powerpc/64: remove support for kernel-mode syscalls powerpc/64: system call remove non-volatile GPR save optimisation powerpc/64: system call implement the bulk of the logic in CQuoted 6 lines
Would it be possible to split in the following parts:
1/ Implement in C whatever can be implemented without removing non-volatile GPR save optimisation
2/ Remove non-volatile GPR save optimisation
3/ Implement in C everything elseHmm. I'll have a look but I would rather not go back and add the intermediate state I was hoping to avoid. I'll think about it and if it's not too difficult I will try to add something. I have an idea.
Quoted 5 lines
Quoted 2 lines
With your nvregs performance test on ppc32, are you doing the nvgpr restore? The fast path should be able to avoid that.
I only added the
SAVE_NVGPRScall in the syscall entry macro just after the saving of volatile regs, and changed the trap from \trapno+1 to \trapno
So... this actually seems to work. Haven't booted it, but the compiler seems to do what we want.
This may be the way to go for ppc32 I think. I had a look at various ways you could save nvgprs with some early tests and returning early from the C call if it hits trouble without all registers saved. Most of it becomes quite ugly.
Thanks,
Nick
arch/powerpc/kernel/syscall_64.c +48 -0
@@ -14,6 +14,52 @@ extern void __noreturn tabort_syscall(void);typedef long (*syscall_fn )(long ,long ,long ,long ,long ,long ); +registerunsigned long r31 asm("r31" ); +registerunsigned long r30 asm("r30" ); +registerunsigned long r29 asm("r29" ); +registerunsigned long r28 asm("r28" ); +registerunsigned long r27 asm("r27" ); +registerunsigned long r26 asm("r26" ); +registerunsigned long r25 asm("r25" ); +registerunsigned long r24 asm("r24" ); +registerunsigned long r23 asm("r23" ); +registerunsigned long r22 asm("r22" ); +registerunsigned long r21 asm("r21" ); +registerunsigned long r20 asm("r20" ); +registerunsigned long r19 asm("r19" ); +registerunsigned long r18 asm("r18" ); +registerunsigned long r17 asm("r17" ); +registerunsigned long r16 asm("r16" ); +registerunsigned long r15 asm("r15" ); +registerunsigned long r14 asm("r14" ); + +static inline void save_nvgprs (struct pt_regs *regs) +{ +if (!(regs->trap &1 )) +return ; + + regs->gpr[14 ] = r14; + regs->gpr[15 ] = r15; + regs->gpr[16 ] = r16; + regs->gpr[17 ] = r17; + regs->gpr[18 ] = r18; + regs->gpr[19 ] = r19; + regs->gpr[20 ] = r20; + regs->gpr[21 ] = r21; + regs->gpr[22 ] = r22; + regs->gpr[23 ] = r23; + regs->gpr[24 ] = r24; + regs->gpr[25 ] = r25; + regs->gpr[26 ] = r26; + regs->gpr[27 ] = r27; + regs->gpr[28 ] = r28; + regs->gpr[29 ] = r29; + regs->gpr[30 ] = r30; + regs->gpr[31 ] = r31; + + regs->trap &=0x1 ; +} +long system_call_exception (long r3,long r4,long r5,long r6,long r7,long r8,unsigned long r0,struct pt_regs *regs) {unsigned long ti_flags;
@@ -66,6 +112,7 @@ long system_call_exception(long r3, long r4, long r5, long r6, long r7, long r8,* do_syscall_trace_enter() returns an invalid syscall number * and the test below against NR_syscalls will fail. */ +save_nvgprs (regs); r0 =do_syscall_trace_enter (regs); }
@@ -132,6 +179,7 @@ unsigned long syscall_exit_prepare(unsigned long r3, struct pt_regs *regs)if (ti_flags & _TIF_NEED_RESCHED) {schedule (); }else { +save_nvgprs (regs);/* * SIGPENDING must restore signal handler function * argument GPRs, and some non-volatiles (e.g., r1).
Nicholas Piggin
reviewerNicholas Piggin's on August 29, 2019 7:38 pm:
Quoted 28 lines
Christophe Leroy's on August 28, 2019 7:55 pm:
Quoted 27 lines
Le 28/08/2019 à 11:49, Nicholas Piggin a écrit :
Quoted 24 lines
Christophe Leroy's on August 28, 2019 7:06 pm:
Quoted 11 lines
Le 27/08/2019 à 15:55, Nicholas Piggin a écrit :
Quoted 7 lines
Accounted for some feedback.
Nicholas Piggin (4):
powerpc: convert tocopy_thread_tls
powerpc/64: remove support for kernel-mode syscalls powerpc/64: system call remove non-volatile GPR save optimisation powerpc/64: system call implement the bulk of the logic in CQuoted 6 lines
Would it be possible to split in the following parts:
1/ Implement in C whatever can be implemented without removing non-volatile GPR save optimisation
2/ Remove non-volatile GPR save optimisation
3/ Implement in C everything elseHmm. I'll have a look but I would rather not go back and add the intermediate state I was hoping to avoid. I'll think about it and if it's not too difficult I will try to add something. I have an idea.
Quoted 9 lines
Quoted 5 lines
Quoted 2 lines
With your nvregs performance test on ppc32, are you doing the nvgpr restore? The fast path should be able to avoid that.
I only added the
SAVE_NVGPRScall in the syscall entry macro just after the saving of volatile regs, and changed the trap from \trapno+1 to \trapnoSo... this actually seems to work. Haven't booted it, but the compiler seems to do what we want.
Here's a really quick start for ppc32. The interrupt handling is different enough it may be hard to merge entirely with ppc64, but it's not really much code anyway.
Unfortunately can't restore full registers using the same method, because we have some others like lr and cr, so the exit still must return a code to asm.
--- arch/powerpc/kernel/Makefile | 2 +- arch/powerpc/kernel/syscall_32.c | 167 +++++++++++++++++++++++++++++++ 2 files changed, 168 insertions(+), 1 deletion(-) create mode 100644 arch/powerpc/kernel/syscall_32.c
arch/powerpc/kernel/Makefile +1 -1
@@ -107,7 +107,7 @@ extra-y += vmlinux.lds obj-$(CONFIG_RELOCATABLE) += reloc_$(BITS).o -obj-$(CONFIG_PPC32) += entry_32.o setup_32.o early_32.o +obj-$(CONFIG_PPC32) += entry_32.o setup_32.o early_32.o syscall_32.o obj-$(CONFIG_PPC64) += dma-iommu.o iommu.o obj-$(CONFIG_KGDB) += kgdb.o obj-$(CONFIG_BOOTX_TEXT) += btext.o
arch/powerpc/kernel/syscall_32.c new file +167 -0
167 lines not on this page · raw message
Signature · 1 line
2.22.0
Quoted 2 lines · Nicholas Piggin
So... this actually seems to work. Haven't booted it, but the compiler seems to do what we want.
From the GCC manual:
After defining a global register variable, for the current compilation
unit:
* If the register is a call-saved register, call ABI is affected: the register will not be restored in function epilogue sequences after the variable has been assigned. Therefore, functions cannot safely return to callers that assume standard ABI.
and
* Accesses to the variable may be optimized as usual and the register remains available for allocation and use in any computations, provided that observable values of the variable are not affected.
This doesn't do what you think, or what you want, or what you think you want ;-)
(And if you make all those regs -ffixed-* you are in for a world of hurt).
Segher
Nicholas Piggin
reviewerSegher Boessenkool's on August 29, 2019 9:51 pm:
Quoted 14 lines · Nicholas Piggin
Quoted 2 lines
So... this actually seems to work. Haven't booted it, but the compiler seems to do what we want.
From the GCC manual:
After defining a global register variable, for the current compilation
unit:* If the register is a call-saved register, call ABI is affected: the register will not be restored in function epilogue sequences after the variable has been assigned. Therefore, functions cannot safely return to callers that assume standard ABI.
Quoted 8 lines
and
* Accesses to the variable may be optimized as usual and the register remains available for allocation and use in any computations, provided that observable values of the variable are not affected.
This doesn't do what you think, or what you want, or what you think you want ;-)
After reading gcc docs from gcc 4 to 9, I think it does.
We want this to apply to all functions in the compilaition unit. It's fine to use the regs temporarily, and so it's fine for called functions in other units to call them (because they will be restored), and we don't want them restored for our caller.
Quoted 1 line
(And if you make all those regs -ffixed-* you are in for a world of hurt).
From the look of it, -ffixed would be a little bit stronger in that it will never allow the register to be used, wheras the global register variable allows it to be allocated and used elsewhere so long as its observable value is not affected. The latter is actually preferred because it's fine for the compiler to use the regs if it wants to. It does not even have to use r15 register when I reference r15 variable, only whatever value it had.
Thanks,
Nick
Nicholas Piggin
reviewerNicholas Piggin's on August 30, 2019 1:49 am:
Quoted 12 lines
Quoted 2 lines
This doesn't do what you think, or what you want, or what you think you want ;-)
After reading gcc docs from gcc 4 to 9, I think it does.
We want this to apply to all functions in the compilaition unit. It's fine to use the regs temporarily, and so it's fine for called functions in other units to call them (because they will be restored), and we don't want them restored for our caller.
Quoted 1 line
(And if you make all those regs -ffixed-* you are in for a world of hurt).
Quoted 7 lines
From the look of it, -ffixed would be a little bit stronger in that it will never allow the register to be used, wheras the global register variable allows it to be allocated and used elsewhere so long as its observable value is not affected. The latter is actually preferred because it's fine for the compiler to use the regs if it wants to. It does not even have to use r15 register when I reference r15 variable, only whatever value it had.
Ah, of course that means all of them need to be inputs to the stmw asm (which makes sense and is documented in gcc-9 doc, which is when the compiler was allowed to allocate them for other things.
Thanks,
Nick