Linux powerpc new system call instruction and ABI
Subsystems (4)
Nicholas Piggin
reviewerThanks to everyone who has given feedback on the proposed new system call instruction and ABI, I think it has reached agreement and the implementation can be merged into Linux.
I have a hacked glibc implementation (that doesn't do all the right HWCAP detection and misses a few things) that I've tested several things including some kernel selftests (involving signals and syscalls) with.
System Call Vectored (scv) ABI
==============================
The scv instruction is introduced with POWER9 / ISA3, it comes with an rfscv counter-part. The benefit of these instructions is performance (trading slower SRR0/1 with faster LR/CTR registers, and entering the kernel with MSR[EE] and MSR[RI] left enabled, which can reduce MSR updates. The scv instruction has 128 levels (not enough to cover the Linux system call space).
Assignment and advertisement
----------------------------
The proposal is to assign scv levels conservatively, and advertise them with HWCAP feature bits as we add support for more.
Linux has not enabled FSCR[SCV] yet, so executing the scv instruction will cause the kernel to log a "SCV facility unavilable" message, and deliver a SIGILL with ILL_ILLOPC to the process. Linux has defined a HWCAP2 bit PPC_FEATURE2_SCV for SCV support, but does not set it.
This change allocates the zero level ('scv 0'), advertised with PPC_FEATURE2_SCV, which will be used to provide normal Linux system calls (equivalent to 'sc').
Attempting to execute scv with other levels will cause a SIGILL to be delivered the same as before, but will not log a "SCV facility unavailable" message (because the processor facility is enabled).
Calling convention
------------------
The proposal is for scv 0 to provide the standard Linux system call ABI with the following differences from sc convention[1]:
- lr is to be volatile across scv calls. This is necessary because the scv instruction clobbers lr. From previous discussion, this should be possible to deal with in GCC clobbers and CFI.
- cr1 and cr5-cr7 are volatile. This matches the C ABI and would allow the kernel system call exit to avoid restoring the volatile cr registers (although we probably still would anyway to avoid information leaks).
- Error handling: The consensus among kernel, glibc, and musl is to move to using negative return values in r3 rather than CR0[SO]=1 to indicate error, which matches most other architectures, and is closer to a function call.
Notes
-----
- r0,r4-r8 are documented as volatile in the ABI, but the kernel patch as submitted currently preserves them. This is to leave room for deciding which way to go with these. Some small benefit was found by preserving them[1] but I'm not convinced it's worth deviating from the C function call ABI just for this. Release code should follow the ABI.
Previous discussions:
https://lists.ozlabs.org/pipermail/linuxppc-dev/2020-April/208691.html
https://lists.ozlabs.org/pipermail/linuxppc-dev/2020-April/209268.html
[1] https://github.com/torvalds/linux/blob/master/Documentation/powerpc/syscall64-abi.rst
[2] https://lists.ozlabs.org/pipermail/linuxppc-dev/2020-April/209263.html
The following patches to add scv support to Linux are posted to
https://lists.ozlabs.org/pipermail/linuxppc-dev/
Nicholas Piggin (2):
powerpc/64s/exception: treat NIA below __end_interrupts as soft-masked powerpc/64s: system call support for scv/rfscv instructions
Thanks,
Nick
Signature · 1 line
2.23.0
Nicholas Piggin
reviewerPATCH 1/2 powerpc/64s/exception: treat NIA below __end_interrupts as soft-masked
The scv instruction causes an interrupt which can enter the kernel with MSR[EE]=1, thus allowing interrupts to hit at any time. These must not be taken as normal interrupts, because they come from MSR[PR]=0 context, and yet the kernel stack is not yet set up and r13 is not set to the PACA).
Treat this as a soft-masked interrupt regardless of the soft masked state. This does not affect behaviour yet, because currently all interrupts are taken with MSR[EE]=0.
--- arch/powerpc/kernel/exceptions-64s.S | 27 ++++++++++++++++++++++++--- 1 file changed, 24 insertions(+), 3 deletions(-)
arch/powerpc/kernel/exceptions-64s.S +24 -3
@@ -508,8 +508,24 @@ DEFINE_FIXED_SYMBOL(\name\()_common_real) .macro __GEN_COMMON_BODY name .if IMASK + .if ! ISTACK + .error "No support for masked interrupt to use custom stack" + .endif + + /* If coming from user, skip soft-mask tests. */ + andi. r10,r12,MSR_PR + bne 2f + + /* Kernel code running below __end_interrupts is implicitly + * soft-masked */ + LOAD_HANDLER(r10, __end_interrupts) + cmpld r11,r10 + li r10,IMASK + blt- 1f + + /* Test the soft mask state against our interrupt's bit */ lbz r10,PACAIRQSOFTMASK(r13) - andi. r10,r10,IMASK +1: andi. r10,r10,IMASK /* Associate vector numbers with bits in paca->irq_happened */ .if IVEC == 0x500 || IVEC == 0xea0 li r10,PACA_IRQ_EE
@@ -540,7 +556,7 @@ DEFINE_FIXED_SYMBOL(\name\()_common_real) .if ISTACK andi. r10,r12,MSR_PR /* See if coming from user */ - mr r10,r1 /* Save r1 */ +2: mr r10,r1 /* Save r1 */ subi r1,r1,INT_FRAME_SIZE /* alloc frame on kernel stack */ beq- 100f ld r1,PACAKSAVE(r13) /* kernel stack to use */
@@ -2838,7 +2854,8 @@ masked_interrupt: ld r10,PACA_EXGEN+EX_R10(r13) ld r11,PACA_EXGEN+EX_R11(r13) ld r12,PACA_EXGEN+EX_R12(r13) - /* returns to kernel where r13 must be set up, so don't restore it */ + ld r13,PACA_EXGEN+EX_R13(r13) + /* May return to masked low address where r13 is not set up */ .if \hsrr HRFI_TO_KERNEL .else
@@ -2997,6 +3014,10 @@ EXC_COMMON_BEGIN(ppc64_runlatch_on_trampoline) USE_FIXED_SECTION(virt_trampolines) /* + * All code below __end_interrupts is treated as soft-masked. If + * any code runs here with MSR[EE]=1, it must then cope with pending + * soft interrupt being raised (i.e., by ensuring it is replayed). + * * The __end_interrupts marker must be past the out-of-line (OOL) * handlers, so that they are copied to real address 0x100 when running * a relocatable kernel. This ensures they can be reached from the short
Signature · 1 line
2.23.0
Michael Ellerman
AppliedRe: [PATCH 1/2] powerpc/64s/exception: treat NIA below __end_interrupts as soft-masked
Quoted 9 lines · Nicholas Piggin
The scv instruction causes an interrupt which can enter the kernel with MSR[EE]=1, thus allowing interrupts to hit at any time. These must not be taken as normal interrupts, because they come from MSR[PR]=0 context, and yet the kernel stack is not yet set up and r13 is not set to the PACA).
Treat this as a soft-masked interrupt regardless of the soft masked state. This does not affect behaviour yet, because currently all interrupts are taken with MSR[EE]=0.
Applied to powerpc/next.
[1/2] powerpc/64s/exception: treat NIA below __end_interrupts as soft-masked
https://git.kernel.org/powerpc/c/b2dc2977cba48990df45e0a96150663d4f342700 [2/2] powerpc/64s: system call support for scv/rfscv instructions https://git.kernel.org/powerpc/c/7fa95f9adaee7e5cbb195d3359741120829e488b
cheers
Nicholas Piggin
reviewerPATCH 2/2 powerpc/64s: system call support for scv/rfscv instructions
In reply to Nicholas PigginAdd support for the scv instruction on POWER9 and later CPUs.
For now this implements the zeroth scv vector 'scv 0', as identical to 'sc' system calls, with the exception that lr is not preserved, nor are volatile cr registers, and error is not indicated with CR0[SO], but by returning a negative errno.
rfscv is implemented to return from scv type system calls. It can not be used to return from sc system calls because those are defined to preserve lr.
getpid syscall throughput on POWER9 is improved by 26% (428 to 318 cycles), largely due to reducing mtmsr and mtspr.
--- Documentation/powerpc/syscall64-abi.rst | 42 ++++-- arch/powerpc/include/asm/asm-prototypes.h | 2 +- arch/powerpc/include/asm/exception-64s.h | 6 + arch/powerpc/include/asm/head-64.h | 2 +- arch/powerpc/include/asm/ppc-opcode.h | 2 + arch/powerpc/include/asm/ppc_asm.h | 2 + arch/powerpc/include/asm/ptrace.h | 7 +- arch/powerpc/include/asm/setup.h | 4 +- arch/powerpc/include/asm/sstep.h | 1 + arch/powerpc/kernel/cpu_setup_power.S | 10 +- arch/powerpc/kernel/cputable.c | 3 +- arch/powerpc/kernel/dt_cpu_ftrs.c | 1 + arch/powerpc/kernel/entry_64.S | 171 +++++++++++++++++++++- arch/powerpc/kernel/exceptions-64s.S | 123 +++++++++++++++- arch/powerpc/kernel/process.c | 10 +- arch/powerpc/kernel/setup_64.c | 5 +- arch/powerpc/kernel/signal.c | 19 ++- arch/powerpc/kernel/syscall_64.c | 32 +++- arch/powerpc/lib/sstep.c | 14 ++ arch/powerpc/platforms/pseries/setup.c | 8 +- arch/powerpc/xmon/xmon.c | 1 + 21 files changed, 421 insertions(+), 44 deletions(-)
Documentation/powerpc/syscall64-abi.rst +30 -12
@@ -5,6 +5,15 @@ Power Architecture 64-bit Linux system call ABI syscall ======= +Invocation +---------- +The syscall is made with the sc instruction, and returns with execution +continuing at the instruction following the sc instruction. + +If PPC_FEATURE2_SCV appears in the AT_HWCAP2 ELF auxiliary vector, the +scv 0 instruction is an alternative that may provide better performance, +with some differences to calling sequence. + syscall calling sequence\ [1]_ matches the Power Architecture 64-bit ELF ABI specification C function calling sequence, including register preservation rules, with the following differences.
@@ -12,16 +21,23 @@ rules, with the following differences. .. [1] Some syscalls (typically low-level management functions) may have different calling sequences (e.g., rt_sigreturn). -Parameters and return value ---------------------------- +Parameters +---------- The system call number is specified in r0. There is a maximum of 6 integer parameters to a syscall, passed in r3-r8. -Both a return value and a return error code are returned. cr0.SO is the return -error code, and r3 is the return value or error code. When cr0.SO is clear, -the syscall succeeded and r3 is the return value. When cr0.SO is set, the -syscall failed and r3 is the error code that generally corresponds to errno. +Return value +------------ +- For the sc instruction, both a value and an error condition are returned. + cr0.SO is the error condition, and r3 is the return value. When cr0.SO is + clear, the syscall succeeded and r3 is the return value. When cr0.SO is set, + the syscall failed and r3 is the error value (that normally corresponds to + errno). + +- For the scv 0 instruction, the return value indicates failure if it is + -4095..-1 (i.e., it is >= -MAX_ERRNO (-4095) as an unsigned comparison), + in which case the error value is the negated return value. Stack -----
@@ -34,22 +50,23 @@ Register preservation rules match the ELF ABI calling sequence with the following differences: =========== ============= ======================================== +--- For the sc instruction, differences with the ELF ABI --- r0 Volatile (System call number.) r3 Volatile (Parameter 1, and return value.) r4-r8 Volatile (Parameters 2-6.) -cr0 Volatile (cr0.SO is the return error condition) +cr0 Volatile (cr0.SO is the return error condition.) cr1, cr5-7 Nonvolatile lr Nonvolatile + +--- For the scv 0 instruction, differences with the ELF ABI --- +r0 Volatile (System call number.) +r3 Volatile (Parameter 1, and return value.) +r4-r8 Volatile (Parameters 2-6.) =========== ============= ======================================== All floating point and vector data registers as well as control and status registers are nonvolatile. -Invocation ----------- -The syscall is performed with the sc instruction, and returns with execution -continuing at the instruction following the sc instruction. - Transactional Memory -------------------- Syscall behavior can change if the processor is in transactional or suspended
@@ -75,6 +92,7 @@ auxiliary vector. returning to the caller. This case is not well defined or supported, so this behavior should not be relied upon. +scv 0 syscalls will always behave as PPC_FEATURE2_HTM_NOSC. vsyscall ========
arch/powerpc/include/asm/asm-prototypes.h +1 -1
@@ -98,7 +98,7 @@ unsigned long __init early_init(unsigned long dt_ptr);void __init machine_init (u64 dt_ptr);#endif long system_call_exception (long r3,long r4,long r5,long r6,long r7,long r8,unsigned long r0,struct pt_regs *regs); -notrace unsigned long syscall_exit_prepare (unsigned long r3,struct pt_regs *regs); +notrace unsigned long syscall_exit_prepare (unsigned long r3,struct pt_regs *regs,long scv);notrace unsigned long interrupt_exit_user_prepare (struct pt_regs *regs,unsigned long msr);notrace unsigned long interrupt_exit_kernel_prepare (struct pt_regs *regs,unsigned long msr);
arch/powerpc/include/asm/exception-64s.h +6 -0
@@ -123,6 +123,12 @@hrfid ; \ b hrfi_flush_fallback +#define RFSCV_TO_USER \ + STF_EXIT_BARRIER_SLOT; \ + RFI_FLUSH_SLOT; \ + RFSCV; \ + b rfscv_flush_fallback +#endif /* __ASSEMBLY__ */ #endif /* _ASM_POWERPC_EXCEPTION_H */
arch/powerpc/include/asm/head-64.h +1 -1
@@ -128,7 +128,7 @@ end_##sname: .if ((start) % (size) != 0); \ .error "Fixed section exception vector misalignment"; \ .endif; \ - .if ((size) != 0x20) && ((size) != 0x80) && ((size) != 0x100); \ + .if ((size) != 0x20) && ((size) != 0x80) && ((size) != 0x100) && ((size) != 0x1000); \ .error "Fixed section exception vector bad size"; \ .endif; \ .if (start) < sname##_start; \
arch/powerpc/include/asm/ppc-opcode.h +2 -0
@@ -257,6 +257,7 @@#define PPC_INST_MFVSRD 0x7c000066#define PPC_INST_MTVSRD 0x7c000166#define PPC_INST_SC 0x44000002 +#define PPC_INST_SCV 0x44000001#define PPC_INST_SLBFEE 0x7c0007a7#define PPC_INST_SLBIA 0x7c0003e4
@@ -411,6 +412,7 @@#define __PPC_CT (t) (((t) & 0x0f) << 21)#define __PPC_SPR (r) ((((r) & 0x1f) << 16) | ((((r) >> 5) & 0x1f) << 11))#define __PPC_RC21 (0x1 << 10) +#define __PPC_LEV (l) (((l) & 0x7f) << 5)/* * Both low and high 16 bits are added as SIGNED additions, so if low 16 bits
arch/powerpc/include/asm/ppc_asm.h +2 -0
@@ -755,6 +755,8 @@ END_FTR_SECTION_NESTED(CPU_FTR_CELL_TB_BUG, CPU_FTR_CELL_TB_BUG, 96)#define N_SLINE 68#define N_SO 100 +#define RFSCV .long 0x4c0000a4 +/* * Create an endian fixup trampoline *
arch/powerpc/include/asm/ptrace.h +6 -1
@@ -222,9 +222,14 @@ static inline void set_trap(struct pt_regs *regs, unsigned long val) regs->trap = (regs->trap &TRAP_FLAGS_MASK ) | (val & ~TRAP_FLAGS_MASK ); } +static inline bool trap_is_scv (struct pt_regs *regs) +{ +return (IS_ENABLED (CONFIG_PPC_BOOK3S_64 ) &&TRAP (regs) ==0x3000 ); +} +static inline bool trap_is_syscall (struct pt_regs *regs) { -return TRAP (regs) ==0xc00 ; +return (trap_is_scv (regs) ||TRAP (regs) ==0xc00 ); }static inline bool trap_norestart (struct pt_regs *regs)
arch/powerpc/include/asm/setup.h +2 -2
@@ -30,12 +30,12 @@ void setup_panic(void);#define ARCH_PANIC_TIMEOUT 180#ifdef CONFIG_PPC_PSERIES -extern void pseries_enable_reloc_on_exc (void ); +extern bool pseries_enable_reloc_on_exc (void );extern void pseries_disable_reloc_on_exc (void );extern void pseries_big_endian_exceptions (void );extern void pseries_little_endian_exceptions (void );#else -static inline void pseries_enable_reloc_on_exc (void ) {} +static inline bool pseries_enable_reloc_on_exc (void ) {return false; }static inline void pseries_disable_reloc_on_exc (void ) {}static inline void pseries_big_endian_exceptions (void ) {}static inline void pseries_little_endian_exceptions (void ) {}
arch/powerpc/include/asm/sstep.h +1 -0
@@ -40,6 +40,7 @@ enum instruction_type {CACHEOP ,BARRIER ,SYSCALL , +SYSCALL_VECTORED_0 ,MFMSR ,MTMSR , RFI,
arch/powerpc/kernel/cpu_setup_power.S +8 -2
@@ -98,7 +98,7 @@ _GLOBAL(__setup_cpu_power10) _GLOBAL(__setup_cpu_power9) mflr r11 - bl __init_FSCR + bl __init_FSCR_power9 1: bl __init_PMU bl __init_hvmode_206 mtlr r11
@@ -128,7 +128,7 @@ _GLOBAL(__restore_cpu_power10) _GLOBAL(__restore_cpu_power9) mflr r11 - bl __init_FSCR + bl __init_FSCR_power9 1: bl __init_PMU mfmsr r3 rldicl. r0,r3,4,63
@@ -198,6 +198,12 @@ __init_FSCR_power10: mtspr SPRN_FSCR, r3 // fall through +__init_FSCR_power9: + mfspr r3, SPRN_FSCR + ori r3, r3, FSCR_SCV + mtspr SPRN_FSCR, r3 + // fall through + __init_FSCR: mfspr r3,SPRN_FSCR ori r3,r3,FSCR_TAR|FSCR_EBB
arch/powerpc/kernel/cputable.c +2 -1
@@ -120,7 +120,8 @@ extern void __restore_cpu_e6500(void);#define COMMON_USER2_POWER9 (COMMON_USER2_POWER8 | \ PPC_FEATURE2_ARCH_3_00 | \ PPC_FEATURE2_HAS_IEEE128 | \ - PPC_FEATURE2_DARN ) + PPC_FEATURE2_DARN | \ + PPC_FEATURE2_SCV)#define COMMON_USER_POWER10 COMMON_USER_POWER9#define COMMON_USER2_POWER10 (COMMON_USER2_POWER9 | \ PPC_FEATURE2_ARCH_3_1 | \
arch/powerpc/kernel/dt_cpu_ftrs.c +1 -0
@@ -587,6 +587,7 @@ static struct dt_cpu_feature_match __initdata {"little-endian", feat_enable_le, CPU_FTR_REAL_LE}, {"smt", feat_enable_smt, 0}, {"interrupt-facilities", feat_enable, 0}, + {"system-call-vectored", feat_enable, 0}, {"timer-facilities", feat_enable, 0}, {"timer-facilities-v3", feat_enable, 0}, {"debug-facilities", feat_enable, 0},
arch/powerpc/kernel/entry_64.S +169 -2
200 lines not on this page · raw message
arch/powerpc/kernel/exceptions-64s.S +119 -4
153 lines not on this page · raw message
arch/powerpc/kernel/process.c +8 -2
@@ -1596,6 +1596,7 @@ int copy_thread_tls(unsigned long clone_flags, unsigned long usp, {struct pt_regs *childregs, *kregs;extern void ret_from_fork (void ); +extern void ret_from_fork_scv (void );extern void ret_from_kernel_thread (void );void (*f)(void );unsigned long sp = (unsigned long )task_stack_page (p) +THREAD_SIZE ;
@@ -1632,7 +1633,9 @@ int copy_thread_tls(unsigned long clone_flags, unsigned long usp,if (usp) childregs->gpr[1 ] = usp; p->thread.regs = childregs; - childregs->gpr[3 ] =0 ;/* Result from fork() */ +/* 64s sets this in ret_from_fork */ +if (!IS_ENABLED (CONFIG_PPC_BOOK3S_64 )) + childregs->gpr[3 ] =0 ;/* Result from fork() */ if (clone_flags &CLONE_SETTLS ) {if (!is_32bit_task ()) childregs->gpr[13 ] = tls;
@@ -1640,7 +1643,10 @@ int copy_thread_tls(unsigned long clone_flags, unsigned long usp, childregs->gpr[2 ] = tls; } - f = ret_from_fork; +if (trap_is_scv (regs)) + f = ret_from_fork_scv; +else + f = ret_from_fork; } childregs->msr &= ~(MSR_FP |MSR_VEC |MSR_VSX ); sp -=STACK_FRAME_OVERHEAD ;
arch/powerpc/kernel/setup_64.c +4 -1
@@ -196,7 +196,10 @@ static void __init configure_exceptions(void)/* Under a PAPR hypervisor, we need hypercalls */ if (firmware_has_feature (FW_FEATURE_SET_MODE )) {/* Enable AIL if possible */ -pseries_enable_reloc_on_exc (); +if (!pseries_enable_reloc_on_exc ()) { + init_task.thread.fscr &= ~FSCR_SCV ; + cur_cpu_spec->cpu_user_features2 &= ~PPC_FEATURE2_SCV ; + }/* * Tell the hypervisor that we want our exceptions to
arch/powerpc/kernel/signal.c +15 -4
@@ -205,8 +205,14 @@ static void check_syscall_restart(struct pt_regs *regs, struct k_sigaction *ka,return ;/* error signalled ? */ -if (!(regs->ccr &0x10000000 )) +if (trap_is_scv (regs)) { +/* 32-bit compat mode sign extend? */ +if (!IS_ERR_VALUE (ret)) +return ; + ret = -ret; + }else if (!(regs->ccr &0x10000000 )) {return ; + }switch (ret) {case ERESTART_RESTARTBLOCK :
@@ -239,9 +245,14 @@ static void check_syscall_restart(struct pt_regs *regs, struct k_sigaction *ka, regs->nip -=4 ; regs->result =0 ; }else { - regs->result = -EINTR ; - regs->gpr[3 ] =EINTR ; - regs->ccr |=0x10000000 ; +if (trap_is_scv (regs)) { + regs->result = -EINTR ; + regs->gpr[3 ] = -EINTR ; + }else { + regs->result = -EINTR ; + regs->gpr[3 ] =EINTR ; + regs->ccr |=0x10000000 ; + } } }
arch/powerpc/kernel/syscall_64.c +24 -8
@@ -60,6 +60,11 @@ notrace long system_call_exception(long r3, long r4, long r5,local_irq_enable ();if (unlikely (current_thread_info ()->flags & _TIF_SYSCALL_DOTRACE)) { +if (unlikely (regs->trap ==0x7ff0 )) { +/* Unsupported scv vector */ +_exception (SIGILL , regs,ILL_ILLOPC , regs->nip); +return regs->gpr[3 ]; + }/* * We use the return value of do_syscall_trace_enter() as the * syscall number. If the syscall was rejected for any reason
@@ -78,6 +83,11 @@ notrace long system_call_exception(long r3, long r4, long r5, r8 = regs->gpr[8 ]; }else if (unlikely (r0 >= NR_syscalls)) { +if (unlikely (regs->trap ==0x7ff0 )) { +/* Unsupported scv vector */ +_exception (SIGILL , regs,ILL_ILLOPC , regs->nip); +return regs->gpr[3 ]; + }return -ENOSYS ; }
@@ -105,16 +115,20 @@ notrace long system_call_exception(long r3, long r4, long r5,* local irqs must be disabled. Returns false if the caller must re-enable * them, check for new work, and try again. */ -static notrace inline bool prep_irq_for_enabled_exit(void) +static notrace inline bool prep_irq_for_enabled_exit(bool clear_ri) {/* This must be done with RI=1 because tracing may touch vmaps */ trace_hardirqs_on ();/* This pattern matches prep_irq_for_idle */ -__hard_EE_RI_disable (); +if (clear_ri) +__hard_EE_RI_disable (); +else +__hard_irq_disable ();if (unlikely (lazy_irq_pending_nocheck ())) {/* Took an interrupt, may have more exit work to do. */ -__hard_RI_enable (); +if (clear_ri) +__hard_RI_enable ();trace_hardirqs_off (); local_paca->irq_happened |=PACA_IRQ_HARD_DIS ;
@@ -136,7 +150,8 @@ static notrace inline bool prep_irq_for_enabled_exit(void)* because RI=0 and soft mask state is "unreconciled", so it is marked notrace. */ notrace unsigned long syscall_exit_prepare (unsigned long r3, -struct pt_regs *regs) +struct pt_regs *regs, +long scv) {unsigned long *ti_flagsp = ¤t_thread_info ()->flags;unsigned long ti_flags;
@@ -151,7 +166,7 @@ notrace unsigned long syscall_exit_prepare(unsigned long r3, ti_flags = *ti_flagsp; -if (unlikely (r3 >= (unsigned long )-MAX_ERRNO )) { +if (unlikely (r3 >= (unsigned long )-MAX_ERRNO ) && !scv) {if (likely (!(ti_flags & (_TIF_NOERROR | _TIF_RESTOREALL)))) { r3 = -r3; regs->ccr |=0x10000000 ;/* Set SO bit in CR */
@@ -211,7 +226,8 @@ notrace unsigned long syscall_exit_prepare(unsigned long r3, } } -if (unlikely (!prep_irq_for_enabled_exit ())) { +/* scv need not set RI=0 because SRRs are not used */ +if (unlikely (!prep_irq_for_enabled_exit (!scv))) {local_irq_enable (); goto again; }
@@ -282,7 +298,7 @@ notrace unsigned long interrupt_exit_user_prepare(struct pt_regs *regs, unsigned } } -if (unlikely (!prep_irq_for_enabled_exit ())) { +if (unlikely (!prep_irq_for_enabled_exit (true))) {local_irq_enable ();local_irq_disable (); goto again;
@@ -345,7 +361,7 @@ notrace unsigned long interrupt_exit_kernel_prepare(struct pt_regs *regs, unsign } } -if (unlikely (!prep_irq_for_enabled_exit ())) { +if (unlikely (!prep_irq_for_enabled_exit (true))) {/* * Can't local_irq_restore to replay if we were in * interrupt context. Must replay directly.
arch/powerpc/lib/sstep.c +14 -0
@@ -16,6 +16,7 @@#include <asm/disassemble.h> extern char system_call_common[]; +extern char system_call_vectored_emulate[];#ifdef CONFIG_PPC64 /* Bits in SRR1 that are copied from MSR */
@@ -1236,6 +1237,9 @@ int analyse_instr(struct instruction_op *op, const struct pt_regs *regs,case 17 :/* sc */ if ((word &0xfe2 ) ==2 ) op->type =SYSCALL ; +else if (IS_ENABLED (CONFIG_PPC_BOOK3S_64 ) && + (word &0xfe3 ) ==1 ) + op->type =SYSCALL_VECTORED_0 ;else op->type =UNKNOWN ;return 0 ;
@@ -3378,6 +3382,16 @@ int emulate_step(struct pt_regs *regs, struct ppc_inst instr) regs->msr =MSR_KERNEL ;return 1 ; +case SYSCALL_VECTORED_0 :/* scv 0 */ + regs->gpr[9 ] = regs->gpr[13 ]; + regs->gpr[10 ] =MSR_KERNEL ; + regs->gpr[11 ] = regs->nip +4 ; + regs->gpr[12 ] = regs->msr &MSR_MASK ; + regs->gpr[13 ] = (unsigned long )get_paca (); + regs->nip = (unsigned long ) &system_call_vectored_emulate; + regs->msr =MSR_KERNEL ; +return 1 ; +case RFI :return -1 ;#endif
arch/powerpc/platforms/pseries/setup.c +5 -3
@@ -358,7 +358,7 @@ static void pseries_lpar_idle(void)* to ever be a problem in practice we can move this into a kernel thread to * finish off the process later in boot. */ -void pseries_enable_reloc_on_exc (void ) +bool pseries_enable_reloc_on_exc (void ) {long rc;unsigned int delay, total_delay =0 ;
@@ -369,11 +369,13 @@ void pseries_enable_reloc_on_exc(void)if (rc ==H_P2 ) {pr_info ("Relocation on exceptions not" " supported\n" ); +return false; }else if (rc !=H_SUCCESS ) {pr_warn ("Unable to enable relocation" " on exceptions: %ld\n" , rc); +return false; } -break ; +return true; } delay =get_longbusy_msecs (rc);
@@ -382,7 +384,7 @@ void pseries_enable_reloc_on_exc(void)pr_warn ("Warning: Giving up waiting to enable " "relocation on exceptions (%u msec)!\n" , total_delay); -return ; +return false; }mdelay (delay);
arch/powerpc/xmon/xmon.c +1 -0
@@ -1593,6 +1593,7 @@ const char *getvecname(unsigned long vec)case 0x1300 : ret ="(Instruction Breakpoint)" ;break ;case 0x1500 : ret ="(Denormalisation)" ;break ;case 0x1700 : ret ="(Altivec Assist)" ;break ; +case 0x3000 : ret ="(System Call Vectored)" ;break ;default : ret ="" ; }return ret;
Signature · 1 line
2.23.0
Michael Ellerman
reviewerRe: [PATCH 2/2] powerpc/64s: system call support for scv/rfscv instructions
Quoted 9 lines · Nicholas Piggin
arch/powerpc/include/asm/ppc-opcode.h +1
-0@@ -257,6 +257,7 @@#define PPC_INST_MFVSRD 0x7c000066#define PPC_INST_MTVSRD 0x7c000166#define PPC_INST_SC 0x44000002 +#define PPC_INST_SCV 0x44000001
...
Quoted 1 line
arch/powerpc/include/asm/ppc-opcode.h +0
-0@@ -411,6 +412,7 @@
...
Quoted 1 line
+#define
__PPC_LEV(l)(((l) &0x7f) << 5)
These conflicted and didn't seem to be used so I dropped them.
Quoted 8 lines
arch/powerpc/lib/sstep.c +0
-0@@ -3378,6 +3382,16 @@ int emulate_step(struct pt_regs *regs, struct ppc_inst instr) regs->msr =MSR_KERNEL ;return 1 ;
arch/powerpc/lib/sstep.c int emulate_step(struct pt_regs *regs, struct ppc_inst instr) · line 3394
Quoted 10 lines
arch/powerpc/lib/sstep.c +10
-0+case SYSCALL_VECTORED_0 :/* scv 0 */ + regs->gpr[9 ] = regs->gpr[13 ]; + regs->gpr[10 ] =MSR_KERNEL ; + regs->gpr[11 ] = regs->nip +4 ; + regs->gpr[12 ] = regs->msr &MSR_MASK ; + regs->gpr[13 ] = (unsigned long )get_paca (); + regs->nip = (unsigned long ) &system_call_vectored_emulate; + regs->msr =MSR_KERNEL ; +return 1 ; +
This broke the ppc64e build:
ld: arch/powerpc/lib/sstep.o:(.toc+0x0): undefined reference to `system_call_vectored_emulate' make[1]: *** [/home/michael/linux/Makefile:1139: vmlinux] Error 1
I wrapped it in #ifdef CONFIG_PPC64_BOOK3S.
cheers
Re: [PATCH 2/2] powerpc/64s: system call support for scv/rfscv instructions
Michael Ellerman <[email protected]> a écrit :
Quoted 39 lines
Quoted 9 lines · Nicholas Piggin
arch/powerpc/include/asm/ppc-opcode.h +1
-0@@ -257,6 +257,7 @@#define PPC_INST_MFVSRD 0x7c000066#define PPC_INST_MTVSRD 0x7c000166#define PPC_INST_SC 0x44000002 +#define PPC_INST_SCV 0x44000001...
Quoted 1 line
arch/powerpc/include/asm/ppc-opcode.h +0
-0@@ -411,6 +412,7 @@...
Quoted 1 line
+#define
__PPC_LEV(l)(((l) &0x7f) << 5)These conflicted and didn't seem to be used so I dropped them.
Quoted 9 lines
arch/powerpc/lib/sstep.c +0
-0@@ -3378,6 +3382,16 @@ int emulate_step(struct pt_regs *regs, struct ppc_inst instr) regs->msr = MSR_KERNEL;return 1 ;arch/powerpc/lib/sstep.c int emulate_step(struct pt_regs *regs, · line 3395
Quoted 10 lines
arch/powerpc/lib/sstep.c +10
-0+case SYSCALL_VECTORED_0 :/* scv 0 */ + regs->gpr[9 ] = regs->gpr[13 ]; + regs->gpr[10 ] =MSR_KERNEL ; + regs->gpr[11 ] = regs->nip +4 ; + regs->gpr[12 ] = regs->msr &MSR_MASK ; + regs->gpr[13 ] = (unsigned long )get_paca (); + regs->nip = (unsigned long ) &system_call_vectored_emulate; + regs->msr =MSR_KERNEL ; +return 1 ; +This broke the ppc64e build:
Quoted 5 lines
ld: arch/powerpc/lib/sstep.o:(.toc+0x0): undefined reference to
`system_call_vectored_emulate'
make[1]: *** [/home/michael/linux/Makefile:1139: vmlinux] Error 1I wrapped it in #ifdef
CONFIG_PPC64_BOOK3S.
You mean CONFIG_PPC_BOOK3S_64 ?
Christophe
Quoted 2 lines
cheers
Michael Ellerman
reviewerRe: [PATCH 2/2] powerpc/64s: system call support for scv/rfscv instructions
Quoted 42 lines · Christophe Leroy
Michael Ellerman <[email protected]> a écrit :
Quoted 40 lines
Quoted 10 lines · Nicholas Piggin
arch/powerpc/include/asm/ppc-opcode.h +1
-0@@ -257,6 +257,7 @@#define PPC_INST_MFVSRD 0x7c000066#define PPC_INST_MTVSRD 0x7c000166#define PPC_INST_SC 0x44000002 +#define PPC_INST_SCV 0x44000001...
Quoted 1 line
arch/powerpc/include/asm/ppc-opcode.h +0
-0@@ -411,6 +412,7 @@...
Quoted 1 line
+#define
__PPC_LEV(l)(((l) &0x7f) << 5)These conflicted and didn't seem to be used so I dropped them.
Quoted 9 lines
arch/powerpc/lib/sstep.c +0
-0@@ -3378,6 +3382,16 @@ int emulate_step(struct pt_regs *regs, struct ppc_inst instr) regs->msr = MSR_KERNEL;return 1 ;arch/powerpc/lib/sstep.c int emulate_step(struct pt_regs *regs, · line 3395
Quoted 10 lines
arch/powerpc/lib/sstep.c +10
-0+case SYSCALL_VECTORED_0 :/* scv 0 */ + regs->gpr[9 ] = regs->gpr[13 ]; + regs->gpr[10 ] =MSR_KERNEL ; + regs->gpr[11 ] = regs->nip +4 ; + regs->gpr[12 ] = regs->msr &MSR_MASK ; + regs->gpr[13 ] = (unsigned long )get_paca (); + regs->nip = (unsigned long ) &system_call_vectored_emulate; + regs->msr =MSR_KERNEL ; +return 1 ; +This broke the ppc64e build:
Quoted 7 lines
Quoted 5 lines
ld: arch/powerpc/lib/sstep.o:(.toc+0x0): undefined reference to
`system_call_vectored_emulate'
make[1]: *** [/home/michael/linux/Makefile:1139: vmlinux] Error 1I wrapped it in #ifdef
CONFIG_PPC64_BOOK3S.You mean
CONFIG_PPC_BOOK3S_64?
I hope so ...
#### ## ####.
Will send a fixup. Thanks for noticing.
cheers
Hi!
Quoted 9 lines · Nicholas Piggin
Calling convention
------------------
The proposal is for scv 0 to provide the standard Linux system call ABI with the following differences from sc convention[1]:- lr is to be volatile across scv calls. This is necessary because the scv instruction clobbers lr. From previous discussion, this should be possible to deal with in GCC clobbers and CFI.
Quoted 7 lines
- cr1 and cr5-cr7 are volatile. This matches the C ABI and would allow the kernel system call exit to avoid restoring the volatile cr registers (although we probably still would anyway to avoid information leaks).
- Error handling: The consensus among kernel, glibc, and musl is to move to using negative return values in r3 rather than CR0[SO]=1 to indicate error, which matches most other architectures, and is closer to a function call.
What about cr0 then? Will it be volatile as well (exactly like for function calls)?
Quoted 5 lines
Notes
-----
- r0,r4-r8 are documented as volatile in the ABI, but the kernel patch as submitted currently preserves them. This is to leave room for deciding which way to go with these.
The kernel has to set it to *something* that doesn't leak information ;-)
Segher
Nicholas Piggin
reviewerExcerpts from Segher Boessenkool's message of June 12, 2020 7:02 am:
Quoted 16 lines
Hi!
Quoted 13 lines · Nicholas Piggin
Calling convention
------------------
The proposal is for scv 0 to provide the standard Linux system call ABI with the following differences from sc convention[1]:- lr is to be volatile across scv calls. This is necessary because the scv instruction clobbers lr. From previous discussion, this should be possible to deal with in GCC clobbers and CFI.
- cr1 and cr5-cr7 are volatile. This matches the C ABI and would allow the kernel system call exit to avoid restoring the volatile cr registers (although we probably still would anyway to avoid information leaks).
Quoted 6 lines
Quoted 3 lines
- Error handling: The consensus among kernel, glibc, and musl is to move to using negative return values in r3 rather than CR0[SO]=1 to indicate error, which matches most other architectures, and is closer to a function call.
What about cr0 then? Will it be volatile as well (exactly like for function calls)?
Yes, same as for sc (except for SO bit). Which is a bit unclear in this section.
Quoted 7 lines
Quoted 5 lines
Notes
-----
- r0,r4-r8 are documented as volatile in the ABI, but the kernel patch as submitted currently preserves them. This is to leave room for deciding which way to go with these.The kernel has to set it to *something* that doesn't leak information ;-)
For "sc" system calls these were defined as volatile (and used to just leak information), so now we just zero them.
Thanks,
Nick
Hi,
On Thu, Jun 11, 2020 at 06:12:01PM +1000, Nicholas Piggin wrote:
[...]
Quoted 3 lines
- Error handling: The consensus among kernel, glibc, and musl is to move to using negative return values in r3 rather than CR0[SO]=1 to indicate error, which matches most other architectures, and is closer to a function call.
Apparently, the patchset merged by commit v5.9-rc1~100^2~164 was
incomplete: all functions defined in arch/powerpc/include/asm/ptrace.h and arch/powerpc/include/asm/syscall.h that use ccr are broken when scv is used. This includes syscall_get_error() and all its users including PTRACE_GET_SYSCALL_INFO API, which in turn makes strace unusable when scv is used.
See also https://bugzilla.redhat.com/1929836
Signature · 1 line
ldv
Nicholas Piggin
reviewerExcerpts from Dmitry V. Levin's message of May 19, 2021 9:13 am:
Quoted 8 lines
Hi,
On Thu, Jun 11, 2020 at 06:12:01PM +1000, Nicholas Piggin wrote:
[...]Quoted 3 lines
- Error handling: The consensus among kernel, glibc, and musl is to move to using negative return values in r3 rather than CR0[SO]=1 to indicate error, which matches most other architectures, and is closer to a function call.
Quoted 8 lines
Apparently, the patchset merged by commit v5.9-rc1~100^2~164 was
incomplete: all functions defined inarch/powerpc/include/asm/ptrace.handarch/powerpc/include/asm/syscall.hthat use ccr are broken when scv is used. This includessyscall_get_error()and all its users includingPTRACE_GET_SYSCALL_INFOAPI, which in turn makes strace unusable when scv is used.See also https://bugzilla.redhat.com/1929836
I see, thanks. Using latest strace from github.com, the attached kernel patch makes strace -k check results a lot greener.
Some of the remaining failing tests look like this (I didn't look at all of them yet):
signal(SIGUSR1, 0xfacefeeddeadbeef) = 0 (SIG_DFL)
write(1, "signal(SIGUSR1, 0xfacefeeddeadbe"..., 50signal(SIGUSR1, 0xfacefeeddeadbeef) = 0 (SIG_DFL) ) = 50signal(SIGUSR1, SIG_IGN) = 0xfacefeeddeadbeefwrite(2, "errno2name.c:461: unknown errno "..., 41errno2name.c:461: unknown errno 559038737) = 41 write(2, ": Unknown error 559038737\n", 26: Unknown error 559038737
) = 26exit_group(1) = ?
I think the problem is glibc testing for -ve, but it should be comparing against -4095 (+cc Matheus)
#define RET_SCV \
cmpdi r3,0; \
bgelr+; \
neg r3,r3;With this patch, I think the ptrace ABI should mostly be fixed. I think a problem remains with applications that look at system call return registers directly and have powerpc specific error cases. Those probably will just need to be updated unfortunately. Michael thought it might be possible to return an indication via ptrace somehow that the syscall is using a new ABI, so such apps can be updated to test for it. I don't know how that would be done.
Thanks,
Nick
--
arch/powerpc/include/asm/ptrace.h +26 -19
@@ -19,6 +19,7 @@#ifndef _ASM_POWERPC_PTRACE_H#define _ASM_POWERPC_PTRACE_H +#include <linux/err.h> #include <uapi/asm/ptrace.h> #include <asm/asm-const.h>
@@ -152,25 +153,6 @@ extern unsigned long profile_pc(struct pt_regs *regs);long do_syscall_trace_enter (struct pt_regs *regs);void do_syscall_trace_leave (struct pt_regs *regs); -#define kernel_stack_pointer (regs) ((regs)->gpr[1]) -static inline int is_syscall_success (struct pt_regs *regs) -{ -return !(regs->ccr &0x10000000 ); -} - -static inline long regs_return_value (struct pt_regs *regs) -{ -if (is_syscall_success (regs)) -return regs->gpr[3 ]; -else -return -regs->gpr[3 ]; -} - -static inline void regs_set_return_value (struct pt_regs *regs,unsigned long rc) -{ - regs->gpr[3 ] = rc; -} -#ifdef __powerpc64__#define user_mode (regs) ((((regs)->msr) >> MSR_PR_LG) & 0x1)#else
@@ -235,6 +217,31 @@ static __always_inline void set_trap_norestart(struct pt_regs *regs) regs->trap |=0x1 ; } +#define kernel_stack_pointer (regs) ((regs)->gpr[1]) +static inline int is_syscall_success (struct pt_regs *regs) +{ +if (trap_is_scv (regs)) +return !IS_ERR_VALUE ((unsigned long )regs->gpr[3 ]); +else +return !(regs->ccr &0x10000000 ); +} + +static inline long regs_return_value (struct pt_regs *regs) +{ +if (trap_is_scv (regs)) +return regs->gpr[3 ]; + +if (is_syscall_success (regs)) +return regs->gpr[3 ]; +else +return -regs->gpr[3 ]; +} + +static inline void regs_set_return_value (struct pt_regs *regs,unsigned long rc) +{ + regs->gpr[3 ] = rc; +} +#define arch_has_single_step () (1)#define arch_has_block_step () (true)#define ARCH_HAS_USER_SINGLE_STEP_REPORT
arch/powerpc/include/asm/syscall.h +29 -16
58 lines not on this page · raw message
arch/powerpc/include/asm/thread_info.h +2 -0
@@ -165,8 +165,10 @@ static inline bool test_thread_local_flags(unsigned int flags)#ifdef CONFIG_COMPAT #define is_32bit_task () (test_thread_flag(TIF_32BIT)) +#define task_is_32bit (task) (test_tsk_thread_flag(task, TIF_32BIT))#else #define is_32bit_task () (IS_ENABLED(CONFIG_PPC32)) +#define task_is_32bit (task) (IS_ENABLED(CONFIG_PPC32))#endif #if defined(CONFIG_PPC64 )
Nicholas Piggin
reviewerExcerpts from Nicholas Piggin's message of May 19, 2021 12:50 pm:
Quoted 33 lines
Excerpts from Dmitry V. Levin's message of May 19, 2021 9:13 am:
Quoted 8 lines
Hi,
On Thu, Jun 11, 2020 at 06:12:01PM +1000, Nicholas Piggin wrote:
[...]Quoted 3 lines
- Error handling: The consensus among kernel, glibc, and musl is to move to using negative return values in r3 rather than CR0[SO]=1 to indicate error, which matches most other architectures, and is closer to a function call.
Quoted 8 lines
Apparently, the patchset merged by commit v5.9-rc1~100^2~164 was
incomplete: all functions defined inarch/powerpc/include/asm/ptrace.handarch/powerpc/include/asm/syscall.hthat use ccr are broken when scv is used. This includessyscall_get_error()and all its users includingPTRACE_GET_SYSCALL_INFOAPI, which in turn makes strace unusable when scv is used.See also https://bugzilla.redhat.com/1929836
I see, thanks. Using latest strace from github.com, the attached kernel patch makes strace -k check results a lot greener.
Some of the remaining failing tests look like this (I didn't look at all of them yet):
signal(SIGUSR1, 0xfacefeeddeadbeef)= 0 (SIG_DFL)
write(1, "signal(SIGUSR1, 0xfacefeeddeadbe"..., 50signal(SIGUSR1, 0xfacefeeddeadbeef) = 0 (SIG_DFL) )= 50signal(SIGUSR1, SIG_IGN)=0xfacefeeddeadbeefwrite(2, "errno2name.c:461: unknown errno "..., 41errno2name.c:461: unknown errno 559038737)= 41 write(2, ": Unknown error 559038737\n", 26: Unknown error 559038737
) = 26exit_group(1)= ?
Quoted 7 lines
I think the problem is glibc testing for -ve, but it should be comparing against -4095 (+cc Matheus)
#define RET_SCV \ cmpdi r3,0; \ bgelr+; \ neg r3,r3;
This glibc patch at least gets that signal test working. Haven't run the full suite yet because of trouble making it work with a local glibc install...
Thanks,
Nick
---
sysdeps/powerpc/powerpc64/sysdep.h +3 -2
@@ -398,8 +398,9 @@ LT_LABELSUFFIX(name,_name_end): ; \#endif #define RET_SCV \ - cmpdi r3,0; \ - bgelr+; \ + li r9,-4095; \ + cmpld r3,r9; \ + bltlr+; \ neg r3,r3;#define RET_SC \
Matheus Castanho
Reviewed-byQuoted 57 lines · Nicholas Piggin
Excerpts from Nicholas Piggin's message of May 19, 2021 12:50 pm:
Quoted 33 lines
Excerpts from Dmitry V. Levin's message of May 19, 2021 9:13 am:
Quoted 8 lines
Hi,
On Thu, Jun 11, 2020 at 06:12:01PM +1000, Nicholas Piggin wrote:
[...]Quoted 3 lines
- Error handling: The consensus among kernel, glibc, and musl is to move to using negative return values in r3 rather than CR0[SO]=1 to indicate error, which matches most other architectures, and is closer to a function call.
Quoted 8 lines
Apparently, the patchset merged by commit v5.9-rc1~100^2~164 was
incomplete: all functions defined inarch/powerpc/include/asm/ptrace.handarch/powerpc/include/asm/syscall.hthat use ccr are broken when scv is used. This includessyscall_get_error()and all its users includingPTRACE_GET_SYSCALL_INFOAPI, which in turn makes strace unusable when scv is used.See also https://bugzilla.redhat.com/1929836
I see, thanks. Using latest strace from github.com, the attached kernel patch makes strace -k check results a lot greener.
Some of the remaining failing tests look like this (I didn't look at all of them yet):
signal(SIGUSR1, 0xfacefeeddeadbeef)= 0 (SIG_DFL)
write(1, "signal(SIGUSR1, 0xfacefeeddeadbe"..., 50signal(SIGUSR1, 0xfacefeeddeadbeef) = 0 (SIG_DFL) )= 50signal(SIGUSR1, SIG_IGN)=0xfacefeeddeadbeefwrite(2, "errno2name.c:461: unknown errno "..., 41errno2name.c:461: unknown errno 559038737)= 41 write(2, ": Unknown error 559038737\n", 26: Unknown error 559038737
) = 26exit_group(1)= ?Quoted 7 lines
I think the problem is glibc testing for -ve, but it should be comparing against -4095 (+cc Matheus)
#define RET_SCV \ cmpdi r3,0; \ bgelr+; \ neg r3,r3;This glibc patch at least gets that signal test working. Haven't run the full suite yet because of trouble making it work with a local glibc install...
Thanks,
Nick---
sysdeps/powerpc/powerpc64/sysdep.h +0
-0@@ -398,8 +398,9 @@ LT_LABELSUFFIX(name,_name_end): ; \#endif
sysdeps/powerpc/powerpc64/sysdep.h LT_LABELSUFFIX(name,_name_end): ; \ · line 406
Quoted 10 lines
sysdeps/powerpc/powerpc64/sysdep.h +3
-2#define RET_SCV \ - cmpdi r3,0; \ - bgelr+; \ + li r9,-4095; \ + cmpld r3,r9; \ + bltlr+; \ neg r3,r3;#define RET_SC \
Hi Nick,
I agree the current code is accepting more values as errors than it should. This change looks good to me. All glibc tests are passing with it. I also built strace and checked one of the failing tests against a glibc with your patch:
~/src/strace/tests$ uname -r
5.10.16-1-default
~/src/strace/tests$ /lib64/libc.so.6
GNU C Library (GNU libc) release release version 2.33 (git 9826b03b74). [...]
~/src/strace/tests$ ./signal.gen.test
errno2name.c:461: unknown errno 559038737: Unknown error 559038737 signal.gen.test: failed test: ../signal failed with code 1
~/src/strace/tests$ ./signalsignal(SIGUSR1, SIG_IGN) = 0 (SIG_DFL)signal(SIGUSR1, SIG_DFL) = 0x1 (SIG_IGN) signal(SIGUSR1, 0xfacefeeddeadbeef) = 0 (SIG_DFL) errno2name.c:461: unknown errno 559038737: Unknown error 559038737
Running with glibc containing the patch:
~/src/strace/tests$ ~/build/glibc/testrun.sh ./signal signal(SIGUSR1, SIG_IGN) = 0 (SIG_DFL) signal(SIGUSR1, SIG_DFL) = 0x1 (SIG_IGN) signal(SIGUSR1, 0xfacefeeddeadbeef) = 0 (SIG_DFL) signal(SIGUSR1, SIG_IGN) = 0xfacefeeddeadbeef signal(-559038737, SIG_IGN) = -1 EINVAL (Invalid argument) +++ exited with 0 +++
If the patch below looks OK to you and no one objects, I'll commit it to glibc on Monday.
Thanks,
Matheus Castanho
---
From: Nicholas Piggin <[email protected]>
Subject: [PATCH 1/1] powerpc: Fix handling of scv return error codes
When using scv on templated ASM syscalls, current code interprets any negative return value as error, but the only valid error codes are in the range -4095..-1 according to the ABI.
--- sysdeps/powerpc/powerpc64/sysdep.h | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
sysdeps/powerpc/powerpc64/sysdep.h +3 -2
@@ -398,8 +398,9 @@ LT_LABELSUFFIX(name,_name_end): ; \#endif #define RET_SCV \ - cmpdi r3,0; \ - bgelr+; \ + li r9,-4095; \ + cmpld r3,r9; \ + bltlr+; \ neg r3,r3;#define RET_SC \
--
2.31.1
* Matheus Castanho via Libc-alpha:
Quoted 8 lines
From: Nicholas Piggin <[email protected]>
Subject: [PATCH 1/1] powerpc: Fix handling of scv return error codesWhen using scv on templated ASM syscalls, current code interprets any negative return value as error, but the only valid error codes are in the range -4095..-1 according to the ABI.
Reviewed-by Matheus Castanho <[email protected]>
Please reference bug 27892 in the commit message. I'd also appreciate a backport to the 2.33 release branch (where you need to add NEWS manually to add the bug reference).
Thanks,
Florian
Quoted 9 lines · Florian Weimer
* Matheus Castanho via Libc-alpha:
Quoted 7 lines
From: Nicholas Piggin <[email protected]>
Subject: [PATCH 1/1] powerpc: Fix handling of scv return error codesWhen using scv on templated ASM syscalls, current code interprets any negative return value as error, but the only valid error codes are in the range -4095..-1 according to the ABI.
Quoted 5 lines
Quoted 1 line
Reviewed-by Matheus Castanho <[email protected]>Please reference bug 27892 in the commit message. I'd also appreciate a backport to the 2.33 release branch (where you need to add NEWS manually to add the bug reference).
No problem. [BZ #27892] appended to the commit title. I'll make sure to backport to 2.33 as well.
Thanks
--
Matheus Castanho
Quoted 9 lines · Matheus Castanho
Quoted 8 lines · Florian Weimer
* Matheus Castanho via Libc-alpha:
Quoted 7 lines
From: Nicholas Piggin <[email protected]>
Subject: [PATCH 1/1] powerpc: Fix handling of scv return error codesWhen using scv on templated ASM syscalls, current code interprets any negative return value as error, but the only valid error codes are in the range -4095..-1 according to the ABI.
Quoted 8 lines
Quoted 5 lines
Quoted 1 line
Reviewed-by Matheus Castanho <[email protected]>Please reference bug 27892 in the commit message. I'd also appreciate a backport to the 2.33 release branch (where you need to add NEWS manually to add the bug reference).
No problem. [BZ #27892] appended to the commit title. I'll make sure to backport to 2.33 as well.
Could you also mention in the commit message that the change fixes 'signal.gen.test' strace test where it was observed initially?
Signature · 1 line
ldv
Quoted 16 lines · Dmitry V. Levin
Quoted 15 lines · Matheus Castanho
Quoted 13 lines · Florian Weimer
* Matheus Castanho via Libc-alpha:
Quoted 8 lines
From: Nicholas Piggin <[email protected]>
Subject: [PATCH 1/1] powerpc: Fix handling of scv return error codesWhen using scv on templated ASM syscalls, current code interprets any negative return value as error, but the only valid error codes are in the range -4095..-1 according to the ABI.
Reviewed-by Matheus Castanho <[email protected]>Please reference bug 27892 in the commit message. I'd also appreciate a backport to the 2.33 release branch (where you need to add NEWS manually to add the bug reference).
Quoted 5 lines
Quoted 2 lines
No problem. [BZ #27892] appended to the commit title. I'll make sure to backport to 2.33 as well.
Could you also mention in the commit message that the change fixes 'signal.gen.test' strace test where it was observed initially?
Sure, no problem. I'll commit it later today.
--
Matheus Castanho
Matheus Castanho
AppliedQuoted 18 lines · Matheus Castanho
Quoted 16 lines · Dmitry V. Levin
Quoted 15 lines · Matheus Castanho
Quoted 13 lines · Florian Weimer
* Matheus Castanho via Libc-alpha:
Quoted 8 lines
From: Nicholas Piggin <[email protected]>
Subject: [PATCH 1/1] powerpc: Fix handling of scv return error codesWhen using scv on templated ASM syscalls, current code interprets any negative return value as error, but the only valid error codes are in the range -4095..-1 according to the ABI.
Reviewed-by Matheus Castanho <[email protected]>Please reference bug 27892 in the commit message. I'd also appreciate a backport to the 2.33 release branch (where you need to add NEWS manually to add the bug reference).
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
No problem. [BZ #27892] appended to the commit title. I'll make sure to backport to 2.33 as well.
Could you also mention in the commit message that the change fixes 'signal.gen.test' strace test where it was observed initially?
Sure, no problem. I'll commit it later today.
Since the patch falls into the less-than-15-LOC category and this is Nick's first contribution to glibc, looks like he doesn't need a copyright assignment.
Pushed to master as 7de36744ee1325f35d3fe0ca079dd33c40b12267
Backported to 2.33 via commit 0ef0e6de7fdfa18328b09ba2afb4f0112d4bdab4
Thanks,
Matheus Castanho
On Wed, May 19, 2021 at 12:50:24PM +1000, Nicholas Piggin wrote:
[...]
Quoted 7 lines
With this patch, I think the ptrace ABI should mostly be fixed. I think a problem remains with applications that look at system call return registers directly and have powerpc specific error cases. Those probably will just need to be updated unfortunately. Michael thought it might be possible to return an indication via ptrace somehow that the syscall is using a new ABI, so such apps can be updated to test for it. I don't know how that would be done.
Is there any sane way for these applications to handle the scv case? How can they tell that the scv semantics is being used for the given syscall invocation? Can this information be obtained e.g. from struct pt_regs?
For example, in strace we have the following powerpc-specific code used for syscall tampering:
$ cat src/linux/powerpc/set_error.c /* * Copyright (c) 2016-2021 The strace developers. * All rights reserved. * * SPDX-License-Identifier: LGPL-2.1-or-later */
static int
arch_set_r3_ccr(struct tcb *tcp, const unsigned long r3,
const unsigned long ccr_set, const unsigned long ccr_clear)
{
if (ptrace_syscall_info_is_valid() &&
upeek(tcp, sizeof(long) * PT_CCR, &ppc_regs.ccr))
return -1;
const unsigned long old_ccr = ppc_regs.ccr;
ppc_regs.gpr[3] = r3;
ppc_regs.ccr |= ccr_set;
ppc_regs.ccr &= ~ccr_clear;
if (ppc_regs.ccr != old_ccr &&
upoke(tcp, sizeof(long) * PT_CCR, ppc_regs.ccr))
return -1;
return upoke(tcp, sizeof(long) * (PT_R0 + 3), ppc_regs.gpr[3]);
}static intarch_set_error(struct tcb *tcp)
{
return arch_set_r3_ccr(tcp, tcp->u_error, 0x10000000, 0);
}static intarch_set_success(struct tcb *tcp)
{
return arch_set_r3_ccr(tcp, tcp->u_rval, 0, 0x10000000);
}Signature · 1 line
ldv
Nicholas Piggin
reviewerExcerpts from Dmitry V. Levin's message of May 19, 2021 8:24 pm:
Quoted 10 lines · Nicholas Piggin
[...]
Quoted 7 lines
With this patch, I think the ptrace ABI should mostly be fixed. I think a problem remains with applications that look at system call return registers directly and have powerpc specific error cases. Those probably will just need to be updated unfortunately. Michael thought it might be possible to return an indication via ptrace somehow that the syscall is using a new ABI, so such apps can be updated to test for it. I don't know how that would be done.
Quoted 4 lines
Is there any sane way for these applications to handle the scv case? How can they tell that the scv semantics is being used for the given syscall invocation? Can this information be obtained e.g. from struct
pt_regs?
Not that I know of. Michael suggested there might be a way to add something. ptrace_syscall_info has some pad bytes, could we use one for flags bits and set a bit for "new system call ABI"?
As a more hacky thing you could make a syscall with -1 and see how the error looks, and then assume all syscalls will be the same.
Thanks,
Nick
Quoted 45 lines
For example, in strace we have the following powerpc-specific code used for syscall tampering:
$ cat src/linux/powerpc/set_error.c /* * Copyright (c) 2016-2021 The strace developers. * All rights reserved. * * SPDX-License-Identifier: LGPL-2.1-or-later */static int
arch_set_r3_ccr(struct tcb *tcp, const unsigned long r3,
const unsigned longccr_set, const unsigned longccr_clear){ if (ptrace_syscall_info_is_valid() && upeek(tcp, sizeof(long) * PT_CCR, &ppc_regs.ccr)) return -1; const unsigned long old_ccr = ppc_regs.ccr; ppc_regs.gpr[3] = r3; ppc_regs.ccr |= ccr_set; ppc_regs.ccr &= ~ccr_clear; if (ppc_regs.ccr != old_ccr && upoke(tcp, sizeof(long) * PT_CCR, ppc_regs.ccr)) return -1; return upoke(tcp, sizeof(long) * (PT_R0 + 3), ppc_regs.gpr[3]); }static int
arch_set_error(struct tcb *tcp){ return arch_set_r3_ccr(tcp, tcp->u_error, 0x10000000, 0); }static int
arch_set_success(struct tcb *tcp){ return arch_set_r3_ccr(tcp, tcp->u_rval, 0, 0x10000000); }Signature · 1 line
ldv
Quoted 2 lines · Nicholas Piggin via Libc-alpha
As a more hacky thing you could make a syscall with -1 and see how the error looks, and then assume all syscalls will be the same.
I'm not sure this would work.
Even in glibc, it's expected that early syscalls will use sc while scv is used later in the execution.
Signature · 1 line
Tulio Magno
Quoted 7 lines · Dmitry V. Levin
Hi,
On Thu, Jun 11, 2020 at 06:12:01PM +1000, Nicholas Piggin wrote:
[...]Quoted 3 lines
- Error handling: The consensus among kernel, glibc, and musl is to move to using negative return values in r3 rather than CR0[SO]=1 to indicate error, which matches most other architectures, and is closer to a function call.
What about syscalls like times(2) which can return -1 without it being an error?
Jocke
Nicholas Piggin
reviewerExcerpts from Joakim Tjernlund's message of May 19, 2021 5:33 pm:
Quoted 10 lines · Dmitry V. Levin
Quoted 7 lines
Hi,
On Thu, Jun 11, 2020 at 06:12:01PM +1000, Nicholas Piggin wrote:
[...]Quoted 3 lines
- Error handling: The consensus among kernel, glibc, and musl is to move to using negative return values in r3 rather than CR0[SO]=1 to indicate error, which matches most other architectures, and is closer to a function call.
What about syscalls like times(2) which can return -1 without it being an error?
They do become errors / indistinguishable and have to be dealt with by libc or userspace. Which does follow what most architectures do (all except ia64, mips, sparc, and powerpc actually).
Interesting question though, it should have been noted.
Thanks,
Nick
Quoted 12 lines · Nicholas Piggin
Excerpts from Joakim Tjernlund's message of May 19, 2021 5:33 pm:
Quoted 10 lines · Dmitry V. Levin
Quoted 7 lines
Hi,
On Thu, Jun 11, 2020 at 06:12:01PM +1000, Nicholas Piggin wrote:
[...]Quoted 3 lines
- Error handling: The consensus among kernel, glibc, and musl is to move to using negative return values in r3 rather than CR0[SO]=1 to indicate error, which matches most other architectures, and is closer to a function call.
What about syscalls like times(2) which can return -1 without it being an error?
Quoted 8 lines
They do become errors / indistinguishable and have to be dealt with by libc or userspace. Which does follow what most architectures do (all except ia64, mips, sparc, and powerpc actually).
Interesting question though, it should have been noted.
Thanks,
Nick
I always figured the ppc way was superior. It begs the question if not the other archs should change instead?
Jocke
Nicholas Piggin
reviewerExcerpts from Joakim Tjernlund's message of May 19, 2021 6:08 pm:
Quoted 17 lines · Nicholas Piggin
Quoted 16 lines
Excerpts from Joakim Tjernlund's message of May 19, 2021 5:33 pm:
Quoted 10 lines · Dmitry V. Levin
Quoted 7 lines
Hi,
On Thu, Jun 11, 2020 at 06:12:01PM +1000, Nicholas Piggin wrote:
[...]Quoted 3 lines
- Error handling: The consensus among kernel, glibc, and musl is to move to using negative return values in r3 rather than CR0[SO]=1 to indicate error, which matches most other architectures, and is closer to a function call.
What about syscalls like times(2) which can return -1 without it being an error?
They do become errors / indistinguishable and have to be dealt with by libc or userspace. Which does follow what most architectures do (all except ia64, mips, sparc, and powerpc actually).
Quoted 7 lines
Quoted 4 lines
Interesting question though, it should have been noted.
Thanks,
NickI always figured the ppc way was superior. It begs the question if not the other archs should change instead?
It is superior in some ways, not enough to be worth being different.
Other archs are unlikely to change because it would be painful for not much benefit. New system calls just should be made to not return error numbers. If we ever had a big new version of syscall ABI in Linux, we can always use another scv vector number for it.
Thanks,
Nick
Nicholas Piggin
reviewerExcerpts from Nicholas Piggin's message of May 19, 2021 6:42 pm:
Quoted 26 lines
Excerpts from Joakim Tjernlund's message of May 19, 2021 6:08 pm:
Quoted 17 lines · Nicholas Piggin
Quoted 16 lines
Excerpts from Joakim Tjernlund's message of May 19, 2021 5:33 pm:
Quoted 10 lines · Dmitry V. Levin
Quoted 7 lines
Hi,
On Thu, Jun 11, 2020 at 06:12:01PM +1000, Nicholas Piggin wrote:
[...]Quoted 3 lines
- Error handling: The consensus among kernel, glibc, and musl is to move to using negative return values in r3 rather than CR0[SO]=1 to indicate error, which matches most other architectures, and is closer to a function call.
What about syscalls like times(2) which can return -1 without it being an error?
They do become errors / indistinguishable and have to be dealt with by libc or userspace. Which does follow what most architectures do (all except ia64, mips, sparc, and powerpc actually).
Quoted 7 lines
Quoted 4 lines
Interesting question though, it should have been noted.
Thanks,
NickI always figured the ppc way was superior. It begs the question if not the other archs should change instead?
Quoted 6 lines
It is superior in some ways, not enough to be worth being different.
Other archs are unlikely to change because it would be painful for not much benefit. New system calls just should be made to not return error numbers. If we ever had a big new version of syscall ABI in Linux, we can always use another scv vector number for it.
Or... is it possible at syscall entry to peek the address of the instruction which caused the call and see if that was a scv instruction? That would be about as reliable as possible without having that new flag bit.
Thanks,
Nick
Quoted 11 lines · Nicholas Piggin
Excerpts from Dmitry V. Levin's message of May 19, 2021 8:24 pm:
Quoted 10 lines · Nicholas Piggin
[...]
Quoted 7 lines
With this patch, I think the ptrace ABI should mostly be fixed. I think a problem remains with applications that look at system call return registers directly and have powerpc specific error cases. Those probably will just need to be updated unfortunately. Michael thought it might be possible to return an indication via ptrace somehow that the syscall is using a new ABI, so such apps can be updated to test for it. I don't know how that would be done.
Quoted 8 lines
Quoted 4 lines
Is there any sane way for these applications to handle the scv case? How can they tell that the scv semantics is being used for the given syscall invocation? Can this information be obtained e.g. from struct
pt_regs?Not that I know of. Michael suggested there might be a way to add something.
ptrace_syscall_infohas some pad bytes, could we use one for flags bits and set a bit for "new system call ABI"?
PTRACE_GET_SYSCALL_INFO is an architecture-agnostic API, it hides all architecture-specific details behind struct ptrace_syscall_info which has the same meaning on all architectures. ptrace_syscall_info.exit contains both rval and is_error fields to support every architecture regardless of its syscall ABI.
ptrace_syscall_info.exit is extensible, but every architecture would have to define a method of telling whether the system call follows the "new system call ABI" conventions to export this bit of information.
This essentially means implementing something like
static inline long syscall_get_error_abi(struct task_struct *task, struct pt_regs *regs) for every architecture, and using it along with syscall_get_error
in ptrace_get_syscall_info_exit to initialize the new field in ptrace_syscall_info.exit structure.
Quoted 2 lines
As a more hacky thing you could make a syscall with -1 and see how the error looks, and then assume all syscalls will be the same.
This would be very unreliable because sc and scv are allowed to intermingle, so every syscall invocation can follow any of these two error handling conventions.
Quoted 4 lines
Or... is it possible at syscall entry to peek the address of the instruction which caused the call and see if that was a scv instruction? That would be about as reliable as possible without having that new flag bit.
No other architecture requires peeking into tracee memory just to find out the syscall ABI. This would make powerpc the most ugly architecture for ptracing.
I wonder why can't this information be just exported to the tracer via struct pt_regs?
Signature · 1 line
ldv
Nicholas Piggin
reviewerExcerpts from Dmitry V. Levin's message of May 19, 2021 11:26 pm:
Quoted 27 lines · Nicholas Piggin
Quoted 11 lines
Excerpts from Dmitry V. Levin's message of May 19, 2021 8:24 pm:
Quoted 10 lines · Nicholas Piggin
[...]
Quoted 7 lines
With this patch, I think the ptrace ABI should mostly be fixed. I think a problem remains with applications that look at system call return registers directly and have powerpc specific error cases. Those probably will just need to be updated unfortunately. Michael thought it might be possible to return an indication via ptrace somehow that the syscall is using a new ABI, so such apps can be updated to test for it. I don't know how that would be done.
Quoted 8 lines
Quoted 4 lines
Is there any sane way for these applications to handle the scv case? How can they tell that the scv semantics is being used for the given syscall invocation? Can this information be obtained e.g. from struct
pt_regs?Not that I know of. Michael suggested there might be a way to add something.
ptrace_syscall_infohas some pad bytes, could we use one for flags bits and set a bit for "new system call ABI"?
PTRACE_GET_SYSCALL_INFOis an architecture-agnostic API, it hides all architecture-specific details behind structptrace_syscall_infowhich has the same meaning on all architectures. ptrace_syscall_info.exit contains both rval andis_errorfields to support every architecture regardless of its syscall ABI.
Quoted 3 lines
ptrace_syscall_info.exit is extensible, but every architecture would have to define a method of telling whether the system call follows the "new system call ABI" conventions to export this bit of information.
It's already architecture speicfic if you look at registers of syscall exit state so I don't see a problem with a flag that ppc can use for ABI.
Quoted 6 lines
This essentially means implementing something like
static inline longsyscall_get_error_abi(struct task_struct *task, struct pt_regs *regs)for every architecture, and using it along withsyscall_get_error
inptrace_get_syscall_info_exitto initialize the new field in ptrace_syscall_info.exit structure.
Yes this could work. Other architectures can just use a generic implementation if they don't define their own so that's easy. And in userspace they can continue to ignore the flag.
Quoted 13 lines
Quoted 3 lines
As a more hacky thing you could make a syscall with -1 and see how the error looks, and then assume all syscalls will be the same.
This would be very unreliable because sc and scv are allowed to intermingle, so every syscall invocation can follow any of these two error handling conventions.
Quoted 4 lines
Or... is it possible at syscall entry to peek the address of the instruction which caused the call and see if that was a scv instruction? That would be about as reliable as possible without having that new flag bit.
Quoted 6 lines
No other architecture requires peeking into tracee memory just to find out the syscall ABI. This would make powerpc the most ugly architecture for ptracing.
I wonder why can't this information be just exported to the tracer via struct
pt_regs?
It might be able to, I don't see why that would be superior though.
Where could you put it... I guess it could go in the trap field in a high bit. But could that break things that just test for syscall trap number (and don't care about register ABI)? I'm not sure.
Thanks,
Nick
Quoted 28 lines · Nicholas Piggin
Excerpts from Dmitry V. Levin's message of May 19, 2021 11:26 pm:
Quoted 27 lines · Nicholas Piggin
Quoted 11 lines
Excerpts from Dmitry V. Levin's message of May 19, 2021 8:24 pm:
Quoted 10 lines · Nicholas Piggin
[...]
Quoted 7 lines
With this patch, I think the ptrace ABI should mostly be fixed. I think a problem remains with applications that look at system call return registers directly and have powerpc specific error cases. Those probably will just need to be updated unfortunately. Michael thought it might be possible to return an indication via ptrace somehow that the syscall is using a new ABI, so such apps can be updated to test for it. I don't know how that would be done.
Quoted 8 lines
Quoted 4 lines
Is there any sane way for these applications to handle the scv case? How can they tell that the scv semantics is being used for the given syscall invocation? Can this information be obtained e.g. from struct
pt_regs?Not that I know of. Michael suggested there might be a way to add something.
ptrace_syscall_infohas some pad bytes, could we use one for flags bits and set a bit for "new system call ABI"?
PTRACE_GET_SYSCALL_INFOis an architecture-agnostic API, it hides all architecture-specific details behind structptrace_syscall_infowhich has the same meaning on all architectures. ptrace_syscall_info.exit contains both rval andis_errorfields to support every architecture regardless of its syscall ABI.
Quoted 7 lines
Quoted 3 lines
ptrace_syscall_info.exit is extensible, but every architecture would have to define a method of telling whether the system call follows the "new system call ABI" conventions to export this bit of information.
It's already architecture speicfic if you look at registers of syscall exit state so I don't see a problem with a flag that ppc can use for ABI.
To be honest, I don't see anything architecture-specific in PTRACE_GET_SYSCALL_INFO API. Yes, it's implementation uses various functions defined in asm/syscall.h, but this doesn't make the interface architecture-specific.
PTRACE_GET_SYSCALL_INFO saves its users from necessity to be aware of tracee registers. That's why the only place where strace has to deal with tracee registers nowadays is syscall tampering. The most reliable solution is to introduce PTRACE_SET_SYSCALL_INFO, this would make the whole syscall abi issue irrelevant for ptracers, maybe the time has come to implement it.
Unfortunately, extending ptrace API takes time, and it's not going to be backported to older kernels anyway, but scv-enabled kernels are already in the wild, so we need a quick powerpc-specific fix that would be backported to all maintained scv-enabled kernels.
[...]
Quoted 8 lines
Quoted 2 lines
I wonder why can't this information be just exported to the tracer via struct
pt_regs?It might be able to, I don't see why that would be superior though.
Where could you put it... I guess it could go in the trap field in a high bit. But could that break things that just test for syscall trap number (and don't care about register ABI)? I'm not sure.
Looks like struct pt_regs.trap already contains the information that could be used to tell 'sc' from 'scv': if (pt_regs.trap & ~0xf) == 0x3000, then it's scv. Is my reading of arch/powerpc/include/asm/ptrace.h correct?
Signature · 1 line
ldv
Nicholas Piggin
reviewerExcerpts from Dmitry V. Levin's message of May 20, 2021 9:27 am:
Quoted 33 lines · Nicholas Piggin
Quoted 32 lines
Excerpts from Dmitry V. Levin's message of May 19, 2021 11:26 pm:
Quoted 30 lines · Nicholas Piggin
Quoted 11 lines
Excerpts from Dmitry V. Levin's message of May 19, 2021 8:24 pm:
Quoted 10 lines · Nicholas Piggin
[...]
Quoted 7 lines
With this patch, I think the ptrace ABI should mostly be fixed. I think a problem remains with applications that look at system call return registers directly and have powerpc specific error cases. Those probably will just need to be updated unfortunately. Michael thought it might be possible to return an indication via ptrace somehow that the syscall is using a new ABI, so such apps can be updated to test for it. I don't know how that would be done.
Quoted 8 lines
Quoted 4 lines
Is there any sane way for these applications to handle the scv case? How can they tell that the scv semantics is being used for the given syscall invocation? Can this information be obtained e.g. from struct
pt_regs?Not that I know of. Michael suggested there might be a way to add something.
ptrace_syscall_infohas some pad bytes, could we use one for flags bits and set a bit for "new system call ABI"?
PTRACE_GET_SYSCALL_INFOis an architecture-agnostic API, it hides all architecture-specific details behind structptrace_syscall_infowhich has the same meaning on all architectures. ptrace_syscall_info.exit contains both rval andis_errorfields to support every architecture regardless of its syscall ABI.ptrace_syscall_info.exit is extensible, but every architecture would have to define a method of telling whether the system call follows the "new system call ABI" conventions to export this bit of information.
Quoted 8 lines
Quoted 3 lines
It's already architecture speicfic if you look at registers of syscall exit state so I don't see a problem with a flag that ppc can use for ABI.
To be honest, I don't see anything architecture-specific in
PTRACE_GET_SYSCALL_INFOAPI. Yes, it's implementation uses various functions defined inasm/syscall.h, but this doesn't make the interface architecture-specific.
No. But a field or flag it exports could be architecture dependent. It doesn't detract independence from the rest of the ABI. That said...
Quoted 18 lines
PTRACE_GET_SYSCALL_INFOsaves its users from necessity to be aware of tracee registers. That's why the only place where strace has to deal with tracee registers nowadays is syscall tampering. The most reliable solution is to introducePTRACE_SET_SYSCALL_INFO, this would make the whole syscall abi issue irrelevant for ptracers, maybe the time has come to implement it.Unfortunately, extending ptrace API takes time, and it's not going to be backported to older kernels anyway, but scv-enabled kernels are already in the wild, so we need a quick powerpc-specific fix that would be backported to all maintained scv-enabled kernels.
[...]
Quoted 5 lines
Quoted 2 lines
I wonder why can't this information be just exported to the tracer via struct
pt_regs?It might be able to, I don't see why that would be superior though.
Quoted 7 lines
Quoted 3 lines
Where could you put it... I guess it could go in the trap field in a high bit. But could that break things that just test for syscall trap number (and don't care about register ABI)? I'm not sure.
Looks like struct pt_regs.trap already contains the information that could be used to tell 'sc' from 'scv': if (pt_regs.trap & ~0xf) ==
0x3000, then it's scv. Is my reading ofarch/powerpc/include/asm/ptrace.hcorrect?
Hmm, I think it is. Certainly in the kernel regs struct it is, I had in my mind that we put it to 0xc00 when populating the user struct for compatibility, but it seems not. So I guess this would work.
Thanks,
Nick
On Thu, May 20, 2021 at 12:40:36PM +1000, Nicholas Piggin wrote:
[...]
Quoted 7 lines
Quoted 3 lines
Looks like struct pt_regs.trap already contains the information that could be used to tell 'sc' from 'scv': if (pt_regs.trap & ~0xf) ==
0x3000, then it's scv. Is my reading ofarch/powerpc/include/asm/ptrace.hcorrect?Hmm, I think it is. Certainly in the kernel regs struct it is, I had in my mind that we put it to
0xc00when populating the user struct for compatibility, but it seems not. So I guess this would work.
OK, can we state that (pt_regs.trap & ~0xf) == 0x3000 is a part of the scv ABI, so it's not going to change and could be relied upon by userspace? Could this be documented in Documentation/powerpc/syscall64-abi.rst, please?
Signature · 1 line
ldv
Nicholas Piggin
reviewerExcerpts from Dmitry V. Levin's message of May 20, 2021 1:06 pm:
Quoted 6 lines · Nicholas Piggin
[...]
Quoted 4 lines
Quoted 3 lines
Looks like struct pt_regs.trap already contains the information that could be used to tell 'sc' from 'scv': if (pt_regs.trap & ~0xf) ==
0x3000, then it's scv. Is my reading ofarch/powerpc/include/asm/ptrace.hcorrect?
Quoted 8 lines
Quoted 3 lines
Hmm, I think it is. Certainly in the kernel regs struct it is, I had in my mind that we put it to
0xc00when populating the user struct for compatibility, but it seems not. So I guess this would work.OK, can we state that (pt_regs.trap & ~0xf) ==
0x3000is a part of the scv ABI, so it's not going to change and could be relied upon by userspace? Could this be documented inDocumentation/powerpc/syscall64-abi.rst, please?
Yeah I think we can do that. The kernel doesn't care what is put in the userspace pt_regs.trap too much so if this is your preferred approach then I will document it in the ABI.
Thanks,
Nick
Quoted 5 lines · Nicholas Piggin
Excerpts from Joakim Tjernlund's message of May 19, 2021 6:08 pm:
Quoted 2 lines
I always figured the ppc way was superior. It begs the question if not the other archs should change instead?
It is superior in some ways, not enough to be worth being different.
The PowerPC syscall ABI *requires* using cr0.3 for indicating errors, you will have to do that whether you conflate the concepts of return code and error indicator or not!
Quoted 2 lines
Other archs are unlikely to change because it would be painful for not much benefit.
Other archs cannot easily change for much the same reason :-)
Quoted 2 lines
New system calls just should be made to not return error numbers.
Which sometimes is a difficult / non-natural / clumsy thing to do.
Segher
Quoted 7 lines · Segher Boessenkool
Quoted 5 lines · Nicholas Piggin
Excerpts from Joakim Tjernlund's message of May 19, 2021 6:08 pm:
Quoted 2 lines
I always figured the ppc way was superior. It begs the question if not the other archs should change instead?
It is superior in some ways, not enough to be worth being different.
Quoted 8 lines
The PowerPC syscall ABI *requires* using cr0.3 for indicating errors, you will have to do that whether you conflate the concepts of return code and error indicator or not!
Quoted 2 lines
Other archs are unlikely to change because it would be painful for not much benefit.
Other archs cannot easily change for much the same reason :-)
Really? I figured you could just add extra error indication in kernel syscall I/F. Eventually user space could migrate to the new indication.
Jocke
Quoted 12 lines · Joakim Tjernlund
Quoted 11 lines · Segher Boessenkool
Quoted 5 lines · Nicholas Piggin
Excerpts from Joakim Tjernlund's message of May 19, 2021 6:08 pm:
Quoted 2 lines
I always figured the ppc way was superior. It begs the question if not the other archs should change instead?
It is superior in some ways, not enough to be worth being different.
The PowerPC syscall ABI *requires* using cr0.3 for indicating errors, you will have to do that whether you conflate the concepts of return code and error indicator or not!
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
Other archs are unlikely to change because it would be painful for not much benefit.
Other archs cannot easily change for much the same reason :-)
Really? I figured you could just add extra error indication in kernel syscall I/F. Eventually user space could migrate to the new indication.
You seem to assume all user space uses glibc, or *any* libc even? This is false. Some programs do system calls directly. Do not break the kernel ABI :-)
Segher
Quoted 16 lines · Segher Boessenkool
Quoted 15 lines · Joakim Tjernlund
Quoted 14 lines · Segher Boessenkool
Quoted 5 lines · Nicholas Piggin
Excerpts from Joakim Tjernlund's message of May 19, 2021 6:08 pm:
Quoted 2 lines
I always figured the ppc way was superior. It begs the question if not the other archs should change instead?
It is superior in some ways, not enough to be worth being different.
The PowerPC syscall ABI *requires* using cr0.3 for indicating errors, you will have to do that whether you conflate the concepts of return code and error indicator or not!
Quoted 2 lines
Other archs are unlikely to change because it would be painful for not much benefit.
Quoted 8 lines
Quoted 4 lines
Quoted 1 line
Other archs cannot easily change for much the same reason :-)
Really? I figured you could just add extra error indication in kernel syscall I/F. Eventually user space could migrate to the new indication.
You seem to assume all user space uses glibc, or *any* libc even? This is false. Some programs do system calls directly. Do not break the kernel ABI :-)
Even if it were easy to change, the old ppc ABI with a separate error indicator is much worse to use. In musl we paper over archs that do this silliness by converting to a normal negated errno code. There are literally no syscalls that need the ability to return negative values in addition to error codes; historically there were one or two (I only recall one fcntl command) but there were ways to disambiguate and they're only fallbacks for ancient kernels nowadays, if used at all.
Rich
Quoted 16 lines · Segher Boessenkool
Quoted 15 lines · Joakim Tjernlund
Quoted 14 lines · Segher Boessenkool
Quoted 5 lines · Nicholas Piggin
Excerpts from Joakim Tjernlund's message of May 19, 2021 6:08 pm:
Quoted 2 lines
I always figured the ppc way was superior. It begs the question if not the other archs should change instead?
It is superior in some ways, not enough to be worth being different.
The PowerPC syscall ABI *requires* using cr0.3 for indicating errors, you will have to do that whether you conflate the concepts of return code and error indicator or not!
Quoted 2 lines
Other archs are unlikely to change because it would be painful for not much benefit.
Quoted 8 lines
Quoted 4 lines
Quoted 1 line
Other archs cannot easily change for much the same reason :-)
Really? I figured you could just add extra error indication in kernel syscall I/F. Eventually user space could migrate to the new indication.
You seem to assume all user space uses glibc, or *any* libc even? This is false. Some programs do system calls directly. Do not break the kernel ABI :-)
Adding an extra indicator does not break ABI, does it ?
W.r.t breaking ABI, isn't that what PowerPC is trying to do with the new syscall I/F?
Jocke
Quoted 19 lines · Joakim Tjernlund
Quoted 18 lines · Segher Boessenkool
Quoted 17 lines · Joakim Tjernlund
Quoted 15 lines · Segher Boessenkool
Quoted 5 lines · Nicholas Piggin
Excerpts from Joakim Tjernlund's message of May 19, 2021 6:08 pm:
Quoted 2 lines
I always figured the ppc way was superior. It begs the question if not the other archs should change instead?
It is superior in some ways, not enough to be worth being different.
The PowerPC syscall ABI *requires* using cr0.3 for indicating errors, you will have to do that whether you conflate the concepts of return code and error indicator or not!
Quoted 2 lines
Other archs are unlikely to change because it would be painful for not much benefit.
Other archs cannot easily change for much the same reason :-)
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
Really? I figured you could just add extra error indication in kernel syscall I/F. Eventually user space could migrate to the new indication.
You seem to assume all user space uses glibc, or *any* libc even? This is false. Some programs do system calls directly. Do not break the kernel ABI :-)
Adding an extra indicator does not break ABI, does it ?
It does, because the old ABI on most archs has no clobbers except the return value register. Some archs though have additional syscall-clobbered regs that could be repurposed as extra return values, but you could only rely on them being meaningful after probing for a kernel that speaks the new variant. This just makes things more complicated, not more useful.
Quoted 1 line
W.r.t breaking ABI, isn't that what PowerPC is trying to do with the new syscall I/F?
No, it's a new independent interface.
Rich
Quoted 1 line · Rich Felker
On Wed, May 19, 2021 at 06:09:25PM +0000, Joakim Tjernlund wrote:
[...]
Quoted 3 lines
Quoted 1 line
W.r.t breaking ABI, isn't that what PowerPC is trying to do with the new syscall I/F?
No, it's a new independent interface.
Unfortunately, being a new independent interface doesn't mean it isn't an ABI break. In fact, it was a severe ABI break, and this thread is an attempt to find a hotfix.
Signature · 1 line
ldv
Nicholas Piggin
reviewerExcerpts from Dmitry V. Levin's message of May 20, 2021 11:06 am:
Quoted 10 lines · Rich Felker
Quoted 1 line · Joakim Tjernlund
[...]
Quoted 3 lines
Quoted 1 line
W.r.t breaking ABI, isn't that what PowerPC is trying to do with the new syscall I/F?
No, it's a new independent interface.
Unfortunately, being a new independent interface doesn't mean it isn't an ABI break. In fact, it was a severe ABI break, and this thread is an attempt to find a hotfix.
It is an ABI break, that was known. The ptrace info stuff I fixed with the patch earlier was obviously a bug in my initial implementation and not intended (sorry my ptrace testing was not sufficient, and thanks for reporting it, by the way).
But the register ABI was always a known break. The issue is that rfscv clobbers LR, so it can not support the old ABI. If the old ABI did not preserve LR, then we may have chosen to not change register ABI at all.
Thanks,
Nick
Quoted 8 lines · Nicholas Piggin
Excerpts from Dmitry V. Levin's message of May 20, 2021 11:06 am:
Quoted 7 lines · Rich Felker
Quoted 1 line · Joakim Tjernlund
[...]
Quoted 3 lines
Quoted 1 line
W.r.t breaking ABI, isn't that what PowerPC is trying to do with the new syscall I/F?
No, it's a new independent interface.
Quoted 8 lines
Quoted 3 lines
Unfortunately, being a new independent interface doesn't mean it isn't an ABI break. In fact, it was a severe ABI break, and this thread is an attempt to find a hotfix.
It is an ABI break, that was known. The ptrace info stuff I fixed with the patch earlier was obviously a bug in my initial implementation and not intended (sorry my ptrace testing was not sufficient, and thanks for reporting it, by the way).
Could you check whether tools/testing/selftests/ptrace/get_syscall_info.c passes again with your fix, please?
If yes, then PTRACE_GET_SYSCALL_INFO is fixed.
By the way, kernel tracing and audit subsystems also use those functions from asm/syscall.h and asm/ptrace.h, so your ptrace fix is likely to fix these subsystems as well.
Signature · 1 line
ldv
Nicholas Piggin
reviewerExcerpts from Dmitry V. Levin's message of May 20, 2021 12:59 pm:
Quoted 13 lines · Nicholas Piggin
Quoted 12 lines
Excerpts from Dmitry V. Levin's message of May 20, 2021 11:06 am:
Quoted 10 lines · Rich Felker
Quoted 1 line · Joakim Tjernlund
[...]
Quoted 3 lines
Quoted 1 line
W.r.t breaking ABI, isn't that what PowerPC is trying to do with the new syscall I/F?
No, it's a new independent interface.
Unfortunately, being a new independent interface doesn't mean it isn't an ABI break. In fact, it was a severe ABI break, and this thread is an attempt to find a hotfix.
Quoted 7 lines
Quoted 4 lines
It is an ABI break, that was known. The ptrace info stuff I fixed with the patch earlier was obviously a bug in my initial implementation and not intended (sorry my ptrace testing was not sufficient, and thanks for reporting it, by the way).
Could you check whether
tools/testing/selftests/ptrace/get_syscall_info.cpasses again with your fix, please?
It does.
Thanks,
Nick
Quoted 5 lines
If yes, then
PTRACE_GET_SYSCALL_INFOis fixed.By the way, kernel tracing and audit subsystems also use those functions from
asm/syscall.handasm/ptrace.h, so your ptrace fix is likely to fix these subsystems as well.