ARM tracehook support
Hi all,
A few months back, Steven Walter posted some patches that implemement syscall tracing support for ARM:
http://lists.arm.linux.org.uk/lurker/message/20111129.162812.bd17d9b5.en.html#linux-arm-kernel
I've been working on SystemTap support for ARM, and the tracehook patch in particular is interesting in that respect. With that patch applied, it would allow utrace to be supported on ARM, which is necessary for userspace stap probes to function properly.
From what testing I've done, the patches don't appear to be harmful.
So, I'm wondering what's preventing this from being pushed into mainline? Are there still outstanding issues that need to be addressed?
I'd be happy to help solve them if it would help this patch get into
mainline quicker.
Thanks in advance!
Wade Farnsworth
Quoted 1 line · Wade Farnsworth
Hi all,
Hello Wade,
Quoted 5 lines
A few months back, Steven Walter posted some patches that implemement syscall tracing support for ARM:
http://lists.arm.linux.org.uk/lurker/message/20111129.162812.bd17d9b5.en.html#linux-arm-kernel
Quoted 8 lines
I've been working on SystemTap support for ARM, and the tracehook patch in particular is interesting in that respect. With that patch applied, it would allow utrace to be supported on ARM, which is necessary for userspace stap probes to function properly.
From what testing I've done, the patches don't appear to be harmful.
So, I'm wondering what's preventing this from being pushed into mainline? Are there still outstanding issues that need to be addressed?
Yup, I posted some questions here:
http://lists.infradead.org/pipermail/linux-arm-kernel/2011-December/074802.html
but I haven't heard anything from Steven since then, so it's all stalled at the moment. They'll also obviously need rebasing onto latest mainline, although that shouldn't be too hard since they're still fairly recent.
Will
Quoted 15 lines · Will Deacon
Quoted 1 line · Wade Farnsworth
Hi all,
Hello Wade,
Quoted 10 lines
A few months back, Steven Walter posted some patches that implemement syscall tracing support for ARM:
http://lists.arm.linux.org.uk/lurker/message/20111129.162812.bd17d9b5.en.html#linux-arm-kernel
I've been working on SystemTap support for ARM, and the tracehook patch in particular is interesting in that respect. With that patch applied, it would allow utrace to be supported on ARM, which is necessary for userspace stap probes to function properly.
Quoted 7 lines
Quoted 3 lines
From what testing I've done, the patches don't appear to be harmful.
So, I'm wondering what's preventing this from being pushed into mainline? Are there still outstanding issues that need to be addressed?Yup, I posted some questions here:
http://lists.infradead.org/pipermail/linux-arm-kernel/2011-December/074802.html
OK, so to continue that conversation:
Quoted 12 lines · Will Deacon
Review
Quoted 6 lines · Steven Walter
(quoted diff) +6
-0+static inline void syscall_get_arguments(struct task_struct *task, + struct pt_regs *regs, + unsigned int i, unsigned int n, + unsigned long *args) +{ + BUG_ON(i + n > 6);So I guess 6 is the maximum number of registers that are used for syscall passing. That sounds about right to me, but I wondered how you worked it out (and whether or not it should be defined somewhere?).
I believe the 6 argument constraint is a specific to syscall_get_arguments(). Notice the comment in include/asm-generic/syscall.h:
/*
[...]
*
* It's only valid to call this when @task is stopped for tracing on
* entry to a system call, due to %TIF_SYSCALL_TRACE or %TIF_SYSCALL_AUDIT.
* It's invalid to call this with @i + @n > 6; we only support system calls
* taking up to 6 arguments.
*/
Additionally, if you'll look at the other architectures' implementations you'll see similar code.
Quoted 5 lines
In fact, how are these things supposed to deal with 64-bit arguments that straddle two registers? I think we always pack arguments such that we don't get holes in the register layout, but it might be worth checking (EABI requires 64-bit arguments to be passed in even registers).
Hmm, I do believe that 32-bit powerpc has similar alignment issues (64-bit args must be passed in odd/even pairs), but I don't see any special handling of this in that architecture's syscall_get/set_arguments(). So I'm wondering if the handling of this is or should be handled elsewhere. I'll keep digging on this.
Quoted 5 lines
but I haven't heard anything from Steven since then, so it's all stalled at the moment. They'll also obviously need rebasing onto latest mainline, although that shouldn't be too hard since they're still fairly recent.
Yes, it does not appear to be too difficult. I've already made some progress here, and will post my patches once the other issues have been resolved.
Thanks again,
-Wade
Quoted 64 lines · Wade Farnsworth
Quoted 16 lines · Will Deacon
Quoted 1 line · Wade Farnsworth
Hi all,
Hello Wade,
Quoted 11 lines
A few months back, Steven Walter posted some patches that implemement syscall tracing support for ARM:
http://lists.arm.linux.org.uk/lurker/message/20111129.162812.bd17d9b5.en.html#linux-arm-kernel
I've been working on SystemTap support for ARM, and the tracehook patch in particular is interesting in that respect. With that patch applied, it would allow utrace to be supported on ARM, which is necessary for userspace stap probes to function properly.
Quoted 8 lines
Quoted 3 lines
From what testing I've done, the patches don't appear to be harmful. So, I'm wondering what's preventing this from being pushed into mainline? Are there still outstanding issues that need to be addressed?
Yup, I posted some questions here:
http://lists.infradead.org/pipermail/linux-arm-kernel/2011-December/074802.html
OK, so to continue that conversation:
Quoted 12 lines · Will Deacon
Review
Quoted 6 lines · Steven Walter
(quoted diff) +6
-0+static inline void syscall_get_arguments(struct task_struct *task, + struct pt_regs *regs, + unsigned int i, unsigned int n, + unsigned long *args) +{ + BUG_ON(i + n > 6);So I guess 6 is the maximum number of registers that are used for syscall passing. That sounds about right to me, but I wondered how you worked it out (and whether or not it should be defined somewhere?).
I believe the 6 argument constraint is a specific to
syscall_get_arguments(). Notice the comment ininclude/asm-generic/syscall.h:/*
[...]
*
* It's only valid to call this when @task is stopped for tracing on
* entry to a system call, due to %TIF_SYSCALL_TRACE or %TIF_SYSCALL_AUDIT.
* It's invalid to call this with @i + @n > 6; we only support system calls
* taking up to 6 arguments.
*/Additionally, if you'll look at the other architectures' implementations you'll see similar code.
Quoted 5 lines
In fact, how are these things supposed to deal with 64-bit arguments that straddle two registers? I think we always pack arguments such that we don't get holes in the register layout, but it might be worth checking (EABI requires 64-bit arguments to be passed in even registers).
Quoted 6 lines
Hmm, I do believe that 32-bit powerpc has similar alignment issues (64-bit args must be passed in odd/even pairs), but I don't see any special handling of this in that architecture's syscall_get/set_arguments(). So I'm wondering if the handling of this is or should be handled elsewhere. I'll keep digging on this.
This question also came up from an older attempt at ARM tracehook by Roland McGrath:
http://lists.openwall.net/linux-kernel/2009/06/24/150
In a nutshell, the argument is that syscall_get_arguments() doesn't need to know about any of the argument semantics, they just pass all registers that may contain arguments back up the stack. It's then up to the tracer to interpret the arguments. The argument is similar for syscall_set_arguments(). Since the same register set is used for both EABI and OABI, there doesn't need to be any specific handling of one versus the other here.
Thanks,
-Wade
Hi Wade,
Thanks for following this up.
Quoted 14 lines · Wade Farnsworth
Quoted 14 lines
I believe the 6 argument constraint is a specific to
syscall_get_arguments(). Notice the comment ininclude/asm-generic/syscall.h:/*
[...]
*
* It's only valid to call this when @task is stopped for tracing on
* entry to a system call, due to %TIF_SYSCALL_TRACE or %TIF_SYSCALL_AUDIT.
* It's invalid to call this with @i + @n > 6; we only support system calls
* taking up to 6 arguments.
*/
Quoted 2 lines
Quoted 2 lines
Additionally, if you'll look at the other architectures' implementations you'll see similar code.
In which case could we #define this somewhere to make it more obvious?
Quoted 7 lines
In a nutshell, the argument is that
syscall_get_arguments()doesn't need to know about any of the argument semantics, they just pass all registers that may contain arguments back up the stack. It's then up to the tracer to interpret the arguments. The argument is similar forsyscall_set_arguments(). Since the same register set is used for both EABI and OABI, there doesn't need to be any specific handling of one versus the other here.
Ok, if the low-level code doesn't need to interpret arguments then that should be fine. sys_arm_fadvise64_64 would be the tricky case, but the argument ordering seems to have been chosen explicitly to place the 64-bit offsets naturally on even registers, fitting it neatly into 6 registers.
Once you've rebased the code, please post a new version so it can be reviewed again.
Cheers,
Will
Quoted 14 lines · Wade Farnsworth
Quoted 12 lines · Will Deacon
Review
Quoted 6 lines · Steven Walter
(quoted diff) +6
-0+static inline void syscall_get_arguments(struct task_struct *task, + struct pt_regs *regs, + unsigned int i, unsigned int n, + unsigned long *args) +{ + BUG_ON(i + n > 6);So I guess 6 is the maximum number of registers that are used for syscall passing. That sounds about right to me, but I wondered how you worked it out (and whether or not it should be defined somewhere?).
Quoted 3 lines
I believe the 6 argument constraint is a specific to
syscall_get_arguments(). Notice the comment ininclude/asm-generic/syscall.h:
Well, there's two things here. Is a BUG_ON() really suitable here? What controls how many arguments are fetched? Userspace? If so, that's a nice way to oops the kernel.
Secondly, there is a 7 argument syscall - sys_syscall, which we use on OABI to deal with calling a syscall by number. That really does show up as a unique syscall there, so if you want to parse the last argument to such a syscall you need to be able to read up to and including ARM register 7.
Quoted 18 lines
/*
[...]
*
* It's only valid to call this when @task is stopped for tracing on
* entry to a system call, due to %TIF_SYSCALL_TRACE or %TIF_SYSCALL_AUDIT.
* It's invalid to call this with @i + @n > 6; we only support system calls
* taking up to 6 arguments.
*/Additionally, if you'll look at the other architectures' implementations you'll see similar code.
Quoted 5 lines
In fact, how are these things supposed to deal with 64-bit arguments that straddle two registers? I think we always pack arguments such that we don't get holes in the register layout, but it might be worth checking (EABI requires 64-bit arguments to be passed in even registers).
Quoted 5 lines
Hmm, I do believe that 32-bit powerpc has similar alignment issues (64-bit args must be passed in odd/even pairs), but I don't see any special handling of this in that architecture's syscall_get/set_arguments(). So I'm wondering if the handling of this is or should be handled elsewhere. I'll keep digging on this.
I did point that issue out when it first came up, and I think Roland had an answer for it, though I forget what it was.
The big stumbling block to this is OABI, and as I continue to be wholely OABI based here, it's extremely important that nothing in OABI land gets broken. I don't see there's any chance of me ever getting off OABI given the range of platforms I have, and the restriction that EABI was designed to be impossible on ARMv4 architectures.
Quoted 4 lines · Will Deacon
Ok, if the low-level code doesn't need to interpret arguments then that should be fine.
sys_arm_fadvise64_64would be the tricky case, but the argument ordering seems to have been chosen explicitly to place the 64-bit offsets naturally on even registers, fitting it neatly into 6 registers.
That was done because the standard argument order was impossible to pass in the available register set with EABI. It would have meant having to move the syscall number into r8 or higher, which would've been rather unfriendly to Thumb code.
Quoted 15 lines · Russell King - ARM Linux
Quoted 14 lines · Wade Farnsworth
Quoted 12 lines · Will Deacon
Review
Quoted 6 lines · Steven Walter
(quoted diff) +6
-0+static inline void syscall_get_arguments(struct task_struct *task, + struct pt_regs *regs, + unsigned int i, unsigned int n, + unsigned long *args) +{ + BUG_ON(i + n> 6);So I guess 6 is the maximum number of registers that are used for syscall passing. That sounds about right to me, but I wondered how you worked it out (and whether or not it should be defined somewhere?).
Quoted 6 lines
Quoted 3 lines
I believe the 6 argument constraint is a specific to
syscall_get_arguments(). Notice the comment ininclude/asm-generic/syscall.h:Well, there's two things here. Is a
BUG_ON()really suitable here? What controls how many arguments are fetched? Userspace?
Yeah, it could be userspace, or a kernel tracer such as ftrace.
Quoted 2 lines
If so, that's
a nice way to oops the kernel.
I agree that a BUG_ON is probably not ideal, though I note that the other arch's tend to BUG as well. Since there's no way to return an error from this function, what if we just used a pr_warning() and backfill the bogus elements of args[] with zero?
Quoted 6 lines
Secondly, there is a 7 argument syscall -
sys_syscall, which we use on OABI to deal with calling a syscall by number. That really does show up as a unique syscall there, so if you want to parse the last argument to such a syscall you need to be able to read up to and including ARM register 7.
I think we can change it to allow up to 7 arguments in this function easily enough. I think changing the tracers to support an additional argument should be done separatels from this patchset, however.
Quoted 19 lines
Quoted 19 lines
/*
[...]
*
* It's only valid to call this when @task is stopped for tracing on
* entry to a system call, due to %TIF_SYSCALL_TRACE or %TIF_SYSCALL_AUDIT.
* It's invalid to call this with @i + @n> 6; we only support system calls
* taking up to 6 arguments.
*/Additionally, if you'll look at the other architectures' implementations you'll see similar code.
Quoted 5 lines
In fact, how are these things supposed to deal with 64-bit arguments that straddle two registers? I think we always pack arguments such that we don't get holes in the register layout, but it might be worth checking (EABI requires 64-bit arguments to be passed in even registers).
Quoted 8 lines
Quoted 5 lines
Hmm, I do believe that 32-bit powerpc has similar alignment issues (64-bit args must be passed in odd/even pairs), but I don't see any special handling of this in that architecture's syscall_get/set_arguments(). So I'm wondering if the handling of this is or should be handled elsewhere. I'll keep digging on this.
I did point that issue out when it first came up, and I think Roland had an answer for it, though I forget what it was.
Yes, see my other mail for a link to Roland's answer.
Quoted 6 lines
The big stumbling block to this is OABI, and as I continue to be wholely OABI based here, it's extremely important that nothing in OABI land gets broken. I don't see there's any chance of me ever getting off OABI given the range of platforms I have, and the restriction that EABI was designed to be impossible on ARMv4 architectures.
Right, I think most if not all of the issues w.r.t. OABI that were present in Steven's original patches have been ironed out.
I'll post the patches once I've addressed the current crop of concerns.
Thanks, Russell and Will, for the comments.
-Wade