Skip to contentSkip to conversation

ARM tracehook support

DISCUSSION Wade Farnsworth 3 participants 8 messages Feb 20, 2012 – Feb 21, 2012

lore
Share & tools
lore
#0 lore raw

Hi all,

A few months back, Steven Walter posted some patches that implemement syscall tracing support for ARM:

http://lists.arm.linux.org.uk/lurker/message/20111129.162812.bd17d9b5.en.html#linux-arm-kernel

I've been working on SystemTap support for ARM, and the tracehook patch in particular is interesting in that respect. With that patch applied, it would allow utrace to be supported on ARM, which is necessary for userspace stap probes to function properly.

From what testing I've done, the patches don't appear to be harmful.
So, I'm wondering what's preventing this from being pushed into mainline? Are there still outstanding issues that need to be addressed?
I'd be happy to help solve them if it would help this patch get into
mainline quicker.

Thanks in advance!

Wade Farnsworth

#1 lore raw
Quoted 1 line · Wade Farnsworth

Hi all,

Hello Wade,

Quoted 5 lines

A few months back, Steven Walter posted some patches that implemement syscall tracing support for ARM:

http://lists.arm.linux.org.uk/lurker/message/20111129.162812.bd17d9b5.en.html#linux-arm-kernel

Quoted 8 lines

I've been working on SystemTap support for ARM, and the tracehook patch in particular is interesting in that respect. With that patch applied, it would allow utrace to be supported on ARM, which is necessary for userspace stap probes to function properly.

From what testing I've done, the patches don't appear to be harmful.
So, I'm wondering what's preventing this from being pushed into mainline? Are there still outstanding issues that need to be addressed?

Yup, I posted some questions here:

http://lists.infradead.org/pipermail/linux-arm-kernel/2011-December/074802.html

but I haven't heard anything from Steven since then, so it's all stalled at the moment. They'll also obviously need rebasing onto latest mainline, although that shouldn't be too hard since they're still fairly recent.

Will

#2 lore raw
Quoted 15 lines · Will Deacon
Quoted 1 line · Wade Farnsworth

Hi all,

Hello Wade,

Quoted 10 lines

A few months back, Steven Walter posted some patches that implemement syscall tracing support for ARM:

http://lists.arm.linux.org.uk/lurker/message/20111129.162812.bd17d9b5.en.html#linux-arm-kernel

I've been working on SystemTap support for ARM, and the tracehook patch in particular is interesting in that respect. With that patch applied, it would allow utrace to be supported on ARM, which is necessary for userspace stap probes to function properly.

Quoted 7 lines
Quoted 3 lines

From what testing I've done, the patches don't appear to be harmful.
So, I'm wondering what's preventing this from being pushed into mainline? Are there still outstanding issues that need to be addressed?

Yup, I posted some questions here:

http://lists.infradead.org/pipermail/linux-arm-kernel/2011-December/074802.html

OK, so to continue that conversation:

Quoted 12 lines · Will Deacon

Review

Quoted 6 lines · Steven Walter
(quoted diff) +6 -0
+static inline void syscall_get_arguments(struct task_struct *task,
+					 struct pt_regs *regs,
+					 unsigned int i, unsigned int n,
+					 unsigned long *args)
+{
+	BUG_ON(i + n > 6);

So I guess 6 is the maximum number of registers that are used for syscall passing. That sounds about right to me, but I wondered how you worked it out (and whether or not it should be defined somewhere?).

I believe the 6 argument constraint is a specific to syscall_get_arguments(). Notice the comment in include/asm-generic/syscall.h:

/*
[...]
*
* It's only valid to call this when @task is stopped for tracing on
* entry to a system call, due to %TIF_SYSCALL_TRACE or %TIF_SYSCALL_AUDIT.
* It's invalid to call this with @i + @n > 6; we only support system calls
* taking up to 6 arguments.
*/

Additionally, if you'll look at the other architectures' implementations you'll see similar code.

Quoted 5 lines

In fact, how are these things supposed to deal with 64-bit arguments that straddle two registers? I think we always pack arguments such that we don't get holes in the register layout, but it might be worth checking (EABI requires 64-bit arguments to be passed in even registers).

Hmm, I do believe that 32-bit powerpc has similar alignment issues (64-bit args must be passed in odd/even pairs), but I don't see any special handling of this in that architecture's syscall_get/set_arguments(). So I'm wondering if the handling of this is or should be handled elsewhere. I'll keep digging on this.

Quoted 5 lines

but I haven't heard anything from Steven since then, so it's all stalled at the moment. They'll also obviously need rebasing onto latest mainline, although that shouldn't be too hard since they're still fairly recent.

Yes, it does not appear to be too difficult. I've already made some progress here, and will post my patches once the other issues have been resolved.

Thanks again,

-Wade

#3 lore raw
Quoted 64 lines · Wade Farnsworth
Quoted 16 lines · Will Deacon
Quoted 1 line · Wade Farnsworth

Hi all,

Hello Wade,

Quoted 11 lines

A few months back, Steven Walter posted some patches that implemement syscall tracing support for ARM:

http://lists.arm.linux.org.uk/lurker/message/20111129.162812.bd17d9b5.en.html#linux-arm-kernel

I've been working on SystemTap support for ARM, and the tracehook patch in particular is interesting in that respect. With that patch applied, it would allow utrace to be supported on ARM, which is necessary for userspace stap probes to function properly.

Quoted 8 lines
Quoted 3 lines

From what testing I've done, the patches don't appear to be harmful. So, I'm wondering what's preventing this from being pushed into mainline? Are there still outstanding issues that need to be addressed?

Yup, I posted some questions here:

http://lists.infradead.org/pipermail/linux-arm-kernel/2011-December/074802.html

OK, so to continue that conversation:

Quoted 12 lines · Will Deacon

Review

Quoted 6 lines · Steven Walter
(quoted diff) +6 -0
+static inline void syscall_get_arguments(struct task_struct *task,
+ struct pt_regs *regs,
+ unsigned int i, unsigned int n,
+ unsigned long *args)
+{
+ BUG_ON(i + n > 6);

So I guess 6 is the maximum number of registers that are used for syscall passing. That sounds about right to me, but I wondered how you worked it out (and whether or not it should be defined somewhere?).

I believe the 6 argument constraint is a specific to syscall_get_arguments(). Notice the comment in include/asm-generic/syscall.h:

/*
[...]
*
* It's only valid to call this when @task is stopped for tracing on
* entry to a system call, due to %TIF_SYSCALL_TRACE or %TIF_SYSCALL_AUDIT.
* It's invalid to call this with @i + @n > 6; we only support system calls
* taking up to 6 arguments.
*/

Additionally, if you'll look at the other architectures' implementations you'll see similar code.

Quoted 5 lines

In fact, how are these things supposed to deal with 64-bit arguments that straddle two registers? I think we always pack arguments such that we don't get holes in the register layout, but it might be worth checking (EABI requires 64-bit arguments to be passed in even registers).

Quoted 6 lines

Hmm, I do believe that 32-bit powerpc has similar alignment issues (64-bit args must be passed in odd/even pairs), but I don't see any special handling of this in that architecture's syscall_get/set_arguments(). So I'm wondering if the handling of this is or should be handled elsewhere. I'll keep digging on this.

This question also came up from an older attempt at ARM tracehook by Roland McGrath:

http://lists.openwall.net/linux-kernel/2009/06/24/150

In a nutshell, the argument is that syscall_get_arguments() doesn't need to know about any of the argument semantics, they just pass all registers that may contain arguments back up the stack. It's then up to the tracer to interpret the arguments. The argument is similar for syscall_set_arguments(). Since the same register set is used for both EABI and OABI, there doesn't need to be any specific handling of one versus the other here.

Thanks,

-Wade

#4 lore raw

Hi Wade,

Thanks for following this up.

Quoted 14 lines · Wade Farnsworth
Quoted 14 lines

I believe the 6 argument constraint is a specific to syscall_get_arguments(). Notice the comment in include/asm-generic/syscall.h:

/*
[...]
*
* It's only valid to call this when @task is stopped for tracing on
* entry to a system call, due to %TIF_SYSCALL_TRACE or %TIF_SYSCALL_AUDIT.
* It's invalid to call this with @i + @n > 6; we only support system calls
* taking up to 6 arguments.
*/

Quoted 2 lines
Quoted 2 lines

Additionally, if you'll look at the other architectures' implementations you'll see similar code.

In which case could we #define this somewhere to make it more obvious?

Quoted 7 lines

In a nutshell, the argument is that syscall_get_arguments() doesn't need to know about any of the argument semantics, they just pass all registers that may contain arguments back up the stack. It's then up to the tracer to interpret the arguments. The argument is similar for syscall_set_arguments(). Since the same register set is used for both EABI and OABI, there doesn't need to be any specific handling of one versus the other here.

Ok, if the low-level code doesn't need to interpret arguments then that should be fine. sys_arm_fadvise64_64 would be the tricky case, but the argument ordering seems to have been chosen explicitly to place the 64-bit offsets naturally on even registers, fitting it neatly into 6 registers.

Once you've rebased the code, please post a new version so it can be reviewed again.

Cheers,

Will

#6 lore raw
In reply to Wade Farnsworth
Quoted 14 lines · Wade Farnsworth
Quoted 12 lines · Will Deacon

Review

Quoted 6 lines · Steven Walter
(quoted diff) +6 -0
+static inline void syscall_get_arguments(struct task_struct *task,
+					 struct pt_regs *regs,
+					 unsigned int i, unsigned int n,
+					 unsigned long *args)
+{
+	BUG_ON(i + n > 6);

So I guess 6 is the maximum number of registers that are used for syscall passing. That sounds about right to me, but I wondered how you worked it out (and whether or not it should be defined somewhere?).

Quoted 3 lines

I believe the 6 argument constraint is a specific to syscall_get_arguments(). Notice the comment in include/asm-generic/syscall.h:

Well, there's two things here. Is a BUG_ON() really suitable here? What controls how many arguments are fetched? Userspace? If so, that's a nice way to oops the kernel.

Secondly, there is a 7 argument syscall - sys_syscall, which we use on OABI to deal with calling a syscall by number. That really does show up as a unique syscall there, so if you want to parse the last argument to such a syscall you need to be able to read up to and including ARM register 7.

Quoted 18 lines

/*
[...]
*
* It's only valid to call this when @task is stopped for tracing on
* entry to a system call, due to %TIF_SYSCALL_TRACE or %TIF_SYSCALL_AUDIT.
* It's invalid to call this with @i + @n > 6; we only support system calls
* taking up to 6 arguments.
*/

Additionally, if you'll look at the other architectures' implementations you'll see similar code.

Quoted 5 lines

In fact, how are these things supposed to deal with 64-bit arguments that straddle two registers? I think we always pack arguments such that we don't get holes in the register layout, but it might be worth checking (EABI requires 64-bit arguments to be passed in even registers).

Quoted 5 lines

Hmm, I do believe that 32-bit powerpc has similar alignment issues (64-bit args must be passed in odd/even pairs), but I don't see any special handling of this in that architecture's syscall_get/set_arguments(). So I'm wondering if the handling of this is or should be handled elsewhere. I'll keep digging on this.

I did point that issue out when it first came up, and I think Roland had an answer for it, though I forget what it was.

The big stumbling block to this is OABI, and as I continue to be wholely OABI based here, it's extremely important that nothing in OABI land gets broken. I don't see there's any chance of me ever getting off OABI given the range of platforms I have, and the restriction that EABI was designed to be impossible on ARMv4 architectures.

#5 lore raw
In reply to Will Deacon
Quoted 4 lines · Will Deacon

Ok, if the low-level code doesn't need to interpret arguments then that should be fine. sys_arm_fadvise64_64 would be the tricky case, but the argument ordering seems to have been chosen explicitly to place the 64-bit offsets naturally on even registers, fitting it neatly into 6 registers.

That was done because the standard argument order was impossible to pass in the available register set with EABI. It would have meant having to move the syscall number into r8 or higher, which would've been rather unfriendly to Thumb code.

#7 lore raw
In reply to Russell King - ARM Linux
Quoted 15 lines · Russell King - ARM Linux
Quoted 14 lines · Wade Farnsworth
Quoted 12 lines · Will Deacon

Review

Quoted 6 lines · Steven Walter
(quoted diff) +6 -0
+static inline void syscall_get_arguments(struct task_struct *task,
+					 struct pt_regs *regs,
+					 unsigned int i, unsigned int n,
+					 unsigned long *args)
+{
+	BUG_ON(i + n>  6);

So I guess 6 is the maximum number of registers that are used for syscall passing. That sounds about right to me, but I wondered how you worked it out (and whether or not it should be defined somewhere?).

Quoted 6 lines
Quoted 3 lines

I believe the 6 argument constraint is a specific to syscall_get_arguments(). Notice the comment in include/asm-generic/syscall.h:

Well, there's two things here. Is a BUG_ON() really suitable here? What controls how many arguments are fetched? Userspace?

Yeah, it could be userspace, or a kernel tracer such as ftrace.

Quoted 2 lines

If so, that's
a nice way to oops the kernel.

I agree that a BUG_ON is probably not ideal, though I note that the other arch's tend to BUG as well. Since there's no way to return an error from this function, what if we just used a pr_warning() and backfill the bogus elements of args[] with zero?

Quoted 6 lines

Secondly, there is a 7 argument syscall - sys_syscall, which we use on OABI to deal with calling a syscall by number. That really does show up as a unique syscall there, so if you want to parse the last argument to such a syscall you need to be able to read up to and including ARM register 7.

I think we can change it to allow up to 7 arguments in this function easily enough. I think changing the tracers to support an additional argument should be done separatels from this patchset, however.

Quoted 19 lines
Quoted 19 lines

/*
[...]
*
* It's only valid to call this when @task is stopped for tracing on
* entry to a system call, due to %TIF_SYSCALL_TRACE or %TIF_SYSCALL_AUDIT.
* It's invalid to call this with @i + @n> 6; we only support system calls
* taking up to 6 arguments.
*/

Additionally, if you'll look at the other architectures' implementations you'll see similar code.

Quoted 5 lines

In fact, how are these things supposed to deal with 64-bit arguments that straddle two registers? I think we always pack arguments such that we don't get holes in the register layout, but it might be worth checking (EABI requires 64-bit arguments to be passed in even registers).

Quoted 8 lines
Quoted 5 lines

Hmm, I do believe that 32-bit powerpc has similar alignment issues (64-bit args must be passed in odd/even pairs), but I don't see any special handling of this in that architecture's syscall_get/set_arguments(). So I'm wondering if the handling of this is or should be handled elsewhere. I'll keep digging on this.

I did point that issue out when it first came up, and I think Roland had an answer for it, though I forget what it was.

Yes, see my other mail for a link to Roland's answer.

Quoted 6 lines

The big stumbling block to this is OABI, and as I continue to be wholely OABI based here, it's extremely important that nothing in OABI land gets broken. I don't see there's any chance of me ever getting off OABI given the range of platforms I have, and the restriction that EABI was designed to be impossible on ARMv4 architectures.

Right, I think most if not all of the issues w.r.t. OABI that were present in Steven's original patches have been ironed out.

I'll post the patches once I've addressed the current crop of concerns.

Thanks, Russell and Will, for the comments.

-Wade