uprobes/x86: emulate push insns for uprobe on x86
Subsystems (4)
PATCH v4 1/1 uprobes/x86: emulate push insns for uprobe on x86
Uprobe is a tracing mechanism for userspace programs. Typical uprobe will incur overhead of two traps. First trap is caused by replaced trap insn, and the second trap is to execute the original displaced insn in user space.
To reduce the overhead, kernel provides hooks for architectures to emulate the original insn and skip the second trap. In x86, emulation
is done for certain branch insns.
This patch extends the emulation to "push <reg>" insns. These insns are typical in the beginning of the function. For example, bcc
in https://github.com/iovisor/bcc repo provides tools to measure funclantency, detect memleak, etc. The tools will place uprobes in the beginning of function and possibly uretprobes at the end of function. This patch is able to reduce the trap overhead for uprobe from 2 to 1.
Without this patch, uretprobe will typically incur three traps. With this patch, if the function starts with "push" insn, the number of traps can be reduced from 3 to 2.
An experiment was conducted on two local VMs, fedora 26 64-bit VM and 32-bit VM, both 4 processors and 4GB memory, booted with latest tip repo (and this patch). The host is MacBook with intel i7 processor.
The test program looks like
#include <stdio.h>
#include <stdlib.h>
#include <time.h>
#include <sys/time.h>
static void test() __attribute__((noinline));
void test() {}
int main() {
struct timeval start, end; gettimeofday(&start, NULL);
for (int i = 0; i < 1000000; i++) {
test();
}
gettimeofday(&end, NULL);
printf("%ld\n", ((end.tv_sec * 1000000 + end.tv_usec)
- (start.tv_sec * 1000000 + start.tv_usec)));
return 0;}
The program is compiled without optimization, and the first insn for function "test" is "push %rbp". The host is relatively idle.
Before the test run, the uprobe is inserted as below for uprobe:
echo 'p <binary>:<test_func_offset>' > /sys/kernel/debug/tracing/uprobe_events echo 1 > /sys/kernel/debug/tracing/events/uprobes/enable
and for uretprobe:
echo 'r <binary>:<test_func_offset>' > /sys/kernel/debug/tracing/uprobe_events echo 1 > /sys/kernel/debug/tracing/events/uprobes/enable
Unit: microsecond(usec) per loop iteration
x86_64 W/ this patch W/O this patch uprobe 1.55 3.1 uretprobe 2.0 3.6
x86_32 W/ this patch W/O this patch uprobe 1.41 3.5 uretprobe 1.75 4.0
You can see that this patch significantly reduced the overhead, 50% for uprobe and 44% for uretprobe on x86_64, and even more on x86_32.
--- arch/x86/include/asm/uprobes.h | 4 ++ arch/x86/kernel/uprobes.c | 107 +++++++++++++++++++++++++++++++++++++++-- 2 files changed, 107 insertions(+), 4 deletions(-)
Changelogs:
v3 -> v4:
. Revert most of v3 change as 32bit emulation is not really working
on x86_64 platform as among other issues, function emulate_push_stack() needs to account for 32bit app on 64bit platform.
A separate effort is ongoing to address this issue.
v2 -> v3:
. Do not emulate 32bit application on x86_64 platforms
v1 -> v2:
. Address Oleg's comments
arch/x86/include/asm/uprobes.h +4 -0
@@ -53,6 +53,10 @@ struct arch_uprobe {u8 fixups;u8 ilen; } defparam; +struct { +u8 reg_offset;/* to the start of pt_regs */ +u8 ilen; + } push; }; };
arch/x86/kernel/uprobes.c +103 -4
152 lines not on this page · raw message
Signature · 1 line
2.9.5
Quoted 6 lines · Yonghong Song
v3 -> v4:
. Revert most of v3 change as 32bit emulation is not really working
onx86_64platform as among other issues, functionemulate_push_stack()needs to account for 32bit app on 64bit platform.
A separate effort is ongoing to address this issue.
Please test your patch with the fix below, in this particular case the TIF_IA32 check should be fine. Although this is not what we really want, we should probably use user_64bit_mode(regs) which checks ->cs. But this needs more changes and doesn't solve other problems (get_unmapped_area) so I still can't decide what should we do right now...
Oleg.
arch/x86/kernel/uprobes.c +1 -1
@@ -516,7 +516,7 @@ struct uprobe_xol_ops {static inline int sizeof_long (void ) { -return in_ia32_syscall () ?4 :8 ; +return test_thread_flag (TIF_IA32 ) ?4 :8 ; }static int default_pre_xol_op (struct arch_uprobe *auprobe,struct pt_regs *regs)
Quoted 12 lines · Oleg Nesterov
Quoted 6 lines · Yonghong Song
v3 -> v4:
. Revert most of v3 change as 32bit emulation is not really working
onx86_64platform as among other issues, functionemulate_push_stack()needs to account for 32bit app on 64bit platform.
A separate effort is ongoing to address this issue.Reviewed-by Oleg Nesterov <[email protected]>
Quoted 5 lines
Please test your patch with the fix below, in this particular case the
TIF_IA32check should be fine. Although this is not what we really want, we should probably useuser_64bit_mode(regs)which checks ->cs. But this needs more changes and doesn't solve other problems (get_unmapped_area) so I still can't decide what should we do right now...
I tested the below change with my patch. On x86_64, both 64bit and 32bit program can be uprobe emulated properly. On x86_32, however, there is a compilation error like below:
In function ‘check_copy_size’,
inlined from ‘copy_to_user’ at/home/yhs/work/tip/include/linux/uaccess.h:154:6,
inlined from ‘emulate_push_stack.isra.9’ at/home/yhs/work/tip/arch/x86/kernel/uprobes.c:535:6:/home/yhs/work/tip/include/linux/thread_info.h:139:4: error: call to ‘__bad_copy_from’ declared with attribute error: copy source size is too small
__bad_copy_from();
Basically, test_thread_flag(TIF_IA32) returns 0 on x86_32 system.
Quoted 15 lines
Oleg.
arch/x86/kernel/uprobes.c +1
-1@@ -516,7 +516,7 @@ struct uprobe_xol_ops {static inline int sizeof_long (void ) { -return in_ia32_syscall () ?4 :8 ; +return test_thread_flag (TIF_IA32 ) ?4 :8 ; }static int default_pre_xol_op (struct arch_uprobe *auprobe,struct pt_regs *regs)
Oleg Nesterov
maintainerQuoted 14 lines · Yonghong Song
Quoted 12 lines · Oleg Nesterov
Quoted 6 lines · Yonghong Song
v3 -> v4:
. Revert most of v3 change as 32bit emulation is not really working
onx86_64platform as among other issues, functionemulate_push_stack()needs to account for 32bit app on 64bit platform.
A separate effort is ongoing to address this issue.Reviewed-by Oleg Nesterov <[email protected]>
Quoted 8 lines
Quoted 5 lines
Please test your patch with the fix below, in this particular case the
TIF_IA32check should be fine. Although this is not what we really want, we should probably useuser_64bit_mode(regs)which checks ->cs. But this needs more changes and doesn't solve other problems (get_unmapped_area) so I still can't decide what should we do right now...I tested the below change with my patch. On
x86_64, both 64bit and 32bit program can be uprobe emulated properly.
Good, so your patch is fine.
Quoted 2 lines
On
x86_32, however, there is a
compilation error like below:
Yes, yes, when I said "in this particular case" I meant x86_64 system only.
Sorry for confusion, I asked you to test this additional change just to ensure that we didn't miss something and your patch has no problems with 32bit tasks on 64bit system, except those we need to fix anyway.
Oleg.
Quoted 21 lines · Oleg Nesterov
Quoted 20 lines · Yonghong Song
Quoted 17 lines · Oleg Nesterov
Quoted 6 lines · Yonghong Song
v3 -> v4:
. Revert most of v3 change as 32bit emulation is not really working
onx86_64platform as among other issues, functionemulate_push_stack()needs to account for 32bit app on 64bit platform.
A separate effort is ongoing to address this issue.Reviewed-by Oleg Nesterov <[email protected]>Please test your patch with the fix below, in this particular case the
TIF_IA32check should be fine. Although this is not what we really want, we should probably useuser_64bit_mode(regs)which checks ->cs. But this needs more changes and doesn't solve other problems (get_unmapped_area) so I still can't decide what should we do right now...
Quoted 4 lines
Quoted 2 lines
I tested the below change with my patch. On
x86_64, both 64bit and 32bit program can be uprobe emulated properly.Good, so your patch is fine.
Thanks!
Quoted 9 lines
Quoted 3 lines
On
x86_32, however, there is a
compilation error like below:Yes, yes, when I said "in this particular case" I meant
x86_64system only.Sorry for confusion, I asked you to test this additional change just to ensure that we didn't miss something and your patch has no problems with 32bit tasks on 64bit system, except those we need to fix anyway.
Understood. I actually tried a little to see whether I could have a simple way to fix 32bit compilation error without using ugly "#ifdef CONFIG_X86_64". Maybe is_64bit_mm is a good choice. But we could defer this until you have a comprehensive fix for 32bit app uprobe on 64bit systems as there are multiple issues for this.
Quoted 3 lines
Oleg.
Hi, Ingo and Peter,
This patch has been reviewed by Oleg Nesterov. Could you take a look and help merge it upstream?
Thanks!
Yonghong
Quoted 46 lines · Yonghong Song
Quoted 21 lines · Oleg Nesterov
Quoted 20 lines · Yonghong Song
Quoted 17 lines · Oleg Nesterov
Quoted 6 lines · Yonghong Song
v3 -> v4:
. Revert most of v3 change as 32bit emulation is not really working onx86_64platform as among other issues, functionemulate_push_stack()needs to account for 32bit app on 64bit platform.
A separate effort is ongoing to address this issue.Reviewed-by Oleg Nesterov <[email protected]>Please test your patch with the fix below, in this particular case the
TIF_IA32check should be fine. Although this is not what we really want, we should probably useuser_64bit_mode(regs)which checks ->cs. But this needs more changes and doesn't solve other problems (get_unmapped_area) so I still can't decide what should we do right now...Quoted 4 lines
Quoted 2 lines
I tested the below change with my patch. On
x86_64, both 64bit and 32bit program can be uprobe emulated properly.Good, so your patch is fine.
Thanks!
Quoted 9 lines
Quoted 3 lines
On
x86_32, however, there is a
compilation error like below:Yes, yes, when I said "in this particular case" I meant
x86_64system only.Sorry for confusion, I asked you to test this additional change just to ensure that we didn't miss something and your patch has no problems with 32bit tasks on 64bit system, except those we need to fix anyway.
Understood. I actually tried a little to see whether I could have a simple way to fix 32bit compilation error without using ugly "#ifdef
CONFIG_X86_64". Maybeis_64bit_mmis a good choice. But we could defer this until you have a comprehensive fix for 32bit app uprobe on 64bit systems as there are multiple issues for this.Quoted 3 lines
Oleg.