uprobes/x86: emulate push insns for uprobe on x86
Subsystems (4)
PATCH v2 1/1 uprobes/x86: emulate push insns for uprobe on x86
Uprobe is a tracing mechanism for userspace programs. Typical uprobe will incur overhead of two traps. First trap is caused by replaced trap insn, and the second trap is to execute the original displaced insn in user space.
To reduce the overhead, kernel provides hooks for architectures to emulate the original insn and skip the second trap. In x86, emulation
is done for certain branch insns.
This patch extends the emulation to "push <reg>" insns. These insns are typical in the beginning of the function. For example, bcc
in https://github.com/iovisor/bcc repo provides tools to measure funclantency, detect memleak, etc. The tools will place uprobes in the beginning of function and possibly uretprobes at the end of function. This patch is able to reduce the trap overhead for uprobe from 2 to 1.
Without this patch, uretprobe will typically incur three traps. With this patch, if the function starts with "push" insn, the number of traps can be reduced from 3 to 2.
An experiment was conducted on two local VMs, fedora 26 64-bit VM and 32-bit VM, both 4 processors and 4GB memory, booted with latest tip repo (and this patch). The host is MacBook with intel i7 processor.
The test program looks like
#include <stdio.h>
#include <stdlib.h>
#include <time.h>
#include <sys/time.h>
static void test() __attribute__((noinline));
void test() {}
int main() {
struct timeval start, end; gettimeofday(&start, NULL);
for (int i = 0; i < 1000000; i++) {
test();
}
gettimeofday(&end, NULL);
printf("%ld\n", ((end.tv_sec * 1000000 + end.tv_usec)
- (start.tv_sec * 1000000 + start.tv_usec)));
return 0;}
The program is compiled without optimization, and the first insn for function "test" is "push %rbp". The host is relatively idle.
Before the test run, the uprobe is inserted as below for uprobe:
echo 'p <binary>:<test_func_offset>' > /sys/kernel/debug/tracing/uprobe_events echo 1 > /sys/kernel/debug/tracing/events/uprobes/enable
and for uretprobe:
echo 'r <binary>:<test_func_offset>' > /sys/kernel/debug/tracing/uprobe_events echo 1 > /sys/kernel/debug/tracing/events/uprobes/enable
Unit: microsecond(usec) per loop iteration
x86_64 W/ this patch W/O this patch uprobe 1.55 3.1 uretprobe 2.0 3.6
x86_32 W/ this patch W/O this patch uprobe 1.41 3.5 uretprobe 1.75 4.0
You can see that this patch significantly reduced the overhead, 50% for uprobe and 44% for uretprobe on x86_64, and even more on x86_32.
--- arch/x86/include/asm/uprobes.h | 4 ++ arch/x86/kernel/uprobes.c | 108 +++++++++++++++++++++++++++++++++++++++-- 2 files changed, 108 insertions(+), 4 deletions(-)
v1 -> v2:
. Address Oleg's comments
arch/x86/include/asm/uprobes.h +4 -0
@@ -53,6 +53,10 @@ struct arch_uprobe {u8 fixups;u8 ilen; } defparam; +struct { +u8 reg_offset;/* to the start of pt_regs */ +u8 ilen; + } push; }; };
arch/x86/kernel/uprobes.c +104 -4
153 lines not on this page · raw message
Signature · 1 line
2.9.5
Oleg Nesterov
maintainerThe patch looks good to me, but I have a question because I know nothing about insn encoding,
Quoted 8 lines · Yonghong Song
arch/x86/kernel/uprobes.c +7
-0+static int push_setup_xol_ops (struct arch_uprobe *auprobe,struct insn *insn) +{ +u8 opc1 =OPCODE1 (insn), reg_offset =0 ; + +if (opc1 <0x50 || opc1 >0x57 ) +return -ENOSYS ; +
arch/x86/kernel/uprobes.c static int branch_setup_xol_ops(struct arch_uprobe *auprobe, struct insn *insn) · line 784
Quoted 10 lines
arch/x86/kernel/uprobes.c +10
-0+if (insn->length >2 ) +return -ENOSYS ; +if (insn->length ==2 ) { +/* only support rex_prefix 0x41 (x64 only) */ +#ifdef CONFIG_X86_64 +if (insn->rex_prefix.nbytes !=1 || + insn->rex_prefix.bytes[0 ] !=0x41 ) +return -ENOSYS ; + + auprobe->push.ilen =2 ;
and the "else" branch does
auprobe->push.ilen = 1;
you could add
auprobe->push.ilen = insn->length;
at the end of push_setup_xol_ops() instead, but this is minor/cosmetic,
Quoted 19 lines
arch/x86/kernel/uprobes.c +19
-0+switch (opc1) { +case 0x50 : + reg_offset = offsetof(struct pt_regs , r8); +break ; +case 0x51 : + reg_offset = offsetof(struct pt_regs , r9); +break ; +case 0x52 : + reg_offset = offsetof(struct pt_regs , r10); +break ; +case 0x53 : + reg_offset = offsetof(struct pt_regs , r11); +break ; +case 0x54 : + reg_offset = offsetof(struct pt_regs , r12); +break ; +case 0x55 : + reg_offset = offsetof(struct pt_regs , r13); +break ;
arch/x86/kernel/uprobes.c static int branch_setup_xol_ops(struct arch_uprobe *auprobe, struct insn *insn) · line 813
Quoted 10 lines
arch/x86/kernel/uprobes.c +10
-0+case 0x56 : + reg_offset = offsetof(struct pt_regs , r14); +break ; +case 0x57 : + reg_offset = offsetof(struct pt_regs , r15); + break; + } +#else +return -ENOSYS ; +#endif
OK, but shouldn't we also return ENOSYS if CONFIG_X86_64=y but the probed task is 32bit?
Or in this case uprobe_init_insn(x86_64 => false) should fail and push_setup_xol_ops() won't be called?
Oleg.
Quoted 25 lines · Oleg Nesterov
The patch looks good to me, but I have a question because I know nothing about insn encoding,
Quoted 8 lines · Yonghong Song
arch/x86/kernel/uprobes.c +7
-0+static int push_setup_xol_ops (struct arch_uprobe *auprobe,struct insn *insn) +{ +u8 opc1 =OPCODE1 (insn), reg_offset =0 ; + +if (opc1 <0x50 || opc1 >0x57 ) +return -ENOSYS ; +arch/x86/kernel/uprobes.c static int branch_setup_xol_ops(struct arch_uprobe *auprobe, struct insn *insn) · line 784
Quoted 10 lines
arch/x86/kernel/uprobes.c +10
-0+if (insn->length >2 ) +return -ENOSYS ; +if (insn->length ==2 ) { +/* only support rex_prefix 0x41 (x64 only) */ +#ifdef CONFIG_X86_64 +if (insn->rex_prefix.nbytes !=1 || + insn->rex_prefix.bytes[0 ] !=0x41 ) +return -ENOSYS ; + + auprobe->push.ilen =2 ;and the "else" branch does
Quoted 6 lines
auprobe->push.ilen = 1;you could add
auprobe->push.ilen = insn->length;at the end of
push_setup_xol_ops()instead, but this is minor/cosmetic,
Will make this change in the next revision.
Quoted 25 lines
Quoted 25 lines
arch/x86/kernel/uprobes.c +23
-0+switch (opc1) { +case 0x50 : + reg_offset = offsetof(struct pt_regs , r8); +break ; +case 0x51 : + reg_offset = offsetof(struct pt_regs , r9); +break ; +case 0x52 : + reg_offset = offsetof(struct pt_regs , r10); +break ; +case 0x53 : + reg_offset = offsetof(struct pt_regs , r11); +break ; +case 0x54 : + reg_offset = offsetof(struct pt_regs , r12); +break ; +case 0x55 : + reg_offset = offsetof(struct pt_regs , r13); +break ; +case 0x56 : + reg_offset = offsetof(struct pt_regs , r14); +break ; +case 0x57 :
Quoted 8 lines
arch/x86/kernel/uprobes.c static int branch_setup_xol_ops(struct arch_uprobe *auprobe, struct insn *insn) · line 813
Quoted 6 lines
arch/x86/kernel/uprobes.c +6
-0+ reg_offset = offsetof(struct pt_regs , r15); + break; + } +#else +return -ENOSYS ; +#endif OK, but shouldn't we also return ENOSYS if
CONFIG_X86_64=ybut the probed task is 32bit?
Just tested with a 32bit app on x86 box and segfaults. Yes, we would need to return ENOSYS if the app is 32bit on 64bit system. I may not be worthwhile to emulate this uncommon case.
I will use mmap_is_ia32 or a variant to test whether the app is 32bit or not. Please let me know whether this is correct approach or not.
Quoted 6 lines
Or in this case
uprobe_init_insn(x86_64 => false)should fail andpush_setup_xol_ops()won't be called?Oleg.
Oleg Nesterov
maintainerQuoted 34 lines · Yonghong Song
Quoted 23 lines · Oleg Nesterov
Quoted 23 lines
arch/x86/kernel/uprobes.c +23
-0+switch (opc1) { +case 0x50 : + reg_offset = offsetof(struct pt_regs , r8); +break ; +case 0x51 : + reg_offset = offsetof(struct pt_regs , r9); +break ; +case 0x52 : + reg_offset = offsetof(struct pt_regs , r10); +break ; +case 0x53 : + reg_offset = offsetof(struct pt_regs , r11); +break ; +case 0x54 : + reg_offset = offsetof(struct pt_regs , r12); +break ; +case 0x55 : + reg_offset = offsetof(struct pt_regs , r13); +break ; +case 0x56 : + reg_offset = offsetof(struct pt_regs , r14); +break ; +case 0x57 :Quoted 8 lines
arch/x86/kernel/uprobes.c static int branch_setup_xol_ops(struct arch_uprobe *auprobe, struct insn *insn) · line 813
Quoted 6 lines
arch/x86/kernel/uprobes.c +6
-0+ reg_offset = offsetof(struct pt_regs , r15); + break; + } +#else +return -ENOSYS ; +#endif OK, but shouldn't we also return ENOSYS if
CONFIG_X86_64=ybut the probed task is 32bit?
Quoted 1 line
Just tested with a 32bit app on x86 box and segfaults.
Hmm. How did you verify this?
Your v3 doesn't look right and it seems you misunderstood me...
Quoted 2 lines
Yes, we would need to
return ENOSYS if the app is 32bit on 64bit system.
Only if insn->length == 2. "push bp" and other valid 32bit push'es should be emulated correctly or your patch is wrong. Confused...
Quoted 2 lines
Quoted 2 lines
Or in this case
uprobe_init_insn(x86_64 => false)should fail andpush_setup_xol_ops()won't be called?
So it doesn't fail?
Oleg.
Quoted 35 lines · Oleg Nesterov
Quoted 34 lines · Yonghong Song
Quoted 31 lines · Oleg Nesterov
Quoted 19 lines
arch/x86/kernel/uprobes.c +19
-0+switch (opc1) { +case 0x50 : + reg_offset = offsetof(struct pt_regs , r8); +break ; +case 0x51 : + reg_offset = offsetof(struct pt_regs , r9); +break ; +case 0x52 : + reg_offset = offsetof(struct pt_regs , r10); +break ; +case 0x53 : + reg_offset = offsetof(struct pt_regs , r11); +break ; +case 0x54 : + reg_offset = offsetof(struct pt_regs , r12); +break ; +case 0x55 : + reg_offset = offsetof(struct pt_regs , r13); +break ;arch/x86/kernel/uprobes.c static int branch_setup_xol_ops(struct arch_uprobe *auprobe, struct insn *insn) · line 813
Quoted 10 lines
arch/x86/kernel/uprobes.c +10
-0+case 0x56 : + reg_offset = offsetof(struct pt_regs , r14); +break ; +case 0x57 : + reg_offset = offsetof(struct pt_regs , r15); + break; + } +#else +return -ENOSYS ; +#endif OK, but shouldn't we also return ENOSYS if
CONFIG_X86_64=ybut the probed task is 32bit?
Quoted 3 lines
Quoted 1 line
Just tested with a 32bit app on x86 box and segfaults.
Hmm. How did you verify this?
On a x86_32 box, I compiled the test case with static libraries (including static libc). And I then run this binary on x86_64 with uprobe enabled. You will need to install glibc-static package to make it work.
Quoted 15 lines
Your v3 doesn't look right and it seems you misunderstood me...
Quoted 2 lines
Yes, we would need to
return ENOSYS if the app is 32bit on 64bit system.Only if
insn->length== 2. "push bp" and other valid 32bit push'es should be emulated correctly or your patch is wrong. Confused... >Quoted 2 lines
Quoted 2 lines
Or in this case
uprobe_init_insn(x86_64 => false)should fail andpush_setup_xol_ops()won't be called?So it doesn't fail?
Oleg.