tracing/kprobes: Syntax updates, introduce perf probe
Subsystems (3)
Ingo,
I've tested a little this set and it seems to work well. I can't test the C expression parser (need some libdwarf cross-version compatiblity), but I can create kprobes via low-level parameters, then launch perf record and perf trace successfully. I'll need to update perf trace to handle variables that start with '$' characters though.
Anyway, it's still unstable but I guess pulling it would help Masami to continue without the need to handle the whole batch.
Please pull the tracing/kprobes branch that can be found at:
git://git.kernel.org/pub/scm/linux/kernel/git/frederic/random-tracing.git
tracing/kprobes
Thanks,
Frederic.
Masami Hiramatsu (7):
tracing/kprobes: Add $ prefix to special variables
tracing/kprobes: Remove '$ra' special variable
tracing/kprobes: Make special variable names more self-explainable tracing/kprobes: Avoid field name confliction
tracing/kprobes: Robustify fixed field names against variable field names conflicts
perf: Add perf probe subcommand, a kprobe-event setup helper
perf probe: Add perf probe command support without libdwarf
Documentation/trace/kprobetrace.txt | 27 +- kernel/trace/trace_kprobe.c | 160 +++++--- tools/perf/Makefile | 10 + tools/perf/builtin-probe.c | 386 +++++++++++++++++++ tools/perf/builtin.h | 1 + tools/perf/perf.c | 1 + tools/perf/util/probe-finder.c | 690 +++++++++++++++++++++++++++++++++++ tools/perf/util/probe-finder.h | 70 ++++ 8 files changed, 1271 insertions(+), 74 deletions(-) create mode 100644 tools/perf/builtin-probe.c create mode 100644 tools/perf/util/probe-finder.c create mode 100644 tools/perf/util/probe-finder.h
PATCH 1/1 kill-the-bkl/reiserfs: fix reiserfs lock to cpu_add_remove_lock dependency
While creating the reiserfs workqueue during the journal initialization, we are holding the reiserfs lock, but create_workqueue() also holds the cpu_add_remove_lock, creating then the following dependency:
- reiserfs lock -> cpu_add_remove_lock
But we also have the following existing dependencies:
- mm->mmap_sem -> reiserfs lock
- cpu_add_remove_lock -> cpu_hotplug.lock -> slub_lock -> sysfs_mutex
The merged dependency chain then becomes:
- mm->mmap_sem -> reiserfs lock -> cpu_add_remove_lock ->
cpu_hotplug.lock -> slub_lock -> sysfs_mutex
But when we fill a dir entry in sysfs_readir(), we are holding the sysfs_mutex and we also might fault while copying the directory entry to the user, leading to the following dependency:
- sysfs_mutex -> mm->mmap_sem
The end result is then a lock inversion between sysfs_mutex and mm->mmap_sem, as reported in the following lockdep warning:
[ INFO: possible circular locking dependency detected ] 2.6.31-07095-g25a3912 #4
-------------------------------------------------------
udevadm/790 is trying to acquire lock:
(&mm->mmap_sem){++++++}, at: [<c1098942>] might_fault+0x72/0xc0
but task is already holding lock:
(sysfs_mutex){+.+.+.}, at: [<c110813c>] sysfs_readdir+0x7c/0x260
which lock already depends on the new lock.
the existing dependency chain (in reverse order) is:
-> #5 (sysfs_mutex){+.+.+.}:
[...]
-> #4 (slub_lock){+++++.}:
[...]
-> #3 (cpu_hotplug.lock){+.+.+.}:
[...]
-> #2 (cpu_add_remove_lock){+.+.+.}:
[...]
-> #1 (&REISERFS_SB(s)->lock){+.+.+.}:
[...]
-> #0 (&mm->mmap_sem){++++++}:
[...]
This can be fixed by relaxing the reiserfs lock while creating the workqueue.
This is fine to relax the lock here, we just keep it around to pass through reiserfs lock checks and for paranoid reasons.
--- fs/reiserfs/journal.c | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-)
fs/reiserfs/journal.c +4 -1
@@ -2933,8 +2933,11 @@ int journal_init(struct super_block *sb, const char *j_dev_name, } reiserfs_mounted_fs_count++; -if (reiserfs_mounted_fs_count <=1 ) +if (reiserfs_mounted_fs_count <=1 ) { +reiserfs_write_unlock (sb); commit_wq =create_workqueue ("reiserfs" ); +reiserfs_write_lock (sb); + }INIT_DELAYED_WORK (&journal->j_work, flush_async_commits); journal->j_work_sb = sb;
Signature · 1 line
1.6.2.3
Re: [PATCH] kill-the-bkl/reiserfs: fix reiserfs lock to cpu_add_remove_lock dependency
Quoted 4 lines · Frederic Weisbecker
This can be fixed by relaxing the reiserfs lock while creating the workqueue.
This is fine to relax the lock here, we just keep it around to pass through reiserfs lock checks and for paranoid reasons.
Nothing related to kprobes that said...
Sorry for the noise. I've already posted this patch, but it looks like these locking inversions don't want to live their own life and follow me everywhere...
PATCH 1/7 tracing/kprobes: Add $ prefix to special variables
In reply to Frederic WeisbeckerFrom: Masami Hiramatsu <[email protected]>
Add $ prefix to the special variables(e.g. sa, rv) of kprobe-tracer. This resolves consistency issues between kprobe_events and perf-kprobe.
The main goal is to avoid conflicts between local variable names of probed functions, used by perf probe, and special variables used in the kprobe event creation interface (stack values, etc...) and also available from perf probe.
ie: we don't want rv (return value) to conflict with a local variable named rv in a probed function.
LKML-Reference: <[email protected]>
--- Documentation/trace/kprobetrace.txt | 20 ++++++------ kernel/trace/trace_kprobe.c | 60 +++++++++++++++++++++------------- 2 files changed, 47 insertions(+), 33 deletions(-)
Documentation/trace/kprobetrace.txt +10 -10
@@ -36,13 +36,13 @@ Synopsis of kprobe_events FETCHARGS : Arguments. Each probe can have up to 128 args. %REG : Fetch register REG - sN : Fetch Nth entry of stack (N >= 0) - sa : Fetch stack address. @ADDR : Fetch memory at ADDR (ADDR should be in kernel) @SYM[+|-offs] : Fetch memory at SYM +|- offs (SYM should be a data symbol) - aN : Fetch function argument. (N >= 0)(*) - rv : Fetch return value.(**) - ra : Fetch return address.(**) + $sN : Fetch Nth entry of stack (N >= 0) + $sa : Fetch stack address. + $aN : Fetch function argument. (N >= 0)(*) + $rv : Fetch return value.(**) + $ra : Fetch return address.(**) +|-offs(FETCHARG) : Fetch memory at FETCHARG +|- offs address.(***) NAME=FETCHARG: Set NAME as the argument name of FETCHARG.
@@ -85,13 +85,13 @@ Usage examples To add a probe as a new event, write a new definition to kprobe_events as below. - echo p:myprobe do_sys_open dfd=a0 filename=a1 flags=a2 mode=a3 > /sys/kernel/debug/tracing/kprobe_events + echo p:myprobe do_sys_open dfd=$a0 filename=$a1 flags=$a2 mode=$a3 > /sys/kernel/debug/tracing/kprobe_events This sets a kprobe on the top of do_sys_open() function with recording 1st to 4th arguments as "myprobe" event. As this example shows, users can choose more familiar names for each arguments. - echo r:myretprobe do_sys_open rv ra >> /sys/kernel/debug/tracing/kprobe_events + echo r:myretprobe do_sys_open $rv $ra >> /sys/kernel/debug/tracing/kprobe_events This sets a kretprobe on the return point of do_sys_open() function with recording return value and return address as "myretprobe" event.
@@ -138,11 +138,11 @@ events, you need to enable it. # TASK-PID CPU# TIMESTAMP FUNCTION # | | | | | <...>-1447 [001] 1038282.286875: myprobe: (do_sys_open+0x0/0xd6) dfd=3 filename=7fffd1ec4440 flags=8000 mode=0 - <...>-1447 [001] 1038282.286878: myretprobe: (sys_openat+0xc/0xe <- do_sys_open) rv=fffffffffffffffe ra=ffffffff81367a3a + <...>-1447 [001] 1038282.286878: myretprobe: (sys_openat+0xc/0xe <- do_sys_open) $rv=fffffffffffffffe $ra=ffffffff81367a3a <...>-1447 [001] 1038282.286885: myprobe: (do_sys_open+0x0/0xd6) dfd=ffffff9c filename=40413c flags=8000 mode=1b6 - <...>-1447 [001] 1038282.286915: myretprobe: (sys_open+0x1b/0x1d <- do_sys_open) rv=3 ra=ffffffff81367a3a + <...>-1447 [001] 1038282.286915: myretprobe: (sys_open+0x1b/0x1d <- do_sys_open) $rv=3 $ra=ffffffff81367a3a <...>-1447 [001] 1038282.286969: myprobe: (do_sys_open+0x0/0xd6) dfd=ffffff9c filename=4041c6 flags=98800 mode=10 - <...>-1447 [001] 1038282.286976: myretprobe: (sys_open+0x1b/0x1d <- do_sys_open) rv=3 ra=ffffffff81367a3a + <...>-1447 [001] 1038282.286976: myretprobe: (sys_open+0x1b/0x1d <- do_sys_open) $rv=3 $ra=ffffffff81367a3a Each line shows when the kernel hits an event, and <- SYMBOL means kernel
kernel/trace/trace_kprobe.c +37 -23
112 lines not on this page · raw message
Signature · 1 line
1.6.2.3
PATCH 2/7 tracing/kprobes: Remove '$ra' special variable
In reply to Frederic WeisbeckerFrom: Masami Hiramatsu <[email protected]>
Remove '$ra' (return address) because it is already shown at the head of each entry.
LKML-Reference: <[email protected]>
--- Documentation/trace/kprobetrace.txt | 13 ++++++------- kernel/trace/trace_kprobe.c | 11 ----------- 2 files changed, 6 insertions(+), 18 deletions(-)
Documentation/trace/kprobetrace.txt +6 -7
@@ -1,4 +1,4 @@ - Kprobe-based Event Tracer + Kprobe-based Event Tracer ========================= Documentation is written by Masami Hiramatsu
@@ -42,7 +42,6 @@ Synopsis of kprobe_events $sa : Fetch stack address. $aN : Fetch function argument. (N >= 0)(*) $rv : Fetch return value.(**) - $ra : Fetch return address.(**) +|-offs(FETCHARG) : Fetch memory at FETCHARG +|- offs address.(***) NAME=FETCHARG: Set NAME as the argument name of FETCHARG.
@@ -91,10 +90,10 @@ as below. 1st to 4th arguments as "myprobe" event. As this example shows, users can choose more familiar names for each arguments. - echo r:myretprobe do_sys_open $rv $ra >> /sys/kernel/debug/tracing/kprobe_events + echo r:myretprobe do_sys_open $rv >> /sys/kernel/debug/tracing/kprobe_events This sets a kretprobe on the return point of do_sys_open() function with -recording return value and return address as "myretprobe" event. +recording return value as "myretprobe" event. You can see the format of these events via /sys/kernel/debug/tracing/events/kprobes/<EVENT>/format.
@@ -138,11 +137,11 @@ events, you need to enable it. # TASK-PID CPU# TIMESTAMP FUNCTION # | | | | | <...>-1447 [001] 1038282.286875: myprobe: (do_sys_open+0x0/0xd6) dfd=3 filename=7fffd1ec4440 flags=8000 mode=0 - <...>-1447 [001] 1038282.286878: myretprobe: (sys_openat+0xc/0xe <- do_sys_open) $rv=fffffffffffffffe $ra=ffffffff81367a3a + <...>-1447 [001] 1038282.286878: myretprobe: (sys_openat+0xc/0xe <- do_sys_open) $rv=fffffffffffffffe <...>-1447 [001] 1038282.286885: myprobe: (do_sys_open+0x0/0xd6) dfd=ffffff9c filename=40413c flags=8000 mode=1b6 - <...>-1447 [001] 1038282.286915: myretprobe: (sys_open+0x1b/0x1d <- do_sys_open) $rv=3 $ra=ffffffff81367a3a + <...>-1447 [001] 1038282.286915: myretprobe: (sys_open+0x1b/0x1d <- do_sys_open) $rv=3 <...>-1447 [001] 1038282.286969: myprobe: (do_sys_open+0x0/0xd6) dfd=ffffff9c filename=4041c6 flags=98800 mode=10 - <...>-1447 [001] 1038282.286976: myretprobe: (sys_open+0x1b/0x1d <- do_sys_open) $rv=3 $ra=ffffffff81367a3a + <...>-1447 [001] 1038282.286976: myretprobe: (sys_open+0x1b/0x1d <- do_sys_open) $rv=3 Each line shows when the kernel hits an event, and <- SYMBOL means kernel
kernel/trace/trace_kprobe.c +0 -11
@@ -85,11 +85,6 @@ static __kprobes unsigned long fetch_retvalue(struct pt_regs *regs,return regs_return_value (regs); } -static __kprobes unsigned long fetch_ip (struct pt_regs *regs,void *dummy) -{ -return instruction_pointer (regs); -} -static __kprobes unsigned long fetch_stack_address (struct pt_regs *regs,void *dummy) {
@@ -234,8 +229,6 @@ static int probe_arg_string(char *buf, size_t n, struct fetch_func *ff) ret =snprintf (buf, n,"@%s%+ld" , sc->symbol, sc->offset); }else if (ff->func == fetch_retvalue) ret =snprintf (buf, n,"$rv" ); -else if (ff->func == fetch_ip) - ret =snprintf (buf, n,"$ra" );else if (ff->func == fetch_stack_address) ret =snprintf (buf, n,"$sa" );else if (ff->func == fetch_indirect) {
@@ -448,9 +441,6 @@ static int parse_probe_vars(char *arg, struct fetch_func *ff, int is_return)if (is_return && arg[1 ] =='v' ) { ff->func = fetch_retvalue; ff->data =NULL ; - }else if (is_return && arg[1 ] =='a' ) { - ff->func = fetch_ip; - ff->data =NULL ; }else ret = -EINVAL ;break ;
@@ -560,7 +550,6 @@ static int create_trace_probe(int argc, char **argv)* Fetch args: * $aN : fetch Nth of function argument. (N:0-) * $rv : fetch return value -* $ra : fetch return address * $sa : fetch stack address * $sN : fetch Nth of stack (N:0-) * @ADDR : fetch memory at ADDR (ADDR should be in kernel)
Signature · 1 line
1.6.2.3
PATCH 3/7 tracing/kprobes: Make special variable names more self-explainable
In reply to Frederic WeisbeckerFrom: Masami Hiramatsu <[email protected]>
Rename special variables to more self-explainable names as below:
- $rv to $retval
- $sa to $stack
- $aN to $argN
- $sN to $stackN
LKML-Reference: <[email protected]>
--- Documentation/trace/kprobetrace.txt | 22 +++++++------- kernel/trace/trace_kprobe.c | 52 ++++++++++++++++------------------- 2 files changed, 35 insertions(+), 39 deletions(-)
Documentation/trace/kprobetrace.txt +11 -11
@@ -35,13 +35,13 @@ Synopsis of kprobe_events MEMADDR : Address where the probe is inserted. FETCHARGS : Arguments. Each probe can have up to 128 args. - %REG : Fetch register REG - @ADDR : Fetch memory at ADDR (ADDR should be in kernel) + %REG : Fetch register REG + @ADDR : Fetch memory at ADDR (ADDR should be in kernel) @SYM[+|-offs] : Fetch memory at SYM +|- offs (SYM should be a data symbol) - $sN : Fetch Nth entry of stack (N >= 0) - $sa : Fetch stack address. - $aN : Fetch function argument. (N >= 0)(*) - $rv : Fetch return value.(**) + $stackN : Fetch Nth entry of stack (N >= 0) + $stack : Fetch stack address. + $argN : Fetch function argument. (N >= 0)(*) + $retval : Fetch return value.(**) +|-offs(FETCHARG) : Fetch memory at FETCHARG +|- offs address.(***) NAME=FETCHARG: Set NAME as the argument name of FETCHARG.
@@ -84,13 +84,13 @@ Usage examples To add a probe as a new event, write a new definition to kprobe_events as below. - echo p:myprobe do_sys_open dfd=$a0 filename=$a1 flags=$a2 mode=$a3 > /sys/kernel/debug/tracing/kprobe_events + echo p:myprobe do_sys_open dfd=$arg0 filename=$arg1 flags=$arg2 mode=$arg3 > /sys/kernel/debug/tracing/kprobe_events This sets a kprobe on the top of do_sys_open() function with recording 1st to 4th arguments as "myprobe" event. As this example shows, users can choose more familiar names for each arguments. - echo r:myretprobe do_sys_open $rv >> /sys/kernel/debug/tracing/kprobe_events + echo r:myretprobe do_sys_open $retval >> /sys/kernel/debug/tracing/kprobe_events This sets a kretprobe on the return point of do_sys_open() function with recording return value as "myretprobe" event.
@@ -137,11 +137,11 @@ events, you need to enable it. # TASK-PID CPU# TIMESTAMP FUNCTION # | | | | | <...>-1447 [001] 1038282.286875: myprobe: (do_sys_open+0x0/0xd6) dfd=3 filename=7fffd1ec4440 flags=8000 mode=0 - <...>-1447 [001] 1038282.286878: myretprobe: (sys_openat+0xc/0xe <- do_sys_open) $rv=fffffffffffffffe + <...>-1447 [001] 1038282.286878: myretprobe: (sys_openat+0xc/0xe <- do_sys_open) $retval=fffffffffffffffe <...>-1447 [001] 1038282.286885: myprobe: (do_sys_open+0x0/0xd6) dfd=ffffff9c filename=40413c flags=8000 mode=1b6 - <...>-1447 [001] 1038282.286915: myretprobe: (sys_open+0x1b/0x1d <- do_sys_open) $rv=3 + <...>-1447 [001] 1038282.286915: myretprobe: (sys_open+0x1b/0x1d <- do_sys_open) $retval=3 <...>-1447 [001] 1038282.286969: myprobe: (do_sys_open+0x0/0xd6) dfd=ffffff9c filename=4041c6 flags=98800 mode=10 - <...>-1447 [001] 1038282.286976: myretprobe: (sys_open+0x1b/0x1d <- do_sys_open) $rv=3 + <...>-1447 [001] 1038282.286976: myretprobe: (sys_open+0x1b/0x1d <- do_sys_open) $retval=3 Each line shows when the kernel hits an event, and <- SYMBOL means kernel
kernel/trace/trace_kprobe.c +24 -28
96 lines not on this page · raw message
Signature · 1 line
1.6.2.3
PATCH 4/7 tracing/kprobes: Avoid field name confliction
In reply to Frederic WeisbeckerFrom: Masami Hiramatsu <[email protected]>
Check whether the argument name is in conflict with other field names while creating a kprobe through the debugfs interface.
Changes in v3:
- Check strcmp() == 0 instead of !strcmp().
Changes in v2:
- Add common_lock_depth to reserved name list.
LKML-Reference: <[email protected]>
--- kernel/trace/trace_kprobe.c | 65 +++++++++++++++++++++++++++++++++++-------- 1 files changed, 53 insertions(+), 12 deletions(-)
kernel/trace/trace_kprobe.c +53 -12
@@ -38,6 +38,25 @@#define MAX_EVENT_NAME_LEN 64#define KPROBE_EVENT_SYSTEM "kprobes" +/* Reserved field names */ +#define FIELD_STRING_IP "ip" +#define FIELD_STRING_NARGS "nargs" +#define FIELD_STRING_RETIP "ret_ip" +#define FIELD_STRING_FUNC "func" + +const char *reserved_field_names[] = { +"common_type" , +"common_flags" , +"common_preempt_count" , +"common_pid" , +"common_tgid" , +"common_lock_depth" , +FIELD_STRING_IP , +FIELD_STRING_NARGS , +FIELD_STRING_RETIP , +FIELD_STRING_FUNC , +}; +/* currently, trace_kprobe only supports X86. */ struct fetch_func {
@@ -537,6 +556,20 @@ static int parse_probe_arg(char *arg, struct fetch_func *ff, int is_return)return ret; } +/* Return 1 if name is reserved or already used by another argument */ +static int conflict_field_name (const char *name, +struct probe_arg *args,int narg) +{ +int i; +for (i =0 ; i <ARRAY_SIZE (reserved_field_names); i++) +if (strcmp (reserved_field_names[i], name) ==0 ) +return 1 ; +for (i =0 ; i < narg; i++) +if (strcmp (args[i].name, name) ==0 ) +return 1 ; +return 0 ; +} +static int create_trace_probe (int argc,char **argv) {/*
@@ -637,6 +670,12 @@ static int create_trace_probe(int argc, char **argv) *arg++ ='\0' ; else arg = argv[i]; + +if (conflict_field_name (argv[i], tp->args, i)) { + ret = -EINVAL ; + goto error; + } + tp->args[i].name =kstrdup (argv[i],GFP_KERNEL );/* Parse fetch argument */
@@ -1039,8 +1078,8 @@ static int kprobe_event_define_fields(struct ftrace_event_call *event_call)if (!ret)return ret; - DEFINE_FIELD(unsigned long, ip, "ip", 0); -DEFINE_FIELD (int, nargs,"nargs" ,1 ); + DEFINE_FIELD(unsigned long, ip, FIELD_STRING_IP, 0); +DEFINE_FIELD (int, nargs,FIELD_STRING_NARGS ,1 );/* Set argument names as fields */ for (i =0 ; i < tp->nr_args; i++) DEFINE_FIELD(unsigned long, args[i], tp->args[i].name, 0);
@@ -1057,9 +1096,9 @@ static int kretprobe_event_define_fields(struct ftrace_event_call *event_call)if (!ret)return ret; - DEFINE_FIELD(unsigned long, func, "func", 0); - DEFINE_FIELD(unsigned long, ret_ip, "ret_ip", 0); -DEFINE_FIELD (int, nargs,"nargs" ,1 ); + DEFINE_FIELD(unsigned long, func, FIELD_STRING_FUNC, 0); + DEFINE_FIELD(unsigned long, ret_ip, FIELD_STRING_RETIP, 0); +DEFINE_FIELD (int, nargs,FIELD_STRING_NARGS ,1 );/* Set argument names as fields */ for (i =0 ; i < tp->nr_args; i++) DEFINE_FIELD(unsigned long, args[i], tp->args[i].name, 0);
@@ -1108,15 +1147,16 @@ static int kprobe_event_show_format(struct ftrace_event_call *call,int ret, i;struct trace_probe *tp = (struct trace_probe *)call->data; - SHOW_FIELD(unsigned long, ip, "ip"); -SHOW_FIELD (int, nargs,"nargs" ); + SHOW_FIELD(unsigned long, ip, FIELD_STRING_IP); +SHOW_FIELD (int, nargs,FIELD_STRING_NARGS );/* Show fields */ for (i =0 ; i < tp->nr_args; i++) SHOW_FIELD(unsigned long, args[i], tp->args[i].name);trace_seq_puts (s,"\n" ); -return __probe_event_show_format (s, tp,"(%lx)" ,"REC->ip" ); +return __probe_event_show_format (s, tp,"(%lx)" , +"REC->" FIELD_STRING_IP ); } static int kretprobe_event_show_format(struct ftrace_event_call *call,
@@ -1126,9 +1166,9 @@ static int kretprobe_event_show_format(struct ftrace_event_call *call,int ret, i;struct trace_probe *tp = (struct trace_probe *)call->data; - SHOW_FIELD(unsigned long, func, "func"); - SHOW_FIELD(unsigned long, ret_ip, "ret_ip"); -SHOW_FIELD (int, nargs,"nargs" ); + SHOW_FIELD(unsigned long, func, FIELD_STRING_FUNC); + SHOW_FIELD(unsigned long, ret_ip, FIELD_STRING_RETIP); +SHOW_FIELD (int, nargs,FIELD_STRING_NARGS );/* Show fields */ for (i =0 ; i < tp->nr_args; i++)
@@ -1136,7 +1176,8 @@ static int kretprobe_event_show_format(struct ftrace_event_call *call,trace_seq_puts (s,"\n" );return __probe_event_show_format (s, tp,"(%lx <- %lx)" , -"REC->func, REC->ret_ip" ); +"REC->" FIELD_STRING_FUNC +", REC->" FIELD_STRING_RETIP ); }#ifdef CONFIG_EVENT_PROFILE
Signature · 1 line
1.6.2.3
PATCH 5/7 tracing/kprobes: Robustify fixed field names against variable field names conflicts
In reply to Frederic WeisbeckerFrom: Masami Hiramatsu <[email protected]>
Rename probe-common fixed field names to harder conflictable names, because current 'ip', 'func', and other probe field names are easily in conflict with user-specified variable names.
LKML-Reference: <[email protected]>
--- kernel/trace/trace_kprobe.c | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-)
kernel/trace/trace_kprobe.c +4 -4
@@ -39,10 +39,10 @@#define KPROBE_EVENT_SYSTEM "kprobes"/* Reserved field names */ -#define FIELD_STRING_IP "ip" -#define FIELD_STRING_NARGS "nargs" -#define FIELD_STRING_RETIP "ret_ip" -#define FIELD_STRING_FUNC "func" +#define FIELD_STRING_IP "__probe_ip" +#define FIELD_STRING_NARGS "__probe_nargs" +#define FIELD_STRING_RETIP "__probe_ret_ip" +#define FIELD_STRING_FUNC "__probe_func"const char *reserved_field_names[] = {"common_type" ,
Signature · 1 line
1.6.2.3
PATCH 6/7 perf: Add perf probe subcommand, a kprobe-event setup helper
In reply to Frederic WeisbeckerFrom: Masami Hiramatsu <[email protected]>
Add perf probe subcommand that implements a kprobe-event setup helper to the perf command.
This allows user to define kprobe events using C expressions (C line numbers, C function names, and C local variables).
Usage
-----
perf probe [<options>] -P 'PROBEDEF' [-P 'PROBEDEF' ...]
-k, --vmlinux <file> vmlinux/module pathname
-P, --probe <p|r:[GRP/]NAME FUNC[+OFFS][@SRC]|@SRC:LINE [ARG ...]>
probe point definition, where
p: kprobe probe
r: kretprobe probe
GRP: Group name (optional)
NAME: Event name
FUNC: Function name
OFFS: Offset from function entry (in byte)
SRC: Source code path
LINE: Line number
ARG: Probe argument (local variable name or
kprobe-tracer argument format is supported.)
Changes in v4:
- Add _GNU_SOURCE macro for strndup().
Changes in v3:
- Remove -r option because perf always be used for online kernel.
- Check malloc/calloc results.
Changes in v2:
- Check synthesized string length.
- Rename perf kprobe to perf probe.
- Use spaces for separator and update usage comment.
- Check error paths in parse_probepoint().
- Check optimized-out variables.
LKML-Reference: <[email protected]>
--- tools/perf/Makefile | 10 + tools/perf/builtin-probe.c | 358 +++++++++++++++++++++ tools/perf/builtin.h | 1 + tools/perf/perf.c | 3 + tools/perf/util/probe-finder.c | 690 ++++++++++++++++++++++++++++++++++++++++ tools/perf/util/probe-finder.h | 68 ++++ 6 files changed, 1130 insertions(+), 0 deletions(-) create mode 100644 tools/perf/builtin-probe.c create mode 100644 tools/perf/util/probe-finder.c create mode 100644 tools/perf/util/probe-finder.h
tools/perf/Makefile +10 -0
@@ -419,6 +419,16 @@ ifneq ($(shell sh -c "(echo '\#include <libelf.h>'; echo 'int main(void) { Elf * msg := $(error No libelf.h/libelf found, please install libelf-dev/elfutils-libelf-devel); endif +ifneq ($(shell sh -c "(echo '\#include <libdwarf/dwarf.h>'; echo '\#include <libdwarf/libdwarf.h>'; echo 'int main(void) { Dwarf_Debug dbg; Dwarf_Error err; dwarf_init(0, DW_DLC_READ, 0, 0, &dbg, &err); return (long)dbg; }') | $(CC) -x c - $(ALL_CFLAGS) -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -ldwarf -lelf -o /dev/null $(ALL_LDFLAGS) > /dev/null 2>&1 && echo y"), y) + msg := $(warning No libdwarf.h found, disables probe subcommand. Please install libdwarf-dev/libdwarf-devel); +else + EXTLIBS += -lelf -ldwarf + LIB_H += util/probe-finder.h + LIB_OBJS += util/probe-finder.o + BUILTIN_OBJS += builtin-probe.o + BASIC_CFLAGS += -DSUPPORT_DWARF +endif + ifdef NO_DEMANGLE BASIC_CFLAGS += -DNO_DEMANGLE else
tools/perf/builtin-probe.c new file +358 -0
358 lines not on this page · raw message
tools/perf/builtin.h +1 -0
@@ -25,5 +25,6 @@ extern int cmd_timechart(int argc, const char **argv, const char *prefix);extern int cmd_top (int argc,const char **argv,const char *prefix);extern int cmd_trace (int argc,const char **argv,const char *prefix);extern int cmd_version (int argc,const char **argv,const char *prefix); +extern int cmd_probe (int argc,const char **argv,const char *prefix);#endif
tools/perf/perf.c +3 -0
@@ -295,6 +295,9 @@ static void handle_internal_command(int argc, const char **argv) { "version", cmd_version, 0 }, { "trace", cmd_trace, 0 }, { "sched", cmd_sched, 0 }, +#ifdef SUPPORT_DWARF + { "probe", cmd_probe, 0 }, +#endif };unsigned int i;static const char ext[] =STRIP_EXTENSION ;
tools/perf/util/probe-finder.c new file +690 -0
690 lines not on this page · raw message
tools/perf/util/probe-finder.h new file +68 -0
68 lines not on this page · raw message
Signature · 1 line
1.6.2.3
PATCH 7/7 perf probe: Add perf probe command support without libdwarf
In reply to Frederic WeisbeckerFrom: Masami Hiramatsu <[email protected]>
Enables 'perf probe' even if libdwarf is not installed. If libdwarf is not found, 'perf probe' just disables dwarf support. Users can use 'perf probe' to set up new events by using kprobe_events format.
LKML-Reference: <[email protected]>
--- tools/perf/Makefile | 6 ++-- tools/perf/builtin-probe.c | 42 +++++++++++++++++++++++++++++++++------ tools/perf/perf.c | 2 - tools/perf/util/probe-finder.h | 2 + 4 files changed, 40 insertions(+), 12 deletions(-)
tools/perf/Makefile +3 -3
@@ -385,6 +385,7 @@ BUILTIN_OBJS += builtin-stat.o BUILTIN_OBJS += builtin-timechart.o BUILTIN_OBJS += builtin-top.o BUILTIN_OBJS += builtin-trace.o +BUILTIN_OBJS += builtin-probe.o PERFLIBS = $(LIB_FILE)
@@ -420,13 +421,12 @@ ifneq ($(shell sh -c "(echo '\#include <libelf.h>'; echo 'int main(void) { Elf * endif ifneq ($(shell sh -c "(echo '\#include <libdwarf/dwarf.h>'; echo '\#include <libdwarf/libdwarf.h>'; echo 'int main(void) { Dwarf_Debug dbg; Dwarf_Error err; dwarf_init(0, DW_DLC_READ, 0, 0, &dbg, &err); return (long)dbg; }') | $(CC) -x c - $(ALL_CFLAGS) -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -ldwarf -lelf -o /dev/null $(ALL_LDFLAGS) > /dev/null 2>&1 && echo y"), y) - msg := $(warning No libdwarf.h found, disables probe subcommand. Please install libdwarf-dev/libdwarf-devel); + msg := $(warning No libdwarf.h found, disables dwarf support. Please install libdwarf-dev/libdwarf-devel); + BASIC_CFLAGS += -DNO_LIBDWARF else EXTLIBS += -lelf -ldwarf LIB_H += util/probe-finder.h LIB_OBJS += util/probe-finder.o - BUILTIN_OBJS += builtin-probe.o - BASIC_CFLAGS += -DSUPPORT_DWARF endif ifdef NO_DEMANGLE
tools/perf/builtin-probe.c +35 -7
119 lines not on this page · raw message
tools/perf/perf.c +0 -2
@@ -295,9 +295,7 @@ static void handle_internal_command(int argc, const char **argv) { "version", cmd_version, 0 }, { "trace", cmd_trace, 0 }, { "sched", cmd_sched, 0 }, -#ifdef SUPPORT_DWARF { "probe", cmd_probe, 0 }, -#endif };unsigned int i;static const char ext[] =STRIP_EXTENSION ;
tools/perf/util/probe-finder.h +2 -0
@@ -44,6 +44,7 @@ struct probe_point {char *probes[MAX_PROBES ];/* Output buffers (will be allocated)*/ }; +#ifndef NO_LIBDWARF extern int find_probepoint (int fd,struct probe_point *pp);#include <libdwarf/dwarf.h>
@@ -64,5 +65,6 @@ struct probe_finder {char *buf;/* Current output buffer */ int len;/* Length of output buffer */ }; +#endif /* NO_LIBDWARF */ #endif /*_PROBE_FINDER_H */
Signature · 1 line
1.6.2.3
Quoted 8 lines · Frederic Weisbecker
Ingo,
I've tested a little this set and it seems to work well. I can't test the C expression parser (need some libdwarf cross-version compatiblity), but I can create kprobes via low-level parameters, then launch perf record and perf trace successfully. I'll need to update perf trace to handle variables that start with '$' characters though.
Quoted 7 lines
Anyway, it's still unstable but I guess pulling it would help Masami to continue without the need to handle the whole batch.
Please pull the tracing/kprobes branch that can be found at:
git://git.kernel.org/pub/scm/linux/kernel/git/frederic/random-tracing.git
tracing/kprobes
Pulled, thanks Frederic.
Quoted 4 lines
Documentation/trace/kprobetrace.txt | 27 +- kernel/trace/trace_kprobe.c | 160 +++++--- tools/perf/Makefile | 10 + tools/perf/builtin-probe.c | 386 +++++++++++++++++++
Quoted 8 lines
tools/perf/builtin.h | 1 + tools/perf/perf.c | 1 + tools/perf/util/probe-finder.c | 690 +++++++++++++++++++++++++++++++++++ tools/perf/util/probe-finder.h | 70 ++++ 8 files changed, 1271 insertions(+), 74 deletions(-) create mode 100644 tools/perf/builtin-probe.c create mode 100644 tools/perf/util/probe-finder.c create mode 100644 tools/perf/util/probe-finder.h
Hm, there's no x86 decoder change here - does this lot fixes the instruction decoder bug i reported some time ago? That is which keeps tracing/kprobes out of tip:master.
Ingo
Quoted 11 lines · Ingo Molnar
Quoted 10 lines
tools/perf/Makefile | 10 + tools/perf/builtin-probe.c | 386 +++++++++++++++++++ tools/perf/builtin.h | 1 + tools/perf/perf.c | 1 + tools/perf/util/probe-finder.c | 690 +++++++++++++++++++++++++++++++++++ tools/perf/util/probe-finder.h | 70 ++++ 8 files changed, 1271 insertions(+), 74 deletions(-) create mode 100644 tools/perf/builtin-probe.c create mode 100644 tools/perf/util/probe-finder.c create mode 100644 tools/perf/util/probe-finder.h
Quoted 3 lines
Hm, there's no x86 decoder change here - does this lot fixes the instruction decoder bug i reported some time ago? That is which keeps tracing/kprobes out of tip:master.
See my previous report below.
Ingo
----- Forwarded message from Ingo Molnar <[email protected]> -----
Date: Sat, 3 Oct 2009 12:49:23 +0200
From: Ingo Molnar <[email protected]>
To: Frederic Weisbecker <[email protected]>
Subject: Re: [GIT PULL] tracing/kprobes: kprobes fixes
Cc: LKML <[email protected]>,
Steven Rostedt <[email protected]>, Li Zefan <[email protected]>
ok, -tip testing found this new failure:
TEST posttest
Error: ffffffff8106ac00: 66 0f 73 fd 04 pslldq $0x4,%xmm5
Error: objdump says 5 bytes, but insn_get_length() says 4 (attr:8000) make[1]: *** [posttest] Error 2
make: *** [bzImage] Error 2
Ingo
----- End forwarded message -----
Quoted 35 lines · Ingo Molnar
Quoted 11 lines · Ingo Molnar
Quoted 10 lines
tools/perf/Makefile | 10 + tools/perf/builtin-probe.c | 386 +++++++++++++++++++ tools/perf/builtin.h | 1 + tools/perf/perf.c | 1 + tools/perf/util/probe-finder.c | 690 +++++++++++++++++++++++++++++++++++ tools/perf/util/probe-finder.h | 70 ++++ 8 files changed, 1271 insertions(+), 74 deletions(-) create mode 100644 tools/perf/builtin-probe.c create mode 100644 tools/perf/util/probe-finder.c create mode 100644 tools/perf/util/probe-finder.hQuoted 3 lines
Hm, there's no x86 decoder change here - does this lot fixes the instruction decoder bug i reported some time ago? That is which keeps tracing/kprobes out of tip:master.
See my previous report below.
Ingo
----- Forwarded message from Ingo Molnar <[email protected]> -----
Date: Sat, 3 Oct 2009 12:49:23 +0200
From: Ingo Molnar <[email protected]>
To: Frederic Weisbecker <[email protected]>
Subject: Re: [GIT PULL] tracing/kprobes: kprobes fixes
Cc: LKML <[email protected]>,
Steven Rostedt <[email protected]>, Li Zefan <[email protected]>ok, -tip testing found this new failure:
Quoted 8 lines
TEST posttest
Error:ffffffff8106ac00: 66 0f 73 fd 04 pslldq $0x4,%xmm5
Error: objdump says 5 bytes, butinsn_get_length()says 4 (attr:8000) make[1]: *** [posttest] Error 2
make: *** [bzImage] Error 2Ingo
Hmm, thank you for reporting, I've missed some group opcode...
Thank you again,
Signature · 7 lines
Masami Hiramatsu Software Engineer Hitachi Computer Products (America), Inc. Software Solutions Division e-mail: [email protected]
Quoted 34 lines · Ingo Molnar
Quoted 8 lines · Frederic Weisbecker
Ingo,
I've tested a little this set and it seems to work well. I can't test the C expression parser (need some libdwarf cross-version compatiblity), but I can create kprobes via low-level parameters, then launch perf record and perf trace successfully. I'll need to update perf trace to handle variables that start with '$' characters though.
Quoted 7 lines
Anyway, it's still unstable but I guess pulling it would help Masami to continue without the need to handle the whole batch.
Please pull the tracing/kprobes branch that can be found at:
git://git.kernel.org/pub/scm/linux/kernel/git/frederic/random-tracing.git
tracing/kprobesPulled, thanks Frederic.
Quoted 4 lines
Documentation/trace/kprobetrace.txt | 27 +- kernel/trace/trace_kprobe.c | 160 +++++--- tools/perf/Makefile | 10 + tools/perf/builtin-probe.c | 386 +++++++++++++++++++Quoted 8 lines
tools/perf/builtin.h | 1 + tools/perf/perf.c | 1 + tools/perf/util/probe-finder.c | 690 +++++++++++++++++++++++++++++++++++ tools/perf/util/probe-finder.h | 70 ++++ 8 files changed, 1271 insertions(+), 74 deletions(-) create mode 100644 tools/perf/builtin-probe.c create mode 100644 tools/perf/util/probe-finder.c create mode 100644 tools/perf/util/probe-finder.h
Quoted 5 lines
Hm, there's no x86 decoder change here - does this lot fixes the instruction decoder bug i reported some time ago? That is which keeps tracing/kprobes out of tip:master.
Ingo
Not yet, I hope it's part of the last family of instruction set we are missing.
Quoted 39 lines · Frederic Weisbecker
Quoted 38 lines · Ingo Molnar
Quoted 8 lines · Frederic Weisbecker
Ingo,
I've tested a little this set and it seems to work well. I can't test the C expression parser (need some libdwarf cross-version compatiblity), but I can create kprobes via low-level parameters, then launch perf record and perf trace successfully. I'll need to update perf trace to handle variables that start with '$' characters though.
Quoted 7 lines
Anyway, it's still unstable but I guess pulling it would help Masami to continue without the need to handle the whole batch.
Please pull the tracing/kprobes branch that can be found at:
git://git.kernel.org/pub/scm/linux/kernel/git/frederic/random-tracing.git
tracing/kprobesPulled, thanks Frederic.
Quoted 4 lines
Documentation/trace/kprobetrace.txt | 27 +- kernel/trace/trace_kprobe.c | 160 +++++--- tools/perf/Makefile | 10 + tools/perf/builtin-probe.c | 386 +++++++++++++++++++Quoted 8 lines
tools/perf/builtin.h | 1 + tools/perf/perf.c | 1 + tools/perf/util/probe-finder.c | 690 +++++++++++++++++++++++++++++++++++ tools/perf/util/probe-finder.h | 70 ++++ 8 files changed, 1271 insertions(+), 74 deletions(-) create mode 100644 tools/perf/builtin-probe.c create mode 100644 tools/perf/util/probe-finder.c create mode 100644 tools/perf/util/probe-finder.hHm, there's no x86 decoder change here - does this lot fixes the instruction decoder bug i reported some time ago? That is which keeps tracing/kprobes out of tip:master.
Quoted 5 lines
Quoted 1 line
Ingo
Not yet, I hope it's part of the last family of instruction set we are missing.
ok.
We dont want topics to not be in tip:master for too long - we dont grow conflict resolutions nor does it get tested widely.
Ingo