Skip to contentSkip to conversation

tracing/kprobes: Syntax updates, introduce perf probe

GIT PULL In review Frederic Weisbecker 3 participants 15 messages Oct 12, 2009 – Oct 13, 2009

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

Ingo,

I've tested a little this set and it seems to work well. I can't test the C expression parser (need some libdwarf cross-version compatiblity), but I can create kprobes via low-level parameters, then launch perf record and perf trace successfully. I'll need to update perf trace to handle variables that start with '$' characters though.

Anyway, it's still unstable but I guess pulling it would help Masami to continue without the need to handle the whole batch.

Please pull the tracing/kprobes branch that can be found at:

git://git.kernel.org/pub/scm/linux/kernel/git/frederic/random-tracing.git
tracing/kprobes

Thanks,
Frederic.

Masami Hiramatsu (7):
tracing/kprobes: Add $ prefix to special variables
tracing/kprobes: Remove '$ra' special variable
tracing/kprobes: Make special variable names more self-explainable tracing/kprobes: Avoid field name confliction
tracing/kprobes: Robustify fixed field names against variable field names conflicts
perf: Add perf probe subcommand, a kprobe-event setup helper
perf probe: Add perf probe command support without libdwarf

 Documentation/trace/kprobetrace.txt |   27 +-
 kernel/trace/trace_kprobe.c         |  160 +++++---
 tools/perf/Makefile                 |   10 +
 tools/perf/builtin-probe.c          |  386 +++++++++++++++++++
 tools/perf/builtin.h                |    1 +
 tools/perf/perf.c                   |    1 +
 tools/perf/util/probe-finder.c      |  690 +++++++++++++++++++++++++++++++++++
 tools/perf/util/probe-finder.h      |   70 ++++
 8 files changed, 1271 insertions(+), 74 deletions(-)
 create mode 100644 tools/perf/builtin-probe.c
 create mode 100644 tools/perf/util/probe-finder.c
 create mode 100644 tools/perf/util/probe-finder.h
#1 lore raw

PATCH 1/1 kill-the-bkl/reiserfs: fix reiserfs lock to cpu_add_remove_lock dependency

While creating the reiserfs workqueue during the journal initialization, we are holding the reiserfs lock, but create_workqueue() also holds the cpu_add_remove_lock, creating then the following dependency:

- reiserfs lock -> cpu_add_remove_lock

But we also have the following existing dependencies:

- mm->mmap_sem -> reiserfs lock
- cpu_add_remove_lock -> cpu_hotplug.lock -> slub_lock -> sysfs_mutex

The merged dependency chain then becomes:

- mm->mmap_sem -> reiserfs lock -> cpu_add_remove_lock ->

	cpu_hotplug.lock -> slub_lock -> sysfs_mutex

But when we fill a dir entry in sysfs_readir(), we are holding the sysfs_mutex and we also might fault while copying the directory entry to the user, leading to the following dependency:

- sysfs_mutex -> mm->mmap_sem

The end result is then a lock inversion between sysfs_mutex and mm->mmap_sem, as reported in the following lockdep warning:

[ INFO: possible circular locking dependency detected ] 2.6.31-07095-g25a3912 #4
-------------------------------------------------------
udevadm/790 is trying to acquire lock:
(&mm->mmap_sem){++++++}, at: [<c1098942>] might_fault+0x72/0xc0

but task is already holding lock:
(sysfs_mutex){+.+.+.}, at: [<c110813c>] sysfs_readdir+0x7c/0x260

which lock already depends on the new lock.

the existing dependency chain (in reverse order) is:

-> #5 (sysfs_mutex){+.+.+.}:

      [...]

-> #4 (slub_lock){+++++.}:

      [...]

-> #3 (cpu_hotplug.lock){+.+.+.}:

      [...]

-> #2 (cpu_add_remove_lock){+.+.+.}:

      [...]

-> #1 (&REISERFS_SB(s)->lock){+.+.+.}:

      [...]

-> #0 (&mm->mmap_sem){++++++}:

      [...]

This can be fixed by relaxing the reiserfs lock while creating the workqueue.
This is fine to relax the lock here, we just keep it around to pass through reiserfs lock checks and for paranoid reasons.

---
 fs/reiserfs/journal.c |    5 ++++-
 1 files changed, 4 insertions(+), 1 deletions(-)
fs/reiserfs/journal.c +4 -1
index 04e3c42..2f8a7e7 100644
@@ -2933,8 +2933,11 @@ int journal_init(struct super_block *sb, const char *j_dev_name,
 	}
 
 	reiserfs_mounted_fs_count++;
-	if (reiserfs_mounted_fs_count <= 1)
+	if (reiserfs_mounted_fs_count <= 1) {
+		reiserfs_write_unlock(sb);
 		commit_wq = create_workqueue("reiserfs");
+		reiserfs_write_lock(sb);
+	}
 
 	INIT_DELAYED_WORK(&journal->j_work, flush_async_commits);
 	journal->j_work_sb = sb;
Signature · 1 line
1.6.2.3
#2 lore raw

Re: [PATCH] kill-the-bkl/reiserfs: fix reiserfs lock to cpu_add_remove_lock dependency

Quoted 4 lines · Frederic Weisbecker

This can be fixed by relaxing the reiserfs lock while creating the workqueue.
This is fine to relax the lock here, we just keep it around to pass through reiserfs lock checks and for paranoid reasons.

Nothing related to kprobes that said...
Sorry for the noise. I've already posted this patch, but it looks like these locking inversions don't want to live their own life and follow me everywhere...

#3 lore raw

PATCH 1/7 tracing/kprobes: Add $ prefix to special variables

In reply to Frederic Weisbecker

From: Masami Hiramatsu <[email protected]>

Add $ prefix to the special variables(e.g. sa, rv) of kprobe-tracer. This resolves consistency issues between kprobe_events and perf-kprobe.

The main goal is to avoid conflicts between local variable names of probed functions, used by perf probe, and special variables used in the kprobe event creation interface (stack values, etc...) and also available from perf probe.

ie: we don't want rv (return value) to conflict with a local variable named rv in a probed function.

Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Mike Galbraith <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Christoph Hellwig <[email protected]>
Cc Ananth N Mavinakayanahalli <[email protected]>
Cc Jim Keniston <[email protected]>
Cc Frank Ch. Eigler <[email protected]>

LKML-Reference: <[email protected]>

---
 Documentation/trace/kprobetrace.txt |   20 ++++++------
 kernel/trace/trace_kprobe.c         |   60 +++++++++++++++++++++-------------
 2 files changed, 47 insertions(+), 33 deletions(-)
Documentation/trace/kprobetrace.txt +10 -10
index 9b8f7c6..33f5318 100644
@@ -36,13 +36,13 @@ Synopsis of kprobe_events
 
  FETCHARGS	: Arguments. Each probe can have up to 128 args.
   %REG	: Fetch register REG
-  sN	: Fetch Nth entry of stack (N >= 0)
-  sa	: Fetch stack address.
   @ADDR	: Fetch memory at ADDR (ADDR should be in kernel)
   @SYM[+|-offs]	: Fetch memory at SYM +|- offs (SYM should be a data symbol)
-  aN	: Fetch function argument. (N >= 0)(*)
-  rv	: Fetch return value.(**)
-  ra	: Fetch return address.(**)
+  $sN	: Fetch Nth entry of stack (N >= 0)
+  $sa	: Fetch stack address.
+  $aN	: Fetch function argument. (N >= 0)(*)
+  $rv	: Fetch return value.(**)
+  $ra	: Fetch return address.(**)
   +|-offs(FETCHARG) : Fetch memory at FETCHARG +|- offs address.(***)
   NAME=FETCHARG: Set NAME as the argument name of FETCHARG.
 
@@ -85,13 +85,13 @@ Usage examples
 To add a probe as a new event, write a new definition to kprobe_events
 as below.
 
-  echo p:myprobe do_sys_open dfd=a0 filename=a1 flags=a2 mode=a3 > /sys/kernel/debug/tracing/kprobe_events
+  echo p:myprobe do_sys_open dfd=$a0 filename=$a1 flags=$a2 mode=$a3 > /sys/kernel/debug/tracing/kprobe_events
 
  This sets a kprobe on the top of do_sys_open() function with recording
 1st to 4th arguments as "myprobe" event. As this example shows, users can
 choose more familiar names for each arguments.
 
-  echo r:myretprobe do_sys_open rv ra >> /sys/kernel/debug/tracing/kprobe_events
+  echo r:myretprobe do_sys_open $rv $ra >> /sys/kernel/debug/tracing/kprobe_events
 
  This sets a kretprobe on the return point of do_sys_open() function with
 recording return value and return address as "myretprobe" event.
@@ -138,11 +138,11 @@ events, you need to enable it.
 #           TASK-PID    CPU#    TIMESTAMP  FUNCTION
 #              | |       |          |         |
            <...>-1447  [001] 1038282.286875: myprobe: (do_sys_open+0x0/0xd6) dfd=3 filename=7fffd1ec4440 flags=8000 mode=0
-           <...>-1447  [001] 1038282.286878: myretprobe: (sys_openat+0xc/0xe <- do_sys_open) rv=fffffffffffffffe ra=ffffffff81367a3a
+           <...>-1447  [001] 1038282.286878: myretprobe: (sys_openat+0xc/0xe <- do_sys_open) $rv=fffffffffffffffe $ra=ffffffff81367a3a
            <...>-1447  [001] 1038282.286885: myprobe: (do_sys_open+0x0/0xd6) dfd=ffffff9c filename=40413c flags=8000 mode=1b6
-           <...>-1447  [001] 1038282.286915: myretprobe: (sys_open+0x1b/0x1d <- do_sys_open) rv=3 ra=ffffffff81367a3a
+           <...>-1447  [001] 1038282.286915: myretprobe: (sys_open+0x1b/0x1d <- do_sys_open) $rv=3 $ra=ffffffff81367a3a
            <...>-1447  [001] 1038282.286969: myprobe: (do_sys_open+0x0/0xd6) dfd=ffffff9c filename=4041c6 flags=98800 mode=10
-           <...>-1447  [001] 1038282.286976: myretprobe: (sys_open+0x1b/0x1d <- do_sys_open) rv=3 ra=ffffffff81367a3a
+           <...>-1447  [001] 1038282.286976: myretprobe: (sys_open+0x1b/0x1d <- do_sys_open) $rv=3 $ra=ffffffff81367a3a
 
 
  Each line shows when the kernel hits an event, and <- SYMBOL means kernel
kernel/trace/trace_kprobe.c +37 -23

112 lines not on this page · raw message

Signature · 1 line
1.6.2.3
#4 lore raw

PATCH 2/7 tracing/kprobes: Remove '$ra' special variable

In reply to Frederic Weisbecker

From: Masami Hiramatsu <[email protected]>

Remove '$ra' (return address) because it is already shown at the head of each entry.

Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Mike Galbraith <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Christoph Hellwig <[email protected]>
Cc Ananth N Mavinakayanahalli <[email protected]>
Cc Jim Keniston <[email protected]>
Cc Frank Ch. Eigler <[email protected]>

LKML-Reference: <[email protected]>

---
 Documentation/trace/kprobetrace.txt |   13 ++++++-------
 kernel/trace/trace_kprobe.c         |   11 -----------
 2 files changed, 6 insertions(+), 18 deletions(-)
Documentation/trace/kprobetrace.txt +6 -7
index 33f5318..4208253 100644
@@ -1,4 +1,4 @@
-                         Kprobe-based Event Tracer
+                        Kprobe-based Event Tracer
                          =========================
 
                  Documentation is written by Masami Hiramatsu
@@ -42,7 +42,6 @@ Synopsis of kprobe_events
   $sa	: Fetch stack address.
   $aN	: Fetch function argument. (N >= 0)(*)
   $rv	: Fetch return value.(**)
-  $ra	: Fetch return address.(**)
   +|-offs(FETCHARG) : Fetch memory at FETCHARG +|- offs address.(***)
   NAME=FETCHARG: Set NAME as the argument name of FETCHARG.
 
@@ -91,10 +90,10 @@ as below.
 1st to 4th arguments as "myprobe" event. As this example shows, users can
 choose more familiar names for each arguments.
 
-  echo r:myretprobe do_sys_open $rv $ra >> /sys/kernel/debug/tracing/kprobe_events
+  echo r:myretprobe do_sys_open $rv >> /sys/kernel/debug/tracing/kprobe_events
 
  This sets a kretprobe on the return point of do_sys_open() function with
-recording return value and return address as "myretprobe" event.
+recording return value as "myretprobe" event.
  You can see the format of these events via
 /sys/kernel/debug/tracing/events/kprobes/<EVENT>/format.
 
@@ -138,11 +137,11 @@ events, you need to enable it.
 #           TASK-PID    CPU#    TIMESTAMP  FUNCTION
 #              | |       |          |         |
            <...>-1447  [001] 1038282.286875: myprobe: (do_sys_open+0x0/0xd6) dfd=3 filename=7fffd1ec4440 flags=8000 mode=0
-           <...>-1447  [001] 1038282.286878: myretprobe: (sys_openat+0xc/0xe <- do_sys_open) $rv=fffffffffffffffe $ra=ffffffff81367a3a
+           <...>-1447  [001] 1038282.286878: myretprobe: (sys_openat+0xc/0xe <- do_sys_open) $rv=fffffffffffffffe
            <...>-1447  [001] 1038282.286885: myprobe: (do_sys_open+0x0/0xd6) dfd=ffffff9c filename=40413c flags=8000 mode=1b6
-           <...>-1447  [001] 1038282.286915: myretprobe: (sys_open+0x1b/0x1d <- do_sys_open) $rv=3 $ra=ffffffff81367a3a
+           <...>-1447  [001] 1038282.286915: myretprobe: (sys_open+0x1b/0x1d <- do_sys_open) $rv=3
            <...>-1447  [001] 1038282.286969: myprobe: (do_sys_open+0x0/0xd6) dfd=ffffff9c filename=4041c6 flags=98800 mode=10
-           <...>-1447  [001] 1038282.286976: myretprobe: (sys_open+0x1b/0x1d <- do_sys_open) $rv=3 $ra=ffffffff81367a3a
+           <...>-1447  [001] 1038282.286976: myretprobe: (sys_open+0x1b/0x1d <- do_sys_open) $rv=3
 
 
  Each line shows when the kernel hits an event, and <- SYMBOL means kernel
kernel/trace/trace_kprobe.c +0 -11
index f63ead0..ba6d3bd 100644
@@ -85,11 +85,6 @@ static __kprobes unsigned long fetch_retvalue(struct pt_regs *regs,
 	return regs_return_value(regs);
 }
 
-static __kprobes unsigned long fetch_ip(struct pt_regs *regs, void *dummy)
-{
-	return instruction_pointer(regs);
-}
-
 static __kprobes unsigned long fetch_stack_address(struct pt_regs *regs,
 						   void *dummy)
 {
@@ -234,8 +229,6 @@ static int probe_arg_string(char *buf, size_t n, struct fetch_func *ff)
 		ret = snprintf(buf, n, "@%s%+ld", sc->symbol, sc->offset);
 	} else if (ff->func == fetch_retvalue)
 		ret = snprintf(buf, n, "$rv");
-	else if (ff->func == fetch_ip)
-		ret = snprintf(buf, n, "$ra");
 	else if (ff->func == fetch_stack_address)
 		ret = snprintf(buf, n, "$sa");
 	else if (ff->func == fetch_indirect) {
@@ -448,9 +441,6 @@ static int parse_probe_vars(char *arg, struct fetch_func *ff, int is_return)
 		if (is_return && arg[1] == 'v') {
 			ff->func = fetch_retvalue;
 			ff->data = NULL;
-		} else if (is_return && arg[1] == 'a') {
-			ff->func = fetch_ip;
-			ff->data = NULL;
 		} else
 			ret = -EINVAL;
 		break;
@@ -560,7 +550,6 @@ static int create_trace_probe(int argc, char **argv)
 	 * Fetch args:
 	 *  $aN	: fetch Nth of function argument. (N:0-)
 	 *  $rv	: fetch return value
-	 *  $ra	: fetch return address
 	 *  $sa	: fetch stack address
 	 *  $sN	: fetch Nth of stack (N:0-)
 	 *  @ADDR	: fetch memory at ADDR (ADDR should be in kernel)
Signature · 1 line
1.6.2.3
#5 lore raw

PATCH 3/7 tracing/kprobes: Make special variable names more self-explainable

In reply to Frederic Weisbecker

From: Masami Hiramatsu <[email protected]>

Rename special variables to more self-explainable names as below:
- $rv to $retval
- $sa to $stack
- $aN to $argN
- $sN to $stackN

Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Mike Galbraith <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Christoph Hellwig <[email protected]>
Cc Ananth N Mavinakayanahalli <[email protected]>
Cc Jim Keniston <[email protected]>
Cc Frank Ch. Eigler <[email protected]>

LKML-Reference: <[email protected]>

---
 Documentation/trace/kprobetrace.txt |   22 +++++++-------
 kernel/trace/trace_kprobe.c         |   52 ++++++++++++++++-------------------
 2 files changed, 35 insertions(+), 39 deletions(-)
Documentation/trace/kprobetrace.txt +11 -11
index 4208253..1541524 100644
@@ -35,13 +35,13 @@ Synopsis of kprobe_events
  MEMADDR	: Address where the probe is inserted.
 
  FETCHARGS	: Arguments. Each probe can have up to 128 args.
-  %REG	: Fetch register REG
-  @ADDR	: Fetch memory at ADDR (ADDR should be in kernel)
+  %REG		: Fetch register REG
+  @ADDR		: Fetch memory at ADDR (ADDR should be in kernel)
   @SYM[+|-offs]	: Fetch memory at SYM +|- offs (SYM should be a data symbol)
-  $sN	: Fetch Nth entry of stack (N >= 0)
-  $sa	: Fetch stack address.
-  $aN	: Fetch function argument. (N >= 0)(*)
-  $rv	: Fetch return value.(**)
+  $stackN	: Fetch Nth entry of stack (N >= 0)
+  $stack	: Fetch stack address.
+  $argN		: Fetch function argument. (N >= 0)(*)
+  $retval	: Fetch return value.(**)
   +|-offs(FETCHARG) : Fetch memory at FETCHARG +|- offs address.(***)
   NAME=FETCHARG: Set NAME as the argument name of FETCHARG.
 
@@ -84,13 +84,13 @@ Usage examples
 To add a probe as a new event, write a new definition to kprobe_events
 as below.
 
-  echo p:myprobe do_sys_open dfd=$a0 filename=$a1 flags=$a2 mode=$a3 > /sys/kernel/debug/tracing/kprobe_events
+  echo p:myprobe do_sys_open dfd=$arg0 filename=$arg1 flags=$arg2 mode=$arg3 > /sys/kernel/debug/tracing/kprobe_events
 
  This sets a kprobe on the top of do_sys_open() function with recording
 1st to 4th arguments as "myprobe" event. As this example shows, users can
 choose more familiar names for each arguments.
 
-  echo r:myretprobe do_sys_open $rv >> /sys/kernel/debug/tracing/kprobe_events
+  echo r:myretprobe do_sys_open $retval >> /sys/kernel/debug/tracing/kprobe_events
 
  This sets a kretprobe on the return point of do_sys_open() function with
 recording return value as "myretprobe" event.
@@ -137,11 +137,11 @@ events, you need to enable it.
 #           TASK-PID    CPU#    TIMESTAMP  FUNCTION
 #              | |       |          |         |
            <...>-1447  [001] 1038282.286875: myprobe: (do_sys_open+0x0/0xd6) dfd=3 filename=7fffd1ec4440 flags=8000 mode=0
-           <...>-1447  [001] 1038282.286878: myretprobe: (sys_openat+0xc/0xe <- do_sys_open) $rv=fffffffffffffffe
+           <...>-1447  [001] 1038282.286878: myretprobe: (sys_openat+0xc/0xe <- do_sys_open) $retval=fffffffffffffffe
            <...>-1447  [001] 1038282.286885: myprobe: (do_sys_open+0x0/0xd6) dfd=ffffff9c filename=40413c flags=8000 mode=1b6
-           <...>-1447  [001] 1038282.286915: myretprobe: (sys_open+0x1b/0x1d <- do_sys_open) $rv=3
+           <...>-1447  [001] 1038282.286915: myretprobe: (sys_open+0x1b/0x1d <- do_sys_open) $retval=3
            <...>-1447  [001] 1038282.286969: myprobe: (do_sys_open+0x0/0xd6) dfd=ffffff9c filename=4041c6 flags=98800 mode=10
-           <...>-1447  [001] 1038282.286976: myretprobe: (sys_open+0x1b/0x1d <- do_sys_open) $rv=3
+           <...>-1447  [001] 1038282.286976: myretprobe: (sys_open+0x1b/0x1d <- do_sys_open) $retval=3
 
 
  Each line shows when the kernel hits an event, and <- SYMBOL means kernel
kernel/trace/trace_kprobe.c +24 -28

96 lines not on this page · raw message

Signature · 1 line
1.6.2.3
#6 lore raw

PATCH 4/7 tracing/kprobes: Avoid field name confliction

In reply to Frederic Weisbecker

From: Masami Hiramatsu <[email protected]>

Check whether the argument name is in conflict with other field names while creating a kprobe through the debugfs interface.

Changes in v3:
- Check strcmp() == 0 instead of !strcmp().

Changes in v2:
- Add common_lock_depth to reserved name list.

Cc Frederic Weisbecker <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Mike Galbraith <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Christoph Hellwig <[email protected]>
Cc Ananth N Mavinakayanahalli <[email protected]>
Cc Jim Keniston <[email protected]>
Cc Frank Ch. Eigler <[email protected]>

LKML-Reference: <[email protected]>

---
 kernel/trace/trace_kprobe.c |   65 +++++++++++++++++++++++++++++++++++--------
 1 files changed, 53 insertions(+), 12 deletions(-)
kernel/trace/trace_kprobe.c +53 -12
index 3313fa7..bb6cb2b 100644
@@ -38,6 +38,25 @@
 #define MAX_EVENT_NAME_LEN 64
 #define KPROBE_EVENT_SYSTEM "kprobes"
 
+/* Reserved field names */
+#define FIELD_STRING_IP "ip"
+#define FIELD_STRING_NARGS "nargs"
+#define FIELD_STRING_RETIP "ret_ip"
+#define FIELD_STRING_FUNC "func"
+
+const char *reserved_field_names[] = {
+	"common_type",
+	"common_flags",
+	"common_preempt_count",
+	"common_pid",
+	"common_tgid",
+	"common_lock_depth",
+	FIELD_STRING_IP,
+	FIELD_STRING_NARGS,
+	FIELD_STRING_RETIP,
+	FIELD_STRING_FUNC,
+};
+
 /* currently, trace_kprobe only supports X86. */
 
 struct fetch_func {
@@ -537,6 +556,20 @@ static int parse_probe_arg(char *arg, struct fetch_func *ff, int is_return)
 	return ret;
 }
 
+/* Return 1 if name is reserved or already used by another argument */
+static int conflict_field_name(const char *name,
+			       struct probe_arg *args, int narg)
+{
+	int i;
+	for (i = 0; i < ARRAY_SIZE(reserved_field_names); i++)
+		if (strcmp(reserved_field_names[i], name) == 0)
+			return 1;
+	for (i = 0; i < narg; i++)
+		if (strcmp(args[i].name, name) == 0)
+			return 1;
+	return 0;
+}
+
 static int create_trace_probe(int argc, char **argv)
 {
 	/*
@@ -637,6 +670,12 @@ static int create_trace_probe(int argc, char **argv)
 			*arg++ = '\0';
 		else
 			arg = argv[i];
+
+		if (conflict_field_name(argv[i], tp->args, i)) {
+			ret = -EINVAL;
+			goto error;
+		}
+
 		tp->args[i].name = kstrdup(argv[i], GFP_KERNEL);
 
 		/* Parse fetch argument */
@@ -1039,8 +1078,8 @@ static int kprobe_event_define_fields(struct ftrace_event_call *event_call)
 	if (!ret)
 		return ret;
 
-	DEFINE_FIELD(unsigned long, ip, "ip", 0);
-	DEFINE_FIELD(int, nargs, "nargs", 1);
+	DEFINE_FIELD(unsigned long, ip, FIELD_STRING_IP, 0);
+	DEFINE_FIELD(int, nargs, FIELD_STRING_NARGS, 1);
 	/* Set argument names as fields */
 	for (i = 0; i < tp->nr_args; i++)
 		DEFINE_FIELD(unsigned long, args[i], tp->args[i].name, 0);
@@ -1057,9 +1096,9 @@ static int kretprobe_event_define_fields(struct ftrace_event_call *event_call)
 	if (!ret)
 		return ret;
 
-	DEFINE_FIELD(unsigned long, func, "func", 0);
-	DEFINE_FIELD(unsigned long, ret_ip, "ret_ip", 0);
-	DEFINE_FIELD(int, nargs, "nargs", 1);
+	DEFINE_FIELD(unsigned long, func, FIELD_STRING_FUNC, 0);
+	DEFINE_FIELD(unsigned long, ret_ip, FIELD_STRING_RETIP, 0);
+	DEFINE_FIELD(int, nargs, FIELD_STRING_NARGS, 1);
 	/* Set argument names as fields */
 	for (i = 0; i < tp->nr_args; i++)
 		DEFINE_FIELD(unsigned long, args[i], tp->args[i].name, 0);
@@ -1108,15 +1147,16 @@ static int kprobe_event_show_format(struct ftrace_event_call *call,
 	int ret, i;
 	struct trace_probe *tp = (struct trace_probe *)call->data;
 
-	SHOW_FIELD(unsigned long, ip, "ip");
-	SHOW_FIELD(int, nargs, "nargs");
+	SHOW_FIELD(unsigned long, ip, FIELD_STRING_IP);
+	SHOW_FIELD(int, nargs, FIELD_STRING_NARGS);
 
 	/* Show fields */
 	for (i = 0; i < tp->nr_args; i++)
 		SHOW_FIELD(unsigned long, args[i], tp->args[i].name);
 	trace_seq_puts(s, "\n");
 
-	return __probe_event_show_format(s, tp, "(%lx)", "REC->ip");
+	return __probe_event_show_format(s, tp, "(%lx)",
+					 "REC->" FIELD_STRING_IP);
 }
 
 static int kretprobe_event_show_format(struct ftrace_event_call *call,
@@ -1126,9 +1166,9 @@ static int kretprobe_event_show_format(struct ftrace_event_call *call,
 	int ret, i;
 	struct trace_probe *tp = (struct trace_probe *)call->data;
 
-	SHOW_FIELD(unsigned long, func, "func");
-	SHOW_FIELD(unsigned long, ret_ip, "ret_ip");
-	SHOW_FIELD(int, nargs, "nargs");
+	SHOW_FIELD(unsigned long, func, FIELD_STRING_FUNC);
+	SHOW_FIELD(unsigned long, ret_ip, FIELD_STRING_RETIP);
+	SHOW_FIELD(int, nargs, FIELD_STRING_NARGS);
 
 	/* Show fields */
 	for (i = 0; i < tp->nr_args; i++)
@@ -1136,7 +1176,8 @@ static int kretprobe_event_show_format(struct ftrace_event_call *call,
 	trace_seq_puts(s, "\n");
 
 	return __probe_event_show_format(s, tp, "(%lx <- %lx)",
-					  "REC->func, REC->ret_ip");
+					 "REC->" FIELD_STRING_FUNC
+					 ", REC->" FIELD_STRING_RETIP);
 }
 
 #ifdef CONFIG_EVENT_PROFILE
Signature · 1 line
1.6.2.3
#7 lore raw

PATCH 5/7 tracing/kprobes: Robustify fixed field names against variable field names conflicts

In reply to Frederic Weisbecker

From: Masami Hiramatsu <[email protected]>

Rename probe-common fixed field names to harder conflictable names, because current 'ip', 'func', and other probe field names are easily in conflict with user-specified variable names.

Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Mike Galbraith <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Christoph Hellwig <[email protected]>
Cc Ananth N Mavinakayanahalli <[email protected]>
Cc Jim Keniston <[email protected]>
Cc Frank Ch. Eigler <[email protected]>

LKML-Reference: <[email protected]>

---
 kernel/trace/trace_kprobe.c |    8 ++++----
 1 files changed, 4 insertions(+), 4 deletions(-)
kernel/trace/trace_kprobe.c +4 -4
index bb6cb2b..739f70e 100644
@@ -39,10 +39,10 @@
 #define KPROBE_EVENT_SYSTEM "kprobes"
 
 /* Reserved field names */
-#define FIELD_STRING_IP "ip"
-#define FIELD_STRING_NARGS "nargs"
-#define FIELD_STRING_RETIP "ret_ip"
-#define FIELD_STRING_FUNC "func"
+#define FIELD_STRING_IP "__probe_ip"
+#define FIELD_STRING_NARGS "__probe_nargs"
+#define FIELD_STRING_RETIP "__probe_ret_ip"
+#define FIELD_STRING_FUNC "__probe_func"
 
 const char *reserved_field_names[] = {
 	"common_type",
Signature · 1 line
1.6.2.3
#8 lore raw

PATCH 6/7 perf: Add perf probe subcommand, a kprobe-event setup helper

In reply to Frederic Weisbecker

From: Masami Hiramatsu <[email protected]>

Add perf probe subcommand that implements a kprobe-event setup helper to the perf command.
This allows user to define kprobe events using C expressions (C line numbers, C function names, and C local variables).

Usage
-----
perf probe [<options>] -P 'PROBEDEF' [-P 'PROBEDEF' ...]

-k, --vmlinux <file> vmlinux/module pathname
-P, --probe <p|r:[GRP/]NAME FUNC[+OFFS][@SRC]|@SRC:LINE [ARG ...]>
probe point definition, where
p: kprobe probe
r: kretprobe probe
GRP: Group name (optional)
NAME: Event name
FUNC: Function name
OFFS: Offset from function entry (in byte)
SRC: Source code path
LINE: Line number
ARG: Probe argument (local variable name or
kprobe-tracer argument format is supported.)

Changes in v4:
- Add _GNU_SOURCE macro for strndup().

Changes in v3:
- Remove -r option because perf always be used for online kernel.
- Check malloc/calloc results.

Changes in v2:
- Check synthesized string length.
- Rename perf kprobe to perf probe.
- Use spaces for separator and update usage comment.
- Check error paths in parse_probepoint().
- Check optimized-out variables.

Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Mike Galbraith <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Christoph Hellwig <[email protected]>
Cc Ananth N Mavinakayanahalli <[email protected]>
Cc Jim Keniston <[email protected]>
Cc Frank Ch. Eigler <[email protected]>

LKML-Reference: <[email protected]>

---
 tools/perf/Makefile            |   10 +
 tools/perf/builtin-probe.c     |  358 +++++++++++++++++++++
 tools/perf/builtin.h           |    1 +
 tools/perf/perf.c              |    3 +
 tools/perf/util/probe-finder.c |  690 ++++++++++++++++++++++++++++++++++++++++
 tools/perf/util/probe-finder.h |   68 ++++
 6 files changed, 1130 insertions(+), 0 deletions(-)
 create mode 100644 tools/perf/builtin-probe.c
 create mode 100644 tools/perf/util/probe-finder.c
 create mode 100644 tools/perf/util/probe-finder.h
tools/perf/Makefile +10 -0
index b5f1953..6dabcf1 100644
@@ -419,6 +419,16 @@ ifneq ($(shell sh -c "(echo '\#include <libelf.h>'; echo 'int main(void) { Elf *
 	msg := $(error No libelf.h/libelf found, please install libelf-dev/elfutils-libelf-devel);
 endif
 
+ifneq ($(shell sh -c "(echo '\#include <libdwarf/dwarf.h>'; echo '\#include <libdwarf/libdwarf.h>'; echo 'int main(void) { Dwarf_Debug dbg; Dwarf_Error err; dwarf_init(0, DW_DLC_READ, 0, 0, &dbg, &err); return (long)dbg; }') | $(CC) -x c - $(ALL_CFLAGS) -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -ldwarf -lelf -o /dev/null $(ALL_LDFLAGS) > /dev/null 2>&1 && echo y"), y)
+	msg := $(warning No libdwarf.h found, disables probe subcommand. Please install libdwarf-dev/libdwarf-devel);
+else
+	EXTLIBS += -lelf -ldwarf
+	LIB_H += util/probe-finder.h
+	LIB_OBJS += util/probe-finder.o
+	BUILTIN_OBJS += builtin-probe.o
+	BASIC_CFLAGS += -DSUPPORT_DWARF
+endif
+
 ifdef NO_DEMANGLE
 	BASIC_CFLAGS += -DNO_DEMANGLE
 else
tools/perf/builtin-probe.c new file +358 -0

358 lines not on this page · raw message

tools/perf/builtin.h +1 -0
index e11d8d2..ad5f0f4 100644
@@ -25,5 +25,6 @@ extern int cmd_timechart(int argc, const char **argv, const char *prefix);
 extern int cmd_top(int argc, const char **argv, const char *prefix);
 extern int cmd_trace(int argc, const char **argv, const char *prefix);
 extern int cmd_version(int argc, const char **argv, const char *prefix);
+extern int cmd_probe(int argc, const char **argv, const char *prefix);
 
 #endif
tools/perf/perf.c +3 -0
index 19fc7fe..f598120 100644
@@ -295,6 +295,9 @@ static void handle_internal_command(int argc, const char **argv)
 		{ "version", cmd_version, 0 },
 		{ "trace", cmd_trace, 0 },
 		{ "sched", cmd_sched, 0 },
+#ifdef SUPPORT_DWARF
+		{ "probe", cmd_probe, 0 },
+#endif
 	};
 	unsigned int i;
 	static const char ext[] = STRIP_EXTENSION;
tools/perf/util/probe-finder.c new file +690 -0

690 lines not on this page · raw message

tools/perf/util/probe-finder.h new file +68 -0

68 lines not on this page · raw message

Signature · 1 line
1.6.2.3
#9 lore raw

PATCH 7/7 perf probe: Add perf probe command support without libdwarf

In reply to Frederic Weisbecker

From: Masami Hiramatsu <[email protected]>

Enables 'perf probe' even if libdwarf is not installed. If libdwarf is not found, 'perf probe' just disables dwarf support. Users can use 'perf probe' to set up new events by using kprobe_events format.

Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Mike Galbraith <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Christoph Hellwig <[email protected]>
Cc Ananth N Mavinakayanahalli <[email protected]>
Cc Jim Keniston <[email protected]>
Cc Frank Ch. Eigler <[email protected]>

LKML-Reference: <[email protected]>

---
 tools/perf/Makefile            |    6 ++--
 tools/perf/builtin-probe.c     |   42 +++++++++++++++++++++++++++++++++------
 tools/perf/perf.c              |    2 -
 tools/perf/util/probe-finder.h |    2 +
 4 files changed, 40 insertions(+), 12 deletions(-)
tools/perf/Makefile +3 -3
index 6dabcf1..52b1f43 100644
@@ -385,6 +385,7 @@ BUILTIN_OBJS += builtin-stat.o
 BUILTIN_OBJS += builtin-timechart.o
 BUILTIN_OBJS += builtin-top.o
 BUILTIN_OBJS += builtin-trace.o
+BUILTIN_OBJS += builtin-probe.o
 
 PERFLIBS = $(LIB_FILE)
 
@@ -420,13 +421,12 @@ ifneq ($(shell sh -c "(echo '\#include <libelf.h>'; echo 'int main(void) { Elf *
 endif
 
 ifneq ($(shell sh -c "(echo '\#include <libdwarf/dwarf.h>'; echo '\#include <libdwarf/libdwarf.h>'; echo 'int main(void) { Dwarf_Debug dbg; Dwarf_Error err; dwarf_init(0, DW_DLC_READ, 0, 0, &dbg, &err); return (long)dbg; }') | $(CC) -x c - $(ALL_CFLAGS) -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -ldwarf -lelf -o /dev/null $(ALL_LDFLAGS) > /dev/null 2>&1 && echo y"), y)
-	msg := $(warning No libdwarf.h found, disables probe subcommand. Please install libdwarf-dev/libdwarf-devel);
+	msg := $(warning No libdwarf.h found, disables dwarf support. Please install libdwarf-dev/libdwarf-devel);
+	BASIC_CFLAGS += -DNO_LIBDWARF
 else
 	EXTLIBS += -lelf -ldwarf
 	LIB_H += util/probe-finder.h
 	LIB_OBJS += util/probe-finder.o
-	BUILTIN_OBJS += builtin-probe.o
-	BASIC_CFLAGS += -DSUPPORT_DWARF
 endif
 
 ifdef NO_DEMANGLE
tools/perf/builtin-probe.c +35 -7

119 lines not on this page · raw message

tools/perf/perf.c +0 -2
index f598120..c570d17 100644
@@ -295,9 +295,7 @@ static void handle_internal_command(int argc, const char **argv)
 		{ "version", cmd_version, 0 },
 		{ "trace", cmd_trace, 0 },
 		{ "sched", cmd_sched, 0 },
-#ifdef SUPPORT_DWARF
 		{ "probe", cmd_probe, 0 },
-#endif
 	};
 	unsigned int i;
 	static const char ext[] = STRIP_EXTENSION;
tools/perf/util/probe-finder.h +2 -0
index af920de..306810c 100644
@@ -44,6 +44,7 @@ struct probe_point {
 	char	*probes[MAX_PROBES];	/* Output buffers (will be allocated)*/
 };
 
+#ifndef NO_LIBDWARF
 extern int find_probepoint(int fd, struct probe_point *pp);
 
 #include <libdwarf/dwarf.h>
@@ -64,5 +65,6 @@ struct probe_finder {
 	char		*buf;		/* Current output buffer */
 	int		len;		/* Length of output buffer */
 };
+#endif /* NO_LIBDWARF */
 
 #endif /*_PROBE_FINDER_H */
Signature · 1 line
1.6.2.3
#10 lore raw
In reply to Frederic Weisbecker
Quoted 8 lines · Frederic Weisbecker

Ingo,

I've tested a little this set and it seems to work well. I can't test the C expression parser (need some libdwarf cross-version compatiblity), but I can create kprobes via low-level parameters, then launch perf record and perf trace successfully. I'll need to update perf trace to handle variables that start with '$' characters though.

Quoted 7 lines

Anyway, it's still unstable but I guess pulling it would help Masami to continue without the need to handle the whole batch.

Please pull the tracing/kprobes branch that can be found at:

git://git.kernel.org/pub/scm/linux/kernel/git/frederic/random-tracing.git
tracing/kprobes

Pulled, thanks Frederic.

Quoted 4 lines
 Documentation/trace/kprobetrace.txt |   27 +-
 kernel/trace/trace_kprobe.c         |  160 +++++---
 tools/perf/Makefile                 |   10 +
 tools/perf/builtin-probe.c          |  386 +++++++++++++++++++
Quoted 8 lines
 tools/perf/builtin.h                |    1 +
 tools/perf/perf.c                   |    1 +
 tools/perf/util/probe-finder.c      |  690 +++++++++++++++++++++++++++++++++++
 tools/perf/util/probe-finder.h      |   70 ++++
 8 files changed, 1271 insertions(+), 74 deletions(-)
 create mode 100644 tools/perf/builtin-probe.c
 create mode 100644 tools/perf/util/probe-finder.c
 create mode 100644 tools/perf/util/probe-finder.h

Hm, there's no x86 decoder change here - does this lot fixes the instruction decoder bug i reported some time ago? That is which keeps tracing/kprobes out of tip:master.

Ingo

#11 lore raw
Quoted 11 lines · Ingo Molnar
Quoted 10 lines
 tools/perf/Makefile                 |   10 +
 tools/perf/builtin-probe.c          |  386 +++++++++++++++++++
 tools/perf/builtin.h                |    1 +
 tools/perf/perf.c                   |    1 +
 tools/perf/util/probe-finder.c      |  690 +++++++++++++++++++++++++++++++++++
 tools/perf/util/probe-finder.h      |   70 ++++
 8 files changed, 1271 insertions(+), 74 deletions(-)
 create mode 100644 tools/perf/builtin-probe.c
 create mode 100644 tools/perf/util/probe-finder.c
 create mode 100644 tools/perf/util/probe-finder.h
Quoted 3 lines

Hm, there's no x86 decoder change here - does this lot fixes the instruction decoder bug i reported some time ago? That is which keeps tracing/kprobes out of tip:master.

See my previous report below.

Ingo

----- Forwarded message from Ingo Molnar <[email protected]> -----

Date: Sat, 3 Oct 2009 12:49:23 +0200
From: Ingo Molnar <[email protected]>
To: Frederic Weisbecker <[email protected]>
Subject: Re: [GIT PULL] tracing/kprobes: kprobes fixes
Cc: LKML <[email protected]>,
Steven Rostedt <[email protected]>, Li Zefan <[email protected]>

ok, -tip testing found this new failure:

TEST posttest
Error: ffffffff8106ac00: 66 0f 73 fd 04 pslldq $0x4,%xmm5
Error: objdump says 5 bytes, but insn_get_length() says 4 (attr:8000) make[1]: *** [posttest] Error 2
make: *** [bzImage] Error 2

Ingo

----- End forwarded message -----

#12 lore raw
Quoted 35 lines · Ingo Molnar
Quoted 11 lines · Ingo Molnar
Quoted 10 lines
 tools/perf/Makefile                 |   10 +
 tools/perf/builtin-probe.c          |  386 +++++++++++++++++++
 tools/perf/builtin.h                |    1 +
 tools/perf/perf.c                   |    1 +
 tools/perf/util/probe-finder.c      |  690 +++++++++++++++++++++++++++++++++++
 tools/perf/util/probe-finder.h      |   70 ++++
 8 files changed, 1271 insertions(+), 74 deletions(-)
 create mode 100644 tools/perf/builtin-probe.c
 create mode 100644 tools/perf/util/probe-finder.c
 create mode 100644 tools/perf/util/probe-finder.h
Quoted 3 lines

Hm, there's no x86 decoder change here - does this lot fixes the instruction decoder bug i reported some time ago? That is which keeps tracing/kprobes out of tip:master.

See my previous report below.

Ingo

----- Forwarded message from Ingo Molnar <[email protected]> -----

Date: Sat, 3 Oct 2009 12:49:23 +0200
From: Ingo Molnar <[email protected]>
To: Frederic Weisbecker <[email protected]>
Subject: Re: [GIT PULL] tracing/kprobes: kprobes fixes
Cc: LKML <[email protected]>,
Steven Rostedt <[email protected]>, Li Zefan <[email protected]>

ok, -tip testing found this new failure:

Quoted 8 lines

TEST posttest
Error: ffffffff8106ac00: 66 0f 73 fd 04 pslldq $0x4,%xmm5
Error: objdump says 5 bytes, but insn_get_length() says 4 (attr:8000) make[1]: *** [posttest] Error 2
make: *** [bzImage] Error 2

Ingo

Hmm, thank you for reporting, I've missed some group opcode...

Thank you again,

Signature · 7 lines
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America), Inc.
Software Solutions Division

e-mail: [email protected]
#13 lore raw
In reply to Ingo Molnar
Quoted 34 lines · Ingo Molnar
Quoted 8 lines · Frederic Weisbecker

Ingo,

I've tested a little this set and it seems to work well. I can't test the C expression parser (need some libdwarf cross-version compatiblity), but I can create kprobes via low-level parameters, then launch perf record and perf trace successfully. I'll need to update perf trace to handle variables that start with '$' characters though.

Quoted 7 lines

Anyway, it's still unstable but I guess pulling it would help Masami to continue without the need to handle the whole batch.

Please pull the tracing/kprobes branch that can be found at:

git://git.kernel.org/pub/scm/linux/kernel/git/frederic/random-tracing.git
tracing/kprobes

Pulled, thanks Frederic.

Quoted 4 lines
 Documentation/trace/kprobetrace.txt |   27 +-
 kernel/trace/trace_kprobe.c         |  160 +++++---
 tools/perf/Makefile                 |   10 +
 tools/perf/builtin-probe.c          |  386 +++++++++++++++++++
Quoted 8 lines
 tools/perf/builtin.h                |    1 +
 tools/perf/perf.c                   |    1 +
 tools/perf/util/probe-finder.c      |  690 +++++++++++++++++++++++++++++++++++
 tools/perf/util/probe-finder.h      |   70 ++++
 8 files changed, 1271 insertions(+), 74 deletions(-)
 create mode 100644 tools/perf/builtin-probe.c
 create mode 100644 tools/perf/util/probe-finder.c
 create mode 100644 tools/perf/util/probe-finder.h
Quoted 5 lines

Hm, there's no x86 decoder change here - does this lot fixes the instruction decoder bug i reported some time ago? That is which keeps tracing/kprobes out of tip:master.

Ingo

Not yet, I hope it's part of the last family of instruction set we are missing.

#14 lore raw
Quoted 39 lines · Frederic Weisbecker
Quoted 38 lines · Ingo Molnar
Quoted 8 lines · Frederic Weisbecker

Ingo,

I've tested a little this set and it seems to work well. I can't test the C expression parser (need some libdwarf cross-version compatiblity), but I can create kprobes via low-level parameters, then launch perf record and perf trace successfully. I'll need to update perf trace to handle variables that start with '$' characters though.

Quoted 7 lines

Anyway, it's still unstable but I guess pulling it would help Masami to continue without the need to handle the whole batch.

Please pull the tracing/kprobes branch that can be found at:

git://git.kernel.org/pub/scm/linux/kernel/git/frederic/random-tracing.git
tracing/kprobes

Pulled, thanks Frederic.

Quoted 4 lines
 Documentation/trace/kprobetrace.txt |   27 +-
 kernel/trace/trace_kprobe.c         |  160 +++++---
 tools/perf/Makefile                 |   10 +
 tools/perf/builtin-probe.c          |  386 +++++++++++++++++++
Quoted 8 lines
 tools/perf/builtin.h                |    1 +
 tools/perf/perf.c                   |    1 +
 tools/perf/util/probe-finder.c      |  690 +++++++++++++++++++++++++++++++++++
 tools/perf/util/probe-finder.h      |   70 ++++
 8 files changed, 1271 insertions(+), 74 deletions(-)
 create mode 100644 tools/perf/builtin-probe.c
 create mode 100644 tools/perf/util/probe-finder.c
 create mode 100644 tools/perf/util/probe-finder.h

Hm, there's no x86 decoder change here - does this lot fixes the instruction decoder bug i reported some time ago? That is which keeps tracing/kprobes out of tip:master.

Quoted 5 lines
Quoted 1 line

Ingo

Not yet, I hope it's part of the last family of instruction set we are missing.

ok.

We dont want topics to not be in tip:master for too long - we dont grow conflict resolutions nor does it get tested widely.

Ingo