Skip to contentSkip to conversation

perf regression: failed to stat syscalls events

DISCUSSION Wu Fengguang 2 participants 4 messages Jun 24, 2010 Tracing

lore
Share & tools
lore
#0 lore raw

Greetings,

I noticed that

        # perf stat -e syscalls:sys_enter_mmap true

refuse to work on 2.6.35-rc1. The last working kernel is 2.6.34.

Any suggestions before me jumping into debug?

Thanks,
Fengguang

#1 lore raw
Quoted 7 lines · Wu Fengguang

Greetings,

I noticed that

        # perf stat -e syscalls:sys_enter_mmap true

refuse to work on 2.6.35-rc1. The last working kernel is 2.6.34.

Other events like lock:* or block:* or whatever continue to work. So this is a problem specific to syscalls:*.

Thanks,
Fengguang

Tested-by Srikar Dronamraju Acked-by
#2 lore raw
Quoted 11 lines · Wu Fengguang
Quoted 7 lines · Wu Fengguang

Greetings,

I noticed that

        # perf stat -e syscalls:sys_enter_mmap true

refuse to work on 2.6.35-rc1. The last working kernel is 2.6.34.

Other events like lock:* or block:* or whatever continue to work. So this is a problem specific to syscalls:*.

Does your tree contain:

---
commit a8fb2608053547bc3152ea61a5ec7cdfce5d942c

Author: Steven Rostedt <[email protected]>
Date:   Thu Jun 10 14:53:16 2010 -0400

perf/tracing: Fix regression of perf losing kprobe events

With the addition of the code to shrink the kernel tracepoint infrastructure, we lost kprobes being traced by perf. The reason is that I tested if the "tp_event->class->perf_probe" existed before enabling it. This prevents "ftrace only" events (like the function trace events) from being enabled by perf.

Unfortunately, kprobe events do not use perf_probe. This causes kprobes to be missed by perf. To fix this, we add the test to see if "tp_event->class->reg" exists as well as perf_probe.

Normal trace events have only "perf_probe" but no "reg" function, and kprobes and syscalls have the "reg" but no "perf_probe". The ftrace unique events do not have either, so this is a valid test. If a kprobe or syscall is not to be probed by perf, the "reg" function is called anyway, and will return a failure and prevent perf from probing it.

kernel/trace/trace_event_perf.c +3 -1
index e6f6588..8a2b73f 100644
@@ -96,7 +96,9 @@ int perf_trace_init(struct perf_event *p_event)
 	mutex_lock(&event_mutex);
 	list_for_each_entry(tp_event, &ftrace_events, list) {
 		if (tp_event->event.type == event_id &&
-		    tp_event->class && tp_event->class->perf_probe &&
+		    tp_event->class &&
+		    (tp_event->class->perf_probe ||
+		     tp_event->class->reg) &&
 		    try_module_get(tp_event->mod)) {
 			ret = perf_trace_event_init(tp_event, p_event);
 			break;
#3 lore raw
Quoted 6 lines · Peter Zijlstra
Quoted 5 lines · Wu Fengguang
Quoted 4 lines · Wu Fengguang

Greetings,

I noticed that

Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
        # perf stat -e syscalls:sys_enter_mmap true

refuse to work on 2.6.35-rc1. The last working kernel is 2.6.34.

Other events like lock:* or block:* or whatever continue to work. So this is a problem specific to syscalls:*.

Does your tree contain:

Thanks, it worked! This patch is missing from linux-2.6.git for now.

Thanks,
Fengguang

Quoted 45 lines

---
commit a8fb2608053547bc3152ea61a5ec7cdfce5d942c

Author: Steven Rostedt <[email protected]>
Date:   Thu Jun 10 14:53:16 2010 -0400

perf/tracing: Fix regression of perf losing kprobe events

With the addition of the code to shrink the kernel tracepoint infrastructure, we lost kprobes being traced by perf. The reason is that I tested if the "tp_event->class->perf_probe" existed before enabling it. This prevents "ftrace only" events (like the function trace events) from being enabled by perf.

Unfortunately, kprobe events do not use perf_probe. This causes kprobes to be missed by perf. To fix this, we add the test to see if "tp_event->class->reg" exists as well as perf_probe.

Normal trace events have only "perf_probe" but no "reg" function, and kprobes and syscalls have the "reg" but no "perf_probe". The ftrace unique events do not have either, so this is a valid test. If a kprobe or syscall is not to be probed by perf, the "reg" function is called anyway, and will return a failure and prevent perf from probing it.

kernel/trace/trace_event_perf.c +3 -1
index e6f6588..8a2b73f 100644
@@ -96,7 +96,9 @@ int perf_trace_init(struct perf_event *p_event)
 	mutex_lock(&event_mutex);
 	list_for_each_entry(tp_event, &ftrace_events, list) {
 		if (tp_event->event.type == event_id &&
-		    tp_event->class && tp_event->class->perf_probe &&
+		    tp_event->class &&
+		    (tp_event->class->perf_probe ||
+		     tp_event->class->reg) &&
 		    try_module_get(tp_event->mod)) {
 			ret = perf_trace_event_init(tp_event, p_event);
 			break;